HUD Information Systems: Improved Management Practices Needed to Control Integration Cost and Schedule
Released: 1998-12-18T00:00:00

Letter

Background

HUD is the principal federal agency responsible for programs dealing with housing, community development, and fair housing opportunities. Its mission includes making housing affordable through the Federal Housing Administration’s (FHA) mortgage insurance for multifamily housing, providing rental assistance for about 4.5 million lower income residents, helping to revitalize over 4,000 localities through community development programs, and encouraging home ownership by providing mortgage insurance. HUD is one of the nation’s largest financial institutions, responsible for managing more than a reported $454 billion in mortgage insurance and, as of September 30, 1997, a reported $531 billion in guarantees of mortgage-backed securities. For fiscal year 1998, the agency’s budget authority was about $24 billion, and its information technology budget was $222 million.

HUD’s major program areas are managed by the Office of Housing, which includes FHA’s insurance and project-based rental assistance programs; the Office of Community Planning and Development, which includes programs for Community Development Block Grants, empowerment zones/enterprise communities, and assistance for the homeless; the Office of Public and Indian Housing, which provides funds to help operate and modernize public and Indian housing and administers tenant-based rental assistance programs; and the Office of Fair Housing and Equal Opportunity, which is responsible for investigating complaints and ensuring compliance with fair housing laws.

In 1984, we reported that HUD lacked adequate information and financial management systems necessary to ensure accountability for, and control over, departmental programs. In 1989, HUD was involved in highly publicized scandals that included instances in which private real estate agents were able to steal millions of dollars by retaining the proceeds from the sale of FHA-owned properties, rather than transferring the funds to the Treasury. In 1992, we reported that these scandals were attributed, in large part, to fundamental deficiencies in the department’s information and financial management systems. In particular, HUD’s systems were inadequate, lacked credibility and internal controls, and failed to meet program managers’ needs or provide adequate support for oversight of housing and community development programs.

To address fundamental deficiencies in the department’s information and financial systems and meet the requirements of the Chief Financial Officers Act of 1990, which called for financial management reform across the federal government, the Secretary of Housing and Urban Development initiated a number of actions. These actions included the appointment of a Chief Financial Officer (CFO) to oversee the department’s financial operations and initiation of a major Financial Systems Integration (FSI) effort to strengthen its financial management systems. Although HUD proceeded with this high priority effort, it continued to be affected by poorly integrated, ineffective, and generally unreliable information systems that did not satisfy management needs or provide adequate support to control housing and community development programs.

In 1994, we designated the department a high-risk area, in part because of its inadequate information and financial management systems and slow progress in correcting fundamental management weaknesses that had allowed the 1989 scandals to occur. By 1997, we reported that HUD had formulated approaches and initiated actions to address departmentwide deficiencies, including information and financial management systems problems, but many of these actions were far from being completed. In the meantime, HUD continues to rely on unintegrated and inadequate program and financial management systems, some of which are not yet Year 2000 compliant.

Recognizing the need to better manage information technology, recent legislative reforms—the Clinger-Cohen Act of 1996, the Paperwork Reduction Act of 1995, and the Federal Acquisition Streamlining Act of 1994—provide guidance to federal agencies on how to plan, manage, and acquire information technology as part of their overall information resources management (IRM) responsibilities. These legislative reforms highlight the need to ensure that IRM programs and decisions are integrated with organizational planning, budgeting, and financial management.

Integration Costs Have Increased and Expected Completion Has Been Delayed

While HUD revised its FSI plan in 1993 and again in 1997, its primary objective—implementing an integrated financial management system to meet the department’s program and financial management needs—remained unchanged. At the same time, HUD’s implementation strategy and the cost and schedule estimates to develop and deploy FSI continue to change. HUD has not yet finalized the cost and schedule estimates for its 1997 FSI strategy and has not performed the detailed analyses needed to determine whether the strategy is cost beneficial.

HUD’s 1991 FSI Plan

HUD adopted its first Financial Management Systems Strategic Integration Plan in November 1991 to address and resolve material weaknesses in its financial systems. In this plan, HUD acknowledged that inadequate and unintegrated financial management systems rendered it unable to properly manage its programs and financial resources. The plan’s primary objective was, therefore, to implement an integrated financial management system that would meet the department’s program and financial management needs. The 1991 plan contained specific objectives to establish sound financial management controls, correct material weaknesses, improve financial management, provide timely and accurate information to managers to enable them to meet their organizational objectives, meet the goals of section 4 of the Federal Managers’ Financial Integrity Act (FMFIA) of 1982 and comply with the Office of Management and Budget’s (OMB) Circular A-127.

HUD’s strategy for achieving the FSI objectives was to replace about 100 separate financial and mixed systems with nine new fully integrated systems. This strategy was based on an analysis which concluded that HUD did not have the basic financial management systems to serve as the foundation for an integrated systems environment. The design of the 1991 FSI plan required that eight financial systems be integrated with the new core accounting system and this design was consistent with the Joint Financial Management Improvement Program’s (JFMIP) framework for financial management systems. Once the nine new standard systems were deployed, program offices were to use them to support their business operations. In addition, the plan noted that it would be necessary to make interim improvements to existing systems, since the integration effort would be a long-term project. The interim improvements would be needed to manage programs, comply with legal mandates, and correct material weaknesses until the nine new systems became available. The department estimated that it would cost about $103 million to develop and deploy the nine systems called for in the 1991 FSI plan by September 1998. Table 1 shows the objectives, estimated development and deployment costs, and scheduled deployment dates for the nine planned integrated systems.

According to HUD’s CFO, in fiscal years 1992 and 1993, HUD spent about $58 million on FSI. Specifically, $48 million was spent on interim improvements to legacy systems, and $10 million was spent on the Core Accounting System and Mortgage Insurance System projects. During this period, HUD (1) procured Federal Financial System (FFS), a commercial off-the-shelf software package to serve as the department’s Core Accounting System, (2) halted its efforts to develop the Mortgage Insurance System as a result of poor planning, and (3) terminated the Grants, Subsidies, and Loans project because the department could not streamline its grants process. Work on the remaining six projects was not scheduled to begin until 1994.

1993 Revision of HUD’s 1991 FSI Strategy

In September 1993, HUD fundamentally changed its FSI strategy. The revised strategy was in response to (1) slow progress in implementing systems integration, (2) the need to comply with revisions to OMB Circular A-127, and (3) senior management’s serious doubt about the viability of creating nine new fully integrated systems and having program offices adapt their business operations to meet the requirements of these systems by 1998.

The 1993 FSI plan included the same primary objective of implementing an integrated financial management system to meet the department’s program and financial management needs (which was delineated in the 1991 FSI plan), as well as objectives for improving HUD’s financial systems and bringing HUD into compliance with the provisions of FMFIA and the revised OMB Circular A-127. In addition, the 1993 plan included new objectives, such as eliminating the department’s financial systems from OMB’s list of high-risk areas for management improvement, being consistent with HUD’s reform plan, and meeting the requirements of the Government Performance and Results Act of 1993.

The implementation strategy for the 1993 FSI plan was markedly different from that of the 1991 plan. Under the 1993 FSI plan, the CFO’s office was required to complete the core financial system project initiated under the 1991 strategy and program offices were required to develop (1) new systems that would support program management priorities, financial and management information needs, and business needs and (2) integrate these systems with the core financial system. Plans for the remaining eight standard systems called for in the 1991 strategy were cancelled. As in 1991, the conceptual design of the 1993 FSI strategy was consistent with the Joint Financial Management Improvement Program’s requirement that the core financial system receive data from other financial and mixed systems. By 1996, HUD had initiated 10 systems integration projects.

Despite differences in the strategies, HUD did not perform a cost-benefit analysis on the 1993 strategy. Therefore, HUD had no assurance that it had selected the most cost-beneficial solution for FSI. In 1995, the department estimated that the development and deployment cost of the 1993 strategy would be about $209 million. Also, the department extended the deployment date to December 1998, 3 months after the initial scheduled completion date of September 1998. Table 2 describes the 10 major projects under the 1993 FSI plan, estimated development and deployment costs, and initial scheduled deployment dates.

From fiscal years 1994 through 1997, the department spent about $181 million to develop, deploy, and maintain various functions of the 10 major FSI projects, according to the CFO. These expenditures are in addition to the $58 million reportedly spent on FSI between fiscal years 1992 and 1993 and do not include additional FSI costs that may have been incurred by program offices. According to the Office of the CFO and FSI project managers, the status of the 10 systems integration projects under the 1993 FSI plan is as follows:

The Office of Public and Indian Housing deployed the HUD Central Accounting and Program System (HUDCAPS) in fiscal year 1995 to support its tenant-based Section 8 program.

The Office of Community Planning and Development’s Integrated Disbursement and Information System (IDIS) was deployed and was being used to monitor an estimated 950 community development grantees as of September 1998.

The Office of Housing deployed three of the four Tenants Rental Assistance Certification System (TRACS) modules since fiscal year 1997. These modules are being used for contract processing, tenant voucher processing, and budget development and analysis. The Office of Housing also developed and deployed a computer income-matching module to verify the income of tenants receiving rental subsidy from the department.

The departmental Grants Management System (GMS) project was terminated in 1997.In April 1997, the Office of Administration deployed the HUD Procurement System (HPS), which is being used to track and manage the department’s procurement activities.

The Office of Public and Indian Housing (PIH) deployed five of six modules of the Integrated Business System (IBS) as of February 1998. This system is being used by PIH and the Office of Native Americans to monitor their programs, including information related to all housing authorities in the country.

The Office of Fair Housing and Equal Opportunity deployed one of the three modules for its Grants Evaluation Management System (GEMS) in fiscal year 1996.

The Office of Budget deployed the first module of its Budget Formulation System in May 1997. This system is used by the CFO to formulate, prepare and monitor the annual budget.

The CFO’s Office deployed a portion of HUDCAPS to support the department’s administrative accounting functions in fiscal year 1994.

The Office of Housing’s Federal Housing Administration Mortgage Insurance System (FHAMIS) deployed a data warehouse for multifamily data in March 1997, developed information strategy plans and performed business process reviews to lay the foundation for future FHAMIS systems development efforts.

FSI Revised Again in 1997

In 1997, HUD again revised its FSI strategy after concluding that (1) it could not fully deploy HUDCAPS—the core financial system—by September 1998 and (2) the systems integration effort had to conform to the HUD 2020 Management Reform Plan. According to the CFO, HUDCAPS could not be completed as scheduled. Specifically, the system had been deployed on schedule to support the department’s administrative and Section 8 accounting functions, but it had not been deployed to replace the three remaining general ledger systems as planned. Also, program offices had not yet developed the interfaces between the mixed systems and the core financial system. The CFO also stated that the revised FSI strategy had to conform with the HUD 2020 Management Reform Plan, which established an October 1999 deadline for fully deploying an integrated core financial system and called for repairing or replacing the department’s existing mixed systems and developing new systems to support the reforms. The primary objective of the 1997 FSI plan was consistent with the 1991 and 1993 FSI plans—to implement an integrated financial management system, consisting of both financial and mixed systems, that would provide the information necessary to carry out financial and programmatic missions of the department. However, as in 1993, HUD did not perform any cost-benefit analyses despite the additional systems and schedule changes required by the plan. As a result, HUD cannot assure that the 1997 strategy is the most cost-beneficial alternative.

As of May 1998, the CFO had identified nine projects for the 1997 FSI strategy, including five that were started under the 1993 FSI plan (i.e., HUDCAPS, FHAMIS, GEMS, IDIS, and IBS), and four new projects required to support HUD’s 2020 Management Reform Plan. The objective of the HUDCAPS project, however, was expanded to centralize the development of interfaces between mixed systems and the core financial management system. New projects in the 1997 plan include developing interfaces between HUD’s geographic information system and data warehouses, and deploying an executive information system, Real Estate Management System, and FHA’s Financial Data Warehouse.

The cost and schedule estimates to complete the 1997 FSI strategy have not been finalized, and the FSI cost estimate through September 1999 has fluctuated considerably. For example, HUD’s FSI estimate has varied from $540 million in June 1998, to $255 million on November 12, 1998, to $239 million a week later. However, we found that the $255 million and the $239 million estimates do not include at least $132 million associated with maintaining FSI systems. Until HUD finalizes its plans and cost and schedule estimates to complete the 1997 strategy, the expected FSI cost will remain uncertain. In May 1998, the Office of the CFO said that the cost and schedule estimates to complete the 1997 FSI strategy would be finalized by September 30, 1998. However, as of October 19, 1998, these estimates had not been approved. Table 3 displays the 9 projects included in the 1997 FSI strategy as of May 1998, as well as their corresponding initial development and deployment cost and schedule estimates.

HUD has been working to implement the 1997 FSI strategy. The CFO stated that a large number of FSI systems or system modules have been deployed and are being used to manage and monitor the department’s programs. According to the CFO, the status of the nine 1997 systems integration projects as of October 1998 is as follows: In addition to the work completed under the 1993 FSI plan, the Office of Community Planning and Development has also deployed the Integrated Disbursement and Information System (IDIS) in nine states and the District of Columbia.In addition to the work completed under the 1993 FSI plan, the Office of Public and Indian Housing has developed and deployed the sixth module to support the 1993 Integrated Business System (IBS) requirements and implemented a new module to support the business requirements of the Office of Native American programs.

The FHA Financial Data Warehouse is still being developed by the Office of Housing. Therefore, the July 1998 estimated deployment date was not met. In addition to the work completed under the 1993 FSI plan, the Office of Fair Housing and Equal Opportunity (FHEO) deployed an enhanced version of the first module for the Grants Evaluation Management System (GEMS) that supports the pre-award process and completed the development of the second module to support grantee tracking in fiscal year 1998. This system is being used by FHEO to monitor its two major grant programs. In addition to the work completed under the 1993 FSI plan and to carry out Federal Housing Administration Mortgage Insurance System (FHAMIS) goals in the information strategy plans and resulting from business process reviews, the Office of Housing deployed and is using the Single Family Premium Collection Subsystem to collect and account for premiums. The Office of Housing also deployed the Single Family Data Warehouse and a multifamily data quality system to support its FHAMIS project.

The Office of Housing deployed the first phase of the Real Estate Management System (REMS) in March 1998. The Office of Housing is using REMS to collect and monitor data related to all multifamily structures in the department. In addition to the work completed under the 1993 FSI plan, the Office of the CFO has developed and deployed a consolidated HUD-wide general ledger for fiscal year 1999 that will include summary transactions for the department, including FHA and the Government National Mortgage Association; and developed an interface to the Office of Public and Indian Housing’s Section 8 HUDCAPS.

The Office of the CFO developed and deployed the first phase of the department’s Executive Information System (EIS). This system was prototyped using selected data from HUD’s program and financial systems.

The Office of the CFO deployed HUD’s Community 2020 geographic information system (GIS) to provide program and management information in a geographical referenced format to users of HUD’s programs.

During a November 9, 1998 meeting, the Deputy Secretary and CFO told us that HUD is in the process of assessing the conceptual design for a departmental grants management system. While the study has not yet been completed, the officials stated that if a departmental grants management system is deployed, some of the functions performed by IDIS may be replaced by the new system.

Status of 1997 FSI Strategy Implementation

Project management plans help managers monitor projects and ensure that activities are completed within specified costs and schedules. HUD’s system development methodology specifically requires that project plans be developed to document project activities and cost and schedule estimates before a project is initiated. The methodology also requires that project plans be updated if project objectives change or significant budget or schedule variances occur.

Although HUD extended the HUDCAPS implementation date by 13 months from September 1998 to October 1999, the department does not yet have a final project plan that shows whether it can successfully deploy the core financial system and integrate it with mixed systems by the new target date. The 1998 HUDCAPS project plan included tasks, costs, and schedules for fiscal year 1998 activities, but the 1999 plan does not include a schedule that shows key milestones, tasks, task dependencies, and a critical path demonstrating how and when fiscal year 1999 activities necessary to integrate HUDCAPS with the mixed systems will be completed by October 1999. For example, the HUDCAPS project plan does not show how or when the FHA Financial Data Warehouse will be interfaced to the core financial system. In addition, HUD has not yet finalized project plans that are necessary to establish new milestones for FSI projects, such as (1) GEMS, which missed its initial scheduled completion date, and (2) FHAMIS, which will not meet its initial scheduled completion date.These plans should include tasks, task dependencies, and a critical path, as well as development and deployment cost and schedule estimates for individual FSI projects.

According to the Director of IRM Planning and Management, the department required that detailed project plans be developed for each FSI project by September 30, 1998. However, as of October 19, 1998, these plans had not yet been finalized. In addition, the Director of IRM Planning and Management expressed concern over the quality of the project plans that had been submitted. This is an important matter since the department has spent hundreds of millions of dollars on FSI and expects to deploy an integrated core financial management system that will rely extensively on data from the mixed systems by October 1999.

Ineffective Management and Oversight Have Contributed to Cost Increases and Schedule Delays

Ineffective project management and oversight have contributed to numerous problems resulting in FSI cost increases and schedule delays. In 1994, we reported that HUD did not adequately oversee the planning and development of individual FSI projects. As a result, the first two FSI projects suffered delays and rising project costs. To resolve these problems, we recommended that HUD strengthen the management and oversight of individual FSI projects to ensure that significant problems would be brought to the attention of senior managers and corrected in a timely manner. We stated that these measures must continue throughout the integration effort.

Between 1993 and 1997, HUD formed various committees to strengthen project management and the oversight of projects initiated under the 1993 FSI plan. Nevertheless, we found that ineffective project management and oversight continued to contribute to cost increases and schedule delays on individual projects. For example, the Single Family Acquired Asset Management System (SAMS) replacement system which was developed and deployed as part of the FHAMIS 1993 FSI project, was delivered late and over budget and did not meet critical user needs because it was poorly managed. HUD estimated that the SAMS replacement system would be developed and deployed for about $3.2 million in 6 months. However, HUD awarded the contract to develop SAMS before adequately defining the system’s requirements. As a result, the cost of SAMS grew tenfold to over $32 million, the system was deployed 10 months late, and the system did not meet some critical user needs. To meet these needs, HUD was forced to spend an additional $8 million to enhance the system.

In April 1997, the HUD OIG also cited inadequate project management and oversight as factors that contributed to the cost increases and schedule delays for several projects initiated under the 1993 FSI plan. The OIG recommended that the Deputy Secretary take over the direction of FSI to provide the needed management oversight and ensure that project managers receive adequate project management training. In February 1998, HUD responded to these recommendations by establishing new management teams to strengthen the oversight of FSI projects and increasing its project management training program. However, as discussed below, HUD lacks (1) the essential disciplined processes required to effectively manage and oversee FSI projects and other information technology investments and (2) objective data to identify and resolve problems as they arise.

FSI Projects Are Not Being Managed as Investments

The department is not using recognized best practices for selecting, controlling, and evaluating its investments as required by the Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995. The problems HUD has experienced in developing and deploying an integrated financial management system are a direct result of not managing information technology projects properly as investments.

HUD’s investment selection process is not complete and has not provided decisionmakers with key information necessary to make investment decisions and monitor investments. For example, decisionmakers have not had reliable, up-to-date information on project costs, benefits, and risks to make well-informed decisions. Further, HUD lacks an adequate process for monitoring and controlling its FSI investments and does not have a process for evaluating FSI information technology investments once they have been completed. Therefore, the department cannot fully (1) determine whether its investments have achieved expected benefits, (2) identify whether major differences have occurred between actual and expected results in terms of cost, schedule, and risks, or (3) revise its investment management processes on the basis of lessons learned. As a result, the department does not know whether it is making the right investments, how to control these investments effectively, or whether these investments have provided expected mission-related benefits within estimated costs.

In reviewing HUD’s investment management process, we also found that the preparation of software cost estimates—key data required to make good investment decisions—is not consistent with best practices. Also, HUD does not follow best practices requiring that cost-benefit analyses be updated to reflect the current status of investments.

Legislative Reforms and Best Practices Require a Disciplined Process to Select, Control, and Evaluate Investments

The Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995 require agency heads to implement an approach that maximizes the value and assesses and manages the risks of information technology investments. The acts stipulate that this approach be integrated with the agency’s budget, financial, and program management processes.

An information technology investment process is an integrated approach that provides for data-driven selection, control, and evaluation of information technology investments. The investment process is comprised of three phases. The first phase involves selecting investments using quantitative and qualitative criteria for comparing and setting priorities for information technology projects. The second phase includes monitoring and controlling selected projects through progress reviews at key milestones to compare the expected costs, risks, and benefits of earlier phases with the actual costs incurred, risks encountered, and performance benefits realized to date. These progress reviews are essential for senior managers to decide whether to continue, accelerate, modify, or terminate a selected project. The third phase involves a post-implementation review or evaluation of fully implemented projects to compare actuals against estimates, assess performance, and identify areas where future decision-making can be improved. Overall, information from one phase is used to support activities in other phases.

Reliable cost estimates are also needed to allow effective investment decision-making. OMB’s Circular A-130 requires agencies to prepare cost-benefit analyses for system development projects and update them as necessary throughout the life of the systems.

HUD’s Process for Selecting Investments Is Inadequate

As stated in our investment guide, proposed investments should be screened to ensure that they meet minimum acceptance criteria, such as a return-on-investment thresholds, linkage to an organization’s strategic objectives and compliance with an organization’s information technology architecture. Projects that pass the screening process undergo an in-depth analysis. To help make good decisions on information technology investments, best practices require that the in-depth analysis be based on accurate, reliable, up-to-date project information. This information includes cost-benefit analyses, risk assessments, and implementation plans for both new and ongoing projects. Once the information is analyzed, projects are ranked based on their relative benefits, costs, and risks. This ranking should determine which projects should be funded and is the essence of information technology portfolio analysis. After investment decisions have been made, schedules should be established at key milestones to regularly monitor and track the cost, schedule, benefits, and risks of selected projects.

In 1997, HUD implemented a new process to improve how the department screens, ranks, and selects information technology investments. First, proposed investments were screened to determine if they met explicit criteria described in the department’s strategic ranking mechanism document. Although these criteria called for information on the duration of the project, cost for fiscal year 1997 through full deployment, technical risks, and impact on HUD’s mission and customer needs, the screening criteria did not include return-on-investment thresholds or full life-cycle cost estimates as required by OMB’s guidance on evaluating information technology investments.

Investment proposals were then analyzed, scored and ranked using the same criteria and data that were used in the screening process. However, the screening criteria did not require accurate and complete data on life-cycle costs, benefits, risks, project schedules or the corresponding analyses that were conducted to develop these estimates. For example, after reviewing investment proposal data for six FSI projects, we found that only two included life-cycle cost estimates and none included cost-benefit analyses or risk mitigation plans. Therefore, HUD made investment decisions without the information needed for a thorough understanding of the projects to make the necessary trade-offs among them. Finally, HUD’s selection process was also insufficient since it did not establish project review schedules for selected projects as required by best practices.

In fiscal year 1998, HUD did not use its selection process because the Secretary required that investment decisions be based on whether the proposed projects supported the department management and organizational changes called for in the HUD 2020 Management Reform Plan. As in fiscal year 1997, decisions made in 1998 were not based on reliable estimates of life-cycle costs, benefits, and return on investment. HUD is using its fiscal year 1997 selection process to make fiscal year 1999 investment decisions. In addition, HUD has deployed the Information Technology Investment Portfolio System (I-TIPS)—a generic system developed by the Department of Energy—to automate and support the management of information technology capital planning and integrate this planning with the department’s budget process. HUD is using I-TIPS to support its processes to screen, score, and select information technology proposals for fiscal year 1999 and plans to use I-TIPS when making investment decisions for fiscal year 2000. However, because HUD has not corrected its selection process and still does not require complete, accurate, and current data to select information technology investments, there is no assurance that HUD’s 1999 investments decisions will be better than they have been in the past.

HUD Lacks Disciplined Processes to Control and Evaluate Selected Information Technology Investments

Once information technology projects are selected, they should be consistently monitored and controlled through progress reviews at key milestone dates. Progress reviews should assess several aspects of the project, including deliverables, methodology, technical issues, schedule, costs, benefits, and risks. Further, once a project has been fully implemented, it should be evaluated through post-implementation reviews. The post-implementation reviews should provide (1) a project assessment, including an evaluation of customer/user satisfaction and how well the project met its estimated cost and schedule and provided mission-related benefits, and (2) lessons learned so that the investment decision-making processes can be improved.

HUD does not have an adequate process to control investments or a process to evaluate investments. In 1997, the Technology Investment Board working group was established to monitor approved projects and advise the Technology Investment Board executive committee whether to continue, modify, or terminate them. However, the Director for IRM Planning and Management stated that the working group mostly monitors annual project expenditures and the rate of expenditure for any given fiscal year. This degree of oversight is not adequate because it is not based on the project-specific measures required to effectively monitor and control information technology projects. These measures include (1) an accumulation of actual cost data and comparisons to estimated cost levels, (2) a comparison of the estimated and actual schedule, (3) a comparison of expected and actual benefits realized, and (4) an assessment of risks. The information should be regularly collected, updated, and provided to decisionmakers to support effective project monitoring.

The department also lacks a method for evaluating investments and thus does not perform post-implementation reviews or use lessons learned to improve the investment process. HUD’s Director for IRM Planning and Management acknowledged these weaknesses in both the control and evaluation phases of the investment process and added that HUD plans to define these processes by the spring of 1999, before it deploys future releases of I-TIPS. Without processes to control and evaluate investments, HUD cannot (1) determine if projects should be modified, continued, accelerated, or terminated, (2) determine whether a project has met its objectives, (3) compare projected costs and schedules to actual costs incurred and implementation dates, and (4) identify ways to modify or improve its investment management process.

HUD’s Cost Estimating Processes Do Not Meet Accepted Practices

Reliable cost estimates are essential for making effective information technology investment decisions. The reliability of cost estimates is dependent on the thoroughness and discipline of an organization’s estimating processes. Consistently producing reliable estimates requires defined institutional processes for deriving cost estimates, archiving them, and measuring actual performance against them.

Based on its research of leading government and private-sector estimating practices, Carnegie Mellon University’s Software Engineering Institute(SEI) identified six requisites for developing cost estimates. According to SEI, an organization must have all six requisite processes to consistently produce reliable cost estimates. These requisites are the following: a corporate memory (or historical database), which includes cost estimates, revisions, reasons for revisions, actuals, and relevant contextual information; structured processes for estimating software size and the amount and complexity of existing software that can be reused; cost models calibrated and tuned to reflect demonstrated accomplishments on similar past projects; audit trails that record and explain the values used as cost model inputs; processes for dealing with externally imposed cost or schedule constraints in order to ensure the integrity of the estimating process; and data collection and feedback processes that foster capturing and correctly interpreting data from work performed.

The Director of HUD’s Systems Engineering Group stated that the department’s processes do not satisfy SEI’s software cost estimating criteria. As shown in table 4, HUD’s cost estimating processes for FSI projects partially meet one, but do not meet the remaining five institutional process requisites that experts say are embedded in leading information technology development and acquisition organizations.

According to the Director of the Systems Engineering Group, HUD uses its experience in working with the program offices on software development efforts, rather than cost models, to develop cost estimates. The director acknowledged that HUD does not have an automated historical database to use when developing estimates for new FSI projects; instead, separate project files are kept with historical data on individual projects. The director was unsure of the usefulness of these files because they are not updated to identify and correct inconsistencies. Finally, HUD does not update or regularly review its initial cost estimates. As a result, HUD does not have adequate assurance that FSI cost estimates are consistently reliable. This increases the risk of poor FSI investment decisions throughout the project’s life cycle and the likelihood of additional cost overruns.

HUD’s Cost-Benefit Analyses Are Not Updated

OMB Circular A-130 requires agencies to prepare cost-benefit analyses for systems development projects and update them as necessary throughout the life of the systems. HUD’s systems development methodology has required the preparation and updating of cost-benefit analyses since at least September 1992.

In reviewing three cost-benefit analyses for ongoing FSI projects, we found that none had been updated as required in OMB Circular A-130. According to several FSI project managers, cost-benefit analyses are performed only once—to initiate a new information technology project. The project managers stated that the analyses are not updated, although they do prepare yearly project funding requests as part of the budget process. These requests, however, do not reflect any changes to the costs or benefits of a project. Therefore, HUD cannot compare current cost estimates and actual expenditures to determine whether unfavorable cost or benefit variances exist. As a result, HUD may continue to invest in a system without knowing whether costs or benefits have changed enough to warrant discontinuing further investment.

The Office of the Inspector General found similar problems in 1996 and recommended that HUD’s Office of Information Technology establish guidance and define management responsibilities for updating the cost-benefit analysis at appropriate intervals. HUD responded to this recommendation by stating that its September 1995 Benefit/Cost Analysis Methodology, Volume I and Benefit/Cost Analysis Workbook, Volume II define and guide the development of the required components of a cost-benefit analysis and management’s responsibility for periodically updating an analysis. The requirement to use and document cost-benefit analyses in accordance with the methodology and workbook was included in the March 1997 revision of HUD’s system development methodology.

According to HUD’s IRM Director for Planning and Management, although the system development methodology requires the use of both the cost-benefit analysis methodology and the workbook, the department has not officially mandated the use of either one. In addition, several FSI project managers told us that these standards are generally not followed. The director added that the quality, depth, and documentation supporting cost-benefit analyses for FSI projects have been inconsistent. For example, we found the IDIS cost-benefit analysis was well documented and included a discussion of the assumptions and constraints used in performing the analysis, information on recurring and nonrecurring costs, and the estimated life-cycle cost of the system. In contrast, we found that the cost-benefit analysis for the Office of Fair Housing and Equal Opportunity Grants Evaluation Management System was inadequate because it did not quantify benefits.

Year 2000 Computing Crisis May Further Impact FSI

FSI cost and schedule estimates may be impacted by HUD’s Year 2000 program, a priority effort that must be completed on time. In March 1998, we reviewed the status of HUD’s Year 2000 effort and reported that 42 of 63 mission-critical systems were not yet Year 2000 compliant. HUD has attempted to mitigate its Year 2000 risks, but three mission-critical FHAMIS systems undergoing renovations, testing, and certification are behind schedule. To better ensure that these mission-critical systems are corrected on time, HUD suspended systems integration work on these systems so that the department could focus its resources on completing Year 2000 software renovations. According to the project manager, this will cause a major impact to the schedule for completing the FHAMIS systems integration work.

In commenting on this report, HUD stated that it successfully completed all of its Year 2000 renovations for both mission-critical and nonmission-critical systems. HUD expects to complete the Year 2000 certification and validation process by January 31, 1999.

Conclusions

HUD has spent hundreds of millions of dollars on its efforts to develop and deploy an integrated financial management system over the past 7 years. While this effort has not yet been completed, the department has developed and deployed various modules and systems for 12 of the 14 different projects initiated under the 1993 and 1997 FSI strategies. The department, however, does not have the rigorous processes needed to accurately determine how much more it will cost or how much longer it will take to achieve the FSI objective, whether its efforts to date have achieved expected results, or whether its latest strategy is cost beneficial. HUD has not yet finalized project plans or cost and schedule estimates for completing all of the components of the latest FSI plan. Without such plans, the department is likely to continue to spend millions of dollars more, miss milestones, and still not fully meet its objective of developing and fully deploying an integrated financial management system.

Cost increases and schedule delays have been caused by (1) changes to the FSI strategy that were not supported by thorough analyses and (2) inadequate project management and oversight. In addition, the Year 2000 computing crisis has impacted the schedule for the FHAMIS effort. Further, HUD’s latest actions to establish new FSI management teams and increase its project management training program do not address and cannot correct the root cause of the problems—the lack of a data-driven management process to properly oversee and control information technology investments such as FSI.

HUD has not yet implemented a disciplined investment management process to select, control, and evaluate FSI projects in accordance with industry best practices and as required by the Clinger-Cohen Act and the Paperwork Reduction Act. In the absence of such a process, HUD decisionmakers (1) continue to make FSI investment decisions without reliable, complete, and up-to-date data on expected and actual costs, benefits, and risks, (2) cannot adequately monitor and control investments and detect and correct problems early, and (3) cannot evaluate completed projects to determine whether they have achieved expected benefits and improve the investment management process based on lessons learned. Also, HUD does not have well-defined, structured cost estimating processes that are in accordance with industry best practices for developing reliable software cost estimates. Finally, HUD does not follow best practices since it does not require that cost estimates or cost-benefit analyses be updated periodically for decision-making purposes.

Recommendations

In order to strengthen FSI management and oversight and HUD’s information technology investment management decisions, we recommend that the Secretary of Housing and Urban Development ensure that the department takes the following actions:

Prepare complete and reliable estimates of the life-cycle costs and benefits of the overall 1997 FSI strategy and individual FSI projects. In addition, HUD should finalize the detailed project plan for the core financial management system (HUDCAPS) to establish the milestones, tasks, task dependencies, a critical path, and staffing requirements and demonstrate that it is cost-effective to meet the October 1999 scheduled implementation date called for in HUD’s 2020 Management Reform Plan and finalize detailed project plans for individual FSI projects (mixed systems) that establish the milestones, tasks, task dependencies and critical paths, and staffing requirements to complete the 1997 FSI strategy.

Fully implement and institutionalize a disciplined and documented process consistent with provisions of the Clinger-Cohen Act and the Paperwork Reduction Act, as well as our and OMB’s guidance for selecting, controlling, and evaluating information technology investments. This process should, at a minimum, include steps to select information technology investments based on complete, accurate, reliable, and up-to-date project-level information, including estimated life-cycle costs, expected benefits, projected schedule, and risks; conduct formal in-process reviews at key milestones in a project’s life cycle—including comparing actual and estimated project costs, benefits, schedule, and risks—and provide these results to decisionmakers, who will determine whether to continue, accelerate, modify, or terminate FSI projects; and initiate post-implementation reviews within 12 months of deployment to compare completed project cost, schedule, and benefits with original estimates and provide the results of these reviews to decisionmakers so that improvements can be made to HUD’s information technology investment and management processes.

Develop and use defined processes for estimating FSI costs. At a minimum, these processes should include the following SEI requisites: a corporate memory (or historical database), which includes cost and schedule estimates, revisions, reasons for revisions, actuals, and relevant contextual information; structured processes for estimating software size and the amount and complexity of existing software that can be reused; cost models calibrated to reflect demonstrated accomplishments on audit trails that record and explain the values used as cost model inputs; processes for dealing with externally imposed cost or schedule constraints in order to ensure the integrity of the estimating process; and data collection and feedback processes that foster capturing and correctly interpreting data from work performed.

Agency Comments and Our Evaluation

In commenting on a draft of this report, HUD agreed that the management and oversight of FSI could be improved by fully implementing and institutionalizing the provisions of the Clinger-Cohen Act and the Paperwork Reduction Act. In this regard, HUD agreed with our recommendations to implement defined processes for selecting, controlling, and evaluating its information technology investments and for estimating costs. The department also said it agreed that it needs to prepare complete life-cycle costs and benefits estimates for its systems strategy, but it did not specifically address our recommendation to finalize the detailed project plans for HUDCAPS and other individual FSI projects included in the 1997 strategy.

HUD expressed concern that the $540 million FSI estimate through fiscal year 1999 mentioned in our draft report included non-FSI costs and that a more accurate FSI estimate would be approximately $255 million. As noted in our report, HUD has not yet finalized the plans, cost, and schedule to complete its current FSI strategy and, therefore, FSI costs continue to be uncertain. Accordingly, HUD’s estimates through September 1999 have fluctuated considerably, as reflected in various documents received from the CFO and his staff. For example, cost estimates have changed from $540 million reported by HUD in June 1998, to $255 million cited in the department’s November 12, 1998 comments to our draft report, to $239 million that HUD reported a week later. However, we found that the $255 million and the $239 million estimates do not include at least $132 million associated with maintaining FSI systems. HUD’s continuing uncertainty as to what is the FSI cost estimate through September 1999 further demonstrates the department’s need to develop and use well-defined cost estimating processes to prepare reliable cost estimates.

HUD said our report does not properly compare like systems when making year-to-year comparisons. The question we were asked to address was to identify the initial objectives, development, deployment and maintenance costs, and completion dates for HUD’s FSI effort and how they have changed. In order to respond to that question, we describe the systems and the estimated systems costs that were included as part of the three plans and strategies for achieving integrated financial management systems and carefully explain that HUD’s underlying strategy to implement an integrated financial management system has changed three times. In addition, to avoid any misunderstandings, we added language to clarify what the estimates for the FSI strategies and the expected FSI costs include through fiscal year 1999.

Finally, HUD described its FSI accomplishments and stated that our conclusions do not summarize or emphasize the importance of actions taken to improve its mission-critical financial management systems. To address this issue, we noted the actions taken by HUD to date and added information to our discussion of various FSI systems throughout the report.

We are sending copies of this report to the Vice Chair and the Ranking Minority Member of the Subcommittee on Housing and Community Opportunity, House Committee on Banking and Financial Services, and the Chairman and Ranking Minority Member of the Subcommittee on Human Resources, House Committee on Government Reform and Oversight. We are also providing copies to the Secretary of Housing and Urban Development and the Director of the Office of Management and Budget. We will make copies available to others upon request.

Please contact me at (202) 512-6253 or by e-mail at willemssenj.aimd@gao.gov if you have any questions concerning this report. Major contributors to this report are listed in appendix III.

Objectives, Scope, and Methodology

Our objectives were to identify (1) the initial objectives, development, deployment and maintenance costs, and completion dates for HUD’s FSI effort and how they have changed, (2) the factors that have contributed to FSI cost increases and schedule delays, and (3) whether HUD is following industry best practices and has implemented provisions of the Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995 required to manage FSI projects as investments. We were also asked to identify whether HUD’s Year 2000 program would impact its FSI activities.

To identify the objectives, development and deployment costs, and completion dates for HUD’s initial FSI effort and how they have changed, we reviewed the 1991, 1993, and 1997 FSI plans. To identify initial and revised cost and schedule estimates for major FSI projects, we reviewed initial cost-benefit analyses, project plans, budget documents provided to the OMB and FSI cost estimates for fiscal years 1998 and 1999 provided by HUD’s CFO. To identify cost and schedule estimates to complete HUD’s 1997 FSI strategy, we met with program managers for each of the major systems integration projects and representatives from the Office of the CFO. We also reviewed the fiscal years 1998 and 1999 project management plans to deploy an integrated core financial management system (HUDCAPS).

To determine whether FSI costs had increased, we reviewed the (1) initial FSI development and deployment cost estimates reported in both the 1991 and 1993 FSI plans, (2) OMB budget submissions, and (3) CFO’s reports on actual systems integration expenditures between fiscal years 1992 and 1997 and development, deployment and maintenance cost estimates for fiscal years 1998 and 1999. To determine whether schedule delays had occurred, we identified the initial scheduled deployment dates for major FSI projects—including HUDCAPS, FHAMIS, GEMS, IBS, IDIS, and TRACS—and met with their respective project managers to determine whether those dates had been or would be met. We met with project managers for FHAMIS, GEMS, HUDCAPS, IBS, IDIS, and TRACS and officials from the Office of the CFO and reviewed audit reports to determine what factors had contributed to FSI cost increases and schedule delays. We reviewed HUD’s responses to audit recommendations to determine whether HUD had taken any actions to address management problems. Further, we discussed these actions with FSI project managers and with OIG officials to determine whether or not they had been effectively implemented. To determine whether addressing Year 2000 requirements would impact FSI cost and schedule estimates, we met with project managers for individual FSI projects. We also reviewed reports on the status of HUD’s Year 2000 effort to determine whether this effort would affect the development and deployment schedule of any FSI project.

To determine whether HUD was following best practices in managing FSI projects as investments, we compared HUD’s information technology investment procedures and information resources management policies with criteria in our guidance Assessing Risks and Returns: A Guide for Evaluating Federal Agencies’ Information Technology Investment Decision-making (GAO/AIMD-10.1.13, February 1997), OMB’s guidance Evaluating Information Technology Investments: A Practical Guide (November 1995), and OMB’s Capital Programming Guide (July 1997), as well as provisions of the Clinger-Cohen Act of 1996 and the Paperwork Reduction Act of 1995. We determined whether HUD was following best practices for selecting investments by reviewing (1) criteria used to make information technology investment decisions during fiscal years 1997 through 1999, (2) documents for individual FSI projects that were used to make investment decisions for fiscal years 1997 and 1998, (3) HUD’s information technology investment portfolio for fiscal years 1997 and 1998, and (4) minutes from the Technology Investment Board working group, which document meetings on FSI investments. Further, we met with key officials of HUD’s Office of the CFO and Office of Information Technology to obtain additional details on the investment management process and the department’s plans to implement the Information Technology Investment Portfolio System.

In addition, we compared the processes and practices HUD used to develop FSI project cost estimates with the key components of cost estimating practices publicized by Carnegie Mellon University’s SEI. We also reviewed cost-benefit analyses for several major FSI projects and met with FSI project managers to determine whether these analyses had been updated, as required by OMB Circular A-130.

We did not independently verify the accuracy of FSI cost or schedule data provided by HUD. Also, the scope of our review was not intended to, and does not, provide a basis for concluding whether or not HUD’s FSI efforts will achieve their intended results.

Comments From the Department of Housing and Urban Development

The following are GAO’s comments on the Department of Housing and Urban Development’s letter dated November 12, 1998.

GAO Comments

1. As discussed in the “Agency Comments and Our Evaluation” section, HUD agreed with most of our recommendations. HUD also stated that the department has had a structured process in place since 1990 for selecting information and technology investments and monitoring the major system development through the Technology Investment Board Executive Committee, which is chaired by the Secretary. We reviewed HUD’s recent selection and control processes beginning with fiscal year 1997 and found that both processes are incomplete and inadequate to make sound investment decisions and properly manage selected investments. The major deficiencies we found with HUD’s processes were that (1) investment decisions were made without reliable, complete, up-to-date project level information and (2) project oversight was not based on project-specific measures required to effectively monitor and control information technology projects. 2. HUD provided us with a copy of the HUDCAPS project plan for fiscal year 1998 activities and a plan for fiscal year 1999 activities, but the second year project plan was not presented to us as a draft. Furthermore, as we discuss in the report, the fiscal year 1999 HUDCAPS plan was not complete because it did not include a schedule that showed key milestones, tasks, task dependencies, and a critical path demonstrating how HUDCAPS would be completed and interfaced with the mixed systems by October 1999. 3. We added a sentence to the conclusions that summarizes the status of HUD’s FSI effort to date, and we expanded the report’s discussion of individual FSI projects to reflect the new information provided by HUD. 4. We revised the report to indicate that HUD reported that it completed Year 2000 renovation work for all of its mission-critical and nonmission-critical systems. 5. We incorporated additional language in our report to avoid any misunderstanding between what is included in (1) estimates for the FSI plans and (2) expected FSI costs through fiscal year 1999. 6. Discussed in the “Agency Comments and Our Evaluation” section. As noted in our report, HUD has not yet finalized the plans, cost, and schedule to complete its current FSI strategy and, therefore, FSI costs continue to be uncertain. In addition, HUD’s FSI cost estimate through September 1999 has varied considerably, as reflected in various letters received from the CFO. For example, FSI cost estimates have changed from $540 million reported by HUD in June 1998, to $255 million reported on November 12, 1998, to $239 million reported a week later. However, the $255 million and $239 million estimates do not include at least $132 million in maintenance costs. HUD’s continuing uncertainty regarding the FSI cost estimate through September 1999 further demonstrates the department’s need to develop and use well-defined cost estimating processes for preparing reliable FSI cost estimates. Finally, as we note in appendix I, we did not independently verify the accuracy of FSI cost data provided by HUD.

HUD’s statement that the $103 million for the 1991 FSI strategy includes development costs only is inconsistent with its 1991 FSI plan, which states that the $103 million included both development and deployment costs.

Major Contributors to This Report

Accounting and Information Management Division, Washington, D.C.

David G. Gill, Assistant Director Yvette R. Banks, ADP Telecommunications Analyst-in-Charge Madhav S. Panwar, Senior Technical Advisor Teresa L. Jones, Issue Area Assistant The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.