Information Technology: HUD Can Take Additional Actions to Improve Its Governance
Released: 2014-12-10T00:00:00

Letter

Background

To accomplish its mission, HUD administers community and housing programs that affect millions of households each year. Among other things, the department provides affordable rental housing opportunities and helps homeless families and chronically homeless individuals and veterans. The department also administers mortgage insurance programs for single-family housing, multifamily housing, and health care facilities.

HUD is organized into the following five main organizational components:

Housing/Federal Housing Administration: Programs within this office are responsible for contributing to building healthy communities, maintaining and expanding housing opportunities, and stabilizing credit markets in times of economic disruption. This office also regulates certain aspects of the housing industry.

Community Planning and Development: The office’s mission is to provide decent housing, a suitable living environment, and economic opportunities for people of low and moderate income. To accomplish this, it engages in partnerships with all levels of government, the private sector, and nonprofit organizations.

Fair Housing and Equal Opportunity: The office’s mission is to administer and enforce federal laws that prohibit discrimination in housing, such as the Fair Housing Act and the Civil Rights Act of To accomplish this, the office establishes and enforces policies 1964.intended to ensure that all Americans have equal access to housing of their choice.

Public and Indian Housing: Programs within this office are responsible for creating opportunities for residents’ self-sufficiency and economic independence. Toward this end, this office currently oversees a housing choice voucher program to subsidize housing for approximately 2.2 million low-income families; a public housing program that subsidizes about 1.3 million housing units for vulnerable low-income families; and block grants and guarantee programs for Native American groups.

Government National Mortgage Association (Ginnie Mae): This HUD-owned corporation’s mission is to provide support for affordable housing by bringing global capital into the housing finance market while minimizing risk to the taxpayer. While Ginnie Mae does not issue loans or mortgage-backed securities, it guarantees investors timely payment and interest on mortgage-backed securities supported by federally insured or guaranteed loans.

HUD also consists of a number of program offices, such as the Office of Healthy Homes and Lead Hazard Control and the Office of Faith-Based and Neighborhood Partnerships. In addition, the department has administrative offices that report to executive leadership through the Chief Operating Officer, such as the Office of Strategic Planning and Management and the Office of the Chief Information Officer (CIO).

A simplified view of the department’s organization structure is provided in figure 1.

HUD’s Deputy Secretary is responsible for managing the department’s daily operations, annual operating budget, and approximately 8,700 employees. As part of this role, the Deputy Secretary conducts meetings with stakeholders to discuss the Secretary’s priorities. During these meetings, the scope, milestones, risks, and status of action items related to priority issues are discussed.

Overview of HUD’s IT Environment

IT plays a critical role in the ability of the department’s organizational components to perform needed business functions. For example, HUD’s IT environment consists of multiple systems that, among other things, are intended to help the department coordinate with lending institutions to insure mortgages, collect and manage state and local housing data, process applications for community development, and process vouchers for different rental assistance programs. Its systems also support the processing of applications for, and the management of, more than 50 grant programs administered by the department.

However, according to the department, its IT environment has not been sufficient to effectively support its business operations because its systems are overlapping and duplicative, not integrated, necessitate manual workloads, and employ antiquated technologies that are costly to maintain. For example, the department has reported that its environment consisted of:

Over 200 information systems, many of which perform the same function and, thus, are overlapping and duplicative. Specifically, different systems perform the same task to separately support grants management, loan processing, and subsidies management.

Stove-piped, nonintegrated systems that result in identical data existing in multiple systems. For example, two organizational components store about 80 percent of similar data in separate databases that provide information on rental assistance participants.

Manual processing for business functions due to a lack of systems to support these processes. For example, specific Public and Indian Housing IT projects are intended to replace existing ad hoc analyses performed in spreadsheets and databases with systems that automate and standardize those functions.

Antiquated technology (15 to 30 years old) and complex systems that are costly to maintain. For example, the department relies on different obsolete programming languages and operating systems, which require specialized skills to operate and maintain.

Further, contractors engaged by HUD to assess the department’s environment reported in January 2011 that unclear reporting relationships hindered the enforcement of IT policies; contractor performance information was not used to inform management decisions; technical standards were lacking or not enforced; and data management practices did not support business needs.

HUD IT Management and Governance Structure

In October 2011, the Secretary of HUD delegated responsibility for the management of IT resources and for meeting requirements established by the Clinger-Cohen Act of 1996 to the department’s Chief Information Officer (CIO). Among other duties, the CIO was charged with promoting the effective and efficient design and operation of all major IT processes for the department; designing, implementing, and maintaining processes (consistent with the roles and responsibilities of governance boards) for maximizing the value and assessing and managing the risks of IT acquisitions; and advising the Secretary and governance boards regarding whether to continue, modify, or terminate programs or projects.

To guide the department’s efforts in this area, the CIO developed a management framework and policies, including a policy for governance, based on the IT investment management framework developed by GAO and other federal guidance. In addition, to establish management and governance capabilities, the Office of the CIO chartered four investment review boards in 2011:

The Executive Investment Board is to be made up of senior executives, including the department’s Secretary (chair), Deputy Secretary, Chief of Staff, Chief Operating Officer, Chief Financial Officer, General Counsel, and CIO. As the department’s highest-level board, the executive board is charged with responsibility for, among other things, overseeing and approving HUD’s high-cost, high-risk projects, allocating resources, and monitoring enterprise risk.

The Customer Care Committee is made up of executives including the CIO, the Chief Procurement Officer, Chief Human Capital Officer, and deputy assistant secretaries. The responsibilities of this committee include reviewing and submitting investment recommendations to the Executive Investment Board.

The Investment Review Subcommittee is made up of business area personnel from across the department, including representatives from the Office of the CIO, Fair Housing and Equal Opportunity, and Public and Indian Housing, who focus on investment management oversight with respect to business cases and budget information for the Office of Management and Budget (OMB).

The Technical Review Subcommittee consists of personnel from within the Office of the CIO, including the Chief Technology Officer, the Chief Architect, and the Chief Information Security Officer, and is focused on providing technical, project, and architecture subject matter expertise.

The hierarchy of HUD’s investment review boards is depicted in figure 2.

The CIO and the investment review boards are supported by staff in the Office of the CIO. Through coordination with the other organizational components, the office manages IT resources and provides support for the department’s infrastructure, security, and ongoing projects. The office also provides project management guidance and technical expertise, and supports the department’s efforts to govern IT investments.

In 2012, to improve the Office of the CIO’s ability to implement its authorities as required by Congress and OMB, the department initiated a reorganization of the office to, among other things, streamline IT governance practices, improve the alignment of IT investments with mission needs, and ensure customer satisfaction through enhanced service delivery and continuous performance management. To date, HUD has taken steps to complete the effort, including determining the specific offices, divisions, and branches that make up the restructured office. Specifically, the department established three Deputy CIOs with responsibility for managing the following offices created within the Office of the CIO: the Office of Business and IT Resources Management, which is responsible for human and financial resource management, including funds control, budget execution, internal controls, and acquisition management for the Office of the CIO; the Office of Infrastructure and Operations, which is responsible for providing data center service, technical support, training, service delivery assistance for the network, desktop computers, and telephone and e-mail services to HUD staff in headquarters and field offices; and the Office of Customer Relationship and Performance Management, which is responsible for meeting HUD’s programmatic needs through customer engagement, marketing, product development, and capital planning.

In particular, the Office of Customer Relationship and Performance Management consists of three divisions responsible for IT management functions, including enterprise program management, investment management, and enterprise architecture.view of HUD’s restructured Office of the CIO, including the Office of Customer Relationship and Performance Management and the related divisions and branches that directly support the department’s IT governance functions.

Key Governance Practices Outlined in GAO’s IT Investment Management Framework

GAO assessed best practices to develop the IT investment management framework to provide a method for evaluating and assessing how well an agency is selecting and managing its IT resources. The framework consists of five progressive stages of maturity that an agency can achieve in its investment management capabilities. The maturity stages are cumulative; that is, in order to attain a higher stage, an agency must institutionalize all of the critical processes at the lower stages, in addition to the higher stage critical processes. Each maturity stage is composed of critical processes that must be implemented and institutionalized; these critical processes are further broken down into key practices that describe the types of activities that an organization should be performing to successfully implement the critical processes. Only after implementing key practices for each stage’s critical processes are organizations positioned to mature their investment management and move from managing individual projects to helping the organization evaluate the effectiveness of its overall portfolio of investments.

Efforts to build a foundation for IT governance involve establishing specific critical processes, such as those for instituting investment boards, selecting investments, and providing investment oversight. Further, certain key practices must be performed by an organization in order to implement and institutionalize the critical processes effectively. For example, one key practice associated with the critical process of instituting the investment board involves documenting policies and procedures to direct board activities. Table 1 provides a summary description of the specific critical processes discussed in this report, along with examples of related key practices.

HUD’s IT Governance Activities Do Not Yet Fully Address Key Practices

Since 2011, the department has taken steps to build a foundation for IT governance, including instituting investment review boards, establishing elements of a process for selecting investments, and providing investment oversight. However, the department’s governance activities do not yet fully address key practices. In particular, its boards are not operating as intended and its processes for investment selection and oversight lack essential elements. HUD faces risk that its governance decisions will not reflect the needs of the department and that it will be unable to realize planned improvements to its IT environment and systems.

HUD’s Investment Boards and Their Guiding Policies and Procedures Are Not Fully Established and Operational

The establishment of decision-making bodies or boards is a foundational component of effective IT investment management. According to the IT investment management framework developed by GAO, an organization should, among other things, establish one or more investment boards to manage and select projects, including an enterprise-wide investment board composed of senior executives that is responsible for defining and implementing the organization’s investment management process. In cases where additional investment boards are chartered to support the enterprise-wide investment board, the enterprise-wide board should remain responsible for the investment management process and be actively involved in all IT projects and proposals that are high cost or high risk or have significant scope and duration. The authorities, guiding policies, roles, responsibilities, and operations of each board should be defined to ensure that consistent and effective investment management practices are implemented across the organization. We have previously reported that effectively implementing a governance framework involves having a robust implementation plan that specifies—in addition to goals and objectives—milestones for the effort.

Consistent with the IT investment management framework developed by GAO and as described previously in this report, HUD’s Office of the CIO chartered four investment boards in 2011 with defined authorities, roles and responsibilities, and operations for managing and selecting the department’s IT projects. Also during that year, the department issued an IT management framework that was intended to provide guidance to direct the investment boards in fulfilling their investment management responsibilities.

However, the department has not ensured that its investment boards are fully operating according to their designated authority and responsibility. Specifically, HUD chartered an enterprise-wide investment board—the Executive Investment Board—that was to be composed of senior leaders from across the department, act as the highest decision-making authority, and have direct purview of the department’s most complex, costly, and visible IT projects. Nonetheless, as of October 2014, this board had never met. According to officials from the Office of the CIO, operating without a functioning Executive Investment Board reflects the Secretary and Deputy Secretary’s preferred approach to investment management. The officials stated that the Deputy Secretary has assumed the board’s designated authority and roles and responsibilities. The officials added that the Deputy Secretary determines the department’s IT priorities and selects, as needed, specific individuals to participate in discussions regarding which projects should receive funding.

As discussed above, these were the Executive Investment Board, which is to act as the enterprise-wide board; the Customer Care Committee, which is to review and submit investment recommendations to the executive board; the Investment Review Subcommittee, which is to focus on investment management oversight; and the Technical Review Subcommittee, which is to focus on providing subject matter expertise.

The lack of a functioning Executive Investment Board has affected the ability of the department’s other active investment boards–the Customer Care Committee, Investment Review Subcommittee, and Technical Review Subcommittee–to fully operate in accordance with their assigned responsibilities. Specifically, according to the department’s investment review board charters, the Executive Investment Board was to establish key criteria for these boards to use in identifying (1) which IT projects best support HUD’s strategic goals and provide value to the department and (2) which projects were underperforming and should be considered for termination. However, because the Executive Investment Board has not met or conducted any business, such criteria were not established.

As a result, instead of making funding recommendations based on criteria representing enterprise-wide goals for the fiscal year 2014 budget cycle, the Customer Care Committee, for example, recommended a set of IT projects to the Deputy Secretary for funding. According to officials from the Office of the CIO, the recommended projects were based on considerations such as the priorities of individual board members or programmatic concerns. The officials from the Office of the CIO considered this approach to be appropriate because, in their view, it helps to ensure that the concerns and goals of each member of the Customer Care Committee are considered during decision making. Regarding criteria for identifying underperforming projects, the officials stated that the department’s investment boards rarely terminate IT projects because they provide needed services that must continue until a replacement project can be found and implemented.

Further, HUD’s active investment boards are not fully adhering to the operating procedures outlined in their charters concerning meeting frequency and documentation requirements. For example, according to their charters, two of the boards—the Customer Care Committee and Investment Review Subcommittee—are expected to hold monthly meetings and document and distribute the results of the meetings to stakeholders. However, the Customer Care Committee had documented meeting minutes for just 3 months in fiscal year 2014, and the Investment Review Subcommittee had not documented any of its meetings during this time period. In addressing this matter, officials from the Office of the CIO told us that the investment boards meet as needed to fulfill their responsibilities.

Moreover, HUD has not yet developed all of the policies that support its IT management framework. Specifically, the Office of the CIO’s framework identifies 11 key policies that the office was to develop to influence and determine actions and decisions in IT management areas such as acquisitions, capital planning, and project planning. To date, the Office of the CIO has developed 8 such policies, including those previously described and policies for IT governance and enterprise architecture. However, 3 of the planned policies—for performance, privacy, and risk management—have not yet been developed, and HUD has not set a time frame for doing so. Officials from the Office of the CIO stated that they had taken a phased approach to developing the policies and did not intend to have completed the 3 outstanding policies by this time. However, without a timeline supporting this approach or a date by which the 3 outstanding policies will be completed, the department lacks assurance that it will take timely action to implement the additional policies needed to fully establish the IT management framework.

Additionally, the department’s IT management framework has not been updated to reflect significant changes to HUD’s project planning and management practices and OMB requirements for conducting reviews of projects. Officials from the Office of the CIO stated that the framework had not been updated because the department’s investment management practices have evolved every year and a definitive set of practices had not been identified. The officials added that updating the framework had not been a priority because it was written at a sufficiently high level to remain relevant. They stated that they hope to make annual updates to the framework to ensure that it remains current. However, a date by which an updated version of the framework will be completed that incorporates, among other things, OMB requirements, has not been established.

Until HUD’s investment boards operate according to their designated authority and responsibilities and criteria are established to guide investment decision making, the department cannot ensure that, instead of individual interests, a corporate responsibility is reflected in IT decision making. Additionally, the department cannot ensure that projects will be selected or terminated based on their ability to meet strategic goals and bring value to the entire organization. Further, without complete and current policies and procedures to guide its investment boards, the department lacks assurance that its investment management practices will be implemented consistently and effectively across the department.

HUD’s Investment Selection Process Does Not Yet Include Specific Key Practices

According to the IT investment management framework developed by GAO, to support well-informed investment decision making, organizations should document and implement a well-defined process for selecting new proposals and reselecting ongoing investments. Documenting and implementing the process is a basic step toward achieving mature IT project selection. Elements of such a process include key practices, such as those discussed in the IT investment management framework and cost estimation guidance developed by GAO—practices for identifying, evaluating, and prioritizing IT proposals for funding. Specifically, with respect to proposal evaluation, agencies should define how data (including cost estimates) are to be developed, verified, and validated, including detailed information to explain the basis for the cost estimate and how the estimated funding will be spent; criteria for how proposed projects are to be analyzed in terms of benefits, cost, schedule, and project risk; and a scoring mechanism that compares proposed projects to one another in terms of investment size (cost), project longevity (schedule), technical difficulty, project risk, and cost-benefit analysis, in order to help the boards analyze and prioritize projects based on their strengths and weaknesses.

Further, final selection decisions should be made by senior decision makers and should be documented and supported by the evaluation actions described above. In addition, another key practice involves predefining a method for reselecting ongoing projects for continued funding, including establishing predetermined criteria for analyzing ongoing operations and maintenance projects.

As recommended by the guidance GAO developed, HUD has established practices for identifying new IT proposals. The Office of the CIO has taken steps to identify new IT proposals by developing standardized templates for proposed IT projects. Among other things, the templates address how proposed projects will meet business needs by requiring information about strategic and agency priority goals to be addressed by projects and descriptions of specific business needs or requirements that projects are intended to support.

However, the department has not yet fully addressed key practices for evaluating and prioritizing IT proposals. With regard to IT proposal evaluation, the department has not documented how data supporting proposed projects are to be developed, verified, and validated. Specifically, the office has not established detailed requirements or standardized guidance for how sponsors of proposed projects are to develop key elements of proposals. For example, the office has not yet defined how project cost estimates should be created or evaluated, including what detailed information is required to explain the basis for the estimate and how the estimated funding will be spent. In addition, the office has not developed procedures for verifying and validating the data in proposals or required project sponsors to provide supporting documentation that the Office of the CIO could use to verify or validate the data submitted. During fiscal years 2014 and 2015, for example, the office did not validate whether cost information submitted was based on any standardized cost estimation practices. In addition, in evaluating IT proposals, the investment management division does not require deficiencies in IT proposals to be addressed before proposals can be recommended for funding and has not implemented procedures to track whether deficiencies were addressed and whether scores were improved as a result.

Further, while the Office of the CIO has established and used criteria to analyze proposed projects in terms of benefits, the criteria do not address how cost, schedule, or project risk are to be analyzed. The office established criteria for scoring proposals that are used by staff within the office and subject matter experts to score and prioritize proposals for selection and inclusion in the department’s IT budget each year. According to the criteria, to receive a high score in the mission benefits category, IT proposals should identify one or more benefits of the project that directly link to HUD’s mission and clearly define a consistent method by which benefits are measured. However, while the Office of the CIO requires that schedule milestones for contracts be provided, it has not established requirements for the reporting of planned schedules or criteria for analyzing project schedules. Further, regarding risk, although business cases are to report how projects will reduce certain department risks, the Office of the CIO has not established criteria for evaluating proposals based on project risk, including whether all relevant project risks have been identified or adequate plans for mitigating risk have been developed.

Although the Office of the CIO developed and is using a scoring mechanism, it does not allow the department to compare projects to one another in terms of investment size (cost), schedule, technical difficulty, project risk, or cost-benefit analysis in order to consider relative strengths and weaknesses when prioritizing projects. For example, in analyzing reported mission benefits in business case proposals, the investment management division scores an IT proposal based on whether it has identified mission benefits. Similarly, with regard to risk, the Office of the CIO does not incorporate analyses of project risk into its scoring. The IT management division scores a proposal based on whether it identifies technical risks the department faces that the project intends to reduce and whether there is a clear description of a strategy or plan for how such risk may be reduced. However, proposals that went beyond that requirement to identify specific project-related risks and plans for mitigating them could receive the same score as proposals that did not identify any project-related risks. Thus, proposed projects are not compared in terms of their relative benefits, risks, or potential return on investment. Further, scores are not lowered if proposals are incomplete; that is, scores are based only on the data that were reported, and any items left incomplete are not factored into the scoring. As a result, a partially complete proposal could potentially be given a similar or an even higher score than a proposal that included all of the requested information.

Moreover, the extent to which IT proposals selected by senior decision makers are supported by key evaluation practices is unclear, and the decisions are not consistently documented. In particular, officials from the Office of the CIO stated that senior executives on the boards make final decisions about which projects to fund based on qualitative factors that are not standardized. Board members are not provided with additional information such as analyses comparing the relative costs, time frames, benefits, or risks of projects being considered. Further, the Office of the CIO documents board members’ agreement on a final list of projects to be funded, but does not document the rationale used, including explanations for why projects with lower priority scores may have been funded, why any other exceptions to the initial funding priority were made, or whether supporting analysis was used by the various boards to reach a final decision, as called for by GAO’s investment management framework.

Finally, the Office of the CIO has not yet established a predefined method for reselecting ongoing projects for continued funding or defined criteria for analyzing projects that are in the operations and maintenance phase. While the office collects different data in business cases for projects in operations and maintenance, it ranks and scores such projects along with newly proposed projects. Projects are prioritized based upon their total scores—without considering whether projects are new or ongoing. Consequently, a poorly performing project that had a detailed business case proposal meeting established criteria could receive a high score and be ranked high on the list of priorities. Moreover, because the office has not established a reselection method that includes predetermined criteria for analyzing ongoing projects for continued funding, decisions to reselect projects are not consistently based on key practices such as analysis of progress or project outcomes or assessments of the potential risk or return of continuing to invest in a project. While the Office of the CIO or the investment boards may discuss ongoing projects’ progress in investment selection deliberations, without a consistent method and predefined criteria for making decisions about reselecting ongoing projects, HUD may continue to invest in projects that are not performing as needed.

Officials from the Office of the CIO acknowledged that they have not yet developed a standard and well-documented selection process for new IT proposals or for reselecting ongoing investments that fully addresses key practices, including outlining requirements for cost estimates. Officials from the Office of Strategic Planning and Management stated that new guidance for cost estimation is under development and that more robust practices will be used to create cost estimates for the fiscal year 2016 budget process. The officials attributed weaknesses in the selection process to, among other things, efforts to minimize the burden on those requesting funding, which result in limiting requirements for submitting detailed information to support project proposals; lack of an established priority for developing life-cycle cost estimates for projects (until recently); changes in senior leadership, departmental priorities, and approaches; lack of departmental oversight of the decision-making process by senior executives; and reliance on qualitative, judgmental data and inadequate consideration of key quantitative measures, such as return on investment.

Until HUD documents a complete selection process that incorporates key practices for identifying, evaluating, and prioritizing investments, the department is at risk that its selection practices will lack the standardization, transparency, and consistency needed to ensure effective decision making. In addition, key stakeholders may not have common understanding of the practices or qualitative factors considered in decision making, and executives may miss opportunities to consider the relative risks and returns and strengths and weaknesses of proposed investments. As a result, given that the final decisions are not data driven; HUD is at greater risk of not selecting the appropriate mix of IT investments that best meet its organizational and technology needs and priorities for improvement.

Elements of a Process for Investment Oversight Have Been Established but Not Yet Institutionalized

As with investment selection, organizations should have a documented, well-defined process for overseeing ongoing investments once they have been selected. Effective investment oversight and evaluation involves, among other things, documenting the process for oversight, including predefined criteria and checkpoints for reviewing the progress ongoing projects have made in meeting cost, schedule, risk, and benefit expectations; comparing actual performance against estimates; and identifying areas where future decision making can be improved. Specifically, key practices call for predefined thresholds for project performance designed to increase oversight of underperforming projects. In addition, once the project has transitioned from the development phase to the operations and maintenance phase, organizations should conduct post-implementation reviews to compare actual investment results with decision makers’ expectations for cost, schedule, performance, and mission improvement outcomes. The lessons learned from these reviews can be used to modify future investment management decision making.

Since 2011, the Office of the CIO has established elements of a process for managing individual projects after they are selected. Specifically, revisions made in 2014 to the office’s project planning and management framework require project managers to develop management plans that are intended to outline, among other things, performance expectations for projects once they are selected and initiated. The office has also assigned specific investment oversight responsibilities to two of the department’s investment boards—the Technical Review Subcommittee and the Investment Review Subcommittee. Finally, once projects are completed, the project planning and management framework requires the Office of the CIO to conduct post-implementation reviews no later than 1 year after systems are implemented to evaluate results of completed projects.

Nonetheless, the office has not yet institutionalized all of the established practices, and in some cases, has not established key practices. For example, to fulfill its oversight responsibilities, the Technical Review Subcommittee is supposed to collect certain cost, schedule, benefit, and risk data for each IT project at specific life-cycle checkpoints established by the project planning and management framework; conduct reviews at each checkpoint designed to ensure that IT projects are planned, budgeted, and scheduled in alignment with HUD’s strategic goals and approach to technology management; and provide support to other boards, such as technical and architecture analysis, in monitoring and analyzing investment performance.

As required, the Technical Review Subcommittee collects data on each IT project’s cost, schedule, benefits, and risk at established life-cycle checkpoints and conducts reviews at each checkpoint. During the reviews, the subcommittee analyzes projects’ compliance with project documentation requirements and technical and architectural standards.

However, the subcommittee is not yet positioned to fully execute its oversight responsibilities due to certain limitations. First, the project planning and management artifacts and data required for these reviews have recently been revised and the related project management practices have not yet been institutionalized. Officials in the Office of the CIO stated that efforts to institutionalize the new process and strengthen the subcommittee’s oversight function for all investments are under way, but the office has not yet established expected time frames for when these efforts will be completed. Moreover, efforts by the Office of the CIO to improve project oversight practices may continue to be constrained by project management deficiencies we identified in our 2013 report, including practices designed to provide data that could be used for investment oversight.

Second, the subcommittee has not consistently used data collected about projects to monitor progress against the expectations established. In August 2014, officials from the Office of the CIO stated that the subcommittee had begun conducting assessments for cost and schedule; however, they could not produce the results of any of these assessments or provide any evidence that such reviews had actually assessed performance against expected benefits or risks. Moreover, the Office of the CIO’s documented policies and procedures requiring reviews by the subcommittee do not establish cost-, schedule-, or performance-based thresholds that would automatically trigger remedial action or referrals to other investment boards. Officials from the office acknowledged that practices for monitoring progress had not yet matured and that the performance data currently collected and maintained at the project level needed improvement. Moreover, because of this immaturity, current efforts to monitor progress are limited. Further, without such thresholds, project oversight may not be consistent or troubled projects may not receive additional oversight by the investment boards.

Officials from the Office of the CIO also acknowledged that the office has not yet established and documented a well-defined process for investment oversight and attributed weaknesses in oversight to, among other things, limitations in its ability to develop reliable cost, schedule, and benefit estimates to use for monitoring progress and evaluating performance. Until HUD addresses gaps in its processes for monitoring progress of projects and documents policies and procedures to sustain and consistently implement oversight practices, it cannot ensure that its management of IT will achieve desired results.

As another investment oversight mechanism, the Investment Review Subcommittee is responsible, under its charter, for monitoring investment and portfolio performance and taking action to (1) terminate investments consistently experiencing variances in cost or performance or (2) bring them back within acceptable cost or performance limits. However, to date, the Investment Review Subcommittee has not fulfilled these responsibilities. Officials from the Office of the CIO acknowledged that the subcommittee has not performed these functions and said that they are reviewing the roles and responsibilities of the subcommittee to determine whether changes are needed. The officials also stated that, as part of early actions to define an oversight process, the Office of the CIO is working to implement more mature practices and developing an investment-level view of performance, including a strategy for identifying performance expectations. Specifically, the officials provided plans to develop an enterprise-wide portfolio performance management process, which includes implementing a new portfolio management tool with portfolio and project management functionality planned to be operational by February 2015.

Officials from the Office of the CIO attributed weaknesses in oversight to, among other things, lack of a consistent, enterprise-wide way to collect and compare actual cost, benefit, schedule, or risk to estimates. Without progress reviews that compare investments against estimated cost, estimated schedule time frames, and expected benefits using predefined thresholds, HUD will not have insight into whether the projects are meeting mission needs during their development or after they are completed.

Finally, the Office of the CIO has not consistently conducted post- implementation reviews to evaluate results of projects after they are completed. Although the office’s project planning and management process calls for such reviews no later than 1 year after system implementation, the office had not scheduled or conducted any such reviews until recently. In August 2014, officials from the Office of the CIO reported that the Technical Review Subcommittee had begun undertaking post-implementation reviews of its approximately 200 operational systems. They explained that, after conducting several initial reviews, the subcommittee determined that the data in system documentation available to the office were insufficient to support these reviews. As a result, the Chief Technology Officer collected additional data needed to support post-implementation reviews of all 200 operational systems. According to officials from the Office of the CIO, the reviews are scheduled to be completed by the end of November 2014.

When explaining why post-implementation reviews had not been conducted until recently, officials in the Office of the CIO said that, among other reasons, relatively few IT projects had been completed. Nonetheless, while many of HUD’s recent modernization projects remain under way and have not reached a point at which post-implementation reviews would be appropriate, establishing effective practices to collect and maintain the data needed to support such reviews remains important. The Office of the CIO will not be positioned to conduct effective post- implementation reviews until it identifies the data needed for such reviews and requires that it be collected consistently for all projects. Officials from the Office of the CIO acknowledged that the office experienced deficiencies in capturing and communicating lessons learned and translating them into revised decision-making processes. Without post- implementation reviews, HUD will not be able to compare the outcomes of completed projects with expectations.

Reported Governance-Related Cost Savings and Efficiencies Were Not Adequately Supported, and Comprehensive Data on Governance Outcomes Are Not Yet Identified or Tracked

OMB guidance requires agencies to identify and communicate anticipated cost savings and portfolio improvements realized. Specifically, the guidance requires agencies to identify and report cost savings gained from, for example, retiring low-value and duplicative investments, eliminating costly support contracts, moving to shared services such as cloud computing, and reducing IT commodity spending. Additionally, agencies are directed to seek operational efficiencies such as automating or streamlining processes and providing mobile technologies that result in improved services to taxpayers. OMB’s guidance also calls for agency leadership to use high-quality data in these efforts. Among other things, OMB’s guidance on performance reporting calls for agencies to ensure that supporting documentation is maintained and readily available, data are verified as appropriate to the needed level of accuracy, and data limitations are explained and documented. We have also reported that it is important to have assurance that the data collected and reported by agencies are complete and accurate. Specifically, we have reported that agencies should explain the procedures used to verify or validate their data and ensure that data are sufficiently complete, accurate, and consistent.

HUD’s reported cost savings and operational efficiencies were not supported by data that were complete, accurate, and consistent. To date, through various mechanisms such as its fiscal year 2014 expenditure planinformation, the Office of the CIO identified 14 instances of cost savings and operational efficiencies reportedly accomplished through selected investments and governance decisions. From its varied efforts, HUD reported about $23 million in cost savings during fiscal year 2014. Sources of the savings identified included and reports to OMB, as well as in response to our requests for initiatives to consolidate commodity IT; decisions to eliminate several system-support contracts (resulting in annual cost avoidance); projects selected and implemented to automate manual processes and program reporting requirements; and actions to deactivate three legacy systems.

However, the information reported from these four sources was deficient in the following ways.

Data were not validated: HUD did not take steps to verify or validate the data. Specifically, officials from the Office of the CIO reported that they did not verify or validate data reported or require that those reporting cost savings verify or validate the data provided.

Supporting data were not provided: The Office of the CIO could not provide documentation needed to substantiate that the cost savings data reported were reliable. Specifically, neither the list of deactivated systems nor the fiscal year 2014 expenditure plan provided the basis for savings reported, links to supporting documentation for those calculations, or clear information about when the savings were achieved. In addition, although certain reported savings could be traced to OMB requirements for reporting on specific governance activities and others could be presumed to have been generated from the investment selection process funding specific projects, it was unclear whether other savings resulted from specific governance decisions because HUD did not provide meeting minutes or other documentation of governance decisions authorizing or recognizing the savings or efficiencies.

Regarding the lack of data validation, officials from the Office of the CIO told us that they accept data as reported because they lack the resources needed to validate the data submitted. With regard to the lack of adequate supporting documentation, CIO officials acknowledged that not all governance decisions result in savings or efficiencies and that clearly linking cost savings and efficiencies to specific governance decisions is a challenge. However, the Deputy CIO recognized that Office of the CIO officials could do a better job of documenting specific decisions made by investment review boards in meeting minutes.

Additionally, the quality of the information reported about actions to deactivate legacy systems was questionable for the following reasons:

Data were not complete or accurate: HUD reported that it had deactivated 19 systems. However, the agency’s reported cost savings associated with deactivated systems were not complete. For example, savings for 10 of the systems were either listed as “unknown,” or information about estimated or actual savings associated with deactivated systems was missing. Further, 2 of the systems reported as deactivated remained operational but were moved off external contracts to in-house support.

Data were not consistently reported: Specifically, the period of time over which certain savings should extend was not consistently specified. For example, the 2014 expenditure plan attributed to a mapping system for grants management reported savings of an estimated $50,000 for each grantee every 5 years. However, for the other deactivated systems, the data did not clearly show for how many years estimated savings or actual savings would accrue.

HUD also reported during fiscal year 2014 that it had achieved specific operational efficiencies from selected IT projects under way or completed. Specific sources of reported efficiencies identified included the following:

The Office of the CIO reported that selection of and investment in human capital management systems had resulted in a number of quantified business benefits such as reductions in hiring time; improved data processing accuracy and quality; improved timeliness of performance reviews and overall communication delivery; increases in assessments of employee training needs; and an increase in the response rate for exit surveys for outgoing employees.

The office also reported operational efficiencies achieved through investments in specific projects but did not quantify the benefits reported to have been generated by those systems:

The Integrated Budget Forecasting Model and its functionality designed to continually monitor short-term funding needs and automatically detect excess funding on contracts were associated with reductions in payment delays, fewer disruptions caused by inaccurate projections of needed funding, and increased ability of multifamily housing staff to perform timely recaptures of overpayments to housing authorities and cancel excess account balances.

The Federal Housing Administration Transformation effort was reported to have increased data quality and streamlined delivery to financial services customers.

Implementation of a modernized enterprise-wide integrated acquisitions management system was reported to have reduced inefficiency and processing time and improved service delivery by eliminating duplication in the acquisition/procurement process.

However, as with reported cost savings, the quality of the reported information on operational efficiencies achieved is questionable. In this regard, the Office of the CIO could not provide evidence that operational efficiencies achieved had been verified or validated. It also could not provide supporting documentation to substantiate the reliability of the data about reported efficiencies or to show that they were supported by data that were complete, accurate, or consistent.

The questionable quality of HUD’s reported cost savings and operational efficiencies may be attributable, in part, to specific governance weaknesses previously discussed in this report. Among other things, insufficient requirements for collecting data in IT proposals about expected benefits or reporting of progress made toward those goals may limit the information available about savings and efficiencies for individual IT projects. In addition, the lack of comprehensive information for IT investment management leaves the Office of the CIO dependent upon ad hoc reporting efforts and searches for data reported for other purposes to identify cost savings and operational efficiencies. Finally, deficiencies in documentation of governance decision making may limit the office’s capacity to attribute cost savings and operational efficiencies to its IT governance practices.

According to officials from the Office of the CIO, including the Acting Deputy CIO for Customer Relationship and Performance Management and the investment management division director, HUD lacks a systematized view of cost savings and a formal written policy for identifying and reporting on cost savings and operational efficiencies. They explained that savings are not documented in one place, are not tracked, and are measured in different ways for different projects. Further, the Office of the CIO does not systematically assess whether planned savings and efficiencies (including those associated with management of its enterprise architecture) have been achieved.

Lacking a process for identifying savings and efficiencies from all of its governance decision making, the Office of the CIO cannot obtain a comprehensive view of savings and efficiencies planned for or accomplished by its portfolio of IT investments and, thus, does not have assurance that investments are delivering expected benefits. The office may also be missing opportunities to expedite investments that are producing greater than expected savings and efficiencies. Finally, without comprehensive data about expected or actual savings and efficiencies, the department will be limited in its ability to gauge the effectiveness of its overall governance efforts and report to Congress on the progress it is making in improving its IT environment.

Conclusions

HUD has made important progress in establishing IT governance practices. Specifically, the Office of the CIO has established investment review boards and guiding policies for governance. Additionally, the office has partially implemented processes for selecting and overseeing investments.

The need to address long-standing shortcomings in HUD’s IT management capability and improve the overlapping, duplicative, antiquated, and costly-to-maintain systems it uses to support the department’s mission make it critical that the Office of the CIO fully implement and sustain effective IT governance practices. Lacking boards that operate as intended, as well as complete and current policies for IT management, the department faces risk that its governance efforts will fall short of ensuring effective decision making. Further, maturing selection processes to ensure consistent application of selection criteria, comparison of proposed projects based on key factors, and adequate documentation of the entire process has the potential to bolster HUD’s investment management and better position leaders to implement more strategic management of the investment portfolio. Developing and sustaining robust processes for monitoring the progress of investments and evaluating their performance against expected outcomes would also aid the Office of the CIO in better understanding what it is obtaining for the investments made in IT each year. Until investment management practices are fully implemented, HUD cannot be assured that its IT investments are providing planned functionality efficiently and effectively and the department may be missing opportunities to improve the quality and outcomes of IT investments.

Finally, in the absence of capability to estimate and monitor governance- related cost savings and operational efficiencies, HUD has limited ability to monitor the outcomes of its IT governance activities.

Recommendations for Executive Action

To ensure that HUD fully implements and sustains effective IT governance practices, we recommend that the Secretary of Housing and Urban Development direct the Deputy Secretary and the department’s Chief Information Officer to place a high priority on taking the following actions:

Ensure that the executive-level investment review board meets as outlined in its charter, documents criteria for use by the other boards, and distributes its decisions to appropriate stakeholders.

Fully establish and maintain a complete set of governance policies, establish time frames for establishing policies planned but not yet developed, and update key governance documents to reflect changes made to established practices.

Fully establish an IT investment selection process that includes (1) articulating how reviews of project proposals are to be conducted; (2) planning how data (including cost estimates) are to be developed and verified and validated; (3) establishing criteria for how cost, schedule, and project risk are to be analyzed; (4) developing procedures for how proposed projects are to be compared to one another in terms of investment size (cost), project longevity (schedule), technical difficulty, project risk, and cost-benefit analysis; and (5) ensuring that final selection decisions made by senior decision makers and governance boards are supported by analysis, consider predefined quantitative measures, and are consistently documented.

Fully establish a well-defined process that incorporates key practices for overseeing investments, including (1) monitoring actual project performance against expected outcomes for project cost, schedule, benefit, and risk; (2) establishing and documenting cost-, schedule-, and performance-based thresholds for triggering remedial actions or elevating project review to higher-level investment boards; and (3) conducting post-implementation reviews to evaluate results of projects after they are completed.

Further, to establish an enterprise-wide view of cost savings and operational efficiencies generated by investments and governance processes, we recommend that the Secretary of Housing and Urban Development direct the Deputy Secretary and Chief Information Officer to place a higher priority on identifying governance-related cost savings and efficiencies and establish and institutionalize a process for identifying and tracking comprehensive, high-quality data on savings and efficiencies resulting from IT investments and the IT governance process.

Agency Comments

We received written comments on a draft of this report from the Department of Housing and Urban Development (reproduced in app. II). In its comments, the department stated that it concurred with our recommendations and intends to provide more information, including timelines for planned actions, once the final report has been issued.

We are sending copies of this report to interested congressional committees. We are also sending copies to the Secretary of Housing and Urban Development. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine (1) to what extent the Department of Housing and Urban Development (HUD) has implemented key information technology (IT) governance practices, including effective cost estimation, and (2) what, if any, cost savings or operational efficiencies HUD has reported achieving as a result of its IT governance practices.

To address our first objective, we reviewed and analyzed documentation on HUD’s IT governance practices. This documentation included HUD’s IT Governance Policy that called for the establishment of an IT management framework and four investment review boards; the IT Management Framework and Governance Concept of Operations that identified key IT policies and processes to support IT governance and investment decision making; and charters that established the roles, responsibilities, authorities, and operations of the department’s investment review boards.

We also reviewed documents showing how HUD determines which IT projects will be recommended and then selected to receive funding, including business cases, the criteria for assigning scores to competing investments, and master lists of final scoring results. Additionally, we reviewed available meeting minutes and presentations from HUD’s investment review boards related to investment decision making. Further, we reviewed documents regarding the Office of the Chief Information Officer’s (CIO) recent efforts to establish an enterprise-wide cost estimation methodology, including a cost element dictionary and memorandum to provide guidance for formulation of HUD’s fiscal year 2016 IT budget.

To assess HUD’s IT governance practices, we compared the evidence collected from our document reviews and analysis against critical processes in the information technology investment management framework developed by GAO that were most relevant to HUD’s efforts. These processes establish basic capabilities that lay the foundation for implementing more mature governance capabilities in the future, specifically instituting governance boards and establishing processes for selecting and overseeing investments. In addition, we identified supplementary criteria for assessing cost-estimating practices that were derived from GAO’s Cost Estimating and Assessment Guide.determined that the information provided by the Office of the CIO was sufficiently reliable for addressing this objective.

To address our second objective, we obtained and evaluated HUD- reported information on achieved cost savings and operational efficiencies accomplished through selected investments and governance decisions. Various sources of reported savings and efficiencies included the department’s fiscal year 2014 expenditure plan, which identified recent accomplishments and associated benefits from IT modernization efforts, a list of systems that had been deactivated since the beginning of fiscal year 2010 that identified resultant actual or estimated annual cost savings, and reports submitted to the Office of Management and Budget on cost savings achieved through the consolidation of commodity IT and cost avoidance achieved through decisions to eliminate system-support contracts.

Because reporting of savings and efficiencies was fragmented and not clearly attributed to governance decision making, we summarized the savings and efficiencies included in disparate Office of the CIO reports and asked officials to determine whether the information, as summarized, was accurate and complete. The officials confirmed that the list of savings we compiled from various reports accurately depicted its governance- related savings and efficiencies; however, we determined that the data provided were questionable because the information was not complete, accurate, consistent, or verified. Further, although requested, the Office of the CIO could not provide documentation to substantiate the information that it reported. As a result, we determined that the information provided by the office to report governance-related cost savings and operational efficiencies was not sufficiently reliable for the purposes of our review. Specifically, the data provided were not consistent and complete, and the Office of the CIO did not provide supporting documentation we could have used to assess the reliability of the information, such as documentation showing how reported cost savings and operational efficiencies were achieved or how, if at all, they related to specific governance decisions.

To address both objectives, we interviewed officials from the department’s Office of the CIO, including the Deputy Chief Information Officer (formerly the Acting Chief Information Officer), the Acting Deputy CIO for Customer Relationship and Performance Management, the Acting Chief Technology Officer, the Acting Chief IT Transformation Officer, the Investment Management Division Director, and the Acting Enterprise Architecture Director, to obtain information on HUD’s IT governance process. We also interviewed HUD officials from the department’s Office of Strategic Planning and Management Office, including the Acting Director.

We conducted this performance audit from February 2014 to December 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Housing and Urban Development

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact above, Mark T. Bird (Assistant Director), Donald A. Baca, Kami J. Corbett, Amanda C. Gill, Lee A. McCracken, and Roger M. Smith made significant contributions to this report.