Information Technology: HUD Needs to Improve Key Project Management Practices for Its Modernization Efforts
Released: 2013-06-12T00:00:00

Letter

Background

To accomplish its mission, HUD administers community and housing programs that benefit millions of households each year. Among other things, the department provides affordable rental housing opportunities and helps homeless families and chronically homeless individuals and veterans. The department also administers mortgage insurance programs for single-family housing, multifamily housing, and health care.

HUD relies on five main organizational components to carry out its mission. Of these, two components have lead responsibility for improving access to housing and are the business owners for related IT modernization efforts:

Housing/Federal Housing Administration (FHA): Programs within this office are responsible for contributing to building healthy communities, maintaining and expanding housing opportunities, and stabilizing credit markets in times of economic disruption. This office also regulates certain aspects of the housing industry. For example, the department currently reports that it provides insurance on loans made by its approved lenders for 4.8 million single-family mortgages and 13,000 multifamily projects, including manufactured homes and hospitals. The FHA Transformation modernization effort is managed within this office.

Public and Indian Housing: Programs within this office are responsible for creating opportunities for residents’ self-sufficiency and economic independence. Toward this end, this office currently oversees a housing choice voucher program to subsidize housing for approximately 2.1 million low-income, elderly, and disabled families; a public housing program that subsidizes about 1.3 million housing units for vulnerable low-income families; and block grants and guarantee programs for Native American groups. The NGMS modernization effort is managed within this office.

In addition, to support these organizational components, the department relies on various administrative offices to provide guidance and tools. These include the department’s Office of the Chief Information Officer (OCIO) and the Office of the Chief Procurement Officer (OCPO). Through coordination with the organizational components, OCIO manages IT resources and provides support for the department’s infrastructure, security, and ongoing projects. This office also provides project management guidance and technical expertise to modernization efforts. For its part, OCPO is responsible for obtaining contracted goods and services required by the department to meet its strategic objectives. This office is involved with initiating acquisition actions upon request by the organizational components.

Further, HUD’s Deputy Secretary is responsible for managing the department’s daily operations, annual operating budget, and approximately 8,900 employees. As part of this role, the Deputy Secretary conducts biweekly meetings with stakeholders to discuss the Secretary’s priorities. During these meetings, the scope, milestones, risks, and status of action items related to priority issues are discussed. The FHA Transformation and NGMS modernization efforts are designated as priority and each has its own biweekly meeting.

A simplified view of the department’s housing organization structure and the offices responsible for FHA Transformation and NGMS is provided in figure 1.

According to the fiscal year 2014 President’s Budget request for HUD, $285.1 million is expected to be spent on IT investments. HUD’s IT environment consists of multiple systems that, among other things, are intended to help the department coordinate with lending institutions to insure mortgages, collect and manage state and local housing data, process applications for community development, and issue vouchers that provide access to subsidized housing. In particular, the department’s housing programs rely on systems for processing and managing these business operations. For example, systems within the Office of Housing are expected to process mortgage insurance applications, bill and collect premiums, pay claims, manage receivables and other assets, track delinquencies and defaults, and support staff in providing counseling to first-time home buyers and existing homeowners. Additionally, Public and Indian Housing programs that use systems are intended to process vouchers for different rental assistance programs, as well as to support the processing of applications for, and the management of, more than 50 grant programs administered by the department.

However, HUD’s current IT environment has not effectively supported its business operations because its systems are overlapping and duplicative, not integrated, necessitate manual workloads, and employ antiquated technologies that are costly to maintain. For example, the department reported from 2008 to 2012 that its IT environment consisted of:

Over 200 information systems, many of which perform the same function and, thus, are overlapping and duplicative. Specifically, different systems perform the same task to separately support grants management, loan processing, and subsidies management.

Stovepiped, nonintegrated systems that result in identical data existing in multiple systems. For example, two organizational components store about 80 percent of similar data in separate databases that provide information on rental assistance participants.

Manual processing for business functions due to a lack of systems to support these processes. For example, specific NGMS projects are intended to replace existing ad hoc analyses performed in spreadsheets and databases with systems that automate and standardize those functions.

Antiquated technology (15- to 30- years old) and complex systems that are costly to maintain. For example, the department relies on different programming languages and operating systems, which requires specialized skills to operate and maintain.

Additionally, HUD engaged contractors to conduct an assessment of the department’s environment. This assessment, issued in January 2011, concluded that unclear reporting relationships hindered the enforcement of IT policies; contractor performance information was not used to inform management decisions; technical standards were lacking or not enforced; and data management practices did not support business needs.

HUD’s Transformation Initiative IT Modernization Efforts

Through the Transformation Initiative’s IT component, HUD has begun addressing challenges to its environment and modernizing its systems. In this regard, the department initiated seven IT modernization efforts, of which FHA Transformation and NGMS are the two largest. For fiscal years 2010 and 2011, the department reported that the Transformation Initiative funding made available for FHA Transformation and NGMS was $58.5 and $41.1 million, respectively. (See later discussion in this report regarding costs associated with the 14 projects in our study.)

FHA Transformation was initiated to improve the department’s management of insurance programs through the development and implementation of a modern financial services IT environment that is expected to improve loan endorsement processes, collateral risk capabilities, and fraud prevention. In August 2009, HUD published the FHA Office of Housing Information Technology Strategy and Improvement Plan, which identified and prioritized 25 IT areas with performance gaps for its single-family housing, multifamily housing development and rental assistance, health care facilities programs, and enterprise applications. In May 2010, FHA Transformation began planning and executing modernization efforts aimed at addressing the gaps identified in the plan. Specifically, the modernization initiative is intended to implement technology within the following four functional areas aimed at addressing changes in FHA’s business model, operating environment, and components of the loan life cycle: Infrastructure and legacy migration: Provide a scalable infrastructure to support rules engines, analytics, and reporting systems, as well as a mechanism for transferring legacy applications to the new platform. Specifically, the Federal Financial Services Platform project is intended to provide hardware and standard software to support case management and migration of legacy applications (e.g., the Computerized Homes Underwriting Reporting System) for all lines of business.

Borrower/collateral risk management and fraud monitoring: Provide tools for analyzing, monitoring, and managing emerging issues and trends in the housing market, including borrower and collateral risk, appraisals, and fraud, as it relates to the FHA portfolio. For example, the Legacy Application Transformation project is expected to implement a software service tool that aggregates data to identify emerging issues and trends in borrower risk and fraud by analyzing the accuracy and validity of verified assets, income, and employment on individual loans. Other projects within this functional area include business process reengineering and a pilot designed to automate and streamline the multifamily housing underwriting process. Using the new infrastructure, an automated underwriting tool is expected to be deployed to expand the capabilities for processing loan applications for insurance programs and replace current systems (e.g., the Development Application Processing System).

Counterparty management: Provide applications for improved performance and compliance of lenders and appraisers through more proactively identifying risk trends and improving loan file review techniques. Specific projects include the Lender Electronic Assessment Portal, a web-based automated delivery of electronic applications and storage of lender application data that assists with reviewing new lender applications and requests for annual recertification to participate in FHA programs. Future plans call for replacing seven legacy applications.

Portfolio analysis: Provide tools intended to augment risk monitoring and management; enhance predictive analytics; provide timely and flexible reporting; and deliver more accurate, detailed information to decision makers. For example, the Portfolio Risk Reporting & Analytics project is intended to provide a web-based software service tool for modeling FHA program risks. While initial use of the software is to include receiving hard-copy reports from the third-party vendor, HUD also expects to deploy the tool within the department’s infrastructure in order for FHA employees to have access to reports and the analytics dashboard data electronically.

Overall leadership for FHA is provided by the Assistant Secretary for Housing/Federal Housing Commissioner, who chairs the modernization effort steering committee; the General Deputy Assistant Secretary for Housing; and the Director for the Office of Program Systems Management, who is the executive sponsor. The modernization effort also has a project management office that is responsible for executing and managing the associated projects. As of April 2013, FHA Transformation consisted of 10 projects, 9 of which were included in our study. Table 1 summarizes the 9 FHA Transformation projects that we assessed as part of our study.

The NGMS modernization effort is intended to provide an integrated system with a seamless view of financial and program data currently warehoused in disparate data sources and a new set of monitoring, oversight, and software tools directed at ensuring that funds are used to assist affordable housing participants and reduce improper payment errors. In November 2011, the department used contractors to develop four prototype software tools aimed at demonstrating anticipated NGMS functionality for voucher programs. However, in July 2012, the department determined that the prototypes that had been developed would not address its business needs. As a result, the department initiated planning efforts to restructure the modernization effort and expand the scope to include all Public and Indian Housing lines of business.

HUD has reported that the aim of the restructured effort is to enhance the department’s affordable housing program, improve end-user satisfaction, streamline complex business processes, and integrate disparate IT systems into a common, modernized platform. The department intends for NGMS to support efforts to improve HUD’s financial accountability by more accurately quantifying budgetary data resources, measuring program effectiveness, and justifying the agency’s budget formulations and requests. NGMS is expected to help department personnel reduce improper payments by identifying anomalies in operating costs, reserves, and subsidy payments. Once implemented, NGMS is intended to provide staff with a new set of monitoring, oversight, and analysis tools to ensure that allocated federal funds are used efficiently to assist affordable housing participants. The department is taking an incremental approach to developing NGMS and expects to deliver initial functionality by August 2013. NGMS system and software development projects are designed to support four functional areas:

Financial management: Provide automated processes for budget forecasting and formulation and cash management based on real-time data that are expected to allow the department to anticipate cash flow needs through precise scenarios and disburse funds on the basis of project and tenant records, eliminating reconciliations. For example, the Budget Forecasting and Formulation project is intended to develop a solution that will include forecasting functionality, data aggregation, and analytics to support the budget development process for Public and Indian Housing programs such as vouchers, administrative fees, family self-sufficiency, mainstream vouchers, and housing assistance programs. In addition, this functional area is expected to migrate data from HUD’s Central Accounting and Program system and utilize information gathered from Public Housing Authorities regarding subsidized housing programs through an interface with the department’s New Core system.

HUD operations: Provide a single point of access to data and information to improve efficiency and reduce administrative burden through a New Data Collection system that is expected to replace legacy systems (e.g., Public and Indian Housing Information Center) and provide new functionality for subsidized housing programs, geospatial data on physical housing, real-time occupancy information, and energy conservation measures for properties. In the interim, the Portfolio and Risk Management Tool project is expected to provide aggregated data about Public Housing Authorities through a standard business intelligence solution and is expected to expand its use to partner operations in the future.

Partner operations: Expand the department’s operations system to provide a web-based single point of access for gathering consistent and accurate information from families and landlords to be used in the operation of public housing and voucher programs administered by Public Housing Authorities.

Business support: Provide expanded access and use of NGMS IT solutions; grant HUD and program participants better access to information and technical assistance through a central point of access with live help and self-paced guides; and develop the necessary infrastructure and processes to enable timely and accurate answers to end users’ inquiries.

Overall leadership for NGMS is provided by the General Deputy Assistant Secretary for Public and Indian Housing, who is the executive sponsor and chair of the modernization effort steering committee. The modernization effort also has a project management office that is responsible for executing and managing the associated projects. As of April 2013, NGMS consisted of six projects, of which five were included in our study. Table 2 summarizes the NGMS projects that we assessed.

Effective use of project planning and management practices is essential for the success of modernization efforts such as those being undertaken by HUD. Our prior reviews of federal agencies have shown that, when effectively implemented, these practices can significantly increase the likelihood of delivering promised capabilities on time and within budget. Moreover, project management maturity is dependent on an agency’s standardization and institutionalization of such practices. PMI reported in its March 2013 annual survey of project management professionals that high-performing organizations are almost three times more likely than low-performing organizations to use standardized practices throughout the organization, and generate better project outcomes.

To guide the application of best practices, we and others, including PMI and SEI at Carnegie Mellon University, have issued reports and frameworks for effective project management. These reports and frameworks emphasize practices that include the development of essential documentation needed for the execution and management of projects in the areas of project planning (charters, work breakdown structures, and project management plans), requirements management (requirements management plans and traceability matrixes), and acquisition planning (acquisition strategies).

Project planning: This practice helps establish project objectives and outline the course of action required to attain those objectives. It also provides a means to track, review, and report progress and performance of the project by defining project activities and developing cost and schedule estimates, among other things. Project planning involves, for example, creating a charter to authorize project work, developing a work breakdown structure, and establishing project management plans that provide processes for measuring progress.

Requirements management: Having a documented strategy for developing and managing requirements can help ensure that the final product will function as intended. Effective management of requirements involves assigning responsibility for them, tracking them, and controlling changes to them over the course of the project. It also ensures that each requirement traces back to the business need and forward to its design and testing. Requirements management practices call for the use of requirements management plans to provide a mechanism for documenting the process for managing requirements and associated traceability matrixes, which are intended to facilitate efforts to link requirements to identified business needs to help ensure that they will be satisfied by the end product.

Acquisition planning: Effective IT project management also involves creating strategies to serve as the road map for acquisition planning. Such road maps are used for early planning of procurements and are developed by a project manager. Among other things, acquisition strategies should address plans for how projects will manage risks, deliverables, and reporting on contractor performance.

In addition to calling for agencies to apply best practices, federal guidance, along with our framework for managing IT investments and our prior reviews of federal investments, outlines the importance of having reviews conducted by management at various points throughout a project’s life cycle. Such reviews are critical to helping ensure that cost, schedule, and performance goals for a project are satisfied, and they can provide early detection of risks and problems that could impede progress toward those goals. Further, management reviews can help ensure that appropriate quality standards are achieved and provide input for areas that need improvement.

HUD’s IT Project Management Framework and Governance Structure

In order to better manage its modernization efforts, during 2011 HUD established new policies and procedures for executing and governing IT investments. Specifically, in April 2011, the department developed a Project Planning and Management (PPM) framework to provide guidance for managing a project’s life cycle in accordance with best practices. Using the framework, projects—such as those related to FHA Transformation and NGMS modernization efforts—are expected to proceed through life-cycle phases that require specific documents to demonstrate project activities and outcomes. The framework provides guidance through sample templates, with associated instructions and checklists that projects can use in developing their documentation. The framework also calls for management reviews that are intended to help ensure projects are aligned with the department’s architecture and technical standards, and that they have developed required information before committing resources to the next life-cycle phase. For example, at the initiation of a project, among other things, a charter and schedule are expected to be developed and approved by a review committee. In addition, during a project’s definition phase critical documents, such as a project management plan, requirements management plan, requirements traceability matrix, and an acquisition strategy, are also expected to be developed and approved by a review committee.

In July 2011, HUD also established a governance policy that set forth processes, standards, roles, and responsibilities to facilitate decision making around investments, stakeholder relationships, project life-cycle management, and other important IT operational areas. In particular, the policy established an IT governance structure consisting of the Executive Investment Board, Customer Care Committee, Investment Review Subcommittee, and Technical Review Subcommittee. Figure 2 provides a simplified depiction of this governance structure.

According to HUD’s Policy for Information Technology Governance handbook, these governance bodies have the following composition and responsibilities.

Executive Investment Board: Comprised of senior leaders, including the HUD Secretary, Deputy Secretary, and Chief Information Officer, with responsibilities for providing strategic direction, managing the IT investment portfolio, and overseeing and approving projects that cost more than $5 million.

Customer Care Committee: Comprised of executives, including the Chief Information Officer, the Chief Procurement Officer, and deputy assistant secretaries, who manage IT investments and perform project management oversight by reviewing and submitting recommendations to the Executive Investment Board, and coordinating with the subcommittees responsible for approving projects that cost between $500,000 and $5 million. respect to business cases and budget information for the Office of Management and Budget (OMB).

Technical Review Subcommittee: Comprised of personnel from within OCIO, including the Chief Technology Officer, the Chief Architect, and the Chief Information Security Officer, with a focus on ensuring that the technical architecture is aligned with the department’s strategic goals and monitoring IT projects through conducting control gate reviews that assess whether all necessary documentation has been produced. The subcommittee is also responsible for approving projects that cost less than $500,000.

HUD Has Begun Applying Key Practices for Its Modernization Efforts but Has Been Hindered by Inadequacies in Its Project Management Framework and Governance

For its FHA Transformation and NGMS modernization efforts, HUD has taken initial steps toward applying key project management practices in the areas of project planning, requirements management, and acquisition planning. However, the department has not yet fully implemented any of these practices in managing the 14 projects in our review. In large part, these deficiencies can be attributed to inadequate development and use of the department’s project management framework and governance structure. Without fully implementing these practices and effectively developing and using its framework and governance structure, HUD risks investing its resources on projects that may not meet critical mission needs.

According to the Project Management Institute and the Software Engineering Institute, disciplined project management practices call for the development of project details such as objectives, scope of work, schedules, costs, and requirements against which projects can be managed and executed. This step can be facilitated by developing project artifacts that include, among other things, charters to authorize projects and assign responsibility for their execution, work breakdown structures to define the work that needs to be done to accomplish project objectives, project management plans to define how projects are to be executed and controlled, and requirements management plans to document the processes and methods to be used for developing and managing project requirements. Further, developing requirements traceability matrixes that provide linkages between business objectives and detailed system requirements, and establishing strategies to ensure adequate acquisition planning are practices that contribute to effective project management. Our prior reviews of federal agencies have shown that applying these practices can significantly increase the likelihood of delivering promised capabilities on time and within budget.

HUD had taken initial steps in applying key project management practices by developing artifacts, to varying degrees, for the 9 FHA Transformation and 5 NGMS modernization efforts in our review. Nevertheless, the department lacked information needed for managing and executing the projects because the documentation developed did not contain a number of essential details that best practices stress as being critical to effectively defining a project and measuring its success. In this regard, none of the documentation included all of the critical information that could facilitate effective project management, such as full descriptions of the work necessary to complete the projects, cost and schedule baselines, or prioritized requirements.

Table 3 summarizes the key project management practices in the areas of project planning (charters, work breakdown structures, and project management plans), requirements management (requirements management plans and traceability matrixes), and acquisition planning (acquisition strategies) for the 14 projects that we assessed. In addition, appendix III provides our more detailed assessment against best practices.

A project charter formally authorizes a project and identifies high-level information that constitutes and assigns responsibility for project success. According to project management practices, to be effective, a charter should include, among other things, a project’s purpose or justification; high-level information on such factors as requirements and risks, measurable objectives, and related success criteria; a summary schedule and budget; project approval requirements (e.g., information on what factors will define project success and who will be responsible for final sign off at the completion of the project); and names, responsibilities, and authority levels of assigned leadership such as the project manager and sponsor.

Of the 14 projects in our review, all 9 FHA Transformation and 3 of the NGMS projects had developed charters that included most of the relevant high-level information. For example, all of the charters included such information as the project purpose, description, and high-level risks and requirements, as well as the names of the assigned project managers. Regarding measurable objectives and success criteria, 10 of the projects included objectives, while 6 had related success criteria. Lastly, 10 project charters included a summary schedule, and 1 included a summary budget.

While most of the charters contained high-level information, other essential details were not included such as the authority levels of project leaders and the requirements for approving the completion of the projects. Specifically, while each of the charters generally referenced HUD’s PPM framework and the associated governance committees, the charters did not explicitly state what results would constitute project success (e.g., a specified number of project objectives met) or what individuals or entities would be responsible for final sign-off at the completion of the project. For example, FHA Transformation’s LEAP Institution Manager project charter included the project’s purpose, high- level risks and requirements, and measurable objectives and related criteria. Specifically, the charter noted that, by the end of fiscal year 2014, the project would result in the retirement of four systems, eliminating the associated costs for operations and maintenance. The charter also incorporated a summary schedule and the names of its project manager and sponsor. However, while the charter included a total budget figure, it did not include details regarding the breakdown of the budget provided, the responsibilities and levels of authority given to the manager and sponsor identified, and the requirements for approving the completion of the projects.

Additionally, the NGMS Integrated Budget Forecasting Model project charter provided the project’s purpose, high-level risks and requirements, the names of the project sponsors and managers, a summary schedule, and measurable objectives with related success criteria. In particular, regarding the measurable objectives, the charter stated that the project would reduce the average time to respond to ad hoc requests for budgetary reports and data from 3 days to 1 day. However, while the charter referenced the OMB Capital Asset Plan and Business Case Summary (Exhibit 300) for a list of associated costs, it did not include a summary of the project’s expected budget. This charter also did not include the responsibilities and the authority levels of the sponsors and managers or the project completion requirements.

FHA Transformation and NGMS officials acknowledged the absence of these details in the charters and attributed the deficiencies to the general immaturity of the department’s project management practices. Regarding the remaining two NGMS projects for which charters had not yet been developed, project officials stated in April 2013, that one project was in the process of developing a charter, while relevant information for the other project was expected to be incorporated into the Budget Forecasting and Formulation charter.

In the absence of charters that reflect all of the essential elements, HUD lacks clear definitions of what will constitute success for its modernization projects and has less ability to hold the responsible officials accountable for this success. Moreover, the lack of important details that a charter is intended to provide at the initial authorization of a project makes it more difficult to undertake other project planning activities, such as developing work breakdown structures, project management plans, and requirements.

Work Breakdown Structures Had Not Been Fully Developed

A work breakdown structure is the cornerstone of every project because it defines in detail the work necessary to accomplish a project’s objectives and provides a basic framework for a variety of related activities like estimating costs, developing schedules, identifying resources, and determining where risks may occur. According to best practices, this artifact should be a deliverable-oriented hierarchical decomposition of the work to be executed by the project team to accomplish the project’s objectives. Moreover, best practices state that it should represent the entire scope of the project and product work, including project management, and it should be standardized to enable an organization to collect and share data among projects. In addition, it should be accompanied by a dictionary that describes in brief narrative form what work is to be performed in each of the various work breakdown structure elements.

Of the 9 FHA Transformation and 5 NGMS projects, none had developed complete work breakdown structures and associated dictionaries; only one NGMS project—Budget Forecasting and Formulation—had a draft work breakdown structure and associated dictionary. However, while this draft work breakdown structure included details regarding the first increment of the project, neither it nor the associated dictionary provided details for any of the future planned increments. Thus, it did not reflect the entire scope of the project and lacked descriptions of the work that would be performed following the first increment, which is expected to deploy initial functionality by late summer of 2013. Further, rather than being organized by deliverables—that is, unique and verifiable products, results, or capabilities—the draft was organized by life-cycle phases such as definition and design. As a result, it did not allow for progress to be measured by deliverable, which would enable more precise identification and effective mitigation of the root causes for any cost or schedule overruns. Moreover, developing a deliverable-oriented work breakdown structure would show how deliverables relate to one another as well as to the overall end product.

According to NGMS officials, plans are under way to fully develop work breakdown structures that represent the first and second increments of all projects in late spring 2013. Notwithstanding these plans, as of April 2013, a specific time frame for developing the work breakdown structures and associated dictionaries for the third increment of NGMS projects had not yet been determined.

Further, regarding the 9 FHA Transformation projects, in April 2013 officials stated that a work breakdown structure and dictionary to represent the entire modernization effort is being developed. However, the department was not able to provide a specific date for when this documentation would be completed.

NGMS and FHA Transformation officials stated that work breakdown structures were not initially developed for the projects because the PPM framework did not require the completion of this artifact. The officials added that, in addition to HUD project management practices lacking maturity, their staff had not yet developed the expertise required to create this artifact.

Until FHA Transformation and NGMS develop deliverable-oriented work breakdown structures and associated dictionaries for all of their projects, these efforts will lack critical information for understanding the detailed work that needs to be performed to accomplish project objectives. Further, by not defining the work to be performed, HUD cannot provide reasonable assurance that cost and schedule estimates will capture all the relevant information needed for the management of these efforts.

Project Management Plans Included Important Details but Lacked Baselines and Management Processes

According to project management practices, a project management plan is the primary source that defines how a project is to be executed and controlled. Best practices emphasize the importance of having such plans in place to, among other things, establish a complete description that ties together all project activities and evolves over time to continuously reflect the current status and desired end point of the project. Moreover, these practices state that to be effective, a project management plan should identify life-cycle processes to be applied, outline plans for project tailoring (i.e., determining what processes and documentation would be necessary to accomplish project objectives), provide communication techniques to be used, and list management reviews. Further, building on the initial summary schedule and budget in the charter, the project management plan should include baseline cost and schedule estimates developed during planning activities. Moreover, this baseline information should be updated as needed and periodically compared with actual performance data in order to track and report progress. Finally, the plan should include, or make reference to, subsidiary management plans that describe how subordinate activities are to be carried out for the project.

Six FHA Transformation and 5 NGMS projects had drafted or completed project management plans that outlined life-cycle processes, identified communication techniques and management reviews, and incorporated certain subsidiary management plans. For example, the plan for FHA Transformation’s Legacy Application Transformation project included an approach for tailoring the life-cycle processes to be used; contained a communication table with details about what techniques would be used; and described different types of management reviews, including an official team review and a structured walkthrough. Similarly, the plan for NGMS’s Integrated Budget Forecasting Model project indicated that the project was following HUD’s PPM framework, which includes tailoring life- cycle processes; contained communication techniques; and identified different types of management reviews, including audit reviews and post- project reviews.

However, the plans provided for the 11 projects lacked other essential information. Specifically, they did not clearly identify cost and schedule baselines or consistently incorporate subsidiary plans. For example, the plan for FHA Transformation’s MFH Development & Underwriting Business Process Reengineering/Automated Underwriting Solution project listed milestones, such as implementing a solution by July 2013, and referenced a total cost of ownership artifact that indicated a cost of $1.9 million. Yet, the plan did not indicate if these were considered to be the project’s schedule and cost baselines against which progress would be measured. The impact of the lack of clear baselines was evidenced by inconsistencies between the project management plan and other project documentation. For example, weekly status reports indicated that the solution would be implemented by November 2013, rather than by the July 2013 date identified in the project management plan. This lack of clarity regarding the project’s cost and schedule baseline makes it difficult to accurately measure and report progress against commitments made to deliver functionality.

In a similar example, the project management plan for NGMS’s Integrated Budget Forecasting Model provided project milestones and identified cost estimates by life-cycle phase, but it did not specify if these figures represented cost or schedule baselines developed as part of planning activities. Additionally, the plan that reflected the other 4 NGMS projects in our study included subsidiary plans for requirements, scope, schedule, cost, quality, human resources, and risk management, but it did not incorporate necessary details in the acquisition strategy and lacked one for process improvement.

According to FHA Transformation and NGMS officials, the project management plans did not include cost and schedule baselines, in part, because the baseline information had been included in the updates provided to OMB. However, including a project’s cost and schedule baseline in a project management plan is important because the plan serves as a primary source of information used to execute and manage the project. In addition, such baseline information provides managers and sponsors the foundational basis for measuring project progress. Relying on information reported to an external entity such as OMB rather than on cost and schedule baselines to manage projects may not allow the project manager to have accurate real-time information available when responding to stakeholder interests regarding the status of project progress. Regarding the incorporation of subsidiary plans, the officials stated that these plans were not required by the PPM framework to complete control gate reviews and, as a result, were not fully addressed in all of the project plans. Further, for the remaining 3 FHA projects that had not yet developed project management plans, FHA Transformation officials said the projects were still completing initial planning activities. In accordance with the PPM framework, the projects would be expected to develop plans when those activities are completed.

Until FHA Transformation and NGMS have comprehensive project management plans that reflect cost and schedule baselines and fully incorporate subsidiary plans for process improvement and acquisition management, these modernization efforts will continue to lack a foundational tool needed for successfully managing their projects and for providing stakeholders with insight into the status of the projects.

Requirements Management Plans Lacked Details for Prioritizing and Measuring Requirements

According to project management practices, effective planning of requirements includes documenting the processes and methods to be used for developing and managing requirements from initial identification through implementation. Such practices state that requirements management plans should incorporate the approach for how requirements development activities (e.g., collecting requirements) will be conducted and how changes will be managed; identify methods for prioritizing requirements; and specify the metrics to be used to measure products against identified requirements, among other things. As we previously reported, effective planning for requirements development and management activities can reduce the risk of cost overruns, schedule delays, and limitations in system functionality.

Seven of the FHA Transformation and all 5 of the NGMS projects in our review had developed requirements management plans that documented how requirements development activities would be conducted, including managing changes. For example, the FHA Transformation’s Healthcare Automated Lender Application Pilot project plan outlined processes for how changes to requirements would be, among other things, evaluated upon submission and analyzed for determining their impact on original requirements in order for decisions to be made regarding proposed changes. In addition, the NGMS Integrated Budget Forecasting Model project plan indicated that requirements would be gathered through interviews between the contractor and stakeholders, and provided detailed policies and procedures for developing and maintaining requirements. However, only 1 of the 12 projects—LEAP Automation of Lender Approval Workflow—identified methods for prioritizing requirements, and none of the projects established metrics for determining the extent to which the products developed addressed requirements. The over 2,400 functional requirements identified for the NGMS Budget Forecasting and Formulation project illustrates the significance of this point as the lack of prioritization could heighten the difficultly developers may face in determining which among the many requirements to focus on first.

FHA Transformation and NGMS officials stated that they had followed the PPM framework template to develop the requirements management plans for their projects. However, they added that the framework did not call for prioritization methods to be identified in the requirements management plan, and the department’s governance committee responsible for project oversight did not provide feedback to indicate that the plans needed to include this information. As of late April 2013, according to FHA Transformation officials, the remaining 2 projects had not yet developed requirements management plans because the projects were still in initial planning.

Without establishing methods for prioritizing requirements, the department will lack vital information needed to allocate resources in a manner which ensures that higher-priority requirements are addressed before lower-priority ones. In addition, until metrics for determining how products address requirements are established, the department lacks the ability to ensure that products will address business needs. As a result of these missing details, HUD increases the risk that implemented solutions may not effectively support the department’s mission.

Requirements Matrixes Provided Limited Traceability

According to best practices, the development of a requirements traceability matrix is intended to link business needs outlined in high-level requirements to more detailed requirements. Traceability refers to the ability to follow a requirement from origin to implementation and is critical to understanding the interconnections and dependencies among the individual requirements and the impact when a requirement is changed. Requirement matrixes provide tracing to, among other things, business needs and the criteria used to evaluate and accept requirements. Further, the use of attributes (e.g., a unique identifier, priority level, status, and completion date) in the matrix helps define the requirement to facilitate traceability. As we have reported, establishing and maintaining traceability is important for understanding the relationships among requirements—from the point at which business requirements are initially established through the execution of test cases to validate the resulting product.

Six FHA Transformation projects and 2 NGMS projects had developed requirements traceability matrixes to track their requirements. However, the eight matrixes that had been developed varied in the extent to which project requirements linked detailed functional requirements backwards to high-level business needs and forward to implementation. Further, attributes intended to allow the original business needs to be traced to detailed functional requirements were incomplete or missing. For example, the FHA Transformation Portfolio Risk Reporting & Analytics project matrix supported traceability of requirements back to higher-level business goals and also provided specific attributes such as unique identifiers. However, the matrix did not link the high-level requirements outlined in the project’s requirements definition documentation to more detailed requirements or trace to documentation that described criteria to be used for evaluation and acceptance of requirements.

In addition, the NGMS Integrated Budget Forecasting Model matrix included requirements that were traced from high-level to more detailed requirements and recorded specific attributes such as a unique identifier and the current status. However, the matrix did not provide traceability to criteria for evaluating and accepting requirements or consistently record accurate information regarding the current status of the requirements. For instance, the matrix included two identical requirements, but each stated requirement had a different disposition: the information for one requirement indicated that it had been “completed” while information for the other requirement included a notation of “discontinued,” but without any associated dates to clarify which disposition accurately described whether the requirement had been implemented. Further, the NGMS Budget Forecasting and Formulation project developed a matrix that used a unique identifier that allowed traceability from 15 high-level requirements to more detailed functional requirements. In addition, these requirements and the traceability matrix were approved by the appropriate stakeholders. However, the matrix did not document several other attributes, including status, or provide traceability to criteria for the evaluation and acceptance of these requirements. In particular, the matrix did not establish priorities for requirements to aid in ensuring that those of highest priority are addressed first.

According to FHA Transformation and NGMS officials, the PPM framework guidance was used in creating the matrixes and, in many cases, the projects relied on contractors to complete the artifacts. In addition, according to these officials, project resources were focused on providing the documentation required by the framework and associated governance committee. As a result, information that was not explicitly identified as being required in an artifact, such as matrixes that demonstrate traceability, was not developed. With regard to the remaining 3 FHA Transformation and 3 NGMS projects, department officials said the projects were still completing initial planning activities and had not reached a point where requirements had been defined to populate a matrix.

The incomplete state of the requirements traceability matrixes makes it unclear what mission needs have been addressed by project functional requirements and are planned to be implemented in a solution. Without fully traceable requirements for each project, the FHA Transformation and NGMS modernization efforts are limited in their ability to know whether necessary requirements are being implemented or if those being implemented support defined business needs.

Strategies Lacked Details for Guiding Project Acquisitions

Best practices also state that effective IT project management involves creating a strategy for acquisition planning. The strategy should be based on the needs of each individual project and can be formal or informal, and highly detailed or broadly framed. The strategy should also be incorporated as a subsidiary component of the project management plan. An acquisition strategy serves as the road map for effective acquisition planning and should document the types of contracts to be used, address contract risks, determine dates for deliverables, and coordinate contracts with other processes, such as scheduling and performance reporting. Additionally, the strategy should reflect early identification of metrics to be used in managing and evaluating contractors to help ensure that business needs are addressed through contracted support.

FHA Transformation and NGMS each developed one acquisition strategy that was intended to represent all the projects being undertaken by their respective modernization efforts. In addition, while the acquisition strategy for NGMS was intended to represent all the projects, one project— Integrated Budget Forecasting Model—also developed its own individual strategy. These three strategies identified the types of contracts (e.g., time and materials, firm-fixed price, or interagency agreement) that were planned to be awarded for their associated projects. For instance, the FHA Transformation strategy stated that indefinite-delivery, indefinite- quantity contracts would be awarded and associated task orders would be firm-fixed price, time and materials, or labor hour. The NGMS strategy stated that it would utilize existing interagency agreements and work with small disadvantaged businesses for its contract needs. Further, the NGMS Integrated Budget Forecasting Model project’s separate acquisition strategy identified the type of contract to be used (i.e., blanket purchase agreement with firm-fixed price task orders), addressed contract risks (e.g., the unavailability of server space), and determined dates for deliverables (e.g., create and update detailed functional requirements between January 20 and February 10, 2011).

Nonetheless, the three strategies did not fully document details needed for effective acquisition planning, such as information on how risks would be addressed, determining dates for deliverables, coordinating with other processes, and identifying metrics needed for evaluating contractors. For example, the FHA Transformation acquisition strategy identified dates for projects, but did not state how contract dates would be coordinated with schedule processes. Moreover, both the NGMS strategy and the individual strategy for the Integrated Budget Forecasting Model project did not state how other project processes, such as requirements development, would be coordinated with acquisitions or identify metrics for assessing contractors’ performance.

FHA Transformation and NGMS officials stated that the strategies developed were based on the PPM framework template and that the strategies had been approved by the Technical Review Subcommittee, which did not identify the deficiencies. Further, while a strategy should guide acquisition planning, OCIO officials said the requirement in the PPM framework did not call for developing strategies prior to awarding contracts.

Without strategies that guide planning activities in order to ensure that acquisitions are managed in accordance with other processes and provide performance metrics, the department increases the risk that acquisitions associated with its modernization efforts will not be effectively managed and that acquired services or products will not meet its needs.

Inadequate Development and Use of HUD’s Project Management Framework and Governance Contributed to FHA Transformation and NGMS Weaknesses

As previously discussed, HUD’s project management framework and associated governance structure was established to provide policies and procedures for managing the department’s IT investments. Specifically, the framework provides instructions, templates, and checklists intended to help ensure important details are incorporated for use during the execution and management of project activities. The department’s governance structure is responsible for ensuring that all necessary documentation is produced for all IT projects through control gate reviews conducted by the Technical Review Subcommittee.

Officials responsible for the 9 FHA Transformation and 5 NGMS projects in our review stated that they relied on the department’s PPM framework to implement project management practices and the artifacts discussed in this report. However, guidance discussed in the framework did not always include essential information called for by best practices. For example, the guidance for developing requirements management plans did not specifically direct the projects to identify methods for prioritizing requirements. In addition, the projects did not develop strategies early enough to guide acquisitions because the framework did not call for the strategy to be developed until after projects completed initial planning activities.

In other cases, where guidance was provided, FHA Transformation and NGMS projects did not always follow the guidance provided or adequately implement the tools provided by the framework in developing the documentation we examined. This was particularly evident in the development of work breakdown structures. For example, the PPM guidance included specific details regarding the importance of developing work breakdown structures as the basis for defining project work and establishing reliable cost and schedule baselines. However, as noted earlier, only 1 of the 14 projects in our study had drafted a work breakdown structure. Further, the project management plan template and guidance call for incorporation of cost and schedule baselines and approaches for how those will be managed for any given project. However, the project management plans we examined did not clearly incorporate such baselines or how they would be managed.

Compounding the issue of inadequate development and use of the framework was the lack of evidence that the department’s governance bodies had provided adequate oversight to ensure compliance with project management practices. In particular, the department’s Technical Review Subcommittee did not express concerns regarding the alignment of FHA Transformation or NGMS documentation with the framework and, when issues were raised, the subcommittee had nonetheless allowed the projects to proceed. In doing so, the projects were able to move to the next control gate review without critical information—a practice which could result in projects proceeding for months without correcting flaws or inadequacies in information that was vital to effective project management. Specifically, in examining documentation for control gate reviews, we found that the Technical Review Subcommittee did not consistently operate as intended or use the guidance provided in the department’s framework. While the department’s framework outlines processes for conducting control gate reviews of projects and provides templates to be used, the reviews were conducted without using the framework guidance. For example, the control gate review procedures state that documentation should be assessed based on (1) its accuracy in capturing necessary information for the project’s development, (2) its completeness with a level of detail sufficient to provide correct and relevant information, and (3) the adequacy of information in the artifact to make it actionable and informative. The framework also provided a decision document intended to capture any issues or concerns identified by the subcommittee.

However, it was not evident that any of the control gate reviews conducted from 2011 through 2012 had assessed the documentation against the outlined criteria or that the decision document was used. For example, during this time, none of the control gate review documents provided for FHA Transformation and NGMS included an assessment of the documentation against the criteria in the control gate review procedures, and meeting minutes or e-mails were used to record high- level issues or concerns identified instead of the more detailed information called for in the decision document. According to responsible OCIO officials, the subcommittee did not assess compliance with the framework, but was focused on reviewing the technical aspects of IT projects. The officials also noted that the subcommittee did not have the staff needed to fully implement the control gate review guidance included in the framework, but that it did look to see if the identified artifacts were developed for each project. Further, these officials stated that it was the responsibility of the project managers and their teams to address issues identified before the next control gate review, but that the subcommittee did not enforce any specific deadlines. Based on our assessment of the control gate review documentation, as well as interviews with OCIO and modernization effort officials, it was not clear that the subcommittee consistently considered its role to include a full assessment of the artifacts for compliance with the framework outlined in control gate review guidance and templates.

According to OCIO officials, the initial implementation of the framework focused on attempting to get projects to understand basic project management, and as a result, the department limited the focus of the first version of the framework. In April 2013, the department reported that it was working on a revised version of the framework that would be released in September 2013. However, the preliminary information provided regarding the revisions planned for the framework did not incorporate information to address all the deficiencies identified by project officials or highlighted in this report. For example, draft documents regarding the planned revisions did not explicitly state whether work breakdown structures and associate dictionaries would be required documentation to serve as the basis of cost and schedule baselines. Further, the preliminary information did not specify if methods for prioritizing requirements are to be incorporated into the requirements guidance. Additionally, as of February 2013, the department had assigned new leadership for managing the control gate reviews. According to this official, the control gates are expected to be revised to ensure that artifacts are evaluated and that the subcommittee takes a more active role in assessing the application of project management practices. However, the department did not state if it would clarify the Technical Review Subcommittee’s role or associated guidance outlined in the PPM framework or identify time lines for implementing the anticipated changes.

Until HUD has a PPM framework for managing its projects that incorporates the abovementioned details, including clarifying the role of the Technical Review Subcommittee, and is appropriately used in managing its modernization efforts, the department increases the risk of continuing to inadequately apply project management practices and will not be positioned to effectively manage or report progress of its modernization efforts.

Conclusions

HUD has taken steps toward applying best practices by establishing a framework for standardizing project management, and to varying degrees, the FHA Transformation and NGMS modernization efforts have developed basic documentation in the areas of project planning, requirements management, and acquisition planning. Notwithstanding these initial actions, the limited extent to which its modernization efforts implemented key practices in these areas puts its projects at an increased risk of failure. Specifically, the absence of complete information in foundational documentation intended to guide these efforts—such as project charters that define project success, deliverable-oriented work breakdown structures that detail the work needed to be accomplished, project management plans that include cost and schedule baselines, requirements management plans that provide methods for prioritizing requirements, traceable requirements to desired capabilities, and sound acquisition strategies that guide planning activities—means that HUD has not taken the steps to fully define its modernization efforts in terms of what they will accomplish, what steps are necessary to complete them, what they will cost, when they will be completed, what specific functionality is needed to meet their goals, and how contractors will be held accountable for performance. This indicates that, despite the steps that have been taken, the maturity of HUD’s project management practices does not sufficiently position the department to successfully carry out these efforts.

Contributing to these deficiencies is that the department has not developed and used its project management framework in a manner that ensured the quality or completeness of project management documentation. Additionally, the lack of adequate oversight from the Technical Review Subcommittee resulted in projects not fully understanding how to develop complete artifacts. Until it addresses these weaknesses in applying project management practices, HUD may continue to invest resources in modernization projects that will not satisfy business needs and support its mission. Moreover, fully implementing effective project management practices is critical not only for the success of these modernization efforts, but also for that of the other five IT Transformation Initiatives or any other projects under way or undertaken in the future.

Recommendations for Executive Action

To ensure that HUD effectively and efficiently manages its modernization efforts aimed at improving its IT environment to support mission needs, we recommend that the Secretary of Housing and Urban Development direct the Deputy Secretary to establish a plan of action that identifies specific time frames for correcting the deficiencies highlighted in this report for both its ongoing projects, as applicable, and its planned projects, to include developing charters that define what constitutes project success and establish accountability, finalizing deliverable-oriented work breakdown structures and associated dictionaries that define the detailed work needed to accomplish project objectives, completing comprehensive project management plans that reflect cost and schedule baselines and fully incorporate subsidiary management plans, establishing requirements management plans that include prioritization methods to be applied and metrics for determining how products address requirements, completing matrixes to include requirements traceability from mission needs through implementation, and establishing strategies to guide how acquisitions are managed in accordance with other processes and that performance metrics are established.

Further, to improve development and use of the department’s project management framework, we recommend that the Secretary direct the FHA Transformation and NGMS steering committees to ensure that project management expertise needed to apply the guidance outlined in the framework is provided to execute and manage their respective projects; the Chief Information Officer to ensure that revisions to the framework incorporate specific information to address the areas of deficiency in project planning, requirements management, and acquisition planning identified in this report; and the Customer Care Committee to review the role and responsibilities of the Technical Review Subcommittee and ensure that the department’s governance structure operates as intended and adequately oversees the management of all of its modernization efforts.

Agency Comments and Our Evaluation

We provided a draft of this report to HUD for review. In response, HUD provided a letter, signed by the Acting Chief Information Officer, which included a chart containing the department’s written comments on the draft report. In the chart, the department outlined its views related to our four recommendations, and provided other comments and technical corrections on information in specific sections of the draft report, including the background and appendix I, our discussion of the findings on the development and use of HUD’s project management framework, and the report title page. The department’s comments are reprinted in their entirety in appendix IV.

In commenting on our recommendations, the department discussed actions it was taking on various aspects of the first recommendation, but did not state whether or not it concurred with the entirety of the recommendation; stated that our conclusion leading to the second recommendation did not follow from the premises established in the draft report; and concurred with our third and fourth recommendations. Summaries of HUD’s comments for each recommendation, along with our responses, follow.

With regard to the first recommendation—which called for the Deputy Secretary to establish a plan of action that identifies specific time frames for correcting the deficiencies highlighted in this report for both of its ongoing projects, as applicable, and its planned projects—the department noted activities that FHA Transformation expects to undertake in addressing the deficiencies for the six specific items listed as part of this recommendation. In this regard, the department stated that FHA Transformation acknowledged the need to update project charters and project management plans, develop deliverable- oriented work breakdown structures, examine and correct the requirements management plans and traceability matrixes, and work with support offices to ensure acquisition planning occurs at the earliest possible opportunity in the project’s life cycle. The department added that FHA Transformation had recognized the need to update its charters and project management plans well ahead of our draft report. Nonetheless, updated artifacts for FHA Transformation were not provided during our review. Moreover, the department did not address whether or how it intends to address deficiencies for its ongoing or planned projects, including those associated with the NGMS modernization effort. Accordingly, we maintain that it is important for HUD to establish a plan of action that identifies specific time frames for addressing the deficiencies in its IT projects. As acknowledged in the department’s comments, efforts to improve these project management practices could be applied to the other five IT Transformation Initiatives or any other projects under way or undertaken in the future.

For our second recommendation, which called for the FHA Transformation and NGMS steering committees to ensure that project management expertise needed to apply the guidance outlined in the framework is provided to execute and manage their respective projects, the department contended that our conclusion leading to this recommendation did not follow from the premises established in the report. The department stated that it has ample talent and that providing additional talent would likely yield similar results regarding its deficiencies until the underlying steps are taken to apply effective project management practices.

We agree that applying effective project management practices is important; however, in our view, it is essential for the FHA Transformation and NGMS steering committees to ensure that their respective modernization efforts have the expertise needed to do so, as it pertains to the development of tools such as work breakdown structures and requirements traceability matrixes. During our study, department officials stated on multiple occasions, that certain artifacts and practices were not implemented because staff lacked expertise in these areas. For example, both FHA Transformation and NGMS officials stated that their staff had not developed the expertise required to create work breakdown structures. Similarly, these officials stated that projects had relied on contractors to complete requirements traceability matrixes. Additionally, as we noted, the officials acknowledged that a lack of project management maturity was the cause of many of the deficiencies identified. Moreover, in its comments on this report, the department stated that staff training for the transition to applying the framework was limited. Thus, for these reasons, we believe our recommendation is valid and should be implemented.

The department concurred with our third recommendation that the Chief Information Officer ensure that revisions to the framework incorporate specific information to address the areas of deficiency in project planning, requirements management, and acquisition planning identified in this report.

In commenting on the fourth recommendation, the department concurred with the need for the Customer Care Committee to review the role and responsibilities of the Technical Review Subcommittee and ensure that the department’s governance structure operates as intended and adequately oversees the management of its modernization efforts.

In other comments, the department stated that the discussion of the department’s project management framework did not recognize the difficulty of implementing this framework over the past 2 years. It stressed that tremendous effort had been made by the FHA Transformation and NGMS modernization efforts toward applying the framework while continuing to make progress on their related projects. It also stated that time is needed to fully incorporate the framework throughout the department on projects other than these modernization efforts.

Acknowledging that the department has continued to take actions to improve its environment, the focus of our work for this report was on the implementation of project management practices for FHA Transformation and NGMS, specifically. As such, we did not assess the difficulties associated with improving the department’s overall capacity to manage its IT projects. We do agree that there are difficulties associated with applying project management practices while concurrently undertaking multiple modernization efforts and have previously reported on the progress HUD has made in addressing its limited capacity to manage and modernize its IT environment.

Regarding the title page, HUD commented that modernization efforts historically account for a relatively small percentage of IT projects at the department, and that a more comprehensive perspective that accounts for all IT investments should be considered in the title of our report. Our objective for this report was specifically to identify the extent to which key project management practices were implemented for the FHA Transformation and NGMS modernization efforts. As such, this report did not evaluate all of the department’s IT investments. However, in this report, we do acknowledge the value of HUD applying these practices to all of its IT projects and moreover, we plan to undertake future work to evaluate the department’s institutionalization of its IT governance that we anticipate will be more comprehensive in assessing the department’s management of IT investments.

Lastly, the department stated that the report should contain historical information illustrating the distribution of modernization funding in contrast to funding available for the operation and maintenance of IT. Toward this end, we assessed all relevant data that the department provided to us regarding its IT funding against the data that it reported to OMB. However, we found these data to lack consistency and concluded they were not sufficiently reliable for inclusion in our report.

With respect to HUD’s technical corrections on the draft report, we have incorporated revisions, as appropriate. Specifically, in the background section and appendix I, we included a footnote to clarify that the Office of Program Systems Management is within the Office of Housing/FHA Office of Multifamily Housing Programs. We also updated the report section that discussed the development and use of HUD’s project management framework by removing the specific reference to the Deputy Chief Information Officer for IT Operations. In this same section, the department stated that OCIO did not concur with statements attributed to officials from the Technical Review Subcommittee. We modified the statements and the attribution in that section to represent more specifically what the officials stated. In doing so, we also further clarified the activities conducted by members of the Technical Review Subcommittee and comments provided by officials from the two modernization efforts.

We are sending copies of this report to interested congressional committees. We are also sending copies to the Secretary of the Department of Housing and Urban Development and the Director of the Office of Management and Budget. Copies of this report will also be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made significant contributions to this report are acknowledged in appendix IV.

Appendix I: Objective, Scope, and Methodology

Our objective was to identify the extent to which the Department of Housing and Urban Development (HUD) has implemented key project management practices for the Federal Housing Administration Transformation Initiative (FHA Transformation) and the Next Generation Management System (NGMS) modernization efforts. To address this objective, we examined all 14 projects for FHA Transformation and NGMS that had been identified in the department’s fiscal year 2011 expenditure plan. This included 9 FHA Transformation and 5 NGMS projects, which are identified in tables 1 and 2 of this report.

Because HUD recently began implementing project management practices for its information technology (IT) modernization projects, we reviewed the implementation of practices during the initial phases of the projects’ life cycles; these practices establish the foundational plans and processes for managing projects throughout their life cycles. Specifically, we reviewed project planning and management practices essential for the success of modernization efforts in three areas: project planning, requirements management, and acquisition planning. We identified best practices in these areas included in the Project Management Institute’s (PMI) A Guide to the Project Management Body of Knowledge (PMBOK® Guide), the Software Engineering Institute’s (SEI) Capability Maturity Model® Integration for Development (CMMI-DEV) and for Acquisition (CMMI-ACQ), and GAO’s Cost Estimating and Assessment Guide (Cost Guide).

For the 14 projects in our study, we assessed the three project management areas by reviewing six relevant documents to determine whether they contained essential information called for by best practices. Our assessment evaluated to what extent these documents (1) were developed and contained essential information, (2) were developed but lacked essential information, or (3) had not yet been developed. Specifically:

To assess project planning activities, we determined whether projects had developed project charters, work breakdown structures, and project management plans, and when they had, we compared the contents of these documents with project management practices in order to determine the extent to which critical elements were incorporated or executed on the projects. Specifically, we assessed whether project charters addressed important elements such as the project purpose or justification, the project manager’s responsibility and authority level, and the name and responsibility of the project sponsor. We assessed whether the work breakdown structures were deliverable-oriented hierarchical decompositions of the work to be executed and had associated dictionaries. Finally, we assessed whether project management plans addressed important elements such as the project life cycle, results of project tailoring, cost and schedule baselines, and subsidiary management plans.

To assess requirements management, we determined whether projects had developed requirements management plans and requirements traceability matrixes, and when they had, we compared the contents of these documents with best practices in order to determine the extent to which each program was applying specific elements. Specifically, we assessed whether requirements management plans addressed important elements such as configuration management activities, methods used to prioritize requirements, metrics, and a traceability structure. In addition, we assessed whether requirements identified in matrixes were, among other things, traceable to business needs, opportunities, goals, and objectives and whether the matrixes included essential information such as requirements change requests and status.

To assess acquisition planning, we determined whether the modernization initiatives had developed acquisition strategies and, when they had, compared the contents of these documents with key practices to determine the actions HUD is taking to ensure that the acquisitions for FHA Transformation and NGMS are planned in accordance with best practices and guidance. Specifically, we assessed whether acquisition strategies addressed important elements such as the established dates for the contract deliverables, and procurement metrics.

We interviewed relevant HUD officials and staff in the FHA Transformation and NGMS project offices, including the General Deputy Assistant Secretary for Public and Indian Housing, the Director for the Office of Program Systems and Management, the Deputy Director of FHA Transformation, and the NGMS Program Manager. In addition, we interviewed officials from the department’s Chief Procurement Office, including the Deputy Chief Procurement Officer, and the Office of the Chief Information Officer, including the Acting Deputy Chief Information Officer for Business and IT Modernization, to obtain information on how these offices support the work of the two modernization efforts. Further, we attended and observed project status meetings, and related review sessions conducted by senior leadership, including HUD’s Deputy Secretary.

We determined that information provided by the department, such as work breakdown structures and requirements traceability matrixes, was sufficiently reliable for the purposes of our review. To arrive at this assessment, we conducted reliability testing by comparing information with statements from relevant department officials to identify discrepancies. However, we did not test the quality of certain information, such as cost and schedule data provided by the program offices.

We conducted this performance audit from June 2012 to June 2013, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provided a reasonable basis for findings and conclusions based on our audit objective.

Appendix II: Summary of HUD’s Transformation Initiative Information Technology Modernization Efforts

Appendix II: Summary of HUD’s Transformation Initiative Information Technology Modernization Efforts Description Develop and implement a modern financial services IT environment to better manage and mitigate risk across FHA’s insurance programs for single-family housing, multifamily housing, and the insured healthcare portfolio.

Reengineer processes and implement an automated system for managing grants that will reduce application time, eliminate manual paper processes, and increase the transparency of grant management information.

Provide business intelligence and geospatial tools for documenting and assessing progress toward achieving strategic goals that will enhance transparency, reduce workload, increase employee productivity, and improve data quality.

Integrate human resources systems and tools to allow for automated recruitment and hiring documentation, reduction of manual data entry, and accelerated candidate decision making.

Integrate an acquisition management system that is compliant with federal regulations to reduce inefficiencies, time, and duplication in the procurement process across office locations to expedite services rendered to the public.

Modernize and replace financial management systems through an outsourced shared services provider.

Reengineer management processes to establish a technical infrastructure that will integrate disparate systems and provide consistent information in order to support rental housing assistance services.

Appendix III: Detailed Assessment of FHA Transformation and NGMS Project Management Documentation

Our assessment of FHA Transformation and NGMS implementation of key project management documentation in the areas of project planning (charters, work breakdown structures, and project management plans), requirements management (requirements management plans and traceability matrixes) and acquisition planning (acquisition strategies) is described below.

Project Charters

Best practices recognize the development of a project charter as an integral step in project planning for establishing and maintaining project teams. A charter formally authorizes a project and identifies high-level information that constitutes and assigns responsibility for project success. This is a critical artifact for creating project management plans, documenting business needs, and outlining the result a project is intended to achieve. Specifically, to be effective, project management practices state that a charter should include high-level requirements; high-level project description; high-level risks; measurable objectives and related success criteria; project approval requirements (e.g., what results in project success the project’s purpose or justification; summary milestone schedule; summary budget; and who is responsible for final sign off); assigned project manager, responsibility and authority level; and the name and authority of the sponsor or other person(s) authorizing the project charter.

According to best practices, a work breakdown structure is the cornerstone of every project because it defines in detail the work necessary to accomplish a project’s objectives and provides a basic framework for a variety of related activities like estimating costs, developing schedules, identifying resources, and determining where risks may occur. In the context of the work breakdown structure, work refers to work products or deliverables that are the result of effort and not to the effort itself. Creating a work breakdown structure involves subdividing (or decomposing) project deliverables and work into smaller, more manageable components (called work packages) that can be scheduled, cost estimated, and managed. According to best practices, the work breakdown structure is a deliverable-oriented hierarchical decomposition of the work to be executed by the project team to accomplish the project’s objectives and create the required deliverables. Further, these practices state that a work breakdown structure should also represent the entire scope of the project and product work, including project management, and it should be standardized to enable an organization to collect and share data among projects. In addition, it should be accompanied by a dictionary of the various work breakdown structure elements that describes in brief narrative form what work is to be performed in each element.

As we have previously reported, agencies need to develop comprehensive project management plans, and best practices emphasize the importance of having a plan in place that, among other things, establishes a complete description that ties together all activities and evolves over time to continuously reflect the current status and desired end point of the project. According to project management practices, a project management plan is the primary source that defines, among other things, how the project is to be executed and controlled. Project management plans can be either summary level or detailed and can be composed of one or more subsidiary plans to address elements of project management. To be effective, best practices state that a project management plan integrates cost and schedule baselines from planning activities, and this baseline information should be updated as needed and periodically compared with actual performance data in order to track and report progress. While the content of a project management plan will vary depending upon the type and complexity of a project, it is developed through a series of integrated processes and is progressively elaborated by updates during the execution and management of a project. Such plans include identification of the projected life cycle and processes to be applied to each phase; results of project tailoring; how the team will execute the work to accomplish project objectives; project cost and schedule baselines; how the team will maintain the integrity of performance measurement a change management plan that documents how changes will be baselines; the needs and techniques for communicating among stakeholders; key management reviews; a configuration management plan to define those items that are configurable, those items that require formal change control, and the process for controlling change to such items; and subsidiary management plans (scope, requirements, schedule, cost, quality, process improvement, human resources, communication, risk and procurement).

According to project management practices, effective planning of requirements includes documenting the processes and methods to be used for effectively developing and managing requirements from initial identification through implementation. A project’s success is directly influenced by the care taken in capturing and managing requirements. Other essential planning activities such as developing a work breakdown structure or estimating a project’s cost and schedule are built upon requirements developed. Best practices state that in establishing requirements, project teams should plan requirements collection activities such as conducting interviews, focus groups, facilitated workshops, or other techniques, including surveys and prototypes. Depending on the type of project, the approach for managing requirements can vary, but requirements management plans document the approach for how requirements development activities will be conducted. In particular, a plan includes how requirements activities (e.g., collecting requirements) will be planned, tracked, and reported; configuration management activities such as how changes will be initiated, analyzed, and managed; requirements prioritization methods; product metrics that will be used and the rationale for using them; and a traceability structure outlining attributes for a traceability matrix and identifying what other project documents requirements will be traced to.

Project management practices state that requirements traceability matrixes are designed to support backward traceability by linking each requirement to the broader business objective it supports and forward traceability by linking these requirements to more detailed functional requirements. Traceability refers to the ability to follow a requirement from origin to implementation and is critical to understanding the interconnections and dependencies among the individual requirements and the impact when a requirement is changed. This bidirectional traceability can help management determine whether the project addresses all requirements and that those requirements and the related deliverables are traceable back to valid business needs. According to best practices, requirement matrixes provide tracing to business needs, opportunities, goals and objectives; high-level requirements to more detailed requirements; criteria used for evaluation and acceptance of the requirements; a set of approved requirements; and status of requirement changes and requests.

Further, matrixes should have specific attributes recorded for each requirement. Attributes associated with each requirement—such as a unique identifier, textual description, priority, version, current status, and date completed—should be recorded.

According to best practices, effective IT project management also involves early planning for and management of acquisitions. The planning process begins with the identification of project needs which can best be, or must be, met by acquiring products, services, or results outside of the organization. During planning, coordination of the acquisition with other project management activities, such as budgeting, scheduling, resource estimating, risk identification, and requirements definition, should be discussed and documented. Most organizations have documented policies and procedures specifically defining mandatory acquisition activities for obtaining contracted goods or services. The acquisition planning process should result in a plan or strategy that describes how management decisions will be applied for a particular project. Such strategies serve as the road map for effectively planning and managing acquisitions from initiation through contract closure. In particular, project management practices indicate that acquisition strategies should provide guidance for defining the types of contracts to be used; addressing risk management issues; coordinating procurement with other project aspects, such as scheduling and performance reporting; setting the scheduled dates in contracts for determining dates for deliverables and coordinating with other project management processes; and establishing procurement metrics to be used in managing and evaluating contractors.

Appendix IV: Comments from the Department of Housing and Urban Development

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact above, Teresa M. Neven (Assistant Director), Kami J. Corbett, Amanda C. Gill, Lee A. McCracken, John Ockay, and Shannin G. O’Neill made significant contributions to this report.