Information Technology: HUD Needs to Improve Key Project Management Practices for Its Modernization Efforts
Published: 2013-06-12T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: HUD relies extensively on IT to carry out its mission of strengthening communities and ensuring affordable housing and has reported that efforts are under way to modernize its aging, duplicative, and poorly integrated systems. Committee report language mandated GAO to evaluate the implementation of project management practices for HUDs IT modernization efforts. The objective was to identify the extent to which the department implemented key project management practices for the FHA Transformation and NGMS modernization efforts. GAO assessed project management artifacts for 9 FHA Transformation and 5 NGMS projects in the areas of project planning (charters, work breakdown structures, and project management plans), requirements management (requirements management plans and traceability matrixes), and acquisition planning (acquisition strategies) against best practices. GAO also interviewed officials.
section_title: What GAO Found
paragraphs: The Department of Housing and Urban Development (HUD) has taken initial steps toward applying key project management practices in the areas of project planning, requirements management, and acquisition planning for its Federal Housing Administration Transformation (FHA Transformation) Initiative to address performance gaps in housing insurance programs and its Next Generation Management System (NGMS) to improve management of its affordable housing programs. However, HUD has not yet fully implemented any of these practices in executing and managing the information technology (IT) projects associated with these efforts. Specifically, while the department had developed project management artifacts such as charters and requirements management plans, none of these documents included all of the key details that could facilitate effective management of its projects such as full descriptions of the work necessary to complete the projects, cost and schedule baselines, or prioritized requirements, among other things. Department officials attributed these deficiencies to a lack of project management expertise.
Because HUD has not taken these foundational steps to fully define its modernization efforts, the department is not well positioned to successfully manage or execute the associated projects. These incomplete documents limit the department's ability to fully understand the work to be completed or accurately report project progress. A major reason for these information deficiencies is HUD's inadequate development and use of its project management framework, which did not ensure the quality or completeness of artifacts developed. Specifically, the framework did not always include essential guidance and, in other cases, the projects did not always implement the guidance provided. Further, the governance structure did not consistently operate as intended to provide adequate oversight to ensure compliance with key project management practices. As a result, the department increases the risk of continuing to inadequately apply project management practices and may not be positioned to effectively manage or report progress of its key modernization efforts. Fully implementing effective project management practices is critical for the success of these two modernization efforts and others under way or planned.
section_title: What GAO Recommends
paragraphs: GAO recommends that HUD establish a plan of action to fully implement best practices, provide needed project management expertise, and improve the development and use of its project management framework and governance structure. In written comments, HUD concurred with the recommendations to improve its framework and governance, but did not concur with the entirety of the recommendation to develop a plan of action, and contended that the need for project management expertise did not follow from the premises established in the draft report. GAO maintains that these actions are necessary as discussed in this report.

section_title: Letter
paragraphs: The Department of Housing and Urban Development (HUD) relies  extensively on information technology (IT) to carry out its mission of  creating strong, sustainable, inclusive communities and ensuring  affordable housing. According to the department, in fiscal year 2012, IT  systems supported HUD programs valued at more than $1 trillion.  However, the department has reported that its systems are overlapping,  duplicative, not integrated, necessitate manual workloads, and employ  antiquated, costly-to-maintain technologies. Further, as we and others  have reported, the department has long experienced shortcomings in its  IT management capability. For example, a 2010 assessment of the  department’s IT environment concluded that HUD lacked sufficient  capability to execute basic management functions, including project  management, which inhibited effective IT operations and the successful  delivery of IT solutions.
For fiscal years 2010 and 2011, HUD invested in developing and  modernizing its systems through funding provided by the department’s  Transformation Initiative. The department has been pursuing seven IT  modernization efforts under this initiative, the two largest of which are the  Federal Housing Administration Transformation (FHA Transformation)  Initiative to address performance gaps in housing insurance programs  and the Next Generation Management System (NGMS) to improve  management of the department’s affordable housing programs. To  provide oversight and inform decision making, Congress established  limitations on funding for the Transformation Initiative IT modernization  efforts and required the department to submit expenditure plans that  satisfied statutory conditions before fully obligating the available funds.  We were required to review the department’s expenditure plans and,  accordingly, have reported on the results of our assessments.  Concurrent with the modernization funding, the department also began  implementing reforms to improve its IT management controls and guide  its investments, including the establishment of a framework for planning  and managing such projects.
Given the important role that HUD’s modernized systems are to have in  supporting its mission, the Senate Report accompanying the  department’s fiscal year 2012 appropriation, as approved by the  Conference report, mandated us to evaluate the implementation of project  management practices for its IT modernization efforts. In this regard, our  specific objective was to identify the extent to which the department has  implemented key project management practices for the FHA  Transformation and NGMS modernization efforts.
To address this objective, we examined all of the 14 projects for FHA  Transformation and NGMS that had been identified in the department’s  fiscal year 2011 expenditure plan. We assessed these projects against  key IT management practices in the areas of (1) project planning, (2)  requirements management, and (3) acquisition planning. As part of our  analysis, we reviewed HUD project management documentation for the  14 FHA Transformation and NGMS projects, including charters, work  breakdown structures (which define the necessary work and provide a  basis for cost and schedule estimates), project management plans,  requirements management plans and traceability matrixes (which provide  the ability to follow a requirement from origin to implementation), and  acquisition strategies. We assessed whether this documentation was  consistent with project management practices identified by the Project  Management Institute (PMI) and the Software Engineering Institute (SEI),  and best practices we previously identified for cost estimating and project  schedules. We also attended the department’s project management  meetings to observe how the identified practices were being applied. In  addition, we interviewed responsible HUD officials, including FHA  Transformation and NGMS management officials, procurement officials,  and responsible IT officials. Appendix I provides additional details  regarding our objective, scope, and methodology.
We conducted this performance audit from June 2012 to June 2013 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objective. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objective.
subsections: section_title: Background
paragraphs: To accomplish its mission, HUD administers community and housing  programs that benefit millions of households each year. Among other  things, the department provides affordable rental housing opportunities  and helps homeless families and chronically homeless individuals and  veterans. The department also administers mortgage insurance programs  for single-family housing, multifamily housing, and health care.
HUD relies on five main organizational components to carry out its  mission. Of these, two components have lead responsibility for improving  access to housing and are the business owners for related IT  modernization efforts:
Housing/Federal Housing Administration (FHA): Programs within  this office are responsible for contributing to building healthy  communities, maintaining and expanding housing opportunities, and  stabilizing credit markets in times of economic disruption. This office  also regulates certain aspects of the housing industry. For example,  the department currently reports that it provides insurance on loans  made by its approved lenders for 4.8 million single-family mortgages  and 13,000 multifamily projects, including manufactured homes and  hospitals. The FHA Transformation modernization effort is managed  within this office.
Public and Indian Housing: Programs within this office are  responsible for creating opportunities for residents’ self-sufficiency  and economic independence. Toward this end, this office currently  oversees a housing choice voucher program to subsidize housing for  approximately 2.1 million low-income, elderly, and disabled families; a  public housing program that subsidizes about 1.3 million housing units  for vulnerable low-income families; and block grants and guarantee  programs for Native American groups. The NGMS modernization  effort is managed within this office.
In addition, to support these organizational components, the department  relies on various administrative offices to provide guidance and tools.  These include the department’s Office of the Chief Information Officer  (OCIO) and the Office of the Chief Procurement Officer (OCPO). Through  coordination with the organizational components, OCIO manages IT  resources and provides support for the department’s infrastructure,  security, and ongoing projects. This office also provides project  management guidance and technical expertise to modernization efforts.  For its part, OCPO is responsible for obtaining contracted goods and  services required by the department to meet its strategic objectives. This  office is involved with initiating acquisition actions upon request by the  organizational components.
Further, HUD’s Deputy Secretary is responsible for managing the  department’s daily operations, annual operating budget, and  approximately 8,900 employees. As part of this role, the Deputy Secretary  conducts biweekly meetings with stakeholders to discuss the Secretary’s  priorities. During these meetings, the scope, milestones, risks, and status  of action items related to priority issues are discussed. The FHA  Transformation and NGMS modernization efforts are designated as  priority and each has its own biweekly meeting.
A simplified view of the department’s housing organization structure and  the offices responsible for FHA Transformation and NGMS is provided in  figure 1.
According to the fiscal year 2014 President’s Budget request for HUD,  $285.1 million is expected to be spent on IT investments. HUD’s IT  environment consists of multiple systems that, among other things, are  intended to help the department coordinate with lending institutions to  insure mortgages, collect and manage state and local housing data,  process applications for community development, and issue vouchers  that provide access to subsidized housing. In particular, the department’s  housing programs rely on systems for processing and managing these  business operations. For example, systems within the Office of Housing  are expected to process mortgage insurance applications, bill and collect  premiums, pay claims, manage receivables and other assets, track  delinquencies and defaults, and support staff in providing counseling to  first-time home buyers and existing homeowners. Additionally, Public and  Indian Housing programs that use systems are intended to process  vouchers for different rental assistance programs, as well as to support  the processing of applications for, and the management of, more than 50  grant programs administered by the department.
However, HUD’s current IT environment has not effectively supported its  business operations because its systems are overlapping and duplicative,  not integrated, necessitate manual workloads, and employ antiquated  technologies that are costly to maintain. For example, the department  reported from 2008 to 2012 that its IT environment consisted of:
Over 200 information systems, many of which perform the same  function and, thus, are overlapping and duplicative. Specifically,  different systems perform the same task to separately support grants  management, loan processing, and subsidies management.
Stovepiped, nonintegrated systems that result in identical data  existing in multiple systems. For example, two organizational  components store about 80 percent of similar data in separate  databases that provide information on rental assistance participants.
Manual processing for business functions due to a lack of systems to  support these processes. For example, specific NGMS projects are  intended to replace existing ad hoc analyses performed in  spreadsheets and databases with systems that automate and  standardize those functions.
Antiquated technology (15- to 30- years old) and complex systems  that are costly to maintain. For example, the department relies on  different programming languages and operating systems, which  requires specialized skills to operate and maintain.
Additionally, HUD engaged contractors to conduct an assessment of the  department’s environment. This assessment, issued in January 2011,  concluded that unclear reporting relationships hindered the enforcement  of IT policies; contractor performance information was not used to inform  management decisions; technical standards were lacking or not enforced;  and data management practices did not support business needs.
subsections: section_title: HUD’s Transformation Initiative IT Modernization Efforts
paragraphs: Through the Transformation Initiative’s IT component, HUD has begun  addressing challenges to its environment and modernizing its systems. In  this regard, the department initiated seven IT modernization efforts, of  which FHA Transformation and NGMS are the two largest. For fiscal  years 2010 and 2011, the department reported that the Transformation  Initiative funding made available for FHA Transformation and NGMS was  $58.5 and $41.1 million, respectively. (See later discussion in this report  regarding costs associated with the 14 projects in our study.)
FHA Transformation was initiated to improve the department’s  management of insurance programs through the development and  implementation of a modern financial services IT environment that is  expected to improve loan endorsement processes, collateral risk  capabilities, and fraud prevention. In August 2009, HUD published the  FHA Office of Housing Information Technology Strategy and  Improvement Plan, which identified and prioritized 25 IT areas with  performance gaps for its single-family housing, multifamily housing  development and rental assistance, health care facilities programs, and  enterprise applications. In May 2010, FHA Transformation began  planning and executing modernization efforts aimed at addressing the  gaps identified in the plan. Specifically, the modernization initiative is  intended to implement technology within the following four functional  areas aimed at addressing changes in FHA’s business model, operating  environment, and components of the loan life cycle:  Infrastructure and legacy migration: Provide a scalable  infrastructure to support rules engines, analytics, and reporting  systems, as well as a mechanism for transferring legacy applications  to the new platform. Specifically, the Federal Financial Services  Platform project is intended to provide hardware and standard  software to support case management and migration of legacy  applications (e.g., the Computerized Homes Underwriting Reporting  System) for all lines of business.
Borrower/collateral risk management and fraud monitoring:  Provide tools for analyzing, monitoring, and managing emerging  issues and trends in the housing market, including borrower and  collateral risk, appraisals, and fraud, as it relates to the FHA portfolio.  For example, the Legacy Application Transformation project is  expected to implement a software service tool that aggregates data to  identify emerging issues and trends in borrower risk and fraud by  analyzing the accuracy and validity of verified assets, income, and  employment on individual loans. Other projects within this functional  area include business process reengineering and a pilot designed to  automate and streamline the multifamily housing underwriting  process. Using the new infrastructure, an automated underwriting tool  is expected to be deployed to expand the capabilities for processing  loan applications for insurance programs and replace current systems  (e.g., the Development Application Processing System).
Counterparty management: Provide applications for improved  performance and compliance of lenders and appraisers through more  proactively identifying risk trends and improving loan file review  techniques. Specific projects include the Lender Electronic  Assessment Portal, a web-based automated delivery of electronic  applications and storage of lender application data that assists with  reviewing new lender applications and requests for annual  recertification to participate in FHA programs. Future plans call for  replacing seven legacy applications.
Portfolio analysis: Provide tools intended to augment risk monitoring  and management; enhance predictive analytics; provide timely and  flexible reporting; and deliver more accurate, detailed information to  decision makers. For example, the Portfolio Risk Reporting &  Analytics project is intended to provide a web-based software service  tool for modeling FHA program risks. While initial use of the software  is to include receiving hard-copy reports from the third-party vendor,  HUD also expects to deploy the tool within the department’s  infrastructure in order for FHA employees to have access to reports  and the analytics dashboard data electronically.
Overall leadership for FHA is provided by the Assistant Secretary for  Housing/Federal Housing Commissioner, who chairs the modernization  effort steering committee; the General Deputy Assistant Secretary for  Housing; and the Director for the Office of Program Systems  Management, who is the executive sponsor. The modernization effort  also has a project management office that is responsible for executing  and managing the associated projects. As of April 2013, FHA  Transformation consisted of 10 projects, 9 of which were included in our  study. Table 1 summarizes the 9 FHA Transformation projects that we  assessed as part of our study.
The NGMS modernization effort is intended to provide an integrated  system with a seamless view of financial and program data currently  warehoused in disparate data sources and a new set of monitoring,  oversight, and software tools directed at ensuring that funds are used to  assist affordable housing participants and reduce improper payment  errors. In November 2011, the department used contractors to develop  four prototype software tools aimed at demonstrating anticipated NGMS  functionality for voucher programs. However, in July 2012, the  department determined that the prototypes that had been developed  would not address its business needs. As a result, the department  initiated planning efforts to restructure the modernization effort and  expand the scope to include all Public and Indian Housing lines of  business.
HUD has reported that the aim of the restructured effort is to enhance the  department’s affordable housing program, improve end-user satisfaction,  streamline complex business processes, and integrate disparate IT  systems into a common, modernized platform. The department intends  for NGMS to support efforts to improve HUD’s financial accountability by  more accurately quantifying budgetary data resources, measuring  program effectiveness, and justifying the agency’s budget formulations  and requests. NGMS is expected to help department personnel reduce  improper payments by identifying anomalies in operating costs, reserves,  and subsidy payments. Once implemented, NGMS is intended to provide  staff with a new set of monitoring, oversight, and analysis tools to ensure  that allocated federal funds are used efficiently to assist affordable  housing participants. The department is taking an incremental approach  to developing NGMS and expects to deliver initial functionality by August  2013. NGMS system and software development projects are designed to  support four functional areas:
Financial management: Provide automated processes for budget  forecasting and formulation and cash management based on real-time  data that are expected to allow the department to anticipate cash flow  needs through precise scenarios and disburse funds on the basis of  project and tenant records, eliminating reconciliations. For example,  the Budget Forecasting and Formulation project is intended to  develop a solution that will include forecasting functionality, data  aggregation, and analytics to support the budget development  process for Public and Indian Housing programs such as vouchers,  administrative fees, family self-sufficiency, mainstream vouchers, and  housing assistance programs. In addition, this functional area is  expected to migrate data from HUD’s Central Accounting and  Program system and utilize information gathered from Public Housing  Authorities regarding subsidized housing programs through an  interface with the department’s New Core system.
HUD operations: Provide a single point of access to data and  information to improve efficiency and reduce administrative burden  through a New Data Collection system that is expected to replace  legacy systems (e.g., Public and Indian Housing Information Center)  and provide new functionality for subsidized housing programs,  geospatial data on physical housing, real-time occupancy information,  and energy conservation measures for properties. In the interim, the  Portfolio and Risk Management Tool project is expected to provide  aggregated data about Public Housing Authorities through a standard  business intelligence solution and is expected to expand its use to  partner operations in the future.
Partner operations: Expand the department’s operations system to  provide a web-based single point of access for gathering consistent  and accurate information from families and landlords to be used in the  operation of public housing and voucher programs administered by  Public Housing Authorities.
Business support: Provide expanded access and use of NGMS IT  solutions; grant HUD and program participants better access to  information and technical assistance through a central point of access  with live help and self-paced guides; and develop the necessary  infrastructure and processes to enable timely and accurate answers to  end users’ inquiries.
Overall leadership for NGMS is provided by the General Deputy Assistant  Secretary for Public and Indian Housing, who is the executive sponsor  and chair of the modernization effort steering committee. The  modernization effort also has a project management office that is  responsible for executing and managing the associated projects. As of  April 2013, NGMS consisted of six projects, of which five were included in  our study. Table 2 summarizes the NGMS projects that we assessed.
Effective use of project planning and management practices is essential  for the success of modernization efforts such as those being undertaken  by HUD. Our prior reviews of federal agencies have shown that, when  effectively implemented, these practices can significantly increase the  likelihood of delivering promised capabilities on time and within budget.  Moreover, project management maturity is dependent on an agency’s  standardization and institutionalization of such practices. PMI reported in  its March 2013 annual survey of project management professionals that  high-performing organizations are almost three times more likely than  low-performing organizations to use standardized practices throughout  the organization, and generate better project outcomes.
To guide the application of best practices, we and others, including PMI  and SEI at Carnegie Mellon University, have issued reports and  frameworks for effective project management. These reports and  frameworks emphasize practices that include the development of  essential documentation needed for the execution and management of  projects in the areas of project planning (charters, work breakdown  structures, and project management plans), requirements management  (requirements management plans and traceability matrixes), and  acquisition planning (acquisition strategies).
Project planning: This practice helps establish project objectives and  outline the course of action required to attain those objectives. It also  provides a means to track, review, and report progress and  performance of the project by defining project activities and  developing cost and schedule estimates, among other things. Project  planning involves, for example, creating a charter to authorize project  work, developing a work breakdown structure, and establishing  project management plans that provide processes for measuring  progress.
Requirements management: Having a documented strategy for  developing and managing requirements can help ensure that the final  product will function as intended. Effective management of  requirements involves assigning responsibility for them, tracking them,  and controlling changes to them over the course of the project. It also  ensures that each requirement traces back to the business need and  forward to its design and testing. Requirements management  practices call for the use of requirements management plans to  provide a mechanism for documenting the process for managing  requirements and associated traceability matrixes, which are intended  to facilitate efforts to link requirements to identified business needs to  help ensure that they will be satisfied by the end product.
Acquisition planning: Effective IT project management also involves  creating strategies to serve as the road map for acquisition planning.  Such road maps are used for early planning of procurements and are  developed by a project manager. Among other things, acquisition  strategies should address plans for how projects will manage risks,  deliverables, and reporting on contractor performance.
In addition to calling for agencies to apply best practices, federal  guidance, along with our framework for managing IT investments and our  prior reviews of federal investments, outlines the importance of having  reviews conducted by management at various points throughout a  project’s life cycle. Such reviews are critical to helping ensure that cost,  schedule, and performance goals for a project are satisfied, and they can  provide early detection of risks and problems that could impede progress  toward those goals. Further, management reviews can help ensure that  appropriate quality standards are achieved and provide input for areas  that need improvement.
section_title: HUD’s IT Project Management Framework and Governance Structure
paragraphs: In order to better manage its modernization efforts, during 2011 HUD  established new policies and procedures for executing and governing IT  investments. Specifically, in April 2011, the department developed a  Project Planning and Management (PPM) framework to provide  guidance for managing a project’s life cycle in accordance with best  practices. Using the framework, projects—such as those related to FHA  Transformation and NGMS modernization efforts—are expected to  proceed through life-cycle phases that require specific documents to  demonstrate project activities and outcomes. The framework provides  guidance through sample templates, with associated instructions and  checklists that projects can use in developing their documentation. The  framework also calls for management reviews that are intended to help  ensure projects are aligned with the department’s architecture and  technical standards, and that they have developed required information  before committing resources to the next life-cycle phase. For example, at  the initiation of a project, among other things, a charter and schedule are  expected to be developed and approved by a review committee. In  addition, during a project’s definition phase critical documents, such as a  project management plan, requirements management plan, requirements  traceability matrix, and an acquisition strategy, are also expected to be  developed and approved by a review committee.
In July 2011, HUD also established a governance policy that set forth  processes, standards, roles, and responsibilities to facilitate decision  making around investments, stakeholder relationships, project life-cycle  management, and other important IT operational areas. In particular, the  policy established an IT governance structure consisting of the Executive  Investment Board, Customer Care Committee, Investment Review  Subcommittee, and Technical Review Subcommittee. Figure 2 provides a  simplified depiction of this governance structure.
According to HUD’s Policy for Information Technology Governance  handbook, these governance bodies have the following composition and  responsibilities.
Executive Investment Board: Comprised of senior leaders, including  the HUD Secretary, Deputy Secretary, and Chief Information Officer,  with responsibilities for providing strategic direction, managing the IT  investment portfolio, and overseeing and approving projects that cost  more than $5 million.
Customer Care Committee: Comprised of executives, including the  Chief Information Officer, the Chief Procurement Officer, and deputy  assistant secretaries, who manage IT investments and perform  project management oversight by reviewing and submitting  recommendations to the Executive Investment Board, and  coordinating with the subcommittees responsible for approving  projects that cost between $500,000 and $5 million. respect to business cases and budget information for the Office of  Management and Budget (OMB).
Technical Review Subcommittee: Comprised of personnel from  within OCIO, including the Chief Technology Officer, the Chief  Architect, and the Chief Information Security Officer, with a focus on  ensuring that the technical architecture is aligned with the  department’s strategic goals and monitoring IT projects through  conducting control gate reviews that assess whether all necessary  documentation has been produced. The subcommittee is also  responsible for approving projects that cost less than $500,000.
section_title: HUD Has Begun Applying Key Practices for Its Modernization Efforts but Has Been Hindered by Inadequacies in Its Project Management Framework and Governance
paragraphs: For its FHA Transformation and NGMS modernization efforts, HUD has  taken initial steps toward applying key project management practices in  the areas of project planning, requirements management, and acquisition  planning. However, the department has not yet fully implemented any of  these practices in managing the 14 projects in our review. In large part,  these deficiencies can be attributed to inadequate development and use  of the department’s project management framework and governance  structure. Without fully implementing these practices and effectively  developing and using its framework and governance structure, HUD risks  investing its resources on projects that may not meet critical mission  needs.
According to the Project Management Institute and the Software  Engineering Institute, disciplined project management practices call for  the development of project details such as objectives, scope of work,  schedules, costs, and requirements against which projects can be  managed and executed. This step can be facilitated by developing  project artifacts that include, among other things, charters to authorize  projects and assign responsibility for their execution, work breakdown  structures to define the work that needs to be done to accomplish project  objectives, project management plans to define how projects are to be  executed and controlled, and requirements management plans to  document the processes and methods to be used for developing and  managing project requirements. Further, developing requirements  traceability matrixes that provide linkages between business objectives  and detailed system requirements, and establishing strategies to ensure  adequate acquisition planning are practices that contribute to effective  project management. Our prior reviews of federal agencies have shown  that applying these practices can significantly increase the likelihood of  delivering promised capabilities on time and within budget.
HUD had taken initial steps in applying key project management practices  by developing artifacts, to varying degrees, for the 9 FHA Transformation  and 5 NGMS modernization efforts in our review. Nevertheless, the  department lacked information needed for managing and executing the  projects because the documentation developed did not contain a number  of essential details that best practices stress as being critical to effectively  defining a project and measuring its success. In this regard, none of the  documentation included all of the critical information that could facilitate  effective project management, such as full descriptions of the work  necessary to complete the projects, cost and schedule baselines, or  prioritized requirements.
Table 3 summarizes the key project management practices in the areas  of project planning (charters, work breakdown structures, and project  management plans), requirements management (requirements  management plans and traceability matrixes), and acquisition planning  (acquisition strategies) for the 14 projects that we assessed. In addition,  appendix III provides our more detailed assessment against best  practices.
A project charter formally authorizes a project and identifies high-level  information that constitutes and assigns responsibility for project success.  According to project management practices, to be effective, a charter  should include, among other things, a project’s purpose or justification;  high-level information on such factors as requirements and risks,  measurable objectives, and related success criteria; a summary schedule  and budget; project approval requirements (e.g., information on what  factors will define project success and who will be responsible for final  sign off at the completion of the project); and names, responsibilities, and  authority levels of assigned leadership such as the project manager and  sponsor.
Of the 14 projects in our review, all 9 FHA Transformation and 3 of the  NGMS projects had developed charters that included most of the relevant  high-level information. For example, all of the charters included such  information as the project purpose, description, and high-level risks and  requirements, as well as the names of the assigned project managers.  Regarding measurable objectives and success criteria, 10 of the projects  included objectives, while 6 had related success criteria. Lastly, 10 project  charters included a summary schedule, and 1 included a summary  budget.
While most of the charters contained high-level information, other  essential details were not included such as the authority levels of project  leaders and the requirements for approving the completion of the  projects. Specifically, while each of the charters generally referenced  HUD’s PPM framework and the associated governance committees, the  charters did not explicitly state what results would constitute project  success (e.g., a specified number of project objectives met) or what  individuals or entities would be responsible for final sign-off at the  completion of the project. For example, FHA Transformation’s LEAP  Institution Manager project charter included the project’s purpose, high- level risks and requirements, and measurable objectives and related  criteria. Specifically, the charter noted that, by the end of fiscal year 2014,  the project would result in the retirement of four systems, eliminating the  associated costs for operations and maintenance. The charter also  incorporated a summary schedule and the names of its project manager  and sponsor. However, while the charter included a total budget figure, it  did not include details regarding the breakdown of the budget provided,  the responsibilities and levels of authority given to the manager and  sponsor identified, and the requirements for approving the completion of  the projects.
Additionally, the NGMS Integrated Budget Forecasting Model project  charter provided the project’s purpose, high-level risks and requirements,  the names of the project sponsors and managers, a summary schedule,  and measurable objectives with related success criteria. In particular,  regarding the measurable objectives, the charter stated that the project  would reduce the average time to respond to ad hoc requests for  budgetary reports and data from 3 days to 1 day. However, while the  charter referenced the OMB Capital Asset Plan and Business Case  Summary (Exhibit 300) for a list of associated costs, it did not include a  summary of the project’s expected budget. This charter also did not  include the responsibilities and the authority levels of the sponsors and  managers or the project completion requirements.
FHA Transformation and NGMS officials acknowledged the absence of  these details in the charters and attributed the deficiencies to the general  immaturity of the department’s project management practices. Regarding  the remaining two NGMS projects for which charters had not yet been  developed, project officials stated in April 2013, that one project was in  the process of developing a charter, while relevant information for the  other project was expected to be incorporated into the Budget  Forecasting and Formulation charter.
In the absence of charters that reflect all of the essential elements, HUD  lacks clear definitions of what will constitute success for its modernization  projects and has less ability to hold the responsible officials accountable  for this success. Moreover, the lack of important details that a charter is  intended to provide at the initial authorization of a project makes it more  difficult to undertake other project planning activities, such as developing  work breakdown structures, project management plans, and  requirements.
subsections: section_title: Work Breakdown Structures Had Not Been Fully Developed
paragraphs: A work breakdown structure is the cornerstone of every project because it  defines in detail the work necessary to accomplish a project’s objectives  and provides a basic framework for a variety of related activities like  estimating costs, developing schedules, identifying resources, and  determining where risks may occur. According to best practices, this  artifact should be a deliverable-oriented hierarchical decomposition of  the work to be executed by the project team to accomplish the project’s  objectives. Moreover, best practices state that it should represent the  entire scope of the project and product work, including project  management, and it should be standardized to enable an organization to  collect and share data among projects. In addition, it should be  accompanied by a dictionary that describes in brief narrative form what  work is to be performed in each of the various work breakdown structure  elements.
Of the 9 FHA Transformation and 5 NGMS projects, none had developed  complete work breakdown structures and associated dictionaries; only  one NGMS project—Budget Forecasting and Formulation—had a draft  work breakdown structure and associated dictionary. However, while this  draft work breakdown structure included details regarding the first  increment of the project, neither it nor the associated dictionary provided  details for any of the future planned increments. Thus, it did not reflect the  entire scope of the project and lacked descriptions of the work that would  be performed following the first increment, which is expected to deploy  initial functionality by late summer of 2013. Further, rather than being  organized by deliverables—that is, unique and verifiable products, results,  or capabilities—the draft was organized by life-cycle phases such as  definition and design. As a result, it did not allow for progress to be  measured by deliverable, which would enable more precise identification  and effective mitigation of the root causes for any cost or schedule  overruns. Moreover, developing a deliverable-oriented work breakdown  structure would show how deliverables relate to one another as well as to  the overall end product.
According to NGMS officials, plans are under way to fully develop work  breakdown structures that represent the first and second increments of all  projects in late spring 2013. Notwithstanding these plans, as of April  2013, a specific time frame for developing the work breakdown structures  and associated dictionaries for the third increment of NGMS projects had  not yet been determined.
Further, regarding the 9 FHA Transformation projects, in April 2013  officials stated that a work breakdown structure and dictionary to  represent the entire modernization effort is being developed. However,  the department was not able to provide a specific date for when this  documentation would be completed.
NGMS and FHA Transformation officials stated that work breakdown  structures were not initially developed for the projects because the PPM  framework did not require the completion of this artifact. The officials  added that, in addition to HUD project management practices lacking  maturity, their staff had not yet developed the expertise required to create  this artifact.
Until FHA Transformation and NGMS develop deliverable-oriented work  breakdown structures and associated dictionaries for all of their projects,  these efforts will lack critical information for understanding the detailed  work that needs to be performed to accomplish project objectives.  Further, by not defining the work to be performed, HUD cannot provide  reasonable assurance that cost and schedule estimates will capture all  the relevant information needed for the management of these efforts.
section_title: Project Management Plans Included Important Details but Lacked Baselines and Management Processes
paragraphs: According to project management practices, a project management plan  is the primary source that defines how a project is to be executed and  controlled. Best practices emphasize the importance of having such  plans in place to, among other things, establish a complete description  that ties together all project activities and evolves over time to  continuously reflect the current status and desired end point of the  project. Moreover, these practices state that to be effective, a project  management plan should identify life-cycle processes to be applied,  outline plans for project tailoring (i.e., determining what processes and  documentation would be necessary to accomplish project objectives),  provide communication techniques to be used, and list management  reviews. Further, building on the initial summary schedule and budget in  the charter, the project management plan should include baseline cost  and schedule estimates developed during planning activities. Moreover,  this baseline information should be updated as needed and periodically  compared with actual performance data in order to track and report  progress. Finally, the plan should include, or make reference to,  subsidiary management plans that describe how subordinate activities  are to be carried out for the project.
Six FHA Transformation and 5 NGMS projects had drafted or completed  project management plans that outlined life-cycle processes, identified  communication techniques and management reviews, and incorporated  certain subsidiary management plans. For example, the plan for FHA  Transformation’s Legacy Application Transformation project included an  approach for tailoring the life-cycle processes to be used; contained a  communication table with details about what techniques would be used;  and described different types of management reviews, including an  official team review and a structured walkthrough. Similarly, the plan for  NGMS’s Integrated Budget Forecasting Model project indicated that the  project was following HUD’s PPM framework, which includes tailoring life- cycle processes; contained communication techniques; and identified  different types of management reviews, including audit reviews and post- project reviews.
However, the plans provided for the 11 projects lacked other essential  information. Specifically, they did not clearly identify cost and schedule  baselines or consistently incorporate subsidiary plans. For example, the  plan for FHA Transformation’s MFH Development & Underwriting  Business Process Reengineering/Automated Underwriting Solution  project listed milestones, such as implementing a solution by July 2013,  and referenced a total cost of ownership artifact that indicated a cost of  $1.9 million. Yet, the plan did not indicate if these were considered to be  the project’s schedule and cost baselines against which progress would  be measured. The impact of the lack of clear baselines was evidenced by  inconsistencies between the project management plan and other project  documentation. For example, weekly status reports indicated that the  solution would be implemented by November 2013, rather than by the  July 2013 date identified in the project management plan. This lack of  clarity regarding the project’s cost and schedule baseline makes it difficult  to accurately measure and report progress against commitments made to  deliver functionality.
In a similar example, the project management plan for NGMS’s Integrated  Budget Forecasting Model provided project milestones and identified cost  estimates by life-cycle phase, but it did not specify if these figures  represented cost or schedule baselines developed as part of planning  activities. Additionally, the plan that reflected the other 4 NGMS projects  in our study included subsidiary plans for requirements, scope, schedule,  cost, quality, human resources, and risk management, but it did not  incorporate necessary details in the acquisition strategy and lacked one  for process improvement.
According to FHA Transformation and NGMS officials, the project  management plans did not include cost and schedule baselines, in part,  because the baseline information had been included in the updates  provided to OMB. However, including a project’s cost and schedule  baseline in a project management plan is important because the plan  serves as a primary source of information used to execute and manage  the project. In addition, such baseline information provides managers and  sponsors the foundational basis for measuring project progress. Relying  on information reported to an external entity such as OMB rather than on  cost and schedule baselines to manage projects may not allow the project  manager to have accurate real-time information available when  responding to stakeholder interests regarding the status of project  progress. Regarding the incorporation of subsidiary plans, the officials  stated that these plans were not required by the PPM framework to  complete control gate reviews and, as a result, were not fully addressed  in all of the project plans. Further, for the remaining 3 FHA projects that  had not yet developed project management plans, FHA Transformation  officials said the projects were still completing initial planning activities. In  accordance with the PPM framework, the projects would be expected to  develop plans when those activities are completed.
Until FHA Transformation and NGMS have comprehensive project  management plans that reflect cost and schedule baselines and fully  incorporate subsidiary plans for process improvement and acquisition  management, these modernization efforts will continue to lack a  foundational tool needed for successfully managing their projects and for  providing stakeholders with insight into the status of the projects.
section_title: Requirements Management Plans Lacked Details for Prioritizing and Measuring Requirements
paragraphs: According to project management practices, effective planning of  requirements includes documenting the processes and methods to be  used for developing and managing requirements from initial identification  through implementation. Such practices state that requirements  management plans should incorporate the approach for how  requirements development activities (e.g., collecting requirements) will be  conducted and how changes will be managed; identify methods for  prioritizing requirements; and specify the metrics to be used to measure  products against identified requirements, among other things. As we  previously reported, effective planning for requirements development and  management activities can reduce the risk of cost overruns, schedule  delays, and limitations in system functionality.
Seven of the FHA Transformation and all 5 of the NGMS projects in our  review had developed requirements management plans that documented  how requirements development activities would be conducted, including  managing changes. For example, the FHA Transformation’s Healthcare  Automated Lender Application Pilot project plan outlined processes for  how changes to requirements would be, among other things, evaluated  upon submission and analyzed for determining their impact on original  requirements in order for decisions to be made regarding proposed  changes. In addition, the NGMS Integrated Budget Forecasting Model  project plan indicated that requirements would be gathered through  interviews between the contractor and stakeholders, and provided  detailed policies and procedures for developing and maintaining  requirements. However, only 1 of the 12 projects—LEAP Automation of  Lender Approval Workflow—identified methods for prioritizing  requirements, and none of the projects established metrics for  determining the extent to which the products developed addressed  requirements. The over 2,400 functional requirements identified for the  NGMS Budget Forecasting and Formulation project illustrates the  significance of this point as the lack of prioritization could heighten the  difficultly developers may face in determining which among the many  requirements to focus on first.
FHA Transformation and NGMS officials stated that they had followed the  PPM framework template to develop the requirements management plans  for their projects. However, they added that the framework did not call for  prioritization methods to be identified in the requirements management  plan, and the department’s governance committee responsible for project  oversight did not provide feedback to indicate that the plans needed to  include this information. As of late April 2013, according to FHA  Transformation officials, the remaining 2 projects had not yet developed  requirements management plans because the projects were still in initial  planning.
Without establishing methods for prioritizing requirements, the  department will lack vital information needed to allocate resources in a  manner which ensures that higher-priority requirements are addressed  before lower-priority ones. In addition, until metrics for determining how  products address requirements are established, the department lacks the  ability to ensure that products will address business needs. As a result of  these missing details, HUD increases the risk that implemented solutions  may not effectively support the department’s mission.
section_title: Requirements Matrixes Provided Limited Traceability
paragraphs: According to best practices, the development of a requirements  traceability matrix is intended to link business needs outlined in high-level  requirements to more detailed requirements. Traceability refers to the  ability to follow a requirement from origin to implementation and is critical  to understanding the interconnections and dependencies among the  individual requirements and the impact when a requirement is changed.  Requirement matrixes provide tracing to, among other things, business  needs and the criteria used to evaluate and accept requirements. Further,  the use of attributes (e.g., a unique identifier, priority level, status, and  completion date) in the matrix helps define the requirement to facilitate  traceability. As we have reported, establishing and maintaining  traceability is important for understanding the relationships among  requirements—from the point at which business requirements are initially  established through the execution of test cases to validate the resulting  product.
Six FHA Transformation projects and 2 NGMS projects had developed  requirements traceability matrixes to track their requirements. However,  the eight matrixes that had been developed varied in the extent to which  project requirements linked detailed functional requirements backwards to  high-level business needs and forward to implementation. Further,  attributes intended to allow the original business needs to be traced to  detailed functional requirements were incomplete or missing. For  example, the FHA Transformation Portfolio Risk Reporting & Analytics  project matrix supported traceability of requirements back to higher-level  business goals and also provided specific attributes such as unique  identifiers. However, the matrix did not link the high-level requirements  outlined in the project’s requirements definition documentation to more  detailed requirements or trace to documentation that described criteria to  be used for evaluation and acceptance of requirements.
In addition, the NGMS Integrated Budget Forecasting Model matrix  included requirements that were traced from high-level to more detailed  requirements and recorded specific attributes such as a unique identifier  and the current status. However, the matrix did not provide traceability to  criteria for evaluating and accepting requirements or consistently record  accurate information regarding the current status of the requirements. For  instance, the matrix included two identical requirements, but each stated  requirement had a different disposition: the information for one  requirement indicated that it had been “completed” while information for  the other requirement included a notation of “discontinued,” but without  any associated dates to clarify which disposition accurately described  whether the requirement had been implemented. Further, the NGMS  Budget Forecasting and Formulation project developed a matrix that used  a unique identifier that allowed traceability from 15 high-level  requirements to more detailed functional requirements. In addition, these  requirements and the traceability matrix were approved by the  appropriate stakeholders. However, the matrix did not document several  other attributes, including status, or provide traceability to criteria for the  evaluation and acceptance of these requirements. In particular, the matrix  did not establish priorities for requirements to aid in ensuring that those of  highest priority are addressed first.
According to FHA Transformation and NGMS officials, the PPM  framework guidance was used in creating the matrixes and, in many  cases, the projects relied on contractors to complete the artifacts. In  addition, according to these officials, project resources were focused on  providing the documentation required by the framework and associated  governance committee. As a result, information that was not explicitly  identified as being required in an artifact, such as matrixes that  demonstrate traceability, was not developed. With regard to the remaining  3 FHA Transformation and 3 NGMS projects, department officials said the  projects were still completing initial planning activities and had not  reached a point where requirements had been defined to populate a  matrix.
The incomplete state of the requirements traceability matrixes makes it  unclear what mission needs have been addressed by project functional  requirements and are planned to be implemented in a solution. Without  fully traceable requirements for each project, the FHA Transformation and  NGMS modernization efforts are limited in their ability to know whether  necessary requirements are being implemented or if those being  implemented support defined business needs.
section_title: Strategies Lacked Details for Guiding Project Acquisitions
paragraphs: Best practices also state that effective IT project management involves  creating a strategy for acquisition planning. The strategy should be  based on the needs of each individual project and can be formal or  informal, and highly detailed or broadly framed. The strategy should also  be incorporated as a subsidiary component of the project management  plan. An acquisition strategy serves as the road map for effective  acquisition planning and should document the types of contracts to be  used, address contract risks, determine dates for deliverables, and  coordinate contracts with other processes, such as scheduling and  performance reporting. Additionally, the strategy should reflect early  identification of metrics to be used in managing and evaluating  contractors to help ensure that business needs are addressed through  contracted support.
FHA Transformation and NGMS each developed one acquisition strategy  that was intended to represent all the projects being undertaken by their  respective modernization efforts. In addition, while the acquisition strategy  for NGMS was intended to represent all the projects, one project— Integrated Budget Forecasting Model—also developed its own individual  strategy. These three strategies identified the types of contracts (e.g.,  time and materials, firm-fixed price, or interagency agreement) that were  planned to be awarded for their associated projects. For instance, the  FHA Transformation strategy stated that indefinite-delivery, indefinite- quantity contracts would be awarded and associated task orders would  be firm-fixed price, time and materials, or labor hour. The NGMS strategy  stated that it would utilize existing interagency agreements and work with  small disadvantaged businesses for its contract needs. Further, the  NGMS Integrated Budget Forecasting Model project’s separate  acquisition strategy identified the type of contract to be used (i.e., blanket  purchase agreement with firm-fixed price task orders), addressed contract  risks (e.g., the unavailability of server space), and determined dates for  deliverables (e.g., create and update detailed functional requirements  between January 20 and February 10, 2011).
Nonetheless, the three strategies did not fully document details needed  for effective acquisition planning, such as information on how risks would  be addressed, determining dates for deliverables, coordinating with other  processes, and identifying metrics needed for evaluating contractors. For  example, the FHA Transformation acquisition strategy identified dates for  projects, but did not state how contract dates would be coordinated with  schedule processes. Moreover, both the NGMS strategy and the  individual strategy for the Integrated Budget Forecasting Model project  did not state how other project processes, such as requirements  development, would be coordinated with acquisitions or identify metrics  for assessing contractors’ performance.
FHA Transformation and NGMS officials stated that the strategies  developed were based on the PPM framework template and that the  strategies had been approved by the Technical Review Subcommittee,  which did not identify the deficiencies. Further, while a strategy should  guide acquisition planning, OCIO officials said the requirement in the  PPM framework did not call for developing strategies prior to awarding  contracts.
Without strategies that guide planning activities in order to ensure that  acquisitions are managed in accordance with other processes and  provide performance metrics, the department increases the risk that  acquisitions associated with its modernization efforts will not be effectively  managed and that acquired services or products will not meet its needs.
section_title: Inadequate Development and Use of HUD’s Project Management Framework and Governance Contributed to FHA Transformation and NGMS Weaknesses
paragraphs: As previously discussed, HUD’s project management framework and  associated governance structure was established to provide policies and  procedures for managing the department’s IT investments. Specifically,  the framework provides instructions, templates, and checklists intended to  help ensure important details are incorporated for use during the  execution and management of project activities. The department’s  governance structure is responsible for ensuring that all necessary  documentation is produced for all IT projects through control gate reviews  conducted by the Technical Review Subcommittee.
Officials responsible for the 9 FHA Transformation and 5 NGMS projects  in our review stated that they relied on the department’s PPM framework  to implement project management practices and the artifacts discussed in  this report. However, guidance discussed in the framework did not always  include essential information called for by best practices. For example,  the guidance for developing requirements management plans did not  specifically direct the projects to identify methods for prioritizing  requirements. In addition, the projects did not develop strategies early  enough to guide acquisitions because the framework did not call for the  strategy to be developed until after projects completed initial planning  activities.
In other cases, where guidance was provided, FHA Transformation and  NGMS projects did not always follow the guidance provided or adequately  implement the tools provided by the framework in developing the  documentation we examined. This was particularly evident in the  development of work breakdown structures. For example, the PPM  guidance included specific details regarding the importance of developing  work breakdown structures as the basis for defining project work and  establishing reliable cost and schedule baselines. However, as noted  earlier, only 1 of the 14 projects in our study had drafted a work  breakdown structure. Further, the project management plan template and  guidance call for incorporation of cost and schedule baselines and  approaches for how those will be managed for any given project.  However, the project management plans we examined did not clearly  incorporate such baselines or how they would be managed.
Compounding the issue of inadequate development and use of the  framework was the lack of evidence that the department’s governance  bodies had provided adequate oversight to ensure compliance with  project management practices. In particular, the department’s Technical  Review Subcommittee did not express concerns regarding the alignment  of FHA Transformation or NGMS documentation with the framework and,  when issues were raised, the subcommittee had nonetheless allowed the  projects to proceed. In doing so, the projects were able to move to the  next control gate review without critical information—a practice which  could result in projects proceeding for months without correcting flaws or  inadequacies in information that was vital to effective project  management. Specifically, in examining documentation for control gate  reviews, we found that the Technical Review Subcommittee did not  consistently operate as intended or use the guidance provided in the  department’s framework. While the department’s framework outlines  processes for conducting control gate reviews of projects and provides  templates to be used, the reviews were conducted without using the  framework guidance. For example, the control gate review procedures  state that documentation should be assessed based on (1) its accuracy in  capturing necessary information for the project’s development, (2) its  completeness with a level of detail sufficient to provide correct and  relevant information, and (3) the adequacy of information in the artifact to  make it actionable and informative. The framework also provided a  decision document intended to capture any issues or concerns identified  by the subcommittee.
However, it was not evident that any of the control gate reviews  conducted from 2011 through 2012 had assessed the documentation  against the outlined criteria or that the decision document was used. For  example, during this time, none of the control gate review documents  provided for FHA Transformation and NGMS included an assessment of  the documentation against the criteria in the control gate review  procedures, and meeting minutes or e-mails were used to record high- level issues or concerns identified instead of the more detailed  information called for in the decision document. According to responsible  OCIO officials, the subcommittee did not assess compliance with the  framework, but was focused on reviewing the technical aspects of IT  projects. The officials also noted that the subcommittee did not have the  staff needed to fully implement the control gate review guidance included  in the framework, but that it did look to see if the identified artifacts were  developed for each project. Further, these officials stated that it was the  responsibility of the project managers and their teams to address issues  identified before the next control gate review, but that the subcommittee  did not enforce any specific deadlines. Based on our assessment of the  control gate review documentation, as well as interviews with OCIO and  modernization effort officials, it was not clear that the subcommittee  consistently considered its role to include a full assessment of the  artifacts for compliance with the framework outlined in control gate review  guidance and templates.
According to OCIO officials, the initial implementation of the framework  focused on attempting to get projects to understand basic project  management, and as a result, the department limited the focus of the first  version of the framework. In April 2013, the department reported that it  was working on a revised version of the framework that would be  released in September 2013. However, the preliminary information  provided regarding the revisions planned for the framework did not  incorporate information to address all the deficiencies identified by project  officials or highlighted in this report. For example, draft documents  regarding the planned revisions did not explicitly state whether work  breakdown structures and associate dictionaries would be required  documentation to serve as the basis of cost and schedule baselines.  Further, the preliminary information did not specify if methods for  prioritizing requirements are to be incorporated into the requirements  guidance. Additionally, as of February 2013, the department had  assigned new leadership for managing the control gate reviews.  According to this official, the control gates are expected to be revised to  ensure that artifacts are evaluated and that the subcommittee takes a  more active role in assessing the application of project management  practices. However, the department did not state if it would clarify the  Technical Review Subcommittee’s role or associated guidance outlined in  the PPM framework or identify time lines for implementing the anticipated  changes.
Until HUD has a PPM framework for managing its projects that  incorporates the abovementioned details, including clarifying the role of  the Technical Review Subcommittee, and is appropriately used in  managing its modernization efforts, the department increases the risk of  continuing to inadequately apply project management practices and will  not be positioned to effectively manage or report progress of its  modernization efforts.
section_title: Conclusions
paragraphs: HUD has taken steps toward applying best practices by establishing a  framework for standardizing project management, and to varying  degrees, the FHA Transformation and NGMS modernization efforts have  developed basic documentation in the areas of project planning,  requirements management, and acquisition planning. Notwithstanding  these initial actions, the limited extent to which its modernization efforts  implemented key practices in these areas puts its projects at an  increased risk of failure. Specifically, the absence of complete information  in foundational documentation intended to guide these efforts—such as  project charters that define project success, deliverable-oriented work  breakdown structures that detail the work needed to be accomplished,  project management plans that include cost and schedule baselines,  requirements management plans that provide methods for prioritizing  requirements, traceable requirements to desired capabilities, and sound  acquisition strategies that guide planning activities—means that HUD has  not taken the steps to fully define its modernization efforts in terms of  what they will accomplish, what steps are necessary to complete them,  what they will cost, when they will be completed, what specific  functionality is needed to meet their goals, and how contractors will be  held accountable for performance. This indicates that, despite the steps  that have been taken, the maturity of HUD’s project management  practices does not sufficiently position the department to successfully  carry out these efforts.
Contributing to these deficiencies is that the department has not  developed and used its project management framework in a manner that  ensured the quality or completeness of project management  documentation. Additionally, the lack of adequate oversight from the  Technical Review Subcommittee resulted in projects not fully  understanding how to develop complete artifacts. Until it addresses these  weaknesses in applying project management practices, HUD may  continue to invest resources in modernization projects that will not satisfy  business needs and support its mission. Moreover, fully implementing  effective project management practices is critical not only for the success  of these modernization efforts, but also for that of the other five IT  Transformation Initiatives or any other projects under way or undertaken  in the future.
section_title: Recommendations for Executive Action
paragraphs: To ensure that HUD effectively and efficiently manages its modernization  efforts aimed at improving its IT environment to support mission needs,  we recommend that the Secretary of Housing and Urban Development  direct the Deputy Secretary to establish a plan of action that identifies  specific time frames for correcting the deficiencies highlighted in this  report for both its ongoing projects, as applicable, and its planned  projects, to include developing charters that define what constitutes project success and  establish accountability,  finalizing deliverable-oriented work breakdown structures and  associated dictionaries that define the detailed work needed to  accomplish project objectives,  completing comprehensive project management plans that reflect cost  and schedule baselines and fully incorporate subsidiary management  plans, establishing requirements management plans that include  prioritization methods to be applied and metrics for determining how  products address requirements,  completing matrixes to include requirements traceability from mission  needs through implementation, and establishing strategies to guide how acquisitions are managed in  accordance with other processes and that performance metrics are  established.
Further, to improve development and use of the department’s project  management framework, we recommend that the Secretary direct  the FHA Transformation and NGMS steering committees to ensure  that project management expertise needed to apply the guidance  outlined in the framework is provided to execute and manage their  respective projects;   the Chief Information Officer to ensure that revisions to the framework  incorporate specific information to address the areas of deficiency in  project planning, requirements management, and acquisition planning  identified in this report; and  the Customer Care Committee to review the role and responsibilities  of the Technical Review Subcommittee and ensure that the  department’s governance structure operates as intended and  adequately oversees the management of all of its modernization  efforts.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to HUD for review. In response, HUD  provided a letter, signed by the Acting Chief Information Officer, which  included a chart containing the department’s written comments on the  draft report. In the chart, the department outlined its views related to our  four recommendations, and provided other comments and technical  corrections on information in specific sections of the draft report, including  the background and appendix I, our discussion of the findings on the  development and use of HUD’s project management framework, and the  report title page. The department’s comments are reprinted in their  entirety in appendix IV.
In commenting on our recommendations, the department discussed  actions it was taking on various aspects of the first recommendation, but  did not state whether or not it concurred with the entirety of the  recommendation; stated that our conclusion leading to the second  recommendation did not follow from the premises established in the draft  report; and concurred with our third and fourth recommendations.  Summaries of HUD’s comments for each recommendation, along with our  responses, follow.
With regard to the first recommendation—which called for the Deputy  Secretary to establish a plan of action that identifies specific time  frames for correcting the deficiencies highlighted in this report for both  of its ongoing projects, as applicable, and its planned projects—the  department noted activities that FHA Transformation expects to  undertake in addressing the deficiencies for the six specific items  listed as part of this recommendation. In this regard, the department  stated that FHA Transformation acknowledged the need to update  project charters and project management plans, develop deliverable- oriented work breakdown structures, examine and correct the  requirements management plans and traceability matrixes, and work  with support offices to ensure acquisition planning occurs at the  earliest possible opportunity in the project’s life cycle. The department  added that FHA Transformation had recognized the need to update its  charters and project management plans well ahead of our draft report.   Nonetheless, updated artifacts for FHA Transformation were not  provided during our review. Moreover, the department did not address  whether or how it intends to address deficiencies for its ongoing or  planned projects, including those associated with the NGMS  modernization effort. Accordingly, we maintain that it is important for  HUD to establish a plan of action that identifies specific time frames  for addressing the deficiencies in its IT projects. As acknowledged in  the department’s comments, efforts to improve these project  management practices could be applied to the other five IT  Transformation Initiatives or any other projects under way or  undertaken in the future.
For our second recommendation, which called for the FHA  Transformation and NGMS steering committees to ensure that project  management expertise needed to apply the guidance outlined in the  framework is provided to execute and manage their respective  projects, the department contended that our conclusion leading to this  recommendation did not follow from the premises established in the  report. The department stated that it has ample talent and that  providing additional talent would likely yield similar results regarding  its deficiencies until the underlying steps are taken to apply effective  project management practices.
We agree that applying effective project management practices is  important; however, in our view, it is essential for the FHA  Transformation and NGMS steering committees to ensure that their  respective modernization efforts have the expertise needed to do so,  as it pertains to the development of tools such as work breakdown  structures and requirements traceability matrixes. During our study,  department officials stated on multiple occasions, that certain artifacts  and practices were not implemented because staff lacked expertise in  these areas. For example, both FHA Transformation and NGMS  officials stated that their staff had not developed the expertise  required to create work breakdown structures. Similarly, these officials  stated that projects had relied on contractors to complete  requirements traceability matrixes. Additionally, as we noted, the  officials acknowledged that a lack of project management maturity  was the cause of many of the deficiencies identified. Moreover, in its  comments on this report, the department stated that staff training for  the transition to applying the framework was limited. Thus, for these  reasons, we believe our recommendation is valid and should be  implemented.
The department concurred with our third recommendation that the  Chief Information Officer ensure that revisions to the framework  incorporate specific information to address the areas of deficiency in  project planning, requirements management, and acquisition planning  identified in this report.
In commenting on the fourth recommendation, the department  concurred with the need for the Customer Care Committee to review  the role and responsibilities of the Technical Review Subcommittee  and ensure that the department’s governance structure operates as  intended and adequately oversees the management of its  modernization efforts.
In other comments, the department stated that the discussion of the  department’s project management framework did not recognize the  difficulty of implementing this framework over the past 2 years. It stressed  that tremendous effort had been made by the FHA Transformation and  NGMS modernization efforts toward applying the framework while  continuing to make progress on their related projects. It also stated that  time is needed to fully incorporate the framework throughout the  department on projects other than these modernization efforts.
Acknowledging that the department has continued to take actions to  improve its environment, the focus of our work for this report was on  the implementation of project management practices for FHA  Transformation and NGMS, specifically. As such, we did not assess  the difficulties associated with improving the department’s overall  capacity to manage its IT projects. We do agree that there are  difficulties associated with applying project management practices  while concurrently undertaking multiple modernization efforts and  have previously reported on the progress HUD has made in  addressing its limited capacity to manage and modernize its IT  environment.
Regarding the title page, HUD commented that modernization efforts  historically account for a relatively small percentage of IT projects at  the department, and that a more comprehensive perspective that  accounts for all IT investments should be considered in the title of our  report. Our objective for this report was specifically to identify the  extent to which key project management practices were implemented  for the FHA Transformation and NGMS modernization efforts. As  such, this report did not evaluate all of the department’s IT  investments. However, in this report, we do acknowledge the value of  HUD applying these practices to all of its IT projects and moreover,  we plan to undertake future work to evaluate the department’s  institutionalization of its IT governance that we anticipate will be more  comprehensive in assessing the department’s management of IT  investments.
Lastly, the department stated that the report should contain historical  information illustrating the distribution of modernization funding in  contrast to funding available for the operation and maintenance of IT.   Toward this end, we assessed all relevant data that the department  provided to us regarding its IT funding against the data that it reported  to OMB. However, we found these data to lack consistency and  concluded they were not sufficiently reliable for inclusion in our report.
With respect to HUD’s technical corrections on the draft report, we have  incorporated revisions, as appropriate. Specifically, in the background  section and appendix I, we included a footnote to clarify that the Office of  Program Systems Management is within the Office of Housing/FHA Office  of Multifamily Housing Programs. We also updated the report section that  discussed the development and use of HUD’s project management  framework by removing the specific reference to the Deputy Chief  Information Officer for IT Operations. In this same section, the department  stated that OCIO did not concur with statements attributed to officials  from the Technical Review Subcommittee. We modified the statements  and the attribution in that section to represent more specifically what the  officials stated. In doing so, we also further clarified the activities  conducted by members of the Technical Review Subcommittee and  comments provided by officials from the two modernization efforts.
We are sending copies of this report to interested congressional  committees. We are also sending copies to the Secretary of the  Department of Housing and Urban Development and the Director of the  Office of Management and Budget. Copies of this report will also be  available at no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made significant contributions to this  report are acknowledged in appendix IV.
section_title: Appendix I: Objective, Scope, and Methodology
paragraphs: Our objective was to identify the extent to which the Department of  Housing and Urban Development (HUD) has implemented key project  management practices for the Federal Housing Administration  Transformation Initiative (FHA Transformation) and the Next Generation  Management System (NGMS) modernization efforts. To address this  objective, we examined all 14 projects for FHA Transformation and  NGMS that had been identified in the department’s fiscal year 2011  expenditure plan. This included 9 FHA Transformation and 5 NGMS  projects, which are identified in tables 1 and 2 of this report.
Because HUD recently began implementing project management  practices for its information technology (IT) modernization projects, we  reviewed the implementation of practices during the initial phases of the  projects’ life cycles; these practices establish the foundational plans and  processes for managing projects throughout their life cycles. Specifically,  we reviewed project planning and management practices essential for the  success of modernization efforts in three areas: project planning,  requirements management, and acquisition planning. We identified best  practices in these areas included in the Project Management Institute’s  (PMI) A Guide to the Project Management Body of Knowledge (PMBOK®  Guide), the Software Engineering Institute’s (SEI) Capability Maturity  Model® Integration for Development (CMMI-DEV) and for Acquisition  (CMMI-ACQ), and GAO’s Cost Estimating and Assessment Guide (Cost  Guide).
For the 14 projects in our study, we assessed the three project  management areas by reviewing six relevant documents to determine  whether they contained essential information called for by best practices.  Our assessment evaluated to what extent these documents (1) were  developed and contained essential information, (2) were developed but  lacked essential information, or (3) had not yet been developed.  Specifically:
To assess project planning activities, we determined whether projects  had developed project charters, work breakdown structures, and  project management plans, and when they had, we compared the  contents of these documents with project management practices in  order to determine the extent to which critical elements were  incorporated or executed on the projects. Specifically, we assessed  whether project charters addressed important elements such as the  project purpose or justification, the project manager’s responsibility  and authority level, and the name and responsibility of the project  sponsor. We assessed whether the work breakdown structures were  deliverable-oriented hierarchical decompositions of the work to be  executed and had associated dictionaries. Finally, we assessed  whether project management plans addressed important elements  such as the project life cycle, results of project tailoring, cost and  schedule baselines, and subsidiary management plans.
To assess requirements management, we determined whether  projects had developed requirements management plans and  requirements traceability matrixes, and when they had, we compared  the contents of these documents with best practices in order to  determine the extent to which each program was applying specific  elements. Specifically, we assessed whether requirements  management plans addressed important elements such as  configuration management activities, methods used to prioritize  requirements, metrics, and a traceability structure. In addition, we  assessed whether requirements identified in matrixes were, among  other things, traceable to business needs, opportunities, goals, and  objectives and whether the matrixes included essential information  such as requirements change requests and status.
To assess acquisition planning, we determined whether the  modernization initiatives had developed acquisition strategies and,  when they had, compared the contents of these documents with key  practices to determine the actions HUD is taking to ensure that the  acquisitions for FHA Transformation and NGMS are planned in  accordance with best practices and guidance. Specifically, we  assessed whether acquisition strategies addressed important  elements such as the established dates for the contract deliverables,  and procurement metrics.
We interviewed relevant HUD officials and staff in the FHA  Transformation and NGMS project offices, including the General Deputy  Assistant Secretary for Public and Indian Housing, the Director for the  Office of Program Systems and Management, the Deputy Director of  FHA Transformation, and the NGMS Program Manager. In addition, we  interviewed officials from the department’s Chief Procurement Office,  including the Deputy Chief Procurement Officer, and the Office of the  Chief Information Officer, including the Acting Deputy Chief Information  Officer for Business and IT Modernization, to obtain information on how  these offices support the work of the two modernization efforts. Further,  we attended and observed project status meetings, and related review  sessions conducted by senior leadership, including HUD’s Deputy  Secretary.
We determined that information provided by the department, such as  work breakdown structures and requirements traceability matrixes, was  sufficiently reliable for the purposes of our review. To arrive at this  assessment, we conducted reliability testing by comparing information  with statements from relevant department officials to identify  discrepancies. However, we did not test the quality of certain information,  such as cost and schedule data provided by the program offices.
We conducted this performance audit from June 2012 to June 2013, in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objective. We believe that  the evidence obtained provided a reasonable basis for findings and  conclusions based on our audit objective.
section_title: Appendix II: Summary of HUD’s Transformation Initiative Information Technology Modernization Efforts
paragraphs: Appendix II: Summary of HUD’s  Transformation Initiative Information  Technology Modernization Efforts Description  Develop and implement a modern financial services IT environment to better manage  and mitigate risk across FHA’s insurance programs for single-family housing, multifamily  housing, and the insured healthcare portfolio.
Reengineer processes and implement an automated system for managing grants that will  reduce application time, eliminate manual paper processes, and increase the  transparency of grant management information.
Provide business intelligence and geospatial tools for documenting and assessing  progress toward achieving strategic goals that will enhance transparency, reduce  workload, increase employee productivity, and improve data quality.
Integrate human resources systems and tools to allow for automated recruitment and  hiring documentation, reduction of manual data entry, and accelerated candidate  decision making.
Integrate an acquisition management system that is compliant with federal regulations to  reduce inefficiencies, time, and duplication in the procurement process across office  locations to expedite services rendered to the public.
Modernize and replace financial management systems through an outsourced shared  services provider.
Reengineer management processes to establish a technical infrastructure that will  integrate disparate systems and provide consistent information in order to support rental  housing assistance services.
section_title: Appendix III: Detailed Assessment of FHA Transformation and NGMS Project Management Documentation
paragraphs: Our assessment of FHA Transformation and NGMS implementation of  key project management documentation in the areas of project planning  (charters, work breakdown structures, and project management plans),  requirements management (requirements management plans and  traceability matrixes) and acquisition planning (acquisition strategies) is  described below.
subsections: section_title: Project Charters
paragraphs: Best practices recognize the development of a project charter as an  integral step in project planning for establishing and maintaining project  teams. A charter formally authorizes a project and identifies high-level  information that constitutes and assigns responsibility for project success.  This is a critical artifact for creating project management plans,  documenting business needs, and outlining the result a project is  intended to achieve. Specifically, to be effective, project management  practices state that a charter should include high-level requirements; high-level project description; high-level risks; measurable objectives and related success criteria; project approval requirements (e.g., what results in project success  the project’s purpose or justification;  summary milestone schedule;  summary budget;  and who is responsible for final sign off); assigned project manager, responsibility and authority level; and the name and authority of the sponsor or other person(s) authorizing  the project charter.
According to best practices, a work breakdown structure is the  cornerstone of every project because it defines in detail the work  necessary to accomplish a project’s objectives and provides a basic  framework for a variety of related activities like estimating costs,  developing schedules, identifying resources, and determining where  risks may occur. In the context of the work breakdown structure, work  refers to work products or deliverables that are the result of effort and not  to the effort itself. Creating a work breakdown structure involves  subdividing (or decomposing) project deliverables and work into smaller,  more manageable components (called work packages) that can be  scheduled, cost estimated, and managed. According to best practices,  the work breakdown structure is a deliverable-oriented hierarchical  decomposition of the work to be executed by the project team to  accomplish the project’s objectives and create the required deliverables.  Further, these practices state that a work breakdown structure should  also represent the entire scope of the project and product work, including  project management, and it should be standardized to enable an  organization to collect and share data among projects. In addition, it  should be accompanied by a dictionary of the various work breakdown  structure elements that describes in brief narrative form what work is to  be performed in each element.
As we have previously reported, agencies need to develop  comprehensive project management plans, and best practices emphasize  the importance of having a plan in place that, among other things,  establishes a complete description that ties together all activities and  evolves over time to continuously reflect the current status and desired  end point of the project. According to project management practices, a  project management plan is the primary source that defines, among other  things, how the project is to be executed and controlled. Project  management plans can be either summary level or detailed and can be  composed of one or more subsidiary plans to address elements of project  management. To be effective, best practices state that a project  management plan integrates cost and schedule baselines from planning  activities, and this baseline information should be updated as needed and  periodically compared with actual performance data in order to track and  report progress. While the content of a project management plan will vary  depending upon the type and complexity of a project, it is developed  through a series of integrated processes and is progressively elaborated  by updates during the execution and management of a project. Such  plans include  identification of the projected life cycle and processes to be applied to  each phase;  results of project tailoring; how the team will execute the work to accomplish project objectives; project cost and schedule baselines; how the team will maintain the integrity of performance measurement a change management plan that documents how changes will be  baselines;  the needs and techniques for communicating among stakeholders;  key management reviews; a configuration management plan to define those items that are  configurable, those items that require formal change control, and the  process for controlling change to such items; and  subsidiary management plans (scope, requirements, schedule, cost,  quality, process improvement, human resources, communication, risk  and procurement).
According to project management practices, effective planning of  requirements includes documenting the processes and methods to be  used for effectively developing and managing requirements from initial  identification through implementation. A project’s success is directly  influenced by the care taken in capturing and managing requirements.  Other essential planning activities such as developing a work breakdown  structure or estimating a project’s cost and schedule are built upon  requirements developed. Best practices state that in establishing  requirements, project teams should plan requirements collection activities  such as conducting interviews, focus groups, facilitated workshops, or  other techniques, including surveys and prototypes. Depending on the  type of project, the approach for managing requirements can vary, but  requirements management plans document the approach for how  requirements development activities will be conducted. In particular, a  plan includes how requirements activities (e.g., collecting requirements) will be  planned, tracked, and reported;  configuration management activities such as how changes will be  initiated, analyzed, and managed;  requirements prioritization methods; product metrics that will be used and the rationale for using them; and a traceability structure outlining attributes for a traceability matrix and  identifying what other project documents requirements will be traced  to.
Project management practices state that requirements traceability  matrixes are designed to support backward traceability by linking each  requirement to the broader business objective it supports and forward  traceability by linking these requirements to more detailed functional  requirements. Traceability refers to the ability to follow a requirement from  origin to implementation and is critical to understanding the  interconnections and dependencies among the individual requirements  and the impact when a requirement is changed. This bidirectional  traceability can help management determine whether the project  addresses all requirements and that those requirements and the related  deliverables are traceable back to valid business needs. According to  best practices, requirement matrixes provide tracing to business needs, opportunities, goals and objectives; high-level requirements to more detailed requirements; criteria used for evaluation and acceptance of the requirements; a set of approved requirements; and status of requirement changes and requests.
Further, matrixes should have specific attributes recorded for each  requirement. Attributes associated with each requirement—such as a  unique identifier, textual description, priority, version, current status, and  date completed—should be recorded.
According to best practices, effective IT project management also  involves early planning for and management of acquisitions. The  planning process begins with the identification of project needs which can  best be, or must be, met by acquiring products, services, or results  outside of the organization. During planning, coordination of the  acquisition with other project management activities, such as budgeting,  scheduling, resource estimating, risk identification, and requirements  definition, should be discussed and documented. Most organizations  have documented policies and procedures specifically defining mandatory  acquisition activities for obtaining contracted goods or services. The  acquisition planning process should result in a plan or strategy that  describes how management decisions will be applied for a particular  project. Such strategies serve as the road map for effectively planning  and managing acquisitions from initiation through contract closure. In  particular, project management practices indicate that acquisition  strategies should provide guidance for defining the types of contracts to be used; addressing risk management issues; coordinating procurement with other project aspects, such as  scheduling and performance reporting;  setting the scheduled dates in contracts for determining dates for  deliverables and coordinating with other project management  processes; and establishing procurement metrics to be used in managing and  evaluating contractors.
section_title: Appendix IV: Comments from the Department of Housing and Urban Development
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact above, Teresa M. Neven (Assistant Director),  Kami J. Corbett, Amanda C. Gill, Lee A. McCracken, John Ockay, and  Shannin G. O’Neill made significant contributions to this report.