Information Technology: HUD Can Take Additional Actions to Improve Its Governance
Published: 2014-12-10T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: HUD relies on IT to deliver services and manage programs in support of its mission of strengthening communities and ensuring access to affordable housing. However, the department has experienced shortcomings in its IT management capability and limitations in the systems supporting its mission.
A Senate report accompanying HUD's fiscal year 2012 appropriation mandated GAO to evaluate, among other things, the department's institutionalization of IT governance. In response, GAO reported on HUD's IT project management in June 2013.
GAO's objectives for this second review were to determine (1) the extent to which HUD implemented key IT governance practices, including effective cost estimation, and (2) what, if any, cost savings or operational efficiencies HUD has reported achieving as a result of its IT governance practices. To accomplish this, GAO compared HUD's approach to IT governance with best practices and the department's policies and procedures. GAO also analyzed reported cost savings and operational efficiencies, along with any available supporting documentation.
section_title: What GAO Found
paragraphs: The Department of Housing and Urban Development (HUD) has partially established elements of key practices for effective information technology (IT) governance, as identified by GAO's IT investment management guide. However, several shortcomings remain:
Investment boards, policies, and procedures were not fully established: HUD chartered four review boards to manage the department's IT investments; however, the executive-level board, which is to be responsible for overall definition and implementation of the investment management process, has never met. Instead, the department's Deputy Secretary makes decisions about which investments to fund. The lack of an operational executive-level board has affected HUD's other active investment boards, which are operating without criteria the executive-level board was to have established for evaluating proposed investments. In addition, HUD has not yet developed all of the policies that it has identified as needed to support its IT management framework. Specifically, the department has not set a schedule for developing policies for IT investment performance, privacy, and risk management. Office of the Chief Information Officer (CIO) officials explained that operating without an executive-level board represents the preferred investment management approach of HUD's Secretary and Deputy Secretary.
Process for selecting investments lacks key elements: HUD has developed elements of a process for selecting investments based on defined criteria; however, it has not fully defined and implemented practices for identifying, evaluating, and prioritizing proposed IT projects for funding, as recommended by GAO's IT investment management guide. CIO officials acknowledged that they have not yet fully developed a standard and well-documented process and attributed weaknesses to a variety of factors, including changes in leadership, priorities, and approaches.
Process for overseeing investments has not been fully developed: The department has not consistently compared the performance of projects to pre-defined expectations, established thresholds to trigger remedial action for underperforming investments, or reviewed projects after implementation to compare actual investment results with decision makers' expectations. These weaknesses were attributed by CIO officials to, among other things, the lack of a consistent, enterprise-wide way to collect and compare actual data with estimates.
Until effective governance practices are institutionalized, there is risk that HUD's investments in IT may not reflect department-wide goals and priorities or effectively support the department's mission.
While HUD has reported governance-related cost savings and operational efficiencies, the data to support such reports were not always accurate, consistent, or substantiated. This is due, in part, to the lack of a department-wide approach, as called for in Office of Management and Budget guidance, to identify and collect cost-savings information. Thus, it is unclear to what extent HUD has realized savings or operational efficiencies from its IT governance.
section_title: What GAO Recommends
paragraphs: GAO recommends that HUD ensure that its investment review boards operate as intended and complete and update associated policies; fully establish processes including key elements for selecting and overseeing investments; and fully establish a process for identifying, collecting, and reporting data about cost savings and efficiencies. HUD agreed with GAO's recommendations.

section_title: Letter
paragraphs: The Department of Housing and Urban Development (HUD) relies  extensively on information technology (IT) to deliver services and manage  programs in support of its mission of creating strong, sustainable,  inclusive communities and ensuring affordable housing. Among other  purposes, HUD employs IT to support programs servicing mortgage  insurance for $1.2 trillion in loans, providing about $35.4 billion in rental  housing assistance to needy families each year, and administering  community development grant programs with a combined value of about  $6.6 billion annually.
HUD has long experienced shortcomings in its IT management capability  and limitations in the systems supporting its mission. Specifically, HUD  has reported since 2008 that its systems are overlapping, duplicative, and  not integrated; necessitate manual workarounds; and employ antiquated,  costly-to-maintain technologies. Further, we have reported since 2009 on  the need for the department to improve IT management controls (e.g.,  enterprise architecture, investment management, and human capital).
The Senate Report accompanying HUD’s fiscal year 2012 appropriation, as approved by the Conference report, included a mandate that directed  us to evaluate HUD’s IT project management practices and the agency’s  institutionalization of IT governance, including any achieved cost savings  or operational efficiencies that have resulted. We reported on HUD’s IT  project management in June 2013.
Our objectives for this second review in response to the mandate were to  determine (1) to what extent HUD has implemented key IT governance  practices, including effective cost estimation, and (2) what, if any, cost  savings or operational efficiencies HUD has reported achieving as a  result of its IT governance practices.
Consolidated and Further Continuing Appropriations Act, 2012, Pub. L. No. 112-55, 125  Stat. 552, 691-92 (Nov. 18, 2011). research and experience across the federal government have shown are  hallmarks of successful organizations. For each critical process for  building a foundation for investment management, we compared pertinent  agency documentation, such as governance policies, procedures, and  plans, to federal guidance. Specifically, we assessed whether this  documentation was consistent with IT investment management best  practices as identified by GAO in the IT investment management  framework. We also obtained and analyzed data reported by HUD about  governance-related cost savings and operational efficiencies. We took  steps to determine the reliability of the data HUD reported and  determined that the agency did not provide sufficient supporting  documentation to substantiate the cost savings and efficiencies that they  reported. Additional details on our objectives, scope, and methodology  are discussed in appendix I.
We conducted this performance audit from February 2014 to December  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: To accomplish its mission, HUD administers community and housing  programs that affect millions of households each year. Among other  things, the department provides affordable rental housing opportunities  and helps homeless families and chronically homeless individuals and  veterans. The department also administers mortgage insurance programs  for single-family housing, multifamily housing, and health care facilities.
HUD is organized into the following five main organizational components:
Housing/Federal Housing Administration: Programs within this  office are responsible for contributing to building healthy communities,  maintaining and expanding housing opportunities, and stabilizing  credit markets in times of economic disruption. This office also  regulates certain aspects of the housing industry.
Community Planning and Development: The office’s mission is to  provide decent housing, a suitable living environment, and economic  opportunities for people of low and moderate income. To accomplish  this, it engages in partnerships with all levels of government, the  private sector, and nonprofit organizations.
Fair Housing and Equal Opportunity: The office’s mission is to  administer and enforce federal laws that prohibit discrimination in  housing, such as the Fair Housing Act and the Civil Rights Act of   To accomplish this, the office establishes and enforces policies  1964.intended to ensure that all Americans have equal access to housing of  their choice.
Public and Indian Housing: Programs within this office are  responsible for creating opportunities for residents’ self-sufficiency  and economic independence. Toward this end, this office currently  oversees a housing choice voucher program to subsidize housing for  approximately 2.2 million low-income families; a public housing  program that subsidizes about 1.3 million housing units for vulnerable  low-income families; and block grants and guarantee programs for  Native American groups.
Government National Mortgage Association (Ginnie Mae): This  HUD-owned corporation’s mission is to provide support for affordable  housing by bringing global capital into the housing finance market  while minimizing risk to the taxpayer. While Ginnie Mae does not  issue loans or mortgage-backed securities, it guarantees investors  timely payment and interest on mortgage-backed securities supported  by federally insured or guaranteed loans.
HUD also consists of a number of program offices, such as the Office of  Healthy Homes and Lead Hazard Control and the Office of Faith-Based  and Neighborhood Partnerships. In addition, the department has  administrative offices that report to executive leadership through the Chief  Operating Officer, such as the Office of Strategic Planning and  Management and the Office of the Chief Information Officer (CIO).
A simplified view of the department’s organization structure is provided in  figure 1.
HUD’s Deputy Secretary is responsible for managing the department’s  daily operations, annual operating budget, and approximately 8,700  employees. As part of this role, the Deputy Secretary conducts meetings  with stakeholders to discuss the Secretary’s priorities. During these  meetings, the scope, milestones, risks, and status of action items related  to priority issues are discussed.
subsections: section_title: Overview of HUD’s IT Environment
paragraphs: IT plays a critical role in the ability of the department’s organizational  components to perform needed business functions. For example, HUD’s  IT environment consists of multiple systems that, among other things, are  intended to help the department coordinate with lending institutions to  insure mortgages, collect and manage state and local housing data,  process applications for community development, and process vouchers  for different rental assistance programs. Its systems also support the  processing of applications for, and the management of, more than 50  grant programs administered by the department.
However, according to the department, its IT environment has not been  sufficient to effectively support its business operations because its  systems are overlapping and duplicative, not integrated, necessitate  manual workloads, and employ antiquated technologies that are costly to  maintain. For example, the department has reported that its  environment consisted of:
Over 200 information systems, many of which perform the same  function and, thus, are overlapping and duplicative. Specifically,  different systems perform the same task to separately support grants  management, loan processing, and subsidies management.
Stove-piped, nonintegrated systems that result in identical data  existing in multiple systems. For example, two organizational  components store about 80 percent of similar data in separate  databases that provide information on rental assistance participants.
Manual processing for business functions due to a lack of systems to  support these processes. For example, specific Public and Indian  Housing IT projects are intended to replace existing ad hoc analyses  performed in spreadsheets and databases with systems that  automate and standardize those functions.
Antiquated technology (15 to 30 years old) and complex systems that  are costly to maintain. For example, the department relies on different  obsolete programming languages and operating systems, which  require specialized skills to operate and maintain.
Further, contractors engaged by HUD to assess the department’s  environment reported in January 2011 that unclear reporting relationships  hindered the enforcement of IT policies; contractor performance  information was not used to inform management decisions; technical  standards were lacking or not enforced; and data management practices  did not support business needs.
section_title: HUD IT Management and Governance Structure
paragraphs: In October 2011, the Secretary of HUD delegated responsibility for the  management of IT resources and for meeting requirements established  by the Clinger-Cohen Act of 1996 to the department’s Chief Information  Officer (CIO). Among other duties, the CIO was charged with promoting  the effective and efficient design and operation of all major IT processes  for the department; designing, implementing, and maintaining processes  (consistent with the roles and responsibilities of governance boards) for  maximizing the value and assessing and managing the risks of IT  acquisitions; and advising the Secretary and governance boards  regarding whether to continue, modify, or terminate programs or projects.
To guide the department’s efforts in this area, the CIO developed a  management framework and policies, including a policy for governance,  based on the IT investment management framework developed by GAO  and other federal guidance. In addition, to establish management and  governance capabilities, the Office of the CIO chartered four investment  review boards in 2011:
The Executive Investment Board is to be made up of senior  executives, including the department’s Secretary (chair), Deputy  Secretary, Chief of Staff, Chief Operating Officer, Chief Financial  Officer, General Counsel, and CIO. As the department’s highest-level  board, the executive board is charged with responsibility for, among  other things, overseeing and approving HUD’s high-cost, high-risk  projects, allocating resources, and monitoring enterprise risk.
The Customer Care Committee is made up of executives including  the CIO, the Chief Procurement Officer, Chief Human Capital Officer,  and deputy assistant secretaries. The responsibilities of this  committee include reviewing and submitting investment  recommendations to the Executive Investment Board.
The Investment Review Subcommittee is made up of business area  personnel from across the department, including representatives from  the Office of the CIO, Fair Housing and Equal Opportunity, and Public  and Indian Housing, who focus on investment management oversight  with respect to business cases and budget information for the Office  of Management and Budget (OMB).
The Technical Review Subcommittee consists of personnel from  within the Office of the CIO, including the Chief Technology Officer,  the Chief Architect, and the Chief Information Security Officer, and is  focused on providing technical, project, and architecture subject  matter expertise.
The hierarchy of HUD’s investment review boards is depicted in figure 2.
The CIO and the investment review boards are supported by staff in the  Office of the CIO. Through coordination with the other organizational  components, the office manages IT resources and provides support for  the department’s infrastructure, security, and ongoing projects. The office  also provides project management guidance and technical expertise,  and supports the department’s efforts to govern IT investments.
In 2012, to improve the Office of the CIO’s ability to implement its  authorities as required by Congress and OMB, the department initiated a  reorganization of the office to, among other things, streamline IT  governance practices, improve the alignment of IT investments with  mission needs, and ensure customer satisfaction through enhanced  service delivery and continuous performance management. To date, HUD  has taken steps to complete the effort, including determining the specific  offices, divisions, and branches that make up the restructured office.  Specifically, the department established three Deputy CIOs with  responsibility for managing the following offices created within the Office  of the CIO:  the Office of Business and IT Resources Management, which is  responsible for human and financial resource management, including  funds control, budget execution, internal controls, and acquisition  management for the Office of the CIO; the Office of Infrastructure and Operations, which is responsible for  providing data center service, technical support, training, service  delivery assistance for the network, desktop computers, and  telephone and e-mail services to HUD staff in headquarters and field  offices; and the Office of Customer Relationship and Performance Management,  which is responsible for meeting HUD’s programmatic needs through  customer engagement, marketing, product development, and capital  planning.
In particular, the Office of Customer Relationship and Performance  Management consists of three divisions responsible for IT management  functions, including enterprise program management, investment  management, and enterprise architecture.view of HUD’s restructured Office of the CIO, including the Office of  Customer Relationship and Performance Management and the related  divisions and branches that directly support the department’s IT  governance functions.
section_title: Key Governance Practices Outlined in GAO’s IT Investment Management Framework
paragraphs: GAO assessed best practices to develop the IT investment management  framework to provide a method for evaluating and assessing how well an  agency is selecting and managing its IT resources. The framework  consists of five progressive stages of maturity that an agency can achieve  in its investment management capabilities. The maturity stages are  cumulative; that is, in order to attain a higher stage, an agency must  institutionalize all of the critical processes at the lower stages, in addition  to the higher stage critical processes. Each maturity stage is composed  of critical processes that must be implemented and institutionalized; these  critical processes are further broken down into key practices that describe  the types of activities that an organization should be performing to  successfully implement the critical processes. Only after implementing  key practices for each stage’s critical processes are organizations  positioned to mature their investment management and move from  managing individual projects to helping the organization evaluate the  effectiveness of its overall portfolio of investments.
Efforts to build a foundation for IT governance involve establishing  specific critical processes, such as those for instituting investment boards,  selecting investments, and providing investment oversight. Further,  certain key practices must be performed by an organization in order to  implement and institutionalize the critical processes effectively. For  example, one key practice associated with the critical process of  instituting the investment board involves documenting policies and  procedures to direct board activities. Table 1 provides a summary  description of the specific critical processes discussed in this report, along  with examples of related key practices.
section_title: HUD’s IT Governance Activities Do Not Yet Fully Address Key Practices
paragraphs: Since 2011, the department has taken steps to build a foundation for IT  governance, including instituting investment review boards, establishing  elements of a process for selecting investments, and providing  investment oversight. However, the department’s governance activities do  not yet fully address key practices. In particular, its boards are not  operating as intended and its processes for investment selection and  oversight lack essential elements. HUD faces risk that its governance  decisions will not reflect the needs of the department and that it will be  unable to realize planned improvements to its IT environment and  systems.
subsections: section_title: HUD’s Investment Boards and Their Guiding Policies and Procedures Are Not Fully Established and Operational
paragraphs: The establishment of decision-making bodies or boards is a foundational  component of effective IT investment management. According to the IT  investment management framework developed by GAO, an organization  should, among other things, establish one or more investment boards to  manage and select projects, including an enterprise-wide investment  board composed of senior executives that is responsible for defining and  implementing the organization’s investment management process. In  cases where additional investment boards are chartered to support the  enterprise-wide investment board, the enterprise-wide board should  remain responsible for the investment management process and be  actively involved in all IT projects and proposals that are high cost or high  risk or have significant scope and duration. The authorities, guiding  policies, roles, responsibilities, and operations of each board should be  defined to ensure that consistent and effective investment management  practices are implemented across the organization. We have previously  reported that effectively implementing a governance framework involves  having a robust implementation plan that specifies—in addition to goals  and objectives—milestones for the effort.
Consistent with the IT investment management framework developed by  GAO and as described previously in this report, HUD’s Office of the CIO  chartered four investment boards in 2011 with defined authorities, roles  and responsibilities, and operations for managing and selecting the  department’s IT projects.  Also during that year, the department issued  an IT management framework that was intended to provide guidance to  direct the investment boards in fulfilling their investment management  responsibilities.
However, the department has not ensured that its investment boards are  fully operating according to their designated authority and responsibility.  Specifically, HUD chartered an enterprise-wide investment board—the  Executive Investment Board—that was to be composed of senior leaders  from across the department, act as the highest decision-making authority,  and have direct purview of the department’s most complex, costly, and  visible IT projects. Nonetheless, as of October 2014, this board had never  met. According to officials from the Office of the CIO, operating without a  functioning Executive Investment Board reflects the Secretary and Deputy  Secretary’s preferred approach to investment management. The officials  stated that the Deputy Secretary has assumed the board’s designated  authority and roles and responsibilities. The officials added that the  Deputy Secretary determines the department’s IT priorities and selects,  as needed, specific individuals to participate in discussions regarding  which projects should receive funding.
As discussed above, these were the Executive Investment Board, which is to act as the  enterprise-wide board; the Customer Care Committee, which is to review and submit  investment recommendations to the executive board; the Investment Review  Subcommittee, which is to focus on investment management oversight; and the Technical  Review Subcommittee, which is to focus on providing subject matter expertise.
The lack of a functioning Executive Investment Board has affected the  ability of the department’s other active investment boards–the Customer  Care Committee, Investment Review Subcommittee, and Technical  Review Subcommittee–to fully operate in accordance with their assigned  responsibilities. Specifically, according to the department’s investment  review board charters, the Executive Investment Board was to establish  key criteria for these boards to use in identifying (1) which IT projects best  support HUD’s strategic goals and provide value to the department and  (2) which projects were underperforming and should be considered for  termination. However, because the Executive Investment Board has not  met or conducted any business, such criteria were not established.
As a result, instead of making funding recommendations based on criteria  representing enterprise-wide goals for the fiscal year 2014 budget cycle,  the Customer Care Committee, for example, recommended a set of IT  projects to the Deputy Secretary for funding. According to officials from  the Office of the CIO, the recommended projects were based on  considerations such as the priorities of individual board members or  programmatic concerns. The officials from the Office of the CIO  considered this approach to be appropriate because, in their view, it helps  to ensure that the concerns and goals of each member of the Customer  Care Committee are considered during decision making. Regarding  criteria for identifying underperforming projects, the officials stated that  the department’s investment boards rarely terminate IT projects because  they provide needed services that must continue until a replacement  project can be found and implemented.
Further, HUD’s active investment boards are not fully adhering to the  operating procedures outlined in their charters concerning meeting  frequency and documentation requirements. For example, according to  their charters, two of the boards—the Customer Care Committee and  Investment Review Subcommittee—are expected to hold monthly  meetings and document and distribute the results of the meetings to  stakeholders. However, the Customer Care Committee had documented  meeting minutes for just 3 months in fiscal year 2014, and the Investment  Review Subcommittee had not documented any of its meetings during  this time period. In addressing this matter, officials from the Office of the  CIO told us that the investment boards meet as needed to fulfill their  responsibilities.
Moreover, HUD has not yet developed all of the policies that support its IT  management framework. Specifically, the Office of the CIO’s framework  identifies 11 key policies that the office was to develop to influence and  determine actions and decisions in IT management areas such as  acquisitions, capital planning, and project planning. To date, the Office of  the CIO has developed 8 such policies, including those previously  described and policies for IT governance and enterprise architecture.  However, 3 of the planned policies—for performance, privacy, and risk  management—have not yet been developed, and HUD has not set a time  frame for doing so. Officials from the Office of the CIO stated that they  had taken a phased approach to developing the policies and did not  intend to have completed the 3 outstanding policies by this time.  However, without a timeline supporting this approach or a date by which  the 3 outstanding policies will be completed, the department lacks  assurance that it will take timely action to implement the additional  policies needed to fully establish the IT management framework.
Additionally, the department’s IT management framework has not been  updated to reflect significant changes to HUD’s project planning and  management practices and OMB requirements for conducting reviews of  projects. Officials from the Office of the CIO stated that the framework  had not been updated because the department’s investment  management practices have evolved every year and a definitive set of  practices had not been identified. The officials added that updating the  framework had not been a priority because it was written at a sufficiently  high level to remain relevant. They stated that they hope to make annual  updates to the framework to ensure that it remains current. However, a  date by which an updated version of the framework will be completed that  incorporates, among other things, OMB requirements, has not been  established.
Until HUD’s investment boards operate according to their designated  authority and responsibilities and criteria are established to guide  investment decision making, the department cannot ensure that, instead  of individual interests, a corporate responsibility is reflected in IT decision  making. Additionally, the department cannot ensure that projects will be  selected or terminated based on their ability to meet strategic goals and  bring value to the entire organization. Further, without complete and  current policies and procedures to guide its investment boards, the  department lacks assurance that its investment management practices  will be implemented consistently and effectively across the department.
section_title: HUD’s Investment Selection Process Does Not Yet Include Specific Key Practices
paragraphs: According to the IT investment management framework developed by  GAO, to support well-informed investment decision making,  organizations should document and implement a well-defined process for  selecting new proposals and reselecting ongoing investments.  Documenting and implementing the process is a basic step toward  achieving mature IT project selection. Elements of such a process include  key practices, such as those discussed in the IT investment management  framework and cost estimation guidance developed by GAO—practices  for identifying, evaluating, and prioritizing IT proposals for funding.  Specifically, with respect to proposal evaluation, agencies should define how data (including cost estimates) are to be developed, verified, and  validated, including detailed information to explain the basis for the  cost estimate and how the estimated funding will be spent; criteria for how proposed projects are to be analyzed in terms of  benefits, cost, schedule, and project risk; and a scoring mechanism that compares proposed projects to one another  in terms of investment size (cost), project longevity (schedule),  technical difficulty, project risk, and cost-benefit analysis, in order to  help the boards analyze and prioritize projects based on their  strengths and weaknesses.
Further, final selection decisions should be made by senior decision  makers and should be documented and supported by the evaluation  actions described above. In addition, another key practice involves  predefining a method for reselecting ongoing projects for continued  funding, including establishing predetermined criteria for analyzing  ongoing operations and maintenance projects.
As recommended by the guidance GAO developed, HUD has established  practices for identifying new IT proposals. The Office of the CIO has  taken steps to identify new IT proposals by developing standardized  templates for proposed IT projects. Among other things, the templates  address how proposed projects will meet business needs by requiring  information about strategic and agency priority goals to be addressed by  projects and descriptions of specific business needs or requirements that  projects are intended to support.
However, the department has not yet fully addressed key practices for  evaluating and prioritizing IT proposals. With regard to IT proposal  evaluation, the department has not documented how data supporting  proposed projects are to be developed, verified, and validated.  Specifically, the office has not established detailed requirements or  standardized guidance for how sponsors of proposed projects are to  develop key elements of proposals. For example, the office has not yet  defined how project cost estimates should be created or evaluated,  including what detailed information is required to explain the basis for the  estimate and how the estimated funding will be spent. In addition, the  office has not developed procedures for verifying and validating the data  in proposals or required project sponsors to provide supporting  documentation that the Office of the CIO could use to verify or validate  the data submitted. During fiscal years 2014 and 2015, for example, the  office did not validate whether cost information submitted was based on  any standardized cost estimation practices. In addition, in evaluating IT  proposals, the investment management division does not require  deficiencies in IT proposals to be addressed before proposals can be  recommended for funding and has not implemented procedures to track  whether deficiencies were addressed and whether scores were improved  as a result.
Further, while the Office of the CIO has established and used criteria to  analyze proposed projects in terms of benefits, the criteria do not address  how cost, schedule, or project risk are to be analyzed. The office  established criteria for scoring proposals that are used by staff within the  office and subject matter experts to score and prioritize proposals for  selection and inclusion in the department’s IT budget each year.  According to the criteria, to receive a high score in the mission benefits  category, IT proposals should identify one or more benefits of the project  that directly link to HUD’s mission and clearly define a consistent method  by which benefits are measured. However, while the Office of the CIO  requires that schedule milestones for contracts be provided, it has not  established requirements for the reporting of planned schedules or criteria  for analyzing project schedules. Further, regarding risk, although  business cases are to report how projects will reduce certain department  risks, the Office of the CIO has not established criteria for evaluating  proposals based on project risk, including whether all relevant project  risks have been identified or adequate plans for mitigating risk have been  developed.
Although the Office of the CIO developed and is using a scoring  mechanism, it does not allow the department to compare projects to one  another in terms of investment size (cost), schedule, technical difficulty,  project risk, or cost-benefit analysis in order to consider relative strengths  and weaknesses when prioritizing projects. For example, in analyzing  reported mission benefits in business case proposals, the investment  management division scores an IT proposal based on whether it has  identified mission benefits. Similarly, with regard to risk, the Office of the  CIO does not incorporate analyses of project risk into its scoring. The IT  management division scores a proposal based on whether it identifies  technical risks the department faces that the project intends to reduce  and whether there is a clear description of a strategy or plan for how such  risk may be reduced. However, proposals that went beyond that  requirement to identify specific project-related risks and plans for  mitigating them could receive the same score as proposals that did not  identify any project-related risks. Thus, proposed projects are not  compared in terms of their relative benefits, risks, or potential return on  investment. Further, scores are not lowered if proposals are incomplete;  that is, scores are based only on the data that were reported, and any  items left incomplete are not factored into the scoring. As a result, a  partially complete proposal could potentially be given a similar or an even  higher score than a proposal that included all of the requested  information.
Moreover, the extent to which IT proposals selected by senior decision  makers are supported by key evaluation practices is unclear, and the  decisions are not consistently documented. In particular, officials from the  Office of the CIO stated that senior executives on the boards make final  decisions about which projects to fund based on qualitative factors that  are not standardized. Board members are not provided with additional  information such as analyses comparing the relative costs, time frames,  benefits, or risks of projects being considered. Further, the Office of the  CIO documents board members’ agreement on a final list of projects to  be funded, but does not document the rationale used, including  explanations for why projects with lower priority scores may have been  funded, why any other exceptions to the initial funding priority were made,  or whether supporting analysis was used by the various boards to reach a  final decision, as called for by GAO’s investment management  framework.
Finally, the Office of the CIO has not yet established a predefined method  for reselecting ongoing projects for continued funding or defined criteria  for analyzing projects that are in the operations and maintenance phase.  While the office collects different data in business cases for projects in  operations and maintenance, it ranks and scores such projects along with  newly proposed projects. Projects are prioritized based upon their total  scores—without considering whether projects are new or ongoing.  Consequently, a poorly performing project that had a detailed business  case proposal meeting established criteria could receive a high score and  be ranked high on the list of priorities. Moreover, because the office has  not established a reselection method that includes predetermined criteria  for analyzing ongoing projects for continued funding, decisions to reselect  projects are not consistently based on key practices such as analysis of  progress or project outcomes or assessments of the potential risk or  return of continuing to invest in a project. While the Office of the CIO or  the investment boards may discuss ongoing projects’ progress in  investment selection deliberations, without a consistent method and  predefined criteria for making decisions about reselecting ongoing  projects, HUD may continue to invest in projects that are not performing  as needed.
Officials from the Office of the CIO acknowledged that they have not yet  developed a standard and well-documented selection process for new IT  proposals or for reselecting ongoing investments that fully addresses key  practices, including outlining requirements for cost estimates. Officials  from the Office of Strategic Planning and Management stated that new  guidance for cost estimation is under development and that more robust  practices will be used to create cost estimates for the fiscal year 2016  budget process. The officials attributed weaknesses in the selection  process to, among other things, efforts to minimize the burden on those requesting funding, which  result in limiting requirements for submitting detailed information to  support project proposals;  lack of an established priority for developing life-cycle cost estimates  for projects (until recently); changes in senior leadership, departmental priorities, and  approaches; lack of departmental oversight of the decision-making process by  senior executives; and reliance on qualitative, judgmental data and inadequate consideration  of key quantitative measures, such as return on investment.
Until HUD documents a complete selection process that incorporates key  practices for identifying, evaluating, and prioritizing investments, the  department is at risk that its selection practices will lack the  standardization, transparency, and consistency needed to ensure  effective decision making. In addition, key stakeholders may not have  common understanding of the practices or qualitative factors considered  in decision making, and executives may miss opportunities to consider  the relative risks and returns and strengths and weaknesses of proposed  investments. As a result, given that the final decisions are not data driven;  HUD is at greater risk of not selecting the appropriate mix of IT  investments that best meet its organizational and technology needs and  priorities for improvement.
section_title: Elements of a Process for Investment Oversight Have Been Established but Not Yet Institutionalized
paragraphs: As with investment selection, organizations should have a documented,  well-defined process for overseeing ongoing investments once they have  been selected. Effective investment oversight and evaluation involves,  among other things, documenting the process for oversight, including  predefined criteria and checkpoints for reviewing the progress ongoing  projects have made in meeting cost, schedule, risk, and benefit  expectations; comparing actual performance against estimates; and  identifying areas where future decision making can be improved.  Specifically, key practices call for predefined thresholds for project  performance designed to increase oversight of underperforming projects.  In addition, once the project has transitioned from the development phase  to the operations and maintenance phase, organizations should conduct  post-implementation reviews to compare actual investment results with  decision makers’ expectations for cost, schedule, performance, and  mission improvement outcomes. The lessons learned from these reviews  can be used to modify future investment management decision making.
Since 2011, the Office of the CIO has established elements of a process  for managing individual projects after they are selected. Specifically,  revisions made in 2014 to the office’s project planning and management  framework require project managers to develop management plans that  are intended to outline, among other things, performance expectations for  projects once they are selected and initiated. The office has also  assigned specific investment oversight responsibilities to two of the  department’s investment boards—the Technical Review Subcommittee  and the Investment Review Subcommittee. Finally, once projects are  completed, the project planning and management framework requires the  Office of the CIO to conduct post-implementation reviews no later than 1  year after systems are implemented to evaluate results of completed  projects.
Nonetheless, the office has not yet institutionalized all of the established  practices, and in some cases, has not established key practices. For  example, to fulfill its oversight responsibilities, the Technical Review  Subcommittee is supposed to  collect certain cost, schedule, benefit, and risk data for each IT project  at specific life-cycle checkpoints established by the project planning  and management framework; conduct reviews at each checkpoint designed to ensure that IT  projects are planned, budgeted, and scheduled in alignment with  HUD’s strategic goals and approach to technology management; and provide support to other boards, such as technical and architecture  analysis, in monitoring and analyzing investment performance.
As required, the Technical Review Subcommittee collects data on each IT  project’s cost, schedule, benefits, and risk at established life-cycle  checkpoints and conducts reviews at each checkpoint. During the  reviews, the subcommittee analyzes projects’ compliance with project  documentation requirements and technical and architectural standards.
However, the subcommittee is not yet positioned to fully execute its  oversight responsibilities due to certain limitations. First, the project  planning and management artifacts and data required for these reviews  have recently been revised and the related project management practices  have not yet been institutionalized. Officials in the Office of the CIO stated  that efforts to institutionalize the new process and strengthen the  subcommittee’s oversight function for all investments are under way, but  the office has not yet established expected time frames for when these  efforts will be completed. Moreover, efforts by the Office of the CIO to  improve project oversight practices may continue to be constrained by  project management deficiencies we identified in our 2013 report,  including practices designed to provide data that could be used for  investment oversight.
Second, the subcommittee has not consistently used data collected about  projects to monitor progress against the expectations established. In  August 2014, officials from the Office of the CIO stated that the  subcommittee had begun conducting assessments for cost and schedule;  however, they could not produce the results of any of these assessments  or provide any evidence that such reviews had actually assessed  performance against expected benefits or risks. Moreover, the Office of  the CIO’s documented policies and procedures requiring reviews by the  subcommittee do not establish cost-, schedule-, or performance-based  thresholds that would automatically trigger remedial action or referrals to  other investment boards. Officials from the office acknowledged that  practices for monitoring progress had not yet matured and that the  performance data currently collected and maintained at the project level  needed improvement. Moreover, because of this immaturity, current  efforts to monitor progress are limited. Further, without such thresholds,  project oversight may not be consistent or troubled projects may not  receive additional oversight by the investment boards.
Officials from the Office of the CIO also acknowledged that the office has  not yet established and documented a well-defined process for  investment oversight and attributed weaknesses in oversight to, among  other things, limitations in its ability to develop reliable cost, schedule, and  benefit estimates to use for monitoring progress and evaluating  performance. Until HUD addresses gaps in its processes for monitoring  progress of projects and documents policies and procedures to sustain  and consistently implement oversight practices, it cannot ensure that its  management of IT will achieve desired results.
As another investment oversight mechanism, the Investment Review  Subcommittee is responsible, under its charter, for monitoring investment  and portfolio performance and taking action to (1) terminate investments  consistently experiencing variances in cost or performance or (2) bring  them back within acceptable cost or performance limits. However, to date,  the Investment Review Subcommittee has not fulfilled these  responsibilities. Officials from the Office of the CIO acknowledged that the  subcommittee has not performed these functions and said that they are  reviewing the roles and responsibilities of the subcommittee to determine  whether changes are needed. The officials also stated that, as part of  early actions to define an oversight process, the Office of the CIO is  working to implement more mature practices and developing an  investment-level view of performance, including a strategy for identifying  performance expectations. Specifically, the officials provided plans to  develop an enterprise-wide portfolio performance management process,  which includes implementing a new portfolio management tool with  portfolio and project management functionality planned to be operational  by February 2015.
Officials from the Office of the CIO attributed weaknesses in oversight to,  among other things, lack of a consistent, enterprise-wide way to collect  and compare actual cost, benefit, schedule, or risk to estimates. Without  progress reviews that compare investments against estimated cost,  estimated schedule time frames, and expected benefits using predefined  thresholds, HUD will not have insight into whether the projects are  meeting mission needs during their development or after they are  completed.
Finally, the Office of the CIO has not consistently conducted post- implementation reviews to evaluate results of projects after they are  completed. Although the office’s project planning and management  process calls for such reviews no later than 1 year after system  implementation, the office had not scheduled or conducted any such  reviews until recently. In August 2014, officials from the Office of the CIO  reported that the Technical Review Subcommittee had begun undertaking  post-implementation reviews of its approximately 200 operational  systems. They explained that, after conducting several initial reviews, the  subcommittee determined that the data in system documentation  available to the office were insufficient to support these reviews. As a  result, the Chief Technology Officer collected additional data needed to  support post-implementation reviews of all 200 operational systems.  According to officials from the Office of the CIO, the reviews are  scheduled to be completed by the end of November 2014.
When explaining why post-implementation reviews had not been  conducted until recently, officials in the Office of the CIO said that, among  other reasons, relatively few IT projects had been completed.  Nonetheless, while many of HUD’s recent modernization projects remain  under way and have not reached a point at which post-implementation  reviews would be appropriate, establishing effective practices to collect  and maintain the data needed to support such reviews remains important.  The Office of the CIO will not be positioned to conduct effective post- implementation reviews until it identifies the data needed for such reviews  and requires that it be collected consistently for all projects. Officials from  the Office of the CIO acknowledged that the office experienced  deficiencies in capturing and communicating lessons learned and  translating them into revised decision-making processes. Without post- implementation reviews, HUD will not be able to compare the outcomes  of completed projects with expectations.
section_title: Reported Governance-Related Cost Savings and Efficiencies Were Not Adequately Supported, and Comprehensive Data on Governance Outcomes Are Not Yet Identified or Tracked
paragraphs: OMB guidance requires agencies to identify and communicate anticipated  cost savings and portfolio improvements realized. Specifically, the  guidance requires agencies to identify and report cost savings gained  from, for example, retiring low-value and duplicative investments,  eliminating costly support contracts, moving to shared services such as  cloud computing, and reducing IT commodity spending. Additionally,  agencies are directed to seek operational efficiencies such as automating  or streamlining processes and providing mobile technologies that result in  improved services to taxpayers. OMB’s guidance also calls for agency  leadership to use high-quality data in these efforts. Among other things,  OMB’s guidance on performance reporting calls for agencies to ensure  that supporting documentation is maintained and readily available, data  are verified as appropriate to the needed level of accuracy, and data  limitations are explained and documented. We have also reported that it  is important to have assurance that the data collected and reported by  agencies are complete and accurate. Specifically, we have reported that  agencies should explain the procedures used to verify or validate their  data and ensure that data are sufficiently complete, accurate, and  consistent.
HUD’s reported cost savings and operational efficiencies were not  supported by data that were complete, accurate, and consistent. To date,  through various mechanisms such as its fiscal year 2014 expenditure  planinformation, the Office of the CIO identified 14 instances of cost savings  and operational efficiencies reportedly accomplished through selected  investments and governance decisions. From its varied efforts, HUD  reported about $23 million in cost savings during fiscal year 2014.  Sources of the savings identified included   and reports to OMB, as well as in response to our requests for  initiatives to consolidate commodity IT; decisions to eliminate several system-support contracts (resulting in  annual cost avoidance); projects selected and implemented to automate manual processes  and program reporting requirements; and actions to deactivate three legacy systems.
However, the information reported from these four sources was deficient  in the following ways.
Data were not validated: HUD did not take steps to verify or validate  the data. Specifically, officials from the Office of the CIO reported that  they did not verify or validate data reported or require that those  reporting cost savings verify or validate the data provided.
Supporting data were not provided: The Office of the CIO could not  provide documentation needed to substantiate that the cost savings  data reported were reliable. Specifically, neither the list of deactivated  systems nor the fiscal year 2014 expenditure plan provided the basis  for savings reported, links to supporting documentation for those  calculations, or clear information about when the savings were  achieved. In addition, although certain reported savings could be  traced to OMB requirements for reporting on specific governance  activities and others could be presumed to have been generated from  the investment selection process funding specific projects, it was  unclear whether other savings resulted from specific governance  decisions because HUD did not provide meeting minutes or other  documentation of governance decisions authorizing or recognizing the  savings or efficiencies.
Regarding the lack of data validation, officials from the Office of the CIO  told us that they accept data as reported because they lack the resources  needed to validate the data submitted. With regard to the lack of  adequate supporting documentation, CIO officials acknowledged that not  all governance decisions result in savings or efficiencies and that clearly  linking cost savings and efficiencies to specific governance decisions is a  challenge. However, the Deputy CIO recognized that Office of the CIO  officials could do a better job of documenting specific decisions made by  investment review boards in meeting minutes.
Additionally, the quality of the information reported about actions to  deactivate legacy systems was questionable for the following reasons:
Data were not complete or accurate: HUD reported that it had  deactivated 19 systems. However, the agency’s reported cost savings  associated with deactivated systems were not complete. For example,  savings for 10 of the systems were either listed as “unknown,” or  information about estimated or actual savings associated with  deactivated systems was missing. Further, 2 of the systems reported  as deactivated remained operational but were moved off external  contracts to in-house support.
Data were not consistently reported: Specifically, the period of time  over which certain savings should extend was not consistently  specified. For example, the 2014 expenditure plan attributed to a  mapping system for grants management reported savings of an  estimated $50,000 for each grantee every 5 years. However, for the  other deactivated systems, the data did not clearly show for how  many years estimated savings or actual savings would accrue.
HUD also reported during fiscal year 2014 that it had achieved specific  operational efficiencies from selected IT projects under way or completed.  Specific sources of reported efficiencies identified included the following:
The Office of the CIO reported that selection of and investment in  human capital management systems had resulted in a number of  quantified business benefits such as reductions in hiring time;  improved data processing accuracy and quality; improved timeliness  of performance reviews and overall communication delivery;  increases in assessments of employee training needs; and an  increase in the response rate for exit surveys for outgoing employees.
The office also reported operational efficiencies achieved through  investments in specific projects but did not quantify the benefits  reported to have been generated by those systems:
The Integrated Budget Forecasting Model and its functionality  designed to continually monitor short-term funding needs and  automatically detect excess funding on contracts were associated  with reductions in payment delays, fewer disruptions caused by  inaccurate projections of needed funding, and increased ability of  multifamily housing staff to perform timely recaptures of  overpayments to housing authorities and cancel excess account  balances.
The Federal Housing Administration Transformation effort was  reported to have increased data quality and streamlined delivery  to financial services customers.
Implementation of a modernized enterprise-wide integrated  acquisitions management system was reported to have reduced  inefficiency and processing time and improved service delivery by  eliminating duplication in the acquisition/procurement process.
However, as with reported cost savings, the quality of the reported  information on operational efficiencies achieved is questionable. In this  regard, the Office of the CIO could not provide evidence that operational  efficiencies achieved had been verified or validated. It also could not  provide supporting documentation to substantiate the reliability of the data  about reported efficiencies or to show that they were supported by data  that were complete, accurate, or consistent.
The questionable quality of HUD’s reported cost savings and operational  efficiencies may be attributable, in part, to specific governance  weaknesses previously discussed in this report. Among other things,  insufficient requirements for collecting data in IT proposals about  expected benefits or reporting of progress made toward those goals may  limit the information available about savings and efficiencies for individual  IT projects. In addition, the lack of comprehensive information for IT  investment management leaves the Office of the CIO dependent upon ad  hoc reporting efforts and searches for data reported for other purposes to  identify cost savings and operational efficiencies. Finally, deficiencies in  documentation of governance decision making may limit the office’s  capacity to attribute cost savings and operational efficiencies to its IT  governance practices.
According to officials from the Office of the CIO, including the Acting  Deputy CIO for Customer Relationship and Performance Management  and the investment management division director, HUD lacks a  systematized view of cost savings and a formal written policy for  identifying and reporting on cost savings and operational efficiencies.  They explained that savings are not documented in one place, are not  tracked, and are measured in different ways for different projects. Further,  the Office of the CIO does not systematically assess whether planned  savings and efficiencies (including those associated with management of  its enterprise architecture) have been achieved.
Lacking a process for identifying savings and efficiencies from all of its  governance decision making, the Office of the CIO cannot obtain a  comprehensive view of savings and efficiencies planned for or  accomplished by its portfolio of IT investments and, thus, does not have  assurance that investments are delivering expected benefits. The office  may also be missing opportunities to expedite investments that are  producing greater than expected savings and efficiencies. Finally, without  comprehensive data about expected or actual savings and efficiencies,  the department will be limited in its ability to gauge the effectiveness of its  overall governance efforts and report to Congress on the progress it is  making in improving its IT environment.
section_title: Conclusions
paragraphs: HUD has made important progress in establishing IT governance  practices. Specifically, the Office of the CIO has established investment  review boards and guiding policies for governance. Additionally, the office  has partially implemented processes for selecting and overseeing  investments.
The need to address long-standing shortcomings in HUD’s IT  management capability and improve the overlapping, duplicative,  antiquated, and costly-to-maintain systems it uses to support the  department’s mission make it critical that the Office of the CIO fully  implement and sustain effective IT governance practices. Lacking boards  that operate as intended, as well as complete and current policies for IT  management, the department faces risk that its governance efforts will fall  short of ensuring effective decision making. Further, maturing selection  processes to ensure consistent application of selection criteria,  comparison of proposed projects based on key factors, and adequate  documentation of the entire process has the potential to bolster HUD’s  investment management and better position leaders to implement more  strategic management of the investment portfolio. Developing and  sustaining robust processes for monitoring the progress of investments  and evaluating their performance against expected outcomes would also  aid the Office of the CIO in better understanding what it is obtaining for  the investments made in IT each year. Until investment management  practices are fully implemented, HUD cannot be assured that its IT  investments are providing planned functionality efficiently and effectively  and the department may be missing opportunities to improve the quality  and outcomes of IT investments.
Finally, in the absence of capability to estimate and monitor governance- related cost savings and operational efficiencies, HUD has limited ability  to monitor the outcomes of its IT governance activities.
section_title: Recommendations for Executive Action
paragraphs: To ensure that HUD fully implements and sustains effective IT  governance practices, we recommend that the Secretary of Housing and  Urban Development direct the Deputy Secretary and the department’s  Chief Information Officer to place a high priority on taking the following  actions:
Ensure that the executive-level investment review board meets as  outlined in its charter, documents criteria for use by the other boards,  and distributes its decisions to appropriate stakeholders.
Fully establish and maintain a complete set of governance policies,  establish time frames for establishing policies planned but not yet  developed, and update key governance documents to reflect changes  made to established practices.
Fully establish an IT investment selection process that includes (1)  articulating how reviews of project proposals are to be conducted; (2)  planning how data (including cost estimates) are to be developed and  verified and validated; (3) establishing criteria for how cost, schedule,  and project risk are to be analyzed; (4) developing procedures for how  proposed projects are to be compared to one another in terms of  investment size (cost), project longevity (schedule), technical difficulty,  project risk, and cost-benefit analysis; and (5) ensuring that final  selection decisions made by senior decision makers and governance  boards are supported by analysis, consider predefined quantitative  measures, and are consistently documented.
Fully establish a well-defined process that incorporates key practices  for overseeing investments, including (1) monitoring actual project  performance against expected outcomes for project cost, schedule,  benefit, and risk; (2) establishing and documenting cost-, schedule-,  and performance-based thresholds for triggering remedial actions or  elevating project review to higher-level investment boards; and (3)  conducting post-implementation reviews to evaluate results of projects  after they are completed.
Further, to establish an enterprise-wide view of cost savings and  operational efficiencies generated by investments and governance  processes, we recommend that the Secretary of Housing and Urban  Development direct the Deputy Secretary and Chief Information Officer to  place a higher priority on identifying governance-related cost savings and  efficiencies and establish and institutionalize a process for identifying and  tracking comprehensive, high-quality data on savings and efficiencies  resulting from IT investments and the IT governance process.
section_title: Agency Comments
paragraphs: We received written comments on a draft of this report from the  Department of Housing and Urban Development (reproduced in app. II).  In its comments, the department stated that it concurred with our  recommendations and intends to provide more information, including  timelines for planned actions, once the final report has been issued.
We are sending copies of this report to interested congressional  committees. We are also sending copies to the Secretary of Housing and  Urban Development. In addition, the report is available at no charge on  the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made key  contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine (1) to what extent the Department of  Housing and Urban Development (HUD) has implemented key  information technology (IT) governance practices, including effective cost  estimation, and (2) what, if any, cost savings or operational efficiencies  HUD has reported achieving as a result of its IT governance practices.
To address our first objective, we reviewed and analyzed documentation  on HUD’s IT governance practices. This documentation included HUD’s  IT Governance Policy that called for the establishment of an IT  management framework and four investment review boards; the IT  Management Framework and Governance Concept of Operations that  identified key IT policies and processes to support IT governance and  investment decision making; and charters that established the roles,  responsibilities, authorities, and operations of the department’s  investment review boards.
We also reviewed documents showing how HUD determines which IT  projects will be recommended and then selected to receive funding,  including business cases, the criteria for assigning scores to competing  investments, and master lists of final scoring results. Additionally, we  reviewed available meeting minutes and presentations from HUD’s  investment review boards related to investment decision making. Further,  we reviewed documents regarding the Office of the Chief Information  Officer’s (CIO) recent efforts to establish an enterprise-wide cost  estimation methodology, including a cost element dictionary and  memorandum to provide guidance for formulation of HUD’s fiscal year  2016 IT budget.
To assess HUD’s IT governance practices, we compared the evidence  collected from our document reviews and analysis against critical  processes in the information technology investment management  framework developed by GAO that were most relevant to HUD’s efforts.  These processes establish basic capabilities that lay the foundation for  implementing more mature governance capabilities in the future,  specifically instituting governance boards and establishing processes for  selecting and overseeing investments. In addition, we identified  supplementary criteria for assessing cost-estimating practices that were  derived from GAO’s Cost Estimating and Assessment Guide.determined that the information provided by the Office of the CIO was  sufficiently reliable for addressing this objective.
To address our second objective, we obtained and evaluated HUD- reported information on achieved cost savings and operational  efficiencies accomplished through selected investments and governance  decisions. Various sources of reported savings and efficiencies included  the department’s fiscal year 2014 expenditure plan, which identified  recent accomplishments and associated benefits from IT modernization  efforts, a list of systems that had been deactivated since the beginning of  fiscal year 2010 that identified resultant actual or estimated annual cost  savings, and reports submitted to the Office of Management and Budget  on cost savings achieved through the consolidation of commodity IT and  cost avoidance achieved through decisions to eliminate system-support  contracts.
Because reporting of savings and efficiencies was fragmented and not  clearly attributed to governance decision making, we summarized the  savings and efficiencies included in disparate Office of the CIO reports  and asked officials to determine whether the information, as summarized,  was accurate and complete. The officials confirmed that the list of savings  we compiled from various reports accurately depicted its governance- related savings and efficiencies; however, we determined that the data  provided were questionable because the information was not complete,  accurate, consistent, or verified. Further, although requested, the Office of  the CIO could not provide documentation to substantiate the information  that it reported. As a result, we determined that the information provided  by the office to report governance-related cost savings and operational  efficiencies was not sufficiently reliable for the purposes of our review.  Specifically, the data provided were not consistent and complete, and the  Office of the CIO did not provide supporting documentation we could  have used to assess the reliability of the information, such as  documentation showing how reported cost savings and operational  efficiencies were achieved or how, if at all, they related to specific  governance decisions.
To address both objectives, we interviewed officials from the  department’s Office of the CIO, including the Deputy Chief Information  Officer (formerly the Acting Chief Information Officer), the Acting Deputy  CIO for Customer Relationship and Performance Management, the Acting  Chief Technology Officer, the Acting Chief IT Transformation Officer, the  Investment Management Division Director, and the Acting Enterprise  Architecture Director, to obtain information on HUD’s IT governance  process. We also interviewed HUD officials from the department’s Office  of Strategic Planning and Management Office, including the Acting  Director.
We conducted this performance audit from February 2014 to December  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objective. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Housing and Urban Development
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact above, Mark T. Bird (Assistant Director), Donald  A. Baca, Kami J. Corbett, Amanda C. Gill, Lee A. McCracken, and Roger  M. Smith made significant contributions to this report.