Information Technology: DHS Needs to Further Define and Implement Its New Governance Process
Published: 2012-07-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DHS has one of the largest IT budgets in the federal government. In fiscal year 2012, DHS plans to spend about $5.6 billion to, among other things, acquire, implement, and operate approximately 360 IT programs, including about 83 major programs, which are intended to assist in carrying out its diverse missions. With such a large portfolio of IT programs, it is important to ensure that the appropriate governance exists so that the programs meet their cost, schedule, and performance goals and continue to support the departments strategies and objectives. In line with this, DHS has been working to define and implement a new IT governance process.
GAO was asked to (1) describe DHS's new IT governance process and associated policies and procedures, and assess them against best practices; and (2) determine progress made in implementing the new process and how DHSs implementation efforts comport with relevant best practices. To do so, GAO analyzed relevant documentation and interviewed DHS officials responsible for defining and implementing the new governance process.
section_title: What GAO Found
paragraphs: The Department of Homeland Security (DHS) has defined a vision for its new information technology (IT) governance process, which includes a tiered oversight structure that defines distinct roles and responsibilities throughout the department. The new governance framework and the associated policies and procedures are generally consistent with recent Office of Management and Budget (OMB) guidance and with best practices for managing projects and portfolios identified in GAOs IT Investment Management framework, with two practices partially addressed and seven others fully addressed. For example, consistent with OMB guidance calling for the Chief Information Officer (CIO) to play a significant role in overseeing programs, DHSs draft procedures require that lower-level boards overseeing IT programs include the DHS CIO, a component CIO, or a designated executive representative from a CIO office. In addition, consistent with practices identified in GAOs IT Investment Management framework, DHSs draft procedures identify key performance indicators for gauging portfolio performance. However, DHSs policies and procedures have not yet been finalized, because, according to officials, the focus has been on piloting the new governance process. While it is important to conduct pilots to test processes and identify lessons learned, until the department finalizes the policies and procedures associated with the new IT governance, it will have less assurance that its new IT governance will be consistent with best practices and address previously identified weaknesses in investment management.
DHS has begun to implement aspects of its new governance process. For example, it has established several governance entities and conducted program health assessment reviews for all of its major IT programs. In implementing its new governance, the department has generally followed key industry best practices, such as establishing an implementation team; however, the department has not fully followed other practices, including developing a mechanism to capture lessons learned. The table below summarizes GAOs assessment of DHSs implementation efforts. Until the department fully addresses these practices, its implementation approach may be less effective than intended.
section_title: What GAO Recommends
paragraphs: To implement an effective IT governance process, GAO recommends that DHS finalize associated policies and procedures, and fully follow best practices for implementing the process. In comments on a draft of this report, DHS concurred with GAOs recommendations and estimated it would address them by September 2013.

section_title: Letter
paragraphs: The Department of Homeland Security (DHS) has one of the largest  information technology (IT) budgets in the federal government. With a  fiscal year 2012 IT budget of about $5.6 billion, DHS plans to use these  funds to, among other things, acquire, implement, and operate  approximately 360 IT programs, including about 83 major programs, to  assist in carrying out its diverse missions. Given the size of DHS’s IT  portfolio and its importance to the department’s mission, it is important to  ensure that the appropriate governance exists so that the programs meet  their cost, schedule, and performance goals and continue to support the  department’s strategies and objectives. According to the IT Governance  Institute’s Control Objectives for Information and related Technology 4.1,IT governance is the responsibility of executives, and consists of the  leadership, organizational structures, and processes that ensure that an  enterprise’s IT sustains and extends the organization’s strategies and  objectives.
In 2003, we designated implementing and transforming DHS as high risk,  in part due to weaknesses related to the department’s management of its  IT programs. improve and streamline its IT governance and address continuing  weaknesses. This report responds to your request that we conduct a  review of DHS’s new IT governance process. Specifically, our objectives  were to (1) describe DHS’s new IT governance process and associated  policies and procedures, and assess them against best practices; and (2)  determine progress made in implementing the new process and how  DHS’s implementation efforts comport with relevant best practices.
GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: February 2011).  GAO designated implementing and transforming DHS as high risk because DHS had to  transform 22 agencies—several with major management challenges—into one  department, and failure to effectively address DHS’s management and mission risks could  have serious consequences for U.S. national and economic security. The high-risk area  includes challenges in strengthening DHS’s management functions, including acquisition,  IT (which included IT governance), financial, and human capital management; the impact  of those challenges on DHS’s mission implementation; and challenges in integrating  management functions within and across the department and its components. and the implementation of the framework against industry best practices.  Details on our scope and methodology can be found in appendix I.
We conducted this performance audit from October 2011 to July 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: DHS’s mission is to lead the unified national effort to secure America by  preventing and deterring terrorist attacks and protecting against and  responding to threats and hazards to the nation. The department is also  responsible for ensuring that the nation’s borders are safe and secure,  welcoming lawful immigrants and visitors, and promoting the free flow of  commerce. Created in 2002, DHS assumed control of about 209,000  civilian and military positions from 22 agencies and offices that specialize  in one or more aspects of homeland security. The intent behind the  merger that created DHS was to improve coordination, communication,  and information sharing among these multiple federal agencies. Each of  these agencies is responsible for specific homeland security missions and  for coordinating related efforts with its sibling components, as well as  external entities. Figure 1 shows the department-level organizations  which are responsible for or share responsibility for IT acquisition  management activities.
Within the department’s Management Directorate, headed by the Under  Secretary for Management, is the Office of the CIO. The CIO’s  responsibilities include setting departmental IT policies, processes, and  standards, and ensuring that IT acquisitions comply with DHS IT  management processes, technical requirements, and approved enterprise  architecture, among other things. Additionally, the CIO chairs DHS’s Chief  Information Officer Council, which is responsible for ensuring the  development of IT resource management policies, processes, best  practices, performance measures, and decision criteria for managing the  delivery of IT services and investments, while controlling costs and  mitigating risks.
Within the Office of the CIO, EBMO has been given primary responsibility  for ensuring that the department’s IT investments align with its missions  and objectives. EBMO was recently reorganized to include a new  Enterprise Portfolio Governance Division dedicated to executing portfolio  reviews. This division is to provide support to portfolio stakeholders to  administer portfolio activities, such as aligning programs with portfolios,  creating baseline portfolios, and establishing portfolio pilot efforts.
In October 2011, DHS realigned its acquisition management functions  previously performed by divisions within the Office of the Chief  Procurement Officer to establish PARM. The office, which reports  directly to the Under Secretary for Management, is to ensure the  effectiveness of the overall program execution governance process in  support of the department’s Investment Review Board (IRB), and has the  responsibility for developing and maintaining DHS’s Acquisition  Management Directive. PARM is also responsible for providing  independent assessments of major investment programs, and monitoring  programs between formal reviews to identify any emerging issues that  DHS needs to address to keep the programs on track.
subsections: section_title: DHS’s Acquisition Management Process
paragraphs: DHS acquisitions—which are expected to total about $18 billion in fiscal  year 2012—support a wide range of missions and investments, including  ships and aircraft, border surveillance and screening equipment, nuclear  detection equipment, and systems to track the department’s financial and  human resources. In support of its diverse missions, DHS plans to spend  about $5.6 billion in fiscal year 2012 to deploy and maintain over 360 IT  programs to perform both mission-critical and support functions, which  frequently must be coordinated among components, as well as among  external entities. In 2003, DHS established an investment review process  to help reduce risk and increase the chances for successful acquisition  outcomes by providing departmental oversight of major investments  throughout their life cycle and to help ensure that funds allocated for  investments through the budget process are being spent wisely,  efficiently, and effectively. In October 2010, DHS updated the acquisition  guidance, which outlined the acquisition life cycle phases and called for  senior-level approval of each major acquisition program at key decision  events during a program’s acquisition life cycle.
DHS’s Acquisition Review Board —renamed the IRB in October 2011— was established to review and approve major acquisition programs, at  key stages in their life cycles before the acquisition program could move  to the next phase. DHS’s guidance establishes four phases that  constitute the acquisition life cycle:  1.  The need phase—during which a problem is defined and the needed  capability is identified. This phase concludes with the IRB granting the  acquisition program approval to proceed at Acquisition Decision   Event 1. 2.  The analyze/select phase—during which it is determined how to  provide the needed capability. This phase concludes with the IRB  granting the acquisition program approval to proceed at Acquisition  Decision Event 2A. 3.  The obtain phase—during which the needed capability is obtained.
The IRB may review the acquisition program multiple times before  granting the acquisition program approval to proceed with particular  acquisition activities. Between Acquisition Decision Event 2A and  Acquisition Decision Event 2B, the project manager formulates the  acquisition into types of acquisition (e.g., capital investment projects,  services procurements). The phase concludes with the IRB granting  the program approval to proceed at Acquisition Decision Event 3.  4.  The produce/deploy/support phase—during which the process to  produce, deploy, and support the needed capability takes place.  Although the IRB does not have a standard, defined role in this phase,  it may conduct additional reviews as necessary.
Figure 2 presents the four DHS acquisition phases defined in DHS’s  acquisition directive.
DHS recently articulated a vision for an Integrated Investment Lifecycle  Model as part of a broad effort to improve its overall acquisition  management process. The model is intended to help integrate the  department’s planning, programming, budgeting, and execution  processes with the goal of strengthening strategic decision making by  implementing a repeatable process at critical phases throughout the  investment life cycle. In June 2012, DHS noted that with the early phases  of the model’s development, a key challenge was developing  standardized business terms, data elements, and a central portal to  collect, store, and report data. DHS also reported that it planned to  complete a concept of operations document and an implementation plan  for the Integrated Investment Lifecycle Model concept of operations by  the second quarter of fiscal year 2013.
section_title: GAO and DHS Have Reported on the Department’s Acquisition Management Efforts
paragraphs: Over the years, GAO, the department, and DHS’s Office of the Inspector  General have reviewed DHS’s overall acquisition process, as well as the  processes specifically related to its IT acquisitions, identified weaknesses,  and provided recommendations for improvements. For example, in April  2007, we reported that although DHS had established the management  structure to effectively manage its IT investments, the department had yet  to define most of the policies specifically associated with managing its IT  projects as investment portfolios. We also reported that DHS had not  fully implemented the key practices needed to actually control  investments—either at the project level or at the portfolio level. We  recommended that DHS fully define the project-level and portfolio-level  policies and procedures defined in GAO’s ITIM frameworkimplement the practices needed to effectively control investments. DHS  agreed with our findings and recommendations, and took action to  address the majority of them, including augmenting the resources for  providing project oversight.
In November 2008, we reported that DHS had not provided the oversight  needed to identify and address cost, schedule, and performance  problems for its major acquisitions. Specifically, we reported that of the  48 major investments reviewed that required milestone or annual reviews,  45 were not reviewed in accordance with the department’s investment  review policy, and 18 were not reviewed at all. Four of these investments  had transitioned into a late acquisition phase—production and  deployment—without any required reviews. We recommended—and DHS  concurred—that the department identify and align sufficient management  resources to implement oversight reviews in a timely manner throughout  the investment life cycle.
In 2010, DHS’s Office of the CIO conducted a series of reviews of its  major IT programs. According to DHS’s draft Portfolio Governance  Concept of Operations, these reviews, plus additional insights from the CIO, highlighted several problems with the existing IT investment  management process, including the following:
Governance was based on a program-by-program approach that did  not reflect the reality that many programs are inter-related in  delivering mission outcomes.
Programs took too long, were often over cost, and did not meet  performance objectives.
Decision makers were often far removed from program details; many  decisions were reserved for the IRB, which lacks the capacity and  time to be familiar with all mission needs and program issues.
In addition, the department again held portfolio reviews in 2011 and found  that while some progress had been made, many of the aforementioned  problems continued.
Following the February 2011 update to our high-risk series report on  implementing and transforming DHS, in which we reported that while the  department had improved its policies and procedures for investment  management, more work remained, DHS began providing us with bi- annual updates on its progress in addressing weaknesses, including IT  management.
Further, we recently reported that the portfolio reviews DHS conducted in  2011 to avoid investing in systems that were duplicative or overlapping,  and to identify and leverage investments across the department  contributed to the identification and consolidation of duplicative  functionality within four investments. further consolidate systems by 2014, which is expected to produce  approximately $41 million in cost savings. The portfolio reviews also  contributed to the identification of 38 additional systems that are  duplicative.
GAO, Information Technology: Departments of Defense and Energy Need to Address  Potentially Duplicative Investments, GAO-12-241 (Washington, D.C.: Feb. 17, 2012).
section_title: OMB Has Established Initiatives to Reform Federal IT Investment Management through Portfolio Governance and Accountability
paragraphs: In December 2010, the Office of Management and Budget (OMB) issued  its 25 Point Implementation Plan to Reform Federal Information  Technology Management, a document outlining activities spanning 18  months to reform IT management throughout the federal government. A  key goal of the plan—referred to as the IT Reform Plan—is to foster more  effective management of large-scale IT programs. One way the plan  recommends this be done is through streamlining governance and  improving accountability. According to the plan, this involves reforming  and strengthening IRBs to enable them to more adequately manage  agency portfolios, redefining the role of agency CIOs and the federal CIO  Council to focus on portfolio management, and rolling out “TechStat”  reviews at the agency and bureau levels to focus attention on IT  investments, including those that are poorly performing. In April 2012,  we reported that OMB and key federal agencies had made progress on  selected action items identified in the IT Reform Plan, but several areas,  including strengthening the role of the CIO, were not yet completed.
OMB, Chief Information Officer Authorities, M-11-29 (Washington, D.C.: August 8,  2011).
section_title: DHS’s New IT Governance Process Is Generally Consistent with Best Practices
paragraphs: DHS has defined a vision for its new IT governance process, which  includes a three-tiered oversight structure that defines distinct roles and  responsibilities from the program through the department. The new  governance process is generally consistent with recent OMB guidance  and with best practices for managing projects and portfolios, with two  practices partially addressed and seven others fully addressed. However,  the supporting policies and procedures have yet to be finalized.   According to officials, this is because the focus has been on piloting the  process. While it is important to conduct pilots to test processes and  identify lessons learned, until DHS finalizes its procedures associated  with the new IT governance, it will have less assurance that the  governance will be consistent with best practices and address previously  identified weaknesses in investment management.
subsections: section_title: DHS’s Vision for New IT Governance Process is Based on a Tiered Oversight Structure
paragraphs: In December 2011, the Under Secretary for Management’s Integrated  Strategy for High Risk Management introduced an initiative to improve  and streamline IT program execution governance. The initiative  established a tiered governance structure for program execution, with  specific decision responsibilities for each tier. This structure is intended to  supplement, not replace, DHS’s existing policies and procedures for  managing acquisitions. It also includes several of the governance entities  represented in the Integrated Investment Lifecycle Model the department  is currently defining to integrate the planning, programming, budgeting,  and execution processes.
According to the strategy, the governance process is intended to improve  IT investment management across DHS by providing enterprise-level  governance and oversight based on functional IT portfolios;  the integration of enterprisewide processes for strategic planning,  program management, budget planning, acquisition, and program  execution by establishing a tiered governance structure—enterprise,  portfolio, and program governance; program health by continuing initiatives to enable IT programs to  manage to budget and schedule, mitigate risks, and deliver desired  functionality; and IT investment performance reporting across the department by  leveraging a business intelligence tool—known as the Decision  Support Tool—that will provide standardization of data and consistent  monitoring of IT portfolios and programs.
To meet these goals, DHS has defined a scalable, tiered structure among  the governance entities that establishes distinct roles and responsibilities  from the program through the department. (This structure is illustrated in  figure 3.)
Enterprise level governance—Enterprise governance will focus on  setting strategic priorities and requirements to meet the enterprise  mission.
Portfolio level governance—Portfolio governance will manage groups  of related programs to ensure that programs are properly aligned and  function to meet DHS mission needs.
Program level governance—Program governance is the primary  decision-making authority for individual programs.
Table 1 lists the entities involved in the tiered governance process, and  provides a description of their membership. While some of the entities are  new, others already existed.
In addition to the tiered structure intended to improve oversight of  programs and portfolios, the IT governance vision also more broadly  addresses the process for determining what investments should be  selected to address mission gaps (during the planning phase) and  ultimately, deliver capabilities (during the execution phase). The following  describes these two phases:
During the planning phase, DHS expects to consider IT investments  from a departmentwide portfolio perspective, and determine which  investments should be funded. To do this, DHS envisions the use of  functional portfolios. The portfolios are to align to the 13 functional  segments of the department’s enterprise architecture and will be  governed by the Portfolio Governance Boards. The Department  Strategy Council is to provide these portfolios with strategic guidance.  Using input from the portfolio managers, the Capabilities and  Requirements Council is to adjudicate issues across portfolios and set  overall department priorities. In addition, the Portfolio Governance  Boards will provide guidance and investment recommendations to the  Program Review Board for future year planning, programming, and  budgeting. The Program Review Board will then formulate the budget  and communicate with OMB and the IRB.
The execution phase is intended to ensure the investments continue  to be aligned with the department’s strategic goals and fully support  mission performance, and ultimately provide the intended capabilities.  DHS envisions an interaction among the entities that provide  oversight, control, and assistance during program execution—the IRB,  the ESCs, and COEs. The ESCs are to oversee major programs or  groups of closely related programs. However, if a major program runs  into difficultly in its schedule, budget, or scope performance, the  program manager would be required to present the status and action  plan to the IRB for assessment and approval. This requirement is in  addition to the ongoing governance provided by the ESCs. The COEs  would provide expert support in core program management  disciplines, such as requirements engineering. The department also  plans to standardize reviews, including the TechStats and program  health assessments. The TechStat reviews are intended to evaluate  programs, identify any issues, develop a corrective action plan, and  ensure that the program receives proper support to fix the issues. The  program health assessments look at individual programs’ health in  executing and carrying out the program. In fiscal year 2011, DHS  performed reviews of 47 IT programs, primarily focusing on those  programs with poor performance ratings or deemed highly critical or  visible by the CIO. Under the new governance, the Office of the CIO  plans to conduct reviews of all 83 major IT investments anywhere  from monthly to semi-annually, depending on the health of the project.
According to officials, while the Under Secretary for Management has not  yet approved the new IT governance model, DHS has started  implementing it. Figure 4 shows the recommended process for  determining what investments should go forward, providing oversight, and  delivering capabilities.
In addition, DHS plans to use its Decision Support Tool to provide  updated program information to all governance bodies on the status of  major programs. The Decision Support Tool is to provide the ability to  integrate data resident within multiple existing source systems including  the Next Generation Periodic Reporting System and the Investment  Management System, to generate reports, charts and graphs on program  performance.
section_title: DHS’s New IT Governance Framework Is Generally Consistent with Guidance, Best Practices, but Policies and Procedures Have Not Yet Been Finalized
paragraphs: DHS’s new IT governance framework is generally consistent with recent  OMB guidance. Specifically, consistent with OMB guidance calling for the  CIO to play a significant role in the oversight of the portfolio of IT  programs, DHS’s draft procedures note that ESCs overseeing IT  programs must include the DHS CIO, a component CIO, or a designated  executive representative from a CIO office. In addition, for programs that  the DHS CIO Council designates as most critical, the DHS CIO or an  appropriate component CIO will co-chair the ESC. Further, consistent  with recent OMB guidance to focus on portfolio management, the new  governance framework includes the establishment of portfolio governance  boards to oversee functional portfolios with the goals of eliminating  duplication and leveraging services and programs across the department.
In addition, DHS’s new IT governance framework and the associated  policies and procedures are generally consistent with best practices for  managing projects and portfolios identified in GAO’s ITIM framework, with  two practices partially addressed and seven others fully addressed;  however, the procedures have not been finalized. Tables 2 and 3  summarize our assessment of DHS’s draft procedures against relevant  practices of our ITIM.
According to officials, the policies and procedures supporting DHS’s new  IT governance have not been finalized because the focus has been on  piloting the new governance process. To the department’s credit,  according to officials, the Portfolio Governance Concept of Operations,  which focuses more on the role of the Portfolio Governance Boards, is  expected to be approved by the end of July. In addition, the Program  Governance Concept of Operations, which is intended to specify how  programs are to be overseen, is currently being drafted and expected to  be finalized in August 2012 (and approved by the CIO later). Further,  according to officials, resources have recently been assigned to updating  DHS’s current Information Technology Integration and Management  directive to reflect the new framework, and work has also begun to update  the Acquisition Management Directive. While the use of pilots is valuable  in testing processes and identifying lessons learned, until DHS finalizes  the policies and procedures associated with the new IT governance, the  department will have less assurance that its new IT governance will be  consistent with best practices and address previously identified  weaknesses in investment management.
section_title: DHS Has Implemented Aspects of Its New Structure, but Has Not Fully Followed Best Practices
paragraphs: DHS has begun to implement components of the IT governance  framework. Specifically, the department has primarily focused its  implementation efforts on the “execution” phase of the IT governance  operating model (see fig. 4) because, according to officials, it includes the  department’s more mature processes. Efforts to implement other aspects  of the model have been more limited.
Table 4 below shows the status of the implementation of the governance  bodies.
In addition to progress made with the governance entities above, DHS  has also taken other steps to implement the new IT governance structure.
Portfolio reviews: According to the CIO, the department is currently  performing portfolio reviews in collaboration with PARM and the Office  of the Chief Financial Officer. These reviews look at the alignment of a  functional grouping of investments and the mission effectiveness, or  value, those investments deliver. The focus of these reviews is to  identify overlaps and redundancies in existing investments, and to  identify gaps in existing capabilities. According to the Deputy CIO,  these portfolio reviews represent a new way of operating for the  components, as they were not used to looking at functions across the  department, only portfolios within their respective components.  According to the CIO, as the Portfolio Governance Boards are  established, they are expected to take over the role of reviewing these  portfolios.
Program reviews: In fiscal year 2011, the CIO’s office performed  program health assessment reviews of 47 IT programs, primarily  focusing on those programs with poor performance ratings or  determined to be highly critical or visible by the CIO. According to  officials, all 83 major IT investments have been reviewed for this fiscal   year.
TechStat reviews: According to officials, the Office of the CIO also  performed two TechStat reviews in fiscal year 2011, and three of the  four TechStats planned for fiscal year 2012. In addition, officials  stated they have completed TechStat training for 12 components.
subsections: section_title: DHS Is Taking Steps to Obtain Organizational Buy- in
paragraphs: According to industry best practices, in order to effectively implement a  new IT governance framework, or any large organizational change,  organizations should obtain buy-in by involving all key stakeholders to  ensure key perspectives are considered and facilitate adoption. This  includes having top management support and creating forums for  involving business representatives.
DHS has top management support, as evidenced by the Under Secretary  for Management approving key documents supporting the new  governance process. For example, the December 2011 Integrated  Strategy for High Risk Management, which introduces the vision for the IT  governance structure, was approved by the Under Secretary for  Management. The Under Secretary also issued a memorandum calling  for the use of pilot ESCs and providing these ESCs authority to oversee  investment performance. In addition, to secure buy-in, EBMO also sought  comments on a draft of its Portfolio Governance Concept of Operations,  and, according to officials communicated its efforts to improve IT  investment management to senior executives (including the Chief  Financial Officer and Chief Procurement Officer) and to the components.  According to the EBMO Director, these stakeholders’ participation on  ESCs or portfolio review boards and involvement in conducting the  portfolio reviews has also helped to secure their buy-in. Taking these  steps increases the likelihood that the new IT governance process will be  adopted despite the significant cultural change it represents. The CIO and  Executive Director for EBMO both noted that the new governance is  gradually gaining acceptance.
section_title: An Implementation Team Has Been Established, but the Department Lacks an Implementation Plan
paragraphs: Effectively implementing a new IT governance process also requires  developing an effective implementation team and plan. According to best  practices, an effective implementation team should be put in place that  includes key stakeholders from both business and IT components. In  addition, we have previously reported that to effectively implement IT  investment management processes, organizations need to be guided by  a plan that builds on existing strengths and weaknesses; specifies  measurable goals, objectives, and milestones; specifies needed  resources; assigns clear responsibility and accountability for  accomplishing tasks; and is approved by senior-level management.
Such a plan is instrumental in helping agencies coordinate and guide  improvement efforts.
DHS has established an integrated product team to guide the  implementation of the new IT governance structure. According to DHS  officials, the department’s integrated product team is made up of officials  from EBMO, PARM, and the department’s chief executives (e.g., CIO,  Chief Financial Officer, etc.). Further, according to the department, the  EBMO senior staff directly involved with the implementation of the  governance structure have both program-level and portfolio-level  governance experience. According to officials, the integrated product  team is supported by subject matter experts and the Component  Acquisition Executives.
However, the department has not yet developed an implementation plan.  While DHS’s June 2012 Integrated Strategy for High Risk Management  includes components of an implementation plan, such as high-level goals  and activities to be completed by the end of fiscal year 2012, DHS does  not have an implementation plan addressing the elements mentioned  above. While officials recognized the value of an implementation plan,  they stated they had not yet developed one because they are still piloting  the new governance process.  Further, officials stated that they planned  on developing a 2-year implementation plan that would incorporate best  practices identified in the ITIM framework for effectively managing  projects and portfolios. They stated that this plan would draw from the  department’s revised IT strategic plan due to be completed by the end of  the summer of 2012. To mitigate for the limited resources available, DHS  states it will adopt a federated model for resources—drawing resources  from components—and use contractor support, as needed. Until a plan is  developed, the department may not be able to effectively implement the  new IT governance process to ensure that it addresses previously  identified weaknesses, and effectively uses the department’s limited  resources.
section_title: DHS Has Yet to Fully Define Processes for Evaluating Its Implementation Efforts
paragraphs: According to best practices, when implementing an IT governance  framework, it is important to evaluate the implementation efforts by,  among other things, developing measures to assess progress in meeting  objectives. In addition, according to best practices, when implementing a  new governance framework, there should be mechanisms in place to  document lessons learned for subsequent governance improvement  initiatives. According to our ITIM framework, among other things, lessons  learned and recommendations for improving the investment process  should be developed and documented, and then distributed to all  stakeholders.
DHS has defined measures in the June 2012 update to the Integrated  Strategy for High Risk Management. For example, DHS plans to measure  the (1) percentage of DHS IT program reviews completed in fiscal year  2012; (2) percentage of DHS IT programs rated as low risk; (3)  percentage of IT investment portfolio governance boards and ESCs  established and chartered; and (4) percentage of components trained and  conducting component-led TechStats. In addition, the department has  identified measures to assess the effectiveness of the ESC program  reviews. Further, according to officials, draft measures for the COE  functions have also been developed; however, they have not yet been  documented and officials did not have any time frames for doing so.  Without fully defining and documenting measures of success, it is unlikely  that the department will be able to determine if it has successfully and  efficiently established its new IT governance framework.
DHS officials stated that they are using lessons learned from their piloting  activities to improve their governance process; however, they have not  established a mechanism for capturing lessons learned. Without this  mechanism, the department risks not being able to continue to build on  the experiences and lessons learned from prior initiatives, which could  help to identify, introduce, and sustain additional efficiency gains on a  more systematic basis.
section_title: Conclusions
paragraphs: DHS’s new approach to IT governance shows promise in establishing  mechanisms for greater oversight, involving key staff in program  decisions, and focusing on functional portfolios in order to avoid  unnecessary duplication. To the department’s credit, the vision is  generally consistent with guidance and with best practices for managing  projects and portfolios. However, we identified two practices which were  only partially addressed in policies and procedures—ensuring that the  IRB maintains visibility into the activities of the ESCs, and defining project  selection and prioritization criteria—which could limit the extent to which  these practices are institutionalized. In addition, the policies and  procedures have not yet been finalized. To its credit, DHS has efforts  underway to address this. However, until DHS finalizes these policies and  procedures and ensures that they fully address practices for managing  programs and portfolios of investments, the department will have less  assurance that its new approach will address identified weaknesses and  facilitate effective governance.
DHS has implemented aspects of its new approach, such as establishing  governance entities associated with the execution phase of its approach  and conducting portfolio and program reviews. However, because the  agency has not developed an implementation plan, fully documented  performance measures, or established a mechanism for capturing  lessons learned, there is a risk that the approach may be less effective  than intended.
section_title: Recommendations for Executive Action
paragraphs: To implement an effective IT governance strategy, we recommend that  the Secretary of Homeland Security direct the appropriate officials to  finish defining the new IT governance process by  finalizing the IT governance policies and procedures and ensuring  they fully address or reference existing documents that address the  following: how the IRB is to maintain responsibility for lower-level board  activities; and investment selection and prioritization criteria.
In addition, to assist in implementing the new IT governance strategy, we  recommend that the Secretary of Homeland Security direct the  appropriate officials to develop an implementation plan that draws together ongoing and  additional efforts needed to implement the new IT governance  process. The plan should: 1.  build on existing strengths and weaknesses;  2.  specify measurable goals, objectives, and milestones;  3.  specify needed resources;  4.  assign clear responsibility and accountability for accomplishing  5.  be approved by senior-level management.  fully define and document key measures to monitor the  implementation process; and establish mechanisms for capturing lessons learned.
section_title: Agency Comments and Our Evaluation
paragraphs: DHS’s Director for the Departmental GAO-OIG Liaison Office provided  written comments on a draft of this report (reprinted in appendix II). He  stated that the department was pleased to note GAO’s positive  acknowledgement that DHS’s IT governance framework was consistent  with OMB guidance and GAO’s ITIM framework. He also stated that the  department concurred with the recommendations and estimated it would  address them by September 30, 2013. The department also provided  technical comments, which we have incorporated where appropriate.
We are sending copies of this report to interested congressional  committees and the Secretary of Homeland Security. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staffs have any questions on the matters discussed in this  report, please contact me at (202) 512-9286 or pownerd@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. GAO staff who made  major contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to (1) describe the Department of  Homeland Security’s (DHS) new information technology (IT) governance  process and associated policies and procedures, and assess them  against best practices; and (2) determine progress made in implementing  the new approach and how DHS’s implementation efforts comport with  relevant best practices.
To address our first objective, we reviewed and analyzed documentation  on DHS’s newly initiated IT governance process. This documentation  included the Under Secretary for Management’s Program Management &  Execution Playbook, the December 2011 biannual update to the  Implementation and Transformation section of DHS’s Integrated Strategy  for High Risk Management, the Office of the Chief Information Officer’s  Concept of Operations documents for Program and Portfolio Governance,  and Chief Information Officer quarterly update briefings presented to  GAO. We also reviewed various memorandums from the Under Secretary  for Management, which covered such issues as the chartering of  Executive Steering Committees and the implementation of the  department’s Decision Support Tool. We also interviewed officials from  DHS’s Office of the Chief Information Officer’s Enterprise Business  Management Office (EBMO), and from the Office of Program  Accountability and Risk Management (PARM). To assess DHS’s vision of  its planned governance process against GAO’s Information Technology  Investment Management (ITIM) guide, we identified the ITIM stage 2 and  stage 3 critical process areas that were most relevant to DHS’s efforts,  and compared the evidence collected from our document reviews and  interviews to the practices associated with these areas.
To address our second objective, we obtained and evaluated  documentation showing the department’s efforts in implementing the  governance process. This included documentation defining the roles of  various governance entities, such as the Executive Steering Committees  and Portfolio Governance Boards, to be involved in the process. We  reviewed available charters and meeting minutes for several of these  entities to determine if they were functioning. We also reviewed  documentation on the Centers of Excellence established by DHS to  provide subject matter expertise to support program management. In  addition, we interviewed officials from EBMO and PARM to determine the  status of the department’s effort’s in implementing the governance  process. We also requested information from the five programs for which  Executive Steering Committees were first established on the various  reviews they had undergone since January 2010—including type of  review, time frames for each review, and information requested—to  determine the extent of duplication or overlap among the reviews. In order  to assess DHS’s implementation of the IT governance approach against  accepted best practices, we first used content-analysis software to  identify best practices shared by industry and the federal government. We  identified these practices from prior our reports, Office of Management  and Budget guidance, and guidance from recognized experts in IT  governance. We grouped the practices into three categories—(1)  organizational buy-in, (2) development of an implementation team and  implementation plan, and (3) evaluation— and determined from the  evidence collected from our document reviews and interviews the extent  to which DHS was following these best practices.
We conducted this performance audit from October 2011 to July 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Homeland Security
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, the following staff also made key  contributions to this report: Sabine R. Paul, Assistant Director; William G.  Barrick; Sairah R. Ijaz; Lee A. McCracken; and Tomas Ramirez.