Information Technology: DHS Needs to Further Define and Implement Its New Governance Process
Released: 2012-07-25T00:00:00

Letter

Background

DHS’s mission is to lead the unified national effort to secure America by preventing and deterring terrorist attacks and protecting against and responding to threats and hazards to the nation. The department is also responsible for ensuring that the nation’s borders are safe and secure, welcoming lawful immigrants and visitors, and promoting the free flow of commerce. Created in 2002, DHS assumed control of about 209,000 civilian and military positions from 22 agencies and offices that specialize in one or more aspects of homeland security. The intent behind the merger that created DHS was to improve coordination, communication, and information sharing among these multiple federal agencies. Each of these agencies is responsible for specific homeland security missions and for coordinating related efforts with its sibling components, as well as external entities. Figure 1 shows the department-level organizations which are responsible for or share responsibility for IT acquisition management activities.

Within the department’s Management Directorate, headed by the Under Secretary for Management, is the Office of the CIO. The CIO’s responsibilities include setting departmental IT policies, processes, and standards, and ensuring that IT acquisitions comply with DHS IT management processes, technical requirements, and approved enterprise architecture, among other things. Additionally, the CIO chairs DHS’s Chief Information Officer Council, which is responsible for ensuring the development of IT resource management policies, processes, best practices, performance measures, and decision criteria for managing the delivery of IT services and investments, while controlling costs and mitigating risks.

Within the Office of the CIO, EBMO has been given primary responsibility for ensuring that the department’s IT investments align with its missions and objectives. EBMO was recently reorganized to include a new Enterprise Portfolio Governance Division dedicated to executing portfolio reviews. This division is to provide support to portfolio stakeholders to administer portfolio activities, such as aligning programs with portfolios, creating baseline portfolios, and establishing portfolio pilot efforts.

In October 2011, DHS realigned its acquisition management functions previously performed by divisions within the Office of the Chief Procurement Officer to establish PARM. The office, which reports directly to the Under Secretary for Management, is to ensure the effectiveness of the overall program execution governance process in support of the department’s Investment Review Board (IRB), and has the responsibility for developing and maintaining DHS’s Acquisition Management Directive. PARM is also responsible for providing independent assessments of major investment programs, and monitoring programs between formal reviews to identify any emerging issues that DHS needs to address to keep the programs on track.

DHS’s Acquisition Management Process

DHS acquisitions—which are expected to total about $18 billion in fiscal year 2012—support a wide range of missions and investments, including ships and aircraft, border surveillance and screening equipment, nuclear detection equipment, and systems to track the department’s financial and human resources. In support of its diverse missions, DHS plans to spend about $5.6 billion in fiscal year 2012 to deploy and maintain over 360 IT programs to perform both mission-critical and support functions, which frequently must be coordinated among components, as well as among external entities. In 2003, DHS established an investment review process to help reduce risk and increase the chances for successful acquisition outcomes by providing departmental oversight of major investments throughout their life cycle and to help ensure that funds allocated for investments through the budget process are being spent wisely, efficiently, and effectively. In October 2010, DHS updated the acquisition guidance, which outlined the acquisition life cycle phases and called for senior-level approval of each major acquisition program at key decision events during a program’s acquisition life cycle.

DHS’s Acquisition Review Board —renamed the IRB in October 2011— was established to review and approve major acquisition programs, at key stages in their life cycles before the acquisition program could move to the next phase. DHS’s guidance establishes four phases that constitute the acquisition life cycle: 1. The need phase—during which a problem is defined and the needed capability is identified. This phase concludes with the IRB granting the acquisition program approval to proceed at Acquisition Decision Event 1. 2. The analyze/select phase—during which it is determined how to provide the needed capability. This phase concludes with the IRB granting the acquisition program approval to proceed at Acquisition Decision Event 2A. 3. The obtain phase—during which the needed capability is obtained.

The IRB may review the acquisition program multiple times before granting the acquisition program approval to proceed with particular acquisition activities. Between Acquisition Decision Event 2A and Acquisition Decision Event 2B, the project manager formulates the acquisition into types of acquisition (e.g., capital investment projects, services procurements). The phase concludes with the IRB granting the program approval to proceed at Acquisition Decision Event 3. 4. The produce/deploy/support phase—during which the process to produce, deploy, and support the needed capability takes place. Although the IRB does not have a standard, defined role in this phase, it may conduct additional reviews as necessary.

Figure 2 presents the four DHS acquisition phases defined in DHS’s acquisition directive.

DHS recently articulated a vision for an Integrated Investment Lifecycle Model as part of a broad effort to improve its overall acquisition management process. The model is intended to help integrate the department’s planning, programming, budgeting, and execution processes with the goal of strengthening strategic decision making by implementing a repeatable process at critical phases throughout the investment life cycle. In June 2012, DHS noted that with the early phases of the model’s development, a key challenge was developing standardized business terms, data elements, and a central portal to collect, store, and report data. DHS also reported that it planned to complete a concept of operations document and an implementation plan for the Integrated Investment Lifecycle Model concept of operations by the second quarter of fiscal year 2013.

GAO and DHS Have Reported on the Department’s Acquisition Management Efforts

Over the years, GAO, the department, and DHS’s Office of the Inspector General have reviewed DHS’s overall acquisition process, as well as the processes specifically related to its IT acquisitions, identified weaknesses, and provided recommendations for improvements. For example, in April 2007, we reported that although DHS had established the management structure to effectively manage its IT investments, the department had yet to define most of the policies specifically associated with managing its IT projects as investment portfolios. We also reported that DHS had not fully implemented the key practices needed to actually control investments—either at the project level or at the portfolio level. We recommended that DHS fully define the project-level and portfolio-level policies and procedures defined in GAO’s ITIM frameworkimplement the practices needed to effectively control investments. DHS agreed with our findings and recommendations, and took action to address the majority of them, including augmenting the resources for providing project oversight.

In November 2008, we reported that DHS had not provided the oversight needed to identify and address cost, schedule, and performance problems for its major acquisitions. Specifically, we reported that of the 48 major investments reviewed that required milestone or annual reviews, 45 were not reviewed in accordance with the department’s investment review policy, and 18 were not reviewed at all. Four of these investments had transitioned into a late acquisition phase—production and deployment—without any required reviews. We recommended—and DHS concurred—that the department identify and align sufficient management resources to implement oversight reviews in a timely manner throughout the investment life cycle.

In 2010, DHS’s Office of the CIO conducted a series of reviews of its major IT programs. According to DHS’s draft Portfolio Governance Concept of Operations, these reviews, plus additional insights from the CIO, highlighted several problems with the existing IT investment management process, including the following:

Governance was based on a program-by-program approach that did not reflect the reality that many programs are inter-related in delivering mission outcomes.

Programs took too long, were often over cost, and did not meet performance objectives.

Decision makers were often far removed from program details; many decisions were reserved for the IRB, which lacks the capacity and time to be familiar with all mission needs and program issues.

In addition, the department again held portfolio reviews in 2011 and found that while some progress had been made, many of the aforementioned problems continued.

Following the February 2011 update to our high-risk series report on implementing and transforming DHS, in which we reported that while the department had improved its policies and procedures for investment management, more work remained, DHS began providing us with bi- annual updates on its progress in addressing weaknesses, including IT management.

Further, we recently reported that the portfolio reviews DHS conducted in 2011 to avoid investing in systems that were duplicative or overlapping, and to identify and leverage investments across the department contributed to the identification and consolidation of duplicative functionality within four investments. further consolidate systems by 2014, which is expected to produce approximately $41 million in cost savings. The portfolio reviews also contributed to the identification of 38 additional systems that are duplicative.

GAO, Information Technology: Departments of Defense and Energy Need to Address Potentially Duplicative Investments, GAO-12-241 (Washington, D.C.: Feb. 17, 2012).

OMB Has Established Initiatives to Reform Federal IT Investment Management through Portfolio Governance and Accountability

In December 2010, the Office of Management and Budget (OMB) issued its 25 Point Implementation Plan to Reform Federal Information Technology Management, a document outlining activities spanning 18 months to reform IT management throughout the federal government. A key goal of the plan—referred to as the IT Reform Plan—is to foster more effective management of large-scale IT programs. One way the plan recommends this be done is through streamlining governance and improving accountability. According to the plan, this involves reforming and strengthening IRBs to enable them to more adequately manage agency portfolios, redefining the role of agency CIOs and the federal CIO Council to focus on portfolio management, and rolling out “TechStat” reviews at the agency and bureau levels to focus attention on IT investments, including those that are poorly performing. In April 2012, we reported that OMB and key federal agencies had made progress on selected action items identified in the IT Reform Plan, but several areas, including strengthening the role of the CIO, were not yet completed.

OMB, Chief Information Officer Authorities, M-11-29 (Washington, D.C.: August 8, 2011).

DHS’s New IT Governance Process Is Generally Consistent with Best Practices

DHS has defined a vision for its new IT governance process, which includes a three-tiered oversight structure that defines distinct roles and responsibilities from the program through the department. The new governance process is generally consistent with recent OMB guidance and with best practices for managing projects and portfolios, with two practices partially addressed and seven others fully addressed. However, the supporting policies and procedures have yet to be finalized. According to officials, this is because the focus has been on piloting the process. While it is important to conduct pilots to test processes and identify lessons learned, until DHS finalizes its procedures associated with the new IT governance, it will have less assurance that the governance will be consistent with best practices and address previously identified weaknesses in investment management.

DHS’s Vision for New IT Governance Process is Based on a Tiered Oversight Structure

In December 2011, the Under Secretary for Management’s Integrated Strategy for High Risk Management introduced an initiative to improve and streamline IT program execution governance. The initiative established a tiered governance structure for program execution, with specific decision responsibilities for each tier. This structure is intended to supplement, not replace, DHS’s existing policies and procedures for managing acquisitions. It also includes several of the governance entities represented in the Integrated Investment Lifecycle Model the department is currently defining to integrate the planning, programming, budgeting, and execution processes.

According to the strategy, the governance process is intended to improve IT investment management across DHS by providing enterprise-level governance and oversight based on functional IT portfolios; the integration of enterprisewide processes for strategic planning, program management, budget planning, acquisition, and program execution by establishing a tiered governance structure—enterprise, portfolio, and program governance; program health by continuing initiatives to enable IT programs to manage to budget and schedule, mitigate risks, and deliver desired functionality; and IT investment performance reporting across the department by leveraging a business intelligence tool—known as the Decision Support Tool—that will provide standardization of data and consistent monitoring of IT portfolios and programs.

To meet these goals, DHS has defined a scalable, tiered structure among the governance entities that establishes distinct roles and responsibilities from the program through the department. (This structure is illustrated in figure 3.)

Enterprise level governance—Enterprise governance will focus on setting strategic priorities and requirements to meet the enterprise mission.

Portfolio level governance—Portfolio governance will manage groups of related programs to ensure that programs are properly aligned and function to meet DHS mission needs.

Program level governance—Program governance is the primary decision-making authority for individual programs.

Table 1 lists the entities involved in the tiered governance process, and provides a description of their membership. While some of the entities are new, others already existed.

In addition to the tiered structure intended to improve oversight of programs and portfolios, the IT governance vision also more broadly addresses the process for determining what investments should be selected to address mission gaps (during the planning phase) and ultimately, deliver capabilities (during the execution phase). The following describes these two phases:

During the planning phase, DHS expects to consider IT investments from a departmentwide portfolio perspective, and determine which investments should be funded. To do this, DHS envisions the use of functional portfolios. The portfolios are to align to the 13 functional segments of the department’s enterprise architecture and will be governed by the Portfolio Governance Boards. The Department Strategy Council is to provide these portfolios with strategic guidance. Using input from the portfolio managers, the Capabilities and Requirements Council is to adjudicate issues across portfolios and set overall department priorities. In addition, the Portfolio Governance Boards will provide guidance and investment recommendations to the Program Review Board for future year planning, programming, and budgeting. The Program Review Board will then formulate the budget and communicate with OMB and the IRB.

The execution phase is intended to ensure the investments continue to be aligned with the department’s strategic goals and fully support mission performance, and ultimately provide the intended capabilities. DHS envisions an interaction among the entities that provide oversight, control, and assistance during program execution—the IRB, the ESCs, and COEs. The ESCs are to oversee major programs or groups of closely related programs. However, if a major program runs into difficultly in its schedule, budget, or scope performance, the program manager would be required to present the status and action plan to the IRB for assessment and approval. This requirement is in addition to the ongoing governance provided by the ESCs. The COEs would provide expert support in core program management disciplines, such as requirements engineering. The department also plans to standardize reviews, including the TechStats and program health assessments. The TechStat reviews are intended to evaluate programs, identify any issues, develop a corrective action plan, and ensure that the program receives proper support to fix the issues. The program health assessments look at individual programs’ health in executing and carrying out the program. In fiscal year 2011, DHS performed reviews of 47 IT programs, primarily focusing on those programs with poor performance ratings or deemed highly critical or visible by the CIO. Under the new governance, the Office of the CIO plans to conduct reviews of all 83 major IT investments anywhere from monthly to semi-annually, depending on the health of the project.

According to officials, while the Under Secretary for Management has not yet approved the new IT governance model, DHS has started implementing it. Figure 4 shows the recommended process for determining what investments should go forward, providing oversight, and delivering capabilities.

In addition, DHS plans to use its Decision Support Tool to provide updated program information to all governance bodies on the status of major programs. The Decision Support Tool is to provide the ability to integrate data resident within multiple existing source systems including the Next Generation Periodic Reporting System and the Investment Management System, to generate reports, charts and graphs on program performance.

DHS’s New IT Governance Framework Is Generally Consistent with Guidance, Best Practices, but Policies and Procedures Have Not Yet Been Finalized

DHS’s new IT governance framework is generally consistent with recent OMB guidance. Specifically, consistent with OMB guidance calling for the CIO to play a significant role in the oversight of the portfolio of IT programs, DHS’s draft procedures note that ESCs overseeing IT programs must include the DHS CIO, a component CIO, or a designated executive representative from a CIO office. In addition, for programs that the DHS CIO Council designates as most critical, the DHS CIO or an appropriate component CIO will co-chair the ESC. Further, consistent with recent OMB guidance to focus on portfolio management, the new governance framework includes the establishment of portfolio governance boards to oversee functional portfolios with the goals of eliminating duplication and leveraging services and programs across the department.

In addition, DHS’s new IT governance framework and the associated policies and procedures are generally consistent with best practices for managing projects and portfolios identified in GAO’s ITIM framework, with two practices partially addressed and seven others fully addressed; however, the procedures have not been finalized. Tables 2 and 3 summarize our assessment of DHS’s draft procedures against relevant practices of our ITIM.

According to officials, the policies and procedures supporting DHS’s new IT governance have not been finalized because the focus has been on piloting the new governance process. To the department’s credit, according to officials, the Portfolio Governance Concept of Operations, which focuses more on the role of the Portfolio Governance Boards, is expected to be approved by the end of July. In addition, the Program Governance Concept of Operations, which is intended to specify how programs are to be overseen, is currently being drafted and expected to be finalized in August 2012 (and approved by the CIO later). Further, according to officials, resources have recently been assigned to updating DHS’s current Information Technology Integration and Management directive to reflect the new framework, and work has also begun to update the Acquisition Management Directive. While the use of pilots is valuable in testing processes and identifying lessons learned, until DHS finalizes the policies and procedures associated with the new IT governance, the department will have less assurance that its new IT governance will be consistent with best practices and address previously identified weaknesses in investment management.

DHS Has Implemented Aspects of Its New Structure, but Has Not Fully Followed Best Practices

DHS has begun to implement components of the IT governance framework. Specifically, the department has primarily focused its implementation efforts on the “execution” phase of the IT governance operating model (see fig. 4) because, according to officials, it includes the department’s more mature processes. Efforts to implement other aspects of the model have been more limited.

Table 4 below shows the status of the implementation of the governance bodies.

In addition to progress made with the governance entities above, DHS has also taken other steps to implement the new IT governance structure.

Portfolio reviews: According to the CIO, the department is currently performing portfolio reviews in collaboration with PARM and the Office of the Chief Financial Officer. These reviews look at the alignment of a functional grouping of investments and the mission effectiveness, or value, those investments deliver. The focus of these reviews is to identify overlaps and redundancies in existing investments, and to identify gaps in existing capabilities. According to the Deputy CIO, these portfolio reviews represent a new way of operating for the components, as they were not used to looking at functions across the department, only portfolios within their respective components. According to the CIO, as the Portfolio Governance Boards are established, they are expected to take over the role of reviewing these portfolios.

Program reviews: In fiscal year 2011, the CIO’s office performed program health assessment reviews of 47 IT programs, primarily focusing on those programs with poor performance ratings or determined to be highly critical or visible by the CIO. According to officials, all 83 major IT investments have been reviewed for this fiscal year.

TechStat reviews: According to officials, the Office of the CIO also performed two TechStat reviews in fiscal year 2011, and three of the four TechStats planned for fiscal year 2012. In addition, officials stated they have completed TechStat training for 12 components.

DHS Is Taking Steps to Obtain Organizational Buy- in

According to industry best practices, in order to effectively implement a new IT governance framework, or any large organizational change, organizations should obtain buy-in by involving all key stakeholders to ensure key perspectives are considered and facilitate adoption. This includes having top management support and creating forums for involving business representatives.

DHS has top management support, as evidenced by the Under Secretary for Management approving key documents supporting the new governance process. For example, the December 2011 Integrated Strategy for High Risk Management, which introduces the vision for the IT governance structure, was approved by the Under Secretary for Management. The Under Secretary also issued a memorandum calling for the use of pilot ESCs and providing these ESCs authority to oversee investment performance. In addition, to secure buy-in, EBMO also sought comments on a draft of its Portfolio Governance Concept of Operations, and, according to officials communicated its efforts to improve IT investment management to senior executives (including the Chief Financial Officer and Chief Procurement Officer) and to the components. According to the EBMO Director, these stakeholders’ participation on ESCs or portfolio review boards and involvement in conducting the portfolio reviews has also helped to secure their buy-in. Taking these steps increases the likelihood that the new IT governance process will be adopted despite the significant cultural change it represents. The CIO and Executive Director for EBMO both noted that the new governance is gradually gaining acceptance.

An Implementation Team Has Been Established, but the Department Lacks an Implementation Plan

Effectively implementing a new IT governance process also requires developing an effective implementation team and plan. According to best practices, an effective implementation team should be put in place that includes key stakeholders from both business and IT components. In addition, we have previously reported that to effectively implement IT investment management processes, organizations need to be guided by a plan that builds on existing strengths and weaknesses; specifies measurable goals, objectives, and milestones; specifies needed resources; assigns clear responsibility and accountability for accomplishing tasks; and is approved by senior-level management.

Such a plan is instrumental in helping agencies coordinate and guide improvement efforts.

DHS has established an integrated product team to guide the implementation of the new IT governance structure. According to DHS officials, the department’s integrated product team is made up of officials from EBMO, PARM, and the department’s chief executives (e.g., CIO, Chief Financial Officer, etc.). Further, according to the department, the EBMO senior staff directly involved with the implementation of the governance structure have both program-level and portfolio-level governance experience. According to officials, the integrated product team is supported by subject matter experts and the Component Acquisition Executives.

However, the department has not yet developed an implementation plan. While DHS’s June 2012 Integrated Strategy for High Risk Management includes components of an implementation plan, such as high-level goals and activities to be completed by the end of fiscal year 2012, DHS does not have an implementation plan addressing the elements mentioned above. While officials recognized the value of an implementation plan, they stated they had not yet developed one because they are still piloting the new governance process. Further, officials stated that they planned on developing a 2-year implementation plan that would incorporate best practices identified in the ITIM framework for effectively managing projects and portfolios. They stated that this plan would draw from the department’s revised IT strategic plan due to be completed by the end of the summer of 2012. To mitigate for the limited resources available, DHS states it will adopt a federated model for resources—drawing resources from components—and use contractor support, as needed. Until a plan is developed, the department may not be able to effectively implement the new IT governance process to ensure that it addresses previously identified weaknesses, and effectively uses the department’s limited resources.

DHS Has Yet to Fully Define Processes for Evaluating Its Implementation Efforts

According to best practices, when implementing an IT governance framework, it is important to evaluate the implementation efforts by, among other things, developing measures to assess progress in meeting objectives. In addition, according to best practices, when implementing a new governance framework, there should be mechanisms in place to document lessons learned for subsequent governance improvement initiatives. According to our ITIM framework, among other things, lessons learned and recommendations for improving the investment process should be developed and documented, and then distributed to all stakeholders.

DHS has defined measures in the June 2012 update to the Integrated Strategy for High Risk Management. For example, DHS plans to measure the (1) percentage of DHS IT program reviews completed in fiscal year 2012; (2) percentage of DHS IT programs rated as low risk; (3) percentage of IT investment portfolio governance boards and ESCs established and chartered; and (4) percentage of components trained and conducting component-led TechStats. In addition, the department has identified measures to assess the effectiveness of the ESC program reviews. Further, according to officials, draft measures for the COE functions have also been developed; however, they have not yet been documented and officials did not have any time frames for doing so. Without fully defining and documenting measures of success, it is unlikely that the department will be able to determine if it has successfully and efficiently established its new IT governance framework.

DHS officials stated that they are using lessons learned from their piloting activities to improve their governance process; however, they have not established a mechanism for capturing lessons learned. Without this mechanism, the department risks not being able to continue to build on the experiences and lessons learned from prior initiatives, which could help to identify, introduce, and sustain additional efficiency gains on a more systematic basis.

Conclusions

DHS’s new approach to IT governance shows promise in establishing mechanisms for greater oversight, involving key staff in program decisions, and focusing on functional portfolios in order to avoid unnecessary duplication. To the department’s credit, the vision is generally consistent with guidance and with best practices for managing projects and portfolios. However, we identified two practices which were only partially addressed in policies and procedures—ensuring that the IRB maintains visibility into the activities of the ESCs, and defining project selection and prioritization criteria—which could limit the extent to which these practices are institutionalized. In addition, the policies and procedures have not yet been finalized. To its credit, DHS has efforts underway to address this. However, until DHS finalizes these policies and procedures and ensures that they fully address practices for managing programs and portfolios of investments, the department will have less assurance that its new approach will address identified weaknesses and facilitate effective governance.

DHS has implemented aspects of its new approach, such as establishing governance entities associated with the execution phase of its approach and conducting portfolio and program reviews. However, because the agency has not developed an implementation plan, fully documented performance measures, or established a mechanism for capturing lessons learned, there is a risk that the approach may be less effective than intended.

Recommendations for Executive Action

To implement an effective IT governance strategy, we recommend that the Secretary of Homeland Security direct the appropriate officials to finish defining the new IT governance process by finalizing the IT governance policies and procedures and ensuring they fully address or reference existing documents that address the following: how the IRB is to maintain responsibility for lower-level board activities; and investment selection and prioritization criteria.

In addition, to assist in implementing the new IT governance strategy, we recommend that the Secretary of Homeland Security direct the appropriate officials to develop an implementation plan that draws together ongoing and additional efforts needed to implement the new IT governance process. The plan should: 1. build on existing strengths and weaknesses; 2. specify measurable goals, objectives, and milestones; 3. specify needed resources; 4. assign clear responsibility and accountability for accomplishing 5. be approved by senior-level management. fully define and document key measures to monitor the implementation process; and establish mechanisms for capturing lessons learned.

Agency Comments and Our Evaluation

DHS’s Director for the Departmental GAO-OIG Liaison Office provided written comments on a draft of this report (reprinted in appendix II). He stated that the department was pleased to note GAO’s positive acknowledgement that DHS’s IT governance framework was consistent with OMB guidance and GAO’s ITIM framework. He also stated that the department concurred with the recommendations and estimated it would address them by September 30, 2013. The department also provided technical comments, which we have incorporated where appropriate.

We are sending copies of this report to interested congressional committees and the Secretary of Homeland Security. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to (1) describe the Department of Homeland Security’s (DHS) new information technology (IT) governance process and associated policies and procedures, and assess them against best practices; and (2) determine progress made in implementing the new approach and how DHS’s implementation efforts comport with relevant best practices.

To address our first objective, we reviewed and analyzed documentation on DHS’s newly initiated IT governance process. This documentation included the Under Secretary for Management’s Program Management & Execution Playbook, the December 2011 biannual update to the Implementation and Transformation section of DHS’s Integrated Strategy for High Risk Management, the Office of the Chief Information Officer’s Concept of Operations documents for Program and Portfolio Governance, and Chief Information Officer quarterly update briefings presented to GAO. We also reviewed various memorandums from the Under Secretary for Management, which covered such issues as the chartering of Executive Steering Committees and the implementation of the department’s Decision Support Tool. We also interviewed officials from DHS’s Office of the Chief Information Officer’s Enterprise Business Management Office (EBMO), and from the Office of Program Accountability and Risk Management (PARM). To assess DHS’s vision of its planned governance process against GAO’s Information Technology Investment Management (ITIM) guide, we identified the ITIM stage 2 and stage 3 critical process areas that were most relevant to DHS’s efforts, and compared the evidence collected from our document reviews and interviews to the practices associated with these areas.

To address our second objective, we obtained and evaluated documentation showing the department’s efforts in implementing the governance process. This included documentation defining the roles of various governance entities, such as the Executive Steering Committees and Portfolio Governance Boards, to be involved in the process. We reviewed available charters and meeting minutes for several of these entities to determine if they were functioning. We also reviewed documentation on the Centers of Excellence established by DHS to provide subject matter expertise to support program management. In addition, we interviewed officials from EBMO and PARM to determine the status of the department’s effort’s in implementing the governance process. We also requested information from the five programs for which Executive Steering Committees were first established on the various reviews they had undergone since January 2010—including type of review, time frames for each review, and information requested—to determine the extent of duplication or overlap among the reviews. In order to assess DHS’s implementation of the IT governance approach against accepted best practices, we first used content-analysis software to identify best practices shared by industry and the federal government. We identified these practices from prior our reports, Office of Management and Budget guidance, and guidance from recognized experts in IT governance. We grouped the practices into three categories—(1) organizational buy-in, (2) development of an implementation team and implementation plan, and (3) evaluation— and determined from the evidence collected from our document reviews and interviews the extent to which DHS was following these best practices.

We conducted this performance audit from October 2011 to July 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contacts and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, the following staff also made key contributions to this report: Sabine R. Paul, Assistant Director; William G. Barrick; Sairah R. Ijaz; Lee A. McCracken; and Tomas Ramirez.