Information Technology: Federal Agencies Need to Strengthen Investment Board Oversight of Poorly Planned and Performing Projects
Released: 2009-07-30T00:00:00

Letter

Background

OMB plays a key role in helping federal agencies manage their IT investments by working with them to better plan, justify, and determine how much they need to spend on IT projects and how to manage approved projects. In particular, the Clinger-Cohen Act of 1996 requires OMB to establish processes to analyze, track, and evaluate the risks and results of major capital investments in information systems made by federal agencies and report to Congress on the net program performance benefits achieved as a result of these investments. In addition, the Clinger-Cohen Act places responsibility for managing IT investments with the heads of agencies and establishes chief information officers to advise and assist agency heads in carrying out this responsibility.

To help carry out its oversight role and assist the agencies in carrying out their responsibilities, OMB developed its Management Watch List in 2003 and its High-Risk List in 2005 to focus executive attention and to ensure better planning and tracking of the major IT investments. The Management Watch List identifies projects at federal agencies that are poorly planned, i.e., projects with weaknesses in their funding justifications, which are known as exhibit 300s. Because of the focus on the funding justifications, projects on the Management Watch List specifically concern the process by which agencies select projects to invest in. OMB places projects on the High-Risk List when they require special attention from oversight authorities and the highest level of agency management. These projects are not necessarily “at risk” of failure, but may be on the list because of one or more of the following four reasons: The agency has not consistently demonstrated the ability to manage complex projects.

The project has exceptionally high development, operating, or maintenance costs, either in absolute terms or as a percentage of the agency’s total IT portfolio.

The project is being undertaken to correct recognized deficiencies in the adequate performance of an essential mission program or function of the agency, a component of the agency, or another organization.

Delay or failure of the project would introduce for the first time unacceptable or inadequate performance or failure of an essential mission function of the agency, a component of the agency, or another organization.

The High-Risk List also includes projects that are performing poorly (i.e., high-risk projects with reported performance shortfalls). High-risk projects are identified as having performance shortfalls if one or more of the following performance evaluation criteria are not met: (1) establishing baselines with clear cost, schedule, and performance goals; (2) maintaining the project’s cost and schedule variances within 10 percent; (3) assigning a qualified project manager; and (4) avoiding duplication by leveraging inter-agency and governmentwide investments. Projects on the High-Risk List, therefore, require disciplined and effective oversight to ensure that performance shortfalls, if any, are addressed.

GAO, Information Technology: OMB and Agencies Need to Improve Planning, Management, and Oversight of Projects Totaling Billions of Dollars, GAO-08-1051T (Washington, D.C.: July 31, 2008). projects (totaling about $3 billion) were considered both poorly planned and poorly performing.

OMB took several steps to address our recommendations to improve the identification and oversight of Management Watch List and High-Risk List projects; however, further action is needed, including, for example, identifying the deficiencies (i.e., performance shortfalls) associated with the high-risk projects. On April 28, 2009, we testified that the future of the Management Watch List and High-Risk List was uncertain because OMB officials stated that they had not decided if the agency plans to continue to use these lists. We noted that OMB needs to decide if it is going to continue to use the Management Watch List and High-Risk List and, if not, that OMB should promptly implement other appropriate mechanisms to help direct and oversee IT investments in the future. In response, the Federal Chief Information Officer testified that OMB would determine how to better oversee poorly planned and performing projects by the end of June 2009.

Investment Management Framework Calls for Boards to Select and Oversee IT Investments

Federal agencies face significant challenges in planning for and managing their IT systems and networks. These challenges can be addressed, in part, by the use of systematic management processes to select, control, and evaluate the investments. To further support the implementation of such processes, we developed an IT investment management (ITIM) framework for agencies to use. It is based on our research of IT investment management practices of leading private and public sector organizations and can be used to determine both the status of an agency’s current IT investment management capabilities and the additional steps that are needed to establish more effective processes. The framework consists of progressive stages of maturity for any given organization relative to its selection and oversight responsibilities. We have used the framework in many of our reports, and a number of agencies have adopted it.

The ITIM maturity framework cites the establishment of “one or more IT investment management boards” as a fundamental step in establishing a mature capital planning process. The framework states that a departmentwide IT investment review board (IRB) composed of senior executives from both IT and business units should be responsible for defining and implementing the department’s IT investment governance process. This department-level IRB is to provide selection and oversight of department IT projects to ensure that the department’s portfolio of projects meets mission needs at expected levels of cost and risk. Selecting projects involves identifying and analyzing projects’ risks and returns before committing any significant funds to them and selecting those that will best support the agency’s mission needs; overseeing projects involves reviewing the progress of projects against expectations and taking corrective action when these expectations are not being met.

To ensure that agencies’ department-level boards are using a disciplined selection and oversight process, the ITIM framework also states that, among other things, the department-level board should: select new investments and reselect ongoing investments; perform regular reviews of each project’s performance against stated expectations; and receive data associated with a project’s actual performance (including cost, schedule, benefit, and risk performance). Importantly, according to the ITIM framework, while these functions can be performed by subordinate boards, the department-level IRBs must maintain ultimate responsibility for and visibility into the subordinate boards’ activities.

Prior Reviews Have Identified Weaknesses in Executive-Level Board Involvement in Selection and Oversight

We have previously reported that federal agencies face challenges in effectively managing their IT investments. Specifically, in January 2004, we reported that, although most of the major agencies in our review had IRBs responsible for defining and implementing their investment management processes, the agencies did not always have the mechanisms in place for these boards to effectively control their investments. We made recommendations to the agencies regarding those practices that were not fully in place. More recently, in 2008, we reported that the Social Security Administration had not fully developed policies and procedures for management oversight of its IT projects and systems, such as elevating problems to the department-level IRB. We also reported that the Social Security Administration had not tracked corrective actions for underperforming investments and had not reported the actions to the department-level IRB. To address these weaknesses, we recommended that the agency strengthen and expand the board’s oversight responsibilities for underperforming projects and evaluations of projects and establish a mechanism for tracking corrective actions for underperforming investments.

Major Federal Agencies Have Guidance for Selection and Oversight of IT Investments, but Two Agency Boards Lack Business Unit Representation

The 24 major federal agencies have guidance calling for department-level IRBs to select and oversee IT investments pursuant to OMB guidance required by the Clinger-Cohen Act, and specified in practices laid out in the ITIM framework. However, while all of the agencies had department- level IRBs, the board membership for two agencies did not include business unit (i.e., mission) representation.

Agency Guidance Calls for Department-Level IRBs to Select Projects

Each of the agencies had documented guidance that called for a department-level IRB to perform selection of the projects to be included in the agency’s IT investments. For example, according to the Department of the Treasury’s guidance, its department-level IRB is to consider investment scoring results and recommendations that are provided to it by the Chief Information Officer Council (a subordinate board) and select which investments will be included in Treasury’s IT investment portfolio. The Department of Transportation’s recently issued IT investment management policy delegates responsibility for project selection, as well as project oversight, to its component-level investment review boards, but requires its components to establish and/or document the existence of their boards, specifies the roles and responsibilities these boards are to have, and establishes specific metrics to be used by the department-level IRB to measure the performance of the component boards.

Agency Guidance Calls for Department-Level IRBs to Oversee Projects

As with project selection, each of the agencies had documented guidance that called for the department-level IRB to conduct an oversight reviews of projects, and the frequency of these reviews varied (see fig. 1 for a breakdown of the frequency of oversight reviews specified in agencies’ guidance).

For 20 of the 24 agencies, the guidance allowed the delegation of oversight reviews to other entities. In these cases, the agencies had guidance in place to help ensure that these other entities were effectively carrying out their responsibilities. At the remaining four agencies—the National Science Foundation, Small Business Administration, Department of State, and the U.S. Agency for International Development —project oversight was to be primarily performed by the department-level IRB. By having guidance specifying department-level IRB selection and oversight of projects, agencies recognize the importance of involving those who have the ultimate responsibility and accountability for the organization’s success in key project decisions.

Two Agencies’ Department- Level Boards Lack Business Unit Representation

It should be noted, however, that while all of the agencies had guidance requiring department-level IRBs to be responsible for selecting and overseeing projects, the boards at the Departments of Commerce and Labor did not include senior executives from business units (e.g., line or mission units) as called for in the ITIM framework. Specifically, these boards consisted of executives from IT and other department mission support units, such as the Chief Financial Officer, Director of Budget, or Controller, as well as administrative officers, but did not have appropriate line or mission representation from the organizations’ business units. We have previously reported that because allocating resources among major IT investments may require fundamental trade-offs among a multitude of business objectives, portfolio management decisions are essentially business decisions and therefore require sufficient business representation on the department-level IRB.

The two agencies with boards that did not include senior executives from business units offered the following rationales for this practice.

The Department of Commerce reported that it does not include nontechnical program representatives on its department-level IRB because it would be impractical to have fair representation of all 12 of the major agencies and the dozens of major programs comprising the department. In addition, Commerce reported that it is run on a federated basis, putting responsibility on each of the department’s operating units to prioritize its own investments in determining which should be reviewed by the department. Finally, Commerce stated that it does not prioritize among investments from its different operating units; instead, departmental officials work with each operating unit to ensure that the investment and investment strategy being recommended is optimum for meeting that operating unit’s mission. We have previously reported that using this approach of giving responsibility to subordinate units should include appropriate department-level involvement, either through review and approval of their investments that meet certain criteria or through awareness of the subordinate unit’s investment management activities. We believe that this corporate visibility should be provided by a board composed of executives from both business and IT units to ensure that decisions made are in the best interest of the entire department. In addition, while Commerce’s practice may not be to prioritize among the investments at the department level, the department has ultimate responsibility for the success of its operating units’ investments and the department-level IRB should therefore include business representation to ensure that decisions made are in the best interest of the agency.

The Department of Labor reported that the senior IT and administrative executives who serve on its department-level IRB, have in-depth, detailed, and expert knowledge of their units’ missions and business objectives and are capable of representing their units’ interests. However, we have previously reported that IT and administrative executives responsible for mission support functions do not constitute sufficient business representation because, by virtue of their responsibilities, they are not in the best position to make business decisions.

Until these agencies adjust their board memberships to include representation from their business units, they will not have assurance that the department-level IRB includes those executives who are in the best position to make the full range of decisions needed to enable the agency to carry out its mission most effectively.

Many Projects Did Not Receive a Department-Level IRB Selection or Oversight Review

Although all the major agencies had guidance calling for a department- level IRB selection or oversight review, many of the projects we examined did not receive one of these reviews. Specifically, 12 of the 24 projects identified by OMB as being poorly planned in 2007 (accounting for about $4.9 billion) did not receive a selection review, and 13 of 28 poorly performing projects in 2007 (amounting to about $4.4 billion) did not receive an oversight review by the department-level IRB. Furthermore, 6 of the 11 projects identified as being both poorly planned and poorly performing, with nearly $3.7 billion in funding in the President’s fiscal year 2008 budget request, received neither a selection review nor an oversight review.

Half of the Poorly Planned Projects Did Not Receive a Selection Review by a Department-Level IRB

Of the 24 poorly planned projects in 2007 that we reviewed, 12 projects did not receive a selection review, while 12 were reviewed by the department- level IRB. The requested funding level for these 24 poorly planned projects was about $7.3 billion. The 12 projects that were reviewed by a department-level IRB accounted for approximately $2.4 billion, while the 12 projects not reviewed accounted for about $4.9 billion, about two thirds out two thirds of the total requested funding for the 24 projects (see fig. 2 and table 1). of the total requested funding for the 24 projects (see fig. 2 and table 1).

We assessed five projects as not having received department-level IRB selection reviews because the agencies did not provide evidence of such reviews. Agencies offered varying reasons for why selection reviews had not been performed for the remaining seven. Table 1 shows whether projects we reviewed received a selection review from the department- level IRB and lists reported reasons why no review was performed, where applicable.

Following are details on the reasons why the 12 projects did not receive a department-level IRB review: A project belonging to Commerce’s USPTO was not reviewed by the department-level IRB, according to the agency, because the USPTO is a performance-based organization (PBO), and therefore its projects are not required to be reviewed by the department-level IRB. According to the legislation that established the USPTO as a PBO, the office is subject to the policy direction of the Secretary of Commerce, but it otherwise retains responsibility for decisions regarding the management and administration of its operations and exercises independent control of its budget allocations and expenditures, personnel decisions and processes, procurements, and other administrative and management functions.

According to the Department of Education, the Common Services for Borrowers project did not receive a selection review by the department- level board because it is under the oversight of the Federal Student Aid Executive Leadership Team. In written comments on a draft of this report, however, the department stated that it plans to bring all of its IT investments under the department-level board’s oversight.

The Department of Homeland Security did not provide evidence of a selection review for its two projects but noted that it was reengineering its investment management process to include department-level IRB reviews of projects at key milestone decision points.

Although NASA stated that its three projects were governed by oversight bodies, the documentation provided did not show evidence that reviews had been performed by the appropriate department-level review board.

At the Nuclear Regulatory Commission, a lower-level board performed the selection reviews. According to the agency’s guidance, the department- level board should have performed the reviews. It stated that this board only gets involved when the lower-level board believes issues need to be elevated. However, NRC’s guidance does not specify when issues need to be elevated to the department-level IRB. In addition, the agency did not provide any examples of cases when issues had been elevated to the department-level IRB.

Officials from the Department of Transportation’s Office of the Chief Information Officer could not provide a reason why a department-level board selection review of its projects had not been performed. In commenting on a draft of this report, the agency stated that it planned to have this project reviewed in detail by its departmental-level board.

The Department of the Treasury’s projects did not receive a department- level IRB selection review because this board was not active during the time frame we considered during our review. The department, however, has since then reestablished its department-level IRB.

About Half of the Poorly Performing Projects Did Not Receive an Oversight Review by the Department- Level IRB

About half of the poorly performing projects in 2007 we reviewed did not receive an oversight review by a department-level IRB. Of the 28 projects, 13 did not receive an oversight review by the department-level IRB, while 15 did. The President’s requested fiscal year 2008 funding for the 28 projects totaled approximately $4.7 billion. The 15 projects that received a review represented approximately $0.3 billion, or 7 percent of the total $4.7 billion funding request, while the 13 poorly performing projects that were not reviewed totaled nearly $4.4 billion, or 93 percent of the total requested funding. (See fig. 3 and table 2.)

Table 2 shows whether projects received oversight reviews, as well as reported reasons why no review was performed, where applicable.

Agencies provided several reasons why the 13 projects did not receive oversight reviews, including some which were not consistent with sound management practices: One Defense project’s funding was below the financial threshold required for a review by the department-level IRB, consistent with the agency’s guidance. However, in May 2007 and May 2009, we reported that DOD’s guidance and practices did not provide for sufficient oversight and visibility into component-level investment management activities, including component reviews of investments such as this project. We made recommendations to DOD to address these weaknesses, which DOD has yet to fully implement.

Another Defense project was reportedly being rebaselined (meaning that its cost, schedule, and performance goals were being modified to reflect a change in the scope of the work) and therefore had not received a review by the department-level IRB. This project, however, continues to be funded and therefore could have benefited from a department-level oversight review.

According to the Department of Education, the two projects we reviewed did not receive oversight reviews by the department-level IRB because they were under the oversight of the Federal Student Aid Executive Leadership Team. As noted earlier, in written comments on a draft of this report, the department stated it plans to bring all of its IT investments under the department-level board’s oversight.

While DHS provided evidence that a lower-level board had agreed to submit the DHS-Infrastructure project to the department-level IRB for review, the agency did not provide evidence that this review had been performed. The department also stated that SBInet and US-VISIT projects had received an oversight review by the department-level IRB, but did not provide sufficient evidence to support this, including information presented to the board for review. In March 2009, however, DHS officials told us that they had recently made changes to their investment review process and, as part of these changes, were planning to improve the documentation associated with department-level IRB reviews.

A Nuclear Regulatory Commission project should have received a review by the department-level IRB according to the agency’s guidance, but officials told us that, in practice, this board only gets involved when the lower-level board elevates issues. However, agency officials were unable to provide us with any examples where the lower-level board had elevated issues about the project to the IRB.

The Department of the Treasury’s projects did not receive a department- level IRB oversight review because this board was not active during the time frame we considered during our review. The department, however, has since then reestablished its department-level IRB.

According to the U.S. Agency for International Development, its project did not receive an oversight review because it has not been able to proceed due to lack of funding. We agree that an oversight review was not warranted since there was no activity on the project.

A Veterans Affairs project was not reviewed because the IRB is not required to review projects in the operations and maintenance stage. Instead, oversight of projects in this stage is the responsibility of the Office of the Chief Information Officer. However, the IRB does not oversee this office’s review activities. According to the ITIM framework, boards should ensure projects are reviewed throughout their life cycle. In addition, they must maintain ultimate responsibility for and visibility into the activities of groups that carry out their functions.

About Half of the Projects That Were Both Poorly Planned and Poorly Performing Received Neither a Selection Review Nor an Oversight Review

Six of the 11 projects that were identified as being both poorly planned and poorly performing in 2007 did not receive a selection or an oversight review by the departmental-level IRB. Funding requests for fiscal year 2008 for these 6 projects accounted for about $3.7 billion (see table 3).

Without consistent involvement of department-level IRBs in selecting and overseeing projects that have been identified as poorly planned or poorly performing, agencies incur the risk that these projects will not improve, which could lead to potentially billions of federal taxpayer dollars being wasted.

Conclusions

Department-level investment review boards’ involvement in selecting and overseeing their agencies’ IT projects is critical to ensuring that these projects meet mission needs and that federal funds are not wasted. To their credit, the 24 major federal agencies have established guidance calling for department-level boards to perform project selection and oversight reviews. However, department-level boards for two agencies did not include representation from their business units and therefore did not have assurance that the board included all of the executives who are in the best position to make the full range of decisions needed to enable the agency to carry out its mission most effectively.

While having selection and oversight guidance is a good step, it is only worthwhile if effectively implemented. The fact that many poorly-planned or performing projects were not reviewed by department-level boards is particularly alarming considering that they represent, in total, about $6 billion in funding and that the Management Watch List and High-Risk List were established specifically to draw management attention to such projects. Until agencies ensure that their department-level review boards are consistently involved in selecting and overseeing these projects, they will continue to incur the risk that the projects will not improve and that potentially billions of federal taxpayer dollars will be wasted.

Recommendations for Executive Action

To ensure that IT projects are effectively managed, we are making recommendations to the agencies whose practices were not consistent with sound management practices. Specifically, we recommend that the Secretaries of Commerce and Labor ensure their department-level review boards include business unit (i.e., mission) representation; the Chairman of the Nuclear Regulatory Commission direct the Executive Director for Operations to define conditions for elevating issues related to project selection and oversight to its department-level IRB; and the Secretary of Veterans Affairs define and implement responsibilities for the department-level IRB to oversee projects in operations and maintenance.

In addition, we are recommending that the Secretaries of the Departments of Defense, Education, Homeland Security, Transportation, Treasury, and Veterans Affairs, the Administrator for the National Aeronautics and Space Administration, the Chairman of the Nuclear Regulatory Commission, and the Administrator for the U.S. Agency for International Development ensure that the projects that are identified in this report as not having received departmental-IRB selection or oversight reviews receive these reviews.

Agency Comments and Our Evaluation

We sent a draft of this report to the 24 major agencies and received a response from 20. Of these 20, 15 provided comments, and 5 stated they did not have any comments (we had not made any recommendations to these agencies, which were the Department of Health and Human Services, the Department of State, the Environmental Protection Agency, the National Science Foundation, and the Office of Personnel Management). Of the 15 agencies that provided comments, 11 generally agreed with our recommendations, and 1 (the Department of Justice) did not. Three agencies (the Department of Housing and Urban Development, the Department of the Interior, and the Social Security Administration) provided views on various aspects of our report. Several agencies also provided technical comments, which we incorporated as appropriate.

The agencies’ comments and our response are summarized below: In written comments on a draft of the report, the Department of Commerce’s Chief Information Officer, addressing our recommendation that the department ensure that its department-level review board include business unit (i.e. mission) representation, stated that the department had modified the membership structure of its investment review board to provide operating unit management with latitude in identifying senior managers most able to provide effective representation and, as a result had broadened its membership to include chief financial officers from certain operating units as well as the Deputy Director of the Bureau of the Census. The Department of Commerce’s comments are printed in appendix II.

In written comments on a draft of the report, the Department of Defense’s Deputy Chief Information Officer concurred with our recommendation to ensure that the Defense Information System for Security receive an oversight review, stating that, going forward, it will ensure that the project receives all required IRB reviews. The department partially concurred with our recommendation to ensure its Integrated Acquisition Environment Shared Services Provider-Past Performance Information Retrieval System receive an oversight review, stating, as indicated in the report, that the project is below the threshold required for department-level IRB oversight. The department stated, however, that the project will be brought before the appropriate department-level IRB for compliance review if, and when it meets the financial threshold. The department also provided technical comments which we have incorporated as appropriate. The Department of Defense’s comments are printed in appendix III.

In written comments on a draft of the report, the Department of Education’s Chief Information Officer, agreed with our recommendation to ensure that the two projects we identified in the report as not having received departmental-level IRB selection or oversight reviews receive such reviews, stating that the IRB will review the investments, render decisions as appropriate, and incorporate the results in the IT portfolio currently under review. The department also noted that, while the projects we reviewed were under the oversight of the Federal Student Aid’s Executive Leadership Team, they would be brought under the department’s oversight along with all other investments. The department disagreed with the statement that the projects reviewed did not receive a selection or oversight review, stating that they had been selected and reviewed by the Federal Student Aid’s Executive Leadership Team. In our report, we have clarified the discussion of these reviews by the Executive Leadership Team where appropriate. The Department of Education’s comments are reprinted in appendix IV.

In written comments on a draft of this report, the Department of Homeland Security’s Director for Departmental GAO/OIG Liaison Office agreed with the recommendation to conduct department-level reviews of the three programs we reviewed and provided evidence of department Acquisition Review Board reviews for these programs during fiscal year 2008. The department disagreed with the assertion that the department- level review boards were not active in overseeing the three projects we examined during our review and provided decision memoranda—three of which we had not been provided before—as evidence of reviews by the boards in place for 2007, the time period we considered. However, in our report, we do not state that the department-level boards were not active. Rather, we note that the department did not provide sufficient evidence of department-level IRB reviews. We did not change our assessments for the three projects because the additional documentation received still did not provide sufficient evidence documenting the 2007 reviews.

The documentation we have seen from more recent reviews more completely documents departmental-level IRB reviews and we have noted this in our report. The department also provided technical comments. The department’s comments are reprinted in appendix V.

In written comments on a draft of this report, the Acting Chief Information Officer of the Department of Housing and Urban Development stated that the department-level IRB will maintain its disciplined process for program executives to participate in selecting and overseeing projects. We did not make any recommendations to the department. The Department of Housing and Urban Development’s comments are reprinted in appendix VI.

In written comments on a draft of this report, the Department of the Interior’s Deputy Assistant Secretary for Budget and Business Management agreed with our conclusions that consistent involvement of department-level review boards in selecting and overseeing projects, particularly poorly performing projects, is important in safeguarding federal taxpayer dollars. The department also asked that the definition of high-risk projects reflect the fact that some investments designated as such are performing within acceptable thresholds but require heightened awareness and oversight by investment review boards because of their importance. To address this comment, we have added OMB’s criteria for designating projects as high-risk to our report background. We did not make any recommendations to the Department of the Interior. The Department of the Interior’s comments are reprinted in appendix VII.

In written comments on a draft of this report, the Department of Justice’s Assistant Attorney General for Administration disagreed with our recommendation that it ensure its department-level review board include business unit representation and provided clarification on the role and responsibilities of the Deputy Attorney General who chairs the board and on the participation of component executives in the board’s decisionmaking process. Based on this clarification, we agree that the board provides adequate business unit representation. We have noted this change in our report and removed the related recommendation. In its comments, the department also took issue with our use of the term “poorly performing” to characterize the projects we reviewed. We are not implying as the department states that these projects are “near failing.” We have clarified our use of the term in the report and, in the case of the Sentinel project—which we have reviewed— acknowledged progress made in managing the project. The Department of Justice’s comments are reprinted in appendix VIII.

In written comments on a draft of this report, the Department of Labor’s Assistant Secretary for Administration and Management addressed our recommendation to ensure that its department-level review board include business unit representation by acknowledging that the board does not include senior executives from business units and stating that, while it believes the executives on the board effectively represented the business interests of their respective organizations, it will consider appropriate and efficient steps for including senior executives from business units as part of the board’s process. The Department of Labor’s comments are reprinted in appendix IX.

In e-mail comments on a draft of this report, the Department of Transportation’s Director of Audit Relations addressed our recommendation to ensure that the projects we identified as not having received department-level IRB selection or oversight reviews receive these reviews by stating that actions are underway to schedule a summer IRB meeting to review the entire budget year 2011 portfolio of IT investments, and that the Combined IT Infrastructure investment which we reviewed is expected to be reviewed in detail.

In written comments on a draft of this report, the Department of the Treasury’s Deputy Assistant Secretary for Information Systems and Chief Information Officer addressed our recommendation to ensure that the projects we identified as not having received department-level IRB selection or oversight reviews receive these reviews by noting recent efforts to reconstitute a department-level Executive Investment Review Board, increase the oversight role of its Chief Information Officer Council, and remediate weaknesses associated with the three projects we reviewed. The Department of the Treasury’s comments are reprinted in appendix X.

In written comments on a draft of this report, the Secretary of the Department of Veterans Affairs concurred with our recommendations to define and implement responsibilities for the department-level IRB to oversee projects in operations and maintenance by noting that the Programming and Long Term Issues Board will include operational programs/projects in its program reviews for fiscal year 2010. The department also concurred with our recommendation to ensure that the project which we identified as not having received department-level IRB oversight reviews receive these reviews and stated that it will address actions to ensure this in its plan to address our recommendations. The Department of Veterans Affairs’ comments are reprinted in appendix XI.

In written comments on a draft of this report, the National Aeronautics and Space Administration’s Associate Deputy Administrator partially concurred with our recommendation that projects which are identified in this report as not having received department-level IRB selection or oversight reviews receive these reviews stating that the departmental board will continue to review major IT investments that are not highly specialized in nature (this includes two of the four projects we reviewed), while another governing body will maintain responsibility for ensuring the overall successful performance of NASA’s program portfolio, including the highly specialized IT investments. We received information about the second governing body after we sent our report to NASA for comment. During the comment period, the agency also provided us additional documentation on the projects we reviewed. After reviewing this documentation, we have changed the reported reason column in table 1 from “department-level board was not active (i.e., it had not yet been established)” to “NASA did not provide evidence that a selection review had been performed by the appropriate department-level IRB” for the three projects we reviewed for selection. In addition, we changed the department-level IRB review column in table 2 for the Integrated Financial Management Improvement program from a “no” to a “yes.” NASA’s comments are reprinted in appendix XII.

In written comments on a draft of this report, the Nuclear Regulatory Commission’s Deputy Executive Director for Corporate Management, Office of the Executive Director for Operations, agreed with our recommendation to define conditions for elevating issues related to project selection and oversight to its department-level IRB stating that the commission will review and enhance the existing guidance for project selection and oversight to ensure that its process is compliant with the intent of the Clinger-Cohen Act. This will include updating the Information Technology Business Council charter for project oversight reviews to include any necessary changes to the process or criteria for review by the Information Technology Senior Advisory Council. The commission also agreed with our recommendation to ensure that the National Source Tracking System which we identified as not having received a selection or oversight review by the department-level IRB receive such review. The Nuclear Regulatory Commission’s comments are reprinted in appendix XIII.

In written comments on a draft of this report, the Commissioner of the Social Security Administration asked that we remove the Information Technology Operations Assurance project we reviewed from our report because it is not a poorly planned or poorly performing project. During the agency comment period, we informed the agency that we would be removing the project from our sample, and, based on clarification provided by the Associate Chief Information Officer that the project reported a positive cost variance, agreed that it should not be considered poorly performing. We did not make any recommendations to the agency. The Social Security Administration’s comments are reprinted in appendix XIV.

In e-mail comments on a draft of this report, the U.S. Agency for International Development concurred with our recommendation to ensure that the project which we identified as not having received a department- level IRB oversight review receive this review. The agency noted, however, that the review might not occur if the project is not funded.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to other interested congressional committees, the Director of the Office of Management and Budget, and other interested parties. The report also will be available at no charge on the GAO Web site at http://www.gao.gov. Should you or your offices have questions on matters discussed in this report, please contact me at (202) 512-9286 or at pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix XV.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine whether (1) federal departments/agencies have guidance on the role of their department-level investment review boards (IRB) in selecting and overseeing information technology (IT) projects and (2) these boards are performing selection and oversight reviews of poorly planned and performing projects.

To address the first objective, we reviewed the investment management guidance (including policy documents and board charters) of each of 24 agencies listed in the Chief Financial Officers (CFO) Act of 1990 (referred to in our report as “the 24 major agencies”). In reviewing the guidance, we determined the role department-level IRBs are expected to play in selecting and overseeing IT projects, updating the findings from our 2004 governmentwide review of agencies’ use of key investment management practices. We also reviewed the composition of the boards to determine whether they included senior executives from both IT and business (i.e., mission) units, in accordance with the GAO IT Investment Management framework which identifies the key practices for creating and maintaining successful investment management processes.

For the second objective, we selected a sample of 48 IT projects that were identified as being poorly planned according to the Office of Management and Budget’s Management Watch List or reported as poorly performing on the High-Risk Lists or both. To provide a governmentwide perspective, we attempted to select one project from the 2007 Management Watch List and one project from the High-Risk List with performance shortfalls during 2007 for each of the 24 major agencies. We focused on the high-risk projects with performance shortfalls in the areas of cost and schedule since we had reported in September 2007 that these were the most frequently reported shortfalls. To obtain broader representation of agencies with high-risk projects, we also selected three High-Risk projects that had performance shortfalls in 2006. From these lists, we selected those projects with the highest funding levels according to the fiscal year 2008 President’s budget request. When an agency had a project on only one of the lists (i.e., only the Management Watch List or High-Risk List), we selected at least 2 projects from that list. For example, we selected 2 high-risk projects with shortfalls for the Environmental Protection Agency because the agency did not have any projects on the Management Watch List for the time frame we considered.

Our selection process resulted in 26 projects from the Management Watch List, totaling about $7.4 billion in the fiscal year 2008 budget request, and 33 projects from the High-Risk List, totaling about $5.2 billion in the fiscal year 2008 budget request. Eleven of these projects, totaling about $4 billion, were on both lists. The Department of Energy and the National Science Foundation did not have any projects on the Management Watch List or on the High-Risk List with shortfalls and, therefore, we did not select any projects from these agencies. We removed two Management Watch List projects and five high-risk projects from our initial sample after sending the draft report to agency comment because we determined after further review and discussion with agencies that these projects had not been on the Management Watch List during 2007 or reported negative cost or schedule variances exceeding 10 percent between December 2006 and December 2007. This brought our sample of Management Watch List projects to 24 projects, totaling about $7.3 billion in the fiscal year 2008 budget request and 28 high-risk projects totaling about $4.7 billion in the fiscal year 2008 budget request and the number of projects on both lists to 11 projects totaling $4 billion in the fiscal year 2008 budget request.

To determine whether department-level IRBs were performing selection and oversight reviews of poorly planned and performing projects, we requested evidence of board reviews for the 48 projects in our sample during the time they were either on the Management Watch List or High- Risk List. We analyzed the documentation obtained, and, when reviews had not been performed, we followed up with agencies to determine why the required reviews were not performed. For the oversight reviews, we determined whether project cost, benefit, schedule and risk data had been provided to the board, but we did not assess the reliability of this information.

We conducted this performance audit from January 2008 to June 2009 in Washington, D.C., in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Commerce

Appendix III: Comments from the Department of Defense

Appendix IV: Comments from the Department of Education

Appendix V: Comments from the Department of Homeland Security

Appendix VI: Comments from the Department of Housing and Urban Development

Appendix VII: Comments from the Department of the Interior

Appendix VIII: Comments from the Department of Justice

Appendix IX: Comments from the Department of Labor

Appendix X: Comments from the Department of the Treasury

Appendix XI: Comments from the Department of Veterans Affairs

Appendix XII: Comments from the National Aeronautics and Space Administration

Appendix XIII: Comments from the Nuclear Regulatory Commission

Appendix XIV: Comments from the Social Security Administration

Appendix XV: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to the individual named above, Sabine R. Paul, Assistant Director; William G. Barrick; Neil J. Doherty; Nancy E. Glover; Robert G. Kershaw; Lee A. McCracken; Tomas Ramirez; and Kevin C. Walsh made key contributions to this report.