Information Technology: OMB Can More Effectively Use Its Investment Reviews
Released: 2005-04-21T00:00:00

Letter

Background

The President’s Budget for Fiscal Year 2005 identified approximately $60 billion for IT projects. In that budget, OMB stated that, of approximately 1,200 major IT projects, about half—621 projects, representing about $22 billion—were on a Management Watch List. In testimony in March 2004, OMB officials explained that the fiscal year 2005 budget process required agencies to successfully correct project weaknesses and business case deficiencies of projects on the Management Watch List; otherwise, OMB would limit agencies’ spending on new starts and other developmental activities.

In the most recent budget, that for fiscal year 2006, OMB continued its use of a Management Watch List. This budget includes 1,087 IT projects, totaling about $65 billion. Of this total, 342 projects, representing about $15 billion, are on the Management Watch List. The budget also stated that projects on the Management Watch List had to address performance, security, or other related issues before funding would be obligated in fiscal year 2006.

According to OMB officials, the office identifies projects for the Management Watch List through their evaluation of justifications for funding that agencies submit for major IT projects as part of the budget development process. This evaluation is carried out as part of OMB’s predominant mission: to assist the President in overseeing the preparation of the federal budget and to supervise budget administration in executive branch agencies. OMB is also responsible for evaluating the effectiveness of agency programs, policies, and procedures; assessing competing funding demands among agencies; and setting funding priorities. Finally, OMB is responsible for overseeing and coordinating the administration’s policies regarding procurement, financial management, information, and regulations. In each of these three areas of responsibility, OMB’s role is to help improve administrative management, to develop better performance measures and coordinating mechanisms, and to reduce unnecessary burden on the public.

To drive improvement in the implementation and management of IT projects, the Congress enacted the Clinger-Cohen Act in 1996, which expanded the responsibilities of the agencies and OMB under the Paperwork Reduction Act. Under the act, agencies are required to engage in capital planning and performance- and results-based management. OMB is required to establish processes to analyze, track, and evaluate the risks and results of major capital investments in information systems made by executive agencies. OMB is also required to report to the Congress on the net program performance benefits achieved as a result of major capital investments in information systems that are made by executive agencies.

In response to the Clinger-Cohen Act and other statutes, OMB developed section 300 of Circular A-11. This section provides policy for planning, budgeting, acquisition, and management of federal capital assets and instructs agencies on budget justification and reporting requirements for major IT investments. Section 300 defines the budget exhibit 300 as a document that agencies submit to OMB to justify resource requests for major IT investments. This reporting mechanism (part of the budget formulation and review process) is intended to enable an agency to demonstrate to its own management, as well as to OMB, that it has employed the disciplines of good project management; developed a strong business case for the investment; and met other Administration priorities in defining the cost, schedule, and performance goals proposed for the investment. The exhibit 300 includes information that is intended, among other things, to help OMB and the agencies identify and correct poorly planned or performing investments (i.e., investments that are behind schedule, over budget, or not delivering expected results) and real or potential systemic weaknesses in federal information resource management (e.g., project manager qualifications).

According to OMB’s description of its processes, agencies’ exhibit 300 business cases are reviewed by OMB analysts from its four statutory offices—the Offices of E-Government and Information Technology (e-Gov), Information and Regulatory Affairs (OIRA), Federal Financial Management, and Federal Procurement Policy— and its Resource Management Offices (RMO). Within OIRA, each of about 12 analysts is responsible for overseeing IT projects for a specific agency or (more commonly) several agencies. According to OMB officials, the OIRA and e-Gov analysts, along with RMO program examiners, evaluate and score agency exhibit 300 business cases as part of the development of the President’s Budget. The results of this review are provided to agencies through what is called the “passback” process. That is, OMB passes the requests back to agencies with its evaluation, which identifies any areas requiring remediation.

The integrity of this review process presupposes that the exhibit 300s are accurate. In response to a request from this committee, we are currently reviewing the quality of the information that underlies exhibit 300s at several agencies. We will be reporting on this work in the fall of this year.

OMB Established Processes and Criteria for Identifying Weak Projects, but It Did Not Use an Aggregate List to Perform Its Analysis or Oversight

According to OMB officials, including the Deputy Administrator of OIRA and the Chief of the Information Technology and Policy Branch, OMB staff identified projects for the Management Watch List through their evaluation of the exhibit 300s that agencies submit for major IT projects as part of the budget development process. The OMB officials added that the scoring of agency exhibit 300s is based on guidance in OMB Circular A-11 that is intended to ensure that agency planning and management of capital assets are consistent with OMB policy and guidance.

As described in Circular A-11, the scoring of a business case consists of individual scoring for 10 categories, as well as a total composite score of all the categories. (Examples of these 10 categories are performance goals, security and privacy, performance-based management system—including the earned value management system—and support of the President’s Management Agenda.) According to Circular A-11, scores range from 1 to 5, with 5 indicating investments whose business cases provided the best justification and 1 the least.

OMB officials said that, for fiscal year 2005, an IT project was placed on the Management Watch List if its exhibit 300 business case received a total composite score of 3 or less, or if it received a score of 3 or less in the areas of performance goals, performance-based management systems, or security and privacy, even if its overall score was a 4 or 5. OMB reported that agencies with weaknesses in these three areas were to submit remediation plans addressing the weaknesses.

According to OMB management, individual analysts were responsible for evaluating projects and determining which projects met the criteria to be on the Management Watch List for their assigned agencies. To derive the total number of projects on the list that were reported for fiscal year 2005, OMB polled the individual analysts and compiled the numbers.

OMB officials said that they did not aggregate these projects into a single list describing projects and their weaknesses, because they did not see such an activity as necessary in performing OMB’s predominant mission. Further, OMB officials stated that the limited number of analysts involved enabled them to explore governmentwide issues using ad hoc queries and to develop approaches to address systemic problems without the use of an aggregate list. They pointed at successes in improving IT management, such as better compliance with security requirements, as examples of the effectiveness of their current approach.

Nevertheless, OMB has not fully exploited the opportunity to use its Management Watch List as a tool for analyzing IT investments on a governmentwide basis. According to the Clinger-Cohen Act, OMB is required to establish processes to analyze, track, and evaluate the risks and results of major IT capital investments made by executive agencies, which aggregation of the Management Watch List would facilitate. Without aggregation, OMB’s ability to conduct governmentwide analysis is limited, since no governmentwide dataset exists—only a set of subordinate datasets in the hands of individual analysts. In addition, each time an up-to-date report is required, OMB must query all its analysts to assemble an aggregate response; thus, the office cannot efficiently identify, analyze, and report on the full set of IT investments requiring corrective actions.

OMB’s Monitoring of Projects Was Inconsistent, and Agency Follow- up Activities Were Not Tracked Centrally

OMB asked agencies to take corrective actions to address weaknesses associated with projects on the Management Watch List, but it did not develop a structured, consistent process or criteria for deciding how to follow up on these actions. Instead, OMB officials, including the Deputy Administrator of OIRA and the Chief of the Information Technology and Policy Branch, said that the decision on whether and how to follow up on a specific project was typically made jointly between the OIRA analyst and the RMO program examiner who had responsibility for the individual agency, and that follow-up on specific projects was driven by a number of factors, only one of which was inclusion on the Management Watch List. According to these officials, those Management Watch List projects that did receive specific follow-up attention received feedback through the passback process, through targeted evaluation of remediation plans designed to address weaknesses, and through the apportioning of funds so that the use of budgeted dollars was conditional on appropriate remediation plans being in place.

These officials also said that follow-up of some Management Watch List projects was done through quarterly e-Gov Scorecards; these are reports that use a red/yellow/green scoring system to illustrate the results of OMB’s evaluation of the agencies’ implementation of e-government criteria in the President’s Management Agenda. OMB determines the scores in quarterly reviews, in which it evaluates agency progress toward agreed-upon goals along several dimensions. The e-gov scores are part of the input to the quarterly reporting on the President’s Management Agenda.

OMB officials also stated that those Management Watch List projects that did receive follow-up attention were not tracked centrally, but only by the individual OMB analysts. Accordingly, OMB could not readily tell us which of the 621 watch list projects for fiscal year 2005 were followed up on, nor could it use the list to describe the relationship between its follow-up activities and the changes in the numbers of projects on the watch list between fiscal year 2005 (621 projects) and fiscal year 2006 (342 projects).

OMB does not have specific criteria for prioritizing follow-up on Management Watch List projects. Without specific criteria, OMB staff may be agreeing to commit resources to follow up on projects that did not represent OMB’s top priorities from a governmentwide perspective. For example, inconsistent attention to OMB priorities, such as earned value management, could undermine the objectives that OMB set in these areas. In addition, major projects with significant management deficiencies may have continued to absorb critical agency resources.

In order for OMB management to have assurance that IT program deficiencies are addressed, it is critical that corrective actions associated with Management Watch List projects be monitored. Such monitoring is instrumental in ensuring that agencies address and resolve weaknesses found in exhibit 300s, which may indicate underlying weaknesses in project planning or management. Tracking agency follow-up activities is essential to enabling OMB to determine progress on both specific projects and governmentwide trends. Without tracking specific follow-up activities, OMB could not readily ascertain whether the risks that it identified through its Management Watch List were being managed effectively; if they were not, funds were potentially being spent on poorly planned and managed projects.

In summary, OMB’s scoring of agency IT budget submissions and identification of weaknesses has resulted in opportunities to strengthen investments. However, the office has not taken the next step—to develop a single, aggregate list identifying the projects and their weaknesses—and it has not developed a structured, consistent process for deciding how to follow up on corrective actions. OMB’s approach does not fully exploit the insights developed through the scoring process, and it may leave unattended weak projects consuming significant budget dollars. Developing an aggregated list would help OMB to realize more fully the potential benefits of using the Management Watch List as a tool for monitoring and analyzing IT investments governmentwide. Accordingly, in our report we recommended that the Director of OMB take the following four actions: ● Develop a central list of projects and their deficiencies. ● Use the list as the basis for selecting projects for follow-up and for ● to guide follow-up, develop specific criteria for prioritizing the IT projects included on the list, taking into consideration such factors as the relative potential financial and program benefits of these IT projects, as well as potential risks. ● Analyze the prioritized list to develop governmentwide and agency assessments of the progress and risks of IT investments, identifying opportunities for continued improvement. ● Report to the Congress on progress made in addressing risks of major IT investments and management areas needing attention.

In commenting on a draft of this report, OMB’s Administrator of the Office of E-Government and Information Technology expressed appreciation for our review of OMB’s use of its Management Watch List. However, the Administrator disagreed with our assessment that an aggregated governmentwide list is necessary to perform adequate oversight and management, and that OMB does not know whether risks are being addressed. Nonetheless, based on OMB’s inability to easily report which of the 621 investments on the Management Watch List remained deficient or how much of the $22 billion cited in the President’s Budget remained at risk, we continue to believe that an aggregate list would facilitate OMB’s ability to track progress.

Mr. Chairman, that concludes my testimony. I would be pleased to answer any questions that you and the other Members of the Committee may have.

Contact and Acknowledgements

For further information, please contact David A. Powner at (202) 512-9286 or Lester Diamond at (202) 512-7957. We can also be reached by e-mail at pownerd@gao.gov or diamondl@gao.gov. Key contributors to this testimony were William G. Barrick, Barbara Collier, Lester Diamond, and Sandra Kerr.

This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.