Information Technology: OMB Can More Effectively Use Its Investment Reviews
Published: 2005-04-21T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Federal spending on information technology (IT) is over $60 billion this year and is expected to continue to rise. Accordingly, it is essential that federal IT investments are managed efficiently. Of the 1,200 major IT projects in the President's Budget for Fiscal Year 2005, OMB stated that it had placed about half--621 projects, representing about $22 billion--on a Management Watch List to focus attention on mission-critical IT investments that need management improvements. GAO was asked to testify on the findings and recommendations made in a report that it recently completed (GAO-05-276), which describes and assesses OMB's processes for (1) placing projects on its Management Watch List and (2) following up on corrective actions established for projects on the list.
section_title: What GAO Found
paragraphs: For the fiscal year 2005 budget, OMB developed processes and criteria for including investments on its Management Watch List. In doing so, it identified opportunities to strengthen investments and promote improvements in IT management. However, it did not develop a single, aggregate list identifying the projects and their weaknesses. Instead, OMB officials told GAO that to identify projects with weaknesses, individual analysts used scoring criteria that the office established for evaluating the justifications for funding that federal agencies submit for major projects. These analysts, each of whom is typically responsible for several federal agencies, were then responsible for maintaining information on these projects. To derive the total number of projects on the list for fiscal year 2005, the office polled its individual analysts and compiled the result. However, OMB officials told GAO that because they did not see such an activity as necessary, they did not compile a single list. Accordingly, OMB has not fully exploited the opportunity to use its watch list as a tool for analyzing IT investments on a governmentwide basis. OMB asked agencies to take corrective actions to address weaknesses associated with projects on the Management Watch List, but it did not develop a structured, consistent process for deciding how to monitor agency corrective actions. According to OMB officials, decisions on monitoring of progress were typically made by the staff with responsibility for reviewing individual agency budget submissions, depending on the staff's insights into agency operations and objectives. Because it did not consistently require or monitor agency follow-up activities, OMB did not know whether the project risks that it identified through its Management Watch List were being managed effectively, potentially leaving resources at risk of being committed to poorly planned and managed projects. In addition, because it did not consistently monitor the follow-up performed on projects on the Management Watch List, OMB could not readily tell GAO which of the 621 projects received follow-up attention. To help enable OMB to take advantage of the potential benefits of using the Management Watch List as a tool for analyzing and following up on investments, GAO's report included recommendations that OMB develop a single, aggregate Management Watch List and that it develop and use criteria for prioritizing and monitoring the projects on the list. GAO also recommended that the office use the prioritized list for reporting to the Congress as part of its statutory reporting responsibilities. In commenting on a draft of this report, OMB did not agree that the aggregated governmentwide list recommended by GAO is necessary for adequate oversight and management. However, GAO continues to believe that an aggregated Management Watch List would contribute to OMB's ability to analyze IT investments governmentwide and track progress in addressing deficiencies.

section_title: Letter
paragraphs: Thank you for the opportunity to participate in the Committee’s  hearing on processes that the Office of Management and Budget  (OMB) has developed as part of its efforts to identify and follow up  on information technology (IT) projects that need management  improvements.
As you know, the President’s Budget for Fiscal Year 2005 requested  over $60 billion to fund IT, and that figure is expected to rise  throughout the rest of the decade. OMB stated that of the nearly  1,200 major IT projects in the fiscal year 2005 budget, it had placed  about half—621 projects, representing about $22 billion—on its  Management Watch List. For fiscal year 2006, 342 of 1,087 IT  projects (representing about $15 billion) were placed on the watch  list.
At your request, we performed a review of OMB’s processes for  (1) placing projects on its Management Watch List and (2) following  up on corrective actions established for projects on the list. Today I  am summarizing the findings and recommendations of that report,  which is being released today.
subsections: section_title: Background
paragraphs: The President’s Budget for Fiscal Year 2005 identified  approximately $60 billion for IT projects. In that budget, OMB stated  that, of approximately 1,200 major IT projects, about half—621  projects, representing about $22 billion—were on a Management  Watch List. In testimony in March 2004, OMB officials explained  that the fiscal year 2005 budget process required agencies to  successfully correct project weaknesses and business case  deficiencies of projects on the Management Watch List; otherwise,  OMB would limit agencies’ spending on new starts and other  developmental activities.
In the most recent budget, that for fiscal year 2006, OMB continued  its use of a Management Watch List. This budget includes 1,087 IT  projects, totaling about $65 billion. Of this total, 342 projects,  representing about $15 billion, are on the Management Watch List.  The budget also stated that projects on the Management Watch List  had to address performance, security, or other related issues before  funding would be obligated in fiscal year 2006.
According to OMB officials, the office identifies projects for the  Management Watch List through their evaluation of justifications for  funding that agencies submit for major IT projects as part of the  budget development process. This evaluation is carried out as part  of OMB’s predominant mission: to assist the President in overseeing  the preparation of the federal budget and to supervise budget  administration in executive branch agencies. OMB is also  responsible for evaluating the effectiveness of agency programs,  policies, and procedures; assessing competing funding demands  among agencies; and setting funding priorities. Finally, OMB is  responsible for overseeing and coordinating the administration’s  policies regarding procurement, financial management, information,  and regulations. In each of these three areas of responsibility,  OMB’s role is to help improve administrative management, to  develop better performance measures and coordinating  mechanisms, and to reduce unnecessary burden on the public.
To drive improvement in the implementation and management of IT  projects, the Congress enacted the Clinger-Cohen Act in 1996, which  expanded the responsibilities of the agencies and OMB under the  Paperwork Reduction Act. Under the act, agencies are required to  engage in capital planning and performance- and results-based  management. OMB is required to establish processes to analyze,  track, and evaluate the risks and results of major capital  investments in information systems made by executive agencies.  OMB is also required to report to the Congress on the net program  performance benefits achieved as a result of major capital  investments in information systems that are made by executive  agencies.
In response to the Clinger-Cohen Act and other statutes, OMB  developed section 300 of Circular A-11. This section provides policy  for planning, budgeting, acquisition, and management of federal  capital assets and instructs agencies on budget justification and  reporting requirements for major IT investments. Section 300  defines the budget exhibit 300 as a document that agencies submit  to OMB to justify resource requests for major IT investments. This  reporting mechanism (part of the budget formulation and review  process) is intended to enable an agency to demonstrate to its own  management, as well as to OMB, that it has employed the disciplines  of good project management; developed a strong business case for  the investment; and met other Administration priorities in defining  the cost, schedule, and performance goals proposed for the  investment. The exhibit 300 includes information that is intended,  among other things, to help OMB and the agencies identify and  correct poorly planned or performing investments (i.e., investments  that are behind schedule, over budget, or not delivering expected  results) and real or potential systemic weaknesses in federal  information resource management (e.g., project manager  qualifications).
According to OMB’s description of its processes, agencies’ exhibit  300 business cases are reviewed by OMB analysts from its four  statutory offices—the Offices of E-Government and Information  Technology (e-Gov), Information and Regulatory Affairs (OIRA),  Federal Financial Management, and Federal Procurement Policy— and its Resource Management Offices (RMO). Within OIRA, each of  about 12 analysts is responsible for overseeing IT projects for a  specific agency or (more commonly) several agencies. According to  OMB officials, the OIRA and e-Gov analysts, along with RMO  program examiners, evaluate and score agency exhibit 300 business  cases as part of the development of the President’s Budget. The  results of this review are provided to agencies through what is  called the “passback” process. That is, OMB passes the requests  back to agencies with its evaluation, which identifies any areas  requiring remediation.
The integrity of this review process presupposes that the exhibit  300s are accurate. In response to a request from this committee, we  are currently reviewing the quality of the information that underlies  exhibit 300s at several agencies. We will be reporting on this work in  the fall of this year.
section_title: OMB Established Processes and Criteria for Identifying Weak Projects, but It Did Not Use an Aggregate List to Perform Its Analysis or Oversight
paragraphs: According to OMB officials, including the Deputy Administrator of  OIRA and the Chief of the Information Technology and Policy  Branch, OMB staff identified projects for the Management Watch  List through their evaluation of the exhibit 300s that agencies submit  for major IT projects as part of the budget development process.  The OMB officials added that the scoring of agency exhibit 300s is  based on guidance in OMB Circular A-11 that is intended to ensure  that agency planning and management of capital assets are  consistent with OMB policy and guidance.
As described in Circular A-11, the scoring of a business case  consists of individual scoring for 10 categories, as well as a total  composite score of all the categories. (Examples of these 10  categories are performance goals, security and privacy,  performance-based management system—including the earned  value management system—and support of the President’s  Management Agenda.) According to Circular A-11, scores range  from 1 to 5, with 5 indicating investments whose business cases  provided the best justification and 1 the least.
OMB officials said that, for fiscal year 2005, an IT project was placed  on the Management Watch List if its exhibit 300 business case  received a total composite score of 3 or less, or if it received a score  of 3 or less in the areas of performance goals, performance-based  management systems, or security and privacy, even if its overall  score was a 4 or 5. OMB reported that agencies with weaknesses in  these three areas were to submit remediation plans addressing the  weaknesses.
According to OMB management, individual analysts were  responsible for evaluating projects and determining which projects  met the criteria to be on the Management Watch List for their  assigned agencies. To derive the total number of projects on the list  that were reported for fiscal year 2005, OMB polled the individual  analysts and compiled the numbers.
OMB officials said that they did not aggregate these projects into a  single list describing projects and their weaknesses, because they  did not see such an activity as necessary in performing OMB’s  predominant mission. Further, OMB officials stated that the limited  number of analysts involved enabled them to explore  governmentwide issues using ad hoc queries and to develop  approaches to address systemic problems without the use of an  aggregate list. They pointed at successes in improving IT  management, such as better compliance with security requirements,  as examples of the effectiveness of their current approach.
Nevertheless, OMB has not fully exploited the opportunity to use its  Management Watch List as a tool for analyzing IT investments on a  governmentwide basis. According to the Clinger-Cohen Act, OMB is  required to establish processes to analyze, track, and evaluate the  risks and results of major IT capital investments made by executive  agencies, which aggregation of the Management Watch List would  facilitate. Without aggregation, OMB’s ability to conduct  governmentwide analysis is limited, since no governmentwide  dataset exists—only a set of subordinate datasets in the hands of  individual analysts. In addition, each time an up-to-date report is  required, OMB must query all its analysts to assemble an aggregate  response; thus, the office cannot efficiently identify, analyze, and  report on the full set of IT investments requiring corrective actions.
section_title: OMB’s Monitoring of Projects Was Inconsistent, and Agency Follow- up Activities Were Not Tracked Centrally
paragraphs: OMB asked agencies to take corrective actions to address  weaknesses associated with projects on the Management Watch  List, but it did not develop a structured, consistent process or  criteria for deciding how to follow up on these actions. Instead,  OMB officials, including the Deputy Administrator of OIRA and the  Chief of the Information Technology and Policy Branch, said that  the decision on whether and how to follow up on a specific project  was typically made jointly between the OIRA analyst and the RMO  program examiner who had responsibility for the individual agency,  and that follow-up on specific projects was driven by a number of  factors, only one of which was inclusion on the Management Watch  List. According to these officials, those Management Watch List  projects that did receive specific follow-up attention received  feedback through the passback process, through targeted evaluation  of remediation plans designed to address weaknesses, and through  the apportioning of funds so that the use of budgeted dollars was  conditional on appropriate remediation plans being in place.
These officials also said that follow-up of some Management Watch  List projects was done through quarterly e-Gov Scorecards; these  are reports that use a red/yellow/green scoring system to illustrate  the results of OMB’s evaluation of the agencies’ implementation of  e-government criteria in the President’s Management Agenda. OMB  determines the scores in quarterly reviews, in which it evaluates  agency progress toward agreed-upon goals along several  dimensions. The e-gov scores are part of the input to the quarterly  reporting on the President’s Management Agenda.
OMB officials also stated that those Management Watch List  projects that did receive follow-up attention were not tracked  centrally, but only by the individual OMB analysts. Accordingly,  OMB could not readily tell us which of the 621 watch list projects  for fiscal year 2005 were followed up on, nor could it use the list to  describe the relationship between its follow-up activities and the  changes in the numbers of projects on the watch list between fiscal  year 2005 (621 projects) and fiscal year 2006 (342 projects).
OMB does not have specific criteria for prioritizing follow-up on  Management Watch List projects. Without specific criteria, OMB  staff may be agreeing to commit resources to follow up on projects  that did not represent OMB’s top priorities from a governmentwide  perspective. For example, inconsistent attention to OMB priorities,  such as earned value management, could undermine the objectives  that OMB set in these areas. In addition, major projects with  significant management deficiencies may have continued to absorb  critical agency resources.
In order for OMB management to have assurance that IT program  deficiencies are addressed, it is critical that corrective actions  associated with Management Watch List projects be monitored.  Such monitoring is instrumental in ensuring that agencies address  and resolve weaknesses found in exhibit 300s, which may indicate  underlying weaknesses in project planning or management.  Tracking agency follow-up activities is essential to enabling OMB to  determine progress on both specific projects and governmentwide  trends. Without tracking specific follow-up activities, OMB could  not readily ascertain whether the risks that it identified through its  Management Watch List were being managed effectively; if they  were not, funds were potentially being spent on poorly planned and  managed projects.
In summary, OMB’s scoring of agency IT budget submissions and  identification of weaknesses has resulted in opportunities to  strengthen investments. However, the office has not taken the next  step—to develop a single, aggregate list identifying the projects and  their weaknesses—and it has not developed a structured, consistent  process for deciding how to follow up on corrective actions. OMB’s  approach does not fully exploit the insights developed through the  scoring process, and it may leave unattended weak projects  consuming significant budget dollars. Developing an aggregated list  would help OMB to realize more fully the potential benefits of using  the Management Watch List as a tool for monitoring and analyzing  IT investments governmentwide. Accordingly, in our report we  recommended that the Director of OMB take the following four  actions:  ●  Develop a central list of projects and their deficiencies.  ●  Use the list as the basis for selecting projects for follow-up and for  ●  to guide follow-up, develop specific criteria for prioritizing the IT  projects included on the list, taking into consideration such  factors as the relative potential financial and program benefits of  these IT projects, as well as potential risks.   ●  Analyze the prioritized list to develop governmentwide and agency  assessments of the progress and risks of IT investments, identifying  opportunities for continued improvement.  ●  Report to the Congress on progress made in addressing risks of  major IT investments and management areas needing attention.
In commenting on a draft of this report, OMB’s Administrator of the  Office of E-Government and Information Technology expressed  appreciation for our review of OMB’s use of its Management Watch  List. However, the Administrator disagreed with our assessment that  an aggregated governmentwide list is necessary to perform adequate  oversight and management, and that OMB does not know whether  risks are being addressed. Nonetheless, based on OMB’s inability to  easily report which of the 621 investments on the Management  Watch List remained deficient or how much of the $22 billion cited  in the President’s Budget remained at risk, we continue to believe  that an aggregate list would facilitate OMB’s ability to track  progress.
Mr. Chairman, that concludes my testimony. I would be pleased to  answer any questions that you and the other Members of the  Committee may have.
section_title: Contact and Acknowledgements
paragraphs: For further information, please contact David A. Powner at (202)  512-9286 or Lester Diamond at (202) 512-7957. We can also be  reached by e-mail at pownerd@gao.gov or diamondl@gao.gov. Key  contributors to this testimony were William G. Barrick, Barbara  Collier, Lester Diamond, and Sandra Kerr.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.