Information Technology: OMB and Agencies Need to Improve Planning, Management, and Oversight of Projects Totaling Billions of Dollars
Published: 2008-07-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The federal government spends billons of dollars on information technology (IT) projects each year. Consequently, it is important that projects be managed effectively to ensure that public resources are wisely invested. To this end, the Office of Management and Budget (OMB), which plays a key role in overseeing the federal government's IT investments, identifies major projects that are poorly planned by placing them on a Management Watch List and requires agencies to identify high-risk projects that are performing poorly (i.e., have performance shortfalls). Having accurate and transparent project cost and schedule information is also essential to effective oversight. At times, changes to this information--called a rebaselining-- are made to reflect changed development circumstances. These changes can be done for valid reasons, but can also be used to mask cost overruns and schedule delays. GAO has previously testified on the Management Watch List and high risk projects. GAO was asked to (1) provide an update on these projects, (2) identify OMB's efforts to improve the identification and oversight of these projects, and (3) summarize the results of GAO's IT project rebaselining report being released today. In preparing this testimony, GAO analyzed current Management Watch List and high risk project information.
section_title: What GAO Found
paragraphs: OMB and federal agencies have identified approximately 413 IT projects--totaling at least $25.2 billion in expenditures for fiscal year 2008--as being poorly planned, poorly performing, or both. Specifically, through the Management Watch List process, OMB determined that 352 projects (totaling about $23.4 billion) are poorly planned. In addition, agencies reported that 87 of their high risk projects (totaling about $4.8 billion) were poorly performing. Twenty-six projects (totaling about $3 billion) are considered both poorly planned and poorly performing. OMB has taken steps to improve the identification of the Management Watch List and high-risk projects since GAO testified last September, including publicly disclosing reasons for placement on the Management Watch List and clarifying high-risk project criteria. However, more needs to be done by both OMB and the agencies to address recommendations GAO has previously made to improve the planning, management, and oversight of poorly planned and performing projects so that potentially billions in taxpayer dollars are not wasted. In its rebaselining review, GAO reports that 48 percent of the federal government's major IT projects have been rebaselined for several reasons, including changes in project goals and changes in funding. Of those rebaselined projects, 51 percent were rebaselined at least twice and about 11 percent were rebaselined 4 times or more. In addition, while the major agencies have all established rebaselining policies, these policies are not comprehensive. Specifically, none of the policies were fully consistent with best practices, including describing a process for developing a new baseline and requiring the validation of the new baseline. Agencies' policies varied in part because OMB has not issued guidance specifying what elements these policies are to include. In its report, GAO makes recommendations to OMB to issue guidance for rebaselining policies and to the major agencies to develop comprehensive rebaselining policies that address identified weaknesses.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the federal government’s  processes for improving the management of information technology (IT)  investments. As you know, billions of taxpayer dollars are spent on these  projects each year. This number is expected to reach $71 billion for fiscal  year 2009. Given the size of these investments and the criticality of many  of the systems to the health, economy, and security of the nation, it is  important that they be effectively managed.
To this end, the Office of Management and Budget (OMB), which plays a  key role in overseeing the federal government’s IT investments, identifies  major projects that are poorly planned by placing them on a Management  Watch List and requires agencies to identify high-risk projects that are  performing poorly. Having accurate and transparent project cost and  schedule information is also essential to effective oversight. At times,  changes to this information—called a rebaselining— are made to reflect  changed development circumstances. These changes can be done for valid  reasons, but can also be used to mask cost overruns and schedule delays.
We have testified on the Management Watch List and high-risk projects for  the past 2 years, highlighting the number and dollar value of the projects  identified as poorly planned and/or poorly performing. You asked us to (1)  provide an update on OMB’s Management Watch List and list of high-risk  projects, (2) identify OMB’s efforts to improve the identification and  oversight of these projects, and (3) summarize our IT project rebaselining  report, which is being released today.  In preparing this testimony, we  analyzed the current Management Watch List and high-risk project  information and reviewed recent actions taken by OMB to better identify  and oversee these projects. In completing our rebaselining review we  surveyed the managers of a random sample of 180 projects selected from  the 778 major IT projects the 24 major agencies plan to invest in during  fiscal year 2008 and compared agencies’ rebaselining policies to best  practices identified in our Cost Assessment Guide. We performed our  work in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Each year, OMB and federal agencies work together to determine how  much the government plans to spend on IT projects and how these funds  are to be allocated. Federal IT spending has risen to an estimated $71  billion for fiscal year 2009.
OMB plays a key role in overseeing the implementation and management  of federal IT investments. To improve this oversight, Congress enacted the  Clinger-Cohen Act in 1996, expanding the responsibilities delegated to  OMB and agencies under the Paperwork Reduction Act. Among other  things, Clinger-Cohen requires agencies to better link their IT planning and  investment decisions to program missions and goals and to implement and  enforce IT management policies, procedures, standards, and guidelines.  The act also requires that agencies engage in capital planning and  performance and results-based management. OMB’s responsibilities under  the act include establishing processes to analyze, track, and evaluate the  risks and results of major capital investments in information systems made  by executive agencies. OMB must also report to Congress on the net  program performance benefits achieved as a result of these investments.
In response to the Clinger-Cohen Act and other statutes, OMB developed  policy for the planning, budgeting, acquisition, and management of federal  capital assets. This policy is set forth in OMB Circular A-11 (section 300)  and in OMB’s Capital Programming Guide (supplement to Part 7 of  Circular A-11), which direct agencies to develop, implement, and use a  capital programming process to build their capital asset portfolios. Among  other things, OMB’s Capital Programming Guide directs agencies to    evaluate and select capital asset investments that will support core  mission functions and demonstrate projected returns on investment that  are clearly equal to or better than alternative uses of available public  resources,  institute performance measures and management processes that monitor  actual performance and compare it to planned results, and    establish oversight mechanisms that require periodic review of operational  capital assets to determine if mission requirements have changed and  whether the asset continues to fulfill those requirements and deliver its  intended benefits.
To further support the implementation of IT capital planning practices as  required by statute and directed in OMB’s Capital Programming Guide,  we have developed an IT investment management framework that  agencies can use in developing a stable and effective capital planning  process. It is a tool that can be used to determine both the status of an  agency’s current IT investment management capabilities and the additional  steps that are needed to establish more effective processes. Mature and  effective management of IT investments can vastly improve government  performance and accountability, while poor management can result in  wasteful spending and lost opportunities for improving delivery of services  to the public.
subsections: section_title: Prior Reviews on Federal IT Investment Management Have Identified Weaknesses
paragraphs: We have previously reported that the federal government faces enduring  challenges in effectively managing IT investments. For example, in  January 2004, we reported on the mixed results of federal agencies’ use of  IT investment management practices. Specifically, we reported that,  although most of the agencies had IT investment boards responsible for  defining and implementing the agencies’ investment management  processes, they did not always have important mechanisms in place for  these boards to effectively control investments, including decision-making  rules for project oversight, early warning mechanisms, and requirements  that corrective actions for underperforming projects be agreed upon and  tracked. Accordingly, we made several recommendations to agencies to  improve their practices.
In previous work using our investment management framework, we  reported that the use of IT investment management practices by agencies  was mixed. For example, a few agencies that have followed the framework  in implementing capital planning processes have made significant  improvements. In contrast, however, we and others have continued to  identify weaknesses at agencies in many areas, including immature  management processes to support both the selection and oversight of  major IT investments and the measurement of actual versus expected  performance in meeting established performance measures. For example,  in 2007, we reported that the Department of Homeland Security and the  Department of the Treasury did not have the processes in place to  effectively select and oversee their major investments.
section_title: OMB’s Management Watch List Is Intended to Correct Project Weaknesses and Business Case Deficiencies
paragraphs: To help ensure that investments of public resources are justified and that  public resources are wisely invested, OMB began using its Management  Watch List in the President’s fiscal year 2004 budget request as a means to  oversee the justification for and planning of agencies’ IT investments. This  list was derived based on a detailed review of each investment’s Capital  Asset Plan and Business Case, also known as the exhibit 300.
The exhibit 300 is a reporting mechanism intended to enable an agency to  demonstrate to its own management, as well as to OMB, that a major  project is well planned in that it has employed the disciplines of good  project management; developed a strong business case for the investment;  and met other Administration priorities in defining the cost, schedule, and  performance goals proposed for the investment.
In April 2005, we reported that OMB analysts evaluated agency exhibit  300s by assigning scores to each exhibit 300 based on guidance presented  in OMB Circular A-11. As described in this circular, the scoring of a  business case consisted of individual scoring for 10 categories (on a scale  from 1 to 5 with 5 being the highest), as well as a total composite score of  all the categories. The 10 scoring categories are:     support of the President’s Management Agenda,    project (investment) management,    cost/ schedule/ performance.
When we reported on the Management Watch List in 2005, projects were  placed on the Management Watch List if they received low scores (3 or  less) in the areas of performance goals, performance-based management  systems, security and privacy or if they received a low composite score.
For the fiscal year 2009 budget, OMB used more stringent criteria.  Specifically, OMB placed projects on the Management Watch List if they  had (1) an overall score of 30 or less, (2) a security score of 3 or less, or (3)  a non-security score of 2 or less. Projects were also placed on the list if  other sources (such as an Inspector General’s Federal Information  Security Management Act report or the agency’s President’s Management  Agenda E-government scorecard) indicated that the agency did not have  qualified project managers or had weaknesses in its implementation of  security, privacy, or earned value management techniques.
According to OMB, agencies with projects on the Management Watch List  are to submit remediation plans addressing the weaknesses. Those  projects that receive specific follow-up attention receive feedback through  what is known as the passback process, targeted evaluation of  remediation plans designed to address weaknesses, the apportioning of  funds made conditional on appropriate remediation plans being in place,  and the quarterly e-Government Scorecards. According to OMB, it  removes projects from the Management Watch List as agencies remediate  the weaknesses identified with these projects’ business cases.
section_title: OMB’s High-Risk Projects Process Is Intended to Correct and Improve Project Performance
paragraphs: As originally defined in OMB Circular A-11 and subsequently reiterated in  an August 2005 memorandum, high-risk projects are those that require  special attention from oversight authorities and the highest levels of  agency management. These projects are not necessarily at risk of failure,  but may be on the list because of one or more of the following four  reasons:    The agency has not consistently demonstrated the ability to manage  complex projects.
The project has exceptionally high development, operating, or  maintenance costs, either in absolute terms or as a percentage of the  agency’s total IT portfolio.
The project is being undertaken to correct recognized deficiencies in the  adequate performance of an essential mission program or function of the  agency, a component of the agency, or another organization.
Delay or failure of the project would introduce for the first time  unacceptable or inadequate performance or failure of an essential mission  function of the agency, a component of the agency, or another  organization.
In 2006, we reported that, to identify high-risk projects, staff from each  agency’s Office of the Chief Information Officer compare the criteria  against their current portfolio to determine which projects met OMB’s  definition. They then submit the list to OMB for review. According to OMB  and agency officials, after the submission of the initial list, examiners at  OMB work with individual agencies to identify or remove projects as  appropriate. According to most agencies, the final list is then approved by  their Chief Information Officer.
This year, OMB clarified and expanded the high-risk project criteria.  Specifically, in the materials supplementing the President’s budget for  fiscal year 2009, OMB listed the following criteria for identifying high-risk  projects:    projects with a high degree of political or citizen interest,    projects with cross-organizational or agency impact or interdependencies  with other systems efforts,    major systems on the Management Watch List at the conclusion of the  prior fiscal year that continue to warrant heightened attention during  project execution,    major systems formally designated as an E-Government or Line of  Business Shared Service Provider,    E-Government initiative migration projects that are planned or underway  (which are removed upon completion),    existing or legacy agency systems retiring once their functionality has  been migrated to a common solution (also removed once retired), and    program or program management office activities supporting government- wide common solutions.
For the identified high-risk projects, Chief Information Officers are to  assess, confirm, and document projects’ performance. Specifically,  agencies are required to determine, for each of their high-risk projects,  whether the project was meeting one or more of four performance  evaluation criteria, which include:    establishing baselines with clear cost, schedule, and performance goals;    maintaining the project’s cost and schedule variances within 10 percent;    assigning a qualified project manager; and    avoiding duplication by leveraging inter-agency and governmentwide  investments.
High risk projects failing to meet any of these four performance evaluation  criteria are considered to have “performance shortfalls.” Agencies are  instructed to document these shortfalls using a standard template  provided by OMB and provide this template to oversight authorities on  request. Upon submission, individual analysts review the quarterly  performance reports of projects with shortfalls to determine how well the  projects are progressing and, using other performance data already  received, whether the actions described in the planned improvement  efforts are adequate.
section_title: OMB’s Role and Laws and Guidance on IT Project Rebaselining
paragraphs: At times, a major IT project’s cost, schedule, and performance goals— known as a baseline—need to be modified to reflect new circumstances.  While these changes—generally referred to as rebaselining—can be done  for valid reasons—including, for example, changes in a project’s  objectives, scope, requirements, or funding stream—they can also be used  to mask cost overruns and schedule delays. The purpose of a rebaselining  is to ensure that project managers have realistic benchmarks for tracking  the status of the project.
OMB requires that all proposed changes to baselines be submitted to it  prior to an agency’s budget request (and that proposed changes should not  be assumed to be approved). The information OMB requires from agencies  includes costs and milestones from both the initial and current baselines  (if the program has been rebaselined). It also asks agencies whether the  investment was rebaselined during the past fiscal year and, if so, if the new  baseline was approved by the agency head. The Capital Programming  Guide also notes that OMB reviews the reasons for deviation from goals,  the reasonableness of the corrective actions proposed, and the validity of  increased cost estimates. The guide further states that OMB is to consider  approving a rebaseline proposal only when the agency has provided  justification based on an integrated baseline review, demonstrates that  the new goals have a high probability of attainability, and shows that the  acquisition will still have a benefit-cost ratio that justifies continued  funding after comparing it with the other projects in the portfolio and  considering budget limitations.
Staff from OMB’s Office of E-government and Information Technology and  the Acting Chief of OMB’s Information Policy and Technology Branch told  us that they review agencies’ earned value management policies to  determine their compliance with the provisions of the Presidential  Management Agenda for E-government. They stated that, in reviewing  these policies, they determine whether rebaselining is adequately  addressed.
In addition, the Department of Defense (DOD) has statutory requirements  involving rebaselining. Each major defense acquisition program is required  by statute to establish an approved program baseline before entering into  the system development and demonstration phase of the acquisition cycle.  For such programs, a revised baseline is also required for each subsequent  milestone authorizing entry into the next phase of the acquisition cycle.  The statute also requires DOD to prescribe regulations addressing the  content of the baseline, reports of deviations from the baseline,  procedures for reviewing such deviations within DOD, and procedures for  submission to and approval by the Secretary of Defense of revised  baselines.
We also recently issued a draft Cost Assessment Guide on best practices  for estimating and managing program costs which, among other things,  discusses considerations in rebaselining programs. For example, the guide  identifies key cost, schedule, project execution risk, and data accuracy  indicators that can serve as warning signs that a program may need to be  rebaselined. The guide also identifies best practices that are relevant to  rebaselining policies. These practices are: (1) describing reasons when a  rebaseline is warranted, (2) describing the process for developing a new  baseline, (3) requiring validation of the new baseline, (4) requiring  management review, and (5) requiring that decisions associated with the  rebaselining process are documented.
section_title: Hundreds of Projects Totaling Billions of Dollars in Estimated Expenditures for Fiscal Year 2009 Are Poorly Planned or Poorly Performing
paragraphs: OMB and federal agencies have identified approximately 413 IT projects— totaling at least $25.2 billion in expenditures for fiscal year 2009—as being  poorly planned, poorly performing, or both. Specifically, hundreds of  projects totaling billions of dollars have been placed on OMB’s  Management Watch List for fiscal year 2009. In addition, projects identified  as poorly performing under OMB’s high-risk process total about $4.8  billion in estimated expenditures for fiscal year 2009. Finally, 26 projects  totaling $3 billion have been identified as both poorly planned and poorly  performing.
Figure 1 shows the distribution of these projects and their associated  dollar values.
subsections: section_title: Hundreds of Projects Totaling Billions of Dollars Were Placed on the Management Watch List for Fiscal Year 2009
paragraphs: Each year, OMB has placed hundreds of projects totaling billions of  dollars on the Management Watch List. Table 1 provides a historical  perspective of the number of these projects and their associated budgets  since OMB started reporting on the Management Watch List in the  President’s budget request for 2004. The table shows that while the  number of projects and their associated budgets have generally decreased  since then, they increased by 239 projects and $13 billion dollars for fiscal  year 2009, and represent a significant percentage of the total budget.
As of July 2008, OMB reported that 352 of the 585 projects, representing  $23.4 billion, still remained on the Management Watch List (see appendix I  for complete list). Table 2 shows the number of projects each agency has  on the watch list as of July 2008.
According to OMB’s evaluation of the exhibit 300s, investments were  placed on the watch list primarily because of weaknesses in the way they  addressed (1) cost, schedule, and performance; (2) security; (3) privacy;  and (4) acquisition strategy. Figure 2 illustrates the frequency of these  reasons for the projects that remained on list as of July 2008. Appendix II  provides additional detail by agency.
In addition, according to OMB, thirty-two of these projects have been on  the Management Watch List since fiscal year 2006. The Departments of  Veterans Affairs and Commerce have had the most projects on the list  since then: 14 and 7, respectively. Table 3 identifies the 32 projects that  have been on the Management Watch List since fiscal year 2006.
section_title: Poorly Performing Projects Total About $4.8 Billion in Estimated Expenditures for Fiscal Year 2009
paragraphs: As of June 2008, the 24 major agencies identified 472 IT projects as high  risk, at least 87 of which had performance shortfalls collectively totaling  about $4.8 billion in funding requested for fiscal year 2009. Table 4 shows  that the number of projects increased, while the number of projects with  shortfalls decreased this year. The fact that the Department of Veterans  Affairs has not yet provided information on its number of projects with  shortfalls may be a contributing factor.
The majority of projects were not reported to have had performance  shortfalls. Further, seven agencies—the Department of Housing and Urban  Development, the Department of State, the National Aeronautics and  Space Administration, the Nuclear Regulatory Commission, the National  Science Foundation, the Small Business Administration, and the Social  Security Administration—reported that none of their high-risk projects  experienced any performance shortfalls. Figure 3 illustrates the number of  high-risk projects by agency as of June 2008, with and without shortfalls.
Agencies reported cost and schedule variances that exceeded 10 percent  as the most common shortfall. This is consistent with what they reported  about a year ago, and the distribution of shortfall types is similar to that of  last year. Figure 4 illustrates the reported number and type of performance  shortfalls associated with high-risk projects, and appendix III provides  additional details of the shortfalls associated with each of the poorly  performing projects.
Seventeen high-risk projects have experienced performance shortfalls for  the past four quarters (see figure 5).
Of these projects, two projects have had shortfalls since the list of high- risk projects was established in September 2005:    DHS’s Customs and Border Patrol Secure Border Initiative Network  Technology Program, which is expected to provide on-scene agents  near real-time information on attempted border crossings by illegal  aliens, terrorists, or smugglers.
DHS’s Transportation Security Administration Transportation Worker  Identification Credentialing, which is to establish a system-wide  common secure biometric credential, used by all transportation modes,  for personnel requiring unescorted physical and/or logical access to  secure areas of the transportation system.
section_title: Several Projects Are Both Poorly Planned and Poorly Performing
paragraphs: As of July 2008, 26 projects are on both the Management Watch List and  list of high-risk projects with shortfalls, meaning that they are both poorly  planned and poorly performing. They total about $3 billion in estimated  expenditures for fiscal year 2009. This is an increase of 5 projects but a  decrease of $1.1 billion from when we reported last year. These projects  are listed in table 5 below. (The project names were taken from OMB’s  Management Watch List released in July 2008 and matched to those in  agencies’ June 2008 quarterly high-risk reports.)
section_title: Steps Have Been Taken to Improve the Identification of Management Watch List and High Risk Projects, but Projects Totaling Billions of Dollars Still Require Oversight
paragraphs: OMB has taken steps to improve the identification of the Management  Watch List and high-risk projects since we testified last September,  including publicly disclosing reasons for placement on the Management  Watch List, and clarifying high-risk project criteria, however, more needs  to be done by both OMB and the agencies to fully address  recommendations we have previously made to improve the planning,  management, and oversight of the poorly planned and poorly performing  projects so that potentially billions in taxpayer dollars are not wasted.
Management Watch List: In order for OMB to take advantage of the  potential benefits of using the Management Watch List as a tool for  analyzing and following up on IT investments on a governmentwide basis,  in 2005 we recommended that the agency take the following four actions:  (1) develop a central list of Management Watch List projects and their  deficiencies; (2) use the list as the basis for selecting projects for follow-up  and for tracking follow-up activities (including developing specific criteria  for prioritizing the IT projects included on the list, taking into  consideration such factors as their relative potential financial and program  benefits, as well as potential risks); (3) analyze the prioritized list to  develop governmentwide and agency assessments of the progress and  risks of IT investments, identifying opportunities for continued  improvement; and (4) report to Congress on progress made in addressing  risks of major IT investments and management areas needing attention.
OMB has taken steps to address our recommendations for developing a  central list of projects and their deficiencies and developing  governmentwide and agency assessments. Specifically, as previously  noted, OMB started issuing a central list of Management Watch List  projects in September 2006, and publicly disclosing these projects’  deficiencies (i.e., the reasons for inclusion on the Management Watch List)  in April. In addition, OMB performed governmentwide and agency-specific  analyses of projects’ deficiencies in April and in July of this year, which it  reported to Congress and disclosed publicly. However, OMB needs to  continue to use the Management Watch List to prioritize the projects  needing follow-up action and to keep reporting to Congress on  management areas needing attention.
High-Risk Projects: To improve the identification and oversight of the  high-risk projects, in 2006 we recommended, among other things, that  OMB establish a structured, consistent process to update the list of high- risk projects on a regular basis, including identifying new projects and  removing previous ones to ensure that the list is current and complete.  We also recommended that OMB develop a single aggregate list of high- risk projects and their deficiencies and use that list to report to Congress  progress made in correcting high-risk problems, actions under way, and  further actions that may be needed.
OMB took several steps to address these recommendations. As previously  noted, the agency clarified the high-risk project criteria this year. It also  asked agencies to identify, in their quarterly reports, reasons for  placement on the list and reasons for removal, thereby adding structure  and consistency to the process for updating the list. In addition, as  previously reported, OMB also started publicly releasing aggregate lists of  the high-risk projects in September 2006, and has been releasing them on  their website on a quarterly basis since then. However, OMB has yet to  identify the deficiencies (i.e., performance shortfalls) associated with the  high-risk projects as we have done in this report (see appendix III). As we  have stated before, doing so would allow OMB and others to better  analyze the reasons projects are poorly performing, take corrective  actions, and track these projects on a governmentwide basis. Such  information would also help to highlight progress made by agencies or  projects, identify management issues that transcend individual agencies,  and highlight the root causes of governmentwide issues and trends.
In addition, as noted earlier, our prior reviews of federal IT management  practices have identified (and continue to identify) weaknesses at  agencies that have yet to be addressed. While these agencies have taken  action to address the many recommendations we have made to improve  their practices, more needs to be done as evidenced by the large number  of projects that are still poorly planned and poorly performing.
While the actions taken have resulted in better data on the poorly planned  and performing projects, additional steps need to be taken by both OMB  and the agencies to address recommendations we have previously made to  improve the planning, management, and oversight of these projects. These  steps include using the Management Watch List to prioritize follow up  activities. Until these additional steps are taken, potentially billion of  taxpayer dollars are at risk of being wasted.
section_title: About Half of Major IT Projects Have Been Rebaselined Using Policies that Are Not Fully Consistent with Best Practices
paragraphs: Given that cost and schedule variances are the primary reason for poorly  performing projects, having accurate and transparent cost and schedule  information is essential to effective oversight. In a report being released  today, we estimate that about 48 percent of the federal government’s  major IT projects have been rebaselined. Of those rebaselined projects, 51  percent were rebaselined at least twice, and about 11 percent were  rebaselined 4 times or more. These projects were rebaselined for several  reasons, including changes in project goals and changes in funding. While  the major agencies have all established rebaselining policies, these  policies are not comprehensive. Specifically, none of the policies are fully  consistent with best practices, including describing a process for  developing a new baseline and requiring the validation of the new  baseline, identified in our cost assessment guide. Agencies’ policies vary in  part because OMB has not issued guidance specifying what elements these  policies are to include.
subsections: section_title: About Half of IT Projects Were Rebaselined for Several Reasons
paragraphs: In our report we project that 48 percent of the major projects federal  agencies plan to fund in fiscal year 2008 have been rebaselined, and about  half of those have been rebaselined at least twice. Figure 6 summarizes the  percentage of projects rebaselined and figure 7 summarizes the estimated  frequencies of the number of times rebaselined major IT projects were  rebaselined.
Table 6 lists the nine projects in our sample that agencies reported having  been rebaselined four or more times.
Agency officials reported that the key reasons for the most recent  rebaselinings were changes in project requirements, objectives, or scope,  and changes in funding stream. Table 7 shows the estimated frequencies of  each of these reasons.
Several rebaselined projects we have performed detailed reviews of have  experienced significant cost or schedule changes. For example, the U.S.  Coast Guard’s Rescue 21 program is projected to have cost increases of  184 percent and schedule delays of 5 years after rebaselining. Table 8  provides additional examples of projects we have reviewed that  experienced significant cost or schedule changes.
section_title: Agencies’ Rebaselining Policies Are Not Comprehensive
paragraphs: We are also reporting that, although the 24 major agencies have  rebaselined about half of their major IT projects that they planned to  invest in during fiscal year 2008, they have not been guided by  comprehensive rebaselining policies. Specifically, while major agencies  have all established rebaselining policies, none of the policies are fully  consistent with best practices such as describing a process for developing  a new baseline.
Our recently issued draft Cost Assessment Guide includes five practices  that are relevant to rebaselining policies:  1.  Describe reasons when a rebaseline is warranted. A rebaselining  policy should require valid reasons for rebaselining such as that the  baseline is no longer useful as a management tool (e.g., cost/schedule  variances are so high that they lose meaning; program scope has  significantly changed).  2.  Describe the process for developing a new baseline. A rebaselining  policy should describe the development of a new cost estimate and a  new project plan that details the scope of the remaining work along  with schedule and resource allocation.  3.  Require validating the new baseline. A rebaselining policy should  identify who can validate the new baseline and how the validation is to  be done.  4.  Require management review. A rebaselining policy should identify the  authority who decides whether the rebaselining is warranted and the  rebaselining plan is acceptable. In addition, the policy should outline  decision criteria used by the decision authority to determine if the  rebaseline plan is acceptable.  5.  Require that the process is documented. A rebaselining policy should  identify and document rebaselining decisions, including the reasons  for rebaselining; changes to the approved baseline cost, schedule, and  scope; management review of the rebaseline request; and approval of  new baseline. The policy should also require an explanation of why the  current plan is no longer feasible; identify the problems that led to the  need for a new plan of the remaining work; and discuss measures in  place to prevent recurrence.
Our analysis shows that agencies do not have comprehensive rebaselining  policies. Specifically, none of the agencies’ rebaselining policies are fully  consistent with all of the five practices mentioned above. Most policies  fully or partially addressed reasons for rebaselining, requiring  management review, and requiring that the rebaselining process be  documented (79 percent, 96 percent, and 88 percent, respectively), while  describing the process for developing the new baseline and requiring  validation of the new baseline were addressed the least (46 percent and 54  percent of the policies, respectively, did not address these practices).  Table 9 summarizes our assessment of agencies’ rebaselining polices.
Agencies’ policies vary in part because no guidance has been issued  specifying what elements these policies are to include. As previously  noted, OMB has issued guidance which, among other things, requires  baseline change requests to be approved by the agency heads and to be  submitted to OMB for approval. However, this guidance does not  specifically address how agencies are to implement their rebaselining  activities, including the key elements that should be addressed in their  policies. In addition, officials from OMB’s Office of E-government and  Information Technology and the Acting Chief of OMB’s Information Policy  and Technology Branch told us that, in their oversight function, they  review agencies’ earned value management policies, and in doing so  determine whether these policies address rebaselining. However, they  noted that they have not established specific criteria to evaluate the  earned value management policies (and therefore their rebaselining  aspects) and acknowledged that having such criteria would improve  consistency among the policies and facilitate their oversight process.  Without comprehensive policies to guide their rebaselining activities,  agencies may not be optimizing the effectiveness of rebaselining as a tool  to improve performance management. In addition, their rebaselining  processes may lack the transparency needed to ensure effective oversight.
To address the weaknesses identified with agencies’ rebaselining policies,  we made recommendations to the Director of OMB and to the 24 major  agencies. Specifically, we recommended that  the Director of OMB issue guidance for rebaselining policies that  would include a minimum set of key elements, taking into  consideration the criteria used in our report, and each of the heads of the 24 major agencies direct the development of  comprehensive rebaselining policies that address the weaknesses we  identified.
We received comments on a draft of our report from 20 of the major  agencies--4 of which stated that they had no comments. Of the remaining  16 agencies, 10 generally agreed with our findings and/or  recommendations, and 6 disagreed with our assessment of certain  practices associated with their rebaselining policies.
In summary, effective management and oversight of federal IT projects  remains a crucial task for OMB and executive branch agencies. Hundreds  of projects, amounting to billions of dollars in expenditures, have been  identified as poorly managed, poorly performing, or both. While OMB has  taken steps to improve the identification of poorly managed and poorly  performing projects, more needs to be done to improve management and  oversight, as evidenced by the number of recurring Management Watch  List projects and the surge of these projects at the beginning of every fiscal  year.
In addition, without sound policies guiding agencies’ rebaselining efforts,  changes to projects’ cost and schedule goals are not as transparent as  desired and may in fact mask cost overruns and schedule delays. Having  sound rebaselining guidance from OMB and more diligent oversight of  rebaselining efforts from federal agencies will result in more accurate  information on cost and schedule performance shortfalls and provide the  necessary transparency to agency officials, OMB, and other oversight  organizations. As we transition to a new administration, it is essential to  maintain the current momentum of identifying troubled projects and the  reasons they are poorly planned and/or performing and to continue to  focus attention more keenly on solutions and long-term improvement  efforts.
Mr. Chairman, this concludes my statement. I would be happy to answer  any questions at this time.
section_title: Appendix I: GAO Contact and Staff Acknowledgements
paragraphs: If you should have any questions about this testimony, please contact me  at (202) 512-9286 or by e-mail at pownerd@gao.gov. Individuals who made  key contributions to this testimony are Sabine Paul, Assistant Director;  Neil Doherty; Lee McCracken; Kevin Walsh and Eric Winter.
section_title: Appendix II: Management Watch List Projects
paragraphs: The following provides additional detail on the investments comprising  OMB’s Management Watch List as of July 2008. The project names were  taken from OMB’s Management Watch List released earlier this month and  matched to the list of projects in OMB’s Report on IT Spending for Fiscal  Years 2007, 2008, and 2009 to derive the requested amounts for fiscal  year 2009.
section_title: Appendix II: Reasons for Inclusion on the Management Watch List by Agency
paragraphs: Table 11 provides additional detail on the frequency of the reasons for  inclusion on the Management Watch List for each agency for the projects  remaining on the list as of July 2008. It shows security and cost and  schedule performance as being the most common reasons.
section_title: Appendix III: High-Risk Projects with Shortfalls
paragraphs: Table 12 provides additional detail on the high-risk projects that have  performance shortfalls as of June 2008. These shortfalls were identified by  agencies in June 2008 high-risk reports. (The Department of Veterans  Affairs has not yet provided its report.) The project names were taken  from OMB’s list of Management Watch List released earlier this month and  matched to the list of projects on agencies’ June 2008 high-risk reports.