Information Technology: Implementing Best Practices and Reform Initiatives Can Help Improve the Management of Investments
Released: 2014-05-08T00:00:00

Letter

Background

Information technology should enable government to better serve the American people. However, despite spending hundreds of billions on IT since 2000, the federal government has experienced failed IT projects and has achieved little of the productivity improvements that private industry has realized from IT. Too often, federal IT projects run over budget, behind schedule, or fail to deliver results. In combating this problem, proper oversight is critical.

Both OMB and federal agencies have key roles and responsibilities for overseeing IT investment management, and OMB is responsible for working with agencies to ensure investments are appropriately planned and justified. However, as we have described in numerous reports, although a variety of best practices exist to guide their successful acquisition, federal IT projects too frequently incur cost overruns and schedule slippages while contributing little to mission-related outcomes.

Agencies have reported that poor-performing projects have often used a “big bang” approach—that is, projects that are broadly scoped and aim to deliver capability several years after initiation. For example, in 2009 the Defense Science Board reported that the Department of Defense’s (Defense) acquisition process for IT systems was too long, ineffective, and did not accommodate the rapid evolution of IT. The board reported that the average time to deliver an initial program capability for a major IT system acquisition at Defense was over 7 years.

Each year, OMB and federal agencies work together to determine how much the government plans to spend on IT projects and how these funds are to be allocated. As reported to OMB, federal agencies plan to spend more than $82 billion on IT investments in fiscal year 2014, which is the total expended for not only acquiring such investments, but also the funding to operate and maintain them. Of the reported amount, 27 federal agencies plan to spend about $75 billion: $17 billion on development and acquisition and $58 billion on operations and maintenance (O&M). Figure 1 shows the percentages of total planned spending for 2014 for the $75 billion spent on development and O&M.

However, this $75 billion does not reflect the spending of the entire federal government. We have previously reported that OMB’s figure understates the total amount spent in IT investments. Specifically, it does not include IT investments by 58 independent executive branch agencies, including the Central Intelligence Agency or by the legislative or judicial branches. Further, agencies differed on what they considered an IT investment; for example, some have considered research and development systems as IT investments, while others have not. As a result, not all IT investments are included in the federal government’s estimate of annual IT spending. OMB provided guidance to agencies on how to report on their IT investments, but this guidance did not ensure complete reporting or facilitate the identification of duplicative investments. Consequently, we recommended, among other things, that OMB improve its guidance to agencies on identifying and categorizing IT investments.

Further, over the past several years, we have reported that overlap and fragmentation among government programs or activities could be harbingers of unnecessary duplication. Thus, the reduction or elimination of duplication, overlap, or fragmentation could potentially save billions of tax dollars annually and help agencies provide more efficient and effective services.

OMB Has Launched Major Initiatives for Overseeing Investments

OMB has implemented a series of initiatives to improve the oversight of underperforming investments, more effectively manage IT, and address duplicative investments. These efforts include the following: IT Dashboard. Given the importance of transparency, oversight, and management of the government’s IT investments, in June 2009, OMB established a public website, referred to as the IT Dashboard, that provides detailed information on 760 major IT investments at 27 federal agencies, including ratings of their performance against cost and schedule targets. The public dissemination of this information is intended to allow OMB; other oversight bodies, including Congress; and the general public to hold agencies accountable for results and performance. Among other things, agencies are to submit Chief Information Officer (CIO) ratings, which, according to OMB’s instructions, should reflect the level of risk facing an investment on a scale from 1 (high risk) to 5 (low risk) relative to that investment’s ability to accomplish its goals. Ultimately, CIO ratings are assigned colors for presentation on the Dashboard, according to the five-point rating scale, as illustrated in table 1.

As of April 2014, according to the IT Dashboard, 201 of the federal government’s 760 major IT investments—totaling $12.4 billion—were in need of management attention (rated “yellow” to indicate the need for attention or “red” to indicate significant concerns). (See fig. 2.)

TechStat reviews. In January 2010, the Federal CIO began leading TechStat sessions—face-to-face meetings to terminate or turnaround IT investments that are failing or are not producing results. These meetings involve OMB and agency leadership and are intended to increase accountability and transparency and improve performance. Subsequently, OMB empowered agency CIOs to hold their own TechStat sessions within their respective agencies. According to the former Federal CIO, the efforts of OMB and federal agencies to improve management and oversight of IT investments have resulted in almost $4 billion in savings.

Federal Data Center Consolidation Initiative. Concerned about the growing number of federal data centers, in February 2010 the Federal CIO established the Federal Data Center Consolidation Initiative. This initiative’s four high-level goals are to promote the use of “green IT” by reducing the overall energy and real estate footprint of government data centers; reduce the cost of data center hardware, software, and operations; increase the overall IT security posture of the government; and shift IT investments to more efficient computing platforms and technologies. OMB believes that this initiative has the potential to provide about $3 billion in savings by the end of 2015.

IT Reform Plan. In December 2010, OMB released its 25-point plan to reform federal IT. This document established an ambitious plan for achieving operational efficiencies and effectively managing large- scale IT programs. In particular, as part of an effort to reduce the risk associated with IT acquisitions, the plan calls for federal IT programs to deploy capabilities or functionality in release cycles no longer than 12 months, and ideally, less than 6 months. The plan also identifies key actions that can help agencies implement this incremental development guidance, such as working with Congress to develop IT budget models that align with incremental development and issuing contracting guidance and templates to support incremental development.

PortfolioStat. In order to eliminate duplication, move to shared services, and improve portfolio management processes, in March 2012, OMB launched the PortfolioStat initiative. Specifically, PortfolioStat requires agencies to conduct an annual agency-wide IT portfolio review to, among other things, reduce commodity IT spending and demonstrate how their IT investments align with the agency’s mission and business functions. PortfolioStat is designed to assist agencies in assessing the current maturity of their IT investment management process, making decisions on eliminating duplicative investments, and moving to shared solutions in order to maximize the return on IT investments across the portfolio. OMB believes that the PortfolioStat effort has the potential to save the government $2.5 billion over the next 3 years by, for example, consolidating duplicative systems.

Opportunities Exist to Improve Acquisition and Management of IT Investments

Given the magnitude of the federal government’s annual IT budget, which is expected to be more than $82 billion in fiscal year 2014, it is important that agencies leverage all available opportunities to ensure that their IT investments are acquired in the most effective manner possible. To do so, agencies can rely on IT acquisition best practices, incremental development, and initiatives such as OMB’s IT Dashboard and OMB- mandated TechStat sessions. Additionally, agencies can save billions of dollars by continuing to consolidate federal data centers and by eliminating duplicative investments through OMB’s PortfolioStat initiative.

Best Practices Are Intended to Help Ensure Successful Major Acquisitions In 2011, we identified seven successful acquisitions and nine common factors critical to their success and noted that (1) the factors support OMB’s objective of improving the management of large-scale IT acquisitions across the federal government and (2) wide dissemination of these factors could complement OMB’s efforts. Specifically, we reported that federal agency officials identified seven successful acquisitions, in that they best achieved their respective cost, schedule, scope, and performance goals. Notably, all of these were smaller increments, phases, or releases of larger projects. The common factors critical to the success of three or more of the seven acquisitions are generally consistent with those developed by private industry and are identified in table 2.

These critical factors support OMB’s objective of improving the management of large-scale IT acquisitions across the federal government; wide dissemination of these factors could complement OMB’s efforts.

IT Dashboard Can Improve the Transparency into and Oversight of Major IT Investments The IT Dashboard serves an important role in allowing OMB and other oversight bodies to hold agencies accountable for results and performance. However, we have issued a series of reports highlighting deficiencies with the accuracy and reliability of the data reported on the Dashboard. For example, we reported in October 2012 that Defense had not rated any of its investments as either high or moderately high risk and that, in selected cases, these ratings did not appropriately reflect significant cost, schedule, and performance issues reported by GAO and others. We recommended that Defense ensure that its CIO ratings reflect available investment performance assessments and its risk management guidance. Defense concurred and has revised its process to address these concerns.

Further, while we reported in 2011 that the accuracy of Dashboard cost and schedule data had improved over time, more recently, in December 2013 we found that agencies had removed investments from the Dashboard by reclassifying their investments—representing a troubling trend toward decreased transparency and accountability. Specifically, the Department of Energy reclassified several of its supercomputer investments from IT to facilities and the Department of Commerce decided to reclassify its satellite ground system investments. Additionally, as of December 2013, the public version of the Dashboard was not updated for 15 of the previous 24 months because OMB does not revise it as the President’s budget request is being created.

We also found that, while agencies experienced several issues with reporting the risk of their investments, such as technical problems and delayed updates to the Dashboard, the CIO ratings were mostly or completely consistent with investment risk at seven of the eight selected agencies. Additionally, the agencies had already addressed several of the discrepancies that we identified. The final agency, the Department of Veterans Affairs (VA), did not update 7 of its 10 selected investments because it elected to build, rather than buy, the ability to automatically update the Dashboard and has now resumed updating all investments. To their credit, agencies’ continued attention to reporting the risk of their major IT investments supports the Dashboard’s goal of providing transparency and oversight of federal IT investments.

Nevertheless, the rating issues that we identified with performance reporting and annual baselining, some of which are now corrected, serve to highlight the need for agencies’ continued attention to the timeliness and accuracy of submitted information in order to allow the Dashboard to continue to fulfill its stated purpose. We recommended that agencies appropriately categorize IT investments and that OMB make Dashboard information available independent of the budget process. OMB neither agreed nor disagreed with these recommendations. Six agencies generally agreed with the report or had no comments and two others did not agree, believing their categorizations were appropriate. We continue to believe that our recommendations are valid.

Agencies Need to Establish and Implement Incremental Development Policies to Better Achieve Cost, Schedule, and Performance Goals for IT Investments Incremental development can help agencies to effectively manage IT acquisitions and, as such, OMB has recently placed a renewed emphasis on it. In particular, in 2010 OMB called for IT investments to deliver functionality every 12 months, and since 2012 has required investments to deliver functionality every 6 months.

However, as discussed in our report being released today, most selected agencies have not effectively established and implemented incremental development approaches. Specifically, although all five agencies in our review—the Departments of Defense, Health and Human Services (HHS), Homeland Security (DHS), Transportation (Transportation), and VA—have established policies that address incremental development, the policies usually did not fully address three key components we identified for implementing OMB’s guidance. Table 3 provides an assessment of each agency’s policies against the three key components of an incremental development policy.

Among other things, agencies cited the following reasons that contributed to these weaknesses: (1) OMB’s guidance was not feasible because not all types of investments should deliver functionality in 6 months and (2) the guidance did not identify what agencies’ policies are to include or time frames for completion. We agreed that these concerns have merit.

Additionally, the weaknesses in agency policies have enabled inconsistent implementation of incremental development approaches. Specifically, almost three-quarters of the selected investments we reviewed did not plan to deliver functionality every 6 months and less than half planned to deliver functionality in 12-month cycles. Table 4 shows how many of the selected investments at each agency planned on delivering functionality every 6 and 12 months during fiscal years 2013 and 2014.

Considering agencies’ concerns about delivering functionality every 6 months and given that so few are planning to deliver functionality in that time frame, our report noted that delivering functionality every 12 months, consistent with OMB’s IT Reform Plan, would be an appropriate starting point and a substantial improvement. Until OMB issues realistic and clear guidance and agencies update their policies to reflect this guidance, agencies may not consistently adopt incremental development approaches, and IT expenditures will continue to produce disappointing results—including sizable cost overruns and schedule slippages and questionable progress in meeting mission goals and outcomes. In the report being released today, we recommend that OMB develop and issue realistic and clear guidance on incremental development, and that Defense, HHS, DHS, and Transportation update and implement their incremental development policies, once OMB’s guidance is made available. OMB stated that it agreed with our recommendation to update and issue incremental development guidance, but did not agree that its current guidance is not realistic. However, slightly more than one-fourth of selected investments planned to deliver functionality every 6 months— and less than one-half planned to do so every 12 months. Additionally, there are three types of investments for which it may not always be practical or necessary to expect functionality to be delivered in 6-month cycles. Thus, we continue to believe that delivering functionality every 6 months is not an appropriate requirement for all agencies and that requiring the delivery of functionality every 12 months, consistent with OMB’s IT Reform Plan, is a more appropriate starting point. We therefore maintain that OMB should require projects associated with major IT investments to deliver functionality at least every 12 months.

Four agencies—Defense, HHS, DHS, VA—generally agreed with the report or had no comments and one agency—Transportation—did not agree that its recommendation should be dependent on OMB first taking action. Specifically, the department explained that relying on another agency to concur with one of our recommendations before Transportation can take action leaves the department with the potential challenge of a recommendation that cannot be implemented. However, as previously stated, OMB agrees with our recommendation to update and issue incremental guidance, meaning that OMB has committed to taking the actions necessary to enable Transportation to begin addressing our recommendation. Accordingly, we continue to believe that our recommendations are warranted and can be implemented.

TechStat Reviews Can Help Highlight and Evaluate Poorly Performing Investments TechStat reviews were initiated by OMB to enable the federal government to turnaround, halt, or terminate IT projects that are failing or are not producing results. In 2013, we reported that OMB and selected agencies had held multiple TechStats, but that additional OMB oversight was needed to ensure that these meetings were having the appropriate impact on underperforming projects and that resulting cost savings were valid. Specifically, we determined that, as of April 2013, OMB reported conducting 79 TechStats, which focused on 55 investments at 23 federal agencies. Further, 4 selected agencies—the Departments of Agriculture, Commerce, HHS, and DHS—conducted 37 TechStats covering 28 investments. About 70 percent of the OMB-led and 76 percent of agency- led TechStats on major investments were considered medium to high risk at the time of the TechStat.

However, the number of at-risk TechStats held was relatively small compared to the current number of medium- and high-risk major IT investments. Specifically, the OMB-led TechStats represented roughly 18.5 percent of the investments across the government that had a medium- or high-risk CIO rating. For the 4 selected agencies, the number of TechStats represented about 33 percent of the investments that have a medium- or high-risk CIO rating. We concluded that, until OMB and agencies develop plans to address these weaknesses, the investments would likely remain at risk.

In addition, we reported that OMB and selected agencies had tracked and reported positive results from TechStats, with most resulting in improved governance. Agencies also reported projects with accelerated delivery, reduced scope, or termination. We also found that OMB reported in 2011 that federal agencies achieved almost $4 billion in life-cycle cost savings as a result of TechStat sessions. However, we were unable to validate OMB’s reported results because OMB did not provide artifacts showing that it ensured the results were valid. Among other things, we recommended that OMB require agencies to report on how they validated the outcomes. OMB generally agreed with this recommendation.

Continued Oversight Needed to Consolidate Federal Data Centers and Achieve Cost Savings In an effort to consolidate the growing number of federal data centers, in 2010, OMB launched a consolidation initiative intended to close 40 percent of government data centers by 2015, and, in doing so, save $3 billion. Since 2011, we have issued a series of reports on the efforts of agencies to consolidate their data centers. For example, in July 2011 and July 2012, we reported that agencies had developed plans to consolidate data centers; however, these plans were incomplete and did not include best practices. In addition, although we reported that agencies had made progress on their data center closures, OMB had not determined initiative-wide cost savings, and oversight of the initiative was not being performed in all key areas. Among other things, we recommended that OMB track and report on key performance measures, such as cost savings to date, and improve the execution of important oversight responsibilities. We also recommended that agencies complete inventories and plans. OMB agreed with these two recommendations, and most agencies agreed with our recommendations to them.

Additionally, as part of ongoing follow-up work, we have determined that while agencies had closed data centers, the number of federal data centers was significantly higher than previously estimated by OMB.

Specifically, as of May 2013, agencies had reported closing 484 data centers by the end of April 2013 and were planning to close an additional 571 data centers—for a total of 1,055—by September 2014. However, as of July 2013, 22 of the 24 agencies participating in the initiative had collectively reported 6,836 data centers in their inventories— approximately 3,700 data centers more than OMB’s previous estimate from December 2011. This dramatic increase in the count of data centers highlights the need for continued oversight of agencies’ consolidation efforts.

We have ongoing work looking at OMB’s data center consolidation initiative, including evaluating the extent to which agencies have achieved planned cost savings through their consolidation efforts, identifying agencies’ notable consolidation successes and challenges in achieving cost savings, and evaluating the extent to which data center optimization metrics have been established.

Agencies’ PortfolioStat Efforts Have the Potential to Save Billions of Dollars OMB launched the PortfolioStat initiative in March 2012, which required 26 executive agencies to, among other things, reduce commodity IT spending and demonstrate how their IT investments align with the agencies’ mission and business functions. In November 2013, we reported on agencies’ efforts to complete key required PortfolioStat actions and make portfolio improvements. We noted that all 26 agencies that were required to implement the PortfolioStat initiative took actions to address OMB’s requirements. However, there were shortcomings in their implementation of selected requirements, such as addressing all required elements of an action plan to consolidate commodity IT and migrating two commodity areas to a shared service by the end of 2012. In addition, several agencies had weaknesses in selected areas such as the CIO’s authority to review and approve the entire portfolio and ensuring a complete baseline of information relative to commodity IT. Further, we observed that OMB’s estimate of about 100 consolidation opportunities and a potential $2.5 billion in savings from the PortfolioStat initiative was understated because, among other things, it did not include estimates from Defense and the Department of Justice. Our analysis, which included these estimates, showed that, collectively, the 26 agencies reported about 200 opportunities and at least $5.8 billion in potential savings through fiscal year 2015—at least $3.3 billion more than the number initially reported by OMB.

In March 2013, OMB issued a memorandum commencing the second iteration of its PortfolioStat initiative. This memorandum identified a number of improvements that should help strengthen IT portfolio management and address key issues we have identified. However, we concluded that selected OMB efforts could be strengthened to improve the PortfolioStat initiative and ensure agencies achieve identified cost savings, including addressing issues related to existing CIO authority at federal agencies and publicly reporting on agency-provided data. We recommended, among other things, that OMB require agencies to fully disclose limitations with respect to CIO authority. In addition, we made several recommendations to improve agencies’ implementation of PortfolioStat requirements. OMB partially agreed with these recommendations, and responses from 20 of the agencies commenting on the report varied.

We have ongoing work looking at the second iteration of OMB’s PortfolioStat initiative, including identifying action items and associated time frames from joint OMB-agency PortfolioStat meetings, determining agencies’ progress in addressing these action items, and evaluating the extent to which agencies have realized planned savings.

In summary, OMB’s and agencies’ recent efforts have resulted in greater transparency and oversight of federal spending, but continued leadership and attention are necessary to build on the progress that has been made.

The expanded use of the common factors critical to the successful management of large-scale IT acquisitions should result in more effective delivery of mission-critical systems. Additionally, federal agencies need to continue to improve the accuracy and availability of information on the Dashboard to provide greater transparency and even more attention to the billions of dollars invested in troubled projects. Further, agencies need to implement incremental development approaches in order to increase the likelihood that major IT investments meet their cost, schedule, and performance goals. Additionally, agencies should conduct additional TechStat reviews to focus management attention on troubled projects and establish clear action items to turn the projects around or terminate them.

The federal government can also build on the progress of agencies’ data center closures and reduction of commodity IT. With the possibility of over $5.8 billion in savings from the data center consolidation and PortfolioStat initiatives, agencies should continue to identify consolidation opportunities in both data centers and commodity IT. In addition, better support for the estimates of cost savings associated with the opportunities identified would increase the likelihood that these savings will be achieved.

Chairman Carper, Ranking Member Coburn, and Members of the Committee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contact and Staff Acknowledgments

If you or your staffs have any questions about this testimony, please contact me at (202) 512-9286 or at pownerd@gao.gov. Individuals who made key contributions to this testimony are Dave Hinchman (Assistant Director), Deborah A. Davis (Assistant Director), Rebecca Eyler, Kaelin Kuhn, Meredith Raymond, Jamelyn Payan, Bradley Roach, Andrew Stavisky, and Kevin Walsh.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.