Information Technology: Leveraging Best Practices to Help Ensure Successful Major Acquisitions
Published: 2013-11-13T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The federal government reportedly plans to spend at least $82 billion on IT in fiscal year 2014. Given the scale of such planned outlays and the criticality of many of these systems to the health, economy, and security of the nation, it is important that federal agencies successfully acquire these systemsthat is, ensure that the systems are acquired on time and within budget and that they deliver the expected benefits and functionality.
However, GAO has previously reported and testified that federal IT projects too frequently incur cost overruns and schedule slippages while contributing little to mission-related outcomes. To help improve these efforts, OMB has launched several initiatives intended to improve the oversight and management of IT acquisitions. In addition, during the past several years GAO has issued multiple reports and testimonies on federal initiatives to acquire and improve the management of IT investments.
As discussed with committee staff, GAO is testifying today on IT best practices, with a focus on the results of its report issued on the critical success factors of major IT acquisitions. To prepare this statement, GAO drew on previously published work.
section_title: What GAO Found
paragraphs: Information technology (IT) acquisition best practices have been developed by both industry and the federal government. For example, the Software Engineering Institute has developed highly regarded and widely used guidance on best practices, such as requirements development and management, risk management, validation and verification, and project monitoring and control. GAO's own research in IT management best practices led to the development of the Information Technology Investment Management Framework, which describes essential and complementary IT investment management disciplines, such as oversight of system development and acquisition management, and organizes them into a set of critical processes for successful investments.
GAO also recently reported on the critical factors underlying successful IT acquisitions. Officials from federal agencies identified seven investments that were deemed successfully acquired in that they best achieved their respective cost, schedule, scope, and performance goals. Agency officials identified nine common factors that were critical to the success of three or more of the seven investments.
Officials from all seven investments cited active engagement with program stakeholders as a critical factor to the success of those investments. Agency officials stated that stakeholders regularly attended program management office sponsored meetings; were working members of integrated project teams; and were notified of problems and concerns as soon as possible.
Additionally, officials from six investments indicated that knowledge and skills of the program staff, and support from senior department and agency executives were critical to the success of their programs. Further, officials from five of the seven selected investments identified having the end users test and validate the system components prior to formal acceptance testing for deployment as critical to the success of their program. These critical factors support the Office of Management and Budget's (OMB) objective of improving the management of large-scale IT acquisitions across the federal government; wide dissemination of these factors could complement OMB's efforts.
section_title: What GAO Recommends
paragraphs: GAO has made numerous recommendations to OMB and agencies on key aspects of IT acquisition management, as well as the oversight and management of those investments.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the importance of key aspects of  the federal government’s acquisition of information technology (IT)  investments. As reported to the Office of Management and Budget  (OMB), federal agencies plan to spend at least $82 billion on IT in fiscal  year 2014. Given the size of these investments and the criticality of many  of these systems to the health, economy, and security of the nation, it is  important that federal agencies successfully acquire these systems—that  is, ensure that the systems are acquired on time and within budget, and  that they deliver the expected benefits and functionality.
However, as we have previously reported and testified, federal IT projects  too frequently incur cost overruns and schedule slippages while  contributing little to mission-related outcomes. During the past several  years, we have issued multiple reports and testimonies on federal  initiatives to acquire and improve the management of IT investments. In  those reports, we made numerous recommendations to federal agencies  and OMB to further enhance the management and oversight of IT  programs.
As part of its response to our prior work, OMB deployed a public website  in 2009, known as the IT Dashboard, which provides detailed information  on federal agencies’ major IT investments, including assessments of  actual performance against cost and schedule targets (referred to as  ratings) for approximately 700 major federal IT investments. In addition,  OMB has initiated other significant efforts following the creation of the  Dashboard. For example, OMB began leading reviews—known as  TechStat Accountability Sessions (TechStats)—of selected IT  investments to increase accountability and improve performance. Further,  in 2011 we reported on the critical factors underlying successful federal  major IT acquisitions. In that report, we identified seven successful  investment acquisitions and nine common factors critical to their success.
As discussed with committee staff, I am testifying today on IT acquisition  best practices, with a focus on the results of our report on the critical  success factors of major IT acquisitions. Accordingly, my testimony  specifically focuses on those success factors and their importance to  improving IT investment oversight and management. I will also address  several initiatives put into place by OMB to address the transparency of IT  investments and to review troubled projects. All work on which this  testimony is based was performed in accordance with all sections of  GAO’s Quality Assurance Framework that were relevant to our objectives.  The framework requires that we plan and perform the engagement to  obtain sufficient and appropriate evidence to meet our stated objectives  and to discuss any limitations in our work. We believe that the information  and data obtained, and the analysis conducted, provide a reasonable  basis for any findings and conclusions in this product. A more detailed  discussion of the objectives, scope, and methodology of this work is  included in each of the reports on which this testimony is based.
subsections: section_title: Background
paragraphs: Information technology should enable government to better serve the  American people. However, according to OMB, despite spending more  than $600 billion on IT over the past decade, the federal government has  achieved little of the productivity improvements that private industry has  realized from IT. Too often, federal IT projects run over budget, behind  schedule, or fail to deliver promised functionality. In combating this  problem, proper oversight is critical.
Both OMB and federal agencies have key roles and responsibilities for  overseeing IT investment management and OMB is responsible for  working with agencies to ensure investments are appropriately planned  and justified. However, as we have described in numerous reports,  although a variety of best practice documentation exists to guide their  successful acquisition, federal IT projects too frequently incur cost  overruns and schedule slippages while contributing little to mission- related outcomes.
subsections: section_title: IT Acquisition Best Practices Have Been Identified by Industry and Government and Promoted by Legislation
paragraphs: IT acquisition best practices have been developed by both industry and  the federal government. For example, the Software Engineering Institute  has developed highly regarded and widely used guidance on best  practices, such as requirements development and management, risk  management, configuration management, validation and verification, and  project monitoring and control. This guidance also describes disciplined  project management practices that call for the development of project  details, such as objectives, scope of work, schedules, costs, and  requirements against which projects can be managed and executed. In  the federal government, GAO’s own research in IT management best  practices led to the development of the Information Technology  Investment Management Framework, which describes essential and  complementary IT investment management disciplines, such as oversight  of system development and acquisition management, and organizes them  into a set of critical processes for successful investments. This guidance  further describes five progressive stages of maturity that an agency can  achieve in its investment management capabilities, and was developed  on the basis of our research into the IT investment management practices  of leading private- and public-sector organizations. GAO has also  identified opportunities to improve the role played by Chief Information  Officers (CIO) in IT management. In noting that federal law provides  CIOs with adequate authority to manage IT for their agencies, GAO also  reported on limitations that impeded their ability to exercise this authority.  Specifically, CIOs have not always had sufficient control over IT  investments; more consistent implementation of CIOs’ authority could  enhance their effectiveness.
Congress has also enacted legislation that reflects IT management best  practices. For example, the Clinger-Cohen Act of 1996, which was  informed by GAO best practice recommendations, requires federal  agencies to focus more on the results they have achieved through IT  investments, while concurrently improving their IT acquisition processes.  Specifically, the act requires agency heads to implement a process to  maximize the value of the agency’s IT investments and assess, manage,  and evaluate the risks of its IT acquisitions. Further, the act establishes  CIOs to advise and assist agency heads in carrying out these  responsibilities. The act also requires OMB to encourage agencies to  develop and use best practices in IT acquisition.
Additionally, the E-Government Act of 2002 established a CIO Council,  which is led by the Federal CIO, to be the principal interagency forum for  improving agency practices related to the development, acquisition, and  management of information resources, including sharing best practices.
Although these best practices and legislation can have a positive impact  on major IT programs, we have previously testified that the federal  government continues to invest in numerous failed and troubled  projects. We stated that while OMB’s and agencies’ recent efforts had  resulted in greater transparency and oversight of federal spending,  continued leadership and attention was necessary to build on the  progress that had been made.
In an effort to end the recurring cycle of failed IT projects, this committee  has introduced legislation to improve IT acquisition management.  Among other things, this legislation would eliminate duplication and waste in IT acquisition, and increase the authority of agency CIOs,  strengthen and streamline IT acquisition management practices.
We have previously testified in support of this legislation.
section_title: OMB Has Several Initiatives Under Way to Improve the Oversight and Management of IT Investments
paragraphs: OMB plays a key role in helping federal agencies manage their  investments by working with them to better plan, justify, and determine  how much they need to spend on projects and how to manage approved  projects.
In June 2009, OMB established the IT Dashboard to improve the  transparency into and oversight of agencies’ IT investments. According to  OMB officials, agency CIOs are required to update each major investment  in the IT Dashboard with a rating based on the CIO’s evaluation of certain  aspects of the investment, such as risk management, requirements  management, contractor oversight, and human capital. According to  OMB, these data are intended to provide a near real-time perspective of  the performance of these investments, as well as a historical perspective.  Further, the public display of these data is intended to allow OMB,  congressional and other oversight bodies, and the general public to hold  government agencies accountable for results and progress.
In January 2010, the Federal CIO began leading TechStat sessions— reviews of selected IT investments between OMB and agency  leadership—to increase accountability and transparency and improve  performance. OMB has identified factors that may result in an investment  being selected for a TechStat session, such as—but not limited to— evidence of (1) poor performance; (2) duplication with other systems or  projects; (3) unmitigated risks; and (4) misalignment with policies and  best practices.
OMB reported that as of April 2013, 79 TechStat sessions had been held  with federal agencies. According to OMB, these sessions enabled the  government to improve or terminate IT investments that were  experiencing performance problems. For example, in June 2010 the  Federal CIO led a TechStat on the National Archives and Records  Administration’s (NARA) Electronic Records Archives investment that  resulted in six corrective actions, including halting fiscal year 2012  development funding pending the completion of a strategic plan. Similarly,  in January 2011, we reported that NARA had not been positioned to  identify potential cost and schedule problems early, and had not been  able to take timely actions to correct problems, delays, and cost increases  on this system acquisition program. Moreover, we estimated that the  program would likely overrun costs by between $205 and $405 million if  the agency completed the program as originally designed. We made  multiple recommendations to the Archivist of the United States, including  establishing a comprehensive plan for all remaining work, improving the  accuracy of key performance reports, and engaging executive leadership  in correcting negative performance trends.
Drawing on the visibility into federal IT investments provided by the IT  Dashboard and TechStat sessions, in December 2010, OMB issued a  plan to reform IT management throughout the federal government over an  18-month time frame. Among other things, the plan noted the goal of  turning around or terminating at least one-third of underperforming  projects by June 2012. The plan contained two high-level objectives: achieving operational efficiency, and effectively managing large-scale IT programs.
To achieve operational efficiencies, the plan outlined actions required to  adopt cloud solutions and leverage shared services. To effectively  manage IT acquisitions, the plan identified key actions, such as improving  accountability and governance and aligning acquisition processes with  the technology cycle. Our April 2012 report on the federal government’s  progress on implementing the plan found that not all action items had  been completed. These findings are discussed in greater detail later in  the next section.
section_title: Prior GAO Work Has Reported on OMB’s IT Investment Oversight and Management
paragraphs: We have previously reported that OMB has taken significant steps to  enhance the oversight, transparency, and accountability of federal IT  investments by creating its IT Dashboard, by improving the accuracy of  investment ratings, and by creating a plan to reform federal IT. However,  we also found issues with the accuracy and data reliability of cost and  schedule data, and recommended steps that OMB should take to improve  these data.
In July 2010, we reported that the cost and schedule ratings on  OMB’s Dashboard were not always accurate for the investments we  reviewed, because these ratings did not take into consideration  current performance. As a result, the ratings were based on outdated  information. We recommended that OMB report on its planned  changes to the Dashboard to improve the accuracy of performance  information and provide guidance to agencies to standardize  milestone reporting. OMB agreed with our recommendations and, as  a result, updated the Dashboard’s cost and schedule calculations to  include both ongoing and completed activities.
Similarly, our report in March 2011 noted that OMB had initiated  several efforts to increase the Dashboard’s value as an oversight tool  and had used its data to improve federal IT management. However,  we also reported that agency practices and the Dashboard’s  calculations contributed to inaccuracies in the reported investment  performance data. For instance, we found missing data submissions  or erroneous data at each of the five agencies we reviewed, along  with instances of inconsistent program baselines and unreliable  source data. As a result, we recommended that the agencies take  steps to improve the accuracy and reliability of their Dashboard  information, and that OMB improve how it rates investments relative  to current performance and schedule variance. Most agencies  generally concurred with our recommendations and three have taken  steps to address them. OMB agreed with our recommendation for  improving ratings for schedule variance. It disagreed with our  recommendation to improve how it reflects current performance in  cost and schedule ratings, but more recently made changes to  Dashboard calculations to address this while also noting challenges in  comprehensively evaluating cost and schedule data for these  investments.
Subsequently, in November 2011, we noted that the accuracy of  investment cost and schedule ratings had improved since our July  2010 report because OMB refined the Dashboard’s cost and schedule  calculations. Most of the ratings for the eight investments we reviewed  as part of our November 2011 report were accurate, although we  noted that more could be done to inform oversight and decision  making by emphasizing recent performance in the ratings. We  recommended that the General Services Administration comply with  OMB’s guidance for updating its ratings when new information  becomes available (including when investments are rebaselined). The  agency concurred and has since taken actions to address this  recommendation. Since we previously recommended that OMB  improve how it rates investments, we did not make any further  recommendations.
Further, in April 2012, we reported that OMB and key federal  agencies had made progress on implementing actions items from its  plan to reform IT management, but found that there were several  areas where more remained to be done. Specifically, we reviewed 10  actions and found that 3 were complete, while 7 were incomplete. For  example, we found that OMB had reformed and strengthened  investment review boards, but had only partially issued guidance on  modular development. Accordingly, we recommended, among other  things, that OMB ensure that the action items called for in the plan be  completed by the responsible parties prior to the completion of the  plan’s 18-month deadline of June 2012, or if the June 2012 deadline  could not be met, by another clearly defined deadline. OMB agreed to  complete the key action items.
Finally, we reviewed OMB’s efforts to help agencies address IT  projects with cost overruns, schedule delays, and performance  shortfalls in June 2013. In particular, we reported that OMB used  CIO ratings from the Dashboard, among other sources, to select at- risk investments for reviews known as TechStats. OMB initiated  these reviews in January 2010 to further improve investment  performance, and subsequently incorporated the TechStat model into  its plan for reforming IT management. We reported that OMB and  selected agencies had held multiple TechStat sessions but additional  OMB oversight was needed to ensure that these meetings were  having the appropriate impact on underperforming projects and that  resulting cost savings were valid. Among other things, we  recommended that OMB require agencies to address their highest- risk investments and to report on how they validated the outcomes.  OMB generally agreed with our recommendations, and stated that it  and the agencies were taking appropriate steps to address them.
section_title: Critical Factors Underlying Successful Major Acquisitions
paragraphs: Subsequent to the launch of the Dashboard and the TechStat reviews,  and to help the federal agencies address the well-documented acquisition  challenges they face, in 2011, we reported on nine common factors  critical to the success of IT investment acquisitions. Specifically, we  reported that department officials from seven agencies each identified a  successful investment acquisition, in that they best achieved their  respective cost, schedule, scope, and performance goals. To identify  these investments, we interviewed officials from the 10 departments with  the largest planned IT budgets in order for each department to identify  one mission-critical, major IT investment that best achieved its cost,  schedule, scope, and performance goals. Of the 10 departments, 7  identified successful IT investments, for a total of 7 investments.  Officials from the 7 investments cited a number of success factors that  contributed to these investments’ success.
According to federal department officials, the following seven investments  (shown in table 1) best achieved their respective cost, schedule, scope,  and performance goals. The estimated total life-cycle cost of the seven  investments is about $5 billion.
Among these seven IT investments, officials identified nine factors as  critical to the success of three or more of the seven. The factors most  commonly identified include active engagement of stakeholders, program  staff with the necessary knowledge and skills, and senior department and  agency executive support for the program. These nine critical success  factors are consistent with leading industry practices for IT acquisitions.  Table 2 shows how many of the investments reported the nine factors  and selected examples of how agencies implemented them are discussed  below. A more detailed discussion of the investments’ identification of  success factors can be found in our 2011 report.
Officials from all seven selected investments cited active engagement  with program stakeholders—individuals or groups (including, in some  cases, end users) with an interest in the success of the acquisition—as a  critical factor to the success of those investments. Agency officials stated  that stakeholders, among other things, reviewed contractor proposals  during the procurement process, regularly attended program  management office sponsored meetings, were working members of  integrated project teams, and were notified of problems and concerns as  soon as possible. In addition, officials from the two investments at  National Nuclear Security Administration and U.S. Customs and Border  Protection noted that actively engaging with stakeholders created  transparency and trust, and increased the support from the stakeholders.
Additionally, officials from six of the seven selected investments indicated  that the knowledge and skills of the program staff were critical to the  success of the program. This included knowledge of acquisitions and  procurement processes, monitoring of contracts, large-scale  organizational transformation, Agile software development concepts,  and areas of program management such as earned value management  and technical monitoring.
Finally, officials from five of the seven selected investments identified  having the end users test and validate the system components prior to  formal end user acceptance testing for deployment as critical to the  success of their program. Similar to this factor, leading guidance  recommends testing selected products and product components  throughout the program life cycle. Testing of functionality by end users  prior to acceptance demonstrates, earlier rather than later in the program  life cycle, that the functionality will fulfill its intended use. If problems are  found during this testing, programs are typically positioned to make  changes that are less costly and disruptive than ones made later in the  life cycle would be.
In summary, the expanded use of these critical IT acquisition success  factors, in conjunction with industry and government best practices,  should result in the more effective delivery of mission-critical systems.  Further, these factors support OMB’s objective of improving the  management of large-scale IT acquisitions across the federal  government, and wide dissemination of these factors could complement  OMB’s efforts. While OMB’s and agencies’ recent efforts have resulted in  greater transparency and oversight of federal spending, continued  leadership and attention are necessary to build on the progress that has  been made. By improving the accuracy of information on the IT  Dashboard, and holding additional TechStat reviews, management  attention can be better focused on troubled projects and establishing  clear action items to turn these projects around or terminate them.  Further, legislation such as that proposed by this committee can play an  important role in increasing the authority of agency CIOs and improving  federal IT acquisition management practices. Overall, the implementation  of our numerous recommendations regarding key aspects of IT  acquisition management can help OMB and federal agencies continue to  improve the efficiency and transparency with which IT investments are  managed, in order to ensure that the federal government’s substantial  investment in IT is being wisely spent.
Chairman Issa, Ranking Member Cummings, and Members of the  Committee, this completes my prepared statement. I would be pleased to  respond to any questions that you may have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: If you or your staffs have any questions about this testimony, please  contact me at (202) 512-9286 or at pownerd@gao.gov. Individuals who  made key contributions to this testimony are Dave Hinchman (Assistant  Director), Deborah Davis, Rebecca Eyler, Kaelin Kuhn, Thomas Murphy,  Jamelyn Payan, and Jessica Waselkow.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.