Information Technology: Leveraging Best Practices to Help Ensure Successful Major Acquisitions
Released: 2013-11-13T00:00:00

Letter

Background

Information technology should enable government to better serve the American people. However, according to OMB, despite spending more than $600 billion on IT over the past decade, the federal government has achieved little of the productivity improvements that private industry has realized from IT. Too often, federal IT projects run over budget, behind schedule, or fail to deliver promised functionality. In combating this problem, proper oversight is critical.

Both OMB and federal agencies have key roles and responsibilities for overseeing IT investment management and OMB is responsible for working with agencies to ensure investments are appropriately planned and justified. However, as we have described in numerous reports, although a variety of best practice documentation exists to guide their successful acquisition, federal IT projects too frequently incur cost overruns and schedule slippages while contributing little to mission- related outcomes.

IT Acquisition Best Practices Have Been Identified by Industry and Government and Promoted by Legislation

IT acquisition best practices have been developed by both industry and the federal government. For example, the Software Engineering Institute has developed highly regarded and widely used guidance on best practices, such as requirements development and management, risk management, configuration management, validation and verification, and project monitoring and control. This guidance also describes disciplined project management practices that call for the development of project details, such as objectives, scope of work, schedules, costs, and requirements against which projects can be managed and executed. In the federal government, GAO’s own research in IT management best practices led to the development of the Information Technology Investment Management Framework, which describes essential and complementary IT investment management disciplines, such as oversight of system development and acquisition management, and organizes them into a set of critical processes for successful investments. This guidance further describes five progressive stages of maturity that an agency can achieve in its investment management capabilities, and was developed on the basis of our research into the IT investment management practices of leading private- and public-sector organizations. GAO has also identified opportunities to improve the role played by Chief Information Officers (CIO) in IT management. In noting that federal law provides CIOs with adequate authority to manage IT for their agencies, GAO also reported on limitations that impeded their ability to exercise this authority. Specifically, CIOs have not always had sufficient control over IT investments; more consistent implementation of CIOs’ authority could enhance their effectiveness.

Congress has also enacted legislation that reflects IT management best practices. For example, the Clinger-Cohen Act of 1996, which was informed by GAO best practice recommendations, requires federal agencies to focus more on the results they have achieved through IT investments, while concurrently improving their IT acquisition processes. Specifically, the act requires agency heads to implement a process to maximize the value of the agency’s IT investments and assess, manage, and evaluate the risks of its IT acquisitions. Further, the act establishes CIOs to advise and assist agency heads in carrying out these responsibilities. The act also requires OMB to encourage agencies to develop and use best practices in IT acquisition.

Additionally, the E-Government Act of 2002 established a CIO Council, which is led by the Federal CIO, to be the principal interagency forum for improving agency practices related to the development, acquisition, and management of information resources, including sharing best practices.

Although these best practices and legislation can have a positive impact on major IT programs, we have previously testified that the federal government continues to invest in numerous failed and troubled projects. We stated that while OMB’s and agencies’ recent efforts had resulted in greater transparency and oversight of federal spending, continued leadership and attention was necessary to build on the progress that had been made.

In an effort to end the recurring cycle of failed IT projects, this committee has introduced legislation to improve IT acquisition management. Among other things, this legislation would eliminate duplication and waste in IT acquisition, and increase the authority of agency CIOs, strengthen and streamline IT acquisition management practices.

We have previously testified in support of this legislation.

OMB Has Several Initiatives Under Way to Improve the Oversight and Management of IT Investments

OMB plays a key role in helping federal agencies manage their investments by working with them to better plan, justify, and determine how much they need to spend on projects and how to manage approved projects.

In June 2009, OMB established the IT Dashboard to improve the transparency into and oversight of agencies’ IT investments. According to OMB officials, agency CIOs are required to update each major investment in the IT Dashboard with a rating based on the CIO’s evaluation of certain aspects of the investment, such as risk management, requirements management, contractor oversight, and human capital. According to OMB, these data are intended to provide a near real-time perspective of the performance of these investments, as well as a historical perspective. Further, the public display of these data is intended to allow OMB, congressional and other oversight bodies, and the general public to hold government agencies accountable for results and progress.

In January 2010, the Federal CIO began leading TechStat sessions— reviews of selected IT investments between OMB and agency leadership—to increase accountability and transparency and improve performance. OMB has identified factors that may result in an investment being selected for a TechStat session, such as—but not limited to— evidence of (1) poor performance; (2) duplication with other systems or projects; (3) unmitigated risks; and (4) misalignment with policies and best practices.

OMB reported that as of April 2013, 79 TechStat sessions had been held with federal agencies. According to OMB, these sessions enabled the government to improve or terminate IT investments that were experiencing performance problems. For example, in June 2010 the Federal CIO led a TechStat on the National Archives and Records Administration’s (NARA) Electronic Records Archives investment that resulted in six corrective actions, including halting fiscal year 2012 development funding pending the completion of a strategic plan. Similarly, in January 2011, we reported that NARA had not been positioned to identify potential cost and schedule problems early, and had not been able to take timely actions to correct problems, delays, and cost increases on this system acquisition program. Moreover, we estimated that the program would likely overrun costs by between $205 and $405 million if the agency completed the program as originally designed. We made multiple recommendations to the Archivist of the United States, including establishing a comprehensive plan for all remaining work, improving the accuracy of key performance reports, and engaging executive leadership in correcting negative performance trends.

Drawing on the visibility into federal IT investments provided by the IT Dashboard and TechStat sessions, in December 2010, OMB issued a plan to reform IT management throughout the federal government over an 18-month time frame. Among other things, the plan noted the goal of turning around or terminating at least one-third of underperforming projects by June 2012. The plan contained two high-level objectives: achieving operational efficiency, and effectively managing large-scale IT programs.

To achieve operational efficiencies, the plan outlined actions required to adopt cloud solutions and leverage shared services. To effectively manage IT acquisitions, the plan identified key actions, such as improving accountability and governance and aligning acquisition processes with the technology cycle. Our April 2012 report on the federal government’s progress on implementing the plan found that not all action items had been completed. These findings are discussed in greater detail later in the next section.

Prior GAO Work Has Reported on OMB’s IT Investment Oversight and Management

We have previously reported that OMB has taken significant steps to enhance the oversight, transparency, and accountability of federal IT investments by creating its IT Dashboard, by improving the accuracy of investment ratings, and by creating a plan to reform federal IT. However, we also found issues with the accuracy and data reliability of cost and schedule data, and recommended steps that OMB should take to improve these data.

In July 2010, we reported that the cost and schedule ratings on OMB’s Dashboard were not always accurate for the investments we reviewed, because these ratings did not take into consideration current performance. As a result, the ratings were based on outdated information. We recommended that OMB report on its planned changes to the Dashboard to improve the accuracy of performance information and provide guidance to agencies to standardize milestone reporting. OMB agreed with our recommendations and, as a result, updated the Dashboard’s cost and schedule calculations to include both ongoing and completed activities.

Similarly, our report in March 2011 noted that OMB had initiated several efforts to increase the Dashboard’s value as an oversight tool and had used its data to improve federal IT management. However, we also reported that agency practices and the Dashboard’s calculations contributed to inaccuracies in the reported investment performance data. For instance, we found missing data submissions or erroneous data at each of the five agencies we reviewed, along with instances of inconsistent program baselines and unreliable source data. As a result, we recommended that the agencies take steps to improve the accuracy and reliability of their Dashboard information, and that OMB improve how it rates investments relative to current performance and schedule variance. Most agencies generally concurred with our recommendations and three have taken steps to address them. OMB agreed with our recommendation for improving ratings for schedule variance. It disagreed with our recommendation to improve how it reflects current performance in cost and schedule ratings, but more recently made changes to Dashboard calculations to address this while also noting challenges in comprehensively evaluating cost and schedule data for these investments.

Subsequently, in November 2011, we noted that the accuracy of investment cost and schedule ratings had improved since our July 2010 report because OMB refined the Dashboard’s cost and schedule calculations. Most of the ratings for the eight investments we reviewed as part of our November 2011 report were accurate, although we noted that more could be done to inform oversight and decision making by emphasizing recent performance in the ratings. We recommended that the General Services Administration comply with OMB’s guidance for updating its ratings when new information becomes available (including when investments are rebaselined). The agency concurred and has since taken actions to address this recommendation. Since we previously recommended that OMB improve how it rates investments, we did not make any further recommendations.

Further, in April 2012, we reported that OMB and key federal agencies had made progress on implementing actions items from its plan to reform IT management, but found that there were several areas where more remained to be done. Specifically, we reviewed 10 actions and found that 3 were complete, while 7 were incomplete. For example, we found that OMB had reformed and strengthened investment review boards, but had only partially issued guidance on modular development. Accordingly, we recommended, among other things, that OMB ensure that the action items called for in the plan be completed by the responsible parties prior to the completion of the plan’s 18-month deadline of June 2012, or if the June 2012 deadline could not be met, by another clearly defined deadline. OMB agreed to complete the key action items.

Finally, we reviewed OMB’s efforts to help agencies address IT projects with cost overruns, schedule delays, and performance shortfalls in June 2013. In particular, we reported that OMB used CIO ratings from the Dashboard, among other sources, to select at- risk investments for reviews known as TechStats. OMB initiated these reviews in January 2010 to further improve investment performance, and subsequently incorporated the TechStat model into its plan for reforming IT management. We reported that OMB and selected agencies had held multiple TechStat sessions but additional OMB oversight was needed to ensure that these meetings were having the appropriate impact on underperforming projects and that resulting cost savings were valid. Among other things, we recommended that OMB require agencies to address their highest- risk investments and to report on how they validated the outcomes. OMB generally agreed with our recommendations, and stated that it and the agencies were taking appropriate steps to address them.

Critical Factors Underlying Successful Major Acquisitions

Subsequent to the launch of the Dashboard and the TechStat reviews, and to help the federal agencies address the well-documented acquisition challenges they face, in 2011, we reported on nine common factors critical to the success of IT investment acquisitions. Specifically, we reported that department officials from seven agencies each identified a successful investment acquisition, in that they best achieved their respective cost, schedule, scope, and performance goals. To identify these investments, we interviewed officials from the 10 departments with the largest planned IT budgets in order for each department to identify one mission-critical, major IT investment that best achieved its cost, schedule, scope, and performance goals. Of the 10 departments, 7 identified successful IT investments, for a total of 7 investments. Officials from the 7 investments cited a number of success factors that contributed to these investments’ success.

According to federal department officials, the following seven investments (shown in table 1) best achieved their respective cost, schedule, scope, and performance goals. The estimated total life-cycle cost of the seven investments is about $5 billion.

Among these seven IT investments, officials identified nine factors as critical to the success of three or more of the seven. The factors most commonly identified include active engagement of stakeholders, program staff with the necessary knowledge and skills, and senior department and agency executive support for the program. These nine critical success factors are consistent with leading industry practices for IT acquisitions. Table 2 shows how many of the investments reported the nine factors and selected examples of how agencies implemented them are discussed below. A more detailed discussion of the investments’ identification of success factors can be found in our 2011 report.

Officials from all seven selected investments cited active engagement with program stakeholders—individuals or groups (including, in some cases, end users) with an interest in the success of the acquisition—as a critical factor to the success of those investments. Agency officials stated that stakeholders, among other things, reviewed contractor proposals during the procurement process, regularly attended program management office sponsored meetings, were working members of integrated project teams, and were notified of problems and concerns as soon as possible. In addition, officials from the two investments at National Nuclear Security Administration and U.S. Customs and Border Protection noted that actively engaging with stakeholders created transparency and trust, and increased the support from the stakeholders.

Additionally, officials from six of the seven selected investments indicated that the knowledge and skills of the program staff were critical to the success of the program. This included knowledge of acquisitions and procurement processes, monitoring of contracts, large-scale organizational transformation, Agile software development concepts, and areas of program management such as earned value management and technical monitoring.

Finally, officials from five of the seven selected investments identified having the end users test and validate the system components prior to formal end user acceptance testing for deployment as critical to the success of their program. Similar to this factor, leading guidance recommends testing selected products and product components throughout the program life cycle. Testing of functionality by end users prior to acceptance demonstrates, earlier rather than later in the program life cycle, that the functionality will fulfill its intended use. If problems are found during this testing, programs are typically positioned to make changes that are less costly and disruptive than ones made later in the life cycle would be.

In summary, the expanded use of these critical IT acquisition success factors, in conjunction with industry and government best practices, should result in the more effective delivery of mission-critical systems. Further, these factors support OMB’s objective of improving the management of large-scale IT acquisitions across the federal government, and wide dissemination of these factors could complement OMB’s efforts. While OMB’s and agencies’ recent efforts have resulted in greater transparency and oversight of federal spending, continued leadership and attention are necessary to build on the progress that has been made. By improving the accuracy of information on the IT Dashboard, and holding additional TechStat reviews, management attention can be better focused on troubled projects and establishing clear action items to turn these projects around or terminate them. Further, legislation such as that proposed by this committee can play an important role in increasing the authority of agency CIOs and improving federal IT acquisition management practices. Overall, the implementation of our numerous recommendations regarding key aspects of IT acquisition management can help OMB and federal agencies continue to improve the efficiency and transparency with which IT investments are managed, in order to ensure that the federal government’s substantial investment in IT is being wisely spent.

Chairman Issa, Ranking Member Cummings, and Members of the Committee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contact and Staff Acknowledgments

If you or your staffs have any questions about this testimony, please contact me at (202) 512-9286 or at pownerd@gao.gov. Individuals who made key contributions to this testimony are Dave Hinchman (Assistant Director), Deborah Davis, Rebecca Eyler, Kaelin Kuhn, Thomas Murphy, Jamelyn Payan, and Jessica Waselkow.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.