Information Technology: Implementation of IT Reform Law and Related Initiatives Can Help Improve Acquisitions
Released: 2017-03-28T00:00:00

Letter

Background

The federal government is projected to invest more than $89 billion on IT in fiscal year 2017. However, as we have previously reported, investments in federal IT too often result in failed projects that incur cost overruns and schedule slippages, while contributing little to the desired mission-related outcomes. For example:

The Department of Veterans Affairs’ Scheduling Replacement Project was terminated in September 2009 after investing an estimated $127 million over 9 years.

The tri-agency National Polar-orbiting Operational Environmental Satellite System was disbanded in February 2010 at the direction of the White House’s Office of Science and Technology Policy after the program invested 16 years and almost $5 billion.

The Department of Homeland Security’s Secure Border Initiative Network program was ended in January 2011, after the department invested more than $1 billion to the program.

The Office of Personnel Management’s Retirement Systems Modernization program was canceled in February 2011, after investing approximately $231 million on the agency’s third attempt to automate the processing of federal employee retirement claims.

The Department of Veterans Affairs’ Financial and Logistics Integrated Technology Enterprise program was intended to be delivered by 2014 at a total estimated cost of $609 million, but was terminated in October 2011 due to challenges in managing the program.

The Department of Defense’s Expeditionary Combat Support System was canceled in December 2012 after investing more than a billion dollars and failing to deploy within 5 years of initially obligating funds.

The Farm Service Agency’s Modernize and Innovate the Delivery of Agricultural Systems program, which was to replace aging hardware and software applications that process benefits to farmers, was halted in July 2014 after investing about 10 years and at least $423 million, while only delivering about 20 percent of the functionality that was originally planned.

Our past work found that these and other failed IT projects often suffered from a lack of disciplined and effective management, such as project planning, requirements definition, and program oversight and governance. In many instances, agencies had not consistently applied best practices that are critical to successfully acquiring IT.

Federal IT projects have also failed due to a lack of oversight and governance. Executive-level governance and oversight across the government has often been ineffective, specifically from chief information officers (CIO). For example, we reported that some CIOs’ authority was limited in that not all CIOs had the authority to review and approve the entire agency IT portfolio.

Our past work has also identified nine critical factors underlying successful major acquisitions that support the objective of improving the management of large-scale IT acquisitions across the federal government: (1) program officials actively engaging with stakeholders; (2) program staff having the necessary knowledge and skills; (3) senior department and agency executives supporting the programs; (4) end users and stakeholders being involved in the development of requirements; (5) end users participating in the testing of system functionality prior to end user acceptance testing; (6) government and contractor staff being stable and consistent; (7) program staff prioritizing requirements; (8) program officials maintaining regular communication with the prime contractor; and (9) programs receiving sufficient funding.

FITARA Can Improve Agencies’ Acquisition of IT

Recognizing the importance of issues related to government-wide management of IT, FITARA was enacted in December 2014. The law was aimed at improving agencies’ acquisitions of IT and could help enable Congress to monitor agencies’ progress and hold them accountable for reducing duplication and achieving cost savings. FITARA includes specific requirements related to the acquisition of IT, such as

Agency CIO authority enhancements. CIOs at covered agencies are required to (1) approve the IT budget requests of their respective agencies, (2) certify that OMB’s incremental development guidance is being adequately implemented for IT investments, (3) review and approve contracts for IT, and (4) approve the appointment of other agency employees with the title of CIO.

Enhanced transparency and improved risk management. OMB and covered agencies are to make detailed information on federal IT investments publicly available and agency CIOs are to categorize their IT investments by level of risk. Additionally, in the case of major IT investments rated as high risk for 4 consecutive quarters, the law requires that the agency CIO and the investment’s program manager conduct a review aimed at identifying and addressing the causes of the risk.

Expansion of training and use of IT acquisition cadres. Agencies are to update their acquisition human capital plans to address supporting the timely and effective acquisition of IT. In doing so, the law calls for agencies to consider, among other things, establishing IT acquisition cadres or developing agreements with other agencies that have such cadres.

Government-wide software purchasing program. The General Services Administration is to develop a strategic sourcing initiative to enhance government-wide acquisition and management of software. In doing so, the law requires that, to the maximum extent practicable, the General Services Administration should allow for the purchase of a software license agreement that is available for use by all executive branch agencies as a single user.

Maximizing the benefit of the federal strategic sourcing initiative. Federal agencies are required to compare their purchases of services and supplies to what is offered under the federal strategic sourcing initiative. OMB is also required to issue related regulations.

IT Acquisitions and Operations Identified by GAO as a High-Risk Area

In February 2015, we introduced a new government-wide high-risk area, Improving the Management of IT Acquisitions and Operations. This area highlights several critical IT initiatives in need of additional congressional oversight, including (1) reviews of troubled projects; (2) efforts to increase the use of incremental development; (3) efforts to provide transparency relative to the cost, schedule, and risk levels for major IT investments; (4) reviews of agencies’ operational investments; (5) data center consolidation; and (6) efforts to streamline agencies’ portfolios of IT investments. We noted that implementation of these initiatives has been inconsistent and more work remains to demonstrate progress in achieving successful IT acquisitions and operations outcomes.

Further, our February 2015 high-risk report also stated that, beyond implementing FITARA, OMB and agencies needed to continue to implement our prior recommendations in order to improve their ability to effectively and efficiently invest in IT. Specifically, between fiscal years 2010 and 2015, we made 803 recommendations to OMB and federal agencies to address shortcomings in IT acquisitions and operations, including many to improve the implementation of the recent initiatives and other government-wide, cross-cutting efforts. We noted that OMB and agencies should demonstrate government-wide progress in the management of IT investments by, among other things, implementing at least 80 percent of our recommendations related to managing IT acquisitions and operations within 4 years.

In February 2017, we issued an update to our high-risk series and reported that, while progress had been made in improving the management of IT acquisitions and operations, significant work still remained to be completed. For example, as of December 2016, OMB and the agencies had fully implemented 366 (or about 46 percent) of the 803 recommendations. This was a 23 percent increase compared to the percentage we reported as being fully implemented in 2015. Figure 1 summarizes the progress that OMB and the agencies have made in addressing our recommendations, as compared to the 80 percent target.

In addition, in fiscal year 2016, we made 202 new recommendations, thus further reinforcing the need for OMB and agencies to address the shortcomings in IT acquisitions and operations. In addition to addressing our prior recommendations, our 2017 high-risk update also notes the importance of OMB and federal agencies continuing to expeditiously implement the requirements of FITARA.

Opportunities Exist to Improve Acquisition of IT

Given the magnitude of the federal government’s annual IT budget, which is projected to be more than $89 billion in fiscal year 2017, it is important that agencies leverage all available opportunities to ensure that IT investments are made in the most effective manner possible. To do so, agencies can rely on key IT workforce planning activities to facilitate the success of major acquisitions. OMB has also established several initiatives to improve the acquisition of IT, including reviews of troubled IT projects, a key transparency website, and an emphasis on incremental development. However, the implementation of these efforts has been inconsistent and more work remains to demonstrate progress in achieving successful IT acquisition outcomes.

Implementing Key IT Workforce Planning Activities Can Help Ensure Acquisition Skill Gaps Are Addressed

An area where agencies can improve their ability to acquire IT is workforce planning. In November 2016, we reported that IT workforce planning activities, when effectively implemented, can facilitate the success of major acquisitions. As stated earlier, ensuring program staff have the necessary knowledge and skills is a factor commonly identified as critical to the success of major investments. If agencies are to ensure that this critical success factor has been met, then IT skill gaps need to be adequately assessed and addressed through a workforce planning process.

In this regard, we reported that four workforce planning steps and eight key activities can assist agencies in assessing and addressing IT knowledge and skill gaps. Specifically, these four steps are: (1) setting the strategic direction for IT workforce planning, (2) analyzing the workforce to identify skill gaps, (3) developing and implementing strategies to address IT skill gaps, and (4) monitoring and reporting progress in addressing skill gaps. Each of the four steps is supported by key activities (as summarized in table 1).

However, in our November 2016 report, we determined that five agencies that we selected for in-depth analysis had not fully implemented key workforce planning steps and activities. For example, four of these agencies had not demonstrated an established IT workforce planning process. In addition, none of these agencies had fully assessed their workforce competencies and staffing needs regularly or established strategies and plans to address gaps in these areas. Figure 2 illustrates the extent to which the five selected agencies had fully, partially, or not implemented key IT workforce planning activities.

The weaknesses identified were due, in part, to these agencies lacking comprehensive policies that required such activities, or failing to apply the policies to IT workforce planning. We concluded that, until these weaknesses are addressed, the five agencies risk not adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major acquisitions. Accordingly, we made recommendations to each of the five selected agencies to address the weaknesses in their IT workforce planning practices that we identified. Four agencies—the Departments of Commerce, Health and Human Services, Transportation, and Treasury—agreed with our recommendations and one, the Department of Defense, partially agreed.

TechStat Reviews Can Help Highlight and Evaluate Poorly Performing Investments

In January 2010, the Federal CIO began leading TechStat sessions— face-to-face meetings to terminate or turn around IT investments that are failing or are not producing results. These meetings involve OMB and agency leadership and are intended to increase accountability and transparency and improve performance. OMB reported that federal agencies achieved over $3 billion in cost savings or avoidances as a result of these sessions in 2010. Subsequently, OMB empowered agency CIOs to hold their own TechStat sessions within their respective agencies.

In June 2013, we reported that, while OMB and selected agencies continued to hold additional TechStats, more OMB oversight was needed to ensure that these meetings were having the appropriate impact on underperforming projects. Specifically, OMB reported conducting TechStats at 23 federal agencies covering 55 investments, 30 of which were considered medium or high risk at the time of the TechStat. However, these reviews accounted for less than 20 percent of medium- or high-risk investments government-wide. As of August 2012, there were 162 such at-risk investments across the government.

Further, we reviewed four selected agencies and found they had held TechStats on 28 investments. While these reviews were generally conducted in accordance with OMB guidance, we found that areas for improvement existed. For example, these agencies did not consistently create memorandums with responsible parties and due dates for action items. We concluded that, until these agencies fully implemented OMB’s TechStat guidance, they may not be positioned to effectively manage and resolve problems on IT investments. In addition, we noted that, until OMB and agencies develop plans and schedules to review medium- and high- risk investments, the investments would likely remain at risk. Among other things, we recommended that OMB require agencies to conduct TechStats for each IT investment rated with a moderately high- or high- risk rating, unless there is a clear reason for not doing so. OMB generally agreed with this recommendation.

However, when we testified on this issue slightly more than 2 years later in November 2015, we found that OMB had only conducted one TechStat review between March 2013 and October 2015. In addition, we noted that OMB had not listed any savings from TechStats in any of its required quarterly reporting to Congress since June 2012. This issue continues to be a concern and, in January 2017, the Federal CIO Council issued a report titled the State of Federal Information Technology, which noted that while early TechStats saved money and turned around underperforming investments it was unclear if OMB had performed any TechStats in recent years.

IT Dashboard Can Improve the Transparency into and Oversight of Major IT Acquisitions

To facilitate transparency across the government in acquiring and managing IT investments, OMB established a public website—the IT Dashboard—to provide detailed information on major investments at 26 agencies, including ratings of their performance against cost and schedule targets. Among other things, agencies are to submit ratings from their CIOs, which, according to OMB’s instructions, should reflect the level of risk facing an investment relative to that investment’s ability to accomplish its goals. In this regard, FITARA includes a requirement for CIOs to categorize their major IT investment risks in accordance with OMB guidance.

Over the past 6 years, we have issued a series of reports about the IT Dashboard that noted both significant steps OMB has taken to enhance the oversight, transparency, and accountability of federal IT investments by creating its IT Dashboard, as well as issues with the accuracy and reliability of data. In total, we have made 47 recommendations to OMB and federal agencies to help improve the accuracy and reliability of the information on the IT Dashboard and to increase its availability. Most agencies have agreed with our recommendations.

Most recently, in June 2016, we determined that 13 of the 15 agencies selected for in-depth review had not fully considered risks when rating their major investments on the IT Dashboard. Specifically, our assessments of risk for 95 investments at 15 selected agencies matched the CIO ratings posted on the Dashboard 22 times, showed more risk 60 times, and showed less risk 13 times. Figure 3 summarizes how our assessments compared to the selected investments’ CIO ratings.

Aside from the inherently judgmental nature of risk ratings, we identified three factors which contributed to differences between our assessments and the CIO ratings:

Forty of the 95 CIO ratings were not updated during the month we reviewed, which led to more differences between our assessments and the CIOs’ ratings. This underscores the importance of frequent rating updates, which help to ensure that the information on the Dashboard is timely and accurately reflects recent changes to investment status.

Three agencies’ rating processes spanned longer than 1 month.

Longer processes mean that CIO ratings are based on older data, and may not reflect the current level of investment risk.

Seven agencies’ rating processes did not focus on active risks.

According to OMB’s guidance, CIO ratings should reflect the CIO’s assessment of the risk and the investment’s ability to accomplish its goals. CIO ratings that do not incorporate active risks increase the chance that ratings overstate the likelihood of investment success.

As a result, we concluded that the associated risk rating processes used by the 15 agencies were generally understating the level of an investment’s risk, raising the likelihood that critical federal investments in IT are not receiving the appropriate levels of oversight. To better ensure that the Dashboard ratings more accurately reflect risk, we recommended that the 15 agencies take actions to improve the quality and frequency of their CIO ratings. Twelve agencies generally agreed with or did not comment on the recommendations and three agencies disagreed, stating their CIO ratings were adequate. However, we noted that weaknesses in their processes still existed and that we continued to believe our recommendations were appropriate.

Increasing the Use of Incremental Development Practices Can Help Agencies Better Achieve Cost, Schedule, and Performance Goals for IT Acquisitions

OMB has emphasized the need to deliver investments in smaller parts, or increments, in order to reduce risk, deliver capabilities more quickly, and facilitate the adoption of emerging technologies. In 2010, it called for agencies’ major investments to deliver functionality every 12 months and, since 2012, every 6 months. Subsequently, FITARA codified a requirement that agency CIOs certify that IT investments are adequately implementing OMB’s incremental development guidance.

In May 2014, we reported that 66 of 89 selected investments at five major agencies did not plan to deliver capabilities in 6-month cycles, and less than half of these investments planned to deliver functionality in 12-month cycles. We also reported that only one of the five agencies had complete incremental development policies. Accordingly, we recommended that OMB develop and issue clearer guidance on incremental development and that the selected agencies update and implement their associated policies. Four of the six agencies agreed with our recommendations or had no comments; the remaining two agencies partially agreed or disagreed with the recommendations. The agency that disagreed with our recommendation stated that it did not believe that its recommendation should be dependent on OMB first taking action. However, we noted that our recommendation does not require OMB to take action first and that we continued to believe our recommendation was warranted and could be implemented.

Subsequently, in August 2016, we reported that agencies had not fully implemented incremental development practices for their software development projects. Specifically, we noted that, as of August 31, 2015, 22 federal agencies had reported on the IT Dashboard that 300 of 469 active software development projects (approximately 64 percent) were planning to deliver usable functionality every 6 months for fiscal year 2016, as required by OMB guidance. Regarding the remaining 169 projects (or 36 percent) that were reported as not planning to deliver functionality every 6 months, agencies provided a variety of explanations for not achieving that goal. These included project complexity, the lack of an established project release schedule, or that the project was not a software development project. Table 2 lists the total number and percent of federal software development projects for which agencies reported plans to deliver functionality every 6 months for fiscal year 2016.

In conducting an in-depth review of seven selected agencies’ software development projects, we determined that 45 percent of the projects delivered functionality every 6 months for fiscal year 2015 and 55 percent planned to do so in fiscal year 2016. Agency officials reported that management and organizational challenges and project complexity and uniqueness had impacted their ability to deliver incrementally. We concluded that it was critical that agencies continue to improve their use of incremental development to deliver functionality and reduce the risk that these projects will not meet cost, schedule, and performance goals.

In addition, while OMB had issued guidance requiring covered agency CIOs to certify that each major IT investment’s plan for the current year adequately implements incremental development, only three agencies (the Departments of Commerce, Homeland Security, and Transportation) had defined processes and policies intended to ensure that the department CIO certifies that major IT investments are adequately implementing incremental development. Officials from three other agencies (the Departments of Education, Health and Human Services, and the Treasury) reported that they were in the process of updating their existing incremental development policy to address certification, while the Department of Defense’s policies that address incremental development did not include information on CIO certification. We concluded that until all of the agencies we reviewed define processes and policies for the certification of the adequate use of incremental development, they will not be able to fully ensure adequate implementation of, or benefit from, incremental development practices.

Accordingly, we recommended that four agencies establish a policy and process for the certification of major IT investments’ adequate use of incremental development. The Departments of Education and Health and Human Services agreed with our recommendation, while the Department of Defense disagreed and stated that its existing policies address the use of incremental development. However, we noted that the department’s policies did not comply with OMB’s guidance and that we continued to believe our recommendation was appropriate. The Department of the Treasury did not comment on the recommendation.

In conclusion, with the enactment of FITARA, the federal government has an opportunity to improve the transparency and management of IT acquisitions, and to strengthen the authority of CIOs to provide needed direction and oversight. In addition to implementing FITARA, applying key IT workforce planning practices could improve the agencies’ ability to assess and address gaps in knowledge and skills that are critical to the success of major acquisitions. Further, continuing to implement key OMB initiatives can help to improve the acquisition of IT. For example, conducting additional TechStat reviews can help focus management attention on troubled projects and provide a mechanism to establish clear action items to improve project performance or terminate the investment. Additionally, improving the assessment of risks when agencies rate major investments on the IT Dashboard would likely provide greater transparency and oversight of the government’s billions of dollars in IT investments. Lastly, increasing the use of incremental development approaches could improve the likelihood that major IT investments meet cost, schedule, and performance goals.

Chairmen Hurd and Meadows, Ranking Members Kelly and Connolly, and Members of the Subcommittees, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contacts and Staff Acknowledgments

If you or your staffs have any questions about this testimony, please contact me at (202) 512-9286 or at pownerd@gao.gov. Individuals who made key contributions to this testimony are Dave Hinchman (Assistant Director), Chris Businsky, Rebecca Eyler, and Jon Ticehurst (Analyst in Charge).

Related GAO Products

High-Risk Series: Progress on Many High-Risk Areas, While Substantial Efforts Needed on Others. GAO-17-317. Washington, D.C.: February 15, 2017.

IT Workforce: Key Practices Help Ensure Strong Integrated Program Teams; Selected Departments Need to Assess Skill Gaps. GAO-17-8. Washington, D.C.: November 30, 2016.

Information Technology Reform: Agencies Need to Increase Their Use of Incremental Development Practices. GAO-16-469. Washington, D.C.: August 16, 2016.

IT Dashboard: Agencies Need to Fully Consider Risks When Rating Their Major Investments. GAO-16-494. Washington, D.C.: June 2, 2016.

High-Risk Series: An Update. GAO-15-290. Washington, D.C.: February 11, 2015.

Information Technology: Agencies Need to Establish and Implement Incremental Development Policies. GAO-14-361. Washington, D.C.: May 1, 2014.

IT Dashboard: Agencies Are Managing Investment Risk, but Related Ratings Need to Be More Accurate and Available. GAO-14-64. Washington, D.C.: December 12, 2013.

Information Technology: Additional Executive Review Sessions Needed to Address Troubled Projects. GAO-13-524. Washington, D.C.: June 13, 2013.

IT Dashboard: Opportunities Exist to Improve Transparency and Oversight of Investment Risk at Select Agencies. GAO-13-98. Washington, D.C.: October 16, 2012.

IT Dashboard: Accuracy Has Improved, and Additional Efforts Are Under Way to Better Inform Decision Making. GAO-12-210. Washington, D.C.: November 7, 2011.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.