Information Technology: Further Implementation of Recommendations Is Needed to Better Manage Acquisitions and Operations
Released: 2018-03-14T00:00:00

Letter

Background

According to the President’s budget, the federal government plans to invest more than $96 billion for IT in fiscal year 2018—the largest amount ever budgeted. However, as we have previously reported, investments in federal IT too often result in failed projects that incur cost overruns and schedule slippages, while contributing little to the desired mission-related outcomes. For example:

The Department of Veterans Affairs’ Scheduling Replacement Project was terminated in September 2009 after spending an estimated $127 million over 9 years.

The tri-agency National Polar-orbiting Operational Environmental Satellite System was disbanded in February 2010 by the White House’s Office of Science and Technology Policy after the program spent 16 years and almost $5 billion.

The Department of Homeland Security’s Secure Border Initiative Network program was ended in January 2011, after the department obligated more than $1 billion for the program.

The Office of Personnel Management’s Retirement Systems Modernization program was canceled in February 2011, after the agency had spent approximately $231 million on its third attempt to automate the processing of federal employee retirement claims.

The Department of Veterans Affairs’ Financial and Logistics Integrated Technology Enterprise program was intended to be delivered by 2014 at a total estimated cost of $609 million, but was terminated in October 2011.

The Department of Defense’s Expeditionary Combat Support System was canceled in December 2012 after spending more than a billion dollars and failing to deploy within 5 years of initially obligating funds.

Our past work found that these and other failed IT projects often suffered from a lack of disciplined and effective management, such as project planning, requirements definition, and program oversight and governance. In many instances, agencies had not consistently applied best practices that are critical to successfully acquiring IT.

Such projects have also failed due to a lack of oversight and governance. Executive-level governance and oversight across the government has often been ineffective, specifically from chief information officers (CIO). For example, we have reported that some CIOs’ roles were limited because they did not have the authority to review and approve the entire agency IT portfolio.

Implementing FITARA Can Improve Agencies’ Management of IT

FITARA was intended to improve covered agencies’ acquisitions of IT and enable Congress to monitor agencies’ progress and hold them accountable for reducing duplication and achieving cost savings. The law includes specific requirements related to seven areas.

Federal data center consolidation initiative (FDCCI). Agencies covered by FITARA are required to provide OMB with a data center inventory, a strategy for consolidating and optimizing their data centers (to include planned cost savings), and quarterly updates on progress made. The law also requires OMB to develop a goal for how much is to be saved through this initiative, and provide annual reports on cost savings achieved.

Enhanced transparency and improved risk management. OMB and covered agencies are to make detailed information on federal IT investments publicly available, and agency CIOs are to categorize their investments by level of risk. Additionally, in the case of major IT investments rated as high risk for 4 consecutive quarters, the law requires that the agency CIO and the investment’s program manager conduct a review aimed at identifying and addressing the causes of the risk.

Agency CIO authority enhancements. Agency heads at covered agencies are required to ensure that CIOs have authority to (1) approve the IT budget requests of their respective agencies, (2) certify that OMB’s incremental development guidance is being adequately implemented for IT investments, (3) review and approve contracts for IT, and (4) approve the appointment of other agency employees with the title of CIO.

Portfolio review. Covered agencies are to annually review IT investment portfolios in order to, among other things, increase efficiency and effectiveness and identify potential waste and duplication. In establishing the process associated with such portfolio reviews, the law requires OMB to develop standardized performance metrics, to include cost savings, and to submit quarterly reports to Congress on cost savings.

Expansion of training and use of IT acquisition cadres. Covered agencies are to update their acquisition human capital plans to address supporting the timely and effective acquisition of IT. In doing so, the law calls for agencies to consider, among other things, establishing IT acquisition cadres or developing agreements with other agencies that have such cadres.

Government-wide software purchasing program. The General Services Administration is to develop a strategic sourcing initiative to enhance government-wide acquisition and management of software. In doing so, the law requires that, to the maximum extent practicable, the General Services Administration should allow for the purchase of a software license agreement that is available for use by all executive branch agencies as a single user.

Maximizing the benefit of the Federal Strategic Sourcing Initiative. Federal agencies are required to compare their purchases of services and supplies to what is offered under the Federal Strategic Sourcing Initiative. The Administrator for Federal Procurement Policy was also required to issue regulations related to the initiative.

In June 2015, OMB released guidance describing how agencies are to implement FITARA. This guidance is intended to, among other things: assist agencies in aligning their IT resources with statutory establish government-wide IT management controls that will meet the law’s requirements, while providing agencies with flexibility to adapt to unique agency processes and requirements; strengthen the relationship between agency CIOs and bureau CIOs; and strengthen CIO accountability for IT costs, schedules, performance, and security.

The guidance identified several actions that agencies were to take to establish a basic set of roles and responsibilities (referred to as the common baseline) for CIOs and other senior agency officials, which were needed to implement the authorities described in the law. For example, agencies were required to conduct a self-assessment and submit a plan describing the changes they intended to make to ensure that common baseline responsibilities were implemented. Agencies were to submit their plans to OMB’s Office of E-Government and Information Technology by August 15, 2015, and make portions of the plans publicly available on agency websites no later than 30 days after OMB approval. As of November 2016, all agencies had made their plans publicly available.

In addition, in August 2016, OMB released guidance intended to, among other things, define a framework for achieving the data center consolidation and optimization requirements of FITARA. The guidance requires each agency on a quarterly basis to: maintain complete inventories of all data center facilities owned, operated, or maintained by or on behalf of the agency; develop cost savings targets for fiscal years 2016 through 2018 and report any actual realized cost savings; and measure progress toward meeting optimization metrics.

The guidance also directs agencies to develop a data center consolidation and optimization strategic plan that defines the agency’s data center strategy for fiscal years 2016, 2017, and 2018. This strategy is to include, among other things, a statement from the agency CIO indicating whether the agency has complied with all data center reporting requirements in FITARA. Further, the guidance indicates that OMB is to maintain a public dashboard that will display consolidation-related costs savings and optimization performance information for the agencies.

IT Acquisitions and Operations Identified by GAO as a High-Risk Area

In February 2015, we introduced a new government-wide high-risk area, Improving the Management of IT Acquisitions and Operations. This area highlighted several critical IT initiatives in need of additional congressional oversight, including (1) reviews of troubled projects; (2) efforts to increase the use of incremental development; (3) efforts to provide transparency relative to the cost, schedule, and risk levels for major IT investments; (4) reviews of agencies’ operational investments; (5) data center consolidation; and (6) efforts to streamline agencies’ portfolios of IT investments. We noted that implementation of these initiatives was inconsistent and more work remained to demonstrate progress in achieving IT acquisition and operation outcomes.

Further, our February 2015 high-risk report stated that, beyond implementing FITARA, OMB and agencies needed to continue to implement our prior recommendations in order to improve their ability to effectively and efficiently invest in IT. Specifically, from fiscal years 2010 through 2015, we made 803 recommendations to OMB and federal agencies to address shortcomings in IT acquisitions and operations. These recommendations included many to improve the implementation of the aforementioned six critical IT initiatives and other government-wide, cross-cutting efforts. We stressed that OMB and agencies should demonstrate government-wide progress in the management of IT investments by, among other things, implementing at least 80 percent of our recommendations related to managing IT acquisitions and operations within 4 years.

In February 2017, we issued an update to our high-risk series and reported that, while progress had been made in improving the management of IT acquisitions and operations, significant work still remained to be completed. For example, as of March 2018, OMB and agencies had fully implemented 476 (or about 59 percent) of the 803 recommendations. Figure 1 summarizes the progress that OMB and agencies have made in addressing our recommendations as compared to the 80 percent target, as of March 2018.

In addition, in fiscal year 2016, we made 202 new recommendations, thus further reinforcing the need for OMB and agencies to address the shortcomings in IT acquisitions and operations. Also, beyond addressing our prior recommendations, our 2017 high-risk update noted the importance of OMB and covered federal agencies continuing to expeditiously implement the requirements of FITARA.

To further explore the challenges and opportunities to improve federal IT acquisitions and operations, we convened a forum on September 14, 2016, to explore challenges and opportunities for CIOs to improve federal IT acquisitions and operations—with the goal of better informing policymakers and government leadership. Forum participants, which included 13 current and former federal agency CIOs, members of Congress, and private sector IT executives, identified key actions related to seven topics: (1) strengthening FITARA, (2) improving CIO authorities, (3) budget formulation, (4) governance, (5) workforce, (6) operations, and (7) transition planning. A summary of the key actions, by topic area, identified during the forum is provided in figure 2.

In addition, in January 2017, the Federal CIO Council concluded that differing levels of authority over IT-related investments and spending have led to inconsistencies in how IT is executed from agency to agency. According to the Council, for those agencies where the CIO has broad authority to manage all IT investments, great progress has been made to streamline and modernize the federal agency’s footprint. For the others, where agency CIOs are only able to control pieces of the total IT footprint, it has been harder to achieve improvements.

Congress Has Taken Action to Continue Selected FITARA Provisions and Modernize Federal IT

Congress has recognized the importance of covered agencies’ continued implementation of FITARA provisions, and has taken legislative action to extend selected provisions beyond their original dates of expiration. Specifically, Congress and the President enacted laws to: remove the expiration date for enhanced transparency and improved risk management provisions, which were set to expire in 2019; remove the expiration date for portfolio review, which was set to expire in 2019; extend the expiration date for FDCCI from 2018 to 2020; and authorize the availability of funding mechanisms to help further agencies’ efforts to modernize IT.

In particular, a law was enacted to authorize the availability of funding to help further agencies’ efforts to modernize IT. The law, known as the Modernizing Government Technology (MGT) Act, authorizes agencies to establish working capital funds for use in transitioning from legacy IT systems, as well as for addressing evolving threats to information security. The law creates a technology modernization fund within the Department of the Treasury, from which agencies can “borrow” money to retire and replace legacy systems as well as acquire or develop systems.

The Current Administration Has Undertaken Efforts to Improve Federal IT

The current administration has initiated additional efforts aimed at improving federal IT, including digital services. Specifically, in March 2017, the administration established the Office of American Innovation, which has a mission to, among other things, make recommendations to the President on policies and plans aimed at improving federal government operations and services. In doing so, the office is to consult with both OMB and the Office of Science and Technology Policy on policies and plans intended to improve government operations and services, improve the quality of life for Americans, and spur job creation.

In May 2017, the administration also established the American Technology Council, which has a goal of helping to transform and modernize federal agency IT and how the federal government uses and delivers digital services. The President is the chairman of this council, and the Federal CIO and the United States Digital Service Administrator are among the members.

In addition, on May 11, 2017, the President signed Executive Order 13800, Strengthening the Cybersecurity of Federal Networks and Critical Infrastructure. This Executive Order tasked the Director of American Technology Council to coordinate a report to the President from the Secretary of the Department of Homeland Security, the Director of OMB, and the Administrator of the General Services Administration, in consultation with the Secretary of Commerce, regarding the modernization of federal IT. As a result, the Report to the President on Federal IT Modernization was issued on December 13, 2017, and outlined the current and envisioned state of federal IT. The report recognized that agencies have attempted to modernize systems but have been stymied by a variety of factors, including resource prioritization, ability to procure services quickly, and technical issues. The report provided multiple recommendations intended to address these issues through the modernization and consolidation of networks and the use of shared services to enable future network architectures.

In February 2018, OMB issued guidance for agencies to implement the MGT Act. The guidance was intended to provide agencies additional information regarding the Technology Management Fund, and the administration and funding of the related IT Working Capital Funds. Specifically, the guidance allowed agencies to begin submitting initial project proposals for modernization on February 27, 2018. In addition, in accord with the MGT Act, the guidance provides details of the Technology Modernization Board, which is to consist of (1) the Federal CIO; (2) a senior official from the General Services Administration; (3) a member of the Department of Homeland Security’s National Protection and Program Directorate; and (4) four federal employees with technical expertise in IT development, financial management, cyber security and privacy, and acquisition, appointed by the Director of OMB.

Agencies Can Improve IT Acquisitions and Operations

Agencies have taken steps to improve the management of IT acquisitions and operations. However, agencies would be better positioned to realize billions in cost savings and additional management improvements, if they addressed the numerous recommendations we have made aimed at improving data center consolidation, increasing transparency via OMB’s IT Dashboard, implementing incremental development, managing software licenses, reviewing IT acquisitions, implementing key IT workforce activities, and addressing aging legacy systems.

Agencies Have Made Progress in Consolidating Data Centers, but Need to Take Action to Achieve Planned Cost Savings

One of the key initiatives to implement FITARA is data center consolidation. OMB established FDCCI in February 2010 to improve the efficiency, performance, and environmental footprint of federal data center activities, and the enactment of FITARA codified and expanded the initiative. However, in a series of reports that we issued from July 2011 through August 2017, we noted that, while data center consolidation could potentially save the federal government billions of dollars, weaknesses existed in several areas, including agencies’ data center consolidation plans, data center optimization, and OMB’s tracking and reporting on related cost savings. In these reports, we made a matter for Congressional consideration, and a total of 160 recommendations to OMB and 24 agencies to improve the execution and oversight of the initiative. Most agencies and OMB agreed with our recommendations or had no comments. As of March 2018, 83 of these recommendations remained open.

For example, in May 2017, we reported that the 24 agencies participating in FDCCI collectively had made progress on their data center closure efforts. Specifically, as of August 2016, these agencies had identified a total of 9,995 data centers, of which they reported having closed 4,388, and having plans to close a total of 5,597 data centers through fiscal year 2019. Notably, the Departments of Agriculture, Defense, the Interior, and the Treasury accounted for 84 percent of the completed closures.

In addition, that report noted that 18 of the 24 agencies had reported achieving about $2.3 billion collectively in cost savings and avoidances from their data center consolidation and optimization efforts from fiscal year 2012 through August 2016. The Departments of Commerce, Defense, Homeland Security, and the Treasury accounted for approximately $2.0 billion (or 87 percent) of the total.

Further, 23 agencies reported about $656 million collectively in planned savings for fiscal years 2016 through 2018. This is about $3.3 billion less than the estimated $4.0 billion in planned savings for fiscal years 2016 through 2018 that agencies reported to us in November 2015. Figure 3 presents a comparison of the amounts of cost savings and avoidances reported by agencies to OMB and the amounts the agencies reported to us.

As mentioned previously, FITARA required agencies to submit no later than the end of fiscal year 2016 and annually thereafter multi-year strategies to achieve the consolidation and optimization of their data centers. Among other things, this strategy is required to include such information as data center consolidation and optimization metrics, and year-by-year calculations of investments and cost savings through October 1, 2020.

Further, OMB’s August 2016 guidance on data center optimization contained additional information for how agencies are to implement the strategic plan requirements of FITARA, and stated that agencies were required to publicly post their strategic plans to their agency-owned digital strategy websites by September 30, 2016.

As of April 2017, only 7 of the 23 agencies that submitted their strategic plans—the Departments of Agriculture, Education, Homeland Security, and Housing and Urban Development; the General Services Administration; the National Science Foundation; and the Office of Personnel Management—had addressed all five elements required by the OMB memorandum implementing FITARA. The remaining 16 agencies either partially met or did not meet the requirements. For example, most agencies partially met or did not meet the requirements to provide information related to data center closures and cost savings metrics. The Department of Defense did not submit a plan and was rated as not meeting any of the requirements.

To better ensure that federal data center consolidation and optimization efforts improve governmental efficiency and achieve cost savings, in our May 2017 report, we recommended that 11 of the 24 agencies take actions to ensure that the amounts of achieved data center cost savings and avoidances are consistent across all reporting mechanisms. We also recommended that 17 of the 24 agencies each take action to complete missing elements in their strategic plans and submit their plans to OMB in order to optimize their data centers and achieve cost savings. Twelve agencies agreed with our recommendations, 2 did not agree, and 10 agencies and OMB did not state whether they agreed or disagreed.

More recently, in August 2017, we reported that agencies needed to address challenges in optimizing their data centers in order to achieve cost savings. Specifically, we noted that, according to the 24 agencies’ data center consolidation initiative strategic plans as of April 2017, most agencies were not planning to meet OMB’s optimization targets by the end of fiscal year 2018. Further, of the 24 agencies, 5—the Department of Commerce and the Environmental Protection Agency, National Science Foundation, Small Business Administration, and U.S. Agency for International Development—reported plans to fully meet their applicable targets by the end of fiscal year 2018; 13 reported plans to meet some, but not all, of the targets; 4 reported that they did not plan to meet any targets; and 2 did not have a basis to report planned optimization milestones because they do not report having any agency-owned data centers. Figure 4 summarizes agencies’ progress in meeting OMB’s optimization targets as of February 2017, and planned progress to be achieved by September 2017 and September 2018, as of April 2017.

FITARA required OMB to establish a data center optimization metric specific to measuring server efficiency, and required agencies to report on progress in meeting this metric. To effectively measure progress against this metric, OMB directed agencies to replace the manual collection and reporting of systems, software, and hardware inventory housed within agency-owned data centers with automated monitoring tools and to complete this effort no later than the end of fiscal year 2018. Agencies are required to report progress in implementing automated monitoring tools and server utilization averages at each data center as part of their quarterly data center inventory reporting to OMB.

As of February 2017, 4 of the 22 agencies reporting agency-owned data centers in their inventory—the National Aeronautics and Space Administration, National Science Foundation, Social Security Administration, and U.S. Agency for International Development—reported that they had implemented automated monitoring tools at all of their data centers. Further, 10 reported that they had implemented automated monitoring tools at between 1 and 57 percent of their centers, and 8 had not yet begun to report the implementation of these tools. In total, the 22 agencies reported that automated tools were implemented at 123 (or about 3 percent) of the 4,528 total agency-owned data centers, while the remaining 4,405 (or about 97 percent) of these data centers were not reported as having these tools implemented. Figure 5 summarizes the number of agency-reported data centers with automated monitoring tools implemented, including the number of tiered and non-tiered centers.

To address challenges in optimizing federal data centers, in our August 2017 report, we made recommendations to 18 agencies and OMB. Ten agencies agreed with our recommendations, three agencies partially agreed, and six (including OMB) did not state whether they agreed or disagreed.

Risks Need to Be Fully Considered When Agencies Rate Their Major Investments on OMB’s IT Dashboard

To facilitate transparency across the government in acquiring and managing IT investments, OMB established a public website—the IT Dashboard—to provide detailed information on major investments at 26 agencies, including ratings of their performance against cost and schedule targets. Among other things, agencies are to submit ratings from their CIOs, which, according to OMB’s instructions, should reflect the level of risk facing an investment relative to that investment’s ability to accomplish its goals. In this regard, FITARA includes a requirement for covered agency CIOs to categorize their major IT investment risks in accordance with OMB guidance.

Over the past 6 years, we have issued a series of reports about the Dashboard that noted both significant steps OMB has taken to enhance the oversight, transparency, and accountability of federal IT investments by creating its Dashboard, as well as concerns about the accuracy and reliability of the data. In total, we have made 47 recommendations to OMB and federal agencies to help improve the accuracy and reliability of the information on the Dashboard and to increase its availability. Most agencies agreed with our recommendations or had no comments. As of March 2018, 19 recommendations remained open.

In June 2016, we determined that 13 of the 15 agencies selected for in- depth review had not fully considered risks when rating their major investments on the Dashboard. Specifically, our assessments of risk for 95 investments at the 15 selected agencies matched the CIO ratings posted on the Dashboard 22 times, showed more risk 60 times, and showed less risk 13 times. Figure 6 summarizes how our assessments compared to the selected investments’ CIO ratings.

Aside from the inherently judgmental nature of risk ratings, we identified three factors which contributed to differences between our assessments and the CIO ratings:

Forty of the 95 CIO ratings were not updated during April 2015 (the month we conducted our review), which led to differences between our assessments and the CIOs’ ratings. This underscores the importance of frequent rating updates, which help to ensure that the information on the Dashboard is timely and accurately reflects recent changes to investment status.

Three agencies’ rating processes spanned longer than 1 month.

Longer processes mean that CIO ratings are based on older data, and may not reflect the current level of investment risk.

Seven agencies’ rating processes did not focus on active risks.

According to OMB’s guidance, CIO ratings should reflect the CIO’s assessment of the risk and the investment’s ability to accomplish its goals. CIO ratings that do no incorporate active risks increase the chance that ratings overstate the likelihood of investment success.

As a result, we concluded that the associated risk rating processes used by the 15 agencies were generally understating the level of an investment’s risk, raising the likelihood that critical federal investments in IT are not receiving the appropriate levels of oversight.

To better ensure that the Dashboard ratings more accurately reflect risk, we made 25 recommendations to 15 agencies to improve the quality and frequency of their CIO ratings. Twelve agencies generally agreed with or did not comment on the recommendations and three agencies disagreed, stating that their CIO ratings were adequate. However, we noted that weaknesses in these three agencies’ processes still existed and that we continued to believe our recommendations were appropriate.

Agencies Need to Increase Their Use of Incremental Development Practices

OMB has emphasized the need to deliver investments in smaller parts, or increments, in order to reduce risk, deliver capabilities more quickly, and facilitate the adoption of emerging technologies. In 2010, it called for agencies’ major investments to deliver functionality every 12 months and, since 2012, every 6 months. Subsequently, FITARA codified a requirement that covered agency CIOs certify that IT investments are adequately implementing incremental development, as defined in the capital planning guidance issued by OMB. Further, subsequent OMB guidance on the law’s implementation, issued in June 2015, directed agency CIOs to define processes and policies for their agencies which ensure that they certify that IT resources are adequately implementing incremental development.

However, in May 2014, we reported that 66 of 89 selected investments at five major agencies did not plan to deliver capabilities in 6-month cycles, and less than half of these investments planned to deliver functionality in 12-month cycles. We also reported that only one of the five agencies had complete incremental development policies. Accordingly, we recommended that OMB clarify its guidance on incremental development and that the selected agencies update their associated policies to comply with OMB’s revised guidance (once made available), and consider the factors identified in our report when doing so.

Four of the six agencies agreed with our recommendations or had no comments, one agency partially agreed, and the remaining agency disagreed with the recommendations. The agency that disagreed did not believe that its recommendations should be dependent upon OMB taking action to update guidance. In response, we noted that only one of the recommendations to that agency depended upon OMB action, and we maintained that the action was warranted and could be implemented.

Subsequently, in August 2016, we reported that agencies had not fully implemented incremental development practices for their software development projects. Specifically, we noted that, as of August 31, 2015, 22 federal agencies had reported on the Dashboard that 300 of 469 active software development projects (64 percent) were planning to deliver usable functionality every 6 months for fiscal year 2016, as required by OMB guidance. The remaining 169 projects (or 36 percent) that were reported as not planning to deliver functionality every 6 months, agencies provided a variety of explanations for not achieving that goal. These included project complexity, the lack of an established project release schedule, or that the project was not a software development project.

Further, in conducting an in-depth review of seven selected agencies’ software development projects, we determined that 129 out of 287 software development projects delivered functionality every 6 months for fiscal year 2015 (45 percent) and 113 out of 206 software projects (55 percent) planned to do so in fiscal year 2016. However, significant differences existed between the delivery rates that the agencies reported to us and what they reported on the Dashboard. For example, for four agencies (the Departments of Commerce, Education, Health and Human Services, and the Treasury), the percentage of delivery reported to us was at least 10 percentage points lower than what was reported on the Dashboard. These differences were due to (1) our identification of fewer software development projects than agencies reported on the Dashboard and (2) the fact that information reported to us was generally more current than the information reported on the Dashboard.

We concluded that, by not having up-to-date information on the Dashboard about whether the project is a software development project and about the extent to which projects are delivering functionality, these seven agencies were at risk that OMB and key stakeholders may make decisions regarding the agencies’ investments without the most current and accurate information. As such, we recommended that the seven selected agencies review major IT investment project data reported on the Dashboard and update the information as appropriate, ensuring that these data are consistent across all reporting channels.

Finally, while OMB has issued guidance requiring agency CIOs to certify that each major IT investment’s plan for the current year adequately implements incremental development, only three agencies (the Departments of Commerce, Homeland Security, and Transportation) had defined processes and policies intended to ensure that the CIOs certify that major IT investments are adequately implementing incremental development. Accordingly, we recommended that the remaining four agencies—the Departments of Defense, Education, Health and Human Services, and the Treasury—establish policies and processes for certifying that major IT investments adequately use incremental development.

The Departments of Education and Health and Human Services agreed with our recommendation, while the Department of Defense disagreed and stated that its existing policies address the use of incremental development. However, we noted that the department’s policies did not comply with OMB’s guidance and that we continued to believe our recommendation was appropriate. The Department of the Treasury did not comment on its recommendation.

More recently, in November 2017, we reported that agencies needed to improve their certification of incremental development. Specifically, agencies reported that 103 of 166 major IT software development investments (62 percent) were certified by the agency CIO for implementing adequate incremental development in fiscal year 2017, as required by FITARA as of August 2016. Table 1 identifies the number of federal agency major IT software development investments certified for adequate incremental development, as reported on the IT Dashboard for fiscal year 2017.

Officials from 21 of the 24 agencies in our review reported that challenges hindered their ability to implement incremental development, which included: (1) inefficient governance processes; (2) procurement delays; and (3) organizational changes associated with transitioning from a traditional software methodology that takes years to deliver a product, to incremental development, which delivers products in shorter time frames. Nevertheless, 21 agencies reported that the certification process was beneficial because they used the information from the process to assist with identifying investments that could more effectively use an incremental approach, and used lessons learned to improve the agencies’ incremental processes.

In addition, as of August 2017, only 4 of the 24 agencies had clearly defined CIO incremental development certification policies and processes that contained descriptions of the role of the CIO in the process and how the CIO’s certification will be documented; and included definitions of incremental development and time frames for delivering functionality consistent with OMB guidance. Figure 7 summarizes our analysis of agencies’ policies for CIO certification of the adequate use of incremental development in IT investments.

Lastly, we reported that OMB’s capital planning guidance for fiscal year 2018 (issued in June 2016) lacked clarity regarding how agencies were to address the requirement for certifying adequate incremental development. While the 2018 guidance stated that agency CIOs are to provide the certifications needed to demonstrate compliance with FITARA, the guidance did not include a specific reference to the provision requiring CIO certification of adequate incremental development. We noted that, as a result of this change, OMB placed the burden on agencies to know and understand how to demonstrate compliance with FITARA’s incremental development provision. Further, because of the lack of clarity in the guidance as to what agencies were to provide, OMB could not demonstrate how the fiscal year 2018 guidance ensured that agencies provided the certifications specifically called for in the law.

In August 2017, OMB issued its fiscal year 2019 guidance, which addressed the weaknesses we identified in the previous fiscal year’s guidance. Specifically, the revised guidance requires agency CIOs to make an explicit statement regarding the extent to which the CIO is able to certify the use of incremental development, and to include a copy of that statement in the agency’s public congressional budget justification materials. As part of the statement, an agency CIO must also identify which specific bureaus or offices are using incremental development on all of their investments.

In our November 2017 report, we made 19 recommendations to 17 agencies to improve reporting and certification of incremental development. Eleven agencies agreed with our recommendations, 1 partially agreed, and 5 did not state whether they agreed or disagreed. OMB disagreed with several of our conclusions, which we continued to believe were valid.

In total, from May 2014 through November 2017, we made 42 recommendations to OMB and agencies to improve their implementation of incremental development. As of March 2018, 34 of our recommendations remained open.

Agencies Need to Better Manage Software Licenses to Achieve Savings

Federal agencies engage in thousands of software licensing agreements annually. The objective of software license management is to manage, control, and protect an organization’s software assets. Effective management of these licenses can help avoid purchasing too many licenses, which can result in unused software, as well as too few licenses, which can result in noncompliance with license terms and cause the imposition of additional fees.

As part of its PortfolioStat initiative, OMB has developed policy that addresses software licenses. This policy requires agencies to conduct an annual, agency-wide IT portfolio review to, among other things, reduce commodity IT spending. Such areas of spending could include software licenses.

In May 2014, we reported on federal agencies’ management of software licenses and determined that better management was needed to achieve significant savings government-wide. In particular, 22 of the 24 major agencies did not have comprehensive license policies and only 2 had comprehensive license inventories. In addition, we identified five leading software license management practices, and the agencies’ implementation of these practices varied.

As a result of agencies’ mixed management of software licensing, agencies’ oversight of software license spending was limited or lacking, thus potentially leading to missed savings. However, the potential savings could be significant considering that, in fiscal year 2012, 1 major federal agency reported saving approximately $181 million by consolidating its enterprise license agreements, even when its oversight process was ad hoc. Accordingly, we recommended that OMB issue needed guidance to agencies; we also made 135 recommendations to the 24 agencies to improve their policies and practices for managing licenses. Among other things, we recommended that the agencies regularly track and maintain a comprehensive inventory of software licenses and analyze the inventory to identify opportunities to reduce costs and better inform investment decision making.

Most agencies generally agreed with the recommendations or had no comments. As of March 2018, 95 of the recommendations had not been implemented. Table 2 reflects the extent to which agencies implemented recommendations in these areas.

Agencies Need to Ensure That IT Acquisitions Are Reviewed and Approved by Chief Information Officers

FITARA includes a provision to enhance covered agency CIOs’ authority through, among other things, requiring agency heads to ensure that CIOs review and approve IT contracts. OMB’s FITARA implementation guidance expanded upon this section of FITARA in a number of ways. Specifically, according to the guidance:

CIOs may review and approve IT acquisition strategies and plans, rather than individual IT contracts;

CIOs can designate other agency officials to act as their representatives, but the CIOs must retain accountability;

Chief Acquisition Officers (CAO) are responsible for ensuring that all IT contract actions are consistent with CIO-approved acquisition strategies and plans; and

CAOs are to indicate to the CIOs when planned acquisition strategies and acquisition plans include IT.

In January 2018, we reported that most of the CIOs at the 22 selected agencies were not adequately involved in reviewing billions of dollars of IT acquisitions. For instance, most of the 22 selected agencies did not identify all of their IT contracts. The selected agencies identified 78,249 IT-related contracts, to which they obligated $14.7 billion in fiscal year 2016. However, we identified 31,493 additional contracts with $4.5 billion obligated, raising the total amount obligated to IT contracts in fiscal year 2016 to at least $19.2 billion. Figure 8 reflects the obligations agencies reported to us relative to the obligations we identified.

The percentage of additional IT contract obligations we identified varied among the selected agencies. For example, the Department of State did not identify 1 percent of its IT contract obligation dollars. Conversely, 8 agencies did not identify over 40 percent of their IT-related contract obligation dollars. Many of the selected agencies that did not identify these IT acquisitions did not follow OMB guidance. Specifically, 14 of the 22 agencies did not involve the acquisition office in their process to identify IT acquisitions for CIO review, as required by OMB. In addition, 7 agencies did not establish guidance to aid officials in recognizing IT. Until agencies involve the acquisitions office in their IT identification processes and establish supporting guidance, they cannot ensure that they will identify all IT acquisitions. Without proper identification of IT acquisitions, agencies and CIOs cannot effectively provide oversight of these acquisitions.

In addition to not identifying all IT contracts, 14 of the 22 selected agencies did not fully satisfy OMB’s requirement that the CIO review and approve IT acquisition plans or strategies. Further, only 11 of 96 randomly selected IT contracts at 10 agencies that we evaluated were CIO- reviewed and approved as required by OMB’s guidance. The 85 IT contracts not reviewed had a total possible value of approximately $23.8 billion.

Until agencies ensure that CIOs are able to review and approve all IT acquisitions, CIOs will continue to have limited visibility and input into their agencies’ planned IT expenditures and will not be able to use the increased authority that FITARA’s contract approval provision is intended to provide. Further, agencies will likely miss an opportunity to strengthen CIOs’ authority and the oversight of IT acquisitions. As a result, agencies may award IT contracts that are duplicative, wasteful, or poorly conceived.

As a result of this report, we made 39 recommendations, including that agencies ensure that acquisition offices are involved in identifying IT and issue related guidance and ensure that IT acquisitions are reviewed according to OMB guidance. OMB and 20 agencies generally agreed with or did not comment on the recommendations. One agency agreed with one recommendation, but disagreed with another. The remaining agency disagreed with two recommendations. We subsequently removed one of these recommendations from the final report, but not the other. As of March 2018, all 39 recommendations remain open.

Implementing Key IT Workforce Planning Activities Can Help Ensure Acquisition Skill Gaps Are Addressed

An area where agencies can improve their ability to acquire IT is workforce planning. In November 2016, we reported that IT workforce planning activities, when effectively implemented, can facilitate the success of major acquisitions. Ensuring program staff have the necessary knowledge and skills is a factor commonly identified as critical to the success of major investments. If agencies are to ensure that this critical success factor has been met, then IT skill gaps need to be adequately assessed and addressed through a workforce planning process.

In this regard, we reported that four workforce planning steps and eight key activities can assist agencies in assessing and addressing IT knowledge and skill gaps. Specifically, these four steps are: (1) setting the strategic direction for IT workforce planning, (2) analyzing the workforce to identify skill gaps, (3) developing and implementing strategies to address IT skill gaps, and (4) monitoring and reporting progress in addressing skill gaps. Each of the four steps is supported by key activities (as summarized in table 3).

However, in our November 2016 report, we determined that the five agencies that we selected for in-depth analysis had not fully implemented key workforce planning steps and activities.For example, four of these agencies had not demonstrated an established IT workforce planning process. In addition, none of these agencies had fully assessed their workforce competencies and staffing needs regularly or established strategies and plans to address gaps in these areas. Figure 9 illustrates the extent to which the five selected agencies had fully, partially, or not implemented key IT workforce planning activities.

The weaknesses identified were due, in part, to these agencies lacking comprehensive policies that required such activities, or failing to apply the policies to IT workforce planning. We concluded that, until these weaknesses are addressed, the five agencies risk not adequately assessing and addressing gaps in knowledge and skills that are critical to the success of major acquisitions. Accordingly, we made five recommendations to the five selected agencies to address the weaknesses in their IT workforce planning practices that we identified. Four agencies—the Departments of Commerce, Health and Human Services, Transportation, and the Treasury—agreed with our recommendations and one, the Department of Defense, partially agreed. As of March 2018, the agencies had not addressed the five recommendations.

Agencies Need to Address Aging Legacy Systems

IT investments across the federal government are becoming increasingly obsolete. Specifically, in May 2016, we reported that many agencies were using systems which had components that were, in some cases, at least 50 years old. For example, we determined that the Department of Defense was using 8-inch floppy disks in a legacy system that coordinates the operational functions of the nation’s nuclear forces. In addition, the Department of the Treasury was using assembly language code—a computer language initially used in the 1950s and typically tied to the hardware for which it was developed. Further, in some cases, the vendors were no longer providing support for hardware or software. For example, each of the 12 agencies in our review reported using unsupported operating systems and components. At the time, five of the selected agencies reported using 1980s and 1990s Microsoft operating systems that stopped being supported by the vendor more than a decade ago. Table 4 provides examples of legacy systems across the federal government that agencies report are 30 years old or older and use obsolete software or hardware, and identifies those that do not have specific plans with time frames to modernize or replace these investments.

To address this issue, we recommended that 12 agencies identify and plan to modernize or replace legacy systems, including establishing time frames, activities to be performed, and functions to be replaced or enhanced. Most agencies agreed with our recommendations or had no comment. As of March 2018, all of the recommendations remained open.

In conclusion, the federal government has an opportunity to save billions of dollars; improve the transparency and management of IT acquisitions and operations; and to strengthen the authority of CIOs to provide needed direction and oversight. The forum we held also recommended that CIOs be given more authority, and noted the important role played by the Federal CIO.

Most agencies have taken steps to improve the management of IT acquisitions and operations by implementing key initiatives, including data center consolidation, efforts to increase transparency via OMB’s IT Dashboard, incremental development, management of software licenses, approval of IT acquisitions, implementation of IT workforce key practices, and addressing legacy IT; and they have continued to address recommendations we have made over the past several years. However, additional improvements are needed, and further efforts by OMB and federal agencies to implement our previous recommendations would better position them to improve the management of IT acquisitions and operations.

To help ensure that these efforts succeed, OMB’s and agencies’ continued implementation of recommendations is essential. In addition, we will continue to monitor agencies’ implementation of our previous recommendations.

Chairmen Meadows and Hurd, Ranking Members Connolly and Kelly, and Members of the Subcommittees, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contacts and Staff Acknowledgments

If you or your staff have any questions about this testimony, please contact Dave Powner, Director, Information Technology at (202) 512- 9286 or pownerd@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this testimony are Kevin Walsh (Assistant Director), Chris Businsky, Rebecca Eyler, Meredith Raymond, and Jessica Waselkow (Analyst in Charge).

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.