Information Technology: Agencies Need to Improve Their Application Inventories to Achieve Additional Savings
Published: 2016-09-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The federal government is expected to spend more than $90 billion on IT in fiscal year 2017. This includes a variety of software applications supporting agencies' enterprise needs. Since 2013, OMB has advocated the use of application rationalization. This is a process by which an agency streamlines its portfolio of software applications with the goal of improving efficiency, reducing complexity and redundancy, and lowering the cost of ownership.
GAO's objectives were to determine (1) whether agencies have established complete application inventories and (2) to what extent selected agencies have developed and implemented processes for rationalizing their portfolio of applications. To do this, GAO assessed the inventories of the 24 CFO Act agencies against four key practices and selected six agencies—the Departments of Defense, Homeland Security, the Interior, Labor, and NASA and NSF—due to their IT spending, among other factors, to determine whether they had processes addressing applications.
section_title: What GAO Found
paragraphs: Most of the 24 Chief Financial Officers (CFO) Act of 1990 agencies in the review fully met at least three of the four practices GAO identified to determine if agencies had complete software application inventories. To be considered complete, an inventory should (1) include business and enterprise information technology (IT) systems as defined by the Office of Management and Budget (OMB); (2) include these systems from all organizational components; (3) specify application name, description, owner, and function supported; and (4) be regularly updated. Of the 24 agencies, 4 (the Departments of Defense, Homeland Security, and Justice, and the General Services Administration) fully met all four practices, 9 fully met three practices, 6 fully met two practices, 2 fully met one practice, and 3 did not fully meet any practice (see figure).
A January 2016 OMB requirement to complete an IT asset inventory by the end of May 2016 contributed to most of the agencies fully meeting the first three practices. Agencies that did not fully address these practices stated, among other things, their focus on major and high risk investments as a reason for not having complete inventories. However, not accounting for all applications may result in missed opportunities to identify savings and efficiencies. It is also inconsistent with OMB guidance regarding implementation of IT acquisition reform law, referred to as the Federal Information Technology Acquisition Reform Act, which requires that Chief Information Officers at covered agencies have increased visibility into all IT resources. Not accounting for all applications also presents a security risk since agencies can only secure assets if they are aware of them.
Each of the six selected agencies relied on their investment management processes and, in some cases, supplemental processes to rationalize their applications to varying degrees. However, five of the six agencies acknowledged that their processes did not always allow for collecting or reviewing the information needed to effectively rationalize all their applications. The sixth agency, the National Science Foundation (NSF), stated its processes allow it to effectively rationalize its applications, but agency documentation supporting this assertion was incomplete. Only one agency—the National Aeronautics and Space Administration (NASA)—had plans to address shortcomings. Taking action to address identified weaknesses with agencies' existing processes for rationalizing applications would assist with identifying additional opportunities to reduce duplication and achieve savings.
section_title: What GAO Recommends
paragraphs: GAO is recommending that 20 agencies improve their inventories and five of the selected agencies take actions to improve their processes to rationalize their applications more completely. The Department of Defense disagreed with both recommendations made to it. After reviewing additional evidence, GAO removed the recommendation associated with improving the inventory but maintained the other. The other agencies agreed to or had no comments on the draft report.

section_title: Letter
paragraphs: In fiscal year 2017, the federal government is expected to spend more  than $90 billion on information technology (IT), including software  applications. Applications are software components and supporting  software hosted on an operating system that create, use, modify, share,  or store data in order to enable a business or mission function to be  performed. This includes custom, commercial off-the-shelf, government  off-the-shelf, or open-sourced software. In a memorandum issued in  March 2013, the Office of Management and Budget (OMB) advocated the  use of application rationalization—streamlining the portfolio with the goal  of improving efficiency, reducing complexity and redundancy, and  lowering the cost of ownership. Through this process agencies can  identify duplicative, wasteful, and low-value applications and identify  opportunities for savings.
You asked us to review federal agencies’ efforts to rationalize their  portfolio of applications. Our objectives were to determine (1) whether  agencies have established complete application inventories and (2) to  what extent selected agencies have developed and implemented  processes for rationalizing their portfolio of applications. For consistency,  we defined applications as those commodity IT assets associated with  enterprise IT systems and business systems commodity IT categories  identified in OMB guidance.
To address the first objective, we identified four practices for establishing  complete inventories. We derived them primarily from our guide for  assessing the reliability of computer-processed data and best practices  identified in our 2014 report on federal software licenses. These practices  are (1) including the business and enterprise IT systems defined by OMB,  (2) including systems from all organizational components, (3) specifying  basic application attributes—namely application name, description,  owner, and function supported—, and (4) regularly updating the inventory  with quality controls to ensure the reliability of the data in the inventory.  The 24 Chief Financial Officers (CFO) Act of 1990 agencies provided us  with their software application inventories. We analyzed the inventories,  reviewed documentation, and interviewed agency staff to determine the  extent to which agencies implemented the practices we identified.
To address the second objective, we selected six agencies—the  Departments of Defense (DOD), Homeland Security (DHS), Labor  (Labor), and the Interior (Interior); the National Aeronautics and Space  Administration (NASA); and National Science Foundation (NSF). We  selected them based on their fiscal year 2015 IT spending—we selected  two large agencies, two medium agencies, and two small agencies—and  whether they claimed to have an application rationalization process. We  also included agencies recognized for effective rationalization efforts  based on our research and OMB observations. We identified a set of  common application rationalization practices, reviewed documentation,  and interviewed agency officials to determine whether the agencies had  processes addressing these practices.
We conducted this performance audit from May 2015 to September 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix I for a more  detailed discussion of our objectives, scope, and methodology.
subsections: section_title: Background
paragraphs: In March 2012, OMB launched the PortfolioStat initiative which required  agencies to conduct an annual review of their commodity IT portfolio to,  among other things, achieve savings by identifying opportunities to  consolidate investments or move to shared services. For PortfolioStat,  OMB defined broad categories of commodity IT: enterprise IT systems, which include e-mail, identity and access  management, IT security, web infrastructure, and collaboration tools; business systems, which include finance, human resources, and other  IT infrastructure, which includes data centers, networks, desktop  computers, and mobile devices.
Of those categories, the first two include software applications, which are  software components and supporting software hosted on an operating  system that create, use, modify, share, or store data in order to enable a  business or mission function to be performed. This includes custom,  commercial off-the-shelf, government off-the-shelf, or open-sourced  software. The memorandum establishing the PortfolioStat initiative also  required agencies to develop a commodity IT baseline including the  number, types, and costs of investments for all commodity IT categories.
In a subsequent memorandum, OMB advocated the use of application  rationalization to inform data center optimization efforts. Application  rationalization is the process of streamlining the portfolio to improve  efficiency, reduce complexity and redundancy, and lower the cost of  ownership. It can be done by retiring aging and low-value applications,  modernizing aging and high-value applications, eliminating redundant  applications, standardizing on common technology platform and version  (as is the case for moving to shared services), or consolidating  applications. OMB stated in its memorandum that application  rationalization would be a focus of PortfolioStat sessions and required  agencies to describe their approach to maturing the IT portfolio, including  rationalizing applications, in the information resource management plans  and enterprise roadmaps that are required to be updated annually.
In December 2014, the law commonly referred to as the Federal  Information Technology Acquisition Reform Act (FITARA) was enacted  and required covered executive branch agencies (except for DOD) to  ensure that Chief Information Officers (CIO) have a significant role in the  decision making process for IT budgeting, as well as the management,  governance, and oversight processes related to IT. The act also required  that CIOs (in each covered agency except DOD) review and approve (1)  all contracts for IT services prior to their execution and (2) the  appointment of any other employee with the title of CIO, or who functions  in the capacity of a CIO, for any component organization within the  agency. OMB issued guidance in June 2015 that reinforces the  importance of agency CIOs and describes how agencies are to  implement the law.
In that same memorandum, OMB changed PortfolioStat from being an  annual review session to quarterly reviews including a discussion of  portfolio optimization efforts and focus on commodity IT. Specifically, the  memorandum stated that agencies are to discuss how they use category  management to consolidate commodity IT assets; eliminate duplication  between assets; and improve procurement and management of  hardware, software, network, and telecom services during the sessions.  Furthermore, agencies are to share lessons-learned related to commodity  IT procurement policies and efforts to establish enterprise-wide  inventories of related information. The memorandum also specified key  responsibilities for CIOs—including having increased visibility into all IT  resources—and required agencies to develop plans to implement these  responsibilities by December 2015.
Further, during the course of our review, in January 2016, OMB updated  guidance to agencies requiring that they provide information regarding  their IT asset inventories when making integrated data collection  submissions. The guidance required agencies to provide a preliminary  inventory by the end of February 2016 and a complete IT asset inventory,  including information on systems, sub-systems, and applications by the  end of May 2016 to OMB.
Finally, federal law and guidance specify requirements for protecting  federal information and systems. Specifically, the Federal Information  Security Management Act (FISMA)  of 2002, among other things,  requires agencies to maintain and update an inventory of major  information systems at least annually, and the National Institute of  Standards and Technology specifies that this should include an accurate  inventory of software components, including the software applications  which are the subject of our review. OMB plays a key role in monitoring  and overseeing agencies’ security activities and their FISMA  implementation. This includes tracking how well agencies are managing  their inventories of hardware and software assets and protecting them.
subsections: section_title: GAO Has Reported on Efforts Related to Application Rationalization
paragraphs: In November 2013, we reported that agency commodity IT baselines  were not all complete and recommended that 12 agencies complete their  commodity IT baselines. As of March 2016, 6 of the 12 agencies—the  Departments of Agriculture, Commerce, Housing and Urban  Development, and Labor; the Social Security Administration; and the U.S.  Agency for International  Development—reported that they had completed  their commodity IT baseline. The remaining 6 agencies reported making  progress towards completion.
In May 2014, in a review examining federal agencies’ management of  software licenses (which are types of enterprise IT applications), we  determined, among other things, that only 2 of the 24 CFO Act  agencies—the Department of Housing and Urban Development and the  National Science Foundation—had comprehensive software license  inventories. Twenty had partially complete inventories and two did not  have any inventory. We recommended that agencies complete their  inventories. We also recommended that OMB issue a directive to help  guide agencies in managing licenses and that the 24 agencies improve  their policies and practices for managing licenses. In June 2016, OMB  issued a memorandum that is intended to improve agencies’ acquisition  and management of enterprise software, consistent with our May 2014  recommendation. The memorandum contains elements related to  having a comprehensive policy, such as developing  and implementing a  plan for centralizing the management of software licenses.
section_title: Most Agencies Fully Met at Least Three of the Four Practices for Establishing Complete Application Inventories
paragraphs: We identified four practices to determine whether agencies had a  complete software application inventory. To do so, we primarily relied on  best practices used in our recent report on federal software licenses  which determined, among other things, whether agencies had a  comprehensive software license inventory, and our guide for assessing  the reliability  of computer-processed data. We determined that to be  considered complete agencies’ inventories should:  include business systems and enterprise IT systems, as defined by  OMB;  include these systems from all organizational components; specify basic attributes, namely application name, description, owner,  and function supported; and be regularly updated with quality controls in place to ensure the  reliability of the information collected.
Most of the agencies fully met at least three of the four practices.  Specifically,
4 agencies fully met all four practices;
9 agencies fully met three practices and 8 of these partially met the
6 agencies fully met two practices and 5 of these partially met the
2 agencies fully met one practice and partially met the three others,
3 agencies did not fully meet any practice.
subsections: section_title: U.S. Agency for International Development
paragraphs: The following are examples of how we assessed agencies against our  practices. See appendix II for a detailed assessment of all the agencies.
The Environmental Protection Agency fully met three practices  and partially met one. The agency fully met the first practice  because its inventory includes enterprise IT and business systems,  with the exception of very small systems. In addition, it included  applications from all offices and regions in the organization. The  agency partially met the practice for including application attributes in  the inventory because, although it identifies the application name, and  description, component managing the applications, and the business  function associated with its applications, it does not identify the  business function for every application. Officials stated that they are  working to have this information populated for all applications.  Lastly,  the agency fully met the fourth practice of regularly updating the  inventory because it has processes to update its inventory through the  agency’s software life cycle management procedure and provided  evidence of the annual data call issued by the CIO to ensure that the  inventory is current.
The U.S. Agency for International Development fully met two  practices and partially met two. Specifically, the agency’s inventory  includes business and enterprise IT systems and the inventory  includes basic application attributes. However, the agency’s inventory  does not include systems from all organizational components because  officials stated that coordination and communication in the  geographically-widespread agency is difficult. In addition, the agency  has processes for updating its inventory; however, it relies on manual  processes to maintain it.
The Department of Transportation partially met all four practices.
While the department’s inventory for the common operating  environment includes all business and enterprise IT systems and its  inventory of applications includes business systems, the inventory of  applications does not include all enterprise IT systems. Furthermore,  both of its inventories do not include applications used by all of its  components. Specifically, the inventory does not include applications  used by the Federal Highway, Federal Railroad, and Federal Transit  Administrations, among others, and the inventory for its common  operating environment does not include applications used by the  Federal Aviation Administration. The department also partially met the  practice of including basic application attributes because, although the  department’s inventory includes these attributes, its common  operating environment does not provide the business function that the  applications support. Further, while the Department of Transportation  has a process for its partners to provide information on its individual  inventories in order to update the inventory of applications, it does not  have processes in place to ensure the reliability and accuracy of the  reported information, and thus partially met this practice.
Regarding the four practices, the majority of the agencies fully met the  practices of including business systems and enterprise IT system;  including these systems from all organizational components; and  specifying the application name, description, owner, and business  function supported. Only five agencies fully met the practice of regularly  updating the inventory and implementing quality controls for ensuring the  reliability of the inventory data because they provided evidence of  performing both of these activities. Table 3 shows the number of agencies  who fully met, partially met, and did not meet the practices.
OMB’s requirement for agencies to complete an IT asset inventory by the  end of May 2016 greatly contributed to most of the agencies including  business systems and enterprise IT systems for all of their organizational  components and specifying key attributes for them. Those agencies that  did not fully address these practices provided various reasons for not  doing so. For example, one agency stated that it has not made its  software application inventory a priority because it has been focusing on  major and high risk investments, while delegating applications to the  component level. Others noted that the lack of automated processes  make collecting complete inventory information difficult. Further, others  noted that it is challenging to capture applications acquired by  components in the department-wide inventory.
While it is reasonable to expect that priority be given to major and high  risk investments, applications are nevertheless part of the portfolio and  should be accounted for as such. Not accounting for them may result in  missed opportunities to identify savings and efficiencies. It is also  inconsistent with OMB guidance for implementing FITARA  which requires  that CIOs have increased visibility into all IT resources.
In addition, the lack of a comprehensive inventory presents a security  risk. If agencies are not aware of all of their assets, they cannot secure  them, resulting in a vulnerable posture. Given the importance of securing  federal systems and data to ensuring public confidence and the nation’s  safety, prosperity, and well-being, we designated federal information  security as a government-wide high-risk area in 1997. In 2003, we  expanded this area to include computerized systems supporting the  nation’s critical infrastructure. In our high risk update in February 2015,  we further expanded this area to include protecting the privacy of  personal information that is collected, maintained, and shared by both  federal and nonfederal entities.
section_title: Agencies Rationalize Some, but Not All Applications through Existing Investment Management Processes
paragraphs: As previously noted, application rationalization is the process of  streamlining the portfolio to improve efficiency, reduce complexity and  redundancy, and lower the cost of ownership. It can be done in many  ways, including retiring aging and low-value applications, modernizing  aging and high-value applications, eliminating redundant applications,  standardizing on common technology platform and version (as is the case  for moving to shared services), or consolidating applications. Based on  common practices identified in technical papers from industry experts, to  effectively perform rationalization, an agency should first establish a  complete inventory of applications. It should then collect and review cost,  technical, and business value information for each application, and use  that information to make rationalization decisions. These practices are  consistent with those used to manage investment portfolios. Therefore an  agency can achieve application rationalization through established  practices related to investment management, including budget  formulation, security, or enterprise architecture.
Each of the six selected agencies relied on their investment management  processes and, in some cases, supplemental processes to rationalize  their applications to varying degrees. However, five of the six agencies  acknowledged that their processes did not always allow for collecting or  reviewing the information needed to effectively rationalize all their  applications. The sixth agency, NSF, stated its processes allow it to  effectively rationalize its applications, but we found supporting  documentation to be incomplete. Only one agency, NASA,  had plans to  address shortcomings. The following describes the six selected agencies’  processes for rationalizing their applications, provides rationalization  examples, identifies weaknesses and challenges, and addresses plans, if  any, the agencies have for addressing them.
DOD: The department uses its investment management process for  defense business systems  to annually review its applications.  Officials noted that the department’s enterprise architecture is also  used to identify duplication and overlap among these applications. In  addition, the department has identified eight enterprise common  services for collaboration, content discovery, and content delivery it is  requiring its components to use to, among other things, improve  warfighting efficiency and reduce costs.
One example of rationalization that DOD provided resulting from its  efforts with Executive Business Information System that was replaced  by the Navy Enterprise Resource Planning system in a full migration  in 2014. Estimated cost savings or avoidances were estimated at  $268,000 in fiscal year 2012 and almost $200,000 per year in fiscal  years 2013 through 2015. In addition, in an effort to improve its  financial management systems, the department has efforts underway  to reduce the number of financial management systems from 327 to  120 by fiscal year 2019.
However, officials acknowledged that its processes do not address all  applications. Specifically, according to information provided by the  department, about 1,200 enterprise IT and business systems which  are associated with the Enterprise Information Environment Mission  Area are not reviewed by the department—though they are reviewed  by components—because they do not meet the definition of a defense  business system.
Officials cited several challenges with implementing systematic  rationalization efforts, including the department’s organizational  structure and contractual agreements. As an example, they noted that  the Navy’s Next Generation e-mail system is being procured through  a contract with a particular vendor and as such would be difficult to  consolidate with other department e-mail systems. They also noted  that the cost of collecting additional cost, technical, and business  value information, along with maintaining even more data at greater  granularity, may outweigh the benefits.
The department does not have plans at this time to further enhance its  processes to rationalize its applications. While we recognize the  challenges and costs that may be associated with systematic  rationalization efforts, the Enterprise Information Environment Mission  Area could be considered as a near-term target for rationalization  given the large number of enterprise IT and business systems  associated with it. Modifying  existing processes to allow for the  collection, review, and evaluation of cost, technical, and business  information of these systems at the department level could help  identify opportunities for savings and efficiencies.
DHS: DHS has several processes for rationalizing applications. For  example, through its investment management process, portfolios are  regularly assessed against criteria which help identify duplication. In  addition, the department uses its DHS Collaborative Architecture  Methodology in conjunction with its segment architectures to help  identify duplication and fragmentation, at different levels, including at  the application level. The DHS IT Duplication Reduction Act of 2015  mandated the department to report on a strategy for reducing  duplicative IT systems  and the department used the DHS  Collaborative Architecture Methodology process to address this  mandate, including about 700 commodity IT and back-office  applications in the scope of the effort.
Further, the department recently established an Application Services  Council, chaired by its Enterprise Business Management Office.  According to its charter, the council is a cross-component and cross- disciplined leadership team responsible for developing,  maintaining,  and overseeing the Enterprise Information Technology Services  Portfolio, Lifecycle Governance Model, and Roadmap. It is expected  to take a strategic approach to evaluating existing and future IT  service offerings—including software, platform, and infrastructure  services—and provide a forum to identify strategies, best practices,  processes, and approaches for enterprise IT services, cloud  computing, and shared service challenges. For example, officials  reported the council is currently developing a standard service level  agreement template and guidance, as well as a cloud adoption  strategy. The department also reported other mechanisms related to  rationalization include its Joint Requirements Council, strategic  sourcing initiatives, IT acquisition reviews, and executive-level  portfolio reviews.
In addition, it reported that it uses its DHS Enterprise Architecture  Information Repository Technical Reference Model to track  application products and software versions—mainly consisting of  commercial off-the-shelf software. The product information is gathered  through the use of continuous network discovery scans.
Examples of rationalization include the consolidation of learning  management systems and the consolidation of site services, including  help desk operations. The consolidation of learning management  systems was identified through the segment architecture process and  is expected to result in projected savings of 10 to 20 percent in fiscal  year 2016 after transition costs are addressed. The modernization of  the department’s help desk and on-site operations resulted in savings  that cumulatively accrued to $202 million by fiscal year 2015 due to  similar efforts among all department components.
However, DHS’s processes do not address all applications because,  while the components may carry out their own rationalization efforts,  the department does not always collect the application-level  cost,  technical, or business information for applications used by its  components. Specifically, officials reported challenges tracking  product level information for deployed applications and difficulty  gaining visibility into all the supporting application products for large  systems. Officials particularly noted they have been challenged to  collect such information and cited a general lack of visibility into the  components’ budget and their spending. They also noted it was not  clear whether there was a good return on investment for the  resources needed to collect additional technical, cost, and business  value data for systematic application rationalization efforts. Officials  reported the department had a financial systems modernization effort  underway which would provide greater visibility into components’  spending but they did not have a plan to address the collection and  review of technical and business value information.
While we recognize that collecting additional details on all applications  may not be cost-beneficial, the department could consider taking a  segmented approach and initially identify one high-cost function it is  currently not collecting or reviewing detailed cost, technical, and  business information for across the department. It could then modify  existing processes to collect and review this information. These  actions would assist the CIO in gaining visibility into all IT resources  as specified in the OMB implementation guidance for FITARA  and  also help identify additional opportunities for savings and efficiencies.
NASA:  NASA uses its current investment management process—the  Capital Planning and Investment Control process—and its  configuration management tools—to review its applications.
NASA reported examples of rationalization resulting in significant  savings according to NASA officials. These included the NASA.gov  Portal Cloud Transition which resulted in estimated savings of $4  million and the Enterprise Business Portal Transition/Consolidation  which resulted in estimated savings of about $184,000 per year.
However, NASA officials acknowledged that their current processes  do not provide the level of detail needed to effectively rationalize the  agency’s applications.
In terms of challenges to rationalizing applications, officials stated that  it is difficult to obtain transparency on all applications since each of  the agency’s centers runs independently.  In addition, officials stated  that determining application business value is currently subjective to  users because the agency’s process for obtaining this information is  to ask the application owner the impact on the agency if the  application did not exist, whereas application technical health  information is more concrete. Furthermore, NASA officials stated that  there is no systematic process to review applications facing end-of-life  issues due to flat budgets and budget cuts.
NASA has developed a plan for a supplemental process (the annual  capital investment review process) that is to allow the agency to,  among other things, collect detailed data about its applications. The  agency has begun to implement the plan and has completed the first  milestone of the process, which included conducting a data call to  gather and validate application information provided by the various  centers and agency stakeholders. At the time of our review, NASA  had also performed an initial review and analysis of the information  collected and identified optimization opportunities, including  developing a plan to consolidate, decommission, or invest to achieve  maximum cost efficiencies and process effectiveness across the  application program. Fully implementing the annual capital investment  review process could better position the agency to identify additional  opportunities for savings and efficiencies.
Interior: As part of its budget formulation process, Interior performs  rationalization through annual reviews of its portfolio of investments  (and supporting applications) against criteria which measure business  value and technical fit.
Reported examples of application rationalization include Interior’s  cloud e-mail and collaboration services initiative, which consolidated  14 disparate systems into a single enterprise system and achieved a  cost savings/avoidance of $13.56 million, and the consolidation of the  Enterprise eArchive System with the eMail Electronic Records and  Document Management System which resulted in cost  savings/avoidance of $6.1 million.
However, the department reported that its portfolio review process is  not standardized because it has not been fully defined or established  in policy. In addition, it has only been used at the department level,  not at the bureaus or offices, and there is a lack of confidence in the  data that is collected to support the analyses. In comments on a draft  of this report, the department noted that it has also yet to document a  plan to implement policy associated with these efforts which they  believe would establish a standard analytical technique for  rationalizing the investment portfolio. Such a plan would also help  secure the commitment needed to carry out planned efforts.
The department reported several challenges to rationalizing its  applications, including (1) ensuring the quality and accuracy of data  collected since it relies largely on manual processes for collecting  information and (2) the lack of standard portfolio evaluation  techniques to support information resource management decision- making across the department.
The department has efforts underway which should help address  these challenges. Specifically, it is making changes to its information  resource management governance. According to the department,  these changes, combined with efforts to implement the CIO  responsibilities specified in FITARA, should help to address the  challenges to rationalizing its applications and allow for rationalization  of all applications. However, while the department has defined and  begun to implement criteria to assess whether or not an investment  and its underlying applications are wasteful, low-value, or duplicative,  it has not documented its plan for improving its governance—which,  according the department, would support application rationalization.  Such a plan would help secure the commitment needed to carry out  planned efforts.
Labor: Similar to the other agencies, the department uses its  investment management process to review the majority of its business  and enterprise IT applications. In addition, officials stated that the  department initiated an enterprise-wide budget formulation and  Information Technology Acquisition Review Board approval function  beginning in fiscal year 2013 which has helped with rationalization.
Officials stated that their efforts have resulted in rationalization of  commodity applications and on a case-by-case basis the  rationalization of other applications, such as for a case management  platform and an acquisition management system. Additional examples  of application rationalization include the deployment of a web-based  conferencing and collaboration shared service to employees which  resulted in cost avoidance of travel costs of about $2.3 million. The  department also noted benefits of moving to a cloud e-mail solution,  such as saved time and increased user satisfaction.
However, officials identified weaknesses and challenges with  rationalizing their applications. Specifically, they reported that, in most  cases, IT investments are associated with a group of IT assets,  including applications, and individual application information is  therefore not reviewed, making it difficult to effectively rationalize. In  addition, officials stated that the fact that each bureau-level agency  has had authority and responsibility for managing its own applications  and that the department has over 600 locations present challenges.  Further, though senior officials including the CIO, agreed with the  benefits of rationalization, they did not have any plans to rationalize.  They questioned the value of developing such plans stating that (1)  maintaining mission critical applications and the department’s aging  infrastructure are current priorities and (2) funding may not be  available to implement rationalization plans. While we agree that  mission critical applications should be given priority, rationalizing  mission support applications, including enterprise IT and business  systems, could result in solutions which allow agencies to focus more  on mission capabilities and at the same time generate savings which  could be reinvested. As we noted for DHS, the department could  consider taking a segmented approach to further rationalize and  identify a function for which it could modify existing processes to  collect and review detailed application cost, technical, and business  value information.
NSF: NSF also uses its investment management processes and  supporting budget formulation process—with key stakeholders such  as the Executive IT Resources Board, Capital Planning and  Investment Control Working Group, and Enterprise Architecture  Working Group—to collect and review information for its investments.  In addition, NSF’s Enterprise Modernization Roadmap—which is  updated annually—identifies applications along with their associated  business segment and modernization status and plans.
NSF identified its e-mail migration to a new platform, which was  completed in July 2013, as an example of an application  rationalization effort with the highest savings. According to the  agency’s November 2015 integrated data collection submission to  OMB, the migration effort resulted in cost avoidances of $60,000 in  2014. Other examples of application rationalization include  modernization and consolidation of NSF’s grant systems, the 2014  retirement of the financial functions of a legacy system, and the  implementation of its financial system modernization initiative.
However, while officials told us that evaluations for all applications  meeting the scope of our review would be included in the roadmap,  we only identified half of the applications (9 out of 18). In addition,  cost information was only provided in the roadmap for three individual  applications.
NSF officials told us that because they are a relatively small agency  with a single mission in a single location, many of their processes are  handled informally and not thoroughly documented but they are able  to discuss all the applications with each other on a regular basis and  as a result there is no duplication. Nevertheless, consistently  documenting the evaluations and costs for all applications in the  roadmap would improve transparency.
section_title: Conclusions
paragraphs: While it is encouraging that 13 of the 24 CFO Act agencies fully met at  least three of the four practices for establishing a complete software  application inventory, most could improve their software applications  inventories—albeit to varying degrees—by taking steps to fully meet the  practices we identified as being either partially met or not met. Doing so  would better position them to identify opportunities to rationalize their  applications, which could lead to savings and efficiencies. In addition,  they would be better positioned to comply with OMB issued  implementation guidance for the recent IT acquisition reform law which  requires that CIOs have increased visibility into all IT resources and  ensure they are effectively securing their IT assets.
Six selected agencies used their investment management processes and  sometimes supplemental processes to rationalize their applications. Of  the six agencies, one—NSF—had processes that allowed it to rationalize  all applications, though the supporting documentation was not always  complete. In addition, while the remaining five agencies’ processes did  not allow for rationalizing all applications, only one—NASA—had plans to  address identified weaknesses. While these agencies all had examples of  rationalization resulting in savings and efficiencies, modifying their  existing processes to more completely address their applications would  help identify additional opportunities to achieve such savings and  efficiencies, which even small, would add up across agencies and over  time.
section_title: Recommendations for Executive Action
paragraphs: To improve federal agencies’ efforts to rationalize their portfolio of  applications, we are recommending that:  the heads of the Departments of Agriculture, Commerce, Education,  Energy, Health and Human Services, Housing and Urban  Development, the Interior, Labor, State, Transportation, the Treasury,  and Veterans Affairs; and heads of the Environmental Protection  Agency; National Aeronautics and Space Administration; National  Science Foundation; Nuclear Regulatory Commission; Office of  Personnel Management; Small Business Administration; Social  Security Administration; and U.S. Agency for International  Development direct their CIOs and other responsible officials to  improve their inventories by taking steps to fully address the practices  we identified as being partially met or not met; and  the Secretaries of Defense, Homeland Security, the Interior, and  Labor; and the Director of the National Science Foundation to direct  the CIOs and other responsible officials to modify existing investment  management processes to address applications more completely.  Specifically,  the Secretary of Defense should direct the responsible official to  modify the department’s existing processes to collect and review  cost, technical, and business information for the enterprise and  business IT systems within the Enterprise Information  Environment Mission Area applications which are currently not  reviewed as part of the department’s process for business  systems;  the Secretary of Homeland Security should direct the  department’s CIO to identify one high-cost function it could collect  detailed cost, technical, and business information for and modify  existing processes to collect and review this information;  the Secretary of the Interior should direct the department’s CIO to   document and implement a plan for establishing policy that would   define a standard analytical technique for rationalizing the  investment portfolio;  the Secretary of Labor should direct the department’s CIO to  consider a segmented approach to further rationalize and identify  a function for which it would modify existing processes to collect  and review application-specific cost, technical, and business value  information; and  the Director of the National Science Foundation should direct the  CIO to consistently document evaluations for all applications and  report cost information for them in the roadmap or other  documentation.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the 24 CFO Act agencies in our  review for comment and received responses from all 24. Of the 24, 17  agreed with the recommendations directed to them; one (the Department  of Defense) disagreed with the recommendations directed to it; five (the  Department of the Treasury, the National Science Foundation, the  Nuclear Regulatory Commission, the Small Business Administration, and  the and U.S. Agency for International  Development) stated that they had  no comments; and one (the Department of Justice) agreed with the  assessment and conclusion for three of the four practices associated with  establishing a complete software application inventory and provided  clarifying information on the two other practices. Several agencies also  provided technical comments, which we incorporated as appropriate. The  agencies’ comments and our responses are summarized below.
In e-mail comments, the Department of Agriculture’s Senior Advisor  for Oversight and Compliance Enterprise Management stated that the  department concurred with our recommendation. The department also  provided technical comments which we incorporated as appropriate.  As a result of these comments and additional documentation  provided, we changed our evaluation of the practice associated with  updating the software application inventory from not met to partially  met.
In written comments, the Department of Commerce concurred with  our recommendation and stated that the department is committed to  implementing a more efficient process by regularly updating its  application inventory to ensure the reliability of the data collected. The  department also specified actions it plans to take to provide regular  updates of its inventory. The department’s comments are reprinted in  appendix III.
In written comments, the Department of Defense disagreed with both  of our recommendations to the department. For the first  recommendation, the department provided evidence showing that it  updated its inventory subsequent to us sending the report for  comment. As a result, we changed the rating for the related practice  from partially met to fully met and removed the associated  recommendation.
For the second recommendation, the department stated that 53  percent of the inventory records for the Enterprise Information  Environment Mission Area we focused on were IT infrastructure  assets (specifically network enclaves or circuits) and not applications  subject to rationalization. The mission area nevertheless includes  enterprise and business IT applications which could benefit from  rationalization, as evidenced by the example of e-mail system  consolidation provided in the comments. Given the number of  systems involved (at least 1,200), collecting and reviewing cost,  technical, and business information for them would help identify  additional rationalization opportunities  which could yield savings and  efficiencies. We therefore believe a recommendation to address these  systems is still warranted.
The department also stated that our draft implied that major IT  infrastructure modernization efforts, many of which involve the  Enterprise Information Environment Mission Area, were not reviewed  or properly managed by the department. However, as noted in our  report, we did not include IT infrastructure assets in the scope of our  review and therefore made no comment on how these assets are  being managed. We have restated our emphasis on enterprise and  business IT systems as it relates to the mission area where  appropriate.
Finally, in its comments the department stated that our report ignored  significant Enterprise Information Environment Mission Area  application rationalization efforts, such as the Pentagon IT  consolidation under the Joint Service Provider, the Business Process  and System Review, and ongoing efforts concerning public-facing  websites and associated systems. While we were not informed of  these efforts during our review, our intent was to highlight additional  opportunities for rationalization, not discount any that might have  already been implemented. The department also provided technical  comments, which we incorporated into the report as appropriate. The  department’s comments are reprinted in appendix IV.
In written comments, the Department of Education concurred with our  recommendation and described actions it plans to take to address it.  The department’s comments are reprinted in appendix V.
In written comments, the Department of Energy concurred with our  recommendation. In addition, the department stated that it partially  met the four practices associated with establishing a complete  software application inventory and provided the IT Asset Inventory it  submitted to OMB in May 2016 and other documentation supporting  this claim. Our review of the documentation found that the inventory  includes business and enterprise IT systems; however, it does not  include those systems from all organizational components and it is  missing the business function code for a large number of systems.  Furthermore, while the department is updating the IT Asset inventory  in response to OMB guidance for the fiscal year 2016 integrated data  collection submission process, it has not implemented quality control  processes to ensure the reliability of the data within the inventory. As  a result, we changed the department’s rating for the practice  associated with including business and enterprise IT systems from not  met to fully met and from not met to partially met for the remaining  three practices. We modified sections of the report specific to the  department accordingly. The department’s comments are reprinted in  appendix VI.
In written comments, the Department of Health and Human Services  concurred with our recommendation and stated that that it would  review the feasibility of fully addressing the practices it partially met.  The department’s comments are reprinted in appendix VII.
In written comments, the Department of Homeland Security concurred  with our recommendation and described actions it plans to take to  implement it. The department’s comments are reprinted in appendix  VIII.
In written comments, the Department of Housing and Urban  Development concurred with our recommendation and stated that  more definitive information with timelines will be provided once the  final report has been issued. The department’s comments are  reprinted in appendix IX.
In written comments, the Department of the Interior stated that it  would agree with the recommendations if we made its requested  changes. However, we disagreed with the request to change the  rating for the practice associated with regularly updating the inventory  from not met to partially met because, while the department provided  evidence supporting its claim that it recently updated its inventory, the  evidence was not sufficient. Specifically, the department provided an  e-mail requesting the bureaus and offices to complete an inventory  survey. However, the department did not show how the survey  resulted in updates to the inventory. We incorporated the remaining  requested changes in the report as appropriate. The department’s  comments are reprinted in appendix X.
In written comments, the Department of Justice stated that it  concurred with our assessment and conclusions. The department also  provided clarifying information regarding its procedures to ensure  application inventory accuracy and provided documentation showing  that it updates the inventory and implements quality controls to ensure  its reliability. As a result, we changed the rating for the related practice  from partially met to fully met and removed the recommendation made  to the department. The department’s comments are reprinted in  appendix XI.
In written comments, the Department of Labor concurred with our  recommendations to the department and stated that it would take the  necessary steps to address the recommendations. The department’s  comments are reprinted in appendix XII.
In written comments, the Department of State concurred with our  recommendation to the department, and described current and  planned actions to fully address it. The department’s comments are  reprinted in appendix XIII.
In e-mail comments, the Department of Transportation’s Audit Liaison  stated that the department concurred with our findings and  recommendation.
In e-mail comments, the Department of the Treasury’s Audit Liaison  stated that the department did not have any comments.
In written comments, the Department of Veterans Affairs concurred  with our conclusions and recommendation.  The department also  provided information on the actions it plans to take to address the  recommendation. The department’s comments are reprinted in  appendix XIV.
In written comments, the Environmental Protection Agency generally  agreed with our recommendation. The agency also asked that we  include some of the language from the detailed evaluation in appendix  II of the report to the example we have in the body to provide the full  context of its practices. We added the language as requested. The  agency’s comments are reprinted in appendix XV.
In e-mail comments, the General Services Administration’s Associate  CIO of Enterprise Planning and Governance concurred with the  report. The agency also provided evidence of its processes to update  the inventory and ensure the reliability of the data in the inventory,  including the coordination between its Enterprise Architecture Team  and subject matter experts. As a result, we changed the agency’s  rating for the related practice from partially met to fully met and  removed our recommendation to the agency.
In written comments, the National Aeronautics and Space  Administration concurred with our recommendation and stated that it  would utilize the capital investment review process it is currently  implementing to improve its inventory. The agency’s comments are  reprinted in appendix XVI.
In e-mail comments, the National Science Foundation Office of  Integrated Activities’ Program Analyst stated that it had no comments  on the draft report.
In written comments, the Nuclear Regulatory Commission stated that  it is in general agreement with the report. The agency’s comments are  reprinted in appendix XVII.
In written comments, the Office of Personnel Management concurred  with our recommendation and described plans to fully address it. The  agency’s comments are reprinted in appendix XVIII.
In e-mail comments, the Small Business Administration Office of  Congressional and Legislative Affairs’ Program Manager stated that  the Office of the Chief Information Officer believes the report captures  its current posture.
In written comments, the Social Security Administration agreed with  our recommendation to the agency, but disagreed with the partially  met rating for regularly updating the inventory, including implementing  quality controls, stating that it had provided evidence supporting its  implementation of the practice. However, as noted in the report, the  Social Security Administration reported that its systems development  lifecycle contains steps for maintaining the inventory but did not  provide evidence showing that it is using this process to regularly  update the inventory. Therefore we did not change our rating. The  agency’s comments are reprinted in appendix XIX.
In an e-mail, the U.S. Agency for International  Development Audit,  Performance and Compliance Division’s Management Analyst stated  that the agency did not have any comments.
We are sending copies of this report to interested congressional  committees; the heads of the Departments of Agriculture, Commerce,  Defense, Education, Energy, Health and Human Services, Homeland  Security, Housing and Urban Development, the Interior, Justice, Labor,  State, Transportation, the Treasury, and Veterans Affairs; the  Environmental Protection Agency; the General Services Administration;  the National Aeronautics and Space Administration; the National Science  Foundation; the Nuclear Regulatory Commission; the Office of Personnel  Management; the Small Business Administration; the Social Security  Administration; the U.S. Agency for International  Development; the  Director of the Office of Management and Budget; and other interested  parties. This report will also be available at no charge on our website at  http://www.gao.gov.
If you or your staff have any questions on matters discussed in this report,  please contact me at (202) 512-9286 or pownerd@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made major  contributions to this report are listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine (1) whether agencies have established  complete application inventories and (2) to what extent selected agencies  have developed and implemented processes for rationalizing their  portfolio of applications.
For the first objective, we reviewed the 24 major agencies covered by the  Chief Financial Officers (CFO) Act of 1990. To ensure consistency, we  decided to focus on the software applications associated with the  business and enterprise information technology (IT) commodity IT  categories defined in the Office of Management and Budget (OMB)  guidance since they would be familiar to the agencies in our scope. OMB  defines enterprise IT systems as e-mail, identity and access  management, IT security, web infrastructure, and collaboration tools; and  business systems as finance, human resources, and other administrative  functions.
We then identified practices to assess whether agencies had a complete  software application inventory. To identify these practices, we primarily  relied on our guide for assessing the reliability of computer-processed  data which addresses questions about the currency of the data and how  often it is updated, procedures for ensuring the completeness of the data,  and quality control processes in place to ensure the accuracy of the data;  and on criteria used in our recent report on federal software licenses  which determined whether agencies had a comprehensive software  license inventory, among other things. To be considered complete, we  determined an inventory should:  include business systems and enterprise IT systems, as defined by  OMB;  include these systems from all organizational components; specify basic attributes, namely, application name, description, owner,  and function supported; and be regularly updated with quality controls  in place to ensure the reliability  of the information collected.
Following the identification of these four practices, we asked the 24 CFO  Act agencies for their software application inventories. We used a set of  structured questions to determine whether the agencies implemented the  practices and identify lessons learned and challenges faced in  establishing a complete software application inventory. We analyzed  supporting documentation, such as agency and departmental guidance,  policies, and procedures for updating the inventories, and interviewed  relevant agency officials, as needed. We compared the information  received to the four practices. We determined a practice to be fully met if  agencies provided evidence that they fully or largely implemented the  practice for establishing a complete application inventory; partially met if  agencies provided evidence that they addressed some, but not all, of the  practice for establishing a complete application inventory; and not met if  the agencies did not provide any evidence that they implemented the  practice for establishing a complete application inventory.
To verify the inclusion of business and enterprise IT systems, we  analyzed agencies’ inventories and looked for examples of each type of  system identified by OMB in the business and enterprise IT commodity  categories. We followed up with agencies when we were not able to  identify a type of system to determine the reason for the omission. We  considered the practice to be fully met if agencies’ inventories included all  of the business and enterprise IT system types or if agencies provided  valid reasons for excluding them. We considered the practice to be  partially met if agencies acknowledged they were missing applications or  if we determined system types to be missing and agencies did not provide  a valid reason for this. Although we followed up with agencies to  determine whether they maintained separate inventories of software  licenses when they were not included in the inventories provided, we did  not consider the inclusion of these applications in determining our rating  because software licenses are expected to be tracked separately by  OMB.
To verify the inclusion of systems from all organizational components, we  analyzed agencies’ inventories against the list of organizational  components to determine whether they were included. We followed up  with agency officials to determine causes, if any, for missing components.  We considered the practice to be fully met if inventories included  applications from all organizational components or if agencies provided  valid reasons for excluding them. We considered the practice to be  partially met if agencies acknowledged they were missing organizational  components or if we determined several components to be missing and  agencies did not provide a valid reason for this.
Regarding application attributes, we determined that, at a minimum,  agencies should have a name, a description, an owner, and function  supported for each application. We considered the practice to be fully met  if inventories included these attributes for all or most applications or the  agencies provided evidence that attributes not included in the inventory  provided were being tracked separately. We determined the practice to  be partially met if agencies acknowledged that they were missing any of  the attributes or if we determined them to be missing from the inventory  and agencies did not provide alternate sources for them.
For the last practice, we determined whether agencies (1) used relevant  methods to update and maintain the application inventory and (2)  implemented controls to ensure the reliability of the information collected.  Regarding these controls, we looked for the use of automated tools to  collect and track information as their use increases reliability. We  determined the practice to be fully met if agencies provided evidence that  they regularly updated the inventory and had controls for ensuring the  reliability of information collected, including the use of automated tools, or  if agencies had mitigating factors when these processes were not in  place. We determined the practice to be partially implemented when  agencies provided policies and procedures but no evidence of actual  inventory updates or quality controls. We also determined the practice to  be partially implemented if agencies provided evidence of either regular  updates or controls for ensuring reliability  but not both or did not make  use of automated tools for collecting or maintaining information and had  no mitigation factors. Finally, we also determined the practice to be  partially implemented if agencies provided draft policy and guidance of  their processes.
For our second objective, we selected 6 of the 24 CFO Act agencies—the  Departments of Defense, Homeland Security, the Interior, and Labor; and  the National Aeronautics and Space Administration and National Science  Foundation—to assess their application rationalization plans and efforts  to implement them. We selected the agencies based on three factors: whether they had an application rationalization process; in our initial  set of structured questions to agencies, we asked whether they had a  plan or process for rationalizing applications and selected those that  reported having one;  the size of the agency based on fiscal year 2015 IT spending; we  selected two large agencies (i.e., with spending equal to or greater  than $3 billion), two medium agencies (i.e., with spending between $1  billion and $3 billion), and two small agencies (i.e., with spending of  less than $1 billion) for a full range of IT spending; and  if they were known for effectively rationalizing their applications based  on OMB observations and our research on IT acquisition reform  recognizing agencies for their application rationalization efforts.
We identified key practices for effectively rationalizing applications. To do  so, we reviewed OMB guidance on federal IT management. We also  reviewed technical reports on application rationalization from industry  experts. We synthesized the information collected, looked for themes,  and determined that, to effectively rationalize applications, agencies  should have a process addressing the following four key practices: establish an application inventory; collect information on each application, such as total cost, technical  details, and business value; evaluate the portfolio and make application rationalization decisions  based on a review of collected information and determine what  applications to retain, retire, replace, eliminate, modernize, or  consolidate/move to shared services; and execute and manage the process by implementing decisions from the  evaluation and evaluate process outcomes against defined metrics  and adjust, as needed.
While our research identified specific processes for rationalizing  applications, the principles of collecting application information and  reviewing it to inform decision making are consistent with those used to  manage investment portfolios. Therefore we considered established  practices related to investment management, budget formulation,  security, or enterprise architecture.
Since the first key practice was addressed in our first objective, we  focused on the last three practices. To do so, we interviewed relevant  officials using a structured set of questions that were developed in  conjunction with internal experts. We also reviewed documentation to  determine the extent to which agencies had processes addressing these  practices. We also asked agencies to provide their two best examples of  application rationalization in terms of savings or cost avoidance—to  illustrate the results of rationalization. When agencies did not provide two  examples meeting these conditions—the case for DOD, DHS, and NSF— we drew examples from other documentation they had provided.
Finally, we interviewed staff from OMB’s Office of the Federal Chief  Information Officer to determine whether and how the office monitors  agencies’ efforts to rationalize their portfolio of applications as  recommended in OMB guidance. We also interviewed the staff to  determine the impetus for the IT asset data inventory guidance and the  planned used for the information collected.
We conducted this performance audit from May 2015 to September 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: GAO’s Evaluation of Agencies’ Application Inventories
paragraphs: The following tables provide our evaluation of the 24 agencies’ application  inventories.
section_title: Appendix III: Comments from the Department of Commerce
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: Comments from the Department of Education
section_title: Appendix VI: Comments from the Department of Energy
section_title: Appendix VII: Comments from the Department of Health and Human Services
section_title: Appendix VIII: Comments from the Department of Homeland Security
section_title: Appendix IX: Comments from the Department of Housing and Urban Development
section_title: Appendix X: Comments from the Department of the Interior
section_title: Appendix XI: Department of Justice
section_title: Appendix XII: Comments from the Department of Labor
section_title: Appendix XIII: Comments from the Department of State
section_title: Appendix XIV: Comments from the Department of Veterans Affairs
section_title: Appendix XV: Comments from the Environmental Protection Agency
section_title: Appendix XVI: Comments from the National Aeronautics and Space Administration
section_title: Appendix XVII: Nuclear Regulatory Commission
section_title: Appendix XVIII: Comments from the Office of Personnel Management
section_title: Appendix XIX: Comments from the Social Security Administration
section_title: Appendix XX: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, the following staff made key  contributions to this report: Sabine Paul (Assistant Director), Chris  Businsky, Rebecca Eyler, Dan Gordon, James MacAulay, Lori Martinez,  Paul Middleton,  and Di’Mond Spencer.