DOD Business Transformation: Air Force's Current Approach Increases Risk That Asset Visibility Goals and Transformation Priorities Will Not Be Achieved
Published: 2008-08-08T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) established a goal to achieve total asset visibility over 30 years ago. This initiative aims to provide timely, accurate information on the location, movement, status, and identity of equipment and supplies. To date, the effort has been unsuccessful. GAO was requested to determine (1) the implementation status of the Air Force's business system initiatives to achieve total asset visibility, and whether the Air Force has implemented related best practices, and (2) whether the Air Force's business transformation efforts to achieve total asset visibility are aligned within the Air Force and with DOD's broader business transformation priorities. GAO interviewed Air Force officials and reviewed Air Force documentation to obtain an understanding of the Air Force's system initiatives and strategy for achieving total asset visibility and to identify areas for improvement.
section_title: What GAO Found
paragraphs: The Air Force has identified the Expeditionary Combat Support System (ECSS) and the Defense Enterprise Accounting and Management System (DEAMS) as key technology enablers of the Air Force's efforts to transform its logistics and financial management operations and achieve total asset visibility--a key DOD priority. ECSS is expected to provide a single, integrated logistics system, including transportation, supply, maintenance and repair, and other key business functions directly related to logistics such as engineering and acquisition. Additionally, ECSS will perform financial management and accounting for the Air Force working capital fund operations. ECSS is expected to be fully operational in fiscal year 2013, and replace about 250 legacy logistics and procurement systems. DEAMS is expected to provide the entire spectrum of core financial management capabilities, including collections, commitments/obligations, cost accounting, general ledger, funds control, receipts and acceptance, accounts payable and disbursement, billing, and financial reporting for the Air Force general fund operations. DEAMS is expected to replace seven legacy systems and be fully operational in fiscal year 2014. GAO identified several areas in which the Air Force had not fully implemented best practices related to risk management and system testing. These findings increase the risk that these business system initiatives will not meet their stated functionality, cost, and milestone goals, thereby limiting the Air Force's efforts to achieve total asset visibility and other DOD business transformation priorities. Further, key Air Force business transformation strategic plans and documents were not aligned within the Air Force nor with DOD's broader business transformation priorities. While each individual Air Force plan was intended to support the Air Force's business transformation efforts, the plans did not reflect a coordinated effort toward achieving a stated Air Force or DOD goal. For example, neither the Air Force's Military Equipment Accountability Improvement Plan for supporting DOD's military equipment valuation effort, nor the Air Force Logistics Enterprise Architecture Concept of Operations, its key strategic transformation plan for logistics, identified a shared relationship, including metrics, in supporting Air Force and DOD logistics and financial management transformation goals. As a result, neither the Air Force nor DOD will have the performance data needed to oversee efforts intended to improve the Air Force's ability to locate, manage, and account for assets throughout their life cycle.

section_title: Letter
paragraphs: Over 30 years ago, the Department of Defense (DOD) established a goal to  achieve total asset visibility in recognition of its importance to successful  operations. To date, DOD’s efforts have been unsuccessful. DOD defines  total asset visibility as the capability to provide timely, accurate  information on the location, movement, status or condition, and identity of  units, personnel, equipment, and supplies DOD-wide, and having the  capability to act on that information. If the information contained in the  asset accountability systems is not accurate, complete, and timely, DOD’s  day-to-day operations could be adversely affected by, for example,  investing in inventory that is not needed to meet current needs.
Asset visibility is one of three focus areas identified by the department as  critical to effective and efficient supply chain management operations.  The other two focus areas are inventory management and distribution.  Collectively, these three areas are integral to department logistical  operations and improvements are needed in each to address the problems  associated with DOD’s supply chain management, which has been on our  high-risk list since 1990. Of the 27 areas on our high-risk list, DOD          has 8 high-risk areas of its own, and shares responsibility for 7  governmentwide high-risk areas. The nature and severity of DOD’s  deficiencies in these high-risk areas impedes the ability of DOD managers  to receive the full range of information needed to effectively manage day- to-day operations. DOD’s efforts to achieve total asset visibility should  assist the department in addressing several of its high-risk areas, including  supply-chain management, business system modernizations, and financial  management by providing DOD management and the Congress with more  accurate and reliable information for decision making purposes.
Similar to the department as a whole, the Air Force continues to struggle  to achieve and maintain efficient and effective management control over  the hundreds of billions of dollars in inventory and other assets with  which it is entrusted. For example, in 1996, we reported that information  systems do not always provide Air Force managers and employees with  accurate, real-time data on the cost, amount, location, condition, and  usage of inventory—elements that are required to successfully plan,  control, and measure inventory management. Furthermore, in April 2007  we reported that more than half of the Air Force’s spare parts inventory  worth an average of $31.4 billion was not needed. Our report further  noted that the Air Force has continued to purchase unneeded inventory  because its policies do not provide incentives to manage the amount of  inventory on order that may not be needed to support its logistics  operations. The Air Force has acknowledged that its logistics operations  have largely been reactionary, rather than anticipatory, because  stovepiped business processes and systems impede visibility. Importantly,  Air Force logisticians have historically met the challenge of fulfilling their  mission within the constraints of archaic business processes and  nonintegrated systems, through increased personnel and financial  resources.
The Air Force’s vision for transforming its logistics operations to meet  both the current and future threat environment is broad in scope and  touches on all logistics functions, including transportation, maintenance,  procurement, inventory management, and product life-cycle management,  and crosses all commodity lines (such as fuel, munitions, aircraft, and  vehicles) and organizational boundaries. The Air Force has determined  that logistics must operate with an enterprise (Air Force-wide) view,  across integrated processes, while optimizing resources and leveraging  integrated technology “to deliver the right support, to the right place, at  the right time, every time.” The Air Force has a number of ongoing  technology, policy, organization, and process engineering initiatives to  enhance asset visibility. Among these are asset marking and tracking  initiatives, which are intended to enable more accurate and timely  recording of asset information such as location, condition, and status  through the use of unique item identification and radio frequency  identification of individual or groups of assets. In addition, the Air Force  has undertaken two business system initiatives, the Expeditionary Combat  Support System (ECSS) and the Defense Enterprise Accounting and  Management System (DEAMS) that it has identified as key to its efforts to  transform its logistics and financial management operations, and achieve  total asset visibility. ECSS will primarily be responsible for performing the  logistical functions for both the Air Force’s general and working capital  funds and the financial management functions for the Air Force’s working  capital funds. DEAMS will perform the financial management functions for  the Air Force’s general funds. Although ECSS and DEAMS are important  technology enablers, their ability to provide the information intended by  Air Force to support total asset visibility, including asset accountability  and decision making support, is largely dependent upon the Air Force’s  success in integrating, managing, and completing other transformation  initiatives outlined in a variety of Air Force strategic plans and documents.
This report provides information in support of your continuing oversight  of DOD’s progress towards resolving the department’s long-standing  problems in achieving total asset visibility. In July 2007, we reported on  the Army’s efforts to achieve total asset visibility. Our objectives were to  determine (1) the implementation status of the Air Force’s current  business system initiatives to achieve total asset visibility, and whether the  Air Force has implemented related best practices, and (2) whether the Air  Force’s business transformation efforts to achieve total asset visibility are  aligned within the Air Force and with DOD’s broader business  transformation priorities.
To address the first objective, we analyzed documentation and met with  Air Force and DOD Business Transformation Agency officials to identify  and determine the implementation status of key Air Force business system  initiatives intended to support the Air Force’s goal of achieving total asset  visibility. Further, we interviewed and obtained briefings from Air Force  logistics and financial management officials and others on ECSS and  DEAMS program management and oversight. Specifically, we obtained  and reviewed documentation related to each system initiative, including  costs, implementation schedules, and risk management programs, to  assess their acquisition status and to determine whether improvements  were needed in the Air Force’s approach for acquiring and implementing  these systems. We did not review ECSS and DEAMS compliance with the  Air Force’s enterprise architecture because our related work focused on  ascertaining the status of the military services’ efforts to develop and use  an enterprise architecture. The results of this work are discussed in our  May 2008 report, which noted that while the Air Force’s efforts to develop  an enterprise architecture were ahead of the Army’s and the Navy’s efforts,  the Air Force’s architecture was not sufficiently developed to guide and  constrain its business systems modernization investments. To address  the second objective, we obtained and analyzed key Air Force business  transformation strategic plans to assess integration and utilization of  metrics in supporting and managing the Air Force’s efforts to transform its  business operations and achieve total asset visibility. Additionally, we  compared key Air Force business transformation plans with DOD’s  Enterprise Transition Plan to determine if DOD’s business enterprise  transformation priorities were incorporated into the Air Force’s plans.
We conducted this performance audit from July 2007 through August 2008  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Details on our scope and  methodology are included in appendix I. We requested comments on a  draft of this report from the Secretary of Defense or his designee. We  received written comments from the Deputy Under Secretary of Defense  (Business Transformation), which are reprinted in appendix II.
subsections: section_title: Background
paragraphs: In April 2003, the Secretary of Defense charged the military services with  supporting six transformational objectives. These objectives not only  included a reiteration of the department’s goal to fully implement total  asset visibility, but also clearly reflected a growing recognition of the  importance of cost information in fulfilling the logistics mission. Further,  when DOD released its first Enterprise Transition Plan in 2005, these  objectives, including the importance of financial information visibility for  use in decision making, were embodied into the department’s six strategic  business enterprise transformation priorities. Both DOD and the Air  Force have initiatives under way to improve their ability to link financial  resources to associated assets, programs, and activities or missions. For  example, both the department and the Air Force have efforts underway,  such as the Unique Item Identification and Radio Frequency Identification  initiatives and the Standard Financial Information Structure (SFIS)  initiative, to improve their ability to identify and track assets, including  costs, throughout their life cycle.
The Air Force, as a DOD component, is confronted with similar  management challenges that must be effectively resolved if it is to improve  its business operations and in turn provide better support to the  warfighter. The following highlights some of the asset management  challenges the Air Force is attempting to resolve to achieve total asset  visibility.
Excess inventory. We have previously reported that more than half of the  Air Force’s secondary inventory (spare parts), worth an average of       $31.4 billion, was not needed to support required on-hand and on-order  inventory levels from fiscal years 2002 through 2005. The Air Force has  continued to purchase unneeded on-order inventory because its policies  do not provide incentives to reduce the amount of inventory on order that  is not needed to support requirements.
Financial management and reporting. The DOD Inspector General  reported in November 2007, and the Air Force acknowledged, that the Air  Force continues to have significant internal control deficiencies that  impede the ability of its general and working capital funds to produce  accurate and reliable information on the results of their operations.   Deficiencies were found in the following areas: (1) financial management  systems, (2) government furnished and contractor-acquired materiel  (general fund), (3) environmental liabilities, (4) operating materials and  supplies, (5) accounting entries, (6) property, plant, and equipment, and  (7) in-transit inventory (working capital fund).
Deployed assets. In January 2007, the Air Force Audit Agency reported  that the Air Force had lost control and accountability over 5,800 assets,  valued at approximately $108 million, in part because its logistical systems  did not provide Air Force personnel with the capability to effectively  manage, track, and monitor deployed assets. For example, the system  incorrectly reported that assets were deployed to closed bases.  Additionally, the systems did not provide reliable asset information, such  as asset quantities and location. As a result of these weaknesses, Air Force  management did not have total asset visibility and was not able to  determine if the right assets were at the right location to meet mission  requirements.
Government furnished material. In January 2007, the Air Force Audit  Agency also reported that the Air Force did not effectively manage  government furnished material. More specifically, the Air Force Audit  Agency reported that the Air Force logistics personnel inappropriately  provided government furnished material to contractors that were not  authorized by contract documentation to receive this material. This  problem could adversely affect mission support if the Air Force loses  assets that should be in inventory. In addition, poor accountability  controls increase the Air Force’s susceptibility to fraud and misuse of  government resources.
subsections: section_title: Key DOD and Air Force Business Transformation Plans
paragraphs: Successful transformation of DOD’s business operations, including the  achievement of total asset visibility, will require a multifaceted, cross- organizational approach that addresses the contribution and alignment of  key elements, including strategic plans, people, processes, and technology.  The following highlights key DOD and Air Force transformation plans that  are aimed at enhancing business operations and supporting the  department’s total asset visibility goal.
Enterprise Transition Plan. DOD guidance states that the Enterprise  Transition Plan is intended to provide a road map for achieving DOD’s  business transformation through technology, process, and governance  improvements. According to DOD, the Enterprise Transition Plan is  intended to summarize all levels of transition planning information  (milestones, metrics, resource needs, and system migrations) as an  integrated product for communicating and monitoring progress—resulting  in a consistent framework for setting priorities and evaluating plans,  programs, and investments. DOD updates the Enterprise Transition Plan  twice a year, once in March as part of DOD’s annual report to Congress  and again in September. Although the Enterprise Transition Plan provides  an overall strategy and corresponding metrics for achieving each of the  department’s six business enterprise priorities, DOD officials have  acknowledged improvements are needed in the plan to provide a clearer  assessment of the department’s transformation effort. DOD officials have  also acknowledged the need for an integrated planning process and  results-oriented measures to assess overall business transformation.
Financial Improvement and Audit Readiness (FIAR) Plan. A major  component of DOD’s business transformation effort is the Defense FIAR  Plan. The FIAR Plan is updated twice a year and is intended to provide  DOD components with a framework (audit readiness strategy) for  resolving problems affecting the accuracy, reliability, and timeliness of  financial information and obtaining clean financial statement audit  opinions. The FIAR Plan’s audit readiness strategy consists of six phases:  (1) discovery and correction, (2) segment assertion, (3) audit readiness  validation, (4) audit readiness sustainment, (5) financial statement  assertion, and (6) financial statement audit. Each military service is  required to develop subordinate plans that are to support the FIAR Plan in  achieving its objectives.
Air Force Financial Management Strategic Plan for Fiscal Years  2007-2012. This plan identifies seven financial management goals for  transforming the Air Force’s financial management operations. Those  goals are (1) foster mutual respect and integrity, (2) reduce Air Force cost  structure, (3) expand partnership in strategic Air Force decisions,             (4) recruit, prepare, and retain a well-trained and highly educated  professional team for today and tomorrow, (5) provide customers          with world-class financial services, (6) implement open, transparent  business practices, and achieve a clean financial statement audit,            and (7) continuously streamline financial management processes and  increase capabilities. In addition, the plan also identifies specific  objectives for each goal, some of the actions that will be taken to  accomplish the objectives, and 13 financial management metrics.
Air Force Logistics Enterprise Concept of Operations. This  document presents a collection of high-level requirements for  transforming Air Force logistics. It establishes the process framework,  standards, and guidelines to define the environment in which future  logistics systems can be identified, acquired, or built. Further, it aims to  serve as a catalyst for developing doctrine, policies, and organizational  structure consistent with the vision outlined in the Air Force  Expeditionary Logistics for the 21st Century Campaign Plan, needed to  enable logistics transformation.
Air Force Information Reliability and Integration Action Plan/  Financial Improvement Plan. This plan describes actions planned to  identify and address impediments to the Air Force’s ability to achieve  clean financial statement audit opinions. The Air Force Information  Reliability and Integration Action Plan, commonly referred to as the Air  Force Financial Improvement Plan, includes specific tasks, completion  dates, start dates, owner/lead components, and points of contact for  addressing weaknesses adversely affecting the reliability of individual Air  Force financial statement line items and is intended to support the  department’s FIAR Plan.
Air Force Military Equipment Accountability Improvement Plan.  This plan is intended to define how the Air Force will implement measures  to properly collect, account for, track, and report military equipment  values. This plan is intended to identify the actions required to resolve any  existing problems or impediments to achieving auditable values for  military equipment items. The Air Force Military Equipment  Accountability Improvement Plan is intended to be incorporated into the  Air Force’s Financial Improvement Plan and DOD’s FIAR Plan.
section_title: Improvements Are Needed in the Air Force’s Approach for Acquiring and Implementing ECSS and DEAMS
paragraphs: ECSS and DEAMS are two business systems initiatives identified by the  Air Force that are intended to help it address asset accountability  weaknesses and achieve its total asset visibility goal. While these  programs are intended to provide the Air Force with the full spectrum of  logistics and financial management capabilities, our review identified  areas where the Air Force had not fully implemented key best practices  related to risk management for ECSS and DEAMS and system testing for  DEAMS.
subsections: section_title: ECSS and DEAMS Are Intended to Help the Air Force Achieve Total Asset Visibility
paragraphs: ECSS and DEAMS are intended to support the Air Force’s efforts to  transform its business operations and provide accurate, reliable, and  timely information to support decision making and management of the Air  Force’s business operations, including total asset visibility. The ECSS  program was initiated in January 2004 and is expected to provide a single,  integrated logistics system, including transportation, supply, maintenance  and repair, and other key business functions directly related to logistics,  such as engineering and acquisition, at a total life-cycle cost over               $3 billion. Initially, the Air Force anticipated achieving full operational  capability of ECSS during fiscal year 2012. Due to delays as a result of two  contract award protests, the Air Force now expects ECSS to reach full  operational capability in fiscal year 2013. When fully implemented, ECSS is  expected to replace about 250 legacy logistics and procurement  (acquisition) systems and support over 250,000 users Air Force-wide.  ECSS is considered a key element in the Air Force’s efforts to reengineer  and transform its supply chain operations from a reactive posture to a  more predictive posture that facilitates greater effectiveness and efficiency  in the Air Force’s logistics operations that support the warfighter.
ECSS is intended to interface with DEAMS to provide the Air Force with  improved financial visibility over Air Force assets. Additionally,  implementation of ECSS is expected to address long-standing weaknesses  in supply chain management, a DOD issue that has been on our high-risk  list since 1990. In this regard, the redesign of the Air Force’s supply chain  operations, in part through implementation of ECSS, is expected to  address four broad Air Force logistical issues: (1) lack of an enterprise  view, (2) fragmented planning processes, (3) lack of process integration,  and (4) no enterprise-level systems strategy. Figure 1 provides information  related to ECSS’s timeline for implementation and funding.
Currently, the program is undergoing a process referred to as  “blueprinting” to identify needed interfaces and data requirements. After  blueprinting is completed in fiscal year 2009, the Air Force will begin  system testing and initial implementation of ECSS. As of December 2007,  the Air Force reported that approximately $250 million had been obligated  in total for the ECSS effort.
As shown in figure 1, the Air Force estimates a total life-cycle cost of        $3 billion; however, the total life-cycle cost of ECSS is likely to increase  due to an Air Force decision to add functionality. In January 2008, Air  Force ECSS program management officials informed us that ECSS would  assume financial management control and accountability, including  invoice processing and financial reporting responsibility, for the Air  Force’s working capital fund operations. Prior to this decision, the Air  Force had designated DEAMS as the business system initiative it intended  to use to improve the financial management capabilities of both the Air  Force’s working capital and general funds. The Air Force is currently in  the process of determining the cost of this decision and how much it will  add to its already recognized funding shortfall for ECSS of approximately    $697 million. According to ECSS program management office officials,  ECSS’s funding shortfall resulted from contract order award protests that  caused stop-work actions. As a result of the stop-work actions, the ECSS  program management office was not able to spend money for work as  planned, which caused the Air Force to reallocate the money to other Air  Force requirements, ultimately resulting in unfunded ECSS requirements.
The DEAMS program was initiated in August 2003 and is expected to  provide general fund accounting for the entire Air Force at a total           life-cycle cost of over $1 billion. In the past, lack of integration between  business systems, including logistics and financial management, have  adversely affected the ability of DOD and the Air Force to control costs,  ensure basic accountability, anticipate future costs and claims on the  budget, measure performance, maintain funds control, and prevent fraud.  If the information contained in asset and financial accountability systems  is not accurate, complete, and timely, the Air Force’s day-to-day  operations could be adversely affected by, for example, investment in  inventory that is not needed to meet current needs or for which the Air  Force had not allocated sufficient resources or authority to purchase. Both  physical and financial accountability are essential to achieving total asset  visibility and DOD’s objective of providing information to support decision  making.
According to Air Force officials, DEAMS will replace seven legacy  accounting systems. As depicted in figure 2, Air Force program  management officials expect DEAMS to reach initial operational  capability during fiscal year 2011 and full operational capability by fiscal  year 2014 with a total life-cycle cost of about $1.1 billion. DOD defines  total life-cycle cost as the total cost to the government of acquisition and  ownership of that system over its useful life. It includes the cost of  acquisition, operations, and support (to include manpower), and where  applicable, disposal. Figure 2 provides information related to DEAMS’s  timeline for implementation and funding.
The DEAMS business system initiative was approved by the Office of the  Secretary of Defense Business Management Modernization Program’s  Financial Management Transformation Team as a joint United States  Transportation Command (Transportation Command), Defense Finance  and Accounting Service, and Air Force project. According to Air Force  officials, DEAMS will be implemented in two increments—the first at the  Transportation Command and the second at the Air Force.
During the first incremental deployment of DEAMS, which began at Scott  Air Force Base, Illinois, on July 27, 2007, approximately 200 users within  the Transportation Command, the Air Force’s Air Mobility Command  component, and other selected tenant organizations at Scott Air Force  Base, began to receive limited accounting capabilities (starting with  commitment accounting). As of December 2007, the Air Force reported  that approximately $119 million had been obligated for this system. By the  end of the increment 1 deployment phase, which is expected to be  completed by December 2010, DEAMS is intended to provide Scott Air  Force Base with the entire spectrum of core financial management  capabilities, including collections, commitments/obligations, cost  accounting, general ledger, funds control, receipt and acceptance,  accounts payable and disbursement, billing, and financial reporting.  Deployment of DEAMS to an estimated 28,000 users at other Air Force  locations will occur during the DEAMS increment 2 deployment phase.
section_title: ECSS and DEAMS Programs Did Not Fully Embrace or Implement Key Business System Best Practices
paragraphs: The Air Force had not yet fully embraced or implemented key business  system best practices in several areas. Best practices are tried and proven  methods, processes, techniques, and activities that organizations define  and use to minimize program risks and maximize the chances of a  program’s success. Collectively, these practices are intended to  reasonably ensure that the investment in a given system represents the  right solution to fill a mission need—and if the solution is right, that  acquisition and deployment are done the right way, meaning that they  maximize the chances of delivering defined system capabilities on time  and within budget. Specifically, we found that the Air Force had not fully  implemented key best practices related to risk management for ECSS and  DEAMS and system testing for DEAMS. These findings increase the risk  that these two business systems will not meet their stated functionality,  cost, and milestone goals or effectively further the Air Force’s efforts to  achieve total asset visibility.
subsections: section_title: ECSS and DEAMS Risk Management Programs Are Not Comprehensive and Do Not Provide Sufficient Detail to Effectively Oversee the Programs
paragraphs: The Air Force did not have reasonable assurance that its risk management  process would accomplish its primary purpose—managing a program’s  risks to acceptable levels by taking the actions necessary to identify and  mitigate the adverse effects of risks before they affect the program. The  objective of a well-managed risk management program is to provide a  repeatable process for balancing cost, schedule, and performance goals  within program funding. According to DOD’s Risk Management Guide for  DOD Acquisition, risk management is most effective if it is fully  integrated within a program. Our analysis of the ECSS and DEAMS risk  management programs found that neither program used a comprehensive  and fully integrated risk management process. Program risk was  monitored, overseen, and managed independently by various groups or  activities within the program without adequate visibility, at the program  management level. Without adequate visibility of risk management  activities programwide, the program management office has little  assurance of the sufficiency of actions taken by its subordinate groups or  activities to identify, analyze, and mitigate risk that may affect other  groups or the program itself. A single risk management process for each  program with clear linkages to subordinate risk management activities  throughout the program would provide greater visibility and assurance  that appropriate actions are taken to identify and address risks. Acquiring  software is a risky endeavor and risk management processes are intended  to help the program manager and senior leadership ensure that actions are  taken to mitigate the adverse effects of each determined program risk. If  program risks are not effectively communicated and managed, then the  risks will manage the program, potentially leading to increased costs to  ultimately address the impact of a realized risk or implement a program  that does not provide the intended capabilities. The following highlights  specific risk management issues that we identified within the Air Force’s  current approach.
Interfaces. Our analysis of ECSS and DEAMS risk management processes  found that even when risks were identified at lower levels within a  program, the level of detail at the program level was not always sufficient  to provide program managers with the visibility needed to effectively  assess and manage certain risks at those levels. Although the ECSS and  DEAMS program management offices identified interfaces as potential  areas of risk at lower levels within the program, we found that neither  program management office consistently identified interfaces as a risk at  the program level. In the case of DEAMS, the information in the program  level risk management system did not disclose that 70 key interfaces must  be dealt with in order to implement the system, even though this level of  detail was maintained at a lower level by the DEAMS Interface and  Conversion Group. Without visibility of risks identified at all levels of a  program, it is difficult, if not impossible, for the program manager or other  senior-level officials to ascertain if the various risks that are associated  with a program of this magnitude are effectively identified and managed.
We have previously reported that interfaces are critical elements  necessary to successfully implement a new system and failure to properly  address risk in interface areas has contributed to the system failures of  previous agency efforts.
Data conversion. In implementing ECSS and DEAMS, the Air Force will  have to expend considerable resources to clean-up and transfer the data in  the existing legacy systems to ECSS or DEAMS. However, we found that  only the ECSS risk management program identified data quality as an issue  in its discussion of data conversion. Much like system interfaces, each  effort to convert data needs to be separately identified and managed so  that (1) the risks associated with a given effort can be identified,   (2) adequate mitigating actions can be developed for those risks, and   (3) the effectiveness of the mitigating actions can be monitored. For  example, in June 2005, we reported that data conversion problems  seriously affected the Army’s ability to implement its Logistics  Modernization Program at the Tobyhanna Army Depot, Tobyhanna,  Pennsylvania. These problems affected reporting of revenue earned,  accountability over orders received from customers, and prepared billings.  As discussed in our July 2007 report, the Army and its contractor still had  not resolved the issues of customers being improperly billed.
Change management. The DEAMS program management office did not  identify change management as a risk in its risk management system;  however, it was included as a risk by the ECSS program management  office. Change management is the process of preparing users for the  changes that should occur with the implementation of a new system. It  involves engaging users and communicating the nature of anticipated  changes to system users through training on how jobs will change. This is  necessary because commercial products are created with the developers’  expectations of how they will be used, and the products’ functionality may  require the organization implementing the system to change existing  business processes. However, neither the ECSS or DEAMS program had  identified training as a potential change management risk at the program  level. As discussed previously, the lack of sufficient transparency of risks  identified by the lower levels at the program level may impede the ability  of ECSS and DEAMS program managers and senior-level officials to  ensure that risks are effectively mitigated. Further, the lack of centralized  visibility may also minimize program efficiencies that could be gained  through shared knowledge of risks identified by other groups within the  program and actions planned or taken to mitigate them. As we have  previously reported, having staff with the appropriate skills is a key  element for achieving financial management improvement. The  implementation of a new system is intended to bring about improvements  in the way an entity performs its day-to-day business operations. We have  issued several reports that associated the lack of effective change  management to program schedule slippages. Unless those intended  changes are clearly identified and communicated to the affected  employees, the changes in the organization’s business processes may not  occur or be less effective and efficient than envisioned.
Contractor oversight. The Air Force’s ability to manage these two  programs—including oversight of contractors—is critical to reducing the  risks to acceptable levels. Both ECSS and DEAMS program management  officials identified staffing shortfalls within their respective offices as  program risks. In addition, both offices identified actions needed to  mitigate the impact the shortfalls may have on their programs. However,  neither program management office considered whether their programs  had staff with the appropriate skill sets to effectively oversee and manage  their respective contractors. Since the contractors for each program are  performing many of the key tasks, including how the system will perform  and what information or capabilities it will provide, it is critical that the  Air Force have an effective monitoring process to oversee the contractors  and ensure that the project management processes employed by  contractors were effectively implemented.
During discussions on their respective programs, in March 2008, both  ECSS and DEAMS program management officials stated that they thought  their existing risk management programs provided adequate visibility over  risks within their respective programs. However, after discussing our  concerns with the program management officials, they agreed with us that  their program level risk management programs could be improved to  provide better links to the various risks identified and the risk  management processes used by the groups within their programs. They  also agreed that this would help them achieve reasonable assurance that  their decentralized risk management program is achieving the objectives  of a more traditional centralized risk management process.
section_title: More Robust Testing of the Operating Environments at Planned Deployment Locations Is Needed to Minimize Delays
paragraphs: A limited version of DEAMS was deployed at Scott Air Force Base in July  2007. A follow-on deployment intended to provide DEAMS functionality  to additional users, originally scheduled for October 2007, was placed on  hold to address a series of software and connectivity issues that were  identified after the initial deployment. According to Air Force DEAMS  program management officials, DEAMS was functioning as intended on  the older Air Force standard computer desktop configuration; however,  problems occurred when the system was deployed to offices that were  utilizing a newer computer desktop configuration than the one the  program management office had utilized in its initial tests. Air Force  DEAMS program management officials stated that they did not include the  potential of encountering different operating environments at deployment  locations as a potential program risk because they thought that there was  a standard computer desktop configuration across the Air Force and  therefore the risk was remote. DEAMS program management officials  acknowledged that the standardization of computer desktops across the  Air Force is a major challenge and that encountering it during the DEAMS  deployment at Scott Air Force Base was a “lessons learned.”
Further, DEAMS program management officials stated that system  “patches” to address the problem have been tested on multiple computer  desktop configurations at Scott Air Force Base to ensure that DEAMS  operates as intended at that location. According to DEAMS program  management officials, they started the redeployment of DEAMS at the end  of March 2008, and they do not anticipate that this will result in a  significant delay, if any, toward achieving full deployment of DEAMS  within fiscal year 2014.
However, unless the DEAMS program management office obtains a clear  understanding of the environment in which DEAMS will be deployed,  DEAMS will likely suffer additional implementation delays. Further, ECSS  is also likely to encounter nonstandardized computer desktop  configurations during its deployment. Both ECSS and DEAMS program  management officials acknowledged that nonstandardized computer  desktop configurations will continue to represent a potential program risk  and indicated that they intend to test desktop configurations at each  deployment location in the future. DEAMS program management officials  are working with its contractor and other Air Force personnel to develop a  long-term solution for the DEAMS program.
section_title: Better Integration Is Needed to Ensure That the Air Force’s Business Transformation Plans Support DOD Business Transformation Priorities and Total Asset Visibility
paragraphs: Viewed from a broad perspective, the Air Force does not have a single  comprehensive plan or integrated set of plans to support DOD business  transformation priorities, transform Air Force business operations, and  achieve total asset visibility. Rather, the Air Force is utilizing several  individual business transformation plans and efforts. Our analysis of these  plans disclosed that they are neither fully integrated with each other nor  are they fully aligned with business transformation priorities and related  performance measures or metrics outlined in DOD’s Enterprise Transition  Plan. Integration and coordination of improvement efforts within a  component and clear alignment of those efforts with DOD’s Enterprise  Transition Plan is necessary to achieve both the components’ and DOD’s  business transformation priorities and goals, including total asset visibility.  Without clear alignment of transformation plans, priorities, and metrics,  both DOD and the Air Force will have difficulty (1) ensuring that  transformation efforts, such as ECSS and DEAMS, are efficiently and  effectively directed at achieving DOD’s business transformation  priorities/goals, including total asset visibility, and (2) measuring and  reporting on progress toward the capabilities necessary for achieving an  intended business transformation priority, such as financial and materiel  visibility. Air Force officials acknowledged that integration of their plans  within the Air Force and with the DOD’s Enterprise Transition Plan could  be improved and indicated that they intend to make improvements to their  plans. By not fully aligning and integrating these transformation strategies  and plans, the Air Force risks falling short of significantly enhancing its  ability to provide the right equipment and materiel, in the right condition,  at the correct place, when needed to support the warfighter.
subsections: section_title: DOD Business Priorities Are Not Clearly Identified in Key Air Force Business Transformation Plans
paragraphs: Our review of several Air Force strategic documents and plans, such as its  Financial Management Strategic Plan, Accountability Improvement Plan,  and Logistics Enterprise Architecture Concept of Operations, found that  the plans were not clearly linked to each other or with DOD’s Enterprise  Transition Plan.
Air Force Financial Management Strategic Plan. This plan outlines  seven goals for transforming Air Force financial management. However,  the plan contains no reference to the priorities, objectives, or capabilities  identified in DOD’s Enterprise Transition Plan. Additionally, the Air Force  Financial Management Strategic Plan does not identify any performance  measures or metrics that the Air Force intends to use to measure  incremental progress toward achieving its own stated financial  management goals or DOD’s business transformation priorities. It is also  unclear how certain Air Force financial management goals, such as to  “foster mutual respect and integrity” or “recruit, prepare, and retain a well- trained and highly educated professional team for today and tomorrow,”  specifically relate to achieving the four financial visibility objectives  identified in DOD’s Enterprise Transition Plan: (1) produce and interpret  relevant, accurate, and timely financial information that is readily available  for analyses and decision making, (2) link resource allocation to planned  and actual business outcomes and warfighter missions, (3) produce  comparable financial information across organizations, and (4) achieve  audit readiness and prepare auditable financial statements.
Air Force Military Equipment Accountability Improvement Plan.  This plan is intended to support the department’s valuation of military  equipment and the Air Force’s and DOD’s goal to obtain auditable financial  statements. However, the relationship between the Air Force Military  Equipment Accountability Improvement Plan to other Air Force  transformation plans or initiatives, such as the Air Force Logistics  Enterprise Architecture Concept of Operations, in transforming the Air  Force’s business operations is not articulated in the plan. For example,  although the Under Secretary of Defense for Acquisition, Logistics, and  Technology tasked the Air Force and other military components with  preparing a military equipment accountability improvement plan, the plan  does not explain how resolution of these problems will support the Air  Force’s logistics goals to improve operational capability, while minimizing  the cost to deliver capability. Further, the Air Force Military Equipment  Accountability Improvement Plan does not discuss how its efforts  contribute, individually or as part of a collective Air Force effort, to  incremental and measurable improvements in the visibility of Air Force  logistical and financial information for decision making, analysis, and  reporting—a key transformation priority identified in DOD’s Enterprise  Transition Plan.
section_title: The Air Force Lacks Business Transformation Performance Metrics Consistent with DOD’s Enterprise Transition Plan
paragraphs: None of the various Air Force strategic plans we analyzed included  performance measures or metrics that could be used to systematically  assess and report on transformation progress. Without adequate metrics,  both Air Force and DOD management face a difficult challenge in  monitoring implementation of Air Force plans and assessing the Air  Force’s progress in improving its processes, controls, and systems and  achieving DOD’s business transformation priorities, including total asset  visibility. Our prior work has identified at least four characteristics  common to successful hierarchies of performance measures or metrics:  (1) demonstrated results, (2) limited to a vital few, (3) corresponding to  multiple priorities, and (4) linked to responsible programs. Simply stated,  performance measures should tell each organizational level how well it is  achieving its own and shared goals and priorities. Examples of the lack of  consistent metrics follow.
Air Force Logistics Enterprise Architecture Concept of Operations.  None of the six materiel visibility business capability improvement metrics  included in the DOD Enterprise Transition Plan are identified in the Air  Force Logistics Enterprise Architecture Concept of Operations. Further,  the Air Force Logistics Enterprise Architecture Concept of Operations  identified only two measures or goals: (1) increase equipment availability  by 20 percent no later than fiscal year 2011 and (2) reduce annual  operating and support cost by 10 percent no later than fiscal year 2011.  While these are notable goals, these metrics do not provide a means to  measure incremental progress in improving the Air Force’s ability to locate  and account for materiel assets throughout their life cycle.
Air Force Financial Management Strategic Plan. This plan identified  13 metrics, some of which pertained to reducing interest penalties paid,  lost discounts, and unmatched disbursements, to support an assessment of  the current state of the Air Force’s financial management. However, the  Air Force Financial Management Strategic Plan did not include metrics  that the Air Force can use to measure the progress of its various financial  management initiatives in transforming the Air Force’s financial  management and related business operations and achieving DOD business  transformation priorities. For example, none of the 13 metrics outlined in  the Air Force Financial Management Strategic Plan could be used to  measure, monitor, or report incremental progress toward producing and  interpreting relevant, accurate and timely financial information that is  readily available for analyses and decision making—a key financial  visibility objective identified in DOD’s Enterprise Transition Plan.
Air Force Military Equipment Accountability Improvement Plan  and its Financial Improvement Plan. Neither plan included  performance metrics to measure the effectiveness of planned actions to  resolve identified weaknesses that have adversely affected the reliability  of reported financial and physical accountability information. Specifically,  we found that the Air Force’s status reporting for both initiatives consisted  primarily of the completion of milestone dates associated with steps  outlined by DOD in its FIAR Plan for achieving auditability of its financial  statements. As a result, the Accountability Improvement Plan and the  Financial Improvement Plan provide little information on incremental  improvements made in the Air Force’s financial management capabilities,  including decision making support. Moreover, when we compared the  Financial Improvement Plans dated August 1, 2007, and October 11, 2007,  we identified numerous inconsistencies that raise concerns regarding the  oversight and monitoring provided to these plans and their reported  progress. For example, we found    211 of the total 1,762 tasks in the October 2007 Financial Improvement  Plan had completion dates identified as prior to October 1, 2007; however,  the reported progress toward completion for each of these tasks was  identified as zero, and    61 of the total 1,279 tasks that were included in both the August 2007 and  October 2007 Financial Improvement Plans showed a decline in the  percentage completion total reported for the same tasks between the two  plans.
section_title: Conclusion
paragraphs: The Air Force’s efforts to transform its logistics and financial management  operations through system, process, and control changes are being guided  by numerous strategies and plans that are not fully integrated within the  Air Force and with DOD’s business enterprise transformation priorities. As  the Air Force deploys ECSS and DEAMS, it is important that it utilize a  comprehensive and integrated risk management process to identify,  analyze, and mitigate risks and configuration issues that may impede  successful deployment of these systems throughout the Air Force, such as  testing computer desktop configurations at each deployment location.  Additionally, successful transformation will require a comprehensive plan  or integrated set of plans and effective processes and tools, such as  results-oriented performance measures that link enterprise and unit goals  and expectations, for measuring, monitoring, and reporting progress in  accomplishing the department’s priorities. Until the Air Force’s efforts are  aligned within the Air Force and with DOD’s business transformation  priorities, and best practices are fully adopted to minimize risk and  maximize chances for success, the risk increases that billions of dollars  will be wasted and the efforts will not achieve the transformation  envisioned for the future.
section_title: Recommendations for Executive Action
paragraphs: To improve the department’s efforts to achieve total asset visibility and  further enhance its efforts to improve control and accountability over  business system investments and achieve its business transformation  priorities, we recommend that the Secretary of Defense direct the  Secretary of the Air Force to take the following three actions:    Direct Air Force program management officials for ECSS and DEAMS to  ensure that risk management activities at all levels of the program are  identified and communicated to program management to facilitate  oversight and monitoring. Key risks described at the appropriate level of  detail should include and not be limited to risks associated with interfaces,  data conversion, change management, and contractor oversight.
Direct the Air Force program management offices to test ECSS and  DEAMS on relevant computer desktop configurations prior to deployment  at a given location.
Direct Air Force organizations responsible for the business transformation  plans discussed in this report to align their respective plans, including  efforts aimed at achieving total asset visibility, with priorities included in  DOD’s Enterprise Transition Plan. Further, these plans should include  metrics to measure, monitor, and report progress in accomplishing the  business priorities identified in DOD’s Enterprise Transition Plan.
section_title: Agency Comments and Our Evaluation
paragraphs: We received written comments on a draft of this report from the Deputy  Under Secretary of Defense (Business Transformation), which are  reprinted in appendix II. DOD concurred with our recommendations and  identified specific actions it plans to take to implement these  recommendations. For example, the ECSS program management office  has added GAO-identified risks to its inventory of program risks.   Additionally, the DEAMS program management office intends to centralize  two subordinate risk management activities into a single program-level  risk management process.  Further, in its rewrite of the DEAMS program  charter for the department’s Business Capability Lifecycle process, the  DEAMS program management office stated its intent to implement a  program-based risk management process that addresses all risk areas  noted by GAO. In addition, the department noted that the Air Force is  updating its Financial Improvement Plan to assure alignment with the  department’s Financial Improvement and Audit Readiness plan. DOD  stated that the Air Force will ensure that the Financial Improvement Plan  is aligned to the Air Force Financial Management Strategic Plan and  DOD’s Enterprise Transition Plan.
We are sending copies of this report to the Secretary of Defense; Secretary  of the Air Force; Deputy Under Secretary of Defense (Business  Transformation); Assistant Secretary of the Air Force (Financial  Management and Comptroller); Air Force Chief Information Officer; Air  Force Deputy Chief of Staff (Logistics); and other interested congressional  committees and members. Copies of this report will be made available to  others upon request. In addition, this report is available at no charge on  the GAO Web site at http://www.gao.gov.
Please contact Paula M. Rascona at (202) 512-9095 or rasconap@gao.gov,  Nabajyoti Barkakati at (202) 512-4499 or barkakatin@gao.gov, or William  M. Solis at (202) 512-8365 or solisw@gao.gov if you or your staff have  questions on matters discussed in this report. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: In order to determine the implementation status of the Air Force’s current  business system initiatives to achieve total asset visibility, and whether the  Air Force has implemented related best practices, we reviewed Air Force  business system budget documentation and met with Air Force Chief  Information Officer personnel and DOD Business Transformation officials.  Most of the financial information in this report related to ECSS and  DEAMS was obtained from the respective program management offices  and is presented for informational purposes only; it was not used to  develop our findings and recommendations. We interviewed, obtained  briefings, and reviewed documentation provided by ECSS and DEAMS Air  Force program management officials, Business Transformation Agency  officials, and Air Force Financial Management and Comptroller officials to  further our understanding of the intended purpose of each system and  their respective roles in supporting the Air Force’s efforts to achieve total  asset visibility and transform its business operations. During this audit, we  did not review ECSS and DEAMS compliance with the Air Force’s  enterprise architecture because of ongoing GAO work focused on  ascertaining the status of the military services’ efforts to develop and  utilize an enterprise architecture. The results of our work are discussed in  our May 2008 report, which noted that while the Air Force’s efforts to  develop an enterprise architecture were further ahead of Army and Navy  efforts, the Air Force’s architecture was not sufficiently developed to  guide and constrain its business systems modernization investments.
To determine whether any improvements were needed in the Air Force’s  approach for acquiring and implementing these business systems, we  evaluated the ECSS and DEAMS risk management programs, reviewed Air  Force guidance related to risk management, and obtained an explanation  from each program management office on how they managed their  respective risk management program. Additionally, we analyzed risk  management reports that were prepared by each program management  office and reviewed risk management briefings that were presented to  senior Air Force management. We compared risk management reports for  both programs with applicable Air Force guidance to ascertain if each  program identified the risks that are associated with the acquisition and  implementation of a system.
To determine whether the Air Force’s business transformation efforts to  achieve total asset visibility are aligned within the Air Force and with  DOD’s broader business transformation priorities, we interviewed officials  from the Air Force’s Financial Management and Comptroller Office and  the Air Force Logistics Enterprise Architecture and ECSS Transformation  Management Division. There are many DOD and Air Force transformation  plans and initiatives, such as DOD’s Enterprise Transition Plan and  Quadrennial Defense Review Report, and the Air Force Strategic Plan and  the Air Force Smart Operations for the 21st Century. However, following  discussions with Air Force officials, we focused our review on the Air  Force Financial Management Strategic Plan for fiscal years 2007-2012,  Logistics Enterprise Architecture Concept of Operations, Financial  Improvement Plan for August 2007 and October 2007, and Military  Equipment Accountability Improvement Plan issued in December 2006  because they are more directly related to total asset visibility and related  business transformation efforts. We analyzed and compared these  documents to assess consistency among the plans and approaches both  within the Air Force and with DOD’s Enterprise Transition Plan’s business  transformation priorities and metrics.
We conducted this performance audit from July 2007 through August 2008  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. Most of the financial  information in this report related to the ECSS and DEAMS programs was  obtained from the respective program management offices and is  presented for informational purposes only and was not used to develop  our findings and recommendations. To assess the reliability of the funding  data, we interviewed Air Force program management office officials  knowledgeable about funding and reviewed budgetary data on the Air  Force’s investment in ECSS and DEAMS. We conducted our work at the  DOD Business Transformation Agency, the Air Force Chief Information  Officer Office, the Air Force Financial Management and Comptroller  Office, and the Air Force Logistics Enterprise Architecture and ECSS  Transformation Management Division in Arlington, Virginia. Additionally,  we made site visits to the Air Force program management offices for ECSS  and DEAMS at Wright-Patterson Air Force Base in Dayton, Ohio. We  requested comments on a draft of this report from the Secretary of  Defense or his designee. We received written comments from the Deputy  Under Secretary of Defense (Business Transformation), which are  reprinted in appendix II.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: In addition to the above contacts, the following individuals made key  contributions to this report: J. Christopher Martin, Senior-Level  Technologist; Darby Smith, Assistant Director; Evelyn Logue, Assistant  Director; F. Abe Dymond, Assistant General Counsel; Beatrice Alff; Harold  Brumm, Jr.; Francine DelVecchio; Jason Kelly; Jason Kirwan; Chanetta  Reed; Debra Rucker; and Tory Wudtke.