DOD Business Transformation: Air Force's Current Approach Increases Risk That Asset Visibility Goals and Transformation Priorities Will Not Be Achieved
Released: 2008-08-08T00:00:00

Letter

Background

In April 2003, the Secretary of Defense charged the military services with supporting six transformational objectives. These objectives not only included a reiteration of the department’s goal to fully implement total asset visibility, but also clearly reflected a growing recognition of the importance of cost information in fulfilling the logistics mission. Further, when DOD released its first Enterprise Transition Plan in 2005, these objectives, including the importance of financial information visibility for use in decision making, were embodied into the department’s six strategic business enterprise transformation priorities. Both DOD and the Air Force have initiatives under way to improve their ability to link financial resources to associated assets, programs, and activities or missions. For example, both the department and the Air Force have efforts underway, such as the Unique Item Identification and Radio Frequency Identification initiatives and the Standard Financial Information Structure (SFIS) initiative, to improve their ability to identify and track assets, including costs, throughout their life cycle.

The Air Force, as a DOD component, is confronted with similar management challenges that must be effectively resolved if it is to improve its business operations and in turn provide better support to the warfighter. The following highlights some of the asset management challenges the Air Force is attempting to resolve to achieve total asset visibility.

Excess inventory. We have previously reported that more than half of the Air Force’s secondary inventory (spare parts), worth an average of $31.4 billion, was not needed to support required on-hand and on-order inventory levels from fiscal years 2002 through 2005. The Air Force has continued to purchase unneeded on-order inventory because its policies do not provide incentives to reduce the amount of inventory on order that is not needed to support requirements.

Financial management and reporting. The DOD Inspector General reported in November 2007, and the Air Force acknowledged, that the Air Force continues to have significant internal control deficiencies that impede the ability of its general and working capital funds to produce accurate and reliable information on the results of their operations. Deficiencies were found in the following areas: (1) financial management systems, (2) government furnished and contractor-acquired materiel (general fund), (3) environmental liabilities, (4) operating materials and supplies, (5) accounting entries, (6) property, plant, and equipment, and (7) in-transit inventory (working capital fund).

Deployed assets. In January 2007, the Air Force Audit Agency reported that the Air Force had lost control and accountability over 5,800 assets, valued at approximately $108 million, in part because its logistical systems did not provide Air Force personnel with the capability to effectively manage, track, and monitor deployed assets. For example, the system incorrectly reported that assets were deployed to closed bases. Additionally, the systems did not provide reliable asset information, such as asset quantities and location. As a result of these weaknesses, Air Force management did not have total asset visibility and was not able to determine if the right assets were at the right location to meet mission requirements.

Government furnished material. In January 2007, the Air Force Audit Agency also reported that the Air Force did not effectively manage government furnished material. More specifically, the Air Force Audit Agency reported that the Air Force logistics personnel inappropriately provided government furnished material to contractors that were not authorized by contract documentation to receive this material. This problem could adversely affect mission support if the Air Force loses assets that should be in inventory. In addition, poor accountability controls increase the Air Force’s susceptibility to fraud and misuse of government resources.

Key DOD and Air Force Business Transformation Plans

Successful transformation of DOD’s business operations, including the achievement of total asset visibility, will require a multifaceted, cross- organizational approach that addresses the contribution and alignment of key elements, including strategic plans, people, processes, and technology. The following highlights key DOD and Air Force transformation plans that are aimed at enhancing business operations and supporting the department’s total asset visibility goal.

Enterprise Transition Plan. DOD guidance states that the Enterprise Transition Plan is intended to provide a road map for achieving DOD’s business transformation through technology, process, and governance improvements. According to DOD, the Enterprise Transition Plan is intended to summarize all levels of transition planning information (milestones, metrics, resource needs, and system migrations) as an integrated product for communicating and monitoring progress—resulting in a consistent framework for setting priorities and evaluating plans, programs, and investments. DOD updates the Enterprise Transition Plan twice a year, once in March as part of DOD’s annual report to Congress and again in September. Although the Enterprise Transition Plan provides an overall strategy and corresponding metrics for achieving each of the department’s six business enterprise priorities, DOD officials have acknowledged improvements are needed in the plan to provide a clearer assessment of the department’s transformation effort. DOD officials have also acknowledged the need for an integrated planning process and results-oriented measures to assess overall business transformation.

Financial Improvement and Audit Readiness (FIAR) Plan. A major component of DOD’s business transformation effort is the Defense FIAR Plan. The FIAR Plan is updated twice a year and is intended to provide DOD components with a framework (audit readiness strategy) for resolving problems affecting the accuracy, reliability, and timeliness of financial information and obtaining clean financial statement audit opinions. The FIAR Plan’s audit readiness strategy consists of six phases: (1) discovery and correction, (2) segment assertion, (3) audit readiness validation, (4) audit readiness sustainment, (5) financial statement assertion, and (6) financial statement audit. Each military service is required to develop subordinate plans that are to support the FIAR Plan in achieving its objectives.

Air Force Financial Management Strategic Plan for Fiscal Years 2007-2012. This plan identifies seven financial management goals for transforming the Air Force’s financial management operations. Those goals are (1) foster mutual respect and integrity, (2) reduce Air Force cost structure, (3) expand partnership in strategic Air Force decisions, (4) recruit, prepare, and retain a well-trained and highly educated professional team for today and tomorrow, (5) provide customers with world-class financial services, (6) implement open, transparent business practices, and achieve a clean financial statement audit, and (7) continuously streamline financial management processes and increase capabilities. In addition, the plan also identifies specific objectives for each goal, some of the actions that will be taken to accomplish the objectives, and 13 financial management metrics.

Air Force Logistics Enterprise Concept of Operations. This document presents a collection of high-level requirements for transforming Air Force logistics. It establishes the process framework, standards, and guidelines to define the environment in which future logistics systems can be identified, acquired, or built. Further, it aims to serve as a catalyst for developing doctrine, policies, and organizational structure consistent with the vision outlined in the Air Force Expeditionary Logistics for the 21st Century Campaign Plan, needed to enable logistics transformation.

Air Force Information Reliability and Integration Action Plan/ Financial Improvement Plan. This plan describes actions planned to identify and address impediments to the Air Force’s ability to achieve clean financial statement audit opinions. The Air Force Information Reliability and Integration Action Plan, commonly referred to as the Air Force Financial Improvement Plan, includes specific tasks, completion dates, start dates, owner/lead components, and points of contact for addressing weaknesses adversely affecting the reliability of individual Air Force financial statement line items and is intended to support the department’s FIAR Plan.

Air Force Military Equipment Accountability Improvement Plan. This plan is intended to define how the Air Force will implement measures to properly collect, account for, track, and report military equipment values. This plan is intended to identify the actions required to resolve any existing problems or impediments to achieving auditable values for military equipment items. The Air Force Military Equipment Accountability Improvement Plan is intended to be incorporated into the Air Force’s Financial Improvement Plan and DOD’s FIAR Plan.

Improvements Are Needed in the Air Force’s Approach for Acquiring and Implementing ECSS and DEAMS

ECSS and DEAMS are two business systems initiatives identified by the Air Force that are intended to help it address asset accountability weaknesses and achieve its total asset visibility goal. While these programs are intended to provide the Air Force with the full spectrum of logistics and financial management capabilities, our review identified areas where the Air Force had not fully implemented key best practices related to risk management for ECSS and DEAMS and system testing for DEAMS.

ECSS and DEAMS Are Intended to Help the Air Force Achieve Total Asset Visibility

ECSS and DEAMS are intended to support the Air Force’s efforts to transform its business operations and provide accurate, reliable, and timely information to support decision making and management of the Air Force’s business operations, including total asset visibility. The ECSS program was initiated in January 2004 and is expected to provide a single, integrated logistics system, including transportation, supply, maintenance and repair, and other key business functions directly related to logistics, such as engineering and acquisition, at a total life-cycle cost over $3 billion. Initially, the Air Force anticipated achieving full operational capability of ECSS during fiscal year 2012. Due to delays as a result of two contract award protests, the Air Force now expects ECSS to reach full operational capability in fiscal year 2013. When fully implemented, ECSS is expected to replace about 250 legacy logistics and procurement (acquisition) systems and support over 250,000 users Air Force-wide. ECSS is considered a key element in the Air Force’s efforts to reengineer and transform its supply chain operations from a reactive posture to a more predictive posture that facilitates greater effectiveness and efficiency in the Air Force’s logistics operations that support the warfighter.

ECSS is intended to interface with DEAMS to provide the Air Force with improved financial visibility over Air Force assets. Additionally, implementation of ECSS is expected to address long-standing weaknesses in supply chain management, a DOD issue that has been on our high-risk list since 1990. In this regard, the redesign of the Air Force’s supply chain operations, in part through implementation of ECSS, is expected to address four broad Air Force logistical issues: (1) lack of an enterprise view, (2) fragmented planning processes, (3) lack of process integration, and (4) no enterprise-level systems strategy. Figure 1 provides information related to ECSS’s timeline for implementation and funding.

Currently, the program is undergoing a process referred to as “blueprinting” to identify needed interfaces and data requirements. After blueprinting is completed in fiscal year 2009, the Air Force will begin system testing and initial implementation of ECSS. As of December 2007, the Air Force reported that approximately $250 million had been obligated in total for the ECSS effort.

As shown in figure 1, the Air Force estimates a total life-cycle cost of $3 billion; however, the total life-cycle cost of ECSS is likely to increase due to an Air Force decision to add functionality. In January 2008, Air Force ECSS program management officials informed us that ECSS would assume financial management control and accountability, including invoice processing and financial reporting responsibility, for the Air Force’s working capital fund operations. Prior to this decision, the Air Force had designated DEAMS as the business system initiative it intended to use to improve the financial management capabilities of both the Air Force’s working capital and general funds. The Air Force is currently in the process of determining the cost of this decision and how much it will add to its already recognized funding shortfall for ECSS of approximately $697 million. According to ECSS program management office officials, ECSS’s funding shortfall resulted from contract order award protests that caused stop-work actions. As a result of the stop-work actions, the ECSS program management office was not able to spend money for work as planned, which caused the Air Force to reallocate the money to other Air Force requirements, ultimately resulting in unfunded ECSS requirements.

The DEAMS program was initiated in August 2003 and is expected to provide general fund accounting for the entire Air Force at a total life-cycle cost of over $1 billion. In the past, lack of integration between business systems, including logistics and financial management, have adversely affected the ability of DOD and the Air Force to control costs, ensure basic accountability, anticipate future costs and claims on the budget, measure performance, maintain funds control, and prevent fraud. If the information contained in asset and financial accountability systems is not accurate, complete, and timely, the Air Force’s day-to-day operations could be adversely affected by, for example, investment in inventory that is not needed to meet current needs or for which the Air Force had not allocated sufficient resources or authority to purchase. Both physical and financial accountability are essential to achieving total asset visibility and DOD’s objective of providing information to support decision making.

According to Air Force officials, DEAMS will replace seven legacy accounting systems. As depicted in figure 2, Air Force program management officials expect DEAMS to reach initial operational capability during fiscal year 2011 and full operational capability by fiscal year 2014 with a total life-cycle cost of about $1.1 billion. DOD defines total life-cycle cost as the total cost to the government of acquisition and ownership of that system over its useful life. It includes the cost of acquisition, operations, and support (to include manpower), and where applicable, disposal. Figure 2 provides information related to DEAMS’s timeline for implementation and funding.

The DEAMS business system initiative was approved by the Office of the Secretary of Defense Business Management Modernization Program’s Financial Management Transformation Team as a joint United States Transportation Command (Transportation Command), Defense Finance and Accounting Service, and Air Force project. According to Air Force officials, DEAMS will be implemented in two increments—the first at the Transportation Command and the second at the Air Force.

During the first incremental deployment of DEAMS, which began at Scott Air Force Base, Illinois, on July 27, 2007, approximately 200 users within the Transportation Command, the Air Force’s Air Mobility Command component, and other selected tenant organizations at Scott Air Force Base, began to receive limited accounting capabilities (starting with commitment accounting). As of December 2007, the Air Force reported that approximately $119 million had been obligated for this system. By the end of the increment 1 deployment phase, which is expected to be completed by December 2010, DEAMS is intended to provide Scott Air Force Base with the entire spectrum of core financial management capabilities, including collections, commitments/obligations, cost accounting, general ledger, funds control, receipt and acceptance, accounts payable and disbursement, billing, and financial reporting. Deployment of DEAMS to an estimated 28,000 users at other Air Force locations will occur during the DEAMS increment 2 deployment phase.

ECSS and DEAMS Programs Did Not Fully Embrace or Implement Key Business System Best Practices

The Air Force had not yet fully embraced or implemented key business system best practices in several areas. Best practices are tried and proven methods, processes, techniques, and activities that organizations define and use to minimize program risks and maximize the chances of a program’s success. Collectively, these practices are intended to reasonably ensure that the investment in a given system represents the right solution to fill a mission need—and if the solution is right, that acquisition and deployment are done the right way, meaning that they maximize the chances of delivering defined system capabilities on time and within budget. Specifically, we found that the Air Force had not fully implemented key best practices related to risk management for ECSS and DEAMS and system testing for DEAMS. These findings increase the risk that these two business systems will not meet their stated functionality, cost, and milestone goals or effectively further the Air Force’s efforts to achieve total asset visibility.

ECSS and DEAMS Risk Management Programs Are Not Comprehensive and Do Not Provide Sufficient Detail to Effectively Oversee the Programs

The Air Force did not have reasonable assurance that its risk management process would accomplish its primary purpose—managing a program’s risks to acceptable levels by taking the actions necessary to identify and mitigate the adverse effects of risks before they affect the program. The objective of a well-managed risk management program is to provide a repeatable process for balancing cost, schedule, and performance goals within program funding. According to DOD’s Risk Management Guide for DOD Acquisition, risk management is most effective if it is fully integrated within a program. Our analysis of the ECSS and DEAMS risk management programs found that neither program used a comprehensive and fully integrated risk management process. Program risk was monitored, overseen, and managed independently by various groups or activities within the program without adequate visibility, at the program management level. Without adequate visibility of risk management activities programwide, the program management office has little assurance of the sufficiency of actions taken by its subordinate groups or activities to identify, analyze, and mitigate risk that may affect other groups or the program itself. A single risk management process for each program with clear linkages to subordinate risk management activities throughout the program would provide greater visibility and assurance that appropriate actions are taken to identify and address risks. Acquiring software is a risky endeavor and risk management processes are intended to help the program manager and senior leadership ensure that actions are taken to mitigate the adverse effects of each determined program risk. If program risks are not effectively communicated and managed, then the risks will manage the program, potentially leading to increased costs to ultimately address the impact of a realized risk or implement a program that does not provide the intended capabilities. The following highlights specific risk management issues that we identified within the Air Force’s current approach.

Interfaces. Our analysis of ECSS and DEAMS risk management processes found that even when risks were identified at lower levels within a program, the level of detail at the program level was not always sufficient to provide program managers with the visibility needed to effectively assess and manage certain risks at those levels. Although the ECSS and DEAMS program management offices identified interfaces as potential areas of risk at lower levels within the program, we found that neither program management office consistently identified interfaces as a risk at the program level. In the case of DEAMS, the information in the program level risk management system did not disclose that 70 key interfaces must be dealt with in order to implement the system, even though this level of detail was maintained at a lower level by the DEAMS Interface and Conversion Group. Without visibility of risks identified at all levels of a program, it is difficult, if not impossible, for the program manager or other senior-level officials to ascertain if the various risks that are associated with a program of this magnitude are effectively identified and managed.

We have previously reported that interfaces are critical elements necessary to successfully implement a new system and failure to properly address risk in interface areas has contributed to the system failures of previous agency efforts.

Data conversion. In implementing ECSS and DEAMS, the Air Force will have to expend considerable resources to clean-up and transfer the data in the existing legacy systems to ECSS or DEAMS. However, we found that only the ECSS risk management program identified data quality as an issue in its discussion of data conversion. Much like system interfaces, each effort to convert data needs to be separately identified and managed so that (1) the risks associated with a given effort can be identified, (2) adequate mitigating actions can be developed for those risks, and (3) the effectiveness of the mitigating actions can be monitored. For example, in June 2005, we reported that data conversion problems seriously affected the Army’s ability to implement its Logistics Modernization Program at the Tobyhanna Army Depot, Tobyhanna, Pennsylvania. These problems affected reporting of revenue earned, accountability over orders received from customers, and prepared billings. As discussed in our July 2007 report, the Army and its contractor still had not resolved the issues of customers being improperly billed.

Change management. The DEAMS program management office did not identify change management as a risk in its risk management system; however, it was included as a risk by the ECSS program management office. Change management is the process of preparing users for the changes that should occur with the implementation of a new system. It involves engaging users and communicating the nature of anticipated changes to system users through training on how jobs will change. This is necessary because commercial products are created with the developers’ expectations of how they will be used, and the products’ functionality may require the organization implementing the system to change existing business processes. However, neither the ECSS or DEAMS program had identified training as a potential change management risk at the program level. As discussed previously, the lack of sufficient transparency of risks identified by the lower levels at the program level may impede the ability of ECSS and DEAMS program managers and senior-level officials to ensure that risks are effectively mitigated. Further, the lack of centralized visibility may also minimize program efficiencies that could be gained through shared knowledge of risks identified by other groups within the program and actions planned or taken to mitigate them. As we have previously reported, having staff with the appropriate skills is a key element for achieving financial management improvement. The implementation of a new system is intended to bring about improvements in the way an entity performs its day-to-day business operations. We have issued several reports that associated the lack of effective change management to program schedule slippages. Unless those intended changes are clearly identified and communicated to the affected employees, the changes in the organization’s business processes may not occur or be less effective and efficient than envisioned.

Contractor oversight. The Air Force’s ability to manage these two programs—including oversight of contractors—is critical to reducing the risks to acceptable levels. Both ECSS and DEAMS program management officials identified staffing shortfalls within their respective offices as program risks. In addition, both offices identified actions needed to mitigate the impact the shortfalls may have on their programs. However, neither program management office considered whether their programs had staff with the appropriate skill sets to effectively oversee and manage their respective contractors. Since the contractors for each program are performing many of the key tasks, including how the system will perform and what information or capabilities it will provide, it is critical that the Air Force have an effective monitoring process to oversee the contractors and ensure that the project management processes employed by contractors were effectively implemented.

During discussions on their respective programs, in March 2008, both ECSS and DEAMS program management officials stated that they thought their existing risk management programs provided adequate visibility over risks within their respective programs. However, after discussing our concerns with the program management officials, they agreed with us that their program level risk management programs could be improved to provide better links to the various risks identified and the risk management processes used by the groups within their programs. They also agreed that this would help them achieve reasonable assurance that their decentralized risk management program is achieving the objectives of a more traditional centralized risk management process.

More Robust Testing of the Operating Environments at Planned Deployment Locations Is Needed to Minimize Delays

A limited version of DEAMS was deployed at Scott Air Force Base in July 2007. A follow-on deployment intended to provide DEAMS functionality to additional users, originally scheduled for October 2007, was placed on hold to address a series of software and connectivity issues that were identified after the initial deployment. According to Air Force DEAMS program management officials, DEAMS was functioning as intended on the older Air Force standard computer desktop configuration; however, problems occurred when the system was deployed to offices that were utilizing a newer computer desktop configuration than the one the program management office had utilized in its initial tests. Air Force DEAMS program management officials stated that they did not include the potential of encountering different operating environments at deployment locations as a potential program risk because they thought that there was a standard computer desktop configuration across the Air Force and therefore the risk was remote. DEAMS program management officials acknowledged that the standardization of computer desktops across the Air Force is a major challenge and that encountering it during the DEAMS deployment at Scott Air Force Base was a “lessons learned.”

Further, DEAMS program management officials stated that system “patches” to address the problem have been tested on multiple computer desktop configurations at Scott Air Force Base to ensure that DEAMS operates as intended at that location. According to DEAMS program management officials, they started the redeployment of DEAMS at the end of March 2008, and they do not anticipate that this will result in a significant delay, if any, toward achieving full deployment of DEAMS within fiscal year 2014.

However, unless the DEAMS program management office obtains a clear understanding of the environment in which DEAMS will be deployed, DEAMS will likely suffer additional implementation delays. Further, ECSS is also likely to encounter nonstandardized computer desktop configurations during its deployment. Both ECSS and DEAMS program management officials acknowledged that nonstandardized computer desktop configurations will continue to represent a potential program risk and indicated that they intend to test desktop configurations at each deployment location in the future. DEAMS program management officials are working with its contractor and other Air Force personnel to develop a long-term solution for the DEAMS program.

Better Integration Is Needed to Ensure That the Air Force’s Business Transformation Plans Support DOD Business Transformation Priorities and Total Asset Visibility

Viewed from a broad perspective, the Air Force does not have a single comprehensive plan or integrated set of plans to support DOD business transformation priorities, transform Air Force business operations, and achieve total asset visibility. Rather, the Air Force is utilizing several individual business transformation plans and efforts. Our analysis of these plans disclosed that they are neither fully integrated with each other nor are they fully aligned with business transformation priorities and related performance measures or metrics outlined in DOD’s Enterprise Transition Plan. Integration and coordination of improvement efforts within a component and clear alignment of those efforts with DOD’s Enterprise Transition Plan is necessary to achieve both the components’ and DOD’s business transformation priorities and goals, including total asset visibility. Without clear alignment of transformation plans, priorities, and metrics, both DOD and the Air Force will have difficulty (1) ensuring that transformation efforts, such as ECSS and DEAMS, are efficiently and effectively directed at achieving DOD’s business transformation priorities/goals, including total asset visibility, and (2) measuring and reporting on progress toward the capabilities necessary for achieving an intended business transformation priority, such as financial and materiel visibility. Air Force officials acknowledged that integration of their plans within the Air Force and with the DOD’s Enterprise Transition Plan could be improved and indicated that they intend to make improvements to their plans. By not fully aligning and integrating these transformation strategies and plans, the Air Force risks falling short of significantly enhancing its ability to provide the right equipment and materiel, in the right condition, at the correct place, when needed to support the warfighter.

DOD Business Priorities Are Not Clearly Identified in Key Air Force Business Transformation Plans

Our review of several Air Force strategic documents and plans, such as its Financial Management Strategic Plan, Accountability Improvement Plan, and Logistics Enterprise Architecture Concept of Operations, found that the plans were not clearly linked to each other or with DOD’s Enterprise Transition Plan.

Air Force Financial Management Strategic Plan. This plan outlines seven goals for transforming Air Force financial management. However, the plan contains no reference to the priorities, objectives, or capabilities identified in DOD’s Enterprise Transition Plan. Additionally, the Air Force Financial Management Strategic Plan does not identify any performance measures or metrics that the Air Force intends to use to measure incremental progress toward achieving its own stated financial management goals or DOD’s business transformation priorities. It is also unclear how certain Air Force financial management goals, such as to “foster mutual respect and integrity” or “recruit, prepare, and retain a well- trained and highly educated professional team for today and tomorrow,” specifically relate to achieving the four financial visibility objectives identified in DOD’s Enterprise Transition Plan: (1) produce and interpret relevant, accurate, and timely financial information that is readily available for analyses and decision making, (2) link resource allocation to planned and actual business outcomes and warfighter missions, (3) produce comparable financial information across organizations, and (4) achieve audit readiness and prepare auditable financial statements.

Air Force Military Equipment Accountability Improvement Plan. This plan is intended to support the department’s valuation of military equipment and the Air Force’s and DOD’s goal to obtain auditable financial statements. However, the relationship between the Air Force Military Equipment Accountability Improvement Plan to other Air Force transformation plans or initiatives, such as the Air Force Logistics Enterprise Architecture Concept of Operations, in transforming the Air Force’s business operations is not articulated in the plan. For example, although the Under Secretary of Defense for Acquisition, Logistics, and Technology tasked the Air Force and other military components with preparing a military equipment accountability improvement plan, the plan does not explain how resolution of these problems will support the Air Force’s logistics goals to improve operational capability, while minimizing the cost to deliver capability. Further, the Air Force Military Equipment Accountability Improvement Plan does not discuss how its efforts contribute, individually or as part of a collective Air Force effort, to incremental and measurable improvements in the visibility of Air Force logistical and financial information for decision making, analysis, and reporting—a key transformation priority identified in DOD’s Enterprise Transition Plan.

The Air Force Lacks Business Transformation Performance Metrics Consistent with DOD’s Enterprise Transition Plan

None of the various Air Force strategic plans we analyzed included performance measures or metrics that could be used to systematically assess and report on transformation progress. Without adequate metrics, both Air Force and DOD management face a difficult challenge in monitoring implementation of Air Force plans and assessing the Air Force’s progress in improving its processes, controls, and systems and achieving DOD’s business transformation priorities, including total asset visibility. Our prior work has identified at least four characteristics common to successful hierarchies of performance measures or metrics: (1) demonstrated results, (2) limited to a vital few, (3) corresponding to multiple priorities, and (4) linked to responsible programs. Simply stated, performance measures should tell each organizational level how well it is achieving its own and shared goals and priorities. Examples of the lack of consistent metrics follow.

Air Force Logistics Enterprise Architecture Concept of Operations. None of the six materiel visibility business capability improvement metrics included in the DOD Enterprise Transition Plan are identified in the Air Force Logistics Enterprise Architecture Concept of Operations. Further, the Air Force Logistics Enterprise Architecture Concept of Operations identified only two measures or goals: (1) increase equipment availability by 20 percent no later than fiscal year 2011 and (2) reduce annual operating and support cost by 10 percent no later than fiscal year 2011. While these are notable goals, these metrics do not provide a means to measure incremental progress in improving the Air Force’s ability to locate and account for materiel assets throughout their life cycle.

Air Force Financial Management Strategic Plan. This plan identified 13 metrics, some of which pertained to reducing interest penalties paid, lost discounts, and unmatched disbursements, to support an assessment of the current state of the Air Force’s financial management. However, the Air Force Financial Management Strategic Plan did not include metrics that the Air Force can use to measure the progress of its various financial management initiatives in transforming the Air Force’s financial management and related business operations and achieving DOD business transformation priorities. For example, none of the 13 metrics outlined in the Air Force Financial Management Strategic Plan could be used to measure, monitor, or report incremental progress toward producing and interpreting relevant, accurate and timely financial information that is readily available for analyses and decision making—a key financial visibility objective identified in DOD’s Enterprise Transition Plan.

Air Force Military Equipment Accountability Improvement Plan and its Financial Improvement Plan. Neither plan included performance metrics to measure the effectiveness of planned actions to resolve identified weaknesses that have adversely affected the reliability of reported financial and physical accountability information. Specifically, we found that the Air Force’s status reporting for both initiatives consisted primarily of the completion of milestone dates associated with steps outlined by DOD in its FIAR Plan for achieving auditability of its financial statements. As a result, the Accountability Improvement Plan and the Financial Improvement Plan provide little information on incremental improvements made in the Air Force’s financial management capabilities, including decision making support. Moreover, when we compared the Financial Improvement Plans dated August 1, 2007, and October 11, 2007, we identified numerous inconsistencies that raise concerns regarding the oversight and monitoring provided to these plans and their reported progress. For example, we found 211 of the total 1,762 tasks in the October 2007 Financial Improvement Plan had completion dates identified as prior to October 1, 2007; however, the reported progress toward completion for each of these tasks was identified as zero, and 61 of the total 1,279 tasks that were included in both the August 2007 and October 2007 Financial Improvement Plans showed a decline in the percentage completion total reported for the same tasks between the two plans.

Conclusion

The Air Force’s efforts to transform its logistics and financial management operations through system, process, and control changes are being guided by numerous strategies and plans that are not fully integrated within the Air Force and with DOD’s business enterprise transformation priorities. As the Air Force deploys ECSS and DEAMS, it is important that it utilize a comprehensive and integrated risk management process to identify, analyze, and mitigate risks and configuration issues that may impede successful deployment of these systems throughout the Air Force, such as testing computer desktop configurations at each deployment location. Additionally, successful transformation will require a comprehensive plan or integrated set of plans and effective processes and tools, such as results-oriented performance measures that link enterprise and unit goals and expectations, for measuring, monitoring, and reporting progress in accomplishing the department’s priorities. Until the Air Force’s efforts are aligned within the Air Force and with DOD’s business transformation priorities, and best practices are fully adopted to minimize risk and maximize chances for success, the risk increases that billions of dollars will be wasted and the efforts will not achieve the transformation envisioned for the future.

Recommendations for Executive Action

To improve the department’s efforts to achieve total asset visibility and further enhance its efforts to improve control and accountability over business system investments and achieve its business transformation priorities, we recommend that the Secretary of Defense direct the Secretary of the Air Force to take the following three actions: Direct Air Force program management officials for ECSS and DEAMS to ensure that risk management activities at all levels of the program are identified and communicated to program management to facilitate oversight and monitoring. Key risks described at the appropriate level of detail should include and not be limited to risks associated with interfaces, data conversion, change management, and contractor oversight.

Direct the Air Force program management offices to test ECSS and DEAMS on relevant computer desktop configurations prior to deployment at a given location.

Direct Air Force organizations responsible for the business transformation plans discussed in this report to align their respective plans, including efforts aimed at achieving total asset visibility, with priorities included in DOD’s Enterprise Transition Plan. Further, these plans should include metrics to measure, monitor, and report progress in accomplishing the business priorities identified in DOD’s Enterprise Transition Plan.

Agency Comments and Our Evaluation

We received written comments on a draft of this report from the Deputy Under Secretary of Defense (Business Transformation), which are reprinted in appendix II. DOD concurred with our recommendations and identified specific actions it plans to take to implement these recommendations. For example, the ECSS program management office has added GAO-identified risks to its inventory of program risks. Additionally, the DEAMS program management office intends to centralize two subordinate risk management activities into a single program-level risk management process. Further, in its rewrite of the DEAMS program charter for the department’s Business Capability Lifecycle process, the DEAMS program management office stated its intent to implement a program-based risk management process that addresses all risk areas noted by GAO. In addition, the department noted that the Air Force is updating its Financial Improvement Plan to assure alignment with the department’s Financial Improvement and Audit Readiness plan. DOD stated that the Air Force will ensure that the Financial Improvement Plan is aligned to the Air Force Financial Management Strategic Plan and DOD’s Enterprise Transition Plan.

We are sending copies of this report to the Secretary of Defense; Secretary of the Air Force; Deputy Under Secretary of Defense (Business Transformation); Assistant Secretary of the Air Force (Financial Management and Comptroller); Air Force Chief Information Officer; Air Force Deputy Chief of Staff (Logistics); and other interested congressional committees and members. Copies of this report will be made available to others upon request. In addition, this report is available at no charge on the GAO Web site at http://www.gao.gov.

Please contact Paula M. Rascona at (202) 512-9095 or rasconap@gao.gov, Nabajyoti Barkakati at (202) 512-4499 or barkakatin@gao.gov, or William M. Solis at (202) 512-8365 or solisw@gao.gov if you or your staff have questions on matters discussed in this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Scope and Methodology

In order to determine the implementation status of the Air Force’s current business system initiatives to achieve total asset visibility, and whether the Air Force has implemented related best practices, we reviewed Air Force business system budget documentation and met with Air Force Chief Information Officer personnel and DOD Business Transformation officials. Most of the financial information in this report related to ECSS and DEAMS was obtained from the respective program management offices and is presented for informational purposes only; it was not used to develop our findings and recommendations. We interviewed, obtained briefings, and reviewed documentation provided by ECSS and DEAMS Air Force program management officials, Business Transformation Agency officials, and Air Force Financial Management and Comptroller officials to further our understanding of the intended purpose of each system and their respective roles in supporting the Air Force’s efforts to achieve total asset visibility and transform its business operations. During this audit, we did not review ECSS and DEAMS compliance with the Air Force’s enterprise architecture because of ongoing GAO work focused on ascertaining the status of the military services’ efforts to develop and utilize an enterprise architecture. The results of our work are discussed in our May 2008 report, which noted that while the Air Force’s efforts to develop an enterprise architecture were further ahead of Army and Navy efforts, the Air Force’s architecture was not sufficiently developed to guide and constrain its business systems modernization investments.

To determine whether any improvements were needed in the Air Force’s approach for acquiring and implementing these business systems, we evaluated the ECSS and DEAMS risk management programs, reviewed Air Force guidance related to risk management, and obtained an explanation from each program management office on how they managed their respective risk management program. Additionally, we analyzed risk management reports that were prepared by each program management office and reviewed risk management briefings that were presented to senior Air Force management. We compared risk management reports for both programs with applicable Air Force guidance to ascertain if each program identified the risks that are associated with the acquisition and implementation of a system.

To determine whether the Air Force’s business transformation efforts to achieve total asset visibility are aligned within the Air Force and with DOD’s broader business transformation priorities, we interviewed officials from the Air Force’s Financial Management and Comptroller Office and the Air Force Logistics Enterprise Architecture and ECSS Transformation Management Division. There are many DOD and Air Force transformation plans and initiatives, such as DOD’s Enterprise Transition Plan and Quadrennial Defense Review Report, and the Air Force Strategic Plan and the Air Force Smart Operations for the 21st Century. However, following discussions with Air Force officials, we focused our review on the Air Force Financial Management Strategic Plan for fiscal years 2007-2012, Logistics Enterprise Architecture Concept of Operations, Financial Improvement Plan for August 2007 and October 2007, and Military Equipment Accountability Improvement Plan issued in December 2006 because they are more directly related to total asset visibility and related business transformation efforts. We analyzed and compared these documents to assess consistency among the plans and approaches both within the Air Force and with DOD’s Enterprise Transition Plan’s business transformation priorities and metrics.

We conducted this performance audit from July 2007 through August 2008 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Most of the financial information in this report related to the ECSS and DEAMS programs was obtained from the respective program management offices and is presented for informational purposes only and was not used to develop our findings and recommendations. To assess the reliability of the funding data, we interviewed Air Force program management office officials knowledgeable about funding and reviewed budgetary data on the Air Force’s investment in ECSS and DEAMS. We conducted our work at the DOD Business Transformation Agency, the Air Force Chief Information Officer Office, the Air Force Financial Management and Comptroller Office, and the Air Force Logistics Enterprise Architecture and ECSS Transformation Management Division in Arlington, Virginia. Additionally, we made site visits to the Air Force program management offices for ECSS and DEAMS at Wright-Patterson Air Force Base in Dayton, Ohio. We requested comments on a draft of this report from the Secretary of Defense or his designee. We received written comments from the Deputy Under Secretary of Defense (Business Transformation), which are reprinted in appendix II.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contacts and Staff Acknowledgments

GAO Contacts

Acknowledgments

In addition to the above contacts, the following individuals made key contributions to this report: J. Christopher Martin, Senior-Level Technologist; Darby Smith, Assistant Director; Evelyn Logue, Assistant Director; F. Abe Dymond, Assistant General Counsel; Beatrice Alff; Harold Brumm, Jr.; Francine DelVecchio; Jason Kelly; Jason Kirwan; Chanetta Reed; Debra Rucker; and Tory Wudtke.