Defense Inventory: Further Analysis and Enhanced Metrics Could Improve Service Supply and Depot Operations
Released: 2016-06-09T00:00:00

Letter

Background

DOD Organizational Structure for Inventory Management and Industrial Sites

The DOD supply chain is a global network that provides materiel, services, and equipment to the joint force. The Under Secretary of Defense for Acquisition, Technology and Logistics and its subordinate, the Assistant Secretary of Defense for Logistics and Materiel Readiness, prescribe policies and procedures for the conduct of logistics, maintenance, materiel readiness, and sustainment support, to include supply and transportation, and monitor and review these activities. Inventory management, a key component of the DOD supply chain, is the process of determining requirements and procuring, managing, cataloging, distributing, overhauling, and disposing of materiel. Management and oversight of DOD inventory is a responsibility shared among the Under Secretary of Defense for Acquisition, Technology and Logistics, DLA, and the services. Specifically, DLA procures, stores, and distributes mostly consumable items—those that are normally expended or intended to be used up beyond recovery or repair—and provides these items to the services when requisitioned in support of approximately 2,400 weapon systems. On the other hand, the services manage mostly reparable items—items that are generally more cost-effective to repair and reuse than to dispose of and replace.

DOD maintains weapon systems (e.g., aircraft and ships) and equipment (e.g., generators and radars) that require regular and emergency maintenance to continue being available for DOD to meet national security goals. To sustain these systems and equipment at the depot level, the department uses a combination of military depots—public- sector facilities that are government-owned and government-operated— and private-sector contractors. Depot maintenance plays a key role in sustaining the complex weapon systems and equipment both in peacetime and during a mobilization, contingency, or other emergency. DOD’s 17 depot maintenance industrial sites are managed by the Army, Navy, Air Force, and Marine Corps, as shown in figure 1 below. These sites, which are managed by the service materiel commands—Army Materiel Command, Naval Sea Systems Command, Naval Air Systems Command, Air Force Materiel Command, and Marine Corps Logistics Command—repair and overhaul a wide range of vehicles and other military assets, including helicopters, combat vehicles, air defense systems, ships, fighter and bomber aircraft, intercontinental ballistic missiles, jet aircraft engines, components, and software.

DLA’s Management of Supply, Storage, and Distribution Functions at DOD’s Industrial Sites

As a result of the 2005 BRAC round and a June 2005 administrative decision by the Under Secretary of Defense for Acquisition, Technology, and Logistics, the services were required to transfer to DLA all of their supply, storage, and distribution functions at depot-level industrial sites. These actions were intended to improve the efficiency of the supply chain by reducing the number of supply distribution warehouses and related excess capacity while enhancing the effectiveness of logistics support by reconfiguring DOD’s wholesale supply, storage, and distribution network and consolidating functions at several service maintenance sites. Traditionally, DLA had been DOD’s wholesale inventory manager for consumable items; however, the transfer of supply, storage, and distribution functions at the depot maintenance industrial sites, as recommended by BRAC, placed DLA in the role of managing consumable items at the retail level. As a wholesale inventory manager, DLA procures consumable items from commercial vendors and stores them in distribution warehouses to be requisitioned by the retail supply system, which is typically managed by the services. As a retail inventory manager at industrial sites, DLA manages the supply, storage, and distribution of consumable items at the industrial site and provides the items directly to the user (e.g., the depot artisan working to repair a weapon system). While sharing some similarities, the management of inventory at the wholesale and retail levels has a different focus—namely placing DLA in a direct, prominent role supporting service depot-level maintenance. Specifically, DLA, as a wholesale inventory manager, is typically replenishing retail inventory stocks, but as a retail inventory manager at the industrial sites DLA must have the part when it is needed; otherwise, there are operational effects to the depot maintenance being conducted on the end item being repaired.

In 2007, 2008, and 2009, we reviewed the implementation of the supply, storage, and distribution BRAC recommendation and reported that a number of the assumptions used by the BRAC commission to calculate the potential cost savings of this recommendation were not validated and relied on flawed data, and that DLA faced numerous challenges in implementing the recommendation. For example, service officials expressed concerns that inserting DLA into depot operations may hinder their ability to meet depot production schedules and maintain equipment readiness. However, we did not make any recommendations pertaining to the transfer of supply, storage, and distribution functions at the industrial sites in those reports given that DLA’s planning process incorporated several key elements that were intended to provide a smooth transition and mitigate the risk of disrupting depot operations.

The Depot Maintenance Process

The depot maintenance process across the services generally involves three primary steps–planning, disassembly, and rebuilding–as shown in figure 2 below.

Step 1: The depot maintenance process begins by planning the maintenance to be conducted on a particular end item, which could be a weapon system (e.g., C-5 Galaxy cargo plane, M1A2 Abrams tank) or depot-level reparable (e.g., a ship blade propeller, brake assembly). Initially, a detailed statement of the specific work to be performed is developed for the end item. Once this scope of work is decided upon, a number of key planning factors are necessary to identify the materials and spare parts needed for the maintenance: (1) the number of end items to go through maintenance, (2) the schedule for inducting the end items into maintenance, (3) the bill of materials (i.e., the list and quantity of parts needed to conduct the maintenance on the end item), and (4) the replacement factors (i.e., the estimated frequency of replacement based on historical trends and engineering estimates) for the parts on the bill of materials. These planning factors are used to forecast the spare parts needed to conduct the maintenance on the end item, as shown in figure 3 below. Proactive and accurate planning is necessary to ensure the timely availability of spare parts for the maintenance process, especially since the acquisition lead time for spare parts can range from days to years depending upon the specific part. However, in 2011, 2013, and 2015, we found that part shortages was one of several factors negatively affecting the industrial sites’ ability to complete maintenance timely and efficiently. Accurate planning is also important in preventing the accumulation of excess inventory.

Step 2: Once the end item is inducted into the maintenance process it is disassembled. During this step, the end item and its components are inspected to determine the type and degree of repair required or whether the parts need to be replaced. While some parts are always replaced, other parts are assessed for repair. Repairs vary by the time and type of use since the last overhaul. Because usage differs from end item to end item, demands on the supply chain for new and repaired items varies.

Step 3: Following disassembly, the end item is then rebuilt with new and repaired parts. In general, the rebuilding of the end item follows a sequential process, necessitating the timely availability of new and repaired parts to ensure the efficient reassembly of the end item. Part shortages (i.e., backorders) require workarounds–additional efforts to obtain a part (e.g., local manufacturing or obtaining parts from another end item)–that can delay maintenance and negatively affect productivity and costs of depot maintenance. Once the end item is rebuilt, then it is tested and validated for sale to and use by the customer (e.g., a military unit).

Navy F/A-18 Hornet Maintenance One of the primary end items repaired at Fleet Readiness Center (FRC) Southwest and FRC Southeast is the F/A-18 Hornet. The first aircraft was manufactured in the late 1970s and became operational in the early 1980s. As an aircraft ages, it incurs additional inspections and structural repairs. One of those additional inspections occurs when an aircraft reaches 8,000 flying hours.

DOD’s aging weapon systems pose sustainment challenges that affect depot maintenance as well as supply support, especially for aircraft. Depot maintenance on aging weapon systems becomes less predictable as structural fatigue occurs and parts that were not expected to need replacement begin to wear out. Cracking, corrosion, or other unanticipated issues–identified through detailed inspections–must be remedied through the repair or the manufacture of new parts, often requiring engineering support and estimates that can take considerable time to develop. For example, the Navy F/A-18 Hornet jet and Air Force B-52 Stratofortress bomber–both aging weapon systems–have recently experienced unanticipated structural and part replacement issues (see sidebar for information on the F/A-18 Hornet). Additionally, diminishing manufacturing sources and material shortages, and part obsolescence issues increase as weapon systems age, complicating supply support for the depot maintenance process and potentially resulting in a less efficient process if not properly mitigated or resolved. Manufacturers discontinue production and support of needed items because it is no longer profitable to produce the part given low or sporadic demand. DOD’s diminishing manufacturing sources and material shortages guidebook provides guidance on mitigating and managing these issues as well as examples in which the department successfully managed and resolved these issues, such as parts obsolescence for the B-1 Lancer bomber and the Virginia class submarine. of the aircraft to identify structural weaknesses, such as metal fatigue and cracks in the aircraft, and determine what needs to be repaired. engineering support to develop repair solutions for the damaged areas of the aircraft. Structural repairs needed to fix the aircraft are non-standard repairs that must be designed and approved by FRC engineers. According to FRC engineers and engineering information documents we reviewed, it may take as long as a year or more for an engineer to determine and document the step-by- step instructions needed to repair the aircraft.

The services have transferred the management of retail supply, storage, and distribution functions to DLA in varying degrees across service industrial sites, as seen in figure 4. DLA management of all retail supply, storage, and distribution functions at Air Force ALCs and Navy FRCs has provided inventory management benefits. DLA also manages some elements of retail supply, storage, and distribution functions at Army and Marine Corps depots and Navy shipyards, but these sites have not seen the same benefits as the sites that fully transferred their retail supply, storage, and distribution functions to DLA. In addition, the Marine Corps depot and Navy shipyards have not assessed the costs and benefits of further transferring retail management functions to DLA, while the Army only recently began making plans to examine the costs and benefits.

Air Force ALCs and Navy FRCs Transferred All Retail Supply, Storage, and Distribution Functions to DLA

The Air Force between 2007 and 2008 and the Navy between 2008 and 2009 transferred the management of retail supply, storage, and distribution functions to DLA at the ALCs and FRCs. The transfer of the entire retail supply, storage, and distribution functions at ALCs and FRCs involved transferring hundreds of personnel from the services to DLA while remaining largely co-located with the actual maintenance activities at the sites. The ALCs in 2007 and the FRCs in 2008 began to transition retail supply functions to DLA and interfaced their maintenance information systems with DLA information technology systems in 2008 at ALCs and 2013 at FRCs. The ALCs and FRCs now use DLA’s business processes and expertise to manage the supply, storage, and distribution of DLA-managed inventory. Specifically, DLA used an Inventory Management and Stock Positioning system to extend capabilities and system functionality necessary to support retail-level supply, storage and distribution in order to implement the BRAC 2005 recommendation.

Army and Marine Corps Depots and Navy Shipyards Have Retained Some Retail Supply, Storage, and Distribution Functions

In contrast to the ALCs and FRCs, the Army and Marine Corps transferred their retail storage and distribution functions to DLA at industrial sites from 2010 to 2011, but have largely retained the management of their retail supply functions. The Army’s and Marine Corps’ prior reluctance to transfer positions to DLA stemmed from concerns related to work-in-process operations, which comprise highly integrated production and supply functions with many of the same personnel performing both functions. As a result, the Army and Marine Corps officials maintained that they did not have “retail” inventory, and that these positions should not transfer to DLA. The Marine Corps and DLA agreed to the transfer of some storage and distribution functions and positions to DLA in April 2007. Also, after repeated opposition to the transfer of certain positions, the Army agreed to comply with direction from the Office of the Secretary of Defense to transfer similar storage and distribution functions and positions to DLA in July 2007. The Navy transferred retail supply, storage, and distribution functions at Navy shipyards to DLA in 2009 and 2010 and these functions are currently performed by DLA personnel using the same Navy processes and approaches that were used by the Navy prior to the transfer of personnel.

DLA Management of All Retail Supply, Storage, and Distribution Functions at Air Force ALCs and Navy FRCs Has Produced Inventory Management Benefits

The Air Force and Navy, as well as our analysis, have identified a number of benefits resulting from using DLA’s expertise and management in retail functions at the ALCs and FRCs. As part of the transfer of retail functions, DLA purchased the retail inventory it now manages at the ALCs and FRCs from the services and shifted DLA’s “point of sale” from the wholesale warehouse to the production line at the industrial site where inventory items are actually used. According to DLA and service officials, this arrangement has allowed DLA to see real-time demand signals rather than waiting for the demand signal to occur when the retail level requisitions inventory from DLA’s wholesale inventory stocks. This means that DLA can make adjustments to its forecast for parts as soon as changes are known, which can reduce the wait times for parts and help prevent the accumulation of excess inventory. Navy, Air Force, and DLA officials stated that the transfer of retail supply, storage, and distribution functions to DLA at the FRCs and ALCs has allowed them to reduce the amount of on-hand inventory, increase warehouse efficiencies, enhance information-sharing and coordination between maintenance and supply, and improve supply performance outcomes such as decreased backorders or increased throughput.

A 2014 study sponsored by the Assistant Secretary of Defense for Logistics and Materiel Readiness also identified specific benefits from transferring retail supply, storage, and distribution functions at ALCs and FRCs as well, such as reduced inventory levels and lower storage costs. These have been evidenced in aggregate performance measures, as noted in figure 5. Specifically, over a five-year period between January 2010 and April 2015, DLA data shows that the ALCs experienced a 20 percent reduction in on-hand inventory while also reducing backorders by 10 percent, and reducing the number of end items awaiting parts by 20 percent. FRCs saw a 6 percent reduction in on-hand inventory during 2014 without overall degradation to other performance measures such as order fill rates and material availability. In addition, the number of backorders decreased across all Navy FRCs by about 28 percent during fiscal year 2015. Officials we spoke with at the FRCs also stated that DLA participation in retail supply management has resulted in more efficient replenishment of consumable items used on the shop floor and less time awaiting parts. Site-specific improvements have also been realized. For example, Warner Robins ALC officials attributed a decrease in unused spare parts stored on the production line, which increase costs, create excess stock, and take up production floor space, to DLA management of retail supply, storage, and distribution functions.

While many of the benefits realized are quantifiable, officials also stated that there are qualitative benefits from transferring retail supply, storage, and distribution functions to DLA. For example, Air Force and DLA officials we spoke with at Warner Robins ALC stated that having DLA personnel and Air Force personnel integrated and co-located at the shop floor results in a great deal of information sharing. This allows supply planners to identify potential changes to inventory demand patterns before aggregate data may exist to indicate changes affecting procurement actions. This is especially important for items that have long lead times for production, or are relatively expensive to procure. Specifically, identifying an upward tick in demand as early as possible may allow backorders for items with long lead times to be prevented, and identifying downward ticks in demand may allow the purchase of expensive items to be avoided.

According to officials we spoke with, initial transfer efforts at ALCs and FRCs encountered some difficulties related to the adoption of DLA systems and processes, and some personnel were reluctant to trust DLA at first. However, they noted that collaborative efforts were made to overcome challenges and foster an atmosphere of trust. Additionally, DLA tailored its processes and approach for retail supply support to align with the maintenance processes that are unique at the FRCs and ALCs. For example, the metrics used to monitor DLA’s performance at the ALCs and FRCs are different given some differences in the maintenance processes and approaches. Furthermore, based on the lessons learned from these implementations, DLA developed a DLA Retail Supply Chain Materiel Management Policy to establish policies, assign responsibilities, and implement high-level procedures for DLA’s management of retail supply, storage, and distribution functions at applicable industrial sites to incorporate lessons learned from DLA’s management of the retail supply, storage, and distribution functions at the ALCs and FRCs. Additionally, DLA has developed a manual to guide its management of retail supply, storage, and distribution functions at ALCs, and according to officials is in the process of developing a similar manual for the FRCs.

Army and Marine Corps Depots and Navy Shipyards Have Not Experienced the Same Improvements as Industrial Sites That Transferred More Retail Functions to DLA

The Army and Marine Corps depots and Navy shipyards have not experienced the same improvements as the Air Force ALCs and Navy FRCs that transferred retail supply, storage, and distribution functions to DLA. These sites continue to experience (1) delayed response from the DLA wholesale inventory system when there is a change in demand rather than an immediate response from DLA (with corresponding adjustments) and (2) inefficient warehouse practices resulting from multiple levels of storage and inventory processing (i.e., practices requiring an increased number of “touches” to process inventory to the customer). Army and Marine Corps officials recognize that some benefits may be gained by transferring additional retail supply support responsibility to DLA and have been exploring the possibility of doing so, although these initiatives have only recently begun.

While DLA has three to five supply-related personnel at each Army and Marine Corps depot, these officials do not perform retail supply functions and largely act as liaisons between the depots and the relevant wholesale DLA supply chains. The small number of DLA personnel on site makes it difficult to engage in the constant information sharing seen at the Air Force ALCs and Navy FRCs given the volume of parts used by the depots. For example, there are only four DLA personnel on site to perform customer service functions at Anniston Army Depot and three DLA personnel on site to perform similar functions at the Marine Corps industrial site in Albany, Georgia. About 30 DLA personnel perform storage functions at the retail inventory warehouses at each of these locations, but decisions affecting supply functions are made by Army and Marine Corps officials. In contrast, there are about 200 DLA personnel at Warner Robins ALC, and about 120 DLA personnel at Cherry Point FRC participating in all aspects of supply, storage, and distribution.

The Navy nominally transferred its retail supply, storage, and distribution functions at the Navy shipyards to DLA resulting in DLA detachments with hundreds of previously Navy personnel that are largely co-located with the actual maintenance activities at the shipyards. However, the shipyards continue to use the same retail supply, storage, and distribution processes and Navy information systems that existed prior to the transfer. In January 2012, the Navy attempted to fully transfer its retail supply, storage, and distribution functions to DLA at Norfolk Naval Shipyard by implementing DLA information technology systems and transferring ownership of all inventory to DLA. However, after 7 months, the Navy and DLA agreed to revert back to the previous processes, systems, and Navy ownership of the inventory after seeing increases in the time to issue retail inventory to the maintenance area which resulted in some work stoppages.

Ineffective Leadership and Program Management: Inconsistent leadership engagement, insufficient training and communication, and lack of clear responsibilities and accountability for the outcome. For example, DLA had no clear operational control during implementation. Insufficient Preparation: End-to-end testing was insufficient and not focused on end-user functionality, a complete inventory was not conducted prior to conversion of inventory to DLA, and simultaneous testing, training, and modifications to the system led to an unstable data environment.

Cultural Resistance: A shared vision of end-state operations was not established among stakeholders, a perception existed among shipyard personnel that their business processes were not supposed to change, and personnel interviewed stated that there was an atmosphere of distrust between the shipyard and DLA.

According to a Naval Sea Systems Command investigation, the attempt to transfer functions failed for a variety of reasons (see sidebar). In addition, Navy and DLA officials stated that implementation encountered cultural resistance to change at Norfolk Naval Shipyard that hindered successful implementation. Furthermore, the Navy and DLA agreed that any further transfer of functions at the four shipyards would be halted indefinitely as of 2012. As a result, the Navy continues to rely on retail supply, storage and distribution processes in place prior to BRAC 2005. Officials at Norfolk Naval Shipyard confirmed that although personnel transfers did occur to shift retail functions from the Navy to DLA, personnel participating in retail supply, storage, and distribution functions continue to use legacy Navy processes rather than DLA processes.

The investigation concluded that the decision to reverse the transition to DLA means shipyards are foregoing the benefits of automated inventory technology and proven standard distribution processes (storage and job aids, standard training) that are built into DLA processes and systems.

Throughout the effort to implement the 2005 BRAC supply, storage, and distribution recommendation, the Army and Marine Corps maintained that the retail supply functions at the depots are integral to the maintenance process and therefore should not be transferred to DLA. The Army and Marine Corps only transferred responsibility for managing some storage and distribution functions to DLA. As a result, DLA does not make supply decisions such as determining the items to be stocked and the levels for those items. This means that decisions between the wholesale and retail inventory levels are handled by two separate organizations and therefore produce a delay in demand signals and may not be as easily coordinated. The Army and Marine Corps officials stated that they are not certain performance would improve if they transferred additional functions to DLA, though they are exploring further efforts.

Army officials expressed concerns about transferring retail supply functions at the depots to DLA. First, officials stated they were concerned that some of the efficiencies and effectiveness that are provided by its information technology system (i.e., the Logistics Modernization Program) could be lost if it transfers further retail management functions to DLA. However, the Air Force and Navy successfully integrated their information technology systems with DLA’s systems at the ALCs and FRCs, and Army officials told us that the Army has not explored whether this could be done with the Army’s system as well. In addition, officials we interviewed at Anniston Army Depot also expressed concern with DLA’s ability to provide parts to support depot operations, though these concerns were not based on a comprehensive business case analysis.

Marine Corps officials also expressed concern about not having control over supply functions and noted that it plans to take a gradual approach with respect to further transfer of functions to DLA to ensure that DLA performs sufficiently. For example, the Marine Corps and DLA are working together to improve demand planning by establishing a new organization within Marine Corps Logistics Command that interacts directly with DLA to improve supply chain performance, such as the quality of planning information and the availability of spare parts. However, the Marine Corps depot continues to purchase items from DLA wholesale and have them store the inventory until it is needed for maintenance. According to DLA officials this requires the service to invest in additional inventory, incur storage costs, and experience longer waits for parts. While the Marine Corps has been able to reduce its amount of retail inventory since January 2014, the retail inventory warehouse supporting the Albany Production Plant at the end of fiscal year 2015 contained about 67,000 separate inventory items valued at about $126 million, according to DLA. DLA officials stated that this represents a much larger amount of retail inventory than is likely needed. While Marine Corps officials credited this inventory with preventing some backorders, 56 percent of the serviceable retail inventory items at that site had no demand between March 2015 and March 2016, a potential indicator of unneeded inventory, although officials noted that some of this materiel was likely purchased and kept as a result of unexpected schedule changes.

Army and Marine Corps depot and Navy shipyard officials state that their maintenance processes differ from those at the Navy FRCs and Air Force ALCs. However, while maintenance processes may be tailored from one industrial site to another depending on the type of end items being maintained, the basic elements that determine retail inventory needs are the same. These include determining the number and schedule of end items to go through maintenance, the bill of materials (i.e., part lists), and the replacement factors for those parts. Officials also expressed concern over losing visibility of retail inventory if DLA is allowed to manage retail warehouses. However, similar concerns were addressed by DLA and service officials at Warner Robins ALC by allowing officials from both entities to access each other’s information systems. This has allowed the Air Force to benefit from tools and expertise available to DLA while increasing transparency and information-sharing. Additionally, as previously discussed, DLA in coordination with the Air Force and Navy have established extensive written policies, guidance, and processes for the management of the retail supply, storage, and distribution functions at the ALCs and FRCs.

Army and Marine Corps Depots and Navy Shipyards Have Not Assessed Costs and Benefits of Further Transferring Retail Management Functions to DLA

The Army and Marine Corps depots and the Navy shipyards have not assessed the costs and benefits of further transferring retail management functions to DLA. Although benefits may be realized by transferring retail supply, storage, and distribution functions to DLA as evidenced by the experience of the Air Force ALCs and Navy FRCs, the partial transfers of retail management functions have not yielded similar benefits for the Army and Marine Corps depots and Navy shipyards. Congress, in the House Report accompanying a bill for the National Defense Authorization Act for Fiscal Year 2014, mandated the Secretary of Defense conduct an assessment of the roles and missions of DLA. The assessment made several recommendations, one of which was for the Office of the Secretary of Defense and DLA to work with the Army and Marine Corps to prepare business case analyses on the transfer of retail supply, storage, and distribution responsibilities to DLA.

DLA began exploring further transfer of retail management functions at the Army and Marine Corps depots and Navy shipyards in the spring of 2015. DLA and the Marine Corps have agreed to an inventory consolidation and optimization effort in support of the Marine Corps’ Albany and Barstow production plants. As a result, DLA has begun to make adjustments in its approach to supporting these Marine Corps sites, but the Marine Corps will continue to manage their retail supply function. According to DOD officials, DLA has also initiated discussions with Army depot and Navy shipyard officials about the management of retail supply, storage, and distribution functions. For example, in December 2015 the Army and DLA established an integrated project team to identify additional opportunities for retail improvement. Officials state this effort will ultimately include the development of a cost-benefit analysis on shifting DLA’s point-of-sale to the Army forward to the production line while retaining the capabilities provided by Army information systems. However, DLA is taking these preliminary steps without official decisions by the Army, Marine Corps, and Navy on the optimal level of management by DLA at the depots and shipyards, respectively. DLA officials stated that these steps were being taken in an effort to overcome service reluctance to transferring retail functions to DLA at the depots and shipyards and achieve further inventory efficiencies across the department. However, neither the Assistant Secretary of Defense for Logistics and Materiel Readiness nor the services have prepared a comprehensive business case analysis–drawing on lessons learned from previous implementations–that systematically examined the costs, benefits, risks, and challenges associated with fully transferring supply, storage, and distribution functions at the Army and Marine Corps depots and Navy shipyards to DLA.

DOD guidance states that DOD materiel management shall operate as a high-performing and agile supply chain responsive to customer requirements during peacetime and war while balancing risk and total cost and that the Assistant Secretary of Defense for Logistics and Materiel Readiness should monitor the overall effectiveness and efficiency of the DOD materiel management systems and continually develop improvements. The guidance further states that all costs associated with materiel management, including acquiring, distributing, transporting, storing, maintaining, and disposing, shall be considered in making best value decisions throughout the DOD supply chain. Past GAO work indicates a standard way to assist in making best value decisions is the development and use of a comprehensive business case analysis. Our prior work on establishing a “lessons learned” process also found that assessing and using lessons learned from previous experience can provide a powerful method of ensuring that beneficial information is factored into the planning and work processes of future activities.

While DLA continues to pursue expanded retail management, the department has not prepared comprehensive business case analyses identifying the costs and benefits of such transfers in order to inform final decisions. Without conducting comprehensive business case analyses of costs, benefits, risks, and challenges for further transfer of retail management functions to DLA at the Army and Marine Corps depots and Navy shipyards, DOD leadership will not have the information needed to make informed decisions on developing improvements and cannot position the department to efficiently and effectively support and sustain weapon systems for the warfighter. In addition, examining lessons learned from previous transfers of retail supply, storage, and distribution functions–such as the successful Air Force ALC and Navy FRC implementations and the failed implementation at Norfolk Naval Shipyard–could help inform decisions and analysis on the degree of any future transfer efforts at the Army and Marine Corps depots and Navy shipyards. Further, without a decision based on business case analyses regarding the extent to which DLA should manage the supply, storage, and distribution functions at the Army and Marine Corps depots and Naval shipyards, the department risks underutilizing the expertise of DLA and not realizing the benefits DLA can bring to the management of supply, storage, and distribution functions—such as those realized by Air Force ALCs and Navy FRCs. As a result, DOD may continue to have a less efficient and effective supply chain at the Army and Marine Corps depots and Navy shipyards.

DLA and the Services Have Adopted Metrics Assessing Customer Service, but Do Not Have Metrics Needed to More Effectively and Efficiently Manage Operations

DLA and the services have adopted and review customer service metrics that measure the timely availability of spare parts for depot maintenance operations, but do not have metrics that allow them to fully assess the effectiveness and efficiency of supply operations. DOD guidance requires DLA and the services to be responsive to customer requirements while balancing risk and costs, conduct periodic performance and cost evaluations, and adopt metrics that provide information on customer service, internal efficiency, and costs. In an effort to do this, DLA and the services have adopted numerous customer service metrics that assess the timely availability of spare parts for depot maintenance. However, while DLA and the services have some internal efficiency measures, they generally have not adopted metrics that measure the accuracy of planning factors that are necessary to plan efficient and effective support of depot maintenance. Additionally, the services and DLA do not track the potentially significant costs created by a backorder (i.e., disruption costs) to supply and depot maintenance operations, which may prevent DLA and the services from optimizing supply and maintenance operations and may improve the efficiency and effectiveness of depot maintenance.

DOD’s Guidance Requires Performance Evaluations Using Varying Types of Metrics

DOD guidance requires DLA and the services to be responsive to customer materiel requirements while balancing risk and costs, conduct periodic performance and cost evaluations, and adopt metrics that provide information on customer service, internal efficiency, and costs. There are three types of metrics used in evaluating supply support.

Customer service metrics: assess the timely availability of spare parts.

Internal efficiency metrics: generally measure the amounts of excess inventory and the accuracy of the forecast for spare parts. As previously noted, there are a number of planning factors–the number and schedule of end items inducted into maintenance, the bill of materials (i.e., the list and quantity of parts), and the replacement factors (i.e., the estimated frequency of a part needing to be replaced)–that are key to forecasting the needed parts for depot maintenance.

Cost metrics: measure the costs of supply, storage, and distribution functions at industrial sites as well as the cost-effectiveness of resource planning and execution.

As we concluded in June 2014 and April 2015, a balanced approach across these key areas is important because without it any given metric could be optimized at the expense of other metrics. For example, a supply chain could achieve a high level of customer service if it was less focused on the costs of purchasing excess materiel and storing it. Alternatively, a supply chain could reduce its costs if it was less focused on the resulting effect on parts availability, the efficiency of depot maintenance operations, and readiness. Finally, a supply chain may not place enough of an emphasis on internal efficiency metrics, such as forecast accuracy and the accuracy of associated planning factors, resulting in excess inventory or part shortages (i.e., backorders) that are detrimental to customer service and increase costs. See figure 6 below for an overview of the general types of supply chain metrics and the importance of balancing these metrics.

DLA and the Services Have Adopted and Review Customer Service Metrics that Assess the Timely Availability of Spare Parts

DLA and the services have adopted and review numerous customer service metrics that assess the timely availability of spare parts for depot maintenance. However, the specific customer service measures used vary by service and largely depend upon the nature of DLA’s involvement in the retail supply, storage, and distribution functions at the industrial sites. In general, DLA reviews material availability, the number and age of backorders, and critical part shortages for all of the services’ industrial sites as well as a number of storage and distribution metrics, such as the time to receive and stow a part in storage and inventory accuracy. standard (e.g., 92 percent within 2 days)–and delivery response time–the time from a maintenance order being placed to the order being delivered to the depot artisan (e.g., 8 hours). With respect to the FRCs, DLA measures the fill rate for DLA-managed items on the gross demand plan– the monthly plan submitted by the FRCs to DLA as part of a formal collaborative forecasting process for FRCs’ projected spare part needs. Additionally, DLA measures metrics associated with depot-level reparables that are awaiting DLA-managed items in order to complete the repair process for both the ALCs and FRCs.

84 percent of the material and parts were ordered after the start of maintenance on the USS Ronald Reagan (CVN 76) in 2012 and 2013,

69 percent of the material and parts were ordered after the start of maintenance on the USS West Virginia (SSBN 736) in 2011-2013, and

51 percent of the material and parts were ordered after the start of maintenance on the USS Topeka (SSN 754) in 2013 and 2014.

DLA and the Navy shipyards focus on additional customer service metrics that are driven by the unique operating processes for ship maintenance. For example, DLA and the shipyard measure DLA’s performance providing the materials and parts identified by the shipyard at the start of a particular ship’s maintenance. However, these materials and parts only make up a portion of the total needed for maintenance since a large volume of the work on a ship is conducted through “open and inspect” processes, meaning that the need for a new part is not discovered until the ship is being disassembled and inspected in the shipyard, according to Navy officials (see sidebar). With respect to these materials and parts ordered after the ship enters maintenance, DLA and the shipyard track the average days from ordering the part to receiving the part at the shipyard.

Because DLA does not manage the retail supply functions at the Army and Marine Corps depots, the metrics used at these are largely limited to the previously mentioned customer service metrics of material availability, the number and age of backorders, and critical part shortages. In 2014, DLA and the Army began using the Army Supply Plan–a collaborative forecasting process for Army depots’ projected spare part needs–that has increased the complexity of the customer service metrics used to evaluate performance at Army depots. Specifically, DLA’s performance on material availability and backorders is broken out based on whether the collaborative forecast provided by the Army to DLA was used. In 2013, the Marine Corps and DLA began collaborating more closely on demand planning by establishing a new organization within Marine Corps Logistics Command that interacts directly with DLA to improve the accuracy of demand planning information and the availability of spare parts, according to Marine Corps officials. The Marine Corps and DLA have also expanded the number of items that use collaborative forecasting in an effort to improve supply availability. In 2015, DLA and the Marine Corps began an inventory consolidation and optimization effort to reduce DLA’s response time and improve its support to the Albany and Barstow Production Plants. The Marine Corps and DLA are monitoring performance of this new arrangement through a number of metrics, such as customer wait time (i.e., the number of days from ordering an item to receiving the item) and delivery response time.

DLA and the FRCs Have Adopted Internal Efficiency Metrics, but DLA and the Other Industrial Sites Do Not Have Metrics that Assess the Accuracy of Key Planning Factors

DLA and the FRCs have adopted internal efficiency metrics that measure the accuracy of key supply planning factors, but DLA and the other industrial sites generally do not have these types of metrics, which are critical to efficient support of depot maintenance. While the Army and Marine Corps depots, Air Force ALCs, and Navy shipyards have adopted some internal efficiency metrics, such as demand forecasting accuracy or the percentage of excess material, these metrics do not allow them, in conjunction with DLA, to identify inaccurate planning factors that can result in inefficiencies in the depot maintenance process.

Navy FRCs and DLA Have Adopted Metrics Measuring Key Planning Factors

The Navy FRCs and DLA have adopted and review metrics measuring the accuracy of their demand forecasting and key planning factors: (1) the number and schedule of end items inducted into maintenance, (2) the bill of materials (i.e., the list and quantity of parts), and (3) the replacement factors (i.e., the estimated frequency of a part needing to be replaced). Since fiscal year 2013, the Navy and DLA have used a performance- based agreement for depot-level reparables that includes metrics that measure the accuracy and stability of the schedule, bills of material, and the replacement factor as well as other types of performance metrics. Additionally, the Navy FRCs review the accuracy of bills of material and replacement factors for their major weapon systems on a regular basis and developed a metrics guide that defines and provides information on key metrics, such as bill of material and replacement factor accuracy metrics.

FRC Southwest officials noted that the availability and use of these metrics are critical to ensuring efficient and effective supply support. Specifically, FRC Southwest senior depot maintenance leaders emphasized to us that part shortages are often not the fault of DLA, but rather because DLA was not provided the necessary and accurate information by maintenance officials. Also, FRC East officials told us that the availability of the planning factor accuracy metrics has allowed them to better identify the issues that need to be addressed with respect to depot maintenance and supply support planning. For example, FRC East officials told us that the bill of materials and replacement factors are fairly accurate and stable, but that the scheduling of work (specifically for the depot-level reparables) was less than ideal, leading to inefficient depot maintenance processes and inadequate supply support.

Army and Marine Corps Depots, Air Force ALCs, and Navy Shipyards Have Not Adopted Metrics Measuring the Accuracy of Key Planning Factors

By contrast, the Army and Marine Corps depots, Air Force ALCs, and Navy shipyards have not adopted similar internal efficiency metrics that measure the accuracy of key supply planning factors, although department officials told us their accuracy was fundamental to efficient and effective supply support by DLA. For example,

Army depots: We were told by Anniston Army Depot maintenance officials that inaccurate planning factors are negatively affecting efficient and effective supply support for depot maintenance. The officials provided documentation that in fiscal years 2014 and 2015 there was a total of 1,167 program changes to ordered work at the depot valued at $212.1 million and requiring about an additional 1.5 million workload hours at Anniston Army Depot. Also, in June 2013, we found that the Army significantly underestimated the amount of new depot maintenance orders to be received from its customers across fiscal years 2006–2012 by a total of $10.1 billion. According to Army officials, such changes and additions to the workload make it difficult to efficiently and effectively plan supply support for depot maintenance. Furthermore, in September 2014, the DOD Inspector General found that the Army did not provide DLA with reliable forecasts for spare parts needed to support planned depot maintenance largely due to inaccurate and missing information associated with the bill of materials and replacement factors. The Army is in the process of taking steps to address the findings of that report, such as requiring material supportability analyses prior to accepting workload, improving schedule accuracy and stability, developing better policy and processes to review the accuracy of bills of materials, and developing metrics to assess the accuracy of the bill of materials. However, these recommended changes have not been completely implemented as of March 2016.

Marine Corps depot: Marine Corps depot officials told us on our visit to the Albany Production Plant in October 2015 that the depot had not received its complete planned work schedule for fiscal year 2016 even though fiscal year 2016 had already started. Additionally, Marine Corps officials told us that changes to planned work, including additions of unplanned work, occur throughout the year of execution. In 2012, we found that for 45 of the 60 orders for fiscal years 2010 and 2011 that we reviewed, customers increased quantities or added unanticipated workload requirements throughout the fiscal year that delayed completing work on existing orders. The Marine Corps depot maintenance officials noted that these types of changes have a negative effect on supply support given the need for proactive and advanced planning. Officials emphasized that an inaccurate schedule can lead to both excess inventory and part shortages.

Air Force ALCs: The Air Force has processes in place to periodically review and update some planning factors, such as the bill of materials and replacement factors, but has not adopted metrics that are regularly reviewed across the three ALCs to oversee the accuracy of planning factors. Further, information provided by Warner Robins ALC indicates issues associated with the accuracy of bills of materials for weapon systems. Specifically, the information reported by the officials shows a large percentage of the parts ordered for maintenance of weapon systems were not on the bills of materials. Oklahoma City and Warner Robins ALC officials and DLA officials also told us that accurate and stable scheduling of end items for maintenance was important to efficient and effective supply and maintenance operations. For example, improvements in the scheduling– establishing a set and consistent number of repairs being conducted monthly–of depot-level reparables (e.g., constant speed drives) resulted in improvements to supply support as well as overall performance of the repair shops for those end items (see sidebar).

Naval Shipyards: Naval Sea Systems Command and Norfolk Naval Shipyard officials told us that they do not have metrics that track the accuracy of planning factors, except for an excess material metric calculated at the end of a ship’s maintenance. We asked the Navy officials whether any thought had been given to conducting a “what if” analysis of the supply support for a ship’s maintenance to determine if the accuracy of the advanced planning could be improved, resulting in a more efficient supply operation that was less focused on ordering material and parts during maintenance and resolving backorders when material and parts were not available. To their knowledge, the officials could not recall such an assessment being conducted, but noted that the advanced planning for ship maintenance did consider a number of factors, such as past demand and the cost of the parts. Naval Sea Systems Command officials noted that they are currently reviewing the metrics associated with supply support at the shipyards, but this effort was in the very early stages and no decisions had been made as of early 2016.

The 2014 study sponsored by the Assistant Secretary of Defense for Logistics and Materiel Readiness also determined that the accuracy of planning factors, such as the bill of materials, was a problem and recommended that the Office of the Secretary of Defense, DLA and the services invest more effort in maintaining accurate bills of materials. Accurate schedules, bills of materials, and replacement factors are important to efficient and effective supply support across DOD depot maintenance. The less accurate the planning factor the more likely DLA will not be able to provide the correct mix of parts at the right time for depot maintenance to operate efficiently and effectively. Ultimately, the costs of inaccurate planning factors are both excess inventory and backorders. Without relevant metrics on forecast planning factors, DLA and the services are unable to determine the accuracy of their key planning factors and take actions to resolve any issues identified through measuring the accuracy of planning inputs in an effort to improve supply and depot maintenance operations.

Services Have Not Adopted Cost Metrics That Would Allow the Assessment of the Cost- Effectiveness of Supply and Depot Operations

DLA and the services review the general operating costs of supply, storage, and distribution functions that are managed by DLA, but have not adopted other cost metrics, such as the potentially significant costs created by backorders (i.e., disruption costs) to supply and depot maintenance operations. The cost of DLA’s management of retail supply, storage, and distribution functions for depot maintenance operations varies across the services. For example, the Air Force and DLA have agreed to a local recovery rate–a rate set at an estimated level to recover the costs of services–for DLA’s management of retail supply, storage, and distribution at the ALCs. DLA and the Navy are evaluating a similar methodology, but further negotiations are needed to reach agreement, according to the Fiscal Year 2017 Defense Working Capital Fund Budget. The Army and Marine Corps reimburse DLA for the costs associated with operating the storage and distribution functions at the depots, according to Army, Marine Corps, and DLA officials.

However, DLA and the services do not consistently measure or track other cost metrics, such as any cost created by backorders (i.e., disruption costs) to supply support and depot maintenance operations. As previously described, DLA and the services track the number, age, and criticality of backorders, but they do not consistently track the costs associated with workarounds–additional efforts to obtain a part and resolve a backorder so that depot maintenance can be completed. As figure 7 below illustrates, there are four general types of workarounds– cannibalization, expediting, local procurement, and local manufacturing– that all have potential costs to supply support and depot maintenance operations.

In our visits to 7 of 17 depot maintenance industrial sites, we identified examples of workarounds to obtain parts being tracked to some degree. For example, FRC Southwest tracks the parts that are cannibalized from one end item to satisfy a part need on another end item that is further along in the maintenance process. Warner Robins ALC also tracks similar information for the weapon systems (e.g., F-15, C-130, and C-5 aircraft) on which it conducts depot maintenance and reviews this information as part of supportability analyses that it conducts to improve parts support for future maintenance on the weapon systems. The Marine Corps depot maintenance and DLA officials told us that they use an “urgent and emergency requirements impact statement” to request an expedited purchase for a spare part from DLA. This statement includes the effects of the part not being available to depot maintenance operations, such as a line stoppage (i.e., maintenance not being able to continue) and a reduction in the workforce. However, according to DLA officials, there can also be costs associated with the expediting action by DLA, such as paying a premium to the commercial supplier providing the parts.

In addition to the potential costs directly associated with these workarounds, they also require personnel resources to arrange and execute the workaround. Depot maintenance and DLA officials told us that there is a general tendency to focus resources and attention on reacting to backorders (i.e., backorder resolution through workarounds) as opposed to preventing backorders. In our visits to the Anniston Army Depot and Oklahoma City ALC we observed depot maintenance production reviews conducted by senior service officials that demonstrated this heightened attention to backorder resolution. According to service and DLA officials, while these workarounds are needed at times so that work on an end item can be completed in a timely manner without halting or slowing the maintenance production process, DLA and the services generally do not fully track the necessary data– direct costs of the workaround as well as indirect costs such as personnel dedicated to backorder resolution–that would allow them to calculate, report, and assess the financial cost or benefit of the workaround.

In some instances, a backorder cannot be mitigated by a workaround and changes are required in the depot maintenance process, which is another type of disruption cost. In other words, depot maintenance and DLA officials have exhausted all options to obtain a suitable part for maintenance to continue its work and it must wait for the part to arrive. In these cases, depot maintenance officials report that they have several general options for mitigating the effect of an unavailable part.

Effect of Parts Shortages on High-Mobility Multi-purpose Wheeled Vehicle (HMMWV) Depot Maintenance In June 2013, we found that the depot maintenance work at Letterkenny and Red River Army Depots in fiscal years 2011 and 2012 could not be completed when anticipated due to parts shortages.

First, the sequence of maintenance can be adjusted so that maintenance can be continued on the end item. Once the part is received, then the depot artisan will move forward with the work that was stalled due to the lack of part.

Second, depot maintenance officials do as much work as possible, remove the end item from the maintenance production area once no more work can be conducted given the lack of parts, and begin work on another end item up until the same point (see sidebar). Once the parts are received, then the end items are brought back into the maintenance production area to finish the maintenance on the end items.

Third, depot maintenance officials may decide to stop maintenance on the end item(s) entirely, redistributing depot artisans to other maintenance efforts and clearing the production area for new work.

In order to perform the work on these vehicles, Letterkenny had to reestablish its production line which was previously shut down due to the lack of HMMWV work. Further, both depots had to hire contractor personnel to staff the production line and establish a supply chain so that the depots could obtain the parts to perform the work. Both depots encountered problems with obtaining sufficient quantities of parts, such as doors, gunner protection kits, windshields, turret bearings, and half shafts, to perform the work. This parts problem was exacerbated at Red River when the depot went to a double shift on disassembling and assembling the vehicles in April 2012. As a result of the parts shortage, the depots assembled 4,254 vehicles but the vehicles were missing parts and the work had to be carried over into fiscal year 2013 for completion once the parts were available. We made 3 recommendations–all of which have been implemented–related to improving budgeting for working capital funds and reducing carryover. When work has been ordered and funded (obligated) by customers (such as the services), but has not been completed at the end of a fiscal year, it is referred to as carryover.

In each of these cases, the lack of spare parts introduces multiple disruption costs that affect the efficiency and effectiveness of depot operations. important concepts, such as demand forecasting accuracy, across the department by establishing a team of subject matter experts from each of the services and DLA (see sidebar). experts, such as logistics management specialists, information technology and data experts, and operations research analysts (i.e., individuals with expertise in advanced mathematical and analytical methods), representing each of the services and the Defense Logistics Agency (DLA). sources and methods used by the services and DLA and evaluated potential department-wide metrics for measuring demand forecasting accuracy based on the available data sources.

At FRC East, Navy and DLA officials stated that disruption costs have a major effect on the efficiency and effectiveness of depot operations, and thought that it was not only reasonable but necessary for depot maintenance operations to develop metrics that measure them. Furthermore, during our visit to FRC East these officials provided a basic example of how a depot maintenance operation could articulate the disruption costs and the effect of those costs. Based on FRC East and DLA data, FRC East had $26 million in backorders accumulated across fiscal years 2010–2015 that resulted in $131 million in depot-level reparables being unable to complete repair and be provided back to the warfighter. This approximate 1 to 5 ratio of backorders to sales revenue (i.e., the completion of the repair and the sale of the end item to the warfighter) provides an estimate of the disruption cost of backorders to the efficiency and effectiveness of depot operations.

Furthermore, through the process of establishing these metrics, DOD developed additional areas for exploration and improvement, such as improving its guidance on demand forecasting.

However, the officials also stated that this estimate of or proxy for the disruption cost is likely a conservative one because it does not factor in many of the disruption costs discussed above. For example, the estimate does not include the costs associated with (1) the time and efforts to resolve those backorders through workarounds by personnel, (2) packaging up the dissembled pieces of the end item since maintenance could not be completed, (3) moving the end item away from the maintenance area to a storage warehouse, (4) storing the end item until the necessary part(s) arrive, (5) bringing the end item back to the production area once the parts arrive, and, lastly, (6) preparing the end item for the maintenance process to continue.

A 2014 study sponsored by the Assistant Secretary of Defense for Logistics and Materiel Readiness found that the costs of not having parts when they are needed can be considerable and that the department, if they could measure such costs, might find that the disruption costs of not having parts are high relative to the costs of steps that would improve material availability. Based on our work in the area and our visits to 7 of 17 depot maintenance industrial sites, the lack of metrics to track and assess disruption costs prevents decision makers from understanding the financial effect of backorders and taking any necessary actions to systemically mitigate disruption costs. Additionally, service depot maintenance and DLA officials are unable to make tradeoffs with respect to investing in stocking additional retail inventory or making improvements to supply support rather than relying on workarounds that can be costly in terms of financial and personnel resources, as described in figure 8.

As a result the services and DLA lack the information necessary to make decisions about the cost-effective use of limited resources. For example, the total disruption costs to a depot maintenance industrial site created by 85 percent material availability by DLA may be more costly than the investments needed to improve the material availability to 90 percent. However, the department does not track this type of performance information to guide its decisions in optimizing the cost-effectiveness of supply and depot maintenance operations. Without measuring, tracking, and reviewing the disruption costs associated with backorders, DLA and the services are not able to assess whether the right investment in inventory at a particular industrial site is being made to support cost- effective supply and maintenance operations at industrial sites.

Conclusions

Cost-effective and efficient management of inventory is integral to ensuring that the services’ depot-level industrial sites can fulfill their mission to provide the warfighter with reliable weapon systems in a timely manner. DOD actions in response to the 2005 BRAC recommendation to transfer supply, storage, and distribution functions at these sites from the services to DLA have had some positive effects at the Air Force ALCs and the Navy FRCs, but the services could take additional steps to ensure they realize the possible benefits of further transfers of retail functions. For instance, the Army and Marine Corps depots and Navy shipyards may benefit from transferring more of their retail supply functions to DLA. While officials relayed to us the concerns they had about transferring functions to DLA, they have not fully assessed the costs and benefits of doing so. Without conducting business case analyses that, among other things, draw on lessons learned from the experiences at the ALCs and FRCs—as well as the failed implementation at Norfolk Naval Shipyard—the Army and Marine Corps depots and Navy shipyards are not positioned to know what could be gained by further transferring retail functions to DLA and the department is unable to determine the degree to which the retail supply functions should be transferred to DLA.

Another key step to promoting cost effective and efficient supply support at the industrial sites is accurate measurement and monitoring of performance, which can provide insight into an organization’s successful, and less than successful, strategies. For instance, without accurate measures of planning factors—such as how often a part needs to be replaced—the services and DLA will not have information that could aid them in determining the efficiency of their supply and depot maintenance operations, as well as identify root causes of inefficiencies. Similarly, there may be metrics to more accurately determine the disruption costs of backorders than the services and DLA are currently using. As we discuss, disruptions to the orderly flow of work through the maintenance process may sometimes cost more than taking other steps to avoid such disruptions—as well as leading to lost productivity and delayed delivery to the warfighter. Developing metrics to determine the full costs of such disruptions may be difficult given differences across the services and involve considerable effort, but the department has demonstrated the ability to develop and implement metrics that assess complex concepts, such as demand forecasting accuracy, through establishing a team of experts to evaluate available data sources and approaches. Without taking measured steps to implement such metrics, the services and DLA will continue to have an imperfect understanding of the current costs of their processes, and will lack insight into areas where they could take action to achieve greater cost-effectiveness.

Recommendations for Executive Action

To increase department-wide supply chain efficiencies and effectiveness in support of maintenance at the Army and Marine Corps depots and Navy shipyards, we recommend that the Secretary of Defense direct the Assistant Secretary of Defense for Logistics and Materiel Readiness, in conjunction with the Director, Defense Logistics Agency, and the Secretaries of the Army and Navy and the Commandant of the Marine Corps to take the following two actions: assess through a comprehensive business case analysis–drawing on lessons learned from previous efforts–the costs and benefits of DLA managing the retail supply, storage, and distribution functions at the Army and Marine Corps depots and Navy shipyards; and use the analysis to make a decision on the degree to which DLA should manage these functions at the Army and Marine Corps depots and Navy shipyards.

To improve the efficiency and effectiveness of supply and maintenance operations, in accordance with DOD guidance, we recommend that the Secretary of Defense direct the Assistant Secretary of Defense for Logistics and Materiel Readiness, in conjunction with the Director, Defense Logistics Agency, and the Secretaries of the Army, Navy, and Air Force and the Commandant of the Marine Corps to take the following two actions: develop and implement metrics that measure the accuracy of planning factors, such as the schedule, bill of materials, and replacement factors, used for depot maintenance; and take action, as appropriate and necessary, to resolve any issues identified through measuring the accuracy of planning inputs in an effort to improve supply and depot maintenance operations.

To be able to assess the cost-effectiveness of supply and depot maintenance operations, in accordance with DOD supply chain management guidance, we recommend that the Secretary of Defense direct the Assistant Secretary of Defense for Logistics and Materiel Readiness, in conjunction with the Director, Defense Logistics Agency, the Secretaries of the Army, Navy, and the Air Force, and the Commandant of the Marine Corps to take the following two actions: take steps to develop and implement metrics, to the extent feasible, to measure and track disruption costs created by the lack of parts at depot maintenance industrial sites by, for example, establishing a team of supply and depot maintenance experts from DLA and the services to assess potential data sources, approaches, and methods; and take action, as appropriate, to address any inefficiencies identified by the disruption cost metrics in supply and depot maintenance operations.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. In written comments, DOD concurred with our six recommendations. DOD’s comments are also reprinted in their entirety in appendix V. DOD also provided technical comments, which we incorporated into the report as appropriate.

DOD concurred with our recommendations to assess the costs and benefits of allowing DLA to manage retail supply, storage, and distribution functions at the Army and Marine Corps depots and Navy shipyards using a comprehensive business case analysis and to use this analysis to guide future decision making. DOD noted that the Army and DLA are currently working together to investigate and evaluate the costs and benefits of having DLA manage the retail supply, storage, and distribution functions at the Army depots. The Army and DLA plan to have this completed by July 2016 and to share the results of the analysis with the Marine Corps and Navy. We agree that this is a good first step with respect to the Army and DLA, but also believe that, consistent with our recommendation, this effort should be broadened. Specifically, the Assistant Secretary of Defense for Logistics and Materiel Readiness in conjunction with the Navy, Marine Corps, and DLA should also conduct an analysis of the costs and benefits of DLA managing the retail supply, storage, and distribution functions at the Navy shipyards and Marine Corps depot. Furthermore, the Assistant Secretary of Defense for Logistics and Materiel Readiness in collaboration with the Army, Navy, and Marine Corps following this analysis of costs and benefits should make a decision on the degree to which DLA should manage these functions.

DOD concurred with our recommendations related to the improvement of metrics that measure the accuracy of planning factors. DOD stated that it will begin developing planning factor metrics related to schedule variance, delivery time variance, and the accuracy of the bills of materials. DOD also noted that the Army and Navy have already begun developing metrics related to the accuracy of planning factors. DOD expects to have these metrics completed by October 2018.

DOD concurred with our recommendations related to the improvement of metrics to measure and track the disruption costs created by the lack of parts at depot industrial sites. DOD is determining whether measuring such costs is feasible, and plans to include service input to help identify the different costs associated with these disruptions. They are also looking at the disruptions caused by additional factors outside of supply and maintenance, such as unexpected program or funding changes. DOD also expects to have concluded this effort by October 2018.

We are sending copies of this report to the appropriate congressional committees; the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; the Commandant of the Marine Corps; the Under Secretary of Defense for Logistics and Materiel Readiness; and the Director, Defense Logistics Agency. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have questions about this report, please contact me at merrittz@gao.gov or (202) 512-5257. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Scope and Methodology

To assess the extent to which the services transferred retail supply, storage, and distribution functions at Department of Defense (DOD) industrial sites to the Defense Logistics Agency (DLA), and whether the results have been used to inform future efforts, we reviewed Office of the Secretary of Defense (OSD), DLA, and service guidance, and documentation related to retail inventory management at service industrial sites; we conducted interviews with officials from OSD, DLA, service materiel commands, and service industrial sites; and, visited 7 of 17 service industrial sites to observe maintenance operations and retail inventory processes. This non-generalizable sample was selected to provide a mix of services (at least one per type of industrial site–Army depot, Navy shipyard, Navy Fleet Readiness Center, Air Force Air Logistics Complex, and Marine Corps depot), types of weapon systems repaired (a mix of air, ground, and sea vehicles as well including facilities that conduct component-level repair), and the extent to which DLA performs retail supply, storage, and distribution functions at the site (including those that had fully transferred these functions to DLA and those that had not). We also took into consideration the recommendations of service supply chain personnel. We analyzed pertinent documents and interviewed officials responsible for retail inventory management at selected defense industrial sites to determine any challenges related to transferring retail functions, and whether the services have assessed the costs and benefits of further transferring retail functions to DLA. We interviewed officials about the extent to which different information systems used by DLA and the services were used for retail inventory management and how these systems interface, if at all. We reviewed inventory data from 2010 through 2015 using DLA’s Enterprise Business System, the Army’s Logistics Modernization Program, the Navy’s Enterprise Resource Planning system, and the Air Force’s D200 system, which is a legacy system the Air Force uses to track most of its inventory. To assess the reliability of the data, we reviewed summary level inventory data, interviewed officials responsible for generating the data, and observed data entry during retail inventory operations at service industrial sites. We determined that the data we used were sufficiently reliable for the purposes of this report.

To support our analysis on each of the objectives, we contacted or interviewed officials from the following organizations: Office of the Secretary of Defense

Office of the Deputy Assistant Secretary of Defense for Maintenance

Office of the Deputy Assistant Secretary of Defense for Supply Chain

Defense Logistics Agency, Headquarters

Defense Logistics Agency, Aviation

Defense Logistics Agency, Land and Maritime

Defense Logistics Agency, Distribution

Army Materiel Command

TACOM Life Cycle Management Command

Anniston Army Depot

Naval Air Systems Command

Naval Sea Systems Command

Naval Supply Systems Command

Fleet Readiness Center East

Fleet Readiness Center Southwest

Norfolk Naval Shipyard

Marine Corps Logistics Command

Albany Production Plant

Air Force Materiel Command

Air Force Sustainment Command

Oklahoma City Air Logistics Complex

Warner Robins Air Logistics Complex To determine the extent to which DLA and the services have adopted metrics that allow them to effectively and efficiently manage supply and maintenance operations, we analyzed DOD, DLA, and service policies, regulations, and guidance pertaining to the use of metrics for the management of inventory and maintenance operations. We visited 7 of 17 depot maintenance industrial sites, using a non-generalizable sample as previously described, in order to discuss with the DLA and service depot maintenance officials the metrics used to manage supply and depot maintenance operations as well as to understand the application of the metrics at depot maintenance industrial sites. We reviewed documentation, such as performance management briefing slides, and analyzed DLA’s and the services’ use of metrics to manage their inventory in against DOD guidance, which requires DLA and the services to be responsive to customer requirements while balancing risk and costs, conduct periodic performance and cost evaluations, and adopt metrics that provide information on customer service, cost, and internal efficiency. Specifically, we assessed DLA and a service as using a particular type of metric providing information on customer service, cost, or internal efficiency if the metric was a regular part of service inventory management and depot maintenance performance reviews. We also conducted interviews with DLA, service, and Office of the Secretary of Defense officials to understand and corroborate the use of performance metrics used to inform inventory and depot maintenance management decisions. We also observed a number of inventory management meetings held at the services, such as those discussing backorders and backorder resolution, schedules, and the supportability of repair programs.

To assess the status of backorders for DLA-managed items at service industrial sites, we collected DLA’s complete backorder data from its Enterprise Business System for fiscal years 2013 through 2015. From this data, we removed all “unactionable” backorders, as well as backorders that were not tied to a service industrial site. The remaining data were then analyzed by service industrial site, supply chain, and acquisition advice code. In addition, we assessed the reliability of these data by analyzing the policies and processes in place to ensure data collected through this information system are correct and protected from unauthorized modification, conducting interviews on the information system with knowledgeable DLA officials, and examining the data for errors and outliers. We determined that the data were sufficiently reliable for our purposes. To assess the extent to which DLA and each of the services have implemented collaborative forecasting methods at the industrial sites, we analyzed and compared the collaborative processes used by the Army, Navy, Air Force, and Marine Corps. We reviewed the metrics used to monitor collaborative forecasting efforts and interviewed DLA and service representatives involved with collaborative forecasting, as well as the depot customers who rely on accurate forecasts. Finally, to describe service inventory improvement efforts, we analyzed documents and conducted discussions with service personnel, had service and DLA supply personnel identify key improvement efforts undertaken by their respective organizations, and compared inventory improvement efforts among services.

We conducted this performance audit from May 2015 to June 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Backorders for Defense Logistics Agency (DLA) Managed Spare Parts at Service Industrial Sites

Total backorders for DLA-managed spare parts at the service industrial sites decreased by about 15 percent from October 2012 through September 2015, though they are currently higher than the low achieved around February 2014, as shown in figure 9 below. We collected backorder data from DLA for fiscal years 2013 through 2015 and focused our analysis on backorders for DLA-managed spare parts that occurred at one of the service industrial sites.

Backorders for DLA’s Supply Chains Generally Decreased

Backorders associated with each of DLA’s supply chains generally decreased from October 2012 through September 2015, as shown in figure 10 below. DLA has multiple supply chains, which handle the procurement of inventory parts, as well as conduct customer outreach and handle backorders. DLA has designed their business processes such that a given customer (e.g., a service industrial site) only needs to interact with a single supply chain when it needs to order parts, discuss forecasts, or receive updates on backorders. For example, Anniston Army Depot interfaces directly with DLA’s Land supply chain, even for those parts that are not specifically handled by the Land supply chain. DLA’s Land supply chain saw an increase of around 17 percent in backorders over this time period, though the other DLA supply chains saw reductions of at least 15 percent, and the Maritime supply chain exhibited a decrease of nearly 40 percent. The increases and decreases to backorders seen here are discussed in more detail in the service-specific sections below.

Most Backorders Are For Stocked or Insurance Parts

DLA’s backorders, when categorized by how the part is acquired and stocked, have remained generally steady from October 2012 through September 2015, although backorders for parts that are the most regularly stocked have increased since March 2014, as shown in figure 11 below. The Department of Defense (DOD) uses a coding system– known as “acquisition advice codes”–to categorize parts according to its approach for acquiring and stocking them. This includes parts that are regularly stocked (code “D”), not stocked and must be procured (code “J”), directly delivered from vendors (code “H”), or kept on hand in small quantities, referred to as “insurance items” (code “Z”), among others.

As seen in figure 11 above, the majority of DLA’s backorders for spare parts at service industrial sites are actually for parts coded “D,” which represent parts that are described by DLA as “stocked.” These are the parts that, due to their regular demands, are intended to be “on the shelves” on a regular basis. Parts that are supposed to be stocked can experience backorders if there is a sudden spike in demand or difficulties with the supply of inventory. These are parts where an increased investment in inventory may prevent work-disrupting backorders with minimal risk, since they are parts that DLA expects to sell over time, and therefore carry less risk to DLA to purchase. DLA officials attributed the fluctuation in stocked parts and insurance parts in late 2012 and early 2013 to the implementation of new methods for setting inventory levels for certain parts.

The second largest category of backorders is comprised of items coded “Z,” which represent what are called “insurance items.” These are parts that are only needed intermittently, and so DLA keeps a nominal quantity on hand in order to mitigate the effects of long lead times. The relatively low demand for “Z” coded parts suggests that the backorders could be a result of the tradeoffs that DOD has to make with respect to inventory investment and customer service. On the one hand, stocking more of these inventory parts could conceivably lower these types of backorders. However, this would entail a larger investment in inventory and would also increase DLA’s risk of purchasing and paying to store excess inventory. DLA has to make decisions on which parts are most likely to be needed by its customers in order to strike a balance between inventory investment and customer service.

The remaining parts we analyzed were coded either “J” or “H.” Code “J” parts are considered “non-stocked” items, which are not generally on the shelf and are procured only when DLA receives a funded requisition for the part. In other words, DLA generally does not purchase these parts based only on a forecast. Code “H” parts are those that are delivered directly to the customer by a third-party vendor. While DLA centrally manages these parts, they do not always keep a stock of them on hand, relying instead on the commercial vendor to provide the part when requisitioned.

Changes in Backorders Vary by Service

The trend of the number of backorders for DLA-managed spare parts varies across the service industrial sites. With respect to the Army, backorders for DLA-managed spare parts at the depots decreased for almost every location from October 2012 through September 2015, as shown in figure 12 below. With the exception of Red River Army Depot, which saw an increase of about 7 percent, each of the Army industrial sites saw backorders for DLA-managed parts fall by at least 35 percent, while two sites–Corpus Christi and Anniston–saw their backorders fall by more than 70 percent and 49 percent, respectively. DLA officials attributed the reductions to a concerted effort at backorder reduction undertaken within the Land and Maritime supply chains. DLA officials also noted that while backorders at Red River have increased between fiscal years 2013 and 2015, backorders there had decreased in fiscal year 2015, which they attributed partially to the implementation of the Army Supply Plan.

With respect to the Navy shipyards, changes in the backorders for DLA- managed parts varied at every location from October 2012 through September 2015, as shown in figure 13 below. For example, Norfolk Naval Shipyard saw a decrease in backorders of over 60 percent, while Portsmouth saw a decrease of around 20 percent. Meanwhile, Puget Sound held relatively steady with an increase in backorders of less than 2 percent, while Pearl Harbor saw its backorders increase by over 40 percent. DLA officials stated that they were not entirely certain why backorders increased at Pearl Harbor, but speculated that it might be due to the extended repairs of two Virginia-class submarines during this time period. DLA officials noted that the high number of backorders for Norfolk Naval Shipyard in late 2012 were due to the attempted transfer of the supply function at Norfolk Naval Shipyard to DLA’s information systems and processes. As previously discussed in this report, DLA and the Norfolk Naval Shipyard attempted the full transfer of the supply function to DLA’s information systems and processes, but after 7 months decided to revert back to Norfolk Naval Shipyard’s information systems and processes. DLA officials also attributed the spike at Norfolk in September 2014 to increased end-of-year spending by the Navy.

With respect to the Navy Fleet Readiness Centers (FRCs), backorders for DLA-managed parts generally increased from October 2012 through September 2014, as shown in figure 14 below. The Navy FRCs implemented DLA’s Inventory Management and Stock Positioning (IMSP) system during this time frame. However, the FRCs made the transition at different times. San Diego was the first to implement IMSP, in June 2013, while Jacksonville made the transition to IMSP in late October 2013. By contrast, Cherry Point did not move to IMSP until March 2014, which corresponded with a significant increase in backorders. FRC officials at Cherry Point told us that there was a significant increase in unplanned workload during that time due to the transition of the AV-88 program from the commercial sector to Cherry Point. However, the overall trend for all FRCs has been a steady increase of at least around 40 percent during the transition, peaking generally in September 2014. This could reflect the “growing pains” of transitioning to IMSP, as Air Force Air Logistics Complex (ALC) officials noted that it took them more than 2 years to work through the initial implementation challenges of fully transferring the retail supply function to DLA’s information systems and processes. As such, it is not yet clear if the FRCs will experience the same backorder reduction noted by the ALCs over the 5-year period after they transferred retail inventory functions to DLA.

With respect to the Air Force ALCs, backorders for DLA-managed parts decreased, and in some cases significantly, from October 2012 through September 2015, as shown in figure 15 below. For example, DLA- managed backorders at Ogden have dropped by more than 40 percent over the past 3 years. Though not as large, the reductions seen at Oklahoma City and Warner Robins have also been meaningful, at about 13 percent and 38 percent, respectively. As noted in the report, Air Force officials attribute their reduced backorders over the past 5 years to the decision to transfer retail supply, storage, and distribution functions to DLA.

With respect to the Marine Corps Production Plants, backorders for DLA- managed parts increased from October 2012 through September 2015, as shown in figure 16 below. While Albany saw its backorders for DLA- managed items increase by 9 percent over the 3-year period, Barstow’s backorders increased by over 40 percent during that same time. DLA officials noted that DLA supports the Marine Corps by focusing primarily on key items that impact the materiel availability at the depots, but that a majority of their regular business still involves other parts that are not particularly emphasized.

Appendix III: Service Methods to Support Collaborative Forecasting with the Defense Logistics Agency (DLA) at Industrial Sites

Each of the services has implemented a different method for developing and providing collaborative forecasts to DLA, though these methods do share some characteristics. Through collaborative forecasting, DLA and the customer (i.e., the industrial site) work together to evaluate historical demand data for spare parts and tailor forecast plans for those spare parts based on projected future usage. Collaborative forecasting is prospective, in that it uses various planning factors–the repair schedule, the list of parts required for the repair, and the rates at which those parts are replaced–to guide inventory management decisions, rather than DLA’s historical sales data. As part of the 2010 Comprehensive Inventory Management Improvement Plan, the Department of Defense (DOD) created a sub-plan to improve inventory forecasting that included expanding the use of collaborative forecasting. In addition, DOD Manual 4140.01, Vol. 2 requires the use of collaboration to improve the accuracy of forecasts. In June 2014, we found that DLA’s collaborative forecasting program had not improved the aggregate forecasting accuracy or used a comprehensive approach to manage the program and recommended that the department take steps, such as developing additional performance metrics and using those metrics to measure and monitor performance through regular performance management meetings. To address the recommendation, DLA officials stated that they have begun to consider additional metrics to assess the performance of the effort, and senior management has begun to monitor demand forecasting accuracy across DLA.

As noted earlier, service industrial sites rely on several key factors when planning workload, and similar factors–the number of end items that need repair, parts lists (i.e., bills of materials), and replacement rates–are just as important when the services develop a collaborative forecast. Each of the services’ collaborative forecast methods uses these three factors as the basis to develop their submission to DLA. Some service methods also have other similarities, as noted in table 1.

However, while these methods share similar characteristics, they also differ in some key respects. For example, the methods focus on different timeframes. The Army Supply Plan looks out 5 years, as does the Air Force’s method, while Navy Aviation provides data for 2 years. The services also have different levels of experience using the methods. Officials noted that the Air Force has been using Demand Data Exchange since 2009, Navy Aviation has been using its Gross Demand Plan since 2013, and the Army started the Army Supply Plan in late 2014.

Further, the services do not send the same amount of data to DLA. The Navy and Army submit all required parts and quantities for all planned maintenance at the industrial sites to DLA and DLA and the services have automated and manual processes in place to review these submissions. Once reviewed, the collaborative forecast submission is used by DLA in establishing its demand plans. The Air Force and Marine Corps, in collaboration with DLA, identify items through a manual process and establish projected forecasts that are used by DLA in establishing its demand plans. Officials noted that these items are identified based on anticipated changes in demand or the fact that the item has been problematic in some fashion.

Finally, the service forecasting methods are supported in different ways by DLA. For example, the Army and Navy Aviation both submit a full list of depot requirements to DLA. However, DLA uses different methods to support both sets of data; the Army is supported heavily by DLA’s Peak and Next-Gen level-setting methods, which are designed for items that are inherently difficult to forecast. However, these methods initially excluded retail inventory, and so are not as regularly used to support Navy Aviation.

As noted in the above table, the services and DLA have developed a number of different ways of assessing the effectiveness of collaborative forecasting. The collaborative forecasting mechanisms–specifically the inputs, outputs, and performance metrics–are all regularly reviewed by DLA and service personnel. In addition, Office of the Secretary of Defense officials noted that their future plans for inventory management involve monitoring and assessing the effectiveness of these different collaborative forecasting mechanisms.

Background We reviewed a number of depot improvement efforts the services have underway. While some officials described these as “leading” or “best” practices, we did not assess the extent to which these efforts made concrete improvements to inventory outcomes, in part due to the difficulty of separating causal factors from the large number of variables involved. In addition, we only visited 7 of the 17 service industrial sites (Anniston Army Depot, Norfolk Naval Shipyard, Fleet Readiness Center East, Fleet Readiness Center Southwest, Oklahoma City Air Logistics Complex, Warner Robins Air Logistics Complex, and Albany Production Plant). Therefore, there may be additional efforts that have not been identified.

Goals of the Efforts The efforts listed have varying goals, but their overall focus can be grouped into three categories: 1.

Inventory management: Providing better accountability, reducing costs, and distilling lessons learned.

Navy Aviation includes Marine Corps Aviation.

2. Collaboration: Improving the

level of trust and communication between the service and Defense Logistics Agency. 3. Maintenance throughput: Increasing the rate at which the depot can repair end items.

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Zina D. Merritt, (202) 512-5257, or merrittz@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Suzanne Wren (Assistant Director), John Bumgarner, James Lackey, Amie Lesser, Greg Pugnetti, Mike Silver, Sabrina Streagle, John Trubey, and Erik Wilkins-McKee made key contributions to this report.

Related GAO Products

Defense Inventory: Services Generally Have Reduced Excess Inventory, but Additional Actions Are Needed. GAO-15-350. Washington, D.C.: April 20, 2015.

High-Risk Series: An Update. GAO-15-290. Washington, D.C.: February 11, 2015.

Defense Inventory: Actions Needed to Improve the Defense Logistics Agency’s Inventory Management. GAO-14-495. Washington, D.C.: June 19, 2014.

High-Risk Series: An Update. GAO-13-283. Washington, D.C.: February 14, 2013.

Defense Inventory: Actions Underway to Implement Improvement Plan, but Steps Needed to Enhance Efforts. GAO-12-493. Washington, D.C.: May 3, 2012.

Defense Logistics: DOD Needs to Take Additional Actions to Address Challenges in Supply Chain Management. GAO-11-569. Washington, D.C.: July 28, 2011.

Opportunities to Reduce Potential Duplication in Government Programs, Save Tax Dollars, and Enhance Revenue. GAO-11-318SP. Washington, D.C.: March 1, 2011.

High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February 2011.

DOD’s 2010 Comprehensive Inventory Management Improvement Plan Addressed Statutory Requirements, But Faces Implementation Challenges. GAO-11-240R. Washington, D.C.: January 7, 2011.

DOD’s High-Risk Areas: Observations on DOD’s Progress and Challenges in Strategic Planning for Supply Chain Management. GAO-10-929T. Washington, D.C.: July 27, 2010.

Defense Inventory: Defense Logistics Agency Needs to Expand on Efforts to More Effectively Manage Spare Parts. GAO-10-469. Washington, D.C.: May 11, 2010.

High-Risk Series: An Update. GAO-09-271. Washington, D.C.: January 2009.

Defense Inventory: Army Needs to Evaluate Impact of Recent Actions to Improve Demand Forecasts for Spare Parts. GAO-09-199. Washington, D.C.: January 12, 2009.

Defense Logistics: Lack of Key Information May Impede DOD’s Ability to Improve Supply Chain Management. GAO-09-150. Washington D.C.: January 12, 2009.

Defense Inventory: Management Actions Needed to Improve the Cost Efficiency of the Navy’s Spare Parts Inventory. GAO-09-103. Washington, D.C.: December 12, 2008.

DOD’s High-Risk Areas: Efforts to Improve Supply Chain Can Be Enhanced by Linkage to Outcomes, Progress in Transforming Business Operations, and Reexamination of Logistics Governance and Strategy. GAO-07-1064T. Washington, D.C.: July 10, 2007.

Defense Inventory: Opportunities Exist to Save Billions by Reducing Air Force’s Unneeded Spare Parts Inventory. GAO-07-232. Washington, D.C.: April 27, 2007.

Defense Inventory: Opportunities Exist to Improve the Management of DOD’s Acquisition Lead Times for Spare Parts. GAO-07-281. Washington, D.C.: March 2, 2007.

High-Risk Series: An Update. GAO-07-310. Washington, D.C.: January 2007.

DOD’s High-Risk Areas: Progress Made Implementing Supply Chain Management Recommendations, but Full Extent of Improvement Unknown. GAO-07-234. Washington, D.C.: January 17, 2007.

DOD’s High-Risk Areas: Challenges Remain to Achieving and Demonstrating Progress in Supply Chain Management. GAO-06-983T. Washington, D.C.: July 25, 2006.

Defense Inventory: Actions Needed to Improve Inventory Retention Management. GAO-06-512. Washington, D.C.: May 25, 2006.

High-Risk Series: An Update. GAO-05-207. Washington, D.C.: January 2005.