Defense Logistics: Army Should Track Financial Benefits Realized from its Logistics Modernization Program
Released: 2013-11-13T00:00:00

Letter

Background

The Army initiated LMP in 1999 to replace two aging materiel management systems—the Commodity Command Standard System and the Standard Depot System. In replacing these systems, which had been used for more than 30 years to manage inventory, depot maintenance, and arsenal manufacturing operations, the Army expected LMP to help transform its logistics operations. A modified commercial off-the-shelf system, LMP was intended to support AMC industrial operations and improve business processes and practices in areas such as operations at depots and arsenals and inventory management. By providing a single source of data and integrated decision-making tools, LMP was expected to increase efficiencies in AMC operations—such as buying and managing spare and repair parts and conducting depot level maintenance. For example, in the area of planning future maintenance capacity, the Army expected improvements to AMC budget forecasts and adjustments through reduced repair cycle time, better resource allocation, increased production throughput, reduced production cost, and more accurate production schedules. With LMP’s deployment, the Army expected to achieve benefits that included reduced equipment repair times, improved inventory forecasting, and cost savings. Additionally, LMP is part of the Army’s broader strategy to implement enterprise resource planning systems in other areas and, as such, is expected to integrate with other Army systems, to include the General Fund Enterprise Business System and the Global Combat Support System- Army.

The deployment of LMP Increment 1 across AMC occurred in three phases between 2003 and 2010. The first phase of LMP deployment occurred at the Communications-Electronics Command and Tobyhanna Army Depot in July 2003. LMP was originally expected to be fully deployed across AMC by June 2004, but problems experienced during the first phase of deployment caused the Army to delay further deployment until these problems were resolved. The second phase of LMP deployment occurred at the Aviation and Missile Command and Corpus Christi and Letterkenny Army Depots in May 2009. The third and final deployment phase occurred in October 2010 at depots, arsenals, and other sites within the Army Sustainment Command, the Joint Munitions and Lethality Command, and the Tank-automotive and Armaments Command.

The Army has plans to expand the capabilities of LMP. On December 27, 2011, the Under Secretary of Defense for Acquisition, Technology and Logistics signed an Acquisition Decision Memorandum that categorized the deployed components of LMP as Increment 1 and considered all new LMP acquisition activities after December 28, 2011, to be Increment 2. The memorandum directed the Army, in preparing for the next acquisition milestone review of Increment 2, to submit an updated business case along with other specified documents. The Army prepared and submitted documents in response to the memorandum, including an updated business case and an accompanying economic analysis, for an acquisition milestone review that was held on June 25, 2013. As a result of this review, the Army received approval for Increment 2 to move into the engineering development phase of its acquisition. The Army intends to deploy Increment 2 in three waves, beginning in December 2013, and is planning for the deployment to be completed by September 30, 2016. Figure 1 shows a timeline for LMP development and deployment, including both Increment 1 and Increment 2.

As of May 2013, the Army had increased its projections for the system’s total life cycle costs from $2.6 billion (fiscal years 2000 through 2021) to more than $4 billion (fiscal years 2000 through 2026). This estimate includes the fielded components of LMP—Increment 1—as well as the expansion of LMP capabilities under Increment 2 (see table 1).

LMP Supports AMC’s Industrial Operations, but Additional Development Is Needed to Meet Certain Critical Requirements

AMC is using LMP to support its industrial operations, but additional development of LMP is necessary, according to the Army, because the current system does not support certain critical requirements that have emerged since the initial development of LMP and because the current system will not enable the Army to generate auditable financial statements. Officials at the 14 AMC sites we visited stated that LMP provided the core functionality they needed to support their operations. They stated that over time they are improving in their ability to use LMP, and some sites have locally developed tools to augment its capabilities. Army officials stated that although LMP is functional, the current system does not support certain critical requirements, such as requirements related to automatically tracking repair and manufacturing operations on the shop floor of depots and arsenals. In addition, according to Army officials, the current system will not enable the Army to generate financial statements validated as ready for audit by 2017, the statutory deadline for this goal. Increment 2 is intended to address these shortcomings.

AMC Is Using LMP to Support Its Industrial Operations

AMC is using LMP to support operations at its industrial sites. Officials at all 14 of the sites we visited between October 2012 and March 2013 told us that LMP provided the core functionality they needed to conduct their operations. Specifically, officials from each of the five Army depots stated that they were able to perform their maintenance, repair, and overhaul responsibilities using LMP; Rock Island Arsenal officials stated that they could perform their manufacturing operations using LMP; and officials at the Army ammunition sites we visited told us they used LMP to manage their inventory. Similarly, officials at the life cycle management commands stated that they were able to carry out their operations, which include the management of spare parts inventory. Additionally, periodic status reports submitted to AMC headquarters show that industrial sites where LMP has been deployed have been able to conduct their repair and manufacturing operations. These reports—which include data on the workload planned and completed for the current fiscal year—are submitted and discussed during regularly-held meetings to provide AMC headquarters with an update on the status of operations. Furthermore, the Army’s fiscal year 2013 and 2014 working capital fund budget documents state that LMP is functional at each of AMC’s life cycle management commands.

We reported in November 2010 that the Army Sustainment Command, the Joint Munitions and Lethality Command, and arsenals under the Tank-automotive and Armaments Command—all of which had been in the final deployment phase of LMP in October 2010—required additional system functionality to perform their unique operations and that this functionality had not yet been delivered. We followed up on each of these previously reported issues for our current review.

The Army Sustainment Command needed (1) an interface between LMP and the Army War Reserve Deployment System, a separate system used to track inventory and transfer accountability of pre- positioned stocks to units and (2) software functionality to conduct mass uploads—the automated movement of thousands of items of inventory between the Army Sustainment Command and the warfighter. During our current review, we found that the Army Sustainment Command had established an interface between LMP and the Army War Reserve Deployment System, and officials stated that LMP now has the capability to perform a mass upload.

The Joint Munitions and Lethality Command required specific functionality to ship, receive, inventory, and perform stock movements for ammunition. To accomplish these tasks, the Joint Munitions and Lethality Command required the development of an interface between LMP and a system commonly referred to as SmartChain. We found during our current review that the interface between LMP and SmartChain has been established, and officials stated that LMP and SmartChain provide the capability needed to perform these tasks.

The Tank-automotive and Armaments Command required specific functionality at its arsenals to automatically report data for items it manufactures. When Rock Island Arsenal and Watervliet Arsenal switched from their legacy system to LMP, they lost this capability because LMP does not include a manufacturing and execution system that provides this capability. We found during our current review that Rock Island Arsenal has since developed its own software tools and manual processes to provide similar functionality. Specifically, Rock Island Arsenal uses three legacy systems, which together cost approximately $300,000 per year to operate. Arsenal officials told us that they developed an additional system to report manufacturing data and track the status of items being manufactured—at a cost of about $400,000. They hired additional personnel to manage these systems, which includes manually entering data into LMP. According to documents provided by Rock Island officials, annual costs for these systems and personnel will average $3.6 million from fiscal years 2013 through 2016. Arsenal and AMC officials stated that they expect Increment 2 to replace systems they are currently using to manage their shop floor operations. (We did not visit Watervliet Arsenal for this review).

Officials at AMC sites we visited stated that over time they are improving in their ability to use LMP, and some locations have developed tools and processes to better extract and analyze LMP data, or to fill in gaps in existing LMP functionality. In addition to the manufacturing tracking tools and processes developed at Rock Island Arsenal, as discussed above, we found the following examples:

Red River Army Depot officials created a tool that enables users to generate customizable reports based on data extracted from LMP. Officials said they use these reports to brief depot leadership on areas such as schedule performance and inventory. Tobyhanna Army Depot created a tool using Microsoft Excel that extracts data from LMP and develops visual reports—in graphs and charts—to analyze depot workload and perform capacity planning. According to Tobyhanna officials, these reports allow them to manage depot operations by, for example, determining whether workload schedules need to be adjusted or personnel reassigned to higher priority tasks.

Corpus Christi Army Depot, using a combination of legacy systems, manual processes, and a locally-developed software tool, gained, among other things, the ability to automatically track the progress of work on the shop floor and record employee’s labor charges. Depot officials stated that the use of this tool, in conjunction with LMP, has provided them with detailed data that helps them to better manage their operations and achieve efficiencies by improving depot processes, such as decreasing the cost to repair UH-60 helicopters (from $7.58 million per helicopter in fiscal year 2011 to $6.82 million per helicopter in fiscal year 2012) while increasing the quantity of UH- 60 helicopters they can repair (from 48 in fiscal year 2011 to 51 in fiscal year 2012).

Additional LMP Development Needed to Meet Certain Critical Army Requirements

The Army is continuing development of LMP because, as currently deployed, it does not provide all of the functionality the Army needs to meet certain critical requirements, according to Army officials. According to the Army’s business case for Increment 2, the new functionality the Army is developing as part of Increment 2 is intended to address critical requirements that have emerged since the initial development of LMP and that are not addressed by Increment 1. The business case stated that the current system does not support certain critical AMC requirements pertaining to shop floor automation, Army business transformation goals (for example, maintaining data on the maintenance status of equipment), and requirements outlined in DOD guidance (for example, requirements related to item unique identification). Requirements for Increment 2 can be categorized into six components: (1) further integration with other Army and Defense Logistics Agency enterprise resource planning systems, (2) controlling and maintaining visibility over material managed by non-Army sources of supply, (3) additional capability to manage Army prepositioned stocks, (4) managing repair operations at Army installations, (5) expanded ammunition management capability, and (6) tracking repair and manufacturing operations on the shop floor. Table 2 summarizes the six components and the time line for their deployment.

In addition to the critical requirements identified in the Army’s business case for Increment 2, the Army has stated that additional development is necessary because LMP, as currently deployed, will not enable the Army to generate auditable financial statements. The National Defense Authorization Act for Fiscal Year 2010 required the development of a plan to ensure that the financial statements of DOD are validated as ready for audit not later than the end of fiscal year 2017. LMP product office and AMC officials told us that the Army cannot meet this requirement with the functionality currently provided by Increment 1, and the Army’s fiscal year 2013 and 2014 working capital fund budget documents state that although LMP is functional at AMC’s life cycle management commands, it requires the enhancements and upgrades of Increment 2 to generate auditable financial statements. In general, DOD considers the successful implementation of enterprise resource planning systems—which include LMP—critical to addressing long-standing weaknesses in financial and supply chain management, and DOD officials have stated that these systems are critical to ensuring that the department meets its requirement to have auditable consolidated financial statements.

According to Army officials, a key LMP improvement in Increment 2 that supports audit readiness is the addition of the shop floor automation tool (part of the Enhanced Industrial Base component listed in table 2). The tool provides a standardized approach for managing shop floor operations across AMC’s depots and arsenals. This tool, according to Army officials, will provide the capability to track the labor and material expenses to specific work tasks and customers, which is a more detailed reporting capability than what is currently provided by Increment 1. Army officials acknowledged that some locations such as Corpus Christi Army Depot have developed local tools to manage shop floor operations but asserted that such tools cannot support the Army’s achievement of audit readiness. They stated that a significant investment of funds would be needed to make locally developed tools auditable, and that doing so would not be economically feasible.

In addition, according to Army officials, it would not be possible to gain all of the capabilities to be delivered with Increment 2 simply by upgrading legacy systems and modifying processes. For example, the Army is currently continuing to use one of the legacy systems that LMP was intended to replace—the Commodity Command Standard System—to control and maintain visibility over material managed by non-Army sources of supply. According to officials from the LMP product office and AMC, this legacy system cannot currently support the Army’s need for auditable financial statements by fiscal year 2017 and thus needs to be replaced. By implementing this component of Increment 2, the Army expects to fully eliminate its need for the Commodity Command Standard System. Another example is related to item unique identification. According to DOD, financial management information necessary for the management of the department’s mission critical assets is also required to support future financial statement audits. This financial management information includes item unique identification, which cannot be accomplished using existing systems or through a completely manual process because, according to Army officials, some unique item identifiers are not human-readable. Instead, Army officials stated that AMC requires an electronic scanner that is planned to be included in the deployment of Increment 2. As a further justification for expanding LMP, the Army estimates that Increment 2 will cost $730 million through fiscal year 2026 but achieve approximately $1.4 billion in financial benefits. Table 3 shows the financial benefits that the Army has projected for Increment 2.

The financial benefits shown in table 3 include both anticipated cost savings and cost avoidance. The Army is anticipating cost savings totaling approximately $1.1 billion from reduced contractor support, overtime, legacy information technology, legacy operations, and AMC industrial operations personnel. According to officials from the LMP product office and AMC, cost savings will be realized as reductions to the Army’s planned budgets starting in fiscal year 2017. Officials cited an AMC memo which states that approximately $30 million of the cost savings from reduced contractor support, overtime, and AMC industrial operations personnel should be realized in fiscal year 2017; $60 million should be realized in fiscal year 2018; and $90 million should be realized in fiscal year 2019 and annually thereafter for the life of LMP. According to the officials, the memo is referring to reductions in planned Army budgets in the specified fiscal years. The approximately $250 million in remaining financial benefits expected from Increment 2 includes cost avoidances from implementing the National Maintenance Program component of Increment 2, providing automatic item unique identification recording capability that will eliminate the need to manually perform those tasks, and reducing excess non-Army managed item inventory.

LMP Has Provided Some Benefits to the Army, but the Extent of Financial Benefits to Date Is Unknown

The use of LMP has provided the Army some benefits, but whether the system has delivered the expected financial benefits to date is unknown because AMC does not have a process to track these benefits. Since its deployment, LMP has provided some benefits to the Army. For example, because LMP relies on accurate data to perform effectively and efficiently, the Army has made data accuracy a priority. The Army has made progress in improving the accuracy of its data by conducting data assessments, correcting data problems, and placing management emphasis on data accuracy. Additionally, the use of LMP has improved accountability for inventory stored at AMC depots and has increased visibility over Army assets. However, the full extent of the financial benefits realized from LMP—which the Army projected would be over $750 million by the end of fiscal year 2012—is unknown because the Army has not tracked the benefits. Federal guidelines and standards outline the need for assessing whether expected benefits from an investment are achieved. To support the development of Increment 2, the Army is developing a performance baseline for sites that will initially deploy Increment 2. Without a process in place to identify and document financial benefits linked to LMP-driven performance improvements, the Army will be unable to track whether it is achieving the expected financial benefits from its sizeable investment.

LMP Has Provided Some Benefits to the Army

Since its deployment, LMP has provided the Army some benefits. During our prior reviews of LMP before the final deployment phase had occurred, we reported that sites using LMP stated it was an improvement over legacy systems, because it increased visibility over assets and provided a single source of data for decision making. During our visits to AMC headquarters and the 14 AMC sites for the current review, officials provided examples of various benefits resulting from using LMP, such as increased visibility over inventory and maintenance operations.

We also found that, since our prior reviews, AMC has made progress in improving the accuracy of LMP data that it uses to support industrial operations by conducting data assessments, correcting data problems, and placing management emphasis on data accuracy. Data accuracy is necessary in order for enterprise resource planning systems such as LMP to perform effectively and efficiently. For example, to support repairs of vehicles, LMP contains data that identify the number of vehicles scheduled for repair at a depot, the number of parts needed to support these repairs, and the capacity of the depot to perform the repairs. Based on these data, LMP will recommend to item managers that they purchase a specific quantity of spare parts by a certain date in order to support the scheduled repairs. In order for these recommendations to be correct, the data they are based on must be accurate. If the data on which they are based are not accurate, item managers may purchase either too many or too few spare parts, or the purchases may not be made in time to support the scheduled repair—which could lead to repairs not being completed according to the planned schedule. According to the AMC Data Integrity Strategy, a data accuracy rate of 95 percent or greater—depending on the type of data—is consistent with the industry standard.

The Army has instituted several processes to enhance data accuracy. A key effort involves periodic assessments at AMC sites to evaluate the accuracy of LMP data. AMC has determined that these assessments, which are conducted by its Logistics Support Activity, should focus on 29 critical data objects—groupings of data based on function—such as bills of material, purchase orders, or a sequence of repair activities. Between August 2010 and November 2012, the Logistics Support Activity performed 361 assessments across AMC that covered 18 of the 29 critical data objects. The results of these assessments are documented in reports, and the root causes of any problems are identified for correction. Logistics Support Activity officials stated that their goal is to focus attention on these root causes so that data problems do not continue to occur. They will then perform a follow-up assessment, if it is deemed necessary, to assess whether data accuracy has improved. For example, in August 2011, the Logistics Support Activity conducted a review of bills of material at various sites across AMC and assessed the accuracy of AMC’s bills of material at 51 percent—well below their 95-percent standard. In a follow-up review conducted in May 2012, the Logistics Support Activity assessed the overall accuracy of AMC’s bills of material at 92 percent. Furthermore, officials from the Logistics Support Activity stated that follow-up assessments at individual AMC sites have revealed improvements in data accuracy. For example, an initial assessment of work center data at the Letterkenny Munitions Center showed data accuracy to be 27 percent, while the follow-up assessment showed data accuracy to be 100 percent. Officials estimated that sites meet the 95- percent standard during initial data assessments only half of the time, but in follow-up assessments they meet the standard about 90 percent of the time.

AMC sites we visited are also performing their own internal reviews of LMP data accuracy. For example, Letterkenny Army Depot has a data quality team that has conducted a number of internal assessments. One of these assessments, conducted in October 2012 on the bill of material for a TOW missile, found that multiple materials were listed on the bill of material as required but had no or very limited consumption. Depot officials told us that, based on these types of assessments, the depot intends to make changes to its data to more accurately reflect usage of materials during repairs. Similarly, in order to assess and improve the data in LMP, officials at Anniston Army Depot mapped out the entire end- to-end repair process for the M9 Armored Combat Earthmover, which enabled them to identify data errors in LMP that they corrected. Officials at Red River Army Depot told us that they prepare weekly reports for depot management, and that data accuracy is discussed during meetings held by the depot commander. Officials also stated that they are emphasizing the importance of maintaining accurate LMP data and are holding users accountable for their management of LMP data. For example, officials at all five Army depots stated that LMP users and their supervisors are expected to ensure that the data they manage are accurate.

The use of LMP has also increased visibility over inventory. AMC officials stated that LMP provides the capability to automatically calculate the parts needed to conduct a repair at a depot. If the depot does not have these parts, LMP recommends that the needed parts be purchased from some other source of supply. In order for this process to work effectively, all inventory at the depot should be recorded in LMP so the system does not recommend ordering parts the depot already has. Officials at AMC sites we visited told us that prior to LMP, parts might be purchased by the Army and stored at the depot but not necessarily recorded in the legacy systems. Accordingly, item managers could make unnecessary purchases of inventory items that were available but not visible within the legacy systems. AMC stated that, due to the inventory reviews following the deployment of LMP and corrections made to inventory records, it entered more than $200 million worth of inventory into LMP that had already been purchased by the Army and was physically located at the depots but was not recorded in the legacy systems. Accordingly, AMC officials expect that officially tracking inventory that had already been purchased could result in a cost avoidance in the future. Similarly, Aviation and Missile Command officials stated that the better visibility of inventory in LMP enabled them to identify excess inventory that was on hand, which resulted in a cost avoidance of $1.2 million due to reduced inventory storage costs.

The use of a single source of data provided by LMP, according to AMC officials, has resulted in other efficiencies. For example, officials at the AMC War Reserve Division stated that they are now able to provide near real-time updates to address questions on the status of war reserves to Army headquarters. These officials stated that, prior to LMP, the process to answer questions from Army headquarters would require 3 weeks to collect and analyze data from the life cycle commands; now, the process can be completed by extracting a report from LMP in minutes. Additionally, AMC officials stated that the single source of data has reduced the number of meetings that need to be held. For example, AMC officials who manage secondary inventory items stated that they are able to track the status of obligations across each of the life cycle management commands by extracting a consolidated report directly from LMP. The officials stated that, prior to LMP, they would hold bi-weekly meetings with the life cycle commands to capture the same information. Additionally, officials at Red River Army Depot stated that LMP provides visibility of inventory at all AMC locations, which has enhanced their ability to redistribute parts to support production needs. These officials stated that, prior to LMP, they had to call other depots to see if additional parts were available; with LMP, they are able to quickly identify which locations have the additional parts.

Finally, AMC officials stated that LMP has enabled them to develop and begin to implement a set of standardized, enterprise-wide performance measures to better assess the business operations of AMC sites. They told us that the measures previously used to assess AMC performance were inadequate, in part because they were not standardized. For example, officials stated that there was no standard measure for the rate of inventory turnover—which measures the number of times an inventory item is used and replaced during a given period—but that AMC was working on developing such a measure. At the time of our review, the standardized performance measures that AMC had developed included one for inventory turnover, as well as others for depot schedule performance, depot cost performance, forecast accuracy, supplier delivery performance, and direct versus indirect labor hours. According to documents provided by AMC, these measures were being used during regularly-held meetings with AMC headquarters beginning in June 2013.

Extent of Financial Benefits to Date from Deploying LMP Is Unknown

The extent to which expected financial benefits have been realized to date from deploying LMP is unknown, because AMC does not yet have a process to track these benefits. Without a process in place to track financial benefits associated with LMP, the Army does not have a way to determine whether LMP’s projected financial benefits are materializing. The Army expected significant financial benefits from the deployment of LMP Increment 1 across AMC, which was completed in October 2010. According to a 2009 study prepared by the Army to support the fiscal year 2010 Investment Review Board certification of LMP, the system was expected to lead to over $750 million in financial benefits by fiscal year 2012 and eventually achieve more than two dollars in benefits for every dollar spent. In its fiscal year 2014 budget documents, the Army projected that LMP would provide an estimated net financial benefit of nearly $1.3 billion through 2020. The Army expected these financial benefits to be achieved largely by LMP-driven improvements to the performance of AMC operations through, for example, reducing inventory, improving productivity, and reducing costs for legacy systems.

Federal guidelines and standards outline the need for assessing whether expected benefits from an investment are achieved. According to the Office of Management and Budget’s Circular A-94, which provides general guidance for benefit-cost analysis of federal programs, an element of benefit-cost analysis is verification of expected benefits. The circular states that retrospective studies to determine whether anticipated benefits and costs have been realized are potentially valuable. Such studies can be used to determine necessary corrections in existing programs and to improve future estimates of benefits and costs in these programs or related ones. Agencies should have a plan for periodic, results-oriented evaluation of program effectiveness. They should also discuss the results of relevant evaluation studies when proposing reauthorizations or increased program funding. The Office of Management and Budget’s Capital Programming Guide also states that a post-implementation review of an information technology project should evaluate an investment’s efficiency and effectiveness to determine how well the investment achieved the planned functionality and anticipated benefits. Additionally, GAO’s Information Technology Investment Management framework states that a critical process for building a foundation for information technology investment success is providing investment oversight. The purpose of this critical process is to ensure that the relevant organization provides effective oversight for its information technology projects throughout all phases of their life cycles. Such oversight should include observing the project’s progress toward expected cost, schedule, and benefits. Furthermore, federal internal control standards state that managers should compare actual performance to planned or expected results throughout the organization and analyze significant differences.

However, the extent to which the Army has realized the expected financial benefits from LMP is unknown because AMC does not yet have a process to identify and document financial benefits realized as a result of performance improvements gained through the use of LMP. We asked AMC headquarters officials for information on the total financial benefits achieved from using LMP, but they were unable to provide such information because they could not quantify the impact LMP had on inventory value or provide measurements of improved productivity. Officials did not provide an explanation for why they did not have a process to track financial benefits, but they stated that the inability to quantify financial benefits from LMP-driven performance improvements was due in part to the fluctuations in AMC workload resulting from operations in Iraq and Afghanistan. Additionally, most AMC sites we visited reported that LMP had not led to financial benefits that they could quantify. Specifically, officials at 12 of the 14 sites we visited stated that LMP either had not resulted in any quantifiable financial benefits or that they could not provide us with any documentation that quantified the financial benefits.

The LMP product office and two AMC sites reported financial benefits from LMP. According to the LMP product office, an estimated $114 million of financial benefits were attributed to legacy system costs that were avoided through fiscal year 2012 as a result of LMP deployment. Additionally, in March 2013, the Aviation and Missile Command reported a $1.2 million cost avoidance due to reduced storage costs from the disposal of excess inventory. Finally, as noted earlier in this report, Corpus Christi Army Depot reported that use of LMP, in conjunction with a locally-developed software tool and manual processes, enabled it to improve depot processes and decrease the costs to repair UH-60 helicopters by approximately $760,000 per helicopter. Because 51 UH-60 helicopters were repaired in fiscal year 2012, AMC officials estimated a total cost savings of approximately $39 million.

LMP product office and AMC officials told us that there was not an accurate process currently in place to track financial benefits associated with LMP-driven performance improvements. Further, the officials stated that retroactively producing an accurate assessment of benefits realized to date from LMP would be difficult, because the Army had not established a baseline for performance prior to the implementation of LMP against which LMP-driven improvements could then be measured. The officials added that the Army has learned this lesson and has incorporated operational performance metrics in its plans for Increment 2. (These operational metrics are separate from the enterprise-wide performance measures discussed earlier in this report that AMC began to use in June 2013.) Specifically, the Army is in the process of developing an initial operational performance baseline for sites that will pilot Increment 2. According to documents provided by the LMP product office, these operational metrics include assessing the time needed to repair a weapon system or component; the direct labor charges needed to support production at industrial activities; program support overtime (i.e., the amount of overtime for indirect labor—such as schedulers, planners, and resource managers); data entry lag time; tracking the labor and materials expended for rework operations; and inventory visibility managed outside of LMP. Officials from the LMP product office and AMC stated that the process to baseline performance against these metrics is ongoing and primarily focused on the Increment 2 pilot sites, with the intention of assessing performance across all of AMC before the next milestone decision for Increment 2 in May 2015. Because the baseline is still under development, the degree to which these operational metrics will enable the Army to demonstrate financial benefits from deploying Increment 2 is unknown.

Conclusions

Over the last decade, the Army has made progress using LMP to support its industrial operations, has improved data accuracy in LMP, and has realized benefits. However, because the Army has not established a process for tracking LMP’s financial benefits, it is not in a position to determine whether it is realizing a return on its sizeable investment in LMP. The Army plans to spend another $1.7 billion on operating the deployed components of LMP Increment 1 over the course of the system’s life cycle, in addition to spending another $730 million on Increment 2. Given the magnitude of the investment already committed and planned to be committed to LMP, oversight by decision makers in DOD and the Congress would likely improve with a better understanding of what financial benefits have been realized from deploying LMP to determine whether the Army’s goals for the system are being met and resources are being used effectively.

Recommendation for Executive Action

To determine whether the Army is achieving its estimated financial benefits in LMP, we recommend that the Secretary of Defense direct the Secretary of the Army to develop and implement a process to track the extent of financial benefits realized from the use of LMP during the remaining course of its life cycle. This process should be linked with the LMP performance baseline now being developed by the Army for use at AMC industrial sites.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. The Army provided written comments, which are reproduced in appendix II. The Army concurred with our recommendation and stated that it will develop a process to track the extent of financial benefits recognized within LMP, which will be linked to the LMP performance baseline. The Army also stated that it is initiating a series of workshops to establish an enduring process to capture the financial benefits realized from the use of LMP. These actions, when implemented, will meet the intent of the recommendation. In addition, the Army provided technical comments, which we have incorporated as appropriate.

We are sending copies of this report to appropriate congressional committees, the Secretary of Defense, and the Secretary of the Army. In addition, this report will be made available at no charge on the GAO website at http://www.gao.gov.

Should you or your staff have any questions concerning this report, please contact me at (202) 512-5257 or merrittz@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To assess the extent to which LMP supports AMC’s industrial operations, we met with officials at AMC headquarters responsible for overseeing LMP to discuss the system’s deployment and usage. We reviewed our prior work related to the deployment of LMP and followed up on issues that we had previously identified. We also obtained and reviewed pertinent documents, including reports submitted to AMC headquarters by the life cycle management commands and subordinate sites on the status of their industrial operations. We met with officials at 14 AMC sites— including all five AMC life cycle management commands and all five AMC maintenance depots—where LMP is deployed. Specifically, we visited the following individual sites:

Army Sustainment Command, Rock Island, Illinois

Aviation and Missile Command, Huntsville, Alabama

Communications-Electronics Command, Aberdeen, Maryland

Joint Munitions and Lethality Command, Rock Island, Illinois

Tank-automotive and Armaments Command, Warren, Michigan

Anniston Army Depot, Anniston, Alabama

Corpus Christi Army Depot, Corpus Christi, Texas

Letterkenny Army Depot, Chambersburg, Pennsylvania

Red River Army Depot, Texarkana, Texas

Tobyhanna Army Depot, Tobyhanna, Pennsylvania

Rock Island Arsenal (Joint Manufacturing and Technology Center),

Anniston Defense Munitions Center, Anniston, Alabama

Letterkenny Munitions Center, Chambersburg, Pennsylvania

McAlester Army Ammunition Plant, McAlester, Oklahoma We judgmentally selected Rock Island Arsenal to visit based on its proximity to the headquarters of Army Sustainment Command and the Joint Munitions and Lethality Command, McAlester Army Ammunition Plant based on the scope of its manufacturing and storage mission, and Anniston and Letterkenny Munitions Centers based on their proximity to Army depots. During these site visits, we interviewed officials and obtained relevant documentation regarding the extent to which they used LMP to conduct their operations, were taking actions to improve the accuracy of the data used in LMP, and had realized nonfinancial and financial benefits from LMP.

We also obtained and analyzed Army documents, including the business case and an accompanying economic analysis, that were developed to support the Army’s proposal to move forward with the development and acquisition of Increment 2. We met with officials from AMC headquarters and the LMP product office to discuss their plans for Increment 2, and we also discussed the Army’s plans for Increment 2 with officials at individual AMC sites.

To determine the extent to which the Army has realized expected benefits from deploying the system, we obtained and reviewed Army documentation describing the expected benefits to be achieved from deploying LMP, including briefings describing the expected benefits and functionality of the system, a 2009 study supporting the fiscal year 2010 Investment Review Board certification of LMP, Army budget documents, and evidence that AMC headquarters, the individual sites, and the LMP Product Office were able to provide regarding actual benefits, if any, achieved to date. We also met with officials at AMC’s Logistics Support Activity to discuss their efforts to improve the accuracy of data used in LMP, and we obtained and reviewed AMC’s strategy for LMP data accuracy as well as documentation on LMP data accuracy assessments performed by the Logistics Support Activity. To assess the reliability of LMP data, we reviewed related documentation on LMP data accuracy and interviewed officials knowledgeable about LMP data. We determined the data were sufficiently reliable for the purposes of our report.

We conducted this performance audit from August 2012 to November 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of the Army

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Thomas Gosling (Assistant Director), Cynthia Grant, Joanne Landesman, Jim Melton, Geoffrey Peck, Amie Steele, and Michael Willems made key contributions to this report.

Related GAO Products

DOD Financial Management: Reported Status of Department of Defense’s Enterprise Resource Planning Systems. GAO-12-565R. Washington, D.C.: March 30, 2012.

Defense Logistics: Oversight and a Coordinated Strategy Needed to Implement the Army Workload and Performance System. GAO-11-566R. Washington, D.C.: July 14, 2011.

Defense Logistics: Additional Oversight and Reporting for the Army Logistics Modernization Program Are Needed. GAO-11-139. Washington, D.C.: November 18, 2010.

DOD Business Transformation: Improved Management Oversight of Business System Modernization Efforts Needed. GAO-11-53. Washington, D.C.: October 7, 2010.

Defense Logistics: Actions Needed to Improve Implementation of the Army Logistics Modernization Program. GAO-10-461. Washington, D.C.: April 30, 2010.

Defense Logistics: Observations on Army’s Implementation of the Logistics Modernization Program. GAO-09-852R. Washington, D.C.: July 8, 2009.

DOD Business Transformation: Lack of an Integrated Strategy Puts the Army’s Asset Visibility System Investments at Risk. GAO-07-860. Washington, D.C.: July 27, 2007.

Army Depot Maintenance: Ineffective Oversight of Depot Maintenance Operations and System Implementation Efforts. GAO-05-441. Washington, D.C.: June 30, 2005.

DOD Business Systems Modernization: Billions Continue to Be Invested with Inadequate Management Oversight and Accountability. GAO-04-615. Washington, D.C.: May 27, 2004.

DOD Competitive Sourcing: Plan Needed to Mitigate Risks in Army Logistics Modernization Program. GAO/NSIAD-00-19. Washington, D.C.: October 4, 1999.