Defense Logistics: Actions Needed to Improve Implementation of the Army Logistics Modernization Program
Published: 2010-04-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Logistics Modernization Program (LMP) is an Army business system that is intended to replace the aging Army systems that manage inventory and depot repair operations. Through 2009, the Army obligated more than $1 billion for LMP. LMP was originally scheduled to be completed by 2005, but after the first deployment in July 2003, the Army delayed fielding because of significant problems. The Army has since decided to field the system in two additional deployments: the second deployment occurred in May 2009, and the third deployment is scheduled to occur in October 2010. GAO was asked to evaluate the effectiveness of the Army's management processes in enabling the second deployment sites to realize the full benefits of LMP.
section_title: What GAO Found
paragraphs: The Army's management processes that were established prior to the second deployment of LMP were not effective in enabling the second deployment sites to realize the full benefits of LMP. When LMP becomes fully operational at the second deployment locations, the Army expects that it will significantly enhance depot operations. However, the Army was unable to ensure that the data used by LMP were of sufficient quality to enable the depots to perform their day-to-day missions after LMP became operational. As a result of these data quality issues, depot personnel had to develop and use manual work-around processes until they could correct the data in LMP, which prevented the Army from achieving the expected benefits from LMP. Data quality issues occurred despite improvements made by the Army to address data issues experienced during the first deployment of LMP because the Army's testing strategy did not provide reasonable assurance that the data being used by LMP were accurate and reliable. Instead, the Army's testing efforts focused on whether the software was functioning, but did not assess whether the data used by the depots to perform their repair missions were of sufficient quality to work in LMP. According to depot officials, the data problems are being corrected as they are identified. Additionally, the Army's training strategy did not effectively provide LMP users the skills necessary to perform their new tasks. Users at the depots stated that the training they received did not provide a realistic environment that showed them how to perform their expected duties, and did not always match their new responsibilities. However, users at the depots also stated that they had received additional training that resolved the issue. The Army also lacked a comprehensive set of metrics with which to measure the success of LMP implementation. GAO's previous work has shown that successful performance measures should be aligned throughout the organization and cover the activities that an entity is expected to perform. However, the Army did not have common metrics with which to measure success during the second deployment, and the Army's scorecard for measuring LMP implementation focused on the software, but did not assess whether the depots were able to perform their work using LMP as envisioned. Despite these challenges, LMP has provided the Army some benefits, and officials at the second deployment sites provided examples of how LMP had improved their day-to-day operations, for example, through the increased visibility of assets. The third deployment of LMP is scheduled to occur in October 2010, and will involve more commands, occur at locations across the globe, and affect more users than the previous deployments. LMP program management officials stated that they are taking steps to address the issues discussed in this report for the third deployment and are adjusting plans related to data testing and training. However, because these plans are being developed, GAO was unable to verify that the problems have been resolved. Without correcting these issues prior to the third deployment, the Army is likely to face similar, or potentially greater, problems that prevent it from realizing the full benefits of LMP.

section_title: Letter
paragraphs: For decades, the Department of Defense (DOD) has been challenged in  modernizing its business systems.  In 1995, we designated DOD’s business  system modernization program as high risk, and continue to do so because  these systems are fundamental to addressing long-standing weaknesses  related to the management of contracts, finance, and the supply chain.  Moreover, we continue to report on business system investments that fail  to effectively deliver promised benefits and capabilities on time and within  budget.
Organizations that implement enterprise resource planning systems have  faced substantial challenges because of the complexity of the endeavor,  and the Army is no exception. In 1998, the Army initiated the Logistics  Modernization Program (LMP) in order to replace two aging Army systems  used to manage its inventory and its repair operations at the depots. LMP  was originally scheduled to be completed by 2005, but the Army delayed  fielding LMP because of the significant problems faced by the first  deployment sites—the Communications-Electronics Command and  Tobyhanna Army Depot—in July 2003, which we detailed in several  reports. The Army has since determined that fielding of LMP would occur  in two additional phases of deployment: the Aviation and Missile  Command in 2009, which includes Corpus Christi Army Depot and  Letterkenny Army Depot, and the Tank-automotive and Armaments  Command, the Joint Munitions and Lethality Command, and the Army  Sustainment Command in October 2010. Through 2009, the Army has  obligated more than $1 billion to implement LMP, and estimates a total life  cycle cost in excess of $2.6 billion to procure and operate the system. The  Army expects that LMP will reduce redundant and stove piped information  technology investments and assist in driving business transformation  across the Army, which will enable the Army to supply and service the  warfighter more quickly and cost effectively.
We previously assessed the Army’s preparation for the second deployment  of LMP and reported that the Army had begun to implement our prior  recommendations related to past issues on data conversion, billing and  collection, requirements management and testing, and independent  verification and validation. We also reported that the Army had  implemented critical project management processes and controls that  enabled the Army to identify and understand the risks associated with  making a deployment decision. We further noted that these critical  management processes—which addressed data conversion from legacy  systems to LMP, training LMP users, and the ability to rapidly evaluate and  respond to potential issues—were absent during the first deployment of  LMP at Tobyhanna Army Depot. However, the effectiveness of the Army’s  management processes could not be evaluated until the second  deployment had occurred on May 14, 2009.
You asked us to continue monitoring the Army’s efforts to deploy LMP and  evaluate the Army’s progress in addressing the issues that are critical to  successful implementation. You also asked us to monitor the actions taken  by the Army after the system has been deployed to ensure that its stated  processes are adequate and have been effectively implemented.  Accordingly, the objective of this review was to evaluate the effectiveness  of the Army’s management processes in enabling the second deployment  sites to realize the full benefits of LMP.
To address this objective, we reviewed and analyzed the Army plans and  policies that governed LMP implementation. We obtained briefings from  the LMP program management office on the intended purpose of LMP, as  well as information related to the execution of the second deployment. We  also monitored the second deployment of LMP as it occurred and had  personnel observing operations at Corpus Christi Army Depot, Texas;  Letterkenny Army Depot, Pennsylvania; and the LMP program  management office’s command center in Marlton, New Jersey. We met  with officials at each location to discuss how they were using LMP to  perform their missions and attended and observed daily meetings held by  each location and among the locations where they discussed issues that  had arisen and the actions they were taking to resolve the issues. After the  initial deployment, we conducted follow-up visits to Corpus Christi Army  Depot and Letterkenny Army Depot to monitor the progress of  implementation. We also met with officials at the Army Materiel  Command, the Aviation and Missile Command, and the LMP program  management office to discuss how the Army was managing the  implementation and using the system. Lastly, we met with LMP program  management officials to discuss plans for the third deployment of LMP  scheduled to occur in October 2010.
We conducted this performance audit from May 2009 through March 2010  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: In February 1998, the Army Materiel Command began an effort to replace  its existing materiel management systems—the Commodity Command  Standard System and the Standard Depot System—with LMP. The Army  has used these existing systems for over 30 years to manage inventory and  depot maintenance operations. LMP is intended to transform the Army’s  logistics operations in six core processes: order fulfillment, demand and  supply planning, procurement, asset management, materiel maintenance,  and financial management. If effectively implemented, LMP is expected to  provide the Army benefits associated with commercial best practices, such  as inventory reduction, improved repair cycle time, and increased  response time. Additionally, LMP is intended to improve supply and  demand forecast planning and maintenance workload planning and to  provide a single source of data for decision making.
LMP became operational at the Army Communications-Electronics  Command and Tobyhanna Army Depot in July 2003, and was originally  expected to be fully deployed by fiscal year 2005. However, because of  problems experienced at Tobyhanna Army Depot, the Army delayed  implementation until the operational problems were resolved. In May  2009, LMP became operational at the Army Aviation and Missile Command  and Corpus Christi and Letterkenny Army Depots, and the third and final  deployment of LMP is expected to occur in October 2010 at the Army  Sustainment Command, the Joint Munitions and Lethality Command, the  Tank-automotive and Armaments Command, and Anniston and Red River  Army Depots. When LMP is fully implemented, it is expected to include  approximately 21,000 users at 104 locations across the globe (see table 1),  and it will be used to manage more than $40 billion worth of goods and  services, such as inventory managed at the national level and repairs at  depot facilities.
According to LMP program management officials, preparations for the  third deployment of LMP began in December 2008. In September 2009,  leaders from the third deployment sites formally established the Executive  Steering Committee to ensure a successful implementation of LMP during  the third deployment. The committee’s responsibilities include reviewing  plans and schedules, exchanging lessons learned, performing ongoing  assessments of readiness to implement LMP, and making decisions on  important issues. As of January 2010, LMP program management officials  told us that several education courses had been completed and that the  third deployment sites were in the process of completing their initial data  conversion activities.
section_title: Army’s Management Processes Not Effective in Enabling Second Deployment Sites to Achieve LMP Benefits
paragraphs: The Army’s management processes that were established prior to the  second deployment of LMP were not effective in enabling the second  deployment sites to realize the full benefits of LMP. Based on our  observations, we found that data quality issues prevented personnel at  Corpus Christ and Letterkenny Army Depots from using LMP as  envisioned. The Army acknowledged that data quality was one of the most  important and challenging success factors in deriving the optimal business  benefits from LMP. At both depots, LMP users were unable to use LMP to  conduct depot operations and inventory management functions  immediately following the second deployment. Although the depots were  able to continue to repair items and support the warfighter, LMP users had  to rely on manual work-around processes, which are not part of how LMP  is intended to function and hinder the Army’s ability to realize the benefits  expected from LMP. The depots experienced data quality issues, despite  improvements the Army made to address data quality issues experienced  during the first deployment of LMP at Tobyhanna Army Depot, because  the Army’s testing strategy did not provide reasonable assurance that the  data being used by LMP were accurate and reliable. Specifically, the  Army’s testing strategy focused on determining whether the software  worked as designed, but did not assess whether LMP was capable of  functioning in a depot environment using the actual data from the depots.  Additionally, the Army’s training strategy did not effectively provide users  the skills necessary to perform all of their tasks in LMP. Users at the  depots stated that the training they received before LMP became  operational was not conducted in a realistic environment that showed  them how to perform their expected duties. However, users at both depots  also stated that the LMP program management office had provided  additional training after LMP became operational to address their  concerns. Additionally, when monitoring LMP implementation at the  second deployment sites, the Army lacked a comprehensive set of metrics  with which to accurately assess whether LMP was delivering the intended  functionality. Instead, the metrics used by the LMP program management  office focused on whether the software was working but did not measure  whether the deployment sites were performing their day-to-day operations  using LMP as envisioned. Despite these challenges, the second deployment  sites reported achieving some benefits through the use of LMP, such as  increased visibility over assets.
subsections: section_title: Data Quality Issues Prevented Depots from Realizing the Full Benefits of LMP
paragraphs: The Army was unable to realize the full benefits of LMP at the second  deployment sites because of data quality issues. The benefits that LMP was  expected to provide the depots included reducing the amount of time to  repair items, automatically calculating the material requirements for repair  projects, and improving the management of maintenance capacity.  However, the ability of the second deployment sites to realize these  benefits depended on data quality, which according to the Army, is one of  the most important and challenging success factors in deriving optimal  business benefits from an enterprise resource planning system like LMP.
In preparation for the second deployment, the Army focused on  addressing systemic data quality problems associated with the first  deployment of LMP at Tobyhanna Army Depot. As we reported in June  2005, LMP did not always contain the correct unit price or unit of issue  for certain materials, resulting in excess material being ordered and  incorrect prices being charged to jobs. To avoid similar problems during  the second deployment of LMP, the Army developed processes to ensure  that the unit of issue and unit of measure values shown for a given item  were appropriate for that item. According to LMP program management  office officials, out of a target population of more than 12 million data  records, 99.9 percent were loaded into LMP (12.40 million out of 12.41  million data records) before the system became operational at the second  deployment sites. Although the Army’s processes addressed some of the  data issues from the first deployment of LMP, according to LMP program  management office officials, these processes did not assess whether the  overall quality of the data that the system would use was sufficient to  support the LMP processes.
Both Corpus Christi and Letterkenny Army Depots experienced data  quality issues that prevented the Army from realizing the full benefits of  LMP. For example, during our visit to Corpus Christi Army Depot shortly  after LMP became operational in May 2009, we observed that personnel at  the depot could not use LMP to induct a helicopter for repair because of  data quality issues. As a result, LMP could not automatically identify the  materials needed to support the repair and ensure that parts would be  available in time to support the repairs. Furthermore, labor rates were also  missing for some stages of repair, thereby precluding LMP from computing  the expected labor costs for a repair project. Users at Corpus Christi Army  Depot addressed the data quality problems as they arose. However,  because of these data quality problems, some of the enterprise processes  that LMP can perform had to be conducted using manual work-around  processes to ensure that the depot accomplished its repair mission and  thereby continued to meet the needs of the warfighter. Furthermore, these  manual work-around processes are labor intensive and prevent the Army  from achieving the benefits that LMP is expected to provide, such as  increasing the efficiency in ordering parts, determining whether sufficient  funds are available to perform the expected work, and determining  whether the production schedule could be achieved with existing  resources.
Letterkenny Army Depot also experienced issues with data quality. For  example, when we visited Letterkenny Army Depot shortly after LMP  became operational, depot officials told us that they had identified data  quality errors when they attempted to induct a ground vehicle for the  Patriot system for repair. Letterkenny Army Depot officials also stated that  prior to LMP becoming operational, they spent about 18 months rebuilding  the data that were needed to perform repair and supply functions in LMP.  However, after LMP became operational, they realized that the data were  not of sufficient quality to enable the depot to use LMP as envisioned, so  Letterkenny Army Depot officials took steps to correct the data. During  our subsequent visit to Letterkenny Army Depot in August 2009, depot  officials stated that they were continuing to refine the data, and that they  had nearly completed doing so for their most common repair item—the  High Mobility Multi-Purpose Wheeled Vehicle. This refinement would then  serve as a template for the remaining repair items. Officials at Letterkenny  Army Depot noted, however, that they had pre-positioned materials prior  to the transition to LMP to ensure that the depot could perform its repair  mission if difficulties arose, which enabled the depot to perform its day-to- day operations despite the data shortcomings. Nonetheless, officials at  Letterkenny Army Depot acknowledged that they were not using the  preferred automated LMP processes as intended, and needed to use  manual work-around processes to complete their tasks.
section_title: Army’s Testing Strategy Did Not Evaluate Data Quality
paragraphs: The Army’s testing strategy, as illustrated by the problems experienced at  the second deployment locations, was not comprehensive enough to  provide reasonable assurance that the data being used by LMP were  accurate and reliable. As we have previously reported, testing is a critical  process utilized by organizations with the intent of finding errors before a  system is implemented. According to the Army’s Master Test Plan for LMP  implementation, the objective of system testing was to ensure that LMP  operated as intended. However, the Army’s testing efforts focused on  determining whether the software worked as designed. Army officials  stated that Army testing efforts for the second deployment of LMP had  incorporated lessons learned from the first deployment of LMP. With  respect to testing data conversion, the Army noted two improvements in  its testing process: the use of data-specific scenario testing and engaging  users in critical business process tests. According to the Army’s test plan,  data-specific scenario tests were designed to assess the quality, validity,  and integrity of the data to be migrated into LMP, as well as to validate  that data migration processes function as designed. Additionally, critical  business process tests, which were performed by users, involved the  execution of business process-oriented scenarios using the data loaded  into LMP with the intent of assessing the functional readiness of the  software.
In order to assess the functional readiness of the software, the Army used  simulated test data to test the system. For example, when assessing the  functional readiness of the software to perform an induction of an item for  repair, Army officials told us that they did not attempt to induct an item  for repair using the data loaded into LMP. Instead, the Army tested  whether LMP could perform an induction, and performed these tests using  simulated data so that developers would know whether LMP could  provide the intended capability. While this approach is useful and  desirable to determine whether the software can operate as expected, it  does not assess whether the data are of sufficient quality to work in LMP.  Thus, the first attempt to perform a process in LMP using actual data  during the second deployment occurred when the depots attempted to use  the system. Consequently, the Army’s testing strategy did not detect  problems with the quality of the data at the deployment sites.
During lessons learned sessions held in June 2009, Army officials at the  LMP program management office and at the second deployment sites  acknowledged these weaknesses in their testing strategy. Army officials  we interviewed also stated that testing needed to address whether the  deployment sites could perform their work using the system as intended.  For example, officials at Corpus Christi Army Depot agreed that a better  strategy to test whether LMP could perform as intended would be to  induct a representative number of items at each depot using actual data  from each deployment site. By attempting to induct several items into the  depot repair processes that are representative of the majority of their  workloads prior to transitioning to LMP, the Army would have likely  detected problems related to data quality, which could have provided  reasonable assurance that LMP could operate as intended if the tests were  successful. Furthermore, by incorporating the use of actual data into its  testing strategy, the Army could have obtained reasonable assurance that  the data loaded into LMP were of sufficient quality to be used and  increased user confidence in the system. For example, officials at  Letterkenny Army Depot told us that testing the system in a simulated  environment with actual data would not only assess whether the system  would work but also would gauge the effectiveness of training. Had the  Army’s testing strategy used the actual data at the deployment locations,  the Army could have identified issues related to data quality prior to the  transition to LMP.
section_title: Army’s Training Strategy Did Not Fully Meet Needs of LMP Users
paragraphs: Although the Army’s training strategy was designed to provide LMP users  the skills and knowledge to successfully perform their new roles, LMP  users we interviewed at Corpus Christi and Letterkenny Army Depots  stated that the training they received prior to LMP becoming operational  did not fully meet their needs. LMP users we interviewed at Corpus Christi  and Letterkenny Army Depots stated that the training focused on what  LMP was supposed to do rather than on how they were to use the system  to perform their day-to-day missions. Additionally, because the duties of  some LMP users at the depots were changing, the training users received  was not always commensurate with the responsibilities they were  assigned. Consequently, some LMP users told us that they did not always  understand the actions they needed to perform in order to accomplish  their assigned tasks. Furthermore, since the timeline for implementing the  various LMP processes differed at each location, the training sometimes  occurred weeks before it could be actually used, which resulted in users  needing additional training after the system became operational. Despite  these issues, LMP users at both depots stated that the additional training  they received after LMP became operational was effective and addressed  their needs.
According to the Army’s LMP End-User Training and Development  Delivery Plan dated January 30, 2009, most implementation failures are  caused by poor end user training, which is the transfer of knowledge to  the end users who will run the enterprise with the new solution. To  prevent poor end user training, the Army’s plan stated that end user  training must provide users not only with the transactions and tasks  performed in LMP, but also with the ability to recognize the underlying  flow of information through LMP. To meet this goal, the Army used a  blended learning solution designed to provide LMP users with the skills,  process knowledge, and performance support to successfully perform  their new roles after the transition to LMP. The Army’s plan also stated  that training should occur as close as possible to the date of  implementation because the user’s ability to retain information diminishes  each day the user is not able to put into practice the training he or she has  received.
The Army experienced challenges with the quality and timing of its  education and training efforts. According to the Army, education focused  on LMP concepts, while training demonstrated how the system should be  utilized by the user. However, according to Army lessons learned  documents, users received education on LMP concepts too far in advance  of training, which limited the ability of LMP users to understand how the  changes in the business processes that were to occur as a result of LMP  implementation affected their job responsibilities. Additionally, LMP users  we met at the depots questioned the overall quality of the training they  received. For example, these users stated that the instruction focused too  much on concepts, rather than providing them the skills necessary to  perform their day-to-day operations. Users also noted that they had limited  opportunities to enhance their knowledge of the system by actually using  the system in a training environment.
The Army was also unable to deliver the correct training to some users  because of challenges in properly assigning roles. The Army’s plan for  implementing LMP required not only the adoption of a new technological  solution, but also required changes to the duty descriptions for some LMP  users. For example, prior to LMP implementation, production controllers  at Letterkenny Army Depot were responsible for tracking the flow of a  repair item; however, after LMP implementation, production controllers  became asset managers, who were responsible not only for tracking the  flow of the item through the repair process but also for ordering parts.  Additionally, supervisors were generally responsible for assigning roles to  their personnel, and based on these roles, users would receive training.  However, as noted in Army lessons learned documents, the process for  determining roles sometimes occurred before supervisors received any  notable LMP education. As a result, some users did not receive the correct  training.
The Army also faced challenges in its ability to deliver training as close as  possible to the date of implementation because of the decentralized  execution strategy at each deployment site. In order to prepare users, the  Army used a master training calendar and standard training curriculum.  Although this training was standardized across the second deployment  sites, execution of LMP implementation was decentralized. For example,  after LMP implementation occurred on May 14, 2009, Letterkenny Army  Depot consolidated all of its production controllers into one area and  began performing LMP functions in a phased approach. In contrast,  Corpus Christi Army Depot executed LMP implementation by attempting  to perform all tasks shortly after the transition. Although each approach  has merit, the approach used by Letterkenny Army Depot did not match  the Army’s approach to training. That is, LMP users at Letterkenny Army  Depot only performed some of the LMP processes immediately after LMP  became operational. Accordingly, some of the users at Letterkenny Army  Depot had to receive refresher training before they could perform their  assigned duties.
Despite the concerns that LMP users expressed about training, LMP users  also stated that the Army was able to successfully provide refresher  training and ad hoc coursework to address their issues. For example, LMP  users we met with at both Corpus Christi and Letterkenny Army Depots  stated that these courses were effective because the content was focused  and specific. Officials at both depots also stated that the contractor  support personnel that the LMP program management office provided to  assist each depot were also effective in providing information.
section_title: Army Lacked Comprehensive Metrics to Assess LMP Implementation
paragraphs: The Army was unable to determine whether the second deployment sites  had achieved the envisioned functionality of LMP because the Army  lacked a comprehensive set of metrics to measure the success of LMP  implementation. Our previous work has shown that successful  performance measures should be aligned throughout the organization and  cover the activities that an entity is expected to perform to support the  intent of the program. However, based on our review of the second  deployment of LMP, we determined that the Army did not develop a  comprehensive set of metrics. As a result, the scorecards that the LMP  program management office used to measure the progress of LMP  implementation did not measure whether the deployment sites were  performing their day-to-day operations using LMP as envisioned.
The LMP program management office assessed the progress of LMP  implementation using a scorecard that was agreed to by the deployment  sites and the LMP program management office. This scorecard focused on  four elements: (1) validating user access, (2) critical business processes  validated by sites, (3) production support and infrastructure readiness,  and (4) training readiness. Table 2 provides an explanation of each area.
As noted above, in addition to the criteria for each element, the LMP  program management office’s measurements included a legend that  provided detail on the color coding used to assess progress. According to  this scorecard, a “green” rating was assigned if the element was “on track,”  a “yellow” rating if the element had “issues being worked,” a “red” rating if  the element had “significant problems/issues,” and a “white” rating if the  element had not been started. Of the elements on the scorecard, the only  category that assessed whether LMP could be used as intended was  critical business process validation. However, as stated in the critical  business process validation reports submitted by the Aviation and Missile  Command, Corpus Christi Army Depot, and Letterkenny Army Depot, this  validation allowed the use of manual work-around processes that were not  part of the envisioned LMP processes. Accordingly, the Army did not have  an assessment in place to determine whether LMP was delivering the  envisioned capability to the second deployment sites.
Based on our observations at Corpus Christi and Letterkenny Army  Depots, we found that the elements on the LMP program management  office’s scorecard did not accurately reflect the activities that the depots  were expected to perform. One of the primary purposes of implementing  LMP was to gain the capability and efficiencies through automated  processes associated with the software. Accordingly, while the scorecard  measured the functionality of the software after LMP became operational,  the scorecard did not assess whether the depots were able to perform  their work using LMP as envisioned.
In addition to its scorecard, the LMP program management office provided  periodic briefings regarding LMP implementation that identified problems  as “What’s Important Now” items. The briefings described each problem,  the impact of the problem, steps being taken to mitigate the problem, and  an estimated date for when the problem would be resolved. However,  despite the presence of these items, the LMP program management  office’s scorecard reflected the status of LMP implementation as “green.”  For example, a May 28, 2009, briefing that was provided to senior Army  management contained 17 “What’s Important Now” items that identified  problems related to missing data, the ability of the depots to fill customer  requisitions, the ability to correct data in LMP, and challenges related to  issuing materials to the shop floor to support repairs. Additionally, in the  same briefing, the LMP program management office reported that Corpus  Christi Army Depot had inducted an aircraft for repair using LMP. In  actuality, as discussed earlier, the aircraft was inducted by depot  personnel using manual work-around processes that were similar to legacy  processes rather than the envisioned LMP processes. Although the depot  was unable to induct a helicopter using LMP as intended, the LMP  program management office’s scorecard reflected a “green” rating for all of  the elements.
We also found that the LMP program management office’s scorecard did  not accurately reflect the internal assessment of LMP implementation at  the depots. For example, Letterkenny Army Depot developed and used a  scorecard to measure progress of LMP implementation, which included  more than 50 processes that end users had to perform in LMP covering  areas such as supply, maintenance, and finance. According to officials at  Letterkenny Army Depot, a process was identified as “green” once the user  had successfully performed the task in LMP using the envisioned  processes. However, the progress as tracked by the depot did not match  the progress as reported on the LMP program management office’s  scorecard. For example, on May 26, 2009, Letterkenny Army Depot had  identified 48 of its processes as “red” because the depot either had not yet  performed the function in LMP or was unable to perform it successfully in  LMP using the envisioned processes. However, on the same day, the LMP  program management office reported that LMP was “green” in all elements  measured by the LMP program management office’s scorecard. These  differences reflect the lack of a comprehensive set of metrics for  measuring the success of LMP implementation, because while the LMP  program management office was measuring whether the software was  working, Letterkenny Army Depot had identified that it was unable to  conduct its daily operations using LMP as envisioned.
section_title: LMP Has Provided the Army Some Benefits
paragraphs: Although data quality and training issues prevented the second  deployment sites from using the full capabilities of LMP as envisioned, the  use of LMP at the second deployment sites has provided the Army some  benefits that were not available in legacy systems, such as increased  visibility. For example, officials at Corpus Christi Army Depot stated that  LMP has provided them the ability to track and trace individual  transactions to specific end users. With this tool, officials at Corpus Christi  Army Depot stated that they are able to research individual actions, as  well as ensure that individuals are following the procedures at the depot.  Corpus Christi Army Depot officials also stated that LMP provided them  increased visibility over contractor-managed inventories, which was not  available in the legacy systems.
The Army has also achieved benefits through the common picture  provided by LMP. For example, when explaining how the life cycle  management commands were using LMP, an item manager from the  Aviation and Missile Command showed us how the common view  provided by LMP improved communication with the depots. When  attempting to find the location of an item for repair, the item manager  stated that both the depot and the item manager saw that the item to be  repaired had already arrived at the depot, so the depot could then begin  the repair process. According to the item manager, this visibility was not  available in the legacy systems, and the lack of a common picture  sometimes delayed induction of items for repair. The item manager stated  that the delays occurred because the item manager’s system showed that  an item was located at a depot, but the depot’s system did not show the  item as received, so personnel at the depot had to locate the item before it  could be inducted for repair. Furthermore, the item manager  demonstrated how the common view provided by LMP helped locate a  critical part for the Patriot missile system of a deployed unit. In this case,  the item manager was able to identify where the part was stored in order  to support the deployed unit. The item manager stated that this capability  was not available in legacy systems.
section_title: Preparations for Third Deployment of LMP Are Under Way
paragraphs: The Army has taken steps to address some of the concerns we identified  during the second deployment of LMP. LMP program management officials  told us that preparations for the third deployment of LMP had begun in  December 2008, but that the Army had adjusted its plans—specifically in  the areas of testing and training—based on lessons learned from the  second deployment. For example, LMP program management officials told  us that the third deployment sites had already begun data preparation  activities in early 2009, and that these activities would continue through  September 2010. Additionally, LMP program management officials stated  that they had developed changes to their testing strategy and that tests are  scheduled to begin in May 2010. LMP program management officials also  stated that they intended to conduct selective testing using actual depot- specific data and that this testing is scheduled to begin in August 2010.  However, according to LMP program management officials, the exact  processes to be tested and the materials associated with supporting the  tests have yet to be developed.
With respect to training, LMP program management officials told us that  the training for the third deployment will be conducted by a cadre of  personnel from each of the major deployment sites, and that members of  this cadre would both serve as instructors to users at the locations as well  as assist sites with implementing LMP. LMP program management officials  stated that they were in the process of completing instruction for the cadre  of trainers, and that LMP users had already begun receiving instruction on  the LMP process. During a meeting with DOD and Army officials, a  representative from one of the sites preparing for the third deployment  stated that while the instructors had changed from contractors to the  cadre of trainers, LMP users were still being taught with the same training  materials from the second deployment. Additionally, according to the  Army’s schedule for the third deployment of LMP, the Army plans to  update training for users in June 2010 and begin delivery of training to  users in July 2010. Accordingly, because these events have yet to occur,  we were unable to determine whether they adequately addressed the  issues we identified during the second deployment related to data testing  and training.
section_title: Conclusion
paragraphs: Implementation of enterprise resource planning systems, like LMP, is a  complex endeavor, and based on the second LMP deployment, the Army  has improved its ability to manage implementation. Because the Army  improved its management processes based on lessons learned during the  first deployment of LMP at Tobyhanna Army Depot, the Army was  successful in mitigating some of the previous issues. Furthermore, the  second deployment of LMP also demonstrated the potential to provide  benefits for the Army, such as providing better visibility of inventory and  asset management. However, despite the improvements and the benefits  that were achieved, the deployment sites still faced challenges related to  data quality and training, which limited their ability to use LMP as  intended. Additionally, the Army’s lack of a comprehensive set of  performance metrics prevented it from measuring whether the intended  LMP functionality had been achieved at the depots. Although the Army  was effective in addressing these challenges after they had arisen at the  second deployment sites, applying these lessons learned to the Army’s  plans as it prepares to support the third phase of deployment in October  2010 is critical. Unless the Army addresses these challenges, the third  deployment locations are likely to face the same, or even greater,  problems, since the third deployment of LMP will occur at more locations  and affect more users than the previous deployments.
section_title: Recommendations for Executive Action
paragraphs: To improve the third deployment of LMP, we are recommending that the  Secretary of the Army direct the Commanding General, Army Materiel  Command, to take the following three actions:  Improve testing activities to obtain reasonable assurance that the data  used by LMP can support the LMP processes.  Improve training for LMP users by    providing training on actual job processes in a manner that allows  the users to understand how the new processes support their job  responsibilities and the types of work they are expected to perform  and    providing training at the individual deployment sites to match  deployment timelines.
Establish performance metrics that will enable the Army to assess  whether the deployment sites are able to use LMP as intended.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD stated that the Army  concurred with our recommendations and highlighted the corrective  actions it is taking to (1) improve testing activities to obtain reasonable  assurance that the data used by LMP can support the LMP processes,   (2) improve training for LMP users, and (3) establish performance metrics  that will enable the Army to assess whether the deployment sites are able  to use LMP as intended. Regarding our first recommendation, the Army  commented that the third deployment of LMP involves two new test  activities—the Process and Data Integration Test and the Business  Operational Test—that are designed to address lessons learned from the  second deployment of LMP. The Army commented that the Process and  Data Integration Test will evaluate business processes using migrated  business data from critical weapon systems, and that the Business  Operational Test will require expert and select end users to perform  transactions in LMP using local data. According to the Army, these tests  will bring together data, business processes, standard operating  procedures, and end user training materials to ensure success. While we  have not evaluated the effectiveness or sufficiency of these two new tests  in correcting the data testing issues we discuss in this report, we believe  that the steps the Army is taking appropriately address the intent of our  recommendation. With respect to our second recommendation, the Army  commented that it began role mapping and development of the training  calendar earlier in the deployment process, and that training will be  delivered to end users in a just-in-time method to ensure that the training  is timely and focused to meet the needs of the users. The Army also  commented that it will continue to provide refresher training after  deployment. We have not reviewed these new training initiatives, but we  believe that they are a step in the right direction toward addressing the  intent of our recommendation and improving LMP user training. Finally,  regarding our third recommendation, the Army commented that it is  working to improve standard performance measures, and that the metrics  will reflect lessons learned from previous LMP deployments. The Army  commented that the expected date of completion for development of these  measures is July 1, 2010. The Army’s written comments are reprinted in  appendix II.
We are sending copies of this report to interested congressional  committees; the Secretary of Defense; the Secretary of the Army; and the  Director, Office of Management and Budget. The report also is available at  no charge on the GAO Web site at http://www.gao.gov.
Please contact William M. Solis at (202) 512-8365 or solisw@gao.gov,   Asif A. Khan at (202) 512-9869 or khana@gao.gov, or Nabajyoti Barkakati  at (202) 512-4499 or barkakatin@gao.gov if you or your staff have  questions on matters discussed in this report. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this report. Key contributors to this report were J. Chris  Martin, Senior-Level Technologist; David Schmitt, Assistant Director;  Evelyn Logue; Darby Smith; and Jim Melton.
section_title: Appendix I: Scope and Methodology
paragraphs: In order to evaluate the effectiveness of the Army’s management processes  in enabling the second deployment sites to realize the full benefits of the  Logistics Modernization Program (LMP), we reviewed and analyzed Army  plans and policies that governed LMP implementation. Specifically, we  reviewed plans created by the LMP program management office related to  data conversion and migration, system testing, the training curriculum,  and how the Army intended to monitor the implementation of LMP. We  also held several meetings with LMP program management office officials  and received briefings related to LMP implementation. During these  briefings, we also received information on how LMP is intended to  function, as well as the benefits that the Army expects to receive by using  the system.
We also monitored the second deployment of LMP as it occurred and had  personnel observing operations at Corpus Christi Army Depot, Texas;  Letterkenny Army Depot, Pennsylvania; and the LMP program  management office’s command center at Marlton, New Jersey. During our  initial visits to Corpus Christi Army Depot and Letterkenny Army Depot,  we met with depot officials to discuss how they were using LMP and any  problems or successes that had arisen from their usage. We also observed  how personnel at the depots were performing certain processes using LMP  and received documents related to those actions. Additionally, we  reviewed how each of the depots was assessing the progress of LMP  implementation, and attended several of the daily internal meetings held at  Corpus Christi Army Depot and Letterkenny Army Depot. To assess how  the Army was managing the overall progress of implementation, we also  attended daily meetings held by the LMP program management office, the  Aviation and Missile Command, Corpus Christi Army Depot, and  Letterkenny Army Depot.
After the initial deployment, we attended a lessons learned discussion  hosted by the LMP program management office. We conducted follow-up  visits to Letterkenny Army Depot in June 2009 and August 2009 and to  Corpus Christi Army Depot in September 2009 to monitor the progress of  implementation. We also met with officials at the Army Materiel  Command, the Aviation and Missile Command, and the LMP program  management office to discuss how the Army was managing the  implementation, and received documents that were used to inform senior  leaders at the Army Materiel Command on the status of LMP  implementation. We also met with the Army Deputy Chief of Staff for  Logistics in January 2010 to discuss our initial observations.
During meetings we held with officials in the Army Materiel Command and  the LMP program management office, we discussed the steps that the  Army was taking to support the third phase of deployment for LMP. We  received copies of Army briefings assessing the progress of  implementation, as well as revisions to Army plans and drafts of new plans  based on the lessons learned from the second deployment of LMP.  Because of the timing of this review, we did not assess the Army’s plans  for the third deployment locations, nor were we able to observe actions  taken by the third deployment locations to prepare for LMP  implementation.
We conducted this performance audit from May 2009 through March 2010  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense