Army Logistics: Report on Manpower and Workload System Inadequate and System Interface Untested
Released: 2002-10-30T00:00:00

Letter

Background

The Army Workload and Performance System (AWPS) is intended to resolve long-standing systemic problems in the Army’s civilian manpower requirements determination process. It is an information and reporting system that draws production and manpower data from other existing programs, including the Army’s Standard Depot System. Its main purpose is to provide decision support tools for linking workload demands to manpower requirements and the budget process. The system was initially installed at Corpus Christi Army Depot, Texas, in June 1996. Since then, it has been put into operation at the Army’s four other maintenance depots—Anniston, Letterkenny, Red River, and Tobyhanna. In 1999, the Assistant Secretary of the Army certified the system as fully operational for the maintenance mission at the five maintenance depots.

The Army is moving forward with the installation of AWPS at all of its logistics and industrial activities. To date the system is being used as a decision-making tool in other functional areas, including ammunition logistics, base operations, materials usage, working capital fund budgets, and reporting of net operating results. The Secretary of the Army has directed that AWPS be used throughout the Army as the standard Armywide mechanism for determining manpower requirements for all of its logistics and industrial activities.

The first AWPS master plan, submitted to Congress in April 1999, described the Army’s progress and future plans for developing and implementing the system. In our November 1999 report regarding that master plan, we pointed out that the information it contained was limited, and we recommended that the Army develop a more substantial master plan that incorporated all applications for which the system was to be implemented, along with their priorities, costs and benefits, and proposed schedules. We also recommended that the Army make improvements in the existing management and oversight structures. The House Report to the National Defense Authorization Act for Fiscal Year 2001 required the Army to submit a revised master plan, incorporating our recommendations, by February 2001. Subsequently, section 346 of the National Defense Authorization Act for Fiscal Year 2002 required the Army to submit an annual progress report on its implementation of the revised master plan. Section 346 also required that these reports specifically address any changes made to the master plan since the previous report.

In December 1999, the Army contracted with the Computer Sciences Corporation to create the Logistics Modernization Program, which is a new information system for managing the Army’s supply, maintenance, and transportation functions. This system, initially called the Wholesale Logistics Modernization Program, will replace the existing Standard Depot System and many other source data systems, several of which provide data to AWPS. The Logistics Modernization Program is designed to improve readiness and logistics support to the war fighter by (1) reducing requisition response times, (2) improving the availability of supplies, (3) optimizing the use of inventory, and (4) responding more quickly to changing customer requirements. The milestones to the first deployment of the Logistics Modernization Program are shown in table 1.

Once the Logistics Modernization Program becomes operational at the maintenance depots, the Army plans to shut down many of the old information systems that currently support AWPS and it will become the primary source for the data that AWPS needs to function. As a result, section 346 of the National Defense Authorization Act for Fiscal Year 2002 encouraged the Army to set up a process that would permit or enhance data sharing between the two systems. To ensure that the Army’s AWPS capabilities remained intact, section 346 also mandated that the Army retain AWPS as its standard servicewide manpower system, under the Secretary of the Army’s supervision and management. This mandate was further underscored in a letter dated August 9, 2001, from several congressional representatives to the Commander of the U.S. Materiel Command, which further requested that the Army refrain from incorporating the new system into the Logistics Modernization Program.

Report Does Not Contain Adequate Information to Assess Progress

The Army’s May 2002 report on its workload and performance system does not contain the information that Congress needs to assess the Army’s progress in implementing the system. In response to the requirement for a progress report, as specified in section 346 of the Fiscal Year 2002 National Defense Authorization Act, the Army submitted an updated version of its May 2001 master plan. This updated version did not identify or explain the changes that the Army had made to the master plan since the May 2001 version. In addition, the Army’s report did not contain certain cost, schedule, and performance information that would normally be expected. Moreover, the Army’s report did not fully discuss the potential duplication and overlap in functions performed by the Logistics Modernization Program and the workload and performance system.

Report Fails to Address Changes in Milestones and Tasks since May 2001

Although required by section 346, the Army’s 2002 report did not address the changes made to the milestones or tasks set out in the May 2001 AWPS master plan. Appendixes II and III provide tables showing the milestones and tasks identified in both the 2001 and 2002 reports. In comparing the two reports, we found that several milestones had been changed, but the 2002 report did not identify these changes nor did it provide a detailed discussion of the reasons for these changes or their significance. For example, in its 2001 report the Army had scheduled Corpus Christi Army Depot as the first site to prototype the Net Operating Result capability, beginning in August 2001. We found, however, that in the 2002 report this task was set back by 1 year—to the fourth quarter of fiscal year 2002. The same task was also scheduled to be prototyped at one of the ammunition sites by March 2002, but this milestone was later delayed by about 1 year until sometime between January and March 2003. In each case, the 2002 report did not provide an analysis or explanation for the scheduling change.

We also found discrepancies between the two reports related to the phasing of certain tasks involved in implementing the new system. Some tasks that were assigned to a specific phase in the 2001 report were moved to a different phase in the 2002 report, and there was no discussion of why these changes were made or what their impact on the overall implementation schedule might be. For example, phase 1 of the 2001 report involved only the consolidation of ongoing implementation actions, whereas in the 2002 report phase 1 also included non-Army Material Command maintenance activities. The 2002 report, however, does not clearly address the status of tasks previously listed under phase 1.

Report Lacks Cost, Schedule, and Performance Data

The Army’s 2002 plan does not contain the cost, schedule, and performance data that might normally be expected. For example, according to the Department of Defense’s (DOD) Regulation 5000.2-R, progress reports related to the acquisition of major new automated information systems should contain detailed information on such key parameters as cost, schedule, and performance. Army officials stated that the scope and cost of the AWPS system does not meet the minimum threshold to be considered a major information system and, thus, the regulation does not apply to it. While we agree that the AWPS system does not meet the threshold requirements of the regulation, we believe certain criteria in the regulation would provide Congress with the necessary information to properly evaluate the AWPS system and should therefore be addressed in the Army’s progress reports. Consequently, we have analyzed the AWPS report using criteria from the regulation. Additionally, the Clinger-Cohen Act of 1996 requires agencies to have investment management processes and information to help ensure that information technology projects are being implemented at an acceptable cost and within a reasonable and expected time frame. In effect, these requirements and guidance recognize that one cannot manage what one cannot measure. Finally, in our November 1999 report on the Army’s original master plan for AWPS, we identified several shortcomings, including the lack of detailed information on costs and expenditures, milestones, and performance. We recommended in that report that the Army develop a more substantive master plan that included priorities, costs and benefits, and schedules.

In our analysis of the Army’s 2002 plan, we found that, while it addresses some of these elements, it does not provide the detailed or complete data that is needed to adequately assess the Army’s progress in implementing the workload and performance system. As table 2 shows, the 2002 plan contained information on a few parameters identified in DOD’s guidance, including direct costs; dates for certain events, such as reaching initial operating capabilities; and objectives for operational requirements. However, it did not include information on a large number of parameters, such as total procurement costs, critical schedule dates, and measures of performance.

In addition, the 2002 report did not contain necessary cost, scheduling, and performance data for the individual tasks that the Army has assigned to each implementation phase. Phase 1, implementation of the workload and performance system at non-Army Materiel Command maintenance depots; phase 2, expansion of the system into nonmaintenance missions (e.g., base operations, medical); and phase 3, development of decision- support tools for use at the major command and headquarters levels (e.g., working capital fund budget, links to depot maintenance operational system, and cross-organizational activities). As table 3 illustrates, the Army’s report contained cost, scheduling, and performance information for only a small number of these tasks. Furthermore, we could only identify specific costs for one of the tasks and, in most cases, the milestones and performance measures were too broad and did not include interim measures and specific performance targets to measure progress.

Report Lacks Detailed Cost Data

While the Army’s May 2002 report provided some estimated funding requirements for AWPS for fiscal years 2004 through 2006, it did not contain the detailed information that could be used to assess the costs of implementing the system thus far and the costs of expanding it into other functional areas in the future. According to the Army Materiel Command, the total estimated costs for the AWPS program were about $44.8 million for fiscal years 1996 through 2002, and the estimated program costs for fiscal year 2003 are about $8.9 million. The primary source for this funding has been the Army’s working capital fund. These figures and the funding sources, however, were not included in the Army’s report. In addition, the Army’s report did not identify the extent to which actual expenditures relate to the budgeted amounts. The report also did not provide any cost estimates for funding the Army’s plan to expand AWPS to other nonmaintenance activities, such as base operations support. According to Army officials, these expansion plans will require funding through the Army’s appropriated operations and maintenance accounts.

In its report, the Army estimated that it would need about $20.1 million over the next 3 fiscal years (2004 through 2006), to ensure that the remaining tasks are implemented. Table 4 shows the Army’s projected costs for fiscal years 2004 through 2006, which were included in its May 2002 report. According to the report, these future year costs are unfunded and the Army has not yet identified funding sources for them. These officials stated that, other than the funding that has been provided through the working capital fund, the department has not adequately funded the AWPS expansion effort in recent years and that this lack of funding has hampered their ability to plan and implement further expansions.

As table 4 indicates, the Army did not provide a detailed cost analysis regarding the historical and projected costs for AWPS, nor did it provide a complete summary of the estimated costs to complete the tasks listed for each phase. Specifically, the table includes cost estimates for additional system implementation (phase 1) and for the development of decision support tools (phase 3), but it provides no specific estimates for expanding the system into other functional areas (phase 2). Additionally, the Army did not include the associated costs to support the development of all the specific tasks required to complete each phase.

Report Contains Limited Milestone Data

The Army’s May 2002 report contained only limited information on the milestones established to implement the new system and no data on whether earlier milestones had been reached, thereby making it difficult to assess the progress of the system’s development and implementation. Specifically, the report lacked schedules that include implementation and completion dates and interim milestones. For example, the Army is updating the Workload and Performance System applications from the original programming language to a more up-to-date programming language. According to the Army, this upgrade has been installed at all five maintenance depots and will be installed at other installations between May 2002 and May 2003. However, specific dates for implementing or completing this upgrade were not included in the May 2002 report. In another example, the Army indicates that it intends to install AWPS at other nonmaintenance activities outside the Army Materiel Command, but it does not provide specific milestones for each location or the specific tasks associated with the development and installation process. As shown in appendix III, the Army has established expected completion dates for some of the AWPS applications, but the completion dates for other long- term applications have not yet been set.

The Army’s May 2002 report also did not provide milestones for completing the interface between AWPS and the Logistics Modernization Program. Instead, it simply stated that between May 2002 and February 2003 the system has to accept, and operate with, data from the Logistics Modernization Program. The original date (July 2001) set to operationalize the interface at the first site, the Tobyhanna Army Depot, had changed by about 18 months. In addition, the report noted that the Operations Support Command is scheduled to transition to the Logistics Modernization Program 1 year after the Communications and Electronics Command, which is approximately January 2004. This date is about 2 years beyond the original date of October 2000.

Report Lacks Adequate Performance Assessments

The Army’s May 2002 report does not address in detail the extent to which AWPS is providing the Army with the capability to match manpower requirements and workload for which it was initially intended. While the report states that the implementation of AWPS in several mission areas within the Army Materiel Command has shown that the system can efficiently draw data from other existing systems and manipulate this information to link personnel needs with projected workloads, the Army has not demonstrated that AWPS has improved its ability to support its long-term forecasting of civilian personnel requirements based on projected workload. Because the Army did not provide supporting evidence for the statement in its May 2002 report that the system has led to increased operational efficiencies, the extent of the improvements is unclear. We did not independently review the effectiveness of the AWPS system at the depots we visited.

Report Does Not Address Potential Overlap with Logistics Modernization Program

The Army’s report also fails to discuss the potential overlap and duplication that exists between AWPS and the Logistics Modernization Program. Although these two systems were designed to serve different functions, Army and contractor officials point out that there is some potential overlap and redundancy in the systems’ capabilities. For example, the capability of the performance measurement and control module in the AWPS software also exists in the Logistics Modernization Program software configuration. This module allows the user to compare actual resource expenditures against production plans, scheduled workload, and related budgets for specific projects in order to determine the likelihood of completing a project within its estimated time frame and budget. In addition, because the Logistics Modernization Program is not complete, the Army cannot be certain what other capabilities may be duplicated. Army officials at the Tobyhanna Army Depot expressed concerns that the need to operate and maintain both systems could lead to higher costs and duplication of efforts.

A second module in AWPS, however, the strategic planning and forecasting module, is unique to AWPS and does not currently exist within the software configuration for the Logistics Modernization Program. This module provides the user with the capability to forecast manpower and capacity requirements based on future projected workload. More specifically, this module allows the Army the ability to conduct “what if” analyses for manpower and capacity requirements based on future workload projections at each of its maintenance activities. Contractor officials stated that although this capability could be built into the Logistics Modernization Program, it would have to be modified to be compatible with the current software configuration. By incorporating this capability into the Logistics Modernization Program, the Army could eliminate the need to operate and maintain two separate systems. Computer Sciences Corporation submitted a formal proposal to the Army in August 2001 to incorporate all of the capabilities of AWPS into the Logistics Modernization Program for an estimated contract price increase of about $2 million. Contractor officials told us in May 2002, however, that because of the amount of work they have dedicated to building the interface between the two systems, this cost estimate is no longer valid.

Army Has Not Sufficiently Tested the Interface between AWPS and the Logistics Modernization Program

Although the Army has begun developing an interface between AWPS and the Logistics Modernization Program, it has not sufficiently tested the interface to ensure that data can be shared between the two systems and that the AWPS capability will not be adversely affected. Once the Logistics Modernization Program is implemented, the Army plans to shut down several systems, including the Standard Depot System, that currently provide data for AWPS. However, the Army has not demonstrated that the Logistics Modernization Program databases will be able to supply AWPS with the data that it needs to continue to function. Until the Army has placed the interface in operation at several sites, it will be too early to assess its effectiveness.

The Army’s contract with the Computer Sciences Corporation to develop and field the Logistics Modernization Program required that the contractor would create an interface between the two systems, and this work started in 1999. In February 2002, Army and contractor officials developed an interface control document that identified the data elements that AWPS would need from the Logistics Modernization Program databases to maintain its current capabilities. Since that time, contractor personnel have been working to locate the sources within the Logistics Modernization Program databases for each data element and determine the most expedient way to move that data into AWPS. According to Army and contractor officials, about 90 percent of the data elements had been located by May 2002.

While initial testing of the interface began in August 2002, it will be tested at only one of the five Army depots by February 2003 when the Logistics Modernization Program is scheduled to come on line. Specifically, the Army will be testing the interface at Tobyhanna Army Depot between August 2002 and February 2003, and expects that the interface will be fully functional by the time the Logistics Modernization Program is deployed at the depot in February 2003. Subsequently, the Army plans to install the Logistics Modernization Program and the AWPS interface at the four remaining Army maintenance depots, along with the Army’s ammunition maintenance facilities. According to the May 2002 report, the Army expects to shut down the current information systems that support AWPS at the same time as it turns on the Logistics Modernization Program. As a result, there will be no transition period during which the current information systems and the Logistics Modernization Program are in operation at the same time.

Conclusions

The Army’s May 2002 report to Congress on the development and implementation of AWPS has a number of significant limitations. The report does not contain key information regarding the changes to the program since the submission of the May 2001 master plan, and it does not provide adequate information on the costs, schedule, and performance of the system. As a result, the report is of limited use to Congress in evaluating whether the AWPS project is still in line with its original cost, schedule, and performance objectives. The Army has not demonstrated to Congress how well the system has helped it thus far to determine future civilian workload requirements based on projected workloads. Moreover, the report does not contain the information that Congress needs to determine how much funding will be required to complete the initial implementation of the system and expand it into other functional areas.

AWPS provides the Army with a capability for strategic planning and forecasting at its maintenance facilities that currently does not exist within the Logistics Modernization Program. The interface that is being developed between the two systems is intended to allow the workload and performance system to maintain its current capabilities, including its strategic planning and forecasting module. Because each system offers the Army certain unique capabilities, a rationale for operating both systems at the same time exists. However, because the two systems may develop some overlap and redundant capabilities in the future, there is some potential for increased costs or other inefficiencies.

Recommendations for Executive Action

In order to improve the quality of the Army’s annual progress reports to Congress on the implementation of AWPS and to enhance the efficiency and effectiveness of the system, we recommend that the Secretary of Defense direct the Secretary of the Army to: submit to Congress annual progress reports on the implementation of AWPS that contain a complete description of any changes to the master plan since the submission of the previous report and a detailed explanation of the status of the AWPS program in relation to the costs, milestones, and performance data contained in the previous report; ensure that these progress reports contain detailed cost, schedule, and performance information to allow Congress to fully assess the status of the Army’s implementation of the workload and performance system and its interface with the Logistics Modernization Program, and the extent to which the system is providing the Army with the capability to match manpower and workload requirements; undertake a review of the interface between AWPS and the Logistics Modernization Program, once it has been successfully installed at the Army’s five maintenance depots, to ensure that it is the most efficient and cost-effective use of these two systems; and ensure that the data-sharing mechanisms between the Logistics Modernization Program and AWPS are complete and allow for full functionality of AWPS before turning off the information systems that currently support AWPS.

Agency Comments and Our Evaluation

The Department of Defense fully concurred with our finding and recommendations. In response to our recommendation that the Army ensure that future progress reports contain cost, schedule, and performance information as specified in relevant Defense regulations and other congressional guidance, DOD will implement the recommendation in its February 2003 report. However, DOD noted that the workload and performance system is not a major automated information system and, therefore, is not required to strictly adhere to the requirements of Department of Defense Regulation 5000.2-R. We agree that the workload and performance system does not meet the minimum threshold to be considered a major system. However, we believe that the parameters outlined in this regulation provide an appropriate management framework for the types of information that should be included in future progress reports.

DOD also informally provided other suggested revisions to address certain technical and factual information in the text of the draft report. We reviewed these suggested revisions and made changes where appropriate.

We are sending copies of this report to interested congressional committees, the Secretaries of Defense and the Army, and the Director, Office of Management and Budget. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Appendix I: Comments from the Department of Defense

Appendix II: Task List for 2001 and 2002

May 2001 Plan Short-Term Army Workload and Performance System modules

Workload and workforce have been combined as the strategic planning and forecasting module.

This phase is not included in May 2002 plan. First phase of the plan is to cover non-Army Materiel Command maintenance activities.

Base operations at ammunition logistics activities is not included.

This is the first phase in May 2002 plan. Second phase of the plan is expansion into nonmaintenance missions.

This phase is categorized into three parts.

Does not include any information on where mission indirect will be installed.

Does not include any information on the status of the Next generation-AWPS.

Does not include any information on the status of the Base operations-Next generation. .

Appendix III: Milestone Schedule

Army Workload and Performance System modules

No date provided No date provided No date provided No date provided No date provided Installed at maintenance depots and will be installed at other sites over the course of the year.

Completion February 2003 at Tobyhanna and other commodity commands at about 3-month intervals.

As of October 2001, AWPS has been operational at all five maintenance depots, ammunition logistics, ammunition manufacturing (Crane and McAlester), and base operations at all maintenance depots.

Appendix IV: Scope and Methodology

To determine whether the Army’s May 2002 master plan contains adequate information to assess the Army’s progress in implementing AWPS, we reviewed the Army’s May 2001 and May 2002 master plans. We compared the contents of these plans to the key requirements set forth in section 346 of the National Defense Authorization Act for Fiscal Year 2002. In addition, we reviewed the May 2002 master plan to determine the extent to which it addressed the recommendations outlined in our November 1999 report.We also examined the Department of Defense’s regulation outlining the mandatory procedures for the acquisition of major automated information systems to determine specific criteria required for a progress report. We compared the contents of the May 2002 master plan to the criteria outlined in this regulation. Although this regulation does not specifically apply to the development of the AWPS system, we believe that sound management practices support the need to address these parameters in the Army’s progress reports. We also met with officials at the Headquarters, Department of the Army; Headquarters, Army Material Command; and the Operations Support Command in Rock Island, Illinois, to discuss the development and implementation of the AWPS system. In addition, we discussed the benefits and problems that the depots have experienced with AWPS with officials at Tobyhanna Army Depot, Tobyhanna, Pennsylvania; and Corpus Christi Army Depot, Corpus Christi, Texas. We did not, however, independently review the effectiveness of the AWPS system at the depots we visited. Lastly, we relied on prior work done in connection with the implementation of AWPS.

To identify the measures the Army has taken to ensure appropriate coordination and data sharing between AWPS and the Logistics Modernization Program, we reviewed the February 2002 Interface Control Document developed jointly by the Department of the Army and the Computer Sciences Corporation, and discussed the related interface initiatives with appropriate Army and contractor officials. We also reviewed the actions the Army had taken to facilitate the interface and data sharing between the two systems to identify what additional actions were needed before the Army could be assured that the AWPS system would remain fully operational during the transition period. Specifically, we met with officials at the Headquarters, Department of the Army; Headquarters, Army Materiel Command; the Army’s Operations Support Command in Rock Island, Illinois; the Logistics Modernization Project Office in Moorestown, New Jersey; and Tobyhanna Army Depot and Corpus Christi Army Depot. Because the interface between the two systems is still being developed and has not been fully tested, we were unable to assess its effectiveness.

We conducted our review between March 2002 and August 2002 in accordance with generally accepted government auditing standards.