Background

The Army began developing AWPS in 1996 at the direction of the House  National Security Committee (now known as the House Armed Services  Committee), and started fielding the system to its maintenance depots  that year. AWPS was not intended to directly replace any single existing  system; rather it was aimed at resolving deficiencies in the Army’s civilian  manpower requirements determination process by providing analytically  based workload forecasts to support workforce requirements. In 1998,  the committee further directed the Army to develop a master plan for  implementing AWPS. We subsequently reviewed the master plan and  recommended, among other things, that the Army examine its existing  program management and oversight structure as it continued to  implement AWPS. The Department of Defense (DOD) concurred with  the recommendations and cited corrective actions it would take, including  having the Secretary of the Army assess the adequacy of existing  program management and oversight structures responsible for system  development and implementation. In June 2001, the Army sent an  updated master plan to the House and Senate Committees on Armed  Services.
Section 346 of the National Defense Authorization Act for Fiscal Year  2002 encouraged the sharing of data between AWPS and LMP (which  had been in development since 1999) and identified several requirements  related to AWPS. Among other things, the law required: the Army to  continue to use AWPS as a standard Army-wide manpower system under  the supervision and management of the Secretary of the Army; the  Secretary of the Army to submit annual progress reports to Congress on  the implementation of the AWPS master plan until the Secretary certifies  to Congress that AWPS is fully implemented; and GAO to submit  evaluations of each of the Army’s annual reports not later than 60 days  after the Army has issued them to Congress. We evaluated the Army’s  May 2002 progress report to Congress and recommended several actions  to improve the quality of the Army’s annual progress reports and enhance  the efficiency and effectiveness of AWPS. DOD concurred with our  recommendations and stated that it would take steps to implement them.  For example, the Army said it was in the final stages of updating the  AWPS master plan and that it would provide it to Congress by November  2007. However, the Army ultimately did not submit an updated master  plan. In our most recent report on AWPS in July 2011, we found that the  Army was using AWPS to generate reports related to managing its  workload and workforce issues but that the accuracy of those reports  varied. We also found that the Army’s oversight of AWPS was  fragmented among several organizations and that the Army had not been  submitting annual reports on the implementation of the 2002 AWPS  master plan. Accordingly, we recommended that an organization be  identified for overall oversight of AWPS and that annual reports on AWPS  implementation be submitted to Congress, as required by law. The Army  concurred with these recommendations and stated that it would update  the master plan and report to Congress by October 2011. We discuss  later in this report the status of the Army’s efforts to address this  recommendation.
AWPS has several modules that enable users to generate products that  include the workload and resources report, which compares forecast  workload against the available workforce; performance management and  control tables, which assess the cost and schedule performance for  production programs; and a report used by AMC leadership to monitor the  status of efforts to repair and recapitalize equipment returning from  overseas operations. To do so, AWPS pulls LMP data on labor costs as  well as the personnel time required to complete repairs, among other  things. Officials from Army G-4 (Maintenance) also use AWPS reports to  assess whether the Army is maintaining a core depot-level maintenance  and repair capability. As of October 2013, approximately 1,400  personnel had access to the system at 15 AMC industrial sites.
The Army spent approximately $1.4 billion on LMP through fiscal year  2012, and its total life cycle cost from fiscal years 2000 to 2026 is more  than $4 billion. According to the AWPS program management office, the  Army has spent over $90 million on AWPS from its inception in 1996  through the end of fiscal year 2013. Table 1 shows the Army’s reported  actual and projected AWPS costs from fiscal year 2011—when we issued  our last report on AWPS—through fiscal year 2018. The costs include  government and contractor personnel expenses and hardware and  software upgrades.

	The Army Has Not Followed Certain Statutory and Regulatory Requirements for Implementing AWPS

		The Army Has Not Submitted Annual Progress Reports to Congress on AWPS Implementation

The Army has not submitted the required annual progress reports to  Congress on the implementation of the AWPS master plan since 2002.  Section 346 of the National Defense Authorization Act for Fiscal Year  2002 requires the Secretary of the Army to submit annual progress  reports to Congress on the implementation of the AWPS master plan until  the Secretary certifies to Congress that AWPS is fully implemented. The  Army submitted its first AWPS progress report to Congress in 2002 in the  form of a revised version of the AWPS master plan. Although the  Secretary of the Army has not yet certified to Congress that AWPS is fully  implemented, the Army has not submitted any further annual reports.  Because we recommended in July 2011 that the Army submit annual  reports on the implementation of the AWPS master plan, as required by  law, we are not making a new recommendation on this issue in this  report.
The Army has not been submitting annual reports because, according to  Army officials, it does not plan to submit progress reports until the revised  master plan is completed. Army officials stated that AMC, in coordination  with other Army offices, has been drafting an update to the AWPS master  plan since July 2011—when GAO’s last report on AWPS was issued—in  order to reflect changes regarding AWPS status and future use. However,  it has not yet been completed and submitted to Congress, and officials  were uncertain when that would occur. The AWPS master plan is still  important, as AWPS has not yet been certified as fully implemented.  According to officials, the Army has not completed the AWPS master plan  due to oversight challenges arising from a series of internal  reorganizations and multiple transfers of responsibility for AWPS within  the Army. Another contributing factor, according to AMC officials, was the  loss of personnel with knowledge of AWPS that has occurred since the  AWPS master plan was last updated in 2002. In response to one of our  July 2011 recommendations, the Army identified the Office of the  Assistant Secretary of the Army (Manpower and Reserve Affairs) as the  Army organization with overall oversight of AWPS. However, we did not  find evidence that this office has had an active role in overseeing the  completion of the AWPS master plan. Internal control standards identify  the need for oversight, including responsibilities such as setting and  monitoring objectives, as important. In the absence of oversight,  completion of the AWPS master plan could be further delayed, resulting  in continued lack of submission of annual progress reports on AWPS  implementation to Congress.

		The Army Has Not Validated AWPS as a Manpower Requirements Determination Tool

The Army has not validated AWPS as a manpower requirements  determination tool as required by regulation. In October 2001, the  Secretary of the Army designated AWPS as an Army-wide decision  support, manpower determination, and management tool for applicable  workload and workforce in the Army. According to Army regulation,  USAMAA is responsible for reviewing and validating manpower  requirements models. Army Regulation 570-4, which establishes policies  and procedures for manpower management, states that USAMAA is  responsible for reviewing and recommending approval of manpower  requirements models developed by major commands. Army Regulation  71-32, which establishes policy for force development and  documentation, states that the Assistant Secretary of the Army  (Manpower and Reserve Affairs), to which USAMAA provides assistance,  is responsible for validating manpower models. Such reviews are to  occur at least every 3 years, or more often as needed, and are part of  USAMAA’s responsibility to ensure consistent application of Army  manpower policies. Army regulation also states that major commands— such as AMC—are responsible for submitting manpower models to  USAMAA for approval, along with background information to support  assumptions and equations in existing and proposed models. According  to USAMAA officials, AWPS has not been validated as a manpower  requirements determination tool in accordance with Army regulation, and  therefore they cannot be assured that AWPS reports on workforce  requirements are accurate.
USAMAA officials stated that they have previously asked AMC whether it  would submit AWPS for review and validation, but AMC has not yet done  so. USAMAA officials told us they contacted AMC in late 2011 or early  2012 regarding submitting AWPS for validation. At the time, AMC was not  prepared to move forward with validation. However, officials from the  AWPS program management office told us that in April 2012 they asked  for assistance from USAMAA regarding submitting AWPS for validation,  but that additional assistance had not yet been provided. Because  USAMAA officials involved in these earlier contacts subsequently left  USAMAA, we were unable to corroborate that USAMAA did not respond  to AWPS program management officials’ requests for assistance. A  senior official within the Office of the Assistant Secretary of the Army  (Manpower and Reserve Affairs) told us that USAMAA had provided  assistance to the AWPS program management office but that AWPS had  not been formally submitted for review and validation.
USAMAA has published written guidance on how to develop and validate  a manpower modeling tool. This guidance is designed for developing new  tools and is not tailored for submission of an existing tool, such as AWPS,  for validation. For example, the primary validation process identified in the  guidance begins at the outset of the model-development process, which  is not applicable for an existing tool. However, USAMAA officials provided  us with an explanation of what would be necessary to validate AWPS as  a manpower requirements determination tool and indicated that they work  collaboratively with Army organizations to support the model-validation  process. The process would include a briefing from AMC that  demonstrates the AWPS manpower model. USAMAA officials could also  ask AMC for further information on the model. USAMAA would then test  the model to ensure the accuracy of what was presented by AMC. If any  issues were to arise during testing, USAMAA would work with AMC to  address them before validating the AWPS manpower model. Officials  from the AWPS program management office stated that although  USAMAA has not validated AWPS as a manpower requirements  determination tool, AWPS has undergone prior reviews by other entities,  to include an assessment of the data interface between AWPS and LMP.  As of September 2013, the draft of the updated AWPS master plan states  that the Army plans to submit AWPS to USAMAA for validation in the next  1 to 4 years, and AMC officials also acknowledged the need to have  AWPS validated by USAMAA.
Although AWPS has not been validated, Army officials stated that AWPS  reports continue to be used for workforce planning, including the  development of budget estimates for future workload and workforce  needs. For example, AMC industrial sites are using AWPS forecasting  reports to make decisions on how to size their workforce to the sites’  projected workload. We previously reported that the accuracy of reports  generated from AWPS varied. Through USAMAA validation of AWPS,  the Army would have greater assurance that reports of these types,  linking workload demand to workforce requirements, are accurate.

	The Army Has Begun to Assess Unnecessary Overlap between AWPS and LMP, but Progress Has Been Limited

The Army has begun to assess whether unnecessary overlap exists  between AWPS and LMP and has begun its assessment by initiating  data-collection and analysis efforts. However, its overall progress— including the development of a business case analysis—has been limited,  due primarily to the absence of senior-level leadership involvement and  attention and the lack of a fully developed and documented approach for  conducting the assessment, including an established milestone for  completing it. GAO has found that overlap among government programs  or activities can lead to unnecessary duplication and can result in  unnecessary costs and less-efficient and less-effective services.  Eliminating unnecessary overlap is particularly important given the current  budgetary situation that the federal government and DOD face.
In December 2012, the AMC Commanding General and Deputy  Commanding General directed that a plan be developed to “sunset,” or  retire, AWPS and transition its functionality to another system, such as  LMP. AMC officials stated that this task was assigned by AMC  leadership as a result of feedback provided by industrial sites indicating  that overlap existed between LMP and AWPS and questioning whether it  was necessary for sites to continue to pay operating expenses for both  AWPS and LMP. In response, a project team was established within  AMC and started meeting in February 2013 to assess unnecessary  overlap between AWPS and LMP. As part of this effort, the team has  begun to evaluate the extent to which software functionality currently  provided by AWPS can be provided by LMP—either through LMP’s  existing functionality or through the expansion of functionality planned for  LMP Increment 2. AMC officials stated that LMP Increment 2 will include  a software module called Business Objects that will be able to produce  reports similar to some of those produced by AWPS—such as charts  outlining cost performance and schedule performance for projects at AMC  industrial sites. Team members stated they will develop a business case  analysis that will likely recommend one of three courses of action for  providing AWPS functionality: retire AWPS completely and provide  functionality with another system, such as LMP; provide functionality  partially with AWPS and partially with another system, such as LMP; or  continue to provide functionality solely with AWPS.
The project team has begun developing its assessment by initiating data- collection and analysis efforts. Specific actions to complete the  assessment have been identified, such as surveying AWPS users,  holding a design workshop to determine what AWPS functionality can be  provided by LMP or another tool, and developing the business case  analysis mentioned previously. The team has received a demonstration  on the current functionality provided by AWPS and LMP and has  distributed surveys to Army personnel with AWPS accounts to collect  information on how they use the system. Among other things, the survey  asked whether personnel used AWPS, for what purposes, and how often.  The survey also asked whether they used another tool—such as LMP—to  provide similar functionality. The team received survey responses from  several AMC sites, including consolidated responses signed by site  leadership (e.g., depot commander) and individual responses from AWPS  account holders. At the time of our assessment, consolidated responses  were provided by 7 of 15 AMC sites—two life cycle management  commands, three maintenance depots, and two manufacturing arsenals.
Based on our review of the consolidated responses to the project team’s  survey and our previous work, there is potential for some unnecessary  overlap between AWPS and LMP, but further analysis is necessary. The  survey lists several AWPS modules and asks users to report the time  spent using each module, as well as the time spent using other tools to  perform similar functions. All seven consolidated survey responses  reported that AMC sites use some AWPS modules, but they also all  reported using other tools—including LMP—to provide similar  functionality. However, the project team has not assessed the reason  these sites use multiple tools to perform similar functions and therefore it  has not yet determined the extent of unnecessary overlap that may exist.  For example, Anniston Army Depot and Rock Island Arsenal responded  that they use reports available in LMP to assess cost performance and  schedule performance for projects at their sites. Other sites responded  that they use locally developed tools that extract data from LMP and  produce reports similar to those in AWPS. For example, Red River Army  Depot responded that it has developed a depot information visualization  system that extracts data from LMP and links planned workload to  workforce requirements. In addition, Tobyhanna Army Depot is using a  tool in Microsoft Excel that extracts data from LMP and develops  management reports similar to those created by AWPS. According to  Tobyhanna’s survey response, this tool allows the depot to compare  planned workload against its available workforce. Officials from the  AWPS program management office stated that they plan to analyze the  survey results and follow up with survey responders to obtain additional  information. Furthermore, as previously noted, AMC initiated its  assessment as a result of feedback provided by industrial sites indicating  that overlap existed between LMP and AWPS.
Although the project team has made progress on the user survey, its  progress in other areas of the assessment has been more limited. The  team’s charter—which it began drafting in February 2013 to guide its  operations—has not been completed. Team meetings have not been held  regularly, and Army officials stated that attendance at these meetings is  not always a high priority for team members. The AWPS program  management office first provided a demonstration of AWPS functionality  to the team in March 2013, but the LMP product office did not provide a  similar demonstration of LMP functionality until December 2013.  Milestones for some key actions—such as developing a business case for  how to most cost-effectively provide AWPS functionality—have not been  established, while milestones set for other actions have been missed. For  example, the team originally planned to complete its analysis of survey  responses from AWPS users by August 2013. However, as of December  2013, it had not completed this analysis.
Progress on the Army’s assessment of AWPS has been limited due to the  absence of senior-level leadership attention and involvement. Best  practices have shown that sustained leadership attention and involvement  can help organizations make lasting changes and achieve positive  results. In addition, these best practices indicate that a strong and stable  team responsible for managing change is also essential to ensure that the  effort receives the attention it needs to be sustained and successful.  Having a strong, stable team for this assessment is particularly important  because any changes made to AWPS and LMP could potentially affect  numerous functions and activities across AMC. The project team is  composed of representatives from various offices within AMC. The team  leader works in the AMC G-1 (Personnel) office and has provided G-1  management with updates on the team’s progress. The other team  members, however, do not work for G-1 and therefore do not officially  report to the team leader or G-1 management; team members cited this  leadership structure as a factor contributing to the team’s limited  progress. Project team members stated that other than a June 2013  progress update to an AMC committee overseeing the implementation of  LMP, no senior-level AMC manager or committee has requested updates  on the team’s progress in assessing unnecessary overlap between  AWPS and LMP and developing a plan to potentially retire AWPS.
Additionally, the team’s progress has been limited due to the lack of a  fully developed and documented approach, including an established  milestone for completing the assessment and recommending a course of  action for providing AWPS functionality. When the task was originally  assigned in December 2012, the AMC Commanding General and Deputy  Commanding General did not establish a milestone for when the project  team should complete the assessment, and the project team itself has not  yet established a completion date. Standards for Internal Control call for  proper documentation of evaluation processes. In addition, best  practices have shown that a well-planned schedule—with milestones for  completing activities—is a fundamental management tool that can help  government programs use public funds effectively by specifying when  work will be performed in the future and measuring program performance  against an approved plan. The absence of senior-level AMC leadership  involvement and attention, as well as the lack of a fully developed and  documented approach that includes an established milestone for  completion, increases the risk that the Army will not make sustained  progress in its efforts to identify and eliminate unnecessary overlap  between AWPS and LMP.

	Conclusions

The Army has experienced delays in completing an update to the AWPS  master plan. In the absence of strong oversight, completion of the master  plan could be further delayed, resulting in continued lack of submission of  statutorily required annual progress reports to Congress on the system’s  implementation. In addition, AWPS was developed to address  deficiencies in the Army’s manpower requirements determination  process, but the Army has not validated AWPS as a manpower  requirements determination tool in accordance with regulation and  therefore lacks assurance that AWPS is producing valid results for  workforce planning. Further, GAO has found that eliminating unnecessary  overlap among government programs or activities can help prevent  unnecessary duplication and help agencies provide more-efficient and  more-effective services. Eliminating unnecessary overlap is particularly  important given the current budgetary situation that the federal  government and DOD face. The Army’s current effort to assess  unnecessary overlap between AWPS and LMP could facilitate its ability to  make a sound business decision on how to most cost-effectively provide  AWPS functionality—which could include retiring AWPS and transitioning  its functionality to LMP. Timely and effective actions to complete the  assessment could improve the Army’s ability to support industrial  operations in the most-efficient and cost-effective manner by minimizing  unnecessary overlap between these two information systems. In the  absence of increased leadership attention and involvement and without a  fully developed and documented approach for this assessment, including  an established milestone for completion, the Army could continue— unnecessarily—to spend millions of dollars annually on AWPS.

	Recommendations for Executive Action

We are recommending that the Secretary of the Army take four actions.
To ensure completion of the update to the AWPS master plan, we  recommend that the Secretary of the Army direct the Assistant Secretary  of the Army (Manpower and Reserve Affairs) to strengthen oversight of  the plan’s development, review, and approval.
To provide greater assurance of the accuracy of manpower requirements  reports produced by AWPS for use at Army industrial sites, we  recommend that the Secretary of the Army direct AMC—with assistance  as needed from USAMAA—to submit AWPS to USAMAA for review and  validation as a manpower requirements determination tool, in accordance  with Army regulations.
To complete the Army’s assessment of unnecessary overlap between  AWPS and LMP and to ensure that a sound business decision is made  on how to most cost-effectively provide AWPS functionality, we  recommend that the Secretary of the Army direct the Commanding  General, AMC, to take the following two actions:  Identify a specific senior-level AMC manager or committee to provide  increased leadership involvement and attention of the project team’s  efforts, to include ensuring that a strong and stable team exists for  managing change.
Establish a fully developed and documented approach for the team’s  assessment, including a milestone for completing it.

	Agency Comments

We requested comments from the Army, but none were provided.
We are sending copies of this report to appropriate congressional  committees, the Secretary of Defense, and the Secretary of the Army. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-5257 or merrittz@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix I.

Appendix I: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the contact named above, Thomas Gosling (Assistant  Director), Joanne Landesman, Brian Mazanec, Alexis Olson, Geoffrey  Peck, and Sabrina Streagle made key contributions to this report.

Summary:
Why GAO Did This Study

AWPS is an information system that produces management reports intended to link the Army's industrial workload demands to workforce requirements. The Army has spent over $90 million on the system through fiscal year 2013 and plans to spend approximately $35.6 million over the next 5 years. AWPS relies on data from LMP, which was implemented after AWPS, and prior GAO reports have noted the potential overlap between AWPS and LMP. Pub. L. No. 111-139 mandates that GAO identify federal programs, agencies, offices, and initiatives that have duplicative goals or activities. This review evaluates the extent to which the Army has (1) followed certain applicable statutory and regulatory requirements for implementing AWPS and (2) assessed whether there is unnecessary overlap between AWPS and LMP. GAO reviewed laws, regulations, and prior GAO reports; analyzed Army documents related to its assessment of unnecessary overlap between AWPS and LMP; and interviewed Army officials.

What GAO Found

The Army has not followed certain applicable statutory and regulatory requirements for implementing the Army Workload and Performance System (AWPS). The National Defense Authorization Act for Fiscal Year 2002 requires the Secretary of the Army to submit annual progress reports to Congress on the implementation of the AWPS master plan until the Secretary certifies to Congress that AWPS is fully implemented. Although the Secretary has not yet made this certification, the Army has not been submitting progress reports. According to Army officials, the Army does not plan to submit progress reports until a revised master plan for AWPS is completed. However, for many years the Army has been unable to complete an update to the master plan due to a lack of oversight. Additionally, the Army has not validated AWPS as a manpower requirements determination tool. Army regulation states that the U.S. Army Manpower and Analysis Agency (USAMAA) is responsible for reviewing and validating manpower requirements models like AWPS and that major commands—such as the Army Materiel Command (AMC)—are responsible for submitting their models to USAMAA for validation. AMC directs the activities of Army industrial sites and is the primary user of AWPS. USAMAA and AMC officials have discussed the need for AWPS to be validated, but AWPS has not yet been submitted to USAMAA for validation. USAMAA officials initially contacted AMC regarding submitting AWPS for validation; however, at the time, AMC was not prepared to proceed. Later, when ready to seek validation, AWPS officials told GAO they asked for assistance from USAMAA, but additional assistance had not yet been provided. Because USAMAA officials involved in these earlier contacts subsequently left USAMAA, GAO was unable to corroborate that USAMAA did not respond to AWPS officials' request for assistance. Through USAMAA validation, the Army would have greater assurance that AWPS workforce management reports are accurate.
The Army has begun to assess whether unnecessary overlap exists between AWPS and the Logistics Modernization Program (LMP), but its overall progress has been limited. At the direction of AMC leadership, a project team was established and has begun to assess the extent to which AWPS's software functionality can be replaced with existing or future LMP functionality. The team has distributed surveys to AWPS users to collect information on how they use AWPS, and whether they use other tools—such as LMP—to provide similar functionality. Initial responses to this survey indicate the potential for some overlap between AWPS and LMP. Identifying unnecessary overlap among government programs or activities is important because overlap can lead to unnecessary duplication and can result in unnecessary costs and less-efficient and less-effective services. However, the overall progress of the team's assessment has been limited, due primarily to the absence of senior-level leadership attention and involvement and to the lack of a fully developed and documented approach for the assessment. Best practices have shown that sustained leadership attention and involvement can help organizations achieve positive results, and internal control standards call for proper documentation of evaluation processes. The absence of these elements increases the risk that the Army will not make progress in identifying and eliminating unnecessary overlap between AWPS and LMP.

What GAO Recommends

GAO is making four recommendations to the Army to (1) strengthen oversight for completing the update to the AWPS master plan, (2) direct that AWPS be submitted to USAMAA for validation, (3) identify a specific senior-level AMC manager or committee to provide oversight of the project team's efforts, and (4) establish a fully developed and documented approach for the team's assessment that includes a milestone for completing this effort. GAO requested comments from the Army, but none were provided.