Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions for Ship Crewing Requirements and Training
Released: 2010-06-09T00:00:00

Letter

Background

The Navy uses a four-step process and considers various factors in crewing its ships. First, it determines the workload of its ships. Second, it loads workload data into its Manpower Requirements System model to generate workforce requirements. Third, it determines the proportion of its requirements it can fund and then determines the number of authorized positions for each ship. Finally, it assigns individual personnel to its ships. Figure 1 shows the four-step process that generates workforce requirements and authorized positions and distributes personnel.

To determine the workload aboard a ship, the Navy Manpower Analysis Center sends analysts to ships to assess the tasks that must be accomplished and determine who should accomplish them. When determining workload, analysts consider various Navy standards and guidance with a focus on the ship’s required operational capability and projected operating environment. The analysts base their assessments on the amount of work necessary to accomplish the ship’s missions while at sea.

The Navy Manpower Analysis Center converts the workload information its analysts collect into workforce requirements using a computer model called the Naval Manpower Requirements System. This model contains a number of different standards to calculate workforce requirements, with the goal of identifying the most efficient mix of personnel to accomplish the required workload. A key standard used in this model is the Navy Standard Workweek, which is the number of hours per week available to accomplish required workload. The Navy Standard Workweek is used for planning purposes, is not restrictive or binding on commanders or commanding officers in establishing working hours, and is not intended to reflect the limits of personnel endurance. In February 2002, the Navy increased the Navy Standard Workweek from 67 to 70 hours. At the same time, it reduced other standards used in the model, such as the Productivity and Make Ready/Put Away Allowances. These changes to the standards used by the model led to a reduction in the model’s output—the ship’s workforce requirements—for any given ship’s workload information. While these changes affected enlisted workforce requirements, they did not affect those for officers since officer workforce requirements are not based on workload analysis.

After the Navy determines the workforce requirements for its ships, it estimates the level of funding that may be available to fill those requirements. When the Office of the Deputy Chief of Naval Operations (Manpower, Personnel, Training and Education) determines that a requirement can be funded, the Navy then refers to that requirement as an authorized billet (i.e., position) that can be filled.

The Navy’s Distribution Office fills the authorized positions with personnel. Distribution Office officials consider several factors as they assign personnel to authorized positions. These factors include the skills required by the position, personnel preferences, deployment schedules, and the “distributable” inventory of personnel—the number of personnel available to be assigned to ships. This inventory is limited not only by personnel levels but also by factors such as the servicemembers’ health and administrative statuses. Because the inventory of personnel available for assignment is generally less than the number of authorized positions, ships generally do not receive personnel for all of their authorized positions.

In October 2001, under an initiative referred to as optimal manning, the Navy directed pilot projects aboard a cruiser and a destroyer to determine if workload, while a ship is at sea, could be reduced by changing watchstanding requirements and more effectively using technology on board these ships. Based on the results, the Navy made certain adjustments; for example, it eliminated the port and starboard lookouts under typical at-sea conditions. In addition, it combined a number of watchstations, so tasks that had previously been the responsibility of several personnel were now consolidated into a single watchstation. The reduced at-sea workloads were then loaded into the Naval Manpower Requirements System model, which resulted in reduced workforce requirements. The Navy subsequently reduced authorized positions and actual personnel levels aboard its cruisers and destroyers.

The Navy uses both formal off-ship training programs and on-the-job training to improve the knowledge and skill levels its personnel have in their respective occupational specialties or jobs and to provide the knowledge and skills necessary for personnel to qualify to stand various watches, which may or may not relate directly to their occupational specialties. In 2000, the Chief of Naval Operations initiated a review of the Navy’s entire training system, including its organizational structure, curriculum development, and execution. As a result of this review and subsequent studies, the Navy made a number of changes to its training structure and programs. For example, it replaced some classes that were formerly led by instructors with computer-based training. It also established the Naval Education and Training Command in March 2003 to focus on education and training policy and strategy development. From 2003 to 2004, it created learning and support centers to administer training and education programs at the fleet level. These centers were functionally organized around enlisted ratings and mission areas. According to Navy officials, in 2008, the centers were aligned under the Naval Education and Training Command.

The Navy Lacks a Firm Analytical Basis for Some of Its Reductions to Cruiser and Destroyer Crew Sizes

Since 2001, the Navy has reduced the requirements and actual numbers of enlisted personnel aboard its guided-missile cruisers and guided-missile destroyers. The Navy made these adjustments based on its optimal manning initiative as well as a decision to change certain standards it uses to translate estimated workload into workforce requirements. In performing the analysis to support these reductions, the Navy considered various factors and made various assumptions but did not always address the elements called for in Navy guidance and best practices. Additionally, we were told by shipboard personnel that in-port workload is increasing, which raises questions about the Navy’s assumption that workload while a ship is underway exceeds in-port workload.

Requirements, Authorized Positions, and Current Onboard Personnel Have Decreased since 2001

In October 2001, the Navy initiated its optimal manning initiative by conducting pilot projects on a cruiser and a destroyer to determine if it could reduce their workloads while maintaining combat capability and readiness. Based on the results of these projects, the Navy, in March 2004, implemented the optimal manning initiative on its cruisers and destroyers. At the same time the optimal manning initiative was being tested and implemented, the Navy changed several of the standards it uses to translate a ship’s workload into workforce requirements, including increasing the standard workweek from 67 to 70 hours. The cumulative effect of both the optimal manning initiative and the changes to workload standards reduced the average enlisted workforce requirements aboard cruisers by 21 percent and aboard destroyers by 20 percent from fiscal years 2001 to 2009. Because of this decrease in workforce requirements, the number of authorized positions and current onboard enlisted sailors also decreased over this period, with the average number of enlisted personnel listed as current onboard cruisers decreasing by 15 percent and destroyers decreasing by 17 percent. During this time, the number of officers on these ships has experienced little change; therefore, we focused our analysis on the changes to enlisted personnel.

Table 1 shows the changes in average workforce requirements, authorized positions, and current onboard enlisted personnel for cruisers and destroyers from fiscal years 2001 to 2009.

Appendix IV contains additional details about changes in the workforce requirements, authorized positions, and current onboard enlisted personnel for cruisers and destroyers from fiscal years 2001 to 2009.

Some of the Navy’s Crew Size Reductions Were Supported by Analysis While Others Were Not

The Navy’s Total Force Manpower Policies Procedures Instruction requires, among other things, that the workforce requirements determination process methodology be based on data obtained through engineering studies, industry standards, technical and operational evaluations, job task analysis, work study, activity sampling, wartime tasking identified in operational instructions, or through application of staffing standards. Additionally, we have identified valid and reliable data as a critical component in assessing an agency’s workforce requirements as a human capital management best practice.

While Optimal Manning Reductions Were Based on Analysis They Also Relied on an Untested Assumption

In performing the analysis to support its optimal manning initiative, the Navy considered several of the elements called for in the guidelines set forth in the Navy’s Total Force Manpower Policies Procedures instruction and human capital best practices. For example, beginning in October 2001, it conducted pilot tests to reduce the at-sea workload on the guided- missile cruiser USS Mobile Bay and guided-missile destroyer USS Milius. These tests consisted of job task analysis and work studies to verify that proposed reductions to crew size under this initiative would not negatively affect the capabilities of these ships. Based on the results of the pilot tests and consultations with shipboard personnel, the Navy directed that the lessons learned from these tests be applied to other surface ships in 2004. Based on these reductions in at-sea workload, the workforce requirements for cruisers and destroyers decreased.

In assessing workload as part of the optimal manning initiative, the Navy used its long-held assumption that at-sea workload exceeds in-port workload requirements. Although in-port workload has varied over time, the Navy has not collected information to estimate such changes and determine whether it should adjust its assumption. However, shipboard personnel we interviewed consistently told us that in-port workload was increasing, which raises questions about the Navy’s assumption that workload while a ship is underway exceeds in-port workload. Specifically, we conducted interviews with senior personnel, including the commanding officer, executive officer, department heads, division officers, and senior enlisted personnel, on board 11 ships—four guided- missile cruisers, six guided-missile destroyers, and one guided-missile frigate. Personnel aboard these ships consistently told us that in-port workload was increasing primarily because of the increases in antiterrorism force protection requirements and the decrease in the size and function of shore intermediate maintenance activities. Specifically, we were told the following: Antiterrorism force protection workload has been increasing, and imposes additional training and qualification requirements beyond the increased watchstanding requirements. However, because the antiterrorism force protection watches—and the related required training and weapon qualifications—occur in port, the Navy does not calculate this workload when considering what the workforce requirement for a ship should be.

Assistance and support available to the ships from the shore intermediate maintenance activities has decreased. Personnel said that because the number of personnel working at these activities has decreased, some of the workload that the activities previously accomplished was transferred to the ship crews, thus increasing a ship’s in-port workload.

Some personnel also noted other factors that affect a ship’s ability to meet in-port workload requirements. For example, while in port, crew members can be temporarily pulled off their ships to support other waterfront requirements, such as helping other ships to prepare for inspections or conduct local operations. Additionally, the number of crew members available to do work on a ship is generally lower in port than when the ship is at sea because personnel complete many of their off-ship training requirements while they are in port, and personnel generally take their earned leave while their ships are in port. As a result of these factors, the ships may have fewer personnel to perform the in-port workload.

Changes to Standards in the Manpower Requirements System Model Are Not Based on Navy’s Required Analysis

In addition to decreasing the workload entered into its Manpower Requirements System model as a result of the optimal manning initiative, the Navy changed some of the standards this model uses to translate workload into workforce requirements. In February 2002, the Office of the Chief of Naval Operations, Total Force Programming, Manpower and Information Resources Division, directed the Navy Manpower Analysis Center to change some of the standards in the model in order to decrease workforce requirements. The Navy Manpower Analysis Center increased the Navy Standard Workweek from 67 to 70 hours, decreased the Make Ready/Put Away Allowance from 30 to 15 percent and decreased the Productivity Allowance from a uniform 20 percent to a floating range from 2 to 8 percent. Each of these changes led to a reduction in the model’s output, which is workforce requirements.

During our review, we found no evidence that the input changes to the Navy’s Manpower Requirements System model were based on the type of analysis required in the Navy’s Total Force Manpower Policies Procedures Instruction and human capital best practices. For example, when decreasing the Productivity Allowance, the Navy did not conduct the type of analysis called for in its instruction, such as job task analysis or engineering studies, to verify this change. Rather, according to Navy officials, the standards were changed in order to decrease workforce requirements to the level of authorized positions at the time. They said that workforce requirements for cruisers and destroyers were overstated as these ships had historically functioned with crew sizes smaller than their calculated workforce requirements. Additionally, Navy officials said that a 2001 study by the Center for Naval Analyses found that the current workweek, which at the time was 67 hours, could be increased. However, in analyzing this study we found that the study was based on workload data from a single ship, and the study’s authors had stated that the study should not be used to expand the standard workweek for other ships.

In a 2005 report, the Naval Audit Service also found that these changes were not based on verifiable analysis or data. In response to that report, the Office of the Assistant Deputy Chief of Naval Operations (Manpower, Personnel, Training and Education) stated that it had requested funding for the Center for Naval Analyses to conduct a formal study of the Navy Standard Workweek in fiscal year 2006; however, as of April 2010 we found no evidence that this study was conducted. By changing the standards in its model without the analysis required by its instruction, the Navy lacks assurance that the requirements generated by the model are reliable and accurate. Without performing additional analysis to determine that the factors and assumptions it uses to determine personnel requirements are valid, the Navy cannot be assured that it has appropriately sized ship crews.

Overall Impact of Reduced Crew Sizes on the Capabilities and Condition of Ships Is Unclear

While the Navy’s efforts to achieve efficiencies are important given growing compensation costs, personnel aboard the ships we visited told us that they believe the reductions in crew size have been detrimental to both the capabilities and condition of the ships. For example, many shipboard personnel cited changes in the way their ships prepared for material inspections as examples of the impacts of decreased crew sizes. Specifically, they said that their ships were no longer capable of preparing for inspections by the Navy’s independent Board of Inspection and Survey (INSURV) without outside help. INSURV conducts material inspections of Navy ships every 5 years. Shipboard personnel and shore-based Navy officials knowledgeable about these inspections told us that prior to reductions in crew size, cruisers and destroyers would typically prepare for these inspections with just the crew on board the ships. However, they said the crew size of these ships is now insufficient to prepare the ships; therefore personnel from other ships and shore establishments supplement the ship crews in conducting maintenance and preservation tasks to prepare the ships to be inspected. In addition, the shipboard personnel we spoke with said that they thought the reductions in crew sizes were negatively affecting the ships’ material condition and could ultimately lead to an increase in ship mishaps.

In our analysis of the results of the INSURV inspections, we did not find any specific trend showing a decline in the material condition of ships. However, it is unclear in what way the results of these inspections may be affected by the outside support these ships’ crew members told us they were receiving. As part of its inspection process, INSURV identifies and reports material conditions that substantially reduce a ship’s fitness for naval service and its ability to perform its primary and secondary missions. If INSURV determines that the results of its inspection of a particular ship call into question that ship’s ability to conduct prompt and sustained combat operations, it classifies that ship as unfit for sustained combat operations and includes this classification in its report on the ship’s inspection. Table 2 shows the numbers of cruisers and destroyers inspected since 2003, the first year data were available on the classification of ships as fit or unfit.

Regarding ship inspections, as shown in table 2, only one cruiser or destroyer was deemed unfit for sustained combat operations by INSURV from 2003 to 2007, while in both 2008 and 2009 three ships were determined to be unfit. Because of the relatively small number of inspections each year, it is not possible to draw the conclusion that the last 2 years represents a trend in fitness of these ships. However, several Navy officials told us they believe that reductions in crew sizes were having a detrimental effect on the condition of these ships.

Regarding ship mishaps, we examined the number of class A and B mishaps from 2001 to 2009 for cruisers and destroyers and did not find a definitive trend in the number of mishaps. We did not evaluate the circumstances of each reported mishap. Class A and B mishaps are the most severe type of mishaps, with class A representing serious personal injury or material damage of at least $1 million, and class B mishaps representing at least $200,000 in damage. The results of this analysis are shown in figure 2.

Based on the relatively small number of mishaps that occur each year, we were unable to identify a trend in the results.

Shipboard personnel aboard destroyers also told us that prior to reductions in crew sizes, their ships were able to simultaneously operate two replenishment stations while underway but now the ships are only able to operate one station at a time. This means the dangerous underway replenishment evolution, where ships operate side by side while supplies are transferred from one ship to the other, now takes longer to complete.

The Navy Has Evaluated Some Aspects of Its Training Changes but Lacks Performance Measures and Data to Fully Evaluate the Impact of These Changes

The Navy has made significant changes to its training programs and has evaluated some aspects of these changes, specifically, those related to cost and training time, but lacks the performance measures and data necessary to fully evaluate the impact changes to training have had on trainees’ job performance and the time required for personnel to achieve various qualifications.

Since 2003, the Navy Has Made Significant Changes to Its Training Programs

The Navy has made a number of changes to its off-ship training programs, including replacing many formerly instructor-led and lab-based classes with computer-based training. For example, in January 2003, the Navy eliminated the 6-month division officer course, held in Newport, Rhode Island, and replaced it with the current Surface Warfare Officers School-at- Sea course, which consists of six computer discs that junior officers are required to complete after they report to their first ships. After completing the computer-based training program and certain on-the-job training and watchstanding qualifications, junior officers report to the Surface Warfare Officers School in Rhode Island for 3 weeks of advanced proficiency training. After this training, they return to their ships where they may receive additional training before being tested for qualification as surface warfare officers. The Navy’s shift to computer-focused training was intended to reduce both training cost and training time before on-ship duty. Prior to the 2003 changes, instructors for the 6-month division officer course taught fundamental skills such as navigation, communication, maintenance, ship handling, and engineering, and students received hands- on training with simulators and patrol craft. The course also included classified instruction on U.S. and enemy radar, sonar, and weapons characteristics. Upon completion of the course, officers reported to their ships for additional, on-the-job training before they were tested for qualification as surface warfare officers.

In addition, the Center for Naval Engineering made changes to training for enlisted sailors, including changes to basic engineering training. According to Navy officials, prior to 2004, engineering courses were taught in eight separate schools based on the occupational specialty of the enlisted sailor. The instruction largely was instructor-led training held in a classroom or lab. In August 2004, this classroom-based training was replaced by the basic engineering common core, a general course integrating training from eight separate engineering schools. This course uses a blended learning approach with both self-paced computer-based training and instructor-led training in the classroom.

Navy Lacks Performance Measures to Fully Evaluate the Impact of Its Training Programs

The Navy has evaluated the impact that its changes to training programs have had on the length and cost of training, but it lacks a broader range of performance measures needed to evaluate the impact on other key aspects, such as the trainees’ job performance. Our prior work shows that it is important for agencies to incorporate performance measures that can be used to demonstrate the contributions training programs make to improve results. By incorporating valid measures of effectiveness into training and development programs, agencies can better ensure that they adequately address training objectives and thereby increase the likelihood that desired changes will occur in the target population’s skills, knowledge, abilities, attitudes, or behaviors.

In evaluating its training programs, the Navy has, for the most part, developed and applied input- and output-based metrics, such as the number of individuals trained, training hours per individual, and training cost per individual. For example, Navy training officials we spoke with stated that the changes to training have resulted in improvements such as decreases in class length and overall training cost. In addition, officials from the Surface Warfare Officers School said that changing the traditional methods of training for new officers from an on-site to a computer-based training program reduced the time it took for new officers to achieve their surface warfare officer qualification, both in terms of reducing time spent in training and on board their ships before qualifying. Officials also estimated that since implementing these new methods of training for surface warfare officers, the Navy has saved about $50 million annually. Similarly, officials at the Center for Naval Engineering told us that the shift to the basic engineer common core reduced training time. Based on our analysis of data provided by the center, we estimate that the average time to complete the course decreased by 29 percent, from an average of 101 to 71 days, compared to the previous curriculum.

While important, these metrics do not enable the Navy to determine how its training programs are affecting key aspects, such as the trainees’ job performance, knowledge, skills, and abilities once they report to their ships. For example, the Navy could measure the ability of personnel to perform the tasks expected of them once they report to their ships or the amount and types of on-the-job training required for personnel to perform required tasks. While the Navy did initiate a pilot study to measure trainees’ knowledge levels, officials told us that this study was only done once. To conduct this study, the Center for Naval Engineering divided a group of engineering students into two groups. One group received training via the traditional curriculum and the other group received the new curriculum. After completing their training, students from both groups were given a test administered by the Navy’s Afloat Training Group to determine their knowledge retention. The test results showed that students taught under the new curriculum had a higher level of knowledge retention than their counterparts. While this type of test is an example of a potential performance measure the Navy could use to assess training performance, it was a onetime study and did not measure knowledge retention at additional intervals after sailors reported to their ships.

Navy officials told us that the Navy would like to measure the quality of the instruction and its impact on job performance, knowledge, skills, and abilities by observing students when they report to ships and measuring to what extent they are able to use their training. However, they noted that such an evaluation would be difficult and require participation from not only the Navy training program but also operating forces. An official with the Assessment Branch of the Navy Education and Training Command told us that while the command is in the process of identifying potential performance metrics for measuring the quality of instruction and its impact on job performance, this effort is in a preliminary stage, and the official could not estimate when these metrics would be developed or implemented. Without performance measures in place, the Navy cannot fully determine the effectiveness of training changes it has already implemented and is not positioned to determine whether any adjustments need to be made.

During the course of our review, we met with leadership personnel from 11 different ships who generally expressed similar opinions concerning the impact of training changes on crew performance aboard their ships. Many of these leaders said that those sailors and officers who did not receive classroom instruction but instead were taught using new methods, such as computer-based training, required more on-the-job training when they arrived on board than those who had received classroom instruction. In addition, many of the leaders noted that as crew sizes have decreased, it has become more difficult to provide the on-the-job training these newer crew members need to accomplish their duties. One officer equated personnel receiving computer-based training instead of traditional classroom training to searching the Internet to learn how to fix a car versus learning on an actual car. He said that people are not really learning how to use the equipment because they do not touch it. Many of the senior enlisted personnel we spoke with said that because of declines in the knowledge and abilities of junior sailors who had completed computer- based training, they themselves were now directly supervising or actually completing work that junior sailors were previously able to do independently.

To help mitigate the perceived lack of basic knowledge of new officers who had not gone through the 6-month classroom-based division officer course, personnel on one ship told us that they created an on board classroom. The commanding officer told us that he created this program because he recognized the knowledge gap of junior officers when they arrived on board. The program consisted of new officers spending several days with various members of the crew and learning various skills, which the commanding officer said had previously been taught during the 6- month division officer course. In addition, because of a perceived lack of knowledge of junior officers, in 2008 the Commander Naval Surface Forces instructed the Navy’s Afloat Training Groups to develop and implement a 3-week surface warfare officer introduction course to supplement the computer-based training. This course is taught in San Diego, California; Norfolk, Virginia; Mayport, Florida; Everett, Washington; Pearl Harbor, Hawaii; and Yokosuka, Japan, and uses both lectures and simulator training to provide junior officers with the basic knowledge and skills they need on their assigned ships. The instructors for these courses are volunteers from both the Navy’s Afloat Training Groups as well as senior officers from ships in these areas.

The opinions we heard from shipboard personnel were echoed in findings of a 2009 report on computer-based training by the Naval Inspector General. According to the official responsible for this report, it was prompted by the results of a survey that the Naval Inspector General conducted of civilian and military Navy personnel in 2007 and 2008. One of the primary concerns of Navy personnel, according to this survey, was a declining level of knowledge of sailors reporting to their ships after completing computer-based training. The Inspector General’s office interviewed officials at various levels in the Navy’s training infrastructure, including officials at a number of schools as well as the fleet level to gather a cross section of opinions from these groups. Focus groups were conducted with personnel at various naval stations both inside and outside the contiguous United States. The review looked at both the enlisted and officer training. One of the key findings of this study was that speed of completion is the prime motivational factor, with the reward for timely completion being a priority for moving to the next stage of promotion. The study also found that shipboard personnel reported that the use of computer-based training resulted in more on board training in basic knowledge, skills, and abilities than was necessary before the implementation of computer-based training. Additionally, the inspection team was unable to find a valid metric with which to compare the occupational specialty knowledge obtained under computer-based training to the knowledge obtained under the traditional training system, and recommended that the Navy establish a training evaluation model to measure the outcome and effectiveness of training on sailor performance in the fleet.

The Navy Currently Lacks Complete Data to Track Personnel Qualification Times

While the Navy is taking steps to improve the completeness of its personnel qualification data, such as the time it takes for officers to achieve their surface warfare officer qualification, it currently lacks complete data with which to track changes in these times. The time required for personnel to achieve watchstation and warfare qualifications is a potential metric the Navy could use to measure the effects of changes of its training programs. Personnel qualification standards are an integral part of a ship’s training program. They describe the minimum knowledge and skills individuals must demonstrate prior to standing watches, maintaining equipment, or performing other specific duties. To assign qualification standards to shipboard personnel and track their progress, the Navy uses the Relational Administrative Data Management system. However, our assessment of the system’s 2005 through 2009 data found that the data were incomplete. For example, we found that many cruisers and destroyers did not report the qualification data monthly as required by a Navy instruction and some of these ships did not report the data at all. Navy Instruction 3502.1D states that all ships equipped with the Relational Administrative Data Management system should, by the 10th day of every month, upload personnel qualification data from the system manually into the Navy’s official repository for individual skills training called the Navy Training Management and Planning System. To determine the completeness of available personnel qualification data, we compared the actual number of data uploads to the potential number of uploads. The potential number of uploads in a given year equals 12 times the number of ships that have the Relational Administrative Data Management system because each ship is supposed to upload data once a month. Table 3 shows that the upload rate among cruisers and destroyers was only 6 percent in 2005 but was 69 percent by 2009.

Although rates of reporting are now higher than in 2005, the rate of reporting is still too low for us to use the data to make conclusions about qualification times. In addition, we were told that the Navy lacks baseline qualification data from the years prior to when it made changes to its training programs. Therefore, we cannot evaluate whether changes to these formal off-ship training programs have affected personnel qualification times.

To facilitate ships’ reporting of the data, the Navy developed new software that automatically transfers the qualification data from the Relational Administrative Data Management system into the Navy Training Management and Planning System. As of March 31, 2010, this new software was installed on 2 of the Navy’s 22 cruisers and on 9 of its 57 destroyers. According to Navy officials responsible for this program, the Navy plans to install the new release on its remaining cruisers and destroyers by fiscal year 2018.

A Navy working group also found similar issues with the personnel qualification standards data. In October 2006, the Commander, Naval Surface Forces Surface Warfare Enterprise, chartered the Train Through Qualification working group to research training and qualification processes to identify constraints and define barriers preventing the delivery of qualified or nearly qualified sailors to their ships. As a part of its review, the working group attempted to evaluate the amount of time it took for individuals to qualify for specific watchstations. However, the working group determined that the quantity of the data was insufficient to establish the time it took for a watchstander to qualify. According to the working group, having complete data is important for the working group to make appropriate and effective recommendations. Because it did not see any improvements in the data over time, the working group suspended its efforts in December 2009.

Conclusions

To achieve greater efficiencies and reduce costs, the Navy, among other things, has taken steps to reduce the size of the crews on its guided-missile cruisers and guided-missile destroyers and changed its approach to training sailors and surface warfare officers. While achieving efficiencies is very important, particularly given the growing costs to compensate personnel, the Navy must still ensure that it can perform its mission safely and maintain adequate readiness. In performing the analysis to support these reductions, the Navy considered some factors and made various assumptions, but in some cases did not address other key elements called for in its guidance and best practices for determining workforce requirements. Without undertaking certain types of analysis—such as job task analysis and engineering studies—or testing the validity of its long- standing assumptions, such as whether at-sea workload exceeds in-port workload, the Navy will not have all the information it needs to measure the workload of its ships and translate that workload into workforce requirements. Without performing additional analysis to determine that the standards and assumptions it uses to determine personnel requirements are valid, the Navy cannot be assured that its ship crews are appropriately sized to accomplish necessary tasks and maintain the material readiness of ships both at sea and in port on a daily basis.

Additionally, while the Navy was reducing the size of its crews, it changed its approach to training sailors and surface warfare officers by replacing some instructor-led training classes with computer-based training. While the Navy has metrics showing that some of these changes have reduced costs and training time, it lacks outcome-based performance measures to determine the effectiveness of the revised training in terms of trainees’ job performance, knowledge, skills, or abilities. Without outcome-based performance measures in place to measure the effectiveness of training programs, it is unclear what the effect has been. Overall, without an analysis of assumptions and standards used to reduce ship crew sizes— and without outcome-based performance metrics to evaluate the impact of training program changes—the Navy cannot be assured that the sizes of its ship crews are sufficient to operate and maintain its ships and cannot fully determine the effectiveness of the training changes it has implemented and whether further adjustments are necessary.

Recommendations for Executive Action

To improve the analytic basis of the Navy’s workforce requirements determination process, we recommend that the Secretary of the Navy direct the Chief of Naval Operations to take the following two actions: Conduct an assessment to validate the underlying assumptions and standards used to calculate shipboard workforce requirements, including the relative magnitude of in-port and at-sea requirements, the Navy Standard Workweek, and its associated Productivity and Make Ready/Put Away Allowances, while taking into account various other factors that can affect ships’ crews, such as the availability of shore support.

Review the results of this comprehensive assessment and determine whether it is necessary to adjust the personnel requirements for the Navy’s cruisers and destroyers.

To better gauge the impact of changes to the Navy’s training programs, we recommend that the Secretary of the Navy direct the Chief of Naval Operations to develop metrics to be used to measure the impact of training, such as the impact on job performance, knowledge, skills, and abilities as they relate to occupational and watchstanding proficiency.

Agency Comments and Our Evaluation

In written comments on a draft of this report, the Department of the Navy concurred with our recommendations and identified several actions it plans to implement them. Concerning our recommendation to conduct an assessment to validate the underlying assumptions and standards used to calculate shipboard workforce requirements, the Navy agreed to conduct a study and stated that such a study to validate the standards used to calculate workforce requirements, including the Navy Standard Workweek and Make Ready/Put Away Allowances, will provide additional rigor to its current methodologies. Additionally, the Navy agreed to examine some of the tenets of its optimal manning initiative with respect to anticipated, but not achieved, workload reductions.

Concerning our recommendation to develop metrics to measure the impact of training on job performance, knowledge, skills, and abilities as they relate to occupational and watchstanding proficiency, the Navy stated that it would leverage existing Surface Force efforts to measure the impact of training. Specifically, the Navy detailed the process it uses to report training readiness information by ship into the Defense Readiness Reporting System-Navy. While these actions provide the chain of command with information on training readiness once personnel are aboard ship, they do not provide specific information about the impact of off-ship training. Given that the Navy has made a number of changes in its off-ship training programs and that many of the shipboard personnel we interviewed had concerns about the effectiveness of current training programs, it is important for the Navy to determine the impact of its training changes. Additional metrics could be used to identify training areas that could be improved. Therefore, while the Navy should continue collecting and using the metrics it outlined in its comments on this report, it is important for the Navy to develop or identify specific metrics it can use to measure the impact of training at an individual level. The full text of DOD’s written comments is reprinted in appendix V.

We are sending copies of this report to other appropriate congressional committees and the Secretary of Defense. This report also is available at no charge on the GAO Web site at http://www.gao.gov.

Should you or your staff have any questions concerning this report, please contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VI.

Appendix I: Deployment of Individual Augmentees

Military operations in support of the Global War on Terrorism, particularly those in Iraq and Afghanistan, have challenged the ability of the Army and Marine Corps to provide needed ground forces. To help mitigate these challenges, the Navy has provided personnel, known as individual augmentees, to fill or augment units and organizations that support missions such as maritime and port security, civil affairs, airlift support, and detainee operations. U.S. Fleet Forces Command is responsible for administering the Navy’s individual augmentee program.

According to the Navy, at any given time, active duty personnel make up about half of the Navy’s individual augmentee force while the other half are mobilized reserve personnel. According to data provided by the Navy, about 37,000 active duty personnel were deployed as individual augmentees from fiscal year 2006 through fiscal year 2009. The Navy selects active duty personnel to fill individual augmentee assignments through one of two processes. In the first, the Individual Augmentee Manpower Management Assignment process, personnel already assigned to a command are deployed to carry out their individual augmentee assignments, while they remain assigned to their current commands. They then return to their assigned commands after completing their individual augmentee assignments, which can last from a few months to a year or more. The individual augmentee assignments generally occur on relatively short notice and can be disruptive for both the personnel and their commands. In June 2007, the Navy also began using a second process, the Global War on Terrorism Support Assignment process, to provide more predictability for sailors, their families, and Navy commands. Under this process, personnel can negotiate for an individual augmentee assignment when they are between their traditional Navy sea and shore rotational assignments. The personnel detach from their current commands while carrying out the individual augmentee assignments and transfer to their next permanent change of station assignments upon completion of the individual augmentee assignments. These individual augmentee assignments vary in length, but the standard duration is from 8 to 14 months.

Our analysis of data provided by the Navy found that from fiscal year 2006 through fiscal year 2009, about 31,000 active duty personnel (84 percent) were deployed using the Individual Augmentee Manpower Management Assignment process and about 6,000 active duty personnel (16 percent) were deployed using the Global War on Terrorism Support Assignment process. During this time period, of the about 31,000 personnel deployed as individual augmentees under the Individual Augmentee Manpower Management Assignment process, almost 24,000 (76 percent) came from shore commands and about 7,500 (24 percent) came from sea units, such as ships. About 1,200 (16 percent) of the 7,500 personnel coming from ships were deployed from guided-missile cruisers and guided-missile destroyers. Table 4 shows the numbers of individual augmentees deployed in fiscal years 2006 through 2009.

Of the 1,224 personnel deployed from cruisers and destroyers, the majority (about 1,100) were enlisted sailors and the remainder (about 115) were officers. Compared to the total number of sailors serving on these ships, the number of enlisted sailors deployed as individual augmentees is a relatively small percentage. For example, during fiscal year 2009 about 2 percent of the total current onboard enlisted personnel for cruisers were deployed as individual augmentees, while 1.6 percent were deployed from destroyers.

Our analysis also found that enlisted sailors, especially petty officers, constituted most of the individual augmentees deployed from cruisers and destroyers. Of the officers deployed as individual augmentees, most were junior grade. Figure 3 shows the distribution of enlisted personnel and officers deployed as individual augmentees from cruisers and destroyers during fiscal years 2006 through 2009.

As part of our analysis, we examined whether particular occupational fields on cruisers and destroyers were in higher demand than others for the individual augmentee program. Table 5 lists the top job titles and descriptions of the individual augmentees deployed from cruisers and destroyers during fiscal years 2006 through 2009. Overall, personnel from the nine occupational fields listed in the table made up 72 percent or 795 of the 1,102 enlisted individual augmentees deployed from cruisers and destroyers. Among the enlisted occupational fields, we found that information systems technicians topped the list for filling individual augmentee assignments.

To obtain information on the deployment of individual augmentees, we discussed this topic during our ship visits, which covered four cruisers, six destroyers, and one frigate. During our visits, shipboard personnel acknowledged that while the number of individual augmentees deployed from their ships was relatively small, as a percentage of the overall size of the crews, individual augmentee assignments result in staffing losses that ships do not receive additional personnel to fill. They also noted that returning individual augmentees cannot be required to immediately deploy with the ship because Navy policy requires that, subject to certain exceptions, they receive a period of “dwell-time” between deployments. Because of the Navy’s reductions in crew sizes on cruisers and destroyers, shipboard personnel said that every sailor on board is essential to the operation of a ship and the loss of even one crewmember to an individual augmentee assignment can be detrimental to a ship’s ability to carry out its mission. Also, they said that the situation is compounded when the individual augmentee is a middle- to senior-level crew member who normally mentors, coaches, and trains junior staff. Personnel on board the ships we visited said that having fewer sailors means more work for those on board, which can lower morale and quality of life, and that some duties such as ship maintenance, may be delayed or may not get done for an indefinite period.

Appendix II: Distribution of Enlisted and Officer Pay Grades, Fiscal Years 2001 to 2009

The House Armed Services Committee, in its report accompanying the National Defense Authorization Act for Fiscal Year 2010, directed GAO to compare shipboard rank/rate distributions over time and analyze underlying reasons for any changes and their impact on ship capabilities for selected ship types. For this review, we focused on guided-missile cruisers and guided-missile destroyers. In our analysis of the rank distribution for officers and rate distribution for enlisted sailors, we did not find a major change in this distribution from fiscal years 2001 to 2009.

Figure 4 shows the distribution in terms of percentages of enlisted sailors on cruisers from fiscal years 2001 to 2009.

As demonstrated in figure 4, the breakdown among the general categories of enlisted pay grades—general rates (E-1 to E-3), petty officers (E-4 to E- 6), and chief petty officers (E-7 to E-9)—on cruisers showed only slight variations from fiscal year 2001 to fiscal year 2009.

The rate distribution for enlisted sailors on destroyers was similar to the distribution on cruisers—displaying only relatively minor changes from fiscal year 2001 to fiscal year 2009. Specifically, figure 5 shows the distribution in terms of percentages of enlisted sailors on destroyers from fiscal years 2001 to 2009.

Appendix III: Scope and Methodology

To assess the extent to which the Navy used valid assumptions and standards in determining crew sizes for cruisers and destroyers, we analyzed various Navy documents and instructions related to determining crew sizes, including Office of the Chief of Naval Operations Instruction 1000.16K, Navy Total Force Manpower Policies Procedures in order to identify the steps required in the Navy’s process to determine crew sizes. We reviewed relevant prior GAO work on human capital management, as well as various studies and reports from research organizations about the Navy’s process to crew its ships. We analyzed reports from the Naval Audit Service about the Navy’s optimal manning initiative and manpower requirements process for surface ships for information about the ship crewing process.

We also calculated whether workforce requirements and authorized positions had changed from fiscal years 2001 to 2009 by analyzing data provided by the Navy from its Total Force Manpower Management System for cruisers and destroyers. To calculate whether the number of enlisted personnel designated as current onboard personnel for cruisers and destroyers had changed from fiscal years 2001 to 2009, we analyzed data provided by the Navy from its Navy Manpower Program and Budget System. We averaged the workforce requirements, authorized positions, and current onboard data across each ship type for each fiscal year to find an average value.

To assess the extent to which the Navy has measured the impact of changes to its training programs, including qualification times, we reviewed prior GAO work on assessing strategic training and development efforts. We also reviewed Navy instructions on surface force training, personnel qualification standards, and Navy training system requirements, acquisition, and management. To assess the extent to which the Navy is tracking performance measures, we interviewed Navy training officials and ship personnel to obtain their views on how the implementation of new training practices has affected shipboard performance. To assess changes in training times and the cost of training related to occupational specialties, we analyzed data and interviewed individuals from the Center for Naval Engineering located in Norfolk, Virginia and the Surface Warfare Officers School located in Newport, Rhode Island. Furthermore, we reviewed reports from the Naval Inspector General, the Center for Naval Analyses, LMI, and the Office of the Deputy Chief of Naval Operations (Manpower, Personnel, Training and Education) to gain insights about the Revolution in Training.

In addition, we obtained and analyzed personnel qualification standards data that are recorded in the Relational Administrative Data Management system on board ships from the Naval Education and Training Professional Development and Technology Center. To determine the percentage of guided-missile cruisers and guided-missile destroyers that were uploading their data into the Navy Training Management and Planning System on a monthly basis as required, we compared the total number of times cruisers and destroyers uploaded the data to the total number of times these ships should have uploaded the data (i.e., once a month or 12 times per year) for calendar years 2005 through 2009. We also interviewed officials from the Navy’s Train Through Qualification working group responsible for assessing sailor training and qualification processes to discuss their findings. Based on our analysis and discussions with the working group officials, we determined that the personnel qualification standards data were not reliable for evaluating whether changes to formal off-ship training programs have affected personnel qualification times because they were not complete.

In the course of our work, we interviewed officials from the following organizations and offices: Aegis Training and Readiness Center Afloat Training Group, Pacific Center for Naval Engineering Commander, Naval Surface Forces Cruiser Class Squadron Destroyer Class Squadron Naval Education and Training Command Naval Education and Training Professional Development and Technology Naval Inspector General’s Office Naval Postgraduate School Naval Safety Center Navy Manpower Analysis Center Navy Personnel Command, Bureau of Naval Personnel Office of the Deputy Chief of Naval Operations (Manpower, Personnel, Training and Education)

Office of the Under Secretary of Defense, Personnel and Readiness Space and Naval Warfare Systems Center Atlantic Surface Warfare Officers School Train Through Qualification working group U.S. Fleet Forces Command We also met with personnel on board four Navy cruisers, six destroyers, and one frigate. Specifically, we visited the USS Leyte Gulf (CG-55), USS San Jacinto (CG-56), USS Normandy (CG-60), USS Monterey (CG-61), USS Benfold (DDG-65), USS Milius (DDG-69), USS Bulkeley (DDG-84), USS Shoup (DDG-86), USS Momsen (DDG-92), USS Stockdale (DDG-106), and USS Vandegrift (FFG-48). We conducted interviews with senior personnel on board these ships, including the commanding officer, executive officer, department heads, division officers, and senior enlisted personnel.

Because ship personnel told us that they believed reduced crew sizes affected the ships’ material condition and could lead to mishaps, we analyzed inspection and mishap data. Specifically, we analyzed the results of the Navy’s Board of Inspection and Survey (INSURV) inspections for cruisers and destroyers from fiscal year 2001 to fiscal year 2009. The number of categories INSURV evaluates ships across has increased since 2001; therefore, we compared inspection results across the 17 categories, which have consistently been measured since 2001. We averaged the results of the ships inspected each year by category and classified the average score as red, yellow, or green based on the standards INSURV uses in its inspections. We compared the proportion of inspection categories receiving red, yellow, or green scores each fiscal year to assess whether there was a discernable trend in the inspection results over this period. We also obtained mishap data from the Naval Safety Center. We requested data for class A, B, and C mishaps, as these are the types of mishaps that ships are required to report to the Safety Center through the Navy’s Web Enabled Safety System. We compared the number of mishaps, by class that had occurred in each fiscal year from 2001 to 2009, which was the last full year available for analysis, to assess whether there was a discernable trend in mishaps over this period. We reported on the number of A and B mishaps, as these are the most severe types of mishaps in terms of personal injury and material damage.

To assess the extent to which the Navy’s support for individual augmentees and training and transition teams has affected the levels and composition of shipboard manning, we reviewed the Navy’s policies regarding the sourcing and assignment of individual augmentees. In addition, we obtained and analyzed data from the Navy Personnel Command on the number of active duty sailors deployed as individual augmentees from fiscal years 2006 through 2009, which included sailors deployed on training and transition teams. Officials from the command stated that although the data are neither complete nor completely accurate, they represent the Navy’s best available data on individual augmentees. Using these data, we analyzed the rates and ranks of the sailors deployed as individual augmentees from Navy cruisers and destroyers to determine if there were any trends over this period. We also looked at whether particular occupational fields on cruisers and destroyers, such as information systems technicians or fire controlmen, had sailors deployed as individual augmentees more often than other occupational fields. To gain additional insight into the impact individual augmentees have had on shipboard manning, we interviewed officials from U.S. Fleet Forces Command; the Office of the Commander, Naval Surface Forces; and personnel on board four Navy cruisers, six destroyers, and one frigate.

To calculate whether the rank/rate distribution had changed from fiscal years 2001 to 2009, we analyzed the current onboard data for enlisted personnel and requirements data for officers. We used the current onboard data for enlisted personnel provided by the Navy to analyze whether the rate distribution had changed over this period because the workforce requirements for enlisted personnel decreased by about 20 percent for cruisers and destroyers. This decrease caused the authorized positions and current onboard enlisted personnel levels to decrease by lesser amounts. We used the requirements data for officers because workforce requirements for officers on these two ship classes have experienced little change over this time period, and therefore we had no indication that there was any corresponding change to the level of authorized positions or current onboard personnel.

We conducted this performance audit from July 2009 through June 2010 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix IV: Requirements, Authorized Positions, and Current Onboard Enlisted Personnel for Cruisers and Destroyers

Since 2001, in an effort to reduce costs and increase efficiencies, the Navy has reduced the requirements and actual numbers of enlisted personnel aboard its guided-missile cruisers and guided-missile destroyers. Figure 6 shows the average changes in workforce requirements, authorized positions, and current onboard enlisted personnel for cruisers from fiscal years 2001 to 2009.

Figure 6 shows that between fiscal years 2002 and 2004, the enlisted requirements and level of authorized positions decreased. The average number of enlisted sailors listed as currently on board these ships did no decline as rapidly as requirements and authorized positions since 2002, because sailors were not removed from the ships after the decrease in authorized positions. Instead, a Navy official told us that the Navy just waited for sailors to rotate off the ships and did not replace them. As a result, current onboard numbers continued to decline from 2003 through 2008, and in 2009, current oo and authorized positions. and authorized positions. nboard numbers dropped below requirements nboard numbers dropped below requirements Figure 7 shows the average change in workforce requirements, authoriz Figure 7 shows the average change in workforce requirements, authoriz positions, and current onboard enlisted personnel for destroyers from positions, and current onboard enlisted personnel for destroyers from fiscal years 2001 to 2009. fiscal years 2001 to 2009.

Figure 7 shows that between fiscal years 2003 and 2008, the number of enlisted requirements and authorized positions has declined, while the average number of enlisted sailors listed as current onboard has declined since fiscal year 2004.

Appendix V: Comments from the Department of Defense

Appendix VI: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the contact named above, key contributors to this report were Michael Ferren (Assistant Director), Elizabeth Deyo, Mary Jo LaCasse, Tamiya Lunsford, Richard Powelson, Steven Putansu, Jerry Sandau, Grant Sutton, and Ben Thompson.