Military Readiness: Navy's Fleet Response Plan Would Benefit from a Comprehensive Management Approach and Rigorous Testing
Published: 2005-11-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Navy has been transforming itself to better meet 21st century needs. Since 2000, the Congress has appropriated about $50 billion annually for the Navy to operate and maintain its forces and support around 376,000 military personnel. In recognizing that the Navy faces affordability issues in sustaining readiness within its historical share of the defense budget, the Chief of Naval Operations announced a concept called the Fleet Response Plan to enhance its deployment readiness status. The Fleet Response Plan is designed to more rapidly prepare and sustain readiness in ships and squadrons. GAO evaluated the extent to which the Navy has (1) employed a sound management approach in implementing the Fleet Response Plan and (2) tested and evaluated the effectiveness of the plan and shared results to improve implementation.
section_title: What GAO Found
paragraphs: In establishing the Fleet Response Plan, the Navy has embraced a major change in the way it manages its forces. However, the Navy's management approach in implementing the Fleet Response Plan has not fully incorporated sound management practices needed to guide and assess implementation. These practices include (1) establishing a coherent mission and strategic goals, including resource commitments; (2) setting implementation goals and a timeline; and (3) establishing a communication strategy. While the Navy has taken a number of positive actions to implement the plan, it has not provided readiness goals for units other than carrier strike groups; resource and maintenance goals; performance measures and timelines; or a communications strategy. Sound management practices were not fully developed because senior leaders wanted to quickly implement the plan in response to changes in the security environment. However, without an overall management plan containing all of these elements, it may be difficult for the Navy to determine whether its efforts to improve the fleet's readiness are achieving the desired results, adequately measuring overall progress, or identifying what resources are needed to implement the Fleet Response Plan. The Navy has not fully tested and evaluated the Fleet Response Plan or developed lessons learned to identify the effectiveness of its implementation and success over time. Systematic testing and evaluation of new concepts is an established practice to gain insight into how systems and capabilities will perform in actual operations. However, instead of methodically conducting realistic tests to evaluate the Fleet Response Plan, the Navy has tried to demonstrate the viability of the plan by relying on loosely linked events that were not part of an overall test and evaluation strategy. This approach could impair the Navy's ability to validate the plan and evaluate its success over time. In addition, the Navy has not used its lessons learned system to share the results of its Fleet Response Plan events or as an analytical tool to evaluate the progress of the plan and improve implementation, which limits the Navy's ability to identify and correct weaknesses across the fleet.

section_title: Letter
paragraphs: A key goal of Navy senior leadership is to transform the Navy to better  meet 21st century security challenges. Since 2000, Congress has  appropriated about $50 billion annually for the Navy to operate and  maintain its forces and support around 376,000 active military personnel.  Nonetheless, the Navy recognizes it is facing affordability issues related to  sustaining readiness while developing and procuring several types of new  ships within its historical share of the defense budget. One area where the  Navy has made significant changes is in its operational posture. In March  2003, the Chief of Naval Operations initiated the development of a  concept, which became known as the Fleet Response Plan, to enhance  the Navy’s deployment readiness status. The Fleet Response Plan, as  implemented by Fleet Forces Command in May 2003, is designed to more  rapidly prepare and then sustain readiness in ships and squadrons. To  achieve this capability, the plan alters prior manning, maintenance, and  training practices to allow for a more responsive and ready naval force.  The Navy expects this new readiness approach will enable its forces to  provide not only presence and engagement in forward areas, but also  surge a greater number of ships on short-notice in response to significant  crises without increasing the readiness budget. The Fleet Response Plan  modifies the Navy’s pre-2001 rotational deployment policy, replacing   6-month routine deployments with more flexible deployment options for  as many as eight carrier strike groups when and where needed.
The Fleet Response Plan represents a major change in the way the Navy  manages its forces. Implementing large-scale change management  initiatives, such as organizational transformations, can be a complex  endeavor. Our prior work shows that failure to adequately address—and  often even consider—a wide variety of management issues is at the heart  of unsuccessful transformations. We have identified a number of key best  practices and lessons learned from major public and private sector  organizational mergers, acquisitions, and transformations. These sound  management practices include, for example, establishing a coherent  mission and integrated strategic goals to guide the transformation,  including resource commitments; setting implementation goals and a  timeline to build momentum and show progress from day one; and  establishing a communication strategy to create shared expectations and  report related progress.
We prepared this report under the Comptroller General’s authority and are  providing it to you because of your oversight of defense issues. We have  previously reported on the maintenance aspects of the Navy’s Fleet  Response Plan. This report focuses on the following two questions: (1) To  what extent has the Navy employed a sound management approach in  implementing the Fleet Response Plan? (2) To what extent has the Navy  tested and evaluated the effectiveness of its Fleet Response Plan and  shared results to improve its implementation?
To assess the Navy’s management approach in implementing the Fleet  Response Plan, we obtained and analyzed key messages, briefings, and  instructions on the Fleet Response Plan and interviewed Department of  Defense (DOD) and Navy headquarters and fleet officials, and compared  the Navy’s approach with best practices for transformations of large  organizations. To assess the extent to which the Navy has tested the  effectiveness of the Fleet Response Plan and shared results to improve its  implementation, we obtained briefings from and interviewed Navy  officials, reviewed and queried the Navy Lessons Learned System to  determine relevant lessons recorded, and examined Navy guidance on test  and evaluation efforts. We reviewed and validated the Navy Lessons  Learned System data and determined the data were sufficiently reliable for  our analysis. We conducted our review from January 2005 through August  2005 in accordance with generally accepted government auditing  standards. The scope and methodology used in our review are described in  further detail in appendix I.
subsections: section_title: Background
subsections: section_title: Composition of a Carrier Strike Group
paragraphs: Carrier strike groups are typically centered around an aircraft carrier and  its air wing, and also include a guided missile cruiser; two guided missile  destroyers; a frigate; an attack submarine; and one or more supply ships  with ammunition, fuel, and supplies (such as food and spare parts). These  groups are formed and disestablished on an as needed basis, and their  compositions may differ though they contain similar types of ships. Figure  1 shows a carrier strike group sailing in a group formation as it prepares to  participate in an exercise.
section_title: Origin of the Fleet Response Plan
paragraphs: Prior to the September 11, 2001, terrorist attacks, only those Navy ships  and air squadrons at peak readiness were deployed overseas, usually for   6 months at a time. Most of the Navy’s remaining units were not available  because they were in early stages of their maintenance or training cycles,  or because the Navy did not have good visibility of the readiness of these  units. This prompted the Chief of Naval Operations in March 2003 to task  the Commander, Fleet Forces Command, to develop the Fleet Response  Plan concept to enhance the Navy’s surge capability. The Chief of Naval  Operations approved the concept in May 2003 and further directed the  Commander, Fleet Forces Command, to be responsible and accountable  for effectively implementing the plan.
section_title: Fleet Response Plan Believed to Provide Increased and Flexible Readiness
paragraphs: The Fleet Response Plan emphasizes an increased level of readiness and  the ability to quickly deploy naval forces to respond to crises, conflicts, or  homeland defense needs. The plan applies broadly to the entire fleet;  however, it only sets specific requirements for carrier strike groups. For  example, the plan calls for eight carrier strike groups to be ready to deploy  within 90 days of notification. Six of them would be available to deploy  within 30 days and the other two within 90 days. This is commonly  referred to as the 6 + 2 goal. Under the Fleet Response Plan, the Navy has  developed a surge capability schedule that it uses to manage and identify  the level of training a ship has completed and its readiness to deploy. The  schedule contains three progressive readiness goals: emergency surge,  surge-ready, and routine deployable status. Each readiness goal specifies  phases of training that must be completed to achieve the goal. To be  placed in emergency surge status, a ship or an air squadron needs to have  completed its unit-level phase training. Achieving surge-ready status  requires the completion of integrated phase training. Attaining routine  deployable status requires achievement of all necessary capabilities,  completion of underway sustainment phase training, and certification of  the unit for forward deployed operations.
The surge capabilities schedule provides a readiness snapshot for each  ship, allowing decision makers to quickly determine which ships are  available to meet the needs of the mission. Figure 2 illustrates how the  Navy notionally identifies the eight aircraft carriers available for surge  deployments. The carriers numbered 1 through 6 are expected to be ready  to deploy within 30 days notice. The carriers labeled “+1” and “+2” are  expected to able to surge within 90 days notice. The six surge-ready  carriers include two carriers on deployment (numbered 3 and 4), one  carrier that is part of the forward deployed naval force based in Japan  (number 6), and three carriers in the sustainment phase (numbered 1, 2,  and 5). These six carriers are expected to have completed postdeployment  depot-level maintenance and their unit-level phase training. The two  additional surge carriers are expected to have completed depot-level  maintenance but not to have completed unit-level phase training. The  remaining four carriers are in the maintenance phase or deep  maintenance.
section_title: Revised Fleet Response Plan Is Being Developed
paragraphs: Based on the Navy’s experiences during the past 2 years, Fleet Forces  Command has convened a cross-functional working group to develop a  refined version of the Fleet Response Plan. This update, known as Fleet  Response Plan-Enhanced, is intended to further define the Fleet Response  Plan, modify terminology for progressive readiness states to better reflect  their meaning, tie in elements such as a human capital strategy, and  expand the focus of the plan beyond carrier strike groups to the entire  Navy. It may also extend the Fleet Response Plan’s current employment  cycle length of 27 months. The Fleet Response Plan-Enhanced is still  under development at this time.
section_title: Fleet Response Plan Does Not Fully Incorporate Sound Management Practices
paragraphs: The Navy’s management approach in establishing the Fleet Response Plan  as its new readiness construct has not fully incorporated sound  management practices needed to effectively guide, monitor, and assess  implementation. Studies by several organizations have shown that  successful organizations in both the public and private sectors use sound  management practices to assist agencies in measuring performance,  reporting results, and achieving desired outcomes. These practices  provide management with a framework for effectively implementing and  managing programs and shift program management focus from measuring  program activities and processes to measuring program outcomes. Sound  management practices include (1) establishing a coherent mission and  integrated strategic goals to guide the transformation, including resource  commitments; (2) setting implementation goals and a timeline to build  momentum and show progress from day one; and (3) establishing a  communication strategy to create shared expectations and report related  progress.
The Navy’s implementation of the Fleet Response Plan has included some  aspects of these practices. For example, the Navy has established some  strategic goals needed to meet the intent of the plan, such as the  progressive readiness levels of emergency surge, surge-ready, and routine  deployable status. The Navy also has established specific training actions  to support these goals, such as that carrier strike groups must complete  unit-level training to be certified as emergency surge-ready. However,  other actions taken by the Navy do not fully incorporate these practices.  For example, the Navy has identified the 6 + 2 surge capability as a  readiness goal and performance measure for carrier strike groups, but no  such goal was established for the rest of the fleet. The Navy also has some  unofficial goals and performance measures regarding manning and  maintenance, but these unofficial goals and performance measures have  not been formally established. For example, briefings on the Fleet  Response Plan state that the Navy desires and needs fully manned ships  (i.e., manning at 100 percent of a ship’s requirement) for the program to be  successful. Moreover, according to Navy officials, the Navy has not  established milestones for achieving its results.
In addition, 2 years after initiating implementation of the Fleet Response  Plan, the Navy still does not have an official written definition of the Fleet  Response Plan that clearly establishes a coherent mission and integrated  strategic goals to guide the transformation, including resource  commitments. This definition would describe the Fleet Response Plan’s  total scope and contain guidance with formal goals and performance  measures. The Navy recently has taken some action to address this area.  In February 2005, the Navy directed the Center for Naval Analyses to  conduct a study to develop formal definitions and guidance as well as  identify goals and performance measures for the plan. However, it remains  to be seen whether this study will be completed as planned by November  2005; if it will recommend developing and implementing sound  management practices, such as goals, measures, milestones, and timelines;  and whether any management improvement recommendations made in the  study will be implemented by the Fleet Forces Command, the Navy  command responsible for implementing the Fleet Response Plan. Without  goals, performance measures, timelines, milestones, benchmarks, and  guidance to help effectively manage implementation of the Fleet Response  Plan and determine if the plan is achieving its goals, the Navy may find it  more difficult to implement the Fleet Response Plan across the entire  naval force.
Moreover, despite the Navy’s unofficial goal that the Fleet Response Plan  be budget neutral, as articulated in briefings and by senior leaders, the  Navy has not yet clearly identified the resources needed to achieve its  goals or provided a rationale for how these resources will contribute to  achieving the expected level of performance. Navy officials have said that  current operations and maintenance funding levels, as well as manning at  100 percent of required positions, have contributed to successful  implementation of the Fleet Response Plan. However, officials do not  know what level of manning or funding is actually required for program  success over the long term to avoid any unintended consequences, such as  greater amounts of deferred maintenance. According to Navy officials, it is  difficult to attribute costs to the plan because there is no single budget line  item that tracks the costs associated with the Fleet Response Plan.  Without knowing the funding needed, the Navy may not be able to assess  the impact of possible future changes in funding on implementing the plan.
Furthermore, without a comprehensive plan that links costs with  performance measures and outcomes, neither the Navy nor Congress may  be able to determine if the Fleet Response Plan is actually achieving its  unofficial goal of being budget neutral.
Finally, the Navy also has not developed a comprehensive  communications strategy that reaches out to employees, customers, and  stakeholders and seeks to genuinely engage them in a two-way exchange,  which is a critical step in successfully implementing cultural change or  transformation. We looked for formal mechanisms that communicated the  details of the Fleet Response Plan and spoke with personnel from carrier  strike groups, aircraft carriers, air wings and an air squadron, one surface  combatant ship, and other command staff. We found that while the Fleet  Response Plan was communicated extensively to senior-level officers, and  the Navy provided numerous briefings and messages related to the plan,  communication and understanding of the plan did not flow through to the  lower ranks. While the concept of the Fleet Response Plan is generally  understood by some senior-level officials, many of the lower grade  personnel on these ships were unaware of the scope, goals, and other  aspects of the plan. In the absence of clear communication throughout the  fleet via an overall communications strategy that could increase employee  awareness of the Fleet Response Plan, its successful implementation could  be impeded.
Sound management practices, such as those noted above, were not fully  used by the Navy because senior leaders wanted to quickly implement the  Fleet Response Plan in response to the Chief of Naval Operations’ desires.  However, without an overall management plan containing all of these  elements to guide the implementation of such a major change, it may be  difficult for the Navy and Congress to determine the extent to which the  Fleet Response Plan is achieving the desired results, measure its overall  progress, or determine the resources needed to implement the plan.
section_title: Navy Has Not Fully Tested and Evaluated the Fleet Response Plan or Developed Lessons Learned
paragraphs: The Navy has not fully tested and evaluated the Fleet Response Plan or  developed lessons learned to identify the effectiveness of its  implementation and success over time. The methodical testing, exercising,  and evaluation of new doctrines and concepts is an established practice  throughout the military to gain insight into how systems and capabilities  will perform in actual operations. However, instead of methodically  conducting realistic tests to evaluate the Fleet Response Plan, the Navy  has tried to demonstrate the viability of the plan by relying on loosely  linked events that were not part of an overall test and evaluation strategy,  which impairs the Navy’s ability to validate the plan and evaluate its  success over time. In addition, the Navy has not used its lessons learned  system to share the results of its Fleet Response Plan tests or as an  analytical tool to evaluate the progress of the plan and improve  implementation, which limits the Navy’s ability to identify and correct  weaknesses across the fleet.
subsections: section_title: Methodical Tests and Evaluations of New Concepts Are Important
paragraphs: Methodically testing, exercising, and evaluating new doctrines and  concepts is an important and established practice throughout the military.  DOD has long recognized the importance of using tabletop exercises, war  games, and experimentation to explore military doctrine, operational  concepts, and organizational arrangements. Collectively, these tests and  experiments can provide important insight into how systems and  capabilities will perform in actual operations. U.S. Joint Forces Command,  which has lead responsibility for DOD experimentation on new concepts  of operation and technologies, states that its experimental efforts aim to  foster military innovation and improvement by exploring, developing, and  transferring new concepts and organizational ideas into operational  reality.
Particularly large and complex issues may require long-term testing and  evaluation that is guided by study plans. Joint Forces Command’s Joint  Warfighting Center has an electronic handbook that provides guidance for  conducting exercises and lays out the steps in an exercise life cycle:  design; planning; preparation; execution; and analysis, evaluation, and  reports. The Army also has well-established guidance governing service  studies, analyses, and evaluations that the Navy feels is representative of  best practices for military operations research. This provides an important  mechanism through which problems pertaining to critical issues and other  important matters are identified and explored to meet service needs. As  shown in figure 3, the Army’s process involves six major steps that create  a methodical process for developing, conducting, documenting, and  evaluating a study. Following a formal study process enables data  evaluation and development of lessons learned that could be used to build  on the existing knowledge base. In a roundtable discussion with the Fleet  Forces Command on the rationale behind Summer Pulse 2004, the Navy’s  major exercise for the Fleet Response Plan, a senior Navy official stated,  “From the concept, … you need to exercise, … you need to practice, …  you need to demonstrate it to know you got it right and what lessons are  there to learn from how we did it.”
Other governmental agencies, like GAO, and the private sector also rely on  detailed study plans, or data collection and analysis plans, to guide the  development of studies and experiments and the collection and analysis of  data, and to provide a feedback loop that links the outcomes of the study  or experiment event and subsequent analysis to the original goals and  objectives of the study or event. GAO guidance states that data collection  and analysis plans “should carry forward the overall logic of the study so  that the connection between the data that will be collected and the  answers to the study questions will become evident.”
Recent Navy guidance also recognizes the need for a thorough evaluation  of complex initiatives. In April 2005, the Navy issued a Study Planning  and Conduct Guide assembled by the Navy Warfare Development  Command. This guide stresses the importance of establishing a long- range plan for complex and novel problems and lays out the rationale for  detailed study plans for exercises and experiments, as they establish a  structure in which issues are explored and data are collected and analyzed  in relation to the established goals or objectives for the event.  Furthermore, the Navy’s guide notes that random, inadequately prepared  events and a determination just to study the problem do not lead to  successful resolution of problems that may arise in programs and concepts  that the Navy is testing and evaluating.
section_title: Navy Events to Show Viability of the Fleet Response Plan Have Lacked Methodical Testing and Evaluation
paragraphs: The Navy has not methodically conducted realistic tests of the Fleet  Response Plan to demonstrate the plan’s viability and evaluate its progress  and success over time, instead relying on loosely linked events and some  routine data to demonstrate the viability of the plan. The events identified  by the Navy as successful tests of the Fleet Response Plan are Summer  Pulse 2004, the emergency deployment of the U.S.S. Abraham Lincoln,  and Global War on Terrorism Surge 2005, but of these events only Summer  Pulse 2004 was driven by the Fleet Response Plan with the intent of  demonstrating that large numbers of ships could be surged. In addition,  these events were not part of an overall test and evaluation strategy that  yielded specific information from which to assess the value of the plan in  increasing readiness and meeting the new 6 + 2 surge capability goal for  carrier strike groups.
Summer Pulse 2004 encompassed a number of previously scheduled  deployments, exercises, and training events that took place between June  and August of 2004. The intent of Summer Pulse 2004 was to demonstrate  the Fleet Response Plan’s new readiness construct and the Navy’s ability  to deploy multiple carrier strike groups of varying levels of readiness.  However, Summer Pulse 2004 was not a methodical and realistic test of  the Fleet Response Plan for three reasons. First, Summer Pulse 2004 did  not follow best practices regarding study plans and the ability to evaluate  the impact and outcomes of the plan. The Navy did not develop a formal  study plan identifying study objectives, data collection requirements, and  analysis, or produce a comprehensive after-event report describing the  study’s findings. Navy officials have stated that the elements of a formal  study plan were there for the individual deployments, exercises, and  training events constituting Summer Pulse 2004, but were not brought  together in a single package. While the Navy may have had the study  elements present for the individual exercises, they were not directly linked  to testing the Fleet Response Plan. Without such a comprehensive study  plan and overall evaluation, there is no ability to discern potential impacts  on fleet readiness, maintenance, personnel, and other issues that are  critical to the Fleet Response Plan’s long-term success. Second, Summer  Pulse 2004 was not a realistic test because all participating units had  several months’ warning of the event. As a result, five carriers were  already scheduled to be at sea and only two had to surge. Because six  ships are expected to be ready to deploy with as little as 30 days’ notice  under the plan and two additional carriers within 90 days, a more realistic  test of the Fleet Response Plan would include no-notice or short-notice  exercises. Such exercises conducted without advance notification to the  participants would provide the highest degree of challenge and realism.  Without such exercises, the Navy might not be able to realistically practice  and coordinate a full surge deployment. Third, Summer Pulse 2004 was not  a sufficient test because the Navy involved only seven carriers instead of  the eight carriers called for in the plan. Therefore, it did not fully test the  Navy’s ability to meet deployment requirements for the expected force.
Another event cited by the Navy as evidence of the Fleet Response Plan’s  success is the deployment of the U.S.S. Abraham Lincoln carrier strike  group while it was in surge status in October 2004. Originally scheduled to  deploy in the spring of 2005, the Lincoln was deployed early to support  operations in the Pacific Command area of operation and provide aid to  areas devastated by a tsunami in the Indian Ocean in December 2004. Navy  officials said that the Fleet Response Plan enabled the Navy to identify a  carrier to send to the Pacific and to quickly tailor its training package  based on its progressive readiness status. The Navy touted this rapid  response relief work by a strike group deployed during surge status as a  Fleet Response Plan success story. We agree that the Lincoln carrier  strike group was able to respond quickly. However, the extent to which  this event realistically tested the Fleet Response Plan’s expectations for  surging one carrier strike group is not known. As with Summer Pulse 2004,  the Lincoln deployment was not a methodical test of the Fleet Response  Plan because there was no plan to systematically collect or analyze data  that would evaluate the outcomes of the Lincoln deployment against Fleet  Response Plan-related study goals.
The Navy also pointed to a third event, its recent Global War on Terrorism  Surge 2005, as an indicator that the Fleet Response Plan works. The  Global War on Terrorism surge was a response to a request for forces  from which the Navy is looking to glean Fleet Response Plan-related  information about what did and did not work when the ships return.  However, this is not a good test of the Fleet Response Plan because there  is no plan showing what specific data are being collected or what  analytical approaches are being employed to assess the ships’ experiences.  As of September 2005, no other events had been scheduled to further test  and evaluate the Fleet Response Plan.
The Navy has not developed the kind of comprehensive plans to test and  evaluate the Fleet Response Plan as recommended by DOD and Navy  guidance and best practices because Navy officials have stated that  existing readiness reporting processes effectively evaluate the Fleet  Response Plan’s success on a daily basis. They said after-action reports  from training exercises and the Joint Quarterly Readiness Review assist  with this function. Navy officials explained that they implemented the  Fleet Response Plan the same way they had implemented the Inter- Deployment Training Cycle, the predecessor to the Fleet Response Plan’s  Fleet Readiness Training Plan. While this may be true, the Inter- Deployment Training Cycle was focused on the specific training needed to  prepare units for their next deployment, not for implementing a new  readiness construct that emphasized surge versus routine deployments.  Furthermore, the Inter-Deployment Training Cycle did not contain stated  goals whose validity the Navy needed to test. In addition, ongoing  readiness reports do not provide information on important factors such as  costs, long-term maintenance implications, and quality of life issues.
The Summer Pulse 2004, Lincoln surge deployment, and Global War on  Terrorism Surge 2005 testing events were not part of a methodical test and  evaluation approach. Therefore, the Navy is unable to convincingly use  these events to evaluate the Fleet Response Plan and determine whether  the plan has been successful in increasing readiness or achieving other  goals. Moreover, without effective evaluation of the Fleet Response Plan,  the Navy may be unable to identify and correct potential problem areas  across the fleet. Without a comprehensive long-range plan that establishes  methodical and realistic testing of the Fleet Response Plan, the Navy may  be unable to validate the Fleet Response Plan operational concept,  evaluate its progress and success over time, and ensure that it can  effectively meet Navy goals over the long term without any adverse,  unintended consequences for maintenance, quality of life, and fleet  readiness.
section_title: Navy Lessons Learned System’s Repository and Analytic Resources Have Not Been Used to Catalog and Share Fleet Response Plan Lessons
paragraphs: The formal Navy repository for lessons learned, the Navy Lessons Learned  System, has not been used to disseminate Fleet Response Plan-related  lessons learned or to analyze test results to evaluate the progress of the  plan and improve implementation. The Navy Lessons Learned System has  been designated by the Chief of Naval Operations as the singular Navy  program for the collection, validation, and distribution of unit feedback as  well as the correction of problems identified and derived from fleet  operations, exercises, and miscellaneous events. However, there are no  mechanisms or requirements in place to force ships, commands, and  numbered fleet staffs to submit all lessons learned to the Navy Lessons  Learned System, although such mechanisms exist for the submission of  port visit and other reports. For the events that the Navy cites as tests of  the Fleet Response Plan, it did not analyze and evaluate the results and  produce formal lessons learned to submit to the Navy Lessons Learned  System for recordation and analysis. Any evaluation done of the testing  events has not been incorporated into the Lessons Learned System,  preventing comprehensive analyses of lessons learned and identification  of problems and patterns across the fleet that may require a high-level,  Navy-wide response.
Some ship and carrier strike group staff informed us that they prefer  informal means of sharing lessons learned, because they feel the process  through which ships and commands have to submit lessons learned for  validation and inclusion in the database can be complex and indirect. This  may prevent ship and command staffs across the fleet from learning from  the experiences of others, but it also prevents the Navy Lessons Learned  System from performing comprehensive analyses of the lessons learned  and possibly identifying problems and patterns across the fleet that may  require a high-level Navy-wide response. In addition, the lessons learned  are recorded by mission or exercise (e.g., Operation Majestic Eagle) and  not by operational concept (e.g., the Fleet Response Plan), making  identification of Fleet Response Plan-specific lessons learned difficult and  inconsistent.
Over the last 10 years, we have issued several reports related to lessons  learned developed by the military. We have found that service guidance  does not always require standardized reporting of lessons learned and  lessons learned are not being used in training or analyzed to identify  trends and performance weaknesses. We emphasized that effective  guidance and sharing of lessons learned are key tools used to  institutionalize change and facilitate efficient operations. We found that  despite the existence of lessons learned programs in the military services  and the Joint Staff, units repeat many of the same mistakes during major  training exercises and operations. Our current review indicates that the  Navy still does not include all significant information in its lessons learned  database. Therefore, Navy analysts cannot use the database to perform  comprehensive analyses of operational concepts like the Fleet Response  Plan to evaluate progress and improve implementation.
Officials from the Navy Warfare Development Command stated that the  Navy is currently drafting a new Chief of Naval Operations Instruction  governing the Navy Lessons Learned System that will address some of  these issues. Navy Warfare Development Command officials hope that  the new instruction will result in several improvements over the current  system. First, they would like to see a dual reporting system, so that  lessons learned are simultaneously sent to the Navy Lessons Learned  System for preliminary evaluation when they are submitted to the  numbered fleets for validation. This would allow Navy Lessons Learned  analysts to look at unvarnished data for patterns or issues of interest to  the Chief of Naval Operations, without taking away the numbered fleets’  validation processes. In addition, officials would like to establish deadlines  for the submission of lessons learned to ensure timeliness. Not only will  these changes add value to the data stored in the Navy Lessons Learned  System, but they will keep the data flowing while ensuring that data are  actually submitted and not lost as they move up the chain of command.  According to Navy Lessons Learned officials, other branches of the  military already allow operators in the field to submit lessons learned  directly to their lessons learned systems, enabling value-added analysis  and the timely posting of information. By addressing these issues, the  Navy can help ensure that the lessons learned process will become more  efficient, be a command priority, and produce actionable results.
section_title: Conclusions
paragraphs: Two years after implementing a major change in how it expects to operate  in the future, the Navy has not taken all of the steps needed to enable the  Navy or Congress to assess the effectiveness of the Fleet Response Plan.  As the Navy prepares to implement the Fleet Response Plan across the  entire naval force, it becomes increasingly important that the Navy  effectively manages this organizational transformation so that it can  determine if the plan is achieving its goals. The absence of a more  comprehensive overarching management plan to implement the Fleet  Response Plan has left essential questions about definitions, goals,  performance measures, guidance, timelines, milestones, benchmarks, and  resources unanswered, even though sound management practices  recognize the need for such elements to successfully guide activities and  measure outcomes. The absence of these elements could impede effective  implementation of the Fleet Response Plan. Furthermore, without a  comprehensive plan that links costs with performance measures and  outcomes, neither the Navy nor Congress may be able to determine if the  Fleet Response Plan is budget neutral. More effective communications  throughout the fleet using an overall communications strategy could  increase employee awareness of the plan and help ensure successful  implementation.
The Navy also has not developed a comprehensive long-range plan for  testing and evaluating the Fleet Response Plan. Without a well-developed  plan and methodical testing, the Navy may not be aware of all of the  constraints to successfully surging its forces to crises in a timely manner.  Moreover, the absence of an overarching testing and evaluation plan that  provides for data collection and analysis may impede the Navy’s ability to  use its testing events to determine whether the Fleet Response Plan has  been successful in increasing readiness and to identify and correct  problem areas across the fleet. Failure to document and record the results  of testing and evaluation efforts in the Navy Lessons Learned System  could limit the Navy’s ability to validate the value of the concept, identify  and correct performance weaknesses and trends across the fleet, perform  comprehensive analyses of lessons learned, and disseminate these lessons  and analyses throughout the fleet.
section_title: Recommendations for Executive Action
paragraphs: To facilitate successful implementation of the Fleet Response Plan and  enhance readiness and ensure the Navy can determine whether the plan  has been successful in increasing readiness and is able to identify and  correct performance weaknesses and trends across the fleet, we  recommend that the Secretary of Defense take the following two actions:    Direct the Secretary of the Navy to develop a comprehensive overarching  management plan based on sound management practices that will clearly  define goals, measures, guidance, and resources needed for  implementation of the Fleet Response Plan, to include the following  elements:    establishing or revising Fleet Response Plan goals that identify what  Fleet Response Plan results are to be expected and milestones for  achieving these results,    developing implementing guidance and performance measures based  on these goals,  identifying the costs and resources needed to achieve each  performance goal, and    communicating this information throughout the Navy.
Direct the Secretary of the Navy to develop a comprehensive plan for  methodical and realistic testing and evaluation of the Fleet Response Plan.  Such a comprehensive plan should include a description of the following  elements:    how operational tests, exercises, war games, experiments,  deployments, and other similar events will be used to show the  performance of the new readiness plan under a variety of conditions,  including no-notice surges;    how data will be collected and analyzed for these events and  synthesized to evaluate program success and improvements; and    how the Navy Lessons Learned System will collect and synthesize  lessons from these events to avoid repeating mistakes and improve  future operations.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD generally concurred  with our recommendations and cited actions it will take to implement the  recommendations.
DOD concurred with our recommendation that the Navy should develop a  comprehensive overarching management plan based on sound  management practices that would clearly define the goals, measures,  guidance, and resources needed for successful implementation of the Fleet  Response Plan, including communicating this information throughout the  Navy. DOD noted that the Navy has already taken action or has plans in  place to act on this recommendation, and described several specific  accomplishments and ongoing efforts in this regard. DOD also noted that  the Navy intends to communicate through message traffic, white papers,  instructions, lectures, and meetings with Navy leadership. We agree that  these means of communication are an important part of an effective  communication strategy; however, we do not believe that these methods  of communication constitute a systemic strategy to ensure communication  at all personnel levels. We believe the Navy would benefit from a  comprehensive communication strategy that builds on its ongoing efforts,  but encompasses additional actions to ensure awareness of the plan  throughout the Navy.
DOD partially concurred with our recommendation to test and evaluate  the Fleet Response Plan. DOD noted that it plans to use a variety of events  and war games to evaluate the Fleet Response Plan, but it does not see a  need to conduct no-notice surges to test the Fleet Response Plan. DOD  stated that it believes no-notice surges are expensive and unnecessary and  could lead to penalties on overall readiness and the ability to respond to  emergent requirements. DOD also noted that the Navy has surged single  carrier strike groups, expeditionary strike groups, and individual ships or  units under the Fleet Response Plan, and it cited several examples of such  surges. We commend the Navy’s plans to use a variety of events to  evaluate the Fleet Response Plan and its use of the Navy Lessons Learned  System to report and evaluate the lessons learned in the Global War on  Terrorism Surge 2005 exercise held earlier this year. However, we  continue to believe that no-notice surges are critical components of  realistic testing and evaluation plans and that the benefits of such  exercises can outweigh any additional costs associated with conducting  such tests on a no-notice basis. Both we and Congress have long  recognized the importance of no-notice exercises. For example, in a 1989  report, we noted that DOD was instituting no-notice exercises to assess  the preparedness of combatant commands’ state of training of their staffs  and components. In addition, in 1990 the Department of Energy  conducted no-notice tests of security personnel in response to our work  and out of recognition that such tests are the best way to assess a security  force’s ability at any given time. Furthermore, in recent years, the  Department of Homeland Security, Department of Energy, and others have  conducted no-notice exercises because they add realism and demonstrate  how well organizations are actually prepared to respond to a given  situation. Despite the importance of no-notice exercises, the Navy has not  conducted no-notice exercises to test and evaluate the centerpiece surge  goal of 6 + 2 for carrier strike groups. We believe that the smaller surges  cited by DOD can provide insights into the surging process, but we do not  believe that such surges can effectively test the Navy’s readiness for a full  6 + 2 carrier strike group surge.
DOD also provided technical and editorial comments, which we have  incorporated as appropriate. DOD’s comments are reprinted in appendix II  of this report.
We are sending copies of this report to other interested congressional  committees; the Secretary of Defense; the Secretary of the Navy; and the  Director, Office of Management and Budget. We will make copies available  to others upon request. In addition, the report will be available at no  charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-4402 or stlaurentj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the extent to which the Navy has employed a sound  management approach in implementing the Fleet Response Plan, we  interviewed Navy headquarters and fleet officials; received briefings from  relevant officials; and reviewed key program documents. In the absence of  a comprehensive planning document, we compared best practices for  managing and implementing major efforts to key Navy messages,  directives, instructions, and briefings, including, but not limited to, the  Culture of Readiness message sent by the Chief of Naval Operations  (March 2003); the Fleet Response Concept message sent by the Chief of  Naval Operations (May 2003); the Fleet Response Plan Implementation  message sent by the Commander, Fleet Forces Command (May 2003); the  Fleet Response Plan Implementation Progress message sent by the  Commander, Third Fleet (September 2003); and the U.S. Fleet Forces  Command’s Fleet Training Strategy instruction (May 2002 and an undated  draft). We also conducted meetings with several of the commanding  officers, executive officers, and department heads of selected carrier  strike groups, aircraft carriers, and air wings to obtain information on how  the plan had been communicated, how the plan had changed their  maintenance and training processes, the impact on their quality of life, the  cost implications of the plan, and other factors.
To assess the extent to which the Navy has tested the effectiveness of the  Fleet Response Plan and shared results to improve its implementation, we  obtained briefings; interviewed Navy headquarters and fleet officials; and  reviewed test and evaluation guidance for both the Navy and other federal  agencies. To evaluate the three Fleet Response Plan demonstrations  identified by the Navy, we interviewed officials from the Fleet Forces  Command and the Navy Warfare Development Command, reviewed  existing documentation on the demonstrations, queried the Navy Lessons  Learned System for lessons learned from the demonstrations, and  compared our findings to accepted best practices for tests and  evaluations. Further, we reviewed Navy Lessons Learned System  instructions and queried the system to determine recorded lessons learned  pertaining to the Fleet Response Plan.
We validated the Navy Lessons Learned System data and determined the  data were sufficiently reliable for our analysis. We conducted our review  from January 2005 through August 2005 in accordance with generally  accepted government auditing standards at the following locations:    The Joint Staff, Washington, D.C.    U.S. Pacific Command, Camp H. M. Smith, Hawaii    Offices of the Chief of Naval Operations, Washington, D.C.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Richard Payne, Assistant Director;  Renee Brown; Jonathan Clark; Nicole Collier; Dawn Godfrey; David  Marroni; Bethann Ritter; Roderick Rodgers; John Van Schaik; and Rebecca  Shea made significant contributions to this report.