Military Readiness: Navy Is Making Progress Implementing Its Fleet Response Plan, but Has Not Fully Developed Goals, Measures, and Resource Needs
Published: 2008-02-01T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Navy initiated its Fleet Response Plan (FRP) in March 2003 as a critical enabler to help meet the new demands of the twenty-first century security environment. FRP represents a major change in the way the Navy manages its forces, and is intended to more rapidly prepare and then sustain readiness in ships and squadrons. To achieve the desired capabilities under FRP, the Navy has altered its training, maintenance, and manning practices. The Navy expects this new readiness approach will enable its forces to provide presence and engagement in forward areas, as well as surge a greater number of ships on short notice. The Navy intends to achieve this goal without increasing its operations and maintenance budget of about $40 billion for each of the next 5 years. However, GAO previously reported that the Navy had not fully incorporated a sound management approach to guide and assess implementation of FRP. As a result, GAO was asked to examine the extent to which the Navy has (1) made progress in implementing a sound management approach for FRP and (2) evaluated the long-term risks and tradeoffs of FRP-related changes.
section_title: What GAO Found
paragraphs: The Navy has taken several positive steps toward implementing a sound management approach for FRP, but has not developed implementation goals, fully developed performance measures, or comprehensively assessed and identified the resources required to achieve FRP goals. GAO's prior work has shown that key elements of a sound management approach include: defining clear missions and desired outcomes, establishing implementation goals, measuring performance, and aligning activities with resources. The Navy has made progress in implementing FRP since GAO's prior reports. For example, it has established a goal of having three carrier strike groups deployed, three ready to deploy within 30 days of being ordered to do so, and one more within 90 days (referred to as 3+3+1). The Navy also has established a framework to set implementation goals for all forces, established some performance measures that are linked to the FRP phases, and begun efforts to identify needed resources. However, the Navy has not yet established a specific implementation goal for expeditionary strike groups and other forces. In addition, the Navy has not fully developed performance measures to enable it to assess whether carrier strike groups have achieved adequate readiness levels to deploy in support of the 3+3+1 goal. Moreover, the Navy has not fully identified the resources required to achieve FRP goals. Until the Navy's management approach fully incorporates the key elements, the Navy may not be able to measure how well FRP is achieving its goals or develop budget requests based on the resources needed to achieve expected readiness levels. The Navy has not fully considered the long-term risks and tradeoffs associated with the changes made as FRP has been implemented, such as carrier operational and maintenance cycles and force structure. The Navy has extended the intervals between carrier dry-dock maintenance periods from 6 years to 8 years and begun a test program that will extend some carrier dry-dock intervals to as much as 12 years, and it has lengthened operational cycles for carriers and their airwings to 32 months. GAO previously advocated that the Department of Defense adopt a risk management approach to aid in its decision making that includes assessing the risks of various courses of action. However, the Navy has not fully considered the long-term risks and tradeoffs of these recent changes because it has not performed a comprehensive assessment of how the changes, taken as a whole, might affect its ability to meet FRP goals and perform its missions. In addition, while the Navy has developed force structure plans that include two upcoming periods when the number of available aircraft carriers temporarily drops from 11 to 10, the plans included optimistic assumptions about the length of the gaps and the availability of existing carriers and did not fully analyze how the Navy would continue to meet FRP goals with fewer carriers. Until the Navy develops plans that use realistic assumptions and accurately identify the levels of risk the Navy is willing to accept during these gap periods, senior Navy leadership may not have the information it needs to make informed trade-off decisions.

section_title: Letter
paragraphs: The twenty-first century security environment has created new demands  for Navy forces, particularly with continued operations in support of the  Global War on Terrorism. One initiative that the Navy is continuing to  implement and mature to help meet these demands is its Fleet Response  Plan (FRP). In March 2003, the Chief of Naval Operations initiated FRP to  more rapidly prepare and then sustain readiness in ships and squadrons.  Fleet Forces Command, the Navy command that holds principal  responsibility for managing fleet personnel, training, requirements,  maintenance, and operations, began to implement FRP in May 2003. To  achieve the desired capabilities under FRP, the Navy has altered prior  training, maintenance, and manning practices. The Navy expects this new  readiness approach will enable its forces to provide not only presence and  engagement in forward areas, but also surge a greater number of ships on  short notice. The Navy’s operations and maintenance budget is projected  to remain at about $40 billion in constant dollars over the next 5 years.
The Fleet Response Plan modifies the Navy’s pre-2001 deployment pattern,  replacing 6-month routine deployments with more flexible deployment  options intended to enable the quick response of naval forces to crises,  conflicts, or homeland defense needs. FRP set specific requirements for  carrier strike groups, which typically include an aircraft carrier with an  airwing, two destroyers, a frigate, cruiser, and support ship, and possibly,  depending on the mission, a submarine. In June 2007, the Navy began  extending FRP to expeditionary strike groups, which may include  amphibious ships with a Marine Expeditionary Unit, aircraft, and landing  craft; surface combatants such as a cruiser, a destroyer, and a frigate; and  possibly, depending on the mission, a submarine and a land-based  maritime patrol aircraft.
We have previously reported on the Navy’s implementation of FRP. In  November 2005, we reported that while FRP represents a major change in  the way the Navy manages its forces, it had not fully incorporated a sound  management approach needed to guide and assess implementation.  Moreover, the Navy had not fully tested and evaluated FRP or developed  lessons learned to identify the effectiveness of its implementation. In 2005,  we recommended that the Navy develop a comprehensive management  plan with goals and performance measures to facilitate implementation of  FRP and that it develop a comprehensive testing and evaluation plan to  help determine whether FRP has been successful. The Department of  Defense (DOD) generally agreed with our recommendations, citing several  actions it had underway or planned. In 2004 we examined the maintenance  impacts of FRP, and we observed that FRP did not shorten aircraft carrier  maintenance intervals and that the impact of FRP on the Navy’s  maintenance budget was unknown.
The John Warner National Defense Authorization Act for Fiscal Year 2007  required us to report on a variety of issues pertaining to FRP management,  readiness, and program expansion.  These issues included: the Navy’s  management approaches to implementing FRP; the adequacy of Navy  directives and guidance with respect to maintenance and training  requirements and procedures; the adequacy of the Navy’s evaluation  criteria for the plan; the Navy’s progress in identifying the amount of  funding required to effectively implement the operations and maintenance  requirements of the plan and the effect of providing less funding the  identified amount; Navy data on aircraft carriers, destroyers, and cruisers  that participated in the plan with respect to readiness, response time, and  availability for routine or unforeseen deployments; and any  recommendations pertaining to expanding the plan to include  expeditionary strike groups.
In April 2007, we briefed your offices on our preliminary observations.  This report updates the information provided in the briefing and provides  further detail based on the work we have completed since that time.  Specifically, our objectives for this report were to assess the extent to  which the Navy has (1) made progress in implementing a sound  management approach for FRP, and (2) evaluated the long-term risks and  tradeoffs of FRP-related changes.
To assess the Navy’s progress in developing a sound management  approach, we reviewed and analyzed guidance and instructions on  manning, maintenance, and training; and key studies, messages, and  planning documents and compared these to best practices for managing  and implementing major efforts identified in our prior reports. We also  interviewed officials in the offices of the Chief of Naval Operations and the  Chairman of the Joint Chiefs of Staff; Fleet Forces Command and Pacific  Fleet; Naval Air and Sea Systems Commands; and Marine Corps Forces  Command to obtain information about initiatives that the Navy has  undertaken. To assess the extent to which the Navy has evaluated the  long-term risks and tradeoffs of FRP-related changes, we analyzed  historical and long-range carrier maintenance schedules, analyzed the  Navy’s plans for meeting FRP goals during periods when only 10 aircraft  carriers are available, and interviewed Navy readiness officials in the  offices of the Chief of Naval Operations and Fleet Forces Command. We  addressed both of these objectives for both carrier and expeditionary  strike groups. We conducted our review from November 2006 through  October 2007 in accordance with generally accepted government auditing  standards. The scope and methodology used in our review are described in  further detail in appendix I.
subsections: section_title: Background
paragraphs: Prior to the September 11, 2001, terrorist attacks, only those Navy ships  and air squadrons at peak readiness were deployed overseas, usually for 6  months at a time. Most of the Navy’s remaining units were not available  because they were in the early stages of their maintenance or training  cycles, or because the Navy did not have good visibility of the readiness of  these units. This prompted the Chief of Naval Operations in March 2003 to  task the Commander of Fleet Forces Command to develop the FRP  concept to enhance the Navy’s surge capability. The Chief of Naval  Operations approved the concept and directed the Commander of Fleet  Forces Command to be responsible and accountable for effectively  implementing the plan. The Commander of Fleet Forces Command is  responsible for overall coordination, establishment, and implementation of  integrated requirements and policies for manning, equipping, and training  both Atlantic and Pacific fleet units throughout the training cycle and is  responsible for articulating all fleet warfighting and readiness  requirements to the Chief of Naval Operations.
subsections: section_title: Composition of Carrier and Expeditionary Strike Groups
paragraphs: Carrier strike groups are typically centered around an aircraft carrier and  its airwing, and also include a guided missile cruiser; two guided missile  destroyers; a frigate; an attack submarine; and one or more supply ships  with ammunition, fuel, and supplies (such as food and spare parts). The  Navy currently has 11 aircraft carriers in service, with two additional  carriers under construction. (The carriers are listed in app. III.) Generally,  three carrier strike groups are deployed at any given time. The three  deployed carriers include the USS Kitty Hawk, which is home ported in  Japan and is counted as being continuously deployed. This carrier  provides most of the U.S. naval presence in the western Pacific Ocean  region and some in the Indian Ocean/Arabian Sea region. Carriers  originating from the eastern and western United States have traditionally  provided presence in the Mediterranean Sea and Indian Ocean/Arabian Sea  regions, respectively. There are also 10 carrier airwings, one of which  accompanies each deployed carrier. While the composition of each  airwing can be tailored to the carrier’s specific mission, each airwing  typically includes a helicopter squadron as well as squadrons of aircraft  for attack (composed of aircraft such as the F/A-18), electronic warfare  (composed of aircraft such as the EA-6B), and reconnaissance (composed  of aircraft such as the E-2C) missions. The Navy cites a variety of roles in  which carrier strike groups may be employed, for example they (1) are  deployed worldwide in support of U.S. interests and commitments; (2)  respond to global crises from peacetime to full-scale war; (3) can operate  as the cornerstone of joint/allied maritime expeditionary forces in times of  crisis; and (4) can operate and support aircraft attacks on enemies, protect  friendly forces, and engage in sustained independent operations of war.
Expeditionary strike groups are typically centered on amphibious ships  with a Marine Corps Marine Expeditionary Unit, aircraft, and landing craft.  Each expeditionary strike group notionally includes one amphibious  assault ship, one amphibious transport dock ship, and one dock landing  ship. These amphibious ships together can embark a Marine Expeditionary  Unit consisting of about 2,200 Marines, their aircraft, their landing craft,  their combat equipment, and about 15 days worth of supplies. Like a  carrier strike group, the expeditionary strike group may also include  several surface combatants such as a cruiser, destroyer, and frigate; an  attack submarine; and one or more P-3 long-range, land-based, maritime  patrol aircraft. Expeditionary strike groups are designed to be  independently deployable, strike-capable naval formations, but they can  also operate in conjunction with carrier strike groups to form larger naval  task forces. Generally, two or three expeditionary strike groups are  forward-deployed at any given time.
section_title: FRP Cycle
paragraphs: FRP represents a change in the way the Navy manages its forces. The plan  changes the manner in which the Navy maintains, trains, mans, and  deploys its ships to allow a greater number of ships to surge on short  notice while at the same time meeting forward- presence requirements.  Four phases within the FRP cycle serve as the framework to more rapidly  prepare and sustain the readiness of ships, aircraft, and personnel. As  depicted in figure 1, the four FRP phases are (1) basic, or unit-level  training; (2) integrated training; (3) sustainment (which may include one  or more extended periods of deployment); and (4) maintenance.
At the end of the basic phase, a unit is characterized as an “independent  unit ready for tasking” and may be assigned independent operations in  support of homeland security, counternarcotics missions, or assigned to  provide disaster relief or humanitarian assistance. As the training  progresses, the capabilities of the units increase accordingly as do the  roles and missions assigned. Once the basic phase is completed and the  integrated phase begins, training can be tailored to meet a combatant  commander’s request for a specific capability, such as to support  antipiracy operations, and the unit is characterized as “maritime security  surge” capable. Upon completion of the integrated phase, a unit begins the  sustainment phase and is characterized as “major combat operations  surge” capable, meaning the unit is ready for operational employment, but  is not necessarily able to lead combat force operations. Once a unit is  certified through advanced integrated training and is fully capable of  conducting all forward-deployed operations, it attains the status of “major  combat operations ready.” Routine deployments occur during the  sustainment phase. Finally, ships spend time in maintenance phase, when  major shipyard or depot-level repairs, upgrades, and modernization occur.  The Fleet Forces Command and Pacific Fleet commanders have delegated  responsibility to their subordinate force commanders (such as air,  submarine, and surface) for overseeing the FRP’s basic phase; and they  have delegated their geographical fleet commanders (such as the Seventh  Fleet, which operates in the Western Pacific) responsibility for overseeing  the integrated and sustainment phases.
section_title: The Navy Has Taken Several Positive Steps, but Has Not Fully Developed Goals, Measures, and Resource Needs
paragraphs: The Navy has taken several positive steps toward implementing a sound  management approach for FRP, but has not fully developed such an  approach because the Navy’s implementation of FRP is still evolving. The  Navy’s implementation of FRP has included some key elements of a sound  management approach, such as defining clear missions and desired  outcomes, developing some performance measures, and beginning to  identify needed resources. However, the Navy has not fully incorporated  certain key elements, such as fully developing implementation goals or  performance measures and aligning activities with resources.
subsections: section_title: The Navy Has Taken Several Positive Steps to Implement a Sound Management Approach for FRP
paragraphs: The Navy has taken several positive steps toward implementing a sound  management approach for FRP since our prior reports. Our prior work has  shown that key elements of a sound management approach include  defining clear missions and desired outcomes, establishing  implementation goals, measuring performance, and aligning activities with  resources. The Navy’s implementation of FRP has included some of these  elements. For example, the Navy has issued two FRP instructions, one in  August 2006 and another in August 2007. The August 2007 instruction  defined the FRP mission as providing ready Navy forces to meet  combatant commanders’ requests for forces in support of the nation’s  maritime security. These forces consist of forward operating and U.S.- based assets that may rotationally deploy or surge. Both instructions also  defined the four FRP phases, defined notional lengths for each phase, and  established responsibility for oversight and execution of the plan with  Fleet Forces and Pacific Fleet commanders and their subordinate force  and geographic fleet commanders.
The Navy has also updated instructions in the areas of personnel,  maintenance, and training to reflect desired outcomes under FRP. For  example, in January 2007, the Navy revised its personnel instruction that  provided guidelines on personnel deployment length under FRP. The new  instruction sets limits on the length of deployments, dwell times between  deployments, and requirements for time spent in home port. The Chief of  Naval Operations must grant a waiver if those limits are exceeded. In the  absence of a waiver, deployments are limited to 7 months for units with a  single deployment and 6 months for units with multiple deployments  within the same FRP cycle. In addition, units must spend at least as much  time between any two deployments as they did on their most recent  deployment; and units must spend a minimum of 50 percent of the time in  their home port over an FRP cycle. With regard to maintenance, the Navy  has revised notional depot maintenance schedules to reflect the FRP  cycle, and developed guidelines on out-of-depot continuous maintenance.  With regard to training, the Navy revised surface, carrier, and air squadron  training readiness instructions to reflect changes in training phases and to  show the exercises that are expected to be completed in each phase.
In addition, the Navy established a 6+1 implementation goal for carrier  strike groups, meaning that it aims to have six carrier strike groups  available to deploy within 30 days and one more within 90 days. In its  August 2007 instruction, the Navy modified this goal to 3+3+1 and linked  the goal to the FRP phases. As that instruction noted, 3+3+1 means that  the Navy’s goal is to have three carrier strike groups deployed, three ready  to deploy within 30 days of being ordered to do so (in the FRP sustainment  or integrated phases), and one prepared to deploy within 90 days (in the  FRP basic phase). The Navy plans to use this same framework, which took  the form X+Y+Z, to set implementation goals for all forces. The first  variable (X) refers to the number of ships or aircraft deployed, the second  (Y) refers to the number that will be in a surge status, and (Z) refers to the  number that will be in a Ready for Tasking status.
Furthermore, the Navy has developed some performance measures to use  in evaluating its performance under FRP. In its August 2007 instruction,  the Navy established some performance measures for each FRP phase.  For example, two principal measures will be used to assess performance  in the maintenance phase: on-time completion of maintenance periods and  assessment scores for the level of maintenance completed. The  instruction also delineates performance measures for training in FRP  phases. The array of performance measures spelled out in the instruction  broke new ground by reaching across the entire FRP cycle with measures  that can be associated with a numeric goal and evaluated over time. For  example, one measure of maintenance efficiency was “on-time completion  of maintenance,” and the Navy associated the measure with a numeric goal  of delivering ships within 30 days of the scheduled completion date. Lastly,  the Navy’s aviation community has developed the Aviation Readiness  Integrated Improvement Program training and readiness funding profiles.  That program has developed a matrix that sets out expected readiness  levels for aircraft squadrons, expressed in terms of the current readiness  reporting system’s 5-point scale (with 1 being the highest and 5 the  lowest), by squadron type and month of the FRP cycle. Available  resources are allocated based on a tiered readiness profile, with the  highest priority given to deployed units and units in pre-deployment  sustainment. For example, an F-18 A/B/C/D squadron could expect 50  percent of required funding in the first month of its maintenance phase in  order to achieve a training readiness level of 2.9, compared to 80 percent  of required funding in the fifth month of sustainment phase (which  corresponds to the first month of a typical deployment) in order to achieve  a training readiness level of 1.9.
The Navy has also been working on other initiatives that include  identifying and validating the cost of the fleet response training plans,  calculating the notional training costs of a carrier strike group through the  FRP phases, and developing metrics to link operations and maintenance  costs for current and future years to FRP implementation goals.
section_title: The Navy Has Not Developed Comprehensive Implementation Goals, Fully Developed Performance Measures, or Fully Identified Resources Required to Achieve FRP Goals
paragraphs: The Navy has not developed comprehensive implementation goals for all  Navy assets, finished developing performance measures, or fully identified  all the resources required to achieve FRP goals– which are all key  elements of a sound management approach. Specifically, the Navy has not  yet established a specific implementation goal for expeditionary strike  groups and other forces that is analogous to the 3+3+1 goal for carrier  strike groups. In addition, the Navy has not fully developed performance  measures because it has not established required readiness levels that  would enable it to measure whether the carrier strike groups can meet the  3+3+1 goal, and existing performance measures also lack some details  about acceptable levels of performance and scope. Furthermore, the Navy  has not shown that it has identified all resources required to achieve FRP  goals or fully aligned needed resources with FRP activities.
subsections: section_title: The Navy Has Not Established Implementation Goals for Expeditionary Strike Groups or Other Forces
paragraphs: The Navy has not yet established implementation goals for expeditionary  strike groups and other forces. While the Navy began extending FRP to  expeditionary strike groups in June 2007, it has not developed an  implementation goal for expeditionary strike groups that would be  analogous to the 3+3+1 goal for carrier strike groups because it has only  recently begun to apply FRP to expeditionary strike groups. Marine Corps  officials view FRP’s goal of promoting increased readiness of amphibious  ships as promising, so long as it does not have the effect of reducing the  forward presence of the Marine Expeditionary Units that embark on those  ships. The Navy has also applied the FRP concept to ship configurations  that operate outside the umbrella of either type of strike group, for both  missions related to the Global War on Terrorism and relief efforts, such as  Hurricane Katrina. However, the Navy has not established specific FRP  implementation goals for these other forces, other than a general goal to  have some forces deployed, some preparing for deployment, and some in  the basic phase of FRP. Navy officials agree that specific FRP goals for  these other forces are needed, and officials at Fleet Forces Command said  that preliminary goals have been developed for submarines, surface ships,  and other types of forces. However, these goals had not been finalized at  the time of our review and are not yet being used to guide FRP  implementation. Moreover, the preliminary goal for surface ships was  expressed in a single aggregated figure that included destroyers, cruisers,  frigates, amphibious ships, and minesweepers. An aggregate figure may  not provide a meaningful goal, though, as some ships, such as destroyers,  may be used in a variety of roles: as components of carrier or  expeditionary strike groups or as part of smaller task forces. Until the  Navy develops implementation goals for all force configurations, it will not  know whether it has enough ships to be distributed among all possible  roles. Furthermore, without goals that establish how many expeditionary  strike groups and other forces are needed to be ready to deploy, the Navy  may not be able to determine whether it can provide all desired  capabilities under FRP.
section_title: The Navy Has Not Fully Developed FRP Performance Measures
paragraphs: Although the Navy has developed some performance measures in its 2007  FRP instruction and linked these measures to the four FRP phases, its  performance measures are not fully developed for two reasons. First, the  Navy has not established required readiness levels in its current readiness  system that would enable it to measure whether the carrier strike groups  are ready to deploy in support of the 3+3+1 goal. The Navy currently  reports unit readiness in terms of the Status of Resources and Training  System (SORTS), but DOD plans to transition to a new system in 2009.  SORTS uses a 5-point scale to assess units’ ability to meet the full range of  their wartime missions, including major combat operations. Overall  SORTS scores, as well as individual scores in each of five subcategories,  are presented to the Chief of Naval Operations in weekly and monthly  readiness briefings, and readiness is also reported to Congress quarterly,  as required by law.  However, the Navy has not defined what overall  readiness levels are required to move ships from one FRP phase to  another. The Navy has established one phase-specific overall readiness  measure. Units must reach an overall readiness level of C-2, which is the  next-to-highest level, by the end of the integrated phase of FRP. The Navy  has also specified that units must achieve a certain training readiness level  by the end of the basic phase of FRP, but an overall readiness level has not  been specified. Moreover, although units are supposed to be able to  perform major combat operations while in the sustainment phase, the  Navy has not established a readiness level that would enable it to measure  when a unit was ready to transition from surge- to deployment-ready  status. While some Navy officials have said that they believe that the  requirement for deploying beyond the continental United States— regardless of mission or FRP phase—is C-2, they were unaware of any  formal guidance to that effect.
The Navy relies on institutional knowledge and the commanders’ judgment  to make the determination of when a unit such as a carrier strike group is  ready to deploy. Further, the Navy has not fully defined required readiness  levels because Navy officials are reluctant to invest in developing  measures based on SORTS, a legacy system that DOD will discontinue as it  introduces the new Defense Readiness Reporting System by the middle of  fiscal year 2009. Officials plan to continue to report readiness information  to Congress using SORTS until the new system is fully in place. Under the  new system, the Secretary of the Navy is charged with defining the  mission-essential tasks that will be used to assess units’ readiness for a  range of potential missions. However, these tasks have not yet been fully  defined. Without a clear requirement, senior leadership may not be able to  determine whether implementation goals are being met, and might instead  use imprecise proxy measures—such as the raw number of carriers that  are not in depot maintenance—as a way of estimating how many carrier  strike groups are available for deployment. Moreover, planners may not  know what required readiness level to use as a standard when developing  budget requests and making resource allocation decisions.
Second, the August 2007 FRP instruction outlined several performance  measures for specific tasks, but some of these measures lacked details  about acceptable levels of performance and scope. Navy guidance directs  that each complete standard for determining whether a task can be  accomplished should consist of one or more measures as well as a  criterion, or quantitative description of the acceptable level of  performance, for each measure. For example, a measure might be “on-time  maintenance,” and its associated criterion might be “within 30 days of  promised completion date.” However, in some cases the measures  described in the FRP instruction lacked criteria. For example, the  instruction contained five measures of basic phase performance, but  none of these measures specified criteria about how well units must  perform. Although information for one of the measures (the M-rating—  required training readiness under the current readiness reporting system)  could be inferred from Navy training manuals, the other four measures  lacked quantitative descriptions. For example, the “cost performance”  measure did not provide a quantitative description (e.g., percentage of  budgeted funds expended) or a numeric goal (e.g., 95 percent).
The FRP instruction also did not fully address the scope of some of the  performance measures. For example, the maintenance phase performance  measure dealing with on-time maintenance completion did not indicate  whether it was applicable to aircraft as well as ships; nor did four of the  five basic phase performance measures. In addition, while FRP changes  the manner in which the Navy maintains, trains, mans, and deploys its  ships, the instruction outlines performance measures pertaining to  maintenance, training, and deployment, but it did not contain performance  measures pertaining to manning. Navy officials have said that they believe  there is a goal of fully manning all ships and aircraft at all times. Although  this goal has appeared in various Navy briefings, it is not documented in  official Navy guidance. Navy officials observed that FRP is still an evolving  construct, and stated that the Fleet Readiness Enterprise, which includes  representatives from training and readiness directorates, is developing  appropriate criteria that will delineate acceptable levels of performance.  Moreover, a Navy official noted that work that will provide further details  for the established performance measures is underway at lower levels of  command. However, until the FRP performance measures are fully linked  to quantified levels of acceptable performance, encompass ships and  aircraft, as applicable, and provide measures to assess appropriate  manning levels, decision makers may be unable to determine the extent to  which FRP is achieving its goals.
section_title: The Navy Has Not Fully Identified Resources Needed to Implement FRP
paragraphs: The Navy also has not fully identified all resources required to achieve  FRP goals or aligned needed resources with FRP activities. In the past, the  Navy sponsored several studies to identify FRP costs, but these are of  limited utility to the Navy’s understanding of links between resources and  FRP activities because they included a goal that FRP would be cost- neutral, relied on assumptions that are now outdated, and lacked other  details. The Navy’s principal studies consequently did not reflect a  bottom-up assessment of the resources needed to implement FRP and  achieve certain required readiness levels. Additionally, these studies used  assumptions that are now outdated, such as a 12-carrier fleet, the former  “6+2” carrier strike group implementation goal, and a 27-month carrier  strike group operational cycle. These assumptions affect the proportion of  time that carriers spend in maintenance over their life cycles and the  urgency with which any single carrier might have to be readied to surge,  either of which would have resource implications. In addition, the studies  did not analyze the cost impact of relying more heavily on pier-side  continuous maintenance rather than depot maintenance, deploying for  more than 6 months at a time or more than once in the same operational  cycle, or preparing for an actual surge. In case of a surge, for example,  several ships might need to be readied within the same 30-day period,  imposing additional costs on shipyards, and aircraft might need to be  transferred from one airwing to another, raising transportation costs.
More recently, the Navy established several task forces and initiatives to  help identify training requirements and costs, and to link costs to expected  readiness levels. However, these efforts are ongoing and have not yet  produced a comprehensive approach for aligning activities with resources.  For example, the Navy has established at least two bodies, the Fleet  Training Board of Directors and a Task Force on Readiness, whose work  should help the Navy align its FRP activities with its resources and funding  needs. The Fleet Training Board is tasked with establishing a process for  calculating and tracking fleet training costs, which involves validating all  costs associated with training as well as identifying excesses and gaps in  the training process. Although the Fleet Training Board has defined time  frames for presenting briefings to senior Navy leadership, its charter was  still in draft as of September 2007. The Navy’s Task Force on Readiness,  whose charter was approved in April 2007, aims to develop a  comprehensive set of metrics linked to cost in order to inform current- year execution of readiness objectives and future-year planning,  programming, and budgeting of readiness requirements. This linkage will  allow the Navy to identify the operations and maintenance funding needed  to execute various FRP implementation goals, such as the carrier strike  group’s goal of 3+3+1, at specific readiness levels. However, the task force  has not yet shown how the three variables–funding, implementation goals,  and readiness levels–would interact and how a change in one variable  would affect the outcomes for the other two. The task force on readiness  has set a goal of having its results incorporated into the development of  the fiscal year 2010 budget. However, it is not clear how the results of this  task force will be disseminated or whether they will be used to develop  formal guidance that could be used to develop future budgets. While the  groups are taking steps in the right direction, without a thorough analysis  of the costs of FRP that is based on updated information about force  structure, goals, and operations and maintenance cycles and that links  resource inputs to expected readiness outputs, the Navy may be unable to  identify what resources would be needed to achieve the intended benefits  under FRP and to develop its budget requests to reflect those needs.
section_title: The Navy Has Not Fully Considered Long-term Risks and Tradeoffs of Changes
paragraphs: The Navy has not fully considered the long-term risks and tradeoffs  associated with the changes it has made in areas such as carrier  operational and maintenance cycles and force structure. Specifically, the  Navy has made several changes in its maintenance and operational cycles  to facilitate FRP; however, the Navy has not performed an integrated  assessment of how the changes would, taken as a whole, affect its ability  to meet FRP goals and perform its full range of missions. Moreover,  although the Navy has developed plans to show how it would continue to  meet FRP goals during two upcoming periods when the number of  available aircraft carriers temporarily drops from 11 to 10, these plans do  not consider several issues.
subsections: section_title: The Navy Has Not Fully Assessed Changes in Carrier Maintenance and Operational Cycles
paragraphs: The Navy has made several changes to maintenance and operational  cycles to facilitate FRP, but it has not fully assessed the implications of  these changes. Specifically, the Navy has extended the intervals between  carrier dry-dock maintenance periods from 6 years to 8 years and begun a  test program that will extend some carrier dry-dock intervals to as much  as 12 years, or only three times during their life cycles. It has also  lengthened operational cycles for carriers and their airwings to 32 months.  We have previously advocated that DOD adopt a comprehensive risk  management approach to aid in its decision making that includes, among  other things, assessing the risks of various courses of action for both near-  and long-term challenges.
Prior to making changes to its maintenance cycle in the past, the Navy has  conducted assessments of the potential effects of the changes. For  example, when the Navy altered its previous system for scheduling depot  maintenance about a decade ago, the engineering community conducted  a formal study to determine which types of maintenance could be  performed at greater intervals without having a negative impact on the  integrity of the carrier or its expected total service life. A similar study was  completed in 2005 and used as the basis for revising the notional depot  maintenance schedule the following year to extend the dry-docking cycle  from 6 to 8 years. At that time, officials concluded that the proposed  extension was technically acceptable and contained a manageable level of  risk. Since then, the Navy has again extended the dry-docking cycle, with a  12-year cycle planned for the Nimitz as a test case and possible further  extensions for other carriers on a case by case basis. However, the  Nimitz, which is the oldest ship in its class, has spent more time in depot  maintenance during the first half of its service life than is planned for  newer ships and therefore might be an atypical example of the class.  Officials have stated that, while they will not conduct a comprehensive  study of the entire class of ships, they are confident that their test study of  the Nimitz will suffice to collect the data they need to inform their  decision as to whether to extend the cycle for other ships.
The Navy has also extended the carrier operational cycle from the pre-FRP  27 months to 32 months. Operational cycles were extended to 32 months  in tandem with the extension of carrier dry-docking cycles, and the  technical studies that were performed at the time to determine the effect  on carriers applied to both operational and maintenance cycles. However,  these studies did not examine the full impact on carriers’ accompanying  airwings, even though an effect of the extension of the operational cycle is  a corresponding increase in the air squadron operational cycle. As a result  of the extension of the operational cycle, which may now last as long as 32  months, a Navy official stated that one capstone pilot training exercise is  conducted less frequently than in the past. Pilots participate in this  exercise, at Naval Air Station Fallon, once per training cycle, normally  shortly after they complete the basic phase. During the air squadron  training cycle that existed prior to FRP, pilots participated once every 2  years; now they are only required to do so once per FRP cycle, which is  every 27 to 32 months. While Navy officials have told us that they were  unsure what effect less frequent Fallon exercises would have on pilot  skills and are reviewing the extent to which Fallon exercises may need to  be repeated during a multiple-deployment FRP cycle, they did not provide  us with documentation of their review or evidence that they had studied  the issue at the time the decision to extend the operational cycle was  made.
The Navy has not fully considered the long-term risks and tradeoffs of  these changes to its maintenance and operational cycles because it has not  performed a comprehensive assessment of how the changes, interacting  with one another, might affect its ability to meet FRP goals and perform its  full range of missions. As a combined result of increases to both  maintenance and operational cycles, carriers have fewer opportunities to  be inspected in dry dock. In addition, they spend about 22 percent less  time in any type of depot maintenance period compared to the pre-FRP 24- month cycle. Since the oldest Nimitz-class carrier was commissioned  about 32 years ago and the newest is still under construction, neither we  nor the Navy can know, based on experience, all the effects that  maintenance or operational cycle increases could have on the carriers.  Two possibilities have arisen in discussions with Navy officials: extensions  could have an impact on total service life, and extending dry docking  cycles beyond 8 years or operational cycles beyond 32 months could limit  the maintenance community’s ability to respond to problems quickly.  However, there was no consensus about the likelihood or possible severity  of either of these outcomes.
Some evidence suggests that lengthening intervals between depot  maintenance periods may be associated with a reduction in total carrier  service life. For example, the 2006 guidance from the Chief of Naval  Operations on maintenance intervals projected that the service life for  Nimitz-class nuclear carriers would be slightly reduced compared to past  estimates. Moreover, FRP was developed to enable carriers to be deployed  for more time than before and under conditions—such as short-notice  surges with a premium on providing a quick response—that tend to use up  reactor fuel in nuclear carriers more rapidly than would be the case with  longer deployments with longer transit time at slower speeds. In Nimitz- class carriers, reactor fuel is replenished only once, at the midpoint of a  carrier’s life cycle, at about the 23-year mark. Therefore, if reactor fuel is  used up in less than the scheduled time, the carrier may reach both the  midpoint and end of its service life earlier than planned. In addition, the  Navy’s analysis leading to the operational cycle extension from 27 to 32  months assumed that operating tempo would not increase and carriers  would not make multiple deployments in a single FRP cycle. If the Navy  did have to replace carriers sooner than planned, there could potentially  be significant effects on long-term Navy budget requirements. Without  assessing the short- and long-term risks and trade-offs associated with the  changes in maintenance and operational cycles, it will be difficult for the  Navy to determine the extent to which these changes could affect its  ability to meet FRP goals and perform its full range of missions.
section_title: The Navy Has Not Fully Analyzed Ability to Meet FRP Goals during Periods When Less than 11 Carriers Will Be Available
paragraphs: The Navy has reported that FRP is supportable with 11 carriers and has  developed plans to show how it would continue to meet FRP goals during  two upcoming periods when the number of available aircraft carriers  temporarily drops from 11 to 10. However, these plans have not fully  analyzed the risks that could arise because they may make optimistic  assumptions about the length of the gaps and lack some details about how  the Navy would mitigate these gaps. The first period is expected to begin  in fall 2008, after the Kitty Hawk’s scheduled November 2008  decommissioning, and will last until the Bush is prepared for its first  deployment in the summer of 2010, a gap of nearly 2 years. According to  the Navy, the second period is scheduled to begin in fall 2012 when the  Enterprise is decommissioned, and will last for 33 months until the Ford is  commissioned in fiscal year 2015.
The Navy’s plans may have presented optimistic assumptions about the  total length of the gap. For example, the first plan projected that the Bush  would be operationally ready in the summer of 2010, which is about a year  after its scheduled commissioning date, but the second plan did not  address the time between the Ford’s projected commissioning date and its  operational readiness date. According to a December 2006 DOD report on  the Ford’s progress, the carrier is scheduled to reach initial operational  capability in September 2016, for a total gap of 45 months, as opposed to  the gap of 33 months in the Navy’s report. In addition, we have recently  reported that the Ford is encountering delays in technology development  that could affect its delivery schedule. In both cases, there may even be  additional time between operational readiness and actual first deployment.  The average interval between commissioning and deployment for all  Nimitz-class carriers was nearly 2 years, and no carrier since the Vinson  (which first deployed in 1983) has deployed within 1 year of its  commissioning date.
Both of the Navy’s plans lack some details about how the Navy would  mitigate these gaps. The first plan reported that there would be 5 months  between the scheduled decommissioning date of the Kitty Hawk and the  projected operational readiness date of the Bush when carrier readiness  status would fall below 6+1, and noted that these could be mitigated with  adjustments to scheduled maintenance periods or by accelerating  scheduled training. The first plan lacked specific information about the  carriers and their projected FRP phases for each month of the gap period,  so we could not validate the plan’s assumptions about how many carriers  would be surge-ready during any particular month. In addition, the plan  did not link specific mitigations, such as extending a carrier’s maintenance  interval or accelerating unit-level training, to specific months in which  surge-ready availability fell below 6+1. Without such information, we  could not evaluate whether the Navy had weighed the possibilities and  determined the most appropriate way of mitigating a potential shortfall.
Moreover, applying the expected durations of basic and integrated phases  of FRP that the Navy formalized in the August 2007 FRP instruction, we  identified at least 6 months in the first gap period after the Kitty Hawk  retires when there will not be enough carriers in the right FRP phases to  meet the 6+1 or 3+3+1 implementation goals, a difference that could affect  the Navy’s mitigation planning.
The second plan showed the baseline depot maintenance schedule, and  presented several alternative ways in which identified gaps in the Navy’s  ability to deploy 6+1 carriers between the scheduled decommissioning  date of the Enterprise and the projected commissioning date of the Ford  could be filled. These included: extending the carrier depot maintenance  cycle or operational schedules, deferring some global presence  deployments, or delaying the decommissioning date of the Kitty Hawk.  Applying the same expected durations of basic and integrated phases of  FRP to the second gap period, we identified at least 13 months when there  will not be enough carriers to meet the 6+1 or 3+3+1 implementation  goals. The second plan did not address how the Navy could  simultaneously have enough carriers available to surge to meet FRP goals  without stretching out some maintenance intervals beyond currently  approved limits. The Navy’s decision to lengthen intervals between depot  maintenance periods, as discussed, was based on the assumption that  there would be 11 carriers at all times. Therefore, during these periods the  Navy may have to choose among not meeting FRP carrier strike group  goals, further extending carrier maintenance cycles, shortening training, or  some combination of these tradeoffs.
Until the Navy develops plans that use realistic schedule assumptions and  that can depict the likely challenges to implementation goals during these  gap periods, senior Navy leadership may not have the information it needs  to make informed trade-off decisions. As a result, the Navy may not be  able to achieve an optimal balance between maximizing carrier strike  groups’ ability to surge on short notice in support of FRP and performing  the full range of Navy missions.
section_title: Conclusions
paragraphs: The Navy considers FRP to be a critical enabler in meeting challenges of  the twenty-first century security environment. Although the Navy has  taken several important steps toward fully developing a sound  management approach for FRP, such as establishing oversight and  execution responsibility, developing implementation goals for carrier  strike groups, and identifying some key performance measures, this  process is incomplete. Without implementation goals for extending FRP to  expeditionary strike groups and other ship configurations; performance  measures that identify readiness levels and are fully linked to acceptable  levels of performance for FRP phases; and a completed analysis that links  needed resources to FRP phases, goals, and readiness levels, the Navy may  not be able to develop budget requests based on the resources needed to  achieve required readiness levels or demonstrate to senior DOD officials  and Congress whether it can achieve the intended benefits under FRP.
The Navy has studied the effects of some aspects of changes it has made  during FRP’s first years, such as extensions of operational and  maintenance cycles, and has begun to pursue other studies. However, until  the Navy performs a comprehensive risk assessment that addresses the  cumulative impact of changes to operational and maintenance cycles, and  the possible effects of having fewer carriers in the force structure in the  future, it will be unable to weigh the trade offs associated with meeting  FRP goals within projected budgets. Further, assessing risk becomes  increasingly important as the Navy expands FRP to include other forces  and as its force structure faces periods with 10 instead of 11 available  carriers.
section_title: Recommendations for Executive Action
paragraphs: To improve the Navy’s management as FRP continues to evolve and as the  Navy moves forward with implementation, we recommend that the  Secretary of Defense direct the Secretary of the Navy to take the following  four actions:    establish implementation goals for the application of FRP to other    establish required overall readiness levels for each FRP phase in its    develop additional performance measures that identify acceptable  levels of performance and scope; and  identify how resources should be linked to the FRP phases, goals, and  readiness levels and publish appropriate guidance.
To improve the Navy’s ability to weigh the trade offs associated with  meeting FRP goals within current resource and force structure plans, we  recommend that the Secretary of Defense direct the Secretary of the Navy  to take the following action:    perform a risk assessment that integrates consideration of the effects  of changes in carrier strike group operational and maintenance cycles  under a range of force structure assumptions, and that identifies  strategies to mitigate potential risks. Such a risk assessment should  integrate plans to meet FRP goals during two upcoming periods when  the number of available aircraft carriers temporarily drops from 11 to  10.
section_title: Agency Comments and our Evaluation
paragraphs: In written comments on a draft of this report, DOD fully agreed with one  recommendation, and partially agreed with four recommendations. DOD’s  comments are reprinted in their entirety in appendix II.
DOD partially agreed with our recommendation that the Navy establish  implementation goals for the application of the Fleet Response Plan to  other forces. DOD stated that the Navy has developed preliminary goals  for submarines, surface combatants, and other types of forces, and will  continue to include additional Navy forces beyond those of the carrier  strike group. While we laud this progress, we urge the Navy to take steps  to finalize these goals as soon as possible. In addition, we reiterate that the  preliminary goal for surface ships was expressed in a single aggregated  figure that included destroyers, cruisers, frigates, amphibious ships, and  minesweepers. As discussed in the report, an aggregate figure may not  provide a meaningful goal, because some ships, such as destroyers, may be  used in a variety of roles: as components of carrier or expeditionary strike  groups or as part of smaller task forces. Without implementation goals for  all force configurations, the Navy may not know whether it has enough  ships to be distributed among all possible roles and provide all desired  capabilities under FRP. Furthermore, since the Navy’s system for aligning  resource inputs with readiness outputs is linked to its implementation  goals, identifying separate goals is a prerequisite to identifying appropriate  levels of funding.
DOD partially agreed with our recommendation that the Navy develop  required overall readiness levels for each Fleet Response Plan phase in its  readiness reporting system. DOD stated that, under the Defense Readiness  Reporting System, the Navy will be able to rapidly and accurately assess a  unit’s readiness and ability to conduct missions throughout the FRP  continuum. While we hope that DOD can realize its expectations for this  system, we note that it is not scheduled to be fully implemented for  another year or more. Therefore, we urge DOD to direct the Navy to  develop a readiness level requirement within existing systems so that it  will have visibility over whether the Navy’s goals are being met up until the  Defense Readiness Reporting System is fully implemented, and ensure that  this new system allows DOD to assess units’ readiness for each FRP  phase.
DOD partially agreed with our recommendation that the Navy develop  additional performance measures that identify acceptable levels of  performance and scope. DOD stated that the Fleet Readiness Enterprise is  developing criteria that will show acceptable levels of performance. While  we support DOD’s efforts to develop criteria, we urge the Navy to take  steps to finalize these criteria as soon as possible. DOD also stated that the  Navy has Figures of Merit to guide the allocation of resources in specific  areas, and furthermore trusts its unit commanders to accurately report  their ability to meet FRP requirements. At the time of our review, these  measures were still under development, so it is unclear whether they will  be sufficient to provide the necessary information about the scope and  acceptable levels of performance to enable consistent and accurate  measurement of performance. We continue to believe that our  recommendation merits further action and that DOD needs to direct the  Navy to fully link the FRP performance measures to quantified levels of  acceptable performance, encompass ships and aircraft as applicable, and  provide measures to assess appropriate manning levels, so that decision  makers will be able to determine the extent to which FRP is achieving its  goals.
DOD agreed with our recommendation to identify how resources should  be linked to the Fleet Response Plan phases, goals, and readiness levels  and publish appropriate guidance. DOD stated that the Navy is in the  process of developing a comprehensive set of top-level metrics that will  provide the Navy with the ability to link readiness outputs to required  resource inputs. When development is complete, the Navy will have a set  of quantifiable and traceable relationships between its financial system  and Fleet readiness. We agree that the actions cited represent positive  steps and urge the Navy to press forward with these efforts. We note that  we also recommended that the Navy publish appropriate guidance upon  completion.
DOD partially agreed with our recommendation that the Navy perform a  risk assessment that integrates consideration of the effects of changes in  carrier strike group operational and maintenance cycles under a range of  force structure assumptions and identify strategies to mitigate potential  risks. DOD stated that it understands that modifications to existing plans  will have numerous consequences and indicated that the Navy will  continue to update its plans based on current risk assessments. DOD  concluded that there was no need for further direction from the Secretary  of Defense in this matter. We disagree, and continue to believe that the  Navy should conduct an integrated risk assessment and identify strategies  to mitigate potential risks. As discussed in the report, prior to making  changes to its maintenance cycle in the past, the Navy has conducted  assessments of the potential effects of the changes, and we emphasize the  interrelated nature of the changes that the Navy has made to implement  FRP. For example, as we discussed in our report, changing carrier  operational cycles may have repercussions on the opportunities available  for aircraft pilot training. In view of the possible impact on the Navy’s  ability to perform its full range of missions and the potential financial  implications of intensifying operational tempo or decreasing maintenance  of the Navy’s carriers to the point where their planned service life could be  jeopardized, we continue to believe that it is critical for the Navy to fully  consider the long-term risks and trade offs of these changes through a  comprehensive and integrated risk assessment.
We are sending copies of this report to the Secretary of Defense, the  Secretary of the Navy, and the Chief of Naval Operations. We will also  make copies available to other interested parties upon request. In addition,  this report will be made available at no charge on the GAO Web site at  www.gao.gov. If you or your staff have any questions about this report,  please contact me at (202) 512-4402 or stlaurentj@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. Staff members who made key contributions  to this report are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the Navy’s progress in developing a sound management  approach, we reviewed and analyzed guidance and instructions on  manning, maintenance, and training; key studies, messages, and planning  documents. We reviewed prior GAO products to identify best practices for  managing and implementing major efforts and compared these best  practices to the Navy documents we analyzed. We also interviewed  officials in the offices of the Chief of Naval Operations and Chairman of  the Joint Chiefs of Staff, Washington, D.C.; Fleet Forces Command;  Commander, Air Forces Atlantic; Commander, Surface Forces Atlantic;  Commander, Submarine Forces Atlantic, Norfolk, Virginia; Naval Air  Systems Command, Patuxent River, Maryland; Naval Sea Systems  Command, Washington, D.C.; and Marine Corps Forces Command,  Norfolk, Virginia; about initiatives the Navy has undertaken. We also  contacted officials at the office of the Commander, U.S. Pacific Fleet;  Commander, Naval Air Forces, San Diego, California; and Carrier Planning  Activity, Chesapeake, Virginia.
We also analyzed data from the Navy’s current readiness reporting system.  Readiness data include ships’ and airwings’ scores under the Status of  Resources and Training System, which measures inputs such as numbers  of personnel, amount and condition of equipment on-board, and  completion of training exercises, and combines them under a five-point  rating system. Status of Resources and Training System data are limited in  that they only measure readiness against the standard of major combat  operations and do not specifically show readiness at each FRP phase.  DOD has recognized this limitation and has begun to develop a new,  capabilities-based system called the Defense Readiness Reporting System,  which was designed to measure units’ ability to perform specific tasks.  The Navy has begun the transition to the Defense Readiness Reporting  System, full implementation of which it believes will address the gaps in  performance measures that we identified in our review. However, this  transition was not yet complete at the time we conducted our review. The  Navy continues to report Status of Resources and Training System data in  the Type Commanders’ Readiness Management System, and to use Status  of Resources and Training System data as the foundation of its weekly and  monthly readiness reports to the Chief of Naval Operations. With these  noted limitations, we determined that the data were sufficiently reliable  for our purposes.
To assess the extent to which the Navy has considered the long-term risks  and trade offs of FRP-related changes, we interviewed Navy readiness  officials in the offices of the Chief of Naval Operations, Arlington, Virginia;  and program managers at Naval Sea Systems Command, Washington, D.C.,  and Chesapeake, Virginia; and Fleet Forces Command, Norfolk, Virginia.  To assess whether the Navy had sufficiently taken into account the  possibility of not having enough carriers available to meet either a 6+1  implementation goal or a 3+3+1 construct during the years in which 10  carriers will be available, we analyzed the quarterly long-range carrier  maintenance schedule that was published in July 2007 and noted, based on  the notional durations for each of the FRP phases that were published in  the August 2007 FRP instruction, how many carriers would be in each  phase during each month during fiscal years 2008 through 2010 and 2013  through 2016. For purposes of our analysis, we assumed that no carrier  would remain in any FRP phase for longer than the planned time, and that  the George Washington, which is scheduled to be at its home port in Japan  in 2009, would be deployable at any time that it was not in depot  maintenance. We also analyzed a Navy risk mitigation plan for the years  following the decommissioning of the Enterprise and a less detailed plan  that covered the years following the decommissioning of the Kitty Hawk,  and discussed our observations with Navy readiness and carrier program  officials.
We performed our work from November 2006 through October 2007 in  accordance with generally accepted government auditing standards.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: U.S. Navy Aircraft Carriers
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the contact named above, Patricia W. Lentini, Assistant  Director; Renee S. Brown; Janine Cantin; Alissa Czyz; Karen Nicole Harms;  Matthew S. Lee; Sally Newman; Maria-Alaina Rambus; Rebecca Shea; and  Nicole Volchko made key contributions to this report.