text: Background Naval Forces Involved in Amphibious Operations An amphibious operation is a military operation launched from the sea by an amphibious force, embarked in ships or craft, with the primary purpose of introducing a landing force ashore to accomplish an assigned mission. An amphibious force is comprised of an (1) amphibious task force and (2) landing force together with other forces that are trained, organized, and equipped for amphibious operations. The amphibious task force is a group of Navy amphibious ships, most frequently deployed as an Amphibious Ready Group (ARG). The landing force is a Marine Air- Ground Task Force—which includes certain elements, such as command, aviation, ground, and logistics—embarked aboard the Navy amphibious ships. A Marine Expeditionary Unit (MEU) is the most-commonly deployed Marine Air-Ground Task Force. Together, this amphibious force is referred to as an ARG-MEU. The Navy’s amphibious ships are part of its surface force. An ARG consists of a minimum of three amphibious ships, typically an amphibious assault ship, an amphibious transport dock ship, and an amphibious dock landing ship. Figure 1 shows the current number of amphibious ships by class and a description of their capabilities. The primary function of amphibious ships is to provide transport to Marines and their equipment and supplies. The ARG includes an amphibious squadron that is comprised of a squadron staff, tactical air control squadron detachment, and fleet surgical team. This task organization also includes a naval support element that is comprised of a helicopter squadron for search and rescue and antisurface warfare, two landing craft detachments for cargo lift, and a beachmaster unit detachment to control beach traffic. An MEU consists of around 2,000 Marines, their aircraft, their landing craft, their combat equipment, and about 15 days’ worth of supplies. The MEU includes a standing command element; a ground element consisting of a battalion landing team; an aviation element consisting of a composite aviation squadron of multiple types of aircraft; and a logistics element consisting of a combat logistics battalion. Figure 2 provides an overview of the components of a standard ARG-MEU. An amphibious force can be scaled to include a larger amphibious task force, such as an Expeditionary Strike Group, and a larger landing force, such as a Marine Expeditionary Brigade or Marine Expeditionary Force (MEF) for larger operations. A Marine Expeditionary Brigade is comprised of 3,000 to 20,000 personnel and is organized to respond to a full range of crises, such as forcible entry and humanitarian assistance. A MEF is the largest standing Marine Air-Ground Task Force and the principal Marine Corps warfighting organization. Each MEF consists of 20,000 to 90,000 Marines. MEFs are used in major theater war and other missions across the range of military operations. There are three standing MEFs—I MEF at Camp Pendleton, California; II MEF at Camp Lejeune, North Carolina; and III MEF in Okinawa, Japan. Navy and Marine Corps Training for Amphibious Operations Navy ships train to a list of mission-essential tasks that are assigned based on the ship’s required operational capabilities and projected operational environments. Most surface combatants, including cruisers, destroyers, and all amphibious ships, have mission-essential tasks related to amphibious operations. The Navy uses a phased approach to training, known as the Fleet Response Training Plan. The training plan for amphibious ships is broken up into five phases: maintenance, basic, advanced, integrated, and sustainment. The maintenance phase is focused on the completion of ship maintenance, with a secondary focus on individual and team training. The basic phase focuses on development of core capabilities and skills through the completion of basic-level inspections, assessments, and training requirements, among other things. This phase can include certification in areas such as mobility, communications, amphibious well-deck operations, aviation operations, and warfare training. The basic phase of training requires limited Marine Corps involvement—mainly to certify amphibious ships for well-deck and flight-deck operations. The advanced phase focuses on advanced tactical training, including amphibious planning. The integrated phase is where individual units and staffs are aggregated into an Amphibious Ready Group (ARG) and train with an embarked MEU or other combat units. The sustainment phase includes training to sustain core skills and provides an additional opportunity for training with Marine Corps units, when possible. Marine Corps units train to accomplish a set of mission-essential tasks for the designed capabilities of the unit. For example, the mission-essential tasks for a Marine Corps infantry battalion include amphibious operations, offensive operations, defensive operations, and stability operations. Many Marine Corps units within the command, aviation, ground, and logistics elements have an amphibious-related mission-essential task. The Marine Corps uses a building-block approach to accomplish training, progressing from individual through collective training. For example, an assault amphibian vehicle battalion will progress through foundational, individual, and basic amphibious training—such as waterborne movement and ship familiarization—to advanced amphibious training, such as live training involving ship-to-shore movement conducted under realistic conditions. Marine Corps unit commanders use Training and Readiness manuals to help develop their training plans. Training and Readiness manuals describe the training events, frequency of training required to sustain skills, and the conditions and standards that a unit must accomplish to be certified in a mission-essential task. To be certified in the mission- essential task of amphibious operations, Marine Corps units must train to a standard that may require the use of amphibious ships. For example, ground units with amphibious-related mission-essential tasks will not be certified until live training involving sea-based operations and ship-to- shore movement has been conducted under realistic conditions. Similarly, for aviation squadrons, training for amphibious operations (called sea- based aviation operations) will not be certified until live training involving sea-based operations has been conducted under realistic conditions, including aviation operations from an amphibious platform. Similar types of units, such as all infantry battalions, may train on the same mission- essential tasks. However, unit commanders are ultimately responsible for their units’ training, and a variety of factors can lead commanders to adopt different approaches to training, such as the units’ assigned missions or deployment locations. Marine Corps units that are scheduled to deploy as part of an ARG-MEU will follow a standardized 6-month predeployment training program that gradually builds collective skill sets over three phases, as depicted in figure 3. Marine Corps’ Use of Virtual Training Devices The Marine Corps’ use of virtual training devices has increased over time. Virtual training devices were first incorporated into training for the aviation community, which has used simulators for more than half a century. The Marine Corps’ ground units did not begin using simulators and simulations until later. Specifically, until the 1980s, training in the ground community was primarily live training. Further advances in technology resulted in the acquisition of simulators and simulations with additional capabilities designed to help individual Marines and units acquire and refine skills through more concentrated and repetitive training. For example, the Marine Corps began using devices that allowed individual Marines to conduct training in basic and advanced marksmanship and weapons employment tactics. More recently, during operations in Iraq and Afghanistan, the Marine Corps introduced a number of new virtual training devices to prepare Marines for conditions on the ground and for emerging threats. For example, to provide initial and sustainment driver training, the Marine Corps began using simulators that can be reconfigured to replicate a variety of vehicles. In addition, in response to an increase in vehicle rollovers, the Marine Corps began using egress trainers to train Marines to safely evacuate their vehicles. The Marine Corps has also developed virtual training devices that can be used to train Marines in collective training, such as amphibious operations. For example, the Marine Air-Ground Task Force Tactical Warfare Simulation is a constructive simulation that provides training on planning and tactical decision making for the Marine Corps’ command element. See figure 4 for a description of examples of Marine Corps devices that can be used for individual through collective training. Navy and Marine Corps Units Completed Training for Certain Amphibious Operations Priorities but Not Others, and Efforts to Mitigate Training Shortfalls Are Incomplete Navy and Marine Corps units that are deploying as part of an ARG-MEU completed their required training for amphibious operations, but several factors have limited the ability of Marine Corps units to conduct training for other amphibious operations–related priorities. The Navy and Marine Corps have taken steps to identify and address amphibious training shortfalls, but their efforts to mitigate these shortfalls have not prioritized available training resources, systematically evaluated among potential training resource alternatives to accomplish the services’ amphibious operations training priorities, or monitored progress toward achieving the priorities. Navy and Marine Corps ARG-MEU Deploying Units Completed Required Training for Amphibious Operations, but Several Factors Have Limited Training for Other Marine Corps Amphibious Operations Priorities Navy and Marine Corps units deploying as part of ARG-MEUs have completed required training for amphibious operations, but the Marine Corps has been unable to consistently accomplish training for other service amphibious operations priorities. We found that Navy amphibious ships have completed training for amphibious operations. Specifically, based on our review of deployment certification messages from 2014 through 2016, we found that each deploying Navy ARG completed training for the amphibious operations mission in accordance with training standards. Similarly, we found that each MEU completed all of its mission-essential tasks that are required during the predeployment training program. These mission-essential tasks cover areas such as amphibious raid, amphibious assault, and noncombatant evacuation operations, among other operations. However, while the Marine Corps has completed amphibious operations training for the MEU, based on our review of unit-level readiness data from fiscal year 2014 through 2016 we found that the service has been unable to fully accomplish training for its other amphibious operations priorities, which include home-station unit training to support contingency requirements, service-level exercises, and experimentation and concept development for amphibious operations. Specific details of these shortfalls were omitted because the information is classified. Additionally, Marine Corps officials cited shortfalls in their ability to conduct service-level exercises that train individuals and units on amphibious operations-related skills, as well as provide opportunities to conduct experimentation and concept development for amphibious operations. In particular, officials responsible for planning and executing these exercises told us that one of the biggest challenges is aligning enough training resources, such as amphibious ships, to accurately replicate a large-scale amphibious operation. For example, officials from III MEF told us that the large-scale amphibious exercise Ssang Yong is planned to be conducted every other year, but that the exercise requires the availability and alignment of two ARG-MEUs in order to have enough forces to conduct the exercise. These officials stated that this alignment may only happen every 3 years, instead of every other year, as planned. In addition, officials from I MEF and II MEF told us that their large-scale amphibious exercises are intended to be a Marine Expeditionary Brigade–level training exercise, however, these exercises are typically only able to include enough amphibious ships to support a MEU, while the other forces must be simulated. Despite these limitations, Navy and Marine Corps officials have identified these service-level exercises as a critical training venue to support training for the Marine Expeditionary Brigade command element and to rebuild the capability to command and control forces participating in amphibious operations. Based on our analysis of interviews with 23 Marine Corps units, we found that all 23 units cited the lack of available amphibious ships as the primary factor limiting training for home-station units. The Navy’s fleet of amphibious ships has declined by half in the last 25 years, from 62 in 1990 to 31 today, with current shipbuilding plans calling for four additional amphibious ships to be added by fiscal year 2024, increasing the total number of amphibious ships to 35 (see fig. 5). Navy and Marine Corps officials noted a number of issues that can affect the amount of training time that is available with the current amphibious fleet. In particular, the current fleet of ships is in a continuous cycle of maintenance, ARG-MEU predeployment training, and sustainment periods, leaving little additional time for training with home-station units and participation in service-level exercises. Navy officials told us that the Optimized Fleet Response Plan may provide additional training opportunities for Marine Corps units during the amphibious ships’ sustainment periods. Given the availability of the current inventory of amphibious ships, Marine Corps requests to the Navy for amphibious ships and other craft have been difficult to fulfill. For example, data from I MEF showed that the Navy was unable to fulfill 293 of 314 (93 percent) of I MEF requests for Navy ship support for training in fiscal year 2016. Similarly, data from II MEF showed that in fiscal year 2016 the Navy was unable to fulfill 19 of 40 requests for ship services. We identified issues with the completeness of this request data. Specifically, we found that the data may not fully capture the Marine Corps’ demand for amphibious ships. As a result, this information may overstate the ability of the Navy to fulfill these requests. We discuss these data-reliability issues further below. Marine Corps officials from the 23 units we interviewed also cited other factors that limit opportunities for amphibious operations training, such as the following: Access to range space: Seventeen of 23 Marine Corps units we interviewed identified access to range space as a factor that can limit their ability to conduct amphibious operations training. Unit officials told us that priority for training resources, including range access, is given to units that will be part of a MEU deployment, leaving little range time available for other units. In addition, unit officials told us that the amount of range space available can affect the scope and realism of the training that they are able to conduct. Training for amphibious operations can require a large amount of range space, because the operational area extends from the offshore waters onto the landing beach and further inland. A complete range capability requires maneuver space, tactical approaches, and air routes that allow for maneuverability and evasive actions. However, officials from II MEF told us that the size of the landing beach near Camp Lejeune, North Carolina makes conducting beach-clearing operations infeasible. Adequate ranges have been identified as a challenge across DOD. For example, according to DOD’s 2016 Report to Congress on Sustainable Ranges, some Marine Corps installations lack fully developed maneuver corridors, training areas, and airspace to adequately support ground and air maneuver inland from landing beaches. Maintenance delays, bad weather, and transit time: Ten of 23 Marine Corps units told us that changes to an amphibious ship’s schedule resulting from maintenance overruns or bad weather can also reduce the time available for a ship to be used for training. In addition, the transit time a ship needs to reach Marine Corps units can further reduce the time available for training. This is a particular challenge for II MEF units stationed in North Carolina and South Carolina that train with amphibious ships stationed in Virginia and Florida. According to II MEF officials, transit time to Marine Corps units can take up to 18 hours in good weather, using up almost a full day of available training time for transit. High pace of deployments: Five of 23 Marine Corps units told us that the high pace of deployments and need to prepare for upcoming deployments limited their opportunity to conduct training for amphibious operations. For example, II MEF officials told us that an infantry battalion that is scheduled to deploy as part of a Special Purpose Marine Air-Ground Task Force to Africa generally does not embark on an amphibious ship or have amphibious operations as part of its assigned missions. As a result, the unit will likely not conduct amphibious operations during its predeployment training. Efforts to Identify and Address Amphibious Training Shortfalls Lack Strategic Training and Risk-Management Practices The Navy and Marine Corps have taken some steps to mitigate the training shortfall for their amphibious operations priorities, but these efforts are incomplete because they have not prioritized available training resources, systematically evaluated among potential training resource alternatives to accomplish the services’ amphibious operations training priorities, or monitored progress toward achieving the priorities. The Navy and Marine Corps are in the process of identifying (1) the amount of amphibious operations capabilities and capacity that are needed to achieve the services’ wartime requirements, and (2) the training resources and funding required to meet the amphibious operations- related training priorities. First, in December 2016, the Navy conducted a force structure assessment that established a need for a fleet of 38 amphibious ships. Based on the assessment, the Chief of Naval Operations and the Commandant of the Marine Corps determined that increasing the Navy’s amphibious fleet from a 31-ship to a 38-ship amphibious fleet would allow the Marine Corps to meet its wartime needs of having enough combined capacity to transport two Marine Expeditionary Brigades. Specifically, a 38-ship fleet would provide 17 amphibious ships for each Marine Expeditionary Brigade, plus four additional ships to account for ships that are unavailable due to maintenance. According to Navy and Marine Corps officials, an increase in the number of amphibious ships should create additional opportunities for the Navy and Marine Corps to accomplish amphibious operations training. Second, the Marine Corps has also recognized a need to improve the capacity and experience of its forces to conduct amphibious operations and is taking steps to identify the training resources and funding required to meet its amphibious operations–related training priorities. To accomplish this task, in 2016 the Marine Corps initiated the Amphibious Operations Training Requirements review. As a part of this review, the Marine Corps has comprehensively determined units that require amphibious operations training and is in the process of refining the training and readiness manuals for each type of Marine Corps unit to include an amphibious-related mission-essential task as appropriate, and better emphasizing the types of conditions and standards for amphibious training in the manuals. According to officials, as of May 2017, Marine Corps Forces Command has reviewed the mission-essential tasks for 60 unit types and found 31 unit types already had a mission-essential task for amphibious operations, while another 5 unit types required that an amphibious-related mission-essential task be added. The review further found that the other 24 unit types do not require a mission-essential task for amphibious operations. In addition, the Marine Corps Training and Education Command noted in its review that certain training standards within the training manuals are being refined in order to distinguish between levels of training accomplished. For example, for ground-based units, such as infantry battalions, an additional training standard was added for all amphibious-related mission-essential tasks that a unit would not be considered both trained and certified unless live training using amphibious ships has been conducted under realistic conditions. The Amphibious Operations Training Requirements review is also intended to accomplish other actions to better define the services’ amphibious operations training priorities, but these actions were incomplete at the time of our review. Specifically, the review will also establish an objective for the number of Marine Corps forces that must be trained and ready to conduct amphibious operations at a given point in time, and the amount of funding for ship steaming days that is required to provide training for the services’ amphibious operations priorities. According to officials responsible for the Amphibious Operations Training Requirements review, an outcome of the review is expected to be a combined Navy and Marine Corps directive signed by the Chief of Naval Operations and the Commandant of the Marine Corps that should provide guidance to better define a naval objective for amphibious readiness and required ship steaming days. Marine Corps officials estimated that the issuance of the directive will be in the summer of 2017. With these two efforts, the Navy and Marine Corps have been proactive in identifying the underlying problems with training for amphibious operations, and their ongoing efforts indicate that addressing this training shortfall is a key priority for the two services. In particular, the proposed Navy and Marine Corps directive that will result from the Amphibious Operation Training Requirements review should help establish a naval objective for amphibious readiness with the corresponding units that need to be trained and ready in amphibious operations, as well as a basis for estimating the required amount of training resources, such as ship steaming days, to meet amphibious operations training priorities. When completed, the development of this directive is an important first step to clearly identify the total resources needed for amphibious operations training. However, the Navy’s and Marine Corps’ current approach for amphibious operations training does not incorporate strategic training and leading risk-management practices. Specifically, we found the following: The Marine Corps does not prioritize all available training resources: Based on our prior work on strategic training, we found that agencies need to align their training processes and available resources to support outcomes related to the agency’s missions and goals, and that those resources should be prioritized so that the most- important training needs are addressed first. For certain units that are scheduled to deploy as part of an ARG-MEU, the Navy and Marine Corps have a formal training program that specifies the timing and resource needs across all phases of the training, including the number of days embarked on amphibious ships that the Navy and Marine Corps need to complete their training events. Officials stated that available training resources, including access to amphibious ships for training, are prioritized for these units. However, for other Marine Corps units not scheduled for a MEU deployment, officials described an ad hoc process to allocate any remaining availabilities of amphibious ship training time among home- station units. Specifically, officials stated that the current process identifies units that are available for training when an amphibious ship becomes available rather than a process that aligns the next highest- priority units with available training resources. For example, officials at Headquarters Marine Corps told us that the Navy will identify training opportunities with amphibious ships at quarterly scheduling conferences. The Marine Corps will fill these training opportunities with units that are available to accomplish training during that period, but not based on a process that identifies its highest-priority home- station units for training. Similarly, a senior officer with First Marine Division told us that he would prioritize home-station units that have gone the longest without conducting amphibious-related training, which may not be the units with the highest priority for amphibious operations training. The Navy and Marine Corps have recognized the need for reinstituting a recurring training program for home-station units, but efforts to implement such a program have not been started at the time of our review. According to Navy officials, the Navy and Marine Corps have had a recurring training program in the past to provide home- station units with amphibious operations training called the Type Commander Amphibious Training series, or TCAT, but this program was phased out 15 years ago with the implementation of the Fleet Response Training Plan that is more focused on ARG-MEU training. Navy and Marine Corps officials told us that reinstituting a similar training program would allow the services to better prioritize training resources and align units to achieve the services’ proposed naval objective for amphibious readiness. Without establishing a process to prioritize available training resources for home-station units, the Navy and Marine Corps cannot be certain that scarce training opportunities are being aligned with their highest-priority needs. The Navy and Marine Corps do not systematically evaluate a full range of training resource alternatives to achieve amphibious operations training priorities: Our prior work on risk management has found that evaluating and selecting alternatives are critical steps for addressing operational capability gaps. Based on our interviews with officials across the Marine Expeditionary Forces and review of documentation, we identified a number of alternatives that could help mitigate the risk to the services’ amphibious capability due to limited training opportunities. These alternatives include utilizing additional training opportunities during an amphibious ship’s basic phase of training; using alternative platforms for training, such as Marine Prepositioning Force ships, or the amphibious ships of allies; utilizing smaller Navy craft or pier-side ships to meet training requirements; and leveraging developmental and operational test events. However, the Navy and Marine Corps have not developed a systematic approach to explore and incorporate selected training resource alternatives into home-station training plans. Specifically, officials told us that the combined Navy and Marine Corps directive that is expected to be completed later this year will better define a naval objective for amphibious readiness and the required training resources to achieve it, and will provide guidance to the two services to better identify training resource alternatives for home-station training. Based on our review of briefing materials on the Amphibious Operations Training Requirements review, however, we found that the services have discussed using some training resource alternatives to mitigate amphibious operations training shortfalls, such as pier-side ships to minimize the required number of ship steaming days, but the services have not systematically evaluated potential alternatives. Marine Corps officials told us that fully evaluating resource alternatives, particularly the use of simulated training and pier-side ships, could allow for more amphibious training without the need for additional steaming days. Fully exploring alternatives, such as utilizing alterative platforms and pier-side ships, and incorporating a broader range of training resource alternatives into training will be important as the Navy and Marine Corps try to achieve their training priorities and could help bridge the time gap until more amphibious ships are introduced into the fleet. The Navy and Marine Corps have not developed a process or set of metrics to monitor progress toward achieving its amphibious operations training priorities and mitigating existing shortfalls: Our prior work on risk management has found that monitoring the progress made and results achieved are other critical steps for addressing operational capability gaps. Marine Corps officials told us that the service uses the readiness reporting system (Defense Readiness Reporting System—Marine Corps) to measure the capabilities and capacity of its units to perform amphibious operations. While this reporting system allows the Marine Corps to assess the current readiness of units to perform the amphibious operations mission-essential task—an important measure—the system does not provide other information. For example, it does not allow officials to assess the status of service-wide progress in achieving its amphibious operations priorities or monitor efforts by the Marine Expeditionary Forces in establishing comprehensive amphibious operations training programs. Marine Corps officials told us that they may need to capture and track additional information, such as the number of amphibious training events scheduled and completed. However, as noted above, we found that the Marine Corps does not capture complete data that could be used for these assessments, such as demand for training time with amphibious ships. For example, officials from I MEF told us they do not capture the full demand for training time with Navy ships because unit commanders will not always submit a request that they believe is unlikely to be filled. In addition, these officials stated that their requests are prescreened before being submitted to the Navy to ensure that the requests align with known periods of available ship time. As a result, requests for amphibious ships and crafts are supply- driven, instead of demand-driven, which could affect the services’ ability to monitor progress in accomplishing unit training because an underlying metric is incomplete. Establishing a process to monitor progress in achieving amphibious operations training priorities will better enable the Navy and Marine Corps to ensure that their efforts are accomplishing the intended results and help assess the extent to which the services have mitigated any amphibious operations training shortfalls. More Comprehensively Incorporating Collaboration Practices Would Further Naval Integration Efforts for Amphibious Operations The Navy and Marine Corps have taken some steps to improve coordination between the two services, but the services have not fully incorporated leading collaboration practices that would help drive efforts to improve naval integration for amphibious operations. Our prior work on interagency collaboration has found that certain practices can help enhance and sustain collaboration among federal agencies. These key practices include (1) defining and articulating a common outcome; (2) establishing mutually reinforcing or joint strategies; (3) identifying and addressing needs by leveraging resources; (4) agreeing on roles and responsibilities; (5) establishing compatible policies, procedures, systems, and other means to operate across agency boundaries; (6) developing mechanisms to monitor, evaluate, and report on results; and (7) reinforcing agency accountability for collaborative efforts through plans and reports, among others. Common outcomes and joint strategy: The Navy and Marine Corps have issued strategic documents that discuss the importance of improving naval integration, but the services have not developed a joint strategy that defines and articulates common outcomes to achieve naval integration. We have found that collaborative efforts require agency staff working across agency lines to define and articulate the common outcome or purpose they are seeking to achieve that is consistent with their respective agency goals and mission. In addition, collaborating agencies need to develop strategies that work in concert with those of their partners. These strategies can help in aligning the partner agencies’ activities, processes, and resources to accomplish common outcomes. Further, joint strategies can benefit from establishing specific objectives, related actions, and subtasks with measurable outcomes, target audiences, and agency leads. Based on our review of Navy and Marine Corps strategic-level documents, both services identify the importance of improving naval integration, but these documents do not define and articulate outcomes that are common among the services or identify actions and time frames to achieve common outcomes that would be included a joint strategy. Instead, the documents describe naval integration in varying ways, including as a means to improve the capabilities of naval forces to perform essential functions, such as sea control and maritime security; exercise command and control for large-scale operations, including amphibious operations; and establish concepts to conduct naval operations in contested environments, among other areas. For example, strategic documents developed by the Navy only broadly discuss naval integration. In March 2015, the Department of the Navy issued an updated version of A Cooperative Strategy for 21st Century Seapower. This document discusses building the future naval force, including the need to organize and equip the Marine Expeditionary Brigade to exercise command and control of joint and multinational task forces for larger operations and enable the MEF for larger operations. In January 2016, the Department of the Navy published A Design for Maintaining Maritime Superiority, stating the need to deepen operational relationships with other services to include current and future planning, concept and capability development, and assessment. Marine Corps strategic documents provide a more-detailed and expansive list of areas for improved integration with the Navy, but do not provide guidance on how to achieve those areas. For example, in March 2014, the Marine Corps issued Expeditionary Force 21, which describes the need to increase naval integration, including operational integration between the Marine Expeditionary Brigade and the Navy’s Expeditionary Strike Group. Further, in September 2016 the Marine Corps issued a Marine Corps Operating Concept that establishes five tasks needed for the Marine Corps to build its future force, including integrating the naval force to fight at and from the sea. According to Navy and Marine Corps officials, naval integration is a broad term, has different meanings across various service organizations, and is not commonly understood. For example, officials told us that the services have identified the need to develop more-precise language around the term naval integration and articulate common outcomes to create a more- integrated approach to develop naval capabilities. Another senior Marine Corps training official told us that clear guidance is needed on how to define outcomes for naval integration for Navy and Marine Corps command-level staff. In particular, the official stated that without guidance it is unclear how an integrated staff should be composed—whether as two separate Navy and Marine Corps command staffs that should work together, or as one staff composed of both Navy and Marine Corps personnel. The continuing lack of common outcomes and a joint strategy could limit the Navy and Marine Corps ability to achieve their goals for naval integration. Further, joint strategies for improving naval integration could help ensure that services efforts are aligned to maximize available training opportunities and resources. Compatible policies, procedures, and systems: The Navy and Marine Corps have established several mechanisms to better coordinate their respective capabilities for amphibious operations training, but have not fully established compatible policies, procedures, and systems to foster and build naval integration. We have found that agencies need to address the compatibility of standards, policies, procedures, and data systems that will be used in the collaborative effort. These policies can be used to provide clarity about roles and responsibilities, including how the collaborative effort will be led. The Marine Corps has established a working group that provides a forum for collaboration for amphibious operations. Specifically, Marine Corps Forces Command established a Maritime Working Group to develop and manage a continuing Navy–Marine Corps quarterly collaborative process that is comprised of officials from the services’ headquarters, components, and operating forces. According to its mission statement, the Maritime Working Group is intended to align naval amphibious exercise planning to inform force development, war games, experimentation, and coalition participation in order to advance concepts; influence doctrine; inform naval exercise design and sourcing; inform capabilities development; and increase naval warfighting readiness. Based on our observation of the Maritime Working Group in September 2016, we found that the forum covered a broad range of topics including exercise prioritization, experimentation, and planning for future Navy exercises. Following the meeting, a summary of the topics discussed was provided to all participants as well as follow-on actions to be completed. However, we found that the Navy and Marine Corps have not fully established compatible policies and procedures, such as common training tasks and standards and agreed-upon roles and responsibilities, to ensure their efforts to achieve improved naval integration are consistent and sustained. For example, on the West Coast, the Navy and Marine Corps organizations 3rd Fleet and I MEF have issued guidance that formalizes policies that assign 1st Marine Expeditionary Brigade and Expeditionary Strike Group 3 with the responsibilities to conduct joint training. This guidance addresses the importance of Navy and Marine Corps interoperability by formalizing procedures, assigning responsibility, and providing general policy regarding training certification standards for these units. Officials from Fleet Forces Command noted that there is not similar guidance for East Coast–based units for the 2nd Marine Expeditionary Brigade and Expeditionary Strike Group 2. According to a Navy inspection report, Fleet Forces Command officials stated that they did not institute a deployment certification program for Expeditionary Strike Group 2 because of changing priorities at the command. As a result, the services lack clarity on the roles and responsibilities for these organizations—another key collaboration practice—that is needed to ensure these improvements are prioritized to further and sustain the collaborative effort. Both the Navy and Marine Corps have also identified areas where more- compatible training is needed to improve the skills and abilities of naval forces to perform certain missions. For example, Marine Corps training guidance from III MEF identifies a number of areas where Marine Corps units could improve collective naval capabilities by expanding training with the Navy, including areas such as joint maneuver, seizure and defense of forward naval bases, and facilitating maritime maneuver, among others. The Marine Corps Operating Concept also identifies other areas where integration with the Navy should be enhanced, including for intelligence, surveillance, and reconnaissance; operating in a distributed or disaggregated environment; and employment of fifth-generation aviation, such as the F-35. However, the services have been limited in their efforts to improve naval integration in these areas because they have not established compatible training tasks and standards that would institutionalize Navy and Marine Corps unit-level training requirements. Marine Corps officials told us that without compatible training tasks and standards, there is not a mechanism to force continued integration between the services outside of forces deploying as part of an ARG-MEU to help develop integrated naval capabilities. We also found that some of the Navy and Marine Corps’ systems for managing and conducting integrated training are incompatible, leading to inefficiencies in the process to manage training events involving Navy and Marine Corps units. For example, the Marine Corps has developed a system called Playbook to help align Navy and Marine Corps resources for training exercises that have been scheduled through the Force Synchronization process. At the time of our review, the Marine Corps was in the process of inputting data for all of its scheduled training exercises, including experiments and war games, into the system in order to align training resources and capabilities to its highest priority exercises and help build a training and exercise plan through 2020. However, the Navy uses several other data systems to track and capture its training resource requirements, and these systems are incompatible with Playbook. The lack of interface requires the Marine Corps to manually input and reconcile Navy information into its system. This can cause certain inefficiencies in arranging training. For example, officials from III MEF told us that adjustments to the Navy’s maintenance schedule for amphibious ships are not always communicated in advance, which can create a misalignment in the availability of amphibious ships and Marine Corps units to conduct training exercises. The Marine Corps has identified the need to define the Navy’s use of Playbook and explore a potential interface with Navy systems, but, as of May 2017, officials said that any evaluation, including potential cost-benefit analyses for addressing the interoperability issues, had not yet taken place. By having incompatible systems to schedule training, the services remain at risk of missing opportunities to maximize training opportunities for amphibious operations. Leverage resources to maximize training opportunities: The Navy and Marine Corps have identified certain opportunities where the two services can better leverage resources to conduct additional amphibious operations training together, but these opportunities have not been fully maximized. We have found that collaborating agencies should look for opportunities to address needs by leveraging each other’s resources, thus obtaining additional benefits that would not be available if they were working separately. Marine Corps Forces Command and Fleet Forces Command, as well as Marine Corps Forces Pacific and Pacific Fleet, have each established a Campaign Plan for Amphibious Operations Training. The purpose of these plans is to align resources for larger, service-level exercises for amphibious operations over a 5-year period. The goal of these exercises is to develop operational proficiency for a Marine Expeditionary Brigade–level contingency or crisis, but the specific focus of the exercise can change from year to year. For example, in 2017 the Bold Alligator exercise will focus on joint forcible entry operations and anti-access / area denial, whereas in prior years the focus has been on other operational areas, such as crisis response. We found that the Navy and Marine Corps also use mechanisms, such as scheduling conferences, to coordinate and prioritize requests for ship services for these exercises, as well as for other training events. The services are looking to better leverage available training resources for amphibious operations, but enhancing their collaborative efforts could take greater advantage of potential training opportunities. For example, Navy officials have stated that the Surface Warfare Advanced Tactical Training initiative could provide an additional training opportunity for Marine Corps units to train with Navy ships. This initiative is intended to provide amphibious ships with a period of training focused on advanced tactical training, such as defense of the amphibious task force, and multiunit ship-to-shore movement, among other objectives. According to a Navy official responsible for the development of this initiative, its primary focus is on advanced tactical training for Navy personnel, but greater integration with the Marine Corps may be needed to accomplish certain training objectives, such as air defense. Further, it would provide an opportunity for the Marine Corps to achieve additional amphibious operations training. However, according to this official, the Marine Corps did not provide input into how its capabilities could be fully incorporated into the Navy’s advanced tactics training or identify potential opportunities to maximize amphibious operations training for both services. Further, the Marine Corps officials told us that there are opportunities to use transit time during Navy community-relations events, such as port visits, to conduct amphibious training for home-station units, but these events are not always identified with enough lead time to take full advantage of the training opportunity. According to officials at II MEF, Marine Corps units typically need at least 6 months of advance notice to align their forces and equipment for the potential training opportunity. Further, Marine Corps officials told us that the Navy does not always have a fully trained staff with the amphibious ship during these events, which can limit the comprehensiveness of the training that Marine Corps units are able to accomplish. These officials also stated that the flight deck or well deck may not be certified for use at the time of these community- relations events, further limiting their utility for Marine Corps training. Despite these limitations, Marine Corps officials have told us that these events can still provide training benefits, such as ship familiarization for Marines, but that these opportunities still require advanced notice. By improving coordination over its training resources, the services will be better positioned to take full advantage of these scarce training opportunities. Mechanisms to monitor results and reinforce accountability: The Navy and Marine Corps have processes to evaluate and report on the results of specific training exercises, but have not developed mechanisms to monitor, evaluate, and report on results nor jointly reinforced accountability for their naval integration efforts through agency plans and reports. We have found that agencies need to monitor and evaluate their efforts to enable them to identify areas for improvement and help decision makers obtain feedback for improving operational effectiveness. Further, agency plans and reports can reinforce accountability by aligning goals and strategies with the collaborative effort. For large-scale exercises, such as Bold Alligator, the Marine Corps conducts reviews that identify actions that should be sustained moving forward, as well as areas that should be improved in future exercises, including issues related to naval integration. However, the services have not established other processes or mechanisms to monitor, evaluate, and report on results that are needed to measure progress in achieving service-level goals for naval integration and to align efforts to maximize training opportunities for amphibious operations. For example, the Marine Corps does not have a process to monitor and report on results for the critical tasks identified in its Marine Corps Operating Concept, including those tasks related to naval integration, such as integrating command structures, developing concepts for littoral operations in a contested environment, and conducting expeditionary advanced base operations. Monitoring progress against these tasks, as well as common outcomes, once defined, should help the Navy and Marine Corps track progress toward achieving improved naval integration. While the Navy and Marine Corps have taken some steps to improve naval integration in recent years, these efforts are still in the early stages. In particular, Navy and Marine Corps officials stated that the services have not yet defined or articulated common outcomes needed to achieve naval integration because they have not determined who would be responsible for this effort or when to begin its development. Defining and articulating common outcomes for naval integration would allow the services to more effectively incorporate other leading collaboration practices aimed at those common outcomes, to the extent deemed appropriate, such as developing a joint strategy, establishing compatible policies, leveraging resources, and monitoring results. The Marine Corps Has Not Fully Integrated Its Virtual Training Devices into Operational Training The Marine Corps has taken some steps to better integrate virtual training devices into its operational training. However, the Marine Corps’ process to manage the development and use of its virtual training devices in operational training plans has gaps. The Marine Corps Has Taken Some Steps to Integrate Virtual Training Devices into Operational Training The Marine Corps has taken some steps to integrate virtual training devices into operational training and has other efforts under way. In 2013, we reported that the Marine Corps did not have information on the performance and cost of virtual training that would assist the service in assessing and comparing the benefits of virtual training as it sought to optimize the mix of live and virtual training to meet requirements and prioritize training investments. We also found that the Marine Corps had not developed overall metrics or indicators to measure how the use of virtual training devices had contributed to improving the effectiveness of training, or identified a methodology to identify the costs associated with using virtual training. We recommended that the Marine Corps develop outcome-oriented performance metrics for assessing the effect of virtual training on improving performance or proficiency and develop a methodology to identify the costs of virtual training in order to compare the costs of using live and virtual training. Further, in 2015 the Commandant of the Marine Corps issued guidance that stated the service will focus on better leveraging virtual training technology and that all types of Marine Corps forces should make extensive use of virtual training where appropriate. In response to our recommendations and the Commandant’s guidance, in 2015 the Marine Corps Training and Education Command created a Simulation Assessment Working Group with stakeholders from across the Marine Corps to identify training events that could be supported by virtual training devices and incorporate those devices into Training and Readiness manuals. The working group found that over 7,000 of the 12,000 training events reviewed could use a virtual training device to either fully or partially meet the training standard of that event. The group also identified 135 events that may only be performed using the virtual training device or must be performed with the device as a prerequisite to live training. Based on the results of the working group, Training and Education Command updated the corresponding unit-specific Training and Readiness manuals to identify where a training event could be completed using a virtual training device. While this action represents some progress toward better incorporating virtual training devices into operational training, our recommendations remain open because the Marine Corps’ efforts to develop specific outcome-oriented performance metrics to assess virtual training or a methodology to make more- informed comparisons between the costs of live and virtual training are not yet complete. According to a senior Training and Education Command official, the Marine Corps is working to update its training information management system to better capture this information. In 2015, the Marine Corps also issued a Concept of Operations (CONOPS) for the United States Marine Corps Live, Virtual, and Constructive – Training Environment (LVC-TE) (hereafter referred to as Concept of Operations) that is intended to describe the live, virtual, and constructive training environment based on operational requirements in sufficient detail to continue the development of this training capability. According to the Concept of Operations, the goal in implementing the live, virtual, and constructive training environment is to expand training opportunities, reduce training costs, improve safety, and maintain high levels of proficiency and readiness. The Concept of Operations estimates that the live, virtual, and constructive training environment will be implemented in 2022. Lastly, the Marine Corps has an ongoing effort to better inform users of the availability of virtual training devices that support ground-based units. Specifically, the Marine Corps Training and Education Command is developing a Ground Training Simulations Implementation Plan that is intended to provide a framework for the use of current and future virtual training devices for ground units. The Ground Training Simulations Implementation Plan is modeled after the processes used by the Marine Corps’ aviation community to integrate simulators into aviation training. The Marine Corps estimates that the plan will be finalized in the summer of 2017. According to a Training and Education Command official involved in the plan’s development, the plan will help address a challenge the Marine Corps has faced in educating commanders on the availability and capabilities of available virtual training devices. This challenge is consistent with information we gathered during our visit to selected Marine Corps installations. Officials at the two Battle Simulation Centers we visited, for example, told us that unit commanders do not always know what virtual training devices are available and how they can be used to meet training requirements. Marine Corps Process to Manage the Development and Use of Virtual Training Devices in Operational Training Plans Has Gaps The Marine Corps process to manage the development and use of virtual training devices in operational training plans has gaps due to a lack of guidance. Specifically, the Marine Corps does not (1) include consideration of critical factors for integrating virtual training devices into operational training in its front-end planning to support the acquisition of its virtual training devices, (2) consistently consider expected and actual usage data for virtual training devices to support its investment decisions, or (3) consistently evaluate the effectiveness of its virtual training devices for operational training. Front-End Planning The Marine Corps’ process for conducting front-end planning and analysis to support the acquisition of its virtual training devices does not include consideration of critical factors for integrating virtual training devices into operational training, such as the specific training tasks the device is intended to address, how the device would be used to meet proficiency goals, or available time for units to train with the device. DOD’s Strategic Plan for the Next Generation of Training for the Department of Defense states that the right mix of live, virtual, and constructive training capabilities will depend on training tasks and objectives, required proficiency, and available training time, among other factors. In addition, we have previously found that part of the front-end analysis process for training and development programs should include a determination of the skills and competencies in need of training and how training will build proficiency for those skills and competencies. Based on our analysis of the Marine Corps’ front-end planning documents (called system development documents) for the six virtual training devices included in our review, we found that documentation for five of the six devices did not include specific training tasks. In addition, the documentation for two devices specified that specific training tasks would be identified during the verification and validation phase, which is a type of analysis that typically takes place after the device has already been acquired, according to a senior Training and Education Command official. While the documentation for all of the devices included a high- level discussion of relevant mission areas, documentation for five out of six devices did not identify specific training tasks, such as specific training events in a unit’s Training and Readiness manual, that the device was intended to address. For example, documentation for the Combined Arms Command and Control Training Upgrade System includes a high-level discussion of mission areas that the device supports, such as force application, command and control, and battlespace awareness. It also states that the device is to support training events, but it does not specify what those events are. In addition, none of the system development documents we reviewed identified proficiency goals or considered available training time for the units to use the device. According to officials at Training and Education Command, many virtual training devices in the Marine Corps’ inventory were developed based on urgent needs to meet capability gaps identified by warfighters and were not based on training requirements. Of the six devices included in our review, three of the devices were acquired to meet urgent warfighter needs—the Family of Egress Trainers—Modular Amphibious Egress Trainer, the Operator Driver Simulator, and the Supporting Arms Virtual Trainer. However, the system development documents we reviewed for those three devices were completed after the devices had been fielded to meet the urgent needs, but still did not identify specific training tasks or proficiency goals, or consider available training time for the units to use the device. Moreover, the system development documents for two of the remaining three devices we reviewed did not contain this information. While the Marine Corps did not identify and assess these factors in the front-end planning process, the Marine Corps has begun taking steps to identify these factors through efforts such as the Simulation Assessment Working Group. However, these efforts are occurring after the devices have already been acquired and fielded, leading to decisions that have potential cost implications. For example, in its analysis, the Simulation Assessment Working Group did not fully consider alternative devices that could be used to achieve specific training tasks because its methodology was to identify the one virtual training device that was considered the “best in breed” simulator for conducting each training event rather than considering all devices that could be used for the event, including those that might be more cost-effective. Officials at II MEF told us that this methodology did not include an evaluation of the device’s cost compared to other devices that could achieve similar training outcomes. For example, these officials told us that the Supporting Arms Virtual Trainer was identified as a “best in breed” device for a number of training events, including calls for fire and close air support. However, these officials stated that the Deployable Virtual Training Environment device is a lower- cost alternative that could achieve similar outcomes for many of the training events that do not require the level of realism provided by the Supporting Arms Virtual Trainer. Based on information provided by Training and Education Command, the acquisition cost for the Supporting Arms Virtual Trainer is about $4.5 million per system while the acquisition cost for the Deployable Virtual Training Environment laptop is around $3,700 (see fig. 6). The Marine Corps’ front-end planning process to support the acquisition of virtual training devices has gaps because the service does not have specific policies to ensure the process considers key factors. Specifically, Navy and Marine Corps acquisition policies we reviewed do not require that front-end planning consider specific training tasks the device is intended to address, how the device would be used to meet proficiency goals, or available time for units to train with the device. Training and Education Command officials acknowledged the gaps in the Marine Corps’ process and stated that the front-end process for future device acquisitions would identify specific training tasks that a device will address. However, without guidance that specifically addresses these factors, the Marine Corps does not have a reasonable basis to ensure that it is acquiring the right number and type of virtual training devices to meet its operational training needs. Expected and Actual Usage Data The Marine Corps does not consistently consider expected and actual usage data for virtual training devices to support its investment decisions. Our prior work has found that agencies should establish measures that they can use in assessing training programs, such as expected training hours, which reflect the usage rates of the training program. However, the Marine Corps did not establish expected usage rates in its system development documents for five of the six virtual training devices included in our review, and a senior Training and Education Command official said it also has not established expected usage rates since acquiring the devices. For example, the system development document for the Supporting Arms Virtual Trainer stated that the usage of the device could replace up to 33 percent of the live-fire missions required to retain annual currency, but the document does not specify that units are expected to use the device to replace that high of a percentage of the live-fire missions. As a result, the Marine Corps does not have a baseline against which to assess actual usage of the device. Only the system development document for the Marine Air-Ground Task Force Tactical Warfare Simulation included usage targets, stating that usage is expected to be extensive and estimates that the device will be used for 700 hours per system per year. However, the system development documents for the other four devices we reviewed did not include any information on expected usage rates. Additionally, the Marine Corps has not consistently collected actual usage data for its virtual training devices, which could be used to inform continued investments in existing virtual training devices. During our review, a senior Marine Corps Training and Education Command official told us that Training and Education Command collects data for about two- thirds of the Marine Corps’ total inventory of virtual training devices, but usage data are not available for certain devices. More specifically, the Marine Corps provided usage data for three of the six devices that were included in our review, but it was unable to provide usage data for certain systems, such as the Marine Air-Ground Task Force Tactical Warfare Simulation and the Combined Arms Command and Control Training Upgrade System. This official stated that contractors collect data on these devices, but there is no Marine Corps’ system to collect data on the number of Marines or hours trained. Specifically, contractors submit spreadsheets on a monthly basis showing the number of Marines who have used the device, but these data are not included in any formal reports and there is no standard database for collecting or evaluating them. The Marine Corps has not considered actual usage data in its decision making for additional investments in certain virtual training devices, despite low usage rates for a number of those devices. For example, according to available contractor data, actual usage for the Operator Driver Simulator was significantly lower than the current available hours. Based on data provided by Training and Education Command, the Operator Driver Simulator was used for approximately 7,600 hours in fiscal year 2015 and 5,600 hours in fiscal year 2016, but was available for use for approximately 192,000 hours. However, based on the results of the Simulation Assessment Working Group, Training and Education Command estimated that to accomplish all training events linked to the Operator Driver Simulator would require about 570,000 available training hours. As a result, the Simulator Assessment Working Group recommended various investment options for the Operator Driver Simulator that ranged from $56 million to $121 million, despite the current low utilization and excess capacity. Officials from Training and Education Command told us that they anticipate an increase in user demand for the Operator Driver Simulator based on guidance from the Commandant of the Marine Corps to make driver certification more rigorous. However, officials from Marine Corps Systems Command stated that current Operator Driver Simulators have deficiencies in supporting driver training and, therefore, Marines choose to drive live vehicles instead. The Marine Corps has not considered expected and actual usage of its virtual training devices to support investment decisions due to a lack of guidance on establishing and collecting usage data. Marine Corps training guidance for ground units states that virtual training devices shall be used, as applicable, when constraints limit the use of realistic training conditions, but it does not identify the extent to which virtual training devices are expected to be used. Without guidance on setting usage- rate expectations and assessing actual usage, the Marine Corps risks sustained investment in virtual training devices that do not meet operational training needs. Evaluate the Effectiveness of Devices We also found that the Marine Corps was not consistently evaluating the effectiveness of its virtual training devices to accomplish operational training. Our prior work has shown that agencies need to develop processes that systematically plan for and evaluate the effectiveness of their training and development efforts. These evaluations should include data measures, both quantitative and qualitative, to assess training results in areas such as increased user proficiency. Further, evaluations of training effectiveness should be used to make decisions on whether resources should be reallocated or redirected. The Marine Corps uses the verification and validation report process as its primary assessment of a virtual training device after it has been fielded, according to the senior Training and Education Command official with whom we spoke. However, based on our review of postfielding analyses for the virtual training devices included in our review, we found that the Marine Corps does not have a consistent process for selecting devices for which to complete these analyses or how the analysis should be conducted. More specifically, we were provided with verification and validation reports for only three of the six devices in our review—the Supporting Arms Virtual Trainer, the Family of Egress Trainers—Modular Amphibious Egress Trainer, and the Operator Driver Simulator—as well as plans to complete these reports for two other devices. According to a senior Training and Education Command official, Training and Education Command considers certain factors to prioritize the completion of verification and validation reports, such as planned investments for major upgrades on a device. The official also stated that Training and Education Command prioritized completing reports for these virtual training devices to specifically align with recommendations made by the Simulation Assessment Working Group. However, the Simulation Assessment Working Group does not take place on a recurrent basis, and therefore the recommendations from the group do not establish a process for prioritizing future verification and validation reports. Officials from Marine Corps Systems Command told us that program managers are now trying to perform verification and validation reports for future acquisitions prior to full acceptance of the training systems, but that this step is not mandatory. Additionally, there is not a consistent process to include training effectiveness evaluations within the verification and validation report itself. The verification and validation process is not required to include an evaluation of effectiveness based on current guidance, but as noted in the verification and validation report for the Family of Egress Trainers— Modular Amphibious Egress Trainer, such an evaluation is essential to determine whether the capabilities of a virtual training device satisfy requirements to improve training performance and combat readiness. In two instances, the verification and validation reports for the Operator Driver Simulator and Family of Egress Trainers—Modular Amphibious Egress Trainer both included evaluations of the effectiveness of the devices in improving user proficiency, which concluded that the devices enabled Marines to successfully pass related training courses. In another instance, the Marine Corps did not conduct a training effectiveness analysis as part of the verification and validation process. Specifically, for the Supporting Arms Virtual Trainer, Marine Corps Systems Command attempted to conduct a training effectiveness evaluation, but training activity data for a statistically significant sampling of the target training audience were unavailable, which suggests the need for improved data on device usage. We further found that the training effectiveness evaluations that the Marine Corps did complete differed in how they were conducted, which can affect the quality of the information the evaluations provide. For example, the training effectiveness evaluation for the Operator Driver Simulator was conducted to determine whether the device effectively trained Marines to perform tasks required for one specific training and readiness event. The methodology included collecting training activity data from 1 fiscal year in one location and for one of the Operator Driver Simulator vehicle variants. The report noted that conducting a more- complete evaluation, along with additional data collection, would better identify opportunities to improve and enhance training. In contrast, the training effectiveness evaluation for the Family of Egress Trainers— Modular Amphibious Egress Trainer also collected training activity data, but collected data from multiple training sites and for all training courses conducted during the 1-year period used for the evaluation. According to officials from Marine Corps Systems Command, the effectiveness evaluation methods may vary based on the type of training being executed and how well the training requirements are defined. These officials stated that when the device’s training requirements have been more thoroughly defined, the effectiveness evaluation can be more targeted. The Navy and Marine Corps acquisition policy and guidance documents we reviewed do not establish a process to consistently evaluate the training effectiveness of virtual training devices, including identifying the devices to be evaluated and determining what data should be collected and assessed. According to a senior Training and Education Command official, evaluating effectiveness is not a required part of the verification and validation process and is an area that needs to be addressed. The Marine Corps’ Concept of Operations also identified a lack of guidance for conducting effectiveness analyses. Specifically, the Concept of Operations identifies a lack of policy guiding live, virtual, and constructive training capabilities and benefits. It also identifies a training gap on the linkages between live, virtual, and constructive training, as well as a policy gap around the lack of guidance on analysis of virtual training devices after they have been fielded. Without guidance establishing a well-defined process to consistently evaluate the effectiveness of virtual training devices for training—including the selection of devices, guidelines on conducting the analysis, and the data that should be collected and assessed—the Marine Corps risks investing in devices whose value to operational training is undetermined. Conclusions The Navy and Marine Corps have identified the need to rebuild the capability to conduct amphibious operations and to reinvigorate naval integration between the services toward that end. However, the Navy and Marine Corps have not completed efforts needed to mitigate their training shortfalls for amphibious operations. Specifically, the services have not developed an approach to prioritize available training resources, systematically evaluate among training resource alternatives to achieve amphibious operations priorities, and monitor progress toward achieving them. Without such an approach, the services are not well positioned to mitigate existing amphibious operations training shortfalls and begin to rebuild their amphibious capability as the services await the arrival of additional amphibious ships into the fleet. In addition, while the Navy and Marine Corps have taken a number of positive steps to improve coordination between the two services, they need to define and articulate common outcomes for naval integration. This first critical step will enable them to fully incorporate other leading collaboration practices aimed at a common purpose, such as developing a joint strategy; more fully establishing compatible policies, procedures, and systems; better leveraging resources; and establishing mechanisms to monitor results that are needed to achieve service-level goals for naval integration and to align efforts to maximize training opportunities for amphibious operations. Further, the Marine Corps’ process to integrate virtual training devices into operational training has gaps. Developing guidance for the development and use of virtual training devices would help close these gaps, which is critical as virtual training will become increasingly important to the development of the capability of Marines, including the capability for conducting amphibious operations, among other mission areas. Recommendations for Executive Action To better mitigate amphibious operations training shortfalls, we recommend the Secretary of Defense direct the Secretary of the Navy, in coordination with the Chief of Naval Operations and Commandant of the Marine Corps, to develop an approach, such as building upon the Amphibious Operations Training Requirements review, to prioritize available training resources, systematically evaluate among training resource alternatives to achieve amphibious operations priorities, and monitor progress toward achieving them. To achieve desired goals and align efforts to maximize training opportunities for amphibious operations, we recommend the Secretary of Defense direct the Secretary of the Navy, in coordination with the Chief of Naval Operations and Commandant of the Marine Corps, to clarify the organizations responsible and time frames to define and articulate common outcomes for naval integration, and use those outcomes to develop a joint strategy; more fully establish compatible policies, procedures, and systems; better leverage training resources; and establish mechanisms to monitor results. To more effectively and efficiently integrate virtual training devices into operational training, we recommend that the Secretary of Defense direct the Commandant of the Marine Corps to develop guidance for the development and use of virtual training devices that includes developing requirements for virtual training devices that consider and document training tasks and objectives, required proficiency, and available training time; setting target usage rates and collecting usage data; and conducting effectiveness analysis of virtual training devices that defines a consistent process for performing the analysis, including the selection of the devices to be evaluated, guidelines on conducting the analysis, and the data that should be collected and assessed. Agency Comments We provided a draft of the classified report to DOD for review and comment. The department’s comments on the classified report are reprinted in Appendix II. In its comments, DOD concurred with all three recommendations. DOD stated that it will review the status of actions the Navy and Marine Corps plan to take in response to all three recommendations within the next twelve months. We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, the Office of the Under Secretary of Defense for Personnel and Readiness, the Secretary of the Navy, and the Commandant of the Marine Corps. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you have any questions about this report or need additional information, please contact me at (202) 512-5431 or russellc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III. Appendix I: Objectives, Scope, and Methodology The objectives of this report are to determine the extent to which (1) the Navy and Marine Corps have completed training for amphibious operations priorities and taken steps to mitigate any training shortfalls, (2) the Navy’s and Marine Corps’ efforts to improve naval integration for amphibious operations incorporate leading collaborative practices, and (3) the Marine Corps has integrated selected virtual training devices into its operational training. This report is a public version of a classified report that we issued in August 2017. DOD deemed some of the information in our August report to be classified, which must be protected from loss, compromise, or inadvertent disclosure. Therefore, this report omits classified information on select Marine Corps units’ ability to complete training for amphibious operations. Although the information provided in this report is more limited, the report addresses the same objectives as the classified report and uses the same methodology. We focused our review on Navy and Marine Corps organizations and units that have a role in the development and execution of training requirements for amphibious operations. For the Navy, we focused on the training requirements and accomplished training for amphibious ships. For the Marine Corps, we focused on selected active-component units that have identified training requirement for amphibious operations, including Marine Expeditionary Units (MEU) and other units with a mission-essential task for amphibious operations. We selected a nongeneralizable sample of 23 Marine Corps units to speak with in order to interview geographically dispersed units under each Marine Expeditionary Force, as well as units across all elements of the Marine Air-Ground Task Force (i.e., command, ground combat, aviation combat, and logistics combat forces). See below for the list of 23 Marine Corps units. We focused on the Marine Corps’ integration of virtual training devices into operational training because the Navy does not have virtual training devices that simulate amphibious operations, including ship-to- shore movement, according to Navy officials. In addition, we focused on Marine Corps virtual training devices that are used to support the command and ground elements of the Marine Air-Ground Task Force. We selected a nongeneralizable sample of six virtual training devices based on the target training audience, applicability to amphibious operations training, location, and type of training events (individual or collective training) for which the devices are used. The devices included in our review are the Combined Arms Command and Control Training Upgrade System, Marine Air-Ground Task Force Tactical Warfare Simulation, Supporting Arms Virtual Trainer, Amphibious Assault Vehicle Turret Trainer, Family of Egress Trainers—Modular Amphibious Egress Trainer, and Operator Driver Simulator. To determine the extent to which the Navy and Marine Corps have completed training for amphibious operations priorities and taken steps to mitigate any training shortfalls, we analyzed deployment certification reports for all Amphibious Ready Group (ARG)—Marine Expeditionary Unit (MEU) deployments over the most-recent 3-year period. We also analyzed unit-level readiness data for all Marine Corps’ infantry battalions, assault amphibian vehicle battalions, Osprey tilt-rotor aircraft squadrons, and Marine Expeditionary Brigades over the most-recent 3- year period—from fiscal years 2014 through 2016—and compared those data against unit-level training requirements for amphibious operations. We analyzed 3 years of training data because training requirements for Marine Corps units are reviewed and updated on a 3-year cycle. We performed data-reliability procedures on the unit-level readiness data by comparing the data against related documentation and surveying knowledgeable officials on controls over reporting systems and determined that the data presented in our findings were sufficiently reliable for the purposes of this report. We interviewed Navy and Marine Corps officials to discuss any factors that limited their ability to conduct training for amphibious operations. We assessed the reliability of data on amphibious ship requests by speaking with knowledgeable officials and determined the data were sufficiently reliable for the purposes of presenting the number of actual requests submitted and fulfilled. In addition, we reviewed processes and initiatives established by the Navy and Marine Corps to identify and assess training shortfalls for amphibious operations, including the Marine Corps’ Amphibious Operations Training Requirements review, and evaluated these processes and initiatives against our prior work on strategic training and risk management. To determine the extent to which the Navy’s and Marine Corps’ efforts to improve naval integration for amphibious operations incorporate leading collaboration practices, we reviewed the Navy and Marine Corps documents, including A Cooperative Strategy for 21st Century Seapower and the Marine Corps Operating Concept, that discuss the goal of improving naval integration. We also reviewed mechanisms that have been established to coordinate training, including campaign plans for amphibious operations; observed a working group focused on amphibious operations; and interviewed officials with both services to discuss efforts to improve naval integration. We assessed the extent to which the Navy’s and Marine Corps’ efforts toward improving naval integration have followed leading practices for collaboration that we have identified in our prior work. Specifically, we have identified eight practices described in our prior work that can help enhance and sustain collaboration. We selected seven of the eight practices most relevant to issues we identified in our prior work on collaboration to assess the status of Navy and Marine Corps collaborative efforts to improve naval integration. Based on our analysis, we selected the following seven practices: define and articulate a common outcome; establish mutually reinforcing or joint strategies; identify and address needs by leveraging resources; agree on roles and responsibilities; establish compatible policies, procedures, and other means to operate across agency boundaries; develop mechanisms to monitor, evaluate, and report on results; and reinforce agency accountability for collaborative efforts through agency plans and reports. To determine the extent to which the Marine Corps has integrated selected virtual training devices into its operational training, we collected information on the development, usage, and evaluation of virtual training devices, and their integration into operational training plans. We reviewed documentation on actions the Marine Corps has taken to integrate its virtual training devices into operational training, including documentation on the Simulation Assessment Working Groups and the Ground Training Systems Plan. We reviewed DOD and Marine Corps acquisition policies and interviewed Marine Corps officials responsible for the acquisition and oversight of virtual training devices at Training and Education Command and Marine Corps Systems Command and officials responsible for management of the virtual training devices at the Battle Simulation Centers at Camp Lejeune, North Carolina, and Camp Pendleton, California. We reviewed acquisition documents for each of the selected devices, including Capability Production Documents and Capability Development Documents, and assessed the extent to which these documents included key information as identified in leading practices for managing strategic training and DOD’s Strategic Plan for the Next Generation of Training for the Department of Defense. We also reviewed documentation on the Marine Corps process to include expected and actual usage data for virtual training devices to support investment decisions. Further, we reviewed analyses conducted after the selected devices had been fielded through Verification and Validation Reports and evaluated the extent these documents assessed the effectiveness of the virtual training devices for improving user proficiency. The performance audit upon which this report is based was conducted from May 2016 to August 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate, evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. We subsequently worked with DOD from August 2017 to September 2017 to prepare this unclassified version of the original classified report for public release. This public version was also prepared in accordance with these standards. Navy Marine Corps Appendix II: Comments from the Department of Defense Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact name above, Matthew Ullengren, Assistant Director; Russell Bryan; William Carpluk; Ron La Due Lake; Joanne Landesman; Kelly Liptan; Shahrzad Nikoo; and Roxanna Sun made key contributions to this report.
target: The Navy and Marine Corps have identified a need to improve their ability to conduct amphibious operations—an operation launched from the sea by an amphibious force. Senate and House reports accompanying bills for the National Defense Authorization Act for Fiscal Year 2017 included provisions for GAO to review Navy and Marine Corps training. This report examines the extent to which (1) the Navy and Marine Corps have completed training for amphibious operations priorities and taken steps to mitigate any training shortfalls, (2) these services' efforts to improve naval integration for amphibious operations incorporate leading collaboration practices, and (3) the Marine Corps has integrated selected virtual training devices into operational training. GAO analyzed training initiatives; interviewed a nongeneralizable sample of officials from 23 units that were selected based on their training plans; analyzed training completion data; and selected a nongeneralizable sample of six virtual training devices to review based on factors such as target audience. This is a public version of a classified report GAO issued in August 2017. Information that DOD deemed classified has been omitted. Navy and Marine Corps units that are deploying as part of an Amphibious Ready Group and Marine Expeditionary Unit (ARG-MEU) completed their required training for amphibious operations, but other Marine Corps units have been limited in their ability to conduct training for other amphibious operations–related priorities. GAO found that several factors, to include the decline in the fleet of the Navy's amphibious ships from 62 in 1990 to 31 today limited the ability of Marine Corps units to conduct training for other priorities, such as recurring training for home-station units (see figure). As a result, training completion for amphibious operations was low for some but not all Marine Corps units from fiscal years 2014 through 2016. The services have taken steps to address amphibious training shortfalls, such as more comprehensively determining units that require training. However, these efforts are incomplete because the services do not have an approach to prioritize available training resources, evaluate training resource alternatives, and monitor progress towards achieving priorities. Thus, the services are not well positioned to mitigate any training shortfalls. The Navy and Marine Corps have taken some steps to improve coordination between the two services, but have not fully incorporated leading collaboration practices to improve integration of the two services—naval integration—for amphibious operations. For example, the Navy and Marine Corps have not defined and articulated common outcomes for naval integration that would help them align efforts to maximize training opportunities for amphibious operations. The Marine Corps has taken steps to better integrate virtual training devices into operational training, but gaps remain in its process to develop and use them. GAO found that for selected virtual training devices, the Marine Corps did not conduct front-end analysis that considered key factors, such as the specific training tasks that a device would accomplish; consider device usage data to support its investment decisions; or evaluate the effectiveness of existing virtual training devices because of weaknesses in the service's guidance. As a result, the Marine Corps risks investing in devices that are not cost-effective and whose value to operational training is undetermined.