Air Force Training: Further Analysis and Planning Needed to Improve Effectiveness
Published: 2016-09-19T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: For more than a decade, the Air Force focused its training on supporting operations in the Middle East. The Air Force has established goals for its combat aircrews to conduct training for the full range of core missions. Both the Senate and House Reports accompanying bills for the FY 2016 National Defense Authorization Act included a provision for GAO to review the Air Force's training plans.
This report discusses the extent to which the Air Force has (1) determined requirements to train combat aircrews for the full range of core missions, (2) met annual training requirements for combat fighter squadrons across the full range of core missions and evaluated the effectiveness of this training, and (3) established virtual training plans that include desirable characteristics of a comprehensive strategy. GAO reviewed Air Force training requirements and plans and interviewed officials with a non-generalizable sample of units based on the units' range of core missions.
section_title: What GAO Found
paragraphs: The Air Force establishes combat aircrew training requirements for the full range of core missions based on an annual process, but these requirements may not reflect current and emerging training needs, because the Air Force has not comprehensively reassessed the assumptions underlying them. Specifically, assumptions about the total annual live-fly sortie requirements by aircraft, the criteria for designating aircrews as experienced or inexperienced, and the mix between live and simulator training have remained the same since 2012. For example, Air Combat Command has set the same minimum number of live-fly sortie requirements across aircraft platforms, but has not conducted the analysis needed to determine if requirements should differ based on the number of core missions for each platform. Reassessing the assumptions underlying annual training requirements would better position the Air Force to meet its stated goals for its forces to achieve a range of missions for current and emerging threats.
Combat fighter squadrons were generally able to complete mission training requirements for ongoing contingency operations, such as close air support to ground forces, but were unable to meet annual training requirements across the full range of core missions. Further, the Air Force does not systematically evaluate the effectiveness of training that has been completed against established expectations. Selected unit commanders that GAO interviewed cited four common factors that limited their ability to complete training, such as high deployment rates, and other factors that affected the training that aircrews were able to accomplish. However, Air Force processes used to record and monitor annual training do not include a systematic evaluation of training effectiveness against expectations. Specifically, Air Combat Command has not established the desired learning objectives or training support elements needed to accomplish training expectations and does not collect data to assess effectiveness. A more consistent basis for monitoring results is critical in tracking the Air Force's progress in training units for the full range of core missions.
Air Force plans for virtual training do not include all desirable characteristics of a comprehensive strategy, such as a risk-based investment strategy or a time line for addressing training needs. A strategy that included these elements would help ensure that the Air Force's plans addressed its capability needs.
This is a public version of a sensitive report GAO issued in August 2016. It omits sensitive information and data on some of the Air Force's training priorities, completion of annual training requirements for active-duty fighter squadrons, and aircraft maintenance generation capabilities.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Air Force (1) reassess assumptions for annual training requirements, (2) establish and collect data on desired learning objectives and training support elements for its training expectations, and (3) develop a risk-based investment strategy for its virtual training plans. DOD concurred with the third recommendation and did not concur with the first and second recommendations, stating that existing initiatives and policies address these issues. GAO believes the recommendations remain valid for the reasons discussed in this report.

section_title: Letter
paragraphs: For more than a decade, the Air Force focused the training of its forces  on supporting operations in the Middle East, including Iraq and  Afghanistan. Commanders established training requirements that they  deemed necessary to prepare aircrews to conduct missions in these  locations—such as close air support to ground forces—limiting training  opportunities in other areas. According to Air Force reports, in the coming  years, the Air Force will confront an increasingly complex security  environment that will demand a wider range of skill sets and different  capabilities than are currently being employed. For example, aircrews  may be called upon to conduct missions that require freedom of  maneuver in highly-contested air spaces. To meet these emerging needs,  the Air Force has established goals for its fighter aircrews to conduct  training for a full range of core missions.
Replicating complex threats—such as a highly-contested airspace—in a  live training environment is resource intensive and can be constrained by  several factors, including airspace restrictions, weather, and range  capabilities, among others. The Air Force has stated that by integrating  live, virtual, and constructive (LVC) components into training it could  overcome some of the limitations of training in a live-only environment but  notes that it will take time and dedicated resources to address  technological and capacity limitations. The Air Force has further stated  that the integration of these components is particularly important for fifth- generation aircraft, such as the F-35, which will rely more heavily on  virtual training because of the limitations of training in the live  environment and security considerations attributable to the sensitive  capabilities of these aircraft.
We have previously reported on virtual training issues for the Air Force. In  2012, we assessed the Air Force’s virtual training efforts in response to  the Secretary of Defense’s efficiency initiatives. We found that the Air  Force lacked some key elements of an overarching organizational  framework that were needed to fully integrate virtual training into overall  training goals and that it had no methodology for determining the costs of  its virtual training enterprise. We recommended that the Air Force  designate an entity to integrate its virtual training efforts, develop a  strategy to align virtual training initiatives and goals, and develop a  methodology to collect cost data on virtual training. In response, the Air  Force has taken a series of actions to improve the planning and  management of its virtual training efforts. For example, in 2012 the  Secretary of the Air Force issued a guidance letter that restructured the  management of virtual training responsibilities within Headquarters Air  Force and designated the Air Force Agency for Modeling and Simulation  as the lead implementation organization for LVC operational training  integration. According to Air Force officials, the Air Force is also in the  process of updating guidance on the management, acquisition,  modification, and modernization of simulator and other training devices.  We analyze other actions the Air Force has taken related to virtual  training later in this report.
Both the Senate and House reports accompanying the bills for the 2016  National Defense Authorization Act included provisions for us to review  the Air Force’s training plans and requirements. This report discusses  the extent to which the Air Force has (1) determined requirements to train  its combat aircrews for the full range of core missions, (2) met annual  training requirements for combat fighter squadrons across the full range  of core missions and evaluated the effectiveness of this training, and (3)  established virtual training plans that include desirable characteristics of a  comprehensive strategy.
This report is a public version of the prior sensitive report that we issued  in August 2016. The Department of Defense (DOD) deemed some of the  information in the prior report as For Official Use Only (FOUO), which  must be protected from public disclosure. Therefore, this report omits  FOUO information and data on some of the Air Force’s training priorities,  completion of annual training requirements for active-duty fighter  squadrons, and aircraft maintenance generation capabilities. Although the  information provided in this report is more limited in scope, it addresses  the same objectives as the sensitive report. Also, the methodology used  for both reports is the same.
We focused our review on the annual requirements for continuation  training as established in the Ready Aircrew Program for aircrews  assigned to aircraft within the active component of the combat air forces.  The aircraft included in our review were the A-10, F-15, F-16, F-22, and  F-35. To determine the extent to which the Air Force has determined  requirements to train its combat aircrews for the full range of core  missions, we analyzed Air Force documents to identify the Air Force’s  current process for developing annual training requirements and reviewed  changes in these requirements for fiscal years 2012 through 2016. We  evaluated these processes against DOD and Air Force guidance that  prioritizes and establishes training requirements and leading practices  that we have identified for managing strategic training.
To determine the extent to which the Air Force has met annual training  requirements for combat fighter squadrons across the full range of core  missions and evaluated the effectiveness of this training, we analyzed  training completion data from fiscal years 2012 through 2015 for the  combat air forces. We performed data reliability procedures on  information included in Ready Aircrew Program reports by comparing the  data against related documentation and interviewing knowledgeable  officials on controls over reporting systems and determined that the data  presented in our findings were sufficiently reliable for the purposes of this  report. We reviewed Air Force documentation that describes training  challenges and interviewed Air Force officials, including wing and  squadron commanders at selected units, to discuss any factors that  limited the ability of aircrews to complete training for the full range of core  missions. We selected a non-generalizable sample of units to speak with  based on our analysis of training completion data and the range of units’  core missions. We evaluated the Air Force’s process for assessing the  effectiveness of its annual training against leading practices that we have  identified for managing strategic training and the Ready Aircrew Program  tasking memorandums that specify the requirements for annual  continuation training for personnel assigned to combat units.
To determine the extent to which the Air Force has established virtual  training plans that include desirable characteristics of comprehensive  strategies, we reviewed planning documents established by the Air Force  to guide its virtual training efforts and evaluated those actions against our  prior work on desirable characteristics of national strategies. We  determined whether these actions addressed, partially addressed, or did  not address the desirable characteristics of strategy documents, based on  our assessment of whether the planning documents explicitly cited all  elements of a characteristic and the level of specificity and detail  included. Further details on our scope and methodology can be found in  appendix I.
We conducted this performance audit from June 2015 to September 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The Secretary of the Air Force is responsible for recruiting, organizing,  training, and equipping forces in order to meet the current and future  operational requirements of DOD’s combatant commands. The Air  Force is organized into 10 major commands that are assigned a major  part of the Air Force mission. Numbered Air Forces fall directly under  the major commands and are focused on ensuring the readiness of  assigned forces. For example, Pacific Air Forces contains three  numbered air forces that are located in South Korea, Japan, and Alaska.  They provide operational leadership and supervision and are assigned  subordinate units, such as wings, groups, and squadrons. The  component numbered Air Force is a specialized category of numbered Air  Force that is structured to perform an operational and warfighting mission  in support of a combatant command. Air Force units, such as an  operational flying squadron, are designed to provide a specific set of  mission capabilities to meet anticipated operational requirements. See  figure 1 for general characteristics of the combat aircraft platforms  included in our review. Each combat unit has a Designed Operational  Capability statement that it uses for reporting readiness data and  includes, among other things, the full range of core missions that the unit  can be called on to provide. Air Combat Command has responsibility for  developing and managing aircrew training requirements to meet the full  range of core missions.
Throughout their careers, Air Force personnel designated as aircrews  participate in several different types of training. When they first enter the  Air Force, they receive basic military training. After basic military training,  aircrews receive additional training, including undergraduate training by  the Air Force’s Air Education and Training Command. Next, aircrews  receive initial qualification training that is focused on their career  specialties and is provided by the Air Education and Training Command  for the F-16, F-15C, and F-35 platforms and by Air Combat Command for  the A-10, F-15E, and F-22 platforms. After aircrews complete initial  qualification training, they are assigned to a unit, where they complete  mission qualification training—additional training on the skills specific to  the unit’s capabilities. Air Force Instruction (AFI) 11-202 is the  foundational aircrew training instruction that guides training programs for  all Air Force flying training. There is an AFI 11-2, Volume 1 for each  aircraft platform that specifies the training requirements for aircrews who  are assigned to units that operate that platform. For example, F-22 units  would use AFI 11-2F-22A, Volume 1. In addition to AFI 11-202, tasking  memorandums for the Ready Aircrew Program specify annual  continuation training for personnel assigned to combat units. Specifically,  Ready Aircrew Program tasking memorandums establish the minimum  number of live training events (“sorties”) and simulator missions (or virtual  training) that aircrews must complete during the annual training cycle to  be considered combat mission ready or basic mission capable for the  assigned aircraft. Finally, aircrews may complete upgrade training to  qualify for positions such as flight leader, instructor pilot, or mission  commander.  “Combat mission ready” is a squadron commander certification based on  the extent to which aircrews or units have completed and sustained  required training. Combat mission ready status is maintained by  completing the annual continuation training requirements as prescribed  by the unit’s AFI 11-2, Volume 1, and the Ready Aircrew Program tasking  memorandums. These requirements include completing a specific  quantity of relatively advanced individual and unit training events within a  given time period. For example, experienced F-22 aircrews must perform  11 defensive counter air training sorties in a 12-month period. In addition,  aircrews must achieve and maintain “currency” by completing certain  required training events deemed critical to safety of flight, such as  landings, low altitude flight, and formation take-offs. For example,  experienced F-22 aircrews are to land the aircraft every 45 days or else  they would lose landing currency. Currency events are typically achieved  by completing the annual training requirements.
Most Air Force combat units report the status of their unit training through  the Air Combat Command’s Ready Aircrew Program. The Ready Aircrew  Program establishes the minimum number of sorties and simulator  missions that aircrews must accomplish to sustain combat mission  readiness. These sorties and simulator missions are aligned with the  unit’s primary missions, for which the units must maintain “proficiency,”  and secondary missions, for which they must maintain “familiarity.” For  example, the fiscal year 2016 tasking memorandum for the F-15E aircraft  specifies that experienced aircrews are required to complete a minimum  of 96 annual live sorties (8 per month) to be considered combat mission  ready, of which 49 are flown to fulfill primary mission requirements.  Similarly, inexperienced aircrews are to complete 108 annual sorties (9  per month), of which 54 are flown to fulfill primary mission requirements.  Both experienced and inexperienced aircrews are required to complete  36 annual simulator missions (3 simulator missions per month) to be  considered combat mission ready. Units are required to submit an annual  end-of-cycle training report covering the 12-month training cycle. These  reports include information on the percentage of training accomplished by  the unit against the training requirements for its primary and secondary  missions, as well as any factors that limit the unit’s ability to meet its  training requirements, such as the availability of personnel or aircraft.
Virtual training uses aircraft simulators that replicate the live environment  for various missions and events. Aircraft simulators can be standalone or  linked to information networks that bring together geographically- separated simulators in order to conduct training. Training missions that  connect geographically-separated simulators are referred to as distributed  mission operations. Distributed mission operations utilize the integration  of virtual (e.g., a person training in a simulator) and constructive (e.g.,  computer-generated) elements to train aircrews. To date, Air Force virtual  training efforts have focused mainly on the integration of virtual and  constructive aspects of distributed training. However, the Air Force’s goal  is to integrate live training with distributed mission operations, which is  referred to as Live, Virtual, and Constructive (LVC) operational training.
section_title: The Air Force’s Process for Developing Combat Aircrew Annual Training Requirements for the Full Range of Core Missions Is Based on Dated Assumptions
paragraphs: The Air Force establishes combat aircrew training requirements for the  full range of core missions based on an annual process, but these  requirements may not reflect current and emerging training needs,  because the Air Force has not comprehensively reassessed the  assumptions underlying them. The Air Force process to develop annual  training requirements for the aircrews of each type of combat aircraft  includes input from the component numbered Air Forces and concludes  with an annual training review board that brings together stakeholders.  However, these requirements may not reflect current and emerging  training needs, because the Air Force has not comprehensively  reassessed the assumptions underlying the total number of annual live-fly  training sorties by aircraft, the criteria for designating aircrews as  experienced or inexperienced, and the mix between live and simulator  training since 2012.
subsections: section_title: Air Combat Command Has a Process to Develop Annual Training Requirements for the Full Range of Core Missions
paragraphs: The Air Force has established a process to develop annual training  requirements for the aircrews of each type of combat aircraft for the full  range of core missions. This process begins with identifying the capability  needs of the component numbered Air Forces. Every two years the Vice  Commander of Air Combat Command sends a letter to the component  numbered Air Forces requesting that they review their primary and  secondary mission lists and identify their priorities for Air Force units by  mission and type of combat aircraft. The component numbered Air Forces  provide their responses to Air Combat Command, describing their  expectations and priorities.
Annually, Air Combat Command takes the expectations and priorities  identified by the component numbered Air Forces and develops the draft  Ready Aircrew Program tasking memorandums that set the annual  training requirements for each aircraft. Inputs into development of the  draft tasking memorandums include subject-matter experts’ assessments  of available training resources and the mid-year and end-of-year training  and readiness reviews. The draft Ready Aircrew Program tasking  memorandums are then released to the fighter wings to obtain feedback  from the operations group commanders on their units’ ability to execute  the training requirements. Air Combat Command officials consolidate  group commander responses and prepare the agenda for the annual  Realistic Training Review Board. The review board consists of senior  officers who meet over three days during the summer to review the draft  Ready Aircrew Program tasking memorandums and make their final  recommendations to Air Combat Command. The final adjustments are  made by Air Combat Command, and the Ready Aircrew Program tasking  memorandums are published before the start of the next fiscal year.  Figure 2 summarizes Air Combat Command’s process for developing  these annual training requirements.
The annual training requirements are used to estimate the Air Force’s  budget request for the flying-hour program, which funds live training.  These hours are determined using the Air Force Single Flying Hour  Model, which provides the methodology and processes that the major  commands use to build their flying-hour programs. The model calculates  the flying hours needed based on inputs of the force structure (number of  aircraft and aircrew), aircrew data (types and number of aircrew that  require training), and live-fly requirements (established in the Ready  Aircrew Program tasking memorandums for combat air forces).  Headquarters Air Force has overall program management responsibility  for the flying-hour program, and major commands are responsible for  their requirements and for reporting on how those hours are used. Air  Combat Command performs this function for the combat air forces. Once  flying hours are allocated to units, the wing commanders manage those  hours and report monthly to Headquarters Air Force on the number of  hours actually flown.
section_title: Annual Training Requirements Are Based on Dated Assumptions and May Not Reflect Current and Emerging Training Needs
paragraphs: The Air Force strategy, America’s Air Force: A Call to the Future,  discusses the need to ensure a full-spectrum-capable force that can  successfully achieve missions to address a broad range of current and  emerging threats. Further, the document states that the design of  training should move to achieve sufficient readiness across all mission  sets while utilizing virtual training to deliver robust and realistic training  against existing and emerging threats. On an annual basis, Air Combat  Command makes updates to its training plans based on the process  discussed above. These updates can include changes to the aircraft’s  primary and secondary missions, the allocation of sorties among  missions, and the types of required training events, among other  changes. However, based on our review of annual training requirements  for combat aircrews from 2012 through 2016 and discussions with Air  Force officials, we found that Air Combat Command has used the same  underlying assumptions to establish its annual training requirements.  Specifically, the total annual live-fly training sorties by aircraft, the criteria  for designating aircrews as experienced or inexperienced, and the mix  between live and simulator training remained the same during this time  period and may not reflect current and emerging training needs, as  discussed below.
Since 2012, Air Combat Command has set the same requirement for the  minimum number of live-fly sorties across all combat aircraft platforms,  regardless of the number of core missions assigned to each platform. For  fiscal year 2016, for example, the annual requirement for live training  sorties for an F-15C, which has two primary missions and no secondary  missions, is the same as the annual requirement for an F-35, which has  three primary missions and three secondary missions. Officials from Air  Combat Command stated that the live-fly sortie requirements are based  in part on a 2010 Navy analysis of the number of sorties needed to  perform safe flying operations. However, two of four wing commanders  we spoke with noted that differences in the number of core missions  should be considered when establishing minimum requirements for live  training sorties by platform, because of variations in the level of  complexity of each aircraft and mission. Air Combat Command officials  told us that aircrews flying more complex combat aircraft may require  additional training but stated that given the constrained resource  environment that DOD and other federal agencies have operated under  for the past several years, it would be difficult to increase training  requirements and the flying hours required to meet them, even in the face  of a potential need. According to its fiscal year 2017 budget request, the  Air Force capped its flying-hour programs at a level it determined to be  executable and therefore has not been able to grow readiness in the full  range of missions over previous years. The budget request also notes  that the fiscal year 2017 request is intended to align funding for the flying- hour program to maintain current readiness levels and to meet its highest  priorities. However, Air Combat Command officials stated that the  analysis needed to determine the relative differences in annual live-fly  training requirements based on the complexity of various aircraft and  missions had not been conducted.
In addition, the criteria for designating experienced and inexperienced  aircrews are based on assumptions that have been in place for the last 40  years. The annual training requirements differ for aircrews designated as  experienced and those designated as inexperienced. For example,  inexperienced aircrews are required to complete one more training sortie  each month than experienced ones. In most cases, aircrews are  designated as experienced if they have flown 500 hours in their primary  aircraft. Certain wing and squadron officials offered that, as a result of  ongoing deployments to the Middle East, aircrews can achieve 500 hours  in their primary aircraft on their first deployment. However, because the  scope of missions these aircrews conduct while deployed is narrow, they  are not necessarily getting training and experience across the full range  of core missions. As a result, when they return from their deployment,  these aircrews can be designated as experienced, and therefore they will  not get the additional training that they may need. These officials also  noted that aircrew capability within the squadron may suffer and there  could be less capacity to fill squadron positions with sufficiently qualified  aircrews.
Further, annual training requirements since fiscal year 2012 have set the  same mix of live and simulator requirements across most of the platforms  we reviewed. For all but one combat aircraft platform, both experienced  and inexperienced aircrews are required to complete 36 simulator  missions per year to be considered combat mission ready. These  requirements were established based on the capacity of simulators at one  installation and not on an analysis of the training needs of aircrews.  According to Air Combat Command officials, the requirement for aircrews  to fly 36 simulator missions per year was established based in part on the  simulator capacity at Shaw Air Force Base, which had the greatest  demands on its simulators because it supports three active fighter  squadrons and an Air National Guard squadron. However, one wing  commander and two squadron commanders told us that simulator  requirements may need to differ among aircraft, depending on the full  range of core missions that the aircraft is expected to perform.  Additionally, Air Combat Command officials told us that aircrews of fifth- generation aircraft may require additional simulator training. In fact, the  simulator training requirements for F-35 aircrews include one more  simulator mission per month, for a total of 48 simulator missions per year.  However, Air Combat Command did not provide any analysis that  supports the need for an additional simulator mission for the F-35 or  explain why the F-22, another fifth-generation aircraft, would not also  require additional simulator training for its aircrews over what is required  for fourth-generation aircraft. Based on our review of fiscal year 2015 data  provided by Air Combat Command on available simulator hours at  combat air forces bases, we found that there was excess simulator  capacity at a number of locations. For example, F-15E squadrons at  Mountain Home Air Force Base had an annual capacity of approximately  6,000 hours at their simulator facility, but the annual simulator  requirement was approximately 4,000 hours.
According to our prior work on strategic training, training plans should be  designed to determine the skills and competencies a workforce needs to  achieve current, emerging, and future agency missions and to identify  gaps—including those that training and development strategies can help  address. Periodic reassessments should be part of a continual effort to  evaluate and improve the agency’s training and development efforts, and  evidence of timely changes should be reflected in those efforts. While  Air Combat Command makes updates to the annual training plans each  year, it has not comprehensively reassessed the assumptions underlying  its annual training requirements—including, but not limited to, the total  annual training requirements by aircraft, the criteria for designating  aircrews as experienced or inexperienced, and the mix between live and  simulator training—and updated these new assumptions in its training  plans.
Officials at Air Combat Command told us that they had two studies under  way to help inform the needed analysis to address these issues, but the  studies had not been completed at the time of our review. For example,  Air Combat Command has an ongoing study evaluating the current  requirement that aircrews accomplish 500 flying hours in order to be  considered experienced. The study is intended to explore various options  for the F-22, including adjusting the current 500-hour requirement that is  used to determine when to designate an aircrew as experienced. Air  Combat Command officials stated that they may explore applying the  results of this study to other combat aircraft. Additionally, in 2016, Air  Combat Command issued a proposal to study the minimum, safe, and  optimal numbers of live-fly and simulator training sorties aircrews must  complete each year. The study is intended to include an examination of  the minimum number of sorties required by mission type, the definition of  proficient and familiar, and the mix of live and simulator training to support  each mission set, among other areas. According to officials from Air  Combat Command, the results of the study are expected to be available  in 2017.
While these initiatives are a positive step, it is too early to determine how  the results of the two studies will be used to implement changes to the Air  Force’s annual training requirements or whether the scope of the studies  is sufficient to address all of the assumptions that need to be revisited.  Without fully reassessing the assumptions underlying its annual training  requirements, the Air Force cannot be certain that its annual training  plans are aligned with its stated goals to ensure a full-spectrum-capable  force that can successfully achieve missions across a broad range of  current and emerging threats.
section_title: Combat Fighter Squadrons Are Not Meeting All of Their Annual Training Requirements across the Full Range of Core Missions, and Completed Training Is Not Being Evaluated for Effectiveness
subsections: section_title: Combat Fighter Squadrons Are Not Meeting All of Their Annual Training Requirements across the Full Range of Core Missions
paragraphs: Based on our analysis of data on the completion of annual training, we  found that combat fighter squadrons were generally able to complete  mission training requirements for ongoing contingency operations, such  as close air support to ground forces, but were unable to meet annual  training requirements across the full range of core missions.
section_title: Various Factors Affected the Ability of Combat Fighter Squadrons to Complete Annual Training across the Full Range of Core Missions
paragraphs: Wing and squadron commanders cited various factors that have limited  the ability of their squadrons to complete training across the full range of  core missions. We analyzed the information we obtained from our  interviews with four wing commanders and eight squadron  commanders—including commanders overseeing aircrew training for both  fourth- and fifth-generation aircraft—and found that they identified similar  training limitations. They cited four common limitations: the maintenance  unit’s ability to provide adequate numbers of aircraft for training, high  deployment rates, adversary air tasking, and manpower shortfalls in the  squadrons.
Seven of the squadron commanders we interviewed told us that their  maintenance unit’s ability to provide an adequate number of aircraft  affected their squadrons’ ability to complete their annual training  requirements. These commanders stated that in recent years,  maintenance units for U.S.-based squadrons have been consistently  manned below their authorized levels. Air Force officials told us these  shortfalls in maintenance personnel limit their ability to produce the  number of aircraft required to meet annual live-fly training requirements.
Squadron commanders we interviewed also identified factors that can  limit the time aircrews have available to meet annual training  requirements, including the following:  High deployment rates—Four squadron commanders told us that high  deployment rates limited the ability of aircrews to conduct annual  training across the full range of core missions between deployments.  Certain fighter squadrons have been deploying every eighteen  months for a 6-month period each time. During these 6-month  deployments, squadrons have been flying primarily close air support  missions and are unable to be proficient across their full range of core  missions. When they return from deployment, squadrons have 12  months to rebuild proficiency across their full range of core missions  before their next six-month deployment. Some squadron commanders  told us that this high deployment rate typically reduces aircrews’ 12- month training schedule by at least 2 months, because they must  focus on close air support training in preparation for the next  deployment.
Adversary air tasking—Five squadron commanders told us that their  squadrons have been tasked heavily to provide adversary air for large  force exercises, such as joint and multinational “Red Flag” exercises  designed to provide realistic training in a contested or degraded  environment. According to Air Force officials, the demand for  adversary air at large force exercises has increased because in fiscal  year 2015 one of the two dedicated adversary air squadrons at Nellis  Air Force Base was eliminated because of budgetary reductions. To  help mitigate this reduction in adversary air capability, Air Combat  Command increased the tasking of fighter squadrons to provide this  capability. For example, an F-15E squadron commander told us that  his squadron will have been tasked 3 of the first 9 months of fiscal  year 2016 to provide adversary air for various exercises, including a  Red Flag exercise. According to one squadron commander,  participation as adversary air in these large force exercises can take  up to 2 months out of the squadron’s training cycle to prepare for and  execute this role. This limits the time available to conduct annual  training for the squadron’s primary and secondary missions.
Squadron manpower shortfalls—Five squadron commanders told us  that shortfalls in their squadron manning can limit the time aircrews  have to meet their annual training requirements. Specifically, with  fewer aircrews assigned to a squadron, aircrews are required to take  on a larger share of the administrative functions of the squadron,  including scheduling training and weapons evaluation. This challenge  can be greater for squadrons operating under the total force  integration construct, which uses aircrews from reserve component  squadrons to supplement the aircrews in active-duty squadrons. For  example, the two F-22 squadrons in the 1st Fighter Wing are  integrated with F-22 aircrews from the 192nd Fighter Wing of the  Virginia Air National Guard. As a result, there are fewer active-duty  aircrews in the squadron. However, according to both F-22 squadron  commanders, these aircrews are required to perform the  administrative duties for the full squadron, because the Air National  Guard aircrews are typically available only on the weekends, and their  time with the squadron is used for training. Both F-22 squadron  commanders we spoke with told us that even if maintenance  personnel could generate the number of aircraft needed to meet their  annual training requirements, the squadrons would be unable to  conduct additional training, because their aircrews’ availability is  limited by their other assigned responsibilities within the squadrons.
In addition to factors that limit the ability of squadrons to meet their annual  training requirements, the squadron commanders we spoke with also  cited a number of factors that affected the training that aircrews are able  to accomplish. The extent to which these factors affected training differed  between fourth- and fifth-generation aircraft. For fourth-generation  aircraft, five squadron commanders told us that the considerable number  of taskings to perform adversary air at large force exercises do not  provide effective training opportunities. Additionally, some of these  commanders expressed concern that adversary air taskings would  increase and that recent changes to their annual training requirements  may mask this problem. Specifically, prior to fiscal year 2016, the annual  training requirements for certain combat aircrews set a cap on the annual  number of adversary air sorties that could count toward their combat  mission ready status. However, in the fiscal year 2016 training  requirements, that cap was removed, allowing for all adversary air sorties  to count toward the designation of combat mission ready. Two squadron  commanders told us they were concerned that this change could result in  masking the readiness ratings for squadrons, since overflying adversary  aircraft sortie requirements provides limited training compared with their  full range of core missions, but still allows the aircrews to be designated  as combat mission ready. According to Air Combat Command officials,  this change was made to encourage aircrews to accurately log adversary  air sorties. These officials stated that previously some aircrews may have  been flying in an adversary air role, but would complete some other  limited type of training maneuver so that they could record the sortie as a  different mission that would count toward their combat mission ready  status.
For fifth-generation aircraft, all three squadron commanders we  interviewed cited various factors affecting training, including airspace and  training range limitations and the availability of adversary air to fly against.  For example, two F-22 squadron commanders at Langley Air Force Base  told us that the size of the base’s airspace limits their ability to train for  their more complex missions, including offensive counter air and  defensive counter air missions, because the available airspace constrains  the capabilities they are able to deploy. Additionally, all three of the  commanders for squadrons flying fifth-generation aircraft told us that  limits in training range capabilities, such as threat replicators and targets,  affected the training completed at smaller regional training ranges, as well  as at larger training ranges such as the Utah Test and Training Range  and the Nevada Test and Training Range. According to these officials,  the training ranges lack many of the more advanced threat replication  systems that can challenge F-35 and F-22 capabilities and provide  effective training across their full range of core missions. According to  training range officials at the Nevada Test and Training Range, the  current range capabilities are adequate to replicate only a “first Gulf War”- era threat scenario.
Air Combat Command has implemented some initiatives to help increase  the number of training sorties that aircrews are able to conduct. However,  these initiatives can further affect the training that is completed. For  example, Air Combat Command has received increased funding for “Red  Flag” exercises to enable a fourth annual exercise at Nellis Air Force  Base. These exercises can provide a high-end, realistic training  environment for aircrews. However, officials explained that Red Flag  exercises require a significant adversary air presence to provide high  quality training for the participants. As discussed above, these exercises  can result in squadrons being overtasked to fly as the adversary force,  which can further limit aircrews’ availability to train to their full range of  core missions. To help mitigate the burden on squadrons, in December  2015, the Air Force began contracting with a private company to provide  adversary air capabilities on a short-term basis to support exercises at  Nellis Air Force Base. According to Air Force officials, in fiscal year 2017,  the Air Force plans to spend about $25 million to contract for six A-4  aircraft (a legacy fighter platform first developed in the 1950s) to provide  adversary air for two months of exercises.
In addition, squadron commanders, in coordination with maintenance  groups, have developed some initiatives to increase the number of sorties  they are able to produce, but these initiatives can further limit the training  that is completed. For example, seven squadron commanders told us  they have implemented the use of “hot pits”—a practice where the aircrew  flies a sortie, lands and refuels, and then flies another sortie without  shutting down the aircraft. This practice minimizes the aircraft repairs that  may arise between sorties if an aircraft is shut down and then restarted.  Five squadron commanders told us that they also occasionally use tanker  aircraft to refuel their squadrons in the air during training. Both “hot  pitting” and aerial refueling practices produce an additional sortie for the  aircrews that day. However, squadron commanders told us that such  practices prevent the aircrews from getting the full benefit from both  training sorties. Specifically, these practices do not allow the aircrews to  receive a mission brief and debrief for both sorties, a critical learning  component of effective training. In addition, squadron commanders told  us that without the benefit of a full briefing for the second training sortie,  the aircrews are not able to conduct their more complex missions that  require additional instruction to be effective. Figure 5 shows an F-22  aircraft conducting a “hot pit” refueling and an F-15C aircraft conducting  an aerial refueling.
The Air Force has a number of efforts under way to study or address  some of the factors that limit the ability of fighter squadrons to meet  annual training requirements and that affect the training that aircrews are  able to accomplish, but it will likely take time before the full benefit of  these efforts can be assessed. For example, the Air Force has  announced several initiatives to manage the shortage of maintenance  personnel, including hiring additional maintenance personnel, temporarily  transitioning active-duty maintenance units from some legacy aircraft, and  offering selective reenlistment bonuses as an incentive to improve  retention, among other initiatives. According to Air Force officials, it may  take several years before newly hired maintenance personnel will have  the training and experience they need in order to improve aircraft  availability rates. In addition, Air Combat Command officials told us that  there is an ongoing study reviewing adversary air requirements. Further,  the Air Force has taken steps to upgrade its training infrastructure,  including training ranges. For example, in its fiscal year 2017 budget  request, the Air Force requested a funding increase of about $35 million  for the maintenance and sustainment of 21 training ranges, including  installation of additional threat emitters and expansion of communication  capabilities between air and ground forces.
section_title: The Air Force Does Not Systematically Evaluate the Effectiveness of Completed Training against Expectations
paragraphs: Air Force processes used to record and monitor combat fighter  squadrons’ annual training are focused on the frequency of training  completed to meet annual requirements and do not include a systematic  evaluation of the effectiveness of that training against established  expectations. Squadron commanders track and report monthly and 3- month “look-backs” to determine if aircrews are meeting their prescribed  annual training requirements. These assessments emphasize the  frequency of training and inform the designation of aircrews as combat  mission ready. According to Air Force officials, squadron commanders  are delegated with the authority to determine whether a sortie should  “count” toward meeting the annual training requirements. These  determinations are based on Air Force guidance—including AFI 11-2,  Volume 1—that establish common training standards for each type of  combat aircraft and documentation supporting the Ready Aircrew  Program that provides a list of training expectations that include the types  of events that should be included in a training mission to be considered  effective. For example, the Ready Aircrew Program tasking  memorandums establish when a non-effective sortie should be recorded,  for example when training limitations are caused by weather conditions,  aircraft maintenance or weapon malfunctions, or cancellation of training  support elements—which can include adversary air or air-refueling  assets.
Air Combat Command uses quantitative data on the completion of annual  training requirements and qualitative information on broad factors that  affect training in various forums that discuss training. However, our review  of documentation supporting these forums found that the Air Force does  not systematically evaluate the extent to which completed training met  expectations or the degree to which certain challenges have affected  training. In our review, we identified two forums that Air Combat  Command uses to oversee training completion and discuss training  challenges. These forums are focused on a review of training completion  data and include discussions of factors that affect training completion and  the effectiveness of training. For example, at the end of each fiscal year,  Air Combat Command aggregates combat fighter squadrons’ training  completion data for its end-of-year training and readiness review. Tables  are compiled by squadron and by aircraft type to provide a comparison of  annual training completion data against each aircraft’s full range of core  missions. According to Air Combat Command officials, the squadron  commanders are required to submit a list of qualitative factors that, in  general, affected the completion and effectiveness of the squadron’s  training. These factors are then discussed at an end-of-year training and  readiness review, which is an informational briefing used to inform the  state of readiness of the combat air forces.
In addition, the annual Realistic Training Review Board provides another  forum for discussion between Air Combat Command and leadership at  the group commander level on training challenges. According to officials  and our review of documentation discussed during these forums, these  discussions are focused on a number of issues that affect training, such  as funding for the flying-hour program and distributed mission operations.  Further, the Board identifies a number of action items related to training  that need to be addressed. For example, output from the 2015 Board  included actions directed toward improving cruise missile threat  replication, targets for F-22 ranges, and opportunities for certain weapons  employment for F-15C squadrons. While these forums provide an  opportunity for Air Force officials to discuss and identify training  challenges, neither includes a systematic evaluation of the effectiveness  of training against established expectations in order to identify limitations  to the effectiveness of training.
The lack of a systematic evaluation of the effectiveness of training against  established expectations has affected the Air Force’s ability to make  informed resource decisions to improve its training infrastructure. For  example, training range officials from both the Nevada and Utah Test and  Training Ranges told us that without information on the gaps between  aircraft training requirements and current range capabilities, they were not  well positioned to make informed decisions on where to focus their  resources to improve range capabilities. An official from the Nevada Test  and Training Range told us that identifying training gaps is becoming  even more important, because the new F-35 aircraft will have additional  range requirements that exceed those of fourth-generation fighter aircraft.  However, under current Air Force guidance, the Nevada Test and  Training Range is required to provide capabilities only for large force  exercises and does not currently possess the range capabilities needed  to support a number of training events identified in the training  expectations for the F-35, including close air support in an urban terrain  and certain attack events, among others.
We found that the Air Force does not systematically evaluate the  effectiveness of training against established expectations for two reasons.  First, Air Combat Command has not established the desired learning  objectives and training support elements needed to accomplish the  training expectations in its annual Ready Aircrew Program tasking  memorandums. Our prior work on strategic training notes that agencies  should systematically evaluate the effectiveness of their training efforts  and collect data corresponding to established training objectives to  continually improve, deliver, and enhance training and ensure the  effective allocation of resources. In contrast to the Ready Aircrew  Program tasking memorandums, we found that documentation  underpinning other types of aircrew training, such as mission qualification  training and upgrade training, provide more detailed information on critical  features needed to measure the effectiveness of training sorties. For  example, the upgrade training syllabus for the F-22 provides desired  learning objectives for an aircrew to conduct an effective defensive  counter air sortie and information on the number of adversary aircraft  required and the period of time that should be spent defending a target.  However, the Ready Aircrew Program tasking memorandum for the F-22  does not include similar information on an effective defensive counter air  sortie. Instead, it provides only a list of expectations that could be  included in a training mission, along with information on the desired  amounts of adversary air needs.
Second, the current process used by squadrons to report information on  their training completion does not collect data on the effectiveness of the  training against established expectations. Documentation supporting the  Ready Aircrew Program notes that credible and detailed training  accomplishment reporting provides an accurate assessment of readiness  and the data needed to advocate for adequate resources. This  documentation notes that Air Combat Command expects to see data that  document the underlying challenges to providing an effective training  program. However, the form that aircrews are required to complete  following a training sortie requires them to record the number of flying  hours and other information, such as the type of mission, but does not  capture information on the ability of aircrews to meet the training  expectations established in the Ready Aircrew Program. For example, the  form does not provide the opportunity to record the training support  elements involved in the sortie, such as the number of adversary aircraft  involved and the threat replicators available. In the absence of a process  to obtain this information, squadron commanders are also unable to  collect data corresponding to the effectiveness of annual training  completed to meet expectations or the degree to which certain challenges  affected training. One squadron commander we interviewed has  developed internal guidance for aircrews to determine whether a sortie  was effective and should count toward the annual training requirements.  This commander provides an annual guidance letter to be used within the  squadron that sets the minimum requirements that must be met to log an  effective training sortie. However, based on our interviews with other  squadron commanders, this practice is not being consistently performed  across all squadrons.
Officials from Air Combat Command told us that they had not developed  more detailed expectations for combat fighter squadron commanders to  measure and collect data on the effectiveness of annual training for  several reasons. These officials stated that Air Force instructions and the  Ready Aircrew Program provide squadron commanders with the authority  to determine if a training sortie was effective and should be counted  toward meeting annual training requirements. Further, these officials  noted that aircrews have been limited in their ability to complete annual  training requirements due to the factors previously discussed in this report  and that having more detailed expectations for measuring the  effectiveness of annual training could make it more difficult for aircrews to  achieve a combat mission ready status. In addition, Air Combat  Command officials stated that the need to complete training against more  detailed expectations and collect information on training effectiveness  should be balanced against the time and other factors that would be  required to collect the information.
Given the range of factors that have limited the ability of combat fighter  squadrons to complete annual training requirements, having a consistent  basis for monitoring training results is critical in tracking progress toward  achieving the Air Force’s goal of training units for the full range of core  missions. Moreover, Air Combat Command has stated its goal of moving  to a training model based on the effectiveness of training, using  quantifiable levels of performance, rather than the current model focused  on the frequency of training. Without a process to systematically measure  and collect data on the effectiveness of training against established  expectations, the Air Force will not be well positioned to improve, deliver,  and enhance training or allocate resources to address factors that limit  the effectiveness of training.
section_title: Air Force Planning Documents for Virtual Training Do Not Include All Desirable Characteristics of a Comprehensive Strategy
paragraphs: The Air Force’s virtual training plans do not include all desirable  characteristics of a comprehensive strategy, such as a risk-based  investment strategy or a time line for addressing training needs. We have  previously reported that when addressing national or department-wide  issues, it is standard practice for organizations to have a strategy that  includes desirable characteristics such as establishing goals and  objectives, identifying actions for addressing those objectives, allocating  resources, identifying roles and responsibilities, and measuring  performance against objectives, among others.
In recent years, the Air Force has developed some planning documents  to guide its virtual training efforts. In 2013, for example, the Headquarters  Air Force issued its United States Air Force Live Virtual Constructive  Operational Training Flight Plan. The flight plan was the first Air Force  level strategic document to address LVC operational training from an Air  Force-wide perspective. The document established the Air Force’s broad  vision for the future of the LVC operational training environment and  highlighted areas that needed attention to advance the program and  realize its full potential. In addition, in 2014, Air Combat Command issued  the Combat Air Force Live, Virtual, Constructive Vision, which describes  the command’s goals for the integration of LVC operational training  capabilities, along with the priority tasks and general timeframes for  implementation. Further, in 2015, Air Combat Command released the  Combat Air Force Live, Virtual, Constructive Roadmap that sets near-,  mid-, and far-term tasks that are required to better define needed training  capabilities.
While the Air Force has issued these planning documents to help guide  its LVC operational training initiatives for the combat air forces, they do  not include all of the characteristics we have identified as being desirable  in comprehensive strategies. Based on our analysis of the United States  Air Force Live Virtual Constructive Operational Training Flight Plan and  the Combat Air Force Live, Virtual, Constructive Vision, we found that the  Air Force has only partially addressed or has not addressed most of the  desirable characteristics, as shown in table 1.
For one characteristic, we found that both of the planning documents  contained the desirable elements. For example, United States Air Force  Live Virtual Constructive Operational Training Flight Plan establishes the  roles and responsibilities for organizations involved in the implementation  of LVC operational training initiatives, including Headquarters Air Force  and the major commands, such as Air Combat Command. It also  discusses the governance structure and the role of primary support  organizations, such as the LVC Council of Colonels and the Distributed  Warfare Integration Cell, which are intended to identify and prioritize LVC  operational training gaps and shortfalls that cut across the major  commands. For other characteristics, we found that some of the elements  were included, but not all. For example, the Combat Air Force Live,  Virtual, Constructive Vision identifies a number of implementation tasks  across a range of implementation phases, but it does not establish  priorities, specific milestones, or performance measures. Our prior work  has shown that these elements can help to ensure effective  implementation and accountability to achieve results. Further, for some  characteristics, we determined that neither planning document contained  any of the desirable elements. For example, neither of the two documents  identified what the virtual training efforts will cost, the sources and types  of resources required, or the prioritization of needed investments. Our  prior work has shown that a risk-based investment strategy may be used  to define and prioritize related resource and operational requirements, as  well as to develop a time line for obtaining capability needs. A risk-based  investment strategy includes five key phases: (1) setting strategic goals  and objectives and determining constraints, (2) assessing risks, (3)  evaluating alternatives for addressing these risks, (4) selecting the  appropriate alternatives, and (5) implementing the alternatives and  monitoring the progress made and results achieved.
We identified several areas where the Air Force is making significant  investments in virtual training, despite the lack of a comprehensive  strategy needed to guide and prioritize these efforts. In some cases, this  has led to investments that may not maximize available Air Force  resources. For example, the Air Force is building an F-22 simulator facility  at Nellis Air Force Base. The facility is expected to be completed in fiscal  year 2017, at an estimated cost of $14 million. However, according to  officials at Nellis Air Force Base, the Air Force does not expect the four  simulators to arrive there until fiscal year 2019 at the earliest, leaving the  facility potentially unutilized during that interim period. Additionally,  officials identified a number of examples where the Air Force has not  made cost-efficient investments in virtual training systems. For instance,  officials from Headquarters Air Force noted that different versions of the  F-16 simulator have been procured within the active, guard, and reserve  components, which create inefficiencies among the components as they  develop, procure, and sustain these differing simulators.
Air Force officials also provided examples of prior investments in  simulators that do not meet the training needs of aircrews across their full  range of core missions. For example, the Air Force Audit Agency reported  in 2013 that Air Force officials had not adequately defined F-22 simulator  requirements, because the procurement contracts did not include  simulator performance specifications. Instead, all F-22 simulators shared  a single training system specification that provided insufficient technical  detail as to what the Air Force considered an acceptable product. As a  result, F-22 squadrons at Langley Air Force Base have traveled to the Air  Combat Simulator facility in Marietta, Georgia to utilize the contractor- owned F-22 simulators at that location. Officials from the F-22 squadrons  told us that the contractor-owned simulators provide higher-quality  training than the simulators at their home station and allow them to train  in a more realistic threat environment. For 2015, the cost of using the Air  Combat Simulator facility in Marietta was around $10,000 per day, plus  travel costs for the aircrews. According to an official from Air Combat  Command, F-22 squadrons at Langley Air Force Base utilized the Air  Combat Simulator facility for 5 days in calendar year 2015 and, as of  February 2016, have utilized the facility for 11 days in calendar year  2016. In addition, based on our review of simulator utilization data, the F- 22 simulators at Langley Air Force Base remain underutilized.
Air Combat Command has identified significant requirements for LVC  operational training capabilities that will require funding over the next  several years. Specifically, the Air Force estimates that LVC operational  training requirements will total approximately $3.8 billion in funding for  fiscal years 2015 through 2019. This includes funding to maintain  simulator currency with the aircraft, modernize simulator visual systems,  secure virtual training networks, and hire additional simulator instructors.  Air Force officials told us that concern over simulator information  security—particularly for fifth-generation aircraft—will require network  upgrades to ensure that the infrastructure has adequate controls in place  to protect data. According to these officials, network security and  information assurance are currently the most pressing challenges in the  Air Force’s virtual training environment.
While the United States Air Force Live Virtual Constructive Operational  Training Flight Plan and the Combat Air Force Live, Virtual, Constructive  Vision demonstrate some effort toward defining a broad strategy for the  Air Force’s virtual training efforts, the Air Force lacks a comprehensive  strategy, because it has not fully refined its planning documents to  achieve its training vision, for example by developing a risk-based  investment strategy that identifies and prioritizes capability needs and  includes a time line for addressing them. The Flight Plan recognizes the  need for an investment strategy to prioritize and guide investments in  future LVC operational training capabilities; however, officials told us that  development of an investment strategy has not begun. Additionally, as we  reported in 2012, the Air Force does not have a methodology to fully  account for and track the costs of its virtual training enterprise.  Headquarters Air Force officials told us that they have made some  progress in tracking operation and maintenance costs for virtual training  but none toward tracking investment funding. As we reported in 2012,  identifying virtual training costs is challenging, because funds that support  virtual training are dispersed across multiple program elements.  Headquarters Air Force officials told us that they have identified  approximately 75 program elements that support the LVC operational  training program, but that it is difficult to determine if these elements  represent all required funding. Further, these officials told us that these 75  program elements are not consistently monitored, because the  automation required to monitor and conduct any trending analysis on the  program elements does not currently exist.
Given the importance of an integrated LVC operational training  environment to the future of Air Force training, a comprehensive strategy  would help ensure that the Air Force develops the capabilities needed to  meet its virtual training needs. Without a comprehensive approach to  virtual training that includes a risk-based investment strategy and a time  line for prioritizing and addressing capability needs, the Air Force may be  slow to develop needed capabilities and therefore could increase risk for  fifth-generation fighter aircraft, whose training will require a robust,  realistic virtual training environment.
section_title: Conclusions
paragraphs: The Air Force has identified the need to ensure a full-spectrum capable  force that can successfully achieve missions to address a broad range of  current and emerging threats. However, the Air Force has not completed  the analysis and planning needed to ensure that aircrew training  requirements fully align with those goals. Specifically, Air Combat  Command has not comprehensively reassessed the assumptions  underlying its annual training requirements, some of which have been in  place for 40 years. Without fully reassessing the assumptions underlying  its requirements, the Air Force cannot be certain that its annual training  plans are aligned with its stated goals to ensure a full-spectrum capable  force that can successfully achieve missions across a broad range of  current and emerging threats. In addition, Air Combat Command has not  identified desired learning objectives or training support elements  necessary to accomplish the training expectations in its annual Ready  Aircrew Program tasking memorandums. Having a consistent basis for  monitoring training results is critical for tracking progress in achieving the  Air Force’s goal of training units for the full range of core missions. Also,  without a process to collect data to assess the effectiveness of the  training, the Air Force may not be well positioned to improve, deliver, and  enhance training or allocate resources to address factors that limit the  effectiveness of training. Further, the Air Force lacks a comprehensive  strategy for virtual training, because it has not fully refined its planning  documents to achieve its training vision, for example by developing a risk- based investment strategy and a time line for prioritizing its needs.  Achieving this vision is critical; as new fifth-generation aircraft become  operational they will require a robust and realistic virtual training  environment to train to their full capabilities.
section_title: Recommendations for Executive Action
paragraphs: To ensure that annual training plans are aligned with the Air Force’s  stated goals to ensure that its forces can successfully achieve missions  across a broad range of current and emerging threats, we recommend  that the Secretary of Defense direct the Secretary of the Air Force to  comprehensively reassess the assumptions underlying its annual training  requirements—including, but not limited to, the total annual training  requirements by aircraft, the criteria for designating aircrews as  experienced or inexperienced, and the mix between live and simulator  training—and make any appropriate adjustments in future training plans.
To improve the Air Force’s ability to consistently monitor training results  and better position it to allocate resources to address factors that limit the  effectiveness of training, we recommend that the Secretary of Defense  direct the Secretary of the Air Force to establish desired learning objectives and training support elements  needed to accomplish the training expectations in its annual Ready  Aircrew Program tasking memorandums and develop a process to collect data to assess the effectiveness of  annual training against these features.
To improve the Air Force’s ability to develop the capabilities needed to  meet its virtual training needs, we recommend that the Secretary of  Defense direct the Secretary of the Air Force to continue to refine its  planning for virtual training to incorporate the desirable characteristics of  a comprehensive strategy, including developing a risk-based investment  strategy that identifies and prioritizes capability needs and includes a time  line for addressing them.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on the non-public sensitive version of this report,  DOD did not concur with the first and second recommendations, and  concurred with the third recommendation. DOD’s comments are reprinted  in their entirety in appendix II.
In its comments, DOD stated it did not concur with our first  recommendation, but DOD identified some actions it was taking to  address the intent of the recommendation. Specifically, DOD stated that  the Air Force has already begun a comprehensive reassessment of the  assumptions underlying its annual training requirements and that those  results will be used to influence future training plans. More specifically,  DOD stated that the Air Force has directed a study that will reassess the  full spectrum of the Air Force’s Ready Aircrew Program training  requirements and provide a range of readiness levels associated with  varying training levels. According to DOD’s comments, the estimated  completion date of the study is September 2017. DOD also stated that a  second study is reviewing the factors that contribute to the Air Force’s  aircrew experience designation. For example, the Air Force has  completed a review of the F-22 that resulted in substantive changes for  the F-22 experiencing criteria. DOD’s comments further stated that the Air  Force has directed a follow-on study for all fourth-generation fighter  aircraft with an estimated completion date for this study of September  2017. As we noted in our report, the Air Force’s initiatives represent  positive steps to study aspects of the combat aircrew annual training  requirements. However, given that these studies are in the initial stages  or not yet underway, it is unclear whether the scope of the studies is  sufficient to address all of the assumptions that need to be revisited.  Further, it is too early to determine what adjustments, if any, the Air Force  will make in its future training plans in response to the studies’ results, as  we recommended. Therefore, we believe the recommendation remains  valid.
DOD did not concur with our second recommendation, stating that the Air  Force’s Ready Aircrew Program training differs significantly from other  syllabus-directed courses of instruction and that desired learning  objectives for this training are set at the squadron level in accordance  with current Air Force guidance. Specifically, the Ready Aircrew Program  tasking memorandums direct squadron commanders to develop unit  training programs that focus on their primary and secondary missions,  and that in order to be effective, each mission must successfully complete  a sufficient number of events applicable to that mission type, according to  the squadron commander. In addition, DOD’s comments noted that Air  Force guidance provides higher headquarters direction at the appropriate  level while providing local commanders with the flexibility to set detailed  learning objectives for individual sorties that allows them to maximize the  training value and efficiency of each sortie. DOD also stated that a table  in the Ready Aircrew Program tasking memorandums identifies the  desired adversary air ratios, which serves as the training support  elements needed to accomplish training expectations. Finally, DOD  stated that because the effectiveness of annual continuation training is  identified at the local level, only the accomplishment of training needs to  be collected and reported. We agree that squadron commanders should  have the flexibility to inform individual aircrew training needs. However,  establishing a baseline of desired learning objectives to meet Air Combat  Command’s training expectations for the Ready Aircrew Program will  provide squadron commanders information on what features a training  event should include and help ensure that aircrews are receiving  consistent and comparable training across the force. Further, while the  Ready Aircrew Program tasking memorandums identify desired adversary  air ratios, the memorandums do not identify any other training support  elements that are needed to accomplish training expectations. For  example, the tasking memorandums do not provide information on the  training range capabilities needed to meet expectations, such as threat  replicators and targets. As noted in our report, a range of factors, in  addition to the availability of sufficient ratios of adversary air, have limited  the ability of combat fighter squadrons to complete annual training  requirements. The lack of this information can affect the Air Force’s ability  to make informed resource decisions to improve its training infrastructure.  Finally, our report also noted that Air Combat Command guidance states  that it expects to see data that documents the underlying challenges to  providing an effective training program. DOD’s comment that only the  accomplishment of training needs to be collected and reported contradicts  this guidance as well as the Air Force’s stated goal of moving to a training  model based on the effectiveness of training, using quantifiable levels of  performance, rather than the current model focused on the frequency of  training. Without developing a process to collect data and measure the  effectiveness of training against established expectations that includes  information on desired learning objectives and training support elements,  the Air Force will not have the information required to achieve this goal.  Therefore, we believe the recommendation remains valid.
In concurring with the third recommendation, DOD stated that the Air  Force is beginning to take a holistic view of operational training and the  Air Force has stood up an Operational Training Division to better focus  control over the operational training enterprise. One responsibility of the  division will be to update the United States Air Force Live Virtual  Constructive Operational Training Flight Plan with all of the desirable  characteristics of a comprehensive strategy. The Air Force expects to  complete this update by September 2017. By updating this plan to include  the desirable characteristics of a comprehensive strategy, such as a risk- based investment strategy that identifies and prioritizes capability needs  and a time line for addressing them, the Air Force would improve its  ability to develop the capabilities required to achieve its virtual training  needs.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, the Under Secretary of Defense  for Personnel and Readiness, and the Secretary of the Air Force. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have questions about this report, please contact me at  (202) 512-5431 or russellc@gao.gov. Contact points for our Offices of   Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this report are to determine the extent to which the Air  Force has (1) determined requirements to train its combat aircrews for the  full range of core missions, (2) met annual training requirements for  combat fighter squadrons across the full range of core missions and  evaluated the effectiveness of this training, and (3) established virtual  training plans that include desirable characteristics of a comprehensive  strategy.
This report is a public version of the prior sensitive report that we issued  in August 2016. The Department of Defense (DOD) deemed some of the  information in the prior report as For Official Use Only (FOUO), which  must be protected from public disclosure. Therefore, this report omits  FOUO information and data on some of the Air Force’s training priorities,  completion of annual training requirements for active-duty fighter  squadrons, and aircraft maintenance generation capabilities. Although the  information provided in this report is more limited in scope, it addresses  the same objectives as the sensitive report. Also, the methodology used  for both reports is the same.
We focused our review on the annual continuation training requirements  for aircrews assigned to aircraft within the active component of the  combat air forces. We focused on Air Combat Command as the lead  major command for all combat air forces, which has responsibility for  developing and managing annual continuation training requirements  through the Ready Aircrew Program. The aircraft included in our review  were the A-10, F-15, F-16, F-22, and F-35. We selected a non- generalizable sample of units to speak with based on our analysis of  annual training completion data and the range of units’ core missions.  Based on this analysis, we conducted site visits with F-22 units at Langley  Air Force Base, F-16 Block 40 and F-35 units at Hill Air Force Base, F- 15E units at Seymour Johnson Air Force Base, and F-16 Block 50 units at  Shaw Air Force Base.
To determine the extent to which the Air Force has determined  requirements to train its combat aircrews for the full range of core  missions, we analyzed Air Force documents on training to identify the Air  Force’s current process for developing training requirements, including  the Air Force Instruction 11-2, Volume 1 for each aircraft in our review.  We interviewed officials involved in the development of the annual  training requirements, including officials from Air Combat Command, U.S.  Air Forces Europe, Pacific Air Forces, and U.S. Air Forces Central  Command. We also interviewed wing and squadron commanders at  selected units, as described above, to obtain an operational-level  perspective on the training requirements needed to meet the full range of  core missions. We reviewed the annual training requirements established  for each aircraft for fiscal years 2012 through 2016 to determine how the  Air Force establishes the experienced and inexperienced aircrew  designations and how sortie and simulator mission requirements have  changed over time. We evaluated these processes against Department of  Defense (DOD) and Air Force guidance that prioritize and establish  training requirements and against the leading practices we have identified  for managing strategic training.
To determine the extent to which the Air Force has met annual training  requirements for combat fighter squadrons across the full range of core  missions and evaluated the effectiveness of this training, we analyzed  training completion data from fiscal years 2012 through 2015 to identify  core missions for which aircrews were or were not fully trained. We  performed data reliability procedures on the information included in  Ready Aircrew Program completion reports by comparing that information  against related documentation and interviewing knowledgeable officials  on controls over the reporting systems. We determined that the data  presented in our findings were sufficiently reliable for the purposes of this  report. We reviewed Air Force documentation that describes training  challenges and interviewed Air Force officials, including wing and  squadron commanders at selected units, as described above, and  officials at Air Combat Command’s two major range and test facility  bases—the Nevada Test and Training Range and the Utah Test and  Training Range—to discuss any factors that limit the ability of aircrews to  complete training for the full range of core missions. We evaluated the Air  Force’s process for assessing the effectiveness of its annual training  against leading practices we identified for managing strategic training and  against the Ready Aircrew Program tasking memorandums, which specify  annual continuation training for personnel assigned to combat units.
To determine the extent to which the Air Force has established virtual  training plans that include desirable characteristics of a comprehensive  strategy, we reviewed planning documents established by the Air Force  to guide its virtual training efforts and evaluated these actions against our  prior work on desirable characteristics of national strategies. Given that  there is no established set of requirements for strategies, we relied on our  assessments of national strategies and the criteria that were applied to  assess these strategies. We identified six desirable characteristics that  national or department-wide strategies should contain. We assessed  these criteria against the two existing Air Force planning documents we  found most relevant to the combat air forces—United States Air Force  Live Virtual Constructive Operational Training Flight Plan and the Combat  Air Force Live, Virtual, Constructive Vision—to determine the extent to  which these documents contain the elements of a comprehensive  strategy. We determined whether these actions addressed, partially  addressed, or did not address the desirable characteristics of strategy  documents, based on our assessment of whether the planning  documents explicitly cited all elements of a characteristic and the level of  specificity and detail included.
We conducted this performance audit from June 2015 to September 2016  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact name above, Matthew Ullengren, Assistant  Director; Karyn Angulo; Russell Bryan; Joanne Landesman; Kirsten  Lauber; Amie Lesser; Jim Lloyd; Justin Riordan; and Sabrina Streagle  made key contributions to this report.