Air Force Training: Actions Needed to Better Manage and Determine Costs of Virtual Training Efforts
Released: 2012-07-20T00:00:00

Letter

Background

For each of their aircraft, the Air Force lead commands set training requirements that aircrews must complete on an annual basis in order to maintain combat mission readiness. These training requirements include basic tasks such as take-offs and landings and also more-advanced tasks, such as air-to-air combat and ground-attack missions. To help meet these requirements, the Air Force has developed an approach to training that it terms distributed mission operations. This approach is intended to train units as they expect to fight, maintain readiness, and conduct mission rehearsals in a realistic environment. Distributed mission operations utilizes the integration of virtual (e.g., a person training in a simulator) and constructive (e.g., computer generated) elements to train personnel at geographically separated sites by means of a network. For the purposes of this report, we refer to training that includes a simulator as virtual training.

The Air Force has four primary centers that facilitate distributed mission operations by connecting units and simulators from geographically dispersed areas:

Distributed Mission Operations Center in Albuquerque, New Mexico, managed by Air Combat Command;

Distributed Training Operations Center in Des Moines, Iowa, managed by the Air National Guard;

Warrior Preparation Center in Einsiedlerhof, Germany, managed by U.S. Air Forces Europe; and

Korean Air Simulation Center in Osan, Republic of Korea, managed by Pacific Air Forces.

The Distributed Mission Operations Center functions as the lead integrator of virtual systems to conduct theater-level exercises and events that include air, land, space, cyber, and maritime virtual assets for Air Force, joint, and coalition partners. These large-scale events, known as virtual flags, are conducted quarterly and last about 2 weeks. In 2011, the Distributed Mission Operations Center trained over 1,400 personnel from the Air Force, Army, Navy, Marines, and coalition forces through this virtual exercise as well as other, small-scale, events.

The Distributed Training Operations Center plans, builds, and manages small-scale events to meet the learning objectives of its customers, mainly Air Combat Command, Air National Guard, and Air Force Reserve Command. These events are short-term, typically lasting 90 minutes. During 2011, the Distributed Training Operations Center conducted over 4,000 events that trained more than 9,500 personnel, of which at least 60 percent were active-duty personnel.

The two overseas virtual training centers provide different capabilities for the commands they support. The Warrior Preparation Center supports training for joint, coalition, and partner-nation forces in the European and African theaters. In addition, the Warrior Preparation Center supports an Air-to-Ground Operations School and three detachments that provide multinational training opportunities. The Korean Air Simulation Center operates constructive simulations that support the air operations in Korea during U.S. Forces Korea operational-level exercises and supports selected exercises for U.S. Forces Japan.

To train its units and personnel, the Air Force conducts distributed mission operations using several different internal Air Force and Department of Defense (DOD) information networks. Some of these key networks, along with their managing organizations, are shown in table 1.

These networks differ according to such factors as security restrictions, bandwidth capacity, data protocols, and support services.

In May 2010, the Secretary of Defense directed DOD to undertake a department-wide efficiency initiative to reduce excess overhead costs and to reinvest the resulting savings in sustaining force structure and modernization. The Air Force identified a number of areas to improve its efficiency, including an initiative, beginning in fiscal year 2012, to decrease training costs by reducing its live flying hour program for its legacy fighter and bomber aircraft by 5 percent and its Air Force Reserve Command F-16 flying hour program by 10 percent.initiative, the Air Force expects to save a total of $1.7 billion from fiscal years 2012 to 2016. The Air Force estimated savings of about $268 million for fiscal year 2012. In discussing the initiative, the Air Force stated that it expected to offset any effect on readiness caused by a reduction in live flying hours by increasing its use of simulators.

Air Force Major Commands Determine Their Mix of Live and Virtual Training Based on Various Factors

Currently, the three lead Air Force major commands—Air Mobility Command, Air Force Special Operations Command, and Air Combat Command—have similar processes to determine the mix of live and virtual training, but the mix of training differs across the major commands, and among aircraft within the commands. Air Combat Command is responsible for fighters, bombers, and attack aircraft; Air Mobility Command is responsible for transport and tanker aircraft; and Air Force Special Operations Command is responsible for special-operations aircraft. At each command, training-requirement review boards composed of subject-matter experts meet to consider broad sets of training issues and evaluate training requirements for specific aircraft. The boards consider factors such as specific combatant command mission requirements and the capabilities of simulators and networks that have already been fielded, and determine which training requirements can be completed in a virtual environment and which need to be completed in a live environment. The results of their reviews are reflected in updated training guidance for each type of aircraft. In addition, each of the commands also establishes requirements to improve, acquire, or upgrade training devices to meet mission tasks.

While all three lead major commands rely on both live and virtual training to meet aircrews’ training requirements, the mix is different for each major command, as discussed below.

Air Mobility Command

For each aircraft type, Air Mobility Command issues a requirement document that specifies the number of times each task or “event” must be completed for a pilot or aircrew to be certified as mission ready. The document also specifies the percentage of events that can be completed in a simulator. For example, the C-130 requirement document specifies that 50 percent of assault landings may be completed in a simulator and 100 percent of instrument approaches in a simulator. Although live and virtual training requirements vary by aircraft, according to Air Mobility Command officials, approximately 50 percent of aircrew training is conducted in simulators, including all training related to takeoffs, landings, and instrument approaches.however, training for some special qualifications such as aerial refueling, formation flying, airdrops, and assault landings must periodically be conducted live in the actual aircraft. For example, for aerial refueling, currently, there are differences between what the fighter pilots see in their simulators and what air refueling crews see in their simulators. Because the simulators are currently not able to accurately replicate the aerial refueling environment, simulated training cannot yet replace live training. In developing its virtual training program, Air Mobility Command worked with the Federal Aviation Administration to leverage civilian standards, which require simulators to respond like the actual aircraft in order to be certified for training. Air Mobility Command is currently developing a networked distributed training center that would enable more virtual training with combat air forces and coalition partners.

Air Force Special Operations Command

As Air Mobility Command does, for each aircraft type Air Force Special Operations Command issues a requirement document that specifies the number of times each task or “event” must be completed for a pilot or aircrew to be certified as mission ready and the percentage of events that can be completed in a simulator. Air Force Special Operations Command officials stated that the command’s goal is to accomplish up to 50 percent of its aircrew training in simulators depending upon the aircraft. For example, aircrew training requirements for the AC-130U, a close air support aircraft, allows aircrew to accomplish 50 percent of their mission tasks in a simulator. Air Force Special Operations Command based its simulator certification program on the standards and metrics used by the Federal Aviation Administration and Air Mobility Command. Air Force Special Operations Command officials stated that simulators provide training that might not be available in the live environment, such as training for specific locations or adverse weather conditions. Air Force Special Operations Command has a stated goal to perform all qualification and continuation training events in the simulator, while increasing both live and simulator mission rehearsal training.

Air Combat Command

Air Combat Command also issues a requirement document for each type of aircraft on an annual basis. Virtual training requirements vary by aircraft, with large aircraft such as bombers generally able to satisfy more of their training requirements in simulators than fighters. Beginning in fiscal year 2012, Air Combat Command’s training-requirement review board revised each aircraft’s training requirements and specified that approximately 25 percent of training requirements were to be met using virtual training, while the rest of the requirements were to be met using live training. Prior to this, Air Combat Command training guidance specified that virtual training was to be used as a supplement to live training, but it did not set a goal or specific percentage requirement for virtual training. The virtual training that had been done in those years included emergency procedures, instrumentation training, and tactical training rather than mission training. With the availability of more- advanced full mission-training simulators, aircrews are now able to train beyond these basic tasks to more-advanced air-to-air and air-to-ground combat missions, like suppression of enemy air defenses.

According to Air Combat Command officials, the combat air forces face certain challenges that prevent them from conducting the same level of virtual training as forces from the other major commands. Some challenges arise due to differences between unit and simulator locations, difficulties coordinating distributed training events, and a lack of simulator fidelity. For example, officials stated that there are very few simulators collocated with Reserve component units, which means valuable reserve component training time can be lost travelling to and from the simulators. Officials also noted that the software for some aircraft simulators is two or three versions behind the software in the actual aircraft, which could in some cases, affect the performance of aircrews in the actual aircraft. In addition, fighter simulators cannot replicate the extreme physical effects of air combat maneuvers that fighter pilots experience in the actual aircraft. Fighter pilots we interviewed stated that unlike flying other aircraft such as bombers and transports, fighter pilots must effectively make decisions while conducting their missions in a hostile environment and maneuvering the aircraft through high-speed and high-gravity maneuvers that put stress on the human body. We note that the Navy faces similar challenges in conducting virtual training for its fighter aircrews. For example, the crews of the Navy’s F/A-18E/F currently conduct 18 percent of their training through virtual training and plan to increase this to 32 percent by 2020.

The Air Force Has Not Established an Overarching Organizational Framework to Guide, Oversee, and Integrate Virtual Training Efforts

The Air Force has recently taken steps to increase management attention over its virtual training efforts, but its approach to virtual training currently lacks (1) a designated organization with accountability and authority for achieving results and (2) an overarching strategy—key elements of an organizational framework that we have found to be critical for successful transformations in both public and private organizations. In the absence of a framework to structure and guide its virtual training efforts, the Air Force will continue to face challenges in integrating its virtual capabilities and cannot be certain that its efforts align with strategic goals or know whether critical gaps or duplication of efforts exist.

The Air Force Has Increased Management Focus on Virtual Training Efforts, but Oversight Remains Fragmented

According to Air Force leadership, distributed mission operations are the cornerstone of the Air Force training transformation. Additionally, in the Strategic Plan for the Next Generation of Training, DOD has emphasized comprehensive training that integrates service and joint capabilities. The Air Force has increased management attention on virtual training efforts by reorganizing and creating new headquarters offices and establishing working groups, but oversight remains fragmented. For instance, in February 2011, the Air Force Agency for Modeling and Simulation was realigned under Headquarters Air Force Director of Operations (A3O) to serve as the execution arm for integrating and implementing virtual capabilities, resources, and policy. Also, in August 2011, the Air Force established the Headquarters Air Force Director of Operations– Operational Training (A3O-CL) office to provide leadership and support to distributed mission operations users across the Air Force. Additionally, in February 2012, Headquarters Air Force established a working group, composed of subject-matter experts from the Distributed Training Centers and the major commands, to address operational challenges within the virtual training programs. Issues unable to be resolved in this forum are elevated to higher-level working groups including the Headquarters Operations Conference, the Modeling and Simulation Steering Committee, and finally the Air Force Modeling and Simulation General Officer’s Steering Group.

These organizations and working groups have increased management focus on virtual training efforts, but the Air Force has not designated an organization with accountability and oversight authority necessary to integrate all its virtual training efforts, including developing and acquiring interoperable virtual training systems and establishing and enforcing authoritative standards for simulators, constructive elements, and databases. Rather, oversight of standards development, acquisition, sustainment, and integration of training systems is fragmented among various Air Force organizations, as shown in table 2.

In the absence of an organization to guide virtual training efforts, the lead major commands have developed their own standards and acquired and fielded systems that are not interoperable and often require costly, time- consuming work-arounds to be able to train together in large, complex virtual training exercises. For example, in conducting its quarterly virtual flag exercises, the Distributed Mission Operations Center must integrate simulators and networks that have been developed to different standards. To integrate all the participants into the exercises, the center must implement “gateways” that allow dissimilar simulators to translate data; develop databases to provide a common constructive environment; and link numerous Air Force and DOD networks that have different security restrictions, bandwidth limitations, and data transfer protocols. According to officials, developing and implementing these types of solutions takes up to 9 months, and involves significant reliance on contractor personnel. The network configuration must be certified for each virtual flag event and then it must be disassembled, reconfigured, and recertified for subsequent training events. While the Air Force has been developing these work-around solutions to allow interoperability among its older aircraft simulators, similar solutions will be also be needed for Joint Strike Fighter simulators as they are fielded and integrated into distributed mission operations. According to Joint Strike Fighter program office officials, the programs’ operational requirement document specifies that the aircraft and simulators must be interoperable with other aircraft and networks, but interoperability is not scheduled to be achieved until later blocks in the program’s development. The Air Force’s current Joint Strike Fighter simulators are stand-alone and not integrated into distributed mission operations. Like other aircraft simulators, the Joint Strike Fighter simulators will require complex multilevel security guards and gateways to allow them to operate with other aircraft simulators in a distributed mission environment. Joint Strike Fighter program officials stated that since the types of interoperability challenges that they face are not unique but are similar to those of other programs there should not be any unique technical barriers that would prevent solutions to these challenges. Air Combat Command is also currently assessing the technologies needed to integrate live and virtual training for the Joint Strike Fighter.

Our prior work has found that designating an integration team, vested with the necessary authority and resources, is a critical element of managing the transformation of an organization. We note that the Army and the Navy have each designated an organization with centralized oversight over standards development, acquisition, sustainment, and integration of virtual training systems. Further, the Navy has established guiding principles and investment priorities to assist decision makers in selecting the proper simulator solution for specific training requirements and gaps, and to help avoid interoperability issues. One of the principles states that simulators intended to interface with other simulators must be compatible with the Navy Continuous Training Environment network. A Navy training instruction elaborates on this guiding principle by further stating, “Interoperability is a key objective for Fleet simulators. Translator development is expensive and time consuming, translation slows things down (introduces latency), and translation is never perfect. To reduce the need for translators to overcome interoperability challenges, Navy Continuous Training Environment network technical standards have been adopted Fleet-wide and are mandatory for new simulators that will integrate into Fleet synthetic training.”

Office of the Under Secretary of Defense, Strategic Plan for the Next Generation of Training for Department of Defense (Sept. 23, 2010). training environment. However, the Air Force has not yet identified an organization to perform this oversight. Without a dedicated organization with responsibility and accountability to integrate virtual training efforts, the Air Force may continue to face challenges in managing and integrating its virtual training efforts, including interoperability issues that lead to diminished training quality, fewer training opportunities due to lengthy preparation times, and increased costs.

Individual Air Force Initiatives to Enhance Virtual Training Capabilities Are Not Yet Guided by an Overarching Strategy

The Air Force is currently pursuing a number of individual initiatives to enhance its virtual training capabilities. Among these initiatives are the following:

Air Mobility Command is planning to create and staff a Distributed Training Center at Scott Air Force Base in December 2012. The training center is initially planned to provide the integration capability needed for transport aircraft simulators to conduct distributed mission operations on a daily, consistent basis and will also provide the ability for these simulators to participate in Distributed Mission Operations Center events. Air Mobility Command also has plans to expand the center’s capability to include integration of tanker simulators to train air refueling virtually.

The Distributed Training Operations Center has increased its capabilities in response to increased user requirements and mission requests by adding personnel and expanding event availability. Additionally, the Distributed Training Operations Center plans to work with the major commands to establish remote capability sites that would be networked to the center to increase distributed mission operations availability across the Air Force. According to Distributed Training Operations Center officials, remote-capability sites have already been established for the Air National Guard in South Dakota and Pacific Air Forces in Alaska.

Air Combat Command is in the process of establishing a Distributed Training Center at Langley Air Force Base. The training center is intended to provide a focal point for scheduling of combat air forces events not involving the Distributed Training Operations Center or Distributed Mission Operations Center. It is also intended to provide scenario development, focused on training gaps, desired unit missions, operational plan missions, and other tactics, techniques, and procedures. The training center is expected to be fully operational in September 2012.

Air Force Special Operations Command is in the process of establishing a Distributed Training Center at Cannon Air Force Base that will focus on virtual training activities for initial and mission qualification training and unit-level mission essential tasks.

Our prior work has found that strategic planning is a key element of an overarching organizational framework. For example, a leading practice derived from principles established under the Government Performance and Results Act of 1993 is to improve the management of federal agencies by developing comprehensive strategies to address management challenges that threaten their ability to meet long-term goals. We have previously reported that these types of strategies should contain results-oriented goals, performance measures, and expectations with clear linkages to organizational, unit, and individual performance goals to promote accountability and should also be clearly linked to key resource decisions. While the Air Force currently has numerous individual initiatives underway to enhance its virtual training capabilities and is planning to make additional investments, it has not yet developed an overall strategy to guide and integrate these efforts. For example, the Air Force has not outlined overall goals for its virtual training efforts, resource needs, and investment priorities. In the absence of a strategy, the Air Force cannot be certain that its individual initiatives are synchronized and will address its highest priority needs.

Air Force officials stated that they are currently developing a Live, Virtual, Constructive Flight Plan to serve as their strategy for virtual training. They told us the Flight Plan will provide direction to the major commands on the handling of operational issues and will establish an internal structure for how issues are to be raised and resolved. Officials expect that the Flight Plan will be completed by July 2012. Officials stated that a separate effort will be undertaken to develop an acquisition strategy for virtual training systems. At this point, it is unclear the extent to which these plans will contain the necessary elements of an overall strategy that the Air Force can use to manage and integrate its planning and acquisition efforts.

Air Force Lacks Cost Data for Virtual Training and Did Not Account for Virtual Training Costs When Determining Savings from Its Flying-Hour Efficiency Initiative

In outlining its efficiency initiative related to training, the Air Force estimated potential cost savings of $268 million for fiscal year 2012, and a total of $1.7 billion for fiscal years 2012 to 2016 by among other things, reducing legacy combat Air Force flying hours across the board by 5 percent. The flying-hour efficiency initiative also called for a concurrent increase in the use of high-fidelity simulators and virtual training to avoid any effect on aircrews’ mission readiness from the reduction in live flying. However, in estimating costs, the Air Force included the savings associated with reductions in live training but not the potential costs associated with increases in virtual training that were called for to offset the reduction in live training. On the basis of our prior work, cost savings estimates should include all significant costs in order to have a reasonable basis. Additionally, federal internal control standards state that decision makers need visibility over a program’s financial data to determine whether the program is meeting the agencies’ goals and effectively using resources.

Air Force officials told us that the cost savings associated with the flying- hour efficiency initiative were estimated by multiplying the reductions in live training flying hours for each aircraft by the cost per flying hour for that aircraft, and then adding the resultant figures for all the aircraft to determine total savings. For example, according to numbers provided by the Air Force, the live training cost of 1 F-15E flight hour is approximately $17,449 and F-15E flying hours were reduced by 1,782 hours. These amounts were multiplied together to arrive at the Air Force’s projected total savings of approximately $31,094,000 for reductions in F-15E flight hours. Similar calculations were made for each of the other aircraft that had their flight hours reduced and the savings for all the aircraft were summed.

The Air Force did not consider any potential costs associated with the increase in virtual training in its estimate of cost savings because it has not developed a methodology to collect and track information on the cost of its virtual training program. According to Air Force officials, some training costs could increase as a result of increases in virtual training. These costs could include expenses for aircrew to travel to simulator locations, additional contractor personnel to schedule and operate simulators, and the purchase of additional simulators to meet increased demand. Furthermore, according to Air Force officials, identifying virtual training costs is challenging because funds to support virtual training and distributed mission operations are currently dispersed across multiple program elements. For example, our analysis identified a portion of virtual training funding, specifically distributed mission operations funding, in a program element titled “Human Effectiveness Applied Research.” In another case, distributed mission operations funding was part of a program element titled “International Activities,” under an “Armaments Cooperation” subcategory that also included funding for alternative energy among other things. In 2011, the Air Force conducted a onetime study in an attempt to identify the full cost of its virtual efforts. It found that the total investment in virtual capabilities for fiscal year 2012 was at least $1.9 billion. Of that, operational training support accounted for approximately 50 percent of the annual investment, including the largest identified expenditure of $182.3 million for combat air forces distributed mission operations. However, the study noted that its efforts may not have identified all program elements associated with virtual training and therefore further steps would be needed to capture the full value of the Air Force’s virtual training investment. As of May 2012, the Air Force had not taken any additional steps to develop a methodology for identifying virtual training costs. Without a means to collect or calculate its virtual training costs, the Air Force lacks the information it needs to make informed investment decisions in the future regarding the mix of live and virtual training. Furthermore, the Air Force will be unable to determine the potential costs associated with its flying-hour efficiency initiative.

Conclusions

In an effort to achieve greater efficiencies in its training program while maintaining mission readiness, the Air Force has taken various steps to emphasize and increase the use of virtual training. Among other things, the Air Force has implemented various initiatives and established organizations intended to enhance its virtual training capabilities. However, none of these organizations have the authority necessary to ensure the integration of the Air Force’s virtual training efforts, and oversight remains fragmented. Further, the Air Force lacks an overarching organizational framework to guide its current virtual training efforts and the additional investments it plans to make. In the absence of such a framework, the Air Force faces challenges in managing its current inventory of virtual training systems and has experienced delays and costs that stem from the lack of interoperability among its simulators and networks, resulting in workarounds that are required to compensate for these limitations. An overarching management approach, including a single entity responsible for coordinating and integrating all virtual training efforts, as well as a comprehensive strategy that aligns individual efforts with goals and investment priorities, will not be enough if decision makers lack visibility over the potential costs of virtual training—especially as they consider future changes to the mix of live and virtual training. Until the Air Force has a methodology to consistently collect and track its virtual training costs and a management framework to coordinate its efforts, it will continue to face challenges to planning and conducting its virtual training and informing its future investment decisions.

Recommendations for Executive Action

To develop a fully integrated management approach to guide virtual training efforts and investments, we recommend the Secretary of Defense direct the Secretary of the Air Force to designate an entity that is responsible and accountable for integrating all of the Air Force’s virtual training efforts, including the development and enforcement of interoperability standards across virtual training systems, and investment planning; and develop an overarching strategy to align goals and funding for virtual training efforts across all Air Force major commands. This strategy should at a minimum contain elements such as results-oriented goals, performance measures, and a determination of resources needed to achieve stated goals. In addition, this strategy should show clear linkages between existing and planned initiatives and goals.

To improve decision makers’ visibility over the costs related to virtual training, we recommend that the Secretary of Defense direct the Secretary of the Air Force to develop a methodology for collecting and tracking cost data for virtual training and use this cost data to help inform future decisions regarding the mix of live and virtual training.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD stated that it concurred with all of our recommendations. In response to our recommendation to designate an entity that is responsible and accountable for integrating all of the Air Force’s virtual training efforts, including the development and enforcement of interoperability standards across virtual training systems, and investment planning, DOD stated that the Air Force has taken initial steps to designate its Headquarters, Air Force office, AF/A3/5 (Operations, Plans, and Requirements) as the single entity responsible for integrating the Air Force’s virtual training efforts.

In response to our recommendation to develop an overarching strategy to align goals and funding for virtual training efforts across all Air Force major commands, DOD stated that the Air Force is developing an overarching strategy and policy to provide a fully integrated management approach to guide its Live Virtual Constructive-Operational Training efforts and investments. It further stated that operational level guidance will be provided in Air Force Instruction 36-2251, Management of Air Force Training Systems and that investment guidance to link virtual training to “Readiness” was provided in the Air Force’s Fiscal Year 2014 Annual Planning and Programming Guidance and Program Objective Memoranda Preparation Instructions.

In response to our recommendation to develop a methodology for collecting and tracking cost data for virtual training and use this cost data to help inform future decisions regarding the mix of live and virtual training, DOD stated that the Air Force is taking actions to improve visibility related to virtual training to inform decisions regarding the mix of live and virtual training. It also stated that the Air Force Instruction regarding Management of Air Force Training Systems will provide major commands with clear guidance to employ consistent methods to collect and measure virtual training systems data. DOD said the Aviation Resource Management System will be enhanced to provide the capability to capture projected and executed aircraft virtual training and cost data to provide better oversight and management of virtual training funding. Finally, DOD stated that the Air Force is developing a standard methodology of accounting and tracking the programming and execution of program funds through improved visibility into cost categories associated with Live Virtual Constructive-Operational Training. DOD’s comments are included in their entirety in appendix II. DOD also provided a number of technical and clarifying comments, which we have incorporated where appropriate.

We are sending copies of this report to appropriate congressional committees, the Secretary of Defense, the Secretary of the Air Force, and the Under Secretary of Defense for Personnel and Readiness. In addition, this report will be available at no charge on our website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-9619 or pickups@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To address our objectives, we met with officials from the Office of the Secretary of Defense, Joint Staff, Office of the Secretary of the Air Force, Headquarters Air Force, and several Air Force major commands. Our review focused primarily on virtual training systems for manned aircraft from combat air forces, mobility air forces, and special operations forces. Excluded from this review were virtual training programs for unmanned aircraft, space, combat support, and combat service support systems. To determine how the Air Force determines the mix of live and virtual training, we obtained and analyzed training requirement instructions for combat, mobility, and special operations aircraft from each of the three lead major commands—Air Mobility Command, Air Combat Command, and Air Force Special Operations Command. We provided a questionnaire and received written responses from the major commands on the mix of live and virtual training and the benefits, limitations, and challenges of virtual training. We interviewed officials from Air Combat Command, Air Mobility Command, Air Force Special Operations Command, Air National Guard, Air Force Reserve Command, U.S. Air Forces Europe, and Pacific Air Forces.

To determine the extent to which the Air Force has developed an overarching framework to guide, oversee, and integrate its virtual training efforts, we analyzed Air Force studies on virtual training technologies and capabilities. We reviewed relevant Department of Defense (DOD) and Navy training guidance. We also reviewed our ongoing work related to Navy virtual training. We interviewed officials from the Office of the Secretary of the Air Force, Headquarters Air Force, the Joint Staff, the Department of the Navy, Air Force Major Commands, the four primary centers that facilitate distributed mission operations, and joint training officials. We visited Langley Air Force Base, Virginia, to observe F-15 and F-22 simulator operations at the Mission Training Center. We also visited the Distributed Mission Operations Center at Kirtland Air Force Base, New Mexico, to observe a distributed training event and the center’s capabilities.

To determine the extent to which the Air Force considered costs related to virtual training in estimating potential savings from its training efficiency initiative, we obtained and analyzed the Air Force efficiency calculation and compared it with cost saving estimating best practices. We analyzed Air Force budget program elements related to virtual training and distributed mission operations. We also analyzed an Air Force study on the cost associated with Air Force modeling and simulation efforts. We interviewed officials from the Air Force Defense Contracting Management Organization and Headquarters Air Force budget operations office, as well as officials from the major commands listed above.

We conducted this performance audit from August 2011 to July 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Michael J. Ferren, Assistant Director; Russell M. Bryan; Paige A. Muegenburg; Mathew Sakrekoff; and Erik S. Wilkins-McKee made key contributions to this report.