Air Force: Actions Needed to Strengthen Management of Unmanned Aerial System Pilots
Released: 2014-04-10T00:00:00

Letter

Background

Growth in Use of RPAs and Expanded Missions

The Air Force has rapidly expanded its use of RPAs in the last decade to support combat operations in Iraq and Afghanistan. The Air Force flies three types of RPAs—the MQ-1 (Predator), the MQ-9 (Reaper) and the larger RQ-4 (Global Hawk). Beyond the traditional intelligence, surveillance, and reconnaissance capability to analyze evolving battlefield conditions, the MQ-1 and the MQ-9 have been outfitted with missiles to strike targets, with equipment to designate targets for manned aircraft by laser, and with sensors to locate the positions of improvised explosive devices and moving insurgents, among other missions.

The Military Services’ Various Approaches to Assigning RPA Personnel

All the military services operate RPAs, and each uses different approaches to assign personnel to pilot them and operate their sensors. For example, the Air Force (the focus of this review) assigns officers to fly RPAs and enlisted personnel to operate the RPAs’ sensors, which provide intelligence, surveillance, and reconnaissance capabilities. In addition, the Air Force relied solely on manned-aircraft pilots to fly RPAs until 2010, when it established an RPA pilot career field for officers who specialize in flying RPAs and are not qualified to fly manned aircraft. Similarly, the Navy assigns officers to pilot RPAs, and enlisted personnel to operate RPA sensors. However, the Navy has not established a separate career field for pilots who specialize in flying RPAs and instead assigns pilots of manned aircraft to operate them. By contrast, the Army and Marine Corps have opted to assign enlisted personnel to fly RPAs and operate their sensors. Further, in both the Army and Marine Corps, there is no distinction between the pilot and sensor operator.

RPA Pilots’ Current and Future Basing

Air Force RPA pilots carry out their missions and pilot RPAs from eight active-duty bases in the continental United States including Creech, Cannon, and Beale Air Force Bases and from Air National Guard bases in six states including North Dakota, New York, and Ohio. In addition, RPA pilots are trained at some of the bases where RPAs are operated, such as at Beale Air Force Base, as well as at other bases where RPAs are not operated, such as at Holloman Air Force Base. The Air Force plans to add an Air Force Reserve unit at Hurlburt Field as well as Air National Guard RPA bases in Arkansas, Iowa, Michigan, New York, and Pennsylvania (see fig. 1).

Training and Training Costs

The initial training that the Air Force provides to its RPA pilots is designed specifically for flying RPAs and consists of two major components that take about 10 months to complete. The first major component is Undergraduate RPA Training and it consists of a basic flying skills course in which RPA pilots learn to fly a small manned aircraft in Pueblo, Colorado; instrument training in a manned-aircraft flight simulator at Randolph Air Force Base in Texas, and an RPA fundamentals course that is also at Randolph. In the second major component of their initial training, RPA pilots get their first opportunity to fly an RPA at a Formal Training Unit, which for most active-duty pilots takes place at Holloman Air Force Base in New Mexico. During this training, RPA pilots learn basic RPA operations in all mission areas including intelligence, surveillance, and reconnaissance as well as close air support. Following their time in Formal Training Units, RPA pilots finish their training by attending a 2- week joint weapons course in which they learn how to operate with the Army, Navy, and Marine Corps in a joint operational environment.

The Air Force spends considerably less to train RPA pilots than it does to train manned-aircraft pilots. Specifically, Air Education and Training Command officials estimate that the Air Force spends about $65,000 to train each RPA pilot to complete Undergraduate RPA Training. Conversely, these officials estimate that the Air Force spends an average of $557,000 for each manned-aircraft pilot to complete the corresponding portion of manned-aircraft pilot training, which is called Undergraduate Pilot Training.

Remote-Split Operations

The Air Force currently flies the bulk of its RPAs using a concept known as remote-split operations. With remote-split operations, a small number of RPA pilots deploy to operational theaters located overseas to launch and recover RPAs from various locations around the world while other RPA pilots remotely control the RPA for its mission from Air Force bases in the United States (see fig. 2). According to Air Force officials, remote- split operations help the Air Force reduce the personnel and equipment it deploys overseas because the units that launch and recover RPAs are staffed with a relatively small number of pilots, sensor operators, support personnel, and equipment. In addition, remote-split operations provide the Air Force flexibility to change the geographic region of the world where an RPA pilot conducts a mission without moving the pilot, support personnel, or equipment needed to control the RPA. If the Air Force is not able to use one of its launch and recovery sites for various reasons such as poor weather, the Air Force can continue its RPA operations by launching RPAs from a different launch and recovery site.

Promotion Process

The Defense Officer Personnel Management Act (DOPMA) created a system for managing the promotions for the officer corps of each of the military services. DOPMA specifies that the secretaries of the military departments must establish the maximum number of officers in each competitive category that may be recommended for promotion by competitive promotion boards. Career categories, also known as competitive categories, cluster officers with similar education, training, or experience, and these officers compete among themselves for promotion opportunities. Under this system, as currently implemented in the Air Force, there are several competitive categories including one that contains the bulk of Air Force officers called the Line of the Air Force, which includes RPA pilots, as well as pilots of manned aircraft and other operations-oriented careers.

To consider officers for promotion from among those who are eligible, the Air Force assigns groups of senior officers to serve as members of a promotion selection board for each competitive category of officer in the Air Force. Promotion boards consist of at least five active-duty officers who are senior in grade to the eligible officers, but no officer on the board is below the rank of major. In addition, Air Force guidance states that the Air Force attempts to provide a balanced perspective on promotion boards, and hence it selects officers who mirror, as much as possible, the officers they are considering with respect to race, sex, aeronautical rating, career field, and command. Promotion boards typically convene annually at AFPC headquarters to review a variety of records for each eligible officer, including performance and training reports as well as recommendations from supervisors. Board members assess these records using a best-qualified approach and use a variety of methods to score the records and resolve differences among the scoring of the board members, if necessary. An Air Force officer cannot serve as a member of two successive promotion boards considering officers of the same competitive category and rank.

A key feature of DOPMA is its “up-or-out” promotion system. Under this system, as currently implemented in the Air Force, promotion to the first two ranks in an officer’s career is not competitive. Specifically, 100 percent of fully qualified Air Force second lieutenants and first lieutenants are promoted after serving for 2 years in their respective ranks and do not meet with a competitive promotion board. However, as officers advance through the ranks in cohorts that are determined by the year they were commissioned, they compete for promotion against other members of their cohort at set years or zones of consideration for each rank. For example, Air Force officers are generally considered for promotion to major, or the grade of O-4, after 10 years.

Under the DOPMA system, a select group of officers can also be considered for promotion 1 or 2 years early, or “below the zone.” However, because only a limited number of officers below the zone may be promoted, officers have their greatest potential for promotion “in the zone.” If officers in a cohort are not promoted while they are in the zone, they can compete for promotion in the following one or in some instances two years later, which is known as competing “above the zone.” However, if these officers are not selected for promotion above the zone, they could be involuntarily separated from the Air Force.

The Air Force Has Made Efforts to Manage RPA Pilots but Faces Challenges to Recruit, Develop, and Retain Pilots and Build Their Morale

The Air Force has taken some steps toward managing RPA pilots using a strategic human-capital approach but faces several challenges including accurately identifying personnel requirements, limited training time for pilots, recruiting and retention difficulties, and incorporating feedback from RPA pilots into its operations.

The Air Force Has Undertaken Some Workforce Planning Efforts but It May Not Have Identified Its Personnel Requirements Accurately

The Air Force’s effort to meet combatant command RPA requirements has included some elements of strategic human-capital planning, but increasing demand and past experience indicate the Air Force has not accurately identified RPA personnel requirements. High-performing organizations use strategic human-capital planning to help them evaluate the extent to which their human-capital approaches support the accomplishment of programmatic goals. Strategic human-capital planning involves identifying human-capital needs like the necessary “shape,” which involves ensuring that agencies have the right numbers of staff at the right levels of experience, as well as the necessary size of the workforce for accomplishing agency missions while also enabling the workforce to accomplish career-development tasks, which furthers agency goals and objectives.

Air Force Steps to Plan for the Shape and Size of the RPA Pilot Workforce and React to Increased Combat Air Patrol (CAP) Requirements

The Air Force has taken steps to plan for the shape and size of the RPA pilot workforce and react to requirements from the Secretary of Defense, including adding a cadre of experienced officers to mentor officers recruited into a new career the Air Force established for RPA pilots. In order to develop a long-term, sustainable career path for pilots flying RPAs and demonstrate its commitment to RPA pilots, in 2010 the Air Force established an RPA pilot career field with a separate set of training requirements. These officers are qualified only to fly RPAs and are not qualified on Air Force manned aircraft.

In addition, the Air Force recognized that as new officers were recruited into the RPA pilot career field, they would need a group of more-senior officers to serve as mentors and leaders. Therefore, in 2011, the Air Force permanently recategorized around 475 manned-aircraft pilots who were generally serving at the ranks of major and lieutenant colonel to serve as permanent RPA pilots, according to Air Force documentation. Air Force officials stated that these more-senior pilots would help provide a leadership and experience base for the new RPA pilot career field. The officials also stated that additional manned-aircraft pilots have been permanently recategorized as RPA pilots since 2011, and Air Force documentation shows a total of 545 recategorized manned-aircraft pilots.

Furthermore, the Air Force has taken steps to plan for the size of its RPA pilot workforce. According to Headquarters Air Force officials, the number of RPA combat air patrols (CAP), directed by the Secretary of Defense and based on the mission needs of the combatant commands, is a primary factor in determining RPA pilot personnel levels. In 2010, the Secretary of Defense directed the Air Force to fund personnel to reach 65 CAPs by fiscal year 2013 and be prepared to grow beyond that requirement in future years. To determine the number of RPA pilots, the Air Force Manpower Agency conducted a personnel requirements study for MQ-1 Predator squadrons in 2008 and established the number of RPA crews required to fly one CAP for 24 hours, referred to as the crew ratio.

Based on the study, the Air Force concluded that the crew ratio for MQ-1 Predator squadrons would be 10:1, which calls for 10 RPA pilots to sustain a Predator for 24 hours. Air Force officials stated that although the 2008 study did not address the personnel requirements for MQ-9 Reaper squadrons, the Air Force used the study as the basis for establishing a 10:1 crew ratio for MQ-9 units also because MQ-1 and MQ- 9 units have similar requirements. In addition to this crew ratio, the Air Force used Air Force Instruction 38-201 to calculate the required number of additional pilots it needs for support positions such as commanders, and staff positions at various organizational levels including headquarters. Using the crew ratio and the Air Force instruction, the Air Force determined that the total number of RPA pilots required to sustain the 65 CAPs currently required by the Secretary of Defense is between 1,600 and 1,650 pilots, according to a Headquarters Air Force official.

Furthermore, the Air Force has taken steps to react to increased CAP requirements. Until 2009, the Air Force relied solely on manned-aircraft pilots serving assignments as RPA pilots to fill personnel requirements. In fiscal year 2006, manned-aircraft pilots were sustaining 12 CAPs, and the 2006 Quadrennial Defense Review stated that the Predator system alone would grow to 21 CAPs by 2010. However, according to Headquarters Air Force officials, by 2007 the demand from the combatant commands had already exceeded that benchmark. Air Force leadership committed the service to meeting the increased requirements, and the Air Force took actions to provide sufficient personnel. These actions included lengthening the assignments of manned-aircraft pilots in RPA squadrons and then extending those assignments indefinitely, mobilizing pilots from the Air National Guard and Air Force Reserve, delaying the establishment of the RPA weapons school after designating RPA as a formal weapon system, and extending the length of deployments to augment staffing levels of RPA squadrons. In 2009, the Air Force also began assigning manned-aircraft training graduates to RPA assignments as their first assignment after completing Undergraduate Pilot Training. In 2010, the Air Force established the RPA pilot career field. Figure 3 summarizes the steps that the Air Force took to react to increased CAP requirements since 2007.

Using these steps, the Air Force has made progress towards meeting the CAP requirements, but at personnel levels that were below requirements. In addition, the Air Force reduced the capacity of its RPA training unit because instructors were pulled to fly in RPA units. In fiscal year 2012, the Air Force began a reconstitution period intended to staff the training units, restart the weapons school, and increase the overall number of RPA pilots to increase the crew ratios of RPA units. As of December 2013, there were 1,366 RPA pilots, or around 85 percent of the total of 1,600 pilots determined by the Air Force as necessary to sustain RPA operations and training for 65 CAPs. In addition, the Air Force anticipates increasing the number of RPA pilot staff positions across the Air Force from 111 as of December 2013 to 300 by fiscal year 2023 to serve at various Air Force commands, including at Headquarters Air Force and Air Combat Command.

Air Force Has Not Accurately Identified Optimum Personnel Requirements and Has Not Established a Minimum Personnel Requirement

The Air Force has not accurately identified optimum personnel requirements, or crew ratio, for the number of RPA pilots it requires. We have reported that high-performing organizations use complete and current data to inform their strategic human-capital planning and remain open to reevaluating workforce planning efforts. In the 2008 study that the Air Force Manpower Agency conducted to determine the appropriate crew ratios for MQ-1 Predator squadrons, the Air Force did not account for all of the flying and administrative tasks that are required in these squadrons. While the study accounted for some important tasks that RPA pilots perform in MQ-1 squadrons such as performing operational missions, it did not account for other important tasks such as those required to launch and recover RPAs. In addition, the study did not account for some important administrative tasks such as conducting flight-safety evaluations and providing a commander’s support staff. The study acknowledged that due to its reporting time frames, it did not capture the personnel requirements of a variety of tasks.

Headquarters Air Force personnel acknowledged the study’s limitations and said that because the study omitted critical and important tasks from its analysis, the resulting crew ratio that it recommended probably did not provide enough pilots to perform the work in an MQ-1 squadron. These officials stated that, because of the study’s omissions, the 10:1 crew ratio for MQ-1 squadrons established in an Air Force instruction that was based on this study should probably be increased. Similarly, some RPA unit commanders and RPA pilots in some of our focus groups also said that the crew ratio is too low. However, to-date the Air Force has not updated the crew ratio for RPA squadrons. Headquarters Air Force officials stated that updating the crew ratio has not been a top priority. At the same time, these officials noted that more recently they have discussed the need to update the crew ratio and expressed optimism that it would become a priority in the future, though no concrete plans exist to initiate an update to the requirement.

Furthermore, an Air Force instruction states that a crew ratio establishes the number of personnel required to support a unit mission and that if a ratio is too low, combat capability is diminished and flight safety suffers. Such risks can arise when crew-ratio requirements are set too low, as well as when units operate at crew ratios that are too far below optimum crew ratios. However, Air Force documentation shows that crew ratios in RPA units have fluctuated between 7:1 and 8.5:1, and at times have dropped to 6:1, according to Air Force officials. This indicates that the RPA pilot workload is performed by fewer pilots working more hours to accomplish the mission than if the Air Force ensured that its RPA units operated at the required crew ratios. The Air Force has operated at these levels to provide a higher number of CAPs.

According to Headquarters Air Force officials, in the past the Air Force has attempted to deny requests made by combatant commanders for Air Force RPA capabilities because they push crew ratios too low. These officials stated that when the Air Force denies a request it provides justification, which include concerns about crew ratios, to the Joint Staff, which is responsible for resolving differences between combatant commanders’ requests for capabilities and the services that provide them. However, Air Force officials stated that the Joint Staff has overridden some of the Air Force denials in order to accomplish missions, despite the possibility that crew ratios would decrease. Without establishing a minimum crew ratio for RPA units, the Air Force does not have the information it needs to determine when those units are operating at crew ratio levels that expose the Air Force to unacceptable levels of risk to accomplishing its mission and ensuring safety.

A High Pace of Operations and Work Demands Limit Time for Training and Development

As a result of inaccurate crew ratios for Air Force RPA squadrons and a lack of a minimum crew ratio, the RPA pilot workforce has sustained a high pace of operations, which limits its time for training and development. The Air Force Unmanned Aircraft Systems Flight Plan 2009-2047 states that it is imperative to provide the necessary training and opportunities for advancement that will create a cadre of future Air Force leaders. However, unit commanders in each of the three locations we visited and some RPA pilots stated that the high pace of operations and demand for RPA capabilities limited their units’ time to train for the various mission sets that RPA units are required to perform. One unit commander stated that battlefield commanders that his unit supports have pointed out that his RPA pilots need training, and pilots in some focus groups noted that limited training opportunities prevent RPA units from excelling at their missions and becoming experts in their field. In addition, pilots in all 10 focus groups indicated that they are limited in their ability to pursue developmental opportunities.

Furthermore, DOD has noted that the prevalence and use of unmanned systems, including RPAs, will continue to grow at a dramatic pace. As discussed above, the Secretary of Defense has stated specifically that the requirement for 65 CAPs represents a temporary plateau in progress toward an increased enduring requirement. Also, as the national security environment changes, RPA pilots will be expected to conduct a broader range of missions across different conditions and environments, including antiaccess and area-denial environments where the freedom to operate RPAs is contested. By not creating an environment where RPA pilots can receive the training and development opportunities they need to perform their functions effectively, the Air Force may be hindering its ability to perform its mission even if it is able to operate at the optimum crew ratio that is set in the Air Force instruction.

The Air Force Faces Challenges Recruiting RPA Pilots and May Face Challenges Retaining Them in the Future

The Air Force has used a dual strategy to meet its increasing need for RPA pilots: using manned-aircraft pilots and recruiting RPA pilots, the career field established in 2010 for officers trained to only fly RPAs. However, the Air Force has faced challenges in recruiting RPA pilots since it began this career field. High-performing organizations tailor their recruitment and retention strategies to meet their specific mission needs. The Air Force intends to build a cadre of dedicated RPA pilots, and projects that RPA pilots will make up 90 percent of the RPA pilot workforce by fiscal year 2022. However, the Air Force has not been able to achieve its recruiting goals for RPA pilots in fiscal years 2012 and 2013. In fiscal year 2013, the Air Force recruited 110 new RPA pilots, missing its goal of 179 pilots by around 39 percent. Consequently, while the Air Force has made progress in increasing the total number of RPA pilots and staffed its RPA units at about 85 percent of current requirements as of December 2013, around 42 percent of those pilots are manned-aircraft pilots and manned-aircraft pilot training graduates. Both of these groups are temporary RPA pilots who serve only one assignment in an RPA squadron. While the length of these assignments can be extended, these pilots will likely not stay in the RPA squadrons permanently (see fig. 4).

Headquarters Air Force officials believe the Air Force has missed its recruiting goals in 2012 and 2013 for RPA pilots because potential recruits have a limited understanding of the RPA mission and there is a lack of recruiting officials with RPA experience to advise potential recruits. The Air Force may face challenges recruiting officers to serve as RPA pilots because of a negative perception that some in the Air Force associate with flying RPAs. Headquarters Air Force officials, RPA pilots in some of our focus groups, and one unit commander stated that some in the Air Force view flying RPAs negatively, resulting in a stigma. According to these officials one reason some view flying an RPA negatively is because flying an RPA does not require pilots to operate an aircraft while on board an aircraft in-flight. In addition, officials stated that overcoming this stigma may be difficult because publicizing the work that RPA pilots do is often not feasible due to the classified nature of RPA missions. Nonetheless, Headquarters Air Force officials stated that the Air Force projects it will meet its recruiting goals for the RPA pilot career field for fiscal year 2014 on the basis of commitments made by cadets participating in the Air Force Reserve Officer Training Corps.

We have reported that high-performing organizations make use of targeted investments such as recruiting bonuses as part of their strategies to recruit high-quality personnel with the critical skills. However, Headquarters Air Force officials reported that the Air Force is not currently exercising its option to offer a recruiting bonus as an incentive to volunteer for the RPA pilot career field. Officials from the Headquarters Air Force and the Office of the Secretary of Defense stated that such pay incentives are rarely used to recruit officers in the Air Force. Headquarters Air Force officials also stated that due to the current constrained budget environment in which DOD and the federal government are operating, the Air Force would first prefer to exhaust the use of all nonmonetary options for improving recruiting before offering bonuses. As a result, the Air Force may have to continue to rely on manned-aircraft pilots to meet RPA pilot personnel needs. This approach may not be cost-effective because the Air Force spends an average of $557,000 per pilot on traditional Undergraduate Pilot Training, compared to an average of $65,000 for Undergraduate RPA Training, according to Air Education and Training Command officials. Without a more-tailored approach to recruiting RPA pilots that increases the appeal of the new career to potential recruits, the Air Force risks perpetuating personnel shortages and may need to continue relying on manned-aircraft pilots to fill its personnel requirements.

Moreover, the Air Force uses officers as RPA pilots, but it has not evaluated whether using alternative personnel populations such as enlisted or civilian personnel as RPA pilots is a viable option. A report by the House Permanent Select Committee on Intelligence urged the Air Force to study the other military services’ experiences with using enlisted personnel as RPA operators and evaluate whether this approach would degrade mission performance. Headquarters Air Force officials stated that prior to 2010, they decided to assign officers to serve as RPA pilots because they thought officers were more appropriate since RPAs fly in complex airspace, and, in some cases, fire missiles at adversaries. Headquarters Air Force officials also stated that they have, at times, considered the use of enlisted or civilian personnel but have not initiated formal efforts to evaluate whether using such populations would negatively affect the ability of the Air Force to carry out its missions.

However, without an evaluation of the viability of using other sources of personnel, the Air Force may lack valuable information on whether additional options exist for meeting personnel requirements.

With regard to pilot retention, the Air Force has taken some steps but does not have a retention strategy for RPA pilots, though indications suggest that it could face challenges retaining them in the future. Specifically, according to Headquarters Air Force officials, the Air Force has offered assignment incentive payments to RPA pilots since the career field was established in 2010. In addition, the officials stated that manned- aircraft pilots serving assignments in RPA squadrons receive skill-based aviator career incentive pay and can receive aviator retention pay by extending their service commitment in the Air Force. Despite these incentive payments, pilots in 7 of 10 focus groups we conducted indicated that retention of RPA pilots is or will be a challenge. In addition, pilots in some focus groups stated that they are considering their options for leaving active-duty service in the Air Force to go to the Air National Guard, or Air Force Reserve, or the private sector. Unit commanders in one location we visited, pilots in some of our focus groups, and other Air Force officials stated that they were concerned about the future retention rates of RPA pilots.

Headquarters Air Force officials stated that the Air Force’s strategy for meeting personnel requirements has focused on recruiting and that they have not observed indications of a concern with the retention of RPA pilots. However, the Air Force has not evaluated the potential effect of the difficult working conditions, such as long working hours and frequently rotating shifts that we discuss in more detail later in this report, that RPA pilots face and how those conditions may affect the Air Force’s ability to retain RPA pilots, despite the situation that many of these pilots will begin to reach the end of their service commitments in fiscal year 2017. In a 2011 memorandum to the Air Force, the Secretary of Defense directed the Air Force to provide sufficient incentives to retain high-quality RPA personnel. Although the Air Force has made retention payments available to RPA pilots, these efforts may not be enough or appropriate to overcome the challenges the Air Force may face to retain RPA pilots.

The Air Force Has Mechanisms to Collect Feedback from RPA Pilots but Has Not Used That Feedback to Manage Its Human-Capital Strategy for RPA Pilots

While the Air Force has mechanisms in place to collect feedback from RPA pilots, it has not used this feedback to develop its strategic human- capital approach to managing RPA pilots, such as by incorporating their feedback into tailoring a recruiting and retention strategy or by taking actions related to training and development. High-performing organizations involve their employees in their strategic human-capital approaches and planning in order to improve motivation and morale by seeking employee feedback on a periodic basis, and using that input to adjust their human-capital approaches.

The Air Force has mechanisms in place that it has used to collect feedback from RPA pilots. For example, the Air Force solicits feedback from RPA units as well as all other Air Force units during an annual Unit Climate Assessment that gauges discrimination, harassment, and morale issues at the unit level. While this effort is not specific to the RPA units, it does include assessments of RPA units. Unit commanders can use the results of their Unit Climate Assessments to address challenges at the local unit level. However, Headquarters Air Force officials responsible for managing RPA pilots have not obtained information from these assessments to identify whether they include potentially valuable information about any concerns related to establishing the RPA pilot career field. Headquarters Air Force officials stated that the Air Force created this career field more quickly and under greater operational demand than any career field in recent Air Force history. However, these officials also stated that using feedback from the Unit Climate Assessments to address issues at a headquarters level that would affect RPA pilots could undermine unit commanders. They also noted that officials at the headquarters level might lack the proper context for understanding the assessment results.

The Air Force also collected feedback from RPA pilots in studies the Air Force School of Aerospace Medicine published in 2011 and 2013 to assess the level of and reasons for stress in personnel assigned to RPA units, which included surveys and interviews of RPA pilots. In response to these studies, the Air Force took actions designed to address stress in personnel assigned to RPA units. For instance, the studies recommended that the Air Force assign an operational psychologist to each RPA unit, and, in response, local flight surgeons, clinical providers, and aerospace physiologists have created teams to help address stress concerns at the base level. While researchers from the Air Force’s medical research community conducted these studies, they included findings related to personnel shortages that are germane to the Air Force personnel and operations communities. However, Headquarters Air Force officials from the personnel and operations communities stated that, prior to our review, they were unaware of the studies and their findings.

RPA pilots in our focus groups also noted information that suggests that incorporating pilot feedback from existing mechanisms could help improve communication and address issues pilots are facing. For example, pilots in some of our focus groups stated that they did not know what the career path for an RPA pilot is or what steps they should take to advance. Further, in some of our focus groups, manned-aircraft pilots who are serving assignments as RPA pilots expressed uncertainty regarding whether they will be able to return to their manned platforms and what effect, if any, their RPA assignment will have on their careers. Pilots in some focus groups also reported that senior leadership had not communicated to them about this uncertainty, and one pilot specifically noted that the lack of communication negatively affects morale. Without using existing mechanisms to obtain feedback from RPA pilots directly, Headquarters Air Force may be missing an opportunity to obtain information that can help it address recruiting, retention, training, and development challenges related to RPA pilots.

Air Force Has Taken Some Actions to Address the Potentially Challenging Working Conditions RPA Pilots Face but Has Not Fully Analyzed Being Deployed-on-Station

Air Force RPA Pilots Find Their Work Rewarding but Report Multiple Challenging Working Conditions, Some of Which the Air Force Has Taken Actions to Address

RPA pilots find their mission rewarding, but they reported that they face multiple, challenging working conditions. RPA pilots in 8 of the 10 focus groups we conducted reported that they found it rewarding to be able to contribute to combat operations every day through the RPA mission. For instance, one pilot stated that the mission is the reason that he had decided to become a permanent RPA pilot and that it was rewarding to contribute to overseas contingency operations, which he would not be able to do in any other job. Similarly, the Air Force School of Aerospace Medicine published studies in 2011 and 2013 that evaluated the psychological condition of RPA personnel and found that RPA pilots held positive perceptions of the effect and contributions of their work. However, RPA pilots also stated that they face multiple challenging working conditions including: long hours, working shifts that frequently rotate, and remaining in assignments beyond typical lengths. RPA pilots in all of our focus groups reported that these challenging conditions negatively affected their morale and caused them stress. Similarly, the Air Force School of Aerospace Medicine studies found that RPA personnel reported sources of stress that were consistent with the challenges we identified. These challenges include the following:

RPA pilots in 8 of our 10 focus groups stated, and Air Force studies we reviewed show, that RPA pilots work long hours. RPA pilots in 7 of our focus groups described factors that contribute to their long hours including performing administrative duties and attending briefings, in addition to flying shifts. The Air Force studies also found that working long hours was one of the top five reasons for stress among personnel in RPA squadrons. In the studies, over 57 percent of respondents reported that they worked more than 50 hours per week. In addition, the studies found that over 40 percent of respondents reported that performing administrative duties added hours to their work week and was the third-highest reason for stress among active- duty RPA personnel.

RPA pilots also reported that it was challenging to work on shifts that rotate. RPA pilots in 7 of the 10 focus groups we conducted stated that constantly rotating shifts caused sleep problems for them because they must continuously adjust their sleep schedule to accommodate new shifts. In addition, pilots noted that continuously rotating to new shifts disrupted their ability to spend time with their family and friends. Officials told us that it was ideal for pilots working evening or night shifts to maintain a consistent sleep pattern on their off-duty days even though those sleep patterns would require that pilots sleep while their family and friends were awake. However, some RPA pilots reported that they typically adjusted their sleep schedules dramatically on their off-duty days so they could spend time with their families and that these changes to their sleep schedules resulted in significant fatigue both at home and when they returned to work. Similarly, over half of the respondents to the surveys included in the Air Force studies we reviewed reported that shift work caused a moderate to large amount of their stress.

RPA pilots in 5 of our focus groups reported that being assigned to continue flying RPAs for periods extending beyond the typical Air Force assignment was difficult. In all of the focus groups we conducted with RPA pilots, those who plan to return to flying manned aircraft stated that they have been required to stay in their assignments for periods that are longer than a typical Air Force assignment. Air Force officials stated that there is no requirement for officers to move to a new assignment after a specified period. However, pilots in our focus groups and Air Force headquarters officials said that officer assignments typically last 3 to 4 years. Air Force documentation shows that some of these pilots have been in their RPA assignments for over 6 years. Moreover, the Air Force studies also found that one of the most common stressors that RPA personnel cited was the lack of clarity regarding when they would return to their careers in manned aircraft. Specifically, the 2011 study states that the Air Force informed RPA pilots who previously flew manned aircraft that their RPA assignments were temporary and after 3 to 4 years they could return to their manned-aircraft career. The study goes on to state that due to the increasing demand for RPAs and the long-standing surge in RPA operations, many pilots have been unable to return to their manned-aircraft careers and, until recently, the Air Force kept them in these assignments indefinitely.

The Air Force has taken some actions to address some of the challenging working conditions that RPA pilots face. The Air Force studies included over 10 recommendations to address the sources of stress that RPA personnel reported. For example, the studies recommended that the Air Force assign an operational psychologist to each RPA unit to help commanders optimize work-rest schedules and shift cycles, and identify pilots who are reaching elevated levels of fatigue or stress. In response, the Air Force has assigned mental-health providers that are dedicated to RPA squadrons at Beale, Cannon, and Creech Air Force Bases. However, the studies also recommended that the Air Force increase staffing in RPA squadrons to reduce the number of hours that RPA personnel work and to help establish better shift schedules. Air Force researchers stated that increasing staffing levels, or crew ratios, in RPA squadrons would be the most-effective means to reduce RPA pilot stress, but as discussed above, the Air Force has operated its RPA squadrons below the optimum crew ratios.

Air Force Has Not Fully Analyzed Challenges That RPA Pilots Face Related to Being Deployed-on- Station

RPA pilots also face challenges related to being deployed-on-station as they balance their warfighting responsibilities with their personal lives. Because pilots are able to operate RPAs from Air Force bases in the United States and are thus able to live at home—what is known as being deployed-on-station—their dual role juxtaposes stress related to supporting combat operations with the strains that can occur in their personal lives. While these pilots face this challenging working condition that may affect their quality of life, DOD’s Quadrennial Quality of Life Reviews have emphasized DOD’s continued commitment to provide servicemembers with the best quality of life possible.

Being deployed-on-station is a new concept in warfighting, and a 2011 report prepared for the Air Force Medical Support Agency describes five conditions that personnel who are deployed-on-station can experience. The report notes that these personnel (1) experience a justifiable risk of being the target of hostile adversary attacks because they are combatants and their bank accounts, reputations, or physical safety could be targeted; (2) operate in contact with and sometimes kill adversaries, although operations they conduct are out of direct risk from combat; (3) must act with urgency to sometimes kill adversaries and take other time- pressured actions to help ensure combatants they support do not lose their lives; (4) work on a wartime rhythm that includes 24/7 operations 365 days a year; and (5) are required to conceal information from friends and family about their work because their missions are often classified. A Headquarters Air Force official described being deployed-on-station as a status between deployed-in-theater and not deployed and emphasized that personnel who are deployed-on-station are not directly engaged in combat, which is a significant component of being deployed. The official also acknowledged that being deployed-on-station can be more challenging than assignments with more-limited connections to the battlefield.

RPA pilots in each of the 10 focus groups we conducted reported that being deployed-on-station negatively affected their quality of life, as it was challenging for them to balance their warfighting responsibilities with their personal lives for extended periods of time. RPA pilots in some of our focus groups, as well as commanders of RPA squadrons, noted that they would prefer to deploy-in-theater for 6 months with a clear end point and be separated from their family and friends rather than be deployed-on- station for 3 or more years. One commander stated that he preferred being deployed-in-theater and knowing when his deployment would end. In contrast, he stated that in an RPA squadron, it was difficult to juggle his warfighting role with the typical challenges of home life for multiple years. Likewise, the Air Force studies found that being deployed-on-station was one of the most commonly cited stressors that RPA personnel reported.

In addition, RPA pilots in 6 of our 10 focus groups reported that they are expected to do more work than their counterparts who are deployed-in- theater. For example, RPA pilots in some of our focus groups who had previously deployed-in-theater stated that they are expected to complete administrative tasks that are not required of them when they are deployed-in-theater.

Headquarters Air Force officials as well as pilots in some of our focus groups stated that the Air Force provides support to personnel who are deployed-in-theater that it does not provide for personnel who are deployed-on-station. Moreover, the Air Force has surveyed RPA personnel and other deployed-on-station personnel to study their stress and mental health, but it has not fully analyzed the effects of being deployed-on-station. Specifically, it has not fully analyzed whether being deployed-on-station has negative effects on quality of life that are not attributable to the stressors that are related to low unit-staffing levels that we discussed above such as rotating shifts and long assignments. As a result, the Air Force does not have the information it needs to determine whether being deployed-on-station has a negative effect on the quality of life of RPA pilots that is not attributed to the other factors and what steps might be needed to reduce those effects.

The Air Force Monitors RPA Pilot Promotion Rates but Has Not Analyzed Factors Related to Those Rates

The Air Force Monitors RPA Pilot Promotion Rates and Has Found That They Are Lower Than in Other Career Fields

AFPC monitors the promotion rates of RPA pilots and has found that they were promoted below the average rate for active-duty line officers on 20 of 24 officer promotion boards since 2006. We reached the same conclusion based on our review of data for these promotion boards. We also found that RPA pilots were promoted below the average rate of manned-aircraft pilots on 21 of 24 boards. Furthermore, we compared the promotion rates of RPA pilots to those of other career fields and found that RPA pilots were promoted at the lowest rate of any career field on 9 of the 24 boards and were promoted in the lowest 5 percent of the career fields that competed on 5 additional boards. Conversely, RPA pilots were promoted in the top 50 percent of the career fields that competed on only 3 boards of the 24 boards. More specifically, RPA pilots competing for promotion to each rank that we analyzed faced challenges. RPA pilots competing for promotion to major were promoted in the top 50 percent on just one of the seven promotion boards since 2006. RPA pilots competing for promotion to lieutenant colonel were promoted at the lowest or next- to-lowest rate compared to the other career fields that competed on 7 of the 9 boards since 2006. Likewise, RPA pilots competing for promotion to the rank of colonel had the lowest promotion rate of any career field that competed on 4 of the 8 colonel boards since 2006. Figures 5, 6, and 7 display the results of our analyses.

Air Force Has Not Analyzed Factors Related to Lower RPA Pilot Promotion Rates

While AFPC has monitored the promotion rates of RPA pilots, it has not analyzed the factors related to lower promotion rates for these pilots. It is a common statistical practice when analyzing how selected factors are related to a given outcome to account for other key factors that could also be related to the outcome. Although AFPC analyzed the promotions of officers in the Line of the Air Force competitive category, which includes RPA pilots, and identified factors related to promotion outcomes for officers in this category, it has not incorporated a key factor—the career field effect of being an RPA pilot—into its analysis.

AFPC analyzed promotion data of officers in the competitive category that includes RPA pilots called Line of the Air Force and found multiple factors related to promotion outcomes. Specifically, AFPC analyzed these data using logistic regression, which is a statistical method that enables AFPC to analyze the relationships among multiple factors. Using this method, AFPC identified a number of factors that are positively and negatively related to promotions. For example, AFPC found that one of the two factors with the most-substantial positive relationship to promotions was for an officer to have completed a professional military education program by attending an Air Force school in-residence, rather than completing the same professional military education program by correspondence. The other factor with the most-substantial positive relationship was for an officer to have completed an advanced academic degree. By contrast, AFPC found that officers who have unfavorable information, such as performance-related reprimands, in their personnel files are promoted at lower rates, in general, than officers who do not.

AFPC did not include the career field effect of being an RPA pilot as a factor in its analysis. As a result, AFPC does not know whether or how being an RPA pilot is related to promotions for these pilots. AFPC has analyzed other careers and found that most careers are not related to promotion rates. AFPC officials stated that they had not analyzed this effect because most of the officers currently serving as RPA pilots are temporary RPA pilots and AFPC does not typically analyze a career field effect of temporary assignments.

In addition, AFPC assumed that the factors that were substantially related to promotions for the Line of the Air Force category were also substantially related to promotions for the RPA pilot subgroup, but did not confirm that its assumption was warranted. AFPC officials stated that when they analyzed the records of RPA pilots, they focused on the factors identified in the analysis of Line of the Air Force officers, including completing professional military education in-residence and advanced degrees. They found that RPA pilots generally completed professional military education in-residence and advanced degrees at lower rates compared to the average rates for officers who had been promoted since 2006. However, by not including the career field effect of being an RPA pilot into its analysis, the Air Force cannot determine whether these factors have the same relationship with RPA pilot promotions as they do on officer promotions in the broader Line of the Air Force category.

Air Force Reported Reasons for Low RPA Pilot Promotions Rates to Congress and Took Actions in Response without a Comprehensive Understanding

The Air Force reported reasons for low RPA pilot promotions rates to Congress and took actions to raise those rates without a comprehensive understanding of the factors related to the promotion rates of these pilots. Specifically, the Air Force attributed low RPA pilot promotion rates to three factors: (1) RPA pilots completed professional military education at lower rates than average; (2) RPA pilots completed advanced degrees at lower rates than average; and (3) the process the Air Force used to select RPA pilots. As discussed above, the AFPC’s approach to identify the first two factors assumed that their relationships with promotion rates for RPA pilots as a subgroup would be the same as those with the Line of the Air Force as a whole, but this assumption was not confirmed through analysis. Regarding the third factor, Air Force documentation states “lower quality pilots are generally sent to RPA squadrons.”

Headquarters Air Force officials and two commanders of manned-aircraft squadrons explained that commanders select pilots from their squadrons to assign to RPA squadrons and in general most commanders assign less-skilled pilots and less-competent officers to these squadrons. Headquarters officials also stated that less-skilled and less-competent officers generally had fewer of the factors AFPC identified that positively influence promotions in their records than their peers. Air Force officials also explained that because the bulk of RPA pilots who have competed for promotion since 2006 were assigned using this process, they believe these are the reasons that RPA pilots have been promoted at lower rates than their peers. However, the Air Force has not incorporated variables into its analysis to account for RPA pilots or the process to assign them to determine whether they are related to promotions of RPA pilots. Consequently, the Air Force report to Congress may not be accurate because the Air Force does not have comprehensive analysis to demonstrate that these factors are actually related to RPA pilot promotions.

Recently, the Air Force has taken actions to raise promotion rates of RPA pilots. First, to communicate to promotion boards that promoting RPA pilots was important, the Secretary of the Air Force has issued instructions since 2008 to each officer promotion board, directing them to consider the strategic effect made by RPA pilots when evaluating their records for promotion. In the instructions, the Secretary directs board members to consider that RPA pilots’ records may not show the same career progression as their peers because of operational requirements they have had to meet to satisfy the needs of the Air Force. Second, the Air Force intervened on behalf of RPA pilots to enhance their opportunities to achieve one of the two most important factors that AFPC identified in its analysis of all Line of the Air Force officers by reserving 46 in-residence seats in Air Force professional military education schools in 2012 for RPA pilots who were competing to be promoted to major. Moreover, the Air Force stated in its August 2013 report to Congress that its long-term plan to raise promotion rates is to attract “quality” recruits to the RPA pilot career field and to establish a sustainable pace of operations that will enable these pilots time to complete in-residence professional military education and advanced academic degrees. However, because it has not fully analyzed the career field effects of being an RPA pilot, it is unclear whether the Air Force is targeting these corrective actions at the right factors. Consequently, the Air Force’s actions may have limited effect on improving the promotion rates for RPA pilots.

Conclusions

The Air Force has demonstrated a commitment in recent years to the use of RPAs, believing that the capabilities they provide are worth the service’s investment in both platforms and personnel. As the RPA pilot career field evolves, it will be important that Air Force senior leadership demonstrates a commitment to a human-capital management approach that addresses a number of outstanding challenges. For instance, without updating its optimum crew ratio for RPA units, the Air Force may have RPA pilot shortfalls even after its current requirement is met, which could exacerbate existing strains on this workforce. In addition, by not establishing a minimum crew ratio below which RPA units cannot operate, the Air Force does not know when it is operating at unacceptable levels of risk to mission and safety. Further, without developing a strategy tailored to address specific challenges of recruiting and retaining RPA pilots, current pilot shortfalls may persist even longer than expected. Finally, without evaluating the viability of using alternative personnel populations, such as enlisted or civilian personnel, the Air Force may not meet and sustain required RPA pilot staffing levels.

Moreover, without incorporating feedback from RPA pilots using existing feedback mechanisms, the Air Force may be missing opportunities to manage its human-capital strategies effectively for these pilots. Also, RPA pilots face a number of challenging working conditions that can affect their quality of life including those associated with being deployed-on- station. However, without analyzing whether being deployed-on-station has long-term negative effects, the Air Force does not have the information it needs to determine whether it should take any action in response. Finally, while the Air Force has taken action to improve the chances for RPA pilots to be promoted, senior Air Force leaders cannot be assured that the actions are the appropriate ones because the Air Force has not analyzed the effect that being an RPA pilot itself may have on those chances.

Recommendations for Executive Action

We recommend that the Secretary of Defense direct the Secretary of the Air Force to take the following seven actions: update crew ratios for RPA units to help ensure that the Air Force establishes a more-accurate understanding of the required number of RPA pilots needed in its units, establish a minimum crew ratio in Air Force policy below which RPA units cannot operate without running unacceptable levels of risk to accomplishing the mission and ensuring safety, develop a recruiting and retention strategy that is a tailored to the specific needs and challenges of RPA pilots to help ensure that the Air Force can meet and retain required staffing levels to meet its mission, evaluate the viability of using alternative personnel populations including enlisted or civilian personnel as RPA pilots to identify whether such populations could help the Air Force meet and sustain required RPA pilot staffing levels, incorporate feedback from RPA pilots by using existing mechanisms or by collecting direct feedback from RPA pilots, analyze the effects of being deployed-on-station to determine whether there are resulting negative effects on the quality of life of RPA pilots and take responsive actions as appropriate, and include the career field effect of being an RPA pilot into AFPC’s analysis to determine whether and how being an RPA pilot is related to promotions and determine whether the factors AFPC identified in its analysis of Line of the Air Force officers are also related to RPA pilot promotions.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for review and comment. The Deputy Director of Force Management Policy, Headquarters Air Force provided written comments in response to our report. In its written comments, the Air Force concurred with four of our seven recommendations and partially concurred with the remaining three recommendations. The Air Force’s written comments are reprinted in their entirety in appendix III. The Air Force also provided technical comments that we have incorporated into this report where applicable.

In concurring with our first three recommendations, the Air Force stated that it: has an effort underway to update crew ratios for RPA units and expects to have this effort completed by February 2015; a minimum crew ratio would indicate when the Air Force receives a request for forces that would pose risks to the mission and safety and it expects to respond to our recommendation by February 2015; and will develop a recruiting and retention strategy that is tailored to the specific needs and challenges of RPA pilots and expects to have this done by October 2015.

In concurring with our fifth recommendation, to incorporate feedback from RPA pilots by using existing mechanisms or by collecting direct feedback from RPA pilots, the Air Force stated that if it determines that it is appropriate to collect such feedback, it will do so using a survey. We continue to believe that collecting this feedback could be a useful tool for the Air Force to develop a tailored recruiting and retention strategy and to inform actions it may take related to training and developing RPA pilots.

The Air Force partially concurred with our fourth recommendation that it evaluate the viability of using alternative personnel populations as RPA pilots and determine if such populations could help the Air Force meet and sustain required RPA pilot staffing levels. The Air Force stated that it considered assigning enlisted personnel as RPA pilots, but it decided that the responsibilities of piloting an RPA were commensurate with the rank of officers instead. At the same time, the Air Force stated that it has initiated a review of some of its missions and the ranks needed to execute those missions and that it may consider using enlisted airmen in this review.

In our report, we acknowledge that the Air Force had previously considered using enlisted personnel as RPA pilots and that the Air Force decided instead to use officers. However, it is not clear what steps the Air Force took in its previous considerations. We think it is a positive step that the Air Force has initiated a review of Air Force missions and rank requirements to execute those missions. Considering the significant role that RPAs play in the Air Force mission, we believe the Air Force should include RPA pilots in its review to evaluate whether enlisted personnel as well as civilians may provide a means for the Air Force to address shortfalls in the staffing levels of RPA pilots.

In addition, the Air Force partially concurred with our sixth recommendation that it analyze the effects of being deployed-on-station to determine if there are resulting negative effects on RPA pilots’ quality of life and take responsive actions as appropriate. In response to our recommendation, the Air Force stated that it had studied the effects that being deployed-on-station has on RPA pilots and that many of the stressors it identified in these studies were related to low unit staffing levels. In addition, the Air Force asked us to focus our recommendation on an evaluation of these studies.

We acknowledge in our report that the Air Force evaluated the psychological condition of RPA personnel who are deployed-on-station in studies it published in 2011 and 2013. We also acknowledge that the primary recommendation these studies make is to increase staffing levels in RPA units to alleviate the stress of RPA personnel. As we discussed in our report, RPA units have been understaffed and thus increasing staffing levels may be appropriate. However, our finding is focused on whether being deployed-on-station has negative effects on quality of life that are not attributable to the stressors that are related to low unit-staffing levels. We think that a more complete understanding of the effects of being deployed-on-station that are not attributable to low staffing levels will help the Air Force determine if responsive actions are needed that go beyond increasing staffing levels. Further, the 2011 report prepared for the Air Force Medical Support Agency that focuses more directly on the concept of being deployed-on-station is a constructive source of input for the Air Force to understand any negative effects of being deployed-on-station. However, it is not clear that an evaluation of this report and the 2011 and 2013 studies will provide the Air Force with a complete understanding of this new deployment concept’s consequences for its personnel.

Finally, the Air Force partially concurred with our seventh recommendation that it include the career field effect of being an RPA pilot into AFPC’s promotion analysis to determine if being an RPA pilot is related to promotions and determine if other factors that AFPC identified in its analysis of Line of the Air Force officers are also related to RPA pilot promotions. The Air Force stated that the RPA career field is a subsection of the Line of the Air Force and therefore factors related to promotions identified in analysis of the Line of the Air Force are directly related to RPA pilot promotions. In our report, we acknowledge that the Air Force identified factors related to promotion outcomes for officers in the Line of the Air Force competitive category. However, as we discussed in the report, not including the career field effect of being an RPA pilot as a factor in its analysis has several consequences. First, AFPC does not know whether or how being an RPA pilot is related to promotions for these pilots. Second, the Air Force cannot determine whether the factors that it found that are related to promotions for the Line of the Air Force competitive category have the same relationship with RPA pilot promotions. Third, the information the Air Force included in a report to Congress in August 2013 on education, training, and promotion rates of RPA pilots may not be accurate. Finally, it is unclear whether the Air Force is targeting actions to increase RPA promotion rates at the right factors and thus its actions may have limited effect.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Defense, and the Secretary of the Air Force. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3604 or FarrellB@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Scope and Methodology

To understand the context of each of the issues in our review, we analyzed various Department of Defense (DOD) and Air Force documents. This documentation included a report to Congress by the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics on the future of unmanned aerial systems and a report by the Air Force Audit Agency on the Air Force’s personnel management of pilots flying RPAs. We also reviewed reports that we previously issued that address topics related to our review including a 2010 report on DOD planning, training, and doctrine for unmanned aircraft systems.

To evaluate the extent to which the Air Force uses a strategic human- capital approach to manage remotely piloted aircraft (RPA) pilots, we used a model of human-capital management GAO had previously developed that specifies leading practices that high-performing organizations exhibit in their strategic human-capital management. The Model for Strategic Human Capital Management is intended to help federal organizations use their human capital effectively and integrate human-capital considerations into daily decision making and planning for the program results they wish to accomplish. It identifies concepts and leading practices that are organized into strategic human-capital management cornerstones including strategic human-capital planning; acquiring, developing, and retaining talent; and creating results-oriented cultures.

To adapt the criteria to the context of this review, we reviewed the model to identify specific practices that organizations can use to make progress associated with each of the four strategic human-capital management cornerstones. We then analyzed each practice to determine whether it was appropriate and relevant to both the RPA pilot workforce and the military context overall. After identifying the list of practices, we discussed our adaptation with Air Force officials, who agreed they were appropriate and relevant and provided points of contact for obtaining information on each practice.

We interviewed officials from Headquarters Air Force offices including the Officer of Manpower, Personnel, and Services Policy and the Office of Operations, Plans, and Requirements Policy to gather their perspectives and information on practices across all four cornerstones. From these offices, we obtained and analyzed documentation, including strategic DOD and Air Force guidance and data on personnel levels, recruiting, incentive pays, and attrition rates for remotely piloted aircraft (RPA) pilots. In addition, we interviewed knowledgeable officials from the Office of the Under Secretary of Defense for Military Personnel Policy on the Air Force’s use of incentives to recruit and retain RPA pilots. We collected perspectives from RPA pilots and RPA unit commanders on the Air Force’s strategic human-capital planning practices, including the effects of those practices on their training, professional development, quality of life, and retention, as well as any efforts the Air Force has made to solicit feedback from and communicate about key issues with RPA pilots. We also interviewed knowledgeable officials from the Air Force Personnel Center on practices related to results-oriented cultures.

Furthermore, we compared the perspectives and documentation we collected to the GAO criteria and held discussions with Air Force officials to discuss instances in which the Air Force’s management actions were not consistent with these criteria. We discussed challenges raised by the RPA pilots and unit commanders with whom we spoke, including any efforts in place to address the challenges.

To evaluate the extent to which the Air Force has addressed concerns, if any, about the working conditions of RPA pilots that may affect their quality of life, we identified and analyzed criteria included in DOD’s 2009 and 2004 Quadrennial Quality of Life Reviews in which DOD expresses its commitment to provide servicemembers with the best quality of life possible through support and development of responses to emerging servicemember needs. DOD has broadly defined quality of life to include such factors as morale, health and wellness, and work-life balance. To understand these reviews and the commitments, we obtained information from the Office of the Deputy Assistant Secretary of Defense for Military Community & Family Policy, which is responsible for conducting the department’s Quadrennial Quality of Life Reviews. To understand challenges in the working conditions that RPA pilots may face we analyzed studies that the Air Force conducted to assess the stress and mental-health condition of RPA personnel, including RPA pilots.

In particular, we reviewed and analyzed two studies conducted by the Air Force School of Aerospace Medicine published in 2011 and 2013, which identified the sources of stress of RPA personnel. The studies’ results were based on self-administered surveys of Air Force RPA personnel, including pilots, from squadrons in Air Combat Command, Air Force Special Operations Command, the Air National Guard, and the Air Force Reserve. The surveys were administered in 2011 and 2012 with response rates from RPA squadrons that ranged from 24 to 98 percent. The surveys included questions related to exhaustion, distress, and post- traumatic stress disorder.

We also interviewed the researchers who conducted these studies to clarify our understanding of their methods, findings, and recommendations to alleviate the stress of RPA personnel. In addition, we analyzed a report prepared for the Air Force Medical Support Agency that describes the defining characteristics of being deployed-on-station and examines the challenges that personnel who are deployed-on-station face. To obtain a firsthand account of the challenging working conditions that RPA pilots face, we conducted focus groups with pilots at Beale, Cannon, and Creech Air Force Bases. We also interviewed leadership officials at these bases to obtain their perspective on the challenges that RPA pilots in their units face. Moreover, we interviewed mental-health professionals at each of the bases we visited to obtain their perspectives on the working conditions of RPA pilots and any effects on their quality of life.

To evaluate actions the Air Force has taken to address the challenging working conditions RPA pilots face, we analyzed the recommendations that were included in the studies conducted by the Air Force School of Aerospace Medicine and the report prepared for the Air Force Medical Support Agency. We also obtained and analyzed documentation provided by the Air Force Medical Support Agency that describes actions the Air Force has taken in response to these recommendations and we interviewed officials from this agency to further understand these actions. Furthermore, we interviewed and obtained information from officials in the Air Force Office of Manpower, Personnel and Services Policy and the Office of Operations, Plans and Requirements Policy to determine any actions the Air Force has taken to alleviate the challenging working conditions that RPA pilots face. We also obtained information from commanders and mental-health professionals at each of the bases we visited to understand actions they have taken to address the challenging working conditions that RPA pilots face and that affect their quality of life.

To evaluate the extent to which the Air Force analyzes the promotion rates of RPA pilots, we applied criteria from common statistical practices, which indicate that when analyzing relationships between selected factors and a given outcome researchers should account for other key factors that could also explain that relationship. To understand the context of Air Force officer promotions, we reviewed relevant laws and Air Force guidance including the Defense Officer Personnel Management Act and Air Force Instruction 36-2501. To identify the promotion rates of Air Force RPA pilots and how their promotion rates compared to officers in other careers in the Air Force, we analyzed promotion-rate data for officers in the Line of the Air Force competitive category who were promoted “in-the-zone” to the ranks of major, lieutenant colonel, and colonel. We analyzed data from 2006 to the most-recently available data, which for promotion to major and colonel was 2012 and for promotion to lieutenant colonel was 2013.

We focused on Line of the Air Force officers, because RPA pilots are included in this category. We focused on officers promoted in-the-zone because this zone is the point in an officer’s career when his or her opportunity for promotion is the highest. We focused on rates of promotion to the ranks of major, lieutenant colonel, and colonel because the promotion rates from second lieutenant to first lieutenant and from first lieutenant to captain are nearly 100 percent, and hence the first competitive promotion opportunity for an Air Force officer occurs as he or she becomes eligible for promotion to the rank of major. In addition, we did not evaluate promotion rates above colonel because no RPA pilots have been promoted to the general officer ranks in the Air Force yet.

To identify the percentile of RPA pilot promotion rates compared to other line officer career fields, we analyzed data on the range of promotion rates of active-duty officers from the careers that competed in the promotion zone on each promotion board to the ranks of major, lieutenant colonel, and colonel from 2006 to 2013. For this analysis, the promotion rate of RPA pilots includes the rate for permanent RPA pilots (i.e., RPA pilots and recategorized RPA pilots) as well as temporary RPA pilots (i.e., manned-aircraft pilots serving assignments in RPA squadrons and manned-aircraft pilot training graduates). For this analysis all of the listed career fields are mutually exclusive. That is, if a temporary RPA pilot was identified as an RPA pilot in this analysis, the pilot was not included in the data to calculate promotion rates for other careers such as the manned- aircraft career fields. For each promotion board, officers from between 22 and 33 careers competed for promotion. This analysis excludes career fields where fewer than 10 officers were eligible for promotion, because the rate of promotion in these cases is highly sensitive to the outcomes of single individuals. However, we included the results from 8 boards in which fewer than 10 RPA pilots competed for promotion to provide a more-comprehensive account of RPA pilot promotions. The promotion rate that we calculate for these instances should be considered cautiously since the outcome of one or two individuals could have a large effect on the overall rate. Fewer than 10 RPA pilots were eligible for promotion to the rank of lieutenant colonel for the first 2006 board as well as the 2007 and 2008 boards. In addition, fewer than 10 RPA pilots were eligible for promotion to the rank of colonel for the 2006, 2007, 2008, and both of the 2009 promotion boards. We obtained these data from the Air Force Personnel Center (AFPC), and to understand the methods AFPC used to collect, store, and maintain these data, we interviewed officials from AFPC and reviewed documentation they provided, and we found the data to be reliable for our purposes.

To evaluate steps the Air Force took to analyze the promotion rates of RPA pilots and the reasons that these rates have been lower than average, we interviewed Air Force officials in headquarters personnel offices as well as AFPC offices. In addition, we evaluated documentation of AFPC’s analysis of officer promotions rates including the results of AFPC’s logistic regression identifying the factors that are related to officer promotion. We also reviewed the August 2013 report that the Air Force provided to Congress on the promotion rates of RPA pilots in which the Air Force identifies reasons for lower promotion rates of RPA pilots. To identify actions the Air Force took to respond to low RPA pilot promotion rates, we evaluated relevant documentation including instructions the Secretary of the Air Force has provided to promotion board members since 2008 in which the Secretary communicates the importance of promoting RPA pilots. We also reviewed briefings that Air Force headquarters offices as well as AFPC prepared for the Secretary of the Air Force on additional steps the Air Force took to address low RPA pilot promotion rates. We also analyzed the Air Force’s August 2013 report to Congress and additional documentation that the Air Force provided about its plans to raise promotion rates of RPA pilots.

As we noted earlier, to obtain the perspectives of RPA pilots related to each of our three objectives we conducted 10 focus groups that each consisted of between six and nine active-duty RPA pilots during site visits to Beale, Cannon, and Creech Air Force Bases. To conduct these focus groups we randomly selected RPA pilots to participate, asked them a structured set of questions during meetings that lasted about 90 minutes, and took detailed notes. We then evaluated these notes using content analysis to develop our findings. We discuss the methods we used to select our participants, develop questions, conduct the focus-group meetings, and analyze the information we obtained in the focus groups, and the results of our analysis, in more detail in appendix II.

We conducted this performance audit from February 2013 to April 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Focus-Group Methodology and Results

To obtain the perspectives of pilots of remotely piloted aircraft (RPA) related to each of our three objectives, we conducted 10 focus group meetings with active-duty RPA pilots during site visits to Beale, Cannon, and Creech Air Force Bases. We decided to visit the three bases we selected because more RPA pilots are stationed at these bases than other Air Force bases. We specifically included Beale Air Force base because we wanted to obtain the perspectives of the RPA pilots who fly the RQ-4 (Global Hawk) who are stationed there. In addition, we selected Cannon Air Force Base because we wanted to obtain the perspectives of RPA pilots assigned to the Air Force Special Operations Command.

To select specific RPA pilots to participate in our focus groups, we obtained documentation that included lists of the RPA pilots stationed at each base as well as the amount of time each had served flying RPAs, and their ranks. To obtain a variety of perspectives, we randomly selected pilots with various amounts of experience flying RPAs and we included pilots of various ranks in our groups. These groups typically consisted of six to nine participants.

To conduct the focus groups, a GAO moderator followed a protocol that included prompts, instructions to the participants, and a set of three questions, each with several follow-up questions. We pretested this protocol at Beale Air Force base and used it at the remaining two bases. We used the same set of questions from this protocol for each of the 10 focus groups we conducted. These questions are reprinted below. During each focus group, the GAO moderator asked questions related to the topics of our review to participants who, in turn, provided their perspectives on the topics. During the focus-group meetings, three GAO team members took separate sets of detailed notes to document the participants’ comments. See table 2 for the complete list of questions and follow-up questions that we asked during our focus groups.

Following our focus-group meetings, we consolidated our separate sets of detailed notes for each focus group to create a compiled final record of the participant comments from each focus group. To do this, a GAO analyst reviewed the set of detailed notes and compiled them in a final record for each focus group. A key rule of this compilation was that if one analyst recorded a comment, but another did not, we included the material in the final record. To ensure that our compiled final record of each focus group was accurate, a second analyst then reviewed at least 25 percent of each of the final records. In instances where an analyst identified some discrepancies between the detailed notes and the final record, the reviewing analyst corrected the discrepancy and reviewed a higher percentage of the notes for that focus group.

Next, we used content analysis to analyze the final records of each focus group to identify themes that participants expressed across all or most of the groups. To do this, three GAO analysts first met to discuss and agree on a preliminary set of themes. We then analyzed an initial set of the records and counted instances that we observed these initial themes. We then reconvened as a group to discuss and agree on additional themes to add to our analysis and to consolidate and delete others. We then analyzed our records and made coding decisions. Following the initial analysis by one analyst, a second analyst independently reviewed all of the coding decisions that the first analyst made for each of the records. Where there were discrepancies, the analysts reviewed one another’s coding and rationale for their coding decisions and reached a consensus on which codes should be used. See figure 8 for the complete results of our analysis.

When describing the results of our analysis of our focus groups in this report, we use the term “some,” as in “pilots in some focus groups,” to report topics that were discussed by RPA pilots in two to four of our focus groups. The information we present from our focus groups accurately captures the opinions provided by the RPA pilots who attended the 10 focus groups at the three Air Force Bases we visited. However these opinions cannot be generalized to all of the RPA pilots at the three Air Force Bases we visited or to all RPA pilots in the Air Force. The results of our analyses of the opinions of RPA pilots we obtained during our focus groups are not generalizable because the Air Force Bases we selected are not necessarily representative of all of the Air Force Bases that contain RPA squadrons and the RPA pilots included in our focus groups are not necessarily representative of all of the RPA pilots in the Air Force.

Appendix III: Comments from the Department of the Air Force

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Lori Atkinson (Assistant Director), Steve Boyles, Ron La Due Lake, Kelly Liptan, James P. Klein, Steven R. Putansu, Michael Willems, Erik Wilkins-McKee, and Amie Steele made key contributions to this report.