Air Force: Actions Needed to Strengthen Management of Unmanned Aerial System Pilots
Published: 2014-04-10T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since 2008, the Air Force has more than tripled the number of its active-duty pilots flying RPAs, which is the term the Air Force uses to refer to unmanned aerial systems such as the MQ-1 Predator. Due to increases in demand, RPA pilots have had a significant increase in workload since 2007. GAO was asked to evaluate the Air Force's approach to managing its RPA pilots as well as their quality of life and promotion rates. For this review, GAO evaluated the extent to which the Air Force (1) has used a strategic human-capital approach to manage RPA pilots; (2) has addressed concerns, if any, about the working conditions of RPA pilots that may affect their quality of life; and (3) analyzes the promotion rates of RPA pilots.
GAO analyzed personnel planning documents, Air Force studies, and officer promotion data. GAO also interviewed unit commanders at selected Air Force bases and Headquarters Air Force officials and conducted focus groups with RPA pilots. While the results of these focus groups are not generalizable, they provide valuable insights.
section_title: What GAO Found
paragraphs: The Air Force has managed its remotely piloted aircraft (RPA) pilots using some strategic human-capital approaches, such as planning for the different levels of experience that it needs in these pilots. However, it continues to face challenges. High-performing organizations manage human capital to identify the right number of personnel and to target the right sources to fill personnel needs. In 2008, the Air Force determined the optimum number of RPA pilots—the crew ratio—for some units, but it did not account for all tasks these units complete. Air Force officials stated that, as a result, the crew ratio is too low, but the Air Force has not updated it. Air Force guidance states that low crew ratios diminish combat capability and cause flight safety to suffer, but the Air Force has operated below its optimum crew ratio and it has not established a minimum crew ratio. Further, high work demands on RPA pilots limit the time they have available for training and development and negatively affects their work-life balance. In addition, the Air Force faces challenges recruiting officers into the RPA pilot career and may face challenges retaining them in the future. High-performing organizations tailor their recruiting and retention strategies to meet their specific mission needs, but the Air Force has not tailored its approach to recruiting and retaining RPA pilots nor considered the viability of using alternative personnel such as enlisted personnel or civilians. Without developing an approach to recruiting and retaining RPA pilots and evaluating the viability of using alternative personnel populations for the RPA pilot career, the Air Force may continue to face challenges, further exacerbating existing shortfalls of RPA pilots. Moreover, the Air Force has not used direct feedback from RPA pilots via existing mechanisms, or otherwise, to develop its approach to managing challenges related to recruiting, retention, training, and development of RPA pilots.
The Air Force has taken some actions to address potentially difficult working conditions RPA pilots face, but it has not fully analyzed the challenge pilots face to balance their warfighting roles with their personal lives. RPA pilots operate RPAs from bases in the United States and live at home; thus they experience combat alongside their personal lives—known as being deployed-on-station—which RPA pilots stated negatively affects their morale. While the Department of Defense has committed to maintaining high morale for servicemembers, the Air Force has not fully analyzed the effects on morale related to being deployed-on-station, and thus it does not know whether it needs to take actions in response.
The Air Force monitors RPA pilot promotion rates, but has not analyzed factors that may relate to their low promotion rates. Statistical principles call for researchers to account for potential key factors in analysis because when they omit key factors, the relationships between other factors may not be accurately estimated. The Air Force analyzed promotions across a group of officers, including RPA pilots, and found factors that related to promotions in general. However, the Air Force has not analyzed the factors related to RPA pilots' promotions specifically and, as a result, it does not have the information to determine what factors may affect their promotions. Consequently, the Air Force may not be targeting actions it is taking to raise RPA pilot promotion rates at the appropriate factors, and information it has reported to Congress may not be accurate.
section_title: What GAO Recommends
paragraphs: GAO recommends that the Air Force update optimum crew ratios; establish a minimum crew ratio; develop a recruiting and retention strategy; evaluate using alternative personnel populations to be pilots; use feedback from RPA pilots; analyze the effects of being deployed-on-station; and analyze the effect that being an RPA pilot has on promotions. The Air Force concurred with four recommendations and partially concurred with the remaining three recommendations.

section_title: Letter
paragraphs: Remotely piloted aircraft (RPA) are one of the most in-demand  capabilities the Air Force provides to battlefield commanders. Beyond  replacing human beings in aircraft that perform dangerous roles, RPAs  are highly valuable because they possess characteristics that many  manned aircraft do not. For example, they can fly long-duration missions,  thereby providing a sustained presence over the battlefield. In response  to the increased demand, the Air Force has significantly increased the  number of RPAs it uses for intelligence, surveillance, and reconnaissance  and precision strike capabilities, according to Air Force documentation.  Consequently, the Air Force has increased the number of its pilots flying  RPAs from approximately 400 in 2008 to about 1,350 in 2013. Due to the  increased demand for their capabilities, these pilots have served at a high  pace of operations since 2007. Most of these pilots are located on Air  Force bases within the United States and fly the RPAs overseas in  operational environments. The Air Force uses the term RPA to refer to  large unmanned aircraft systems, such as the MQ-1 Predator. The  Department of Defense (DOD) defines an unmanned aerial system as a  system whose components include the necessary equipment, networks,  and personnel to control an unmanned aircraft—that is, an aircraft that  does not carry a human operator and is capable of flight under remote  control or autonomous programming.
Our prior work has found that DOD has faced challenges in the  development and acquisition of unmanned aircraft systems and in the  integration of these systems into combat operations. Regarding  personnel, we have found that the Air Force and the Army identified  limitations in their approaches to provide personnel to meet unmanned  aircraft systems force levels, and they had not fully developed plans to  supply needed personnel. More recently, the National Defense  Authorization Act for Fiscal Year 2013 required the Air Force to report on  the education, training, and promotion rates of RPA pilots.
You requested that we evaluate the Air Force’s approach to managing its  RPA pilot workforce as well as the pilots’ quality of life and their promotion  rates. For this review we evaluated the extent to which the Air Force (1)  has used a strategic human-capital approach to manage RPA pilots; (2)  has addressed concerns, if any, about the working conditions of RPA  pilots that may affect their quality of life; and (3) has analyzed the  promotion rates of RPA pilots.
To evaluate the extent to which the Air Force uses a strategic human- capital approach to manage RPA pilots, we applied criteria from our  model of strategic human-capital management that we previously  reported. The model includes leading practices used by high-performing  organizations and is intended to help federal organizations use their  human capital effectively and integrate human-capital considerations into  daily decision making and planning for the program results they wish to  accomplish. We adapted these criteria by analyzing the practices in the  model to determine which were relevant to RPA pilots and a military  career in general. We reviewed our adaptation with Air Force officials who  agreed that our criteria were relevant to the Air Force’s management of  RPA pilots. To identify the extent to which the Air Force used these  practices, we interviewed officials from offices including the Deputy Chief  of Staff of the Air Force for Manpower, Personnel, and Services and the  Deputy Chief of Staff of the Air Force for Operations, Plans, and  Requirements, as well as RPA pilots and their commanders. In addition,  we analyzed DOD and Air Force guidance and data on personnel levels,  incentive pays, and attrition rates for RPA pilots. Furthermore, we  discussed instances of Air Force personnel-management actions that we  found to be inconsistent with the leading practices we reported on  previously with the appropriate Air Force officials.
To evaluate the extent to which the Air Force has addressed concerns, if  any, about the working conditions of RPA pilots that may affect their  quality of life, we identified and analyzed criteria included in DOD’s 2009  and 2004 Quadrennial Quality of Life Reviews. These reviews include  statements that express DOD’s commitment to provide servicemembers  with the best quality of life possible. DOD has broadly defined quality of  life to include such factors as morale, health and wellness, and work-life  balance. To understand the working conditions of RPA pilots that may  affect their quality of life, we analyzed Air Force studies that evaluated the  stress and mental-health conditions of RPA personnel, which included  pilots. We also interviewed the researchers who conducted these studies  to clarify our understanding of their methods, findings, and  recommendations to alleviate the stress of RPA personnel. We analyzed  the recommendations included in the Air Force studies as well as  documentation provided by officials from various Air Force offices  describing actions the Air Force has taken in response to these  recommendations. In addition, we interviewed officials from the Air Force  Medical Support Agency, Headquarters Air Force officials, as well as RPA  pilots, their commanders, and mental-health professionals during site  visits to Beale, Cannon, and Creech Air Force Bases. In these interviews  we obtained perspectives on the working conditions of RPA pilots that  may affect their quality of life and actions taken in response.
To evaluate the extent to which the Air Force analyzes the promotion  rates of RPA pilots, we applied criteria from our model of strategic  human-capital management regarding using complete data in human- capital decisions. We analyzed data from the Air Force Personnel Center  (AFPC) for active-duty officers promoted to the ranks of major, lieutenant  colonel, and colonel. We analyzed data from 2006 to the most-recently  available data, which for promotion to major and colonel was 2012 and for  promotion to lieutenant colonel was 2013. In addition, we interviewed  officials from AFPC, reviewed documentation they provided, and found  the data to be reliable for our purposes. Furthermore, we analyzed  documentation that officials from Headquarters Air Force and AFPC  provided, including Air Force policy that governs the officer promotion  process, AFPC’s analysis of officer promotion rates, and the Air Force’s  August 2013 report to Congress on the promotion rates of RPA pilots.  Moreover, we evaluated documentation of steps the Air Force has taken  to raise the promotion rates of RPA pilots including instructions the  Secretary of the Air Force provided to promotion board members and  briefings that Headquarters Air Force and AFPC prepared for the  Secretary of the Air Force.
For all our objectives, we also conducted 10 focus groups during site  visits to Beale, Cannon, and Creech Air Force Bases. Each group  generally consisted of six to nine active-duty RPA pilots at ranks ranging  from second lieutenant to lieutenant colonel. The information that we  obtained during these focus groups accurately captures the opinions  provided by the RPA pilots who attended the focus groups at the three Air  Force Bases we visited. However, these opinions cannot be generalized  to all RPA pilots at these bases or to all RPA pilots in the Air Force. We  discuss our scope and methodology in more detail in appendix I and our  approach to conducting our focus groups in appendix II.
We conducted this performance audit from February 2013 to April 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: Growth in Use of RPAs and Expanded Missions
paragraphs: The Air Force has rapidly expanded its use of RPAs in the last decade to  support combat operations in Iraq and Afghanistan. The Air Force flies  three types of RPAs—the MQ-1 (Predator), the MQ-9 (Reaper) and the  larger RQ-4 (Global Hawk). Beyond the traditional intelligence,  surveillance, and reconnaissance capability to analyze evolving battlefield  conditions, the MQ-1 and the MQ-9 have been outfitted with missiles to  strike targets, with equipment to designate targets for manned aircraft by  laser, and with sensors to locate the positions of improvised explosive  devices and moving insurgents, among other missions.
section_title: The Military Services’ Various Approaches to Assigning RPA Personnel
paragraphs: All the military services operate RPAs, and each uses different  approaches to assign personnel to pilot them and operate their sensors.  For example, the Air Force (the focus of this review) assigns officers to fly  RPAs and enlisted personnel to operate the RPAs’ sensors, which  provide intelligence, surveillance, and reconnaissance capabilities. In  addition, the Air Force relied solely on manned-aircraft pilots to fly RPAs  until 2010, when it established an RPA pilot career field for officers who  specialize in flying RPAs and are not qualified to fly manned aircraft.  Similarly, the Navy assigns officers to pilot RPAs, and enlisted personnel  to operate RPA sensors. However, the Navy has not established a  separate career field for pilots who specialize in flying RPAs and instead  assigns pilots of manned aircraft to operate them. By contrast, the Army  and Marine Corps have opted to assign enlisted personnel to fly RPAs  and operate their sensors. Further, in both the Army and Marine Corps,  there is no distinction between the pilot and sensor operator.
section_title: RPA Pilots’ Current and Future Basing
paragraphs: Air Force RPA pilots carry out their missions and pilot RPAs from eight  active-duty bases in the continental United States including Creech,  Cannon, and Beale Air Force Bases and from Air National Guard bases in  six states including North Dakota, New York, and Ohio. In addition, RPA  pilots are trained at some of the bases where RPAs are operated, such  as at Beale Air Force Base, as well as at other bases where RPAs are not  operated, such as at Holloman Air Force Base. The Air Force plans to  add an Air Force Reserve unit at Hurlburt Field as well as Air National  Guard RPA bases in Arkansas, Iowa, Michigan, New York, and  Pennsylvania (see fig. 1).
section_title: Training and Training Costs
paragraphs: The initial training that the Air Force provides to its RPA pilots is designed  specifically for flying RPAs and consists of two major components that  take about 10 months to complete. The first major component is  Undergraduate RPA Training and it consists of a basic flying skills course  in which RPA pilots learn to fly a small manned aircraft in Pueblo,  Colorado; instrument training in a manned-aircraft flight simulator at  Randolph Air Force Base in Texas, and an RPA fundamentals course that  is also at Randolph. In the second major component of their initial  training, RPA pilots get their first opportunity to fly an RPA at a Formal  Training Unit, which for most active-duty pilots takes place at Holloman  Air Force Base in New Mexico. During this training, RPA pilots learn basic  RPA operations in all mission areas including intelligence, surveillance,  and reconnaissance as well as close air support. Following their time in  Formal Training Units, RPA pilots finish their training by attending a 2- week joint weapons course in which they learn how to operate with the  Army, Navy, and Marine Corps in a joint operational environment.
The Air Force spends considerably less to train RPA pilots than it does to  train manned-aircraft pilots. Specifically, Air Education and Training  Command officials estimate that the Air Force spends about $65,000 to  train each RPA pilot to complete Undergraduate RPA Training.  Conversely, these officials estimate that the Air Force spends an average  of $557,000 for each manned-aircraft pilot to complete the  corresponding portion of manned-aircraft pilot training, which is called  Undergraduate Pilot Training.
section_title: Remote-Split Operations
paragraphs: The Air Force currently flies the bulk of its RPAs using a concept known  as remote-split operations. With remote-split operations, a small number  of RPA pilots deploy to operational theaters located overseas to launch  and recover RPAs from various locations around the world while other  RPA pilots remotely control the RPA for its mission from Air Force bases  in the United States (see fig. 2). According to Air Force officials, remote- split operations help the Air Force reduce the personnel and equipment it  deploys overseas because the units that launch and recover RPAs are  staffed with a relatively small number of pilots, sensor operators, support  personnel, and equipment. In addition, remote-split operations provide the  Air Force flexibility to change the geographic region of the world where an  RPA pilot conducts a mission without moving the pilot, support personnel,  or equipment needed to control the RPA. If the Air Force is not able to  use one of its launch and recovery sites for various reasons such as poor  weather, the Air Force can continue its RPA operations by launching  RPAs from a different launch and recovery site.
section_title: Promotion Process
paragraphs: The Defense Officer Personnel Management Act (DOPMA) created a  system for managing the promotions for the officer corps of each of the  military services. DOPMA specifies that the secretaries of the military  departments must establish the maximum number of officers in each  competitive category that may be recommended for promotion by  competitive promotion boards. Career categories, also known as  competitive categories, cluster officers with similar education, training, or  experience, and these officers compete among themselves for promotion  opportunities. Under this system, as currently implemented in the Air  Force, there are several competitive categories including one that  contains the bulk of Air Force officers called the Line of the Air Force,  which includes RPA pilots, as well as pilots of manned aircraft and other  operations-oriented careers.
To consider officers for promotion from among those who are eligible, the  Air Force assigns groups of senior officers to serve as members of a  promotion selection board for each competitive category of officer in the  Air Force. Promotion boards consist of at least five active-duty officers  who are senior in grade to the eligible officers, but no officer on the board  is below the rank of major. In addition, Air Force guidance states that the  Air Force attempts to provide a balanced perspective on promotion  boards, and hence it selects officers who mirror, as much as possible, the  officers they are considering with respect to race, sex, aeronautical rating,  career field, and command. Promotion boards typically convene  annually at AFPC headquarters to review a variety of records for each  eligible officer, including performance and training reports as well as  recommendations from supervisors. Board members assess these  records using a best-qualified approach and use a variety of methods to  score the records and resolve differences among the scoring of the board  members, if necessary. An Air Force officer cannot serve as a member of  two successive promotion boards considering officers of the same  competitive category and rank.
A key feature of DOPMA is its “up-or-out” promotion system. Under this  system, as currently implemented in the Air Force, promotion to the first  two ranks in an officer’s career is not competitive. Specifically, 100  percent of fully qualified Air Force second lieutenants and first lieutenants  are promoted after serving for 2 years in their respective ranks and do not  meet with a competitive promotion board. However, as officers advance  through the ranks in cohorts that are determined by the year they were  commissioned, they compete for promotion against other members of  their cohort at set years or zones of consideration for each rank. For  example, Air Force officers are generally considered for promotion to  major, or the grade of O-4, after 10 years.
Under the DOPMA system, a select group of officers can also be  considered for promotion 1 or 2 years early, or “below the zone.”  However, because only a limited number of officers below the zone may  be promoted, officers have their greatest potential for promotion “in the  zone.” If officers in a cohort are not promoted while they are in the zone,  they can compete for promotion in the following one or in some instances  two years later, which is known as competing “above the zone.” However,  if these officers are not selected for promotion above the zone, they could  be involuntarily separated from the Air Force.
section_title: The Air Force Has Made Efforts to Manage RPA Pilots but Faces Challenges to Recruit, Develop, and Retain Pilots and Build Their Morale
paragraphs: The Air Force has taken some steps toward managing RPA pilots using a  strategic human-capital approach but faces several challenges including  accurately identifying personnel requirements, limited training time for  pilots, recruiting and retention difficulties, and incorporating feedback from  RPA pilots into its operations.
subsections: section_title: The Air Force Has Undertaken Some Workforce Planning Efforts but It May Not Have Identified Its Personnel Requirements Accurately
paragraphs: The Air Force’s effort to meet combatant command RPA requirements  has included some elements of strategic human-capital planning, but  increasing demand and past experience indicate the Air Force has not  accurately identified RPA personnel requirements. High-performing  organizations use strategic human-capital planning to help them evaluate  the extent to which their human-capital approaches support the  accomplishment of programmatic goals. Strategic human-capital planning  involves identifying human-capital needs like the necessary “shape,”  which involves ensuring that agencies have the right numbers of staff at  the right levels of experience, as well as the necessary size of the  workforce for accomplishing agency missions while also enabling the  workforce to accomplish career-development tasks, which furthers  agency goals and objectives.
subsections: section_title: Air Force Steps to Plan for the Shape and Size of the RPA Pilot Workforce and React to Increased Combat Air Patrol (CAP) Requirements
paragraphs: The Air Force has taken steps to plan for the shape and size of the RPA  pilot workforce and react to requirements from the Secretary of Defense,  including adding a cadre of experienced officers to mentor officers  recruited into a new career the Air Force established for RPA pilots. In  order to develop a long-term, sustainable career path for pilots flying  RPAs and demonstrate its commitment to RPA pilots, in 2010 the Air  Force established an RPA pilot career field with a separate set of training  requirements. These officers are qualified only to fly RPAs and are not  qualified on Air Force manned aircraft.
In addition, the Air Force recognized that as new officers were recruited  into the RPA pilot career field, they would need a group of more-senior  officers to serve as mentors and leaders. Therefore, in 2011, the Air  Force permanently recategorized around 475 manned-aircraft pilots who  were generally serving at the ranks of major and lieutenant colonel to  serve as permanent RPA pilots, according to Air Force documentation.  Air Force officials stated that these more-senior pilots would help provide  a leadership and experience base for the new RPA pilot career field. The  officials also stated that additional manned-aircraft pilots have been  permanently recategorized as RPA pilots since 2011, and Air Force  documentation shows a total of 545 recategorized manned-aircraft pilots.
Furthermore, the Air Force has taken steps to plan for the size of its RPA  pilot workforce. According to Headquarters Air Force officials, the number  of RPA combat air patrols (CAP), directed by the Secretary of Defense  and based on the mission needs of the combatant commands, is a  primary factor in determining RPA pilot personnel levels. In 2010, the  Secretary of Defense directed the Air Force to fund personnel to reach 65  CAPs by fiscal year 2013 and be prepared to grow beyond that  requirement in future years. To determine the number of RPA pilots, the  Air Force Manpower Agency conducted a personnel requirements study  for MQ-1 Predator squadrons in 2008 and established the number of RPA  crews required to fly one CAP for 24 hours, referred to as the crew ratio.
Based on the study, the Air Force concluded that the crew ratio for MQ-1  Predator squadrons would be 10:1, which calls for 10 RPA pilots to  sustain a Predator for 24 hours. Air Force officials stated that although  the 2008 study did not address the personnel requirements for MQ-9  Reaper squadrons, the Air Force used the study as the basis for  establishing a 10:1 crew ratio for MQ-9 units also because MQ-1 and MQ- 9 units have similar requirements. In addition to this crew ratio, the Air  Force used Air Force Instruction 38-201 to calculate the required  number of additional pilots it needs for support positions such as  commanders, and staff positions at various organizational levels including  headquarters. Using the crew ratio and the Air Force instruction, the Air  Force determined that the total number of RPA pilots required to sustain  the 65 CAPs currently required by the Secretary of Defense is between  1,600 and 1,650 pilots, according to a Headquarters Air Force official.
Furthermore, the Air Force has taken steps to react to increased CAP  requirements. Until 2009, the Air Force relied solely on manned-aircraft  pilots serving assignments as RPA pilots to fill personnel requirements. In  fiscal year 2006, manned-aircraft pilots were sustaining 12 CAPs, and the  2006 Quadrennial Defense Review stated that the Predator system alone  would grow to 21 CAPs by 2010. However, according to Headquarters Air  Force officials, by 2007 the demand from the combatant commands had  already exceeded that benchmark. Air Force leadership committed the  service to meeting the increased requirements, and the Air Force took  actions to provide sufficient personnel. These actions included  lengthening the assignments of manned-aircraft pilots in RPA squadrons  and then extending those assignments indefinitely, mobilizing pilots from  the Air National Guard and Air Force Reserve, delaying the establishment  of the RPA weapons school after designating RPA as a formal weapon  system, and extending the length of deployments to augment staffing  levels of RPA squadrons. In 2009, the Air Force also began assigning  manned-aircraft training graduates to RPA assignments as their first  assignment after completing Undergraduate Pilot Training. In 2010, the  Air Force established the RPA pilot career field. Figure 3 summarizes the  steps that the Air Force took to react to increased CAP requirements  since 2007.
Using these steps, the Air Force has made progress towards meeting the  CAP requirements, but at personnel levels that were below requirements.  In addition, the Air Force reduced the capacity of its RPA training unit  because instructors were pulled to fly in RPA units. In fiscal year 2012,  the Air Force began a reconstitution period intended to staff the training  units, restart the weapons school, and increase the overall number of  RPA pilots to increase the crew ratios of RPA units. As of December  2013, there were 1,366 RPA pilots, or around 85 percent of the total of  1,600 pilots determined by the Air Force as necessary to sustain RPA  operations and training for 65 CAPs. In addition, the Air Force anticipates  increasing the number of RPA pilot staff positions across the Air Force  from 111 as of December 2013 to 300 by fiscal year 2023 to serve at  various Air Force commands, including at Headquarters Air Force and Air  Combat Command.
section_title: Air Force Has Not Accurately Identified Optimum Personnel Requirements and Has Not Established a Minimum Personnel Requirement
paragraphs: The Air Force has not accurately identified optimum personnel  requirements, or crew ratio, for the number of RPA pilots it requires. We  have reported that high-performing organizations use complete and  current data to inform their strategic human-capital planning and remain  open to reevaluating workforce planning efforts. In the 2008 study that  the Air Force Manpower Agency conducted to determine the appropriate  crew ratios for MQ-1 Predator squadrons, the Air Force did not account  for all of the flying and administrative tasks that are required in these  squadrons. While the study accounted for some important tasks that RPA  pilots perform in MQ-1 squadrons such as performing operational  missions, it did not account for other important tasks such as those  required to launch and recover RPAs. In addition, the study did not  account for some important administrative tasks such as conducting  flight-safety evaluations and providing a commander’s support staff. The  study acknowledged that due to its reporting time frames, it did not  capture the personnel requirements of a variety of tasks.
Headquarters Air Force personnel acknowledged the study’s limitations  and said that because the study omitted critical and important tasks from  its analysis, the resulting crew ratio that it recommended probably did not  provide enough pilots to perform the work in an MQ-1 squadron. These  officials stated that, because of the study’s omissions, the 10:1 crew ratio  for MQ-1 squadrons established in an Air Force instruction that was  based on this study should probably be increased. Similarly, some RPA  unit commanders and RPA pilots in some of our focus groups also said  that the crew ratio is too low. However, to-date the Air Force has not  updated the crew ratio for RPA squadrons. Headquarters Air Force  officials stated that updating the crew ratio has not been a top priority. At  the same time, these officials noted that more recently they have  discussed the need to update the crew ratio and expressed optimism that  it would become a priority in the future, though no concrete plans exist to  initiate an update to the requirement.
Furthermore, an Air Force instruction states that a crew ratio establishes  the number of personnel required to support a unit mission and that if a  ratio is too low, combat capability is diminished and flight safety suffers.  Such risks can arise when crew-ratio requirements are set too low, as  well as when units operate at crew ratios that are too far below optimum  crew ratios. However, Air Force documentation shows that crew ratios in  RPA units have fluctuated between 7:1 and 8.5:1, and at times have  dropped to 6:1, according to Air Force officials. This indicates that the  RPA pilot workload is performed by fewer pilots working more hours to  accomplish the mission than if the Air Force ensured that its RPA units  operated at the required crew ratios. The Air Force has operated at these  levels to provide a higher number of CAPs.
According to Headquarters Air Force officials, in the past the Air Force  has attempted to deny requests made by combatant commanders for Air  Force RPA capabilities because they push crew ratios too low. These  officials stated that when the Air Force denies a request it provides  justification, which include concerns about crew ratios, to the Joint Staff,  which is responsible for resolving differences between combatant  commanders’ requests for capabilities and the services that provide them.  However, Air Force officials stated that the Joint Staff has overridden  some of the Air Force denials in order to accomplish missions, despite the  possibility that crew ratios would decrease. Without establishing a  minimum crew ratio for RPA units, the Air Force does not have the  information it needs to determine when those units are operating at crew  ratio levels that expose the Air Force to unacceptable levels of risk to  accomplishing its mission and ensuring safety.
section_title: A High Pace of Operations and Work Demands Limit Time for Training and Development
paragraphs: As a result of inaccurate crew ratios for Air Force RPA squadrons and a  lack of a minimum crew ratio, the RPA pilot workforce has sustained a  high pace of operations, which limits its time for training and  development. The Air Force Unmanned Aircraft Systems Flight Plan  2009-2047 states that it is imperative to provide the necessary training  and opportunities for advancement that will create a cadre of future Air  Force leaders. However, unit commanders in each of the three locations  we visited and some RPA pilots stated that the high pace of operations  and demand for RPA capabilities limited their units’ time to train for the  various mission sets that RPA units are required to perform. One unit  commander stated that battlefield commanders that his unit supports  have pointed out that his RPA pilots need training, and pilots in some  focus groups noted that limited training opportunities prevent RPA units  from excelling at their missions and becoming experts in their field. In  addition, pilots in all 10 focus groups indicated that they are limited in their  ability to pursue developmental opportunities.
Furthermore, DOD has noted that the prevalence and use of unmanned  systems, including RPAs, will continue to grow at a dramatic pace. As  discussed above, the Secretary of Defense has stated specifically that the  requirement for 65 CAPs represents a temporary plateau in progress  toward an increased enduring requirement. Also, as the national security  environment changes, RPA pilots will be expected to conduct a broader  range of missions across different conditions and environments, including  antiaccess and area-denial environments where the freedom to operate  RPAs is contested. By not creating an environment where RPA pilots  can receive the training and development opportunities they need to  perform their functions effectively, the Air Force may be hindering its  ability to perform its mission even if it is able to operate at the optimum  crew ratio that is set in the Air Force instruction.
section_title: The Air Force Faces Challenges Recruiting RPA Pilots and May Face Challenges Retaining Them in the Future
paragraphs: The Air Force has used a dual strategy to meet its increasing need for  RPA pilots: using manned-aircraft pilots and recruiting RPA pilots, the  career field established in 2010 for officers trained to only fly RPAs.  However, the Air Force has faced challenges in recruiting RPA pilots  since it began this career field. High-performing organizations tailor their  recruitment and retention strategies to meet their specific mission  needs. The Air Force intends to build a cadre of dedicated RPA pilots,  and projects that RPA pilots will make up 90 percent of the RPA pilot  workforce by fiscal year 2022. However, the Air Force has not been able  to achieve its recruiting goals for RPA pilots in fiscal years 2012 and  2013. In fiscal year 2013, the Air Force recruited 110 new RPA pilots,  missing its goal of 179 pilots by around 39 percent. Consequently, while  the Air Force has made progress in increasing the total number of RPA  pilots and staffed its RPA units at about 85 percent of current  requirements as of December 2013, around 42 percent of those pilots are  manned-aircraft pilots and manned-aircraft pilot training graduates. Both  of these groups are temporary RPA pilots who serve only one assignment  in an RPA squadron. While the length of these assignments can be  extended, these pilots will likely not stay in the RPA squadrons  permanently (see fig. 4).
Headquarters Air Force officials believe the Air Force has missed its  recruiting goals in 2012 and 2013 for RPA pilots because potential  recruits have a limited understanding of the RPA mission and there is a  lack of recruiting officials with RPA experience to advise potential recruits.  The Air Force may face challenges recruiting officers to serve as RPA  pilots because of a negative perception that some in the Air Force  associate with flying RPAs. Headquarters Air Force officials, RPA pilots in  some of our focus groups, and one unit commander stated that some in  the Air Force view flying RPAs negatively, resulting in a stigma. According  to these officials one reason some view flying an RPA negatively is  because flying an RPA does not require pilots to operate an aircraft while  on board an aircraft in-flight. In addition, officials stated that overcoming  this stigma may be difficult because publicizing the work that RPA pilots  do is often not feasible due to the classified nature of RPA missions.  Nonetheless, Headquarters Air Force officials stated that the Air Force  projects it will meet its recruiting goals for the RPA pilot career field for  fiscal year 2014 on the basis of commitments made by cadets  participating in the Air Force Reserve Officer Training Corps.
We have reported that high-performing organizations make use of  targeted investments such as recruiting bonuses as part of their  strategies to recruit high-quality personnel with the critical skills. However,  Headquarters Air Force officials reported that the Air Force is not  currently exercising its option to offer a recruiting bonus as an incentive to  volunteer for the RPA pilot career field. Officials from the Headquarters  Air Force and the Office of the Secretary of Defense stated that such pay  incentives are rarely used to recruit officers in the Air Force.  Headquarters Air Force officials also stated that due to the current  constrained budget environment in which DOD and the federal  government are operating, the Air Force would first prefer to exhaust the  use of all nonmonetary options for improving recruiting before offering  bonuses. As a result, the Air Force may have to continue to rely on  manned-aircraft pilots to meet RPA pilot personnel needs. This approach  may not be cost-effective because the Air Force spends an average of  $557,000 per pilot on traditional Undergraduate Pilot Training, compared  to an average of $65,000 for Undergraduate RPA Training, according to  Air Education and Training Command officials. Without a more-tailored  approach to recruiting RPA pilots that increases the appeal of the new  career to potential recruits, the Air Force risks perpetuating personnel  shortages and may need to continue relying on manned-aircraft pilots to  fill its personnel requirements.
Moreover, the Air Force uses officers as RPA pilots, but it has not  evaluated whether using alternative personnel populations such as  enlisted or civilian personnel as RPA pilots is a viable option. A report by  the House Permanent Select Committee on Intelligence urged the Air  Force to study the other military services’ experiences with using enlisted  personnel as RPA operators and evaluate whether this approach would  degrade mission performance. Headquarters Air Force officials stated  that prior to 2010, they decided to assign officers to serve as RPA pilots  because they thought officers were more appropriate since RPAs fly in  complex airspace, and, in some cases, fire missiles at adversaries.  Headquarters Air Force officials also stated that they have, at times,  considered the use of enlisted or civilian personnel but have not initiated  formal efforts to evaluate whether using such populations would  negatively affect the ability of the Air Force to carry out its missions.
However, without an evaluation of the viability of using other sources of  personnel, the Air Force may lack valuable information on whether  additional options exist for meeting personnel requirements.
With regard to pilot retention, the Air Force has taken some steps but  does not have a retention strategy for RPA pilots, though indications  suggest that it could face challenges retaining them in the future.  Specifically, according to Headquarters Air Force officials, the Air Force  has offered assignment incentive payments to RPA pilots since the career  field was established in 2010. In addition, the officials stated that manned- aircraft pilots serving assignments in RPA squadrons receive skill-based  aviator career incentive pay and can receive aviator retention pay by  extending their service commitment in the Air Force. Despite these  incentive payments, pilots in 7 of 10 focus groups we conducted indicated  that retention of RPA pilots is or will be a challenge. In addition, pilots in  some focus groups stated that they are considering their options for  leaving active-duty service in the Air Force to go to the Air National  Guard, or Air Force Reserve, or the private sector. Unit commanders in  one location we visited, pilots in some of our focus groups, and other Air  Force officials stated that they were concerned about the future retention  rates of RPA pilots.
Headquarters Air Force officials stated that the Air Force’s strategy for  meeting personnel requirements has focused on recruiting and that they  have not observed indications of a concern with the retention of RPA  pilots. However, the Air Force has not evaluated the potential effect of the  difficult working conditions, such as long working hours and frequently  rotating shifts that we discuss in more detail later in this report, that RPA  pilots face and how those conditions may affect the Air Force’s ability to  retain RPA pilots, despite the situation that many of these pilots will begin  to reach the end of their service commitments in fiscal year 2017. In a  2011 memorandum to the Air Force, the Secretary of Defense directed  the Air Force to provide sufficient incentives to retain high-quality RPA  personnel. Although the Air Force has made retention payments available  to RPA pilots, these efforts may not be enough or appropriate to  overcome the challenges the Air Force may face to retain RPA pilots.
section_title: The Air Force Has Mechanisms to Collect Feedback from RPA Pilots but Has Not Used That Feedback to Manage Its Human-Capital Strategy for RPA Pilots
paragraphs: While the Air Force has mechanisms in place to collect feedback from  RPA pilots, it has not used this feedback to develop its strategic human- capital approach to managing RPA pilots, such as by incorporating their  feedback into tailoring a recruiting and retention strategy or by taking  actions related to training and development. High-performing  organizations involve their employees in their strategic human-capital  approaches and planning in order to improve motivation and morale by  seeking employee feedback on a periodic basis, and using that input to  adjust their human-capital approaches.
The Air Force has mechanisms in place that it has used to collect  feedback from RPA pilots. For example, the Air Force solicits feedback  from RPA units as well as all other Air Force units during an annual Unit  Climate Assessment that gauges discrimination, harassment, and morale  issues at the unit level. While this effort is not specific to the RPA units, it  does include assessments of RPA units. Unit commanders can use the  results of their Unit Climate Assessments to address challenges at the  local unit level. However, Headquarters Air Force officials responsible for  managing RPA pilots have not obtained information from these  assessments to identify whether they include potentially valuable  information about any concerns related to establishing the RPA pilot  career field. Headquarters Air Force officials stated that the Air Force  created this career field more quickly and under greater operational  demand than any career field in recent Air Force history. However, these  officials also stated that using feedback from the Unit Climate  Assessments to address issues at a headquarters level that would affect  RPA pilots could undermine unit commanders. They also noted that  officials at the headquarters level might lack the proper context for  understanding the assessment results.
The Air Force also collected feedback from RPA pilots in studies the Air  Force School of Aerospace Medicine published in 2011 and 2013 to  assess the level of and reasons for stress in personnel assigned to RPA  units, which included surveys and interviews of RPA pilots. In response to  these studies, the Air Force took actions designed to address stress in  personnel assigned to RPA units. For instance, the studies recommended  that the Air Force assign an operational psychologist to each RPA unit,  and, in response, local flight surgeons, clinical providers, and aerospace  physiologists have created teams to help address stress concerns at the  base level. While researchers from the Air Force’s medical research  community conducted these studies, they included findings related to  personnel shortages that are germane to the Air Force personnel and  operations communities. However, Headquarters Air Force officials from  the personnel and operations communities stated that, prior to our review,  they were unaware of the studies and their findings.
RPA pilots in our focus groups also noted information that suggests that  incorporating pilot feedback from existing mechanisms could help  improve communication and address issues pilots are facing. For  example, pilots in some of our focus groups stated that they did not know  what the career path for an RPA pilot is or what steps they should take to  advance. Further, in some of our focus groups, manned-aircraft pilots  who are serving assignments as RPA pilots expressed uncertainty  regarding whether they will be able to return to their manned platforms  and what effect, if any, their RPA assignment will have on their careers.  Pilots in some focus groups also reported that senior leadership had not  communicated to them about this uncertainty, and one pilot specifically  noted that the lack of communication negatively affects morale. Without  using existing mechanisms to obtain feedback from RPA pilots directly,  Headquarters Air Force may be missing an opportunity to obtain  information that can help it address recruiting, retention, training, and  development challenges related to RPA pilots.
section_title: Air Force Has Taken Some Actions to Address the Potentially Challenging Working Conditions RPA Pilots Face but Has Not Fully Analyzed Being Deployed-on-Station
subsections: section_title: Air Force RPA Pilots Find Their Work Rewarding but Report Multiple Challenging Working Conditions, Some of Which the Air Force Has Taken Actions to Address
paragraphs: RPA pilots find their mission rewarding, but they reported that they face  multiple, challenging working conditions. RPA pilots in 8 of the 10 focus  groups we conducted reported that they found it rewarding to be able to  contribute to combat operations every day through the RPA mission. For  instance, one pilot stated that the mission is the reason that he had  decided to become a permanent RPA pilot and that it was rewarding to  contribute to overseas contingency operations, which he would not be  able to do in any other job. Similarly, the Air Force School of Aerospace  Medicine published studies in 2011 and 2013 that evaluated the  psychological condition of RPA personnel and found that RPA pilots held  positive perceptions of the effect and contributions of their work.  However, RPA pilots also stated that they face multiple challenging  working conditions including: long hours, working shifts that frequently  rotate, and remaining in assignments beyond typical lengths. RPA pilots  in all of our focus groups reported that these challenging conditions  negatively affected their morale and caused them stress. Similarly, the Air  Force School of Aerospace Medicine studies found that RPA personnel  reported sources of stress that were consistent with the challenges we  identified. These challenges include the following:
RPA pilots in 8 of our 10 focus groups stated, and Air Force studies  we reviewed show, that RPA pilots work long hours. RPA pilots in 7 of  our focus groups described factors that contribute to their long hours  including performing administrative duties and attending briefings, in  addition to flying shifts. The Air Force studies also found that working  long hours was one of the top five reasons for stress among  personnel in RPA squadrons. In the studies, over 57 percent of  respondents reported that they worked more than 50 hours per week.  In addition, the studies found that over 40 percent of respondents  reported that performing administrative duties added hours to their  work week and was the third-highest reason for stress among active- duty RPA personnel.
RPA pilots also reported that it was challenging to work on shifts that  rotate. RPA pilots in 7 of the 10 focus groups we conducted stated  that constantly rotating shifts caused sleep problems for them  because they must continuously adjust their sleep schedule to  accommodate new shifts. In addition, pilots noted that continuously  rotating to new shifts disrupted their ability to spend time with their  family and friends. Officials told us that it was ideal for pilots working  evening or night shifts to maintain a consistent sleep pattern on their  off-duty days even though those sleep patterns would require that  pilots sleep while their family and friends were awake. However, some  RPA pilots reported that they typically adjusted their sleep schedules  dramatically on their off-duty days so they could spend time with their  families and that these changes to their sleep schedules resulted in  significant fatigue both at home and when they returned to work.  Similarly, over half of the respondents to the surveys included in the  Air Force studies we reviewed reported that shift work caused a  moderate to large amount of their stress.
RPA pilots in 5 of our focus groups reported that being assigned to  continue flying RPAs for periods extending beyond the typical Air  Force assignment was difficult. In all of the focus groups we  conducted with RPA pilots, those who plan to return to flying manned  aircraft stated that they have been required to stay in their  assignments for periods that are longer than a typical Air Force  assignment. Air Force officials stated that there is no requirement for  officers to move to a new assignment after a specified period.  However, pilots in our focus groups and Air Force headquarters  officials said that officer assignments typically last 3 to 4 years. Air  Force documentation shows that some of these pilots have been in  their RPA assignments for over 6 years. Moreover, the Air Force  studies also found that one of the most common stressors that RPA  personnel cited was the lack of clarity regarding when they would  return to their careers in manned aircraft. Specifically, the 2011 study  states that the Air Force informed RPA pilots who previously flew  manned aircraft that their RPA assignments were temporary and after  3 to 4 years they could return to their manned-aircraft career. The  study goes on to state that due to the increasing demand for RPAs  and the long-standing surge in RPA operations, many pilots have  been unable to return to their manned-aircraft careers and, until  recently, the Air Force kept them in these assignments indefinitely.
The Air Force has taken some actions to address some of the challenging  working conditions that RPA pilots face. The Air Force studies included  over 10 recommendations to address the sources of stress that RPA  personnel reported. For example, the studies recommended that the Air  Force assign an operational psychologist to each RPA unit to help  commanders optimize work-rest schedules and shift cycles, and identify  pilots who are reaching elevated levels of fatigue or stress. In response,  the Air Force has assigned mental-health providers that are dedicated to  RPA squadrons at Beale, Cannon, and Creech Air Force Bases.  However, the studies also recommended that the Air Force increase  staffing in RPA squadrons to reduce the number of hours that RPA  personnel work and to help establish better shift schedules. Air Force  researchers stated that increasing staffing levels, or crew ratios, in RPA  squadrons would be the most-effective means to reduce RPA pilot stress,  but as discussed above, the Air Force has operated its RPA squadrons  below the optimum crew ratios.
section_title: Air Force Has Not Fully Analyzed Challenges That RPA Pilots Face Related to Being Deployed-on- Station
paragraphs: RPA pilots also face challenges related to being deployed-on-station as  they balance their warfighting responsibilities with their personal lives.  Because pilots are able to operate RPAs from Air Force bases in the  United States and are thus able to live at home—what is known as being  deployed-on-station—their dual role juxtaposes stress related to  supporting combat operations with the strains that can occur in their  personal lives. While these pilots face this challenging working condition  that may affect their quality of life, DOD’s Quadrennial Quality of Life  Reviews have emphasized DOD’s continued commitment to provide  servicemembers with the best quality of life possible.
Being deployed-on-station is a new concept in warfighting, and a 2011  report prepared for the Air Force Medical Support Agency describes five  conditions that personnel who are deployed-on-station can experience.  The report notes that these personnel (1) experience a justifiable risk of  being the target of hostile adversary attacks because they are  combatants and their bank accounts, reputations, or physical safety could  be targeted; (2) operate in contact with and sometimes kill adversaries,  although operations they conduct are out of direct risk from combat; (3)  must act with urgency to sometimes kill adversaries and take other time- pressured actions to help ensure combatants they support do not lose  their lives; (4) work on a wartime rhythm that includes 24/7 operations  365 days a year; and (5) are required to conceal information from friends  and family about their work because their missions are often classified. A  Headquarters Air Force official described being deployed-on-station as a  status between deployed-in-theater and not deployed and emphasized  that personnel who are deployed-on-station are not directly engaged in  combat, which is a significant component of being deployed. The official  also acknowledged that being deployed-on-station can be more  challenging than assignments with more-limited connections to the  battlefield.
RPA pilots in each of the 10 focus groups we conducted reported that  being deployed-on-station negatively affected their quality of life, as it was  challenging for them to balance their warfighting responsibilities with their  personal lives for extended periods of time. RPA pilots in some of our  focus groups, as well as commanders of RPA squadrons, noted that they  would prefer to deploy-in-theater for 6 months with a clear end point and  be separated from their family and friends rather than be deployed-on- station for 3 or more years. One commander stated that he preferred  being deployed-in-theater and knowing when his deployment would end.  In contrast, he stated that in an RPA squadron, it was difficult to juggle his  warfighting role with the typical challenges of home life for multiple years.  Likewise, the Air Force studies found that being deployed-on-station was  one of the most commonly cited stressors that RPA personnel reported.
In addition, RPA pilots in 6 of our 10 focus groups reported that they are  expected to do more work than their counterparts who are deployed-in- theater. For example, RPA pilots in some of our focus groups who had  previously deployed-in-theater stated that they are expected to complete  administrative tasks that are not required of them when they are  deployed-in-theater.
Headquarters Air Force officials as well as pilots in some of our focus  groups stated that the Air Force provides support to personnel who are  deployed-in-theater that it does not provide for personnel who are  deployed-on-station. Moreover, the Air Force has surveyed RPA  personnel and other deployed-on-station personnel to study their stress  and mental health, but it has not fully analyzed the effects of being  deployed-on-station. Specifically, it has not fully analyzed whether being  deployed-on-station has negative effects on quality of life that are not  attributable to the stressors that are related to low unit-staffing levels that  we discussed above such as rotating shifts and long assignments. As a  result, the Air Force does not have the information it needs to determine  whether being deployed-on-station has a negative effect on the quality of  life of RPA pilots that is not attributed to the other factors and what steps  might be needed to reduce those effects.
section_title: The Air Force Monitors RPA Pilot Promotion Rates but Has Not Analyzed Factors Related to Those Rates
subsections: section_title: The Air Force Monitors RPA Pilot Promotion Rates and Has Found That They Are Lower Than in Other Career Fields
paragraphs: AFPC monitors the promotion rates of RPA pilots and has found that they  were promoted below the average rate for active-duty line officers on 20  of 24 officer promotion boards since 2006. We reached the same  conclusion based on our review of data for these promotion boards. We  also found that RPA pilots were promoted below the average rate of  manned-aircraft pilots on 21 of 24 boards. Furthermore, we compared  the promotion rates of RPA pilots to those of other career fields and found  that RPA pilots were promoted at the lowest rate of any career field on 9  of the 24 boards and were promoted in the lowest 5 percent of the career  fields that competed on 5 additional boards. Conversely, RPA pilots  were promoted in the top 50 percent of the career fields that competed on  only 3 boards of the 24 boards. More specifically, RPA pilots competing  for promotion to each rank that we analyzed faced challenges. RPA pilots  competing for promotion to major were promoted in the top 50 percent on  just one of the seven promotion boards since 2006. RPA pilots competing  for promotion to lieutenant colonel were promoted at the lowest or next- to-lowest rate compared to the other career fields that competed on 7 of  the 9 boards since 2006. Likewise, RPA pilots competing for promotion to  the rank of colonel had the lowest promotion rate of any career field that  competed on 4 of the 8 colonel boards since 2006. Figures 5, 6, and 7  display the results of our analyses.
section_title: Air Force Has Not Analyzed Factors Related to Lower RPA Pilot Promotion Rates
paragraphs: While AFPC has monitored the promotion rates of RPA pilots, it has not  analyzed the factors related to lower promotion rates for these pilots. It is  a common statistical practice when analyzing how selected factors are  related to a given outcome to account for other key factors that could also  be related to the outcome. Although AFPC analyzed the promotions of  officers in the Line of the Air Force competitive category, which includes  RPA pilots, and identified factors related to promotion outcomes for  officers in this category, it has not incorporated a key factor—the career  field effect of being an RPA pilot—into its analysis.
AFPC analyzed promotion data of officers in the competitive category that  includes RPA pilots called Line of the Air Force and found multiple factors  related to promotion outcomes. Specifically, AFPC analyzed these data  using logistic regression, which is a statistical method that enables AFPC  to analyze the relationships among multiple factors. Using this method,  AFPC identified a number of factors that are positively and negatively  related to promotions. For example, AFPC found that one of the two  factors with the most-substantial positive relationship to promotions was  for an officer to have completed a professional military education program  by attending an Air Force school in-residence, rather than completing the  same professional military education program by correspondence. The  other factor with the most-substantial positive relationship was for an  officer to have completed an advanced academic degree. By contrast,  AFPC found that officers who have unfavorable information, such as  performance-related reprimands, in their personnel files are promoted at  lower rates, in general, than officers who do not.
AFPC did not include the career field effect of being an RPA pilot as a  factor in its analysis. As a result, AFPC does not know whether or how  being an RPA pilot is related to promotions for these pilots. AFPC has  analyzed other careers and found that most careers are not related to  promotion rates. AFPC officials stated that they had not analyzed this  effect because most of the officers currently serving as RPA pilots are  temporary RPA pilots and AFPC does not typically analyze a career field  effect of temporary assignments.
In addition, AFPC assumed that the factors that were substantially related  to promotions for the Line of the Air Force category were also  substantially related to promotions for the RPA pilot subgroup, but did not  confirm that its assumption was warranted. AFPC officials stated that  when they analyzed the records of RPA pilots, they focused on the  factors identified in the analysis of Line of the Air Force officers, including  completing professional military education in-residence and advanced  degrees. They found that RPA pilots generally completed professional  military education in-residence and advanced degrees at lower rates  compared to the average rates for officers who had been promoted since  2006. However, by not including the career field effect of being an RPA  pilot into its analysis, the Air Force cannot determine whether these  factors have the same relationship with RPA pilot promotions as they do  on officer promotions in the broader Line of the Air Force category.
section_title: Air Force Reported Reasons for Low RPA Pilot Promotions Rates to Congress and Took Actions in Response without a Comprehensive Understanding
paragraphs: The Air Force reported reasons for low RPA pilot promotions rates to  Congress and took actions to raise those rates without a comprehensive  understanding of the factors related to the promotion rates of these pilots.  Specifically, the Air Force attributed low RPA pilot promotion rates to  three factors: (1) RPA pilots completed professional military education at  lower rates than average; (2) RPA pilots completed advanced degrees at  lower rates than average; and (3) the process the Air Force used to select  RPA pilots. As discussed above, the AFPC’s approach to identify the first  two factors assumed that their relationships with promotion rates for RPA  pilots as a subgroup would be the same as those with the Line of the Air  Force as a whole, but this assumption was not confirmed through  analysis. Regarding the third factor, Air Force documentation states  “lower quality pilots are generally sent to RPA squadrons.”
Headquarters Air Force officials and two commanders of manned-aircraft  squadrons explained that commanders select pilots from their squadrons  to assign to RPA squadrons and in general most commanders assign  less-skilled pilots and less-competent officers to these squadrons.  Headquarters officials also stated that less-skilled and less-competent  officers generally had fewer of the factors AFPC identified that positively  influence promotions in their records than their peers. Air Force officials  also explained that because the bulk of RPA pilots who have competed  for promotion since 2006 were assigned using this process, they believe  these are the reasons that RPA pilots have been promoted at lower rates  than their peers. However, the Air Force has not incorporated variables  into its analysis to account for RPA pilots or the process to assign them to  determine whether they are related to promotions of RPA pilots.  Consequently, the Air Force report to Congress may not be accurate  because the Air Force does not have comprehensive analysis to  demonstrate that these factors are actually related to RPA pilot  promotions.
Recently, the Air Force has taken actions to raise promotion rates of RPA  pilots. First, to communicate to promotion boards that promoting RPA  pilots was important, the Secretary of the Air Force has issued  instructions since 2008 to each officer promotion board, directing them to  consider the strategic effect made by RPA pilots when evaluating their  records for promotion. In the instructions, the Secretary directs board  members to consider that RPA pilots’ records may not show the same  career progression as their peers because of operational requirements  they have had to meet to satisfy the needs of the Air Force. Second, the  Air Force intervened on behalf of RPA pilots to enhance their  opportunities to achieve one of the two most important factors that AFPC  identified in its analysis of all Line of the Air Force officers by reserving 46  in-residence seats in Air Force professional military education schools in  2012 for RPA pilots who were competing to be promoted to major.  Moreover, the Air Force stated in its August 2013 report to Congress that  its long-term plan to raise promotion rates is to attract “quality” recruits to  the RPA pilot career field and to establish a sustainable pace of  operations that will enable these pilots time to complete in-residence  professional military education and advanced academic degrees.  However, because it has not fully analyzed the career field effects of  being an RPA pilot, it is unclear whether the Air Force is targeting these  corrective actions at the right factors. Consequently, the Air Force’s  actions may have limited effect on improving the promotion rates for RPA  pilots.
section_title: Conclusions
paragraphs: The Air Force has demonstrated a commitment in recent years to the use  of RPAs, believing that the capabilities they provide are worth the  service’s investment in both platforms and personnel. As the RPA pilot  career field evolves, it will be important that Air Force senior leadership  demonstrates a commitment to a human-capital management approach  that addresses a number of outstanding challenges. For instance, without  updating its optimum crew ratio for RPA units, the Air Force may have  RPA pilot shortfalls even after its current requirement is met, which could  exacerbate existing strains on this workforce. In addition, by not  establishing a minimum crew ratio below which RPA units cannot  operate, the Air Force does not know when it is operating at unacceptable  levels of risk to mission and safety. Further, without developing a strategy  tailored to address specific challenges of recruiting and retaining RPA  pilots, current pilot shortfalls may persist even longer than expected.  Finally, without evaluating the viability of using alternative personnel  populations, such as enlisted or civilian personnel, the Air Force may not  meet and sustain required RPA pilot staffing levels.
Moreover, without incorporating feedback from RPA pilots using existing  feedback mechanisms, the Air Force may be missing opportunities to  manage its human-capital strategies effectively for these pilots. Also, RPA  pilots face a number of challenging working conditions that can affect  their quality of life including those associated with being deployed-on- station. However, without analyzing whether being deployed-on-station  has long-term negative effects, the Air Force does not have the  information it needs to determine whether it should take any action in  response. Finally, while the Air Force has taken action to improve the  chances for RPA pilots to be promoted, senior Air Force leaders cannot  be assured that the actions are the appropriate ones because the Air  Force has not analyzed the effect that being an RPA pilot itself may have  on those chances.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Defense direct the Secretary of the  Air Force to take the following seven actions: update crew ratios for RPA units to help ensure that the Air Force  establishes a more-accurate understanding of the required number of  RPA pilots needed in its units, establish a minimum crew ratio in Air Force policy below which RPA  units cannot operate without running unacceptable levels of risk to  accomplishing the mission and ensuring safety, develop a recruiting and retention strategy that is a tailored to the  specific needs and challenges of RPA pilots to help ensure that the  Air Force can meet and retain required staffing levels to meet its  mission, evaluate the viability of using alternative personnel populations  including enlisted or civilian personnel as RPA pilots to identify  whether such populations could help the Air Force meet and sustain  required RPA pilot staffing levels,  incorporate feedback from RPA pilots by using existing mechanisms  or by collecting direct feedback from RPA pilots, analyze the effects of being deployed-on-station to determine whether  there are resulting negative effects on the quality of life of RPA pilots  and take responsive actions as appropriate, and  include the career field effect of being an RPA pilot into AFPC’s  analysis to determine whether and how being an RPA pilot is related  to promotions and determine whether the factors AFPC identified in its  analysis of Line of the Air Force officers are also related to RPA pilot  promotions.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD for review and comment. The  Deputy Director of Force Management Policy, Headquarters Air Force  provided written comments in response to our report. In its written  comments, the Air Force concurred with four of our seven  recommendations and partially concurred with the remaining three  recommendations. The Air Force’s written comments are reprinted in their  entirety in appendix III. The Air Force also provided technical comments  that we have incorporated into this report where applicable.
In concurring with our first three recommendations, the Air Force stated  that it: has an effort underway to update crew ratios for RPA units and  expects to have this effort completed by February 2015; a minimum crew ratio would indicate when the Air Force receives a  request for forces that would pose risks to the mission and safety and  it expects to respond to our recommendation by February 2015; and will develop a recruiting and retention strategy that is tailored to the  specific needs and challenges of RPA pilots and expects to have this  done by October 2015.
In concurring with our fifth recommendation, to incorporate feedback from  RPA pilots by using existing mechanisms or by collecting direct feedback  from RPA pilots, the Air Force stated that if it determines that it is  appropriate to collect such feedback, it will do so using a survey. We  continue to believe that collecting this feedback could be a useful tool for  the Air Force to develop a tailored recruiting and retention strategy and to  inform actions it may take related to training and developing RPA pilots.
The Air Force partially concurred with our fourth recommendation that it  evaluate the viability of using alternative personnel populations as RPA  pilots and determine if such populations could help the Air Force meet  and sustain required RPA pilot staffing levels. The Air Force stated that it  considered assigning enlisted personnel as RPA pilots, but it decided that  the responsibilities of piloting an RPA were commensurate with the rank  of officers instead. At the same time, the Air Force stated that it has  initiated a review of some of its missions and the ranks needed to execute  those missions and that it may consider using enlisted airmen in this  review.
In our report, we acknowledge that the Air Force had previously  considered using enlisted personnel as RPA pilots and that the Air Force  decided instead to use officers. However, it is not clear what steps the Air  Force took in its previous considerations. We think it is a positive step that  the Air Force has initiated a review of Air Force missions and rank  requirements to execute those missions. Considering the significant role  that RPAs play in the Air Force mission, we believe the Air Force should  include RPA pilots in its review to evaluate whether enlisted personnel as  well as civilians may provide a means for the Air Force to address  shortfalls in the staffing levels of RPA pilots.
In addition, the Air Force partially concurred with our sixth  recommendation that it analyze the effects of being deployed-on-station  to determine if there are resulting negative effects on RPA pilots’ quality  of life and take responsive actions as appropriate. In response to our  recommendation, the Air Force stated that it had studied the effects that  being deployed-on-station has on RPA pilots and that many of the  stressors it identified in these studies were related to low unit staffing  levels. In addition, the Air Force asked us to focus our recommendation  on an evaluation of these studies.
We acknowledge in our report that the Air Force evaluated the  psychological condition of RPA personnel who are deployed-on-station in  studies it published in 2011 and 2013. We also acknowledge that the  primary recommendation these studies make is to increase staffing levels  in RPA units to alleviate the stress of RPA personnel. As we discussed in  our report, RPA units have been understaffed and thus increasing staffing  levels may be appropriate. However, our finding is focused on whether  being deployed-on-station has negative effects on quality of life that are  not attributable to the stressors that are related to low unit-staffing levels.  We think that a more complete understanding of the effects of being  deployed-on-station that are not attributable to low staffing levels will help  the Air Force determine if responsive actions are needed that go beyond  increasing staffing levels. Further, the 2011 report prepared for the Air  Force Medical Support Agency that focuses more directly on the concept  of being deployed-on-station is a constructive source of input for the Air  Force to understand any negative effects of being deployed-on-station.  However, it is not clear that an evaluation of this report and the 2011 and  2013 studies will provide the Air Force with a complete understanding of  this new deployment concept’s consequences for its personnel.
Finally, the Air Force partially concurred with our seventh  recommendation that it include the career field effect of being an RPA  pilot into AFPC’s promotion analysis to determine if being an RPA pilot is  related to promotions and determine if other factors that AFPC identified  in its analysis of Line of the Air Force officers are also related to RPA pilot  promotions. The Air Force stated that the RPA career field is a subsection  of the Line of the Air Force and therefore factors related to promotions  identified in analysis of the Line of the Air Force are directly related to  RPA pilot promotions. In our report, we acknowledge that the Air Force  identified factors related to promotion outcomes for officers in the Line of  the Air Force competitive category. However, as we discussed in the  report, not including the career field effect of being an RPA pilot as a  factor in its analysis has several consequences. First, AFPC does not  know whether or how being an RPA pilot is related to promotions for  these pilots. Second, the Air Force cannot determine whether the factors  that it found that are related to promotions for the Line of the Air Force  competitive category have the same relationship with RPA pilot  promotions. Third, the information the Air Force included in a report to  Congress in August 2013 on education, training, and promotion rates of  RPA pilots may not be accurate. Finally, it is unclear whether the Air  Force is targeting actions to increase RPA promotion rates at the right  factors and thus its actions may have limited effect.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, and the Secretary of the Air Force.  In addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3604 or FarrellB@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix IV.
section_title: Appendix I: Scope and Methodology
paragraphs: To understand the context of each of the issues in our review, we  analyzed various Department of Defense (DOD) and Air Force  documents. This documentation included a report to Congress by the  Office of the Under Secretary of Defense for Acquisition, Technology and  Logistics on the future of unmanned aerial systems and a report by the  Air Force Audit Agency on the Air Force’s personnel management of  pilots flying RPAs. We also reviewed reports that we previously issued  that address topics related to our review including a 2010 report on DOD  planning, training, and doctrine for unmanned aircraft systems.
To evaluate the extent to which the Air Force uses a strategic human- capital approach to manage remotely piloted aircraft (RPA) pilots, we  used a model of human-capital management GAO had previously  developed that specifies leading practices that high-performing  organizations exhibit in their strategic human-capital management. The  Model for Strategic Human Capital Management is intended to help  federal organizations use their human capital effectively and integrate  human-capital considerations into daily decision making and planning for  the program results they wish to accomplish. It identifies concepts and  leading practices that are organized into strategic human-capital  management cornerstones including strategic human-capital planning;  acquiring, developing, and retaining talent; and creating results-oriented  cultures.
To adapt the criteria to the context of this review, we reviewed the model  to identify specific practices that organizations can use to make progress  associated with each of the four strategic human-capital management  cornerstones. We then analyzed each practice to determine whether it  was appropriate and relevant to both the RPA pilot workforce and the  military context overall. After identifying the list of practices, we discussed  our adaptation with Air Force officials, who agreed they were appropriate  and relevant and provided points of contact for obtaining information on  each practice.
We interviewed officials from Headquarters Air Force offices including the  Officer of Manpower, Personnel, and Services Policy and the Office of  Operations, Plans, and Requirements Policy to gather their perspectives  and information on practices across all four cornerstones. From these  offices, we obtained and analyzed documentation, including strategic  DOD and Air Force guidance and data on personnel levels, recruiting,  incentive pays, and attrition rates for remotely piloted aircraft (RPA) pilots.  In addition, we interviewed knowledgeable officials from the Office of the  Under Secretary of Defense for Military Personnel Policy on the Air  Force’s use of incentives to recruit and retain RPA pilots. We collected  perspectives from RPA pilots and RPA unit commanders on the Air  Force’s strategic human-capital planning practices, including the effects  of those practices on their training, professional development, quality of  life, and retention, as well as any efforts the Air Force has made to solicit  feedback from and communicate about key issues with RPA pilots. We  also interviewed knowledgeable officials from the Air Force Personnel  Center on practices related to results-oriented cultures.
Furthermore, we compared the perspectives and documentation we  collected to the GAO criteria and held discussions with Air Force officials  to discuss instances in which the Air Force’s management actions were  not consistent with these criteria. We discussed challenges raised by the  RPA pilots and unit commanders with whom we spoke, including any  efforts in place to address the challenges.
To evaluate the extent to which the Air Force has addressed concerns, if  any, about the working conditions of RPA pilots that may affect their  quality of life, we identified and analyzed criteria included in DOD’s 2009  and 2004 Quadrennial Quality of Life Reviews in which DOD expresses  its commitment to provide servicemembers with the best quality of life  possible through support and development of responses to emerging  servicemember needs. DOD has broadly defined quality of life to include  such factors as morale, health and wellness, and work-life balance. To  understand these reviews and the commitments, we obtained information  from the Office of the Deputy Assistant Secretary of Defense for Military  Community & Family Policy, which is responsible for conducting the  department’s Quadrennial Quality of Life Reviews. To understand  challenges in the working conditions that RPA pilots may face we  analyzed studies that the Air Force conducted to assess the stress and  mental-health condition of RPA personnel, including RPA pilots.
In particular, we reviewed and analyzed two studies conducted by the Air  Force School of Aerospace Medicine published in 2011 and 2013, which  identified the sources of stress of RPA personnel. The studies’ results  were based on self-administered surveys of Air Force RPA personnel,  including pilots, from squadrons in Air Combat Command, Air Force  Special Operations Command, the Air National Guard, and the Air Force  Reserve. The surveys were administered in 2011 and 2012 with response  rates from RPA squadrons that ranged from 24 to 98 percent. The  surveys included questions related to exhaustion, distress, and post- traumatic stress disorder.
We also interviewed the researchers who conducted these studies to  clarify our understanding of their methods, findings, and  recommendations to alleviate the stress of RPA personnel. In addition,  we analyzed a report prepared for the Air Force Medical Support Agency  that describes the defining characteristics of being deployed-on-station  and examines the challenges that personnel who are deployed-on-station  face. To obtain a firsthand account of the challenging working conditions  that RPA pilots face, we conducted focus groups with pilots at Beale,  Cannon, and Creech Air Force Bases. We also interviewed leadership  officials at these bases to obtain their perspective on the challenges that  RPA pilots in their units face. Moreover, we interviewed mental-health  professionals at each of the bases we visited to obtain their perspectives  on the working conditions of RPA pilots and any effects on their quality of  life.
To evaluate actions the Air Force has taken to address the challenging  working conditions RPA pilots face, we analyzed the recommendations  that were included in the studies conducted by the Air Force School of  Aerospace Medicine and the report prepared for the Air Force Medical  Support Agency. We also obtained and analyzed documentation provided  by the Air Force Medical Support Agency that describes actions the Air  Force has taken in response to these recommendations and we  interviewed officials from this agency to further understand these actions.  Furthermore, we interviewed and obtained information from officials in the  Air Force Office of Manpower, Personnel and Services Policy and the  Office of Operations, Plans and Requirements Policy to determine any  actions the Air Force has taken to alleviate the challenging working  conditions that RPA pilots face. We also obtained information from  commanders and mental-health professionals at each of the bases we  visited to understand actions they have taken to address the challenging  working conditions that RPA pilots face and that affect their quality of life.
To evaluate the extent to which the Air Force analyzes the promotion  rates of RPA pilots, we applied criteria from common statistical practices,  which indicate that when analyzing relationships between selected factors  and a given outcome researchers should account for other key factors  that could also explain that relationship. To understand the context of Air  Force officer promotions, we reviewed relevant laws and Air Force  guidance including the Defense Officer Personnel Management Act and  Air Force Instruction 36-2501. To identify the promotion rates of Air  Force RPA pilots and how their promotion rates compared to officers in  other careers in the Air Force, we analyzed promotion-rate data for  officers in the Line of the Air Force competitive category who were  promoted “in-the-zone” to the ranks of major, lieutenant colonel, and  colonel. We analyzed data from 2006 to the most-recently available data,  which for promotion to major and colonel was 2012 and for promotion to  lieutenant colonel was 2013.
We focused on Line of the Air Force officers, because RPA pilots are  included in this category. We focused on officers promoted in-the-zone  because this zone is the point in an officer’s career when his or her  opportunity for promotion is the highest. We focused on rates of  promotion to the ranks of major, lieutenant colonel, and colonel because  the promotion rates from second lieutenant to first lieutenant and from  first lieutenant to captain are nearly 100 percent, and hence the first  competitive promotion opportunity for an Air Force officer occurs as he or  she becomes eligible for promotion to the rank of major. In addition, we  did not evaluate promotion rates above colonel because no RPA pilots  have been promoted to the general officer ranks in the Air Force yet.
To identify the percentile of RPA pilot promotion rates compared to other  line officer career fields, we analyzed data on the range of promotion  rates of active-duty officers from the careers that competed in the  promotion zone on each promotion board to the ranks of major, lieutenant  colonel, and colonel from 2006 to 2013. For this analysis, the promotion  rate of RPA pilots includes the rate for permanent RPA pilots (i.e., RPA  pilots and recategorized RPA pilots) as well as temporary RPA pilots (i.e.,  manned-aircraft pilots serving assignments in RPA squadrons and  manned-aircraft pilot training graduates). For this analysis all of the listed  career fields are mutually exclusive. That is, if a temporary RPA pilot was  identified as an RPA pilot in this analysis, the pilot was not included in the  data to calculate promotion rates for other careers such as the manned- aircraft career fields. For each promotion board, officers from between 22  and 33 careers competed for promotion. This analysis excludes career  fields where fewer than 10 officers were eligible for promotion, because  the rate of promotion in these cases is highly sensitive to the outcomes of  single individuals. However, we included the results from 8 boards in  which fewer than 10 RPA pilots competed for promotion to provide a  more-comprehensive account of RPA pilot promotions. The promotion  rate that we calculate for these instances should be considered cautiously  since the outcome of one or two individuals could have a large effect on  the overall rate. Fewer than 10 RPA pilots were eligible for promotion to  the rank of lieutenant colonel for the first 2006 board as well as the 2007  and 2008 boards. In addition, fewer than 10 RPA pilots were eligible for  promotion to the rank of colonel for the 2006, 2007, 2008, and both of the  2009 promotion boards. We obtained these data from the Air Force  Personnel Center (AFPC), and to understand the methods AFPC used to  collect, store, and maintain these data, we interviewed officials from  AFPC and reviewed documentation they provided, and we found the data  to be reliable for our purposes.
To evaluate steps the Air Force took to analyze the promotion rates of  RPA pilots and the reasons that these rates have been lower than  average, we interviewed Air Force officials in headquarters personnel  offices as well as AFPC offices. In addition, we evaluated documentation  of AFPC’s analysis of officer promotions rates including the results of  AFPC’s logistic regression identifying the factors that are related to officer  promotion. We also reviewed the August 2013 report that the Air Force  provided to Congress on the promotion rates of RPA pilots in which the  Air Force identifies reasons for lower promotion rates of RPA pilots. To  identify actions the Air Force took to respond to low RPA pilot promotion  rates, we evaluated relevant documentation including instructions the  Secretary of the Air Force has provided to promotion board members  since 2008 in which the Secretary communicates the importance of  promoting RPA pilots. We also reviewed briefings that Air Force  headquarters offices as well as AFPC prepared for the Secretary of the  Air Force on additional steps the Air Force took to address low RPA pilot  promotion rates. We also analyzed the Air Force’s August 2013 report to  Congress and additional documentation that the Air Force provided about  its plans to raise promotion rates of RPA pilots.
As we noted earlier, to obtain the perspectives of RPA pilots related to  each of our three objectives we conducted 10 focus groups that each  consisted of between six and nine active-duty RPA pilots during site visits  to Beale, Cannon, and Creech Air Force Bases. To conduct these focus  groups we randomly selected RPA pilots to participate, asked them a  structured set of questions during meetings that lasted about 90 minutes,  and took detailed notes. We then evaluated these notes using content  analysis to develop our findings. We discuss the methods we used to  select our participants, develop questions, conduct the focus-group  meetings, and analyze the information we obtained in the focus groups,  and the results of our analysis, in more detail in appendix II.
We conducted this performance audit from February 2013 to April 2014 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Focus-Group Methodology and Results
paragraphs: To obtain the perspectives of pilots of remotely piloted aircraft (RPA)  related to each of our three objectives, we conducted 10 focus group  meetings with active-duty RPA pilots during site visits to Beale, Cannon,  and Creech Air Force Bases. We decided to visit the three bases we  selected because more RPA pilots are stationed at these bases than  other Air Force bases. We specifically included Beale Air Force base  because we wanted to obtain the perspectives of the RPA pilots who fly  the RQ-4 (Global Hawk) who are stationed there. In addition, we selected  Cannon Air Force Base because we wanted to obtain the perspectives of  RPA pilots assigned to the Air Force Special Operations Command.
To select specific RPA pilots to participate in our focus groups, we  obtained documentation that included lists of the RPA pilots stationed at  each base as well as the amount of time each had served flying RPAs,  and their ranks. To obtain a variety of perspectives, we randomly selected  pilots with various amounts of experience flying RPAs and we included  pilots of various ranks in our groups. These groups typically consisted of  six to nine participants.
To conduct the focus groups, a GAO moderator followed a protocol that  included prompts, instructions to the participants, and a set of three  questions, each with several follow-up questions. We pretested this  protocol at Beale Air Force base and used it at the remaining two bases.  We used the same set of questions from this protocol for each of the 10  focus groups we conducted. These questions are reprinted below. During  each focus group, the GAO moderator asked questions related to the  topics of our review to participants who, in turn, provided their  perspectives on the topics. During the focus-group meetings, three GAO  team members took separate sets of detailed notes to document the  participants’ comments. See table 2 for the complete list of questions and  follow-up questions that we asked during our focus groups.
Following our focus-group meetings, we consolidated our separate sets of  detailed notes for each focus group to create a compiled final record of  the participant comments from each focus group. To do this, a GAO  analyst reviewed the set of detailed notes and compiled them in a final  record for each focus group. A key rule of this compilation was that if one  analyst recorded a comment, but another did not, we included the  material in the final record. To ensure that our compiled final record of  each focus group was accurate, a second analyst then reviewed at least  25 percent of each of the final records. In instances where an analyst  identified some discrepancies between the detailed notes and the final  record, the reviewing analyst corrected the discrepancy and reviewed a  higher percentage of the notes for that focus group.
Next, we used content analysis to analyze the final records of each focus  group to identify themes that participants expressed across all or most of  the groups. To do this, three GAO analysts first met to discuss and agree  on a preliminary set of themes. We then analyzed an initial set of the  records and counted instances that we observed these initial themes. We  then reconvened as a group to discuss and agree on additional themes to  add to our analysis and to consolidate and delete others. We then  analyzed our records and made coding decisions. Following the initial  analysis by one analyst, a second analyst independently reviewed all of  the coding decisions that the first analyst made for each of the records.  Where there were discrepancies, the analysts reviewed one another’s  coding and rationale for their coding decisions and reached a consensus  on which codes should be used. See figure 8 for the complete results of  our analysis.
When describing the results of our analysis of our focus groups in this  report, we use the term “some,” as in “pilots in some focus groups,” to  report topics that were discussed by RPA pilots in two to four of our focus  groups. The information we present from our focus groups accurately  captures the opinions provided by the RPA pilots who attended the 10  focus groups at the three Air Force Bases we visited. However these  opinions cannot be generalized to all of the RPA pilots at the three Air  Force Bases we visited or to all RPA pilots in the Air Force. The results of  our analyses of the opinions of RPA pilots we obtained during our focus  groups are not generalizable because the Air Force Bases we selected  are not necessarily representative of all of the Air Force Bases that  contain RPA squadrons and the RPA pilots included in our focus groups  are not necessarily representative of all of the RPA pilots in the Air Force.
section_title: Appendix III: Comments from the Department of the Air Force
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Lori Atkinson (Assistant Director),  Steve Boyles, Ron La Due Lake, Kelly Liptan, James P. Klein, Steven R.  Putansu, Michael Willems, Erik Wilkins-McKee, and Amie Steele made  key contributions to this report.