NASA/TM—2014–216636

Unmanned Aircraft System (UAS) Delegation of Separation 

in NextGen Airspace 

Caitlin Kenny 

Hewlett Packard 

Robert J. Shively 

NASA Ames Research Center 

Kevin Jordan 

San Jose State University 

January 2014 

 
ii 

NASA STI Program ... in Profile 

Since its founding, NASA has been dedicated to 

the advancement of aeronautics and space 

science. The NASA scientific and technical 

information (STI) program plays a key part in 

helping NASA maintain this important role. 

The NASA STI program operates under the 

auspices of the Agency Chief Information 

Officer. It collects, organizes, provides for 

archiving, and disseminates NASA’s STI. The 

NASA STI program provides access to the 

NASA Aeronautics and Space Database and its 

public interface, the NASA Technical Report 

Server, thus providing one of the largest 

collections of aeronautical and space science STI 

in the world. Results are published in both nonNASA channels and by NASA in the NASA STI 

Report Series, which includes the following 

report types: 

• TECHNICAL PUBLICATION. Reports of 

completed research or a major significant 

phase of research that present the results of 

NASA Programs and include extensive data 

or theoretical analysis. Includes compilations 

of significant scientific and technical data 

and information deemed to be of continuing 

reference value. NASA counter-part of peerreviewed formal professional papers but has 

less stringent limitations on manuscript 

length and extent of graphic presentations. 

• TECHNICAL MEMORANDUM. Scientific 

and technical findings that are preliminary or 

of specialized interest, e.g., quick release 

reports, working papers, and bibliographies 

that contain minimal annotation. Does not 

contain extensive analysis. 

 

• CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

• CONFERENCE PUBLICATION. Collected 

papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or co-sponsored by 

NASA. 

• SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest. 

• TECHNICAL TRANSLATION. Englishlanguage translations of foreign scientific and 

technical material pertinent to NASA’s 

mission. 

Specialized services also include organizing and 

publishing research results, distributing 

specialized research announcements and feeds, 

providing help desk and personal search support, 

and enabling data exchange services. 

For more information about the NASA STI 

program, see the following: 

• Access the NASA STI program home page at 

http://www.sti.nasa.gov 

• E-mail your question via the Internet to 

help@sti.nasa.gov 

• Fax your question to the NASA STI Help Desk 

at 443-757-5803 

• Phone the NASA STI Help Desk at 

443-757-5802 

• Write to: 

NASA STI Help Desk 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 

 
iii 

NASA/TM—2014–216636

Unmanned Aircraft System (UAS) Delegation of Separation 

in NextGen Airspace 

Caitlin Kenny 

Hewlett Packard 

Robert J. Shively 

NASA Ames Research Center 

Kevin Jordan 

San Jose State University 

National Aeronautics and 

Space Administration 

Ames Research Center 

Moffett Field, California 

January 2014 

 
iv 

The use of trademarks or names of manufacturers in this report is for 

accurate reporting and does not constitute an official endorsement, either 

expressed or implied, of such products or manufacturers by the National 

Aeronautics and Space Administration

Available from: 

NASA Center for AeroSpace Information (CASI) 

7115 Standard Drive 

Hanover, MD 21076-1320 

(301) 621-0390 

This report is also available in electronic form at 

http://www.sti.nasa.gov or http://ntrs.nasa.gov 

 
v 

Contents 

List of Figures and Tables ..................................................................................................... vii 

Acronyms and Definitions .....................................................................................................viii 

1.0 Introduction ...................................................................................................................... 1 

1.1 The Next Generation Air Transportation System ...................................................... 2 

1.2 Delegated Separation ................................................................................................. 2 

1.2.1 Current Day Delegated Separation ................................................................... 3 

1.2.2 Delegated Separation in NextGen Airspace ..................................................... 4 

1.3 Unmanned Aircraft Systems ...................................................................................... 4 

1.3.1 Types of Unmanned Aerial Vehicles ................................................................ 5 

1.3.2 Civilian UAS Use ............................................................................................. 9 

1.4 Unmanned Aircraft System Integration into the National Airspace .......................... 11 

1.4.1 Sense and Avoid ............................................................................................... 11 

1.5 Challenges and Concerns of UAS Integration ........................................................... 11 

1.6 The Current Experiment ............................................................................................ 14 

2.0 Method ............................................................................................................................. 14 

2.1 Participants ................................................................................................................ 14 

2.1.1 Experimental Participants ................................................................................. 14 

2.1.2 Confederate Participants ................................................................................... 14 

2.2 Experiment Design .................................................................................................... 15 

2.2.1 Level of Delegation .......................................................................................... 15 

2.2.2 Level of Traffic Display Information ............................................................... 16 

2.2.3 Missions ............................................................................................................ 16

2.2.4 Mission Objectives ........................................................................................... 17 

2.3 Experimental Environment ........................................................................................ 17 

2.4 Apparatus ................................................................................................................... 17 

2.4.1 Multiple UAS Simulator ................................................................................... 17 

2.4.2 The Ames 3D CSD ........................................................................................... 18 

2.4.3 Multi Aircraft Control System .......................................................................... 19 

2.5 Procedure ................................................................................................................... 21 

2.6 Metrics ....................................................................................................................... 21 

2.6.1 Objective Metrics .............................................................................................. 21 

2.6.2 Subjective Metrics ............................................................................................ 22 

3.0 Results .............................................................................................................................. 22 

3.1 Loss of Separation ..................................................................................................... 23 

3.2 UAS Operator Workload Probes ............................................................................... 23 

3.2.1 Probe Ratings .................................................................................................... 23 

3.2.2 Response Times ................................................................................................ 24 

3.3 UAS Operator NASA TLX ........................................................................................ 24 

3.3.1 Overall Workload ............................................................................................. 24 

3.3.2 Individual Subscale ........................................................................................... 25 

 
vi 

3.4 UAS Operator Situation Awareness Probes .............................................................. 26 

3.4.1 Probe Accuracy ................................................................................................. 26 

3.4.2 Probe Response Time ....................................................................................... 26 

3.5 UAS Operator Situation Awareness Questionnaires ................................................. 27 

3.6 Air Traffic Controller Workload Probes .................................................................... 27 

283.6.1 Probe Ratings ................................................................................................ 28 

3.6.2 Probe Response Times ...................................................................................... 28 

3.7 Air Traffic Controller NASA TLX ............................................................................ 28 

3.7.1 Overall Workload ............................................................................................. 29 

3.7.2 Individual Subscale ........................................................................................... 29 

3.8 Subjective Questions ................................................................................................. 30 

3.8.1 UAS Operator Subjective Post-Trial Questions ............................................... 30 

3.8.2 UAS Operator Post-Simulation Subjective Questions ..................................... 31 

3.8.3 Air Traffic Controller Post-Trial Subjective Questions .................................... 34 

3.8.4 Air Traffic Controller Post-Simulation Subjective Questions .......................... 36 

4.0 Discussion ........................................................................................................................ 38 

4.1 Loss of Separation ..................................................................................................... 38 

4.2 UAS Operator Workload ........................................................................................... 38 

4.3 UAS Operator Situation Awareness .......................................................................... 39 

4.4 Air Traffic Controller Workload Probes .................................................................... 40 

4.5 Post-Trial Subjective Questions ................................................................................ 41 

4.6 Post-Simulation Subjective Questions ....................................................................... 42 

4.7 Comparison of the Current Unmanned Systems Results to Manned Flight .............. 43 

4.8 Future Research Considerations ................................................................................ 44 

5.0 Conclusion ....................................................................................................................... 45 

6.0 References ........................................................................................................................ 46 

 
vii 

List of Figures and Tables 

Figure 1. Global Hawk UAS. ................................................................................................ 5 

Figure 2. MQ-9 Predator B UAS ........................................................................................... 6 

Figure 3. Shadow Tactical unmanned aircraft system ........................................................... 6 

Figure 4. Aerosonde small unmanned aircraft system ........................................................... 7 

Figure 5. Nano Air Vehicles ..................................................................................................8 

Figure 6. The X-45A UCAV ................................................................................................. 8 

Figure 7. The NOAA DropWindSonde system probe ........................................................... 9 

Figure 8. UAS wildfire infrared imaging .............................................................................. 10 

Figure 9. UAS flight path example ........................................................................................ 13 

Figure 10. The MUSIM ground control station ..................................................................... 19 

Figure 11. The 3D CSD ......................................................................................................... 19 

Figure 12. MACS in pseudo pilot mode ................................................................................ 20 

Figure 13. MACS in air traffic controller mode .................................................................... 20 

Figure 14. UAS operator overall workload NASA TLX ....................................................... 24 

Figure 15. Mean UAS operator temporal workload .............................................................. 25 

Figure 16. Situation awareness probe accuracy ..................................................................... 26 

Figure 17. Post-block situation awareness ratings ................................................................. 27 

Figure 18. ATC workload probe ratings ................................................................................ 28 

Figure 19. ATC overall workload NASA TLX ..................................................................... 29 

Figure 20. UAS operator difficulty interacting with ATC .................................................... 30 

Figure 21. ATC reroute acceptability ratings ........................................................................ 35 

Figure 22. UAS operator perceived situation awareness ....................................................... 40 

Table 1. Loss of Separation Means and Standard Errors ....................................................... 23 

Table 2. UAS Operator NASA TLX Significance Summary Table ...................................... 25 

Table 3. ATC NASA TLX Significance Summary Table ..................................................... 29 

Table 4. UAS Operator Subjective Questions Significance Summary Table ........................ 31 

Table 5. UAS Operator Post-Simulation Perceived Safety Ratings ...................................... 32 

Table 6. UAS Operator Post-Simulation Workload Ratings ................................................. 32 

Table 7. UAS Operator Willingness to Use CSD Ratings ..................................................... 33 

Table 8. UAS Operator CSD Usability Ratings .................................................................... 34 

Table 9. UAS Operator CSD Information Displayed Ratings ............................................... 34 

Table 10. ATC Subjective Questions Significance Summary Table ..................................... 35 

Table 11. ATC Sector Management Questions ..................................................................... 37 

Table 12. ATC Perception of UAS Operators' Capabilities .................................................. 37 

Table 13. ATC Perceived Safety and Preference Ratings ..................................................... 38 

 
viii 

Acronyms and Definitions 

ADS-B ........ Automatic Dependent Surveillance-Broadcast 

AOL ............ Airspace Operations Laboratory at NASA Ames Research Center 

CDTI ........... cockpit display of traffic information 

CSD ............ cockpit situation display 

DoD ............ Department of Defense 

FAA ............ Federal Aviation Administration 

FDDRL ....... Flight Deck Display Research Laboratory (NASA Ames Research Center) 

ft .................. feet 

HALE .......... high altitude long endurance 

ITP .............. in-trail procedure 

JPDO ........... Joint Planning and Development Office 

LOS ............. loss of separation 

MACS ......... multi-aircraft control system 

MALE ......... medium altitude long endurance 

MAV ........... micro UAV 

MUAV ........ mini UAV 

MUSIM ....... multiple UAS simulator 

NAS ............ national airspace system 

NASA ......... National Aeronautics and Space Administration 

NAV ............ nano air vehicles 

NexGen ....... Next Generation Air Transportation System 

nm ............... nautical miles 

NOAA ......... National Oceanic and Atmospheric Administration 

OSED .......... Operational Services and Environmental Definition 

RHP ............ remotely piloted helicoper 

TLX ............. Task Load Index (NASA) 

TUAV ......... tactical (or medium range) UAV 

UAS ............ unmanned aircraft systems 

UAV ............ unmanned aerial vehicle 

UCAV ......... unmanned combat aerial vehicle 

VTOL .......... vertical takeoff and landing 

VTUAV ...... vertical takeoff UAV 
1 

Unmanned Aircraft System (UAS) Delegation of Separation 

in NextGen Airspace 

Caitlin Kenny1, Robert J. Shively2, Kevin Jordan3

ABSTRACT 

The purpose of this study was to determine the feasibility of unmanned aircraft 

systems (UAS) performing delegated separation in the national airspace system 

(NAS). Delegated separation is the transfer of responsibility for maintaining 

separation between aircraft or vehicles from air navigation service providers to 

the relevant pilot or flight operator. The effects of delegated separation and 

traffic display information level were collected through performance, workload, 

and situation awareness measures. 

The results of this study show benefits related to the use of conflict detection 

alerts being shown on the UAS operator’s cockpit situation display (CSD), and to 

the use of full delegation. Overall, changing the level of separation responsibility 

and adding conflict detection alerts on the CSD was not found to have an adverse 

effect on performance as shown by the low amounts of losses of separation. The 

use of conflict detection alerts on the CSD and full delegation responsibilities 

given to the UAS operator were found to create significantly reduced workload, 

significantly increased situation awareness and significantly easier 

communications between the UAS operator and air traffic controller without 

significantly increasing the amount of losses of separation. 

1.0 Introduction 

A simple definition of unmanned aircraft systems (UAS) is to describe them as essentially an aircraft 

with the flight crew and support crew removed from the vehicle and placed at a ground control 

system with a connecting computer system and radio link used for command and control purposes 

(Austin, 2010). The demand for public access of UAS in the U.S. National Airspace (NAS) has 

grown dramatically. Since 2004, the number of public requests to fly UAS has increased over 900% 

(JPDO, 2012b). Projections of development from 2010 to 2019 predict over 20,000 UAS created in 

the United States and 35,000+ created worldwide (Teal Group, 2009). The Federal Aviation 

Administration (FAA) Modernization and Reform Act of 2012 addresses this increasing demand by 

requiring full integration of UAS into the NAS by 2015 (FAA, 2012a). Many concerns exist as to 

the best way to integrate UAS into the NextGen environment within the FAA mandated time frame. 

 

 

1 Hewlett Packard. 

2

 NASA Ames Research Center, Moffett Field, California. 

3

 San Jose State University Foundation, San Jose, California. 
2 

1.1 The Next Generation Air Transportation System 

The Next Generation Air Transportation System (NextGen) is an overhaul of the NAS that is aimed 

to create a safer, more convenient and dependable airspace system using satellite based information 

by 2025 (FAA, 2012b). The use of satellite information will allow for shorter routes, reduce delays 

due to traffic congestion, reduce fuel costs, increase the amount of throughput to two to three times 

the amount of current day traffic, and create technology that will allow air traffic controllers to 

manage airspace traffic safer and more efficiently. 

The air traffic management system used in current day operations will be unable to manage this 

growth. In 2011 alone, out of 3,567,652 flights, 20.46% were late arrivals, 19.13% were late 

departures, and 2.44% were cancelled, with only 76.83% of arrivals on time (Bureau of 

Transportations Statistics, 2012). These amounts of delays in the air traffic management system 

show how the transition from current day operations to the NextGen is vital for efficient airspace 

operations (JPDO, 2012a). 

Fundamental changes in many of the technologies and procedures used today are projected to be 

seen in air traffic automation, communications and navigation for accommodation of the expected 

two to three time increase in airspace traffic in the NAS (FAA, 2012b). As the NextGen airspace 

system will be required to do increasingly more tasks, the roles and responsibilities presently 

performed by the airspace management workforce will be required to change along with these new 

tasks. Research is being performed to develop decision support tools for both the pilots of the 

aircraft and the air traffic controllers, including testing of cockpit situation displays for use by pilots, 

as well as new merging and spacing procedures and algorithms (JPDO, 2011). 

The increase in traffic density and change in the roles and responsibilities will create effects on the 

airspace users, such as an increase in workload and a reduction of availability in air traffic 

controllers (Galster, Duley, Masalonis, & Parasuraman, 2001). One proposed measure to reduce the 

amount of controller workload, increase their availability and to increase the amount of throughput 

per sector is the use of delegated separation. 

1.2 Delegated Separation 

Delegated separation is the transfer of responsibility for maintaining separation between aircraft or 

airspace vehicles from air navigation service providers to the relevant pilot or flight operator (JPDO, 

2010). The high level tasks associated with separation assurance are the identification of potential 

conflicts and losses of separation, the identification of a solution for the problem (how the aircraft 

should maneuver to avoid conflicts), the implementation of the solution (change in heading, speed, 

altitude, etc.), and then monitoring for clear of conflict (Hoffman, Zeghal, Cloerec, Grimaud, & 

Nicolaon,1999). 

Tasks associated with separation assurance are then delegated out to the aircraft pilot or flight 

operator in delegated separation. Three major levels of delegation have been defined: limited 

delegation, extended delegation and full delegation (Zeghal, & Hoffman, 2000). The exact tasks and 

responsibilities associated with delegated separation may vary depending upon airspace type, aircraft 

type, and phase of flight (Domino, Tuomey, Mundra, & Smith, 2010). When operating in oceanic 

airspace, delegated separation may be used for overtaking another aircraft. While in terminal areas, 

aircraft may be performing a longitudinal station keeping task or perform traffic merging. When 

operating in an airport, pilots may be performing runway incursions or avoiding obstacles while 

maneuvering on the ground. 
3 

Limited delegation places the controller in charge for identifying both the potential conflict and the 

appropriate solution. Pilots are responsible for implementing the solution and monitoring for their 

ownship to be clear of conflict so that they can request permission to return to their original flight 

path. Extended delegation places the controller in charge of identifying and notifying the pilot of 

potential conflicts. The pilot is then responsible for identifying the appropriate solution, 

implementing the solution, monitoring for clear of conflict, and requesting permission from the 

controller to return to their original flight path. In full delegation, the air traffic controller assumes 

more of a monitoring role as the pilot assumes responsibility for all tasks related to separation 

assurance: identification of conflicts and appropriate solutions, implementation of the solutions, 

monitoring for clear of conflict, and resuming the original flight path (Zeghal, & Hoffman, 2000). 

A concern with using delegated separation in the NextGen airspace environment is the possibility for 

fundamentally changing the air traffic controller roles and responsibilities, potentially reducing the 

controller’s situation awareness of the sector in their command and creating a change in their 

workload. By handing off separation responsibilities to the aircrafts’ flight crews, the controllers’ 

task changes from being one of active control to a task predominately characterized by monitoring 

the aircraft in their sector (Galster, Duley, Masalonis, & Parasuraman, 2001). The additional 

responsibilities to be delegated to pilots and flight crew cause concern as well. Pilots are not 

currently trained on traffic management (FAA, 2008a), and the amount of traffic information 

required to make successful deviations for avoiding conflicts needs to be researched further. 

While the changing roles and responsibilities of air traffic controllers and pilots are of concern, the 

expected benefits of delegated separation are numerous. On the air traffic controller side, a reduction 

in workload, a reduction in the amount of radio communications between aircraft and controllers, 

and a reduction in the amount of interventions by controllers are expected. A reduction in the 

amount of maneuvers performed by the aircraft is anticipated to also create a decrease in the amount 

and cost of fuel used per flight, increased traffic throughput per sector, and an increase in pilot and 

flight deck crew situation awareness (JPDO, 2012b). 

1.2.1 Current Day Delegated Separation 

While delegated separation is viewed as being a largely NextGen concept, varying types of 

delegated separation are allowed in Class B, C, D, and E airspace in current day operations. Cockpit 

display of traffic information (CDTI) assisted visual separation allows pilots to maintain visual 

approach spacing even if visual contact is lost (JPDO, 2012b). This capability is incredibly useful in 

situations with low or poor visibility, such as stormy weather or nighttime flights. Since 2007, UPS 

has been performing CDTI assisted visual separation out of Lousiville, KY (Henden, 2008).

Visual separation is described in FAA Order JO 7110.65 U (2008b).There are two ways to perform 

visual separation; the controller identifies a conflict and issues instructions to the pilot as necessary, 

or the pilot identifies a conflict aircraft, calls in to air traffic control, and provides their own 

separation by maneuvering their aircraft as necessary. When a conflict is identified the air traffic 

controller will typically give the pilot information on the position, direction and intention of the 

intruding aircraft. If the pilot notices the traffic first, the notification process may be reversed with 

the pilot notifying the controller of the intruder. After the pilot acknowledges the intruding aircraft is 

visually in sight, the controller will instruct the pilot to maintain visual separation. The pilot will 

then accept the order to maintain visual separation, and maintain separation from the aircraft. 
4 

1.2.2 Delegated Separation in NextGen Airspace 

The use of delegated separation in NextGen airspace has been addressed by both the FAA and the 

Joint Planning and Development Office (JPDO) as part of Automatic Dependent SurveillanceBroadcast (ADS-B) uses. ADS-B consists of two parts, ADS-B In and ADS-B Out, and will be used 

as the primary surveillances method in NextGen airspace instead of radar (FAA, 2010). ADS-B Out 

broadcasts real- time information on aircraft state, such as identification information, altitude and 

velocity through a transmitter located onboard the aircraft. ADS-B In receives the information sent 

from ADS-B Out, not only from the ownship but also from nearby equipped aircraft, and has the 

potential to receive additional transmitted data. In many cases, the data provided by ADS-B is more 

accurate than radar information, and will allow the aircraft to perform more precise maneuvers for 

spacing and conflict avoidance. 

ADS-B Out will be required for aircraft operating in most of the NAS by 2020. With the precise 

information provided by ADS-B, an increase in traffic situation awareness and better conflict 

detection alerting will be possible. Through the use of ADS-B In, new spacing and separation 

capabilities will be possible starting as soon as 2017 (FAA, 2012c). Flight deck based interval 

management spacing is a capability to be used for managing spacing between aircraft while flying 

enroute. CDTI assisted visual separation is an emerging use of ADS-B data that is planned for 

expansion. It allows pilots to maintain visual approach spacing even if visual contact is lost, such as 

in situations with stormy weather or nighttime low visibility. 

The concept of delegated separation through use of ADS-B data is addressed as well. The defined 

interval concept discussed in the FAA NextGen Implementation Plan (2012b) is an example of 

extended delegation. In a defined interval task, the air traffic controller is seen to hold responsibility 

for maintaining separation while assigning pilots a spacing task that will be performed within 

defined boundaries. This use of pilot assigned spacing is believed to create a closer baseline for 

interval spacing than is currently possible through use of radar information. Delegated separation is 

again suggested through the Oceanic in-trail procedure (ITP) where a climb-through or descendthrough maneuver is initiated by the flight crew of equipped aircraft in conflict. While performing 

this maneuver, pilots are to maintain the in-trail minimum separation. A full description can be seen 

in RTCA DO-312/EUROCAE ED-159 (RTCA, 2008). 

The JPDO provides NextGen planning organizations with a view of mid-late and far-term (2025) 

operational capabilities expected to be in use in the NAS. Much of the work done by the JPDO is 

expected to be incorporated into the FAA NextGen Implementation Plan in the succeeding years as 

the FAA focuses on the near term aspects of a NextGen implementation. As part of the JPDO’s 

NextGen Avionics Roadmap (2011), delegated separation is seen to provide an enhanced situation 

awareness that is shared between pilots in the air, and the controllers on the ground. This enhanced 

situation awareness will allow delegated separation practices to expand from those currently used in 

visual conditions, to non visual conditions in controlled airspace. 

1.3 Unmanned Aircraft Systems 

Unmanned aircraft systems are more than just an aircraft with the crew removed and placed on the 

ground. UAS are typically comprised of the aircraft (also known as an unmanned aerial vehicle or 

UAV), payloads, launch and recovery subsystems (when applicable), support sub-systems, the 

control station(s), communication devices (typically radio), vehicle operator, sensory operator(s), 

and support crew (Gertler, 2012). The system elements used to operate UAS are typically based 
5 

upon those used in manned aircraft, though slightly different as they are created with the knowledge 

that the aircrew will not be onboard. 

When talking about UAS, it is important to denote the differences between UAS, model aircraft and 

drones as they are easily confused (Austin, 2010). Model aircraft are typically designed to be 

operated for recreation, and restricted to use in sight of the operator through their command link. 

Drone aircraft typically contain no “intelligence,” and no capabilities of sending or receiving any 

mission-pertinent information while in flight such as photographs, signal readings, or even vehicle 

location. Drones will often have a pre-planned flight path with a payload collecting data throughout 

their mission. This data is not accessible until after the drone is recovered and the operator removes 

the payload. 

Unmanned aircraft systems can contain varying levels of intelligence, and are capable of 

communicating with the operator information such as payload data, flight state information 

(position, airspeed, heading, and altitude), and vehicle state information (amount of fuel, engine 

status, vehicle temperature, etc.) throughout their mission task. More intelligent UAS, such as the 

Global Hawk, can even be preprogrammed to perform contingency actions when specific offnominal events, such as a loss of command and communication signal, occur (Mouloua, Gilson, & 

Hancock, 2003). 

1.3.1 Types of Unmanned Aerial Vehicles 

While UAS have many components besides the vehicle itself, the systems are typically categorized 

by the body type (such as fixed wing, turbo prop, or rotorcraft), size and capabilities (such as vertical 

takeoff and landing) of the vehicle used in the mission (Special Committee 203, 2010). The 

categories and specifications used to identify UAS can vary depending upon the agency, though a 

generic break down of UAV types may be used to describe them. 

High Altitude Long Endurance, or HALE, UAVs can perform missions in high altitude (30,000+ ft), 

typically 24 hours or longer in duration. Typically HALE UAVs are used for missions that are longrange, require reconnaissance or surveillance, and have the potential to be armed and used for target 

acquisition and prosecution. The RQ-4 Global Hawk made by Northrop Grumman is an example of 

a HALE UAV (NASA, 2010). 

Figure 1. Global Hawk UAS. Image reproduced with permission from NASA. 

Copyright 2010 by NASA. 

Medium Altitude Long Endurance (MALE) UAVs can perform missions from up to 30,000 ft in 

altitude and within 24 hours or less in duration. MALE UAVs are similar in many ways to HALE 
6 

UAVs, and are often operated in a similar manner. An example of a MALE UAV is the MQ-9 

Predator B made by General Atomics (NASA, 2007). 

Figure 2. MQ-9 Predator B UAS. Image reproduced with permission from NASA. 

 Copyright 2007 by NASA. 

Tactical or Medium Range UAV (TUAV), are smaller than MALE UAVs, and operate on simpler 

systems than the larger vehicles. TUAVs consist of both fixed wing and rotorcraft vehicles, with 

some aircraft having the capability to take off from and land on runways and airstrips. Aircraft may 

be accompanied by a ramp launching system to shoot the aircraft into the sky at flight speed. 

Examples include the Hunter RQ-5A made by Northrop Grumman, and the Shadow 600, made by 

AAI Unmanned Aircraft Systems (AAI Unmanned Aircraft Systems, 2009). 

Figure 3. Shadow® Tactical unmanned aircraft system. Image reproduced with 

permission from AAI Unmanned Aircraft Systems. Copyright 2009 by 

AAI Unmanned Aircraft Systems. 

Close Range UAVs are a prolific subset often used by groups that do not have a set location and are 

moving due to their smaller size. As Close Range UAVs are often used in locations that are lacking 

in runways and airstrips, they are often accompanied by a launching system that typically uses a 

ramp mounted on a vehicle used in transport to shoot the aircraft from the ramp at flight speed, 
7 

similar to those used by some TUAVs. Recovery of the Close Range UAVs is typically done 

through use of a parachute and airbag combination to lessen the damage of a fall, though methods 

may vary such as in the case of the sky-hook used for the Scan Eagle. Their use can range from 

reconnaissance, and target acquisition, to power line inspection, and crop spraying. Examples 

include the Scan Eagle made by Insitu, and the Aerosonde Small Unmanned Aircraft System made 

by Aerosonde Pty Ltd, (2012). 

Figure 4. Aerosonde® small unmanned aircraft system. Image reproduced with 

permission from AAI Unmanned Aircraft Systems and Aerosonde Pty Ltd. 

Copyright 2012 by Aerosonde Pty Ltd. 

Mini UAV (MUAV) is a class of UAVs that is not yet clearly defined. MUAVs are generally hand 

launched, and operated through the use of a laptop ground control station. Like Close Range UAVS, 

MUAVs are often used by groups that are mobilized due to their small size and portability. 

Examples include the Desert Hawk III made by Lockheed Martin, and the Skylite made by BlueBird 

Aero Systems. 

Smaller UAVs have been created for use in urban terrains and research. The smaller sizes of these 

UAVs allow the vehicles to fly inside urban areas as opposed to flying over them. A desire for 

UAVs being used in urban terrains is to have the capability to hover and potentially perch on walls 

or window sills, creating unconventional configurations not commonly seen in the larger UAVS, 

such as the use of flapping wings (ornithopeters). 

The Micro UAV (MAV) was initially defined as having a wing span of less than or equal to 6 

inches, though current definitions may vary. MAVs are preferred for use in urban terrains as their 

small size and maneuverability make them ideal for navigating between and within buildings. 

Examples include the Wasp made by AeroVironment Inc, and the Mosquito made by Israel 

Aerospace Industries. 

Nano Air Vehicles (NAV) is an emerging category populated by small vehicles less than 5 cm in 

any direction. The “nano” in the name comes from the requirement that a vehicle so small will need 

nanotechnology for use in subsystems such as batteries, sensors and motors. NAVs may be used in a 
8 

similar manner as MAVs, though they are also expected to be used in swarms, possibly for radar 

confusion or short range surveillance as most operate for only a few minutes at a time due to their 

small size. Examples include the Prox Dynamics Pico-flyer, Ornithopter, and Black Hornet (Prox 

Dynamics, 2012). 

Figure 5. Nano Air Vehicles. Examples of NAVs include the Pico-flyer (top left), the 

Ornithopter (top right), and the PD-100 Black Hornet (bottom). Image 

reproduced with permission from Prox Dynamics. Respective copyrights 

2005, 2007, and 2009 by Prox Dynamics. 

Some of these categories may utilize rotorcraft vehicles, often called Remotely Piloted Helicopter 

(RHP), Vertical Takeoff UAV (VTUAV) or Vertical Takeoff and Landing (VTOL) UAV. These 

UAVs are valued for their ability to hover during missions, and for being less susceptible to 

turbulence than fixed wing aircraft. It is of interest to note that not all aircraft that are capable of a 

vertical takeoff are capable of vertical landing. Some examples include the Firescout made by 

Northrop Grumman, and the Camcopter made by Schiebel. 

Combat UAS are split into two main categories. Unmanned Combat Aerial Vehicles (UCAV) are 

fixed wing aircraft designed to carry and launch weapons which may be used in aerial combat 

situations. Unmanned Combat Rotorcraft (UCAR) are also in development. Examples include the X45A made by Boeing, and the X-47 Pegasus made by Northrop Grumman (NASA, 2002). 

Figure 6. The X-45A UCAV. Image reproduced with permission from NASA. 

Copyright 2002 by NASA. 
9 

1.3.2 Civilian UAS Use 

Unmanned systems are often used for duties that are deemed to be too dull, dirty or dangerous for 

human completion (Takayama, Ju, & Nass, 2008). In addition to the known military use of UAS for 

training, transit, surveillance, and reconnaissance, there is a wide variety of civil applications. 

Proposed uses include research, disaster monitoring and aid, agriculture, surveillance, and 

conservation among others. 

Research applications include geological and meteorological aspects. Meteorological applications 

often include the sampling of atmospheric components that can be used to further understand 

weather phenomena and create more accurate weather forecasting. Weather monitoring data can 

include information on air temperature, dew point, atmospheric pressure, winds, global warming 

monitoring, and atmospheric pollutions ratings (NOAA, 2012b). 

Currently UAS are utilized for research missions, including the Global Hawk used by NASA and 

National Oceanic and Atmospheric Administration (NOAA) primarily focused on collecting 

information about storm development and tracking (NOAA, 2012a). This Global Hawk utilizes a 

special payload called the DropWindSonde System (Figure 7) designed to be dropped into the top of 

the storm and continuously collect data until the weather dissipates (NOAA, 2012b). 

Figure 7. The NOAA DropWindSonde system probe. Image reproduced with 

permission from NOAA. Copyright 2012 by NOAA. 

The use of UAS for disaster monitoring and aid is done through the use of video and infrared search 

within specific restrictions defined by the FAA (Aviation Unmanned Aircraft Program Office, 

2008). The first time that UAS were requested for search and rescue operations was in 2005, in the 

wake of Hurricane Katrina (Waharte & Trigoni, 2012). At the time, the FAA did not have any 

regulations in place for this type of UAS use, and no way to authorize the mission for flight in the 

national airspace. Regulations have since been passed to grant authorization for relief missions in a 

matter of hours. 

The MQ-1 Predator is now authorized to fly in search and rescue missions in support of disaster 

relief as the UAS has an infrared camera that is able to identify heat source emissions small 

enough to come from a human body up to approximately 10,000 ft away (Waharte, & Trigoni, 

2012). This infrared camera can assist rescuers to identify and locate survivors quickly, as well as 

increase the efficiency of search patterns and crew organizations. The infrared camera is especially 

useful during wildfire flares, and has been used to assist firefighters in locating hidden pockets of 
10 

fire and mapping the movement of fires across extended periods of time (NASA, 2007). An 

infrared image created by the Ikhana Predator of the Harris Fire in San Diego County (2007) can 

be seen in Figure 8. 

Figure 8. UAS wildfire infrared imaging. Thermal infrared image scans from NASA's 

Ikhana Predator UAS of the Harris Fire in San Diego County October 24, 

2007. Image shows wildfire hot spots in yellow along the ridgeline. Image 

reproduced with permission from NASA. Copyright 2007 by NASA. 

Agricultural and fisheries uses are varied and can include crop monitoring and spraying, livestock 

monitoring and herd driving, fisheries protection and water stress, as well as habitat monitoring. 

Using UAS for these purposes creates a number of potential advantages, such as with the cost of 

aerial imagery. Aerial imagery is typically collected from either satellite data, which is often not up 

to date and can often have a low resolution, or from data collected by having a light aircraft fly over 

the specified lands. Both options are relatively expensive, and the use of Micro UAS imaging could 

potentially cut down the cost of acquiring the imagery (Austin, 2010). The FAA currently does not 

allow the use of UAS for agricultural or fisheries purposes, though other countries have used them 

for years. Farmers in Japan, for example, have used small UAS for approximately 10 years to 

perform monitoring and crop spraying tasks. 

While surveillance and monitoring is typically associated with military tasks, there are many civil 

uses. Aerial photography and video recordings are used in a variety ways from land surveying, 

power and gas line inspections, oil pipeline security, news and media use, monitoring and control of 

traffic on roads and highways, and ordinance surveys for creation of detailed maps. Perhaps one of 

the more well known surveillance and monitoring tasks UAS are used for is police and customs 

work. The UAS are capable of searching for mission persons and suspects, checking coastline and 

land bound borders for illegal immigration suspects, illegally imported goods and search and rescue 
11 

missions at sea (DoD, 2011). Currently both the U.S. Border Patrol and the U.S. Coast Guard have 

multiple UAS in use, including the fixed wing Predator B and the Eagle Eye tilt rotor helicopter. 

There also exists the capability for UAS to assist in conservation efforts. Through the use of UAS, 

conservationists are able to monitor atmospheric and land based pollution, forestry services are 

capable of identifying and controlling fires, monitoring deforestation patterns, and wildlife 

monitoring. The use of UAS is especially helpful when observing endangered or threatened species 

that have extensive migratory patterns, or live in hard to access regions. UAS have been successfully 

used in conservation efforts for estimating shrub utilization, identifying and locating invasive 

species, measurement of the biomass and nitrogen levels present in various plan species, 

identification of crop water stress, and mapping various rangeland plant species (Austin, 2010). 

1.4 Unmanned Aircraft System Integration into the National Airspace 

The integration of UAS into the NAS is seen to be one of the critical issues existing between now 

and the mid-term vision of NextGen (JPDO, 2012b). The demand for public access of UAS in the 

U.S. National Airspace (NAS) has grown dramatically. Projections of development from 2010 to 

2019 predict over 20,000 UAS created in the United States and 35,000+ created worldwide (Teal 

Group, 2009). The FAA Modernization and Reform Act of 2012 addresses this increasing demand 

by requiring full integration of UAS into the NAS by September 30, 2015 (FAA, 2012a). 

As part of this bill, the FAA is required to create a plan in order to successfully integrate UAS into 

the national airspace without compromising the safety and efficiency of the existing air traffic 

management system and its users (FAA, 2012a). The plan is required to have at minimum 

recommendations on how to define acceptable operational standards for flight and certification of 

UAS. The purpose of this section of the bill is to ensure that all UAS will have a sense and avoid 

system, and to create the standards for what would be required to act as a UAS operator in the NAS 

as well as the process to achieve certification. 

1.4.1 Sense and Avoid 

Sense and avoid is the capability to maintain separation from intruding aircraft. Sense and avoid 

systems may include a suite of surveillance sensors, trackers, threat detection and/or resolution 

algorithms, a traffic display for the pilot, and potentially resolution guidance or advice (Prinzel, et. 

al., 2011). The addition of a sense and avoid suite on a UAS would create the ability for the UAS to 

avoid collisions through a combination of self-separation and collision avoidance. 

The capability for a UAS to self-separate is deemed to be an essential component of the sense and 

avoid system, and may be the only function required in the sense and avoid system provided a safety 

analysis can demonstrate the target level of safety is met (JPDO, 2012b). While operating in 

positively controlled airspace, separation responsibilities may be delegated to the UAS operator. 

Conflict avoidance alone on a UAS is not an acceptable means to remain well clear as stated in 

14CFR Part 91 (FAA, 2001); even if the target level of safety was met, a self separation capability 

would be required. 

1.5 Challenges and Concerns of UAS Integration 

The NAS was originally designed for use of manned aircraft, and while many procedures and 

general principles from manned aircraft can apply for UAS, there are significant differences is 

capabilities, advances in technology, and operational experience. These performance differences 
12 

between UAS and manned aircraft may cause disruption in the NAS. Four challenges have been 

identified for UAS integration into the NAS; communication, airspace operations, internal systems 

onboard the aircraft, and human systems integration (JPDO, 2012b). 

Communication concerns exist about the transfer of information between the aircraft, the operator, 

and the satellite/technology providing communications relay. Currently, communications relays can 

take on average two to eight seconds to send information from the aircraft to the operator. In a time 

sensitive situation, such as when avoiding a collision, this delay may be the difference between a 

crash and successful conflict avoidance. The security of the communications link is also of great 

concern as this could be hacked into, and control of the unmanned vehicle lost (DoD, 2001). 

Airspace operations concerns are associated with the UAS operating in the same airspace as manned 

aircraft. The quality and availability of surveillance data needs to be assessed. Integrated separation 

concepts need to be developed, requiring an evaluation of performance of different potential human 

and machine roles and responsibilities, and how to properly integrate the varying self separation, 

separation assurance and conflict avoidance functions. With the need for a sense and avoid system 

capable of self-separation, the differences in unmanned and manned aircraft flight become points of 

note. Concerns exist about UAS performing self-separation due to the lack of the out-the-window 

view that manned aircraft pilots have (Gawron, 1998). Without this visual means of traffic 

acquisition, a sense and avoid solution becomes vital for safe UAS flight. UAS self-separation is an 

essential component of the sense and avoid solution, and may be the only function provided safety 

analysis demonstrates the target level of safety is met (JPDO, 2011). 

The unmanned aircraft itself, and systems located onboard, are also items of concern. Airframe 

certification, location and navigation systems, UAS avionics and control systems certification, as 

well as the ability for the UAS operator to have an accurate awareness of vehicle state and be 

capable of real time management are all items that need to be further researched (McCarley, & 

Wickens, 2005). The flight characteristics of UAS are different from manned aircraft in many ways, 

such as being operated at speeds slower than those typically used for manned aircraft and being able 

to fly at higher altitudes than many manned aircraft. 

The overall flight paths of manned and unmanned aircraft are also quite different and important to 

note. Manned aircraft typically aim to fly the shortest distance from Point A to Point B through the 

use of specified routes and arrival corridors. Unmanned aircraft are often required to operate in a 

grid or corkscrew pattern, loiter, hover, and perform frequent heading and altitude changes, as well 

as unplanned aerial work around areas of interest while performing mission roles (Special 

Committee 203, 2010). An example of a UAS flight path can be seen in Figure 9. The differences in 

mission behavior have the potential to cause congestion and an increase of potential conflict in 

sectors as the flight path of a UAS may appear to be erratic and unpredictable when compared to that 

of a manned aircraft. The differences in flight path configuration and mission roles are important to 

take into consideration as they may increase the workload of air traffic controllers managing the 

sector the UAS is flying in. 

Human system integration concerns focus on the interaction between the operator and the ground 

control station’s interfaces and systems (JPDO, 2012b). Currently there is no universal ground 

control station used for unmanned aircraft, nor are there any guidelines for what should be required 

in a ground control station, or of the UAS operator. Research must be conducted to determine the 

optimal performance needs. Areas needing research include: the display of traffic, aircraft and 
13 

airspace information, focusing on the presentation method and quantity of information available; 

how to promote optimal human and automation interaction; system level issues for any interfaces or 

algorithms being utilized; a pilot centric station that includes an ergonomic evaluation for optimal 

placement of displays and controls; a clear definition of roles and responsibilities given to the UAS 

operator and the air traffic controller; and a set of qualification and training requirements needed to 

operate safely in the NAS (DoD, 2011). 

Figure 9. UAS flight path example. This image shows the planned route of a 

ScanEagle UAS in a green corkscrew pattern performing a marine fisheries 

protection and monitoring operation. Image reproduced with permission 

from RTCA, Inc. Copyright 2010 by RTCA, Inc.4

The NextGen airspace will be required to provide a NAS that is flexible and robust enough to 

support routine use of UAS instead of as the exception. This could be accomplished through a 

combination of automation, more precise airspace information provided by ADS-B, and delegated 

separation given to the pilot within positively controlled sectors (Special Committee 203, 2010). It is 

theorized that much of the activity would occur in air traffic controller systems using automation, 

and on systems located in manned aircraft and unmanned ground control stations to advise pilots of 

relevant flight path changes. This will help to maximize airspace efficiency and safety. 

Through use of self-separation, aircraft exchanges would be accepted or rejected based upon the 

situation awareness of the UAS operators and manned pilots. The role of the air traffic controller 

would be changed into one predominated by monitoring, with intervention only when necessary 

(Grimaud, Hoffman, & Zeghal, 2001). Self separation is planned to initially take place in areas with 

low traffic density, with a gradual progression to high density airspace. The UAS flying in NextGen 

airspace could benefit greatly from delegated separation as they will also begin to operate in low 

density areas and perform flight paths and mission tasks in a manner that is not typical for manned 

aircraft (Special Committee 203, 2010). The use of delegated separation by UAS operators could 

 

4 The complete document referenced, DO-320 Operational Services and Environmental Definition (OSED) 

for Unmanned Aircraft Systems (UAS), may be purchased from RTCA, Inc. 1150 18th Street NW, Suite 910, 

Washington, DC 20036. (202) 833-9339 www.rtca.org.
14 

reduce the amount of workload placed on air traffic controllers when monitoring for unexpected 

maneuvers and aircraft flying atypical routes. 

1.6 The Current Experiment 

The purpose of the current experiment is to determine the feasibility of UAS performing delegated 

separation in the NAS. The study was a 2 x 2 within-subjects design, with level of delegation 

(extended delegation vs. full delegation) and traffic display information level (basic, traffic only vs. 

conflict detection present) as independent manipulations. The effects of delegated separation and 

traffic display information level were assessed through objective and subjective measures of 

performance, workload and situation awareness. Experimental data were collected in the Flight Deck 

Display Research Laboratory (FDDRL) at NASA Ames Research Center from the experimental 

participants consisting of 13 pilots and two retired air traffic controllers acting as pseudo-controllers. 

Three pseudo- pilots were used in a confederate role to create sector traffic with no data collected on 

their performance. 

The expected results for air traffic controllers in the full level of delegation include: reduced 

workload, radio communications with the UAS operator to be perceived as easier, and less frequent 

communications with the UAS pilot. The expected results for the UAS operator include an increased 

(but manageable) workload in the full level of delegation, and increased situation awareness within 

both the full level of delegation and the conflict detection present mode of the CSD. It is expected 

that both the air traffic controller and the UAS pilot will have higher levels of workload in extended 

delegation, and subjective scores rating their perception of difficulty in communication will increase 

as well. The amounts of losses of separation are expected to be lower in full delegation and in the 

conflict detection present CSD mode. The overall amount of losses of separation levels are expected 

to be low. 

2.0 METHOD 

2.1 Participants

2.1.1 Experimental Participants 

A total of 15 experimental participants were used for this study; 13 pilots acting as UAS operators, 

and two retired air traffic controllers acting as pseudo controllers. Data was collected from the 13 

pilots and the two pseudo controllers. UAS operator participation was limited to pilots who were 

between 18-40 years old, had normal or corrected to normal vision, and held at minimum a Private 

Pilot Certificate with preference towards those with an Instrument Rating. Pseudo controller 

participation was limited to retired air traffic controllers with previous experience using the MutliAircraft Control System (MACS) controller mode software (AOL, 2008). Each air traffic controller 

worked with approximately half of the UAS operator participants throughout the simulation. 

2.1.2 Confederate Participants 

Three pilots acted as confederate pseudo pilots to create the background airspace traffic and radio 

chatter. Pseudo-pilot participation was limited to pilots who were between 18-40 years old, had 

normal or corrected to normal vision, held at minimum a Private Pilot Certificate and had previous 

experience using the pseudo-pilot mode of MACS. No data was collected from the pseudo pilots as 

they were acting in a confederate role. 
15 

2.2 Experimental Design 

The current study was a 2 x 2 within-subjects design with level of delegation (extended delegation 

vs. full delegation) and traffic display information level (basic—traffic only vs. conflict detection 

present) as independent variables. Counterbalancing was used to control for order effects across the 

four experimental scenarios created as the result of the factorial combination of the two levels of 

delegation and two levels of traffic display information. The scenarios were blocked by traffic 

display condition; the first traffic display condition was presented with the two levels of delegation 

before the second traffic display condition was presented with the two levels of delegation. Order of 

presentation of the traffic display condition (basic or conflict detection) was counterbalanced across 

participants. All experimental data was collected in the Flight Deck Display Research Laboratory 

(FDDRL) at NASA Ames Research Center from the participants consisting of 13 pilots, two retired 

air traffic controllers acting as pseudo-controllers, and three pseudo-pilots. 

Data on the effects of delegation and traffic display information level was collected through 

objective and subjective measures of performance, workload and situation awareness. The level of 

delegation manipulation was used to uncover the effects of delegating the roles and responsibilities 

for maintaining separation partially or fully from the air traffic controller to the UAS operator. The 

level of traffic display information was used to uncover the effects of increasing information 

available about the surrounding airspace on operator performance when delegated with separation 

responsibility. As there were only two air traffic controllers in this experiment, it is important to note 

the analyses performed on their data have low statistical power. 

2.2.1 Level of Delegation 

The level of delegation was separated into two conditions, extended delegation and full delegation, 

which were counterbalanced across missions. In both scenarios, only the UAS was given delegated 

separation and the air traffic controllers maintained positive control of all surrounding aircraft. At 

any point in time throughout the scenarios, the UAS Operator was allowed to request help from the 

air traffic controller by giving the direction, flight level and callsign of the intruder aircraft. 

Example of a UAS Operator requesting help: 

• UAS Operator: “L.A. Center, this is PD-1. Traffic north bound, FL 280, callsign 

SWA242. Cannot perform separation, please advise.” 

• ATC: “PD-1, traffic acquired.” 

– ATC orders a reroute to avoid collision. 

In extended delegation, the air traffic controller was responsible for problem identification and 

notifying the UAS operator of the potential conflict. The UAS operator then located the intruder 

aircraft on their traffic display and told the air traffic controller that the traffic was acquired. The 

controller would then tell the UAS operator to maintain separation from that aircraft. The UAS 

operator then accepted the separation order, and was responsible for identification and 

implementation of conflict solutions and monitoring for clear of traffic after rerouting. 

Extended Delegation example: 

• ATC: “PD-1, traffic SWA749 at 2 o’clock, north bound, FL 280. Advise tracking.” 

• UAS Operator: “PD-1, traffic acquired.” 

• ATC: “PD-1, maneuver to maintain separation from SWA749.” 
16 

• UAS Operator: “PD-1, maintaining separation with SWA749.” 

– UAS makes a reroute to avoid collision and monitors until clear of conflict. 

• UAS Operator: “L.A. Center, this is PD-1. Clear of traffic, request to return to original 

flight path.” 

• ATC: “Roger, PD-1. Permission to return to original flight path approved.” 

– UAS returns to original flight path. 

Full Delegation gave responsibility to the UAS operator for all tasks related to separation assurance, 

including identification of problems and solutions, implementation of solutions and monitoring for 

clear of traffic after rerouting. Air traffic controllers only reclaimed separation responsibilities from 

the UAS operator if collisions became imminent or the operators requested assistance.No 

communication with air traffic control was required to complete these scenarios. 

Full Delegation example: 

• UAS Operator monitors CSD for potential conflicts. 

• UAS Operator identifies a potential conflict. 

• UAS Operator makes a reroute to avoid collision and monitors until clear of conflict . 

• UAS Operator identifies clear of conflict and resumes original flight path. 

2.2.2 Level of Traffic Display Information 

The level of traffic display information was split into two conditions, basic and conflict detection 

present, and counterbalanced across missions. The basic conditions provided a 2-D display of the 

surrounding traffic in an ownship centric manner. Basic information included aircraft call sign, 

altitude and airspeed, as well as the use of color-coding to denote altitude (green for 500+ ft below 

ownship, white for within 500 ft below or above ownship, and blue for 500+ ft above ownship). No 

conflict detection was available in the basic conditions. The conflict detection present conditions 

included the information available in the basic conditions and provided conflict detection alerts in a 

visual and aural manner that was based upon ballistic information. Once the UAS operator 

successfully rerouted to avoid the conflict, the ownship and intruder aircraft stopped glowing yellow. 

2.2.3 Missions 

Two training and four experimental scenarios were used in this experiment. The training scenarios 

were five minutes long and provided practice for all participants. The UAS operators learned how to 

operate their aircraft and how to use their ground control station to perform flight path reroutes while 

practicing under the assigned level of delegated separation. Air traffic controllers practiced 

delegating separation responsibility to the UAS operator in the assigned level of delegated 

separation. 

Experimental scenarios were each 30 minutes long, and consisted of a CO2 emissions monitoring 

task. No conflicts were pre-scripted, though the flight paths of the UAS throughout the trials were 

designed to cross over and through traffic streams so as to increase the chance of a conflict 

occurring. All scenarios began with the UAS in a preprogrammed flight path and included five 

mission messages that required the UAS operators’ attention and flight path reroutes. The timing of 

the mission messages and the four flight paths differed between scenarios and were counterbalanced 

as a means to reduce predictability. 
17 

2.2.4 Mission Objectives 

The UAS Operators were instructed to fly a CO2 emissions monitoring task in southern California 

based upon scenario 3 in RTCA SC-203’s OSED for UAS (2012). The mission objectives that were 

given to the UAS operator included: follow the appropriate level of delegated separation while 

flying the mission routes, reroute in response to mission messages, reroute to maintain separation 

from the surrounding traffic, and maintain communications with ATC as necessary. Air traffic 

controllers were also informed of the level of separation delegated to the UAS operator, and notified 

of the corresponding responsibilities in each of the scenarios. Air traffic controllers always 

maintained positive control over the surrounding aircraft; only the UAS received delegated 

separation responsibilities. 

2.3 Experimental Environment 

The experimental environment was a simulated LA Center airspace. UAS operators and air traffic 

controllers had access to a list of currently used navaids and fixes for use in communications and 

route modifications. The current day air traffic controller will accept and hand off aircraft entering or 

exiting their sector and maintain separation between aircraft to avoid mid-air collisions. Separation 

standards for aircraft are typically a 5 nm mile horizontal distance, and 1000 ft in vertical distance in 

current day operations (FAA, 2008b). If the controller fails to maintain this level of separation, a 

loss of separation (LOS) occurs; collisions may or may not occur depending upon how far the 

intruder penetrates the separation standard. 

In half of the trials, separation responsibilities were partially delegated to the UAS operator, and in 

the other half separation responsibilities were fully delegated to the UAS operator. In all 

experimental conditions, the air traffic controller maintained positive control of the surrounding 

aircraft; only separation responsibilities for the UAS were delegated away from the controllers. The 

UAS operators used a ground control station composed of the U.S. Army Aeroflightdynamics 

Directorate’s Multiple UAS Simulator (MUSIM) (Fern & Shively, 2009) and the Ames 3D CSD 

(Granada, Dao, Wong, Johnson & Battiste, 2005) to display traffic. The air traffic controllers and 

confederate pseudo pilots used the MACS in their respective air traffic controller and pseudo pilot 

modes. 

All experimental participants additionally had a touch screen monitor used to collect workload 

ratings through probes. Workload probes were presented with an auditory chime to alert the 

participants of their presence. UAS operator probes were presented every three minutes with a scale 

from one (low) to seven (high), while air traffic controller probes were presented every three 

minutes on a scale from one (low) to nine (high). UAS operators were given six situation awareness 

probes per trial presented in a chat box that was located on MUSIM. Communications between 

participants and pseudo pilots were given by voice through a simulated radio frequency. 

2.4 Apparatus

2.4.1 Multiple UAS Simulator 

The MUSIM, shown in Figure 10, is a Linux based UAS ground control station simulation 

environment; a full description of its capabilities can be found in Fern & Shively (2009). The 

configuration of MUSIM for this experiment was in a 1:1 operator to vehicle interface with no 

sensor video as the focus was on the basic flight operation of the UAS and not sensor tasks. A 

generic fixed wing flight control model with generic Mid-Altitude Long Endurance UAS parameters 
18 

was used. Ownship speed for the UAS was fixed at 110 knots throughout all missions. Shutter 

screen capture software was utilized to record video of all events occurring on MUSIM. 

Figure 10. The MUSIM ground control station. The ground control station was 

separated into four sections. A map display used to indicate the position 

and flight path of the UAS, the available navaids for route modifications, 

and where the UAS operator performed route modifications (left). A multifunction display to display the UAS primary flight display and mission 

messages (top right). The mIRC Chat box used to display a simulated UAS 

chat group and the operator’s situation awareness probes (middle right), 

and a timer (bottom right). 

2.4.2 The Ames 3D CSD 

The Ames 3D CSD (Figure 11) is a 3D volumetric display designed to provide pilots with an 

increased situation awareness of the surrounding traffic. A full description can be found in Granada, 

Dao, Wong, Johnson & Battiste (2005). 

In this experiment, the Ames CSD was used in its basic 2D planar view to display traffic 

information. The CSD has an ownship-centric view of the surrounding traffic that was color coded. 

Green aircraft were over 500 ft below the ownship’s altitude, white aircraft were within 500 ft above 

or below the ownship’s altitude, and blue aircraft were over 500 ft above the ownship’s altitude. 

Participants were allowed to adjust the horizontal viewing distance from 10–640 nm, and were 

provided with conflict detection alerts in half of the conditions. UAS operators were notified of 

traffic as far as 20 minutes into the future through alerts given in the form of an auditory chime, and 

a visual of their aircraft and the intruding aircraft glowing yellow on the cockpit situation display. 

Camtasia screen capture software was utilized to record video of all events occurring on the CSD. 
19 

Figure 11. The Ames 3D CSD. Green aircraft were 500+ ft below ownship’s 

altitude, white aircraft were within 500 ft above or below ownship’s 

altitude, and blue aircraft were 500+ ft above ownship’s altitude. The 

visual alert of the yellow glowing ownship and intruding aircraft can be 

seen indicating a conflict. 

2.4.3 Multi Aircraft Control System 

Multi Aircraft Control System is a JAVA program created by the Airspace Operations Laboratory 

(AOL) at NASA Ames for simulation of airspace to be used in air traffic management and 

operations research (AOL, 2008). In the current experiment, the MACS pseudo pilot (Figure 12) and 

air traffic controller (Figure 13) modes were used. The sector controlled by the air traffic controllers 

was between 13,000 and 24,000 ft, with the navaids and fixes that were available to the UAS 

operator displayed and boundaries denoted by highlighted lines. Camtasia screen capture software 

was utilized to record video of all events occurring on the pseudo pilots’ and air traffic controllers’ 

monitors. 
20 

Figure 12. MACS in pseudo pilot mode. MACS allowed the pseudo pilots to 

control multiple aircraft at a time through the use of automation. The 

monitor on the left displays the sector, and the monitor to the right 

displays aircraft controls. 

Figure 13. MACS in air traffic controller mode. MACS allowed the air traffic 

controllers to observe their sector and manipulate the traffic flow. The 

sector was between 13,000 and 24,000 ft, with navaids and fixes 

displayed, and boundaries denoted by highlighted lines. 
21 

2.5 Procedure 

After arrival in the lab, all participants were briefed on the purpose of the study, and given informed 

consent and demographic forms to complete. Participants were split into two groups; UAS operators 

and pseudo participants. The pseudo participants (including pseudo controllers and pseudo pilots) 

received their initial briefing and training before the experimental data was collected. This was done 

to reduce the amount of time spent on training as the pseudo participants rotated throughout the 

experiment. The UAS operator participants were briefed on the purpose of the study and provided 

with instructions on how to operate MUSIM and respond to online workload and situation awareness 

probes. 

UAS operator participants were then run through a five-minute practice scenario focusing only on 

the use of MUSIM and responding to probes without the presence of the pseudo controllers or 

pseudo pilots. After completion of the initial MUSIM practice scenario, participants were given 

training appropriate to their first experimental scenario block through the use of self-paced 

PowerPoint slides. After the training was completed, all participants completed a full five-minute 

practice scenario in the appropriate condition containing mission reroute messages and probes. 

After the training scenario was completed, participants then began their first 30 minute experimental 

scenario. After each experimental scenario, UAS operator and pseudo controller participants were 

given a NASA Task Load Index (TLX) to assess their level of workload. The experimental 

participants also responded to post-trial subjective questions asking about workload associated with 

communications and negotiations between the pseudo air traffic controllers and the UAS operators. 

After the questionnaires were completed, participants were given a 10 minute break. There were two 

delegated separation scenarios (extended delegation vs. full delegation) performed for each CSD 

experimental block (basic traffic information vs. conflict detection present). 

After each experimental block, the UAS operator participants completed a subjective questionnaire 

asking about situation awareness. After the first experimental block, the UAS operator participants 

were given the appropriate training for the next two scenarios through a self-paced PowerPoint slide 

presentation. The training was then followed by a five minute practice scenario with all elements 

associated with the experimental scenario given to the UAS operator, pseudo air traffic controllers 

and pseudo pilots. 

Post scenario and post block questionnaires were provided in the same order as given with the first 

experimental block. After the last scenario and associated paperwork was completed, the UAS 

operator and pseudo controller participants were given a post-simulation questionnaire asking 

questions on workload, situation awareness, pseudo controller to UAS operator interactions, as well 

as acceptability, preference and usability ratings. Participants were then debriefed, and the 

experimenter answered any questions the participants had. 

2.6 Metrics 

The effects of delegation and traffic display information level were assessed through objective and 

subjective measures of performance, workload and situation awareness. 

2.6.1 Objective Metrics 

The objective metrics that were collected focused mainly on the UAS operator. Current separation 

standards are defined as having a minimum required distance of 1,000 ft vertical and 5 miles lateral 
22 

(FAA, 2008b). When aircraft are separated by a distance less than this, a LOS has occurred. 

Situation awareness probes were provided to the UAS operator through queries provided in the 

MUSIM chat client. Operators were required to monitor the chat window for their callsign (PD-1) 

and respond to any queries addressed to them. Responses were judged based upon the accuracy of 

the response, with the assumption that correct responses indicate high situation awareness. UAS 

operator response times were also analyzed. 

2.6.2 Subjective Metrics 

The subjective metrics collected included situation awareness ratings and multiple workload ratings 

from both the UAS operator and the pseudo controller. The UAS operators were provided with 

online workload probes every three minutes on a scale from one (low) to seven (high), and the 

pseudo air traffic controllers received online workload probes every three minutes on a scale from 

one (low) to nine (high). Both the UAS operators and the pseudo air traffic controllers were 

provided with a post-trial NASA TLX. 

The NASA TLX is a self-assessed workload measure that includes ratings on six dimensions; 

mental, physical, temporal, performance, effort and frustration (Hart, & Staveland, 1988). Along 

with the NASA TLX, both UAS operators and pseudo controllers were given post-trial and postsimulation questions pertaining to their overall workload associated with their interactions with each 

other (UAS operator to pseudo controller) on a scale from one (low) to seven (high), and additional 

information on UAS operator and pseudo-controller workload, situation awareness, and acceptability 

and preference ratings. 

3.0 RESULTS 

Multiple 2x2 repeated measures ANOVAs were performed on the data collected in this experiment. 

The repeated measures ANOVA is used to test the equality of means when the participants are 

exposed to all treatments (Howell, 2011). This test indicates a real difference between the means in 

the datasets compared to a difference found by chance or sampling error. 

ANOVAs were performed on the following data sets: 

• Loss of separation (within 5 nm horizontal and 1,000 ft vertical) 

• Pilot workload probes (ratings and response times) 

• Pilot NASA TLX (individual subscales and overall) 

• Pilot situation awareness probes (accuracy and response times) 

• ATC workload probes (ratings and response times) 

• ATC NASA TLX (individual subscales and overall) 

• Subjective questions on: 

– Difficulty of interaction between ATC and UAS operator

– Acceptability of final flight path 

– Acceptability of traffic reroutes

– Difficulty for ATC to maintain flow and separation with UAS present 

– ATC level of delegation preference 
23 

3.1 Loss of Separation 

Loss of separation data was analyzed by looking at the closest horizontal and vertical distances 

intruding aircraft reached in regards to the UAS per trial. A loss of separation was determined to 

have occurred when the minimum distances reached between the UAS and the intruder were within 

a distance of 5 nm horizontal and 1,000 ft vertical. 

A 2x2 repeated measures ANOVA was used to determine if there were any significant differences 

between the level of delegation and the level of information shown on the traffic display. Results 

showed no significant differences between the basic and conflict detection present CSD condition, 

F(1, 12) = .274, p = .610, d= -0.327; no significant differences between the extended and full levels 

of delegation, F(1,12) = .085, p = .776, d= 0.111; and no significant interaction between the CSD 

and delegation conditions was found, F(1, 12) = .316, p =.584. For mean losses of separation and 

standard error per condition, see Table 1. 

Table 1. Loss of Separation Means and Standard Errors 

Condition Losses of Separation 

Mean S.E 

Basic CSD, Extended Delegation 0.462 0.215 

Basic CSD, Full Delegation 0.308 0.133 

Conflict Detection CSD, Extended 

Delegation 0.462 0.183 

Conflict Detection CSD, Full Delegation 0.538 0.268 

3.2 UAS Operator Workload Probes 

UAS operator workload probes were analyzed by their ratings and by the response times of the 

pilots. The workload ratings were collected on a scale of one (low) to seven (high). When analyzing 

probe ratings, if the pilot did not respond to the probe within the allotted time span of three minutes, 

the blank scores were adjusted to a score of seven (high). When analyzing probe response times, 

time outs (participant did not answer probe before the next probe was presented) were not included 

in the analysis. 

3.2.1 Probe Ratings 

Results showed no significant differences in the ratings between the basic and conflict detection 

present CSD condition F(1, 12) = 3.390, p = .090, d= -1.107, though ratings did trend towards 

significance with conflict detection present condition (M = 4.012, SE = .247) having higher 

workload ratings than the basic condition (M = 3.527, SE = .218). No significant differences were 

found between the extended and full levels of delegation, F(1,12) = 1.880, p = .195, d= 0.632; and 

no significant interaction between CSD and delegation conditions was found, F(1, 12) = .282, p = 

.605. Across all conditions, the overall workload probe scores were just below the average rating of 

four (M = 3.77, SD = 1.65). 
24 

3.2.2 Response Times 

Results showed no significant differences in the response times to workload probes between the 

basic and conflict detection present CSD condition F(1, 12) = .015, p = .906, d= -2.105; no 

significant differences were found between the extended and full levels of delegation, F(1,12) = 

.195, p = .667, d= 7.314; and no significant interaction between CSD and delegation conditions was 

found, F(1, 12) = 1.305, p = .276. 

3.3 UAS Operator NASA TLX 

The NASA TLX is a self-assessed workload measure that includes ratings on six dimensions; 

mental, physical, temporal, performance, effort and frustration (Hart, & Staveland, 1988). The 

overall and individual subscales of the NASA TLX were analyzed. 

3.3.1 Overall Workload 

Results showed no significant differences in the overall workload ratings between the basic and 

conflict detection present CSD conditions F(1, 12) = .003, p = .960, d= 0.162. No significant 

differences were found between the extended and full levels of delegation, F(1,12) = 3.408, p = 

.090, d= 2.748, though results trended towards significance with the extended delegation condition 

(M = 3.122, SE = .228) having higher workload ratings than the full delegation condition (M = 

2.782, SE = .211). No significant interaction was found between CSD and delegation conditions, 

F(1, 12) = .041, p =.842. While no significant differences were found for overall workload, the 

scores were on average low; M = 2.95, SE = .199 (Figure 14). 

Figure 14. UAS operator overall workload NASA TLX. 
25 

3.3.2 Individual Subscale 

The NASA TLX is separated into six individual subscales: mental, physical, temporal, performance, 

effort and frustration. The level of delegation had a significant effect on temporal workload, F(1, 12) 

= 10.958, p = .006, d= 1.634, with full delegation (M = 2.962, SE = .302) having significantly lower 

workload scores than extended delegation (M = 3.808, SE = .292). See Figure 15. 

Figure 15. Mean UAS operator temporal workload. 

The effect of the level of delegation approached significance for frustration, F(1, 12) = 3.770, p = 

.076, d= 1.125, with full delegation (M = 2.038, SE = .302) having lower workload scores than 

extended delegation (M = 2.696, SE = .440). All other subscale results were not significant (see 

Table 2). 

Table 2. UAS Operator NASA TLX Significance Summary Table 

 NASA TLX Subscale 

 Mental Physical Temporal Performance 

Degradation Effort Frustration 

CSD p = .947 p = .488 p = .719 p = .715 p = .802 p = .706 

Delegation p = .157 p = .137 p = .006 p = .730 p = .279 p = .076 

CSD* 

delegation p = .895 p = .273 p = .656 p = .750 p = .899 p = .917 

While the ratings for mental, physical, effort and performance degradation were not found to be 

significant, the workload scores were found to be on average low to normal. Mental (M = 4.13, SE = 

.256), physical (M = 1.36, SE = .299), effort (M = 4.09, SE = .269) and performance degradation (M

= 2.36, SE = .353) respectively. 
26 

3.4 UAS Operator Situation Awareness Probes 

UAS operator situation awareness probes were analyzed for response accuracy and response time 

using 2x2 within subjects design ANOVAs. When analyzing probe response times, time outs 

(participant did not answer a probe question before the next probe was presented) were removed 

from analysis. 

3.4.1 Probe Accuracy 

There were six probes presented to the UAS operator in each trial. Results were coded by accuracy 

with a score of one given to accurate responses, and a score of zero given to incorrect responses. The 

total amount of accurate responses per trial was then used in analysis. 

Results showed highly significant differences in probe accuracy between the basic and conflict 

detection present CSD conditions F(1, 12) = 452.107, p < .001, d= -11.62, with the conflict detection 

present condition (M = 4.423, SE = .178) having significantly higher accuracy ratings than the basic 

condition (M = .840, SE = .026). See Figure 16. 

Figure 16. Situation awareness probe accuracy. 

No significant differences were found between the extended and full levels of delegation, F(1,12) = 

.082, p = .779, d= -0.146; and no significant interaction between CSD and delegation conditions was 

found, F(1, 12) = .033, p =.859. 

3.4.2 Probe Response Time 

Results showed no significant differences in the response times between the basic and conflict 

detection present CSD condition F(1, 12) = 3.851, p = .073, d= -0.618, although response times did 

approach significance with the conflict detection present mode (M = 25.994 seconds, SE = 3.889) 

having quicker response times than the basic mode (M = 33.631, SE = 5.144). No significant 

differences were found between the extended and full levels of delegation, F(1,12) = .167, p = .690, 
27 

d= 0.891; and no significant interaction between CSD and delegation conditions was found, F(1, 12) 

= .136, p =.719. 

3.5 UAS Operator Situation Awareness Questionnaires 

As the experimental blocks were based on CSD condition, situation awareness questionnaires were 

given out after each block on a scale from 0 (low) to 7 (high). In each block (basic or conflict 

detection present), participants performed one trial each of extended and full delegation. Situation 

awareness scores were analyzed using a 2x2 repeated measures design examining the effect of the 

CSD condition and the order of CSD presentation (condition presented first or second). 

Results showed no significant differences were found between the CSD condition, F(1,5) = .649, p = 

.457, d= 1.17; and no significant interaction between order presented and CSD conditions was 

found, F(1, 5) = .361, p =.574. Significant differences in pilot perceived situation awareness 

between the order of CSD presentation F(1, 5) = 9.826, p = .026, d= -0.444 were found, with the 

first experimental block presented (M = 5, SE = .302) having significantly lower perceived situation 

awareness ratings than the second experimental block presented (M = 5.241, SE = .290). This is not 

a surprising result as the UAS operators had more practice with the simulator by the second 

experimental block, and had better knowledge on procedures associated with the use of the CSD and 

delegated separation. See Figure 17. 

Figure 17. Post-block situation awareness ratings. 

3.6 Air Traffic Controller Workload Probes 

Air traffic controller workload probes were analyzed by their ratings and by the response times of 

the ATC per trial. The workload ratings were collected on a scale of one (low) to nine (high). All 

workload probes were responded to within their allotted time span, so no adjustments were made on 

the scores, and no response times were discarded during analysis. It is of note to point out that since 
28 

there were only two air traffic controllers in this experiment, each controlling for roughly half the 

UAS operator participants, the analyses have low statistical power. 

3.6.1 Probe Ratings 

Results showed no significant differences in the ratings between the basic and conflict detection 

present CSD conditions F(1, 1) = .132, p = .778, d= 0.023. No significant differences were found 

between the extended and full levels of delegation, F(1,1) = 1, p = .50, d= 0.132. No significant 

interaction between the CSD and delegation conditions was found, F(1, 1) = 5.183, p = .263. While 

workload probe ratings were not found to be significantly different, the overall workload ratings 

were manageable (M = 5.913, SE = .220) and on par with typical workload probe ratings from 

previous studies performed in the FDDRL with air traffic controller participants (Johnson el al., 

2012; Battiste et al. 2012). See Figure 18. 

Figure 18. ATC workload probe ratings. 

3.6.2 Probe Response Times 

Results showed no significant differences in the response times between the basic and conflict 

detection present CSD conditions F(1, 1) = 3.817, p = .301, d= 0.212. Results showed no significant 

differences in the response times for the levels of delegation F(1, 1) = .037, p = .879. No significant 

interaction between the CSD and delegation conditions was found, F(1, 1) = .011, p =.933. 

3.7 Air Traffic Controller NASA TLX 

The NASA TLX is a self-assessed workload measure that includes ratings on six dimensions; 

mental, physical, temporal, performance, effort and frustration (Hart, & Staveland, 1988). The 

overall and individual subscales were analyzed. It is of note to point out that since there were only 

two air traffic controllers in this experiment, each controlling for roughly half the UAS operator 

participants, the analyses have low statistical power. 
29 

3.7.1 Overall Workload 

Results showed no significant differences in the overall workload ratings between the basic and 

conflict detection present CSD conditions F(1, 1) = 29.388, p = .116, d= -0.2. No significant 

differences were found between the extended and full levels of delegation, F(1,1) = 25.733, p = 

.124. No significant interaction between the CSD and delegation conditions was found, F(1, 1) = 

.801, p = .535. While NASA TLX workload ratings were not found to be significantly different, the 

overall workload ratings were manageable (M = 4.788, SE = .260). See Figure 19. 

Figure 19. ATC overall workload NASA TLX. 

3.7.2 Individual Subscale 

The effect of CSD condition approached significance for temporal workload, F(1, 1) = 64, p = .079, 

d= -0.195, with the basic condition (M = 4.982, SE = .232) having lower workload scores than the 

conflict detection present condition (M = 5.077, SE = .244). All other subscale results were found to 

be not significant (see Table 3). 

Table 3. ATC NASA TLX Significance Summary Table 

 NASA TLX Subscale 

 Mental Physical Temporal Performance 

Degradation Effort Frustration 

CSD p = .126 p = .500 p = .079 p = .830 p = .357 p = .677 

Delegation p = .304 p = .500 p = .179 p = .170 p = .390 p = .279 

CSD* 

delegation p = .667 p = .500 p = .578 p = .371 p = .637 p = .616 

While the ratings for mental, physical, temporal, performance degradation, effort and frustration 

were not found to be significant, the workload scores were found to be on average normal to 
30 

manageable. Mental (M = 5.76, SE = .467), physical (M = 4.02, SE = 1.02), temporal (M = 5.03, SE 

= .238), performance degradation (M = 4.15, SE = .179), effort (M = 5.27, SE = .63) and frustration 

(M = 4.49, SE = .05) respectively. 

3.8 Subjective Questions 

The UAS operators and pseudo air traffic controllers were given post-trial questions pertaining to 

their overall workload, acceptability of reroutes, acceptability of final flight path, and the workload 

associated with their interactions with each other (UAS operator to pseudo controller) on a scale 

from zero (low) to seven (high). Ratings were analyzed using 2 (CSD condition) x 2 (level of 

delegation) ANOVAs. 

Additional questions on UAS operator and pseudo-controller workload, situation awareness, and 

acceptability and preference ratings were given to the participants at the end of the simulation. The 

post-simulation subjective questions were given on a scale from 0 (easy or acceptable) to 7 (hard or 

unacceptable). As the post-simulation questionnaires were collected at the end of the day after all 

trials were completed, analyses could not be performed to determine the effect of CSD condition and 

level of delegation experienced in each trial. The UAS operator responses are provided with the 

percentage of participants who rated that answer provided. 

3.8.1 UAS Operator Subjective Post-Trial Questions 

Level of delegation was seen to have a significant effect on the difficulty to interact with air traffic 

control, F(1, 12) = 6.789, p = .023, d= 2.369, with full delegation (M = .577, SE = .195) having 

significantly lower difficulty scores than extended delegation (M = 1.808, SE = .469). While there is 

a significant difference in the mean, it is of note that both conditions have relatively low difficulty 

ratings. See Figure 20. 

Figure 20. UAS operator difficulty interacting with ATC. 

 
31 

All other UAS operator post-trial subjective question results were found to be not significant (see 

Table 4). 

Table 4. UAS Operator Subjective Questions Significance Summary Table 

UAS Operator Subjective Questions 

Difficulty to 

Interact with ATC 

Acceptability of 

Traffic Reroutes 

Acceptability of 

Final Flight Path 

Overall 

Workload 

CSD p = .201 p = .797 p = .517 p = .814 

Delegation p = .023 p = .235 p = .337 p = .119 

CSD* Delegation p = .487 p = .919 p = .645 p = .897 

While the ratings for both the acceptability of the UAS operators’ final flight path, and the 

acceptability of their reroutes to avoid traffic conflicts were not found to be significant, this was due 

to overall ratings of acceptability; M = 2.59, SE = .30 and M = 2.63, SE = .28 respectively. 

3.8.2 UAS Operator Post-Simulation Subjective Questions 

UAS operator post-simulation ratings are discussed by question type: perceived safety, workload 

associated with UAS, CSD usefulness and ease of use, CSD workload and willingness to use, and 

the amount of information displayed on the CSD. The tables show the responses given by the UAS 

operators and the percentages of participants who gave that response. 

While the UAS operators were mixed on their opinions of flying in a shared environment with UAS 

in the current NAS, over 60% of the participants were willing to share the airspace with properly 

trained unmanned vehicle operators. The importance of training is further emphasized with all 

participants rating that they gained more confidence operating the UAS and working in delegated 

separation conditions as they gained more practice. See Table 5 for all safety ratings and response 

percentages (rounded to the tenth). 

The post-simulation workload ratings reflect those collected via probes and the NASA TLX with 

overall ratings found to be low to manageable. A trend in level of delegation can be seen in the postsimulation ratings that match with other workload measures; full delegation was found to create 

lower UAS operator workload than extended delegation. See Table 6 for all workload ratings and 

response percentages (rounded to the tenth). 

The willingness to use CSD ratings shows that the UAS operators are more willing to fly a UAS in 

the NAS when using the CSD, with about 76% of the pilots being unwilling to fly a UAS without 

one. UAS operators are also more willing to fly a UAS in the NAS when conflict detection is present 

on the CSD. See Table 7 for CSD use ratings and response percentages (rounded to the tenth). 

 
32 

Table 5. UAS Operator Post-Simulation Perceived Safety Ratings 

Subjective Question Response Percentages 

I believe flying in a shared environment 

with UAS would be acceptable in the 

current environment. 

7.7% Strongly Disagree 

23.1% Somewhat Disagree 

15.4% Neither Disagree nor Agree 

30.8% Somewhat Agree 

23.1% Strongly Agree 

With appropriate training for UAS pilots, I 

would be willing to fly in an airspace that 

included UAS. 

23.1% Somewhat Disagree 

15.4% Neither Disagree nor Agree 

53.9% Somewhat Agree 

7.7% Strongly Agree 

I gained comfort with my ability to respond 

to the reroute orders as I gained more 

practice. 

15.4% Somewhat Agree 

84.6% Strongly Agree 

I gained comfort with my ability to operate 

the UAS under the extended delegated 

separation as I gained more practice. 

69.2% Somewhat Agree 

30.8% Strongly Agree 

I gained comfort with my ability to operate 

the UAS under full delegated separation as I 

gained more practice. 

38.5% Somewhat Agree 

61.5% Strongly Agree 

Table 6. UAS Operator Post-Simulation Workload Ratings 

Subjective Question Response Percentages 

I believe the overall WL associated with the 

concept of UAS in the NAS is manageable. 

7.7% Strongly Disagree 

7.7% Somewhat Disagree 

23.1% Neither Disagree nor Agree 

30.8% Somewhat Agree 

30.8% Strongly Agree 

Flying a UAS in extended delegated 

separation would not be possible due to 

high WL. 

7.7% Strongly Disagree 

46.2% Somewhat Disagree 

7.7% Neither Disagree nor Agree 

23.1% Somewhat Agree 

15.4% Strongly Agree 

Flying a UAS in full delegated separation 

would not be possible due to high WL. 

30.8% Strongly Disagree 

15.4% Somewhat Disagree 

15.4% Neither Disagree nor Agree 

23.1% Somewhat Agree 

15.4% Strongly Agree 

I was unable to successfully perform my 

mission due to the high WL associated with 

the rerouting events. 

23.1% Strongly Disagree 

30.8% Somewhat Disagree 

46.2% Somewhat Agree 

I was unable to successfully perform my 

mission due to the high WL associated with 

avoiding conflicts. 

23.1% Strongly Disagree 

15.4% Somewhat Disagree 

30.8% Neither Disagree nor Agree 

23.1% Somewhat Agree 

7.7% Strongly Agree 
33 

Table 7 UAS Operator Willingness to Use CSD Ratings 

Subjective Question Response Percentages 

I would be willing to fly center airspace 

with a UAS without using the CSD. 

53.9% Strongly Disagree 

23.1% Somewhat Disagree 

15.4% Neither Disagree nor Agree 

7.7% Somewhat Agree 

I would be willing to fly center airspace 

with a UAS using the basic mode of the 

CSD. 

15.4% Strongly Disagree 

30.8% Somewhat Disagree 

23.1% Neither Disagree nor Agree 

30.8% Somewhat 

I would be willing to fly center airspace 

with a UAS using the conflict detection 

present mode of the CSD. 

7.7% Strongly Disagree 

7.7% Somewhat Disagree 

23.1% Neither Disagree nor Agree 

23.1% Somewhat Agree 

30.8% Strongly Agree 

7.7% NA 

Use of the CSD will enhance the safety of 

flying UAS in the NAS. 

23.1% Moderate Enhancement 

46.2% Intermediate Enhancement 

30.8% Large Enhancement 

The CSD usability ratings show an overall favorable view towards the use of the CSD. Across all 

participants, the CSD was used to determine reroutes and the information displayed on the CSD in 

both the basic and conflict detection modes was found to be useful. Overall, more participants found 

the conflict detection mode to be very useful when compared to the basic mode. Over 50% of the 

participants found their workload to be lower in the conflict detection mode than in the basic mode. 

See Table 8 for all CSD usability ratings and response percentages (rounded to the tenth). 

Questions were also asked of the UAS operator participants on the information displayed via the 

CSD. Overall, participants found more necessary information displayed in the conflict detection 

present mode than in the basic mode. See Table 9 for CSD information displayed ratings and 

response percentages. 

Open-ended questions were also asked of the UAS operators for what type of information they 

would like to see that was not available in the scenarios presented. The most common responses 

were to have the full flight plan of the surrounding aircraft available for assistance in conflict 

detection and avoidance, and to have an overlay of instrument and approach airways. While not used 

in this experiment in order to create a simulation environment closer to the airspace systems used in 

the current environment, it is worth noting that both of these capabilities are available on the Ames 

3D CSD and have been found to greatly assist in separation assurance. 

 
34 

Table 8. UAS Operator CSD Usability Ratings 

Subjective Question Response Percentages 

Did you use the CSD when determining 

reroutes. 

100% Yes 

How useful was the display for extracting 

airspace information in basic mode. 

15.4% Neither Useless or Useful 

76.9% Somewhat Useful 

7.7% Very Useful 

How useful was the display for extracting 

airspace information in conflict detection 

mode. 

15.4% Somewhat Useless 

7.7% Neither Useless or Useful 

61.5% Somewhat Useful 

23.1% Very Useful 

How useful was the conflict detection mode 

compared to the basic mode in supporting 

interactions with ATC. 

15.4% Somewhat Useless 

38.5% Neither Useless or Useful 

46.2% Somewhat Easy 

How easy was the display to use in the basic 

mode giving the mission requirements. 

15.4% Somewhat Difficult 

38.5% Neither Difficult or Easy 

46.2% Somewhat Easy 

How easy was the display to use with 

conflict detection present given the mission 

requirements. 

7.7% Somewhat Difficult 

38.5% Neither Difficult or Easy 

30.8% Somewhat Easy 

23.1% Very Easy 

What was your WL when using the conflict 

detection present CSD mode vs. the basic 

mode. 

23.1% Somewhat More Difficult 

23.1% Neither Difficult or Easy 

30.8% Somewhat More Easy 

23.1% Much More Easy 

Table 9. UAS Operator CSD Information Displayed Ratings 

Subjective Question Response Percentages 

Was all of the necessary information 

available from the CSD in basic mode. 

15.4% Somewhat Less Available 

30.8% Neither Missing or Available 

46.2% Somewhat Available 

7.7% Completely Available 

Was all of the necessary information 

available from the CSD in the conflict 

detection present mode. 

15.4% Somewhat Less Available 

23.1% Neither Missing or Available 

30.8% Somewhat Available 

30.8% Completely Available 

3.8.3 Air Traffic Controller Post-Trial Subjective Questions 

It is of note to point out that since there were only two air traffic controllers in this experiment, each 

controlling for roughly half the UAS operator participants, the analyses have low statistical power. 

The level of delegation was seen to have a highly significant effect on how acceptable air traffic 

controllers found the UAS operators’ reroutes to be, F(1, 1) = 5041, p = .009, d= 7.723, with the 
35 

extended delegation condition’s reroutes (M = 1.42, SE = .006) being rated as less acceptable to air 

traffic controllers than the full delegation condition (M = 1, SE = .000). See Figure 21. 

Figure 21. ATC reroute acceptability ratings. 

Across all participants and 100% of the trials, both air traffic controllers preferred full delegation to 

extended delegation and current day operations. All other subjective question results were found to 

be not significant (see Table 10). 

Table 10. ATC Subjective Questions Significance Summary Table 

ATC Subjective Questions 

Difficulty to 

Interact with 

UAS Operator 

Acceptability of 

Traffic Reroutes 

Difficulty Meeting Flow 

and Separation 

Requirements with UAS 

Overall 

Workload 

CSC p = .634 p = .732 p = .105 p = .242 

Delegation p = .105 p = .009 p = .630 p = .264 

CSD* 

Delegation p = .830 p = .883 p = .559 p = .500 

While the post-trial air traffic controllers’ subjective ratings were overall not significant, the ratings 

themselves are worth note. In the difficulty to interact with UAS operator, the scores were overall 

acceptable (M = 1.42, SE = .006). The ratings for difficulty in maintaining flow and separation with 

a UAS present in sector were low to manageable (M = 3.49, SE = 2.10). The overall workload scores 

were high, but manageable (M = 5.81, SE = .399). The higher overall workload scores when 

compared to the UAS operators’ ratings (M = 3.98, SE = .29) are not surprising as the controllers 

maintained positive control over their sector while working with the UAS in either extended or full 

delegation. 
36 

3.8.4 Air Traffic Controller Post-Simulation Subjective Questions 

Air traffic controller questionnaire responses are discussed by question type: sector management 

queries, the controllers’ perceived capabilities of the UAS operators, and the perceived airspace 

safety and level of delegation preference. The tables show the responses given by the air traffic 

controllers, and the percentages of trials (13, one per UAS operator) that were given the responses. 

Multiple questions were asked to investigate the air traffic controllers’ experiences performing 

sector management (Table 11). Workload levels involved with managing a sector with an unmanned 

aircraft were found to be on par with or higher than normal current day operations. Meeting flow and 

separation requirements when the UAS operator was in extended delegation was seen to be between 

somewhat difficult and very difficult, on average. Full delegation was seen to be easier for the 

controllers to maintain flow and separation, with over half of the responses being rated as not at all 

difficult. In both extended and full delegation, controllers believed the UAS operator was able to 

maintain separation 92.3% of the time. 

The responses for whether or not the UAS created problems for the controllers’ managing their 

sectors were split, with just over half of the responses saying the UAS did not create problems. 

When asked if the UAS required special handling, 100% of the responses said yes, with just under 

70% of the responses saying that special handling was used between 0 - 25% of the time. Openended questions were asked on strategies used for special handling, with the average response being 

to use altitude restrictions for aircraft descending into the UAS’s flight level. See Table 11 for air 

traffic controller sector management question ratings and response percentages (rounded to the 

tenth). 

The controllers’ overall perception of UAS operator capabilities were on par with their perception of 

manned aircraft pilot capabilities. All UAS operator participants were found to have enough 

knowledge of the airspace and procedures to communicate and respond to the air traffic controllers’ 

instructions. This is not surprising, as all UAS operator participants were required to have at 

minimum a private pilot certificate. See Table 12 for air traffic controller perception of UAS 

operators’ capabilities ratings and response percentages. 

Across all participants, the air traffic controllers preferred full delegation. The controllers rated the 

perceived level of safety in the air transportation system with a UAS present as having on average 

the same level of safety as compared to normal operations. Matching the workload ratings of the air 

traffic controllers, full delegation was perceived as being safer than extended delegation, with 92.3% 

of the trials being considered the same level of safety as normal operations in the current airspace. 

See Table 13 for air traffic controller perceived safety and preference ratings and response 

percentages (rounded to the tenth). 

The 100% preference across participants by the air traffic controllers for full delegation correlates 

with the 92.3% rating of UAS flying under full delegation having the same perceived level of safety 

compared to the controllers’ perceptions of normal current day operations, as well as the better 

controller acceptability ratings of UAS operator reroutes when in full delegation. 

 
37 

Table 11. ATC Sector Management Questions 

Subjective Question Response 

WL managing a sector with a UAS in it 

compared to normal operations with only 

manned aircraft. 

30.8% Neither Higher or Lower 

46.2% Somewhat Higher 

23.1% Much Higher 

Difficulty maintaining flow and separation 

with the UAS in extended delegated 

separation compared to normal operations. 

7.7% Not at all Difficult 

38.5% Somewhat Difficult 

53.9% Very Difficult 

Difficulty maintaining flow and separation 

with the UAS in full delegated separation 

compared to normal operations. 

53.9% Not at all Difficult 

46.2% Somewhat Difficult 

The UAS operator was able to maintain 

separation in extended delegation. 

92.3% Yes 

7.7% No 

The UAS operator was able to maintain 

separation in full delegation. 

92.3% Yes 

7/7% No 

Did the UAS aircraft performance create 

problems for managing your sector? 

46.2% Yes 

53.9% No 

Did the UAS require special handling? If yes, 

what percentage of the scenario time? 

100% Yes 

69.2% 0–25% of time 

30.8% 25–50% of time 

Table 12. ATC Perception of UAS Operators’ Capabilities 

Subjective Question Response 

How immediately did UAS operators 

respond to instructions compared to 

manned pilots. 

7.7% Much Less Immediately 

38.5% Somewhat Less Immediately 

53.9% Same As Manned 

How appropriately did UAS operators 

respond to instructions compared to 

manned pilots. 

7.7% Much Less Appropriate 

15.4% Somewhat Less Appropriate 

76.9% Same as Manned 

Did UAS operators use correct 

terminology when communication, 

compared to manned pilots. 

7.7% Much Less Use 

61.5% Somewhat Less use 

30.8% Same as Manned 

The UAS operators have enough 

knowledge of the airspace and procedures 

to communicate. 

100% Yes 

The UAS operators have enough 

knowledge of the airspace and procedures 

to respond to instructions. 

100% Yes 

 
38 

Table 13. ATC Perceived Safety and Preference Ratings 

Subjective Question Response 

Preferred level of delegation of separation. 100% Full Delegation 

Perceived level of safety of air transportation 

system with UAS in extended delegated 

separation compared to normal operation. 

53.9% Same Level of Safety 

46.2% Somewhat Less Safe 

Perceived level of safety of air transportation 

system with UAS in full delegated separation 

compared to normal operations. 

92.3% Same Level of Safety 

7.7% Somewhat Less Safe 

4.0 DISCUSSION 

The purpose of this thesis was to determine the feasibility of UAS performing delegated separation 

in the NAS. The study was a 2 x 2 within- subjects design, with level of delegation (extended 

delegation vs. full delegation) and traffic display information level (basic—traffic only vs. conflict 

detection present) as independent manipulations. The effects of delegated separation and traffic 

display information level were collected through objective and subjective measures of performance, 

workload and situation awareness. 

4.1 Loss of Separation 

Loss of separation data was analyzed by looking at the closest horizontal and vertical distances 

intruding aircraft reached in regards to the UAS per trial. A loss of separation was determined to 

have occurred when the minimum distances reached between the UAS and the intruder were within 

a distance of 5 nm horizontal and 1,000 ft vertical. In this experiment, there were no significant 

effects on loss of separation; on average there was less than one loss of separation per trial (M = .44, 

SE = .124). 

The low occurrence of losses of separation across the conditions implies that the addition of more 

information on the CSD, and the addition of more responsibility for separation assurance given to 

the UAS operator does not increase the chance of a loss of separation occurring. This finding is 

important for safety issues associated with UAS flying in the NAS as the potential increase of loss of 

separations associated with UAS flying in a manned sector is a main safety concern. While losses of 

separation did occur, this data should be interpreted cautiously as this study contained experimental 

flight paths specifically designed to create conflicts. Further research should be performed to gain a 

more in depth understanding of how a UAS impacts the overall safety, flow and management of 

flight operations in the NAS. 

4.2 UAS Operator Workload 

Workload was collected from the UAS operator in by using online workload probes and through use 

of the NASA TLX. Pilot workload probes were analyzed by their ratings and by the response times 

of the pilots. No significant differences were found between the levels of delegation or the levels of 

information shown on the CSD, though ratings did trend towards significance with the conflict 

detection present condition having higher workload ratings than the basic condition. No significant 

differences were found on response times. 
39 

This trend towards higher workload probe scores on the UAS operator in the conflict detection 

present mode is not surprising as the operators were presented with a higher amount of information 

that they were required to pay attention to, process, make a decision on whether or not to avoid the 

conflict presented, and perform an action based upon that decision. Overall, workload probe scores 

did not surpass the “average,” score of four. This result implies that even with the added workload of 

conflict detection and higher amounts of separation responsibility, pilots were able to perform their 

task without high workload pressure and with comparable response times across all conditions. 

Overall workload scores on the NASA TLX were also not significant, though the level of delegation 

was seen to have a significant effect on the temporal dimension of workload, and the effect of 

frustration approached significance. The data from all other subscales did not differ significantly 

across the four experimental conditions. In both the temporal and frustration dimensions of 

workload, full delegation had lower scores than extended delegation. These findings are supported 

by previous research done on delegated separation that shows a reduction in workload when pilots 

are given full separation responsibilities (Krozel & Mogford, 2001; Lee, et. al., 2003., Johnson & 

Battiste, 2000). 

4.3 UAS Operator Situation Awareness 

Situation awareness measures were collected from the UAS operator in by using online situation 

awareness probes analyzed by accuracy and response times, and through the use of subjective 

questionnaires given post-CSD condition block. 

Probe results showed highly significant differences in accuracy between the basic and conflict 

detection present CSD conditions, with the conflict detection present condition facilitating the UAS 

operator situation awareness with significantly higher accuracy ratings than the basic condition. The 

increase of UAS operator situation awareness with the introduction of the conflict detection mode of 

the CSD is important to note. The lack of an out the window view associated with unmanned 

vehicles has been listed as a safety concern due to the potential for a decreased UAS operator 

situation awareness. The presence of the conflict detection present mode not only increased UAS 

operator situation awareness without significantly increasing UAS operator workload, but was found 

to induce higher UAS operator created reroute acceptability ratings from the air traffic controllers. 

These results show a shared benefit created by the conflict detection present mode of the CSD for 

both the UAS operator and the air traffic controller. 

Subjective questionnaire data collected post experimental block shows significant differences in 

pilot perceived situation awareness between the orders of CSD presentation, with the first 

experimental block presented having significantly lower perceived situation awareness ratings than 

the second experimental block presented. This finding is not surprising as the UAS operators had 

already completed two full trials and multiple training scenarios before they began the second 

experimental block, and had become accustomed to the apparatus and procedures associated with the 

study. 

Probe results show an increase in situation awareness correlating to the increase of information 

being presented on the CSD, with no significant effects on response times or workload. Subjective 

questionnaire results do not show the same significant effect of CSD on ratings, though the data 

trend of higher conflict detection present ratings supports the probe findings. The finding that the 

second experimental block increases the feeling of subjective situation awareness regardless of 
40 

condition is not surprising as the operator has gained better knowledge of how the display works and 

what to look out for by way of conflicts. 

What is interesting is the trend of the conflict detection present CSD mode having a greater increase 

in situation awareness after being presented second, compared to the basic mode being presented 

second (Figure 22). 

Figure 22. UAS operator perceived situation awareness. 

This increase in situation awareness indicates the potential for an effect of training. Pilots may 

become more adept at their task when initially trained on a basic system, and then trained on a more 

advanced system. This finding is supported by Billinghurst et. al (2011), who found a similar effect 

when training participants on basic systems before introducing them to more complex NextGen 

systems. 

These findings suggest that the addition of a conflict detection tool to the CSD has the potential to 

increase situation awareness without negatively affecting workload or pilot response times. 

Moreover, when training pilots on advanced displays concepts, the addition of learning a basic 

version of the display first may foster a more effective and efficient use of the advanced system. 

Further research is required to more accurately determine the effects of flight deck display training 

and content. 

4.4 Air Traffic Controller Workload Probes 

ATC workload probes were analyzed by their ratings and by the response times of the ATC per trial. 

No significant differences were found in workload probe scores, though both ATC had manageable 

workload ratings throughout the scenarios. No significant differences were found in probe response 

times. 

The NASA TLX results showed no significant differences in the overall workload ratings between 

the basic and conflict detection present CSD condition, no significant differences between extended 

delegation and full delegation, and no significant interaction between CSD and delegation 

conditions. While no significant differences were found between the extended and full levels of 
41 

delegation, results showed higher workload ratings in the extended delegation condition than the full 

delegation condition. 

The effect of CSD condition approached significance for temporal workload, with scores being 

lower in the basic condition. No significant differences or interactions were found for mental, 

frustration, effort, physical or performance degradation. While the subscale results were not 

significant, this might have been due to overall manageable ratings that did not change significantly 

throughout the conditions and trended towards lower scores in full delegation. This air traffic 

controller workload trend correlates to the same finding in UAS operator workload; extended 

delegation created a higher overall workload for participants. It is important to note that for both the 

air traffic controller and the UAS operator, extended delegation created an increase on temporal 

workload, significantly for the UAS operator, and an overall trend for higher ratings in this 

delegation condition. These results imply an advantage of full delegation to extended delegation that 

is shared between the air traffic controller and UAS operator. 

4.5 Post-Trial Subjective Questions 

The level of delegation was seen to have a significant effect on the UAS operator’s perceived 

difficulty to interact with air traffic control, as well as to have a correlating trend on the air traffic 

controllers’ perceived difficulty to interact with the UAS operators. For both types of participants, 

extended delegation created higher difficulty scores than full delegation. This is in support of the 

findings that show an increase in workload on both the air traffic controllers and the UAS operators 

when extended delegation was in use. The correlated workload and difficulty ratings show a shared 

advantage between the controllers and the pilots when full delegation is in use. 

The level of delegation was also seen to significantly affect the level of acceptability air traffic 

controllers had towards the UAS operators’ reroutes. When full delegation was in use, the air traffic 

controllers found the UAS operators’ reroutes to be significantly more acceptable. Higher air traffic 

controller acceptability of flight path reroutes is an additional benefit to reduced workload created by 

the use of full delegation. 

While multiple subjective questions were not found to have significant differences, the results are of 

note. For the UAS operators, the ratings for both the acceptability of their final flight path, and the 

acceptability of their reroutes to avoid traffic conflicts were not found to be significant. This was due 

to overall ratings of acceptability on a scale of zero (acceptable) to seven (unacceptable); M = 2.59, 

SE = .30 and M = 2.63, SE = .28 respectively. These ratings show an overall feeling of acceptability 

in flight path modifications from both the air traffic controllers and the UAS operators. 

Interestingly, while the CSD condition did not have a significant effect on how acceptable the UAS 

operators perceived their final flight path to be, the CSD condition was found to have a trend on how 

difficult it was for the air traffic controllers to maintain flow and separation requirements with the 

UAS present. Air traffic controllers found the basic CSD condition to increase the difficulty in 

maintaining flow and separation requirements in their sector than when the conflict detection present 

CSD condition was in use by the UAS operator. This trend was not significant however, and overall 

difficulty ratings for maintaining flow and separation were low to manageable, M = 3.49, SE = 2.10. 

This finding is of note as the conflict detection present condition made it easier for air traffic 

controllers to maintain flow and separation requirements while increasing the UAS operator 

situation awareness, and without significantly increasing UAS operator workload. 
42 

4.6 Post-Simulation Subjective Questions 

While the UAS operators were mixed on their opinions of flying in a shared environment with UAS 

in the current NAS, over 60% of the participants were willing to share the airspace with properly 

trained unmanned vehicle operators. The importance of training is further emphasized with all 

participants rating that they gained more confidence operating the UAS and working in delegated 

separation conditions as they gained more practice. This finding correlates with the increase in UAS 

operator situation awareness found in the second experimental block of scenarios. 

The post-simulation workload ratings for both UAS operators and air traffic controllers were found 

to correlate with probe and NASA TLX ratings with overall scores found to be manageable. A trend 

in level of delegation can be seen in the post-simulation ratings, and correlate with other workload 

measures; full delegation was found to create lower workload than extended delegation. Air traffic 

controller workload levels involved with managing a sector with an unmanned aircraft were rated to 

be on par with or somewhat higher than normal current day operations. Meeting flow and separation 

requirements when the UAS operator was in extended delegation was seen to be between somewhat 

difficult and very difficult, on average. Full delegation was seen to be easier for the controllers to 

maintain flow and separation, with over half of the responses being rated as not at all difficult. The 

workload ratings show a benefit to the use of full delegation that is shared between UAS operators 

and air traffic controllers. 

The responses for whether or not the UAS created problems for the air traffic controllers managing 

their sectors were split, with just over half of the responses saying the UAS did not create any 

problems. When asked if the UAS required special handling, 100% of the responses said yes, with 

just fewer than 70% saying that special handling was used between 0 - 25% of the time. While 

special handling was used a minor amount of the time, it is important to note that air traffic 

controller workload and losses of separation were not found to be significantly affected by the 

conditions. Air traffic controllers mentioned that the special handling they performed, primarily 

altitude restrictions, was the same type of handling they would do for other special use aircraft, such 

as those used in police patrols, and was not out of the ordinary. 

The UAS operators’ willingness to use the CSD ratings shows that they are more willing to fly a 

UAS in the NAS when using the CSD, with about 76% of the pilots being unwilling to fly a UAS 

without one. UAS operators are also more willing to fly a UAS in the NAS when conflict detection 

is present on the CSD, and all participants felt that the addition of a CSD in a UAS ground control 

station would increase the safety of UAS flight in the NAS. These findings are not surprising as the 

conflict detection present mode was found to significantly increase the accuracy ratings for situation 

awareness probes. Situation awareness is a vital aspect in conflict detection and avoidance, arguably 

increasing in importance for UAS operators, as they do not have an out the window view and rely 

more heavily upon their traffic display. 

The UAS operator CSD usability ratings show an overall favorable view towards the use of the 

CSD. Across all operators, the CSD was used to determine reroutes, and the information displayed 

on the CSD in both the basic and conflict detection modes was found to be useful. Overall, UAS 

operators found the conflict detection mode to be more useful when compared to the basic mode, 

which correlates with the increase in situation awareness and an increased willingness to fly a UAS 

ratings, as well as an increase in the air traffic controllers’ acceptability ratings for the UAS 

operators’ reroutes. Additionally, over 50% of the UAS operators found their workload to be easier 
43 

in the conflict detection mode than in the basic mode, a finding that is correlated across workload 

measures collected. 

UAS operators felt that more necessary information was available on the CSD when in conflict 

detection mode. When asked what information they would like to see that was not available in the 

simulation, UAS operators most commonly responded with being able to see the entire flight path of 

the surrounding aircraft and an overlay of instrument and approach airways. While not used in this 

experiment, it is worth noting that both of these capabilities are available on the Ames 3D CSD and 

have been found to assist in separation assurance and increased situation awareness. Further research 

should be performed to investigate if the display of aircraft trajectories and overlays of airways on 

the UAS operators’ CSD has a positive effect on the air traffic controllers’ reroute acceptability and 

difficulty in maintaining flow and separation ratings. 

The air traffic controllers’ overall perception of UAS operator capabilities were on par with their 

perception of manned aircraft pilot capabilities. All UAS operator participants were found to have 

enough knowledge of the airspace and procedures to communicate and respond to the air traffic 

controllers’ instructions. As the UAS operator participants were required to have at minimum a 

private pilot certificate, this was not a surprising finding. Further research should be done to 

investigate the effects of different levels of flight experience, certifications and video game use on 

UAS operation to better define what the requirements will be for UAS operators in the NAS. 

Across all participants, the air traffic controllers preferred full delegation 100% of the time. The 

controllers rated the perceived level of safety in the air transportation system with a UAS present as 

having on average the same level of safety as compared to normal operations. Full delegation was 

perceived as being safer than extended delegation by the air traffic controllers, a finding that is 

correlated with both the controllers’ preference, and the workload ratings of both the controllers and 

the UAS operators. This again suggests that the use of full delegation creates benefits experienced 

by both the UAS operator and the air traffic controller. 

4.7 Comparison of the Current Unmanned Systems Results to Manned Flight 

An important objective measure collected in this experiment was the number of losses of separation 

that occurred per condition within 5 nm horizontal and 1000 ft vertical. While this study found no 

significant differences between conditions, the number of LOS are comparable to other studies. A 

study by Fern, Kenny, Shively & Johnson (2012) measured baseline compliance rates for UAS 

operating in the current airspace system with the same sector and traffic patterns of this experiment, 

though air traffic control was responsible for maintaining separation. Results showed comparable 

rates of overall LOS (M = 0.292, SD = 0.464) to those collected in this study (M = 0.442, SD = 

0.725). Vu et al. (2012) studied delegated separation for manned aircraft using the same separation 

standards. Results from Vu et al. (2012) interestingly showed a higher rate of overall LOS than this 

study (M = 0.625), also with no significant differences in the rate of LOS between the different 

levels of delegation. While a comparable rate of LOS did occur in these studies, these data points 

should be interpreted cautiously as all flight paths were specifically designed to create conflicts. 

Previous studies focusing on manned aircraft performing delegated separation showed a decrease in 

the number of message exchanges and instructions between ATC and pilots (Vu et al., 2012; 

Grimaud, Hoffman & Zeghal, 2001; Zeghal, Grimaud, Hoffman & Rognin, 2001). The same result 

was found in the current study; there were fewer interactions between the UAS operator and ATC 

under delegated separation. The ATC preference for full delegation in this study is also consistent 
44 

with previous findings of controllers rating delegated separation to be an effective and useful tool in 

maintaining flow and separation (Grimaud, Hoffman & Zeghal, 2001). 

The ATC ratings collected in this study further indicated that controllers had an easier time of 

meeting flow and separation requirements. They also reported that UAS operator created reroutes 

were significantly more acceptable under full delegation. These ratings are consistent with previous 

findings in manned flight that report increases in efficiency based on time, distance, fuel 

consumption, straightness of trajectories, and closest point of approach (Grimaud, Hoffman & 

Zeghal, 2001; Lee et al., 2003). Additionally, faster response times from flight crews in full 

delegation to off nominal scenarios have also been found; flight crews were less likely to be passive 

and wait for the air traffic controller to intervene in the event of a loss of separation when ATC is 

not responsible (Prinzel et al., 2011; Johnson, Battiste, Delzell, Holland, Belcher & Jordan, 1997; 

Vu et al., 2012). 

Full delegation was associated with lower UAS operator workload ratings in this experiment. In the 

temporal and frustration dimensions of the NASA TLX, and post-simulation likert scales, full 

delegation resulted in lower workload ratings than extended delegation. These findings are 

consistent with previous research done on delegated separation when manned pilots are given full 

separation responsibilities (Krozel & Mogford, 2001; Lee et al., 2003, Johnson & Battiste, 2000; 

Prinzel et al., 2011). While ATC workload ratings were not found to be significantly different, the 

overall workload ratings were manageable (M = 5.913, SE= .220). They were also similar to typical 

workload probe ratings from previous studies performed in the FDDRL using the same workload 

probe scale with air traffic controller participants (Johnson el al., 2012; Battiste et al., 2012; Vu et 

al., 2012). 

When using the cockpit situation display, UAS operators reported that the conflict detection mode 

was more useful and comprehensive than the basic mode. This correlated with an increase in 

operator SA, higher ATC reroute acceptability ratings, and lower UAS operator post-sim WL 

ratings. These findings are consistent with previous findings of manned pilots having a high 

confidence in using cockpit displays of traffic information (CDTIs) for delegated separation 

(Domino, Tuomey, Mundra & Smith, 2010; Johnson, Battiste, Delzell, Holland, Belcher & Jordan, 

1997). 

4.8 Future Research Considerations 

Future research should be done to determine UAS operator requirements and training requirements, 

in addition to determining the optimal way to present conflict detection alerts. In this experiment, 

alerts were given to the UAS operator as far as 20 minutes out, which caused many UAS operator 

participants to voice frustrations at such early alerts. It is unknown what the proper timing for alerts 

should be for a UAS to successfully avoid a conflict as their flight characteristics (such as speed and 

maneuverability) are different than manned aircraft. Further research should be done, perhaps 

through the use of parametric studies, to investigate the effects of differing flight characteristics on 

alert usage and UAS operator responses. 

A more in-depth look at the conflict data in this experiment may help to shed light on this by an 

additional analysis determining how far out the intruders were before the alerts were given. It would 

also be useful to examine how long it took the UAS operator to respond to the alert, maneuver and 

avoid the conflict. Finally, it would be important to determine the threshold for distance/time that 

allowed the UAS operator to successfully avoiding the conflict and how many conflicts were 
45 

actively vs. passively avoided (active move or passive change in leg of flight path). These data sets 

could provide useful information for future studies to focus on determining what the proper time and 

distance thresholds are for providing UAS operators with alert notifications. The use of trajectory 

based vs. ballistic based conflict information should also be considered. Trajectory based conflict 

information should help to reduce the amount of false alarms, or unnecessary alerts, such as in the 

cases of passive conflict avoidance. 

5.0 Conclusion 

The purpose of this thesis was to determine the feasibility of UAS performing delegated separation 

in the NAS. The results of this study support the feasibility of UAS performing delegated separation 

in the NAS while providing areas in need of further research. Overall, adjusting the level of 

separation responsibility and amount of information available to the UAS operators on the CSD was 

not found to have an adverse effect on performance as shown by the low amounts of losses of 

separation. The results of the study show benefits related to the use of a more advanced CSD with 

conflict detection capabilities, and to the use of full delegation. 

In the more advanced conflict detection present mode of the CSD, pilots had significantly higher 

situation awareness without a significant negative effect on their reaction times or workload levels. 

While there was a trend toward higher workload ratings associated with the conflict detection 

present mode, it is possible that this slight increase could be ameliorated by training the pilot in a 

basic traffic display first. Post simulation workload ratings showed that over 50% of the UAS 

operators found their workload to be lower in the conflict detection mode as well. In addition to the 

increased UAS operator situation awareness and low workload ratings, air traffic controllers were 

more likely to rate the reroutes performed by the UAS operator as being acceptable. Air traffic 

controllers also reported having an easier time performing flow and separation requirements in their 

sector when the conflict detection present mode of the CSD was in use. 

In the full level of delegation, both UAS operators and air traffic controllers benefited from the 

transition of separation responsibility. A decrease in temporal workload, and a trend towards 

reduced frustration was found for both air traffic controllers and UAS operators in full delegation. 

Subjectively, both air traffic controllers and UAS operators reported that the interaction between 

them was easier in full delegation, and air traffic controllers preferred the UAS operator to have full 

delegation responsibilities 100% of the time across all of the trials. 

The use of a more advanced CSD and full delegation responsibilities given to the UAS operator 

were found to create significantly reduced workload, significantly increased situation awareness and 

significantly easier communications between the UAS operator and air traffic controller without 

significantly increasing the amount of losses of separation. 

 
46 

6.0 REFERENCES 

Airspace Operations Laboratory (AOL) (2008). Airspace Operations Laboratory. Homepage 

retrieved July 13, 2012 from http://humansystems.arc.nasa.gov/groups/AOL. 

Austin, R. (2010). Unmanned aircraft systems: UAVs design, development and deployment. United 

Kingdom: John Wiley & Sons Ltd. 

Battiste, V., Lawton, G., Lachter, J., Brandt, S., Koteskey, R., Dao, A.-Q., Kraut, J., Ligda, S., & 

Johnson, W. W. (2012). Comparison of controller and flight deck algorithm performance 

during interval management with dynamic arrival trees (STARS). Paper submitted to the 

30th EAAP Conference. 

Billinghurst, S. S., Morgan, C., Rorie, R. C., Kiken, A., Bacon, L. P., Vu, K.-P. L., Strybel, T. Z. & 

Battiste, V. (2011). Should students learn general air traffic management skills before 

NextGen tools? In Proceedings of the Human Factors and Ergonomics Society Annual 

Conference. 

Bureau of Transportation Statistics. (2012). On-Time performance year to date through July 2012. 

Research and Innovative Technology Administration. Retrieved September 12, 2012 from 

http://www.bts.gov/programs/airline_ 

information/airline_ontime_tables/2012_07/html/table_01.html. 

Department of Defense, UAS ExCom NAS Access Working Group. (2011). Department of defense 

final report to congress on access to national airspace for unmanned aircraft systems. 

Retrieved from http://www.acq.osd.mil/psa/docs/report-to-congress-ana-for-uas.pdf. 

Department of Defense (2001). Unmanned aerial vehicles roadmap, 2002-2025. Office of the 

Secretary of Defense. Department of Defense: Washington, DC, April 2001. 

Domino, D. A., Tuomey, D., Mundra, A., & Smith, A. (2010). Air ground collaboration through 

delegated separation: Application for departures and arrivals. Proceedings of the Integrated 

Communications Navigation and Surveillance (INCS) Conference, G5-1 -G5-17. 

Federal Aviation Administration, Aviation Safety Unmanned Aircraft Program Office AIR-160. 

(2008). Interim operational approval guidance 08-01: Unmanned aircraft systems 

operations in the U. S. national airspace system. Retrieved from: 

http://www.faa.gov/about/initiatives/uas/reg/ media/uas_guidance08-01.pdf. 

Federal Aviation Administration (2001). 14 CFR part 91: General operating and flight rules. 

Retrieved from: http://www.gpo.gov/fdsys/pkg/CFR-2001- title14-vol1/content-detail.html. 

Federal Aviation Administration. (2010). ADS-B broadcast services. Retrieved from: 

http://www.faa.gov/nextgen/ads-b/broadcastservices/. 

Federal Aviation Administration. (2008a). Airline transport pilot and aircraft type rating: Practical 

test standards for airplane. Retrieved from: 

http://www.faa.gov/training_testing/testing/airmen/test_standards/pilot/ media/FAA-S-8081-

5F.pdf. 

Federal Aviation Administration. (2012a). Modernization and reform act of 2012. Retrieved from: 

http://www.faa.gov/about/office_org/headquarters_offices/ 

apl/aatf/legislative_history/media/FAA_Modernization_Reform_Act_2012_ PLAW112publ95.pdf. 
47 

Federal Aviation Administration. (2012b). NextGen implementation plan: March 2012. Retrieved 

from: http://www.faa.gov/nextgen/implementation/plan/. 

Federal Aviation Administration (2008b). Order JO 7110.65U: Air traffic control. Washington DC: 

U.S. Department of Transportation. 

Federal Aviation Administration. (2012c). Operators and Airport Enablers. Retrieved from: 

https://www.faa.gov/nextgen/implementation/media/ 

nextgen_operator_and_airport_enablers.pdf. 

Fern, L. C., Kenny, C. A., Shively, R. J. & Johnson, W. (2012). UAS integration into the NAS: An 

examination of baseline compliance in the current airspace system. Proceedings of the 2012 

Human Factors and Ergonomics Society Meeting. 

Fern, L., & Shively, R. J. (2009). A comparison of varying levels of automation on the supervisory 

control of multiple UASs. In Proceedings of AUVSI’s Unmanned Systems North America 

2009, Washington, DC. 

Galster, S. M., Duley, J. A., Masalonis, A. J., & Parasuraman, R. (2001). Air traffic controller 

performance and workload under mature free flight: Conflict detection and resolution of 

aircraft self-separation. International Journal of Aviation Psychology, 11, 71-93. 

Gawron, V.J. (1998). Human factors issues in the development, evaluation, and operation of 

uninhabited aerial vehicles. In Proceedings of Association for Unmanned Vehicle Systems 

International, 431-438. 

Gertler, J. (2012). U.S. Unmanned aerial systems. Congressional Research Service. Retrieved from: 

http://fpc.state.gov/documents/organization/ 180677.pdf. 

Granada, S., Dao, A. Q., Wong, D., Johnson, W. W., & Battiste, V. (2005). Development and 

integration of a human-centered volumetric cockpit display for distributed air-ground 

operations. In Proceedings of the 12th International Symposium on Aviation Psychology, 

Oklahoma City, OK. 

Grimaud, I., Hoffman, E., & Zeghal,K. (2001). Limited delegation of separation assurance to the 

flight crew. Preparing for the Global Challenges, Aeronautics Days 2001. Hamburg, 

Germany, 29-31. 

Grimaud, Hoffman, Rognin & Zeghal. (2001) Assessing the impact on the flight deck of delegation 

of separation tasks. 20th Digital Avionics Systems Conference, Daytona Beach, FL, October 

2001. 

Hart, S., & Staveland, L. (1988). Development of NASA TLX (task load index): Results of 

empirical and theoretical research. In P. Hancock and N. Meshkati (Eds.), Human Mental 

Workload (pp. 139-183). Elsevier, Amsterdam, 1988. 

Henden, S. (2008, January 7). [Letter announcing SafeRoute ADS-B Solutions receives operational 

approval for use by United Parcel Service (UPS)]. 

Hoffman, E., Zeghal, K., Cloerec, A., Grimaud, I. & Nicolaon, J.-P. (1999). Operational concepts 

for limited delegation of separation assurance to the cockpit. AIAA Guidance, Navigation and 

Control Conference, Portland, Oregon. 

Howell, D. C. (2011). Fundamental statistics for the behavioral sciences. United States of America: 

Cengage Advantage Books. 
48 

Johnson, W. W., Battiste, V. , Delzell, S. , Holland, S. , Belcher, S. A, & Jordan, K. (1997). 

Development and demonstration of a prototype free flight cockpit display of traffic 

information. Proceedings of the 1997 SAE/AIAA World Aviation Congress. 

Johnson, W. W., Battiste, V. (2000). Enabling Cockpit-Based Self-Separation. Retrieved from: 

http://human-factors.arc.nasa.gov/groups/FDDRL/. 

Johnson, W. W., Lachter, J., Brandt, S., Koteskey, R., Dao, A.-Q., Kraut, J., Ligda, S., & Battiste, V. 

(2012). An evaluation of controller and pilot performance, workload and acceptability under 

a nextgen concept for dynamic weather adapted arrival routing. Paper submitted to the 30th 

EAAP Conference. 

Joint Planning and Development Office. (2011). NextGen avionics roadmap. Retrieved from: 

http://www.jpdo.gov/library/20111005_ARM_complete_ LowRes_v2.0.pdf. 

Joint Planning and Development Office. (2010). NextGen integrated work plan. Retrieved from: 

www.jpdo.gov/library/PartnerAgency/IWP_ED.pdf. 

Joint Planning and Development Office. (2012a). NextGen topics: General information. Retrieved 

from: http://www.jpdo.gov/nextgen_topics.asp. 

Joint Planning and Development Office. (2012b). NextGen UAS research, development and 

demonstration roadmap. Retrieved from: 

http://www.jpdo.gov/library/20120315_UAS%20RDandD%20Roadmap.pdf. 

Krozel, J. & Mogford, R. (2001). Free flight literature survey: Human factors research using 

empirical studies. In Proceedings of the Eleventh International Symposium on Aviation 

Psychology. 

Lee, P., Mercer, J. S, Martin, L. H, Prevot, T. , Shelden, S. , Verma, S. , Smith, N. M, Battiste, V. , 

Johnson, W. W, Mogford, R. , & Palmer, E. A. (2003). Free maneuvering, trajectory 

negotiation, and self-spacing concepts in distributed air-ground traffic management. 

Proceedings of the 5th USA/Europe Air Traffic Management Research and Development 

Seminar, Air-Ground Cooperation Track. 

McCarley, J.S. & Wickens, C.D. (2005). Human factors implications of UAVs in the national 

airspace. Retreived from: http://www.humanfactors.illinois. 

edu/Reports&PapersPDFs/TechReport/05-05.pdf. 

Mouloua, M., Gilson, R., & Hancock, P. (2003). Human-centered design of unmanned aerial 

vehicles. Ergonomics in Design, 11, 6-11. 

National Aeronautics and Space Administration (NASA). (2010). Global hawk ED10-0233-22. 

Retrieved from http://www.nasa.gov/centers/dryden/ 

multimedia/imagegallery/Global_Hawk/index.html. 

National Aeronautics and Space Administration (NASA). (2007). Ikhana. Retrieved from 

http://www.nasa.gov/centers/dryden/multimedia/ imagegallery/Ikhana/index.html. 

National Aeronautics and Space Administration (NASA). (2009). Lake Isabella_AMS 11-19-09. 

Retrieved from http://www.nasa.gov/centers/ 

dryden/multimedia/imagegallery/Ikhana/index.html. 

National Aeronautics and Space Administration (NASA). (2007). Wildfire imaging flights by 

NASA’s ikhana UAV conclude. Retrieved from http://www.nasa. 

gov/centers/dryden/news/Features/2007/wildfire_socal_10_07.html. 
49 

National Aeronautics and Space Administration (NASA). (2007). X-45 Unmanned Combat Air 

Vehicle. Retrieved from http://www.nasa.gov/centers/dryden/ multimedia/imagegallery/X45A/index.html. 

National Oceanic and Atmospheric Administration (NOAA). (2012a). NOAA UAS program. 

Retrieved from: http://uas.noaa.gov/. 

National Oceanic and Atmospheric Administration Aircraft Operations Center (NOAA AOC). 

(2012b). Instrument info: Data from expendable probes. Retrieved from: 

http://www.aoc.noaa.gov/instrumentation.htm. 

Prinzel, L.J., Shelton, K.J., Kramer, L.J., Arthur, J.J., Bailey, R.E., Norman, R.M., Ellis, K.K.E., & 

Barmore, B.E. (2011). Flight deck interval management and delegated separation for 

equivalent visual operations. Digital Avionics Systems Conference (DASC), 2011 IEEE/AIAA 

30th. 6B2- 1 - 6B2-14. 

RTCA, inc. (2008). Safety, performance and interoperability requirements document for the in-trail 

procedures in oceanic airspace (ATSA-ITP) application. Retrieved from http://www.rtca.org, 

DO- 312/EUROCAE ED- 159. 

Special Committee 203 (SC-203) RTCA, inc. (2010) Operational services and environmental 

definition (OSED) for unmanned aircraft systems (UAS). Retrieved October 5, 2010, from 

http://www.rtca.org. DO- 320. 

Takayama, L., Ju, W., & Nass, C. (2008). Beyond dirty, dangerous and dull: What everyday people 

think robots should do. In Proccedings of the 3rd ACM/IEEE International Conference on 

Human Robot Interaction, 25-32. 

Teal Group. (2009). World unmanned aerial vehicle systems 2010 market profile & forecast. In Joint 

Planning and Development Office (pp. 1-8). NextGen UAS Research, Development and 

Demonstration Roadmap, 2012. 

Vu, K-P., Strybel, T. Z., Battiste, V., Lachter, J., Dao, A-Q. V., Brandt, S., Ligda, S., Johnson, W. 

(2012). Pilot performance in trajectory-based operations u nder concepts of operation that 

vary separation responsibility across pilots, air traffic controllers, and automation. 

International Journal of Human-Computer Interaction, 28: 107-118. 

Waharte, S. & Trigoni, N. (2012). Supporting search and rescue operations with UAVs. University 

of Oxford. Retrieved from: http://www.cs.ox.ac.uk/files /3198/submission_waharte.pdf. 

Zeghal, K, Grimaud, I., Hoffman, E, & Rognin, L. (2001). Delegation of sequencing operations to 

the flight crew – First quantitative results. Conference on Guidance, Navigation and Control. 

Montreal, Canada. 

Zeghal, K., & Hoffman, E. (2000). Delegation of separation assurance to aircraft: Towards a 

Framework for analysing the different concepts and underlying principles. Presentation at the 

International Council of the Aeronautical Sciences Congress (ICAS), 192.1-192.10