May 2015 

NASA/TM–2015-218763 

NASA Controller Acceptability Study 1(CAS-1) 

Experiment Description and Initial Observations 

James P Chamberlain, Maria C. Consiglio, James R. Comstock Jr., 

Rania W. Ghatas, and César Muñoz 

NASA Langley Research Center, Hampton, Virginia 
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role. 

The NASA STI program operates under the auspices 

of the Agency Chief Information Officer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NTRS Registered and its public interface, the 

NASA Technical Reports Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in both 

non-NASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types: 

• TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase of 

research that present the results of NASA 

Programs and include extensive data or theoretical 

analysis. Includes compilations of significant 

scientific and technical data and information 

deemed to be of continuing reference value. 

NASA counter-part of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations. 

• TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain minimal 

annotation. Does not contain extensive analysis. 

• CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

• CONFERENCE PUBLICATION. 

Collected papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or 

co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest. 

• TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission. 

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services. 

For more information about the NASA STI program, 

see the following: 

• Access the NASA STI program home page at 

http://www.sti.nasa.gov 

• E-mail your question to help@sti.nasa.gov 

• Phone the NASA STI Information Desk at 

757-864-9658 

• Write to: 

NASA STI Information Desk 

Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 
National Aeronautics and 

Space Administration 

Langley Research Center 

Hampton, Virginia 23681-2199 

 

May 2015 

NASA/TM–2015-218763 

NASA Controller Acceptability Study 1(CAS-1) 

Experiment Description and Initial Observations 

James R. Chamberlain, Maria C. Consiglio, James R. Comstock Jr., 

Rania W. Ghatas and César Muñoz 

NASA Langley Research Center, Hampton, Virginia 
Available from: 

NASA STI Program / Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 

Fax: 757-864-6500 

Acknowledgments 

Large research simulation experiments often have many contributors extending well 

beyond the authors on a corresponding research report, and in this regard the CAS-1 

experiment is certainly no exception. CAS-1 required a tremendous amount of background 

preparation and operations support across a wide variety of technical disciplines, and the 

authors are extremely grateful for the assistance of a dedicated and talented research 

team. Anthony Narkawicz, George Hagen and Jason Upchurch of NASA Langley 

Research Center’s Safety Critical Avionics Systems Branch provided valuable insight and 

contributions to the development of the Stratway+ SS guidance algorithm. Many members 

of NASA’s Langley Information Technology Enhanced Services (LITES) contract 

provided extensive support across a wide array of technical areas, including development 

of display concepts, procedures, software, simulation facilities, pilot and controller training 

materials, and flight scenarios; training support for staff pilots and subject controllers; and 

operations support of numerous simulation data collection sessions as well as dry runs and 

dress rehearsals. The authors would especially like to thank the following LITES research 

team members: Pierre Beaudoin, Anna Dehaven, Keith Hoffler, Steve Hylinski, Joel 

Ilboudo, Kristen Mark, Robb Myer, Gaurev Sharma, Jim Sturdy, Dimitrie Tsakpinis, and 

Paul Volk as well as numerous additional support and management personnel. CAS-1’s 

success would not have been possible without the talent, hard work and dedication of these 

research team members. 

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an 

official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics and 

Space Administration. 
Abstract 

This paper describes the Controller Acceptability Study 1 (CAS-1) experiment that was conducted by 

NASA Langley Research Center personnel from January through March 2014 and presents partial CAS-1 

results. CAS-1 employed 14 air traffic controller volunteers as research subjects to assess the viability of 

simulated future unmanned aircraft systems (UAS) operating alongside manned aircraft in moderatedensity, moderate-complexity Class E airspace. These simulated UAS were equipped with a prototype 

pilot-in-the-loop (PITL) Detect and Avoid (DAA) system, specifically the Self-Separation (SS) function 

of such a system based on Stratway+ software to replace the see-and-avoid capabilities of manned aircraft 

pilots. A quantitative CAS-1 objective was to determine horizontal miss distance (HMD) values for SS 

encounters that were most acceptable to air traffic controllers, specifically HMD values that were 

assessed as neither unsafely small nor disruptively large. HMD values between 0.5 and 3.0 nautical miles 

(nmi) were assessed for a wide array of encounter geometries between UAS and manned aircraft. The 

paper includes brief introductory material about DAA systems and their SS functions, followed by 

descriptions of the CAS-1 simulation environment, prototype PITL SS capability, and experiment design, 

and concludes with presentation and discussion of partial CAS-1 data and results. 

CAS-1 results included a total of 1176 controller assessments of all encounter geometries and HMD, and 

showed that a SS HMD value of 1.5 nmi had the highest acceptability rating with a striking degree of 

agreement among the controller volunteers. More generally, most controllers assessed HMD values 

between 1 and 2 nmi to be acceptable across all encounter geometries. There was more variability in 

controller assessments of 0.5 nmi HMD encounters but a significant number of these assessments 

considered this HMD value to be too small and potentially unsafe. HMD values larger than 2 nmi were 

generally assessed as increasingly disruptive to orderly traffic flow. These results should be useful to 

inform the development of operational performance standards for DAA SS functions. For example, it 

may be appropriate for standards to specify that SS functions should always indicate that SS maneuvering 

is necessary for encounters that will result in projected HMD values less than 1 nmi, and never indicate a 

required SS maneuver if the projected HMD is greater than 2 nmi. 

All 14 air traffic controller volunteers were favorably impressed with the PITL SS concept as simulated 

and presented to them, and considered the concept to be viable from an ATC perspective under the 

assumption that acceptable SS HMD values are employed. CAS-1 staff pilots also assessed the concept 

favorably and considered the simulation time valuable in identifying enhancements for future refinements 

of the concept. These controller and pilot assessments were significant since CAS-1 was the first 

sustained opportunity to exercise the Stratway+ instantiation of the PITL SS concept in a realistic 

controller-pilot simulation that was representative of the moderate-complexity, lower-altitude Class E 

airspace that will pose one of the most challenging environments for DAA-equipped UAS. 

Introduction 

As described in references [1], [2], [3] and other documents, a Detect and Avoid (DAA, also known as 

Sense and Avoid or SAA) system is envisioned to have up to two functions: a self-separation (SS) 

function to enable its associated Unmanned Aircraft System (UAS) to stay well clear of other aircraft, and 

a possibly-optional collision avoidance (CA) function to prevent collisions if all other means of separation 

fail. The SS and CA functions are further envisioned to each be comprised of the following subfunctions: Detect, Track, Evaluate, Prioritize, Declare, Determine, Command and Execute (references [1-

4]; see Figure 1). Some of these sub-functions may be performed by automation and some by the UAS 

pilot, depending on the function (SS or CA) and the overall DAA architecture. As described in references 

[2], [4] and subsequently [5], the SS function is envisioned to allocate at least the Determine and 

Command sub-functions to the UAS pilot, consistent with current manned-aircraft see-and-avoid 

1
operations where the pilot in command (PIC) is responsible for determining and commanding appropriate 

avoidance action, if any, based on relevant visual information and ATC communication. The SS subfunctions preceding Determine will collectively provide the UAS pilot with information elements and/or 

decision aids sufficient to replace the missing out-the-window visual information and enable the pilot to 

make safe and efficient SS maneuver decisions. 

Figure 1. DAA Sub-Functions, Thresholds and Volumes 

As shown in Figure 1, the SS and CA functions will each have a defined space/time threshold – a selfseparation threshold (SST) and collision avoidance threshold (CAT), respectively – which will cause the 

function to Declare that avoidance maneuvering may be necessary if an intruder aircraft crosses the 

respective threshold (note that for convenience the DAA community typically references these thresholds 

or boundaries in the singular, i.e., “the SST” or “the CAT” although in practice a uniquely-shaped SST 

and CAT boundary would be computed for each unique intruder encounter geometry, based on the time 

and space parameters defining the respective thresholds). For a representative CA function such as TCAS 

II (references [6] and [7]) an intruder crossing the CAT would result in presentation of visual and aural 

Resolution Advisory (RA) warning declarations to the flight crew (e.g., “Climb, Climb”) with the 

objective of avoiding collision with the intruder, or more precisely, to avoid entry of the intruder into the 

collision volume (CV) around the Ownship. This CV is typically defined as a cylinder centered about the 

Ownship with a radius of 500 feet and height of 200 feet, and the CV penetration rate is considered a 

2
performance measure for the effectiveness of various TCAS II versions and other CA functions. 

Reference [2] proposed the definition of a self-separation volume (SSV) to be used similarly for the SS 

function; that is, the SS function would provide guidance and/or alerts sufficient to enable the PIC to keep 

intruders clear of the SSV, and the SSV penetration rate would be considered a performance measure for 

the SS function (reference [3] subsequently proposed a similar concept but labeled the SSV functional 

boundary as Well Clear Violation (WCV)). 

Reference [2] asserted that the SSV should be large enough to avoid (or minimize): 1) corrective RA 

issuance by TCAS-equipped intruders (or by a TCAS CA system onboard the Ownship); 2) safety 

concerns for air traffic control (ATC) personnel; and 3) undue concern for pilots of proximate see-andavoid aircraft. The SSV should also be small enough to minimize disruptions to traffic flow. 

Determination of minimum and maximum acceptable SSV sizes will inform the design space for required 

DAA surveillance accuracy and for acceptable overall SS function performance. 

In order to minimize TCAS corrective RA issuance, reference [2] proposed a SSV functional shape and 

minimum size based on the TCAS II corrective RA CAT, with the shape and size parameterized by 

threshold values of the TCAS-like variables of modified tau (ModTau), distance modification (DMOD), 

projected horizontal miss distance (HMD) at closest point of approach (CPA), vertical threshold (ZTHR) 

and time to co-altitude (TCOA). This parameterized SSV shape has been codified in software along with 

a predictive algorithm (Stratway+) that can provide SS maneuver guidance to the UAS PIC to keep this 

SSV clear of intruders (references [8] though [11]), but several challenges still remain. Two of the 

challenges were addressed by the Controller Acceptability Study 1 (CAS-1) experiment, conducted in 

Winter-Spring 2014 by NASA Langley Research Center personnel and described in this paper: 

1. The pilot-in-the-loop (PITL) SS concept described in references [2], [4] and [5] had not been tested 

in a human-in-the-loop ( HITL) simulation representative of challenging cur rent-day National 

Airspace System (NAS) operations with actual pilots and air traffic controllers; and 

2. Threshold values of SSV para meters such as HM D that were generally acceptable to ATC 

personnel, both for safety perceptions and for orderly and efficient traffic flow in a realistic ATC 

environment, were unknown. 

The following sections of this paper describe the CAS-1 simulation environment, PITL SS concept 

instantiation in the simulation, and experiment as designed to address these two challenges, followed by 

additional sections to describe some of the CAS-1 results and conclusions. 

Simulation Environment and PITL SS Concept Instantiation 

The CAS-1 simulation environment was based on the Multi Aircraft Control System (MACS) software 

platform. MACS was originally developed by Dr. Tom Prevot and his team at NASA Ames Research 

Center (references [12] and [13]), and this very capable software platform enables HITL simulations of 

realistic air traffic scenarios with many aircraft operating in structured airspace representative of today’s 

NAS. A MACS simulation environment is typically comprised of multiple PC-based workstations 

networked together with each station running an instantiation of MACS and staffed by an operator. The 

operators can be simulation staff, pilots, and/or controllers, and the MACS instantiations can be 

configured to present simulation manager/traffic generator station, aircraft station, or ATC station 

simulations to the respective operators. The simulation manager/traffic generator station initiates all 

aircraft into the simulation via a pre-defined scenario script, enables the transfer of some of these aircraft 

3
to pilots at aircraft stations via script or human operator intervention (other aircraft are auto-flown via predefined flight plans), enables manual or automated removal of aircraft from the scenario (e.g., upon 

landing) and allows for overall scenario control by the operator. The aircraft stations can be set up either 

to present displays/controls of a single aircraft (such as for a PITL study where the pilot is the subject of 

the experiment) or to enable control/monitoring of multiple aircraft. The multi-aircraft station capability 

is used to enable efficient controller-in-the-loop simulations, so that several staff pilots, each at a multiaircraft station, can collectively simulate many participating aircraft in a subject controller’s sector. The 

ATC stations can simulate either a terminal radar approach control (TRACON) station (i.e., STARS, or 

Standard Terminal Automation Replacement System) or an air route traffic control center (ARTCC) en 

route station (i.e., DSR, or Display System Replacement). Multiple ATC sectors can be simulated, so that 

in a controller-in-the-loop study such as CAS-1 the ATC subject would control traffic at an ATC station 

simulating his or her subject sector, and other staff controllers would operate other ATC stations 

simulating the adjacent TRACON and/or ARTCC sectors. 

The MACS version used for CAS-1 was enhanced with additional capabilities developed by NASA 

Langley personnel to enable simulation of DAA/SS-equipped UAS and the PITL SS concept. These 

capabilities included the addition of UAS vehicle models; configurable simulation of command-andcontrol (C2) link delays; Automatic Dependent Surveillance – Broadcast (ADS-B) In/Out surveillance 

modeling at the message level; and prototype SS guidance capability with associated UAS Ground 

Control Station (GCS) pilot displays, based on Stratway+ software (references [8-11]) and modifications 

to the MACS multi-aircraft station capability, respectively. An extensive multi-channel voice 

communication system was also developed, to simulate aircraft-controller party-line communication on 

the subject sector frequency with configurable delay for UAS aircraft, and also to simulate the land-line 

communication channels between adjacent controllers. These additional capabilities, as they were 

configured for CAS-1, are briefly described next. 

A number of UAS vehicle dynamic models were developed by Intelligent Automation, Inc. (IAI) in 2012-

2013 under contract with NASA Langley and incorporated into the MACS version used for CAS-1, and 

three of these models were used in the CAS-1 simulations. These three models approximated the typical 

operational (versus aerodynamic-performance-limited) performance rates and speeds of the General 

Atomics Predator A and Predator B/Reaper and the Northrop-Grumman Global Hawk aircraft when using 

a terrestrial (versus satellite) C2 link. The simulated C2 link delay was set to zero for CAS-1, which also 

more closely simulates the performance of a terrestrial versus satellite C2 link. The voice communication 

delay for CAS-1 was also configured to zero for UAS and all other simulated aircraft, although the actual 

delay was approximately 100-150 milliseconds due to the voice-over-internet-protocol (VoIP) 

implementation of the communication simulation. This is again consistent with typical nominal 

performance of terrestrial C2 links that also carry relayed voice communications. 

The ADS-B In/Out surveillance modeling capability was used in CAS-1 to simulate the Detect and Track 

sub-functions of the UAS SS function. The modeling capability itself is extensive and includes modeling 

at the message level (versus assembled report level) of both 1090 MHz Extended Squitter (1090-ES)- and 

Universal Access Transceiver (UAT)-equipped ADS-B Out aircraft and associated ADS-Rebroadcast 

(ADS-R) spatial-limit and delay modeling, as well as modeling of Traffic Information Service-Broadcast 

(TIS-B) capability (for detection of unequipped intruders) and various levels of 1090 MHz interference 

modeling. However, for CAS-1 the model was configured such that all aircraft were 1090-ES ADS-B 

In/Out-equipped with ideal detection accuracy, no 1090 MHz interference, and unlimited detection range. 

That is, ideal SS Detect and Track sub-function performance was assumed and simulated for CAS-1. 

SS guidance capability was provided to the UAS pilot in CAS-1 by Stratway+ software. The functional 

parameterized shape of the SSV (references [2, 8-11]) was codified within Stratway+ as well as default 

threshold values for the parameters ModTau, DMOD, HMD, ZTHR and TCOA. These default threshold 

4
values were identical to those for TCAS II (reference [7]) but could be overridden via explicit inputs to 

Stratway+, and in fact for CAS-1 the threshold values for DMOD and HMD were manipulated as 

independent variables (IVs) and explicitly input to Stratway+ for all SS encounters. Additional inputs to 

Stratway+ included Ownship position, velocity, winds, and performance rate data (e.g., nominal turn rate, 

from the vehicle model); intruder positions and velocities; and a specified SS look-ahead time (LAT) 

value. Stratway+ used these data to compute conflict prevention bands (“bands”) for heading, airspeed 

and vertical speed. These computed bands, if any, would then be displayed on the pilot’s heading, 

airspeed, and vertical speed indicators, respectively, to show which respective values would cause one or 

more intruders to penetrate the SSV within the specified LAT. LAT was set to 70 seconds for CAS-1. 

Figure 2 shows a photograph of the navigation display (ND) of a MACS multi-aircraft UAS GCS station 

during a simulation with multiple aircraft near the (currently-selected) Ownship, depicted as the bold 

white chevron at the center of the range rings. The view is track-up but for CAS-1 the wind field was 

always assumed zero, so track and heading are the same (165 degrees magnetic in Figure 2) as are 

groundspeed (GS) and true airspeed (TAS) (157 knots in this case). The range is set to 10 nautical miles 

(nmi), indicated by the white “5” next to the half-scale range ring, and the closest intruder laterally is 

shown at the 9 o’clock position and 3 nmi range, westbound but 2800’ above the Ownship and thus not a 

factor (note that MACS colors intruders white if within 500’ vertically of Ownship, and blue or green if 

more than 500’ above or below, respectively). Stratway+ amber heading bands are visible from 174 to 

273 degrees and are caused by the co-altitude eastbound intruder (N5457B) at 2 o’clock and 5 nmi. 

Figure 2. SS Heading Bands Example: Crossing intruder, no conflict (yet) 

5
The Stratway+ heading bands in Figure 2 can be interpreted by the UAS pilot as follows: the current 

heading of 165 degrees is conflict-free from a SS perspective, but the new heading after the planned turn 

at the VELCI waypoint will cause the SSV to (just) be penetrated by an intruder (N5457B in this case) 

within the LAT of 70 seconds. A slight modification to the right turn, for example to only 173 degrees, 

should avoid SSV penetration, but further right turns will cause SSV penetrations, up until a very 

significant right turn to 273 degrees or greater. Note that this very significant right turn, while probably 

not operationally desirable, can still be made at the nominal turn rate from the vehicle performance model 

(approximately 2 degrees per second in this case) and the SSV will not be penetrated, if the turn is 

Commanded within the (configurable) reaction time allocation for the pilot (this reaction time is also an 

input to Stratway+ for use in bands computations and was set to 5 seconds for CAS-1). 

Figure 3. SS Heading Bands Example: Crossing intruder, with conflict after turn at VELCI 

Figure 3 shows a photograph of the ND a short time after the situation in Figure 2 and zoomed to the 5 

nmi scale, after the pilot ignored the heading bands and executed the unmodified turn at VELCI. 

Stratway+ has now “Declared” a SS conflict with N5457B and its chevron is now colored amber to so 

indicate that it will penetrate the SSV within the LAT. A slight left turn to 173 degrees or less will still 

result in passing ahead of the intruder and avoiding the SSV penetration, but as the encounter has 

progressed the option to turn right and go behind is no longer available. The vertical speed band is 

presented on the vertical speed tape of the Primary Flight Display (PFD) and is not shown here, but in this 

example briefly indicated that a high-vertical-rate climb or descent would have also avoided SSV 

penetration, by achieving ZTHR of vertical separation. Stratway+ showed this climb or descent option 

6
becoming unviable via the vertical speed band shortly after the situation shown in Figure 3, as the 

encounter progressed further and the required vertical rate would exceed the vehicle’s performance 

capability. Note that in this example a slight left-turn modification to the flight plan after VELCI is the 

obvious best operational choice for avoiding SSV penetration by the intruder (i.e., “remaining well 

clear”), but a key aspect of the PITL SS concept described here is to present the UAS pilot with all 

available options – left, right, up, down, even a possible speed change – and allocate the SS Determine 

and Command sub-functions (i.e., what action to take, if any) to the pilot and his or her judgment in 

concert with ATC coordination. 

In addition to the Stratway+ bands, UAS pilots in CAS-1 were presented with an alphanumeric display 

which showed a list of all intruders within 15 nmi of the Ownship and on a converging course. An 

example of this display is shown in Figure 4. The top of the alphanumeric display shows the current 

threshold values of ZTHR and HMD if they have been overridden (-999999 otherwise) so in this case 

Stratway+ is presenting bands for a SSV with 2.5 nmi HMD (note that for CAS-1 and all prior and 

subsequent work to date, the HMD and DMOD thresholds were always set equal to each other, so in this 

example the DMOD threshold is also 2.5 nmi). The list shows converging intruders by call sign along 

with their current range, estimated range at CPA and estimated time to CPA. These last two values 

proved useful to UAS pilots in achieving encounters with very precise HMD, and gave them an 

approximate idea of how much time they had left to negotiate a SS maneuver with ATC before SSV 

penetration was inevitable. 

Figure 4. CAS-1 UAS GCS Display of Nearby Converging Intruders 

7
Both UAS and manned aircraft pilots in CAS-1 were also provided with a separate moving map display 

for enhanced situation awareness. An example of this display is shown in Figure 5 with a close-up view 

shown in Figure 6. This example shows the centered Ownship position overlaid on a North-up view of a 

Visual Flight Rules (VFR) Sectional Chart representing the current airspace, but additional options 

included a Track-up view and various satellite imagery or street map underlays. The airspeed tape, 

altitude tape and vertical velocity indicator (VVI) were positioned on the left and right sides of the 

moving map, respectively, as can be seen in Figure 5, and a heading indicator rose was drawn around the 

Ownship position as can be seen more clearly in Figure 6. Intruder aircraft were also depicted on the 

moving map, such as the intruder shown near the upper right corner of Figure 6 over Rockwall Airport, 

and such depictions were useful if a controller issued a traffic advisory relative to a ground feature such as 

an airport or VFR checkpoint (e.g., “Traffic ... southeast-bound over Rockwall”). In the example shown 

the Ownship was not in a current or potential SS conflict with any intruders and no Stratway+ bands were 

present, but in cases where bands were present they would be redundantly displayed on the moving map’s 

airspeed tape, VVI and heading indicator rose as well as on the MACS PFD and ND, again for enhanced 

pilot situation awareness. 

Figure 5. CAS-1 Moving Map Display for UAS and Manned Aircraft Pilots 

8
Figure 6. Close-Up View of Moving Map Display 

The aggregate objective of the PITL SS concept, primarily implemented by the Stratway+ bands but also 

with the converging-intruder alphanumeric display and moving map display as just described, is to restore 

“intelligent sight” (reference [2]) to the UAS pilot who does not have an out-the-window view of traffic 

and to enable that UAS pilot to make SS decisions and maneuvers in a manner comparable to the see-andavoid decisions and maneuvers made by manned aircraft pilots. That is, during normal operations a 

manned aircraft pilot would execute see-and-avoid responsibilities by maintaining visual awareness of the 

traffic situation and exercising judgment based on that sight picture regarding any required ATC 

coordination and aircraft maneuvering to remain well clear of other aircraft. The PITL SS concept aims 

to enable a comparable level of normal-operations awareness and judgment for the UAS pilot, and in this 

sense the bands and other indications (e.g., amber intruder) are not necessarily intended as caution or 

warning alerts, but as SS guidance to enable normal-operations decisions and maneuvers. 

CAS-1 Experiment Description 

The CAS-1 experiment had both an exploratory and a quantitative objective. The exploratory objective 

was to evaluate the capability and viability of the prototype PITL SS concept instantiation, from the 

perspective of both subject air traffic controllers and CAS-1 staff research pilots, when the concept was 

exercised in a realistic, multi-aircraft simulation in light-to-moderate-density, moderate-complexity Class 

E airspace. The quantitative objective was to determine the range of HMD threshold values for the SSV 

9
that was generally acceptable to air traffic controllers in this same Class E airspace. Significant effort was 

expended to generate simulation scenarios of sufficient complexity and traffic density to adequately 

represent today’s NAS environment and to effectively address these two objectives. That is, the research 

staff recognized that a given PITL SS concept and/or SSV HMD threshold might work acceptably in an 

isolated, pair-wise encounter between a UAS and an intruder but fall short when immersed in a more 

realistic environment, and wanted to take sufficient steps to test in that more realistic environment. 

CAS-1 was designed to address the following research questions for its quantitative objective, from the 

perspective of acceptability to air traffic controllers of a future environment with UAS DAA SS 

operations occurring in airspace shared with manned aircraft: 

1. What DAA SS horizontal maneuvers/HMD are too s mall, resulting in issuance of traffic safety 

alerts or controller perceptions of unsafe conditions? 

2. What DAA SS horizontal m aneuvers/HMD are too large (exc essive “well clear” dista nces), 

resulting in behavior the controller would not expect and/or disruptions to traffic flow? 

3. Is there a ran ge of accepta ble DAA SS HMD that can be applied to the developm ent of DAA 

algorithms? Is this range affected by encounter geometry and/or speed differential, and if so, how? 

To address these research questions, 84 simulated SS encounters between a UAS and a manned aircraft 

were constructed with different encounter geometries, HMD, and speed differentials, and these SS 

encounters were then embedded throughout six one-hour simulated background traffic scenarios (14 

encounters per one-hour scenario) representative of light-to-moderate-workload TRACON traffic (both 

Instrument Flight Rules (IFR) and VFR traffic) on a calm-wind, clear-weather day. All of the SS 

encounters were constructed to occur in lower-altitude Class E airspace (most at 3000’ with some at 

4000’ or 6000’). This lower-altitude, Class E TRACON airspace is arguably some of the most 

challenging for a SS function, other than the traffic pattern area in the immediate vicinity of an airport, 

since it has significant complexity with arrivals, departures, over-flights, flight training, etc.; a mix of IFR 

and VFR traffic with some VFR aircraft not receiving air traffic services (i.e., not on the sector frequency 

and not subject to controller instructions); a high incidence of see-and-avoid/SS encounters; and 

significant traffic flow constraints that limit options for SS maneuvers, particularly for those requiring 

large HMD thresholds. 

All of the six one-hour scenarios (“Hours”) containing the SS encounters and background traffic were 

constructed in the DN/AR7 sector of the D10 TRACON. Dallas-Fort Worth International Airport (DFW) 

is the primary airport for the D10 TRACON; the DN/AR7 sector is in the northeast quadrant of D10 and 

handles south-flow traffic to/from satellite airports including Dallas Love Field (DAL), Addison (ADS) 

and McKinney (TKI) as well as other non-towered airports and lower-altitude en route or training flights 

in the sector (see figure 7). Simulated UAS operations included arrivals to and departures from TKI as 

well as overflights throughout the sector, some with a SS encounter and some not. The physical locations 

of the encounters were varied, i.e., on departure, arrival, or en route in different parts of the sector, not 

only to avoid predictability of encounters in the later Hours but also to allow the six Hours to be 

constructed with the encounters embedded into realistic background traffic flows. For experiment control 

all UAS SS encounters were with VFR aircraft not receiving ATC services (i.e., not on the sector 

frequency) so a subject controller could not preemptively and strategically “fix” a SS encounter before it 

had a chance to occur. In most cases these encounters were also designed so that the controller could not 

see it developing far in advance; for example, the VFR intruder might be departing from a non-towered 

field and “pop up” on radar within a couple of minutes of CPA for the encounter, or alternatively might 

turn from a practice area toward the UAS shortly before CPA. 

10
Figure 7. VFR Sectional Chart View – Approximate Location of DN/AR7 Sector 

The CAS-1 experiment data collection simulations were performed from January through March 2014 at 

NASA Langley Research Center’s Stinger Ghaffarian Technologies (SGT) contractor facilities at 130 

Research Drive in Hampton, Virginia. Fourteen retired air traffic control volunteers, all with D10 Eastside training and experience, were recruited to control the simulated traffic scenarios and assess the 

acceptability of the different SS encounters. Most of the controllers had retired within the previous year 

and/or were still active as contract ATC instructors so were familiar with and proficient in D10 

operations. None of the controllers had previous experience controlling UAS in their sectors. These ATC 

volunteers participated in CAS-1 data collection one-at-a-time, each for a two-day session. The first day 

of a session included approximately three hours of classroom and hands-on training and three one-hour 

data collection scenarios, and the second day included the remaining three one-hour scenarios and a 

debrief discussion of approximately one hour. 

During the one-hour scenarios all simulated UAS and manned aircraft receiving ATC services from the 

subject controller were “flown” by instrument-rated pilots as part of the simulation team, and all adjacent 

sector positions or local controller positions at the towered airports within DN/AR7 were staffed by 

retired air traffic controllers, also part of the simulation team. Additional traffic was also visible to the 

controller but not on the sector frequency, including VFR aircraft that were in the sector but not receiving 

services, landing traffic for which the controller had already approved a frequency change to the tower (or 

common traffic advisory frequency for non-towered airports), traffic in adjacent or overlying sectors, and 

traffic overflying the sector (e.g., DFW arrivals); all of this traffic was handled by simulation staffers 

and/or the MACS automation capabilities. 

11
CAS-1 made the following assumptions for the simulation scenarios and the subject controllers were 

briefed accordingly during their training: 

1. A future UAS operating environment, consistent with assumptions in reference [5] and the FAA 

draft UAS C onOps, where appropriately certif ied DAA-equipped UAS would be operationally 

approved for operations under (only ) IFR with the DAA’s PITL SS function replacing manned 

aircraft pilots’ see-and-avoid capability for th e UAS while away from the immediate airport 

environment. Controllers were instructed to assume that the SS reliability was no better and no 

worse than current manned aircraft pilots’ see-and-avoid ability. 

2. Today’s ATC environment with no new UAS-specific operational improvements and only minimal 

changes to procedures and phraseology to accommodate UAS operations in the same airspace used 

by manned aircraft. 

3. Only nominal UAS DAA operations were simulated, i.e., no lost communications, lost C2 link or 

equipment failures were simulated and the communication and control latencies were minimized 

(approximately 100-150 ms communication delay which was the lower lim it of t he simulation 

environment). 

CAS-1 independent variables included the encounter type (3 values: opposite-direction, overtake, or 

crossing), the SSV HMD (6 values: 0.5, 1.0, 1.5, 2.0, 2.5 or 3.0 nmi) and, for the crossing encounter type, 

the relative speed differential (5 values: intruder same speed as the UAS, or 40 or 80 knots slower or 

faster). This combination of independent variables resulted in 42 test conditions: 6 opposite-direction, 6 

overtake and 30 crossing encounter types. A replicate of each test condition was constructed, resulting in 

the 84 simulated encounters previously mentioned. 

The 42 test conditions were randomized in sequence and assigned 14 per hour to scenario Hours 1, 2 and 

3, which were presented to the subject controllers (in different orders, e.g., 123, 231, etc., for each 

subject) on Day 1, so for Day 1 Hours a subject might see one SS encounter with a 1.0 nmi HMD, the 

next with a 3.0 nmi HMD and the next with a 0.5 nmi HMD, etc. For Hours 4, 5 and 6 on Day 2 the 42 

replicate conditions were blocked with the same HMD values for a half-hour at a time, e.g., the first seven 

encounters in Hour 6 would all have 2.5 nmi HMD and the second seven encounters would all have 3.0 

nmi HMD, with the encounter type and speed differential randomized in sequence. This blocking of 

HMD for Hours 4-6 was done to see if workload ratings were affected if, for example, all UAS in the 

sector required a large HMD and thus potentially had a larger aggregate effect on traffic disruption. As 

with Hours 1-3, Hours 4-6 were presented on Day 2 in different orders to different subjects. 

The primary CAS-1 dependent metric was a direct assessment of HMD acceptability by the controller 

subjects immediately after each SS encounter. The assessment was based on a five-point scale as follows: 

1. Much too close; unsafe or potentially so; cause or potential cause for issuance of a traffic alert 

2. Somewhat close, some cause for concern 

3. Neither unsafely close nor disruptively large, did not perceive the encounter to be an issue 

4. Somewhat wide, a bit un expected; might be disru ptive or p otentially disruptive in conge sted 

airspace and/or with high workload 

5. Excessively wide, unexpected; disruptive or potentially disruptive in congested airspace an d/or 

with high workload 

These assessments were elicited verbally and recorded by a retired controller subject matter expert (SME) 

who was part of the CAS-1 research team and sat next to each controller subject during the data collection 

runs. This SME was aware of each scripted SS encounter and was also monitoring the sector frequency 

and other subject controller tasks, so was able to elicit and record each assessment in a timely and nonobtrusive manner. Fractional assessments, e.g., “1.5” were allowed and sometimes assigned by subjects. 

Subjects were also encouraged, workload-permitting, to vocalize why they assigned a particular numeric 

12
rating, particularly if the reason for the rating was non-obvious and/or if there were extenuating 

circumstances or additional remarks about the encounter. The controller SME also recorded any explicit 

action taken (vector, traffic advisory(s), safety alert, adjacent traffic re-route, etc.) by the subject related to 

each encounter, and noted any operational errors, deviations, significant voice communication errors, resequencing and/or other delays or unusual circumstances throughout all of the data runs. Additional 

dependent metrics included workload self-assessments using the Air Traffic Workload Input Technique 

(ATWIT) methodology at fixed intervals during the simulation and questionnaires at the end of each onehour scenario. 

This paper reports results of the direct controller assessments of HMD, partial findings from controller 

comments during the debriefing sessions, and both subject controller and CAS-1 staff evaluations of 

concept viability. These partial CAS-1 results are presented in the next section. 

Partial Results and Discussion 

A total of 1176 assessments of HMD were collected during the CAS-1 experiment across all subject 

controllers and data collection Hours. As expected there was some variability in controllers’ subjective 

assessments of acceptable HMD for SS encounters, particularly for small-HMD encounters, but there was 

a striking degree of agreement across all encounter geometries that a 1.5 nmi HMD was most acceptable 

(i.e., ratings at or near “3”: neither unsafely close nor disruptively large). More generally, most 

controllers found HMD values between 1 and 2 nmi acceptable across all encounter geometries. Figure 8 

shows average controller acceptability ratings for all opposite-direction (OD) encounters and these trends 

are apparent in this figure. 

Figure 8. Controller Acceptability Rating Averages: Opposite Direction Encounters 

All OD encounters had a “geometric CPA” of zero if no SS action was taken, i.e. these were all initially 

collision course geometries requiring the UAS pilot to execute a SS maneuver in order to achieve the 

desired HMD with the respective intruder, and to deviate off course by the full amount of the respective 

HMD. Due to the high closure rates, a relatively large heading change was required for a SS maneuver to 

achieve a large HMD (e.g., 3 nmi), and this required heading change would quickly increase (i.e., heading 

bands would rapidly “grow”) if the pilot did not react soon after the first appearance of bands at the LAT 

13
of 70 seconds. In some large-HMD cases there was not enough time for the UAS pilot to negotiate a SS 

maneuver with the controller before the bands indicated that a heading change of 40 degrees or more was 

required, and in these cases the UAS pilot made the maneuver first and informed the controller as soon as 

practicable afterward. Controllers stated in both their debrief comments and in contemporaneous 

comments immediately after large-HMD encounters that both the magnitude of deviation and the large 

heading changes were factors in their “excessively wide” ratings (i.e., ratings at or near “5”) for these 

encounters. They also noted that such large deviations and heading changes without an initial negotiation 

by the pilot contributed to the disruptiveness of the maneuver. Even in cases where the pilot negotiated 

the SS maneuver first, large-HMD maneuvers frequently required the controller to “point out” the traffic 

to adjacent sector controllers (e.g., when the UAS was near a sector boundary), question the UAS pilot 

about the required extent of his or her SS maneuver, issue traffic advisories to other aircraft in the vicinity 

or even to modify instructions to these aircraft (e.g., arresting a climb or descent through the UAS’s 

altitude or issuing a vector to keep separated from the UAS). 

The three-dimensional plot in Figure 9 shows more detail behind the average rating data shown in Figure 

8. This plot shows the frequency of a given rating for each of the six HMD values, and shows that nearly

all of the 1.5 nmi HMD OD encounters received an acceptable “3” rating. The plot also shows the large 

number of “5” ratings (“excessively wide, unexpected; disruptive”) for 3 nmi HMD OD encounters. 

Figure 9. Plot of Frequency of Rating Responses: Opposite Direction Encounters 

Much of the variability in controller subjective assessments of acceptable HMD occurred at the smaller 

end of the range, as can be seen on the plot in figure 9 for the 0.5 nmi HMD OD encounters. The plot 

shows that the average acceptability rating very near “2” (“somewhat close, some cause for concern”) 

shown in Figure 8 for 0.5 nmi HMD OD encounters is actually comprised of numerous “1” ratings 

(“much too close... unsafe... cause for issuance of a traffic alert”) as well as offsetting “3” ratings. This 

dichotomy was explored in some detail during the debrief sessions with the controllers and represents 

somewhat of a philosophical split in controller opinion. One cohort of controllers stated that if they had 

issued a traffic advisory to a (manned or UAS) pilot about an intruder and the pilot responded with 

“traffic in sight” (or “traffic detected” as the equivalent CAS-1 phrase used by SS-equipped UAS pilots), 

then the controllers felt that they had fulfilled their obligations and that subsequent separation from the 

14
intruder was entirely the pilot’s responsibility (as supported by regulations), unless the pilot subsequently 

reported losing contact with the intruder. Another cohort stated that even if a pilot reported the traffic in 

sight (or detected for UAS) that the controllers still felt an obligation to monitor and intervene if the 

encounter appeared to be developing in an unsafe manner (one controller stated he felt a “moral 

obligation” to ensure safe operations in spite of the pilot’s legal obligation to ensure separation). This 

latter cohort tended to issue more “2” and “1” ratings to 0.5 nmi HMD encounters. Some of these 

controllers noted during the debrief sessions that they found these small-HMD encounters to be not only 

potentially unsafe but also disruptive, since they unnecessarily diverted controller attention away from 

other separation responsibilities elsewhere in his or her sector. It should be noted that nearly all of the 

participating controllers self-rated themselves during their debrief sessions as likely having a somewhat 

higher tolerance for small-HMD encounters than the controller population at large, due to their long 

experience with relatively high-density operations in the D10 TRACON. This subjective self-rating 

would tend to reinforce a conclusion that SS HMD values of 0.5 nmi for OD encounters may be too close 

for at least a subset of the controller population. 

Figures 10 and 11 show acceptability rating averages and frequency of rating responses, respectively, for 

overtake (OV) encounters. These were encounters where a UAS was overtaking a VFR intruder by 40 

knots on a collision (zero geometric CPA) course and deviated by the specified HMD value to stay clear 

of the intruder as it was passed. The results in these two figures show trends that are qualitatively similar 

to the OD encounters, but the small-HMD encounters have fewer “1” ratings and the large-HMD 

encounters have fewer “5” ratings. This phenomenon was explored to some extent in the debrief sessions 

and the most likely explanation is that the overtaking encounters had only a 40 knot closure rate and 

developed much more gradually than did the opposite-direction encounters. This likely led to a better 

comfort factor for controllers during a gradual, small-HMD overtake, and more time to compensate for 

the deviations of large-HMD overtakes. The slower closure rates also typically resulted in smaller, more 

acceptable heading changes to achieve the large-HMD deviations. 

Figure 10. Controller Acceptability Rating Averages: Overtake Encounters 

It should be noted from Figure 10 that there was slightly more rating variability from encounter to 

encounter with the same HMD, versus the OD encounter results shown in Figure 8. That is, the data 

points in Figures 8 and 10 each represent the average ratings across all 14 controllers for each encounter 

with a given HMD, even though those encounters occurred at different places in the sector across the six 

Hours. For example, as shown in Figure 8, the data points for all OD encounters with a 1.5 nmi HMD 

15
basically lie on top of each other, with an average acceptability rating of “3” regardless of where the 

encounter occurred in the sector. In contrast, Figure 10 shows that some 1.5 nmi HMD OV encounters 

had average ratings slightly higher than “3” and others (occurring elsewhere in the sector) had ratings 

slightly lower than “3.” The reason for this slight variance is not immediately clear but is likely due to a 

deviation in one part of the sector being slightly more disruptive than in another part, due to traffic flow 

constraints and proximity to adjacent sector boundaries. Regardless of the reason, the OV encounter 

results in Figures 10 and 11 appear to support the conclusion that a 1-to-2 nmi HMD range is most 

acceptable to controllers. 

Figure 11. Plot of Frequency of Rating Responses: Overtake Encounters 

Crossing (CR) encounters were treated somewhat differently than OD or OV encounters in two ways. 

First, the geometric CPA was non-zero for all CR encounters, so that if no SS action was taken the UAS 

would pass in front of the intruder by the amount of the geometric CPA. CR encounters with zero 

geometric CPA (i.e., collision course geometries) were not used because in almost all cases the obvious 

operational choice would then be to deviate behind the intruder, and technically there is no “well clear” 

requirement when passing behind an aircraft. However, there is a requirement to remain well clear when 

crossing in front of an aircraft, and setting up all CR encounters with non-zero geometric CPAs ensured 

that the SS encounters would result in the UAS passing in front of the intruder. The interactions between 

geometric CPA and SS HMD for CR encounters are explained further in the next paragraphs. 

The second way that CR encounters were treated differently than OD or OV encounters is that the speed 

differential between the UAS and intruder was manipulated as part of the construction of encounter 

geometries. That is, for each of the six SS HMD values, five CR encounters were constructed with the 

UAS either having the same speed as the intruder, or 40 or 80 knots slower or faster. In all CR 

encounters with a SS HMD of 1.5 nmi or less, the encounter was constructed with a geometric CPA equal 

to the HMD, so that no SS maneuver was required to achieve the HMD. For example, for all CR 

encounter cases with 0.5 nmi SS HMD the encounter geometry was set up with a 0.5 nmi geometric CPA, 

regardless of the speed differential between the UAS and intruder. In these cases the UAS would cross in 

16
front of the intruder with 0.5 nmi HMD and with no SS maneuver action required or requested. If the 

subject controller issued a traffic advisory for the crossing traffic (almost always the case for small-HMD 

encounters) the UAS pilot would respond with either “traffic detected, no factor” or just “traffic detected” 

and take no action (unless subsequently issued a traffic alert instruction by the controller). 

Figures 12 and 13 show acceptability rating averages and frequency of rating responses, respectively, for 

CR encounters where the UAS was faster than its intruder. In all CR encounters where the UAS was the 

same speed or faster than its intruder and the HMD was greater than 1.5 nmi, the geometric CPA of the 

encounter was fixed at 1.5 nmi. For example, in the 3 nmi HMD CR encounter data shown in Figures 12 

and 13, the UAS would have crossed in front of the intruder by 1.5 nmi if no SS action was taken, but the 

bands commanded a further turn away from the intruder and the UAS pilot requested this turn to achieve 

a 3 nmi HMD (i.e., a 1.5 nmi deviation from course). 

Figure 12. Controller Acceptability Rating Averages: Crossing Encounters, UA Faster 

The controller acceptability results shown in Figures 12 and 13 for CR encounter HMD are consistent 

with results for OD and OV encounters: HMD values between 1 and 2 nmi appear most acceptable to 

controllers. Figure 13 shows that even when the UAS was faster than its intruder, a significant number of 

“2” or “1” ratings resulted for 0.5 nmi HMD crossing encounters, indicating significant controller 

discomfort with these small HMD values. Figure 13 also shows a lower number of “5” ratings for 3 nmi 

HMD than resulted for OD or OV encounters, but this is likely due to the smaller total path deviations 

required to achieve 3 nmi of HMD in the CR encounters. That is, a 3 nmi HMD for an OD encounter 

would require a full 3 nmi deviation from the UAS flight path, whereas a 3 nmi HMD for a CR encounter 

would only require a 1.5 nmi deviation due to the 1.5 nmi geometric CPA built into the encounter 

geometry. 

17
Figure 13. Plot of Frequency of Rating Responses: Crossing Encounters, UA Faster 

Figures 14 and 15 show acceptability rating averages and frequency of rating responses, respectively, for 

CR encounters where the UAS was slower than its intruder. In these cases the geometric CPA for the 

encounter was set equal to the SS HMD for all values of HMD, so that no SS maneuver action was 

required to achieve HMD in any case. There were two reasons for this decision. First, it was not clear a 

priori how much HMD would be required for controller comfort when passing in front of a faster intruder 

(significantly faster in the 80 knot speed differential case), so the decision was made to explore HMD 

values all the way out to 3 nmi to find this comfort-factor limit. Second, turning away from a much faster 

intruder to increase HMD generally doesn’t work out well from an encounter geometry perspective and 

thus was not attempted when constructing scenarios. In real-world crossing encounters where insufficient 

distance would exist when passing in front of a faster intruder (e.g., a Piper Cub contemplating a cross in 

front of a Boeing 747 with insufficient HMD) the maneuver is (hopefully) just not attempted, and the 

slower aircraft typically orbits until the faster aircraft passes, or passes (above and) behind. 

18
Figure 14. Controller Acceptability Rating Averages: Crossing Encounters, UA Slower 

The controller acceptability results shown in Figures 14 and 15 for small-HMD CR encounters are 

consistent with results for other encounters with small HMD values. That is, a significant number of “2” 

or “1” ratings resulted for 0.5 nmi HMD crossing encounters, indicating significant controller discomfort 

with these small HMD values. Interestingly, most of the “too close” ratings disappeared for CR 

encounters with at least 1 to 1.5 nmi of HMD, even when passing in front of significantly faster intruders. 

Effectively none of the large-HMD encounters received a rating larger than “3” (i.e., no “too wide” 

ratings) because none of these encounters required a maneuver to achieve the large HMD. In other 

words, the UAS and intruder were just two aircraft in the airspace, flying past each other with a large 

HMD but no requested maneuver. The acceptability rating of “3” is appropriate in this case and fits the 

“3” rating description of “...did not perceive the encounter to be an issue.” The few “4” ratings for these 

large-HMD cases were due to misunderstanding of the training on this issue by the first subject, and 

subsequent training was modified to emphasize that encounters requiring no maneuver by the UAS 

should not receive ratings higher than “3” since by definition the non-maneuver is not disruptive.

19
Figure 15. Plot of Frequency of Rating Responses: Crossing Encounters, UA Faster 

As previously stated, the exploratory objective of CAS-1 was to evaluate the capability and viability of 

the prototype PITL SS concept instantiation, from the perspective of both subject air traffic controllers 

and CAS-1 staff research pilots. To that end, the subject controllers were asked during their debrief 

sessions if they thought the concept was viable from their perspective as a controller, with the caveat that 

the final SS HMD thresholds would be set to values that received acceptable (at or near “3”) ratings. The 

results were unanimously positive. Controller responses included the following: 

- “definitely viable” 

- “absolutely viable” 

- “really impressed, way beyond expectations” [from before arriving and seeing the concept] 

- [concept worked] “surprisingly well” 

- “impressed with it” 

- “don’t see any controller having an issue with” [the concept as presented in CAS-1] 

Many of the remarks such as these were volunteered by the controllers during their debrief sessions prior 

to being explicitly asked about concept viability. In hindsight these results are not overly surprising for 

several reasons. First, the PITL SS concept was intentionally designed to operate as similarly as possible 

to the see-and-avoid operations of manned aircraft pilots – from a subject controller’s perspective the only 

significant difference was phraseology, with “traffic detected” being used by UAS pilots instead of 

“traffic in sight” for manned aircraft. Second, controllers quickly realized that, as configured for CAS-1, 

UAS had perfect surveillance capabilities and always “detected” aircraft issued in a traffic advisory, so a 

sense of confidence was likely built that might not be as great in a real-world deployment of SS-equipped 

UAS that don’t always detect all intruders every time. Third, the UAS pilots were part of the CAS-1 staff 

and knew about all of the scripted encounters in advance, so any SS pilot blunders and “surprise factor” 

reactions that might occur in real-world operations were absent for CAS-1. Still, even with these caveats, 

the CAS-1 results provide an important validation of the PITL SS concept as outlined in reference [2] and 

instantiated as described in this paper. 

20
CAS-1 staff pilots and researchers learned a great deal about operating details of the PITL SS concept in a 

real-world Class E airspace environment and these lessons are being applied to the spiral development of 

future concept refinements and instantiations. Two specific lessons are as follows: first, a means to 

reliably inform the pilot of time remaining, if any, to negotiate a SS maneuver with ATC would be highly 

advantageous. That is, Stratway+ bands appear when an intruder is within the LAT of penetrating the 

SSV, but a reliable indication of time remaining until the so-called SS Execution Threshold (SET), i.e., 

threshold beyond which no SS maneuver can avoid SSV penetration, did not exist in the CAS-1 

instantiation of the concept. Second, Stratway+ has no knowledge of intruder intent and this lack of 

knowledge can lead to some false alerts. A particularly troublesome example of these false alerts is the 

case of an aircraft nearby laterally that is descending to an altitude 500’ or 1000’ above the Ownship and 

then leveling. This will cause Stratway+ to provide bands across all 360 degrees of heading and bands on 

the VVI that indicate a descent is necessary; these bands all disappear (except for a “don’t climb” band on 

the VVI for an intruder 500’ above) once the intruding aircraft levels at its new altitude. In many of these 

cases the UAS pilot is aware of the intruder’s intent to level off, either by monitoring the party-line sector 

frequency and/or through explicit communication with the controller. Possible means for conveying this 

information to the Stratway+ SS guidance algorithm are under consideration for future refinements to the 

concept. 

Conclusions 

The data presented in the previous section show that when the UAS PITL SS capability described in this 

paper was simulated in moderate-complexity Class E airspace operations concurrent with light-tomoderate manned aircraft traffic, a SS HMD value of 1.5 nmi had the highest acceptability rating among 

all CAS-1 air traffic controller volunteers across all encounter geometries – opposite-direction, overtake, 

and crossing encounters with a wide range of speed differentials between the UAS and intruder aircraft. 

A total of 1176 controller assessments were obtained across all encounter geometries with a striking 

degree of agreement among the controller volunteers about the acceptability of 1.5 nmi HMD. More 

generally, most controllers assessed HMD values between 1 and 2 nmi to be acceptable (neither unsafely 

close nor disruptively large) across all encounter geometries. There was more variability in controller 

assessments of 0.5 nmi HMD encounters but a significant number of these assessments considered this 

HMD value to be too small and potentially unsafe. HMD values larger than 2 nmi were generally 

assessed as increasingly disruptive to orderly traffic flow. These results should be useful to inform the 

development of operational performance standards for DAA SS functions. For example, it may be 

appropriate for standards to specify that SS functions should always indicate that SS maneuvering is 

necessary for encounters that will result in projected HMD values less than 1 nmi, and never indicate a 

required SS maneuver if the projected HMD is greater than 2 nmi. 

All 14 air traffic controller volunteers were favorably impressed with the PITL SS concept as simulated 

and presented to them, and considered the concept to be viable from an ATC perspective under the 

assumption that acceptable SS HMD values are employed. CAS-1 staff pilots also assessed the concept 

favorably and considered the simulation time valuable in identifying enhancements for future refinements 

of the concept. These controller and pilot assessments were significant since CAS-1 was the first 

sustained opportunity to exercise the Stratway+ instantiation of the PITL SS concept in a realistic 

controller-pilot simulation that was representative of the moderate-complexity, lower-altitude Class E 

airspace that will pose one of the most challenging environments for DAA-equipped UAS. 

21
References 

1. Sense and Avoid (SAA) for Unmanned Aircraft Systems (UAS). Prepared b y: FAA sponsored 

Sense and Avoid Workshop; aka SAA First Caucus Workshop Report. October 9, 2009 

2. Consiglio M., Chamberlain J., Muñoz C., and Hoffler K. Concept of Integration for UAS 

Operations in the NAS. 28th International Congress of the Aer onautical Sciences, Brisbane, 

Australia, 23-28 September 2012 

3. Sense and Avoid (SAA) for Unmanned Aircraft Systems (UAS) Second Caucus Workshop Report. 

Prepared by: FAA sponsored Sense and Avoid Workshop. January 13, 2013 

4. RTCA SC-228 Working Group 1 Document No. WG1-WP02-09 DAA Architecture Scope 

Considerations. Submitted by: Chamberlain J. and Consiglio M. August 12, 2013 

5. RTCA Paper No. 074-14/PMC-1200 Detect and Avoid (DAA) White Paper. Prepared by: SC-228. 

March 18, 2014 

6. Introduction to TCAS II version 7.1. Prepared by U.S. Department of Tran sportation Federal 

Aviation Administration. February 28, 2011 

7. RTCA-DO-185B, Minimum operational perform ance standards for traffic alert and collision 

avoidance system II (TCAS II). Prepared by RTCA SC-147. July 1, 2009 

8. Anthony Narkawicz, César Muñoz, and Gilles Do wek, Provably Correct Conflict Prevention 

Bands Algorithms, Science of Computer Programming, Volume 77, Issues 10-11, pp. 1039-1057, 

September 2012 

9. César Muñoz, Anthony Narkawicz, and Jam es Chamberlain, A TCAS-II Resolution Advisory 

Algorithm, Proceedings of the AIAA Guidance, Navi gation, and Control Conference (G NC), 

AIAA-2013-4622, Boston, Massachusetts, August 2013 

10. Anthony Narkawicz, César Muñoz, Jason Upchurch, James Chamberlain, and María Consiglio, A 

Well-Clear Volume Based on Time to Entry Point, Technical Mem orandum, NASA/TM-2014-

218155, January 2014 

11. César Muñoz, Anthony Narkawicz, James Chamberlain, María Consiglio, and Jason Upchurch, A 

Family of Well-Clear Boundary Models for the Integration of UAS in the NAS, Proceedings of the 

14th AIAA Aviation Technolo gy, Integration, and Operations (ATIO) Conference, AIAA-2014-

2412, Atlanta, Georgia, 2014 

12. Thomas Prevot, Exploring the Many Perspectives of Distributed Air Traffic Management: The 

Multi Aircraft Control System MACS, HCI-02 Proceedings, American Association for Artificial 

Intelligence (AAAI), 2002. 

13. http://humansystems.arc.nasa.gov/groups/aol/technologies/macs.php

22
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Memorandum

 4. TITLE AND SUBTITLE

NASA Controller Acceptability Study 1(CAS-1) Experiment Description 

and Initial Observation 

5a. CONTRACT NUMBER

 6. AUTHOR(S)

Chamberlain, James P.; Consiglio, Maria C.; Comstock, James R., Jr.; 

Ghatas, Rania W.; Munoz, Cesar A.

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20548

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: NASA STI Program (757) 864-9658

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

This paper describes the Controller Acceptability Study 1 (CAS-1) experiment that was conducted by NASA Langley Research Center personnel from January 

through March 2014 and presents partial CAS-1 results. CAS-1 employed 14 air traffic controller volunteers as research subjects to assess the viability of 

simulated future unmanned aircraft systems (UAS) operating alongside manned aircraft in moderate-density, moderate-complexity Class E airspace. These 

simulated UAS were equipped with a prototype pilot-in-the-loop (PITL) Detect and Avoid (DAA) system, specifically the Self-Separation (SS) function of 

such a system based on Stratway+ software to replace the see-and-avoid capabilities of manned aircraft pilots. A quantitative CAS-1 objective was to 

determine horizontal miss distance (HMD) values for SS encounters that were most acceptable to air traffic controllers, specifically HMD values that were 

assessed as neither unsafely small nor disruptively large. HMD values between 0.5 and 3.0 nautical miles (nmi) were assessed for a wide array of encounter 

geometries between UAS and manned aircraft. The paper includes brief introductory material about DAA systems and their SS functions, followed by 

descriptions of the CAS-1 simulation environment, prototype PITL SS capability, and experiment design, and concludes with presentation and discussion of 

partial CAS-1 data and results. 

15. SUBJECT TERMS

Acceptability; Air traffic controllers; Airspace; Controllers; Experiment design

18. NUMBER

 OF 

 PAGES

27

19b. TELEPHONE NUMBER (Include area code)

(757) 864-9658

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

475122.02.07.08.06.03 

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA-TM-2015-218763

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 05 - 2015