March 2016 

NASA/TM-2016-219177 

Parametric Analysis of Surveillance Quality and 

Level and Quality of Intent Information and 

their Impact on Conflict Detection Performance 

Nelson M. Guerreiro, Ricky W. Butler, George E. Hagen, 

Jeffrey M. Maddalon, and Timothy A. Lewis 

Langley Research Center, Hampton, Virginia 
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role. 

The NASA STI program operates under the auspices 

of the Agency Chief Information Officer. It collects, 

organizes, provides for archiving, and disseminates 

NASA’s STI. The NASA STI program provides access 

to the NTRS Registered and its public interface, the 

NASA Technical Reports Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in both 

non-NASA channels and by NASA in the NASA STI 

Report Series, which includes the following report 

types: 

• TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase of 

research that present the results of NASA 

Programs and include extensive data or theoretical 

analysis. Includes compilations of significant 

scientific and technical data and information 

deemed to be of continuing reference value. 

NASA counter-part of peer-reviewed formal 

professional papers but has less stringent 

limitations on manuscript length and extent of 

graphic presentations. 

• TECHNICAL MEMORANDUM. 

Scientific and technical findings that are 

preliminary or of specialized interest, 

e.g., quick release reports, working 

papers, and bibliographies that contain minimal 

annotation. Does not contain extensive analysis. 

• CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

• CONFERENCE PUBLICATION. 

Collected papers from scientific and technical 

conferences, symposia, seminars, or other 

meetings sponsored or 

co-sponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 

technical, or historical information from NASA 

programs, projects, and missions, often 

concerned with subjects having substantial 

public interest. 

• TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission. 

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services. 

For more information about the NASA STI program, 

see the following: 

• Access the NASA STI program home page at 

http://www.sti.nasa.gov 

• E-mail your question to help@sti.nasa.gov 

• Phone the NASA STI Information Desk at 

757-864-9658 

• Write to: 

NASA STI Information Desk 

Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 
National Aeronautics and 

Space Administration 

Langley Research Center 

Hampton, Virginia 23681-2199 

 

March 2016 

NASA/TM–2016-219177 

Parametric Analysis of Surveillance Quality and 

Level and Quality of Intent Information and 

their Impact on Conflict Detection Performance 

Nelson M. Guerreiro, Ricky W. Butler, George E. Hagen, 

Jeffrey M. Maddalon, and Timothy A. Lewis 

Langley Research Center, Hampton, Virginia 
Available from: 

NASA STI Program / Mail Stop 148 

NASA Langley Research Center 

Hampton, VA 23681-2199 

Fax: 757-864-6500 

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not constitute an 

official endorsement, either expressed or implied, of such products or manufacturers by the National Aeronautics and 

Space Administration. 
iii

Table of Contents

Acronyms and Abbreviations................................................................................................................................iv

Nomenclature .........................................................................................................................................................v

I. Introduction .................................................................................................................................................3

II. Background..................................................................................................................................................3

A. Prior Conflict Detection Research ...........................................................................................................4

B. Research Questions..................................................................................................................................5

III. Analysis Model............................................................................................................................................5

A. Simulation Control...................................................................................................................................6

B. Trajectory Prediction ...............................................................................................................................6

C. Conflict Detection....................................................................................................................................7

D. Metrics Collection ...................................................................................................................................7

E. Surveillance Quality ................................................................................................................................7

F. Level and Quality of Intent Information ..................................................................................................9

G. Analysis Parameters...............................................................................................................................10

H. Assumptions and Scope .........................................................................................................................11

IV. Input Data ..................................................................................................................................................12

A. Data Processing .....................................................................................................................................13

Track Data Processing........................................................................................................................13

Intent Information Processing ............................................................................................................15

B. Scenarios................................................................................................................................................17

Scenarios for Analysis Method 1........................................................................................................17

Scenarios for Analysis Method 2........................................................................................................18

V. Metrics.......................................................................................................................................................20

A. Definitions .............................................................................................................................................20

B. Same Conflict Comparison ....................................................................................................................22

VI. Test Matrices.............................................................................................................................................25

A. Surveillance Quality Test Matrix 1 (SQ1) .............................................................................................25

B. Surveillance Quality Test Matrix 2 (SQ2) .............................................................................................25

C. Surveillance Quality Test Matrix 3 (SQ3) .............................................................................................26

D. Level and Quality of Intent Information Test Matrix 1 (IQ1)................................................................26

E. Level and Quality of Intent Information Test Matrix 2 (IQ2)................................................................27

VII. Results and Discussion ..........................................................................................................................28

A. Surveillance Quality Results and Discussion.........................................................................................29

Surveillance Quality Results Tables...................................................................................................31

Surveillance Quality Results Figures .................................................................................................42

B. Level and Quality of Intent Information Results and Discussion ..........................................................50

Level and Quality of Intent Results Tables ........................................................................................52

Level and Quality of Intent Information Results Figures...................................................................59

References............................................................................................................................................................63

Appendix ..............................................................................................................................................................65

A.1. Surveillance Error Modeling..................................................................................................................65

A.2. Some Metrics Issues Associated with Intent..........................................................................................66
iv

Acronyms and Abbreviations

ADS-B - Automatic Dependent Surveillance-Broadcast

BOC - bottom of climb

BOD - bottom of descent

BOT - beginning of turn 

CD - conflict detection

CONUS - Continental United States

CTAS - Center/TRACON Automation System

EOT - end of turn

FA - false alert

FAA - Federal Aviation Administration

FL - flight level

ICAO - International Civil Aviation Organization 

IFR - instrument flight rules

IQ1 - intent information quality test matrix 1

IQ2 - intent information quality test matrix 2

LOI - level of intent

LOS - loss-of-separation

MA - missed alert

MASPS - Minimum Aviation System Performance Standards

MD - missed detection

NACp - Navigation Accuracy Category for position

NACv - Navigation Accuracy Category for velocity

NAS - National Airspace System

PDF - probability density function

QOI - quality of intent

SA - Separation Assurance

SQ1 - surveillance quality test matrix 1

SQ2 - surveillance quality test matrix 2

SQ3 - surveillance quality test matrix 3

SSR - Secondary Surveillance Radar

TC - trajectory change

TOC - top of climb

TOD - top of descent

TRACON - Terminal Radar Approach Control (Facility)

VFR - visual flight rules

WPT - fly-by waypoint
v

Nomenclature

alt = altitude, as a function of time

C = single conflict or conflict set, true

Ĉ = single conflict or conflict set, predicted

dGCD = great-circle distance function

DH = horizontal distance

DV = vertical distance

δt = time tolerance for conflict comparison criteria

δH = horizontal distance tolerance for conflict comparison criteria

δV = vertical distance tolerance for conflict comparison criteria

∆altQ4 = altitude interval for intent quality 4 vertical points

∆tLOS = relative time-to entry into loss-of-separation

∆tLOS,mean = mean time-to loss-of-separation at first detection

∆tQ4 = time interval for intent quality 4 horizontal points

∆ttraj = track data and trajectory prediction time interval

ε = error vector

fGCD = great-circle projection function

IQH = horizontal quality of intent

IQV = vertical quality of intent

Γ = Gamma function

gs = groundspeed, as a function of time

Hsep = horizontal separation criterion

lat = latitudinal position, as a function of time

lon = longitudinal position, as a function of time

NĈ = number of predicted conflicts

NC = number of true conflicts

NC,sim = number of true losses-of-separation in one simulation run

NFA = number of false conflict alerts

NMA = number of missed conflict alerts

NMD = number of loss-of-separation missed detections

PFA = probability of false alert

PMA = probability of missed alert

PMD = probability of missed detection

Rs = surveillance range

s = position state vector

ŝ = position state vector estimate

SQmodel = surveillance quality parameter set

σr = surveillance horizontal position error standard deviation

σa = surveillance altitude error standard deviation

σgs = surveillance groundspeed error standard deviation

σvs = surveillance vertical speed error standard deviation

σtrk = surveillance track angle error standard deviation

t = time

T = true trajectory

T̂ = predicted trajectory

tcd = conflict detection time

Tdet = conflict detection cycle period

Tint = time horizon for intent information
vi

tLOS = time at entry into loss-of-separation

Tpred = time horizon for trajectory prediction and conflict detection

ts = surveillance time

tshift,max = maximum track time shift

tsl = surveillance lag time

trk = track angle, as a function of time

v = velocity state vector

v̂ = velocity state vector estimate

vs = vertical speed, as a function of time

Vsep = vertical separation criterion
1

Abstract

A loss-of-separation (LOS) is said to occur when two aircraft are spatially too close to one another. A 

LOS is the fundamental unsafe event to be avoided in air traffic management and conflict detection (CD) 

is the function that attempts to predict these LOS events. In general, the effectiveness of conflict detection 

relates to the overall safety and performance of an air traffic management concept. An abstract, parametric 

analysis was conducted to investigate the impact of surveillance quality, level of intent information, and 

quality of intent information on conflict detection performance. Surveillance quality was represented by a 

surveillance error model of current and proposed surveillance technologies through a set of position and 

velocity state accuracy standard deviations. Level of intent information was represented by an intent time 

horizon and quality of intent information was represented by various levels of intent data, each providing 

incrementally more information to define the vertical and horizontal components of a kinematic trajectory. 

CD performance was measured using the primary metrics of false alert probability, missed alert probability, 

missed detection probability, and mean time-to-LOS at first detection. Two complementary analysis 

methods were used to conduct the analysis runs; one method used time-shifted, recorded traffic data from 

the National Airspace System (NAS) as the source for scenarios while the second used circular traffic 

scenarios with flight plans randomly-generated using the characteristics observed in the NAS traffic data. 

Three surveillance quality test matrices were implemented to investigate: (1) the CD performance impact 

of a set of surveillance technologies and several variations of conflict detection and trajectory prediction 

parameters, (2) the sensitivity of the performance metrics to each of the surveillance error and trajectory 

prediction and conflict detection parameters, and (3) the variability of the performance metrics to different 

random traffic scenarios and to Monte Carlo sampling. Two level and quality of intent information test 

matrices were implemented to investigate: (1) the CD performance impact of different levels of intent and 

different qualities of intent information with several variations of trajectory prediction and conflict detection 

parameters, (2) the variability of the performance metrics to different random traffic scenarios and the 

sensitivity of the performance metrics to the time and altitude interval parameters used to represent the 

highest quality of intent information modeled. 

The surveillance quality analysis runs quantified the conflict detection performance of some expected 

trends. State-projection trajectory prediction, used in the surveillance quality runs, exhibits poor 

performance, even in the presence of no surveillance error and with a short trajectory prediction and CD 

horizon. As the trajectory prediction and CD horizon increases, the false alert probability, missed alert 

probability, and mean time-to-LOS increase while the missed detection probability decreases. Surveillance 

technologies with high position and velocity state accuracies result in better CD performance than those 

with low position and velocity state accuracies. The sensitivity curves indicate that false alert and missed 

alert probabilities are most sensitive to CD and trajectory prediction horizon and vertical speed errors. 

Missed detection probability is sensitive to most of the parameters tested, with the lowest sensitivity being 

due to vertical position error. The mean time-to-LOS metric is most sensitive to CD cycle period, CD and 

trajectory prediction horizon, and surveillance range. The variability of the CD performance metrics to 

different traffic scenarios with the same input parameters and to Monte Carlo sampling is small based on 

the parameters and scenarios tested.

The conflict detection performance generally improves as the level and quality of intent information is 

increased. The level of intent trends indicate that conflict detection performance is improved as the intent 

horizon approaches the trajectory prediction and CD horizon. When the CD horizon is equal to the intent 

horizon, CD performance is improved by a shorter detection horizon for the metrics of false alerts, missed 

alerts, and missed detections but there is a negative impact on the mean time-to-LOS metric. The use of 

shorter conflict detection cycle periods has a positive impact on CD performance, primarily by reducing 

the missed detection rate. The variability of the CD performance metrics due to different traffic scenarios 

in the presence of intent information is small based on the parameters and scenarios tested.

The data collected in this parametric analysis can be used to estimate the conflict detection performance 

under alternative future scenarios or alternative allocations of the conflict detection function, based on the 

quality of the surveillance and intent data under those conditions. Alternatively, this data could also be 
2

used to estimate the surveillance and intent information quality required to achieve some desired CD 

performance as part of the design of a new separation assurance system. 
3

I. Introduction

Separation Assurance (SA) has been an area of study for government agencies, research institutions, 

and Universities for a few decades [1]. The motivation for that work stems from the need to provide higher 

levels of automation for the separation assurance functions in order to increase the capacity of the National 

Airspace System, while retaining or increasing the level of safety of the system. Some research has 

explored the potential allocation of the separation assurance functions amongst different agents in the 

system, including air traffic controllers, flight crews, and automation systems. In a broader sense, this area 

of research is called Function Allocation.

Function Allocation research is focused on understanding the impacts of allocating the separation 

assurance functions between air and ground agents and between human and automation systems in different 

ways. All separation assurance systems are comprised of two primary functions: conflict detection and 

conflict resolution. However, other functions may also be provided, such as conflict prioritization, conflict 

prevention, complexity avoidance, etc. The performance of a specific allocation of the separation assurance 

functions has typically been studied as a complete system. In this work, the performance impact of only 

one of the primary functions is investigated, namely, the performance of the conflict detection function. 

More specifically, this work is focused on measuring the performance of conflict detection as a function of 

parameters directly related to the air-ground allocation.

The objective of this work was to develop a set of data characterizing the performance of conflict 

detection in a parametric way as a function of the quantity and quality of data available to perform that 

function. Differences in the quantity and quality of data available to the conflict detection function could 

be considered the primary distinguishing features between an airborne or ground-based allocation of that 

function. Thus, the parametric set of data generated by this work can be used to support the analysis of the 

performance of a complete separation assurance function allocation scheme, or to make recommendations 

regarding the quantity and quality of data that should be used to perform conflict detection with some 

required minimum level of performance.

The organization of this document is as follows: Section II provides some background on conflict 

detection and prior work as it relates to the current work. Section III describes the analysis models used to 

collect the parametric performance data and the specifics of some of the modeling elements. Section IV

describes the input data used in the analysis and the method used for creating traffic scenarios. Section V

presents the metrics and definitions used to characterize conflict detection performance. In section VI, the 

analysis test matrices are presented. The results of the analysis runs are presented and discussed in section

VII. Finally, the Appendix describes additional modeling details.

II. Background

Conflict detection refers to the identification or prediction of a future loss-of-separation between two 

aircraft based on the predicted futures states of those aircraft. The conflict detection function is a core 

function of any separation assurance system, as shown in the simplified separation assurance system 

diagram in Figure 1. Its performance has a direct impact on the effectiveness of the conflict resolution 

function. If the CD function provides an alert to a conflict that does not exist (false alert), then the conflict 

resolution function may compute an unnecessary resolution maneuver. If the CD function does not provide 

an alert for a LOS that does exist (missed alert), then safety issues may result. For these reasons, it is 

important to understand the performance characteristics of the conflict detection function independent from 

other separation assurance functions.

The conflict detection function is often tightly integrated with a trajectory prediction function, which 

can be considered to be the largest contributor to CD performance. The trajectory prediction function and 

the CD performance function define the domain for analysis in this work. The inputs to this sub-system 

are the surveillance information for relevant traffic, the intent information that may be available for that 

traffic, and any other sources of information relevant to trajectory prediction, such as wind information or 

aircraft performance parameters. The outputs of this sub-system are the detected conflicts. Figure 1 shows 
4

the input/output diagram and the trajectory prediction and conflict detection relationship in a separation 

assurance system and also highlights the CD performance analysis domain explored here.

An abstract, parametric approach to analyze conflict detection performance aids in the analysis of 

fundamental large-scale system trade studies of air traffic management concepts. Thus, general trends can 

be evaluated without requiring the detail of particular algorithms, communication technologies, etc. This 

type of analysis is particularly appropriate for analyzing different conflict detection function allocations. 

For example, airborne allocation of conflict detection may have very good information about the ownship’s 

current state and intent but little information about the surrounding traffic’s intent. Similarly, an automated 

conflict detection system running on a centralized server may have a good level of intent information about 

all nearby traffic, based on flight plan information, but may have less accurate current state information for 

any single aircraft. Rather than studying only two specific allocation schemes, an abstract, parametric 

analysis enables the estimation of conflict detection performance for many allocation schemes. Data 

interpolation is one possible approach that can be used when the input data characteristics can be cast into 

a parameterization consistent with that used in this analysis.

A. Prior Conflict Detection Research

An extensive body of work exists in the literature with respect to trajectory prediction [2, 3] and conflict 

detection [4-6]. A large portion of prior work is focused on the design of conflict detection algorithms and 

the performance of those implementations in simulation. Additionally, a significant amount of work has 

been done to study human performance in performing separation assurance functions in the face of different 

conflict alerting systems [7, 8]. 

The performance of various conflict detection algorithms (sometimes called alerting algorithms) has

been analyzed in prior work [9-13]. Paglione [9] evaluated the performance of the Federal Aviation 

Administration’s (FAA) Host computer detection algorithms while Bilimoria [10] performed a similar 

evaluation for the NASA Center/TRACON Automation System’s (CTAS) detection algorithms. Prior work 

has also studied conflict detection independent of an algorithm. This research has shown that a conflict 

detection algorithm can be developed that is correct, within a given approximation bound, for an arbitrary 

trajectory [14]. Therefore, if a correct algorithm is used, the performance of the CD function is dominated 

by factors such as trajectory prediction sophistication, alerting buffers, and other heuristics.

The impact of data quality (described in terms of surveillance quality and navigation error) and data 

content have been discussed in prior work [15, 16]. The impact of intent availability has also been studied 

[17-20] in the context of separation assurance. Chung [15] discusses the implication of surveillance delays 

and how these could be used to determine buffers for the trajectory prediction of future states. Finkelsztein

[18] has looked at the sensitivity of the range at closest point of approach in a parametric analysis. 

Figure 1. Simplified separation assurance diagram and the CD performance analysis domain for this work.
5

Our investigation of the prior work in conflict detection performance revealed that more work was

needed to provide a parametric and broadly-applicable characterization of conflict detection performance, 

which was the focus of this work. In much of the prior work, it is difficult to separate the performance of 

the entire SA system, which is typically measured by observed losses-of-separation, from the performance 

of conflict detection alone, which is typically characterized by false and missed alert rates. Many of these 

studies were performed using a conflict resolution algorithm which had the consequence of directly 

impacting the key performance metrics. Also, prior work consisted of algorithm design, human factors 

implications, and full SA system performance, whereas this work is focused on the performance of conflict 

detection as a function of the quality and content of the input data.

B. Research Questions

This study was focused on collecting the data needed to answer two fundamental questions:

1. What are the impacts of surveillance quality on conflict detection performance?

2. What are the impacts of level and quality of intent information on conflict detection 

performance?

Surveillance quality has a direct impact on the accuracy of the flight state estimate available for 

trajectory prediction. Flight state information received from a radar system will have different accuracy 

than that received from an ADS-B system, primarily due to the differences in the sensing and transmission 

mechanisms. To answer question 1, the parametric impact of flight state accuracy on conflict detection was

analyzed in a way that allows for the evaluation of the performance under various surveillance systems and 

data exchange scenarios. 

Flight intent information can have an even more significant impact on trajectory prediction accuracy 

and conflict detection performance. To answer question 2, the impact of the amount of intent information

available for trajectory prediction and the quality of that intent information on conflict detection 

performance was analyzed. The level of intent information is related to the time horizon for which intent 

information is provided while the quality of intent information is related to the amount of information that 

is contained in that intent for the purposes of re-creating a trajectory. Quality of intent information can 

range from simple target-state information to a more detailed kinematic representation of the trajectory.

III. Analysis Model

The conflict detection performance analysis was completed using two analysis methods. The first 

analysis methodology (Method 1) involved the use of recorded traffic tracks time-shifted to create NASwide scenarios with losses-of-separation. Similar analysis approaches with time-shifted recorded data have 

been used in the past by other researchers [21]. The recorded traffic data contains much of the complexity 

that is typically difficult to replicate in a fast-time simulation, such as the airspace structure and routings, 

the varying performance of many aircraft types, and step climbs and descents. The primary disadvantage 

of using real aircraft traces is that losses of separation are rare events in the airspace system. The only 

appropriate modification that can increase the number of LOS events while maintaining the real-world 

nature of this data is temporal shifting. Method 1 leverages this temporal shifting to create losses-ofseparation that did not exist in the as-flown track data as targets for the conflict detection function.

The second analysis methodology (Method 2) uses randomly generated flight plan scenarios over a 

generic volume of airspace. A circular region of airspace is defined and flight plans are randomly generated 

using a set of distributions for parameters such as track angle change, flight segment length, average 

groundspeed, and others, to traverse the circular region. To retain some level of credibility for the flight 

plans randomly generated, the sampling distributions used for the parameters were derived from the 

observed characteristics in the NAS-recorded traffic data. The advantage of using this second analysis 

methodology is that scenarios can be simulated with conditions that can further stress test the conflict 

detection function, such as with geometries or conditions that may not be present in the time-shifted,
6

recorded data. Also, method 2 had shorter scenario execution times that allowed analysis runs to be 

completed more quickly.

The two analysis methods had the same basic functionality: traffic simulation/playback, trajectory 

prediction, conflict detection, and metrics collection. A scenario of traffic was simulated within some 

airspace region (NAS-wide for method 1 and regional for method 2) and conflict detection and trajectory 

prediction was executed at a given interval or period within that simulation. For method 1, the simulation 

was simply a time-shifted play-back of the recorded track data while in method 2 a kinematic trajectory 

generation step was used to simulate the flight plan prescribed in the scenario. During each CD cycle, a 

trajectory prediction was created for each aircraft using the quantity and quality of data specified by a set 

of input parameters, the predictions were compared pair-wise to determine a set of predicted conflicts, the 

true or simulated aircraft tracks were compared pair-wise to determine the set of true conflicts, and, finally, 

the comparison of the true versus predicted conflicts constituted the metrics collection. 

The use of two analysis methods allowed for the cross-validation of results and for balancing of the 

advantages and disadvantages of each method. The assumptions used in both methods were verified 

through a common set of inputs that produced a comparable set of metrics results. The impact of modeling 

differences between the two analysis methods are highlighted throughout this document.

A. Simulation Control

The traffic simulation was implemented as a simple execution loop that advanced each aircraft to the 

appropriate position in the recorded or emulated track data at each simulation time, until all aircraft tracks 

had completed. The simulation step size was determined by the desired conflict detection cycle period, 

Tdet. The simulation times where trajectory prediction, conflict detection, and metrics collection would 

occur were integer multiples of this parameter.

B. Trajectory Prediction

Trajectory propagation methods for conflict detection can be classified as deterministic, probabilistic, 

and worst-case. In a deterministic case, a single trajectory prediction is assumed to have a 100% probability 

of occurring. In the probabilistic case, each possible trajectory prediction for an aircraft has a probability 

of occurring. Finally, a worst-case approach assumes 100% probability on any possible trajectory for an 

aircraft, thereby defining a bounded region ahead of the aircraft that encompasses all possible future 

trajectories. Each of these methods requires its own set of assumptions for implementation; the choice can 

be made based on the use case. Because, for this study, the results are intended to apply NAS-wide and 

many probabilistic effects are explored through the Monte Carlo approach used, the deterministic trajectory 

propagation method was chosen.

At each conflict detection time, tcd, a deterministic trajectory prediction, Ti

̂ (t), was created for each 

aircraft i, for all times, t, between tcd and tcd + Tpred, where Tpred is the trajectory prediction and CD

time horizon. In the case where no intent information was available, a simple state-projection trajectory 

prediction was used whereby, the current position vector of each aircraft was projected from time tcd to 

time tcd + Tpred using the current velocity vector. In the runs where intent information was available, 

constant velocity or constant acceleration segments were used to build a piece-wise trajectory prediction

up to a future time given by tcd + Tint (using constant radius turn geometry and constant vertical 

acceleration regions, where Tint is the intent horizon time). Separate lateral and vertical trajectory profiles 

were generated and these were synchronized using the time elements of the two predictions. For cases 

where the trajectory prediction horizon was longer than the intent horizon (Tpred > Tint), a state projection 

was used to estimate the states beyond the intent horizon point.

Analysis method 1 used a discretized representation for track data and trajectory predictions, with 

discrete state information at every five seconds (∆ttraj = 5 s). In contrast, analysis method 2 used a 

kinematic description of a trajectory, with state information given as the start or end point of a constant 

velocity or constant acceleration segment – a more concise trajectory representation.
7

C. Conflict Detection

A conflict is a future LOS between two aircraft based on some model of their trajectories. There are 

two types of conflicts and LOS’s reported in this paper: true and predicted, so careful terminology is 

required. The major contribution of this paper is a parametric analysis of the information types and error 

sources going into different trajectory predictions and comparing the conflict detection effectiveness based 

on these predictions to the ideal conflict detection effectiveness based on true trajectories. A predicted LOS 

may never manifest itself because the true trajectories of the aircraft may be different from the trajectory 

predictions. Or, the LOS may not manifest because of a maneuver issued by a conflict resolution function. 

We do not consider this case, because this work is not concerned with conflict resolution.

A true conflict, denoted Cm,i,j, is when a future LOS occurs in the true trajectories of the aircraft, 

denoted, Ti(t) and Tj

(t). A predicted conflict, denoted Ĉn,i,j

, is when a future LOS occurs in the predicted 

trajectories of the aircraft, denoted, T̂i(t) and T̂j

(t). Formally, a LOS is when, simultaneously, the 

horizontal distance between the aircraft trajectories (either true or predicted) is less than the horizontal 

separation criterion, Hsep, and the vertical distance is less than the vertical separation criterion, Vsep. Given 

a pair of trajectories (either true or predicted), a conflict (respectively either true or predicted) exists at tcd

if there exists some time, t, within the time horizon (tcd < t ≤ tcd + Tpred), where those trajectories have 

a LOS. 

Conflict detection was executed at a specified time interval, Tdet, during the simulation/playback of the 

traffic in a given scenario. During each conflict detection cycle, the trajectory predictions of all aircraft 

were compared pair-wise to identify the set of predicted conflicts, Ĉ, for that detection cycle. The true

losses-of-separation between all aircraft, which were pre-processed from the true aircraft tracks at the 

beginning of the scenario run, were used to identify the set of true conflicts, C, for that conflict detection 

cycle.

Surveillance range was simulated during each conflict detection cycle by not performing conflict 

detection between specific pairs of aircraft. If the distance from current position of an aircraft j to the 

current position of another aircraft i was greater than the surveillance range parameters, Rs, then conflict 

detection was not executed for that aircraft pair.

In analysis method 1, conflict detection was performed over a discretized trajectory whereas, in analysis 

method 2, conflict detection was performed by comparing linear trajectory segments. These modeling 

assumptions are one reason for the differences in the results between the two analysis methods, but the 

differences were small. Method 1 could not identify instances of LOS that last less than the trajectory 

sampling interval (5 seconds) and that occur between trajectory sample points whereas method 2 did not 

have this limitation. 

D. Metrics Collection

Metrics collection was done during each conflict detection cycle. The primary metrics (e.g., false alerts 

(FA) and missed alerts (MA)) were collected by comparing the predicted and true conflict sets during each 

detection cycle. FA and MA counts were aggregated over all conflict detection cycles of a single analysis 

scenario run. More details on the full set of metrics used in this analysis can be found in section V.

E. Surveillance Quality

The impacts of surveillance quality (research question 1) on conflict detection performance were 

analyzed using a surveillance error model. The model implements an un-compensated surveillance lag time 

parameter, tsl, as well as a set of standard deviation parameters, (σr, σa, σgs, σvs, σtrk), for the Gaussian 

distributions of error on the position and velocity states of aircraft. The position errors were modeled via a

range error parameter, σr, and an altitude error parameter, σa, while the velocity errors were modeled via 

vertical speed, groundspeed, and track angle error parameters, σgs, σvs, σtrk, respectively. Figure 2 shows 

a graphical representation of the surveillance error modeling approach, while a more detailed description 

of the surveillance error modeling approach can be found in the Appendix.
8

Three surveillance technologies were simulated in the analysis discussed here, as well as the baseline 

case with no added surveillance error. One of the surveillance technologies analyzed was the current-day 

secondary surveillance radar (SSR). The parameter values for the modeling of the position error for this 

surveillance technology are described in an ICAO document [22]. The velocity error parameters were 

obtained from a study that compared Data Link-provided (considered truth) state information with that 

being obtained from ATC data feeds [23]. 

The second surveillance technology analyzed was Automatic Dependent Surveillance-Broadcast 

(ADS-B), in two variations. One of the challenges in defining the ADS-B parameters for the surveillance 

error model is the uncertainty of the accuracy of future avionics equipment over a diverse aircraft fleet. 

The Minimum Aviation System Performance Standards (MASPS) for ADS-B, DO-242A [24], defines the 

accuracy levels that the position and velocity values can have in terms of a 95% error bound and the 

associated, reported NACp and NACv values, but does not make recommendations about which accuracy 

levels to use for any given application. Instead, the MASPS for Airborne Surveillance Applications, DO289 [25], was used to select the parameter values for ADS-B. DO-289 provides a minimum 

recommendation for the position and velocity error bounds (NACp>=5, NACv>=2) as well as desired 

values (NACp>=9, NACv>=3) for conflict detection applications. For the purposes of this study, we refer 

to the minimum ADS-B requirements as ADSB_1 and the desired requirements as ADSB_2 and both were 

simulated in this analysis.

Table 1 lists the values chosen for these three technology levels. The lag time parameter is a fixed 

value that was applied across all surveillance samples in each simulation scenario while the remaining 

parameters are standard deviation of the modeled Gaussian error distributions, each with an assumed zero 

mean value. The parameter SQmodel was used to denote a set of surveillance error parameters (the columns 

of Table 1) for each of the three surveillance error models and the baseline case of no surveillance error; 

possible values for this parameters were: None, Radar, ADSB_1, or ADSB_2.

Figure 2. Position and velocity error components for surveillance modeling (horizontal profile).

Table 1. Surveillance error parameters for the simulated technologies.

Error Parameter None Radar ADSB_1

(NACp=5,NACv=2)

ADSB_2

(NACp=9,NACV=3)

lag time (tsl) 0 seconds 12 seconds 1 second 1 second

horizontal position 

(σr)

0.0 NM 0.3 NM 0.25 NM 0.0081 NM

altitude (σa) 0.0 ft 127.5 ft 108.8 ft 73.8 ft

groundspeed (σgs) 0.0 NM/hr 17.4 NM/hr 2.9 NM/hr 1.0 NM/hr

vertical speed (σvs) 0 ft/min 495 ft/min 450 ft/min 150 ft/min

track angle (σtrk) 0 deg 3.73 deg 1 deg 1 deg
9

F. Level and Quality of Intent Information

The impacts of intent information on conflict detection performance (research question 2) were 

investigated using a representation of level of intent (LOI) and a representation of quality of intent (QOI).

In this analysis, level of intent refers to the relative time horizon over which intent information is available 

and is denoted by the parameter Tint. For example, intent information may be available for trajectory 

prediction for the next five minutes from the current time. LOI parameter values for short, medium, and 

long time horizons (5, 10, and 20 minutes) were used in this analysis. Instead of a time horizon, some 

studies [18] represent the level of intent by the number of communication transmissions, e.g., number of 

trajectory change points [24]. The appeal of using number of communication transmissions is that this 

parameter closely relates to the load on the communication system, which is a significant cost driver. On 

the other hand, the number of conflicts and losses, which are the primary metrics of this study, are more 

closely tied to the time horizon than the number of points; for instance, five trajectory change points could 

represent one minute of a trajectory, or one hour of a trajectory. More pointedly, the number of 

communication transmissions alone cannot be used to determine communication load; the content of the 

message must also be defined. One message with five points is equivalent to five messages with one point 

each; the content of a message mixes parametric variables. In our study, the content of a message is 

described as the quality of intent. To maintain this separation of concerns, only the time horizon is used in 

LOI. 

The quality of intent (QOI) representation refers to the amount of information that is contained in the 

available intent. This representation was partially inspired by the ADS-B MASPS definition of a trajectory 

change (TC) report [24], whereby a lateral TC point, for example, includes a reference point and may 

contain information about the turn geometry, such as turn radius and the track into and out of that reference 

point. The representation differs from the ADS-B MASPS definition in the use of five levels that contain 

more or less information than the proposed MASPS. Quality of intent information was further broken down 

into the vertical and horizontal components and was represented by the parameters IQV and IQH, 

respectively. The separation of the horizontal and vertical components of intent information allowed us to 

perform analysis runs with a DO-242A-equivalent representation of intent information, specifically, with 

IQV1 and IQH2. Table 2 shows the quality of intent representations used in this analysis and the information 

associated with each lateral and vertical intent point.

The vertical intent information available for trajectory prediction ranges from target altitude only to 

fully-defined altitude change points. Vertical intent quality of zero (IQV0) specifies that target altitude 

information will be available during an altitude change maneuver. Vertical quality of one (IQV1) adds to 

the vertical quality of zero by providing 2-dimensional (time and altitude) information for any bottom-ofclimb (BOC) or top-of-climb (TOC) points within the intent horizon, for any climb maneuver. With vertical 

quality two (IQV2), each BOC and TOC point also contains the target climb rate out-of or into those points, 

respectively. Vertical quality of three (IQV3) adds a constant acceleration value used to transition from the 

BOC point to the target climb rate or to transition from the target climb rate to the TOC point. Finally, 

vertical intent quality of four (IQV4) adds additional 2-dimensional vertical profile points at a specified 

altitude interval (∆altQ4) between any BOC and TOC points. The same set of information hierarchy applies 

to the top-of-descent (TOD) and bottom-of-descent (BOD) intent points and the vertical profile of any

descent maneuver.

A similar quality of intent scheme was used for the lateral intent information that ranged from target 

track to fully-defined track change points. Lateral/horizontal intent quality of zero (IQH0) specifies the 

target track that is available during a turning maneuver. Lateral quality of intent one (IQH1) adds the 3-

dimensional reference point (time, latitude, longitude) for the turn, which is assumed to be the equivalent 

of a fly-by waypoint. With lateral quality level two (IQH2), the radius for an assumed constant radius turn

for the fly-by maneuver is provided. The trajectory prediction function uses the turn radius information, 

along with an estimated groundspeed, to determine the turn rate for the turn maneuver. Lateral quality of 

intent three (IQH3) includes a turn rate value which, combined with the turn radius, defines the groundspeed 

for the constant turn rate and constant radius turn. Finally, lateral intent quality of four (IQH4) adds 
10

additional 3-dimensional points at a specified time interval (∆tQ4) between subsequent turn reference 

points.

Additional details regarding the methods used to obtain intent information from the recorded traffic 

data for this study are provided in section IV, sub-section A.

G. Analysis Parameters

A comprehensive set of the parameters used in this analysis of conflict detection performance is 

presented in Table 3, along with the nominal values used for each parameter. The nominal values were 

used in each scenario unless otherwise specified in the experiment test matrices in section VI. Note that 

the horizontal and vertical separation criteria (Hsep and Vsep) were held constant for all analysis runs. 

Similarly, all track data and trajectory predictions used a single resolution (∆ttraj) of five seconds for 

analysis method 1.

Table 2. Quality of intent (QOI) values on the horizontal and vertical profiles.

Horizontal/

Vertical Quality

Target State Position

2-D/3-D

Target Rates Acceleration

IQV0 Target altitude

IQV1 Target altitude Time, altitude

IQV2 Target altitude Time, altitude Target climb 

rate

IQV3 Target altitude Time, altitude Target climb 

rate

Vertical acceleration

IQV4

Includes all IQV3 points plus additional IQV1 points added based on refined vertical profile 

requirements (∆altQ4)

IQH0 Target track

IQH1 Target track Time, latitude, longitude1

IQH2 Target track Time, latitude, longitude Turn radius2

IQH3 Target track Time, latitude, longitude Turn radius Turn rate3

IQH4

Includes all IQH3 points plus additional IQH1 points added based on refined horizontal 

profile requirements (∆tQ4)

Notes: 1 Track in and track out are available with all intent waypoints for horizontal quality levels of 1 and above.

2Combine with estimated turn speed to estimate turn rate. 3Combined with radius provides turn speed and 

centripetal acceleration. 
11

H. Assumptions and Scope

The scope and assumptions used in this analysis help to define the realm of applicability of the results 

obtained. The scope of this analysis was limited to the evaluation of en-route conflict detection performance 

in an architecture- and algorithm-agnostic way. The analysis does not include conflict resolution nor does 

it explore the development of any new, or the refinement of any existing, conflict detection algorithms. 

The primary assumptions used in the analysis modeling are listed below. Note that some assumptions

are also inherent to the data processing algorithms used and are described later in this document.

 Conflict detection is being performed by automation

 Conflict detection uses deterministic trajectory predictions

 Conflict detection performance is dependent on the assumptions used in trajectory prediction

 The true trajectory flown by each aircraft is known perfectly and a priori

 No added atmospheric disturbances, such as wind, are used in simulation or in trajectory prediction

 Only trajectory data above flight level (FL) 180 are used for the recorded traffic tracks

 A horizontal separation criterion of 5 nautical miles is used for CD

 A vertical separation criterion of 800 feet is used for CD1

 The surveillance range is assumed to be aircraft-centric with respect to the first aircraft in the 

aircraft pair

 The surveillance error model assumes zero-mean, Gaussian error distributions

 

1 This value was used instead of 1000 ft to prevent spurious detections caused by small fluctuations and noise in the 

altitude data.

Table 3. Analysis input parameters.

Parameter Description Nominal 

Value

Units

Tdet conflict detection cycle period 60 seconds (s)

Tpred trajectory prediction and conflict detection time horizon 1200 seconds (s)

∆ttraj trajectory sampling interval 5 seconds (s)

Hsep horizontal separation criterion 5 nautical miles (NM)

Vsep vertical separation criterion 800 feet (ft)

Rs

surveillance range 1500 nautical miles (NM)

SQmodel surveillance quality parameter set None -

tsl surveillance lag time 0 seconds (s)

σr

surveillance horizontal position error standard deviation 0 nautical miles (NM)

σa

surveillance altitude error standard deviation 0 feet (ft)

σgs surveillance groundspeed error standard deviation 0 knots (kn)

σvs surveillance vertical speed error standard deviation 0 feet/minute (ft/min)

σtrk surveillance track angle error standard deviation 0 degrees (deg)

Tint intent horizon time (level of intent) 0 seconds (s)

IQV vertical quality of intent None -

IQH horizontal quality of intent None -

∆tQ4

time interval for intent quality 4 horizontal points 120 seconds (s)

∆altQ4

altitude interval for intent quality 4 vertical points 1000 feet (ft)

δt

time tolerance for conflict comparison criteria1 120 seconds (s)

δH horizontal distance tolerance for conflict comparison 

criteria1

10 nautical miles (NM)

δV vertical distance tolerance for conflict comparison criteria1 2000 feet (ft)

Note: 1 See Section V, Sub-section B for a detailed description of the conflict comparison approach.
12

 The intent information was inferred from the track data using a custom curve-fitting algorithm

developed for this project2

 There is no lag in the intent information at conflict detection time

 Blunders in the form of missing intent or non-conformance to intent are not considered

 An intent point is added at the intent horizon time if another intent point does not already exist 

close to that horizon

IV. Input Data

The CD performance analysis leveraged a set of recorded traffic tracks from the NAS. The source data 

is a fused, NAS-wide data set that covers 45 days in 2013 and is stored in the Data Warehouse at the NASA 

Ames Research Center. Surveillance radar and ADS-B feeds from multiple facilities are fused to create 

full visual- and instrument-flights-rules (VFR and IFR) track profiles (latitude, longitude, altitude, and time) 

for flights within the Continental United States (CONUS), and partial track profiles for flights entering and 

exiting the NAS. 

A representative day of traffic was chosen from this data set for use in this analysis. Figure 3 shows 

the Federal Aviation Administration’s Aviation System Performance Metrics (ASPM77) database [26] 

values for total daily operations and total daily delayed operations for one portion of the available data days. 

The representative day of traffic was chosen to be March 28th, 2013 due to the high demand and low level 

of delayed operations, likely indicating a day with low weather impacts on the flown tracks. A 36-hour 

window was selected to capture 6 hours of traffic prior to and 6 hours of traffic after the selected date. This 

allowed for the inclusion of the complete track profile for flights that were already airborne before the start 

of the selected date and those that were still airborne at the conclusion of the selected date. 

The sample set of traffic tracks were used in three different ways within this analysis, after some initial 

processing and conditioning. First, the traffic tracks were used in analysis method 1 as the playback or true 

tracks flown by aircraft within those scenarios. Second, the conditioned track data was processed within a 

custom algorithm to process or infer the intent information for each track. Third, the inferred intent 

 

2

 The extraction of intent information from track data was much more difficult than originally envisioned. Several 

different algorithms were created and compared. We note that our intent results are strongly connected to the 

goodness of the fit of the intent model to the track data. 

Figure 3. ASPM77 daily total operations (top), and ASPM daily delayed operations (bottom). March 28, 

2013 was chosen as a high volume and low delay day from the recorded traffic data available.
13

information was used to build distributions of parameters such as track angle change, average groundspeed, 

average climb rates, and others that were used in the generation of random flight plans for the circular 

scenarios used in analysis method 2. A total of 30,799 recorded flight tracks (time, latitude, longitude, 

altitude) for flights into, within, and out of the Continental United States were used in this analysis.

A. Data Processing

Track Data Processing

The raw track data required pre-conditioning and filtering before it could be used to conduct this 

analysis. The tracks inherently contain the noise from the surveillance sensors that captured the data. 

Additionally, the fusion of multiple data sources can introduce an artificial noise due to the various sampling 

frequencies and timestamps. Figure 4 and Figure 5 show the track and altitude profiles, respectively, for a 

randomly selected flight before and after data processing. Note that, using these figures alone, it is difficult 

to spot the noisy characteristics of the data. However, an inspection of the velocity profiles for this example 

flight computed using a simple finite difference (estimating the derivative) reveals the noise characteristics. 

Figure 6-Figure 8 show the groundspeed, vertical speed, and track angle profiles, respectively, for the same 

sample flight before and after the processing step. Note the high level of noise in these velocity profiles. 

In each of the figures for the sample flight, the raw track data is shown in blue and the final processed track 

data for use in this analysis is shown in red.

 

 

Figure 4. Raw and processed track profile for a 

randomly-selected sample flight.

Figure 5. Raw and processed altitude profile for a 

randomly-selected sample flight.

Figure 6. Raw and processed groundspeed profile 

for a randomly-selected sample flight.

Figure 7. Raw and processed vertical speed profile 

for a randomly-selected sample flight.
14

The processing of the track data was completed in three basic steps. First, a pre-screening step was 

used to discard flights that did not have sufficient data useful for this analysis. This included discarding 

flights with no track data above FL180, discarding flights fully outside the time window of interest, and 

discarding flights with anomalous track data. Next, a conditioning step was used to correct any data dropouts and to trim the flight track data to the portion above FL180. Flight segments below FL180 include 

VFR flights with altitude separations in 500 foot increments and traffic entering or exiting the terminal 

airspace environment where the separation standards are different from the en-route environment and 

special separation rules exist; these conditions were observed in a few preliminary test runs of the analysis 

model. Finally, a filtering step was used to remove the surveillance and data fusion noise in order to create 

a baseline set of track data. A combination of excessive groundspeed and vertical speed data point removal, 

followed by a one-dimensional Kalman Filter [27], generated the final processed track data that can be seen 

in Figure 4-Figure 8. The thresholds and algorithms used for the processing of the raw data were developed 

largely by observations drawn from the raw data. The following is a list of the pre-processing and filtering 

steps that were applied to the raw data:

Pre-screening:

 Discard all flights with no data points above FL180

 Discard all flights with the same origin and destination airport

 Discard flights with un-identified origin or destination airport

 Discard any flights already airborne at the beginning of the 36 hour sample window

 Discard any flights still airborne at the end of the 36 hour sample window

 Discard any flights with data gaps in time greater than 5 minutes

 Discard any flights with a processed track time less than 10 minutes

 Discard any flights with incomplete track data (missing descent or climb portion but fully 

within the CONUS)

Conditioning:

 Trim all track data to retain only the portion above FL180

 Remove any duplicated time data points

 Remove any data drop-outs (identified via vertical speeds in excess of 10,000 ft/min)

Filtering:

 Remove any data points producing excessive groundspeeds and vertical speeds

o Groundspeeds greater than 590 knots

o Vertical speeds greater than 10,000 ft/min

 Remove any data points with large groundspeed and vertical speed deviations away from a 9-

point centered moving average

o Groundspeed differences larger than 100 knots

Figure 8. Raw and processed track angle profile for 

a randomly-selected sample flight.
15

o Vertical speed differences larger than 100 ft/min

 Interpolate the track data for a uniform time sampling of 5 seconds

 Apply a one-dimensional Kalman Filter to the track data (altitude, latitude, and longitude) 

 

The selected traffic demand day of March 28th, 2013, with the additional 6 hours of pre- and 6 hours of 

post-traffic, contained a total of 72,769 recorded traffic tracks. After the processing steps above were 

implemented, the remaining traffic set contained 30,799 recorded traffic tracks. 

Intent Information Processing

The intent information used in the conflict detection performance study was inferred from the recorded 

track data using a custom curve-fitting program developed for this project. Flight plan and flight plan 

amendment information was available for the recorded tracks but using that information as source for intent 

would likely lead to un-realistic intent information. For example, if an aircraft had been vectored for traffic, 

or some other reason, for a brief period of time, that vector information would not have been present in the 

flight plan amendments. It was decided that the scope of this analysis should not include erroneous intent 

information or blunder scenarios, which led to the decision to infer the intent information from the track 

data.

The vertical and lateral components of intent information were identified separately. The lateral intent 

was identified by searching for the start and end of non-turning flight segments using significant changes 

in track rate as the trigger. The track angle was computed using the track positions and the track rate was 

computed as the differential of the track angle. Moving average filters were used to mitigate the noisy 

differential derivatives. A tuned threshold limit for track rate of +/-0.03 deg/s was used to identify the 

lateral states (turning, not turning) of each track data point, thereby creating the boundaries between straight 

and turning flight segments, or the estimated beginning-of-turn (BOT) and end-of-turn (EOT) locations. 

By considering only the non-turning flight segments, the projected intersection of subsequent segments 

constituted the notion of a fly-by waypoint along a flight plan. These identified waypoints are the lateral 

intent points inferred from the track data. Figure 9 shows an example of a recorded lateral profile and the 

identified lateral intent points; note that these waypoints are the intersections of the non-turning segments 

and, thus, do not overlay on the track data.

Inferring the vertical intent information involved identifying the level flight segments from each 

vertical profile (time, altitude). The differential vertical speed was first used to initialize the states (climb, 

descent, level) at each track data point using a threshold limit of +/-100 ft/min. Moving average filters were 

used to mitigate the noisy differential vertical speed while heuristics were used to handle consecutive 

segments of the same type and to fine tune the location of the start and end points of the level flight 

segments. Finally, the start and end of each level flight segment was labeled as a BOC, TOC, TOD, or 

BOD altitude intent point based on the preceding or subsequent segment type. Figure 10 shows a sample 

vertical profile with and the identified altitude change points.

The intent information generation process also included the identification of the data elements required 

to support the various qualities of intent modeled in this study. For the lateral intent points, the turn radius 

was determined using the distance from the identified waypoint to the BOT or EOT, whichever was 

smallest, and the track angle change from the BOT to the EOT. The average groundspeed between the 

BOT and EOT, the track angle change, and the radius were then used to compute a constant turn rate for 

the turn. Figure 11 shows a sample lateral profile, including the constant rate turns that were estimated for 

those track changes. Note that the BOT and EOT points are not part of the intent information that was

available for trajectory prediction and are just depicted here for reference3. For the vertical data elements, 

an optimization routine was used to fit two acceleration regions and a constant vertical rate region for any 

altitude change maneuver by minimizing the mean squared error between the recorded data and the modeled 

 

3 We decided to follow DO-242 as much as possible. The DO-242 approach uses other parameters which enable a 

calculation of the BOT and EOT locations. With full information, the BOT and EOT locations can be accurately 

calculated. With less information, they can only be estimated.
16

maneuver. Figure 12 shows a sample altitude change with the TOD point, a constant vertical acceleration 

region, followed by a constant vertical rate region, then another constant vertical acceleration region, and 

finally the BOD point. Note that the acceleration regions are properties of the intent point that they begin 

from or end at and that each intent point has a vertical rate into and a vertical rate out of that point.

 

 

Table 4 shows the identified horizontal intent information that was available for trajectory prediction 

for the sample track shown in Figure 9 at the horizontal quality level of three. Similarly, Table 5 shows the 

identified vertical intent information that was available for trajectory prediction for the sample vertical 

profile shown in Figure 10 at the vertical quality level of three. These two tables constitute one kinematic 

representation of the recorded track data that was estimated or fitted algorithmically and was used as the 

method for describing intent information in this analysis.

Figure 9. Sample recorded track horizontal profile 

with identified waypoints.

Figure 10. Sample recorded track vertical profile 

with identified altitude change points.

Figure 11. Example identified turn intent and fitted 

constant radius turns.

Figure 12. Example identified vertical intent points 

with vertical acceleration zones and constant vertical 

speed segments.
17

B. Scenarios

Scenarios for Analysis Method 1

The traffic scenarios for this analysis were generated in two ways. For method 1, scenario generation 

was done by randomly time-shifting the original start time of each flight track within some specified limits. 

A test set of ten different scenarios using the full recorded traffic data were generated with a maximum time 

shift parameter, tshift,max, of zero, two, five, and fifteen minutes, and one, two, four, six, twelve, and 

twenty-four hours. These scenarios were analyzed with the nominal analysis model parameter values and 

helped to identify the time shift that produced the highest number of true losses-of-separation. As seen in 

Figure 13, the peak number of true LOS was achieved with a time shift close to fifteen minutes or one hour. 

At these time shift values, the full day of time-shifted traffic produced over 15,000 true LOS. The increase 

followed by a decrease in the number of true LOS is the result of the small increase in time shift values that 

increasingly create LOS events in the recorded tracks until such point as the peaks of traffic start to flattens 

out and the density of traffic is reduced, as shown by the number of simulated traffic over the simulation 

time in Figure 14.

All scenarios for analysis method 1 consisted of the full 30,799 recorded traffic tracks and a random 

time shift of up to one hour for each track. Ten different randomized scenarios were created and used to 

investigate the variability due to the randomization. One of these ten scenarios was selected as the baseline 

scenario for all other analysis runs. 

Table 4. Identified horizontal intent data for sample profile shown in Figure 9.

time latitude longitude trackIn trackOut turnRadius turnRate type

[s] [deg] [deg] [deg] [deg] [NM] [deg/s] [unitless]

0 40.8102 -74.1015 268.6 268.6 0 0 START

169.7 40.80281 -74.4436 268.4 246.6 32.4 -0.1558 WPT

810.81 40.34498 -75.8101 245.8 251.8 85.4 0.0755 WPT

1082.7 40.18667 -76.4341 251.5 267.6 64.3 0.0980 WPT

2641.5 40.01759 -80.1314 265.5 256.2 72.7 -0.0872 WPT

2810 39.9442 -80.5185 255.3 255.3 0 0 END

Note: trackOut is the same as target track for intent level 0.

Table 5. Identified vertical intent data for sample profile shown in Figure 10.

time altitude targetAltitude climbRateIn climbRateOut verticalAcceleration type

[s] [ft] [ft] [ft/min] [ft/min] [ft/s2] [unitless]

0 18028 28000 2664 2973 0.1187 START

260 28000 28000 2973 0 -0.4385 TOC

370 28000 35998 0 1459 0.6194 BOC

765 35998 35998 1459 0 -0.2615 TOC

1380 36000 32005 0 -2072 -1.0191 TOD

1550 32005 32005 -2072 0 0.4618 BOD

2375 32000 18161 0 -1968 -1.4486 TOD

2810 18161 18161 -1968 -1944 0.0014 END
18

 

Scenarios for Analysis Method 2

The scenario generation approach for analysis method 2 was to generate flight plans that traverse a 

circular region of airspace, where the flight plan characteristics were generated using distributions obtained 

from the recorded traffic tracks. After selecting the number of aircraft and the radius of the circular region 

of airspace, flight plans were randomly generated by sampling from a set of distributions for: initial heading 

angle deviation, groundspeed, horizontal leg time, turn angle magnitude, number of altitude segments, 

vertical speed, and level flight segment time. Each flight plan was initialized at the boundary of the circular 

region. An initial heading offset was selected from a uniform distribution and added as a deviation away 

from a heading towards the center of the circular region. A horizontal profile was created by successively 

sampling from a horizontal leg time and a turn angle until the flight plan exited the circular region. A 

vertical profile was created by subsequently sampling a level segment time, an altitude change of 1000 or 

2000 feet up or down, and a vertical speed for that altitude change until the time required to exit the circular 

region was achieved. The fitted distributions derived from the characteristics of the recorded traffic data 

and used in the scenario generation are shown in Figure 16 - Figure 21 below and the associated probability 

density functions defined in equations (1)-(3). The horizontal profile of a 100 aircraft scenario is shown in 

Figure 15.

A LOS-maximizing approach was used in creating the set of scenarios for analysis method 2. A set of 

preliminary runs were used to calibrate the parameters for: number of aircraft, circular region radius, and 

initial heading angle deviation. The analysis results presented in this paper used a set of scenarios with 

1500 aircraft over a circular region with a radius of 300 nautical miles and a uniform distribution of initial 

heading angle deviation within +/-40 degrees. 

PDFNormal(x|μ, σ) =

1

σ√2π

e

−

(x−μ)

2

2σ2

(1)

PDFInverseGaussian(x|μ, λ) = [

λ

2πx

3

]

1

2

⁄

e

−

λ(x−μ)

2

2xμ2 (2)

PDFGamma(x|a, b) =

1

b

aΓ(a)

x

(a−1)

e

−x

b

⁄

(3)

Figure 13. Number of true LOS, 

NC,sim, created as a function of the maximum time 

shift parameter, tshift,max.

Figure 14. Number of flights as a function of the 

simulation clock time for a few of the time shift 

scenarios.
19

 

Figure 15. Example circular scenario horizontal profile with 100 aircraft.

Figure 16. Distribution of the maximum average 

groundspeed for all recorded traffic tracks.

Figure 17. Distribution of the time for all horizontal 

flight segments over all recorded tracks.
20

 

V. Metrics

A. Definitions

In conflict detection, the primary concern is in the accuracy of the alerts provided. An alert is a warning 

provided by the conflict detector at some time regarding the potential for a LOS between two aircraft at 

some future time. The correctness of these alerts has typically been characterized using false alert and 

missed alert statistics. A false alert is an issued alert where there is no corresponding LOS. This kind of 

alert is sometimes described as a nuisance alert because it can lead to unnecessary actions by the agent 

receiving it. A missed alert is the lack of an alert to a LOS that will occur between two aircraft within a 

specified look-ahead time. In most air traffic concepts, a missed alert means that the action to resolve the 

conflict will be delayed, which could result in significant safety issues. Correct alerts are alerts that are 

neither missed alerts nor false alerts.

Table 6 provides a set of definitions for conflict detection-related terms used throughout this document. 

In particular, it’s important to note the difference between an alert and a detection. An alert is a warning 

provided by the conflict detector at any given time or any conflict detection cycle. Conversely, a detection 

is the successful identification of a true LOS event over all time or over all conflict detection cycles. A 

Figure 18. Distribution of the magnitude of the turn 

angle between horizontal flight segments for all 

tracks and all segments.

Figure 19. Distribution of the time in level flight over 

all tracks and all level altitude segments.

Figure 20. Distribution of the average vertical speed 

between vertical level segment changes in climbs for 

all tracks.

Figure 21. Distribution of the average vertical speed 

magnitude between vertical level segment changes in 

descents for all tracks.
21

missed detection means that conflict resolution will never occur because the conflict is never detected; this 

is an important safety consideration. A LOS event in this study is characterized by the position of both 

aircraft at initial entry into LOS and the time at which that entry occurs.

The idea of a “true conflict” was used in the analysis and is possible because of the availability of truth 

track data that is known a priori. A true conflict exists if there is a LOS between the true tracks of two 

aircraft; however, in a real world environment, no such true data exists because the future tracks can change 

for a variety of reasons (e.g., to resolve a conflict). Nonetheless, the idea of a true conflict is a useful 

concept when measuring the performance of trajectory prediction and conflict detection in the absence of 

conflict resolution. One true LOS can correspond to multiple true conflicts depending on the number of 

conflict detection cycles that contain that LOS within the conflict detection horizon.

The true and predicted conflicts identified in this study were based on separation criteria that are 

different from the separation standard for en-route airspace [28]. The horizontal separation criterion of 5 

nautical miles was used but a reduced vertical separation criterion of 800 feet was chosen for this study. In 

the track data processing step, and in preliminary analysis runs, it was observed that the Mode-C resolution 

of +/-100 feet altitude of some of the track data had a significant impact on the number of predicted 

conflicts, thereby justifying the use of a reduced vertical separation criterion.

The metrics for this study were collected in terms of absolute counts, ratios or probabilities, or mean 

values. Table 7 lists the primary conflict detection metrics for this study while Table 8 lists the secondary 

metrics. The secondary metrics are absolute counts of quantities such as false alerts and missed alerts and 

are required to compute the observed probabilities and mean values in the primary metrics. The number of 

false alerts, the number of missed alerts, the number of predicted conflicts, and the number of true conflicts 

were each obtained using the size of the true and predicted conflict sets, C and Ĉ, respectively, identified 

during each conflict detection cycle and aggregated over all detection cycles in a simulation run. For each 

simulation run, the total number of LOS and, of those, the number that were never detected in any detection 

cycle (the number of missed detections) were also collected. Only the primary metrics are reported in the 

results tables for this study.

The primary metrics collected for this study were: probability of false alert, PFA, probability of missed 

alert, PMA, probability of missed detection, PMD, and mean time-to-LOS at first detection, ∆tLOS,mean. The 

probability of false alert represents the observed ratio of total false conflict alerts to the total number of 

conflict alerts and indicates the likelihood that any alert provided by the conflict detector could be false. 

Missed alert probability represents the observed ratio of the total number of missed alerts to the total number 

Table 6. Conflict detection terms and definitions.

Term Definition

loss(es)-of-separation

(LOS)

violation(s) of the minimum separation criteria between two aircraft trajectories

separation criteria vertical and horizontal distances that identify the boundary for a LOS event 

between two trajectories

conflict two aircraft are in conflict if there exists a time, t, with a LOS in their true or 

predicted trajectories

detection the identification of a true LOS event

true conflict a conflict that exists between the true trajectories of two aircraft and that is 

within the conflict detection horizon, in the absence of a resolution maneuver

predicted conflict a conflict that exists between the predicted trajectories of two aircraft and that 

is within the conflict detection horizon

alert a warning provided by the conflict detector

false alert (FA) an alert regarding a LOS that is not present in the true aircraft trajectories within 

the conflict detection horizon

missed alert (MA) the lack of an alert regarding a LOS that is present in the true aircraft trajectories 

within the conflict detection horizon

missed detection (MD) over all detection attempts, the absolute failure to detect a true LOS event
22

of true conflicts and indicates the likelihood that, at any detection cycle, there is a true LOS present within 

the conflict detection horizon that is not alerted. The missed detection probability represents the observed 

ratio of total LOS that were not detected as a true conflict in any detection cycle to the total number of LOS 

present in a given simulation run and indicates the likelihood that the conflict detector will fail to provide 

any alert for a given true LOS. Finally, the mean time-to-LOS represents the average time to entry into 

LOS at the first successful detection for those losses-of-separation that were successfully detected within a 

simulation run; losses-of-separation that were never detected do not contribute to this mean value.

B. Same Conflict Comparison

The notion of two conflicts being the “same conflict” or not was developed for this study in order to 

properly characterize conflict detection performance. The traditional definition of a false alert, as it is most 

commonly defined in other examples of conflict detection-related literature, is that, an alert is false when 

there exists a predicted conflict between the predicted trajectories of two aircraft but there does not exist a 

true conflict between the true trajectories of those same two aircraft, within the same time horizon. 

Similarly, the traditional definition of a missed alert is that, an alert is missed when there does not exist a 

predicted conflict between the predicted trajectories of two aircraft but there does exist a true conflict

between the true trajectories of those same two aircraft, within the same time horizon. Given that conflict 

detection is done with imperfect trajectory predictions, these definitions can lead to the characterization of 

a conflict alert as a correct alert even if the predicted and true conflicts are far from each other in space 

and/or time. The false alert and missed alert definitions need to contain criteria to evaluate whether a 

predicted conflict and a true conflict are “the same;” that is, that their respective losses-of-separation occur 

roughly in the same place and at the same time. 

The schematic in Figure 22 depicts two aircraft with both a predicted conflict and a true conflict. The 

traditional definitions for false alert and missed alert would indicate that there is no false alert and no missed 

alert; that is, this example would be considered a correct alert due to the presence of both a predicted and a 

Table 7. Conflict detection performance primary metrics.

Metric Definition

PFA = NFA⁄NĈ the false alert ratio or false alert probability

PMA = NMA⁄NC

the missed alert ratio or missed alert probability

PMD = NMD⁄NC,sim the missed detection ratio or missed detection probability

∆tLOS,mean = ∑

∆tLOS,m

M

M

m=1,2,...

mean time-to loss-of-separation at first detection for all M successfully 

detected true conflicts

Table 8. Conflict detection performance secondary metrics.

Metric Definition

NFA = ∑ ∑|Ĉ ∖ C|

sim_run tcd

the number of false alerts from the set of predicted conflicts (conflict alerts), 

Ĉ, identified during each detection cycle, summed over every detection cycle 

and over an entire simulation run

NĈ = ∑ ∑|̂C|

sim_run tcd

the number of predicted conflicts (conflict alerts), Ĉ, identified during each 

detection cycle, summed over every detection cycle and over an entire 

simulation run

NMA = ∑ ∑|C ∖ Ĉ|

sim_run tcd

the number of missed alerts from the set of true conflicts, C, within the time 

horizon of each detection cycle, summed over every detection cycle and over 

an entire simulation run

NC = ∑ ∑|C|

sim_run tcd

the number of true conflicts present during each conflict detection cycle, 

summed over every detection cycle and over an entire simulation run

NC,sim = |C| the number of unique true conflicts (number of LOS) in a simulation run

NMD = |C ∖ Ĉ| the number of missed detections – from the set of all true conflicts, C, not in 

the set of predicted conflicts (conflict alerts), Ĉ, over an entire simulation run
23

true conflict. However, the scale of the diagram in Figure 22 might be large and the location of the LOS in 

the two conflicts may be many nautical miles apart. This difference in location is important for functions 

such as conflict resolution where the difference between a conflict ahead-of-track and a conflict to the rightof-track is important. Using the traditional definitions for false alert and missed alert results in artificially 

better conflict detection performance. This is because, if some criteria for evaluating whether two conflicts 

are “the same” is applied to the diagram of Figure 22, either of two conditions would occur: the two conflicts 

are considered to be the same and the false alerts and missed alerts are both zero, just as the traditional 

definitions indicate, or, the two conflict are not the same, in which case, the predicted conflict is considered 

to be a false alert to a non-existent LOS, and the true conflict is considered to be a missed alert of a true 

LOS. Therefore, using criteria for evaluating whether two conflicts are the same within each conflict 

detection cycle, the revised definitions of a false and a missed alert are:

 a false alert is a predicted conflict between the predicted trajectories of two aircraft where the same 

true conflict in the true trajectories of those aircraft does not exist, within the conflict detection time 

horizon, and

 a missed alert is a true conflict between the true trajectories of two aircraft where the same predicted 

conflict in the predicted trajectories of those aircraft does not exist, within the conflict detection 

time horizon.

The criteria chosen to evaluate whether two conflicts are “the same” involves comparing the time and 

position of both aircraft at the entry into LOS. This is an appropriate method to use, given that a LOS event 

in this study is a reference to the time and position of first entry into LOS for a pair of aircraft. A conflict 

comparison for an example such as the one in Figure 22 should determine that the two conflicts are not the 

same because, even though the temporal location of the LOS for the predicted and true LOS events may be 

very close, the spatial location of the two LOS is significantly different. Define a conflict vector, Cm,i,j,

for conflict m between aircraft i and aircraft j as:

Cm,i,j∶= [tLOS,m, si,m(tLOS,m), sj,m(tLOS,m)]

(4)

where the first element is the absolute entry time into LOS, tLOS,m, for conflict m, the next element is the 

3-dimensional position of aircraft i at tLOS,m, and the last element is the 3-dimensional position of aircraft 

j at tLOS,m. Figure 23 shows the actual positions of aircraft i and j at the time-in to LOS, tLOS,n, of the true

conflict n, and the predicted positions of those aircraft at the time-in to LOS, tLOS,m, of the predicted conflict 

m.

Figure 22. Example predicted and true conflicts between a pair of aircraft (20 minute long prediction 

arrows).
24

Comparison of two conflicts is done by evaluating the space/time differences between the conflict 

vectors against a set of threshold values. Let,

DH,i,m,n = dGCD (lati,m(tLOS,m), loni,m(tLOS,m), lati,n(tLOS,n), loni,n(tLOS,n)) (5)

be the horizontal arc-distance between the positions of aircraft i in conflicts m and n, where dGCD is a 

function that computes the great-circle distance between two points on a sphere. Let,

DV,i,m,n = |alti,m(tLOS,m) − alti,n(tLOS,n)| (6)

be the magnitude of the difference between the altitude of aircraft i in conflict m, and the altitude of the 

same aircraft i in conflict n. Then, two conflicts m and n are said to be the same (Cm,i,j ≈ Cn,i,j) if and 

only if all of the conditions in equations (7)-(11) are satisfied:

|tLOS,m −tLOS,n| ≤ δt (7)

DH,i,m,n ≤ δH (8)

DH,j,m,n ≤ δH (9)

DV,i,m,n ≤ δV (10)

DV,j,m,n ≤ δV (11)

where δt, δH, and δV, are threshold values selected for the maximum allowable time difference between 

two LOS entry times, the maximum allowable horizontal distance between the position of each aircraft in 

both conflicts, and the maximum allowable vertical distance between the altitudes of each aircraft in both 

conflicts, respectively. The following logic was used in determining a set of reasonable threshold values, 

based on a lateral separation standard of 5 NM and a vertical separation standard of 1000 ft:

 δt = 120 s : the time it takes to traverse 10 NM at an assumed groundspeed of 300 knots

 δH = 10 NM : the horizontal position of an aircraft i can vary by as much as 10 NM and still be in 

LOS with an aircraft j

 δV = 2000 ft : the vertical position of an aircraft i can vary by as much as 2000 ft and still be in 

LOS with an aircraft j

The comparison criteria defined by equations (7)-(11) lead directly to the computation of the primary 

metrics. At any conflict detection time, a false alert was identified when a predicted conflict was not the 

same as any true conflict within the same conflict detection horizon. Similarly, at any conflict detection 

time, a missed alert was identified when a true conflict was not the same as any predicted conflict within 

the same conflict detection horizon. A missed detection was identified by a true LOS that was not predicted 

Figure 23. True aircraft positions at time-in to LOS, tLOS,n, for a true conflict, n, and predicted aircraft 

positions at, tLOS,m, for a predicted conflict, m.
25

in any conflict detection cycle over an entire simulation run. The time-to entry into loss-of-separation at 

first detection, ∆tLOS,m, for any true conflict, m, was given by:

∆tLOS,m = tLOS,m − tcd (12)

and was recorded at the earliest detection cycle in which a correct alert is issued for that LOS.

VI. Test Matrices

The analysis runs for conflict detection performance using method 1 were divided into five test 

matrices. Analysis runs using method 2 were conducted where appropriate for comparison. In the three 

surveillance quality test matrices, the current state of each aircraft was subjected to the surveillance error 

parameters being analyzed and state-projections were used for trajectory predictions. In the two level and 

quality of intent test matrices, no surveillance error was added to the aircraft states and the available intent 

information was used to inform the trajectory predictions. 

A. Surveillance Quality Test Matrix 1 (SQ1)

Surveillance quality test matrix 1 is a full factorial combination of the primary input parameters for 

surveillance quality and conflict detection. The conflict detection cycle period (Tdet) was tested at 15 and 

60 seconds. Conflict detection and trajectory prediction horizon (Tpred) was tested at short, medium, and 

long values of 5, 10, and 20 minutes, respectively. The surveillance range (Rs) was also tested at short, 

medium, and long values of 50, 250, and 1500 (conservative surrogate for infinite surveillance range) 

nautical miles, respectively. Each combination of the three different surveillance error model parameter 

sets (SQmodel), including the baseline model with no surveillance error, was tested. Trajectory prediction 

was exercised using symmetric and asymmetric approaches where, for the asymmetric cases, perfect 

trajectory prediction was assumed for the first aircraft in the conflict pair where the predicted trajectory was 

equal to the true trajectory of that aircraft, and for the symmetric cases, both aircraft had the same quality 

of imperfect trajectory predictions. A single, time-shifted, recorded traffic scenario with 30,799 aircraft

tracks was used for all runs. Table 9 lists the parameter values analyzed, which correspond to a total of 144

analysis runs.

B. Surveillance Quality Test Matrix 2 (SQ2)

Surveillance quality test matrix 2 provides a set of sensitivity data for each of the conflict detection and 

surveillance quality parameters in the model. The sensitivity runs were conducted with respect to two 

baseline conditions selected from the SQ1 test matrix. Baseline condition 1 had no surveillance error while 

baseline condition 2 implemented the ADSB_2 surveillance error parameters set. For each sensitivity run 

only a single parameter value was changed relative to the corresponding baseline run and that parameter 

was constant over the entire run. Suitable parameter values were selected for conflict detection cycle period, 

detection and prediction horizon, and the various surveillance error model parameters. The conflict 

comparison criteria (δt, δH, δV) were investigated at multiplier values of 0.5, 1.0, 2.0, and infinity (disables 

conflict comparison) with respect to their baseline values. Table 10 lists the parameter values that were

analyzed in this test matrix for a total of 208 analysis runs.

Table 9. Surveillance quality test matrix SQ1 (full-factorial; 144 runs).

Parameter Values Units

CD Period (Tdet ) 15, 60 sec

Prediction Horizon (Tpred) 300, 600, 1200 sec

Surveillance Range (Rs) 50, 250, 1500 NM

Surveillance Error Source (SQmodel) None, Radar, ADSB_1, ADSB_2 -

Trajectory Prediction Symmetry Symmetric, Asymmetric -
26

C. Surveillance Quality Test Matrix 3 (SQ3)

Surveillance quality test matrix 3 was implemented to investigate the variability in the analysis metrics 

due to different traffic scenarios and due to Monte Carlo sampling of the surveillance error parameters. A 

set of 10 different traffic scenario randomizations, each with 30,799 aircraft and using the one hour 

maximum time shift, were used in this test matrix. For scenario variability, the baseline 1 and 2 case runs

from the SQ2 test matrix were each analyzed using the 10 different scenarios with symmetric and 

asymmetric trajectory predictions. For Monte Carlo sampling variability, the baseline 2 case run (with

ADSB_2) was run with 10 different random seeds and the same traffic scenario both with symmetric and 

asymmetric trajectory predictions. Table 11 shows the 60 analysis runs conducted to investigate variability.

D. Level and Quality of Intent Information Test Matrix 1 (IQ1)

Level and quality of intent information test matrix 1 is a partial factorial exploration of the conflict 

detection and level and quality of intent input parameters. The partial factorial matrix is the result of the 

full-factorial matrix with the reduction of the parameter combinations that do not provide meaningful 

results. For example, there was no need to analyze a level of intent (intent horizon) parameter of 20 minutes 

if trajectory prediction was only being performed on a 10 minute horizon; intent information beyond 10 

Table 10. Surveillance quality test matrix SQ2 (sensitivity runs; 208 runs).

Parameter Values Units

Nominal Parameter Values baseline 1: Tdet = 60, Tpred = 600, Rs = 1500, SQmodel = None

baseline 2: Tdet = 60, Tpred = 600, Rs = 1500, SQmodel = ADSB_2

-

CD Period (Tdet ) 15, 30, 60, 120, 300 s

Prediction Horizon (Tpred) 60, 120, 300, 600, 900, 1200 s

Surveillance Range (Rs) 20, 40, 80, 100, 200, 300, 500, 1500 NM

Surveillance Lag (tsl) 0, 1, 2, 5, 10, 15 s

Horizontal Position Error 

(σr)

0.001, 0.01, 0.1, 0.5 NM

Vertical Position Error (σa) 1, 10, 25, 50, 100, 150 ft

Groundspeed Error (σgs) 0.1, 1, 5, 10, 20, 50 NM/hr

Vertical Speed Error (σvs) 10, 25, 50, 100, 500, 1000 ft/min

Track Angle Error (σtrk) 0.1, 0.5, 1, 2, 5 deg

Same Conflict Tolerance 

(δt, δH, δV) Multiplier

0.5X, 1.0X, 2.0X, Inf -

Trajectory Prediction 

Symmetry

Symmetric, Asymmetric -

Table 11. Surveillance quality test matrix SQ3 (variability; 60 runs).

Variability Test Baseline Run Trajectory Prediction 

Symmetry

Number of 

Scenarios

Number of Runs 

Per Scenario

Scenario Variability

baseline 1 Symmetric 10 1

baseline 1 Asymmetric 10 1

baseline 2 Symmetric 10 1

baseline 2 Asymmetric 10 1

Sampling

Variability

baseline 2 Symmetric 1 10

baseline 2 Asymmetric 1 10

baseline 1: Tdet = 60, Tpred = 600, Rs = 1500, SQmodel = None

baseline 2: Tdet = 60, Tpred = 600, Rs = 1500, SQmodel = ADSB_2
27

minutes would have been ignored. The purpose of this test matrix was to generate a family of CD 

performance curves for the level of intent information and quality of intent information parameters.

Table 12 shows the parameter values that were investigated in the IQ1 test matrix. Conflict detection 

cycle period (Tdet) was analyzed at short and long values of 15 and 60 seconds, respectively. The conflict 

detection and trajectory prediction horizon (Tpred), and the level of intent (Tint), were each examined at the 

short, medium and long horizon levels of 5, 10, and 20 minutes. The vertical and horizontal quality of 

intent values were combined at the levels that provide equivalent information along the vertical and 

horizontal profiles (e.g., IQV1+IQH1) with the addition of the IQV1+IQH2 quality combination that is the 

equivalent of the ADS-B trajectory change report definition in the ADS-B MASPS. Trajectory prediction 

symmetry was again investigated using the symmetric and asymmetric approaches where, for the 

asymmetric cases, perfect trajectory prediction was assumed for the first aircraft in the conflict pair by 

setting the predicted trajectory equal to the true trajectory of that aircraft, and for the symmetric cases, both 

aircraft had the same quality of imperfect trajectory predictions. Table 12 lists the test matrix parameters, 

which constitute a total of 132 analysis runs from the meaningful parameter combinations.

E. Level and Quality of Intent Information Test Matrix 2 (IQ2)

Level and quality of intent information test matrix 2 investigated the impact of different traffic scenarios 

on the variability of the output metrics and the sensitivity of the metrics to the parameters associated with 

vertical and horizontal intent quality 4. The same set of 10 random traffic scenarios used in the SQ2 test 

matrix were used in test matrix IQ2a to investigate the variability of the output metrics with respect to 3 

baseline scenario runs, with symmetric and asymmetric trajectory predictions. Baseline runs 3-5 represent 

the conditions of an intent information quality at level 3 and with long, medium, and short intent horizons, 

respectively. Table 13 lists the details of the scenario variability runs for a total of 60 analysis runs.

Test matrix IQ2b, shown in Table 14, lists the parameter settings of baseline run 6 with medium 

prediction and medium intent horizons, and the intent information quality 4 parameters variations. The 

time interval for the horizontal points that the intent information quality 4 adds to the horizontal intent 

information quality 3 points was varied from 15 seconds to 300 seconds. The altitude interval for the 

vertical points that the vertical intent information quality 4 adds to the intent information quality 3 points 

was varied from 100 feet to 5000 feet. The analysis runs of test matrix IQ2b were run with symmetric and 

asymmetric trajectory predictions for a total of 36 analysis runs. 

Table 12. Level and quality of intent test matrix IQ1 (partial-factorial; 132 runs).

Parameter Values Units

CD Period (Tdet ) 15, 60 s

Prediction Horizon (Tpred) 300, 600, 1200 s

Level of Intent (Tint ) 300, 600, 1200 s

Quality of Intent IQV0+IQH0, IQV1+IQH1, IQV1+IQH2

1

, IQV2+IQH2, 

IQV3+IQH3, IQV4+IQH4

2

-

Trajectory Prediction 

Symmetry

Symmetric, Asymmetric -

Note: 1 Represents the closest equivalent to the ADS-B MASPS definition of a trajectory 

change report. 2 Default quality 4 intervals of 120 seconds and 1000 ft, per Table 3.
28

VII. Results and Discussion

The use of two separate conflict detection performance analysis methods required that some additional 

steps be taken to reconcile any differences in assumptions inherent in the analysis codes. Several 

comparison runs were made and the primary metrics compared in order to identify and correct these 

differences. The two analysis methods were shown to produce nearly identical results when run under the 

same input parameters and the same traffic scenarios. Table 15 shows the CD performance metrics for 

methods 1 and 2, and their differences, for a sample set of ten traffic scenarios. Each of the ten traffic 

scenarios is a random sample of 100 aircraft pairs with known LOS events from the set of time-shifted, 

recorded NAS traffic. Note that analysis method 2 was run with the recorded traffic data for these 

comparisons, as opposed to random circular scenarios.

Table 13. Level and quality of intent test matrix IQ2a (scenario variability; 60 runs).

Variability Test Baseline Run Trajectory Prediction 

Symmetry

Number of 

Scenarios

Number of Runs 

Per Scenario

Scenario Variability

baseline 3 Symmetric 10 1

baseline 3 Asymmetric 10 1

baseline 4 Symmetric 10 1

baseline 4 Asymmetric 10 1

baseline 5 Symmetric 10 1

baseline 5 Asymmetric 10 1

baseline 3: Tdet = 60, Tpred = 1200, Tint = 1200, IQV3, IQH3

baseline 4: Tdet = 60, Tpred = 1200, Tint = 600, IQV3, IQH3

baseline 5: Tdet = 60, Tpred = 1200, Tint = 300, IQV3, IQH3

Table 14. Level and quality of intent test matrix IQ2b (Q4 sensitivity runs; 36 runs).

Parameter Values Units

Nominal Parameter Values baseline 6: Tdet = 60, Tpred = 600, Tint = 600, IQV4

, IQH4

-

Time Interval for Intent Quality 4 

Horizontal Points (∆tQ4)

15, 30, 45, 60, 75, 90, 120, 180, 300 s

Altitude Interval for Intent Quality 4 

Vertical Points (∆altQ4)

100, 200, 300, 400, 500, 750, 1000, 2000, 5000 ft

Trajectory Prediction Symmetry Symmetric, Asymmetric -

Table 15. Comparison results for analysis methods 1 and 2 using the same 10 sample scenarios.

Sample 

Run

Method 1 Method 2 Difference

PFA PMA PMD ∆tLOS,mean

[s]

PFA PMA PMD ∆tLOS,mean

[s]

PFA PMA PMD ∆tLOS,mean

[s]

1 0.068 0.323 0.042 641 0.065 0.316 0.050 644 0.003 0.007 -0.008 -3

2 0.077 0.287 0.033 650 0.081 0.286 0.010 637 -0.004 0.001 0.023 13

3 0.081 0.319 0.030 648 0.083 0.321 0.030 632 -0.002 -0.002 0.000 16

4 0.064 0.296 0.053 661 0.074 0.298 0.040 646 -0.010 -0.002 0.013 15

5 0.103 0.348 0.043 613 0.098 0.335 0.060 619 0.005 0.013 -0.017 -6

6 0.083 0.318 0.034 647 0.080 0.307 0.050 658 0.003 0.011 -0.016 -11

7 0.075 0.339 0.008 649 0.075 0.334 0.010 656 0.000 0.005 -0.002 -7

8 0.076 0.290 0.016 605 0.077 0.293 0.020 606 -0.001 -0.003 -0.004 -1

9 0.065 0.305 0.009 678 0.065 0.304 0.020 677 0.000 0.001 -0.011 1

10 0.091 0.321 0.053 689 0.090 0.317 0.030 678 0.001 0.004 0.023 11

Note: analysis method 1 and 2 were both run with recorded traffic data scenarios; random circular scenarios were not used 

with analysis method 2 in these comparison runs.
29

Although the two analysis methods produced nearly identical results under the same parameter and 

scenario conditions, the two methods were expected to produce somewhat different results under the same 

parameter conditions when comparing recorded traffic scenarios to circular traffic scenarios. The primary 

reason for this difference is due to the fact that the recorded data has a small level of surveillance and sensor 

noise remaining even after the processing and filtering steps taken to clean the data. This small level of 

noise can have a significant impact on trajectory prediction accuracy, particularly in state-projection 

trajectory predictions where the predictions rely on velocity states obtained from differential derivatives. 

The second reason for the differences is that, for the circular scenarios, the intent information is perfect (the 

intent is identical to the kinematic flight plan and it is flown perfectly using kinematics). The significance 

of these differences is that, in general, the results produced using the circular scenarios of method 2 show 

better CD performance (lower false alert, missed alert, and missed detection probabilities, and higher values 

for mean time-to-LOS) than the results for the scenarios using recorded traffic data and analysis method 1. 

In some sense, the results from method 2 could be considered the upper bounds of CD performance since 

the conditions are more idealized, whereas, the results from method 1 could be considered closer to what 

could be expected in a real environment with noisy or imperfect data.

The following sections present the full set of results tables for the surveillance quality and intent 

information quality test matrices and the figures for a subset of those results, along with some discussion 

of the results. Comparison data runs using the analysis method 2 and random circular traffic scenarios are

also presented and compared where applicable. 

A. Surveillance Quality Results and Discussion

The results of the analysis runs for surveillance quality test matrix 1 (SQ1) are listed in Table 16-Table 

19. The figures associated with test matrix SQ1 are Figure 24-Figure 29. Figure 24 illustrates that the 

absence of intent information, or the reliance on state-projection trajectory predictions, produces poor 

conflict detection performance in all of the primary metrics. In addition, if surveillance error is present at 

the quality of current surveillance radar technologies, the performance is reduced by 50% or more in terms 

of false alerts and missed detections. Even at short trajectory prediction and conflict detection horizons of 

5 minutes, the CD performance in the absence of surveillance noise is poor with 48% false alerts, 46% 

missed alerts, 7% missed detections, and 158 seconds average time-to-LOS at first detection (Figure 25). 

There is a trade-off between trajectory prediction and the conflict detection horizon because higher 

detection horizons produce worst false alert, missed alert, and mean time-to-LOS metrics while missed 

detections are reduced. A surveillance range beyond 250 nautical miles appears to have minimal impact in 

improving the CD performance (Figure 26 and Figure 27), further solidifying the poor quality of stateprojection trajectory predictions. Conflict detection cycle periods of 15 and 60 seconds (Figure 28) 

produced nearly identical false alert and missed alert metrics for the same input parameters but missed 

detection were reduced by ~60% and mean time-to-LOS at first detection was increased by ~25% by using 

15 seconds instead of 60 seconds. Conflict detection performance using asymmetric predictions (perfect 

prediction for aircraft 1 in a pair) versus symmetric predictions had a positive impact on CD performance 

for all metrics (Figure 29).

The results of the analysis runs for surveillance quality test matrix 2 (SQ2) are listed in Table 20-Table 

23. Table 24 shows the results of a set of comparison runs using circular scenarios and analysis method 2. 

The figures associated with test matrix SQ2 are Figure 30-Figure 40. The comparison results of Table 24

are shown in Figure 31 and Figure 33. The sensitivity curves indicate that false alert and missed alert 

probabilities are most sensitive to conflict detection and trajectory prediction horizon (Figure 31) and 

vertical speed errors (Figure 37). Missed detection probability, in terms of percent change, is sensitive to 

most of the parameters tested, with the lowest sensitivity being to the vertical position error (Figure 35). 

The mean time-to-LOS metric is most sensitive to CD cycle period (Figure 30), conflict detection and 

prediction horizon (Figure 31), and surveillance range (Figure 32). The comparison results from analysis 

method 2 for conflict detection and trajectory prediction horizon sensitivity show trends that are in 

agreement with the results of analysis method 1 (Figure 31). A similar agreement in sensitivity curves 

between the two analysis methods can be seen in Figure 33 for the sensitivity of the metrics to the 
30

surveillance lag time parameter. The impact of the conflict comparison criteria on the CD performance 

metrics versus the conventional metrics definitions is that the conventional measurement would produce a 

less conservative performance (~4% lower false alert probability, ~6% lower missed alerts, ~3% lower 

missed detection, and ~60 second longer mean time-to-LOS metrics). Figure 40 confirms the improved 

conflict detection performance when using asymmetric trajectory predictions.

The results of the analysis runs for surveillance quality test matrix 3 (SQ3) are listed in Table 25 and 

Table 26. Table 27 and Table 28 present the summary statistics for the variability in the metrics due to 

different traffic scenarios and due to Monte Carlo sampling of the surveillance error parameters, including 

one case using analysis method 2. The variability of the CD performance metrics to different traffic 

scenarios with the same input parameters appears to be small with ~0.33% or less of standard deviation in 

the false alert probability, less than 0.3% of standard deviation in missed alert probability, less than 0.5% 

of standard deviation in the missed detection probability, and approximately 2 seconds of standard deviation 

for the mean time-to-LOS metric (Table 27). The variability of the CD performance metrics to the Monte 

Carlo sampling of the surveillance error parameters using the same traffic scenario is also small, with less 

than 0.1% of standard deviation in the false and missed alert probabilities, less than 0.25% of standard 

deviation in the missed detection probability, and less than 1.5 seconds of standard deviation for the mean 

time-to-LOS metric (Table 28). The variability is likely dependent on the number of LOS and the number 

of alerts identified during each scenario run; all of the variability scenario runs using the recorded traffic 

data had ~15,000 true LOS and between 135,000 and 237,000 conflict alerts. 
31

Surveillance Quality Results Tables

Table 16. Surveillance quality run data with symmetric predictions and 60 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

SQmodel Rs

[NM]

PFA PMA PMD ∆tLOS,mean

[s]

sq1_001 60 1200 none 1500 0.752 0.742 0.063 277

sq1_002 60 1200 radar 1500 0.931 0.905 0.182 134

sq1_003 60 1200 ADSB_1 1500 0.914 0.878 0.103 172

sq1_004 60 1200 ADSB_2 1500 0.881 0.810 0.065 259

sq1_005 60 1200 none 250 0.741 0.743 0.063 275

sq1_006 60 1200 radar 250 0.929 0.905 0.184 135

sq1_007 60 1200 ADSB_1 250 0.912 0.879 0.110 169

sq1_008 60 1200 ADSB_2 250 0.875 0.810 0.061 255

sq1_009 60 1200 none 50 0.576 0.803 0.069 188

sq1_010 60 1200 radar 50 0.865 0.910 0.193 109

sq1_011 60 1200 ADSB_1 50 0.833 0.887 0.115 133

sq1_012 60 1200 ADSB_2 50 0.734 0.837 0.069 176

sq1_013 60 600 none 1500 0.647 0.624 0.064 214

sq1_014 60 600 radar 1500 0.904 0.838 0.186 125

sq1_015 60 600 ADSB_1 1500 0.879 0.792 0.109 154

sq1_016 60 600 ADSB_2 1500 0.815 0.687 0.067 211

sq1_017 60 600 none 250 0.647 0.624 0.064 214

sq1_018 60 600 radar 250 0.902 0.835 0.191 129

sq1_019 60 600 ADSB_1 250 0.879 0.793 0.111 156

sq1_020 60 600 ADSB_2 250 0.815 0.686 0.065 211

sq1_021 60 600 none 50 0.523 0.677 0.069 172

sq1_022 60 600 radar 50 0.854 0.844 0.192 109

sq1_023 60 600 ADSB_1 50 0.815 0.805 0.115 129

sq1_024 60 600 ADSB_2 50 0.698 0.721 0.070 167

sq1_025 60 300 none 1500 0.483 0.460 0.069 148

sq1_026 60 300 radar 1500 0.851 0.708 0.193 104

sq1_027 60 300 ADSB_1 1500 0.814 0.641 0.118 119

sq1_028 60 300 ADSB_2 1500 0.703 0.502 0.070 147

sq1_029 60 300 none 250 0.483 0.460 0.069 148

sq1_030 60 300 radar 250 0.851 0.710 0.195 103

sq1_031 60 300 ADSB_1 250 0.812 0.641 0.120 120

sq1_032 60 300 ADSB_2 250 0.702 0.503 0.072 147

sq1_033 60 300 none 50 0.433 0.482 0.070 139

sq1_034 60 300 radar 50 0.830 0.714 0.193 100

sq1_035 60 300 ADSB_1 50 0.783 0.650 0.118 112

sq1_036 60 300 ADSB_2 50 0.644 0.522 0.073 137
32

Table 17. Surveillance quality run data with symmetric predictions and 15 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

SQmodel Rs

[NM]

PFA PMA PMD ∆tLOS,mean

[s]

sq1_037 15 1200 none 1500 0.751 0.742 0.003 316

sq1_038 15 1200 radar 1500 0.929 0.903 0.022 220

sq1_039 15 1200 ADSB_1 1500 0.914 0.878 0.015 278

sq1_040 15 1200 ADSB_2 1500 0.880 0.809 0.009 353

sq1_041 15 1200 none 250 0.739 0.743 0.003 314

sq1_042 15 1200 radar 250 0.927 0.904 0.022 220

sq1_043 15 1200 ADSB_1 250 0.911 0.878 0.011 274

sq1_044 15 1200 ADSB_2 250 0.874 0.809 0.010 347

sq1_045 15 1200 none 50 0.573 0.803 0.004 215

sq1_046 15 1200 radar 50 0.863 0.909 0.022 169

sq1_047 15 1200 ADSB_1 50 0.831 0.886 0.013 203

sq1_048 15 1200 ADSB_2 50 0.731 0.836 0.008 234

sq1_049 15 600 none 1500 0.644 0.624 0.003 246

sq1_050 15 600 radar 1500 0.901 0.834 0.025 202

sq1_051 15 600 ADSB_1 1500 0.878 0.793 0.014 236

sq1_052 15 600 ADSB_2 1500 0.815 0.687 0.010 274

sq1_053 15 600 none 250 0.644 0.624 0.003 246

sq1_054 15 600 radar 250 0.901 0.835 0.022 203

sq1_055 15 600 ADSB_1 250 0.879 0.793 0.014 237

sq1_056 15 600 ADSB_2 250 0.815 0.687 0.010 275

sq1_057 15 600 none 50 0.519 0.677 0.004 196

sq1_058 15 600 radar 50 0.852 0.842 0.024 166

sq1_059 15 600 ADSB_1 50 0.814 0.805 0.012 193

sq1_060 15 600 ADSB_2 50 0.696 0.721 0.009 214

sq1_061 15 300 none 1500 0.479 0.460 0.004 171

sq1_062 15 300 radar 1500 0.849 0.708 0.023 154

sq1_063 15 300 ADSB_1 1500 0.812 0.640 0.013 173

sq1_064 15 300 ADSB_2 1500 0.701 0.502 0.010 186

sq1_065 15 300 none 250 0.479 0.460 0.004 171

sq1_066 15 300 radar 250 0.849 0.707 0.025 155

sq1_067 15 300 ADSB_1 250 0.813 0.640 0.015 174

sq1_068 15 300 ADSB_2 250 0.701 0.503 0.009 184

sq1_069 15 300 none 50 0.427 0.481 0.004 160

sq1_070 15 300 radar 50 0.827 0.713 0.024 146

sq1_071 15 300 ADSB_1 50 0.781 0.649 0.016 163

sq1_072 15 300 ADSB_2 50 0.639 0.522 0.010 172
33

Table 18. Surveillance quality run data with asymmetric predictions and 60 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

SQmodel Rs

[NM]

PFA PMA PMD ∆tLOS,mean

[s]

sq1_073 60 1200 none 1500 0.557 0.586 0.039 420

sq1_074 60 1200 radar 1500 0.896 0.850 0.151 303

sq1_075 60 1200 ADSB_1 1500 0.864 0.797 0.050 354

sq1_076 60 1200 ADSB_2 1500 0.812 0.696 0.038 420

sq1_077 60 1200 none 250 0.544 0.589 0.039 416

sq1_078 60 1200 radar 250 0.892 0.849 0.150 299

sq1_079 60 1200 ADSB_1 250 0.858 0.797 0.049 350

sq1_080 60 1200 ADSB_2 250 0.801 0.698 0.039 414

sq1_081 60 1200 none 50 0.416 0.724 0.042 263

sq1_082 60 1200 radar 50 0.808 0.872 0.166 198

sq1_083 60 1200 ADSB_1 50 0.749 0.831 0.056 227

sq1_084 60 1200 ADSB_2 50 0.612 0.770 0.045 261

sq1_085 60 600 none 1500 0.446 0.456 0.040 297

sq1_086 60 600 radar 1500 0.857 0.754 0.157 230

sq1_087 60 600 ADSB_1 1500 0.814 0.679 0.053 252

sq1_088 60 600 ADSB_2 1500 0.721 0.543 0.040 294

sq1_089 60 600 none 250 0.446 0.456 0.040 297

sq1_090 60 600 radar 250 0.857 0.756 0.159 230

sq1_091 60 600 ADSB_1 250 0.814 0.679 0.052 253

sq1_092 60 600 ADSB_2 250 0.721 0.544 0.041 293

sq1_093 60 600 none 50 0.357 0.569 0.042 227

sq1_094 60 600 radar 50 0.792 0.784 0.171 177

sq1_095 60 600 ADSB_1 50 0.725 0.717 0.056 194

sq1_096 60 600 ADSB_2 50 0.562 0.621 0.043 221

sq1_097 60 300 none 1500 0.311 0.310 0.043 185

sq1_098 60 300 radar 1500 0.796 0.614 0.178 148

sq1_099 60 300 ADSB_1 1500 0.734 0.507 0.061 158

sq1_100 60 300 ADSB_2 1500 0.589 0.361 0.046 182

sq1_101 60 300 none 250 0.311 0.310 0.043 185

sq1_102 60 300 radar 250 0.796 0.614 0.181 149

sq1_103 60 300 ADSB_1 250 0.736 0.511 0.062 156

sq1_104 60 300 ADSB_2 250 0.589 0.363 0.046 181

sq1_105 60 300 none 50 0.278 0.354 0.043 170

sq1_106 60 300 radar 50 0.766 0.632 0.187 137

sq1_107 60 300 ADSB_1 50 0.691 0.528 0.064 146

sq1_108 60 300 ADSB_2 50 0.511 0.400 0.049 167
34

Table 19. Surveillance quality run data with asymmetric predictions and 15 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

SQmodel Rs

[NM]

PFA PMA PMD ∆tLOS,mean

[s]

sq1_109 15 1200 none 1500 0.555 0.585 0.002 458

sq1_110 15 1200 radar 1500 0.895 0.849 0.049 472

sq1_111 15 1200 ADSB_1 1500 0.864 0.797 0.007 515

sq1_112 15 1200 ADSB_2 1500 0.811 0.696 0.006 522

sq1_113 15 1200 none 250 0.542 0.588 0.002 453

sq1_114 15 1200 radar 250 0.891 0.848 0.049 464

sq1_115 15 1200 ADSB_1 250 0.858 0.797 0.007 508

sq1_116 15 1200 ADSB_2 250 0.800 0.697 0.006 513

sq1_117 15 1200 none 50 0.414 0.724 0.002 293

sq1_118 15 1200 radar 50 0.809 0.872 0.051 297

sq1_119 15 1200 ADSB_1 50 0.748 0.831 0.006 327

sq1_120 15 1200 ADSB_2 50 0.609 0.769 0.006 324

sq1_121 15 600 none 1500 0.444 0.455 0.002 325

sq1_122 15 600 radar 1500 0.856 0.754 0.053 332

sq1_123 15 600 ADSB_1 1500 0.814 0.679 0.007 351

sq1_124 15 600 ADSB_2 1500 0.720 0.544 0.006 354

sq1_125 15 600 none 250 0.444 0.455 0.002 325

sq1_126 15 600 radar 250 0.856 0.754 0.049 331

sq1_127 15 600 ADSB_1 250 0.813 0.678 0.007 352

sq1_128 15 600 ADSB_2 250 0.720 0.544 0.006 353

sq1_129 15 600 none 50 0.355 0.568 0.002 252

sq1_130 15 600 radar 50 0.791 0.783 0.052 251

sq1_131 15 600 ADSB_1 50 0.724 0.716 0.008 269

sq1_132 15 600 ADSB_2 50 0.562 0.621 0.007 268

sq1_133 15 300 none 1500 0.308 0.309 0.002 206

sq1_134 15 300 radar 1500 0.793 0.612 0.067 203

sq1_135 15 300 ADSB_1 1500 0.734 0.506 0.008 211

sq1_136 15 300 ADSB_2 1500 0.586 0.361 0.007 214

sq1_137 15 300 none 250 0.308 0.309 0.002 206

sq1_138 15 300 radar 250 0.794 0.613 0.064 202

sq1_139 15 300 ADSB_1 250 0.733 0.506 0.009 212

sq1_140 15 300 ADSB_2 250 0.587 0.361 0.007 214

sq1_141 15 300 none 50 0.274 0.353 0.002 191

sq1_142 15 300 radar 50 0.764 0.630 0.065 187

sq1_143 15 300 ADSB_1 50 0.690 0.528 0.007 195

sq1_144 15 300 ADSB_2 50 0.508 0.398 0.007 198
35

Table 20. Surveillance quality sensitivity run data with symmetric predictions and no baseline surveillance error.

Run Tdet

[s]

Tpred

[s]

Rs

[NM]

tsl

[s]

σr

[NM]

σa

[ft]

σgs

[knots]

σvs

[ft/min]

σtrk

[deg]

Same X PFA PMA PMD ∆tLOS,mean

[s]

sq2_001 60 600 1500 0 0 0 0 0 0 1 0.647 0.624 0.064 214

sq2_002 15 600 1500 0 0 0 0 0 0 1 0.644 0.624 0.003 246

sq2_003 30 600 1500 0 0 0 0 0 0 1 0.645 0.624 0.016 230

sq2_004 120 600 1500 0 0 0 0 0 0 1 0.650 0.624 0.193 205

sq2_005 300 600 1500 0 0 0 0 0 0 1 0.666 0.633 0.463 203

sq2_006 60 60 1500 0 0 0 0 0 0 1 0.146 0.138 0.138 30

sq2_007 60 120 1500 0 0 0 0 0 0 1 0.257 0.245 0.083 70

sq2_008 60 300 1500 0 0 0 0 0 0 1 0.483 0.460 0.069 148

sq2_009 60 900 1500 0 0 0 0 0 0 1 0.712 0.698 0.064 251

sq2_010 60 1200 1500 0 0 0 0 0 0 1 0.752 0.742 0.063 277

sq2_011 60 600 20 0 0 0 0 0 0 1 0.478 0.771 0.081 114

sq2_012 60 600 40 0 0 0 0 0 0 1 0.507 0.699 0.071 158

sq2_013 60 600 80 0 0 0 0 0 0 1 0.573 0.642 0.067 197

sq2_014 60 600 100 0 0 0 0 0 0 1 0.601 0.632 0.065 206

sq2_015 60 600 200 0 0 0 0 0 0 1 0.647 0.624 0.064 214

sq2_016 60 600 300 0 0 0 0 0 0 1 0.647 0.624 0.064 214

sq2_017 60 600 500 0 0 0 0 0 0 1 0.647 0.624 0.064 214

sq2_018 60 600 1500 1 0 0 0 0 0 1 0.647 0.625 0.068 214

sq2_019 60 600 1500 2 0 0 0 0 0 1 0.649 0.627 0.071 213

sq2_020 60 600 1500 5 0 0 0 0 0 1 0.659 0.634 0.083 213

sq2_021 60 600 1500 10 0 0 0 0 0 1 0.670 0.643 0.106 212

sq2_022 60 600 1500 15 0 0 0 0 0 1 0.680 0.652 0.126 212

sq2_023 60 600 1500 0 0.001 0 0 0 0 1 0.647 0.624 0.064 214

sq2_024 60 600 1500 0 0.01 0 0 0 0 1 0.647 0.624 0.064 214

sq2_025 60 600 1500 0 0.1 0 0 0 0 1 0.648 0.626 0.068 215

sq2_026 60 600 1500 0 0.5 0 0 0 0 1 0.663 0.642 0.084 217

sq2_027 60 600 1500 0 0 1 0 0 0 1 0.647 0.624 0.064 213

sq2_028 60 600 1500 0 0 10 0 0 0 1 0.648 0.624 0.066 213

sq2_029 60 600 1500 0 0 25 0 0 0 1 0.648 0.625 0.068 214

sq2_030 60 600 1500 0 0 50 0 0 0 1 0.652 0.626 0.071 215

sq2_031 60 600 1500 0 0 100 0 0 0 1 0.688 0.625 0.070 220

sq2_032 60 600 1500 0 0 150 0 0 0 1 0.724 0.626 0.075 223

sq2_033 60 600 1500 0 0 0 0.1 0 0 1 0.647 0.624 0.065 214

sq2_034 60 600 1500 0 0 0 1 0 0 1 0.647 0.624 0.065 214

sq2_035 60 600 1500 0 0 0 5 0 0 1 0.653 0.630 0.066 213

sq2_036 60 600 1500 0 0 0 10 0 0 1 0.665 0.641 0.069 210

sq2_037 60 600 1500 0 0 0 20 0 0 1 0.691 0.663 0.073 205

sq2_038 60 600 1500 0 0 0 50 0 0 1 0.758 0.717 0.091 187

sq2_039 60 600 1500 0 0 0 0 10 0 1 0.657 0.623 0.064 216

sq2_040 60 600 1500 0 0 0 0 25 0 1 0.703 0.619 0.057 222

sq2_041 60 600 1500 0 0 0 0 50 0 1 0.744 0.624 0.054 222

sq2_042 60 600 1500 0 0 0 0 100 0 1 0.785 0.650 0.052 221

sq2_043 60 600 1500 0 0 0 0 500 0 1 0.877 0.792 0.090 147

sq2_044 60 600 1500 0 0 0 0 1000 0 1 0.912 0.860 0.199 101

sq2_045 60 600 1500 0 0 0 0 0 0.1 1 0.647 0.624 0.064 214

sq2_046 60 600 1500 0 0 0 0 0 0.5 1 0.648 0.626 0.064 213

sq2_047 60 600 1500 0 0 0 0 0 1 1 0.653 0.632 0.065 212

sq2_048 60 600 1500 0 0 0 0 0 2 1 0.664 0.647 0.069 209

sq2_049 60 600 1500 0 0 0 0 0 5 1 0.706 0.698 0.081 190

sq2_050 60 600 1500 0 0 0 0 0 0 0.5 0.684 0.664 0.086 188

sq2_051 60 600 1500 0 0 0 0 0 0 2 0.624 0.595 0.049 234

sq2_052 60 600 1500 0 0 0 0 0 0 Inf. 0.606 0.560 0.036 277
36

Table 21. Surveillance quality sensitivity run data with symmetric predictions and ADSB_2 baseline surveillance 

error.

Run Tdet

[s]

Tpred

[s]

Rs

[NM]

tsl

[s]

σr

[NM]

σa

[ft]

σgs

[knots]

σvs

[ft/min]

σtrk

[deg]

Same X PFA PMA PMD ∆tLOS,mean

[s]

sq2_053 60 600 1500 1 0.0081 73.8 1 150 1 1 0.815 0.684 0.061 211

sq2_054 15 600 1500 1 0.0081 73.8 1 150 1 1 0.815 0.687 0.008 275

sq2_055 30 600 1500 1 0.0081 73.8 1 150 1 1 0.814 0.686 0.021 243

sq2_056 120 600 1500 1 0.0081 73.8 1 150 1 1 0.818 0.689 0.193 182

sq2_057 300 600 1500 1 0.0081 73.8 1 150 1 1 0.826 0.698 0.496 157

sq2_058 60 60 1500 1 0.0081 73.8 1 150 1 1 0.416 0.183 0.183 30

sq2_059 60 120 1500 1 0.0081 73.8 1 150 1 1 0.519 0.275 0.094 69

sq2_060 60 300 1500 1 0.0081 73.8 1 150 1 1 0.703 0.503 0.070 146

sq2_061 60 900 1500 1 0.0081 73.8 1 150 1 1 0.858 0.766 0.066 242

sq2_062 60 1200 1500 1 0.0081 73.8 1 150 1 1 0.881 0.809 0.065 258

sq2_063 60 600 20 1 0.0081 73.8 1 150 1 1 0.622 0.795 0.092 111

sq2_064 60 600 40 1 0.0081 73.8 1 150 1 1 0.673 0.737 0.074 153

sq2_065 60 600 80 1 0.0081 73.8 1 150 1 1 0.754 0.699 0.067 192

sq2_066 60 600 100 1 0.0081 73.8 1 150 1 1 0.780 0.692 0.066 202

sq2_067 60 600 200 1 0.0081 73.8 1 150 1 1 0.816 0.687 0.065 212

sq2_068 60 600 300 1 0.0081 73.8 1 150 1 1 0.816 0.687 0.065 212

sq2_069 60 600 500 1 0.0081 73.8 1 150 1 1 0.816 0.687 0.064 210

sq2_070 60 600 1500 1 0.0081 73.8 1 150 1 1 0.816 0.687 0.064 210

sq2_071 60 600 1500 2 0.0081 73.8 1 150 1 1 0.817 0.689 0.067 209

sq2_072 60 600 1500 5 0.0081 73.8 1 150 1 1 0.821 0.695 0.077 208

sq2_073 60 600 1500 10 0.0081 73.8 1 150 1 1 0.827 0.703 0.092 207

sq2_074 60 600 1500 15 0.0081 73.8 1 150 1 1 0.833 0.712 0.108 205

sq2_075 60 600 1500 1 0.001 73.8 1 150 1 1 0.815 0.686 0.064 211

sq2_076 60 600 1500 1 0.01 73.8 1 150 1 1 0.815 0.685 0.064 211

sq2_077 60 600 1500 1 0.1 73.8 1 150 1 1 0.816 0.687 0.067 211

sq2_078 60 600 1500 1 0.5 73.8 1 150 1 1 0.825 0.701 0.083 211

sq2_079 60 600 1500 1 0.0081 1 1 150 1 1 0.812 0.683 0.058 208

sq2_080 60 600 1500 1 0.0081 10 1 150 1 1 0.814 0.684 0.058 209

sq2_081 60 600 1500 1 0.0081 25 1 150 1 1 0.814 0.684 0.061 208

sq2_082 60 600 1500 1 0.0081 50 1 150 1 1 0.813 0.685 0.063 212

sq2_083 60 600 1500 1 0.0081 100 1 150 1 1 0.818 0.688 0.066 210

sq2_084 60 600 1500 1 0.0081 150 1 150 1 1 0.820 0.693 0.076 213

sq2_085 60 600 1500 1 0.0081 73.8 0.1 150 1 1 0.815 0.686 0.063 210

sq2_086 60 600 1500 1 0.0081 73.8 1 150 1 1 0.817 0.689 0.067 210

sq2_087 60 600 1500 1 0.0081 73.8 5 150 1 1 0.818 0.690 0.065 209

sq2_088 60 600 1500 1 0.0081 73.8 10 150 1 1 0.821 0.695 0.065 206

sq2_089 60 600 1500 1 0.0081 73.8 20 150 1 1 0.830 0.708 0.074 200

sq2_090 60 600 1500 1 0.0081 73.8 50 150 1 1 0.858 0.747 0.094 180

sq2_091 60 600 1500 1 0.0081 73.8 1 10 1 1 0.690 0.635 0.072 216

sq2_092 60 600 1500 1 0.0081 73.8 1 25 1 1 0.725 0.634 0.071 221

sq2_093 60 600 1500 1 0.0081 73.8 1 50 1 1 0.757 0.638 0.063 222

sq2_094 60 600 1500 1 0.0081 73.8 1 100 1 1 0.794 0.661 0.063 220

sq2_095 60 600 1500 1 0.0081 73.8 1 500 1 1 0.881 0.798 0.100 144

sq2_096 60 600 1500 1 0.0081 73.8 1 1000 1 1 0.915 0.865 0.214 99

sq2_097 60 600 1500 1 0.0081 73.8 1 150 0.1 1 0.813 0.681 0.060 212

sq2_098 60 600 1500 1 0.0081 73.8 1 150 0.5 1 0.814 0.684 0.061 212

sq2_099 60 600 1500 1 0.0081 73.8 1 150 1 1 0.815 0.686 0.066 212

sq2_100 60 600 1500 1 0.0081 73.8 1 150 2 1 0.820 0.696 0.066 205

sq2_101 60 600 1500 1 0.0081 73.8 1 150 5 1 0.836 0.733 0.082 184

sq2_102 60 600 1500 1 0.0081 73.8 1 150 1 0.5 0.847 0.741 0.096 166

sq2_103 60 600 1500 1 0.0081 73.8 1 150 1 2 0.799 0.656 0.047 240

sq2_104 60 600 1500 1 0.0081 73.8 1 150 1 Inf. 0.786 0.615 0.026 288
37

Table 22. Surveillance quality sensitivity run data with asymmetric predictions and no baseline surveillance 

error.

Run Tdet

[s]

Tpred

[s]

Rs

[NM]

tsl

[s]

σr

[NM]

σa

[ft]

σgs

[knots]

σvs

[ft/min]

σtrk

[deg]

Same X PFA PMA PMD ∆tLOS,mean

[s]

sq2_105 60 600 1500 0 0 0 0 0 0 1 0.446 0.456 0.040 297

sq2_106 15 600 1500 0 0 0 0 0 0 1 0.444 0.455 0.002 325

sq2_107 30 600 1500 0 0 0 0 0 0 1 0.445 0.455 0.010 311

sq2_108 120 600 1500 0 0 0 0 0 0 1 0.450 0.458 0.121 286

sq2_109 300 600 1500 0 0 0 0 0 0 1 0.465 0.463 0.309 269

sq2_110 60 60 1500 0 0 0 0 0 0 1 0.087 0.084 0.084 31

sq2_111 60 120 1500 0 0 0 0 0 0 1 0.158 0.154 0.051 78

sq2_112 60 300 1500 0 0 0 0 0 0 1 0.311 0.310 0.043 185

sq2_113 60 900 1500 0 0 0 0 0 0 1 0.511 0.534 0.039 369

sq2_114 60 1200 1500 0 0 0 0 0 0 1 0.557 0.586 0.039 420

sq2_115 60 600 20 0 0 0 0 0 0 1 0.338 0.699 0.048 151

sq2_116 60 600 40 0 0 0 0 0 0 1 0.349 0.602 0.043 207

sq2_117 60 600 80 0 0 0 0 0 0 1 0.388 0.505 0.041 266

sq2_118 60 600 100 0 0 0 0 0 0 1 0.409 0.481 0.040 281

sq2_119 60 600 200 0 0 0 0 0 0 1 0.446 0.456 0.040 297

sq2_120 60 600 300 0 0 0 0 0 0 1 0.446 0.456 0.040 297

sq2_121 60 600 500 0 0 0 0 0 0 1 0.446 0.456 0.040 297

sq2_122 60 600 1500 1 0 0 0 0 0 1 0.449 0.459 0.045 298

sq2_123 60 600 1500 2 0 0 0 0 0 1 0.455 0.465 0.057 300

sq2_124 60 600 1500 5 0 0 0 0 0 1 0.482 0.487 0.089 303

sq2_125 60 600 1500 10 0 0 0 0 0 1 0.523 0.526 0.154 307

sq2_126 60 600 1500 15 0 0 0 0 0 1 0.561 0.562 0.213 309

sq2_127 60 600 1500 0 0.001 0 0 0 0 1 0.446 0.456 0.040 297

sq2_128 60 600 1500 0 0.01 0 0 0 0 1 0.446 0.456 0.040 297

sq2_129 60 600 1500 0 0.1 0 0 0 0 1 0.448 0.458 0.042 298

sq2_130 60 600 1500 0 0.5 0 0 0 0 1 0.462 0.471 0.048 300

sq2_131 60 600 1500 0 0 1 0 0 0 1 0.446 0.456 0.040 297

sq2_132 60 600 1500 0 0 10 0 0 0 1 0.447 0.456 0.040 297

sq2_133 60 600 1500 0 0 25 0 0 0 1 0.448 0.457 0.041 298

sq2_134 60 600 1500 0 0 50 0 0 0 1 0.450 0.459 0.043 299

sq2_135 60 600 1500 0 0 100 0 0 0 1 0.479 0.461 0.046 301

sq2_136 60 600 1500 0 0 150 0 0 0 1 0.537 0.464 0.047 303

sq2_137 60 600 1500 0 0 0 0.1 0 0 1 0.446 0.456 0.040 297

sq2_138 60 600 1500 0 0 0 1 0 0 1 0.447 0.456 0.040 297

sq2_139 60 600 1500 0 0 0 5 0 0 1 0.454 0.462 0.040 297

sq2_140 60 600 1500 0 0 0 10 0 0 1 0.467 0.475 0.040 296

sq2_141 60 600 1500 0 0 0 20 0 0 1 0.501 0.503 0.043 290

sq2_142 60 600 1500 0 0 0 50 0 0 1 0.594 0.574 0.052 272

sq2_143 60 600 1500 0 0 0 0 10 0 1 0.453 0.457 0.039 298

sq2_144 60 600 1500 0 0 0 0 25 0 1 0.523 0.457 0.035 302

sq2_145 60 600 1500 0 0 0 0 50 0 1 0.600 0.463 0.033 303

sq2_146 60 600 1500 0 0 0 0 100 0 1 0.672 0.496 0.032 299

sq2_147 60 600 1500 0 0 0 0 500 0 1 0.814 0.681 0.043 247

sq2_148 60 600 1500 0 0 0 0 1000 0 1 0.860 0.772 0.084 199

sq2_149 60 600 1500 0 0 0 0 0 0.1 1 0.446 0.456 0.040 297

sq2_150 60 600 1500 0 0 0 0 0 0.5 1 0.449 0.459 0.040 297

sq2_151 60 600 1500 0 0 0 0 0 1 1 0.456 0.466 0.040 296

sq2_152 60 600 1500 0 0 0 0 0 2 1 0.472 0.484 0.041 294

sq2_153 60 600 1500 0 0 0 0 0 5 1 0.527 0.545 0.045 281

sq2_154 60 600 1500 0 0 0 0 0 0 0.5 0.481 0.491 0.051 278

sq2_155 60 600 1500 0 0 0 0 0 0 2 0.425 0.432 0.031 312

sq2_156 60 600 1500 0 0 0 0 0 0 Inf. 0.409 0.400 0.024 341
38

Table 23. Surveillance quality sensitivity run data with asymmetric predictions and ADSB_2 baseline surveillance 

error.

Run Tdet

[s]

Tpred

[s]

Rs

[NM]

tsl

[s]

σr

[NM]

σa

[ft]

σgs

[knots]

σvs

[ft/min]

σtrk

[deg]

Same X PFA PMA PMD ∆tLOS,mean

[s]

sq2_157 60 600 1500 1 0.0081 73.8 1 150 1 1 0.720 0.543 0.040 294

sq2_158 15 600 1500 1 0.0081 73.8 1 150 1 1 0.719 0.543 0.006 353

sq2_159 30 600 1500 1 0.0081 73.8 1 150 1 1 0.720 0.544 0.014 325

sq2_160 120 600 1500 1 0.0081 73.8 1 150 1 1 0.721 0.545 0.118 261

sq2_161 300 600 1500 1 0.0081 73.8 1 150 1 1 0.732 0.555 0.342 220

sq2_162 60 60 1500 1 0.0081 73.8 1 150 1 1 0.299 0.151 0.151 31

sq2_163 60 120 1500 1 0.0081 73.8 1 150 1 1 0.406 0.199 0.069 76

sq2_164 60 300 1500 1 0.0081 73.8 1 150 1 1 0.587 0.362 0.046 181

sq2_165 60 900 1500 1 0.0081 73.8 1 150 1 1 0.778 0.639 0.040 369

sq2_166 60 1200 1500 1 0.0081 73.8 1 150 1 1 0.812 0.695 0.038 420

sq2_167 60 600 20 1 0.0081 73.8 1 150 1 1 0.480 0.728 0.063 151

sq2_168 60 600 40 1 0.0081 73.8 1 150 1 1 0.534 0.646 0.047 204

sq2_169 60 600 80 1 0.0081 73.8 1 150 1 1 0.632 0.574 0.042 260

sq2_170 60 600 100 1 0.0081 73.8 1 150 1 1 0.667 0.560 0.040 275

sq2_171 60 600 200 1 0.0081 73.8 1 150 1 1 0.721 0.545 0.040 294

sq2_172 60 600 300 1 0.0081 73.8 1 150 1 1 0.721 0.544 0.039 293

sq2_173 60 600 500 1 0.0081 73.8 1 150 1 1 0.721 0.544 0.040 294

sq2_174 60 600 1500 1 0.0081 73.8 1 150 1 1 0.721 0.544 0.040 294

sq2_175 60 600 1500 2 0.0081 73.8 1 150 1 1 0.724 0.549 0.047 294

sq2_176 60 600 1500 5 0.0081 73.8 1 150 1 1 0.736 0.567 0.075 296

sq2_177 60 600 1500 10 0.0081 73.8 1 150 1 1 0.755 0.597 0.134 300

sq2_178 60 600 1500 15 0.0081 73.8 1 150 1 1 0.774 0.628 0.190 301

sq2_179 60 600 1500 1 0.001 73.8 1 150 1 1 0.721 0.544 0.041 294

sq2_180 60 600 1500 1 0.01 73.8 1 150 1 1 0.721 0.543 0.042 295

sq2_181 60 600 1500 1 0.1 73.8 1 150 1 1 0.722 0.546 0.042 293

sq2_182 60 600 1500 1 0.5 73.8 1 150 1 1 0.728 0.555 0.045 293

sq2_183 60 600 1500 1 0.0081 1 1 150 1 1 0.718 0.541 0.038 293

sq2_184 60 600 1500 1 0.0081 10 1 150 1 1 0.718 0.541 0.038 293

sq2_185 60 600 1500 1 0.0081 25 1 150 1 1 0.718 0.542 0.038 293

sq2_186 60 600 1500 1 0.0081 50 1 150 1 1 0.719 0.542 0.040 294

sq2_187 60 600 1500 1 0.0081 100 1 150 1 1 0.722 0.544 0.043 294

sq2_188 60 600 1500 1 0.0081 150 1 150 1 1 0.727 0.547 0.045 295

sq2_189 60 600 1500 1 0.0081 73.8 0.1 150 1 1 0.721 0.543 0.041 294

sq2_190 60 600 1500 1 0.0081 73.8 1 150 1 1 0.721 0.544 0.042 295

sq2_191 60 600 1500 1 0.0081 73.8 5 150 1 1 0.723 0.548 0.041 292

sq2_192 60 600 1500 1 0.0081 73.8 10 150 1 1 0.728 0.555 0.040 291

sq2_193 60 600 1500 1 0.0081 73.8 20 150 1 1 0.740 0.571 0.045 284

sq2_194 60 600 1500 1 0.0081 73.8 50 150 1 1 0.778 0.625 0.054 263

sq2_195 60 600 1500 1 0.0081 73.8 1 10 1 1 0.486 0.473 0.048 300

sq2_196 60 600 1500 1 0.0081 73.8 1 25 1 1 0.551 0.474 0.045 302

sq2_197 60 600 1500 1 0.0081 73.8 1 50 1 1 0.617 0.478 0.043 304

sq2_198 60 600 1500 1 0.0081 73.8 1 100 1 1 0.684 0.510 0.040 299

sq2_199 60 600 1500 1 0.0081 73.8 1 500 1 1 0.819 0.689 0.052 248

sq2_200 60 600 1500 1 0.0081 73.8 1 1000 1 1 0.864 0.777 0.096 199

sq2_201 60 600 1500 1 0.0081 73.8 1 150 0.1 1 0.717 0.538 0.041 295

sq2_202 60 600 1500 1 0.0081 73.8 1 150 0.5 1 0.718 0.540 0.039 294

sq2_203 60 600 1500 1 0.0081 73.8 1 150 1 1 0.721 0.545 0.041 294

sq2_204 60 600 1500 1 0.0081 73.8 1 150 2 1 0.727 0.556 0.041 289

sq2_205 60 600 1500 1 0.0081 73.8 1 150 5 1 0.750 0.601 0.046 272

sq2_206 60 600 1500 1 0.0081 73.8 1 150 1 0.5 0.750 0.592 0.059 266

sq2_207 60 600 1500 1 0.0081 73.8 1 150 1 2 0.706 0.517 0.029 311

sq2_208 60 600 1500 1 0.0081 73.8 1 150 1 Inf. 0.693 0.482 0.018 340
39

Table 24. Comparison sensitivity run data from circular scenarios and analysis method 2, with 

symmetric predictions, no baseline surveillance error and with ADSB_2 error (no surveillance lag).

Tdet

[s]

Tpred

[s]

Rs

[NM]

tsl

[s]

σr

[NM]

σa

[ft]

σgs

[knots]

σvs

[ft/min]

σtrk

[deg]

Same X PFA PMA PMD ∆tLOS,mean

[s]

60 60 500 0 0 0 0 0 0 1 0.052 0.061 0.062 31

60 120 500 0 0 0 0 0 0 1 0.107 0.109 0.053 76

60 300 500 0 0 0 0 0 0 1 0.260 0.229 0.044 157

60 600 500 0 0 0 0 0 0 1 0.444 0.373 0.039 218

60 900 500 0 0 0 0 0 0 1 0.560 0.469 0.037 245

60 1200 500 0 0 0 0 0 0 1 0.638 0.534 0.037 258

60 600 500 0 0 0 0 0 0 1 0.425 0.361 0.037 227

60 600 500 1 0 0 0 0 0 1 0.430 0.365 0.041 228

60 600 500 2 0 0 0 0 0 1 0.436 0.371 0.046 228

60 600 500 5 0 0 0 0 0 1 0.450 0.385 0.056 228

60 600 500 10 0 0 0 0 0 1 0.475 0.411 0.073 227

60 600 500 15 0 0 0 0 0 1 0.499 0.437 0.089 226

60 60 500 0 0.0081 73.8 1 150 1 1 0.148 0.074 0.077 31

60 120 500 0 0.0081 73.8 1 150 1 1 0.226 0.117 0.045 75

60 300 500 0 0.0081 73.8 1 150 1 1 0.376 0.258 0.025 159

60 600 500 0 0.0081 73.8 1 150 1 1 0.528 0.440 0.019 226

60 900 500 0 0.0081 73.8 1 150 1 1 0.608 0.545 0.019 256

60 1200 500 0 0.0081 73.8 1 150 1 1 0.656 0.610 0.018 269
40

Table 25. Surveillance quality run data with 10 different traffic scenarios, no surveillance error or 

ADSB_2 error, and symmetric and asymmetric predictions.

Run Scenario Number SQmodel Symmetric/ 

Asymmetric

PFA PMA PMD ∆tLOS,mean

[s]

sq3_001 1 none symmetric 0.644 0.624 0.069 217

sq3_002 2 none symmetric 0.645 0.620 0.068 217

sq3_003 3 none symmetric 0.648 0.624 0.062 214

sq3_004 4 none symmetric 0.641 0.623 0.068 216

sq3_005 5 none symmetric 0.647 0.624 0.064 214

sq3_006 6 none symmetric 0.645 0.625 0.066 214

sq3_007 7 none symmetric 0.651 0.624 0.063 213

sq3_008 8 none symmetric 0.644 0.619 0.069 218

sq3_009 9 none symmetric 0.650 0.618 0.059 218

sq3_010 10 none symmetric 0.648 0.625 0.066 213

sq3_011 1 none asymmetric 0.450 0.457 0.043 302

sq3_012 2 none asymmetric 0.448 0.454 0.041 299

sq3_013 3 none asymmetric 0.450 0.456 0.039 298

sq3_014 4 none asymmetric 0.446 0.457 0.039 299

sq3_015 5 none asymmetric 0.446 0.456 0.040 297

sq3_016 6 none asymmetric 0.448 0.456 0.041 299

sq3_017 7 none asymmetric 0.453 0.454 0.038 299

sq3_018 8 none asymmetric 0.449 0.454 0.039 299

sq3_019 9 none asymmetric 0.453 0.451 0.035 300

sq3_020 10 none asymmetric 0.451 0.458 0.039 297

sq3_021 1 ADSB_2 symmetric 0.813 0.688 0.066 212

sq3_022 2 ADSB_2 symmetric 0.814 0.685 0.070 215

sq3_023 3 ADSB_2 symmetric 0.814 0.687 0.060 211

sq3_024 4 ADSB_2 symmetric 0.810 0.686 0.070 214

sq3_025 5 ADSB_2 symmetric 0.816 0.688 0.064 210

sq3_026 6 ADSB_2 symmetric 0.814 0.687 0.067 211

sq3_027 7 ADSB_2 symmetric 0.815 0.684 0.062 214

sq3_028 8 ADSB_2 symmetric 0.813 0.683 0.072 215

sq3_029 9 ADSB_2 symmetric 0.814 0.678 0.060 216

sq3_030 10 ADSB_2 symmetric 0.815 0.687 0.061 210

sq3_031 1 ADSB_2 asymmetric 0.718 0.544 0.042 300

sq3_032 2 ADSB_2 asymmetric 0.719 0.542 0.040 297

sq3_033 3 ADSB_2 asymmetric 0.720 0.544 0.040 295

sq3_034 4 ADSB_2 asymmetric 0.717 0.544 0.038 296

sq3_035 5 ADSB_2 asymmetric 0.721 0.545 0.039 294

sq3_036 6 ADSB_2 asymmetric 0.719 0.544 0.041 295

sq3_037 7 ADSB_2 asymmetric 0.723 0.544 0.038 296

sq3_038 8 ADSB_2 asymmetric 0.719 0.542 0.039 295

sq3_039 9 ADSB_2 asymmetric 0.721 0.539 0.035 298

sq3_040 10 ADSB_2 asymmetric 0.723 0.546 0.041 295
41

Table 26. Surveillance quality run data with the same scenario and 10 different random seeds, the 

ADSB_2 error model, and symmetric and asymmetric predictions.

Run SQmodel Symmetric/ 

Asymmetric

PFA PMA PMD ∆tLOS,mean

[s]

sq3_061 ADSB_2 symmetric 0.816 0.688 0.066 209

sq3_062 ADSB_2 symmetric 0.816 0.688 0.066 209

sq3_063 ADSB_2 symmetric 0.816 0.688 0.066 209

sq3_064 ADSB_2 symmetric 0.816 0.687 0.063 211

sq3_065 ADSB_2 symmetric 0.815 0.686 0.068 212

sq3_066 ADSB_2 symmetric 0.815 0.686 0.068 212

sq3_067 ADSB_2 symmetric 0.816 0.688 0.064 210

sq3_068 ADSB_2 symmetric 0.816 0.687 0.062 209

sq3_069 ADSB_2 symmetric 0.816 0.688 0.062 210

sq3_070 ADSB_2 symmetric 0.816 0.687 0.062 211

sq3_071 ADSB_2 asymmetric 0.721 0.544 0.039 293

sq3_072 ADSB_2 asymmetric 0.721 0.544 0.039 293

sq3_073 ADSB_2 asymmetric 0.721 0.544 0.039 293

sq3_074 ADSB_2 asymmetric 0.720 0.543 0.042 294

sq3_075 ADSB_2 asymmetric 0.722 0.544 0.041 295

sq3_076 ADSB_2 asymmetric 0.721 0.543 0.039 295

sq3_077 ADSB_2 asymmetric 0.721 0.545 0.040 293

sq3_078 ADSB_2 asymmetric 0.721 0.545 0.043 293

sq3_079 ADSB_2 asymmetric 0.721 0.544 0.038 295

sq3_080 ADSB_2 asymmetric 0.722 0.544 0.042 295

Table 27. CD performance metric variability to different scenario randomizations with no 

surveillance error and with the ADSB_2 error model, symmetric and asymmetric predictions.

Case

PFA PMA PMD ∆tLOS,mean [s]

Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean

Std. 

Dev.

None - Symmetric 0.646 0.00303 0.623 0.00262 0.065 0.00328 215 2.02

None - Symmetric -

Circular 0.434 0.00342 0.368 0.00232 0.039 0.00200 223 1.39

None - Asymmetric 0.449 0.00269 0.455 0.00207 0.039 0.00209 299 1.45

ADSB_2 - Symmetric 0.814 0.00154 0.685 0.00299 0.065 0.00459 213 2.15

ADSB_2 - Asymmetric 0.720 0.00192 0.543 0.00190 0.039 0.00202 296 1.71

Table 28. CD performance metric variability to surveillance error model Monte Carlo sampling with 

the ADSB_2 error model, symmetric and asymmetric predictions.

Case

PFA PMA PMD ∆tLOS,mean [s]

Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean

Std. 

Dev.

ADSB_2 - Symmetric 0.816 0.00055 0.687 0.00090 0.065 0.00243 210 1.27

ADSB_2 - Asymmetric 0.721 0.00052 0.544 0.00057 0.040 0.00140 294 0.83
42

Surveillance Quality Results Figures

Figure 24. CD performance metrics as a function of the prediction and detection horizon for the 4 

surveillance models, using symmetric predictions and short surveillance range (50 NM).

Figure 25. CD performance metrics as a function of the prediction and detection horizon for the 4 

surveillance models, using symmetric predictions and infinite surveillance range (1500 NM).
43

Figure 26. CD performance metrics as a function of the surveillance range parameter for the 4 

surveillance models, using symmetric predictions and short CD horizon (300 seconds).

Figure 27. CD performance metrics as a function of the surveillance range parameter for the 4 

surveillance models, using symmetric predictions and long CD horizon (1200 seconds).
44

Figure 28. Difference in CD performance metrics between a CD cycle period of 15 seconds and a CD 

cycle period of 60 seconds over all SQ1 surveillance quality analysis runs.

Figure 29. Difference in CD performance metrics between symmetric and asymmetric trajectory 

predictions over all SQ1 surveillance quality analysis runs with CD cycle period of 60 seconds.
45

Figure 30. CD performance metric sensitivities to CD cycle period with no surveillance error (None) 

and with the ADSB_2 surveillance error model.

Figure 31. CD performance metric sensitivities to conflict detection and trajectory prediction time 

horizon with no surveillance error (None), with the ADSB_2 surveillance error model, from a circular 

scenario with no surveillance error (None – Circular), and from a circular scenario with ABDS_2 

surveillance error (ADSB_2 – Circular).
46

Figure 32. CD performance metric sensitivities to the surveillance range parameters with no 

surveillance error (None) and with the ADSB_2 surveillance error model.

Figure 33. CD performance metric sensitivities to the surveillance lag time parameter with no 

surveillance error (None), with the ADSB_2 surveillance error model, and from a circular scenario with 

no surveillance error (None – Circular).
47

Figure 34. CD performance metric sensitivities to surveillance horizontal position error standard 

deviation with no other surveillance error (None) and with the ADSB_2 surveillance error model.

Figure 35. CD performance metric sensitivities to surveillance vertical position error standard 

deviation with no other surveillance error (None) and with the ADSB_2 surveillance error model.
48

Figure 36. CD performance metric sensitivities to surveillance groundspeed error standard deviation 

with no other surveillance error (None) and with the ADSB_2 surveillance error model.

Figure 37. CD performance metric sensitivities to surveillance vertical speed error standard deviation 

with no other surveillance error (None) and with the ADSB_2 surveillance error model.
49

Figure 38. CD performance metric sensitivities to surveillance track angle error standard deviation 

with no other surveillance error (None) and with the ADSB_2 surveillance error model.

Figure 39. CD performance metric sensitivities to the conflict comparison criterion with no other 

surveillance error (None) and with the ADSB_2 surveillance error model.
50

B. Level and Quality of Intent Information Results and Discussion

The results of the analysis runs for the level and quality of intent information test matrix 1 (IQ1) are 

listed in Table 29-Table 32. Table 33 shows the results of some comparison runs performed using the 

circular scenarios and analysis method 2. The figures associated with test matrix IQ1 are Figure 41-Figure 

45. The intent information quality trends indicate that conflict detection performance is improved as the 

intent horizon approaches the trajectory prediction and conflict detection horizon. This is expected because, 

for intent horizons shorter than the CD horizon, the trajectory predictions are less accurate due to the lack 

of complete intent information. Comparing the results where the CD horizon is equal to the intent horizon 

(e.g., Tpred = Tint), CD performance is improved by a shorter detection horizon (e.g., 

Tpred = 300 s and Tint = 300s) for the metrics of false alerts, missed alerts, and missed detections but there 

is a negative impact to the mean time-to-LOS metric.

In general, conflict detection performance is improved as the quality of intent information is increased. 

Reduced false alerts, missed alerts, and missed detections and increased mean time-to-LOS are the trends 

observed as the intent quality increases from none (IQV = 0, IQH = 0) to maximum information (IQV = 4, 

IQH = 4). One exception to these trends are the cases where the vertical and horizontal intent information

quality is of quality 1. These cases show better CD performance compared to the cases where horizontal 

intent information quality is of quality 2. This exception is likely due to the assumptions made in trajectory 

prediction for this analysis where, in absence of a turn radius and turn rate, the turn rate is assumed to be 

0.2 deg/s, based on observations of the recorded data, and the turn radius is computed using the average 

speed of the previous leg and this assumed turn rate. CD performance is shown to be better when only the 

turn reference point is provided (IQH = 1) and reasonable values are assumed for the turn rate and turn 

radius as compared to the cases were a partially specified turn is provided (IQH = 2) and other turn 

parameters have to be assumed. However, a fully specified turn geometry (IQH = 3) still produces better 

Figure 40. Difference in CD performance metrics between symmetric and asymmetric trajectory 

prediction over all SQ2 sensitivity runs (with no other surveillance error (None) and with the ADSB_2 

surveillance error model).
51

CD performance. Notably, one example of a partially specified turn geometry is the definition proposed 

by the ADS-B DO-242 MASPS.

The analysis runs conducted using analysis method 2 and shown in Figure 41-Figure 43 provide an 

upper bound on CD performance for false alerts, missed alerts and missed detections. Because method 2 

uses perfect trajectory prediction, as the intent horizon approaches the trajectory prediction and conflict 

detection horizon, the false alert and missed alert probabilities approach zero. The missed detection 

probabilities are always zero because, as long as there is some intent horizon larger than the conflict 

detection cycle period, there is always at least one CD cycle in which to correctly detect a LOS. The mean 

time-to-LOS parameter, however, does not represent an upper limit on performance because of the random 

nature of the circular scenarios and the fact that some LOS were introduced into the scenarios with a timeto-LOS that was less than the trajectory prediction and conflict detection horizon; the conflict detector never 

had the opportunity to detect those losses at the full detection horizon.

The use of shorter conflict detection cycle period has a positive impact on CD performance, primarily 

by reducing the missed detection rate. For all intent information qualities of 1-4, there was approximately 

a 40 percent reduction of the missed detection probabilities when using CD cycle period of 15 seconds 

versus 60 seconds and little or no change to the other metrics (Figure 44). The shorter cycle period provides 

four times as many detection cycles over which to successfully detect a LOS.

Similar to the surveillance analysis runs, conflict detection performance using asymmetric predictions 

(perfect prediction for aircraft 1 in a pair) versus symmetric prediction had a positive impact on CD 

performance for all metrics (Figure 45). 

The results of the analysis runs for the level and quality of intent information test matrix 2 (IQ2) are 

listed in Table 34 and Table 35. Table 34 presents the results of the analysis runs for the variability due to 

different traffic scenarios; those results are summarized in Table 36. The variability of the CD performance 

metrics due to different traffic scenarios is small with 0.33% or less of standard deviation in the false alert 

probability, 0.38% or less of standard deviation in the missed alert probability, 0.27% or less of standard 

deviation in the missed detection probability, and less than 7 seconds of standard deviation in the mean 

time-to-loss.

Table 35 shows the results of the analysis runs for the sensitivity of the time and altitude interval for 

the intent information quality 4 horizontal and vertical points. The associated figures are shown in Figure 

46 and Figure 47. The sensitivities indicate that the choice of the time and altitude interval for the quality 

4 intent points can be chosen within a fairly wide range with only a small impact to the CD performance. 

The choice of time interval for the additional horizontal points is less critical than the choice of the altitude 

interval for the additional vertical points due to the flatter sensitivity profiles. As expected, all of the intent 

information quality 4 analysis runs showed better CD performance when compared to the equivalent intent 

quality 3 runs. 

It can be noted that one problem with intent information in the form used in this study arises from 

considering the horizontal and vertical trajectory profiles to be, aside from the initial and end points, 

completely independent. Because both horizontal and vertical intent points are indexed only by time, 

variations in groundspeed, especially in long segments between turns, can result in a vertical profile that is 

insufficiently synchronized with the horizontal profile to allow for accurate four-dimensional conflict 

detection. This can have the apparent effect of introducing along-track errors, which can be emphasized

from a detection standpoint for predicted climb and descent segments. Intent information quality 4 tends 

to mitigate this by introducing additional points that allow for localized speed corrections in the absence of 

speed intent information.
52

Level and Quality of Intent Results Tables

Table 29. Intent quality run data with symmetric predictions and 60 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

Tint

[s]

IQV IQH PFA PMA PMD ∆tLOS,mean

[s]

iq1_001 60 1200 1200 4 4 0.037 0.053 0.029 882

iq1_002 60 1200 1200 3 3 0.086 0.109 0.037 871

iq1_003 60 1200 1200 2 2 0.280 0.220 0.067 854

iq1_004 60 1200 1200 1 2 0.308 0.253 0.070 849

iq1_005 60 1200 1200 1 1 0.172 0.195 0.056 851

iq1_006 60 1200 600 4 4 0.191 0.282 0.020 672

iq1_007 60 1200 600 3 3 0.214 0.309 0.027 665

iq1_008 60 1200 600 2 2 0.395 0.395 0.046 635

iq1_009 60 1200 600 1 2 0.405 0.406 0.046 633

iq1_010 60 1200 600 1 1 0.262 0.353 0.040 655

iq1_011 60 1200 300 4 4 0.386 0.486 0.018 484

iq1_012 60 1200 300 3 3 0.393 0.499 0.023 476

iq1_013 60 1200 300 2 2 0.529 0.558 0.034 440

iq1_014 60 1200 300 1 2 0.531 0.559 0.030 438

iq1_015 60 1200 300 1 1 0.414 0.517 0.027 467

iq1_016 60 600 600 4 4 0.038 0.053 0.026 500

iq1_017 60 600 600 3 3 0.072 0.095 0.033 496

iq1_018 60 600 600 2 2 0.251 0.184 0.057 489

iq1_019 60 600 600 1 2 0.264 0.202 0.058 489

iq1_020 60 600 600 1 1 0.133 0.156 0.047 490

iq1_021 60 600 300 4 4 0.206 0.241 0.018 399

iq1_022 60 600 300 3 3 0.220 0.261 0.023 395

iq1_023 60 600 300 2 2 0.393 0.335 0.034 374

iq1_024 60 600 300 1 2 0.388 0.338 0.031 371

iq1_025 60 600 300 1 1 0.250 0.290 0.028 387

iq1_026 60 300 300 4 4 0.036 0.051 0.025 253

iq1_027 60 300 300 3 3 0.054 0.080 0.031 252

iq1_028 60 300 300 2 2 0.218 0.136 0.050 250

iq1_029 60 300 300 1 2 0.213 0.136 0.044 250

iq1_030 60 300 300 1 1 0.083 0.107 0.038 251

iq1_031 60 1200 1200 0 0 0.559 0.710 0.122 301

iq1_032 60 600 600 0 0 0.410 0.590 0.122 232

iq1_033 60 300 300 0 0 0.269 0.440 0.123 157
53

Table 30. Intent quality run data with symmetric predictions and 15 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

Tint

[s]

IQV IQH PFA PMA PMD ∆tLOS,mean

[s]

iq1_034 15 1200 1200 4 4 0.037 0.053 0.016 889

iq1_035 15 1200 1200 3 3 0.086 0.110 0.021 880

iq1_036 15 1200 1200 2 2 0.280 0.220 0.040 859

iq1_037 15 1200 1200 1 2 0.308 0.253 0.039 851

iq1_038 15 1200 1200 1 1 0.173 0.195 0.030 855

iq1_039 15 1200 600 4 4 0.191 0.282 0.011 691

iq1_040 15 1200 600 3 3 0.215 0.310 0.016 681

iq1_041 15 1200 600 2 2 0.395 0.395 0.027 651

iq1_042 15 1200 600 1 2 0.405 0.406 0.026 648

iq1_043 15 1200 600 1 1 0.261 0.353 0.023 670

iq1_044 15 1200 300 4 4 0.386 0.486 0.010 511

iq1_045 15 1200 300 3 3 0.393 0.499 0.012 499

iq1_046 15 1200 300 2 2 0.530 0.558 0.018 462

iq1_047 15 1200 300 1 2 0.531 0.559 0.019 463

iq1_048 15 1200 300 1 1 0.414 0.517 0.017 492

iq1_049 15 600 600 4 4 0.038 0.052 0.014 514

iq1_050 15 600 600 3 3 0.072 0.095 0.018 512

iq1_051 15 600 600 2 2 0.251 0.184 0.033 505

iq1_052 15 600 600 1 2 0.263 0.201 0.030 505

iq1_053 15 600 600 1 1 0.133 0.155 0.025 506

iq1_054 15 600 300 4 4 0.207 0.241 0.010 421

iq1_055 15 600 300 3 3 0.221 0.261 0.012 415

iq1_056 15 600 300 2 2 0.395 0.335 0.018 395

iq1_057 15 600 300 1 2 0.389 0.337 0.019 394

iq1_058 15 600 300 1 1 0.250 0.290 0.017 408

iq1_059 15 300 300 4 4 0.036 0.051 0.013 272

iq1_060 15 300 300 3 3 0.055 0.080 0.015 271

iq1_061 15 300 300 2 2 0.219 0.137 0.026 269

iq1_062 15 300 300 1 2 0.213 0.136 0.023 271

iq1_063 15 300 300 1 1 0.083 0.106 0.019 271

iq1_064 15 1200 1200 0 0 0.553 0.709 0.020 300

iq1_065 15 600 600 0 0 0.402 0.588 0.020 232

iq1_066 15 300 300 0 0 0.261 0.437 0.020 163
54

Table 31. Intent quality run data with asymmetric predictions and 60 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

Tint

[s]

IQV IQH PFA PMA PMD ∆tLOS,mean

[s]

iq1_067 60 1200 1200 4 4 0.024 0.031 0.017 886

iq1_068 60 1200 1200 3 3 0.065 0.075 0.022 878

iq1_069 60 1200 1200 2 2 0.219 0.169 0.050 864

iq1_070 60 1200 1200 1 2 0.243 0.192 0.051 859

iq1_071 60 1200 1200 1 1 0.130 0.137 0.038 865

iq1_072 60 1200 600 4 4 0.133 0.199 0.011 737

iq1_073 60 1200 600 3 3 0.154 0.221 0.016 730

iq1_074 60 1200 600 2 2 0.295 0.295 0.033 703

iq1_075 60 1200 600 1 2 0.307 0.304 0.032 701

iq1_076 60 1200 600 1 1 0.192 0.255 0.027 723

iq1_077 60 1200 300 4 4 0.269 0.364 0.010 589

iq1_078 60 1200 300 3 3 0.278 0.374 0.013 581

iq1_079 60 1200 300 2 2 0.393 0.430 0.022 548

iq1_080 60 1200 300 1 2 0.397 0.432 0.018 546

iq1_081 60 1200 300 1 1 0.296 0.391 0.015 574

iq1_082 60 600 600 4 4 0.024 0.031 0.015 502

iq1_083 60 600 600 3 3 0.051 0.060 0.020 500

iq1_084 60 600 600 2 2 0.186 0.131 0.041 494

iq1_085 60 600 600 1 2 0.201 0.144 0.040 494

iq1_086 60 600 600 1 1 0.101 0.106 0.032 495

iq1_087 60 600 300 4 4 0.133 0.157 0.010 436

iq1_088 60 600 300 3 3 0.146 0.171 0.013 433

iq1_089 60 600 300 2 2 0.276 0.232 0.022 416

iq1_090 60 600 300 1 2 0.277 0.234 0.018 414

iq1_091 60 600 300 1 1 0.172 0.195 0.015 426

iq1_092 60 300 300 4 4 0.023 0.030 0.014 255

iq1_093 60 300 300 3 3 0.038 0.050 0.018 254

iq1_094 60 300 300 2 2 0.154 0.092 0.032 252

iq1_095 60 300 300 1 2 0.154 0.091 0.028 252

iq1_096 60 300 300 1 1 0.062 0.069 0.022 253

iq1_097 60 1200 1200 0 0 0.405 0.540 0.062 453

iq1_098 60 600 600 0 0 0.294 0.410 0.062 318

iq1_099 60 300 300 0 0 0.204 0.282 0.064 193
55

Table 32. Intent quality run data with asymmetric predictions and 15 second CD cycle period.

Run Tdet

[s]

Tpred

[s]

Tint

[s]

IQV IQH PFA PMA PMD ∆tLOS,mean

[s]

iq1_100 15 1200 1200 4 4 0.024 0.031 0.008 894

iq1_101 15 1200 1200 3 3 0.065 0.075 0.011 887

iq1_102 15 1200 1200 2 2 0.219 0.168 0.029 871

iq1_103 15 1200 1200 1 2 0.243 0.192 0.029 867

iq1_104 15 1200 1200 1 1 0.130 0.137 0.021 873

iq1_105 15 1200 600 4 4 0.133 0.200 0.006 753

iq1_106 15 1200 600 3 3 0.154 0.221 0.008 744

iq1_107 15 1200 600 2 2 0.295 0.295 0.019 716

iq1_108 15 1200 600 1 2 0.307 0.304 0.020 717

iq1_109 15 1200 600 1 1 0.193 0.256 0.016 740

iq1_110 15 1200 300 4 4 0.270 0.365 0.005 613

iq1_111 15 1200 300 3 3 0.278 0.375 0.006 602

iq1_112 15 1200 300 2 2 0.394 0.430 0.012 567

iq1_113 15 1200 300 1 2 0.397 0.432 0.011 569

iq1_114 15 1200 300 1 1 0.296 0.391 0.009 599

iq1_115 15 600 600 4 4 0.024 0.031 0.007 517

iq1_116 15 600 600 3 3 0.051 0.060 0.009 515

iq1_117 15 600 600 2 2 0.187 0.130 0.023 510

iq1_118 15 600 600 1 2 0.201 0.144 0.022 510

iq1_119 15 600 600 1 1 0.101 0.106 0.018 511

iq1_120 15 600 300 4 4 0.134 0.157 0.005 456

iq1_121 15 600 300 3 3 0.147 0.171 0.007 452

iq1_122 15 600 300 2 2 0.278 0.232 0.012 435

iq1_123 15 600 300 1 2 0.277 0.234 0.011 435

iq1_124 15 600 300 1 1 0.172 0.195 0.009 447

iq1_125 15 300 300 4 4 0.023 0.029 0.007 274

iq1_126 15 300 300 3 3 0.039 0.049 0.008 273

iq1_127 15 300 300 2 2 0.154 0.091 0.017 271

iq1_128 15 300 300 1 2 0.154 0.091 0.014 272

iq1_129 15 300 300 1 1 0.062 0.068 0.010 273

iq1_130 15 1200 1200 0 0 0.402 0.539 0.012 457

iq1_131 15 600 600 0 0 0.291 0.408 0.012 325

iq1_132 15 300 300 0 0 0.200 0.280 0.012 205
56

Table 33. Comparison level of intent run data using circular method 2 with symmetric predictions, 60 

second CD cycle period, and 3 CD horizons.

Tdet

[s]

Tpred

[s]

Tint

[s]

IQV IQH PFA PMA PMD ∆tLOS,mean

[s]

60 300 60 3 3 0.160 0.158 0.000 167

60 300 120 3 3 0.086 0.098 0.000 180

60 300 300 3 3 0.002 0.000 0.000 202

60 300 600 3 3 0.003 0.000 0.000 202

60 300 900 3 3 0.003 0.000 0.000 202

60 300 1200 3 3 0.003 0.000 0.000 202

60 600 60 3 3 0.354 0.323 0.000 228

60 600 120 3 3 0.270 0.269 0.000 244

60 600 300 3 3 0.097 0.121 0.000 293

60 600 600 3 3 0.003 0.000 0.000 335

60 600 900 3 3 0.003 0.000 0.000 335

60 600 1200 3 3 0.004 0.000 0.000 335

60 1200 60 3 3 0.569 0.493 0.000 266

60 1200 120 3 3 0.500 0.451 0.000 284

60 1200 300 3 3 0.318 0.328 0.000 341

60 1200 600 3 3 0.135 0.163 0.000 422

60 1200 900 3 3 0.038 0.050 0.000 479

60 1200 1200 3 3 0.002 0.000 0.000 505
57

Table 34. Intent quality run data with 10 different traffic scenarios, 3 intent horizons, and symmetric and 

asymmetric predictions.

Run Scenario Number Tpred

[s]

Tint

[s]

Symmetric/

Asymmetric

PFA PMA PMD ∆tLOS,mean

[s]

iq2_001 1 1200 1200 symmetric 0.087 0.111 0.034 894

iq2_002 2 1200 1200 symmetric 0.081 0.111 0.038 879

iq2_003 3 1200 1200 symmetric 0.085 0.102 0.031 876

iq2_004 4 1200 1200 symmetric 0.087 0.115 0.033 881

iq2_005 5 1200 1200 symmetric 0.086 0.109 0.037 871

iq2_006 6 1200 1200 symmetric 0.089 0.106 0.034 879

iq2_007 7 1200 1200 symmetric 0.088 0.104 0.031 879

iq2_008 8 1200 1200 symmetric 0.084 0.110 0.034 880

iq2_009 9 1200 1200 symmetric 0.086 0.108 0.032 874

iq2_010 10 1200 1200 symmetric 0.092 0.107 0.030 881

iq2_011 1 1200 600 symmetric 0.216 0.314 0.026 675

iq2_012 2 1200 600 symmetric 0.217 0.312 0.028 666

iq2_013 3 1200 600 symmetric 0.217 0.309 0.024 664

iq2_014 4 1200 600 symmetric 0.214 0.312 0.023 669

iq2_015 5 1200 600 symmetric 0.214 0.309 0.027 665

iq2_016 6 1200 600 symmetric 0.218 0.310 0.023 666

iq2_017 7 1200 600 symmetric 0.218 0.307 0.023 666

iq2_018 8 1200 600 symmetric 0.214 0.309 0.022 669

iq2_019 9 1200 600 symmetric 0.219 0.306 0.021 669

iq2_020 10 1200 600 symmetric 0.219 0.310 0.021 668

iq2_021 1 1200 300 symmetric 0.394 0.503 0.022 481

iq2_022 2 1200 300 symmetric 0.397 0.500 0.021 476

iq2_023 3 1200 300 symmetric 0.400 0.500 0.019 474

iq2_024 4 1200 300 symmetric 0.392 0.499 0.019 479

iq2_025 5 1200 300 symmetric 0.393 0.499 0.023 476

iq2_026 6 1200 300 symmetric 0.396 0.500 0.019 475

iq2_027 7 1200 300 symmetric 0.398 0.498 0.019 476

iq2_028 8 1200 300 symmetric 0.393 0.497 0.020 478

iq2_029 9 1200 300 symmetric 0.401 0.492 0.019 482

iq2_030 10 1200 300 symmetric 0.397 0.499 0.017 476

iq2_031 1 1200 1200 asymmetric 0.068 0.076 0.021 900

iq2_032 2 1200 1200 asymmetric 0.061 0.074 0.022 883

iq2_033 3 1200 1200 asymmetric 0.064 0.069 0.020 884

iq2_034 4 1200 1200 asymmetric 0.067 0.075 0.020 888

iq2_035 5 1200 1200 asymmetric 0.065 0.075 0.022 878

iq2_036 6 1200 1200 asymmetric 0.066 0.068 0.020 887

iq2_037 7 1200 1200 asymmetric 0.065 0.070 0.020 885

iq2_038 8 1200 1200 asymmetric 0.062 0.075 0.020 883

iq2_039 9 1200 1200 asymmetric 0.063 0.071 0.020 881

iq2_040 10 1200 1200 asymmetric 0.068 0.071 0.018 889

iq2_041 1 1200 600 asymmetric 0.157 0.224 0.016 743

iq2_042 2 1200 600 asymmetric 0.156 0.222 0.017 732

iq2_043 3 1200 600 asymmetric 0.157 0.219 0.015 731

iq2_044 4 1200 600 asymmetric 0.155 0.222 0.014 737

iq2_045 5 1200 600 asymmetric 0.154 0.221 0.016 730

iq2_046 6 1200 600 asymmetric 0.157 0.218 0.015 735

iq2_047 7 1200 600 asymmetric 0.158 0.218 0.014 734

iq2_048 8 1200 600 asymmetric 0.156 0.222 0.013 732

iq2_049 9 1200 600 asymmetric 0.158 0.217 0.013 734

iq2_050 10 1200 600 asymmetric 0.159 0.222 0.012 735

iq2_051 1 1200 300 asymmetric 0.281 0.379 0.013 591

iq2_052 2 1200 300 asymmetric 0.282 0.376 0.013 584

iq2_053 3 1200 300 asymmetric 0.284 0.375 0.012 582

iq2_054 4 1200 300 asymmetric 0.280 0.376 0.011 587

iq2_055 5 1200 300 asymmetric 0.278 0.374 0.013 581

iq2_056 6 1200 300 asymmetric 0.281 0.374 0.012 585

iq2_057 7 1200 300 asymmetric 0.285 0.372 0.012 586

iq2_058 8 1200 300 asymmetric 0.282 0.376 0.011 583

iq2_059 9 1200 300 asymmetric 0.286 0.369 0.011 588

iq2_060 10 1200 300 asymmetric 0.284 0.377 0.010 584
58

Table 35. Intent quality 4 time and altitude interval parameter sensitivity runs with symmetric and 

asymmetric trajectory predictions.

Run Tpred

[s]

Tint

[s]

Symmetric/

Asymmetric

∆tQ4 ∆altQ4 PFA PMA PMD ∆tLOS,mean

[s]

iq2_061 600 600 symmetric 15 1000 0.037 0.049 0.025 501

iq2_062 600 600 symmetric 30 1000 0.037 0.050 0.026 500

iq2_063 600 600 symmetric 45 1000 0.037 0.050 0.026 500

iq2_064 600 600 symmetric 60 1000 0.038 0.051 0.026 500

iq2_065 600 600 symmetric 75 1000 0.038 0.051 0.026 500

iq2_066 600 600 symmetric 90 1000 0.038 0.052 0.026 500

iq2_067 600 600 symmetric 120 1000 0.038 0.053 0.026 500

iq2_068 600 600 symmetric 180 1000 0.039 0.054 0.027 500

iq2_069 600 600 symmetric 300 1000 0.044 0.058 0.028 500

iq2_070 600 600 symmetric 120 100 0.037 0.047 0.022 500

iq2_071 600 600 symmetric 120 200 0.037 0.047 0.023 500

iq2_072 600 600 symmetric 120 300 0.037 0.048 0.024 500

iq2_073 600 600 symmetric 120 400 0.037 0.049 0.024 500

iq2_074 600 600 symmetric 120 500 0.037 0.050 0.025 500

iq2_075 600 600 symmetric 120 750 0.038 0.050 0.025 500

iq2_076 600 600 symmetric 120 1000 0.038 0.053 0.026 500

iq2_077 600 600 symmetric 120 2000 0.040 0.059 0.028 500

iq2_078 600 600 symmetric 120 5000 0.047 0.068 0.030 498

iq2_079 600 600 asymmetric 15 1000 0.023 0.028 0.014 503

iq2_080 600 600 asymmetric 30 1000 0.023 0.029 0.014 503

iq2_081 600 600 asymmetric 45 1000 0.023 0.029 0.015 502

iq2_082 600 600 asymmetric 60 1000 0.024 0.029 0.015 503

iq2_083 600 600 asymmetric 75 1000 0.024 0.030 0.015 502

iq2_084 600 600 asymmetric 90 1000 0.024 0.030 0.015 502

iq2_085 600 600 asymmetric 120 1000 0.024 0.031 0.015 502

iq2_086 600 600 asymmetric 180 1000 0.025 0.032 0.015 502

iq2_087 600 600 asymmetric 300 1000 0.029 0.035 0.016 502

iq2_088 600 600 asymmetric 120 100 0.023 0.027 0.013 502

iq2_089 600 600 asymmetric 120 200 0.023 0.027 0.013 502

iq2_090 600 600 asymmetric 120 300 0.023 0.028 0.013 502

iq2_091 600 600 asymmetric 120 400 0.023 0.028 0.013 502

iq2_092 600 600 asymmetric 120 500 0.023 0.029 0.014 502

iq2_093 600 600 asymmetric 120 750 0.024 0.029 0.014 502

iq2_094 600 600 asymmetric 120 1000 0.024 0.031 0.015 502

iq2_095 600 600 asymmetric 120 2000 0.026 0.035 0.016 502

iq2_096 600 600 asymmetric 120 5000 0.030 0.041 0.017 501
59

Level and Quality of Intent Information Results Figures

Table 36. CD performance metric variability to different scenario randomizations with 1200s CD 

horizon, horizontal and vertical intent quality levels of 3, and 3 intent horizons – symmetric and 

asymmetric predictions.

Case

PFA PMA PMD ∆tLOS,mean [s]

Mean Std. Dev. Mean Std. Dev. Mean Std. Dev. Mean

Std. 

Dev.

1200 s Intent Horizon -

Symmetric 0.086 0.00284 0.108 0.00376 0.033 0.00261 879 6.12

1200 s Intent Horizon -

Asymmetric 0.065 0.00245 0.072 0.00285 0.020 0.00109 886 5.89

600 s Intent Horizon -

Symmetric 0.217 0.00205 0.310 0.00245 0.024 0.00226 668 3.21

600 s Intent Horizon -

Asymmetric 0.157 0.00153 0.221 0.00234 0.014 0.00142 734 3.74

300 s Intent Horizon -

Symmetric 0.396 0.00322 0.499 0.00287 0.020 0.00163 477 2.61

300 s Intent Horizon -

Asymmetric 0.282 0.00246 0.375 0.00277 0.012 0.00106 585 2.86

Figure 41. CD performance metrics versus intent horizon for various intent quality levels, with long CD 

and trajectory prediction horizon (1200 s), symmetric predictions, and 60 second CD cycle period.
60

Figure 42. CD performance metrics versus intent horizon for various intent quality levels, with medium 

CD and trajectory prediction horizon (600 s), symmetric predictions, and 60 second CD cycle period.

Figure 43. CD performance metrics versus intent horizon for various intent quality levels, with short CD 

and trajectory prediction horizon (300 s), symmetric predictions, and 60 second CD cycle period.
61

Figure 44. Difference in CD performance metrics between a CD cycle period of 15 seconds and a CD cycle 

period of 60 seconds over all IQ1 intent quality analysis runs.

Figure 45. Difference in CD performance metrics between symmetric and asymmetric trajectory 

predictions over all IQ1 intent quality analysis runs with CD cycle period of 60 seconds.
62

Figure 46. CD performance metrics sensitivity to the time interval for intent quality 4 horizontal points

with vertical quality 4 altitude interval of 1000 ft.

Figure 47. CD performance metrics sensitivity to the altitude interval for intent quality 4 vertical points

with horizontal quality 4 time interval of 120 seconds.
63

References

1Bilimoria, K.D., “Survey of Air/Ground and Human/Automation Functional Allocation for Separation Assurance,” 12th AIAA 

Aviation Technology, Integration, and Operations (ATIO) Conference and 17th AIAA/ISSM, Indianapolis, Indiana, September 17-

19, 2012.

2Lauderdale, T.A., Cone, A.C., Bowe, A.R., “Relative Significance of Trajectory Prediction Errors on an Automated Separation 

Assurance Algorithm,” Ninth UAS/Europe Air Traffic Management Research and Development Seminar (ATM2011), Berlin, 

Germany, 14-17 June 2011.

3Yang, L.C., Kuchar, J.K., Aircraft Conflict Analysis and Real-Time Conflict Probing Using Probabilistic Trajectory Modeling,

Dept. of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA, 2000.

4Kuchar, J.K., Yang, L.C., “A Review of Conflict Detection and Resolution Modeling Methods,” IEEE Transactions on 

Intelligent Transportation Systems, Vol. 1. No. 4, December, 2000.

5Krozel, J., Peters, M.E., “Conflict Detection and Resolution for Future Air Transportation Management,” NASA Technical 

Report, NAS2-14285, April, 1997.

6Karr, D.A., Vivona, R.A., “Conflict Detection Using Variable Four-Dimensional Uncertainty Bounds to Control Missed 

Alerts,” AIAA Guidance Navigation, and Control Conference and Exhibit, Keystone, Colorado, August 21-24, 2006.

7Wickens, C.D., et.al., “False Alerts in Air Traffic Control Conflict Alerting System: Is There a ‘Cry Wolf’ Effect?” HUMAN 

FACTORS, Vol. 51, No. 4, August 2009, pp. 446-462.

8Thomas, L.C., Wickens, C.D., Rantanen, E.M., “Imperfect Automation in Air Traffic Alerts: A Review of Conflict Detection 

Algorithms and Their Implications for Human Factors Research,” Proceedings of the 47th Annual Meeting of the Human Factors 

and Ergonomics Society, Santa Monica, California, 2003.

9Paglione, M.M., Ryan, H.F., Liu, S., “Evaluation of En Route Host Computer System’s Tactical Alert Processing: Description 

and Methodology,” Technical Note, US Department of Transportation, Federal Aviation Administration, DOT/FAA/TC/TN07/13, 

February 2007.

10Bilimoria, K.D., “Methodology for the Performance Evaluation of a Conflict Probe,” Journal of Guidance, Control, and 

Dynamics, Vol. 24, No. 3, May-June 2001.

11Gaydos, T.L., Liao, B., Smith, E.C., Wang, L., “Analysis of Conflict Detection Performance for Trajectory-Based Descent 

Operations,” Tenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2013), Chicago, Illinois, June 

10-13, 2013.

12Shakarian, A., Haraldsdottir, A., “Required Total System Performance and Results of a Short Term Conflict Alert Simulation 

Study,” 4

th US/Europe Air Traffic Management R&D Seminar, Santa Fe, New Mexico, December 3-7, 2001.

13Paielli, R., Erzberger, H., “Improved Conflict Detection for Reducing Operational Errors in Air Traffic Control,” AIAA 4th

Aviation Technology, Integration and Operations (ATIO) Forum, Chicago, Illinois, September 20-22, 2004.

14Narkawicz, A., Muñoz, C. “A Formally Verified Conflict Detection Algorithm for Polynomial Trajectories” Proceedings of 

the 2015 AIAA Infotech @ Aerospace Conference, AIAA-2015-0795, January 2015.

15Chung, W.W., et. al., “Surveillance and Datalink Communication Performance Analysis for Distributed Separation Assurance 

System Architectures,” NASA Contractor Report, CR-2012-217590, July 2012.

16Cashion, P., Lozito, S., “The Effects of Different Levels of Intent Information on Pilot Self-Separation Performance,” 

Proceedings of International Symposium on Aviation Psychology, 1999.

17Lewis, T.A., Phojanamongkolkij, N., Wing, D.J., “The Effects of Limited Intent Information Availability on Self-Separation 

in Mixed Operations,” 2012 Integrated Communications Navigation and Surveillance (ICNS) Conference, Herndon, Virginia, April 

24-26, 2012.

18Finkelsztein, D.M., Sturdy, J.L., Alaverdi, O., Hochwarth, J.K., “4D Dynamic Required Navigation Performance Final

Report,” NASA Contractor Report, CR-2011-217051, February, 2011.

19Wing, D.J., Barmore, B.E., Krishnamurthy, K., “Use of Traffic Intent Information by Autonomous Aircraft in Constrained 

Operations,” AIAA Guidance Navigation and Control Conference and Exhibit, AIAA-2002-4555, Monterey, California, August 5-

8, 2002.

20Ruigrok, R.C., Clari, M.S., “The Use of Aircraft Intent Information in Airborne Separation Assurance Systems,” FAAEurocontrol Third Technical Interchange Meeting (TIM) on Airborne Separation Assurance Systems, San Francisco, California, 

October 21-23, 2002.

21Paglione, M.M., Oaks, R.D., Bilimoria, K.D., “Methodology for Generating Conflict Scenarios by Time Shifting Recorded 

Traffic Data,” AIAA’s 3rd Annual Aviation Technology, Integration, and Operations (ATIO) Forum, Denver, Colorado, November 

17-19, 2003.

22ICAO, Guidance Material on Comparison of Surveillance Technologies (GMST), Edition 1.0, September 2007.

23Coppenbarger, R.A., Kanning, G., Salcido, R., “Real-Time Data Link of Aircraft Parameters to the Center-TRACON 

Automation System (CTAS),” 4

th USA/Europe ATM R&D Seminar, Santa Fe, New Mexico, December 3-7, 2001.

24RTCA, Inc., Minimum Aviation System Performance Standards for Automatic Dependent Surveillance Broadcast (ADS-B), 

RTCA DO-242A, June, 25, 2002.

25RTCA, Inc., Minimum Aviation System Performance Standards for Aircraft Surveillance Applications (ASA), RTCA DO289, Volume I, December 9, 2003.

26Federal Aviation Administration, Aviation System Performance Metrics (ASPM), https://aspm.faa.gov/.
64

27Welch, G., Bishop, G., “An Introduction to the Kalman Filter,” University of North Carolina at Chapel Hill, UNC-Chapel 

Hill, TR 95-041, July 24, 2006.

28Federal Aviation Administration, Air Traffic Organizational Policy, JO 7110.65V, CHG 2, February 19, 2014.
65

Appendix

A.1. Surveillance Error Modeling

The surveillance error model consisted of standard deviation parameters for a set of Gaussian 

distributions relating to the level of error on the position and velocity states of an aircraft. Each surveillance 

technology has its own unique error signature and the choice of Gaussian distributions was chosen for 

convenience as the simple compromise amongst the many possible error distributions. 

A fixed surveillance lag parameter, tsl, was applied equally over all aircraft tracks to simulate the uncompensated time delay that may exist between surveillance sampling time and the time of trajectory 

prediction. At any conflict detection cycle time, tcd, the true track position states of an aircraft were given 

by s(tcd), where,

and the velocity states were given by v(tcd), where,

Linear interpolation was used to compute a true track data point at the surveillance time when the 

surveillance lag time was not an integer increment of the recorded track data. During the same conflict 

detection cycle, the surveillance position estimate used for trajectory prediction, ŝ(tcd), was given by the 

true position at surveillance time, s(ts

) = s(tcd − tsl), plus the surveillance error components in position 

and velocity, εs and εv, respectively. The surveillance error components were computed using equations 

(15)-(19) as,

[εlat(ts), εlon(ts)] = fGCD(lat(ts

), lon(ts), N(0, σr),U(0,360)) − [lat(ts), lon(ts)] (15)

εalt(ts

) = N(0, σa) (16)

εgs(ts

) = N(0, σgs) (17)

εvs(ts

) = N(0, σvs) (18)

εtrk(ts

) = N(0, σtrk) (19)

where the horizontal position error components, εlat(ts) and εlon(ts), were computed using a great-circle 

projection from the true position at surveillance time, lat(ts

) and lon(ts), with a radial distance error 

sampled from a Gaussian distribution, N(0, σr

), and an azimuth sampled from a Uniform distribution, 

U(0,360). The great-circle projection function, fGCD, computes a latitude and longitude position projected 

some arc-distance along an initial track on the surface of a sphere from a starting latitude and longitude 

position. Finally, the surveillance position and velocity states used at conflict detection time were given 

by:

ŝ(tcd) = [lat(ts

) +εlat (ts), lon(ts)+ εlon(ts), alt(ts)+ εalt(ts)] (20)

v̂(tcd) = [gs(ts

)+ εgs(ts), vs(ts) +εvs(ts),trk(ts) +εtrk(ts)] (21)

Figure 2 shows a diagram of the surveillance error modeling described above. 

s(t) = [lat, lon, alt] (13)

v(t) = [gs, vs,trk] (14)
66

A.2. Some Metrics Issues Associated with Intent

In measuring the performance of an intent-based conflict detector, the problem of deciding whether a 

specific detected conflict matches a true LOS is more complicated than with a state-based conflict detector. 

There are several reasons for this complication. The first reason is that, an intent-based conflict detector

can return a prediction of more than one future losses-of-separation (multiple detected conflicts with the 

same aircraft pair within the conflict detection horizon). The second reason is that, there may be several 

true LOS between a pair of aircraft and a smaller number of predicted LOS. The question arises: which 

predicted LOS correspond to which true LOS, if any?

Figure 48 illustrates the case where a single loss-of-separation exists but where the intent-based conflict 

detector predicts multiple conflicts. In this figure, the x-axis represents time and the transition up represents 

an entry into LOS while a transition down is an exit from LOS, with both the true and predicted cases shown 

for the same time horizon. It is clear that, at least in some sense, the prediction is incorrect. The times do 

not match exactly and the prediction goes in and out of loss several times. But in an abstract sense it is 

really pretty close. An assumption in our metrics is to analyze each predicted loss separately since each 

LOS is characterized by the entry time into that LOS. In the example below there would be 3 detections 

and, after applying the same conflict comparison criterion, any number of those could be could be identified 

as correct alerts. Consequently, the remaining conflicts in a group like this would be labeled as false alerts 

because they would fail to meet the same conflict comparison criterion. For example, the first two predicted 

losses could be considered correct alerts to the true LOS if they both satisfy the same conflict comparison 

criterion whereas, the last predicted conflict does not satisfy the same conflict comparison criterion and is 

labeled as a false alert.

Figure 49 illustrates the case where multiple losses of separation exist but the intent-based conflict 

detector predicts only a single conflict. A similar application of the same conflict comparison criteria is 

used to categorize the predicted and true LOS here. Each predicted LOS is compared sequentially against 

each true LOS and multiple true LOS could be considered “the same” as a single predicted LOS. In this 

example, one possibility is that the first true LOS meets the same conflict comparison criterion with respect 

to the predicted LOS whereas, the second true LOS does not. In this case, the second true LOS would be 

label as a missed alert and the predicted LOS would be labeled as a correct alert. Another possibility is that 

neither true LOS meets the same conflict comparison criterion with respect to the predicted LOS. In that 

case, both true LOS would be labeled as missed alerts and the predicted LOS would be labeled a false alert.

There are many possible and complicated approaches that one could use to try and group the multiple 

predicted or multiple true LOS together but those were not pursued in this study.

Figure 48. Example single true LOS and multiple predicted LOS.
67

Figure 49. Example multiple true LOS and single predicted LOS.
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Memorandum

 4. TITLE AND SUBTITLE

Parametric Analysis of Surveillance Quality and Level and Quality of 

Intent Information and their Impact on Conflict Detection Performance

5a. CONTRACT NUMBER

 6. AUTHOR(S)

Guerreiro, Nelson M.; Butler, Ricky W.; Hagen, George E.; Maddalon, 

Jeffrey M.; Lewis, Timothy A.

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20669

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: NASA STI Program (757) 864-9658

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

A loss-of-separation (LOS) is said to occur when two aircraft are spatially too close to one another. A LOS is the fundamental 

unsafe event to be avoided in air traffic management and conflict detection (CD) is the function that attempts to predict these

LOS events. In general, the effectiveness of conflict detection relates to the overall safety and performance of an air traffic

management concept. An abstract, parametric analysis was conducted to investigate the impact of surveillance quality, level of 

intent information, and quality of intent information on conflict detection performance. The data collected in this analysis can 

be used to estimate the conflict detection performance under alternative future scenarios or alternative allocations of the 

conflict detection function, based on the quality of the surveillance and intent information under those conditions. 

Alternatively, this data could also be used to estimate the surveillance and intent information quality required to achieve some 

desired CD performance as part of the design of a new separation assurance system. 

15. SUBJECT TERMS

Conflict detection; Function allocation; Separation assurance; Surveillance intent

18. NUMBER

 OF 

 PAGES

76

19b. TELEPHONE NUMBER (Include area code)

(757) 864-9658

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

 999182.02.20.07.01

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA-TM-2016-219177

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 03 - 2016