DOT/FAA/AM-02/1

Scott D. Gronlund

John M. Canning

Peter M. Moertl

Joakim Johansson

University of Oklahoma

Department of Psychology

Norman, OK 73019

Michael R. P. Dougherty

Department of Psychology

University of Maryland

College Park, MD 20742

Scott H. Mills

SBC Technology Resources, Inc.

Austin, TX 78759

January 2002

Final Report

This document is available to the public

through the National Technical Information

Service, Springfield, VA 22161.

An Information Tool for

Planning in Air Traffic Control Office of Aerospace Medicine

Washington, DC 20591
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-02/1

4. Title and Subtitle 5. Report Date

An Information Tool for Planning in Air Traffic Control January 2002

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Gronlund, S.D.1, John M. Canning, J.M.1, Peter M. Moertl1, P.M. 1, Joakim

Johansson, J. 1, Dougherty, M.R.P.2, and Mills, S.H.3

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

11. Contract or Grant No.

1

University of Oklahoma, Department of Psychology

Norman, OK 73019

2

University of Maryland, Department of Psychology

College Park, MD 20742

3

SBC Technology Resources, Inc.

Austin, TX 78759

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aerospace Medicine

Federal Aviation Administration

800 Independence Ave., S.W.

Washington, DC 20591 14. Sponsoring Agency Code

15. Supplemental Notes

This work was accomplished under grant # 97-G-037, Approved Subtask # AM-A-00-HRR-516.

16. Abstract

An investigation of how en route air traffic controllers solved sequencing problems using paper flight progress

strips led to the development of a new software interface to aid planning. The primary feature of the new

interface involved replacing the strips with virtual tokens that were dynamically linked to their radar targets.

In addition, an automated tool was developed that provided a tentative sequencing of the tokens. The current

experiment evaluated the automated sequencing tool. Utilization of the tool facilitated the development of the

controllers’ plans and reduced the workload of the tacticians who implemented the plan.

17. Key Words 18. Distribution Statement

Document is available to the public through the

National Technical Information Service;

Air Traffic Control, Automation, Interface, Planning

Springfield, VA 22161

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 14

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

iii

Acknowledgments

We gratefully acknowledge the support of FAA Grant No. 97-G-037. We are indebted to Ron Andrei

(AMA-510) and Paul DeBenedittis (AMA-510) for their help on this project. This manuscript was

improved by the comments of Carol Manning and Scott Shappell of the FAA Civil Aerospace Medical

Institute (AAM-510). We also thank Jorge Mendoza for his statistical counsel.

1

AN INFORMATION ORGANIZATION TOOL FOR PLANNING

IN AIR TRAFFIC CONTROL

Planning is fundamental to the successful completion of many complex, dynamic tasks. For example,

Miller, Copeland, Heaton, and McCloskey (1998)

found planning to be important for successful military operations (see also Pew & Mavor, 1998); Xiao,

Milgram, and Doyle (1997) found it to be an important aspect of anesthesiologists’ preparation for surgery. Although air traffic control has been characterized

as largely involving shorter-term tactical decision

making (Durso & Gronlund, 1999; Hutton,

Olszewski, Thordsen, & Kaempf, 1997), planning is

also an important contributor (e.g., David, 1997;

Dougherty, Gronlund, Durso, Canning, & Mills,

1999). Roughly speaking, by tactics we mean decisions that relate to the current moment and involve

the separation of (usually) pairs of aircraft; planning

decisions occur further in advance than tactics and

involve the consideration of a larger number of aircraft.

An examination of planning in air traffic control is

timely, given future concepts being proposed. For

instance, there have been discussions regarding the

creation of a strategic controller position (N. Lawson

& K. Thompson, personal communication, Dec. 15,

1997; see also Vivona, Ballin, Green, Bach, & McNally,

1996). The proposal provides for one person who

would be responsible for a multiple-sector airspace,

making decisions about traffic in that airspace, and

delegating responsibility for tactical decisions to sector-level controllers. One goal of our project was to

develop possible interface tools for a strategic controller position.

Before developing new interface tools to aid planning or to support a strategic controller position, it is

important to better understand how controllers use

their current tools to develop plans. As pointed out by

David (1997, p. 13), this is nontrivial because it

“appears to be a cognitive task almost invisible to the

outside observer. ” In addition, tactics and planning

are normally confounded in air traffic control because

both types of decision-making often lie within the

same head, even when a team has responsibility for a

sector of airspace.

Dougherty et al. (1999) solved both problems by

assigning the role of the planner and the role of the

tactician, the implementer of the plan, to two different people and requiring that the planner verbalize the

plan for the tactician to implement. The experiment

began with instructions that delineated the roles of

the tactician, the subject matter expert in the experiment, and the planner who served as the participant.

The tactician’s job was to maintain separation between aircraft; that is, he or she made whatever altitude, speed, and heading changes were necessary to

maintain separation. The planner’s job was to give the

tactician a plan for managing the flow of traffic in the

sector (a volume of airspace). The planners conveyed

their plan to the tactician by verbalizing it. The

planners sat beside the radar (the tactician sat in front

of it) and in front of the flight progress strips.1

Twelve en route air traffic controllers serving as

instructors at the FAA Academy in Oklahoma City

completed two different types of scenarios. In one

scenario, the aircraft flew standard routes (so-called

highways in the sky) in which pairs of aircraft would

only intersect at a restricted number of points along

the route. The primary problem to be solved involved

sequencing aircraft into Dallas/Fort Worth. For the

other scenario, aircraft flew direct routes through the

sector, which meant that pairs of aircraft could intersect at any point in the sector. In this scenario, aircraft

were en route to many different destinations. The

scenarios were initially presented in a paused mode

that allowed the planner the necessary time to formulate a plan. In a pilot study, we found that the planner

would immediately fall into a tactical mode if the

problem was active when first viewed.

The direct route scenario required more decisions

relating to the separation of pairs of aircraft rather

than decisions about larger groups of aircraft. However, the latter were much more frequent in the

sequencing scenario, and because these problems are

more the province of a strategic controller, we focused

on sequencing problems in the interface development

effort to follow. In the sequencing scenario, planners

 1Flight progress strips for each aircraft were stacked vertically in a strip bay adjacent to the radar display. Flight strips are 20 x 3 cm rectangular

paper strips. They have 31 fields of information, including the call sign, aircraft type, requested altitude, requested speed, route of flight, etc.

The controllers mark on these strips to update this information. In the field, paper strips are important, in part, because they can function

as a manual backup in case the radar and computers fail.
2

made greater use of the flight progress strips. Ten of

the 12 planners built their sequence of aircraft using

only the strips (one planner used the strips and the

radar, the other used only the radar). In addition, a

greater proportion of the strips were marked in the

sequencing scenario (.45 vs. .24, all p’s < .05 unless

otherwise indicated), and more strips were moved at

the beginning of the problem (5.7 vs. 1.5). This

occurred despite the fact that the planners were instructed that the tactician would not use the strips and

that they were only for the planner’s benefit.

Several sub-tasks performed by the controllers in

Dougherty et al. (1999) related to their decisionmaking in the sequencing scenario. The two most

important tasks were classifying the aircraft into groups

and sequencing the aircraft within each group. The

classifications were needed to identify the set of aircraft that would be affected by a particular constraint

such as “maintain 10 miles in trail between aircraft on

jetroute 107.” The sequences were needed to determine in which order aircraft would pass a common

ground point fix. These fixes were related to the route

or destination of the aircraft (e.g., a hand-off location

to the next sector). To perform their classification and

sequencing sub-tasks, planners needed to: 1) find

current flight status information (position, altitude,

and speed), and 2) find current flight plan information (destination and intermediate points). All of the

planners had grouped their flight progress strips by

aircraft destination and then ordered them within

each group by arrival time at a particular fix. The

strips served as tokens for the aircraft, and the stacking

order of the tokens made the sequence order explicit.

To enhance the functionality of the paper flight

strips, a software interface was designed (see Canning,

Johansson, Gronlund, Dougherty, & Mills, 1999).

Two screens were created for the planner: a radar view

and a flight organizer (see Figures 1 and 2). The

planner’s radar display looked much like the radar

portion of the Display System Replacement (DSR)

screens used by the FAA. However, the radar display

was enhanced to allow for the color coding, marking,

and highlighting of the aircraft in coordination with

the flight organizer display. Classifying and sequencing took place on the flight organizer display. Each

aircraft was represented by a small rectangle called an

aircraft token. Sets of aircraft tokens were placed into

containers called queuing blocks which represented

the groups of classified aircraft. All aircraft tokens

placed in a queuing block were displayed in the same

color. The corresponding aircraft symbol on the radar

view appeared in that same color. Therefore, the

classification was encoded using color on both displays. Planners created and positioned as many queuing blocks as they wished on the flight organizer

screen and selected a fix for each one. Time or distance

to reach the fix (at the option of the planner) was

displayed to the right of each aircraft token. An

automated sequencing tool allowed the aircraft tokens to be automatically reordered so that the lowest

token was the one closest to that block’s fix, and the

other tokens were in ascending order.

The automated sequencing tool did not create an

optimized ordering of tokens (as would a planning aid

like CTAS, Vivona et al., 1996; or URET, Brudnicki

& McFarland, 1997). Rather, it ordered the tokens by

the time or distance to reach the chosen fix, based on

the aircraft’s current position, heading, and speed. It

provided a first attempt at a sequence order that could

then be fine-tuned by the planner. This was in keeping

with a philosophy of automation that keeps the operator in the loop (human-centered automation, e.g.,

Billings, 1996). According to this philosophy, new

tools should allow more traffic to be handled and

better decisions to be made but do so not by

outsourcing the cognitive ability of the operator to the

tools. Rather, through restructuring the information

and speeding access to critical information, an

operator’s cognitive abilities can be enhanced. For a

similar philosophy applied to air traffic control see the

PHARE Highly Interactive Problem Solver developed for Eurocontrol (Meckoff & Gibbs, 1994).

Moertl et al.. (2000) reported the results of an

experiment that evaluated the new interface. The

same 12 planners participated as in the first experiment. They received about two hours of training on

the new interface prior to participating. They completed two sequencing problems: one using the flight

organizer together with the radar, and one using the

strips and the radar (the color-coding functionality of

the radar was lost without the flight organizer). As

before, planners conveyed their plan to the tactician

by verbalizing it. Planner performance (rated by the

planner and tactician and collapsed over the 20-

minute scenario) was judged to be superior in the

flight organizer condition (6.1 vs. 5.2, 9-point scale,

9 being best), although the advantage was only marginal during the first 10-minutes. The flight organizer

also resulted in a lower workload over the 20-minute

scenario for both the planner and the tactician (50.4

vs. 57.8, measured by the average of the six indices of

the NASA Task Load IndeX that includes mental

demand, physical demand, temporal demand, frustration, effort, and the assessment of performance, which

was reverse scored (Hart & Staveland, 1998). Although participants had as much time as they needed
3

Figure 1. The Radar View: One of the two screens available to the planner. See details in the text and

in Canning et al. (1999).
4

to develop their plans, a plan was developed 5.4

minutes faster, on average, using the flight organizer

than using the strips, despite only 2 hours of prior

familiarization with the interface.

The new interface seemed to offer several advantages over the existing paper flight progress strip

environment. The current experiment sought to identify the components of the new interface that were

responsible for the benefit. In particular, the current

experiment evaluated the effectiveness of the automated sequencing tool. In two of the three scenarios,

the automated sequencing tool was fully functional.

In the third scenario, the ability to use fixes with

queuing blocks was disabled. Without a fix, the automated sequencing tool was of no use and planners

would have to construct their own sequences of aircraft. The planners could still create queuing blocks,

but they could not reorder the tokens within a block

at the touch of a button.

One new feature was added to the interface for this

experiment. The planner now had the capability to

mark on the aircraft token the control actions they

decided should be taken to achieve the planned sequence. Planners typed the appropriate letter and then

clicked a token to indicate that a speed, vector, and/

or altitude control action was needed. The letters

appeared within the token in the center of the bottom

row (see Figure 2). The planner could mark more than

one control action for a particular aircraft, and change

it whenever necessary. The planner could also indicate

that no control action was needed for a given aircraft

(i.e., it would fall into place given its current speed,

altitude, and heading). For example, the token for

TWA825 in Figure 2 shows that speed should be used

by the tactician to achieve the appropriate sequencing

of this aircraft, while no action is indicated for

UAL755, which meant that it should fall into place

given its current speed, heading, and altitude. Also,

rather than requiring that the planner verbally convey

the plan to the tactician, the planner simply printed

out a copy of the sequence order for the tactician that

included the accompanying control actions. The tactician remained responsible for issuing the control

actions to the pilots, and for issuing other control

actions needed to maintain the necessary separation

between aircraft.

Although the aforementioned annotation of aircraft tokens facilitated communication of the plan to

the tactician, it also allowed for the introduction of a

second independent variable into the experiment involving plan detail or specificity. Planners were required to create plans at one of two different levels of

specificity. A partial sequence meant that the planner

could sequence as many of the aircraft in a queuing

block as they wished. A complete sequence meant that

the planner had to create a more detailed plan by

sequencing all the aircraft within a queuing block.

Planning aids like URET (Brudnicki & McFarland,

1997) and CTAS (Vivona et al., 1996) allow controllers to make more detailed plans further into the

future. On the surface, this seems like a good idea. But

such plans might be particularly prone to cognitive

rigidity (e.g., Taylor, Finnie, & Hoy, 1997). Cognitive rigidity, a form of functional fixity, would occur

when a planner stuck with an existing plan despite

evidence (as judged by an outside observer) that the

plan required modification. Cognitive rigidity can

also be viewed as a form of sunk costs (Arkes &

Blumer, 1985). More time and effort devoted to

developing a plan might make a planner more willing

to maintain an existing plan, despite evidence that it

should be modified. As a result, planning performance was predicted to be worse when a complete

sequence was required.

Method

Participants

Five en route air traffic controllers participated as

planners. All had participated in our prior interface

experiment, which meant that they had experience

using the flight organizer. None of the remaining

participants from the prior experiment were available,

and time and resources were unavailable to train

additional participants on the interface. All participants were instructors at the FAA Academy and were

familiar with the AeroCenter sector used in the experiment. All were full-performance level (FPL) controllers (i.e., certified to work a sector independently).

They had been FPL controllers for an average of 20.5

years. They last worked in the field 3.8 years previously, on average. The tactician, our subject matter

expert, was also an FPL controller.

Materials

The experiment was conducted at the Civil Aerospace Medical Institute in Oklahoma City. The planner sat at one end of a long room, and the tactician and

two ghost pilots sat at the other end (the ghost pilots

on either side of the tactician). The ghost pilots

controlled the simulated aircraft based on the

tactician’s instructions. The planner and tactician

communicated using an intercom system that simulated a radio. The tactician and the ghost pilots

communicated over the tops of the cubicles that

separated them. The tactician had a single screen on
5

Figure 2. The Flight Organizer: One of the two screens available to the planner. See details in the text

and in Canning et al. (1999).
6

Table 1

Experimental Conditions with Constraints

Aircraft Sequencing Control Action Marking Fix Status

Enabled-Partial Partial Optional Enabled

Disabled-Partial Partial Optional Disabled

Enabled-Complete Complete Mandatory Enabled

which a radar display was shown. The planner had the

flight organizer and the radar display screens described above (see Figures 1 and 2).

Four scenarios were developed by the controller

expert serving as the tactician. The scenarios were

judged to exceed the workload level typically experienced in the field; there was an average of 36.5 aircraft

to be handled during the 10-minute scenario. The

primary problem to be solved involved sequencing

aircraft into two different destination airports. These

were called the primary sequences. Time constraints

limited the experiment to completion of only three of

the scenarios, although participants were rotated

through all four.

Procedure

Each participant completed a 45-minute training

session on the use of the flight organizer functions.

This was followed one or two days later by a 45-minute

practice session with one of the trainers (graduate research assistants who were familiar with the interface).

The experimental session began 2-3 days later.

There were three conditions in the experiment. In

one, the planners created complete sequences for the

two primary sequences in the scenario. Control actions needed to achieve the sequence had to be designated for all aircraft in the two primary sequences. In

the other two conditions, planners created partial

sequences for the two primary sequences. In other

words, it was the planners’ choice which aircraft had

control actions designated and which aircraft had

these decisions deferred. The automated sequencing

tool was disabled for one of these two partial sequence

problems. This meant that the planners had to construct their sequences by manually determining distances. The automated sequencing tool was enabled

in the other partial condition and in the complete

sequence conditions. In sum, the three conditions

were: enabled-partial, disabled-partial, and enabledcomplete. See Table 1 for an overview of the requirements of the different conditions. The ordering of the

conditions was randomized, as was the assignment of

the four scenarios to the three conditions.2

The instructions began by delineating the roles of

the tactician (the controller expert) and the planner

(the participant). The tactician’s job was to maintain

separation between aircraft; he would make whatever

altitude, speed, and heading changes were necessary to

maintain separation. Only the tactician would

communicate with the pilots. The planner’s job was

to sequence the aircraft in the two primary sequences.

The planner was asked to indicate how the sequencing

should be achieved by marking whether speed, altitude, heading or no control action was to be used to

provide the necessary spacing and sequencing. The

planner could print a copy of the plan for the tactician

anytime the planner wished. The print-out contained

an ordered list of the aircraft in the two primary

sequences and the control actions designated by the

planner to achieve the sequences.

The experiment began with the first scenario in the

paused mode. The planners then configured the flight

organizer to their preference. In particular, all the

planners created at least two queuing blocks, one for

each primary sequence. Although the planners were

shown how the interface worked during training, we

did not require that they use the interface in a particular way. Aircraft going to the appropriate destinations

were then placed into the designated queuing blocks

and sequenced either manually or by using the automated sequencing tool, depending on the condition.

After completing the plan, the planners pressed a

button on the interface to print two hardcopies of the

planned sequence. One copy was given to the tactician, who used this information to implement the

plan. The other copy was given to the planner, who

2

 Time constraints forced us to drop the disabled-complete condition. In a pilot study, plan preparation alone took over 20 minutes.
7

Table 2

Tactician’s Subjective Workload and Rating of Plan Quality for the Three Experimental

Conditions (Standard Deviations Given in Parentheses).

Enabled-Complete Enabled-Partial Disabled-Partial

Tactician’s TLXa

51.0

(7.08)

44.9

(14.27)

63.6

(17.39)

Rating of planner’s

plan qualityb

5.25

(0.97)

5.55

(1.35)

4.98

(0.89)

a

 Averaged across the five workload subscales, performance excluded. Scale ranges from 0-100 with 100 indicating

maximum workload.

b

 Averaged across initial plan quality, quality of revisions, strategic awareness, and strategic planning. Scale ranges from

1 to 9, with 9 indicating superior planning. These data represent the average of the tactician’s and the planners’ ratings.

was asked to circle aircraft considered to be in groups.

Then, for each circled group of aircraft, the planners

rated their confidence level that the aircraft in a group

were in their final sequence order (1=very certain to

5=very uncertain). Next, the planners rated their

confidence level that the control actions marked were

sufficient to establish the sequence order (again, 1=very

certain to 5=very uncertain). Once the planners completed this process, the scenario was started and the

tactician began implementing the plan. The planners

were instructed to modify their plan as necessary and

to print out the revised plan for the tactician whenever

they wished.

The scenario was stopped after 10 minutes, and the

tactician completed the NASA-TLX (Hart &

Staveland, 1998), which assessed mental demand,

physical demand, temporal demand, frustration, and

effort during the preceding 10 minutes. The NASATLX assessment of performance was excluded and

replaced by having the planner and the tactician

independently answer five questions related to the

planner’s plan: the planner’s involvement in the tactics, quality of the initial plan, quality of revisions,

planning effectiveness, and strategic awareness. A 10-

minute break followed. After finishing the three scenarios, the planner completed a questionnaire that

collected biographical information. The participants

were then debriefed and released. An experimental

session took approximately 2.5 hours.

Results

The tactician’s TLX workload differed significantly

across conditions (F(2, 3) = 15.82, p = .026, see Table

2), according to Hotelling’s (1931) T2

 test. Hotelling’s

T2

 test was preferred to a one-way ANOVA because it

does makes no assumptions about the variance/covariance matrix or assumptions about the sphericity of

the data. In addition, Marcucci (1986) showed that

the T2

 test was a more powerful test when epsilon

departs appreciably from 1.0 (the Greenhouse-Geisser

epsilon was .585). The tactician’s workload was significantly less in the enabled-partial condition than in

the disabled-partial condition. This comparison revealed that the tactician’s workload was reduced when

the planner had access to the automated sequencing

tool.

The tactician and the planner independently rated

the quality of the planner’s plan. The ratings were

combined in Table 2 and in the following analysis.

Although the plan was judged best in the enabledpartial condition and worst in the disabled-partial

condition, consistent with the tactician’s reduced

workload when the planner used the automated sequencing tool, there were no significant differences

across conditions (F(2, 3) = 1.87, p = .3, according to

Hotelling’s T2

 test). Although the lack of a significant

difference could be a power problem, note that we had

enough power to detect a change in the tacticians’

TLX ratings.
8

The lack of a difference in plan quality was probably because the planners had as much time as they

wished to develop their plans. Therefore, an indication of the effectiveness of the automated tool might

not manifest itself in an increased rating of plan

quality but instead in a reduced time to develop the

plan. In other words, as is often the case in cognitive

psychology, a difference in processing efficiency can

be observed in either the accuracy or the speed, depending on the parameters of the situation. For example, a difference in processing efficiency between

students completing their first year versus their fifth

year of study of a foreign language could result in a

longer time for the first-year student to translate a

passage, although they may eventually be able to

translate it as accurately as the fifth-year student.

However, if we give the first-year and the fifth-year

students the same amount of time, less than what is

needed by the first-year students, the time spent

translating would not differ (by definition), but the

difference in processing efficiency would result in

reduced accuracy for the first-year students.

Use of the automated sequencing tool did reduce

the time needed to complete plan development. Planners created their initial plan an average of 3.75

minutes more quickly when the automated sequencing tool was enabled than when it was disabled (enabled-partial 541 seconds, s = 180.9 vs. disabled-partial

766 seconds, s = 275.8). Although this difference was

of marginal statistical significance (by a one-tailed

test, t(4) = 1.93, p = .063), an improvement in plan

development time of this size is of great practical

significance to the controller.

Of primary interest in assessing the impact of plan

specificity was the comparison of the enabled-complete

to the enabled-partial condition. This manipulation

resulted in control actions being indicated for 94% of

the aircraft in the two primary sequences in the

complete condition but only 65.5% of the aircraft in

the two partial conditions. The tactician’s workload

was greater in the enabled-complete condition relative to the enabled-partial condition, but not significantly so. Although plan quality was judged worse

when the planner created a more detailed plan in the

enabled-complete condition, as was discussed above,

the difference was not significant during the initial 10

minutes of the scenario. The support for the disadvantages of increased plan specificity was much weaker

than the support for the advantages of the automated

sequencing tool.

Small differences across conditions for the remaining results made the mean a sufficient summary. The

planners circled an average of 5.2 aircraft as making

up a group. This was larger than the group size of 2.7

aircraft reported by Means et al. (1988) in a task that

involved circling recalled aircraft on a map. This

difference may reflect the larger groups of aircraft

considered by controllers in a planning role versus

their more typical role. However, it may also reflect

differences in aircraft density, instructions, questions

asked, etc. Planners expressed high confidence in the

sequence order they presented to tacticians (m = 1.68,

1 to 5 scale, 1= very certain). The majority of the

control actions marked were speed (mean frequency

of m = 13.3 aircraft) or vector (m = 9.4). Altitude

changes were not important for solving the sequencing problems, m = 0.33. No control actions were

indicated for 5.8 aircraft on average. There was a

marginal positive correlation between the number of

speed and vector control actions (r = .48, p < .07).

Apparently, rather than a preference for planners to

set-up their sequences using speed or vectors, the

positive correlation suggested that planners tended to

designate more of both types of control actions. However, a median split (all three conditions collapsed

together) on the number of control actions showed

that the tactician judged those planners who designated fewer control actions to have a better plan.

Perhaps a planner with a better “big picture” of the

sector knew the one control action that would solve a

problem, while the planner with a poorer picture

would have to solve the problem in pieces through a

series of control action adjustments.

Discussion

The results of the experiment show consistent and

beneficial effects of using the automated sequencing

tool. The tactician’s workload was decreased. Although plan quality was equivalent across conditions,

plans were developed more rapidly using the automated sequencing tool. The results of Moertl et al.

(2000) were similar. In that experiment, planners

developed plans using the interface (including the

automated sequencing tool) or the paper strips. Although plans were developed far more rapidly using

the interface, there was no significant difference in

plan quality during the initial 10 minutes of the

scenario. Moertl et al. found superior planning performance in the second 10 minutes of the scenario when

planners had to update and adapt their plans to the
9

changing traffic situations. In the present experiment,

the planners only completed 10 minutes per scenario.

Perhaps differences in plan quality would have appeared

if the scenario was run for 10 additional minutes.

The results of the experiment hint at a detriment to

performance as a result of making more detailed plans.

On the one hand, it seems like a good idea for

controllers to use tools like URET and CTAS to look

further into the future. On the other hand, having

more detailed plans, plans that a controller might be

more invested in, could result in greater cognitive

rigidity and hence poorer performance. Additional

studies that manipulate the level of detail of a plan and

the degree of commitment to it should be conducted

to verify that planning aids actually are beneficial.

However, future attempts to examine this construct

should consider other manipulations of plan specificity to include dependent variables better targeted at

cognitive rigidity or, perhaps, situation awareness.

An examination of en route air traffic controller

planning showed that an information organization

tool could enhance the solving of sequencing problems. Prior research showed that the flight organizer

was superior to the existing paper flight strip environment. In the current experiment, a component of the

flight organizer, a sequencing tool that sorts aircraft

tokens within a queuing block, was shown to decrease

the workload of the tactician (who implemented the

plan) and speed plan development. Dissecting a new

interface is an important aspect of interface development as it can help determine what features of the

interface are beneficial to performance and what aspects of performance they enhance. In the present

experiment, we learned how the automated sequencing tool enhanced performance. Surprisingly, it was

the tactician’s workload, rather than the planner’s,

which benefited from the use of this tool. The tool

did, however, reduce the time needed by planners to

develop their initial plan. By dissecting an interface in

this way, we can determine what features of the

interface should be maintained and what those different features contribute to the overall performance of

the system.

The flight organizer interface we designed for a

future strategic controller position represents an alternative approach to air traffic control modernization.

We first tried to understand how the controllers

function in their current environment, and then tried

to systematically enhance that functioning through targeted tools that kept the controller in the loop rather than

outsourcing the planning process to software.

References

Arkes, H.R. & Blumer, C. (1985). The psychology of

sunk cost. Organizational Behavior & Human Decision Processes, 35, 124-140.

Billings, C.E. (1996). Human-centered aviation automation: Principles and guidelines. NASA Technical Memorandum 110381, Moffett Field, CA:

NASA.

Brudnicki, D.J., & McFarland, A.L. (1997). User

Request Evaluation Tool (URET) Conflict Probe

Performance and Benefits Assessment.

(MP97W112). McLean, VA: MITRE

Corporation.

Canning, J.M., Johansson, J., Gronlund, S.D.,

Dougherty, M.R.P., & Mills, S.H. (1999). Effects

of a novel interface design on strategic planning by

en route controllers. In R.S. Jensen and J.D.

Callister (Eds.), Proceedings of the 10th International Symposium on Aviation Psychology (pp. 528-

533). Columbus, OH: Ohio State University Press.

David, H. (1997). Radical revision of en-route air traffic

control, EEC Report No. 307, Eurocontrol Experimental Center, Brussells, Belgium.

Dougherty, M.R.P., Gronlund, S.D., Durso, F.T., Canning, J. & Mills, S.H. (1999). Plan generation in

air traffic control. In R. S. Jensen and J. D. Callister

(Eds.), Proceedings of the 10th International Symposium on Aviation Psychology (pp. 534-539). Columbus, OH: Ohio State University Press.

Durso, F.D., & Gronlund, S.D. (1999). Situation awareness. F.T. Durso, R. Nickerson, R. Schvaneveldt,

S. Dumais, M. Chi, & S. Lindsay (Eds.), The

Handbook of Applied Cognition (pp. 283-314).

New York: Wiley.

Hart, S.G., & Staveland, L.E. (1998). Development of

NASA-TLX (task load index): Results of empirical

and theoretical research. In P.A. Nacock and N.

Meshkati (Eds.), Human Mental Workload (pp.

139-181). Elsevier: North-Holland.

Hotelling, H. (1931). The generalization of Student’s

ratio. In:Annals of Mathematical Statistics, 2,

360-378.

Hutton, R.J.B., Olszewski, R., Thordsen, M.L., &

Kaempf, G.L. (1997). En route air traffic controller

decision making and operational errors. Dayton,

OH: Klein Associates.
10

Marcucci, M. (1986). A comparison of the power of

some tests for repeated measurements. Journal

of Statistical Computation and Simulation, 26,

37-53.

Means, B., Mumaw, R.J., Roth, C., Schlager, M.S.,

McWilliams, E., Gagne’, E., Rice, V., Rosenthal,

D., & Heon, S. (1988). ATC training analysis

study: Design of the next-generation ATC training

system. (Report No. FAA/OPM 342-036) Washington, DC: Department of Transportation/Federal Aviation Administration.

Meckoff, C., & Gibbs, P. (1994). PHARE Highly Interactive Problem Solver, Eurocontrol report 273,

Brussels, Belgium.

Miller, T.E., Copeland, R., Heaton, J.K., & McCloskey,

M.F. (1998). A cognitive approach to developing

decision support tools for air campaign planners.

(Technical report prepared under contract F30602-

95-C-0216 for Rome Laboratory) Fairborn, OH:

Klein Associates Inc.

Moertl, P.M., Canning, J.M., Johansson, J., Gronlund,

S.D., Dougherty, M.R.P., & Mills, S.H. (2000).

Workload and Performance in FOPA: A Strategic

Planning Interface for Air Traffic Control. Proceedings of the 44th annual meeting of the Human

Factors and Ergonomics Society, 3.65 – 3.68.

Pew, R.W., & Mavor, A.S. (1998). Panel on Modeling

Human and Organizational Behavior: Applications

to Military Simulation. Washington, DC: National

Academy Press.

Taylor, R.M., Finnie, S., & Hoy, C. (1997). Cognitive

rigidity: The effects of mission planning and automation on cognitive control in dynamic situations. Paper presented at the 9th International Symposium

on Aviation Psychology, Columbus, OH.

Vivona, R.A., Ballin, M.G., Green, S.M., Bach, R.E, &

McNally, B.D. (1996). A system concept for facilitating user preferences in en route airspace, NASA

Technical Memorandum 4763, Ames Research

Center, Moffett Field, CA.

Xiao, Y., Milgram, P., & Doyle, D. J. (1997). Capturing

and modeling planning expertise in anesthesiology: A field study. In C. E. Zsambok & G. Klein’s

(Eds.) Naturalistic Decision Making, p.p. 197 -

207. Mahwah, NJ: Erlbaum.