Conflic: alerts and MSAWs are, at best, gross
tformance, because they could
inent. Consistent

measures ofcontrolier

occur even w’ F
with Tabie 4, there were no reliable differences in the
ies of these alerts between the FPS and RFPS

frequen

conditions.

Table 4. Frequency of aleris.

Co! 44
(MSAWs _ 2.30 2.84
Compensatory Behaviors
Of the 12 behaviors we tallied from the R-side that

could reflect compensatory behavior, including some
communications events (e.g., requests of pilots for
current speed), only the frequency with which chey
pointed ro the PVD evidenced any difference between
RFEPSs and FPSs, (19) = 2.79: When using reduced
strips, the R-side pointed to the PVD 48% more often
{6.07 vs. 4.10) than he or she otherwise did. Notice
that pointing to the PVD more was not because the
controller was pointing co the strips less. OF the 5
behaviors tallied for the D-side, none were signifi-
cantly effected by the type of strip. The frequencies of
compensatory behaviors appear in Table 5.

Workload
TL

Workload, as assessed by the modified TLX, was
analyzed separately for R-side and D-side. The D-side
controlier reported experiencing comparable work-
load, regardless of whether the FPSs or RFPSs were
used. For the R-side, however, the RFPSs produced
greater self-reported effort, 19) = 2.17, and frustra-

tion, 419) = 2.79.

Granting requests

Another less direct, but mere o'
load, is the number of pilot requests granted and

wo:

the time required co grant chem (see Table 6).

for R-side and

performance

Table 5. Compensatory sshavi
B-side, R-D communication, anc
related controller behaviors.

ICoast tracks 4.85 4.43)
iFlight Plan Readout 1.85 1.82
Route display 8.60 8.91
Point-to-PYD i 449) 8.07
hAiriing i 174.9) 165.2)
Point-to-Strio 34.25 87.35)
‘Offset strip 3.08) 3.88)
View strips sec} | 4057.00) 1697.30)
Vector line £.36 5
Range of PVD 4i5 3G
Pilot requests 48.40) 48.90!
id-ings 1.30) 4.35;
i D-SIDE
Flight Plan Readout 1.68) 2.16
Point-to-PVD 10.80} 41.73
Point-to-Strip 25.15; 26.87
Cfiset strip 42.55) 10.82;
Wiking 147.80} 134.85
R-D COMMUNICATION
Time {sec} 483.55; 188.07
Ralics =p <.10
iBold =p <<

Controllers grented the same percentage of requests,
and granted them in approximately the same amount
of time. Altho the mean grant time for FPSs
appears considerably longer than che mean grant time
for the RFPS, chis 22 second difference is due prima-
tily to one team’s exceptionally long latency.

Opinion

When the controllers were asked to evaluate the 2 strip
formants, the results were quite clear (see Table 7). On the
6 scales, the controllers showed a significant preference
for the cu

Additional open-ended responses about the REPS
were tallied. Controllers were allowed to make as
many of as few comments as they wished; thus, the
following summary, unless i

nt FPS on each scale.

icated, over-represents
o offered more suggestions. When
d about reduced strips, 55%

those controllers wh
asked what they

SONG ALC REESE ES TR RIETESCE



CaN

i

Executive Summary

Currently, en route air traffic control depends
on computer-augmented radar information avail-
able on the Plan View Display (PVD), Computer
Readout Device (CRD), and the Aight informa-
tion available on the Flight Progress Strip (FPS).
Thirty-one fields for information are printed on
the strips. This information supplements the
data available from the PVD. While an aircraft is
in the controller's sector, the controller writes on
the corresponding strip ro reflect the control
instructions, any changes made in the flight plan,
and other contacts with the aircraft.

This report describes an experiment that
compared the effects of using a standard-sized
(i5/is” x 8”) FPS and a smaller (3” x 5”) FPS
containing less information on the performance
and workload of controller teams. The teams,
from Minneapolis ARTCC, controlled simulated
air traffic in a mixed radaz-nonradar environ-
ment. Overall, the 1” x 5” reduced strip yielded
deficits in the control of nonradar flights but not
radar flights. This was evidenced in the subject
matter experts’ evaluation of nonradar separa-
tion, strip processing and board managemeni,
and, to a marginal extent, in the efficiency with
which traffic was moved through the sector. The
Reside controller's awareness was also rated lower
when using the smaller strips. Interestingly, the
controllers’ eva:uation of their own performance
did not reflect this difference between reduced and
normal strips. This may help explain why contro!-
fers did not compensate for the smaller strips to any
great extent. .

Only the radarside (R-side) controller exhibited
any compensatory behaviors and only the R-side
reported increased workload. R-side controllers
pointed to the PVD more often. Although there
was little compensatory activity, the R-side contro!-
fer felt the workload was greater with the smaller

strips. The R-side controlier feitit was more effortful
and more frustrating working with the 1”x 5” strips.
Despite being under a self-reported heavier work-
load, the controllers nevertheless were able to pei-
form secondary tasks, such as granting pilot requests,
as often end as quickly using the smaller strips as
they did using standard strips.

This study also supplied information about
the specific air traffic activities likely to be af
fected by 2 reduction to 2 1” x 5” FPS. Strip
marking, speed of strip processing, and some
aspects of board management seemed especially
affected, inferior steip marking wes evidenced in
et ne expert evaluation and the controllers
often reported that the size of the 1” x 5” strip
prevented writing. The ability to locatea particu
lar strip and to find the information on the strip
once ic was found seems to suffer with a reduction
in the size of the strip used in this study. The on-
line expert evaluation and the controller opin-
ions echoed this problem. Locating strips might
have been especially difficult for the R-side, thus
leading to the large difference in self-reported
frustration. The controliers also noted specific
problems with the strip display, including the use
of shading to replace information that is typically
presented in red. Of board management respon-
sibilities—considezed by the controllers as gen-
erally inferior with 1” x 5” strips—removal of
deadwood seems less likely to be negatively af
fected by reduction in strip size. The on-line
expert evaluation rated the 1” x 5” strips nega-
tively and the subject matter experts recorded
more negative comments about removal of dead-
wood under that condition.

Overall, this study does aot permit recommen-
dation of the 1”x 5” reduced strip as designed for
this study. Suggestions for selecting < less than
standard size FPS are made in this report.



:
|

RST

Separation
Non-radar

Radar

Nee «| Essential Srip Marking
[ i Essential for effective control

i Mandated by 7110.65

Saeed
Difficulty locating strips

Slow strip scan

Non-verbal communications (one verbally which can be done non-verbally)

Awareness

Board Management
Additions to active bay

Removal of deadwood

Effective sequencing

Sample On-Line Expert Evaluation (OLEE) form used by subject matter experts (SMEs) to rate the
effectiveness of controller activities involving strip usage. SMEs rated R-side activities separately from D-

side activities.
S = Satisfactory, NI = Needs improvemeni, U = Unsatisfactory.

lus. GOVERNMENT PRINTING O¥YICE: love -Tep-suareast2



TMOLRCE

‘Two instruments were used to assess co:
formance. One was a version of the standard OJT
evaluation form (FAA Ferm 3120-25) modified to focus
more precisely on strip activity; like an OJT evaluation,
this On-line Expert Evaluation (OLEE) was performed
by expert controllers while the participant controiler was
engagedi the scenario. The other was the Post-scenaric
analysis we developed previously to assess the number of
remaining control actions (Vertac et al., 1993). L
addition, 2 alerts from the contreller’s system were
recorded when they occurred.

oller per-

On-line experteveluation. ‘The SMEs assessed contro!-
jer performance “over the shoulder” while the team was
controlling traffic in the scenario, Each SME had the
OLEE form, as shown in Appendix A. The SME from
the relevant area of specialization always observed the D-
side, while the SME from the other area observed the R-
side. This was for 2 reasons. First, an FPL from a different
area can more easily evaluate mistakes apparent on the
zadar than mistakes on the D-side. Second, the focus of
the study was on changes in flight dete utilization,
suggesting the more knowledgeable controller should
evaluate D-side performance. Of course, thé SME ob-
servers might differ in how they used the CLEE form,
but because scenarios were counterbalanced across str’
condition, such differences cannot affect the resul
systematically. Each observer indicated whether each
performance category was Satisfactory, Needs Improve-
ment, or Unsatisfactory. Observers also kepta tally ofthe
mistakes controllers made within each of the OLEE

categories.

Post scenario analysis. At the end of each scenatio, the
2SMEs consulted in completing the post-scenario analy-
sis. The post-scenaric analysis is 2 quasi-objective mea-
sure of performance that we developed for an earlier set
of studies. The SME determines, for each fight, chose
controller actions still remaining that wili successfully
remove the flight from the controller's sector. Assuming
the same starting point, and the same emount of time
allowed for controlling traffic, a condition that leaves
fewer remaining actions can be thought of as one that
suppozted more efficient contzol. Finally, because FPLs
typically agree on the actions necessary to transit an
aircraft out of the sector, we believe the remaining
actions measure makes a strong appeal to odjecti

‘The curr Tent system alerts ss th e controller
that loss of sepe: of altin
is imminent. Al
separation was |
a larger number of such occurrences in
compared with the oth
croller is perform:

y be lost,
one condition
may again suggest chet the

co.

Controller Behaviors

iors in which controllers engage can p
videvaluable clues to the potential effects ofa new sy

m

on their performance. Some of the behaviors are likely to

behaviors are
erformance

compensate for an inferior display. Och
relevant te performance and suggest poor
in one situation than another. All the behavioral obser-
vations described below were obtained on-line by ob-
servers seated behind the con:

ing the scenarios.
The experimenters used a tally-sheet, and, where appro-
priate, 2 stopwatch, to record elevani data. One
experimenter observed the behavior of the R-side and
the behavior of the D-side. For those t
involving communication between the controller and
pilots, other facilities, or other sectors, the data were
gathered following completion of the scenario from a
multitrack zape recorder that captured the radio and
telephone communications of the controllers.

anothe:

Compensatory Bebaviors

In order to maintain an acceptable ievel of perfor-
mance, controllers may need to compensate for informa-
tion not present on the RFPS, or not easily seen oF
rettieved from the RFPS. Such compensatory beh: vior
are not necessarily negative, but they are cer vainly i
mative. For example, in the Albrig! right © et ai. study using
field controllers from Atlanta ARTCC, controllers wi
out strips compensated by performing more flight plan
readouts. Replacing strip inspection with FPRs hed
interesting consequences. Controllers could spend more

using the quick action keyboard (QAK) to obtain other
flight data, chan when he or she obtained those data from
che strip bay. Thus, compensatory behaviors often pi painted
2 complex picture that must be evaluated carefully.

we recorded several differ-
rs that coul

ent con voller b be!
pensat

serve as com-

mechanisms.



elds that are no longer of value to the controfler— and
this was done in the current study. It is important to
ascertain whether controllers can effectively control traf-
fic when the FPS is physically reduced in size and some
of the fields typically found on the FPS are no longer
present.

There is some e
¢ FPS would not necessarily hurt ATC perfor-
mance (Albright, Treitt, Barile, Vortac, & Manning,
1994; Vortac, Edwards, Fuller, & Manning, 1993;
Vortac, Barile, Albright, Truitt, Manning, & Bain,
1996). Vortac and colleagues have shown that, at lease
in some types of sectors and situations, the amount of
information on the strip can be pared considerably
(Vortac et al., 1993; 1996) or in some cases research
suggests that the strip may be eliminated altogether
(Albright ec al., in press). Although the controller was
often forced to compensate for the lack of an FPS {e.g.,
by doing a Flight Plan Readout (FPR)) there was not
necessarily an increase in workload or decrease in perfor-
mance. In fact, in some cases, the controller was able to
spend more time looking at the PVD (using FPRsin lieu
of strips) than was the case with full FPSs. In another
case, w. h only a one-line 5-field serip and no permitted
sttip-marking, some aspecis of ATC were ectually supe-
rior to 2 full FPS condition in which the controllers were
requized to mark the strips. Finally, the Air Traffic Rules
and Procedures Service has recently given en route
facilities some latitude in the use of scrips. Thus, it is not
unreasonable to assume that removal of information
from the current FPS may be « benign modification of
the way flight data are presented.

One caveat to the above argument is that, to date,
efforts to test the limit of FPS information have used only
sectors in which the flights are continually tracked by the
radar. In mixed environments, where some flights are
never under radar control, and where others enter and
feavezadar control, the role of the FPS and its constituent
fields seems especially critical.

Our team was charged with determining the effects of
reducing the size of the current FPS. We began by
consulting with air traffic specialists at the FAA Academy
and then with specialists and National Air Traffic Con-
ler Association (NATCA) representatives at the Min-
neapolis Air Route Traffic Control Center (ZMP). All
consultants were intimately familiar with controlling
traffic in mixed environments. The purpose of these
‘rations was to devermine what information, ifany,

ence thax removal of informa-

tion on

SL

could be eliminated fom the FPS. Final determination

of the information to be presented on the reduced FPS
(RFPS) was lef to the controller consultants, who as 2
body tended to be conservative in whaz information was
removed. For example, although only one controlter fic
that groundspeed was important, groundspeed was, nev-
ertheless, retained on the RFPS.

The blank RFPS was identical to the current en route
FPS but was ceduced proportionately to 1” x 5° dimen-
sions. Thus, aif line demarcations were retained oa the
REPS. However, a number of fields in which compurer-
generated information typically appeared were le& blank
on the REPS. Specifically, the RFPS no longer contained
the verification symbol, revision number, sector num-
ber, revised ground speed, strip number, previous fx
informacion (including estimated time), and next posted
fix. Font size was also reduced.

y, we were instructed to indicate information
chat currently appeass to the controller in red (e.g., CID,
special routing, end cailsigns) with shading. Although
from a human factors point-of-view, if shading reduces
contrast too dramatically it is unlikely to be a viable
candidate for increasing the readability of critical infor-
mation, the printer targeted to print the RFPS thaz will
be used with the common console will be incapable of
printing color. Nevertheless, to preview one finding of
the report, we recommend that any monochromatic
highlighting of critical information use a method ozher
than shading (e.g., bolding or boxing).

The 1°x5” REPS with eliminated information, and with
shad! ed information,” appears in Figure 1.
ne flight is reproduced for comparison.

‘The current experi
tiveness of air traffic conzrol using the fall FPS with che
ficiency and effectiveness of the RFPS.

Method

Site
he study was conducted at the Minncapolis Center
¢ of 29 Air Route Traffic Control Centers
ind for
airspace from the Canadian border to northern Kansas,
and from eastern Michigan to the western Dakotas.
ZMP has a number of mixed sectors: Air traffic is under
both radar and nonradar contic!. The nonradar volumes
of aizspace exist because of a lack of radar sites through-
he center’s airspace and not, for example, because of

out
geographical impediments to radar coverage.

Ean ee


REDUCED FLIGHT PROGRESS
Route ATC Mixep ENVIRONMENTS

EN

Introduction

The Federal Aviation Administration (FAA) 5
been engaged in an intense effort to modernize the
equipment that controllers use to help eusure the safe
and expeditious movement of the nation’s air traffic.
For reasons detailed elsewhere, attempts to compietely
modernize the system have met with only partial
success. Plans for the Initial Sector Suite System
(ISSS) have been revised significantly. The state-of
the-art com non-consoles that were the technological
underpini:ings of ISSS are available and will be placed
in the field soon.

Unfortunately, the efforts to automate the presenta-
tion of flight data have been less than successful. and the
common console is no longer planned te incorporate
electronic zenditions of aircraft flight progress strips.
Thus, the probiem is to take maximum advantage of the
functional aspects of the common console, while provid-
ing controllers with the flight data information they need
to separate aircraft.

Currently, en route flight progress data are presented
on 1 5/16” x 8” strips of paper called Flight Progress
Strips (FPSs). The FPS includes 31 fields in which
information about a particular flight is presented by the
computer or entered and modified by the controller. A

STRIPS

figure of the current FPS, together with an expla
of the fields, can be found in FAA Order 7110.65 and is
reproduced report. The FPS is i
in a plastic strip holder and these holders are sequenced
ipulated in vertical bays next to the radar scope
retained, a
radar associate's
-console, making access to the
ficult, if ne f

bay of these
side of the common
console’s technology

The 1” x 5” Flight Prayress Strip

One solution to this problem is to present controllers
with Might progress data in much the same way that it is
currently presented, but to reduce the size of the FPS so
that more of the new equipment is accessible by the
controller. However, che functionality of th:
be threatened by reducing its physical size o=
ing some of the 31 information fields. The FPS not on!

supplies information to the contr
and modified by the controller. Thus, a small
its concomitant reduction of con: mation
could affect both the controlles’s ability to locate and
extract information about a flight and the ability zo find
ugh space to update and modif ation on the

¢ q r
58 ' ISE saucy
i eee i
' | i
| i :
Hl | t
fy E z
H E i i
a E q 4
4 i H i :
: i i J
i i ;

Figure 4. Examples of flight progress strips used in this study. On top is the fight progress st
t progress sirip (RFPS), which has the cai

@ bottom, is the 1” x 5” reduced

used today.
and compute:



°

field was less frequent than in our experimental sce-
narios. When asked about the realism of the scenarios,
and nd nozhing unrealistic about the scenarios. The

r typical live traffic mentioned with any frequency
(1) were busy small airports (9) end more nonradar
traffic (5).

The scenz. ios were tested and modified by the SMEs
until they a, peared: -nparable in difficulty, realism,
and complexity. Characteristics of the 4 scenarios appear

in Table 1
On-line Production of Reduced Strips

Plastic strip holders for the RFPSs were created by
removing the middie 3” from the strip holders used in
terminal facilities. The strips used by termina! control-
lers are only i” in height, neatly fitting the requirements
for the current study. Similarly, the paper used to print
terminal strips was used to print the RFPS. The paper,
perforated every inch, was the same color as current en
route FPSs, and was blank on one side, allowing us to
create the RFPS on the blank side of the strip. A wooden
strip bay with metal dowels was constructed to permit
the sequencing and manipulation of the smaller RFPS.

Because it was not possible to interface directly with
the HOST to produce the RFPSs, we utilized a program
Ge., STRIPS) wricten by the staff at ZMP explicitly for
the purpose of quickly translating full FPSs into the
RFPS;, An experimenter sat next to the HOST’s printer
and removed FPSs as they were made available. Thi
experimenter identified the calf siga to another experi-
menter operating the STRIPS program. Thecall sign was

strip from 2 database of
hen changed the
es (hours
and tninutes) 9 on ake stip, and the flight’s beacon code.
Because the CID, time, and beacon code depended on
the time © petiod durin ng which the problem was pre-

0 ae made by controll f ing the scenario, the
experimenters either pulled up a strip from the database,
anticipating that rerouting, or modified an existing strip
in the database to accommodate the route change. Ob-
viously, the process of removing strips from the HOST
printer, translating them, and then printing them on the
terminal forms took lenger than simply removing strips
from the HOST printer. This delay was reduced consid-
erably by having the SMEs indicate which of the several
strips printed for a particular flight was, in fact, one thet
a controller would use for che fight. Nevertheless, it is
undeniable that controllers received strips more slowly
in the RFPS condition of this experiment than they
would when a dedicated printer is interfaced directly to
the HOST. Finelly, the translated RFPS was printed on
an EPSON laser printer and handed to a ZMP develop-
mental who placed the RFPSs in holders and delivered
the strips (i.<., A-side) for the simulations.

Dependent Measures

We collected several dependent measures during the
course of the experiment. The measures fall under 4
categories: Performance, Controller Behaviors, Work-
load, and Controller Opinion.

Table 1. Characteristics of ine 4 scenarios used in the experiment.

aE aE eee eS



wa

Francis P. Durso

‘Todd R. Truitt
Carla A. Hackworth
Chris A. Albright
M. Kathryn Bieckley

Mixed Environments

University of Oklahoma

Norman, CK 73019

Oklahoma City, OK 73125

Cctober 1998

inal Report

This document is avail

through the National Technical Info.
Service, Springfield, Virginia 22161.

lable to the pul



TTT LT

ramets pnnoraiaen

query of the controliers of their effort using a standard
instrument. The other measure was a more complex, less
direct, but more unobtrusive measure of workload.

TEX. We used 2 modified version of the NASA TLX
form. The NASA TLX (Hert & Straveland, 1988) is an
instrument that allows the assessment of several dimen-
sions of the workload construct: mental demand, tempo-
ral demand, physical demand, effort, frustration, and
performance. Subjects placed an “x” on a line ranging
from “low” to “high” to indicate their perception of the
work.oad when conirolling traffic with FPSs and RFPSs.

Granting requests. As part of their job, controllers are
often asked to respond to requests made by pilots, other
sectors, and other facilities. Aichough the frequency with
which controllers grant such requests and the time it
cakes them to grant these requests clearly involves more
than workload (e.g., the particular traffic situation,
feasibility of the request), it is clear that workload would
be one factor influencing the frequency and speed with
which requests are granted. Thus, communication be-
zween the controllers and the pilots, other sectors, and
facilities were recorded. Requests of the controller were
identified and the number of those granted and the time
zeken to grant them were analyzed.

Controller Opinion

Finally, che opinions of the experts employed to
operate any new system are a vital part of its assessment.
‘Thus, we interviewed each controller to determine over-
all opinions of the 2 strip formats. We queried them
using continuous-line scales to determine how well they
liked the formats, how easy they were to use, how useful
they were, the quality of information, their perceived
efficiency in controlling traffic, and their perceived
efficiency in board management. Subjects placed an “x”
ona9.6cm line to indicate theiz opinion on each of the
scales. We also included a number of open-ended ques-
tions designed to explore their particular concerns, if
any, with the reduced sirip, including their opinion asto
how it could be improved.

Procedure

All reams of controllers were sested during a 2-week
petied in January 1995. After giving informed consent
and completing a short biographical questionnaire, the

team decided on who would serve as R-side. The partici-
pants then took their respective places and were allowed
to view and interact with the system and the strips for up
0 5 minutes before the problem started. During that 5-
minute period, che participants received a short briefing
from the SME from that area. The briefing included

idiosyncrasies of control ing simulated air traffic on the
DYSIM and an overview of current craffic parameters,
including “hot” aisspaces, ILS approaches, VOR prob-
ms, and so on. The first of the two 50-minute scenarios
then began.

During the scenaric, an A-side controller placed
FPSs {or RFPSs) in the strip holders. The primary
specialist stood over the shoulder of the D-side partici-
pant and the other SME stood over the shoulder of the
R-side. Hach SME completed an OLEE for the control-
ler they were observing. Behind and farther to the sides
stood the 2 observers responsible for recording R-si
and D-side activity. An observer monitoring our audio-
s of interfacilicy communications was plugged-
in overhead and sat behind the SMEs. During the
reduced strip conditions, a pair of experimenters sat off
to the side next to u.2 HOST printer chat produced the
EPSs. PPSs were removed from the printer and translated
into REPSs.

At the end of 50 minutes, the controllers completed
the modified TLX instrument and were given a 20 to 25-
minute break while che second scenario was set up.
During this time, the SMEs completed the post-scenario
instrument. When the participants returned, the proce-
dure was repeated for the second scenario.

At the end of the second scenario, participants
were separated and interviewed about their experiences
and opinions, using the post-experimental question-
naire. The participants were then thanked and asked not
to discuss the details of the study with other controllers
until the study was completed.

Results

For the analyses in this report, we chose to compare
the FPS condit on with the RFPS condition wu:
corrected Student t-zests at a test-wise alpha of 05. We
made this decision not only because of the temporal
constraints placed on the production of this report, but
also because such a statistical strategy will be liberal in
rejecting the null hypothesis, a bias not without its
benefits when exploring for potential differences.

ng UR-



FPR, if che controller cannot easily access needed
nformation from the strips, he or she may choose
instead, to request che information from the com-
puter vie the system’s quick action keyboard (QAK).
The QAK can be used to deliver the flight plan to a
smali screen, the computer readout display (CRD).
Such a behavior could occur for either the R-side or
the D-side, and thus we kept separate tallies of these
QAK activities for the wo team members.

Time on strip bay. Vf it is difficult to see or find
information on the smaller strips, the controller may
spend more time viewing the strip bay. For the R-side
controller, this of course means that he or she is
spending less time monitoring the PVD. Time-on-
trip-bay for the D-side is, on the other hand, not a
compensatory behavior. Thus, we recorded this be-
havior for the R-side only.

Time in aural R—D communication. Asmaller strip
may be a less valuable aid in communicating between
the two members of the controller team. This deficit
could manifest itself in more R—D aural communi-
cation. For example, the R-side may not be able to
access the information from the RFPS quickly and
subsequently, may verbally request assistance from
the D-side. Thus, for each scenario, we recorded the
cumulative amount of time that the team members
spoke to each other.

Pointing. lf a smaller strip is difficult to locate by
one controller, it may require the other controller to
point to it. This form of nonverbal communication is
quite common, but sharp increases in its occurrence
could be indicative of a problem in searching the strip
bay and detecting the target.

Frings. J-rings are multisided polygons that the
controller can place around 2 fight displayed on the
PVD. These rings aid the controller in discerning
lateral separation between Alights that are at the same
altitude. Disproportionate use of J-rings in one strip
condition could suggest chat che controler is push
separation standards more in that condition than in

the other.

Route displays. Another compensatory operation
availabie co the controller is displaying the route of a
flight across the PVD. Again, if smaller strips make
discerning routes more difficult, we may see controllers

Soe

choosing te obzain route information by req
route display. We recorded the number
displays requested, tegardiess of which con

guested them.

Coast tracks. Por flights not under radar control, the
controller can initiete 2 Coast track on the PVD, Such
ing to the filed Sight plan or
the controller can e data blocks manually. A
controller may choose to ate more Coast tracks
when insufficient information is available fom the
strips. We recorded the number of Coast tracks dis-
played for each scenario.

tracks can move accor

ve

Controller queries. If the controller cannot easily
retrieve the information frem the strip, of if the strip
proves to be an unsuitable medium to record impos-
tant information, the controller may simply choese to
query the pilot about the missing deta. We recorded
all radio and telephone communications during the

scenarios and subsequently tallied the number of such
queries made by the R-side and D-side controllers.

Writing. Finally, we recorded the number of times
hat a controller, either R-side or D-side, wrote on the
sttips. Writing on the strips is a central part of board
management, and as we have shown previously, an
integral part of the behavioral sequences that charac-
terize ATC (Edwards, Fuller, Vortac, & Manning, in
press; Vortac, Edwards, Jones, Manning, & Rotter,
1993; Vortac, Edwards, & Manning, 1994). How-
ever, the effect of smalier strips on the frequency of
writing, while important, is not easily predicted. One
possibility is that, because of the absence of informa-
tion, controllers may choose to write more on the
smailer strips. On the other hand, because the strips
are physically much smalier and have less room on
which to write, the controilers may choose to neglect
oF postpone strip marking that they might otherwise
perform. Thus, although the frequency of writing is
clearly important and worth observing, this measure
alone will not be indicative of the viabiliry of the
smaller strips.

Workload

it mey be the case chat comparable performance can
be maintained by the controllers, bur at 2 cost in the
amount of effort the controller is required to expend. We
assessed workload in 2 ways. One was a relatively direct



Participants

Twenty pairs of controllers participated in che study.
Thirty-nine were full performance level (FPL} controi-
jess and one was a developmental (checked-out for the
sector). Mean time asan FPL et ZMP was 7.43 years, and
ranged from 0 to 21.0 years. The controllers had worked
in their current area for 8 years (range .58 - 13.25)
teams except one comprised controllers who had worked
together operationally as Radar-side (R-side) and Data-
side (D-side) teams orior to the experiment. For the
experiment, one member of each team served as R-side
and one served as D-side for both scenarios. The partici-
pants decided who would perform which duties.

Airspace

Two sectors from ZMP were used in the study (Sector
1, Area 1; Sector 5, Area 2). Both sectors have substantial
amounts of nonradar traffic, in addition to flights under
radar control. The sectors were chosen in consultation
with our subject matter experss (SMEs}, one from Area
Land one fom Area 2.

Ten teame were from ZMP’s Area-1; for the purpose
of this experiment, they controlled simulated treffic in
the Pellston (Sector 1} sector.

The official sector 1 description

Sector 01 works all performances of aircraft from the
ground to 12,000 feet MSL. At Peliston (PLN) it is nonradar
5,000 feet and below. South of PLN this increases to 7000
Feet MSE and below, and to the north of PLN this increases
to 10,000 feet MSL and beiow. The sector becomes active in
the summer months with 2 large amount of recreational
wrefic. The complexity of this sectox becomes evident with the
numerous overlapping approaches coupled with the nonradar
areas. Controllers work with two Approach Controls, Saginaw
(MBS) end Collins (APN). MBS is ARTS equipped for
automated hand-off and Aight plans are passed via computer.
APN requires manual coordinations of fight plansand manual
hand-offs. Sector 01 is bordered on the north and east by
Toronte Center (°¥Z), to which (the controller} must coor-
dinate manually both for Might plens and radar hand-off.
Sector Of is bordered on the south by Cleveland Center, Flint
Sector. Sector 01 is bordered on the west by Minneapolis
Center sectors 02 and 03. (Taken from the facility's sector
narrative, ZMP7220.!).

Ten teams were from ZMP’s Area-2; for the
pose of this experiment, they controlled simulated
traffic in the Eau Claire (Sector 5) sec.or.

The official sector 5 description

Sector 05 is a low eititude sector (6 to 23,009) involved
in handling a broad array of traffic, Though generally small in
area, sector 05 is a fast moving sector. The two airports
genezating the most trafic ere Er. Claire (EAU) and ia
Crosse (LSE). EAU has 4 IER approaches. The 8C Rwy 4and
ILS Rwy 22 are the most commonly used and vectors cen be
provided for the final approach course at the BC Rwy 4. Radar
coverage west of EAU is generally 3000 to 4000 fee: MSL,
while east of the VOR coverage below 6000 fect is marginal at
best. Thirty to 40 miles southeast of EAU coverage does not
exist below 8000 feet. At LSE, there i e WFR control
tower. The ILS Rwy 19, VOR Rwy 13, VOR Rwy 36
approaches are most commonly used. Radar vectors are pro-
vided to the VOR Rwy 13 and VOR Rwy 36 final approach
course while radar coverage north of LSE makes vectoring for
the ILS Rwy 18 approach very difficult. Coverage below 5000
feet north of LSE is very limited. In addition, approach
services are provided for numerous small airports scattered
throughout che sector. Sector 05 works closely with Rochester
(RST) and Waterloo (ALO) epproach controls located in the
scuthwest corner of the sector. RST Approach owns 8006 feet
and down while ALO owns 10,000 fee and down. A part-
time GCA unit operates at VOK field just east of the sector 05
boundary. They own 10,000 feet and down. There is also 2
part-time VFR contre! tower located at the Fort McCoy
{CMY) airport. Two restricted areas end 2 MOA’s are used
within the sector. The traffic flow is generally east to west
consisting of all types of aircraft. Sector 05 remains open ali of
the time. All other sectors within the area combine at Sector
05. (Taken from the facilicy’s sector narrative, ZMP7220.1).

Design

Each team controlled air traffic during two scenarios,
once using full FPSs and once sing the 1” x 5” RFPSs.
Order of the scenarios, order of the strip-condition (FPS
or REPS), and assignment of strip condition to scenario
were counterbalanced across the teams, such that each
scenario appeared equally often as first or second, each
type of strip was used equally often first or second, and
each scenario was controlled using each type of strip
equaily often.

Scenarios

Scenarios were constructed and tested by the area
SME. Two 50-minute scenarios, comparable in diffi-
culty end sealism, were constructed for each area. The
scenarios were intentionally designed to exercise the use
of the flight strips, sometimes including situations expe-
rienced in the field, but at a somewhat higher rate. For
example, departures from small airports make use of the
flight progress strips, but their gypical occurrence in the



Special
Siraj Fetcher, and Doug Mochie.

thanks to: Mike Day, Greg Henderson,

Thanks to our controller volunteers. Area 1:
Fitas, Scott Purvis, Cassie Bunn, Mark
i, Tony Nobach, Doug Rarfield, Jerry

M Carthy, Wayne Goldberg, Steve Weidner,
Peter Brandt, Denny Hunsinger, Terry Pitts, Bill

Szydio, Drew Gaydos, Tom Kloos, Pat Taylor,
Mark Mullane, Karen Ziboiski, Dave Leske, and
Bruce Steeples. Area 2: Craig Boehne, Robin
Thompson, Ethan Piche, Jeanne Huggins, Deron
Hoffmeier, George Skuicalo, Kraig Zibolski, Neil

Wagner, Mike Wehrman, Joe Konietzko, Cathy
Keller, Barb He’

Bolhouse, P.

erson, Mike Benson, Kevin

on, Sasha

selburg, Doug

Goff, Brad Adams, and Jeff

Johnson, Chri
Germain. Remotes: Ken Kamrath, Al Matakovich,
Brad Wahlberg, Rodney Carey, and Cathy Regan.

Dave Hinze, Tom

Support Personne!
Monroe, and Juanita Baldwin. A-Side: Paul
Sickman, Chuck Connley, and Dave Baird.

Azea Supervisors: Dan Callahan, Russell Burks,
and Dave Peterson.

Strip Bay Construction: Hugh Manning.



Table 6. Two measures of workioad. Direct TLX
repors and indirect frequency and delay of

Kent 75.69)
Physical di demand H 58.00
(Temporal demand L 64,75)
fort Hl 87.20)

51.45)

48.40)
(Mental demand 69.55) 76.85;
{Physical demand 53,70 82.30
{Temporal demand 64.95 89.29
lEfort $2.70) 1.55)
Frustration 48,95} 1.45)
Performance 55.80} 48.05
GRANTING REQUESTS i _
Percent Granted 04.99" 5.40)
Time to grant 6.271 38.96)
Bold =p <.05

i eo scan (22)

route {

route {

ude € (2),

Jer to scan th

information recrieval concern

cally, a good. snerhod for
rai control
g; CID seemed especially ai

ther than color.

information, and 8

fected oy cppeari g in shading, © rai
rend 3 comments revolved a

enough room on the , reduce d FP
ents about board m

less (1) iB ead Dp
), hare to remove

ng Q), cannot see when R -side’s
ve percent of the contr
gsand 10% werev

7s stated they

able

[Ease of Use 2.45) 44.02

Usefuiness 3.20) 16.38) seme (48%), with 12% of d

Like Tes Baa, controllers not performing any

Quality of information 12.28 5.04 3.84) resequencing bone

Efficiency of control | 8.93) 4.23) 727 hen askec what ‘ror:

lEfficiency of board management | 8.73) 3.53 6.6, ‘lon they woula remove from
' the reduced FPS, 70% said noth-

Bold =p <.05

found nothing. Negative comments were: hard to see,
had to write smailer, fess room, better chan no strip at
all, good excuse for not using them. Positive comments
ficin the bay, fess time to sequence, less space/room
ele

aa

wer
used, didn’t have to reach as fas, convenient fer
handed person, eye-level, less clutter, less paper.

When asked how reduced strips impaired perfor-
£ responses fell into comments

mance, the majority
about information retrieval and about board manage-
ment. nts about i ieval

Com inform:

ing was suggested
2 one controll
When asked what information
they would add to the reduced FPS, 48% said nothing.
ents were: color (13), previous/next fix (8),
fications {4}, nonrad: dicator (2), size of
sure arrow (3), move beacon code (2), larger
evious fix/next fix information was men-

ing. Only sha
by

more thai

alticude (2).
tioned by a sign’

ority. These concerns often
wansiting from Canada; for such
© appears nowhere else on
yas/ next

dealt with fights
es ig
Rights, information about t

rev’ xX Was removed,

ights had no temporal information.

the strips. Thus, when p
strips for Canadian fi



AT

sens

sca

ATT

ern was in regard to Sights transiting from Cana-
dian airspace. For these flights, important time infor-
mation is routinely available only in the fields used for
fix postings. Thus, while display characteristics should
undoubtedly be changed, this study also suggests that, at
ieast in some cases, valuable information was removed.

‘The problems with the smailer size tested here were
both in terms of finding relevant information quickly
and with writing on rhe strip. Obviously, a strip larger
than the test version would alleviate many of the
problems found here, bur it is unclear what size such
a strip should be, short of using the current FPS. A
larger strip would allow more spece for writing and,
together with optimally formatted information, may
be sufficient. This might be especially true ence the
coniroller gains some experience with smaller strips;
the controllers in this study obviously had had no
prior experience with the 1” x 5” strip. ,

It is also apparent that the reduction will have its
greatest impact on nonradar fights. In fact, other
work by this research team has suggested that in
sectors under radar control, strip marking might be
eliminated completely, and the strip size and informa-
tion reduced substantially. However, the current study
suggests that this is not the case with nonradar flights.
Thus, one solution involves a change in procedures:
Required strip marking could be eliminated and con-
troliers could be given the option to retain only those
strips needed to control nonrada: flights. This proce-
dural change could also be combined with a smaller
strip, although it is unclear from this study what the
size and characteristics of the reduced strip should be.
‘This study does indicate, however, that a 1” x 5” strip
with the characteristics described in this report will be
unlikely to allow controllers te perform their job at
the current levels of efficiency and workload.

References

Albright, C.A.. » Truiee, T R., Barile, A.L., Vortac, O.U.,
& Manning, CA {1 594). How air traffic cont:
lers compensate for the lack of strips. Querteri
Journal of fAir Ti vaffic Control, 2, 229-48,

Edwards, M.B., Fuller, D.K., Vortac, O.U., & Man-

ning, C.A. (1995). The role of flight progress
ips in en route 2ir traffic control: A time series
. International journal of Human-Compuzer
5, £3, 1-13.

Hart, 8.G., & Straveland, LE. (1988). Development of
NASA-TLX (Task Load Index): Results ofempiri-
cal and theoretica! research. In P.A. Hancock and
N. Meshketi (Eds.}, Human Mental Workload (pp.
139-83). Amsterdam: North-Holland.

Vortac, O.U., Barile, ALL., Albright, C.A., Truiet, T.R.,
Manning, C.A., & Bain, D. (1996). Automation
of Sight data in air traffic control. In D. Herman,
C. McEvoy, C. Hertog, & P. Hertel (Eds.), Basie
and Apylied Memory Research, Nehweh, NJ:
Lawrence Erlbaum Associates.

Vortac, O.U., Edwards, M.B., Fuller, D.K., & Man-
sing, CA CL 993). Autromazion and cognii fon in
air traffic contrci: An empirical investigation. a
plied Cognitive Pychology, 7,631-51.

Vortac, O.U., Edwards, M.B., Jones, J.P., Manning,
CA, & Rotter, AJ. (2993). En route air traffic
controllers use of fight progress strips: A graph-
theoretic analysis. International Journal of Aviation
Paychology, 3, 327-43.

Vortac, O.U., Edwards, M.B., & Manning, C.A. (1994).
Sequences of actions for individuals and teams of

air traffic controllers. Human-Competer Fnterac-

tion, 9, 319-43.

13



3. Repo:
i Octobe: 1998
5 Perfonniag Organi

1 &. Pesforming Organization Report No.

fackworth, C.A., Albright, CA. Bi

me and Address 10, Work Unit No. (TRAIS)

niversity of Oklahoma i 2BAA Civil Aeromedical Institute |
Department of Psychology | P.O. Box 25082 :

‘or Grant No.

Nor:

n, OK 73019 { Oklahoma City, OK 73125 j M. Contract

43. Type of Report and Period Covered

72. Sponsoriag Agency neme and Address

Office ion Medicine
Federal Aviation Administration
800 independence Ave., S.W.
Washingzon, D.C. 20591

|
|

Sponsoring Agency Code

Supp
This research was supported by Contract #DTFA-02-93-D-93688, Task Order 9478926

36, Abstract

Currently, en route control of high altitude flights between airports uses computer-augmented radar information available on the
Plan View Display (PVD), Computer Readout Device (CRD), and Right information prinzed on Flight Progress Strips (FPSs}. The
FPS contains thirty-one fields that suppl lement data available on the PVD. While an aircraft is in 2 conti roller's sector, control
instructions, changes to the flight plan, and otier contacts with the aircraft are written on the corresponding strip. This report
describes en experiment that compared the effects of using a standaré- zed (1 5/16” x 8”) FPS end an FPS reduced both in size (x
5”) and information on the performance and workload of controller teams. The teams, from Minneapolis ARTCC, controlled
simulated air waffic in a mixed radar-nonradar environment. Overall, the 1” x 5° reduced strip yielded deficits in the contro! of
nonradar flights but not radar flights. This was evidenced in subject matter experts’ evaluation of nonradar separati
processing ar and board management, and, toa marginal © xtent, in the efficiency of traffic movement through the sector.
side (R-side) controlier’s awareness was also rated lower when using the smaller strips. Interestingly, the controllers’ o
their own performance did not reflect 2 difference between smaller and norme!-sized strips. This may help explain why controllers
did not compensate for the smailer strips to any great extent. Only R-side controllers ext $ bet
increased workload. R-side controllers also pointed to the P
Controllers thought workload was greater with smaller strips. R-si
working with the 1” x 5” strips. Despite the self-reported heavier
tasks, such as granting pilot requests, as often and as oe us!

oad, controllers neverd
ig, smaile:

neless were able to perform secondary |
strips as they did using standard strips. This study aiso
5” Fi rio marking, speed of strip processing,
evidenced in the on-li fe expert
trip
‘ations an

described specific air traffic ac ivities likely to be affected by 2 reduction to a 1” x 3 FP.

an

some aspects of board m: management seems ast especially

and find the information on it seems to safle: wit ion in i . line expert ev
controller opinions echoed this problem
‘erences in self-reported frustration. Controllers also nozed spe
replace information typically ¢ presented in red. Of board manage ities, considered by controllers as generally inferior
with 1” x 5” strips, removai of deadwood seems less likely to be negatively affected by reduction in strip size. The on-fine expert |
evaluation rated the 1” x 5” strips negatively and the subject matter experts recorded more negative comments about removel of |
deadwood under that condition. Overail, this study does not permit recommendation of the 1”x 5” reduced stcip as designed for this
study. Suggestions for improving 2 less than standard size FPS are provided.

17. Key Words P18, Dis
Conirol, Flight Progress Strips, Workload | Document is available to the pub!

Locating strips might bave been Si iily difficult & for the R-side, thus leading to large
y; including the use of shading to

tribution Statement

i National Technical Tnform ion Sei H
| Springfield, Virginia 22161 i
19. Security Clessif. (of this report) 2 20. Securky Classif. (ef this page) } No.0: ages } 22. Price H
ae H : : i
Unclassified i inclassified ig H |
Form DOT F 2700.7 (8-72) Reproduction of ed page auth



NOTICE

s document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of

information exchange. The United States Government

+}

assumes no liability for the contents thereof.



Performance strip resulted
aments about the
© the D-side, the sm

an increased ni

er of nege

-side contro: areness of

struments to assess perfc
ve OLEE form and the Rem:
tions form. The OLEE form contained overall ratings — butelso the ability to scan in!
from the SMEs and 2 count of the number of negative Finally, the D-side’s de:
occurrences or comments by the SMEs. Overall, the
frequency of negative comments was 2 more sen
instrument than the simple ratings of “satisfactory,”
“needs improvement,” and “unsatisfactory.”

in locating

marking ¢ssen-
ip, and removing strips for
2 control. Interestingly, radar
ntly unaffected and there was no
nunication between

OLEE Analysis

formation on the
Although ratings tended to be less sensitive to longer
differences between FPS and RFPS than were fre- rol was appar
quency counts, the Z sets of date were quite clear in indication thec nonverbal co:
negative the members of the pair was hindered.

their overall agreement that tae RFPS:

effects. The mean ratings (Satisfactory = 1; Needs

improvement = 2; Unsatisfactory = 3) and the fre- — Remaining Actions

quency of mistakes noted by the SMEs are shown in

Tables 22 (R-side) and 2b (D-side). Results from the Remaining Actions form showed
Both R-side and D-side evaluations were poorer for all nce between FPSs and RFPSs for the

the RFPS than the FPS condition in the separation of der < eration, or

nenradar (but not radar) flights, in effective strip for the number of proposals zemai: he bay (See

marking essential for control, and in locating the Table 3). We then used the remaining actions date

strips within the bay. In addition, using the 1° x 5° from the post-scenario analysis to analyze sunder

HRedar
INonradar 4.90 2.30) ns| |Nonrader
USTRIP MARKING: pel SRIB MARKING res i i |
‘Essential for control 2.35) 2.75| 2.18) |Essential 41.25] 16.50] 2.6
Nonesse' itiai 1.80 2.15}. 2.30! iNonessential 7.3 9.45) ns)
» SuruP PROCESSING i | STRIP FROCESSING SPEED |
co 3 SPEED ce i Be et t
Locating 4.20) 2.30| 5.77! [Locating 3.45) 7.65
Strip scan F400; 4.05) ns! [Strip sean S| 20!
B-D COMMUNICATION i i po be RD GOMMUNICATION © | i
Nonverbal communication | 1.00 4.05) iNonverbal communicatt: 85} 50!
boot SAWARENESS ie AN |
Awareness 7.15  7.40[  7.75| |Awereness 25) 1.80
BOARD MANAGEMENT. - | 2 SBOARD MANAGEMENT | i
lRemoval of deadwood 4.25) 1.49) ns] (Removal of deadwood $85} 4.30 ns!
Etfective sequencing 4.00] 1.10) ns} | i 30, 80 ng!
Ralics =p <.40

bold = p <.0



°

i When asked <0 identify circumstances that needed
normal strips, controllers indicated:
changes (10), approach/departure indicators (10),
nontadar environment (8), prepla: g (8), holding
m, scanning (6), none (5), vectoring arcs (4), when
busy (4), and resolving conflicts (3). Finally, we asked
controllers if they could safely controi more, less, or
ic with the reduced strips.
None projected an ability to contro! more traffic and
60% believed they would be able to conzrol less.

Conclusions and Recommendations

We compared the performance and workload for
controllers controlling air traffic in a mixed radar-
nonzadar environment using the standard sized FPS
(1 5/16” x 8”) and an FPS reduced in both size (1” x 5”}
and the amount of information it presented. The
results from 20 teams of field controllers from Minne-
apolis center are summarized in this report. Overall,
the 1” x 5” strip yielded deficits in the contro! of
nonradar flights, but not in the control of radar
flights. This was evidenced in the subject matter
experts’ evaluation of nonradar separation, strip pro-
cessing and board management, and, to a marginal
extent, in the efficiency with which traffic was moved
through the sector according to the remaining actions
analysis. Furthermore, the R-side controllers’ aware-
ness was rated lower when using the smaller strips.
Because the problems occurred only with nonradar
flights, such measures as conflict or MSAW alerts of
the number of J-rings used would not be expected to,
and did not, vary with condition.

Interestingly, the controllers’ evaluation of their
own performance, as indicated on the TLX, did not
reflect this difference between reduced and normal
strips. This may help explain why controilers-did not
compensate for the smaller strips to any great extent.
In order to compensate for the reduced strips, control-
lers made only one reliable change in their behavior:
The R-side pointed to the PVD more often. This
behavior was not because the R-side pointed to the
small strips less often. Perhaps the small strips eng
dezéd more nonverbal communication overall, but
this nonverbal communication was not the type thar
alerced our subject matter experts co any problem in
intra-ream communication.

Although chere was little compensatory activity,
the R-side controller, but not the D-side controller,

fele che workload was greater with the 1”x 5” strips. In

rt, the R-side controller felt

effortful and moze Suserating working with the smail
strips. Despite being under a perceived heavier work-
ne controllers nevertheless were abiets perform

it was more

load,
secondary tasks, like granting pilot : requests, as often
and as quickly using small strips as they using

standard strips.

This study also supplied information about the
specific air traffic operations likely to be affected by 2
reduction to a 1” x 5” FPS. Strip marking, strip
processing, and some aspects of board management
seemed especially affected. Inferior strip marking was
evidenced in the OLEE analysis and the controllers often
reported that the size of the strip prevented writing.

The ability xo locate a particular strip and to find
the information on the strip once it was found seems
to suffer with strip reduction. The LEE and the
controller opinions echoed this problem. Locating
strips might have been especially difficult for the R-
side, thus leading to the large difference in perceived
frustration. The controllers also noted specific prob-
lems with the strip Aisplay, including the use of
shading to replace information that is typically pre-
senzed in red.

Of board management responsibilicies—
considered by the controllers as generally inferior
wich small strips — removal of deadwood seems to be
most likely to be affected by strip reduction. 7
OLEE evaluation rated the 1” x 5” strips x
and the subject matter experts recorded more negative
comments about removal of deadwood under chat
condition. Because few controllers compiained about
handling the small strip holders, the deficiency is
likely co lie in an inability to locate those strips ready
for removal.

Overall, it seems clear that the 1” x 5” strip tested
here was inferior to the current FPS, especialiy in the
control of nonradar flights. Thus, based on these data,
we cannot recommend that the i” x 5” strip, as
designed for this study, be considered as a design
option. This, of course, does not mean that the cur-
rent FPS is the only method of presenting flight data.
A smaller strip may be feasible. Toward this end, ic
seems clear chat some method of highlighting critical
information, other than shading, should be used.
Although several controllers offered suggestions about
the display characteristics of the reduced strip, there
was less concern about the substantive information

removed. However, previous/next fix information
was requested by a significant mino:

negative:y