DOT/FAA/AM-00/5

U.S. Department

of Transpor tation

Federal Aviation

Administration

Todd R. Truitt

Francis T. Durso

Jerry M. Crutchfield

Peter Moertl

University of Oklahoma

Norman, Oklahoma 73019

Carol A. Manning

Civil Aeromedical Institute

Federal Aviation Administration

Oklahoma City, Oklahoma 73125

February 2000

Final Report

This document is available to the public

through the National Technical Information

Service, Springfield, Virginia 22161.

Reduced Posting and Marking

Of Flight Progress Strips for

En Route Air Traffic Control Office of Aviation Medicine

Washington, D.C. 20591
NOTICE

This document is disseminated under the sponsorship of

the U.S. Department of Transportation in the interest of

information exchange. The United States Government

assumes no liability for the contents thereof.
i

Technical Report Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.

DOT/FAA/AM-00/5

4. Title and Subtitle 5. Report Date

Reduced Posting and Marking of Flight Progress Strips for En Route Air

Traffic Control

February 2000

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

Truitt, T.R.1, Durso, F.T.1, Crutchfield, J.M.1, Moertl, P.1, and

Manning, C.A.2

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

11. Contract or Grant No.

1

University of Oklahoma, Norman, OK 73019

2

FAA Civil Aeromedical Institute, P.O. Box 25082, Oklahoma City, OK 73125

96-G-040

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered

Office of Aviation Medicine

Federal Aviation Administration 14. Sponsoring Agency Code

800 Independence Ave., S.W.

Washington, DC 20591

15. Supplemental Notes

Work was accomplished under approved task HRR-502.

16. Abstract

The new Display System Replacement (DSR) being implemented in air route traffic control centers (ARTCCs)

will allow the data-side controller less room to post Flight Progress Strips (FPSs). We tested a new FPS marking

and posting procedure designed to reduce the controller’s need for, or reliance on, the FPS. The experiment was

conducted at Cleveland (ZOB) and Jacksonville (ZJX) ARTCCs utilizing individual controllers and controller

teams operating in either high- or low-altitude sectors. Each controller ran two, 30-minute scenarios. Scenarios

were counterbalanced, but sample sizes did not allow counterbalancing of conditions. In the Normal scenario,

controllers worked as they normally would. During the Experimental scenario, controllers were asked to remove

FPSs that were not needed after radar contact and communications were established. Also, FPS marking was not

required for any information that was recorded elsewhere, such as via computer entry or landline

communication. Controllers responded to the Workload Assessment Keypad (WAK) every 5 minutes while a

subject matter expert made performance ratings. Experimenters recorded activity relevant to the plan view

display, computer readout device, and FPSs. At the end of each scenario, controllers provided a position relief

briefing and completed a modified version of the NASA Task Load Index. For individuals and teams at ZOB

and ZJX, results showed that controllers posted fewer FPSs and marked them less often in the experimental

procedure. No detrimental effects on performance, workload, position relief briefings, or team communications

were observed. On-line measures of workload (i.e., the WAK) were comparable and sometimes lower for the

experimental condition. Most controllers reported that they preferred the experimental procedure.

17. Key Words 18. Distribution Statement

Air Traffic Control, Simulation, Field Study, Flight

Progress Strips, Performance Measurement

Document is available to the public through the

National Technical Information Service,

Springfield, Virginia 22161

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price

Unclassified Unclassified 30

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized

1

REDUCED POSTING AND MARKING OF FLIGHT PROGRESS STRIPS

FOR EN ROUTE AIR TRAFFIC CONTROL

Currently, en route air traffic controllers use paper

flight progress strips (FPS) in the constant task of

providing safe and efficient radar service. The controller uses FPS to obtain information about a flight

and to record changes in flight parameters such as

route, speed, or altitude. However, much of the

information on the FPS can be obtained elsewhere,

such as the computer readout device (CRD). Much

of the information recorded by the controller on the

FPS is also recorded elsewhere, such as the host

computer system or ground-to-air audio tapes. Although redundancy can be very beneficial, particularly when a system fails, redundancy in highly reliable

systems may create additional task requirements in

terms of workload or cognitive processing for the

operator. One way to increase task requirements by

redundancy is the execution of unnecessary procedures. For example, writing down information that

has already been entered into a computer system may

increase task requirements. Another way to increase

task requirements is by displaying duplicate or extraneous information that must be searched in order to

find the needed information.

The purpose of the current experiment was to

examine the performance and workload effects of

removing some of the redundant behaviors associated with the FPS that are required for en route air

traffic control (ATC), namely, FPS posting and marking. The outcome is of interest due to impending

replacement of workstations used by en route controllers. The old, vacuum tube dependent workstation, or M-1 console, is being replaced by the new,

more reliable Display System Replacement (DSR).

The DSR was designed to eliminate many of the

problems associated with the poor reliability of the

M-1. The DSR has much more computer power to

allow for future system upgrades, including the use of

color, and additional functions for the controller.

One important difference is the fact that the DSR

workstation provides less room for FPS management.

Therefore, it was reasonable to wonder whether the

benefits provided by the DSR display and its ability

to accommodate new electronic displays and tools

would be diminished by a restricted ability to use the

FPS. It is important to note that during the initial

transition, the DSR will simply replace the old M-1

console without adding any new features. If restricting controller interaction with FPS results in a deficit

(e.g., poorer performance, higher workload) while

using the M-1 console, then it is likely that a similar

deficit will remain during a transition to DSR.

A number of researchers have emphasized the

importance of active FPS usage (e.g., Hopkin, 1988,

1995; Stein, 1991; Stein & Bailey, 1994; Zingale,

Gromelski, & Stein, 1992). Hopkin specifically argued that active control procedures are necessary for

controllers to maintain a sufficient level of knowledge and situation awareness (SA) during the ATC

task. Hopkin emphasized the importance of physical

interaction such as resequencing or writing on the

FPS. Without such physical interaction, he argued,

controller memory, SA, and hence, overall performance would suffer. The views of Hopkin and others

rest on the ideas that memory encoding is important

and that it cannot sufficiently occur without such

meaningful physical activity. The importance of

memory and its relationship to ATC performance in

general, and particularly with regard to FPS usage, is

formally acknowledged in an Federal Aviation Administration publication entitled, The Controller

Memory Guide (Stein & Bailey, 1994). This memory

guide is distributed to controllers in all ATC environments to promote the importance of memory practices and to show how FPS usage can support a

possibly overtaxed or undertaxed cognitive system.

Field controllers from en route centers, terminal

radar control, and airport towers agreed that memory

is an important aspect of ATC and rated the memory

guide high in terms of relevance, realism, and overall

quality (Stein, 1991). Furthermore, interviews of

170 controllers throughout the United States indicated that the three memory aids used most by

controllers involve the FPS (Gromelski, Davidson, &

Stein, 1992). These often-used memory aids are FPS

management (arrangement of FPS), offset or tilted

FPS (indication that further action is needed), and

FPS marking (updating and confirmation of commands issued).
2

The importance of physical interaction with the

FPS for controller memory and performance has

received some empirical support from both basic and

applied research settings. For example, basic research

by Slamecka and Graf (1978) used undergraduates to

show the robustness of the “generation effect.” They

demonstrated that participants had better memory

for words that were generated by the participants

themselves as compared to words that were simply

read. The generation effect held under numerous

conditions including both recall and recognition memory

regardless of whether memory was cued or not.

In a more relevant setting, Zingale, Gromelski,

and Stein (1992) provided some support for the

importance of interaction with FPS. These researchers trained aviation students to use TRACON II, a

simplified, terminal radar ATC simulator. Participants were provided with an FPS for each aircraft in

the simulation. Each participant controlled traffic in

both Writing and No-Writing conditions in which

they either could or could not, respectively, record

control actions on the FPS. Results showed that more

previous control actions were remembered in the

Writing condition than in the No-Writing condition. However, a repeated-measures design was used

such that the No-Writing condition always preceded

the Writing condition. Therefore, it is not clear

whether memory differences were due to practice or

condition effects. Furthermore, the participants used

in this study were not air traffic controllers and thus

results provide only minimal support for the interaction position. No differences in memory for previously performed actions were found when the same

basic experiment was conducted using actual controllers (Zingale, Gromelski, Ahmed, & Stein, 1993).

Further evidence contradicting the hypotheses of the

interactionist view has been provided by studies that

demonstrate a lack of detrimental effects on performance, workload, or cognitive processing when controllers were limited in the amount of interaction they

had with the FPS or when the FPS were completely

removed (Albright, Truitt, Barile, Vortac, & Manning,

1994; Vortac, Edwards, Fuller, & Manning, 1994).

Vortac et al. observed controllers (FAA Academy Instructors) under both normal and restricted FPS conditions. The FPS were posted and visible during the

restricted condition, however, controllers could not

physically manipulate or write on the FPS. Controller

performance, including visual search and recall of flights

and flight data, was not impaired by the restricted

condition. In fact, controllers were more likely to remember to grant requests and did so sooner under the

restricted condition. Vortac et al. concluded that by

restricting interaction with the FPS, the ATC task was

changed such that controllers were now able to assume

a more strategic outlook. The elimination of FPS responsibilities resulted in a reduction in workload, or at

least a redistribution of workload, in regard to the FPS

and more cognitive resources could be directed towards

the task of aircraft separation.

The lack of evidence supporting a detrimental

effect due to restricted interaction with the FPS

suggests that whatever benefit controllers get from

physical interaction with the FPS may be insufficient

to result in a distinct memory or performance improvement. Because experts have better memory for

task-specific information than novices, these results

are in agreement with other research on expertise

(Chase & Simon, 1973; Ericsson & Kintsch, 1995;

Ericsson & Staszewski, 1989). Perhaps controllers

generate enough information, and hence strong

enough memory traces, through cognitive activity

required to perform the ATC task that simply writing

the information down or moving an FPS does not

provide much additional benefit in terms of memory

storage. Furthermore, controllers may not be remembering the verbatim information as written down or

seen on the plan view display (PVD). Instead, they

may be remembering more information regarding

the gist of a situation or command (Gronlund, Ohrt,

Dougherty, Perry & Manning, 1998). Gronlund et

al. also demonstrated that memory for aircraft information was not improved by the level of interaction

with a pilot, but by the importance of the aircraft.

Therefore, the value of FPS interaction may not be in

its benefits to memory storage, but in its ability to aid

retrieval or verification of information that could not

otherwise be quickly retrieved from memory.

If the FPS serve only as aids to retrieval, then limiting

interaction with the FPS, either partially or completely,

should interfere with controller memory and performance to the extent that an alternate means of obtaining

information is unavailable. If the functions served by

the FPS are needed by controllers, then they should find

ways to compensate for the lack of FPS interaction.

Which alternate means controllers use to compensate,

for example, by writing information on a notepad or

retrieving more information from the computer system

via a flight plan readout, tells us more about the role of

the FPS in controller activity.
3

Albright, Truitt, Barile, Vortac, and Manning

(1995) previously examined the role of the FPS by

observing how controllers compensate for the absence of FPS. They observed full performance level,

en route controllers in a simulated, high altitude

sector during both a normal condition and a condition in which all FPS were removed. By removing all

FPS from the controllers and giving them a notepad

on which to write anything they wished, Albright et

al. were able to observe whether controllers could

compensate for the lack of FPS and - if so - how they

compensated. A subject matter expert (SME) evaluated controllers’ performance and controllers provided subjective workload ratings after each scenario.

Results showed no differences in performance or

perceived workload when the FPS were absent. Controllers compensated for the lack of FPS by performing more flight plan readouts on the computer system.

The flight plan readout provides the same basic

information as does an FPS. Although this means of

accessing flight information appeared to have slowed

the time it took controllers to grant a request when

FPS were absent, controllers spent significantly more

time watching the PVD and wrote very little on the

notepad. Given the results of Vortac et al. (1994) and

the way in which controllers compensated for the

absence of FPS in the Albright et al. study, it is

arguable that the primary function of the FPS is to

provide ready access to flight information and, in

terms of ultimate performance and memory, very

little benefit is provided by writing on the FPS.

The results of Vortac et al. (1994) and Albright et

al. (1995) suggest that, given the physical limitations

of the DSR and the space allotted for the FPS, it may

be practical to reduce the amount of writing on the

FPS and the number of FPS that must be posted.

Currently, en route controllers are required to post an

FPS for an aircraft from the time radar contact and

communications are established with that aircraft

until the controller instructs the aircraft to switch

radio frequencies and contact the next controller.

These requirements result in at least one FPS being

posted for each aircraft in a controller’s airspace.

Furthermore, many control actions must be recorded

on the FPS as per the FAA Air Traffic Controller

Handbook, 7110.65.

Reducing the requirements for FPS marking and

posting by making them optional to the controller

thus should not prevent the controller from achieving adequate performance while working within the

current specifications of the DSR. Reducing the FPS

requirements eliminates the redundant recording of

information and potentially results in fewer FPS that

the controller must post and search through in order

to find important information. However, the evidence in support of reduced FPS marking and posting is somewhat limited. The Vortac et al. (1994)

study was limited to individual controllers who were

instructors at the FAA Academy. Although Albright

et al. (1995) used field controllers, their results are

also limited in that they only observed individual

controller behavior in a single, high altitude sector.

In order to provide further support for the viability of

the reduction of FPS posting and marking, questions

similar to those asked by Vortac et al. and Albright et

al. must be addressed in a variety of both high and low

altitude sectors. Furthermore, results must be generalizable beyond the individual controller and the

impact of reduced FPS activities must be assessed for

the en route air traffic control team as well.

The current experiment was designed to answer

two basic questions: First, does providing controllers

with the option of posting and marking FPS result in

significantly fewer FPS posted at any given time?

Second, if the optional posting and marking of FPS

does result in fewer FPS being posted, what, if any,

are the effects on controller performance and

workload? To answer these questions, we compared

controllers under both Normal and Optional FPS

Marking/Posting conditions. During the Normal

condition, controllers had full use of the FPS and

they controlled traffic as they usually would. Under

the Optional FPS condition, with some exceptions,

controllers had to post and mark FPS only until radar

contact and communications were established and

accepted with an aircraft.

The procedures used in the Optional FPS condition were developed by The Strip Reduction Working Group, which met from November 4-6, 1997, in

Washington, D.C. The sole purpose of the meeting

was to identify ways to reduce en route controllers’

use of FPS and FPS marking in anticipation of the

DSR upgrade. The group was organized by FAA Air

Traffic Operations branch and included representatives from ATO-110, the Civil Aeromedical Institute, the National Air Traffic Control Association,

the ATC Supervisors Committee, and the University

of Oklahoma. The varied composition of the Strip

Reduction Working Group allowed for consideration of many possibilities for the reduction of FPS
4

activity. The group discussed and recommended

changes to the current FPS procedure as possible

ways to reduce FPS activity. The result of the group’s

effort was an alternative FPS procedure that is shown

in Appendix A. An important point of the revised

FPS procedure is that although it allows for a reduction in the posting and marking of FPS, such a

reduction is optional and would only be used if the

controller responsible for a sector decided to do so.

Before adjourning, the Strip Reduction Working

Group decided that an empirical study would be

appropriate to test the revised FPS procedure.

The experiment was conducted at two Air Route

Traffic Control Centers (ARTCC), Cleveland (ZOB)

and Jacksonville (ZJX)1

. Data on performance and

workload were collected from individual controllers

and controller teams operating in either high or low

altitude sectors.

Method

 Participants

A total of 48 full performance level (FPL) controllers volunteered to participate in the experiment

(ZOB=24, ZJX=24). Just prior to the experiment,

each controller read and signed an informed consent

statement and then completed a biographical questionnaire (shown in Appendix B). Mean responses to

the biographical questionnaire are shown in Table 1.

Materials and Equipment

Scenarios. All scenarios were selected from training

scenarios that had been developed previously by the

training department of each ARTCC. Once selected,

scenarios were altered if necessary to meet the complexity requirements of the experiment or to insure

the occurrence of particular events of interest. Sectors

used at ZOB were Hudson (high altitude) and

Lichtfield (low altitude). Sectors used at ZJX were

Brewton (high altitude) and Florence (low altitude).

General descriptions of these sectors are provided in

Appendix C. All scenarios were designed to be at least

30 minutes in length. For each ARTCC, two scenarios

were developed for each sector (high or low altitude)

and staffing condition (individual or team) for a total

of eight scenarios per ARTCC. All scenarios for

individual controllers had a complexity rating of

70% and scenarios for controller teams had a complexity rating of 100%. Complexity ratings were

based on the type of events that occurred during the

scenario.

Within the typical occurrences of each scenario, a

strip-critical event was identified in order to give the

SME an opportunity to evaluate how each event was

handled, especially when the related FPS had already

been removed from the board. Each strip-critical

event required the controller to make use of an FPS

and was yoked to the scenario in which it occurred.

Strip-critical events included, 1) providing holding

instructions, 2) a pilot requesting a route change, 3)

an aircraft flying at wrong altitude for direction of

flight, and 4) an aircraft requiring special handling

such as Air Force One.

Dynamic simulator (DYSIM) training facility. The

DYSIM is a high fidelity simulation facility that

closely resembles the real en route ATC environment.

Workstations are fully functional and landline communications are provided. Flight plans for each

aircraft in a scenario are pre-programmed, but controllers or simulation pilots may change any flight

plan during the simulation. Training specialists execute ATC instructions to simulate the roles of pilots

and other controllers during the simulation.

Workload Assessment Keypad (WAK). The WAK is a

computer-controlled, on-line subjective measure of

workload. The WAK is well adapted for use in the field

because it is relatively small, portable, and can collect

ratings from as many as four participants simultaneously. Based on the work of Stein (1985), the WAK

obtains a workload rating by auditorily and visually

prompting the participant. At an adjustable interval, the

WAK emits a high-pitched tone and its seven, numbered buttons illuminate. The participant then makes a

rating by pressing one of the buttons within a specified

amount of time. The WAK records each rating as well

as elapsed time from prompt to response.

1

 Additional data focusing on individual controllers in low altitude sectors were collected at Boston (ZBW). Because there was no effect of the

experimental manipulation at ZBW, and because these data focused on individuals in low altitude sectors only, these data are not presented here.
5

Task Load Index (TLX). A modified version of the

NASA TLX (Hart & Staveland, 1988) was used to

collect subjective ratings of taskload. The TLX contains six separate scales to assess mental demand,

physical demand, temporal demand, effort, frustration, and performance. Each scale was represented as

a 100mm line, anchored from low to high. The TLX

rating form, instructions, and scale descriptors are

shown in Appendix D. Controllers placed an “X” on

each scale after an experimenter described what the scale

was intended to measure.

Procedure

The procedure was the same for both ARTCCs

and is summarized in Table 2. All data were collected

in the DYSIM training facility of the respective

ARTCC. Once controllers arrived at the DYSIM,

they completed an informed consent statement and

biographical questionnaire. An experimenter then

provided instructions on how to use the WAK.

Once controllers understood the WAK instructions, data collection began with the first of two

scenarios. The first scenario was always the Normal

condition in which participants were asked to control

traffic as they normally would. Although condition

(Normal vs. Optional FPS) was not counterbalanced,

the two scenarios corresponding to group (individual

or team and high or low altitude sector) were

conterbalanced, such that each scenario appeared in

each condition an equal number of times. Participants were given a notepad to record anything they

wished. The scenario began with the SME providing

a position relief briefing and then the participant

took full control of the scenario for 30 minutes.

During the scenario, the WAK prompted the participant every 5 minutes for a workload rating (where

1 = very low workload and 7 = very high workload).

The SME used a behavioral event checklist (shown in

Appendix E) to record the occurrence of specific

events related to controller performance. The SME

also observed one strip-critical event (for example,

pilot requests route change) and noted if the participant effectively handled that event. For controller

teams, communication effectiveness was evaluated

by the SME using the form shown in Appendix F.

Two experimenters used microcassette recorders

to archive activity relevant to the plan view display

(PVD), computer readout display (CRD), and flight

progress strips (FPS). In addition, experimenters recorded the time at which each activity occurred. One

experimenter recorded the type of actions performed

by the controller involving the PVD and CRD and

the time these actions were performed. These actions

included request for information, use of a route

display2

, J-ring3, /04, and flight plan readout ( FPR).5

The second experimenter recorded the callsigns of

FPS and the times each was posted and removed. The

second experimenter also recorded D-side activities

regarding the PVD and D-side CRD when controller

teams were being observed. At the end of the first

scenario, the participant used the computer-based

(CRD) checklist to provide a position relief briefing

to the SME. Once completed, the SME evaluated the

quality of the briefing using the form shown in

Appendix G. The participant then completed the

TLX followed by a 15-minute break.

Participants returned to the DYSIM after the break.

Before starting the Optional FPS condition an experimenter reminded the participants how to use the

WAK. A representative of the National Air Traffic

Controllers Association (NATCA) then summarized

the proposed strip marking and posting procedure.

Under the Optional FPS condition, controllers were

instructed by the NATCA representative that they

were to post and mark FPS only until radar contact

and communications were established and accepted

with an aircraft. After that, strip posting and marking

became optional. Participants placed a check mark in

field 21-24 of a strip to indicate that optional posting

2

 A route display is indicated on the PVD as a line drawn from a selected aircraft’s current position to that aircraft’s position at some designated

time, from 1 to 99 minutes, into the future. The route display is based on flight plan information that is currently stored in the computer.

The route display is automatically shown whenever a route amendment is made to an aircraft’s flight plan via computer entry by the controller.

3

 A J-ring (also referred to as a distance reference indicator) is displayed on the PVD as an approximate circle that can be placed around selected

aircraft for means of determining horizontal separation. The J-ring is an approximate circle and has a parameter-defined number of sides. The

radius of the J-ring is typically 5 miles but this parameter can also be set by each ARTCC.

4

 A “slant zero”, /0, is an action performed by the controller via keyboard entry to shorten the length of a leader line (the line connecting

an aircraft position symbol with a datablock as displayed on the PVD). Many, but not all, controllers use the /0 entry as an indicator, or

reminder, that they have instructed an aircraft to switch radio frequencies. Because all aircraft are instructed to switch frequencies when

departing one sector of airspace and entering another, controllers who use the /0 entry performed it on virtually all aircraft departing their sector.
6

and marking could be used for that strip. However,

optional marking only applied to information that

was recorded elsewhere, for example, by computer

entry or voice recordings. The NATCA representative also instructed participants that information that

was not redundant had to be recorded on the FPS and

that an FPS could be removed from the board if it was

no longer needed. Participants were also instructed

that an FPS was required to be posted and marked in

special situations which included, 1) radar contact

would be lost, 2) an aircraft was transitioning from

radar to non-radar, 3) special handling was required,

4) non-radar flight, 5) an aircraft transitioning from

auto to non-auto mode, and 6) holding instructions

issued. The full text of the Optional FPS usage

procedure is shown in Appendix A. Finally, participants were encouraged (but not required) by the

NATCA representative to follow the optional FPS

posting and marking procedures as best they could so

that an adequate test of the procedure could be

conducted. Experimenters then provided each controller with a Procedures Summary Sheet and reviewed each item on the sheet with the controller.

The Summary Sheet is shown in Appendix H. As in

the Normal condition, a notepad was provided for

the controller to record anything he or she wished.

Once the participant indicated that he or she

understood the instructions, the Optional FPS condition began with a position relief briefing from the

SME after which the participant took full control of

the scenario for 30 minutes. As before, the SME and

two experimenters observed the participant’s activity, and controllers were prompted by the WAK every

5 minutes for a workload rating. The participant used

the computer-based (CRD) checklist to provide a

position relief briefing at the end of the scenario and

then completed the TLX a second time. Finally,

controllers were debriefed, thanked, and released.

Results

Data from ZOB and ZJX were combined yielding

a total of 16 individuals (8 per sector type) and 16

teams (8 per sector type) for the analysis. Results of

individuals and teams will be reported separately for

each sector type. Evaluative comparisons will not be

made between ARTCCs, individuals and teams, or

high and low altitude sectors because the objective of

this experiment was to evaluate the Optional FPS

procedure, not to evaluate a particular facility. It

should be noted that scenarios were used an equal

number of times in each condition. Therefore, any

differences between the Normal and Optional FPS

conditions were not due to differences in scenarios.

Statistical values are only reported for tests that were

significant with a level of a = .05.

Number of FPS posted

It would be difficult to assess the reduced posting and

marking procedure without some willingness on the

participant’s part to try the new procedure. An examination of the number of FPS posted during the scenarios

provides information regarding whether or not participants were willing and able to follow instructions and if

in fact the Optional FPS procedure resulted in fewer

FPS being posted over time. Figures 1 through 4 show

the mean number of FPS posted by condition and

scenario time for individuals and controller teams in

both high and low altitude sectors.

Each of the 4 datasets were analyzed using a 2

(Normal vs. Optional FPS) X 30 (1-min intervals)

within-subjects repeated measures analysis of variance (ANOVA). Individuals in low altitude sectors

posted significantly fewer FPS in the Optional FPS

condition (M = 10.6, SD = 2.0) than in the Normal

condition (M = 14.5, SD = 2.6), F(1, 7) = 37.41, and

the number of FPS posted declined over time, F(29,

203) = 12.05. The Condition X Time interaction was

not significant indicating that the reduction was

relatively immediate and was retained throughout

the scenario. Individuals in the high altitude sectors

did not post significantly fewer FPS in the Optional

FPS condition (M = 13.3, SD = 3.0) than in the

Normal condition (M = 14.1, SD = 2.7) and the

Condition X Time interaction was not significant.

Individuals in high altitude sectors did post fewer

FPS over time, F(29, 203) = 11.69. Thus, FPS posting in this condition was not affected by the Optional

FPS procedure.

Controller teams in low altitude sectors posted

significantly fewer FPS in the Optional FPS condition (M = 14.8, SD = 3.2) as compared to the Normal

condition (M = 18.2, SD = 3.5), F(1, 7) = 19.12, and

also posted fewer FPS over time, F(29, 203) = 8.35.

The Condition X Time interaction was significant,

F(29, 203) = 1.81. Therefore, teams in low altitude

sectors used the Optional FPS procedure to reduce

the number of FPS that were posted early in the
7

scenario relative to the Normal condition, but this

reduction took some time. After the first few minutes

of the scenario, the reduction was maintained.

Controller teams in high altitude sectors posted

significantly fewer FPS in the Optional FPS condition (M = 15.0, SD = 5.2) than in the Normal

condition (M = 20.1, SD = 3.5), and posted fewer

FPS over time, F(1, 7) = 25.50 and F(29, 203) =

54.14, respectively. There was a significant Condition X Time interaction, F(29, 203) = 4.25. Like

their low altitude counterparts, teams in high altitude

sectors were able to use the Optional FPS procedure

to reduce the number of FPS posted during the first

few minutes of the scenario and then retained that

reduction. Controller teams in both high and low

altitude sectors removed FPS at a greater rate during

the Optional FPS condition.

Although a significant advantage did not occur for

individuals in high altitude sectors,6

 in general, fewer

FPS were posted in the active bay during the Optional FPS condition as compared to the Normal

condition. Thus, controllers were removing FPS from

the board per the instructions. The advantage of

having fewer FPS posted was achieved within the first

10 to 15 minutes of the scenario, then that advantage

was maintained. It would be reasonable to argue that

the greatest advantage for optional FPS posting and

marking might have occurred for individuals in high

altitude sectors. High altitude operations are typically not as complex as operations at lower altitudes

because there is greater vertical spacing and routes

tend to be less complex. In fact, the research of

Albright et al. (1995) suggests that FPS are not

needed in high altitude sectors. Nevertheless, the

Optional FPS condition did result in fewer FPS being

posted in the more strip intensive, low altitude sectors.

Number of marks per strip

In addition to observing how many FPS were

posted, the average number of marks made on each

strip were also counted. Again, this insured that

participants were in fact using the optional marking

procedure as instructed. Participants did in fact make

about 1 less mark per strip in the Optional FPS

condition than in the Normal condition. Data are

shown in Figures 5-8.

Data were analyzed using a dependent measures ttest for each data set. Figure 5 shows the mean

number of marks per strip for individuals in low

altitude sectors. Significantly fewer marks were made

in the Optional FPS condition (M = 4.19, SD = 0.20)

than in the Normal condition (M = 4.8, SD = 0.15),

t(1, 7) = 2.64. Individuals in high altitude sectors also

made significantly fewer marks per FPS in the Optional FPS condition (M = 2.52 , SD = 0.10) than in

the Normal condition (M = 3.65, SD = 0.13), t(1, 7)

= 3.05. Individuals in both low and high altitude

sectors made significantly fewer marks on the FPS in

the Optional FPS condition as compared to the

Normal condition. Data are shown in Figure 6.

Figure 7 shows that controller teams in low altitude sectors made significantly fewer marks in the

Optional FPS condition (M = 2.39, SD = 0.11) as

compared to the Normal condition (M = 3.74, SD =

0.49), t(1, 7) = 3.19. Controller teams in high altitude sectors made a comparable number of marks per

FPS in the Optional FPS condition (M = 2.93, SD =

0.07) and the Normal condition (M = 3.35, SD =

0.19). Data are shown in Figure 8. With the exception of teams in high altitude sectors, participants

made fewer marks on the FPS under the Optional FPS

condition. This result supports the fact that participants

were implementing the experimental procedure.

On-line workload ratings

It was expected that on-line rating of workload

using the WAK would be lower during the Optional

FPS condition due to the reduced board management

responsibilities. Alternatively, the introduction of a

new procedure could produce more board management duties and hence, more workload. The WAK

data are shown in Figures 9 through14 which present

data from both high and low altitude sectors provided by individuals and team R-sides and D-sides.

5

 The controller can display a FPR on the CRD via a Quick Action Key and keyboard entries. The FPR displays much of the critical

information that is displayed on the FPS including, callsign, computer identification number, beacon code, assigned altitude, and route of

flight information. The FPR is displayed indefinitely until another action resulting in a CRD message overwrites it.

6

 A number of explanations are possible. It is likely that normal operations for individuals in high altitude sectors may already be somewhat

similar to the experimental condition. A second explanation could be that controllers in this group may not have understood or complied

with instructions. Thirdly, the statistical test used may not have had adequate power to detect a significant difference given the small sample

size. Finally, the scenarios may not have provided situations adequate for the removal of the FPS as directed by the procedural guidelines.
8

Missing data due to a participant not responding to

a single WAK prompt were replaced by the appropriate group mean. Replacement by group means artificially reduces variance.7

 All controllers were included

in the analysis even if they failed to respond to more

than one WAK prompt. Each of the 4 datasets was

analyzed separately using a 2 (Normal vs. Optional

FPS) X 6 (5-min intervals) within-subjects repeated

measures ANOVA.

The results, as shown in Figure 9, indicate that

individual controllers in low altitude sectors rated

workload as being significantly lower in the Optional

FPS condition, F(1, 7) = 23.69, and as increasing over

time, F(5, 35) = 19.00. Individual controllers in high

altitude sectors also rated the Optional FPS as having

lower workload, but this difference was not significant.

This result coincides with the finding of no difference

in the number of FPS that were posted by the same

individuals. Individuals in high altitude sectors also

rated workload as increasing significantly over time,

F(5, 35) = 6.47. These data are shown in Figure 10.

Separate WAK ratings were obtained for both the

R-side and D-side controllers when controller teams

were being observed. R-side controllers in low altitude sectors perceived workload as being less on

average in the Optional FPS condition, but not

significantly so. They did perceive workload to be

increasing over time, F(5, 35) = 6.87, as shown in

Figure 11. Likewise, R-side controllers in high altitude sectors, shown in Figure 12, rated the Normal

and Optional FPS conditions as being similar in

workload yet increasing significantly over time, F(5,

35) = 10.52. D-side controllers in both low and high

altitude sectors, shown in Figures 13 and 14, rated

workload as being comparable under the two FPS

procedures. D-side controllers in high altitude sectors rated workload as increasing significantly over

time, F(5, 35) = 6.29.

Overall, participants tended to rate workload as low

to moderate. Workload fluctuated and was generally

perceived by participants, with exception of low-altitude D-sides, as changing throughout the scenario.

Participants judged workload in the Optional FPS

condition as being comparable and occasionally lower

than the Normal condition.

Post-scenario TLX ratings

Participants provided another subjective rating of

workload after each scenario by completing the TLX.

Data were analyzed separately for each group of

participants (Individual, R-side, and D-side in both

high and low altitude sectors) using a 2 (Optional

FPS vs. Normal) X 6 (TLX item) multivariate analysis of variance (MANOVA). None of the omnibus

multivariate analyses reached the level of significance

and no further analyses were pursued. Results are

shown in Figures 15-20. Although results were not

significant, some small differences are visible and the

results were graphed for this purpose. The TLX

results only suggest weak relationships in the data

and were not supported by statistical analyses. Overall, the TLX ratings were similar for the Optional FPS

and Normal conditions.

Compensatory behaviors

Participants could have written on the notepad to

compensate for marking the FPS less often. However,

participants wrote on the notepad infrequently. Participants made a comparable number of marks on the

notepads between the Normal and Optional FPS

conditions. An average of 0.5 marks was made on the

notepad in the Normal condition and an average of

1.1 marks was made on the notepad in the Optional

FPS condition. The small number of notes written

suggest that either participants did not think writing

information down was necessary or that using the

notepad would have required too much work and so

it was not used very often. Notes often referred to

information that was not normally required but was

needed to operate within the DYSIM, for example,

the sector number receiving a hand-off.

In addition to the notepad, participants could

compensate for reduced FPS posting and marking

requirements by utilizing available computer functions such as the flight plan readout (FPR) or route

display. Compensatory actions were analyzed using a

2 (Normal vs. Optional FPS) X 4 (FPR, route display,

J-ring, slant-0) within-subjects MANOVA. No significant differences in the number of compensatory

7

The replacement of missing data in subjective workload measurement remains an unresolved procedural issue. It is preferable to replace

missing WAK data with group means rather than with a maximum workload rating because it was likely that a failure to respond was due

to inadequate auditory and visual prompts rather than extremely high workload.
9

actions between the Normal and Optional FPS conditions were found. Apparently, participants did not

make significant changes in their behavior to compensate for the reduction in FPS activity as evidenced

by their lack of writing on the notepads and use of

computer-based functions. Although not statistically

significant, participants tended to use slightly more

FPRs in the Optional FPS condition than in the

Normal condition. Compensatory behavior data is

shown in Tables 3-5.

Subject Matter Expert Observations

A SME for each sector used the Behavioral and

Event Checklist to record the occurrence of 11 events

including operational errors, operational deviations,

missed handoff, violation of a Letter of Agreement or

other directive, missed readback error, failure to

grant request, failure to direct aircraft to switch

frequency, cause unnecessary delay, inappropriate

request of information, computer entry error, and

failure to complete proper coordination. Subject

matter experts selected these particular events as types

of events that may have a negative impact upon

operations, especially during the Optional FPS condition. Results are shown in Figures 21-24. Events on

the checklist seldom occurred but when one did it

was just as likely to occur in either experimental

condition. One operational error8

 did occur for a

controller team in a high altitude sector but the error

was not related to FPS activity. Overall, the Behavioral and Events Checklist did not detect any significant

differences between the Normal and Optional FPS

conditions for number and type of events that occurred.

Like behavioral events, it was reasoned that the

Optional FPS condition might impact team communication effectiveness and the adequacy of position

relief briefings. On the contrary, these measures

showed no differences between conditions. Team

communication, as noted by our SMEs, was not

adversely impacted. Only one negative comment was

made by a SME regarding team communication.

However, this comment occurred during the Normal

condition and was related to participants having to

repeat an action unnecessarily. Position relief briefings did not suffer either. The SMEs did not note any

deficiencies regarding position relief briefings in either the Normal or Optional FPS condition.

Post-Scenario Questionnaire

Data from the post-scenario questionnaire are shown

in Table 6. The concerns regarding realism typically

referred to situations that were imposed by the idiosyncrasies of the DYSIM facility. Although each pair of

scenarios used were constructed and evaluated by SMEs

to be similar in complexity, some participants perceived

the scenarios they controlled as being similar in complexity, while others did not. Likewise, some participants mentioned that either their counterpart (R-side or

D-side) or a simulation pilot had done something out of

the ordinary during the experiment. None of the participants indicated that the WAK measure interfered with

their ability to control traffic. Finally, most participants

reported that they preferred the optional posting and

marking procedures.

Conclusions

Overall, participants at ZOB and ZJX posted

fewer FPS and made fewer marks on those that were

posted during the Optional FPS condition. Even

though FPS activity was reduced in the Optional FPS

condition, no detrimental effects in performance,

workload, position relief briefings, or team communication were observed. It is important to note that

participants performed similarly in both experimental conditions despite never having practiced using

the optional FPS posting and marking procedure

prior to the experiment. According to the post-experimental questionnaire, most participants preferred

the optional FPS marking and posting procedures

they used during the Optional FPS condition. Participants did not have to compensate for the lack of

FPS by using other tools, such as the FPR or notepad,

to obtain or remember information that would have

otherwise been present on an FPS. No detrimental

effects of the Optional FPS condition were detected.

8

 Aircraft must be separated by 5 nautical miles (NM) laterally or 1,000 feet vertically when flying below 29,000 feet mean sea level (MSL).

When flying at, or above, 29,000 feet MSL aircraft must be separated by 5 NM laterally or 2,000 feet vertically. An operational error occurs

when two or more aircraft violate these separation standards.
10

Therefore, the results of the present experiment suggest

that the Optional FPS condition provided a viable

procedure by which FPS activity could be reduced.

Although the data collected in the present study

provide encouraging support for the use of an optional FPS posting and marking procedure, some

issues remain. Participants were engaged in the

scenarios for a relatively short period of time. Participants generally had a high degree of vigilance given

that experimenters and a SME were observing them.

It is possible that given more time, general patterns of

behavior may change as participants become more

relaxed. Additionally, given more time to familiarize

themselves with the Optional FPS procedure, it is

likely that participants may develop and adopt behaviors unlike those observed in the experiment. In

particular, it is possible that with longer periods of

time to use the Optional FPS procedure, similar

events may be more likely to interfere, that is, be

confused, with one another (McGeoch, 1942) and

some means of compensation, such as cues for memory

retrieval, may become more important. Controllers

who interact with the same pilots and aircraft flying

the same routes every day may be especially susceptible to memory interference because of the high degree

of episodic similarity in temporally distinctive events.

Another pressing question for future research regards how responsibilities are passed from one controller to the next during and immediately after a

position relief briefing. If the reduced requirements

for FPS management are adopted as optional, that is,

FPS use is at the discretion of the controller currently

in charge of a sector, then how would responsibility

be passed given that the controller relieving the

position wishes to use a procedure different from the

previous controller? The obvious problem occurs

when the current controller is using the reduced posting and marking requirements and is then relieved by a

controller who wishes to use and mark all available FPS.

The question of how to effectively transition from one

controller to another controller who wishes to use a

different procedure remains to be answered.

References

Albright, C. A., Truitt, T. R., Barile, A. L, Vortac, O.

U., & Manning, C. A. (1994). Controlling traffic

without flight progress strips: Compensation,

workload, performance, and opinion. Air Traffic

Control Quarterly, 2, 229-48.

Chase, W. G., & Simon, H. A. (1973). Perception in

chess. Cognitive Psychology, 4, 55-81.

Ericsson, K. A., & Kintsch, W. (1995). Long-term

working memory. Psychological Review, 102(2),

211-45.

Ericsson, K. A., & Staszewski, J. J. (1989). Skilled

memory and expertise: Mechanisms of exceptional performance. In D. Klahr and D. Kotovsksy

(Eds.), Complex information processing: The impact of Herbert A. Simon (pp.235-67). Hillsdale,

New Jersey: Lawrence Erlbaum Associates.

Federal Aviation Administration (1995). Air traffic

control handbook (7110.65J). Washington DC:

Author.

Gromelski, S., Davidson, L., & Stein, E. S. (1992).

Controller memory enhancement: Field facility concepts and techniques. (DOT/FAA/CT-TN92/7).

Atlantic City, NJ: U. S. Department of Transportation, Federal Aviation Administration.

Gronlund, S. D., Ohrt, D. D., Dougherty, M. R. P.,

Perry, J. L., & Manning, C. A. (1998). Role of

memory in air traffic control. Journal of Experimental Psychology: Applied, 4, 263-80.

Hart, S. G., & Staveland, L. E. (1988). Development of

NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock

and N. Meshkati (Eds.), Human Mental Workload

(pp. 139-83). Amsterdam: North-Holland.

Hopkin, V. D. (1988). Air traffic control. In E. L.

Wiener and D. C. Nagel (Eds.), Human factors in

aviation (pp. 639-63). San Diego: Academic Press.
11

Hopkin, V. D. (1995). Situational awareness in air

traffic control. In R. D. Gilson, D. J. Garland, &

J. M. Koonce (Eds.), Situational awareness in

complex systems: Proceedings of a CAHFA conference, 171-8. Daytona Beach: Embry Riddle Aeronautical University Press.

McGeoch, J. A. (1942). The psychology of human learning. New York: Longmans, Green.

Slamecka, N. J., & Graf, P. (1988). The generation

effect: Delineation of a phenomenon. Journal of

Experimental Psychology: Human Learning &

Memory, 4(6), 592-604.

Stein, E. S. (1985). Air traffic controller workload: An

examination of workload probe. (DOT/FAA/

CT-TN84/24). Atlantic City, NJ: U. S. Department of Transportation, Federal Aviation

Administration.

Stein, E. S. (1991). Air traffic controller memory - A field

survey. (DOT/FAA/CT-TN90/60). Atlantic City,

NJ: U. S. Department of Transportation, Federal

Aviation Administration.

Stein, E. S., & Bailey, J. (1994). The controller memory

guide. (DOT/FAA/CT-TN94/28). Atlantic City,

NJ: U. S. Department of Transportation, Federal

Aviation Administration.

Vortac, O. U., Edwards, M. B., Fuller, D. K. & Manning, C. A. (1993). Automation and cognition in

air traffic control: An empirical investigation. Applied Cognitive Psychology, 7, 631-51.

Zingale, C., Gromelski, S., Ahmed, S. B., & Stein, E. S.

(1993). Influence of individual experience and flight

strips on air traffic controller memory/situational

awareness. (DOT/FAA/CT-TN93/31). Atlantic

City, NJ: U. S. Department of Transportation,

Federal Aviation Administration.

Zingale, C., Gromelski, S., & Stein, E. S. (1992).

Preliminary studies of planning and flight strip

use as air traffic controller memory aids. (DOT/

FAA/CT-TN92/22). Atlantic City, NJ: U. S.

Department of Transportation, Federal Aviation Administration.
12
13

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Scenario Time (min)

Mean Number of FPS Posted

Normal Optional FPS

Figure 1. Mean number of FPS posted by condition and scenario time for

Individual controllers in Low Altitude sectors.

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Scenario Time (min)

Mean Number of FPS Posted

Normal Optional FPS

Figure 2. Mean number of FPS posted by condition and scenario time for

Individual controllers in High Altitude sectors.

Figures
14

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Scenario Time (min)

Mean Number of FPS Posted

Normal Optional FPS

Figure 3. Mean number of FPS posted by condition and scenario time for

controller Teams in Low Altitude sectors.

0

5

10

15

20

25

30

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Scenario Time (min)

Mean Number of FPS Posted

Normal Optional FPS

Figure 4. Mean number of FPS posted by condition and scenario time for

controller Teams in High Altitude sectors.
15

0

1

2

3

4

5

6

Normal Optional FPS

Condition

Mean Number of Marks per FPS

Figure 5. Mean number of marks per FPS by condition for Individuals in Low

Altitude sectors.

0

1

2

3

4

5

6

Normal Optional FPS

Condition

Mean Number of Marks per FPS

Figure 6. Mean number of marks per FPS by condition for Individuals in High

Altitude sectors.
16

0

1

2

3

4

5

6

Normal Optional FPS

Condition

Mean Number of Marks per FPS

Figure 7. Mean number of marks per FPS by condition for controller Teams

in Low Altitude sectors.

0

1

2

3

4

5

6

Normal Optional FPS

Condition

Mean Number of Marks per FPS

Figure 8. Mean number of marks per FPS by condition for controller Teams

in High Altitude sectors.
17

0

1

2

3

4

5

6

7

5 10 15 20 25 30

Scenario Time (min)

Mean Workload Rating

Normal Optional FPS

Figure 9. Mean workload (WAK) rating by condition and scenario time for

Individual controllers in Low Altitude sectors.

0

1

2

3

4

5

6

7

5 10 15 20 25 30

Scenario Time (min)

Mean Workload Rating

Normal Optional FPS

Figure 10. Mean workload (WAK) rating by condition and scenario time for

Individual controllers in High Altitude sectors.
18
19
20
21
22
23
24
25
26
A1

APPENDIX A

Proposed change to Order 7210.3M Part 2. Chapter 8. Section 1. Paragraph 8-1-6. Flight progress

strip usage/marking procedures.

a. Flight progress strips will continue to be posted, marked, and updated in accordance with the

National directives until radar contact and communications are established and accepted.

b. Once this has been achieved, the radar controller (or, if so designated, the manual controller)

may remove the strip from the board. If the radar controller (or, if so designated, the manual

controller) elects to keep the strip at the sector, a check mark may be placed in box 21-24 to

indicate that further strip marking is unnecessary.

c. The sector team is responsible for all information contained on the flight progress strip.

Standard strip marking is optional for both controllers. However, if the radar controller

chooses to utilize standard strip marking, the associate controller will support and comply

with that request. If the radar controller does not choose to utilize standard strip marking,

nothing in this procedure precludes the associate controller from utilizing standard strip

marking.

d. Partial recording of control information deemed useful to the sector operation is permitted.

e. Strips on aircraft “pointed out” to the sector may be check marked even though

communications are never established.

f. The following flight progress strips are to remain posted and standard strip marking used:

(1) Any aircraft you cannot reasonably expect to remain in radar contact.

(2) Aircraft transitioning from radar to non-radar environment.

(3) Aircraft requiring special handling, i.e., emergencies, radio failures, etc. (Note: Standard

strip marking will begin when the need for special handling is identified.)

(4) All non-radar flights.

(5) All flights transitioning from automated to non-automated modes of operation.

(6) All flights that are issued holding instructions.

a. Departures and proposals are to be considered non-radar until radar contact has been

established and accepted in accordance with FAAH 7110.65.

b. Any control action not recorded via landline, frequency, or computer entry must be marked

on the appropriate strip to record the action.

c. In the event of a back-up system failure, the strips will be posted and marked in accordance

with National and local directives. These systems would include any back-up radar or

malfunctioning recording system and when operating in DARC only.

d. The controller will continue to be responsible for the control of the aircraft and coordination

of information as prescribed in FAAH 7110.65.

e. Blank note pads will be available at every sector for the controller’s use.

f. Standard strip marking must be accomplished when training is in progress. This may be

discontinued with the consent of the training team.
B1

APPENDIX B

Biography

1) At how many En Route Centers have you worked?

_________ (number of centers)

Please list the centers you have worked at beginning with the most recent:

1) _____________________________

2) _____________________________

3) _____________________________

4) _____________________________

2) How long have you worked in your current area?

______ years _______ months

3) How long have you worked at your current ARTCC?

______ years _______ months

4) How many years and months (total) have you worked at an En Route Center as an FPL? ______ years _______

months

5) Please indicate your total number of years as a controller.

______ years _______ months

6) Please indicate all operations in which you have been an FPL (check all that apply)

______ En Route

______ Terminal

______ Flight Service Station

______ Other (please list)

7) Please check below any area in which you served for six months or more. Please indicate whether you were a

manager (M) or a specialist (S). (check all that apply)

______ Flow Control

______ Traffic Management Unit

______ Supervisor at En Route

______ Training Department

______ Automation Specialist

______ Quality Assurance Specialist

______ Area Officer

______ Regional Officer

______ National Headquarters

8) When were you last certified or re-certified?

______ month _______ year

OPTIONAL: Please provide the following:

1) Age _____

Gender _____
C1

APPENDIX C

General descriptions for airspace sectors used.

Cleveland ARTCC (ZOB)

Lichtfield (LFD) Sector – low altitude (ZOB 7220.2B, Appendix 1)

The LFD Sector is a major feeder sector for arrival traffic to the DTW area. Most of this traffic comes in

from the west, southwest, and the south. LFD Sector also works TOL arrivals and departures to or from the

west and northwest.

Hudson (HUD) Sector – high altitude (ZOB 7220.2A, Appendix 2)

The HUD Sector is an intermediate high altitude sector that has two (2) main flows of traffic: One is

primarily eastbound from the CRL VOR. The other flow is westbound via J36.FNT. There is additional

eastbound traffic on J70. With the HUD Sector is an air refuel route, AR206H, which is located along J36.

Jacksonville ARTCC (ZJX)

Florence (FLO) Sector – low altitude (ZJX 7220.4H, Appendix 3)

This sector shall include all airspace from the surface up to but not including FL240.

Brewton Sector – high altitude (ZJX 7220.4H, Appendix 5)

This sector shall include all airspace from the surface up to but not including FL240.
D1

APPENDIX D

TLX rating form and scale descriptors

TLX Instructions

We are interested in finding out your perception of how difficult the task is and how well you perform the

task. Our objective is to measure your perceived “workload” level. The concept of workload is composed of

several different factors. Therefore, we would like you to tell us about several individual factors rather than

one overall workload score.

Here is an example of the rating scales. As you can see, there are six scales on which you will be asked to

provide a rating score: mental demand, physical demand, temporal demand, effort, frustration, and performance.

Rating Scales

Mental demand refers to the level of mental activity like thinking, deciding, and looking that was required

by the task. You will rate this scale from low to high.

Physical demand involves the amount of physical activity required of you, such as controlling or activating.

Temporal demand refers to the time pressure you experienced during the task. In other words, was the pace

slow and leisurely or rapid and frantic? If the pace was rapid and frantic you are experiencing high temporal

demand.

Effort refers to how hard you worked (both mentally and physically) in order to achieve your level of

performance.

Frustration level refers to how secure and relaxed versus stressed and discouraged you felt during the task. If

you feel secure and relaxed, you have low frustration.

Performance level refers to you perception of your own performance level. Your rating here should reflect

your satisfaction with your performance in accomplishing the goals of the task.

Making your response

You should indicate your rating by placing an ‘X’ on the line adjacent to the item.

For example, if you want to give a high rating of stress factor, place an ‘X’ to the right of the half-way mark.

The higher the stress rating, the closer the ’X’ should be to “HIGH”. In contrast, if your stress rating is

low, you would place an ‘X’ closer toward the “LOW” end of the line. Likewise, if the stress rating is

average, place an ‘X’ in the center of the line.

Please give your responses thoughtful consideration, but do not spend too much time deliberating over them. Your

first response will probably accurately reflect your feelings and experiences.
E1

Behavioral and Event Checklist

Event Aircraft identity Totals

Operational Errors

(Describe briefly in this column)

(Write both call signs in one box)

1.

2.

3.

Operational Deviations/SUA violations

(Describe briefly in this column)

(Write call sign in each box)

1.

2.

3.

4.

Behavior Number of events Totals

Failed to accept, initiate handoff

LOA/Directive Violations

Readback/Hearback errors

Failed to accommodate pilot request

Made late frequency change

Unnecessary delays

Asked pilot for information available from

computer or strip

Incorrect information in computer

Failed to pass headings, speeds or coordinate

pointouts

APPENDIX E

SME rating form
F1

APPENDIX F

Team Communication Effectiveness Rating Form

Effectiveness of Team Communications

Sector type: High Low SME ID:

Condition: Baseline Experimental Scenario ID:

Facility: ZOB Participant ID:

Activity Effective Ineffective N/A

Each controller was aware of actions other had taken

Communicated verbally with team member

Pointed at PVD to communicate with team member

Pointed at strips to communicate with team member

Duplicated efforts—repeated actions already taken by

other team member

True Not true N/A

Lack of communication affected safety of operations

Lack of communication affected efficiency of

operations

Notes (comment on specific events):
G1

APPENDIX G

Position Relief Briefing Rating Form

Position Relief Briefing Checklist

To be completed by SME or ATC Task Force Member

Individual / Team

Sector type: High Low

SME ID:

Condition: Baseline Experimental Scenario ID:

Facility: ZOB Participant ID:

Please make a mark under the appropriate column if the participant did not cover the items along

the left side of the form during the position relief briefing.

Items briefed Individual Team R-side Team D-side

Navaids

Equipment

Radar

Airports

Weather

Flow Control

Special Use Airspace

Traffic

Please choose one of the following and comment below, if desired.

In your opinion, was the position relief briefing negatively affected by the absence of strips of the lack of strip

marking?

a. No

b. Yes, it was affected by the absence of strips

c. Yes, it was affected by the lack of strip marking

d. Yes, it was affected by both b and c.
H1

APPENDIX H

Procedures Summary Sheet

FPS Procedures

Post and mark FPS until radar contact and communications are established and accepted.

Must record on FPS any action not recorded elsewhere.

To indicate no further marking required, check box 21-24.

If FPS no longer needed, may remove from board.

Must post and mark FPS for aircraft if:

1) Radar contact may be lost

2) Transitioning from radar to non-radar

3) Special handling required

4) Non-radar flight

5) Transitioning from auto to non-auto mode

6) Holding instructions are issued