**PREFACE**

This investigation evaluated the effectiveness of an out-of-the-window
tower simulation[^1] by studying the first year of training conducted in
the simulated tower. The author wishes to express gratitude to the ten
student controllers whose training was studied in this evaluation. Their
many observations provided an invaluable source of insight about the
simulation.

Jon Bremseth, an O'Hare training specialist, operated the simulation. He
previously developed a training laboratory using videotapes correlated
to flight progress strips, setting the stage for the current,
interactive simulation training facility. Jon served as the facility
contact person for this evaluation. He also assisted this evaluation by
ensuring that one session was videotaped for each trainee each week and
by providing the corresponding flightstrips which were used in data
reduction. He contributed useful observations on the productive use of
out-of-the-window tower simulation, many of which appear in this report.

Many other individuals at O'Hare deserve recognition for their
outstanding contributions to this evaluation. Ellen Jaeger is the
Assistant Manager for Training at O'Hare. She and Roy Hillen, an O'Hare
training specialist, collected preliminary data on trainee errors during
their simulation training and provided information about the trainees'
prior experience. Tower supervisors Bob Karnick and Tim Fitzgerald, and
area manager Kevin Markwell provided ratings of trainee skills. Matt
Dunne, Acting Assistant Manager of Traffic Management provided typical
taxi delays, taxi times and aircraft rates during an outbound push. The
author also recognizes and appreciates the critical support provided by
Tower Air Traffic Manager, Bill Halleck.

The author thanks three individuals for their help with technical
aspects of this project. Mitch Grossberg of FAA/ACT-500 contributed
valuable suggestions on the format used to collect supervisor ratings.
Patricia Pilanen, a tower training specialist at Logan Airport, answered
many of the author's questions about tower training and drew the
author's attention to the need for ground control trainees to learn to
manage flight progress strips without reducing their attention to the
airport movement areas. Karl Hergenrother of RSPA/Volpe Center created
the data reduction programs used to reduce duration data from videotape,
developed videotape data reduction procedures, and contributed valuable
observations on the simulation process.

Objective data reduced from videotape played a key role in this
evaluation. This exacting work was accomplished by Tufts University
engineering psychology students, Ana Pons, Kathleen Kim, and Trudi
Leone, while they were employed at RSPA/Volpe Center. Bill Voss at
ATZ-200 provided the requirement for this study. Larry Cole at AAR-200
sponsored the effort. Their continuing interest and support were
critical to the completion of this project.

**METRIC/ENGLISH CONVERSION FACTORS**

  ------------------ ------- --------------- ----------------- ------- -----------------
  **ENGLISH TO                               **METRIC TO               
  METRIC**                                   ENGLISH**                 

  **LENGTH                                   **LENGTH                  
  (APPROXIMATE)**                            (APPROXIMATE)**           

  **1 inch (in)**    **=**   **2.5           **1 millimeter    **=**   **0.04 inch
                             centimeters     (mm)**                    (in)**
                             (cm)**                                    

  **1 foot (ft)**    **=**   **30            **1 centimeter    **=**   **0.4 inch (in)**
                             centimeters     (cm)**                    
                             (cm)**                                    

  **1 yard (yd)**    **=**   **0.9 meter     **1 meter (m)**   **=**   **3.3 feet (ft)**
                             (m)**                                     

  **1 mile (mi)**    **=**   **1.6           **1 meter (m)**   **=**   **1.1 yards
                             kilometers                                (yd)**
                             (km)**                                    

                                             **1 kilometer     **=**   **0.6 mile (mi)**
                                             (km)**                    
  ------------------ ------- --------------- ----------------- ------- -----------------

  ------------------ ------- --------------- ----------------- ------- -----------------
  **AREA                                     **AREA                    
  (APPROXIMATE)**                            (APPROXIMATE)**           

  **1 square inch    **=**   **6.5 square    **1 square        **=**   **0.16 square
  (sq in, in^2^)**           centimeters     centimeter                inch (sq in,
                             (cm^2^)**       (cm^2^)**                 in^2^)**

  **1 square foot    **=**   **0.09 square   **1 square meter  **=**   **1.2 square
  (sq ft, ft^2^)**           meter (m^2^)**  (m^2^)**                  yards (sq yd,
                                                                       yd^2^)**

  **1 square yard    **=**   **0.8 square    **1 square        **=**   **0.4 square mile
  (sq yd, yd^2^)**           meter (m^2^)**  kilometer                 (sq mi, mi^2^)**
                                             (km^2^)**                 

  **1 square mile    **=**   **2.6 square    **10,000 square   **=**   **1 hectare (ha)
  (sq mi, mi^2^)**           kilometers      meters (m^2^)**           = 2.5 acres**
                             (km^2^)**                                 

  **1 acre = 0.4     **=**   **4,000 square                            
  hectare (he)**             meters (m^2^)**                           
  ------------------ ------- --------------- ----------------- ------- -----------------

+------------------+---+--------------+---------------+---+----------------+
| **MASS - WEIGHT  |   |              | **MASS -      |   |                |
| (APPROXIMATE)**  |   |              | WEIGHT        |   |                |
|                  |   |              | (A            |   |                |
|                  |   |              | PPROXIMATE)** |   |                |
+------------------+---+--------------+---------------+---+----------------+
| **1 ounce (oz)** | * | **28 grams   | **1 gram      | * | **0.036 ounce  |
|                  | * | (gm)**       | (gm)**        | * | (oz)**         |
|                  | = |              |               | = |                |
|                  | * |              |               | * |                |
|                  | * |              |               | * |                |
+------------------+---+--------------+---------------+---+----------------+
| **1 pound (lb)** | * | **0.45       | **1 kilogram  | * | **2.2 pounds   |
|                  | * | kilogram     | (kg)**        | * | (lb)**         |
|                  | = | (kg)**       |               | = |                |
|                  | * |              |               | * |                |
|                  | * |              |               | * |                |
+------------------+---+--------------+---------------+---+----------------+
| **1 short ton =  | * | **0.9 tonne  | **1 tonne (t) | * | **1.1 short    |
| 2,000 pounds     | * | (t)**        | =**           | * | tons**         |
| (lb)**           | = |              |               | = |                |
|                  | * |              | **1,000       | * |                |
|                  | * |              | kilograms     | * |                |
|                  |   |              | (kg)**        |   |                |
+------------------+---+--------------+---------------+---+----------------+

  ------------------ ------- --------------- ----------------- ------- -----------------
  **VOLUME                                   **VOLUME                  
  (APPROXIMATE)**                            (APPROXIMATE)**           

  **1 teaspoon       **=**   **5 milliliters **1 milliliter    **=**   **0.03 fluid
  (tsp)**                    (ml)**          (ml)**                    ounce (fl oz)**

  **1 tablespoon     **=**   **15            **1 liter (l)**   **=**   **2.1 pints
  (tbsp)**                   milliliters                               (pt)**
                             (ml)**                                    

  **1 fluid ounce    **=**   **30            **1 liter (l)**   **=**   **1.06 quarts
  (fl oz)**                  milliliters                               (qt)**
                             (ml)**                                    

  **1 cup (c)**      **=**   **0.24 liter    **1 liter (l)**   **=**   **0.26 gallon
                             (l)**                                     (gal)**

  **1 pint (pt)**    **=**   **0.47 liter                              
                             (l)**                                     

  **1 quart (qt)**   **=**   **0.96 liter                              
                             (l)**                                     

  **1 gallon (gal)** **=**   **3.8 liters                              
                             (l)**                                     

  **1 cubic foot (cu **=**   **0.03 cubic    **1 cubic meter   **=**   **36 cubic feet
  ft, ft^3^)**               meter (m^3^)**  (m^3^)**                  (cu ft, ft^3^)**

  **1 cubic yard (cu **=**   **0.76 cubic    **1 cubic meter   **=**   **1.3 cubic yards
  yd, yd^3^)**               meter (m^3^)**  (m^3^)**                  (cu yd, yd^3^)**
  ------------------ ------- --------------- ----------------- ------- -----------------

  ------------------- ------- --------------- ---------------- ------- -----------------
  **TEMPERATURE                               **TEMPERATURE            
  (EXACT)**                                   (EXACT)**                

  **\[(x-32)(5/9)\]   **=**   **y °C**        **\[(9/5) y +    **=**   **x °F**
  °F**                                        32\] °C**                
  ------------------- ------- --------------- ---------------- ------- -----------------

![](media/image1.emf)**1**

![](media/image2.emf)**2**

**For more exact and or other conversion factors, see NIST Miscellaneous
Publication 286, Units of Weights and Measures. Price \$2.50 SD Catalog
No. C13 10286 Updated 1/23/95**

**TABLE OF CONTENTS**

[Section]{.underline} [Page]{.underline}

1\. INTRODUCTION 1

> 1.1 Purpose 1
>
> 1.2 Background 1
>
> 1.3 Constraints on the Evaluation 2
>
> 1.3.1 Lack of a Concurrent Control Group 3
>
> 1.3.2 Ongoing Training 3
>
> 1.3.3 Sample Size 3

2\. THE SIMULATION 5

> 2.1 Description 5
>
> 2.2 Limitations of the Simulation 6
>
> 2.2.1 Partial Field of View 6
>
> 2.2.2 No Inbound Aircraft 8
>
> 2.2.3 Automated Controller Speech Recognition/Synthetic Pilot Speech 8
>
> 2.3 Precursor to Simulation Training at O'Hare 9

3\. OUT-OF-THE-WINDOW SIMULATION TRAINING METHODS 11

> 3.1 Trainees and Schedule 11
>
> 3.2 Simulation Training Procedure 11
>
> 3.3 Modification of Training Methods 13

4\. EVALUATION METHOD 15

> 4.1 Measures of Effectiveness 15
>
> 4.2 Videotaped Training Sessions 15

5\. RESULTS AND DISCUSSION 17

> 5.1 Post-Simulation Training Measures 17
>
> 5.1.1 Days-to-Certification 17
>
> 5.1.2 Hours-to-Certification 18
>
> 5.1.3 Tower Supervisor and Area Manager Ratings of Ground Control
>
> Skills 19
>
> 5.2 Simulation Training Measures 21
>
> 5.2.1 Preliminary Data Considerations and Analysis 21
>
> 5.2.2 Taxi Delay 23
>
> 5.2.3 Taxi Time 24
>
> 5.2.4 Window and Stripboard Scan 26
>
> 5.2.5 Analysis of Instructor Assistance 26

**TABLE OF CONTENTS (continued)**

[Section]{.underline} [Page]{.underline}

6\. FINDINGS AND RECOMMENDATIONS 29

> 6.1 Training Using the Simulator was Effective 29
>
> 6.2 Training Using the Simulator was Faster and More Effective then
> Traditional Training 30
>
> 6.3 Training Using Tower Simulation is Likely to Show Increased
> Benefits Upon Upgrading 31
>
> 6.4 Current Cost of Recommended Upgrading 34
>
> 6.5 New Applications of Tower Simulation Technology 34

APPENDIX A: FOCUS GROUP SUMMARY 39

APPENDIX B: BASELINE DEVELOPMENT OF GROUND CONTROL SKILLS 43

APPENDIX C: OUTLINE OF RECOMMENDED REQUIREMENTS FOR A TOWER SYSTEM
ASSESSMENT SIMULATOR 45

**LIST OF FIGURES**

[Figure]{.underline} [Page]{.underline}

3\. SIMULATION EQUIPMENT LAYOUT 5

2\. PARTIAL FIELD OF VIEW 7

3\. AUTOMATED SPEECH RECOGNITION/SYNTHETIC SPEECH: EFFECT ON TAXI DELAY
22

4\. AVERAGE TAXI DELAY 23

5\. TAXI DELAY TIMES LESS THAN OR EQUAL TO 30 SECONDS 24

6\. AVERAGE TAXI TIME 25

7\. SCAN TOWARD WINDOW AND STRIPBOARD 26

8\. ASSISTANCE REQUIRED (MAJOR CATEGORIES) 28

9\. INSTRUCTOR-CORRECTED AND SELF-CORRECTED COMMUNICATIONS 28

**LIST OF TABLES**

[Table]{.underline} [Page]{.underline}

1\. AIRPORT CONFIGURATIONS AND DEPARTURE RUNWAYS AT O'HARE

AIRPORT 11

2\. TRAINING OBJECTIVES 12

3\. DAYS-TO-CERTIFICATION 17

4\. HOURS-TO-CERTIFICATION 18

5\. STRENGTH OF GROUND CONTROL SKILLS FOLLOWING ONE MONTH OF SIMULATION
TRAINING COMPARED TO BASELINE 20

6\. EQUIVALENT MONTHS OF OJT: SIMULATION TRAINING COMPARED TO BASELINE
21

7\. INSTRUCTOR ASSISTANCE BY WEEK OF TRAINING (MEAN NUMBER OF ASSISTS)
27

**EXECUTIVE SUMMARY**

The objective of this evaluation was to gather evidence bearing on the
potential usefulness of out-of-the-window ATC tower simulation for
training tower controllers. The FAA Research and Development Service
(ARD) sponsored the development of an out-of-the-window simulation as a
proof-of-concept demonstration. It was about to be used for the first
time to train developmental controllers at O'Hare International Airport
in Chicago. This simulation was a prototype (beta) version of TowerPro™
(Aviation Simulations, Inc.). It had been developed specifically to
train outbound ground control at O'Hare. The data for this evaluation
were collected on the progress and relative post-training proficiency of
all ten developmental controllers who completed simulation training at
O'Hare during 1994, the first year of its use.

The simulation implemented at O'Hare included one controller position,
outbound ground control. An out-of-the-window view was projected onto
three visual displays which approximated the size of actual tower
windows. Computer-generated aircraft appeared to move in three
dimensions on these displays at rates which varied with the type of
aircraft. The aircraft were representative of the types and companies at
O'Hare. A session included 30 to 90 aircraft in 50 minutes.

The simulation was limited to three screens, so that only a 135 degree
section of the airport could be displayed at any one time. Controllers
at O'Hare must observe aircraft movements on all sides of the tower. In
the simulation, instead of turning to look through a different window,
the trainees used a hand-held control to rotate or "pan" the displayed
section to show the relevant part of the airport. The simulation was
also limited to outbound aircraft: *No inbound aircraft were included in
the simulation scenarios.*

The evaluation was constrained by the lack of a concurrent control group
consisting of

developmental controllers who would have received only traditional
training. Instead, comparisons were made between the trainees who used
the simulation and those who had previously been trained at O'Hare,
prior to the installation of the simulation.

The primary measures of simulation effectiveness concerned how long
(days and total hours) it took a developmental controller to become
certified on the outbound ground control position. The primary measures
were supplemented by ratings made by tower supervisors on specific
ground control skills, following simulation training. These expert
ratings were compared to the supervisors' and an area manager's
*recollections* of the baseline strength of developmental controller
skills after varying amounts of time.

Additional data were reduced from videotapes of the simulation training
sessions. The measures obtained included taxi delay, defined as the time
it took the ground controller to provide taxi instructions to an
aircraft that was ready to taxi; taxi time, which began when an aircraft
received taxi instructions and ended when it was instructed to switch
radio frequencies to a tower frequency; and visual scanning directed
toward the "window" or toward the stripboard. Instructor comments made
during the videotaped sessions were classified according to topic and
tallied to provide additional indications of the development of specific
ground control skills. Also, the first group of trainees' perceptions of
the simulation training were solicited in a focus group session.

The data collected support the following conclusions:

**The out-of-the-window tower simulation used at O'Hare:**

-   **is an effective tool for training many ground control skills and
    knowledge, and appears particularly effective in increasing
    trainees' working speed.**

```{=html}
<!-- -->
```
-   **is more effective for training outbound ground control than more
    traditional training techniques.**

The effectiveness of the simulation was indicated by the following:

**I. Trainees' ground control skills increased noticeably during
simulation training.**

*Taxi delay decreased consistently from the second week to the end of
training.*

*Stripboard scanning began to consistently decrease after the second
week of training, while window scanning began to increase after the
third week of training.*

*The amount of assistance needed for a trainee to properly handle a
scenario decreased consistently throughout training.*

**II. Training using the simulator was more effective than traditional
training.**

*The developmental controllers who were trained using simulation became
certified on the outbound ground control position in 25% fewer
[days]{.underline} than developmental controllers who were trained
without simulation.* *However, using simulation, the trainees needed
about the same (only 5% fewer) [hours]{.underline} to become certified
on this tower position.*

*Expert ratings of eight ground control skills based on actual tower
observations were all higher following simulation training than
following traditional training*.

The difference between the days-to-certification and total
hours-to-certification results can be explained in terms of working
speed:

> ***The Working Speed Hypothesis:** **Simulation training increased
> developmental controllers' working speed, which enabled them to work
> in the actual tower under a wider range of conditions (i.e., under
> heavier or more complex traffic) and hence for more hours per day than
> with traditional training.***

**Recommendations**

Upgrading the simulation could be of value in providing training for the
following:

1.  Inbound ground control and coordination between inbound and outbound
    ground control positions

2.  Reflexive, correct communications

3.  Scanning and situational awareness

4.  Understanding what a pilot can see from the cockpit

5.  Smooth transition between window scan and BRITE/ASDE displays

6.  Teamwork

Simulation training at O'Hare requires use of the simulation facility
for roughly six months per year. The most productive use of tower
simulation at O'Hare would incorporate enhanced capabilities and new
areas of application that would permit more extensive use. The following
enhancements and new application areas are discussed:

1.  Individual performance enhancement for current controllers

2.  Team performance enhancement for current controllers

3.  Training in the handling of unusual situations

4.  Optimizing training through the application of experimental training
    conditions

5.  Tower controller candidate screening

6.  Assessment of new tower equipment, procedures, and airport
    configurations.

7.  

**1. INTRODUCTION**

1.1 PURPOSE

The purpose of this effort was to provide human factors testing and
evaluation of an air traffic control tower (ATCT) out-of-the-window
(OTW) controller training simulation located at O'Hare International
Airport in Chicago. It was intended to analyze and quantitatively
measure, to the extent possible, the potential usefulness of devices of
this type if they were to be installed at selected high density or
complex ATC facilities. Goals included:

-   An evaluation of the impact of tower simulation on
    time-to-certification on position

-   Quantitative estimates of any savings to the user (airlines,
    aircraft operators)

-   Recommendations for the most productive application of OTW
    simulators

-   Recommendations for functional specifications of future simulators
    based on current systems

-   Evaluation of the potential of tower simulation for rapid
    prototyping to assess the impact of new tower equipment, procedures,
    or configurations.

1.2 BACKGROUND

The U.S. air traffic system requires well-trained tower controllers.
Controllers receive

training at the FAA Academy, but the training for high activity
(operational level 5) towers normally follows years of operational
experience as a full performance level controller at less active towers,
plus many months of training at the high activity tower. Until certified
on all tower positions, trainees are termed "developmental controllers."
Developmental controllers do not work independently at a tower position
until they have met the criteria for certification on that position. At
the nations' high activity towers, the tower positions typically include
flight data, clearance delivery, ground control, local control, and
supervisor.

This report concerns outbound ground control training at O'Hare Tower.
At O'Hare, controllers handle the largest number of aircraft operations
of any tower in the country. In 1992, O'Hare recorded nearly thirty
million enplanements, with a seventy to eighty percent increase in
enplanements expected by 2010[^2]. The volume and complexity of its
traffic make outbound ground control at O'Hare arguably the most
challenging position to work at and the most difficult position to learn
of any position at any air traffic facility in the U.S.

During daily periods of high activity, ground control responsibilities
are divided between two positions at O'Hare: the inbound ground control
position is responsible for arrivals taxiing in to their gates; outbound
ground control handles departures taxiing out toward a runway. Weather
accounts for much more delay than any other cause, but according to the
FAA Aviation Capacity Enhancement Plan, taxi-out consistently accounts
for more delay than the other phases of flight[^3]. This statistic
suggests the importance of providing safe and efficient air traffic
services to outbound aircraft. The same source indicates that O'Hare
ranked second in delays of the 55 airports at which the FAA collects air
traffic delay statistics.

The outbound ground control position at O'Hare requires considerable
knowledge and skill. The outbound ground controller must first decide
which aircraft to call, whether to provide taxi instructions or to make
a traffic call. Determining the priority of duty requires almost
continuous attention to the situation on the taxiways, particularly
during an "outbound push" (i.e., when relatively many aircraft are
departing). At the same time, he or she must maintain an awareness of
the location of aircraft that are ready to taxi. This controller decides
which one of O'Hare's eight departure runways to send each departing
aircraft to, and in what sequence, decisions that must take into account
the type of aircraft and its initial route of flight. Then the outbound
ground controller must decide which series of taxiways the aircraft
should use to reach its runway. After transmitting the departure runway
and taxi route to the pilot, ground control must make certain that the
pilot understood the instructions, and follows the taxi route. Often the
outbound ground controller needs to coordinate with the inbound ground
controller and local controllers when the aircraft is expected to enter
their areas of responsibility.

The outbound ground controller has typically required more than a year
of traditional on-the-job training (OJT) before a supervisor certifies
his or her ability to work independently at this position. During this
time, an instructor must supervise the developmental controller
(trainee) while the trainee controls aircraft. At times, the situation
on the airport surface is too busy for the trainee to handle, requiring
the instructor to take over. At other times, there is too little traffic
or complexity for the trainee to increase the level of his or her
skills. The ideal training situation is one with enough traffic and
complexity to allow the trainee to gain confidence in handling difficult
situations, but also one that is not so difficult that safety or the
expeditious flow of aircraft could be reduced. The ideal training
situation does not occur often in actuality, so OJT proceeds more slowly
than if the ideal training situation were always available.

In an effort to improve the speed and quality of training, the training
specialists at O'Hare were provided a prototype tower controller
training simulator, TowerPro™ (Aviation Simulations, Inc.). This
prototype was developed for a proof-of-concept demonstration of its
potential for training outbound ground control at O'Hare. The simulation
operator, who was also a training specialist, prepared simulation
scenarios for training on what he believed would provide the ideal
amount of traffic and complexity for each trainee. Simulation thus
permitted the training of ground controllers to always proceed under
planned conditions. To the extent that this training was sufficiently
realistic to transfer to performance in the tower, simulation training
was expected to reduce training time, particularly for the outbound
ground control position.

1.3 CONSTRAINTS ON THE EVALUATION

This evaluation was conducted as a field study. As such, it was limited
in the use of statistical control groups and independent variables. This
section describes these constraints and their impact on the evaluation.

-   Lack of a concurrent control group

-   Ongoing training

-   Sample size

1.3.1 [Lack of a Concurrent Control Group]{.underline}

A constraint on the evaluation was the lack of a concurrent control
group consisting of developmental controllers who would have received
only traditional training. No concurrent control group was included in
the investigation because it was not considered ethical to withhold what
was expected to be better training from a control group. A possible
alternative involved comparisons of simulation training at O'Hare to
traditional training at another tower. However, the results of such
comparisons would be suspect due to confounding with other differences
between the two towers. Instead, comparisons were made between the
trainees who used the simulation and those who had previously been
trained at O'Hare, prior to the installation of the simulation.

1.3.2 [Ongoing Training]{.underline}

The data gathered for this study were collected during the first year
the training staff at O'Hare used an OTW tower simulator to train tower
controllers. Three groups of controllers received training over the
course of this one-year study. The O'Hare training staff gave
considerable thought to the way they used the simulator and modified
their training practices in an attempt to improve the training of each
successive group. As a result, the training methods used for the three
groups differed. This led to instances where it appeared inadvisable to
combine the training data obtained from the different groups.

Another constraint on the evaluation resulted from the increasingly
difficult demands of the training program. Simply stated, more skills
were required at the end of training than at the start (see section
3.2). Due to this increasing demand, the trainees needed to increase
their skills for their performance to remain at the same level. This
constraint complicated the interpretation of data obtained from
simulation training sessions.

1.3.3 [Sample Size]{.underline}

This evaluation was based on the performance of all ten developmental
controllers who received simulation training during 1994 at O'Hare
Tower. The small size of this sample precluded conducting statistical
tests on many of the differences observed.

**2. THE SIMULATION**

2.1 DESCRIPTION

The training simulation evaluated was TOWER/Pro™, a product of Aviation
Simulations, Inc. It cost approximately \$500,000. It was set up in an
O'Hare Tower training room as shown in Figure 1. There, it was used to
train performance on the outbound ground control position. The three
visual displays showed aircraft moving about the simulated O'Hare
Airport. The displays were approximately the size of actual O'Hare Tower
cab windows, although they were partitioned somewhat differently.
Because it presented the airport in a manner virtually identical to the
airport as seen through the actual tower windows (but see the field of
view limitation described in section 2.2.1), the device is considered an
"out-of-the-window" tower simulator. The simulation showed the entire
airport including all taxiways and runways, permitting practice with all
runway configurations. The computer-generated aircraft appeared to move
in three dimensions at rates which depended upon the type of aircraft.
Aircraft crossed from display to display without interruption, much as
planes seen from the tower cab appear to move along taxiways from window
to window. The simulation included a representative mix of air carriers
and propeller-driven types of aircraft. The airport was displayed as if
seen from slightly above the tower cab because the simulation's airport
image was constructed from photographs taken from the roof of the cab.

![](media/image3.emf)

**FIGURE 1. SIMULATION EQUIPMENT LAYOUT**

The airport and aircraft images were projected from a ceiling-mounted
source that was separated by a curtain from the rest of the training
room. The ambient lighting in the training room was maintained at a dim
level while the simulation was running to minimize reductions in color
contrast on the window displays. Accurate color representation is
important because the colored logos and markings on aircraft provide
cues to an aircarrier's company, and controllers use them to identify
aircraft.

An instructor and the simulation operator provided training in the
simulated O'Hare environment. Each instructor was assigned to a pair of
developmental controllers, and instructed them during simulation
training and during OJT. While one trainee assumed the role of outbound
ground controller, the other served as a "ground meterer." The ground
meterer's role in the simulation was to set up the flight strip for each
flight as it was about to enter the simulation.

During the simulation, the trainee stood behind a flight progress strip
bay facing the three screens. He (all trainees during the one-year data
collection period were male) wore a standard headset for communication
with simulation "pilots." A camcorder used to collect data for this
study was located on the trainee's left and was focused on the trainee
to record his voice and the direction of his gaze. The ground meterer
stood to the trainee's left near a computer display which showed the
airport, current aircraft positions, and the simulation clock. The
instructor stood to the right and behind the trainee. The simulation
operator, who was also a training specialist, sat behind the instructor
at a console with aircraft movement controls and a display showing the
airport and aircraft locations. The trainee could see neither the ground
meterer's nor the simulation operator's display.

2.2 LIMITATIONS OF THE SIMULATION

Every simulation embodies limitations. Following are the limitations of
the simulation upon which this report is based:

-   partial field of view

-   automated controller speech recognition/synthetic pilot speech

-   no inbound aircraft.

2.2.1 [Partial Field of View]{.underline}

O'Hare Tower overlooks the airport on all sides, permitting controllers
to view all of the taxiways and runways, which are found on all sides of
the tower. Since the ground and local controllers must visually locate
each aircraft before providing it with movement instructions and
clearances, their scanning must include every direction. The simulation
could display all of the airport, but only 135 degrees of arc was
visible at a time through the three window displays. At O'Hare, the
ground controllers often move about the tower cab for a better view of
the airport. In the simulation, though, the trainees had to select one
135 degree segment to view at a time (Figure 2). They used a handheld
control similar to a computer mouse to rotate or "pan" the view until
the desired segment of the airport could be seen. Trainees could vary
the panning rate from approximately 45 degrees per second to 150 degrees
per second. Panning from a fixed position thus took the place of moving
around the tower. Thus, trainees needed to learn an artificial simulator
skill (panning) to participate in simulation training.

![](media/image4.emf){width="3.213888888888889in"
height="4.675694444444445in"}

**FIGURE 2. PARTIAL FIELD OF VIEW**

It was probably more difficult for the trainees to locate, identify, and
track aircraft in the simulation than in the real tower because they
needed to pan the displayed 135 degree segment of the airport to the
part that they wanted to see. Some evidence of this difficulty appeared
in a focus group conducted with the first group of trainees (see
Appendix A for a summary). When a controller scans the airport from the
real tower, he or she physically changes his or her view with eye
movements, head turns, and body movement. The perceptual system adjusts
to produce a stable view, a phenomenon known as position constancy[^4].
In the simulation, when the trainees panned the view to bring a target
onto the display, display motion was not accompanied by self-motion, so
relatively fast panning produced an unstable view. The effect of panning
was that the trainees could not identify aircraft while panning quickly,
but instead needed to either pan slowly or wait until the display
stopped.

2.2.2 [No Inbound Aircraft]{.underline}

Inbound aircraft were not simulated. Training focused on the outbound
ground control position and thus required a maximum of practice handling
outbound aircraft. Limiting the simulation to inbound aircraft also
maintained the complexity of the traffic at levels appropriate for new
developmental controllers.

2.2.3 [Automated Controller Speech Recognition/Synthetic Pilot
Speech]{.underline}

The simulation included automated speech recognition and synthetic
speech capabilities. Speech recognition was intended to accept correctly
phrased taxi instructions and then respond automatically with the
corresponding aircraft movements and an automated pilot readback using
synthetic speech. The automated system was available, but its use was
discontinued because it hindered training by placing several artificial
constraints on trainee speech. As a result, speech recognition was
discontinued during the third week of attempted use by the first group
of trainees (see Appendix A for trainee observations regarding speech
recognition). The following constraints were observed:

1.  Trainees needed to remember to pause at the end of each instruction,
    and to not pause in the middle of the instruction. This often
    interfered with a controller's speech cadence.

2.  It was impractical to train the speech recognition system to
    recognize the complete lexicon used by ground controllers to assist
    pilots. If the automated system failed to understand an instruction,
    the utterance did not elicit a simulator response. As an example, it
    did not recognize both grouped and ungrouped numbers in aircraft
    callsigns. Also, unlike real pilots, the simulation did not
    recognize common taxi instructions which identified the relative
    position of aircraft, such as "Join alfa behind the second MD80 on
    your left." When a recognition failure occurred, the trainee needed
    to rephrase and re-issue the instruction, which slowed and disrupted
    performance.

3.  The speech recognition system artificially limited the ways a
    trainee could issue instructions to multiple aircraft because each
    aircraft needed to respond before another could accept an
    instruction. This limitation interfered with training on the
    effective use of the ground control frequency.

4.  Trainees needed to "train" the speech recognition system to
    recognize their voices, requiring several days that were then not
    available for training.

5.  Synthetic pilot responses (readbacks and aircraft movements)
    occurred more slowly than in actual operations.

2.3 PRECURSOR TO SIMULATION TRAINING AT O'HARE

In 1989, the O'Hare training staff began to use a large screen
television to display a videotape of the airport taken with an 8 mm
camera. Flight progress strips were correlated to the videotape. This
non-interactive training aid was used successfully to train
developmental controllers to issue taxi instructions, sequence flight
progress strips, and make traffic calls. This precursor of simulation
training was not interactive. The training specialist could pause the
videotape to ask what the trainee would do next or how he or she would
handle a particular situation, but the trainee could not alter the flow
of traffic. Each group that was trained using this laboratory aid
completed a post-training evaluation. Many respondents found the video
"lab" to be very helpful.

**3. OUT-OF-THE-WINDOW SIMULATION TRAINING METHODS**

3.1 TRAINEES AND SCHEDULE

Ten developmental controllers completed simulation training at O'Hare
during the one-year data collection period. Seven of them later became
certified on outbound ground control. They participated in three groups.
The first two groups each consisted of four trainees, and the third
group initially consisted of three, one of whom was required to
interrupt simulation training for medical reasons. He completed
simulation training after the data collection period had ended. Of the
seven who became certified on outbound ground control, three were in
Group 1, three were in Group 2, and one was in Group 3.

Each trainee participated in the simulation as the outbound ground
controller for one hour daily, four days per week. The trainees worked
in pairs. One pair participated for two hours in the morning, and the
other pair participated in the afternoon. While one pair was in
simulation training the other received OJT in the tower. At first OJT
consisted of work at the clearance delivery and flight data positions.
As the trainees' skills advanced, they began OJT at the ground and local
control positions.

3.2 SIMULATION TRAINING PROCEDURE

Prior to each session, the simulation operator prepared an aircraft
schedule including the type and company of each outbound plane and the
time and gate where it would enter the simulation. The aircraft were one
of two components in each simulation scenario. The second component was
the Plan, which corresponded to an airport configuration. The Plan
indicated the active runways that the trainee was to assign to the
aircraft, depending upon their type, company, and gate. At the start of
a session, the simulation operator announced the Plan that was to be
applied. See Table 1 for a list of the Plans in use at O'Hare during the
year data were collected.

**TABLE 1. AIRPORT CONFIGURATIONS AND DEPARTURE RUNWAYS AT O'HARE
AIRPORT**

  ------------------------ ----------------------------------------------
  **Plan (Configuration)** **Departure Runways**

  Weird                    32L at T10, 22L

  X                        32L at T10, 4L, 9L, some 32R, 14L, 32L full
                           length

  B                        22L, 27L

  14's                     9L, 9R, 27L

  27's                     32L at T10, 32R

  9's                      4L, 9R

  Modified X               4L, 9R, 32R

  14R/9R                   9L, 22L, some 14L
  ------------------------ ----------------------------------------------

Also prior to each session, a paper flight strip was printed for each
aircraft showing its callsign, initial route of flight, gate, and the
time it was scheduled to appear at its gate. The flight strips, in
standard plastic holders, were arrayed on the ground meterer's table in
the order that the aircraft were scheduled to enter the simulation.
After the session began, the ground meterer monitored the simulation
time on his display. When an aircraft was scheduled to enter the
simulation, he set its flight strip in the trainee's (controller's)
strip bay.

An aircraft was ready to taxi in response to the simulation operator's
movement commands after it automatically pushed back from its gate and
moved toward the taxiways. The controller trainee called aircraft and
issued taxi instructions through the headset push-to-talk microphone
while marking and moving the plane's flight strip. The simulation
operator then responded as the pilot of the aircraft, reading back
instructions and taxiing the aircraft. The trainee continued to issue
verbal instructions guiding each aircraft along the taxiways until he
instructed the pilot to switch radio frequencies to a tower frequency.
As the trainee worked the simulated outbound traffic, the instructor and
simulation operator responded to his decisions with suggestions,
questions, and comments. The sessions lasted approximately 50 minutes
and were followed by a brief review.

In the simulation, the simulation operator (a training specialist) and
instructor asked questions, made suggestions, and commented on the
trainee's decisions. The simulation operator also responded as a pilot
(actually, as all of the pilots). In the pilot role, he provided verbal
readbacks and acknowledgments, taxied the aircraft and, following the
handoff to a tower frequency, taxied the aircraft to the assigned runway
and made it depart. The simulation operator paused the simulation when
extended discussion appeared necessary. The simulation was paused
infrequently (once or twice in a typical session); pauses lasted less
than two minutes.

As the trainees' proficiency increased, they were expected to
demonstrate more ground control skills and to handle more aircraft.
Session objectives became more complex with the progressive addition of
more advanced skills, as shown in Table 2. The rate at which aircraft
appeared in the scenarios was gradually increased from one aircraft
every two minutes (30 total) to one every forty-five seconds (80 total).
The airport configuration or Plan was changed daily, but Plan Weird and
Plan B were used more often in training because they were more
frequently encountered in actual operations. A change of configuration
(e.g., Plan 27's to Plan Weird) was practiced during the last week of
training.

**TABLE 2. TRAINING OBJECTIVES**

  ---------------- ------------------------------------------------------
  **Phase of       **Objectives**
  Training**       

  Early            Issue correct taxi routes, master stripboard
                   management

  Middle           Assign appropriate runway, issue correct taxi
                   instructions, master stripboard management

  Late             Assign appropriate runway, issue correct taxi
                   instructions, master stripboard management, make
                   traffic calls, handle configuration changes
  ---------------- ------------------------------------------------------

3.3 MODIFICATION OF TRAINING METHODS

As stated earlier, this study was based on data gathered during the
first year the training staff at O'Hare used an OTW tower simulator to
conduct training. They modified some training techniques over the course
of the year. For example, whereas the first two of the three groups of
trainees obtained practice on all O'Hare air carriers throughout their
training, the training methods used for the third group emphasized one
half of the airport at a time by first concentrating on the American
Airlines rush. One anticipated benefit of this training strategy was
that it would reduce the amount of panning required and thus reduce any
distration panning might have created.

A set amount of training was planned for the first two groups. In
contrast, the third group was given training until a criterion was
reached. As a result, one Group 3 developmental controller received four
weeks of simulation training, while the other received this training for
three weeks. The criterion consisted of a demonstration that the trainee
could:

\- issue correct taxi routes

\- effectively manage the stripboard

\- assign the appropriate runway

\- issue correct taxi instructions

\- make traffic calls when needed.

The simulation's automated speech recognition/pilot response system was
used only during the first three weeks of the first group's four-week
simulation training (see section 2.2.3). The second and third groups
received simulation training without automated speech
recognition/synthetic speech. Group 2 received one week more simulation
training than Group 1 because it was not necessary to take away the time
needed to "train" the automated speech recognition system from
simulation training.

**4. EVALUATION METHOD**

4.1 MEASURES OF EFFECTIVENESS

This report describes an evaluation of the first year of training using
an OTW simulator to teach the skills required to perform outbound ground
control at O'Hare Tower. Since an important goal of using the tower
simulation in training was to reduce training time, **the primary
evidence used to evaluate its effectiveness was the time required for
the simulation-trained controllers to become certified on outbound
ground control, compared to facility records.** To augment these data,
the perceptions of tower supervisors were used to compare
simulation-trained to traditionally trained developmental controllers on
a variety of ground control skills. Their ratings helped to identify the
specific ground control skills that simulation training benefited and
the strength of these skills following simulation training.

The evaluation was also based on objective evidence reduced from
videotapes of the simulation training sessions. Three types of evidence
were reduced from the videotapes:

-   Taxi Delay: how long the pilot waited before receiving taxi
    instructions

-   Taxi Time: how long aircraft were under the control of the ground
    controller

-   Scan: the proportion of time a trainee's gaze was directed toward
    the window displays or toward the flight progress strip bay.

Instructor comments found on the videotapes were quantitatively analyzed
to provide additional indications of the development of specific ground
control skills and to gain a better understanding of the simulation
training process.

4.2 VIDEOTAPED TRAINING SESSIONS

Taxi time, taxi delay, scanning, and instructor comment data were
reduced from videotapes of training sessions. One training session for
each developmental controller was videotaped each week for later
analysis. The videotaped sessions were conducted as normal simulation
training sessions. Some exceptions were made for the purposes of the
analysis: The same Plan (Plan X) was used in the videotaped sessions
during the first and last weeks of simulation training for Group 1. The
same Plan (Plan Weird) and the same aircraft rate (1.5 per minute) was
used in all of the second group's videotaped sessions.

The training staff increased the aircraft rate for Group 1 each week
during the four week simulation training period. The rate increased from
.5 aircraft per minute during the first week to 2 aircraft per minute
during the last week. The videotaped problems for Group 2 all included
90 aircraft and used "Plan Weird," one of the two most common
configurations of the eight configurations used at O'Hare (the other is
Plan X). The roughly 90 aircraft per hour rate approximates the
departing aircraft rate in an actual outbound push. These parameters
were kept constant in the recorded sessions for the purpose of this
investigation. Training specialists determined the parameters used in
the regular training sessions, depending upon the trainee's skill level
and particular training needs.

**5. RESULTS AND DISCUSSION**

5.1 POST-SIMULATION TRAINING MEASURES

5.1.1 [Days-to-Certification]{.underline}

The time (i.e., the number of days and total hours) it took
developmental controllers to become certified to work independently at
the outbound ground control position was the primary measure used in
this study. Days-to-certification was measured from the date a
developmental controller began working full time in the tower, having
completed simulator training, to the date a tower supervisor certified
him or her on outbound ground control. The hypothesis was that the
simulation-trained developmental controllers would take fewer days than
non-simulation-trained developmental controllers to become certified.
The test of this hypothesis involved a comparison between
simulation-trained controllers and non-simulation-trained controllers on
days-to-certification on outbound ground control. The non-simulation or
"traditionally trained" group had received four weeks of training in the
videotape lab described in section 2.3. Their days-to-certification was
also measured from the date they began working full time in the tower.

Only data from those non-simulation-trained controllers who began
training after January 1, 1993 were included in the no-simulation group.
Prior to January, 1993, O'Hare certified controllers on the outbound and
inbound ground control positions together. Since simulator training was
only given on outbound ground control, the days taken for certification
on that position alone were the most relevant. Thus, the seven
controllers who started training in or after January, 1993, were
compared on days-to-certification to the seven controllers who became
certified on outbound ground control following simulator training[^5].
This comparison is illustrated in Table 3.

**TABLE 3. DAYS-TO-CERTIFICATION**

  ----------------- ----------------------- ---------------------- ----------------------
                    Days-to-Certification                          

                    N                       Mean                   Standard Deviation

  Simulation        7                       148.57                 49.65

  No Simulation     7                       198.71                 52.52
  ----------------- ----------------------- ---------------------- ----------------------

Table 3 shows that the simulation-trained developmental controllers took
50.1 days less, a 25.2 % reduction in the number of
days-to-certification on outbound ground control. The difference is
statistically significant, *t* (12) = 1.84 (p \< .05), in a one-tailed
test. This finding supports the hypothesis that developmental
controllers who received simulation-training take fewer days to become
certified on the outbound ground control position than non-simulator
trained developmental controllers.

5.1.2 [Hours-to-Certification]{.underline}

The second part of time-to-certification is the total number of hours
required. The hypothesis was that the developmental controllers who
received simulation-training would take fewer hours to become certified
than non-simulator trained developmental controllers. The test of this
hypothesis involved a comparison between the same two groups of
developmental controllers that were compared in section 5.1.1.

Table 4 gives the number of hours simulation-trained developmental
controllers required for certification on outbound ground control and
the hours-to-certification required by trainees without simulator
training. They are approximately equal (a Mann-Whitney test[^6] failed
to reveal a significant difference, *p* \> .05). Therefore, simulation
training was effective in increasing trainees' skills to a point where
the trainees could acquire the needed amount of OJT in fewer days, but
it did not appear to reduce the total amount of OJT needed.

**TABLE 4. HOURS-TO-CERTIFICATION**

  ----------------- ------------------------ ---------------------- ----------------------
                    Hours-to-Certification                          

                    N                        Mean                   Standard Deviation

  Simulation        7                        84.93                  21.18

  No Simulation     7                        89.42                  12.53
  ----------------- ------------------------ ---------------------- ----------------------

It will be recalled from section 1.2 that when the density of traffic
requires a faster working speed than a developmental controller can
produce, the instructor must take over the position. The developmental
controller then loses an opportunity to increase his or her ground
control skills through practice. Accordingly, a hypothesis that would
explain the time-to-certification results is that simulation training
enables trainees to work more rapidly and thus to handle higher traffic
densities in OJT. This can be called the *working speed hypothesis*.

The working speed hypothesis suggests that with simulation training,
developmental controllers can obtain more hours of OJT per day,
resulting in a need for fewer days of OJT than without simulation
training. That simulation training did not appear to affect
hours-to-certification suggests that simulation training does not
address at least some critical ground control skills. These skills
apparently develop independent of working speed so that an increased
working speed does not reduce the number of hours of OJT needed to
develop these other skills. The post-simulation-training supervisor
ratings presented in the following section (5.1.3) and the training
session measures presented in section 5.2.5 provide evidence related to
the identification of these skills.

5.1.3 [Tower Supervisor and Area Manager Ratings of Ground Control
Skills]{.underline}

The measures of time-to-certification provide overall indices of the
benefits that can result from simulation training, but these measures do
not indicate the effect of simulation training on specific skills.
Supervisory (expert) rating data were obtained to permit an examination
of the skills exhibited during initial outbound ground control
performance following simulation training. The strengths of these skills
were compared to a baseline that represents the development of ground
control skills without simulation training.

The baseline was constructed from ratings of typical OJT progress for
traditionally trained average and fast learners on eight categories of
ground control skill. An O'Hare Tower area manager and two O'Hare Tower
supervisors made the baseline ratings. Each of the raters had worked as
O'Hare instructors, area manager, or supervisors for at least six years.
Thus, all had evaluated ground control skills at O'Hare prior to the use
of simulation training.

The baseline was constructed from the supervisors' and area manager's
*recollections.* This procedure was chosen because it was not possible
to obtain the baseline ratings from direct observation. For each skill
category, the supervisors estimated the strength of the skill for a
developmental controller with one week to 12 months experience in the
tower. They used five-point scales to make their ratings in each of the
eight skill areas. The mean baseline ratings for "average learners" and
"fast learners" and the meaning of each point on each of the 5-point
scales appear in Appendix B .

The supervisors were asked to provide ratings on the same eight skills
for the simulation-trained developmental controllers following each
group's completion of simulation training. The ratings for a trainee
were to be made as soon as possible following sufficient observation of
his work in the tower. In all cases they were completed within three
weeks following the end of simulation training for the individual, after
the supervisor had observed the trainee's ground control skills for at
least one hour. The ratings for Group 1 were made two weeks after the
supervisors and area manager provided the baseline data.

Table 5 presents the rated strengths of the eight skill categories
following one month of simulation training compared to the strengths of
the same skills following one month of traditional training. The table
presents the mean ratings for the seven developmental controllers who
became certified on outbound ground control. Table 6 shows how many
months of OJT it would take average or fast learners to reach the same
level of skill that the simulation-trained controllers attained within
three weeks following simulation training. Both baseline and simulation
groups received approximately one month of training prior to OJT.

**TABLE 5. STRENGTH OF GROUND CONTROL SKILLS FOLLOWING ONE MONTH OF
SIMULATION TRAINING COMPARED TO BASELINE**

  ----------------- ----------------- ----------------- -----------------
                    **Simulation**    **Baseline (No    
                                      Simulation)**     

  **Skill**         **Strength        **Average         **Fast Learner:
                    (1-5)**           Learner: Strength Strength (1-5)**
                                      (1-5)**           

  Visually scans    3.21              2.0               2.7
  airport surface                                       

  Maintains         3.21              2.0               1.7
  efficient traffic                                     
  flow                                                  

  Maintains         3.71              2.0               2.7
  aircraft identity                                     

  Working speed     3.71              1.3               2.7

  Effectively       2.89              1.7               2.0
  manages                                               
  stripboard                                            

  Missed/delayed    3.0               1.0               1.7
  traffic calls                                         

  Communication is  3.36              1.7               3.0
  clear and concise                                     

  Makes unnecessary 2.93              1.0               2.0
  transmissions                                         

  Overall           3.25              1.59              2.31
  ----------------- ----------------- ----------------- -----------------

Table 6 shows that according to the area manager and tower supervisors
whose impressions were obtained, the skills of developmental controllers
who received simulation training were advanced compared to the skills of
those who received traditional training. Early in OJT,
simulation-trained developmental controllers demonstrated ground control
skills considered equivalent to traditionally trained average learners
following five months of OJT.

Simulation training was particularly effective in training *working
speed*, where the mean rating of 3.71 indicated that the trainee could
work at a speed that was more than adequate for moderate workload (3.0)
and close to adequate for high workload (4.0). This finding is
consistent with the **working speed hypothesis** proposed in the
preceding section. This explanation suggested that simulation training
increased working speed, enabled the trainee to handle higher traffic
densities, and hence permitted the trainee to work traffic more hours
per day.

**TABLE 6. EQUIVALENT MONTHS OF OJT: SIMULATION TRAINING COMPARED TO
BASELINE**

  ----------------- ----------------- ----------------- -----------------
                    **Simulation**    **Baseline (No    
                                      Simulation)**     

  **Skill**         **Months of OJT** **Average         **Fast Learner:
                                      Learner: Months   Months of OJT**
                                      of OJT**          

  Visually scans    \<1               4.42              1.51
  airport surface                                       

  Maintains         \<1               5.02              3.7
  efficient traffic                                     
  flow                                                  

  Maintains         \<1               5.42              2.07
  aircraft identity                                     

  Working speed     \<1               6.67              4.02

  Effectively       \<1               4.63              2.38
  manages                                               
  stripboard                                            

  Missed/delayed    \<1               5.0               3.0
  traffic calls                                         

  Communication is  \<1               4.17              2.17
  clear and concise                                     

  Makes unnecessary \<1               4.9               2.46
  transmissions                                         

  Overall           \<1               5.03              2.66
  ----------------- ----------------- ----------------- -----------------

5.2 SIMULATION TRAINING MEASURES

Each developmental controller was videotaped once each week during
simulation training. The following objective data were then reduced from
the videotapes: taxi delay, taxi time, and the percentages of time a
trainee's scan was directed toward the "windows" or toward the
simulation stripboard.

Taxi delay and taxi time provide measures of general planning ability
and efficiency. Training was expected to result in reduced taxi delay
and taxi time. Stripboard scan time is a measure of the attention
required to stay organized by marking and moving flight strips.
Controllers are trained to minimize stripboard scanning because it takes
time from scanning the situation on the airport movement areas. Thus
training was expected to increase window scanning and to decrease
stripboard scanning.

5.2.1 [Preliminary Data Considerations and Analysis]{.underline}

The Group 3 training data were excluded from analysis for several
reasons. In Group 3, only two developmental controllers of the three who
began simulation training completed it: one needed to postpone further
training for medical reasons after three weeks. The use of
training-to-a-criterion instead of a previously set number of weeks of
training (see section 3.2) resulted in different amounts of training
given to the remaining two trainees who completed simulation training.
In view of these and other differences between Group 3 and the other two
groups, it appeared inappropriate to combine the three groups' training
data. Group 3 provided insufficient data (a total of eight sessions for
three trainees) to conduct a meaningful separate analysis of Group 3
alone.

The Group 1 data required preliminary analysis to assess any impact of
the automated speech recognition/synthetic speech system on the measures
of training. The automated system was used during the first two to three
weeks of Group 1 training (see section 2.2.3). It appeared to lengthen
taxi delays when it was operating due to the need for the trainees to
repeat taxi instructions and watch to see if indeed the aircraft began
to move as instructed. By requiring unusually lengthy attention to
individual aircraft it also was likely to have interfered with the scan
measures.

An effort was made to quantify the effect of using the speech
recognition system even though it was necessary to compare across
trainee groups. Figure 3 shows the mean taxi delay for Group 1, during
the first two weeks of automated system use (its use was discontinued
during the third week, resulting in its use by some but not all trainees
that week). The figure also shows taxi delay for the first two weeks of
Group 2 and Group 3 training, combined for comparison to Group 1. The
Week 1 scenario included 30 planes for Group 1, in contrast to 90 planes
for Group 2 and Group 3, so the apparent advantage for the automated
system is most likely an artifact. The Week 2 scenarios were more
comparable: Group 1 was given 80 planes, nearly as many as the 90 planes
given to Group 2 and Group 3. Using the Week 2 data, comparison of the
first group while using the automated system with the second and third
groups suggests that the automated system added an average of 1.77
minutes to the Group 1 taxi delay, roughly doubling this delay. Since
the automated system appears to have adversely affected the Group 1
training data for the first two or three weeks, the Group 1 data was not
analyzed further.

Due to the preceding considerations, the only training data analyzed
came from the videotaped sessions of the second group of trainees. The
four trainees in Group 2 each received simulation training for five
weeks. One session per trainee per week was videotaped. Each session
lasted approximately 50 minutes. Since the taxi delay, taxi time, and
scan measures were all based on many individual observations, these data
were considered to provide a sufficient basis for analysis.

A quantitative analysis of instructor comments on these 20 tapes
supplemented the training data. This analysis was conducted to provide
additional indications of the development of particular ground control
skills and a better understanding of the simulation training process.

5.2.2 [Taxi Delay]{.underline}

The time it takes the ground controller to provide taxi instructions to
an aircraft that is ready to taxi is its taxi delay. At O'Hare, most
aircraft receive taxi instructions in less than 30 seconds, which is
considered a "good" taxi delay time at this facility[^7]. Taxi delay
provides a general indication of a controller's ability, including the
ability to identify the gates and/or alleys used by air carriers at
O'Hare, communicate efficiently, plan efficient taxi routes, and
prioritize actions.

In the simulation, taxi delay was defined as the time interval that
began when an aircraft was metered (i.e., when the ground meterer placed
its flight strip on the strip bay) and ended when the aircraft received
taxi instructions. It did not apply to an aircraft that was handed off
by one ground controller to another ground controller (one aircraft in
the videotaped scenarios).

Figure 4 shows the mean Group 2 taxi delay, by week of training. It
demonstrates that following an initial rise, mean taxi delay decreased
monotonically from a high during Week 2 of almost 2.5 minutes to a low
during Week 5 of under one minute. This considerable decrease in mean
taxi delay was accompanied by a corresponding increase in the percentage
of taxi delays provided in at most 30 seconds (Figure 5).

5.2.3 [Taxi Time]{.underline}

Taxi time is a rough estimate of the time an aircraft spends taxiing and
so it provides an indication of fuel cost to the user as well as an
indication of how well a controller plans taxi routes. Taxi time varies
with the distance from the departure gate to the point of handoff
(usually near the departure runway), and hence with the type and company
of the aircraft. Because it also varies with the distance to the
appropriate, active departure runway, taxi time also varies with airport
configuration. Following are typical taxi times for O'Hare[^8] when the
airport is either in its "Plan B" or its "Plan Weird" configuration
(note that taxi time is reduced in the simulation due to the absence of
inbound aircraft):

Departure Runway Taxi Time (Minutes)

United

Eastbound 22L 8.5

Westbound 32L 3 to 4

American

Eastbound 22L 6

Westbound 32L 8

General Aviation

Eastbound 22L \<1

Westbound 27L 9 to 10

As measured in the simulation videotapes, an aircraft's taxi time began
when it was given taxi instructions and ended when it was handed off to
the local controller. The airport configuration or Plan was held
constant (Plan Weird) in the Group 2 sessions videotaped.

Figure 6 shows the mean taxi times by week of training for Group 2. It
is evident that taxi time remained relatively stable during the five
weeks of training. The taxi times measured were considerably lower than
most of those considered typical at O'Hare. It is likely that the times
were low in part because no inbound aircraft were simulated. It is not
clear why taxi time did not reflect the benefit of simulation training;
however, it is possible that

\- taxi time depends mainly on the distance from the departure gate to
the appropriate runway, with little effect of controller skill.

\- the ground controller has more flexibility in how aircraft are handed
off to the local controller, the event which closed taxi time in this
study. Depending upon the situation, a handoff could occur in the same
message as a taxi instruction, or it could wait until a later time when
the trainee had an opportunity to make several handoffs in a single
transmission. Time-to-runway (if measured) might show better sensitivity
to training effects than time-to-handoff.

\- the longer taxi time durations, compared to the length of taxi
delays, made them more likely to include the time taken by instructors'
assistance and comments.

\- increases in skill were counteracted by increases in training program
demands. Some of the later training demands such as traffic calls and
splitting fixes could affect taxi time more than taxi delay.

5.2.4 [Window and Stripboard Scan]{.underline}

This pair of measures describes the length of time each trainee's visual
scan or gaze was directed toward the simulation's visual displays (i.e.,
toward the simulated tower windows), the stripboard, or elsewhere in the
training room (typically toward an instructor). Window scanning and
stripboard scanning were of particular interest because of anecdotal
reports that some trainees spend too much time looking at flight strips
and consequently insufficient time looking through tower windows at the
aircraft movement areas.

Stripboard management began to require less visual attention after the
second week of training. It decreased between the second and fifth weeks
from 51.4% to 44.8% of the total time. Window scanning began to increase
after the third week from 44.0% to 50.2% of the total time. The
direction of these moderate changes in gaze is consistent with the
hypothesis that simulation training improves stripboard management
skills. This evidence is presented in Figure 7.

5.2.5 [Analysis of Instructor Assistance]{.underline}

The constraints under which this study was conducted (see section 1.3)
require a research strategy in which different types of evidence are
examined for consistent trends. Accordingly, videotaped instances of
instructor assistance were analyzed to provide converging evidence on
the development of specific ground control skills.

Instructors made 782 discernible suggestions and comments during the 20
Group 2 training sessions that were videotaped and used in this
evaluation. This instructor advice was examined to determine whether the
trainees required less advice as training progressed. Each instructor
comment was classified according to topic to provide additional
indications of the development of particular ground control skills.

The instructor comments were classified into twelve categories. Six were
designated as "major categories" and six were designated as "minor" on
the basis of frequency. The major categories included 90.15% of the
comments. For each week of training, the mean amount of advice (number
of comments) in each of the major and minor categories is shown in Table
7. Figure 8 shows the effect of training on the average amount of
assistance (per major category) trainees required to perform the
problem. A trainee required roughly half as much assistance by the end
of simulation training.

Table 7 shows reductions in advice on stripboard management and taxi
routes, which appear to have reached asymptotes (minimum levels) by the
end of simulation training. Priority of duty, (i.e., knowing which
aircraft to call next) required instructor comments primarily during the
first two sessions, after which comments appeared at a much reduced
rate. Situational awareness is the only major topic that appears to have
required a fairly constant level of instructor attention throughout
training.

**TABLE 7. INSTRUCTOR ASSISTANCE BY WEEK OF TRAINING (MEAN NUMBER OF
ASSISTS)**

  -------------------------------------- ------------ ------- ------- ------- -------
  **Categories Of Assistance**           **Weeks of                           
                                         Training**                           

  **Major Categories**                   **1**        **2**   **3**   **4**   **5**

  Stripboard Management                  10           6.25    4.5     3.5     3.25

  Situational Awareness                  5.25         5.25    6.75    4.5     4.75

  Priority of Duty                       10           9       3.25    3       2

  Communications                         9.75         10.25   7.5     8.75    6

  Taxi Routes                            9.75         9       7.25    5.5     5.5

  Traffic Calls                          4.75         3.5     3.5     2.75    1
  -------------------------------------- ------------ ------- ------- ------- -------

  -------------------------------------- ------- ------- ------- ------- -------
  **Minor Categories**                   **1**   **2**   **3**   **4**   **5**

  Splitting Fixes                        .5      3       1.75    .75     1.75

  Timely Taxi Instructions               .5      1       .25     1.25    .75

  Handoffs to Local Controller           .75     .25     0       0       0

  Aircraft Identification                .25     .75     .25     1       .25

  Sequencing Aircraft                    .75     .25     1.25    .5      .5

  Runway Assignment                      .5      0       .25     .5      0
  -------------------------------------- ------- ------- ------- ------- -------

As training progressed, instructors made fewer comments on the
developmental controllers' communications. The highest number of
comments were made during the first and second sessions. Compared to an
average of approximately ten comments made in those initial sessions,
the final session shows an approximate 40% reduction. Over the same
period, the trainees increased the number of communication errors they
caught and corrected themselves by more than 60%. Figure 9 presents
these results.

**6. FINDINGS AND RECOMMENDATIONS**

This section summarizes and draws conclusions from the evidence
presented in the preceding section. This evidence was gathered during
the first year of training using a prototype out-of-the-window tower
training simulator at O'Hare International Airport in Chicago. The
simulator was used to train the skills and knowledge required to perform
outbound ground control at O'Hare. The evidence supports the following
conclusions:

**The out-of-the-window tower simulation used at O'Hare:**

-   **is an effective tool for training many ground control skills and
    knowledge; it appears especially effective in increasing trainees'
    working speed.**

```{=html}
<!-- -->
```
-   **is more effective for training ground control than more
    traditional training techniques.**

6.1 TRAINING USING THE SIMULATOR WAS EFFECTIVE

During training, the ground control performance of the developmental
controllers studied consistently improved, beginning with the second
week of training. Following a month of simulation training, their ground
control performance (in the actual tower) was rated as better than the
performance of traditionally trained developmental controllers (see
section 6.2). The following findings indicate improvements during the
simulation training sessions, but do not permit comparisons between
simulation and traditional training. The findings from comparisons
between simulation and non-simulation training (listed in section 6.2)
suggest that the improvements in ground control skills described in this
section were due either to simulation training alone or to a combination
of simulation and concurrent non-simulation training.

*Finding: Taxi delay decreased consistently from the second week to the
end of training.*

The evaluation found a 50% decrease in average taxi delay duration, with
a tripling of the percentage of taxi delays under 30 seconds in a
comparison of the second and fifth (last) weeks of training. The
monotonic downward trend was evident during each intervening week.

*Finding: Stripboard scanning began to consistently decrease after the
second week of training, while window scanning began to increase after
the third week of training.*

Moderate improvements in these two objective measures were found.
Stripboard management began to require less visual attention after the
second week of training, and ultimately decreased from 51.4% to 44.8% of
the total scan time, a 7.7% decrease. Window scanning increased after
the third session, from 44.0% to 50.2%, a 6.2% increase.

*Finding: The amount of assistance needed for a trainee to properly
handle a scenario decreased consistently throughout training.*

The evaluation found a decrease of approximately 50% in the number of
instructor assists from the first to the last week of training. The
number of assists decreased in every major category of ground control
skill. The major categories included Stripboard Management, Situational
Awareness, Priority of Duty, Communications, Taxi Routes, and Traffic
Calls. Stripboard Management and Priority of Duty showed the largest
decreases.

6.2 TRAINING USING THE SIMULATOR WAS FASTER AND MORE EFFECTIVE THAN
TRADITIONAL TRAINING

Developmental controllers became certified on the outbound ground
control position in fewer days following a month of simulation training.
Supervisors' ratings of their ground control skills indicated that they
were further advanced, compared with the results of traditional
training.

*Finding: The developmental controllers who were trained using
simulation became certified on the outbound ground control position in
fewer days, but required about the same total number of hours, as
developmental controllers who were trained without simulation.*

Using facility records, comparisons were made on the number of days and
the number of hours of OJT taken to become certified on the outbound
ground control position. The evaluation found that it took the
simulation-trained developmental controllers an average (mean) of 149
days to become certified on the outbound ground control position. In
contrast, it took the developmental controllers who became certified on
outbound ground control prior to simulation training 199 days, a
statistically significant difference of approximately 25%. The two
groups involved in these comparisons had received approximately the same
amount of training prior to OJT.

The difference in days-to-certification found between developmental
controllers trained using simulation and those trained without
simulation did not appear to extend to total hours-to-certification. The
evaluation found that simulation-trained developmental controllers took
only 5% less time in terms of total hours-to-certification. This pair of
results provides part of the evidence favoring a working speed
hypothesis.

***The Working Speed Hypothesis:***

*Simulation training increased developmental controllers' working speed,
which enabled them to work in the actual tower under a wider range of
conditions (i.e., under heavier or more complex traffic) and hence for
more hours per day than with traditional training. Thus, simulation
training decreased the number of days but not the total number of hours
needed for certification on position.*

Additional evidence in support of the working speed hypothesis is found
in the supervisor ratings of eight skill categories, where working speed
was rated highest compared to a baseline of traditional training. It
received a mean rating of 3.71, which indicates that the trainee could
work at a speed that is more than adequate for moderate workload (3.0)
and close to adequate for high workload (4.0). Baseline ratings for
controllers given traditional training were 1.3 for average learners and
2.7 for fast learners. The working speed hypothesis is also consistent
with the finding of a 50% drop in taxi delay during the course of
simulation training.

That simulation training did not appear to affect hours-to-certification
suggests that simulation training did not address at least some critical
ground control skills (see the following section 6.3). These skills
apparently develop independent of working speed so that an increased
working speed does not reduce the number of hours of OJT needed to
develop these other skills.

*Finding: Supervisor ratings made after the completion of simulation
training were higher than the ratings of traditionally trained
developmental controllers*.

In general, the ratings made by tower supervisors indicated that within
three weeks following simulation training, developmental controllers
exhibited ground control skills in the actual tower comparable to a
traditionally trained "average learner" after five months of OJT or to a
"fast learner" after 2.7 months of OJT. The supervisors rated all eight
categories of ground control skill higher following simulation training
than following traditional training, even when compared to fast
learners. Simulation training appeared most effective in increasing
working speed.

6.3 TRAINING USING TOWER SIMULATION IS LIKELY TO SHOW INCREASED BENEFITS
UPON UPGRADING

One difficulty of traditional OJT for outbound ground controllers is
that, at times, the volume of traffic and its complexity can become so
high that the trainee cannot proceed without potentially compromising
the safe and efficient flow of aircraft to their runways. At these
times, the trainee must leave his or her post and allow the instructor
to take over. This reduces the amount of time for OJT on that day. The
decreased taxi delays and higher supervisor ratings found in this study
are consistent with the notion that simulation training increased the
working speed of developmental controllers to the point where they could
benefit from OJT even under conditions of moderate-to-high density and
complexity.

The 25% difference in days-to-certification following simulation
training was not accompanied by a commensurate reduction in
hours-to-certification. These results suggest that some crucial ground
control skills are not learned in simulation training as currently
implemented. To the extent that these skills can be identified and
simulation training enhanced, extended[^9], or upgraded to support
teaching them, one can expect additional benefits. Recommendations in
this section are offered under the assumption that the FAA decides to
proceed with tower controller training simulation.

The evidence obtained in this evaluation suggests that upgrades are
required to train or to increase the effectiveness of training on the
following skills:

1.  Inbound ground control and coordination between inbound and outbound
    ground control positions

2.  Reflexive, correct communications

3.  Scanning and situational awareness

4.  Understanding what a pilot can see from the cockpit

5.  Smooth transition between window scan and BRITE/ASDE displays

6.  Teamwork

*Inbound ground control and coordination between inbound and outbound
ground control positions.*

Without inbound aircraft, the simulation could not provide the
complexity of the taxi patterns at O'Hare. This limitation reduced the
maximum difficulty of the training problems, and it thus reduced the
number of weeks the simulation could present challenging problems for
training. **All of the upgraded capabilities listed below that involve
extending the duration of simulation training require a capability for
inbound aircraft.**

Ground control training conducted in a simulation lacking inbound
aircraft could have led to the development of unrealistic expectations
of the pace, decisions, and actions needed to move from one point to
another because the outbound aircraft did not have to stop at the
runways and taxiways used for arrivals. Also, without inbound aircraft,
trainees did not have an opportunity to learn the coordination skills
needed to work outbound ground control in tandem with an inbound ground
controller.

Recommendations:

-   Upgrade simulation technology to support inbound aircraft

-   Upgrade simulation technology to include an additional position to
    be used for inbound ground control

-   Extend the duration of simulation training.

*Reflexive, correct communications.* It appears that extended simulation
training could improve trainee communication skills. As the emphasis
during the month of simulation training changed from fundamental to
advanced skills, so did the content of controller communications and
their associated phraseology. This continual change in what was required
of trainee communications could have produced the finding that more
instructor assistance was provided in the communications category than
in any other category during the last recorded training session. A
related finding was that the total number of corrections in the
communications category (self-corrections and instructor corrections)
remained fairly constant throughout training. A trend toward less
instructor assistance on communications after the second week of
training suggests that trainees were learning better communication
skills, but at end of simulation training, communications skills were
among the least advanced, compared to traditional training, in
supervisors' ratings. Together, these findings suggest that additional
training could further promote the reflexive, correct communications
that controllers require at airports like O'Hare, which are
characterized by high aircraft density and complex traffic patterns.

Recommendation:

-   Extend the duration of simulation training.

*Scanning and situational awareness.* Some results indicated that
technical and/or procedural aspects of simulation training could be
enhanced to increase the amount of time developmental controllers scan
the airport. Objectively, scanning out the simulation windows increased
6.2% during the last two weeks of training. During this time, the amount
of instructor assistance on situational awareness decreased, but it
remained almost as high as at the start of simulation training. In
supervisor ratings made soon after the completion of simulation
training, scanning the airport was a skill rated less than average in
strength, and less advanced compared to the other skills rated. These
results suggest that scanning and situational awareness began to improve
relatively late, and could benefit from additional simulation training.
Possibly, the need to pan the displayed view of the airport may have
slowed the integration of scanning into larger units of ground control
skill, and may have limited the transfer of scanning skills from the
simulation to the actual tower.

Recommendations

-   Enhance the simulation to simultaneously display the entire airport.

-   Extend the duration of simulation training.

*Understanding what a pilot can see from the cockpit.* Situational
awareness for ground and local controllers includes an awareness of what
pilots are likely to see at any given moment. This awareness is
important when communicating instructions which commonly refer to
aircraft relative to the pilot's position, as in "Join alfa behind the
MD80 off your left." Situations arose during simulation training which
suggested that ground controllers need training on what pilots are
likely to see. For example, an instructor asked, "Which company do you
think he's looking at?" On another occasion, an instructor commented on
a trainee's instruction: "He may never see American off his left." A
simulation capable of showing a realistic pilot's eye view could advance
training on this aspect of situational awareness.

Recommendation

Provide a "pilot's eye view" capability that will realistically
represent the view of a pilot from the cockpits of particular types of
aircraft.

*Smooth transition between window scan and BRITE/ASDE displays.*

Developmental controllers could use current tower simulation to learn to
translate aircraft locations as displayed on BRITE and ASDE systems
from/to their locations as seen through the tower windows. Training in
this advanced skill could lead to increased situational awareness.

Recommendations

-   Upgrade the simulation technology to provide an interface to BRITE
    and ASDE displays showing aircraft coordinated to the simulation.

-   Extend the duration of simulation training.

*Teamwork.* The simulation permits training for only one tower position.
With enhancements, the simulation could be used to train developmental
controllers to participate in the coordinated team effort required to
control aircraft at O'Hare. Teaching teamwork and coordination skills
requires the simulation to simultaneously show the entire airport. This
capability would enable two or more controllers to view different parts
of the airport at the same time.

Recommendations

-   Upgrade simulation technology to support inbound aircraft.

-   Upgrade simulation technology to include an additional position for
    inbound ground control or local control.

-   Enhance the simulation to simultaneously display the entire airport.

-   Extend the duration of simulation training.

6.4 CURRENT COST OF RECOMMENDED UPGRADING

The recommended upgrades are all currently available from the U.S.
manufacturer of the prototype, Wesson International (Aviation
Simulatons, Inc. is a joint venture of Wesson International, Inc. and
the British Bruce Artwick Organization, Ltd.). Costs of the personal
computer-based system depend primarily upon the number of controller or
pseudo pilot/simulation operator stations, and on the number of
projection screen displays. All current systems include simulated BRITE
and ASDE displays and "pilot's eye view" capabilities. A system that
includes all recommended upgrades (eight displays for a 360 degree view
and three positions - two controller stations and a pseudo
pilot/simulation operator station) would currently cost \$1,050,950. A
new system representative of the one evaluated at O'Hare (three displays
for a 135 degree view, and one controller position, and a pseudo
pilot/simulation operator station) would currently cost \$495,950.

6.5 NEW APPLICATIONS OF TOWER SIMULATION TECHNOLOGY

Simulation training requires use of the simulator for roughly six months
per year. Out-of-the-window ATC tower simulators offer additional
potential uses which could be scheduled during the rest of the year,
when the simulator is not required for training. The first topics
concern advanced training for current full performance level (FPL)
controllers. Sections follow on the use of tower simulation for
candidate screening, and for the assessment of new tower training
methods and technology and the evaluation of new automation systems,
procedures, and airport configurations.

*Individual performance enhancement for current controllers.* The
simulation evaluated does not appear able to provide sufficient volume
and complexity to increase the skills of current FPL controllers at
O'Hare or at other high density or high complexity airports. It would be
essential to add a large volume of inbound aircraft to increase the
difficulty of the training scenarios if training is to benefit current
FPL controllers. This training could possibly enable them to provide
better services and safety and to develop strategies for handling more
traffic.

*Team performance enhancement for current controllers.* The current
one-position simulation does not provide opportunities to train tower
controllers to work more effectively together as a team. A
multiple-position simulation with sufficient volumes of inbound and
outbound aircraft would permit practice and training on situations
requiring coordination and teamwork. The objectives of this training
would be similar to those of crew resource management or CRM[^10]. For
example, they might include improvements in end-of-shift briefings,
workload distribution, situational awareness, management of abnormal
situations, and error surveillance.

*Training in the handling of unusual situations*. Using simulation, it
would be possible to train tower controllers to recognize and properly
handle situations rarely or infrequently seen in actual operations.
Without operational experience, simulation would appear to offer the
best way for controllers to learn to contend with the overall
consequences of such situations for airport traffic control. Such
situations include emergency landings requiring passenger evacuation,
hijacked aircraft or other aviation security events, landings of
diplomatic aircraft, aircraft taxiing on runways incapable of supporting
their weight, fuel spills, birds or other wildlife on runways, unusual
and dangerous airport weather (e.g., icy runways, windshear emergencies,
or tornadoes), mechanical failures on aircraft, emergency medical
flights, and aircraft with similar callsigns approaching a runway.
Procedures for many of these situations appear amenable to simulation
training and practice.

*The optimal conditions for training*. Out-of-the-window tower
simulation technology offers the ability to control the training
environment to a greater extent than the operational environment
permits. The training staff at O'Hare took advantage of this new
capability during the first year of simulation training by tailoring the
volume of aircraft and the amount of guidance presented to the
individual, and by modifying their training techniques between groups of
trainees (see section 3.3).

This ability to control the training environment could lead to further
efforts toward optimizing tower controller training. Training
specialists could apply laboratory findings on the effects of
experimental training conditions. For instance, in one study (Vidulich,
Yeh, and Schneider, 1983[^11]), a group of subjects that practiced a
simulated air traffic control task that ran at twenty times real time
performed better in some aspects (and the same in all other areas) than
a group that practiced at real time. Algorithms for time-compressed
information presentation have recently been developed[^12]. The
possibility of applying the lessons of such developments clearly depends
upon the availability of adequate tower simulation facilities.

*Tower controller candidate screening.* Training resources are best
allocated to the most highly qualified candidates. Current selection
procedures consider only how a candidate appears on paper. The tower
personnel who must select among candidates for a limited number of
training slots have no direct knowledge of how well the candidates can
perform under realistic task demands. They also lack clear evidence of
motivational factors which can lead a candidate to expend special effort
at preparation, such as studying the layout of the airport and
surrounding airspace. In a realistic simulation, candidates can
demonstrate their current knowledge and skills without affecting
operations. Ideally, the tower staff would observe a candidate's
demonstration of prerequisite knowledge and skills, and the ability to
benefit from training in a simulation screening process which extends
over several days. A draft document which describes the organization of
a training candidate screening process using simulation is available
from O'Hare Tower[^13].

*Assessment of new tower equipment, procedures, and airport
configurations: Cost-benefit studies.*

New tower automation systems, new procedures such as coded taxi routes,
and new airport configurations (often involving new or extended runways)
promise to increase airport capacity and to improve airport safety and
the services provided by tower controllers. Assessment is needed to
determine the extent to which the goals of each new system, procedure or
configuration are reached, to determine its effect on the overall tower
environment, and to assess proposed design modifications. Although the
assessment of each new system can include unique considerations,
controllers' visual, manual, pilot communications and coordination
workload, and their response times, errors, decision making, and
situational awareness are common considerations. Such assessments must
first be made in the context of the current tower environment to provide
a baseline for comparison with the new automation system, airport
configuration, or procedure.

Some pertinent measures cannot be made in an operational environment
because they could interfere with tower operations or because the
necessary control over the situation is lacking. Examples include
testing controller response time to identify the cause of alarms and
respond appropriately, to identify and correct a controller error, or to
identify and correct pilot deviations from controller instructions. In
these cases, the initiating events occur too rarely in actual operations
and would have to be simulated. It would also appear impractical to test
the actual consequences of automated taxi route decisions, or to
determine whether controllers can safely monitor them. The measurement
of situational awareness technically requires stopping the situation to
collect data[^14]. Other measures, such as the frequency of manual
activity and data input errors, are best measured automatically through
direct connections to the device, connections which would require
fabrication. In general, the most accurate and relevant measurements can
be made in an operational situation, in the presence of realistic
workload, distractions, time demands, and stress. However, as these
examples suggest, the operational situation may not be suitable to the
required measures, requiring instead the use of global or subjective
measures which could lead to less accurate conclusions.

Out-of-the-window tower simulation could provide a more convenient and
practical testbed for the assessment of new tower automation systems
than operating towers. It would appear particularly useful in the
assessment of automation systems to be used by local and ground
controllers, whose work depends upon viewing aircraft movement areas. A
list of recommended requirements for an assessment simulator is
presented in Appendix C.

**APPENDIX A: FOCUS GROUP SUMMARY**

This focus group was conducted with four developmental controllers in
the first group after they had 7 to 10 hours working at the outbound
ground control position, following 20 hours (4 weeks) of simulation
training. Following is an edited summary of the session.

*In what ways is actual ground control different from the simulation?*

\- Because of the need to scroll the simulation it is harder to keep
track of AC in peripheral vision and to know when to transfer the AC.

\- Simulation workload is constant because it presents planes at a
regular pace, unlike the real tower where the workload occurs in
\"clumps\". There is no \"breathing time\" in the simulation.

\- The simulation\'s graphic resolution is sometimes \"grainy\" and its
colors less intense, adding a half second to the time it takes to
identify aircraft. For example, one must reason from type (737) to
company (not American, therefore United) to check yourself; upstairs you
would know instantly.

\- More small planes, Eagles and Gulfs, in actual traffic than in
simulated. Maybe needs updating.

*In what ways was the simulation an effective teaching tool?*

\- Learning taxi routes.

\- It eliminates worry about real aircraft moving around and the
training specialist stepping in to fix something instantly or to make
something work. It provides a less pressured, more relaxed environment
instead of needing to fix it instantly.

\- It permits control over the pace of the problem. Upstairs when its
slow its too slow; when busy it is too busy; there seems to be no happy
medium.

*Was the pace of the simulation optimally challenging throughout all
four weeks?*

\- It began a little too slow at 1 plane per minute, and too fast at the
end (1 per 30 sec) when learning to make splits. The speed jumped up
sometimes before I caught on.

*If there was another month of simulation what would you like to cover?*

\- Splits, and more varied scenarios, including surges and rest,
different planes and different rushes, instead of always starting up
west and ending up with west.

-Rehearsal of actual rushes and regularly scheduled flights in all
configurations.

*Any advice for the next group of developmentals to receive simulation
training?*

\- Focus on managing stripboard.

\- Don\'t worry about making mistakes or looking stupid, and ask
questions instead of pretending to understand.

*How might the simulation be used better?*

\- Instructors should use the pause feature more to allow evaluation of
what has just occurred, in smaller chunks, identifying problem and
solution.

\- Speed up the problems at the beginning to make them more realistic.

\- Reduce the speed of a problem when first increasing its complexity.

\- Use 1 or 2 instructors full time instead of many instructors because
they teach different techniques for doing the same thing.

*According to the trainees, the following topics are learned better with
live traffic than through simulation training (although with
\"additional tweaking\" the simulation could help with these):*

\- Split fixes.

\- Relative taxi speeds.

\- Location: It is easier to see planes waiting for departure on the
approach end of a taxiway in the real tower, than on the simulator. -
Visual scan: One learns to move around in the actual tower, whereas one
always looks straight ahead in the simulation. One does learn where to
scan in the simulation.

\- Aircraft identification: The simulation picture is not sharp enough
to identify companies.

*Did the simulation help you learn\...*

*Efficient traffic flow?*

\- Yes. It provides practice on maintaining continuous attention and not
wasting time

*Making traffic calls?*

\- No. \[sorry, I did not ask for elaboration\]

*Increased working speed?*

\- Yes.

*Coordination with inbound ground controller?*

\- No, because there was no inbound traffic to coordinate.

*Flight strip marking and stripboard management?*

\- It provided good help, although more work is needed to get the
instructor, student, and simulation working at the same speed.

*Phraseology and correct frequencies?*

\- It was helpful for learning to use correct frequencies. The voice
recognition was too picky, failing to accept an instruction if the
student said \"the\" instead of \"a\", spoke too loud, soft, or fast. It
distracted from what the student was trying to learn.

*Dealing with similar callsigns?*

\- It never came up.

*Avoiding unnecessary verbiage?*

\- The training helped, but not the simulation\'s voice recognition.

*Any other aspects of your present work with actual traffic where you
are finding that the simulation helped?*

\- The location of gates and where to look for airplanes calling to taxi
out. It sped up the learning process.

\- \"I can\'t really put it in numbers, but I feel a lot better about
approaching the ground control learning after a month here\.... I feel
well ahead of the game here.\"

\- \"I think it was very beneficial\...I think it helped in all
areas\...it greatly helped with stripboard management, stripmarking, and
phraeology, not from voice recognition, but from practice and
repetition, saying the routes over and over.

*How did the speed of the simulation compare with actual traffic?*

\- The speed of the simulation problems was in same range over slow
periods, but it did not get as fast as it does upstairs; the 7:30 rush
does not compare to the simulation.

\- It is hard to compare without inbounds in the simulation because
inbounds can become confused with your \[outbound\] traffic (which adds
complexity). \"You are looking at more than your guys upstairs; you are
looking at everything.\"

*What improvements would be most important?*

\- A 360 degree screen would help the most.

\- Simulation operators to run remote positions; one to run inbounds,
another to be the pilot for the outbounds, and then there would be the
instructor, watching. You could set up traps where certain pilots miss
their instructions.

\- Voice recognition would be valuable if it allowed a student to
practice without imposing on others\' time. This would allow for the
additional practice needed to develop habits which in turn would
translate to time saved. Four weeks were not enough to develop
stripmarking and stripboard management skill habits; \"I didn\'t get to
writing the number on the strip, looking out the window and moving it,
all without thinking. I still had to think about writing the number on
the strip, moving the strip, and sequencing it, and figure out which
one\'s an American and which one\'s a United and putting them between
those two. The habits didn\'t come for me as fast as they said they
should because if you do it only one hour per day for 20 days\....\"

**APPENDIX B: BASELINE DEVELOPMENT OF GROUND CONTROL SKILLS**

**"Average Learner"**

  ------------------------------- ------ ------ ------ ------ ------ ------ ------ ------
  **Ground Control Skill**        **1    **1    **2    **4    **6    **8    **10   **12
                                  wk**   mo**   mo**   mo**   mo**   mo**   mo**   mo**

  ------------------------------- ------ ------ ------ ------ ------ ------ ------ ------

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Visually scans airport surface  2.0   2.0   2.7   3.0   4.0   5.0   5.0   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Maintains efficient traffic     1.3   2.0   2.0   2.7   3.7   4.3   4.7   5.0
  flow                                                                      

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Maintains aircraft identity     1.7   2.0   3.0   3.0   4.0   4.7   4.7   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Effectively manages stripboard  1.3   1.7   2.3   2.7   3.7   4.0   4.7   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Working speed                   1.3   1.3   1.7   2.7   3.3   4.3   4.7   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Very Slow 2=Adequate for Low Workload 3=Adequate for Moderate Workload

4=Adequate for High Workload 5=Adequate for Very High Workload

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Missed/delayed traffic calls    1.0   1.0   1.3   2.7   3.3   4.3   4.3   4.7

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Many 2=Some 3=Few 4=Almost None 5=None

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Communication is clear &        1.7   1.7   2.3   3.3   4.0   4.7   4.7   4.7
  concise                                                                   

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Makes unnecessary transmissions 1.0   1.0   1.7   2.3   3.7   4.0   4.3   4.7

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Many 2=Some 3=Few 4=Almost None 5=None

"Fast Learner"

  ------------------------------- ------ ------ ------ ------ ------ ------ ------ ------
  **Ground Control Skill**        **1    **1    **2    **4    **6    **8    **10   **12
                                  wk**   mo**   mo**   mo**   mo**   mo**   mo**   mo**

  ------------------------------- ------ ------ ------ ------ ------ ------ ------ ------

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Visually scans airport surface  2.0   2.7   3.7   4.0   4.7   5.0   5.0   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Maintains efficient traffic     1.3   1.7   2.7   3.3   4.3   5.0   5.0   5.0
  flow                                                                      

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Maintains aircraft identity     2.3   2.7   3.7   4.0   4.7   5.0   5.0   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Effectively manages stripboard  2.0   2.0   3.0   3.7   4.7   5.0   5.0   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Working speed                   2.0   2.7   2.7   3.7   4.0   4.7   5.0   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Very Slow 2=Adequate for Low Workload 3=Adequate for Moderate Workload

4=Adequate for High Workload 5=Adequate for Very High Workload

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Missed/delayed traffic calls    1.3   1.7   2.7   3.3   4.0   4.3   4.7   4.7

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Many 2=Some 3=Few 4=Almost None 5=None

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Communication is clear &        2.3   3.0   3.3   4.7   4.7   4.7   4.7   5.0
  concise                                                                   

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Never 2=Seldom 3=Often 4=Almost Always 5=Always

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----
  Makes unnecessary transmissions 2.0   2.0   2.7   3.7   4.3   4.7   4.7   5.0

  ------------------------------- ----- ----- ----- ----- ----- ----- ----- -----

1=Many 2=Some 3=Few 4=Almost None 5=None

**APPENDIX C: OUTLINE OF RECOMMENDED REQUIREMENTS FOR A TOWER SYSTEM
ASSESSMENT SIMULATOR**

[A. Display]{.underline}

1.  Room space sufficient to replicate largest tower

2.  Moveable realistic-size window displays each showing the view from
    one side of the actual tower

3.  All tower window views displayed simultaneously

4.  Photographic color images of entire airports, readily updatable

5.  Color photographic images of aircraft of types and companies
    appropriate to airport, readily updatable

6.  Inbound and outbound aircraft in local controller airspace at
    realistic speeds which vary with aircraft type

7.  Aircraft taxiing in and out at realistic speeds which vary with
    aircraft type

8.  Capable of exchanging the photograph of one airport for another
    airport

9.  Maximum volume of aircraft equal to future enplanements predicted
    for simulated airport

10. Aircraft and gates individually scheduled

11. Ability to remove, modify, and rename current taxiways and runways
    and to create new or extended taxiways and runways

12. Sessions recorded for playback

[B. Control (by human operator or automated system)]{.underline}

1.  Individual aircraft movements along specified taxiways and runways

2.  Takeoff/landing roll and liftoff/landing at realistic runway
    positions and speeds

3.  Verbal responses transmitted through controller headsets

[C. Positions (include furniture and devices)]{.underline}

1.  ATC (8): two ground control, two local control, flow manager, flight
    data, clearance delivery, supervisor, and

2.  Simulation operator/pilot positions (4): two respond to local
    control; two respond to ground control, or

3.  Automated speech recognition/synthetic pilot and one simulation
    operator who can override the automated pilot systems

[D. Interface to systems under test]{.underline}

1.  Actual current and new system user interface

2.  Capable of simulating inputs to current or new systems

3.  Capable of simulating faults in new systems

4.  Adaptable to new automation system interfaces

5.  Quickly reconfigurable with different combinations of equipment

[E. Interface to controller performance data logging
equipment]{.underline}

1.  Source of input (device used)

2.  Identity of data or command correctly or incorrectly entered

3.  Time-logging of all data collected, synchronized to simulation

[F. Additional data collection]{.underline}

1.  Videotape from each window display location, synchronized to
    simulation

2.  Audiotape from each control position, synchronized to simulation

3.  Record of simulation operator commands and time entered

[^1]: . Cover photograph of TowerPro™ courtesy of Wesson International,
    Inc.

[^2]: . FAA. (Sept. 1994). Terminal Area Forecasts FY 1993-2010.
    FAA-APO-94-11.

[^3]: . FAA. (1994). Aviation Capacity Enhancement Plan.
    DOT/FAA/ASC-94-1.

[^4]: . Matlin, M.W. (1988). Sensation and Perception (2nd Ed.). Allyn
    and Bacon.

[^5]: . Three developmental controllers were certified on outbound
    ground control following the year in which the data were collected.
    Including this data would have produced a mean of 162.6 days and a
    standard deviation of 49.85 days.

[^6]: . This nonparametric test was used because the homogeneity of
    variance assumption underlying the more powerful parametric t-test
    was not met, as indicated by the standard deviation data presented
    in Table 4.

[^7]: . Matt Dunne. FAA O'Hare Tower. Personal Communication.

[^8]: . Matt Dunne. FAA O'Hare Tower. Personal Communication.

[^9]: . Many of the recommended simulation enhancements and new
    application areas (section 6.4) involve extending simulation
    training. This would increase training staff time and costs. The
    simulation's speech recognition and synthetic speech capability was
    intended to enable a trainee to practice without a simulation
    operator continually present. When the simulation speech system was
    operating, aircraft would automatically follow their assigned
    routes, and pilots would read back instructions and clearances
    automatically, using synthetic voice. Problems encountered in the
    use of the evaluated simulation's speech recognition and synthetic
    speech systems are described in section 2.2.3. If these problems are
    solved, autonomous capabilities could reduce the scheduling and
    financial costs associated with some of these enhancements and
    applications. Autonomous simulation capabilities could also increase
    the accuracy of assessments made in a simulated tower environment by
    increasing control over test conditions.

[^10]: . Taggart, W.R (1994). Crew resource management: Achieving
    enhanced flight operations. In N. Johnson, N. McDonald, and R.
    Fuller (eds), Aviation Psychology in Practice, Aldershot, Hants,
    U.K., Averbury.

[^11]: . Vidulich, M., Yeh, Y., and Schneider, W. (1983).
    Time-compressed components for air-intercept control skills.
    Proceedings of the 27th Meeting of the Human Factors Society.

[^12]: . Guckenberger, D., Guckenberger, L. Luongo, F., Stanney, K., and
    Sepulveda, J. (April, 1995). Above-real-time training and the
    hyper-time algorithm. Dr. Dobb's Journal, 52-61.

[^13]: . FAA/Great Lakes Region/O'Hare Tower. (Undated Draft). O'Hare
    Air Traffic Control Tower Performance Verification Program. Author.

[^14]: . Endsley, M. (1995). Measurement of situational awareness in
    dynamic systems. Human Factors, 37(1), 65-84.