Los Angeles Arrival Enhancement Project

Real-Time Simulation Test Report

Prepared by

System Resources Corporation

and the

William J. Hughes Technical Center

Atlantic City International Airport, NJ

March 2, 1998

EXHIBIT

1-llLL-
Table of Contents

Page

1. INTRODUCTION 1

1.1 Los Angeles Arrival Enhancement Project 1

1.2 Real-Time Simulation Overview 1

2. EXPERIMENTAL APPARATUS 2

2.1 Target Generation Facility 2

2.2 Terminal Radar Approach Control (TRACON) Laboratories 3

2.3 Air Route Traffic Control Center (ARTCC) Laboratory 3

3. SIMULATION PARTICIPANTS 3

3.1 Test Director 3

3.2 Full Performance Level Air Traffic Controllers 4

3.3 Simulation Pilot Operators 4

3.4 Simulation Observers 4

4. EXPERIMENTAL DESIGN 5

4.1 Traffic Scenarios 5

4.2 Daily Simulation Schedule 6

4.3 Procedure 6

5. DATA COLLECTION 6

5.1 Controller Questionnaires 6

5.2 Simulation Observer Records 7

5.3 Computer-Generated Data Files 8

5.4 Audio and Video Recordings 8

6. ASSESSMENT METHODOLOGY 8

ii
Table of Contents (cont'd)

Page

7. SIMULATION RESULTS 8

7.1 Controller Questionnaire Analyses 8

7.2 Simulation Observer Documentation 21

7.3 Computer-Generated Data Analyses 22

7.4 Audio and Video Data Compilation 25

8. SUMMARY 26

9. CONCLUSIONS 27

ACRONYM LIST 28

REFERENCES 29

APPENDIXES

A - Scenario Description by Run

B - Controller Questionnaires

C - Controller Post-Run Questionnaire Comments

D - LAX Arrivals

E - Heading, Altitude, and Speed Commands

F - Aircraft Holding Data

G - BASIN Sector Diagram

H - Triple Arrival Stream Configuration Diagram

III
Table of Illustrations

Page

Tables

1. Terminal Controller Experience 4

2. En Route Controller Experience .....................•........................•...................................•. 4

3. Original Scenario Description 6

4. Terminal Workload Contributing Factors 9

5. En Route Workload Contributing Factors 9

6. TRACON Sector Aircraft Counts per Fifteen-Minute Intervals 22

7. Total Number of Aircraft Worked Per Sector for Entire Run 23

8. Total Number of Aircraft Worked Per Sector for 45 Minutes 24

9. LAAEP Videotape Compilation 25

Figures

1. En Route Average Workload by Run •.........•............................................................... 10

2. En Route Workload by Sector .................................................•................................... 11

3. Terminal Average Workload by Run 12

4. Terminal Workload by Sector 13

5. En Route Mean Ratings for Realism .......•................................................................... 14

6. Terminal Mean Ratings for Realism .........••...•..•.......•....•...........•.......•.•..•........•.........•. 14

iv
1. INTRODUCTION

1.1 Los Angeles Arrival Enhancement Project

In an effort to improve the safety and efficiency of air traffic management at Los Angeles

International Airport (LAX), the Federal Aviation Administration (FAA) proposed to

modify certain jet arrival routes, air traffic control sectors, and air traffic control (ATC)

coordination procedures. The Los Angeles Arrival Enhancement Project (LAAEP)

proposes a change in the location of one arrival route to LAX from the east and

reassignment of airspace and coordination of procedures between the Los Angeles Air

Route Traffic Control Center (ARTCC) and the Southern California (SOCAL) Terminal

Radar Approach Control (TRACON) (FAA, 1997).

LAX has two final approach courses, one heading for the north airport complex

(Runways 24L1R) and one heading for the south airport complex (Runways 25LIR).

The current westbound arrival configuration to LAX consists of three main flows of

aircraft that are funneled into one stream by Los Angeles Center (ZLA). The traffic is

then worked primarily by one TRACON controller who provides sequencing to the

southern approach course. Another TRACON controller handles inbound arrivals from

the north and west, sequencing them to the northern final approach course (FAA, 1997).

Aircraft arrivals from the east typically create a much greater volume of traffic on the

southern arrival course than the northern course with the current routes and procedures.

This traffic configuration results in an imbalance in controller workload, and, at times,

significant aircraft delays and thus increased user costs.

The LAAEP proposes a plan to improve the efficient use of airspace, reduce air traffic

delays, balance controller workload, and improve coordination among controllers by

reassigning a portion of airspace from the Los Angeles ARTCC to the TRACON, where

closer aircraft separation standards are allowed. This new portion of airspace was named

the BASIN sector, and it belongs to the SOCAL TRACON. In addition, the plan

proposes a triple traffic flow further east than present, which eventually merges into a

dual stream for final approach. Appendixes G and H include diagrams of the new

BASIN sector and the proposed arrival stream configuration, respectively.

1.2 Real-Time Simulation Overview

The Western Pacific Region (AWP-500) selected the FAA's William 1. Hughes

Technical Center in Atlantic City, NJ, as a test bed to simulate the proposed LAAEP

procedures. The Technical Center laboratories contain fully operational ATC displays in

both terminal and en route environments that interface with each other to provide a highfidelity environment for ATC experimentation.

The LAAEP simulation was conducted January 12-15, 1998. The goal of the simulation

was to provide a realistic environment for LAAEP representatives and controllers to

exercise, observe, and evaluate the proposed LA Center and TRACON airspace and
procedural changes. It is important to note that a valid experimental design was forgone

to allow a more exploratory type of evaluation of the procedure. No baseline study was

conducted from which to draw comparisons between procedures. But, rather, different

combinations of traffic levels and special events, such as aircraft in holding, runway

closings, mile-in-trail (MIT) changes, visual flight rule (VFR) approaches, different

sequencing strategies, were experienced and evaluated for effectiveness. The ultimate

goal of the test was to first determine if the proposed changes were workable, and then, if

consensus was reached as to the workability, make recommendations for a training

program for all other controllers at ZLA and the SOCAL TRACON.

By simulating three terminal sectors, including one new sector, and four en route sectors

concurrently, controllers could realistically control aircraft from the boundaries of the

high-altitude sectors to the Los Angeles International Airport. Controllers from both the

en route and terminal facilities worked together to test and make recommendations on the

LAAEP proposed procedures.

2. EXPERIMENTAL APPARATUS

2.1 Target Generation Facility

The Target Generation Facility (TGF) is an advanced simulation system designed to

support testing of current and future ATC hardware/software and procedures. The TGF

generates realistic digital radar messages for aircraft targets in a simulated airspace

environment. The simulation system can be adapted to mimic actual National Airspace

System (NAS) and Automated Radar Terminal System (ARTS) characteristics including

radar and environmental characteristics for a specific en route or terminal facility (TGF

web site, 1998). The TGF enables testing of ATC equipment and procedures without live

aircraft. Computer-scripted traffic scenarios provide ATC systems with realistic radar

returns for simulated aircraft based on facility-supplied or originally created flight plans.

For the LAAEP simulation, the TGF was partitioned into three subsystems: simulation

pilot, target generation, and development and support. Simulation Pilot Operators used

simulation pilot workstations (SPWs), or computer workstations, to fly the simulated

aircraft and commanded them in accordance with ATC instructions. The pilots

performed their roles in a separate laboratory from the air traffic controllers. The SPWs

contained a communication system that provided an audio interface with the Center and

TRACON controllers. Controller-pilot voice communications were processed through an

AMECOM voice communications system.

The target generation (TG) subsystem consisted of a TG chassis that performed all

modeling within the TGF and correlated dynamic data, such as aircraft state vectors and

radar performance, with known flight plan and adaptation data (TGF web site, 1998).

Dynamic data within the simulation was updated every second, even though the aircraft

were seen on the terminal radar scopes every 4.8 seconds and the en route scopes every

12 seconds. This ensured the simulation's fidelity and provided a realistic picture in both

the en route and terminal applications (TGF web site, 1998).

2
The development and support subsystem provided the pre-exercise activity of traffic

scenario development and the post-exercise activity of data reduction. Scenario

development involved the process of storing environmental data and flight sample data

into a database and previewing the database for use as the scenario in the simulation

exercise (TGF web site, 1998). Data reduction following the simulation involved

compiling aircraft data, simulation pilot command keyboard entries, and flight plans for

analysis.

2.2 Terminal Radar Approach Control (TRACON) Laboratories

SOCAL provided the Technical Center TRACON Laboratory with a software adaptation

of the fielded configuration of the DOWNE and STADIUM sectors. The adaptation was

the latest operational copy (AJ06) of ARTS IlIA dual systems. Technical Center

laboratory personnel, however, modified the COAST video map and software to add an

experimental BASIN sector. Three radar scopes displaying DOWNE, STADIUM, and

BASIN sectors were simulated throughout the test. The frequencies for each terminal

sector were the following: DOWNE = 124.9, STADIUM = 128.5, BASIN = 128.225.

2.3 Air Route Traffic Control Center (ARTCC) Laboratory

The Technical Center ARTCC Laboratory was driven by A4E1.3 Host software and the

charting release was supplied by LA Center. Airspace modifications were configured

practically identical to the field. Since there were subtle differences in the hardware at

the Technical Center, software adjustments were made so that the systems would accept

radar targets from the TGF. This allowed controllers to control aircraft. Some airspace

adaptation was necessary since the simulation required resectorization for a portion of the

en route airspace to the terminal airspace.

Four sectors were displayed throughout the simulation: two high sectors (37 and 39) and

two low (19 and 20). The frequencies for each en route sector were the following: 37 =

133.55,39 = 133.2, 19 = 126.35,20 = 133.4. An inboard, unmanned, Ghost sector was

set up to initiate traffic that fed into the traffic scenarios. A manned, outboard, Ghost

sector received ATC han doffs outside the scope of the experiment. The Ghost stations

gave the simulation realistic merit.

3. SIMULATION PARTICIPANTS

3.1 Test Director

The test director of the LAAEP simulation was an Air Traffic Control Specialist from the

ATC Simulation and Support Branch (ACT-51 0). He was responsible for the training of

the Simulation Pilot Operators, the installation of communications, the integration of the

laboratories, and the monitoring of computer support. Additionally, the test director

ensured that the proper scenarios were run and was responsible for the initiation and

termination of the scenarios. The test director ensured that any problems encountered

during a simulation were fixed to the satisfaction of the customers.

3
3.2 Full Performance Level Air Traffic Controllers

Six en route controllers from the Los Angeles ARTCC and four terminal controllers from

the SOCAL TRACON participated in the simulation. All controllers had Full

Performance Level (FPL) experience. Tables 1 and 2 show the controllers' average

number of years of ATC experience at Los Angeles and other facilities.

Table 1. Terminal Controller Experience

MEAN STANDARD MAXIMUM MINIMUM

LA FPL (YEARS) 9.1 3.5 12.0 4.0

OTHER FPL (YEARS) 5.9 3.6 11.1 3.0

Table 2. En Route Controller Experience

MEAN STANDARD DEV. MAXIMUM MINIMUM

LA FPL (YEARS) 7.8 4.2 14.3 5.7

OTHER FPL (YEARS) 1.5 2.5 3.0 0.0

During the simulation, controllers from ZLA rotated through four sector positions in the

en route laboratory. Controllers from SOCAL rotated through two sector positions

(SOCAL TRACON) and one new sector position (BASIN) in the terminal laboratory.

3.3 Simulation Pilot Operators

The Simulation Pilot Operators provided the link between the air traffic controllers and

the simulated aircraft in the LAAEP simulation. The pilots reported aircraft information

such as position and status to the controllers through voice communications using

standard ATC phraseology. By use of keyboard devices, pilots entered controller

instructions. Depending upon the complexity level of the traffic scenarios, pilots were

able to control up to 10 aircraft per computer terminal.

Two categories of simulation pilots were utilized in the LAAEP simulation: 13 federal

Air Traffic Assistants in Simulation (ATAs) and 13 contracted certified, general aviation

pilots. All individuals were trained to understand and interpret controller phraseology as

well as understand the keyboard commands that had to be entered into the system to fly

aircraft in accordance with ATC instructions.

3.4 Simulation Observers

Three simulation observers, with human factors, mathematics, and real-time simulation

experience, participated in the LAAEP simulation. The observers documented

simulation-related events, such as procedural modifications, hardware and/or software

problems, and changes to daily and controller position schedules. The observers also

recorded information as requested by controller participants for debrief discussion and

review. In addition, the observers collected certain real-time air traffic-related data

during simulation runs (a simulation run was defined as a l-hour and 15-minute traffic

4
scenario- or up to 1 hour and 15 minutes of controlled traffic). They also administered

post-run questionnaires and post-simulation questionnaires to the controllers. The

information documented by the simulation observers was particularly critical to

interpreting LAAEP data, since frequent modifications were made on the fly to traffic

scenarios and procedural strategies.

4. EXPERIMENTAL DESIGN

Before the simulation began, schedules were developed to randomize and counterbalance

the variables in the test. For instance, the level of traffic in the traffic scenarios varied.

To be experimentally valid, the number of times each scenario was scheduled was

counterbalanced to ensure each scenario was shown the same number of times. In

addition, controllers rotated through sector positions. The number of times each

controller worked a particular sector had to be randomized and counterbalanced. On a

broader scale, the total number of runs for the entire simulation was determined based on

an evenly distributed showing of traffic scenarios and controller position assignments.

The schedules were followed to a degree during the actual simulation. Participating

LAAEP representatives and controllers decided, however, that it was more important to

make decisions as to which traffic scenarios and special events, such as holding aircraft

and conducting VFR approaches, should be simulated on a run to run basis. That way,

particular problems could be introduced to the scenarios in combination with a variety of

conditions. LAAEP participants felt it was more important to stretch the capabilities of

the resectorization and observe the effects than to statistically analyze data. The

following sections explain how the traffic scenarios were originally developed and how

they were actually simulated. The daily simulation schedule is also presented.

4.1 Traffic Scenarios

The aircraft targets displayed on the controller scopes during the simulation were

generated from a sample of System Analysis Recorder (SAR) data from ZLA. The TGF

received flight plans from the Center for a peak traffic level period and converted the

flight data into TGF format. From the flight plan data, one master traffic sample was

developed. From that sample, three traffic scenarios, with different levels of traffic,

were developed. The master traffic sample was Scenario 4, described in Table 3 below.

From that scenario, aircraft were selectively deleted to create Scenarios 3 and 1, also

defined in Table 3. Originally, four scenarios were going to be generated, but due to time

constraints, one scenario was not developed. Each traffic scenario was l-hour and 15-

minutes in length.

During the actual test, the MIT maintained by the Center corresponded directly to the

original scenario description shown in Table 3. The TRACON, however, did not always

conduct VFR approaches throughout an entire run. In addition, the TRACON sometimes

ran visuals in Scenario 1 and sometimes did not run visuals in Scenario 3. Refer to

Appendix A for an exact description of the traffic conditions simulated in each run.

5
Table 3. Original Scenario Description

Scenario ARTCC TRACON Number Airport

aircraft acceptance rate

1 20 MIT IFR 165 68

3 15 MIT VFR+ IFR 175 77

4 15 MIT VFR 182 84

*MIT = Miles in trail

4.2 Daily Simulation Schedule

The LAAEP simulation was conducted from 4 PM to 12 AM over a 4-day period. Three

I-hour and IS-minute runs (or up to 1 hour and 15 minutes) were conducted the first two

days, 4 runs on the third day, and 2 runs on the last day of the test, for a total of 12 runs.

After each run, a short debrief session took place between controllers, LAAEP

representatives, and Technical Center personnel to address simulation-related issues.

Procedural issues and/or strategies were discussed among controllers and technical issues

were addressed by TGF.

4.3 Procedure

ZLA and SOCAL controllers monitored and controlled aircraft in their respective

facilities (i.e., ARTCC and TRACON laboratories) for the duration of each run. They

rotated through working the sectors in their airspace so that they each experienced the

proposed LAAEP airspace and procedural changes from all positions. Controllers used

the standard ATC phraseology to command aircraft to make heading, speed, and altitude

changes as they would in their actual facilities. Simulation pilots responded to controller

instructions via specialized keyboards. Aircraft targets on the controllers' displays

responded as well. Special events, such as the holding of aircraft and runway closings

were simulated on an as needed basis, and controllers responded to those events.

At the end of each traffic scenario, controllers completed workload questionnaires based

on their experiences throughout the run. Following the questionnaire sessions, informal

debriefs were held.

5. DATA COLLECTION

5.1 Controller Questionnaires

Three types of questionnaires were developed for the LAAEP simulation. The purpose of

the questionnaires was to first obtain background information on the participating

controllers. Then, questionnaires were distributed after each run throughout the

simulation to collect human factors data on controller workload and performance.

Lastly, questionnaires were distributed at the conclusion of the simulation to collect short

6
answer-type controller assessments of the LAAEP proposed plan. Examples of the

questionnaires can be found in Appendix B.

5.1.1 Background Information Questionnaire

Prior to participating in any simulation scenarios, controllers were asked to complete

Background Information Questionnaires. The purpose of the Background form was to

collect information on controller experience at current and previous facilities. The form

also asked the controllers to rate seven variables on the degree to which they contributed

to the controller's perception of workload. This information was useful to understanding

subsequent workload ratings on Post-Run Questionnaires.

5.1.2 Controller Post-Run Questionnaire

Controller Post-Run Questionnaires were distributed to the controllers after each

simulation run to collect subjective workload and performance data. Controllers rated the

workload they experienced with the new airspace procedures. They also compared that

workload to the workload they typically experience at their actual facilities (with current

operations). The Post-Run Questionnaire also asked controllers to rate their performance

on the ability to maintain traffic flows and separation, and the need for coordination with

the proposed procedures. In addition, controllers assessed the traffic complexity of each

run. Lastly, controllers rated the realism of each simulated scenario as compared to

actual air traffic control operations at their respective facilities.

5.1.3 Controller Post-Simulation Questionnaire

Controllers completed Controller Post-Simulation Questionnaires at the end of the

simulation. The format of the questionnaire was short-answer to enable the controllers to

express their opinions in various areas regarding the proposed procedures. Questions

addressed the overall safety of the new procedures and whether or not any changes

should be incorporated into the procedures. The distribution of workload across sectors

was also addressed. Controllers were asked to compare the amount of vectoring, speed

adjustments, interfacility and intrafacility coordination necessary with the simulated

procedure as compared to actual current operations at their facilities. The PostSimulation Questionnaire also asked the controller his opinion whether or not the

simulated procedure could accommodate future air traffic growth.

5.2 Simulation Observer Records

The simulation observers noted specific events that occurred during each run of the

simulation (e.g., runway closings, mile-in-trail restrictions, holding of aircraft, stackingaircraft vertically separated- of aircraft) to enable a better understanding of questionnaire

responses and quantitative data. Most runs differed to some extent from each other with

respect to the scripting of special situations. The simulation observer records were

critical to the proper reporting of the resultant data. A Scenario Description by Run table

7
was compiled by the simulation observers and is contained in Appendix A. This table

should be referred to when interpreting simulation data results.

5.3 Computer-Generated Data Files

The TGF, ARTS, and Host systems generated computer data continuously throughout the

simulation in TGF, Continuous Data Recording (CDR), and SAR formats, respectively.

The data were quantitative in nature and provided measures on numerous air traffic

control variables including: aircraft tracks worked; heading, altitude, and speed

commands; aircraft in holds; times spent in sectors; and conflict alerts. Computergenerated data were compiled and presented in this report to provide objective feedback

on simulated LAAEP operations.

5.4 Audio and Video Recordings

Audio and video recordings were made over the course of the simulation at the request of

LAAEP management and controllers. The purpose of the video recordings was to

provide LAX representatives with a means of showing simulation events and procedures

to interested parties and to allow for post-hoc analyses of simulation events. Audio was

recorded on the videotapes. In addition, backup audio for all sectors and all twelve runs

was recorded on a DENRO Digital Voice Recording System (DVRS).

6. ASSESSMENT METHODOLOGY

Participating Center and TRACON controllers were the evaluators of the proposed

LAAEP procedures in this simulation. They drew upon their expertise and understanding

of the nature of daily operations and the full range of traffic contingencies to evaluate the

workability of the procedures. Controllers introduced a variety of situations and/or

potential problems into the traffic scenarios. They then assessed the workability of such

circumstances. By simulating worst-case conditions, the limitations of the new arrival

streams and new terminal sector could be tested, training recommendations devised, and

conclusions drawn regarding the safety of the proposed procedures.

7. SIMULATION RESULTS

At the discretion of the customer, a proper experimental design was forgone to allow the

maximum amount of exposure to different traffic situations for each controller. In

addition, no baseline study was conducted to allow a valid statistical comparison between

current operations and planned operations. Given this situation, only a limited amount of

reporting can be done. Data were examined to see if trends could be identified to help

understand the effects of the proposed procedural changes.

7.1 Controller Questionnaire Analyses

The feedback obtained from the controller questionnaires, particularly from the PostSimulation Questionnaires, was crucial to assessing the workability of the LAAEP

8
procedures. Controllers put forth a great deal of effort to express their workload

assessments and opinions regarding their participation in the simulated LAAEP

scenarios.

7.1.1 Background Questionnaire

Controllers were asked to rate seven factors that contributed to workload levels in their

current ATC environment. A rating of 1 indicated a very low contribution and 8

represented a very high contribution. On the average, both terminal and en route

controllers rated aircraft performance characteristics/aircraft mix as the biggest

contributing factor to controller workload. Tables 4 and 5 show all of the average ratings

for each facility.

Table 4. Terminal Workload Contributing Factors"

Mean Standard Maximum Minimum

Deviation

Route Structure 6.3 1.0 7.0 5.0

Airspace Configuration 6.5 0.6 7.0 6.0

Interfacility Coordination 5.5 1.0 6.0 4.0

Intrafacility Coordination 6.0 0.8 7.0 5.0

Aircraft Mix 1.4 8.0 5.0

Weather 6.0 1.8 8.0 4.0

FacilityProcedures 6.0 1.8 8.0 4.0

a In addition, 2 controllers reported Traffic Management Unit (TMU) Flow

and gave it a rating of 8.

Table 5. En Route Workload Contributing Factors"

Mean Standard Maximum Minimum

Deviation

Route Structure 5.0 1.5 7.0 3.0

Airspace Configuration 5.0 1.1 6.0 3.0

Interfacility Coordination 5.2 1.2 7.0 4.0

Intrafacility Coordination 5.2 1.5 7.0 3.0

Aircraft Mix 0.5 7.0 6.0

Weather 5.5 2.4 8.0 2.0

FacilityProcedures 5.2 1.5 7.0 3.0

a In addition, 1 controller listed sequence order and rated it as 5, another controller

listed destination and rated it as 5, and a different controller listed management

practices and rated it as 7.

9
7.1.2 Post-Run Questionnaire

7.1.2.1 Workload Ratings

The average workload for each run was examined to help identify the effects of various

airspace and procedural changes as well as events simulated during this study on the air

traffic controllers. Some trends can be found if these values are correlated to the traffic

description shown in Appendix A. The following sections discuss the average workload

ratings across sectors for each run (labeled with the scenario). Appendix C contains all

controller workload-related comments recorded on the Post-Run Questionnaires.

7.1.2.1.1 En Route Controller Responses

Trends in en route post-run data suggest that with the proper training and exposure,

controllers should be able to work with the proposed changes. However, the effects of

special situations should be reviewed closely and compared to current operations.

Figure 1 depicts the average workload rating for each run. In addition, each bar is labeled

to indicate which scenario was being run.

7 S·I

Scenario 1 showed a steady decline in the average workload for Runs 1,3,5, and 10.

This behavior was expected because of the repetition of the same problem. However, the

average workload sharply increased during Runs 9 and 11 which had special events, such

as runway closings and holding of aircraft.

2 3 4 6 9 10 11 12

Run

Figure 1. En Route Average Workload by Run

10
The graph shows that the average workload for the first two runs with Scenario 3 (Runs 2

and 4) have similar ratings. It should be noted that Run 2 only lasted approximately 49

minutes as opposed to the other run which lasted an hour. This could explain the

unexpected lower rating in Run 2. During Run 7, an increase in the average controller

workload rating is noted. This was the first run with Scenario 3 that incorporated major

traffic changes (refer to Appendix A) indicating that the proposed changes may have

different effects depending on the traffic.

The same type of trend can be observed for Scenario 4. By Run 6, a steady decrease in

the average workload was still observed. However, when Run 6 was compared to Run 8,

an increase in workload is noted. Again this was the first time Scenario 4 was run with

the new traffic changes. Run 8 has a lower average workload rating than Run 7 because

it is the second run with the new traffic changes. During Run 12 an even lower average

workload was noted for Scenario 4. This can probably be attributed to the repeated

traffic, run time, and increased acceptance rate from the TRACON.

Figure 2 depicts the average controller workload and complexity ratings by sector.

Controllers commented on the Post-Run and Post-Simulation Questionnaires that with the

proposed changes; the workload in sectors 19 and 20 was lower than sectors 37 and 39,

and that sector 19 had less workload and was less complex than sector 20. Controller

comments are supported by trends in the data.

6.

5.5

0[

" 5.

~

§

~"

4.5

4.

3.5

19

_---'---l..---' Complexity

20 37R 39R

Sector

Figure 2. En Route Workload by Sector

7.1.2.1.2 Terminal Controller Responses

The BASIN sector did not actually exist in the field for the terminal controllers at the

time of the simulation. However, the purpose of simulating a BASIN sector was to

assess performance and workload in the newly-defined sector and to look at traffic feeds

11
to the two approach sectors, DOWNE and STADIUM. The BASIN controller

strategically positioned aircraft, easing the workload for the approach sectors. Trends in

TRACON post-run data suggest that with the proper training and exposure, controllers

should be able to acclimate to the new working conditions. However, procedures need to

be clearly defined to ensure a safe and steady traffic flow.

Figure 3 depicts the average workload rating for each run. In addition, each bar is labeled

to indicate which scenario was being run.

7.5 S-3

S-1

7.

-c

'"0

:B

~

§

<l)

::E

5.

9 10 11 12

Controllers reported lower average workload ratings as they gained more experience with

Scenario 1. However, an increase in the average workload was noted during Scenarios 9

and 11. This can be explained by special events scripted into both runs. Controllers

noted that in the field, hand-off controllers are available to assist during peak traffic and

under special circumstances. During the simulation, however, they were not. Controllers

also mentioned that taking aircraft out of holding patterns was workload intensive. It was

difficult to pull aircraft out of holding and maintain standard in-trail separation.

2 6 7

The same trend was seen in Scenarios 3 and 4. Controller average workload steadily

decreased with experience. An increase was seen in Run 12. The purpose of this run was

to find the maximum approach capacity. To accomplish this, the Center flooded the

TRACON with aircraft. Controllers stated that their labor intensified in BASIN when

routing aircraft downwind and east of the SMO VORTAC. Workload increases can be

attributed to the traffic flooding which required less separation (7 MIT), and more

communication between the controllers and visual-approach aircraft.

Run

Figure 3. Terminal Average Workload by Run

12
6.

6.5

OJ)

.5

••• ~

c

'"~ 5.

4.

Basin Downe Stadium

Sector

Figure 4. Terminal Workload by Sector

Figure 4 depicts the average controller workload and complexity ratings by sector.

Controllers noted that offloading aircraft from DOWNE to STADIUM on Runway 24R

was vital to managing traffic on Runway 24R. This process increased DOWNE's

complexity and allowed for a more equitable distribution of traffic between DOWNE and

STADIUM. In addition, using inboard runways with visual-approach traffic helped

maintain a moderate level of complexity for the DOWNE sector. Controllers cautioned

that aircraft offloading to Runway 24R is not a normal operation in the field.

In the BASIN sector, traffic complexity seemed to be a leading factor for higher

workload, possibly because fixed procedures were not in place at the time of the

simulation. Controllers attempted to combat the in-trail compression before handing

aircraft off to the DOWNE and STADIUM sectors. Also contributing to traffic

complexity was high traffic volume, transitioning holding aircraft, instrument flight rule

(IFR) conditions, and interfacility coordination. Controllers also mentioned that in the

BASIN sector, visual-approach traffic ran smooth. This might have been because

DOWNE transferred some aircraft to STADIUM and freed Runway 25L for more

aircraft.

7.1.2.2 Traffic Realism Ratings

7.1.2.2.1 En Route Controller Responses

Figure 5 displays the average realism rating given by the en route controllers for each

run. Examination of the figure indicates that changes made to the scenarios during run

time seemed to increase the realism. If this chart is correlated with Appendix A, some

trends become more obvious. Runs 1,3,5,9, 10, and 11 all had Scenario 1 traffic. Runs

1 and 2 were the exact same scenario, hence had very similar ratings. Run 5 had four

overflights added which resulted in a slight increase in the realism. By Runs 9, 10, and

11, additional changes were made and the average increased even more. The same type

of trends can be noted for Scenarios 3 and 4.

13
6.5

7 9 10 11 12

Figure 5. En Route Mean Ratings for Realism

Run

7.1.2.2.2 Terminal Controller Responses

Figure 6 displays the average realism rating given by the terminal controllers for each

run. Controllers felt that they could not rate the realism of the BASIN sector because it

did not exist in the field at the time of the simulation. Therefore, BASIN ratings are not

included in the data.

Factors that contributed to realism ratings were faster-than-normal speeds and visualapproach aircraft maintaining less separation when approaching the final. Researchers

collaborated with controllers to find unrealistic aircraft characteristics. Following some

corrections, realism ratings increased. Controllers rated Runs 9 and 12 as being the most

realistic during the simulation. It seemed that controllers felt scripted events and IFR

traffic most accurately represented field activities.

7.5

5.

2 3 4 5 6

Run

7 8 9 10 11 12

Figure 6. Terminal Mean Ratings for Realism

14
7.1.3 Controller Post-Simulation Questionnaires

The following questions were answered by all participating controllers at the end of the

simulation. Because the feedback received was detailed and particularly informative with

regard to training recommendations and controller strategies, responses were not

summarized in the following section, but presented in their entirety. The en route

controller responses to questions are given first for each question, followed by the

terminal controller responses.

Q1. Can the proposed Los Angeles arrival routes and procedures be safely controlled in

an actual operational environment?

EN ROUTE CONTROLLERS

Response Comments

Yes Sectors 39 and 37 should be just as safe as they are today. The increased

miles in trail will cause more vectoring, however. I see a problem at sector

19 with KONZL and BANDS being too close, which will require constant

monitoring. Sector 20's workload has increased. J65 traffic and PSP

arrivals and departures are your conflict traffic.

Yes None given.

Yes With definite changes in the proposed training plan, it will work. The

current plans for training at both SCI and ZLA are inadequate and will cause

the system to fail. It is also very dependent on TMU, if they do not protect

the controllers the system will fail. This will effect not only the efficiency

but also the safety margin to the point that an accident will be imminent.

Yes This will be a much better system to run traffic if the union's requested

procedures are fulfilled.

Yes Only with proper preparation of personnel, numerous changes in airspace

and procedures, and we will need more TMU support and much more input

with the command center to explain our needs.

Yes With traffic management and saturation control, this will work safely.

Training must be adequate.

TERMINAL·CONTROLLERS

Response Comments

Yes However, revised procedures, developed by our workgroup, must be

implemented.

Yes Using terminal rules allows the final to become compressed earlier. We

must use the new devised (stacked) procedures for this to work effectively.

Yes Provided we develop sound procedures for offloading to runway 24. Traffic

management and the arrival coordinator must take a proactive role to make

this sector work efficiently.

Yes With the correct miles in trail flow from ZLA sectors, the BASIN sector

does work.

15
Q2. Should any changes be incorporated into the proposed procedures?

EN.RoUTEJ CONTROLLERS

Resnonse Comments

No We have discussed all possible changes.

No None given.

Yes LAX arrivals need to be vertically separated above SNA, LGB, aNT, etc.,

arrivals routed over PSP on a static basis.

Yes None given.

Yes We need to move some airspace around and develop some new holding fixes

and procedures.

Yes BASIN's ability to move airplanes to 24 and Stadium's sector. Compression

stacking for sectors 19 and 20 to give to Basin.

TERMINAL .CONTROLLERS

Response Comments

None Too early to tell - but stacks of aircraft to BASIN help ZLA.

given

Yes AR1, AR2, AR3, AR4, CI2 duties must be redefined. The specifics will be

fully explained, in writing, by the Union's Los Angeles Area Representative.

Yes 24 offloading must be addressed.

Yes BASIN handoffto DOWNE and STADIUM. Visual separation between

these aircraft is best. On days with lower visibility, five miles or more

staggered will be required (& match speed).

Q3. Did you change your usual control and work strategies in any way in order to work

the traffic in this simulation? If yes, what did you do differently?

EN ROUTE CONTROLLERS

Resnonse Comments

Yes In order to get 15 - 20 MIT you must start sooner and use more drastic

vectors. Direct KONZL from south ofPKE gets too close to sector 40's

J212 traffic. Stacking an occasional aircraft did away with compression.

Yes I started spacing the aircraft sooner and descended them sooner in high

altitude. In low, when vectoring was required, I used harder vectors for

shorter periods of time.

Yes My vectors were drastically sharper for less duration. I also descended all

the different airport arrivals much sooner on the average.

Yes None given.

Yes I changed where I would move aircraft toward KONZL at sector 39.

Holding was used more often and I moved numerous en route overflights to

accommodate the new arrival procedures.

Yes Used harder turns to get spacing faster. Holding aircraft sooner in low

acceptance rate times.

16
TERMINAL CONTROLLERS

Response Comments

No Speeds are faster so vectors happen slower however, the same strategies

work.

Yes We used procedures between AR1 and AR2 to safely separate aircraft on

24R and 25L. All aircraft on 24R worked by ARl. All aircraft on 25L

worked by AR 1.

Yes I used slightly different phraseology so the simulation pilot could keep up.

Otherwise, Ijust need to get used to the new timing requirement.

Yes I had to lead turns on STADIUM to run the downwind visual pattern.

Q4. Was the distribution of workload across sectors you controlled during this

simulation the same as the distribution of workload with current operations at your

facility? If no, please explain.

EN ROUTE CONTROLLERS

Response Comments

No Sector 19 was not as difficult because you've taken away over Yz of its

traffic. The airspace is so small that it is more difficult to vector now.

Sector 20's workload has increased due to working aircraft other than LAX

arrivals. Sectors 37 and 39 remain about the same.

No I think this distributes the workload better. Sector 19 and 20 are more

evenly matched in their workload and complexity.

No Sectors 37, 39, and 20 worked harder and sector 19 worked less.

No Workload is shared between sectors 19 and 20 as they are currently

configured. The new sectors placed a higher workload on sector 20. If

sector 20 is in holding, the new configuration could be a oroblem.

No Sector 20's workload is increased yet sector 19's workload was reduced to a

great extent. Sector 37 and 39s' workload is different yet within acceptable

levels.

None None given.

given

TERMINAL CONTROLLERS

Response Comments

No Procedures we developed and used during the simulation resulted in equal

workload for AR1 and ARl. This balance is not the true makeup of

workload at AR1 and ARl. The demand ofairolanes is in AR1's airsoace.

No This was much better. It distributed the load to each complex and

controllers equitably.

No This simulation demonstrated that we can balance the workload between

STADIUM & DOWNE. It made DOWNE'sjob much easier. STADIUM's

workload increased. but it was still very manageable.

No BASIN can get busy, but can't compare this sector since it is new. DOWNE

has less workload with this simulation. STADIUM's workload picks up and

seems more balanced with DOWNE. (This is good).

17
Q5. Was there an equitable distribution of workload between sectors you controlled

during this simulation? Ifno, what sectors produced uneven workload levels?

EN R01.JTE CONTROLLERS

Response Comments

No Today sector 19 is more complex than sector 20. With the new proposal,

sector 20 will be more complex than sector 19.

No Sectors 37 and 39 worked harder than 19 and 20.

No 19 and 20 were not even. Sector 20 was much busier.

None None given.

given

Sometimes When SCT was taking stacks the workload was balanced very well. When

39 and 37 had to provide a solid 20 miles in trail, they were much busier

than 19 and 20.

Yes 19 and 20's traffic is more manageable.

TERMINAL CONTROLLERS

Response Comments

No The feeder, BASIN, does a lot of work. If a handoff could be staffed during

the simulation, BASIN is very manageable.

Yes Due to the use of our procedures the workload was close to equal at AR1 and

AR2. The reduced workload at AR1 was shifted to AR4. However, AR4's

workload was not unmanageable.

Yes We attained the goal of equal distribution by offloading aircraft to the 24

complex at LAX.

Yes This simulation demonstrated that we can balance the workload between

STADIUM & DOWNE. It made DOWNE'sjob much easier. STADIUM's

workload increased but it was still very manaaeable,

Q6. (a) Did the simulated procedure change the amount of vectoring needed as

compared to current operations at your facility? If yes, please explain. (b) Per

aircraft?

EN ROUTE CONTROLLERS

Response Comments

None None given.

aiven

(a) Yes, As stated before more vectoring is needed for 15-20 MIT. But if we can

(h) No stack aircraft it should relieve vectoring almost completely.

(a) Yes, The spacing required by each high altitude sector necessitates more vectors

(b) Yes to get it done in a timely manner. Note: when SCT takes stacks, the amount

of vectoring is reduced.

(a) Yes, To gain the increased spacing my vectors were larger for shorter duration.

(h) Ye.<; This was complicated further by having less airspace in sectors 19 and 20.

(a) Yes, It caused more vectoring in the high altitude sectors except when the

(b) Yes occasional stacks on LAX arrivals were utilized.

(a) Yes, Vectors were needed early during low rate acceptance period. During high

(b) Yes acceptance rate, no vectoring or little vectoring was needed.

18
TERMINAVCONTROLLERs

Response Comments

(a) Yes, BASIN did a lot of vectoring to keep flow/separation for handoffto

(b) None DOWNE and STADIUM. Vectoring for STADIUM/DOWNE had very

given little change.

(a) Yes, Since our new procedures at AR4 result in the sequence to 24R and 25L

(b) Yes being determined by the AR4 controller, instead of by the ARI controller or

C12, ARt and AR2 have minimal need to vector internal aircraft. Our

procedures for AR4, by placing aircraft on 24R and 25L, result in less

vectoring. If the AR4 controller does not have the authority to choose which

aircraft are placed on 24R then vectoring will be increased. If a third party,

such as TMC, C12, or supervisor, directs the AR4 controller to move aircraft

to 24R, as CI2 currently does with ARI, then increased vectoring will result.

Runway changes must be done as far from the airport as possible.

(a) Yes, It reduced the vectoring needed because the BASIN sector balanced the

(b) Yes arrival flow, which allowed ARt & AR2 to fine-tune the final.

(a) Yes- STADIUM had to vector a bit more due to increase in straight in runway 24

Stadium, traffic. Having BASIN put the aircraft on 24 reduced the vectoring DOWNE

(b) No- had to do.

Downe

Q7. (a) Did the simulated procedure change the amount of speed adjustments needed

as compared to current operations at your facility? If yes, please explain. (b) Per

aircraft?

EN ROUTE CONTROLLERS

Response Comments

(a) No, None given.

(b) No

(a) No, You still have to control a sequence of a string of aircraft and the techniques

(b) No and amount of adjustments needed seemed to be the same.

(a) No, None given.

(b) No

None None given.

given

(a) Yes, More speed adjustments because of longer in trail requirements. But again

(b) No stacking aircraft will help.

(a) Yes, To create and maintain the increased flow I issued more drastic speeds. lam

(b) Yes not sure if I issued more total speeds.

19
TERMINALCONTIlOLLERs

Response Comments

(a) No, None given.

(b) No

(a) Yes, Speed adjustment workload increased in BASIN sector. This is normal for a

(b) Yes feeder sector.

(a) Yes, None given.

(b) Yes

(a) Yes, Yes. AIC came to us faster and needed 1 - 2 more speed adjustments to get

(b) Yes them slowed and into sequence.

Q8. Did the simulated procedure change the amount 0/ interfacility and/or intrafacility

coordination needed as compared to current operations at your facility? If yes,

please explain.

ENR()UTE CONTROLLERS

Response Comments

No None given.

No None given.

None None given.

given

Yes Since sectors 37 and 39 run independent flows, there is less coordination

required between the two. There is also less coordination with TMU.

Yes Intrafacility coordination was reduced in reference to the LAX flow.

Interfacility coordination did not change.

Yes Sectors 39 and 37 could use less coordination due to not needing to flow

each other's LAX traffic. 39 and 40 are going to have more point outs to

take care of. Sectors 19 and 20 are going to have an increase in

coordination.

TERMINAL .CONTROLLERS

Response Comments

No None given.

Yes Interfacility was reduced. Intrafacility was slightly increased because of

direct coordination between ARI and ARl.

Yes But not excessive.

Yes ZLA called to APREQ altitude between A/C. Not much change in amount

of coordination for runway assignments.

20
Q9. In your opinion, will the simulated procedure accommodate future air traffic

growth? Airline/user requests? Please explain.

EN ROI1TECONTROLlJERS

Response Comments

Don't I think it will help overall especially in the TRACON. Unless LAX

know improves its capacity it is not really going to make much difference. We are

just setting up procedures to hold aircraft and begin to have controllers and

pilots used to holding in the airspace. We will be able to contain it better.

No We are at capacity now. This procedure will not change that; an increase

will not be able to be accommodated.

None None given.

given

Yes I think it is an improvement and will handle more aircraft because BASIN

sector can accept aircraft, sequence, and then hand off to DOWNE or

STADIUM. Sector 20 now can not do this.

Yes If the procedures developed while here are incorporated with the proposed

changes, throughput will increase and airport acceptance should also

increase.

Yes, No Yes, I will think it will accommodate future growth to some degree because

it allows us to put more airplanes through the Center's sectors. No, it does

not accommodate pilot requests in that they like to stay high as long as they

can and they will need to start down earlier.

TERMINAL CONTROLLERS

Response Comments

Yes If our procedures were implemented it is possible to realize a more efficient

flow of aircraft from the East. Any increase in traffic will remain to be seen.

Yes We may be able to take more aircraft using all four runways during visual

separation.

Yes It will drastically increase airport efficiency.

Yes It will help to fill the airport at times when all AIC come from the East.

7.2 Simulation Observer Documentation

The simulation observers compiled the table in Appendix A to use as a reference for

interpreting questionnaire and computer-generated data. In addition, every fifteen

minutes within a simulation run in the TRACON area, the observers counted the number

of aircraft at each controller sector position. This traffic count provided snapshots of the

distribution of workload and the efficient use of airspace across terminal sectors. It also

provided the controllers with preliminary information regarding the role of the BASIN

sector feeding traffic into the DOWNE and STADIUM sectors. Table 6 shows the 15-

minute interval aircraft counts per run per terminal sector.

21
Table 6. TRACON Sector Aircraft Counts per Fifteen-Minute Intervals*

Run Traffic 15-minutes 30-minutes 45-minutes 60-minutes

Level B D S B I D S B Dj S B D S

1 1 5 7 ! 6 2 10 5 4 5 5 3 I 13 i

i 3

2 3 5

I

I 9 7 4 10 6 4 I 5 7 - - -

3 1 3 5 9 2 6 7 6 3 7 7 7 8

4 3 6 9 ! 6 4 6 6 5 6 7 4

i

9 6

5 1 7

I

7 8 6 6 8 5 7 8 - - - i

6 4 6 10 7 5 8 5 5 6 10 - - -

7 3 5 ! 8 I 7 4 7

I

8 7 i 5 7 5 9 I 9 I I

8 4 6 10 I 9 7 9 ! 6 4

! 6 10 I 8

I

- ( 8 I

9 1 5 8 7 5 8 I 8 4 5 9 5 I 12 , 8 I

10 1 6

I

9 6 5 6 I 7 6 I 6 5 I ! - - -

11 1 8 2 2 7 4 3 5 8 9 9 4 8

12 4 6 8 j 8 6 8 8 - - - -

i - - I

* B = BASIN, D = DOWNE, S = STADIUM

At most IS-minute snapshots, the BASIN sector had control of a slightly smaller number

of aircraft than DOWNE and STADIUM. During Run 11, however, that was not the

case. The higher BASIN numbers in Run 11 were most likely due to the holding of

aircraft in the first 20 minutes of the run.

The snapshot counts for the DOWNE and STADIUM sectors appear to be rather evenly

distributed over all runs. The total DOWNE counts were higher than STADIUM, but this

makes sense when correlated to the Scenario Description by Run table in Appendix A.

DOWNE put more aircraft in hold than STADIUM over the course of the simulation.

Taking that into consideration, the allocation of aircraft by BASIN into the DOWNE and

STADIUM sectors appeared to be well-distributed.

7.3 Computer-Generated Data Analyses

The TGF, ARTS, and Host facilities generate a large quantity of data in TGF, CDR, and

SAR formats, respectively. Such data provides measures of airspace and/or airport

capacity. In addition, some of the data, such as air traffic volume data, heading, speed,

and altitude data entries, and aircraft mean sector flight times can provide information

about the complexity of a particular airspace or sector. Other data, such as the number

and duration of aircraft holds and the number of conflict alerts within a certain period of

time can provide information on the level of controller workload.

At the request of the LAAEP participating controllers, the above mentioned variable

measures were compiled and investigated with the LAAEP simulation data. Because no

baseline study was conducted and because simulation runs varied in special events, no

statistical inferences could be drawn from the data. However, raw data is presented and

discussed. Those individuals knowledgeable of Los Angeles Center and TRACON

22
Table 8. Total Number of Aircraft Worked Per Sector for 45 Minutes*

Run Run Traffic 39 37 20. 19 BASIN STADIUM DOWNE

Length Scenario

1 45m 1 43 31 20 22 26 14 n/a

2 45m 3 41 35 24 24 31 21 n/a

3 45m 1 42 33 22 19 27 19 n/a

4 45m 3 43 35 24 24 33 26 n/a

5 45m 1 42 34 20 23 29 19 n/a

6 45m 4 42 36 24 23 35 31 n/a

7 45m 3 43 36 28 26 34 36 35

8 45m 4 44 36 29 26 36 43 36

9 45m 1 40 31 23 21 24 25 32

10 45m 1 39 34 28 24 30 29 30

11 45m 1 40 33 21 18 22 27 26

12 39m,9s 4 39 34 25 24 30 39 32

* Run 12 was 39m, 9s in length. Figures would be slightly higher for 45 minutes.

7.3.1.2 Number Of Lax Arrivals

Since the total number of aircraft tracks worked included not only arrivals, but departures

and overflights as well, the number of LAX arrivals only was computed. Appendix D

contains a table of LAX arrival numbers by runway for the DOWNE and STADIUM

approach control sectors. The majority of aircraft were cleared for the outboard runways,

24R and 25L.

The numbers presented in Appendix D correspond, as expected, to the I5-minute interval

snapshot data. LAX arrivals, for the most part, were evenly distributed between the

DOWNE and STADIUM sectors, down to the runway thresholds.

7.3.2 Heading, Speed, and Altitude Commands

The number of heading, altitude, and speed commands were computed for each run by

sector. The first table in Appendix E displays the figures. For data analysis, heading

commands not only included Fly heading xxx commands, but also Direct xxx commands

in the en route airspace and Intercept localizer commands in the terminal airspace.

Altitude commands not only included climb and descend commands, but also Cross

intersection xxx commands.

As with the Aircraft Tracks Worked data, numbers are difficult to compare because of the

varying run lengths. The second table in Appendix E shows the frequencies of heading,

altitude, and speed commands for the first 45 minutes of each run- with the exception of

Run 12, which was just over 39 minutes long. Appendix A should be referred to when

24
examining the numbers. It should also be kept in mind that individual controller

differences also may have accounted for some of the variation in the numbers.

7.3.3 Number and Duration of Aircraft Holds

Appendix F details the number and duration of aircraft holds by run and by sector during

the simulation. Appendix A shows which runs had aircraft in holds, but the exact

magnitude of those special-scripted events is detailed in Appendix F. Run 9 and

especially Run II were high-workload runs as far as holding was concerned. Again, the

workload questionnaire data support this observation of the quantitative data.

7.4 Audio and Video Data Compilation

The original plan for videotaping the LAAEP simulation involved recording each sector

each run using microcameras mounted to quick-looked, unmanned displays. The

resolution provided by the cameras, however, was not adequate enough to read the

aircraft data tags on playback. Therefore, a digital scan converter was connected to the

TGF's X-windows Plan View Display (XPVD) containing a map of all seven simulated

sectors. For each run, an area of the map was enlarged (i.e., either BASIN, DOWNE and

STADIUM, 19 and 20, or 37 and 39) and recorded on a videotape. Audio was also

recorded; in stereo for the paired sectors. Table 9 lists the videotape compilation for the

entire simulation.

Table 9. LAAEP Videotape Compilation

Run Sector(s) Traffic Special Events

Sample

1 37 & 39 1 20 MIT; holding in Sector 39

2 Downe 3 10 MIT; holding; IFR; 2 active runways

3 Basin 1 10 MIT; IFR; 2 active runways

4 19 & 20 3 15 MIT

5 Downe & Stadium 1 10 MIT; IFR; 2 active runways

6 37 & 39 4 15 MIT; No D side on 37; stacks over KONZL instead

of in-trail

7 19 & 20 3 15 MIT; stacks over KONZL instead ofin-trail

8 Basin 4 10 MIT; VFR; 4 active runways

9 19 & 20 1 20 MIT with 2 flows to 15 MIT and 1 flow 30 minutes

into the problem; holding

10 Basin 1 10 MIT; IFR; 2 active runways; switched to VFR at

30:00 and to 4 active runways

11 37 & 39 1 20 MIT; holding

12 Downe & Stadium 4 7 MIT; VFR; 4 active runways

25
operations should be able to compare the quantitative data from the simulation to current

operations at their facilities.

7.3.1 Air Traffic Volume

Air traffic volume data were compiled by sector for each run of the simulation. The total

number of aircraft tracks worked, which included arrivals, departures, and overflights,

was determined. In addition, the total number of LAX arrivals solely was computed.

7.3.1.1 Number of Aircraft Tracks Worked

Table 7 depicts the total number of aircraft worked per sector per run of the simulation.

Data were not available for the DOWNE sector for Runs 1-6 due to a software problem.

Although between-sector numbers can be examined for each run, not a great deal can be

inferred, because run lengths varied. Table 8, however, shows throughput data for the

first 45 minutes of each run. Run 12 was only 39 minutes in length, so one would expect

the numbers presented to be slightly higher for a 45-minute period. The counts should

still be compared with caution, due to different events that transpired in each run (refer to

Appendix A).

On the average, despite traffic level and special event differences, Sector 39 controlled

the most aircraft per run. Questionnaire responses from the Center controllers support

this observation.

Table 7. Total Number of Aircraft Worked Per Sector for Entire Run

Run Run Traffic 39 37 20 19 BASIN STADIUM DOWNE

Length Scenario

1 67m, lIs 1 58 39 31 34 37 20 n/a

2 46m,27s 3 43 26 25 24 33 21 n/a

3 71m,6s 1 55 40 31 31 41 31 n/a

4 64m,2s 3 57 41 32 31 46 33 n/a

5 56m,39s 1 53 41 29 30 35 26 n/a

6 51m,31s 4 48 43 32 29 40 36 n/a

7 61m,16s 3 55 44 34 34 45 48 48

8 Slm,29s 4 49 42 32 30 39 48 43

9 60m,6s 1 50 42 32 29 39 46 43

10 49m,37s 1 51 37 30 27 33 33 43

11 62m,43s 1 51 42 31 28 30 39 31

12 39m,9s 4 39 34 25 24 30 39 32

23
8. SUMMARY

The full impact of an ATC procedural change on the ATC system is seldom clear until

the change is actually implemented, or, where technology allows, simulated. The scope

of the LAAEP plan was such that adequate testing and evaluation could not be done in

the facility DYSIM laboratories. LAAEP representatives therefore recruited testing

support from the FAA's William J. Hughes Technical Center, in New Jersey. The

Technical Center's ARTCC and TRACON laboratories provided the air traffic controllers

with fully-operational ATC displays and interfacility interface capabilities.

Six FPL controllers from ZLA and four FPL controllers from the SOCAL TRACON

participated in the real-time simulation. The controllers were not only test subjects, but

also evaluators of the LAAEP arrival route modifications and airspace resectorization

plans. Over a four-day period, controllers rotated through sector positions and

experienced varying levels of traffic complexity. They intentionally scripted special

events into the traffic scenarios, such as runway closings and the holding of aircraft, to

test the limitations of the proposed procedures. Controllers worked together to resolve

issues and make recommendations for a workable operation.

Data were collected from laboratory computer systems and from questionnaires.

However, because simulation run events varied and no baseline data was collected prior

to the test, a limited amount of statistical analyses could be done. Objective data relating

to capacity, sector complexity, and workload were collected in TGF, SAR, and CDR

formats. Specifically, throughput, controller action, and special event data were

compiled and the raw figures presented. The numbers must be correlated with specific

run events, detailed in Appendix A, to be interpreted.

Controller workload data was collected following each of twelve runs comprising the

simulation. For the most part, workload did seem to decrease as simulation experience

progressed. Special scenario events seemed to affect workload more than the number of

aircraft involved in a scenario. This was demonstrated by the high workload ratings

associated with Runs 9 and 11, which were Scenario 1 runs, with a considerable amount

of holding.

For some of the Center controllers, increased mile-in-trail requirements resulted in higher

perceived workload. Several controllers commented that stacking the aircraft in lieu of

in-trail separation was the more effective strategy, which also reduced workload. In the

terminal area, the new BASIN sector was perceived by most controllers to be busy, but

workable. Some difficulty was experienced, however, in keeping the most efficient flow

of aircraft in the absence of clearly defined procedures. Both the DOWNE and

STADIUM sectors were considered very manageable by most controllers.

Since only a limited amount of data analysis could be done, controller feedback on the

LAAEP plan proved to be an invaluable source of information. All ten controllers agreed

the LAAEP proposed procedures could be safely conducted in an actual ATC

environment. New procedures training was considered a crucial component, however.

26
The stacking strategy was recommended by several Center controllers to be part of the

training. All controllers felt they had to change their usual work strategies to some

degree to work the traffic with the modified airspace.

As far as the distribution of workload between sectors was concerned, several controllers

felt Sector 19 was not as busy as pre-LAAEP due to a more shared workload between

Sectors 19 and 20. Most TRACON controllers felt the LAAEP demonstrated a workable

balance of traffic between the DOWNE and STADIUM sectors, with the help of the new

BASIN sector.

The majority of controllers believed the simulated LAAEP procedures would

accommodate future air traffic growth. One controller believed the procedures offered an

overall significant improvement with the TRACON. Another controller stated that the

Center was able to work a higher flow of traffic with the TRACON able to accept the

flow with new procedures. Efficiency was a key simulation outcome mentioned by many

controllers. In summary, the users- the ZLA and SOCAL controllers, supported the

simulated LAAEP procedures.

9. CONCLUSIONS

This simulation tested the proposed LAAEP east arrival route modifications and new

BASIN sector for workability, efficiency, and safety. Seven sectors, four en route and

three terminal, were simulated concurrently. Participating controllers worked arrival

traffic at varying degrees of complexity. Special events, such as runway closings, aircraft

in holding, mile-in-trail restrictions, and various combinations of such events were

simulated and evaluated by the controllers to assess the limitations of the LAAEP plan.

All ten FPL controller participants from the LA Center and SOCAL TRACON facilities

agreed the proposed procedures could be safely controlled in an actual operational

environment. They collectively devised performance strategies and training

recommendations for all LAX controllers to ultimately enable a successful operation.

27
ARTCC

ARTS

ATA

ATC

CDR

DVRS

FAA

FPL

IFR

LAAEP

LAX

MIT

NAS

SAR

SOCAL

SPW

TG

TGF

TMU

TRACON

VFR

XPVD

ZLA

Acronym List

Air Route Traffic Control Center

Automated Radar Terminal System

Air Traffic Assistant

Air Traffic Control

Continuous Data Recording

Digital Voice Recording System

Federal Aviation Administration

Full Performance Level

Instrument Flight Rules

Los Angeles Arrival Enhancement Project

Los Angeles International Airport

Miles in Trail

National Airspace System

System Analysis Recorder

Southern California

Simulation Pilot Workstation

Target Generation

Target Generation Facility

Traffic Management Unit

Terminal Radar Approach Control

Visual Flight Rules

X-Windows Plan View Display

Los Angeles Center

28
References

Federal Aviation Administration (1997). Environmental Assessment: Los Angeles

International Airport Arrival Enhancement Project. Federal Aviation Administration,

Washington, DC.

System Resources Corporation & System Simulation Support Branch (1996). [On-Line].

Target Generation Facility (TGF) Overview. Available:

http://www.tgf.tc.faa.gov/doc/overview/overview .html# 1.

29