## LosAngeles Arrival Enhancement Project Real-Time Simulation Test Report

Prepared by System Resources Corporation

and the William J.Hughes Technical Center Atlantic City International Airport, NJ

March 2, 1998 EXHIBIT



# 1-llLL-

### Table ofContents Page

1. INTRODUCTION 1 1.1 LosAngelesArrival Enhancement Project 1 1.2 Real-Time Simulation Overview 1 

### 2. EXPERIMENTAL APPARATUS 2 2.1 Target Generation Facility 2 2.2 Terminal Radar Approach Control (TRACON) Laboratories 3 2.3 Air Route Traffic Control Center (ARTCC) Laboratory 3 

### 3. SIMULATION PARTICIPANTS 3 3.1 Test Director 3 3.2 Full Performance LevelAir Traffic Controllers 4 3.3 Simulation Pilot Operators 4 3.4 Simulation Observers 4 

### 4. EXPERIMENTAL DESIGN 5 4.1 Traffic Scenarios 5 4.2 Daily Simulation Schedule 6 4.3 Procedure 6 

### 5. DATACOLLECTION 6 5.1 Controller Questionnaires 6 5.2 Simulation Observer Records 7 5.3 Computer-Generated Data Files 8 5.4 Audio and VideoRecordings 8 

### 6. ASSESSMENT METHODOLOGY 8

ii

### Table ofContents (cont'd) Page

7. SIMULATION RESULTS 8 7.1 Controller Questionnaire Analyses 8 7.2 Simulation Observer Documentation 21 7.3 Computer-Generated Data Analyses 22 7.4 Audio and Video Data Compilation 25 

### 8. SUMMARY 26
9. CONCLUSIONS 27
ACRONYM LIST 28 REFERENCES 29 APPENDIXES A

- Scenario Description by Run B

- Controller Questionnaires C

- Controller Post-Run Questionnaire Comments D

- LAX Arrivals E

- Heading, Altitude, and Speed Commands F

- Aircraft Holding Data G

- BASIN Sector Diagram H

- Triple Arrival Stream Configuration Diagram

III

### Table ofIllustrations Page

Tables 1. Terminal Controller Experience 4 2. En Route Controller Experience .....................•........................•...................................•. 4 3. Original Scenario Description 6 4. Terminal Workload Contributing Factors 9 5. En Route Workload Contributing Factors 9 6. TRACON Sector Aircraft Counts per Fifteen-Minute Intervals 22 7. Total Number ofAircraft Worked Per Sector for Entire Run 23 8. Total Number ofAircraft Worked Per Sector for 45 Minutes 24 

### 9. LAAEP Videotape Compilation 

### 25 Figures 1. En Route Average Workload by Run •.........•............................................................... 

### 10 2. En Route Workload by Sector .................................................•................................... 11 

### 3. Terminal Average Workload by Run 

### 12 4. Terminal Workload by Sector 

### 13 5. En Route Mean Ratings for Realism .......•................................................................... 14 6. Terminal Mean Ratings for Realism .........••...•..•.......•....•...........•.......•.•..•........•.........•. 14

iv

### 1. INTRODUCTION

1.1 LosAngeles Arrival Enhancement Project
In an effort to improve the safety and efficiency of air traffic management atLos Angeles International Airport (LAX), the Federal Aviation Administration (FAA) proposed to modify certain jet arrival routes, air traffic control sectors, and air traffic control (ATC) coordination procedures. The Los Angeles Arrival Enhancement Project (LAAEP) proposes achange in the location ofone arrival route to LAX from the east and reassignment of airspace and coordination ofprocedures between the Los Angeles Air Route Traffic Control Center (ARTCC) and the Southern California (SOCAL) Terminal Radar Approach Control (TRACON) (FAA, 1997). LAX has two final approach courses, one heading for the north airport complex (Runways 24L1R) and one heading for the south airport complex (Runways 25LIR). The current westbound arrival configuration to LAX consists ofthree main flows of aircraft that are funneled into one stream by Los Angeles Center (ZLA). The traffic is then worked primarily by one TRACON controller who provides sequencing to the southern approach course. Another TRACON controller handles inbound arrivals from the north and west, sequencing them to the northern final approach course (FAA, 1997). Aircraft arrivals from the east typically create amuch greater volume oftraffic on the southern arrival course than the northern course with the current routes and procedures. This traffic configuration results in an imbalance incontroller workload, and, attimes, significant aircraft delays and thus increased user costs. The LAAEP proposes aplan to improve the efficient use of airspace, reduce air traffic delays, balance controller workload, and improve coordination among controllers by reassigning aportion of airspace from the Los Angeles ARTCC to the TRACON, where closer aircraft separation standards are allowed. This new portion of airspace was named the BASIN sector, and itbelongs to the SOCAL TRACON. In addition, the plan proposes atriple traffic flow further east than present, which eventually merges into a dual stream for final approach. Appendixes G and H include diagrams ofthe new BASIN sector and the proposed arrival stream configuration, respectively. 

### 1.2 Real-Time Simulation Overview The Western Pacific Region (AWP-500) selected the FAA's William 1.Hughes Technical Center inAtlantic City, NJ, as atest bed to simulate the proposed LAAEP procedures. The Technical Center laboratories contain fully operational ATC displays in both terminal and en route environments that interface with each other to provide ahighfidelity environment for ATC experimentation. The LAAEP simulation was conducted January 12-15, 1998. The goal ofthe simulation was to provide arealistic environment for LAAEP representatives and controllers to exercise, observe, and evaluate the proposed LA Center and TRACON airspace andprocedural changes. It is important to note that avalid experimental design was forgone

to allow amore exploratory type ofevaluation ofthe procedure. No baseline study was conducted from which to draw comparisons between procedures. But, rather, different combinations oftraffic levels and special events, such as aircraft inholding, runway closings, mile-in-trail (MIT) changes, visual flight rule (VFR) approaches, different sequencing strategies, were experienced and evaluated for effectiveness. The ultimate goal ofthe test was to first determine ifthe proposed changes were workable, and then, if consensus was reached as to the workability, make recommendations for atraining program for all other controllers atZLA and the SOCAL TRACON. By simulating three terminal sectors, including one new sector, and four en route sectors concurrently, controllers could realistically control aircraft from the boundaries ofthe high-altitude sectors to the Los Angeles International Airport. Controllers from both the en route and terminal facilities worked together to test and make recommendations on the LAAEP proposed procedures. 

### 2. EXPERIMENTAL APPARATUS 2.1 Target Generation Facility The Target Generation Facility (TGF) is an advanced simulation system designed to support testing of current and future ATC hardware/software and procedures. The TGF generates realistic digital radar messages for aircraft targets in a simulated airspace environment. The simulation system can be adapted to mimic actual National Airspace System (NAS) and Automated Radar Terminal System (ARTS) characteristics including radar and environmental characteristics for a specific en route or terminal facility (TGF web site, 1998). The TGF enables testing ofATC equipment and procedures without live aircraft. Computer-scripted traffic scenarios provide ATC systems with realistic radar returns for simulated aircraft based on facility-supplied or originally created flight plans. For the LAAEP simulation, the TGF was partitioned into three subsystems: simulation pilot, target generation, and development and support. Simulation Pilot Operators used simulation pilot workstations (SPWs), or computer workstations, to fly the simulated aircraft and commanded them in accordance with ATC instructions. The pilots performed their roles in a separate laboratory from the air traffic controllers. The SPWs contained acommunication system that provided an audio interface with the Center and TRACON controllers. Controller-pilot voice communications were processed through an AMECOM voice communications system. The target generation (TG) subsystem consisted of aTG chassis that performed all modeling within the TGF and correlated dynamic data, such as aircraft state vectors and radar performance, with known flight plan and adaptation data (TGF web site, 1998). Dynamic data within the simulation was updated every second, even though the aircraft were 

### seen on the terminal radar scopes every 4.8 seconds and the en route scopes every 12 seconds. This ensured the simulation's fidelity and provided arealistic picture inboth the en route and terminal applications (TGF web site, 1998).

2The development and support subsystem provided the pre-exercise activity oftraffic

scenario development and the post-exercise activity ofdata reduction. Scenario development involved the process of storing environmental data and flight sample data into adatabase and previewing the database for use as the scenario inthe simulation exercise (TGF web site, 1998). Data reduction following the simulation involved compiling aircraft data, simulation pilot command keyboard entries, and flight plans for analysis. 

### 2.2 Terminal Radar Approach Control (TRACON)Laboratories SOCAL provided the Technical Center TRACON Laboratory with a software adaptation ofthe fielded configuration ofthe DOWNE and STADIUM sectors. The adaptation was the latest operational copy (AJ06) ofARTS IlIA dual systems. Technical Center laboratory personnel, however, modified the COAST video map and software to add an experimental BASIN sector. Three radar scopes displaying DOWNE, STADIUM, and BASIN sectors were simulated throughout the test. The frequencies for each terminal sector were the following: DOWNE 

### = 124.9, STADIUM 

### = 128.5, BASIN 

### = 128.225. 

### 2.3 AirRoute Traffic Control Center (ARTCC) Laboratory The Technical Center ARTCC Laboratory was driven by A4E1.3 Host software and the charting release was supplied by LA Center. Airspace modifications were configured practically identical to the field. Since there were subtle differences in the hardware at the Technical Center, software adjustments were made so that the systems would accept radar targets from the TGF. This allowed controllers to control aircraft. Some airspace adaptation was necessary since the simulation required resectorization for aportion ofthe en route airspace to the terminal airspace. Four sectors were displayed throughout the simulation: two high sectors (37 and 39) and two low (19 and 20). The frequencies for each en route sector were the following: 37 

### = 133.55,39 

### = 133.2, 19

### = 126.35,20 

### = 133.4. An inboard, unmanned, Ghost sector was set up to initiate traffic that fed into the traffic scenarios. A manned, outboard, Ghost sector received ATC handoffs outside the scope ofthe experiment. The Ghost stations gave the simulation realistic merit. 

### 3. SIMULATION PARTICIPANTS 3.1 TestDirector The test director ofthe LAAEP simulation was an Air Traffic Control Specialist from the ATC Simulation and Support Branch (ACT-51 0). He was responsible for the training of the Simulation Pilot Operators, the installation ofcommunications, the integration ofthe laboratories, and the monitoring ofcomputer support. Additionally, the test director ensured that the proper scenarios were run and was responsible for the initiation and termination ofthe scenarios. The test director ensured that any problems encountered during a simulation were fixed to the satisfaction ofthe customers.

3

### 3.2 Full 

### Performance LevelAir Traffic Controllers

Six en route controllers from the Los Angeles ARTCC and four terminal controllers from the SOCAL TRACON participated inthe simulation. All controllers had Full Performance Level (FPL) experience. Tables 1and 2 show the controllers' average number ofyears ofATC experience at Los Angeles and other facilities. Table 1. Terminal Controller Experience

MEAN STANDARD MAXIMUM MINIMUM LAFPL (YEARS) 9.1 3.5 12.0 4.0 OTHER FPL (YEARS) 5.9 3.6 11.1 3.0

Table 2. En Route Controller Experience

MEAN STANDARD DEV. MAXIMUM MINIMUM LAFPL (YEARS) 7.8 4.2 14.3 5.7 OTHER FPL (YEARS) 1.5 2.5 3.0 0.0 During the simulation, controllers from ZLA rotated through four sector positions inthe en route laboratory. Controllers from SOCAL rotated through two sector positions (SOCAL TRACON) and one new sector position (BASIN) in the terminal laboratory. 

### 3.3 Simulation Pilot Operators The Simulation Pilot Operators provided the link between the air traffic controllers and the simulated aircraft inthe LAAEP simulation. The pilots reported aircraft information such as position and status to the controllers through voice communications using standard ATC phraseology. By use ofkeyboard devices, pilots entered controller instructions. Depending upon the complexity level ofthe traffic scenarios, pilots were able to control up to 10 aircraft per computer terminal. Two categories of simulation pilots were utilized inthe LAAEP simulation: 13 federal Air Traffic Assistants in Simulation (ATAs) and 13contracted certified, general aviation pilots. All individuals were trained to understand and interpret controller phraseology as well as understand the keyboard commands that had to be entered into the system to fly aircraft in accordance with ATC instructions. 

### 3.4 Simulation Observers Three simulation observers, with human factors, mathematics, and real-time simulation experience, participated inthe LAAEP simulation. The observers documented simulation-related events, such as procedural modifications, hardware and/or software problems, and changes to daily and controller position schedules. The observers also recorded information as requested by controller participants for debrief discussion and review. In addition, the observers collected certain real-time air traffic-related data during simulation runs (a simulation run was defined as a l-hour and 15-minute traffic

4scenario- or up to 1hour and 15minutes ofcontrolled traffic). They also administered

post-run questionnaires and post-simulation questionnaires to the controllers. The information documented by the simulation observers was particularly critical to interpreting LAAEP data, since frequent modifications were made on the

### flyto traffic scenarios and procedural strategies. 

### 4. EXPERIMENTAL DESIGN Before the simulation began, schedules were developed to randomize and counterbalance the variables inthe test. For instance, the level oftraffic in the traffic scenarios varied. To be experimentally valid, the number oftimes each scenario was scheduled was counterbalanced to ensure each scenario was shown the same number oftimes. In addition, controllers rotated through sector positions. The number oftimes each controller worked aparticular sector had to be randomized and counterbalanced. On a broader scale, the total number ofruns for the entire simulation was determined based on an evenly distributed showing oftraffic scenarios and controller position assignments. The schedules were followed to adegree during the actual simulation. Participating LAAEP representatives and controllers decided, however, that itwas more important to make decisions as to which traffic scenarios and special events, such as holding aircraft and conducting VFR approaches, should be simulated on arun to run basis. That way, particular problems could be introduced to the scenarios in combination with avariety of conditions. LAAEP participants felt itwas more important to stretch the capabilities of the resectorization and observe the effects than to statistically analyze data. The following sections explain how the traffic scenarios were originally developed and how they were actually simulated. The daily simulation schedule is also presented. 

### 4.1 Traffic Scenarios The aircraft targets displayed on the controller scopes during the simulation were generated from a sample of System Analysis Recorder (SAR) data from ZLA. The TGF received flight plans from the Center for apeak traffic level period and converted the flight data into TGF format. From the flight plan data, one master traffic sample was developed. From that sample, three traffic scenarios, with different levels oftraffic, were developed. The master traffic sample was Scenario 4, described in Table 3 below. From that scenario, aircraft were selectively deleted to create Scenarios 3 and 1, also defined in Table 3. Originally, four scenarios were going to be generated, but due to time constraints, one scenario was not developed. Each traffic scenario was l-hour and 15- minutes in length. During the actual test, the MIT maintained by the Center corresponded directly to the original scenario description shown inTable 3. The TRACON, however, did not always conduct VFR approaches throughout an entire run. In addition, the TRACON sometimes ran visuals in Scenario 1and sometimes did not run visuals in Scenario 3. Refer to Appendix A for an exact description ofthe traffic conditions simulated in each run.



### 5Table 3. Original Scenario Description Scenario ARTCC TRACON Number Airport aircraft acceptance rate 1 20 MIT IFR 165 68 3 15MIT VFR+ IFR 175 77 4 15 MIT VFR 182 84 *MIT =Miles intrail 

### 4.2 DailySimulation Schedule The LAAEP simulation was conducted from 4 PM to 12AM over a4-day period. Three I-hour and IS-minute runs (or up to 1hour and 15minutes) were conducted the first two days, 4 runs on the third day, and 2runs on the last day ofthe test, for atotal of 12runs. After each run, a short debrief session took place between controllers, LAAEP representatives, and Technical Center personnel to address simulation-related issues. Procedural issues and/or strategies were discussed among controllers and technical issues were addressed by TGF. 

### 4.3 Procedure ZLA and SOCAL controllers monitored and controlled aircraft in their respective facilities (i.e., ARTCC and TRACON laboratories) for the duration ofeach run. They rotated through working the sectors in their airspace so that they each experienced the proposed LAAEP airspace and procedural changes from all positions. Controllers used the standard ATC phraseology to command aircraft to make heading, speed, and altitude changes as they would in their actual facilities. Simulation pilots responded to controller instructions via specialized keyboards. Aircraft targets on the controllers' displays responded as well. Special events, such as the holding of aircraft and runway closings were simulated on an as needed basis, and controllers responded to those events. At the end ofeach traffic scenario, controllers completed workload questionnaires based on their experiences throughout the run. Following the questionnaire sessions, informal debriefs were held. 

### 5. DATACOLLECTION 5.1 Controller Questionnaires Three types ofquestionnaires were developed for the LAAEP simulation. The purpose of the questionnaires was to first obtain background information on the participating controllers. Then, questionnaires were distributed after each run throughout the simulation to collect human factors data on controller workload and performance. Lastly, questionnaires were distributed atthe conclusion ofthe simulation to collect short

6answer-type controller assessments ofthe LAAEP proposed plan. Examples ofthe

questionnaires can be found in Appendix B. 

### 5.1.1 Background Information Questionnaire Prior to participating in any simulation scenarios, controllers were asked to complete Background Information Questionnaires. The purpose ofthe Background form was to collect information on controller experience at current and previous facilities. The form also asked the controllers to rate seven variables on the degree to which they contributed to the controller's perception ofworkload. This information was useful to understanding subsequent workload ratings on Post-Run Questionnaires. 

### 5.1.2 Controller Post-Run Questionnaire Controller Post-Run Questionnaires were distributed to the controllers after each simulation run to collect subjective workload and performance data. Controllers rated the workload they experienced with the new airspace procedures. They also compared that workload to the workload they typically experience attheir actual facilities (with current operations). The Post-Run Questionnaire also asked controllers to rate their performance on the ability to maintain traffic flows and separation, and the need for coordination with the proposed procedures. In addition, controllers assessed the traffic complexity ofeach run. Lastly, controllers rated the realism ofeach simulated scenario as compared to actual air traffic control operations attheir respective facilities. 

### 5.1.3 Controller Post-Simulation Questionnaire Controllers completed Controller Post-Simulation Questionnaires atthe end ofthe simulation. The format ofthe questionnaire was short-answer to enable the controllers to express their opinions in various areas regarding the proposed procedures. Questions addressed the overall safety ofthe new procedures and whether or not any changes should be incorporated into the procedures. The distribution ofworkload across sectors was also addressed. Controllers were asked to compare the amount ofvectoring, speed adjustments, interfacility and intrafacility coordination necessary with the simulated procedure as compared to actual current operations attheir facilities. The PostSimulation Questionnaire also asked the controller his opinion whether or not the simulated procedure could accommodate future air traffic growth. 

### 5.2 Simulation Observer Records The simulation observers noted specific events that occurred during each run ofthe simulation (e.g., runway closings, mile-in-trail restrictions, holding of aircraft, stackingaircraft vertically separated- of aircraft) to enable abetter understanding of questionnaire responses and quantitative data. Most runs differed to some extent from each other with respect to the scripting of special situations. The simulation observer records were critical to the proper reporting ofthe resultant data. A Scenario Description by Run table



### 7was compiled by the simulation observers and is contained in Appendix A. This table

should be referred to when interpreting simulation data results. 

### 5.3 Computer-Generated Data Files The TGF, ARTS, and Host systems generated computer data continuously throughout the simulation in TGF, Continuous Data Recording (CDR), and SAR formats, respectively. The data were quantitative in nature and provided measures on numerous air traffic control variables including: aircraft tracks worked; heading, altitude, and speed commands; aircraft inholds; times spent in sectors; and conflict alerts. Computergenerated data were compiled and presented inthis report to provide objective feedback on simulated LAAEP operations. 

### 5.4 Audio and Video Recordings Audio and video recordings were made over the course ofthe simulation atthe request of LAAEP management and controllers. The purpose ofthe video recordings was to provide LAX representatives with ameans of showing simulation events and procedures to interested parties and to allow for post-hoc analyses of simulation events. Audio was recorded on the videotapes. In addition, backup audio for all sectors and all twelve runs was recorded on aDENRO Digital Voice Recording System (DVRS). 

### 6. ASSESSMENT METHODOLOGY Participating Center and TRACON controllers were the evaluators ofthe proposed LAAEP procedures in this simulation. They drew upon their expertise and understanding ofthe nature ofdaily operations and the full range oftraffic contingencies to evaluate the workability ofthe procedures. Controllers introduced avariety of situations and/or potential problems into the traffic scenarios. They then assessed the workability of such circumstances. By simulating worst-case conditions, the limitations ofthe new arrival streams and new terminal sector could be tested, training recommendations devised, and conclusions drawn regarding the safety ofthe proposed procedures. 

### 7. SIMULATION RESULTS At the discretion ofthe customer, aproper experimental design was forgone to allow the maximum amount ofexposure to different traffic situations for each controller. In addition, no baseline study was conducted to allow avalid statistical comparison between current operations and planned operations. Given this situation, only a limited amount of reporting can be done. Data were examined to see iftrends could be identified to help understand the effects ofthe proposed procedural changes. 

### 7.1 Controller Questionnaire Analyses The feedback obtained from the controller questionnaires, particularly from the PostSimulation Questionnaires, was crucial to assessing the workability ofthe LAAEP

8procedures. Controllers put forth a great deal ofeffort to express their workload

assessments and opinions regarding their participation inthe simulated LAAEP scenarios. 

### 7.1.1 Background Questionnaire Controllers were asked to rate seven factors that contributed to workload levels intheir current ATC environment. A rating of 1 indicated avery low contribution and 8 represented avery high contribution. On the average, both terminal and en route controllers rated aircraft performance characteristics/aircraft mix as the biggest contributing factor to controller workload. Tables 4 and 5 show all ofthe average ratings for each facility. Table 

### 4. Terminal Workload Contributing Factors" 

### Mean Standard Maximum Minimum

Deviation RouteStructure 

### 6.3 1.0 7.0 5.0 AirspaceConfiguration 

### 6.5 0.6 7.0 6.0 Interfacility Coordination 

### 5.5 1.0 6.0 4.0 Intrafacility Coordination 

### 6.0 0.8 7.0 5.0 Aircraft Mix 

### 1.4 8.0 5.0 Weather 

### 6.0 1.8 8.0 4.0 FacilityProcedures 

### 6.0 1.8 8.0 4.0

aInaddition, 2controllers reported Traffic Management Unit (TMU) Flow andgave itarating of8.
Table 

### 5. En Route Workload Contributing Factors"



### Mean Standard Maximum Minimum

Deviation RouteStructure 

### 5.0 1.5 7.0 3.0 AirspaceConfiguration 

### 5.0 1.1 6.0 3.0 Interfacility Coordination 

### 5.2 1.2 7.0 4.0 Intrafacility Coordination 

### 5.2 1.5 7.0 3.0 Aircraft Mix 

### 0.5 7.0 6.0 Weather 

### 5.5 2.4 8.0 2.0 FacilityProcedures 

### 5.2 1.5 7.0 3.0

aInaddition, 1 controller listedsequence order andrated itas5,another controller listeddestination andrateditas5,andadifferent controller listedmanagement practices andrated itas7.

9

### 7.1.2 Post-Run Questionnaire

7.1.2.1 Workload Ratings The average workload for each run was examined to help identify the effects ofvarious airspace and procedural changes as well as events simulated during this study on the air traffic controllers. Some trends can be found ifthese values are correlated to the traffic description shown in Appendix A. The following sections discuss the average workload ratings across sectors for each run (labeled with the scenario). Appendix C contains all controller workload-related comments recorded on the Post-Run Questionnaires. 

### 7.1.2.1.1 EnRoute Controller Responses Trends in en route post-run data suggest that with the proper training and exposure, controllers should be able to work with the proposed changes. However, the effects of special situations should be reviewed closely and compared to current operations. Figure 1depicts the average workload rating for each run. In addition, each bar is labeled to indicate which scenario was being run.

7 S·I

Scenario 1 showed a steady decline inthe average workload for Runs 1,3,5, and 10. This behavior was expected because ofthe repetition ofthe same problem. However, the average workload sharply increased during Runs 9 and 11 which had special events, such as runway closings and holding ofaircraft. 2 3 4 6 9 10 11 12 Run Figure 1. En Route Average Workload by Run

10The graph shows that the average workload for the first two runs with Scenario 3 (Runs 2

and 4) have similar ratings. It should be noted that Run 2 only lasted approximately 49 minutes as opposed to the other run which lasted an hour. This could explain the unexpected lower rating in Run 2. During Run 7, an increase in the average controller workload rating is noted. This was the first run with Scenario 3 that incorporated major traffic changes (refer to Appendix A) indicating that the proposed changes may have different effects depending on the traffic. The same type oftrend can be observed for Scenario 4. By Run 6, a steady decrease in the average workload was still observed. However, when Run 6was compared to Run 8, an increase inworkload isnoted. Again this was the first time Scenario 4 was run with the new traffic changes. Run 8has a lower average workload rating than Run 7 because it isthe second run with the new traffic changes. During Run 12 an even lower average workload was noted for Scenario 4. This can probably be attributed to the repeated traffic, run time, and increased acceptance rate from the TRACON. Figure 2 depicts the average controller workload and complexity ratings by sector. Controllers commented on the Post-Run and Post-Simulation Questionnaires that with the proposed changes; the workload in sectors 19and 20 was lower than sectors 37 and 39, and that sector 19had less workload and was less complex than sector 20. Controller comments are supported by trends inthe data.

6.

5.5

0[ 

### " 5.

~ § 

### "
~ 4.5

4.

3.5 19 _---'---l..---' Complexity 20 37R 39R Sector Figure 2. En Route Workload by Sector 

### 7.1.2.1.2 Terminal Controller Responses The BASIN sector did not actually exist inthe field for the terminal controllers atthe time ofthe simulation. However, the purpose of simulating aBASIN sector was to assess performance and workload inthe newly-defined sector and to look attraffic feeds

11to the two approach sectors, DOWNE and STADIUM. The BASIN controller

strategically positioned aircraft, easing the workload for the approach sectors. Trends in TRACON post-run data suggest that with the proper training and exposure, controllers should be able to acclimate to the new working conditions. However, procedures need to be clearly defined to ensure a safe and steady traffic flow. Figure 3 depicts the average workload rating for each run. In addition, each bar is labeled to indicate which scenario was being run.

7.5 S-3
S-1 7.

-c '" 0 :B

~ § <l) ::E

5.

9 10 11 12

Controllers reported lower average workload ratings as they gained more experience with Scenario 1. However, an increase in the average workload was noted during Scenarios 9 and 11. This can be explained by special events scripted into both runs. Controllers noted that in the field, hand-off controllers are available to assist during peak traffic and under special circumstances. During the simulation, however, they were not. Controllers also mentioned that taking aircraft out ofholding patterns was 

### workload intensive. It was difficult to pull aircraft out ofholding and maintain standard in-trail separation. 2 6 7

The same trend was seen in Scenarios 3 and 4. Controller average workload steadily decreased with experience. An increase was seen inRun 12. The purpose ofthis run was to find the maximum approach capacity. To accomplish this, the Center flooded the TRACON with aircraft. Controllers stated that their labor intensified in BASIN when routing aircraft downwind and east ofthe SMO VORTAC. Workload increases can be attributed to the traffic flooding which required less separation (7 MIT), and more communication between the controllers and visual-approach aircraft. Run Figure 3. Terminal Average Workload by Run

126. 6.5

OJ) .5 ••• ~ c 

### '"

~ 5.

4. Basin Downe Stadium Sector Figure 4. Terminal Workload by Sector Figure 4 depicts the average controller workload and complexity ratings by sector. Controllers noted that offloading aircraft from DOWNE to STADIUM on Runway 24R was vital to managing traffic on Runway 24R. This process increased DOWNE's complexity and allowed for amore equitable distribution oftraffic between DOWNE and STADIUM. In addition, using inboard runways with visual-approach traffic helped maintain amoderate level ofcomplexity for the DOWNE sector. Controllers cautioned that aircraft offloading to Runway 24R is not anormal operation inthe field. In the BASIN sector, traffic complexity seemed to be a leading factor for higher workload, possibly because fixed procedures were not inplace atthe time ofthe simulation. Controllers attempted to combat the in-trail compression before handing aircraft off to the DOWNE and STADIUM sectors. Also contributing to traffic complexity was high traffic volume, transitioning holding aircraft, instrument flight rule (IFR) conditions, and interfacility coordination. Controllers also mentioned that inthe BASIN sector, visual-approach traffic ran smooth. This might have been because DOWNE transferred some aircraft to STADIUM and freed Runway 25L for more aircraft. 

### 7.1.2.2 Traffic RealismRatings 7.1.2.2.1 EnRoute Controller Responses Figure 5 displays the average realism rating given by the en route controllers for each run. Examination ofthe figure indicates that changes made to the scenarios during run time seemed to increase the realism. Ifthis chart is correlated with Appendix A, some trends become more obvious. Runs 1,3,5,9, 10, and 11 all had Scenario 1traffic. Runs 1 and 2 were the exact same scenario, hence had very similar ratings. Run 5 had four overflights added which resulted in a slight increase inthe realism. By Runs 9, 10, and 11, additional changes were made and the average increased even more. The same type oftrends can be noted for Scenarios 3 and 4.

136.5

7 9 10 11 12

Figure 5. En Route Mean Ratings for Realism Run

7.1.2.2.2 Terminal Controller Responses Figure 6displays the average realism rating given by the terminal controllers for each run. Controllers felt that they could not rate the realism ofthe BASIN sector because it did not exist inthe field atthe time ofthe simulation. Therefore, BASIN ratings are not included in the data. Factors that contributed to realism ratings were faster-than-normal speeds and visualapproach aircraft maintaining less separation when approaching the final. Researchers collaborated with controllers to find unrealistic aircraft characteristics. Following some corrections, realism ratings increased. Controllers rated Runs 9 and 12 as being the most realistic during the simulation. It seemed that controllers felt scripted events and IFR traffic most accurately represented field activities.

7.5

5.

2 3 4 5 6 Run 7 8 9 11 12 10 Figure 6. Terminal Mean Ratings for Realism

14

### 7.1.3 Controller Post-Simulation Questionnaires

The following questions were answered by all participating controllers atthe end ofthe simulation. Because the feedback received was detailed and particularly informative with regard to training recommendations and controller strategies, responses were not summarized inthe following section, but presented intheir entirety. The en route controller responses to questions are given first for each question, followed by the terminal controller responses. 

### Q1. Can theproposed Los Angeles arrival routes andprocedures be safely controlled in an actual operational environment?
EN ROUTE CONTROLLERS Response Comments Yes Sectors 39 and 37 should bejust as safe as they are today. The increased

miles in trail will cause more vectoring, however. I see aproblem at sector 19 with KONZL and BANDS being too close, which will require constant monitoring. Sector 20's workload has increased. J65 traffic and PSP arrivals and departures are your conflict traffic. Yes None given. Yes With definite changes inthe proposed training plan, itwill work. The

current plans for training atboth SCI and ZLA are inadequate and will cause the system to fail. It is also very dependent on TMU, ifthey do not protect the controllers the system will fail. This will effect not only the efficiency but also the safety margin to the point that an accident will be imminent. Yes This will be amuch better system to run traffic ifthe union's requested

procedures are fulfilled. Yes Only with proper preparation ofpersonnel, numerous changes in airspace

and procedures, and we will need more TMU support and much more input with the command center to explain our needs. Yes With traffic management and saturation control, this will work safely.

Training must be adequate. TERMINAL·CONTROLLERS Response Comments Yes However, revised procedures, developed by our workgroup, must be

implemented. Yes Using terminal rules allows the final to become compressed earlier. We

must use the new devised (stacked) procedures for this to work effectively. Yes Provided we develop sound procedures for offloading to runway 24. Traffic

management and the arrival coordinator must take aproactive role to make this sector work efficiently. Yes With the correct miles intrail flow from ZLA sectors, the BASIN sector

does work.

15

### Q2. Shouldanychanges be incorporated intotheproposedprocedures?
EN.RoUTEJ CONTROLLERS Resnonse Comments No Wehave discussed allpossible changes. No None given. Yes LAX arrivals needtobevertically separated above SNA, LGB, aNT, etc.,

arrivals routed overPSPonastaticbasis. Yes None given. Yes Weneedtomove someairspace around anddevelop somenewholding fixes

andprocedures. Yes BASIN's abilitytomove airplanes to24and Stadium's sector. Compression

stacking forsectors 19and20togivetoBasin. TERMINAL .CONTROLLERS Response Comments None Too earlytotell

-butstacks ofaircrafttoBASIN helpZLA. given Yes AR1, AR2, AR3, AR4, CI2duties mustberedefined. The specifics willbe

fullyexplained, inwriting, bytheUnion's LosAngeles Area Representative. Yes 24 offloading mustbe addressed. Yes BASIN handoffto DOWNE and STADIUM. Visual separation between

these aircraft isbest. Ondayswithlowervisibility, fivemiles ormore staggered willberequired (& match speed). 

### Q3. Didyou changeyour usualcontrol andworkstrategies inanyway inordertowork thetraffic inthissimulation? Ifyes, whatdidyou dodifferently?

EN ROUTE CONTROLLERS Resnonse Comments Yes Inordertoget 15

- 20MITyoumust startsooner andusemore drastic

vectors. Direct KONZL fromsouthofPKE getstoo closeto sector40's J212 traffic. Stacking anoccasional aircraft didawaywith compression. Yes Istarted spacingthe aircraft sooneranddescended them soonerinhigh

altitude. Inlow,whenvectoring wasrequired, Iusedharder vectors for shorterperiods oftime. Yes Myvectors were drastically sharper forlessduration. Ialso descended all

the different airport arrivals much sooner onthe average. Yes None given. Yes Ichanged where Iwould move aircrafttoward KONZL atsector 39.

Holding wasusedmore oftenandImovednumerous enroute overflights to accommodate thenew arrivalprocedures. Yes Usedharderturns togetspacing faster. Holding aircraft sooner inlow

acceptance ratetimes.

16TERMINAL CONTROLLERS Response Comments No Speeds arefaster sovectors happen slowerhowever, the same strategies work. Yes Weusedprocedures betweenAR1 andAR2to safely separate aircraft on

24R and25L. Allaircraft on24RworkedbyARl. All aircraft on25L workedbyAR1. Yes Iused slightly differentphraseology sothe simulation pilotcouldkeep up.

Otherwise, Ijust needtogetusedtothenewtiming requirement. Yes Ihadto leadturns onSTADIUM torunthe downwind visual pattern. 

### Q4. Wasthedistribution ofworkload acrosssectorsyou controlledduring this simulation thesame asthedistribution ofworkloadwith currentoperations atyour facility? Ifno,please explain.
EN ROUTE CONTROLLERS Response Comments No Sector 19wasnotasdifficultbecauseyou've taken away over Yzofits traffic. The airspace issosmallthat itismore difficulttovector now. Sector20's workload has increased duetoworking aircraft otherthan LAX arrivals. Sectors 37 and39remain aboutthe same. No Ithinkthis distributes theworkload better. Sector 19and20 aremore evenly matched intheirworkload andcomplexity. No Sectors 37, 39, and20worked harder andsector 19worked less. No Workload is sharedbetween sectors 19and20 asthey arecurrently configured. Thenew sectorsplaced ahigherworkload onsector20. If sector20isinholding, thenew configuration couldbeaoroblem. No Sector20's workload isincreasedyet sector 19's workload wasreduced to a greatextent. Sector37and39s' workload isdifferent yetwithin acceptable levels. None None given. given TERMINAL CONTROLLERS Response Comments No Procedures wedeveloped andusedduringthe simulation resulted inequal workload forAR1 andARl. Thisbalance isnotthetruemakeup of workload atAR1 andARl. Thedemand ofairolanes isinAR1's airsoace. No Thiswas muchbetter. Itdistributed the loadto eachcomplex and controllers equitably. No This simulation demonstrated thatwecanbalance the workload between STADIUM & DOWNE. ItmadeDOWNE'sjob much easier. STADIUM's workload increased. butitwas stillverymanageable. No BASIN cangetbusy, butcan't comparethis sector since itisnew. DOWNE has lessworkload withthis simulation. STADIUM's workload picksup and seems morebalanced withDOWNE. (This isgood).

17

### Q5. Wasthereanequitable distribution ofworkload between sectorsyou controlled during thissimulation? Ifno, whatsectorsproduced unevenworkload levels? ENR01.JTE CONTROLLERS Response Comments No Today sector 19 is more complex than sector 20. With the new proposal,

sector 20 will be more complex than sector 19. No Sectors 37 and 39 worked harder than 19 and 20. No 19 and 20 were not even. Sector 20 was much busier. None None given. given Sometimes When SCT was taking stacks the workload was balanced very well. When

39 and 37 had to provide a solid 20 miles intrail, they were much busier than 19 and 20. Yes 19 and 20's traffic is more manageable. TERMINAL CONTROLLERS Response Comments No The feeder, BASIN, does a lot of work. If ahandoff could be staffed during

the simulation, BASIN is very manageable. Yes Due to the use of our procedures the workload was close to equal at AR1 and

AR2. The reduced workload atAR1 was shifted to AR4. However, AR4's workload was not unmanageable. Yes We attained the goal of equal distribution by offloading aircraft to the 24

complex at LAX. Yes This simulation demonstrated that we can balance the workload between

STADIUM & DOWNE. Itmade DOWNE'sjob much easier. STADIUM's workload increased but itwas still very manaaeable, 

### Q6. (a)Didthesimulatedprocedure change theamount ofvectoring neededas compared tocurrentoperations atyourfacility? Ifyes,please explain. (b)Per aircraft? ENROUTE CONTROLLERS Response Comments None None given. aiven (a) Yes, As stated before more vectoring is needed for 15-20 MIT. But ifwe can (h) No stack aircraft it should relieve vectoring almost completely. (a) Yes, The spacing required by each high altitude sector necessitates more vectors (b) Yes to get it done in atimely manner. Note: when SCT takes stacks, the amount

of vectoring is reduced. (a) Yes, To gain the increased spacing my vectors were larger for shorter duration. (h) Ye.<; This was complicated further by having less airspace in sectors 19 and 20. (a) Yes, It caused more vectoring inthe high altitude sectors except when the (b) Yes occasional stacks on LAX arrivals were utilized. (a) Yes, Vectors were needed early during low rate acceptance period. During high (b) Yes acceptance rate, no vectoring or little vectoring was needed.

18TERMINAVCONTROLLERs Response Comments (a) Yes, BASIN didalotofvectoring tokeep flow/separation forhandoffto (b) None DOWNE and STADIUM. Vectoring forSTADIUM/DOWNE hadvery given littlechange. (a) Yes, Sinceournewprocedures atAR4result inthe sequence to24R and25L (b) Yes being determined bytheAR4 controller, instead ofbytheARI controller or

C12,ARt andAR2haveminimal needtovector internal aircraft. Our procedures forAR4, byplacing aircraft on24R and25L, result inless vectoring. IftheAR4 controller doesnothavethe authority to choose which aircraft areplaced on24Rthenvectoring willbeincreased. Ifathird party, suchasTMC, C12,orsupervisor, directs theAR4 controller tomove aircraft to24R, asCI2currently does withARI, then increased vectoring willresult. Runway changes mustbedone asfarfromthe airport aspossible. (a) Yes, Itreduced thevectoring neededbecausetheBASIN sectorbalanced the (b) Yes arrival flow, which allowedARt & AR2tofine-tune the final. (a) Yes- STADIUM hadtovector abitmore duetoincrease instraight inrunway 24 Stadium, traffic. Having BASIN puttheaircraft on24reduced thevectoring DOWNE (b) No- hadto do. Downe



### Q7. (a)Didthesimulatedprocedure change theamount of

### speedadjustments 

### needed ascompared tocurrentoperations atyourfacility? Ifyes,please explain. (b)Per aircraft?

EN ROUTE CONTROLLERS Response Comments (a) No, None given. (b) No (a) No, You stillhavetocontrol asequence ofastring ofaircraft andthetechniques (b) No and amount ofadjustments needed seemedtobethe same. (a) No, None given. (b) No None None given. given (a) Yes, More speed adjustments because oflongerintrailrequirements. Butagain (b) No stacking aircraftwillhelp. (a) Yes, Tocreate andmaintaintheincreased flowIissuedmore drastic speeds. lam (b) Yes not sureifIissuedmoretotal speeds.

19TERMINALCONTIlOLLERs Response Comments (a) No, None given. (b) No (a) Yes, Speed adjustment workload increased in BASIN sector. This is normal for a (b) Yes feeder sector. (a) Yes, None given. (b) Yes (a) Yes, Yes. AIC came to us faster and needed 1

-2 more speed adjustments to get (b) Yes them slowed and into sequence.


### Q8. Didthesimulatedprocedure change theamount 

### 0/interfacility and/or intrafacility coordination neededascomparedtocurrentoperations atyourfacility? Ifyes, please explain.
ENR()UTE CONTROLLERS Response Comments No None given. No None given. None None given. given Yes Since sectors 37 and 39 run independent flows, there is less coordination

required between the two. There is also less coordination with TMU. Yes Intrafacility coordination was reduced in reference to the LAX flow.

Interfacility coordination did not change. Yes Sectors 39 and 37 could use less coordination due to not needing to flow

each other's LAX traffic. 39 and 40 are going to have more point outs to take care of. Sectors 19 and 20 are going to have an increase in coordination. TERMINAL .CONTROLLERS Response Comments No None given. Yes Interfacility was reduced. Intrafacility was slightly increased because of

direct coordination between ARI and ARl. Yes But not excessive. Yes ZLA called to APREQ altitude between A/C. Not much change in amount

of coordination for runway assignments.



### 20

### Q9. Inyour opinion, willthesimulatedprocedure accommodatefuture airtraffic growth? Airline/user requests?Please explain.

EN ROI1TECONTROLlJERS Response Comments Don't Ithink itwill help overall especially inthe TRACON. Unless LAX know improves its capacity it is not really going to make much difference. We are

just setting up procedures to hold aircraft and begin to have controllers and pilots used to holding inthe airspace. We will be able to contain itbetter. No We are at capacity now. This procedure will not change that; an increase

will not be able to be accommodated. None None given. given Yes Ithink it is an improvement and will handle more aircraft because BASIN

sector can accept aircraft, sequence, and then hand off to DOWNE or STADIUM. Sector 20 now can not do this. Yes Ifthe procedures developed while here are incorporated with the proposed

changes, throughput will increase and airport acceptance should also increase. Yes, No Yes, Iwill think itwill accommodate future growth to some degree because

it allows us to put more airplanes through the Center's sectors. No, itdoes not accommodate pilot requests inthat they like to stay high as long as they can and they will need to start down earlier. TERMINAL CONTROLLERS Response Comments Yes If our procedures were implemented it is possible to realize amore efficient

flow of aircraft from the East. Any increase in traffic will remain to be seen. Yes We may be able to take more aircraft using all four runways during visual

separation. Yes It will drastically increase airport efficiency. Yes It will help to fill the airport at times when all AIC come from the East.

7.2 Simulation Observer Documentation The simulation observers compiled the table inAppendix A to use as areference for interpreting questionnaire and computer-generated data. In addition, every fifteen minutes within a simulation run inthe TRACON area, the observers counted the number of aircraft at each controller sector position. This traffic count provided snapshots ofthe distribution ofworkload and the efficient use ofairspace across terminal sectors. It also provided the controllers with preliminary information regarding the role ofthe BASIN sector feeding traffic into the DOWNE and STADIUM sectors. Table 6 shows the 15- minute interval aircraft counts per run per terminal sector.

21Table 6. TRACON Sector Aircraft Counts per Fifteen-Minute Intervals* Run Traffic 15-minutes 30-minutes 45-minutes 60-minutes Level B D S B 

# I D S B Dj S B D S 1 1 5 7 ! 6 2 10 5 4 5 5 3 

## I 13 i 3 i 2 3 5 I 9 7 4 10 6 4 

## I 5 7 

## - - I 

### - 3 1 3 5 9 2 6 7 6 3 7 7 7 8 4 3 6 9 

### ! 6 4 6 6 5 6 7 4 i 9 6 5 1 7 I 7 8 6 6 8 5 7 8


### - 

## - - i 6 4 6 10 7 5 8 5 5 6 10


### - 

## - - 7 3 5 

### ! 8 I 7 4 7 I 8 7 i 5 7 5 9 

## I 9 I I 8 4 6 10 I 9 7 9 ! 6 4 ! 6 10 I 8 I 8


### - ( I 9 1 5 8 7 5 8 I 8 4 5 9 5 I 12 , 8 I 10 1 6 I 9 6 5 6 

## I 7 6 I 6 5 I ! 

### - 

## - - 11 1 8 2 2 7 4 3 5 8 9 9 4 8 12 4 6 8 j 8 6 8 8


### - 

## - - 

## - i



### - - I *B =BASIN, D =DOWNE, S = STADIUM At most IS-minute snapshots, the BASIN sector had control of a slightly smaller number of aircraft than DOWNE and STADIUM. During Run 11, however, that was not the case. The higher BASIN numbers in Run 11 were most likely due to the holding of aircraft inthe first 20 minutes ofthe run. The snapshot counts for the DOWNE and STADIUM sectors appear to be rather evenly distributed over all runs. The total DOWNE counts were higher than STADIUM, but this makes sense when correlated to the Scenario Description byRun table inAppendix A. DOWNE put more aircraft inhold than STADIUM over the course ofthe simulation. Taking that into consideration, the allocation ofaircraft by BASIN into the DOWNE and STADIUM sectors appeared to be well-distributed. 

### 7.3 Computer-Generated DataAnalyses The TGF, ARTS, and Host facilities generate alarge quantity ofdata in TGF, CDR, and SAR formats, respectively. Such data provides measures of airspace and/or airport capacity. In addition, some ofthe data, such as air traffic volume data, heading, speed, and altitude data entries, and aircraft mean sector flight times can provide information about the complexity of aparticular airspace or sector. Other data, such as the number and duration of aircraft holds and the number ofconflict alerts within acertain period of time can provide information on the level ofcontroller workload. At the request ofthe LAAEP participating controllers, the above mentioned variable measures were compiled and investigated with the LAAEP simulation data. Because no baseline study was conducted and because simulation runs varied in special events, no statistical inferences could be drawn from the data. However, raw data ispresented and discussed. Those individuals knowledgeable ofLos Angeles Center and TRACON

22Table 8. Total Number ofAircraft Worked Per Sector for 45 Minutes* Run Run Traffic 39 37 20. 

## 19 BASIN STADIUM DOWNE

Length Scenario 1 45m 1 

### 43 31 20 22 26 14 n/a 2 45m 3 

### 41 35 24 24 31 21 n/a 3 45m 

### 1 42 33 22 19 27 19 n/a 4 45m 3 

### 43 35 24 24 33 26 n/a 5 45m 1 

### 42 34 20 23 29 19 n/a 6 45m 4 

### 42 36 24 23 35 31 n/a 7 45m 3 

### 43 36 28 26 34 36 35 8 45m 4 

### 44 36 29 26 36 43 36 9 45m 1 

### 40 31 23 21 24 25 32 10 45m 1 

### 39 34 28 24 30 29 30 11 45m 1 

### 40 33 21 18 22 27 26 12 39m,9s 4 

### 39 34 25 24 30 39 32

*Run 12was 39m, 9s in length. Figures would be slightly higher for 45 minutes.



### 7.3.1.2 Number OfLaxArrivals Since the total number ofaircraft tracks worked included not only arrivals, but departures and overflights as well, the number ofLAX arrivals only was computed. Appendix D contains atable ofLAX arrival numbers by runway for the DOWNE and STADIUM approach control sectors. The majority of aircraft were cleared for the outboard runways, 24R and 25L. The numbers presented in Appendix D correspond, as expected, to the I5-minute interval snapshot data. LAX arrivals, for the most part, were evenly distributed between the DOWNE and STADIUM sectors, down to the runway thresholds. 

### 7.3.2 Heading, Speed,and Altitude Commands The number ofheading, altitude, and speed commands were computed for each run by sector. The first table inAppendix E displays the figures. For data analysis, heading commands not only included Fly heading xxx commands, but also Direct xxx commands inthe en route airspace and Intercept localizer commands in the terminal airspace. Altitude commands not only included climb and descend commands, but also Cross intersection xxx commands. As with theAircraft Tracks Worked data, numbers are difficult to compare because ofthe varying run lengths. The second table in Appendix E shows the frequencies ofheading, altitude, and speed commands for the first 45 minutes ofeach run- with the exception of Run 12, which wasjust over 39 minutes long. Appendix A should be referred to when

24examining the numbers. It should also be kept in mind that individual controller

differences also may have accounted for some ofthe variation inthe numbers. 

### 7.3.3 Number and Duration ofAircraft Holds Appendix F details the number and duration of aircraft holds by run and by sector during the simulation. Appendix A shows which runs had aircraft inholds, but the exact magnitude ofthose special-scripted events is detailed in Appendix F. Run 9 and especially Run II were high-workload runs as far as holding was concerned. Again, the workload questionnaire data support this observation ofthe quantitative data. 

### 7.4 Audio and Video Data Compilation The original plan for videotaping the LAAEP simulation involved recording each sector each run using microcameras mounted to quick-looked, unmanned displays. The resolution provided by the cameras, however, was not adequate enough to read the aircraft data tags on playback. Therefore, adigital scan converter was connected to the TGF's X-windows Plan View Display (XPVD) containing amap of all seven simulated sectors. For each run, an area ofthe map was enlarged (i.e., either BASIN, DOWNE and STADIUM, 19and 20, or 37 and 39) and recorded on avideotape. Audio was also recorded; in stereo for the paired sectors. Table 9 lists the videotape compilation for the entire simulation. Table 9. LAAEP Videotape Compilation Run Sector(s) Traffic Special Events Sample 1 37 & 39 1 20MIT; holding inSector 39 2 Downe 3 10MIT; holding; IFR; 2active runways 3 Basin 1 10MIT; IFR; 2active runways 4 19& 20 3 15MIT 5 Downe & Stadium 1 10MIT; IFR; 2active runways 6 37 & 39 4 15MIT; No D sideon37; stacks over KONZL instead ofin-trail 7 19& 20 3 15MIT; stacks over KONZL instead ofin-trail 8 Basin 4 10MIT; VFR; 4active runways 9 19& 20 1 20MIT with2flows to 15MIT and 1flow 30minutes intotheproblem; holding 10 Basin 1 10MIT; IFR; 2active runways; switched toVFR at 30:00 andto4active runways 11 37 & 39 1 20MIT; holding 12 Downe & Stadium 4 7MIT; VFR; 4active runways

25operations should be able to compare the quantitative data from the simulation to current

operations attheir facilities. 

### 7.3.1 Air Traffic Volume Air traffic volume data were compiled by sector for each run ofthe simulation. The total number of aircraft tracks worked, which included arrivals, departures, and overflights, was determined. In addition, the total number ofLAX arrivals solely was computed. 

### 7.3.1.1 Number ofAircraft Tracks Worked Table 7 depicts the total number of aircraft worked per sector per run ofthe simulation. Data were not available for the DOWNE sector for Runs 1-6 due to a software problem. Although between-sector numbers can be examined for each run, not a great deal can be inferred, because run lengths varied. Table 8, however, shows throughput data for the first 45 minutes ofeach run. Run 12was only 39 minutes in length, so one would expect the numbers presented to be slightly higher for a45-minute period. The counts should still be compared with caution, due to different events that transpired in each run (refer to Appendix A). On the average, despite traffic level and special event differences, Sector 39 controlled the most aircraft per run. Questionnaire responses from the Center controllers support this observation.

Table 7. Total Number ofAircraft Worked Per Sector for Entire Run Run Run Traffic 

### 39 37 

### 20 19 BASIN STADIUM DOWNE

Length Scenario 1 67m, lIs 1 58 39 31 34 37 20 n/a 

### 2 46m,27s 3 43 26 25 24 33 21 n/a 

### 3 71m,6s 1 55 40 31 31 41 31 n/a 4 64m,2s 3 57 41 32 31 46 33 n/a 5 56m,39s 1 53 41 29 30 35 26 n/a 6 51m,31s 4 48 43 32 29 40 36 n/a 7 61m,16s 3 55 44 34 34 45 48 48 8 Slm,29s 4 49 42 32 30 39 48 43 

### 9 60m,6s 1 50 42 32 29 39 46 43 10 49m,37s 1 51 37 30 27 33 33 43 11 62m,43s 1 51 42 31 28 30 39 31 

### 12 39m,9s 4 39 34 25 24 30 39 32

23

### 8. SUMMARY

The full impact of an ATC procedural change on the ATC system is seldom clear until the change is actually implemented, or, where technology allows, simulated. The scope ofthe LAAEP plan was such that adequate testing and evaluation could not be done in the facility DYSIM laboratories. LAAEP representatives therefore recruited testing support from the FAA's William J. Hughes Technical Center, inNew Jersey. The Technical Center's ARTCC and TRACON laboratories provided the air traffic controllers with fully-operational ATC displays and interfacility interface capabilities. Six FPL controllers from ZLA and four FPL controllers from the SOCAL TRACON participated in the real-time simulation. The controllers were not only test subjects, but also evaluators ofthe LAAEP arrival route modifications and airspace resectorization plans. Over afour-day period, controllers rotated through sector positions and experienced varying levels oftraffic complexity. They intentionally scripted special events into the traffic scenarios, such as runway closings and the holding of aircraft, to test the limitations ofthe proposed procedures. Controllers worked together to resolve issues and make recommendations for aworkable operation. Data were collected from laboratory computer systems and from questionnaires. However, because simulation run events varied and no baseline data was collected prior to the test, a limited amount of statistical analyses could be done. Objective data relating to capacity, sector complexity, and workload were collected in TGF, SAR, and CDR formats. Specifically, throughput, controller action, and special event data were compiled and the raw figures presented. The numbers must be correlated with specific run events, detailed inAppendix A, to be interpreted. Controller workload data was collected following each oftwelve runs comprising the simulation. For the most part, workload did seem to decrease as simulation experience progressed. Special scenario events seemed to affect workload more than the number of aircraft involved in a scenario. This was demonstrated by the high workload ratings associated with Runs 9 and 11, which were Scenario 1runs, with aconsiderable amount ofholding. For some ofthe Center controllers, increased mile-in-trail requirements resulted inhigher perceived workload. Several controllers commented that stacking the aircraft in lieu of in-trail separation was the more effective strategy, which also reduced workload. In the terminal area, the new BASIN sector was perceived by most controllers to be busy, but workable. Some difficulty was experienced, however, inkeeping the most efficient flow of aircraft inthe absence ofclearly defined procedures. Both the DOWNE and STADIUM sectors were considered very manageable by most controllers. Since only a limited amount ofdata analysis could be done, controller feedback on the LAAEP plan proved to be an invaluable source of information. All ten controllers agreed the LAAEP proposed procedures could be safely conducted in an actual ATC environment. New procedures training was considered acrucial component, however.



### 26The stacking strategy was recommended by several Center controllers to be part ofthe

training. All controllers felt they had to change their 

### usual work strategies to some degree to work the traffic with the modified airspace. As far as the distribution ofworkload between sectors was concerned, several controllers felt Sector 19was not as busy as pre-LAAEP due to amore shared workload between Sectors 19and 20. Most TRACON controllers felt the LAAEP demonstrated aworkable balance oftraffic between the DOWNE and STADIUM sectors, with the help ofthe new BASIN sector. The majority of controllers believed the simulated LAAEP procedures would accommodate future air traffic growth. One controller believed the procedures offered an overall significant improvement with the TRACON. Another controller stated that the Center was able to work ahigher flow oftraffic with the TRACON able to accept the flow with new procedures. Efficiency was akey simulation outcome mentioned by many controllers. In summary, the users- the ZLA and SOCAL controllers, supported the simulated LAAEP procedures. 

### 9. CONCLUSIONS This simulation tested the proposed LAAEP east arrival route modifications and new BASIN sector for workability, efficiency, and safety. Seven sectors, four en route and three terminal, were simulated concurrently. Participating controllers worked arrival traffic atvarying degrees ofcomplexity. Special events, such as runway closings, aircraft inholding, mile-in-trail restrictions, and various combinations of such events were simulated and evaluated by the controllers to assess the limitations ofthe LAAEP plan. All ten FPL controller participants from the LA Center and SOCAL TRACON facilities agreed the proposed procedures could be safely controlled in an actual operational environment. They collectively devised performance strategies and training recommendations for all LAX controllers to ultimately enable a successful operation.

27ARTCC

ARTS ATA ATC CDR DVRS FAA FPL IFR LAAEP LAX MIT NAS SAR SOCAL SPW TG TGF TMU TRACON VFR XPVD ZLA Acronym List

Air Route Traffic Control Center Automated Radar Terminal System Air Traffic Assistant Air Traffic Control Continuous Data Recording Digital Voice Recording System Federal Aviation Administration Full Performance Level Instrument Flight Rules Los Angeles Arrival Enhancement Project Los Angeles International Airport Miles in Trail National Airspace System System Analysis Recorder Southern California Simulation Pilot Workstation Target Generation Target Generation Facility Traffic Management Unit Terminal Radar Approach Control Visual Flight Rules X-Windows Plan View Display Los Angeles Center



### 28References

Federal Aviation Administration (1997). Environmental Assessment: Los Angeles International Airport Arrival Enhancement Project. Federal Aviation Administration, Washington, DC. System Resources Corporation & System Simulation Support Branch (1996). [On-Line]. Target Generation Facility (TGF) Overview. Available: http://www.tgf.tc.faa.gov/doc/overview/overview .html# 1.

29