Final Report

For

Domestic Reduced Vertical Separation Minima (DRVSM)

Third Simulation

**July 28, 2003**

**Prepared for:**

Federal Aviation Administration

Air Traffic Planning and Procedures Program (ATP-6)

800 Independence Ave

S.W. Washington DC 20591

Prepared By:

Human-in-Loop Simulation & Analysis Group, ACB-330

Human Solutions, Incorporated

TABLE OF CONTENTS

+---+-------------------------------------------------------------+----+
| 1 | Introduction                                                | 1  |
| \ |                                                             |    |
| . |                                                             |    |
+---+-------------------------------------------------------------+----+
|   | 1.1 Background                                              | 1  |
+---+-------------------------------------------------------------+----+
|   | 1.2 Scope of the Report                                     | 1  |
+---+-------------------------------------------------------------+----+
|   | 1.3 Document Organization                                   | 1  |
+---+-------------------------------------------------------------+----+
| 2 | Simulation Overview                                         | 2  |
| \ |                                                             |    |
| . |                                                             |    |
+---+-------------------------------------------------------------+----+
|   | 2.1 Objectives                                              | 2  |
+---+-------------------------------------------------------------+----+
|   | 2.2 Airspace                                                | 2  |
+---+-------------------------------------------------------------+----+
|   | 2.3 Scenarios                                               | 3  |
+---+-------------------------------------------------------------+----+
|   | 2.4 Scenario Conditions                                     | 3  |
+---+-------------------------------------------------------------+----+
|   | 2.5 Participants                                            | 3  |
+---+-------------------------------------------------------------+----+
|   | 2.6 Measures                                                | 5  |
+---+-------------------------------------------------------------+----+
|   | 2.7 Simulation Environment                                  | 6  |
+---+-------------------------------------------------------------+----+
|   | 2.8 Data Collection                                         | 7  |
+---+-------------------------------------------------------------+----+
| 3 | Results for Objective 1: Assess DRVSM Procedures            | 8  |
| \ |                                                             |    |
| . |                                                             |    |
+---+-------------------------------------------------------------+----+
|   | 3.1 Preliminary DRVSM Procedures                            | 8  |
+---+-------------------------------------------------------------+----+
|   | 3.2 Procedures Assessment                                   | 9  |
+---+-------------------------------------------------------------+----+
| 4 | Results for Objective 2: Gain Insight for the Development   | 11 |
| \ | of DRVSM Training Plans                                     |    |
| . |                                                             |    |
+---+-------------------------------------------------------------+----+
|   | 4.1 Verbal Feedback on DRVSM Training Approach              | 11 |
+---+-------------------------------------------------------------+----+
|   | 2.  Verbal Feedback and Written Comments for Controller     | 12 |
|   |     Training Emphasis                                       |    |
+---+-------------------------------------------------------------+----+
|   | 3.  Verbal Feedback for Supervisory Training Emphasis       | 13 |
+---+-------------------------------------------------------------+----+
|   | 4.  Controller Perceptions of DRVSM Skills                  | 13 |
+---+-------------------------------------------------------------+----+
|   | 5.  Analysis of Training Duration Ratings                   | 17 |
+---+-------------------------------------------------------------+----+
| 5 | Results for Objective 3: Assess the Impacts of Using RVSM   | 18 |
| \ | Altitudes in CVS Environment                                |    |
| . |                                                             |    |
+---+-------------------------------------------------------------+----+
|   | 1.  Overview of Tactical Use Results                        | 18 |
+---+-------------------------------------------------------------+----+
|   | 2.  Use of RVSM Altitudes Tactically                        | 19 |
+---+-------------------------------------------------------------+----+
|   | 3.  Impact of Tactical Use                                  | 19 |
+---+-------------------------------------------------------------+----+
|   | 4.  Benefits and Risks for Tactical Use                     | 20 |
+---+-------------------------------------------------------------+----+
|   | 5.  Overall Utility of Implementing Tactical Use            | 21 |
+---+-------------------------------------------------------------+----+
| 6 | Conclusions                                                 | 21 |
| \ |                                                             |    |
| . |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
| A |                                                             | 23 |
| p |                                                             |    |
| p |                                                             |    |
| e |                                                             |    |
| n |                                                             |    |
| d |                                                             |    |
| i |                                                             |    |
| x |                                                             |    |
| A |                                                             |    |
| : |                                                             |    |
| D |                                                             |    |
| r |                                                             |    |
| a |                                                             |    |
| f |                                                             |    |
| t |                                                             |    |
| C |                                                             |    |
| o |                                                             |    |
| n |                                                             |    |
| t |                                                             |    |
| r |                                                             |    |
| o |                                                             |    |
| l |                                                             |    |
| l |                                                             |    |
| e |                                                             |    |
| r |                                                             |    |
| a |                                                             |    |
| n |                                                             |    |
| d |                                                             |    |
| P |                                                             |    |
| i |                                                             |    |
| l |                                                             |    |
| o |                                                             |    |
| t |                                                             |    |
| P |                                                             |    |
| h |                                                             |    |
| r |                                                             |    |
| a |                                                             |    |
| s |                                                             |    |
| e |                                                             |    |
| o |                                                             |    |
| l |                                                             |    |
| o |                                                             |    |
| g |                                                             |    |
| y |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+
|   |                                                             |    |
+---+-------------------------------------------------------------+----+

LIST OF FIGURES AND TABLES

  ---------- -------------------------------------------------------- -----
  Figure 2-1 Simulated ZDC Airspace                                   2

  Table 2-1  Description of Scenario Runs Derived from SAR Data       4,5

  Table 2-2  Measures Per Objective                                   6

  Table 2-3  Summary of Questionnaires                                7

  Table 4-1  DRVSM Skills/Changes Criteria and Rating Scales          14

  Table 4-2  DRVSM Skills/Changes Training Relevance Ratings          15

  Table 4-3  DRVSM Skills/Changes Ease of Training Ratings            16

  Table 4-4  Training Duration Ratings                                17

  Table 4-5  Distribution of Training Duration Ratings                18

  Table 5-1  Impact of Tactical Use                                   19

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      

                                                                      
  ---------- -------------------------------------------------------- -----

# INTRODUCTION 

## 1.1 Background {#background .unnumbered}

Reduced Vertical Separation Minimum (RVSM) is an ICAO-approved concept
that reduces the vertical separation standard from 2000 feet to 1000
feet between flight level (FL) 290 and FL410. RVSM adds six flight
levels between FL290 and FL410, thereby increasing airspace throughput
and allowing more flexibility for controllers to grant user preferred
altitudes. RVSM has been implemented in the North Atlantic and Pacific
oceanic airspace, and was implemented in domestic European airspace in
January 2002. Domestic RVSM (DRVSM) is a high priority for the FAA's
Operational Evolution Plan (OEP); however, the impact on en route
controllers in high-density domestic U.S. airspace needs to be
understood.

Under the auspices of the Air Traffic Planning and Procedures Program
(ATP), a series of human-in-the-loop (HITL) simulations have been
conducted to investigate the operational impacts on en route controllers
of implementing RVSM in domestic U.S. airspace. The first of the series
of HITL simulations took place October 24 -- 30, 2001, at the William J.
Hughes Technical Center (WJHTC) Display System Facility (DSF). This
first simulation focused primarily on identifying and understanding the
impacts of different DRVSM altitude bands on en route controllers. The
second DRVSM simulation took place June 3 - 7, 2002, and focused on
understanding the impact of non-RVSM-approved aircraft under heavy
traffic conditions. The third DRVSM simulation occurred June 9 -- 12,
2003. The primary focus of this simulation was air traffic procedures
and controller training.

## 1.2 Scope of the Report  {#scope-of-the-report .unnumbered}

This Final Report provides the results of the third DRVSM simulation.
Data collection for this simulation consisted primarily of subjective
data in the form of controller responses to questionnaires and
controller comments during debrief sessions following each simulation
run.

## 1.3 Document Organization {#document-organization .unnumbered}

This document is organized in six sections and one appendix. Section 2
provides an overview of the simulation structure, environment, and
conduct. The results of the simulation are provided in Sections 3
through 5, and Section 6 provides conclusions*.* Appendix A provides the
draft phraseology used during the simulation.

# 2. SIMULATION OVERVIEW {#simulation-overview .unnumbered}

## 2.1 Objectives {#objectives .unnumbered}

The objectives for the third DRVSM simulation were to:

1.  Assess DRVSM procedures;

2.  Gain insight for the development of DRVSM training plans; and

3.  Assess the impacts of using the new altitudes tactically in a
    conventional vertical separation (CVS) environment.

## 2.2 Airspace {#airspace .unnumbered}

This third DRVSM simulation was designed as a real-time, high fidelity,
HITL, en route simulation. The simulated airspace was based on four
adjacent sectors in Washington Air Route Traffic Control Center (ZDC).
ZDC sectors 10 (Bay), 12 (Brooke), 16 (Hopewell), and 38 (Tar River) had
the appropriate characteristics for this study. Sector 10, combined at
Sector 12 for the purposes of the simulation, encompasses FL240 and
above. Sector 16 (Hopewell) is a high altitude sector whose area of
responsibility includes FL280 and above. Sector 38 (Tar River High) is
also a high altitude sector (FL240 and above). Figure 2-1 provides a
depiction of the airspace used.

FIGURE 2-1. Simulated ZDC Airspace

##   {#section .unnumbered}

## 2.3 Scenarios

The traffic scenarios provided a range of traffic density and level of
complexity. Four distinct traffic scenarios were developed from flight
plans extracted from data analysis and reduction tool (DART) runs of ZDC
system analysis and recording (SAR) tapes. The data allowed for the
realistic representation of sector boundaries, jet routes, and fixes for
the chosen and adjacent sectors. ZDC personnel and air traffic control
(ATC) subject matter experts assisted in developing and validating the
scenarios, and in ensuring that the traffic levels represented realistic
conditions. A total of eight runs using the four distinct traffic
scenarios were developed, tested, and used for this third DRVSM
simulation. The four traffic scenarios (referred to as traffic samples
1a, 2a, 3a, and 4a) were each used to create an additional scenario
(referred to as 1b, 2b, 3b, and 4b, respectively). Although based on the
same raw traffic sample, the "b" samples differed from their "a"
counterparts in several ways. First, aircraft identities were changed in
the "b" traffic samples to give a different appearance to each scenario.
Second, the aircraft designated as non-RVSM-approved differed between
the "a" and "b" samples. Additionally, unique events were designed into
each of the eight traffic samples in order to allow specific procedures
to be exercised and assessed.

##  2.4 Scenario Conditions {#scenario-conditions .unnumbered}

Table 2-1 provides a definition of the eight different runs performed
during the simulation, including the traffic sample from which each run
was developed, the objective (or objectives) the run addresses, and
scripted events included in each run.

## 2.5 Participants {#participants .unnumbered}

Six certified professional controllers (CPC) from ZDC who work the
simulated sectors (and who were not involved in the definition and
validation of the scenarios) staffed the radar (R) and radar associate
(RA) positions during the simulation runs. The participating controllers
interacted with individuals functioning as pilots (simulation pilots)
and ghost controllers. The simulation pilots manipulated
computer-generated targets in response to controller instructions. Ghost
controllers performed the automation entries and voice communications
associated with the airspace surrounding the simulated sectors. ZDC
controllers who participated in the scenario development and validation
served as expert observers during the simulation to record the
participants' responses to the scripted events, as well as any
procedural or operational questions or concerns that arose during the
runs.

TABLE 2-1. Description of Scenario Runs Derived from SAR Data

+----------+---------+------------+-----------------------+-----------+
| **SC     | **      | **MAPS TO  | > **EVENT**           | *         |
| ENARIO** | TRAFFIC | OB         |                       | *SECTOR** |
|          | S       | JECTIVES** |                       |           |
|          | AMPLE** |            |                       |           |
+==========+=========+============+=======================+===========+
| 1a       | 1       | 1, 2       | > Military            | > ALL     |
|          |         |            | > non-approved        |           |
|          |         |            | > aircraft at RVSM    |           |
|          |         |            | > altitude            |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft unable to  | 16        |
|          |         |            | > maintain RVSM due   |           |
|          |         |            | > to equipment        |           |
|          |         |            | > failure             |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 38        |
|          |         |            | > aircraft requesting |           |
|          |         |            | > climb above FL430   |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 12        |
|          |         |            | > aircraft requesting |           |
|          |         |            | > descent below FL290 |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft            | 12        |
|          |         |            | > experiencing wake   |           |
|          |         |            | > turbulence requests |           |
|          |         |            | > offset              |           |
+----------+---------+------------+-----------------------+-----------+
| 1b       | 1       | 1, 2       | > Military            | ALL       |
|          |         |            | > non-approved        |           |
|          |         |            | > aircraft at FL350   |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 38        |
|          |         |            | > aircraft requesting |           |
|          |         |            | > descent below FL290 |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 38        |
|          |         |            | > aircraft at         |           |
|          |         |            | > base/ceiling        |           |
|          |         |            | > altitude            |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft unable to  | 12        |
|          |         |            | > maintain RVSM due   |           |
|          |         |            | > to equipment        |           |
|          |         |            | > failure             |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 12        |
|          |         |            | > aircraft at         |           |
|          |         |            | > base/ceiling        |           |
|          |         |            | > altitude            |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved ferry  | 12        |
|          |         |            | > flight to be        |           |
|          |         |            | > accommodated at     |           |
|          |         |            | > RVSM altitude       |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 16        |
|          |         |            | > aircraft requesting |           |
|          |         |            | > climb above FL430   |           |
+----------+---------+------------+-----------------------+-----------+
| 2a       | 2       | 3          | > 60% aircraft are    | ALL       |
|          |         |            | > RVSM-approved       |           |
+----------+---------+------------+-----------------------+-----------+
| 2b       | 2       | 3          | > 70% aircraft are    | ALL       |
|          |         |            | > RVSM-approved       |           |
+----------+---------+------------+-----------------------+-----------+
| 3a       | 3       | 1, 2       | > Aircraft unable to  | 38        |
|          |         |            | > maintain RVSM due   |           |
|          |         |            | > to equipment        |           |
|          |         |            | > failure             |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft            | 38        |
|          |         |            | > experiencing wake   |           |
|          |         |            | > turbulence requests |           |
|          |         |            | > offset              |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft            | 16        |
|          |         |            | > experiencing wake   |           |
|          |         |            | > turbulence requests |           |
|          |         |            | > offset              |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 16        |
|          |         |            | > aircraft at         |           |
|          |         |            | > base/ceiling        |           |
|          |         |            | > altitude            |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 12        |
|          |         |            | > lifeguard flight to |           |
|          |         |            | > be accommodated at  |           |
|          |         |            | > RVSM altitude       |           |
+----------+---------+------------+-----------------------+-----------+

TABLE 2-1. Description of Scenario Runs Derived from SAR Data (Cont'd)

+----------+---------+------------+-----------------------+-----------+
| **SC     | **      | **MAPS TO  | > **EVENT**           | *         |
| ENARIO** | TRAFFIC | OB         |                       | *SECTOR** |
|          | S       | JECTIVES** |                       |           |
|          | AMPLE** |            |                       |           |
+==========+=========+============+=======================+===========+
| 3b       | 3       | 1, 2       | > Aircraft            | 38        |
|          |         |            | > experiencing wake   |           |
|          |         |            | > turbulence requests |           |
|          |         |            | > offset              |           |
|          |         |            | >                     |           |
|          |         |            | > or descent          |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft unable to  | 38        |
|          |         |            | > maintain RVSM due   |           |
|          |         |            | > to equipment        |           |
|          |         |            | > failure             |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft            | 16        |
|          |         |            | > experiencing wake   |           |
|          |         |            | > turbulence requests |           |
|          |         |            | > offset              |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved ferry  | 16        |
|          |         |            | > flight to be        |           |
|          |         |            | > accommodated at     |           |
|          |         |            | > RVSM altitude       |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved ferry  | 12        |
|          |         |            | > flight to be        |           |
|          |         |            | > accommodated at     |           |
|          |         |            | > RVSM altitude       |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 12        |
|          |         |            | > aircraft requesting |           |
|          |         |            | > climb above FL430   |           |
+----------+---------+------------+-----------------------+-----------+
| 4a       | 4       |            | > Aircraft unable to  | 12        |
|          |         |            | > maintain RVSM due   |           |
|          |         |            | > to equipment        |           |
|          |         |            | > failure             |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 38        |
|          |         |            | > lifeguard flight to |           |
|          |         |            | > be accommodated at  |           |
|          |         |            | > RVSM altitude       |           |
+----------+---------+------------+-----------------------+-----------+
| 4b       | 4       |            | > Aircraft            | 12        |
|          |         |            | > experiencing wake   |           |
|          |         |            | > turbulence requests |           |
|          |         |            | > offset              |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Aircraft unable to  | 16        |
|          |         |            | > maintain RVSM due   |           |
|          |         |            | > to equipment        |           |
|          |         |            | > failure             |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 16        |
|          |         |            | > aircraft at         |           |
|          |         |            | > base/ceiling        |           |
|          |         |            | > altitude            |           |
+----------+---------+------------+-----------------------+-----------+
|          |         |            | > Non-approved        | 38        |
|          |         |            | > lifeguard flight to |           |
|          |         |            | > be accommodated at  |           |
|          |         |            | > RVSM altitude       |           |
+----------+---------+------------+-----------------------+-----------+

##   {#section-1 .unnumbered}

## 2.6 Measures {#measures .unnumbered}

Table 2-2 depicts the subjective and objective data that were collected
during the simulation.

TABLE 2-2. Measures Per Objective

+------------------+--------------------+-------------+--------------+
| **Objectives**   | **Data to Be       | **Source of | **Measures** |
|                  | Collected**        | Data**      |              |
+==================+====================+=============+==============+
| 1.  Assess DRVSM | -   Operational    | -           | -   Not      |
|     procedures   |     input on draft |  Controller |              |
|                  |     procedures     |             |   applicable |
|                  |                    |    comments |              |
|                  | -   Procedural     |             |              |
|                  |     issues or      | ```{=html}  |              |
|                  |     questions not  | <!-- -->    |              |
|                  |     covered in     | ```         |              |
|                  |     draft          | -   Que     |              |
|                  |     procedures     | stionnaires |              |
|                  |                    |             |              |
|                  | -   Ideas for      |             |              |
|                  |     possible       |             |              |
|                  |     solutions      |             |              |
+------------------+--------------------+-------------+--------------+
| 2.  Gain insight | -   Training       | -           | -   Change   |
|     for the      |     ratings prior  |  Controller |     in       |
|     development  |     to first DRVSM |             |     ratings  |
|     of DRVSM     |     runs to        |    comments |     as       |
|     training     |     capture        |             |     number   |
|     plans        |     perceptions    | -   Que     |     of DRVSM |
|                  |                    | stionnaires |     runs and |
|                  | -   Ratings to     |             |              |
|                  |     assess number  |             | controllers' |
|                  |     of DYSIM       |             |              |
|                  |     problems       |             |   experience |
|                  |     needed for     |             |              |
|                  |     workforce      |             |    increases |
|                  |     training       |             |              |
|                  |                    |             |              |
|                  | -   Suggested      |             |              |
|                  |     improvements   |             |              |
|                  |     for training   |             |              |
+------------------+--------------------+-------------+--------------+
| 3.  Assess the   | -   Operational    | -           | -   Workload |
|     impacts of   |     input on       |  Controller |     ratings  |
|     using the    |     benefit and    |             |              |
|     new          |     risks of       |    comments | -            |
|     altitudes    |     tactical use   |             |   Complexity |
|     tactically   |                    | -   Que     |     ratings  |
|     in a         | -   Ratings of     | stionnaires |              |
|     conventional |     impact of      |             | -            |
|     vertical     |     tactical use   |             |    Potential |
|     separation   |                    |             |     for      |
|     (CVS)        |                    |             |     error    |
|     environment  |                    |             |     ratings  |
+------------------+--------------------+-------------+--------------+

## 2.7 Simulation Environment {#simulation-environment .unnumbered}

The simulation was performed in the DSF at the WJHTC. The display system
replacement (DSR), the host computer system (HCS), the voice switching
and control system (VSCS), and the user request evaluation tool (URET)
were, with few differences, configured identically to ZDC's systems.
Flight progress strips were not used, since ZDC primarily employs the
electronic flight data provided by URET. The target generation facility
(TGF) provided high fidelity target generation and movement.

The DSR/Host/URET environment in the DSF was identical to ZDC's for the
two tactical use runs, since this reflects the tentative plan for
tactical use, should tactical use be implemented. For the six DRVSM
runs, however, three modifications were made so that the systems behave
in the manner desired for the DRVSM implementation. These changes are:

1.  A symbol in the data block was used during the DRVSM scenario runs
    to indicate that an aircraft was not RVSM-approved. Since it is
    expected that significantly more aircraft will be approved than not,
    coding the non-RVSM-approved aircraft reduces clutter on the
    controller's situation display. The indicator was a coral box around
    the fourth character in the second line of the data block for
    non-RVSM-approved aircraft. The Air Traffic DSR Evolution Team
    (ATDET), the team responsible for the final computer human interface
    (CHI) design for DRVSM, provided this design for the indicator.

2.  The conflict alert (CA) logic used by the HCS was updated to
    accurately reflect the revised vertical separation standards for
    DRVSM.

3.  Because the conflict probe logic used by URET had not yet been
    updated, the conflict probe and trial planning feature had to be
    deactivated for the purposes of this simulation to avoid giving
    erroneous alerts for pairs of RVSM-approved aircraft with at least
    1000-foot vertical separation.

## 2.8 Data Collection  {#data-collection .unnumbered}

With the exception of data on separation violations[^1], only subjective
data were collected. The subjective data collected include responses to
questionnaires (completed by the six ZDC participants) and controller
comments during the debrief sessions (from the six ZDC participants and
the ZDC expert observers). Table 2-3 details each type of questionnaire
used during this simulation, the frequency with which each was used, and
their distinct purposes.

TABLE 2-3. Summary of Questionnaires

  ----------------------------------------------------------------------------------------
  **Method**      **Users**      **Frequency**   **Completed**   **Purpose**
  --------------- -------------- --------------- --------------- -------------------------
  Background      Participants   Once            After           Gather controller
  Questionnaire                                  orientation     demographic information.
                                                 briefing        

  Tactical Use    Participants   Once            After the       Provide controller
  Questionnaire                                  completion of   feedback on the
                                                 the second      operational impacts and
                                                 tactical use    benefits of tactical use.
                                                 run             

  Pre-RVSM Run    Participants   Once            After RVSM      Collect controller
  Questionnaire                                  classroom       perceptions of RVSM prior
                                                 training and    to hands-on experience,
                                                 prior to the    so that perceptions can
                                                 first RVSM run  be compared to actual
                                                                 experiences after RVSM
                                                                 runs.

  Post-Run        Participants   Every run       After each run  Elicit controller
  Questionnaire                                                  comments related to
                                                                 procedures and training
                                                                 implications.
  ----------------------------------------------------------------------------------------

# Results for objective 1: Assess drvsm procedures

A primary goal for the third DRVSM simulation was to assess the air
traffic procedures needed for DRVSM. Accordingly, preliminary procedures
and corresponding phraseology were drafted for use during the six DRVSM
runs. Participants were provided with a procedures familiarization
briefing prior to the first DRVSM run.

The preliminary DRVSM procedures used during the simulation are
presented in Section 3.1, while the assessment results are in Section
3.2. The phraseology drafted for use during this simulation is included
in Appendix A.

## 3.1 Preliminary DRVSM Procedures  {#preliminary-drvsm-procedures .unnumbered}

The preliminary procedures assessed during the simulation were based on
the prior efforts of the DRVSM Procedures Workgroup. This group
identified five primary procedural topics and provided suggestions for
each topic. The five procedural topic areas are:

1.  Accommodation of non-RVSM-approved military, lifeguard, or ferry
    flight aircraft.

2.  Non-RVSM-approved aircraft transitioning through RVSM airspace.

3.  Base/ceiling coordination with abutting sectors.

4.  Aircraft loss of equipment required for RVSM operation.

5.  Wake turbulence in RVSM airspace.

To ensure that each participant was able to exercise each of the
procedures, special events were scripted into each run to create the
opportunity to use one or more of the procedures. (See Table 2-1.) The
simulation design was specifically created to ensure that over the
course of the six DRVSM runs, each participant would have the
opportunity to exercise all of the procedures at least once while
staffing the R-side position, and at least once while staffing the
RA-side position.

### Accommodation of Non-RVSM Approved Military or Lifeguard Aircraft

*[Draft Procedure]{.underline}*: Non-RVSM approved military, lifeguard,
and ferry flights are to be accommodated within RVSM airspace, traffic
permitting[^2]. Vertical separation of 2000 feet shall be applied to
such aircraft.

### Non-RVSM Approved Aircraft Transitioning Through RVSM Airspace

*[Draft Procedure]{.underline}*: Non-RVSM approved aircraft, other than
military, lifeguard, or other excepted aircraft, should be permitted to
transition to altitudes above or below RVSM airspace, traffic
permitting^2^. Aircraft shall only be leveled off in climb or descent
for separation purposes, not for the purpose of burning off fuel to
accommodate the user. Prior to climbing into RVSM airspace, aircraft
shall be coordinated one sector ahead to insure system acceptance for
transition. Vertical separation of 2000 feet shall be applied to such
aircraft.

For the purposes of this simulation, participants were instructed to
coordinate non-RVSM-approved aircraft with the next sector. For a
non-RVSM-approved aircraft already in RVSM airspace, coordination was to
occur prior to handoff. For a non-RVSM-approved aircraft requesting
clearance, but not yet cleared into RVSM airspace, coordination with the
next sector was to occur prior to the initial clearance into RVSM
airspace. The participants were also instructed to inform the supervisor
of the presence of a non-RVSM-approved aircraft in their airspace. It is
expected that supervisors will perform forward coordination with
adjacent affected areas or facilities and will coordinate with traffic
flow management personnel, as needed, when a non-approved aircraft is in
RVSM airspace.

### Base/Ceiling Coordination Requirements with Abutting Sectors

*[Draft Procedure]{.underline}*: A non-RVSM approved aircraft operating
at a sector's base or ceiling altitude requires coordination with the
sector that owns the airspace immediately above or below so as to insure
2000 feet separation is maintained from any aircraft operating in the
adjacent airspace.

For a non-RVSM approved aircraft flying at the base or ceiling altitude
of the next sector, participants were instructed to perform a point out
and verbally coordinate with the affected sector. Depending on
stratifications or shelves, this could require coordination with more
than one sector.

### Aircraft Loss of Equipment Required for RVSM Operation

*[Draft Procedure]{.underline}*: An aircraft operating within RVSM
airspace that loses an equipment component required for RVSM flight and
advises ATC "unable RVSM due to equipment" shall be removed from RVSM
airspace. Equipment suffix shall be amended to reflect the revised
status of aircraft equipage.

### Aircraft Experiencing Wake Turbulence in RVSM Airspace

*[Draft Procedure]{.underline}*: Aircraft experiencing wake turbulence
in RVSM airspace shall advise ATC of occurrence and pilot request.
Controller should approve pilot request, traffic permitting.

## Procedures Assessment

Participants were asked to provide both written and verbal feedback on
the draft procedures. The questionnaire the controllers completed after
each DRVSM run requested feedback on the use and effectiveness of the
preliminary procedures, the adequacy of the training provided on the
procedures, and any additional procedural issues not covered in the
draft procedures. Procedural topics were then discussed during the
debrief session following each run.

Overall, participants felt the draft procedures were appropriate and
that the training they received was adequate. Each of the team's
suggested improvements and additions to the draft procedures, detailed
in the following sections, related to the presence of non-RVSM-approved
aircraft in RVSM airspace.

### Pilot Phraseology for Non-RVSM-Approved Aircraft 

Controller awareness of the presence of a non-RVSM-approved aircraft in
RVSM airspace is critical to maintaining the safe operation of the NAS
under DRVSM. Accordingly, the data block for a non-approved aircraft in
RVSM airspace is color coded to allow the controller to discriminate
aircraft that require 2000-foot vertical separation. The verbal
coordination procedures between controllers are designed to reinforce
controller awareness of non-RVSM-approved aircraft entering their
sector. As an added measure, the team suggested that pilots of
non-RVSM-approved aircraft identify themselves as such on initial radio
contact with each sector traversed while in RVSM airspace. During
subsequent DRVSM runs, this procedure was followed and the participants
agreed this additional level of redundancy is appropriate, given the
safety-critical nature of the information.

### Supervisory Notification of Non-RVSM-Approved Aircraft

The participants suggested that the procedure for informing the
operations supervisor when a non-RVSM-approved aircraft is operating in
RVSM airspace be streamlined. Three distinct cases were discussed: (1) a
non-RVSM-approved aircraft to be accommodated at level flight at a RVSM
altitude, (2) a non-RVSM-approved aircraft requesting to transition
through RVSM airspace, and (3) a RVSM-approved aircraft that experiences
an equipment loss that changes its RVSM approval status.

Non-RVSM-approved military, lifeguard, or ferry flights desiring to be
accommodated at an RVSM altitude (e.g., requesting a final altitude of
FL320) will be expected to pre-coordinate their request in advance with
the appropriate traffic flow management personnel (the System Command
Center and/or the Traffic Management Unit in the affected center). In
this case, traffic flow personnel should coordinate the approval of such
a request with the operations supervisors of the affected areas of
specialization. The supervisor would then be expected to alert the
affected sectors, and to adjust sector staffing, if necessary, to
accommodate the additional workload represented by the presence of a
non-RVSM-approved aircraft in RVSM airspace. The operations supervisor
of the first area traversed by the non-approved aircraft would also
forward coordinate with the supervisor of the next area. Thus, the
participants suggested that the only situation that requires a
controller to notify the supervisor is when a non-RVSM-approved flight
in RVSM airspace is inbound from another facility.

In the case of a non-RVSM-approved aircraft requesting a clearance to
transition through RVSM airspace, the controller should gain approval
from the next sector prior to granting a clearance into RVSM airspace,
and then notify the supervisor. Similarly, the controller should notify
the supervisor in the event that an RVSM-approved aircraft loses
equipment required for RVSM operation.

The participants' suggested revision to the procedure is for the
controller to notify the supervisor only in the following cases:

1.  A non-RVSM-approved aircraft is inbound from another facility.

2.  A non-RVSM-approved aircraft has started to climb or descend into
    RVSM airspace.

3.  An approved aircraft loses equipment required for RVSM operation.

### Inter-Sector Coordination of Non-RVSM-Approved Aircraft

The participants suggested that the importance of sector-to-sector
coordination of an approval request (APREQ) for a non-RVSM-approved
aircraft needs to be stressed during classroom training, particularly
since DYSIM does not support interaction between adjacent sectors.

### 3.2.4 Aircraft Loss of Equipment Required for RVSM Operation {#aircraft-loss-of-equipment-required-for-rvsm-operation-1 .unnumbered}

The participants found that sometimes the best solution to achieve
2000-foot vertical separation for an aircraft that has lost its RVSM
capability was to climb the aircraft until traffic cleared, allowing the
aircraft to subsequently be cleared from RVSM airspace. They suggested
clarifying during training that this is an acceptable solution to
dealing with such aircraft situations.

# Results for objective 2: Gain insight for the development of DRVSM training plans

A significant focus of near-term DRVSM program efforts will be the
establishment of the DRVSM training plan for the en route controller
workforce. The third DRVSM simulation was designed to obtain as much
field input as possible on training needs. Three types of subjective
data were collected for training: verbal controller feedback from the
debrief sessions that followed each DRVSM simulation run, written
controller comments, and participants' numerical ratings collected via
questionnaire responses. Insights were gained on training approaches,
areas to emphasize in controller training, areas to emphasize in
supervisory training, specific skills necessary for DRVSM, and the
duration of DRVSM simulation training.

## Verbal Feedback on DRVSM Training Approach

The participants' inputs on training approaches for DRVSM include:

-   Like recent CPC training programs for DSR and URET, DRVSM training
    should encompass computer-based-instruction (CBI), classroom
    training, and simulation training. While these recent training
    programs focused on the ability to interact with new systems and
    practice integrating them into the sector operations, DRVSM training
    is quite different since it needs to focus on separation, scanning
    for non-approved aircraft, applying different separation standards
    for approved and non-approved aircraft, the impact of changes in
    national and local procedures and letters of agreement (LOA), and
    the impact of non-approved aircraft on upstream sectors.

-   Given the necessary focus on separation, the participants felt that
    the DRVSM training program needs to be more consistent across areas
    of specialization and across centers than recent CPC training
    programs.

-   Training simulation problems of 20 to 30 minutes duration would be
    more effective for DRVSM than the usual 45 to 60 minute DYSIM
    problems.

-   Each training problem should be very focused on a small set of
    specific training goals.

-   Revisions to local standard operating procedures (SOP) and LOAs will
    be needed to be complete prior to the start of training so they can
    be integrated into local training.

-   An early CBI or some other form of early familiarization for the
    workforce on DRVSM may be very beneficial. The goal would be to get
    the workforce thinking and talking about the changes inherent in
    DRVSM.

-   The fact that DYSIM currently does not support dynamic interactions
    between adjacent simulated sectors is a significant limitation for
    DRVSM training, particularly given the impact of non-RVSM-approved
    aircraft on upstream sectors and the need for an APREQ.

## Verbal Feedback and Written Comments for Controller Training Emphasis

Although controllers consistently responded (via written questionnaires)
that they saw nothing missing or that needed improvement in the DRVSM
training materials briefed prior to their first DRVSM run, the group
discussions during the debrief sessions were highly effective in
identifying topics that either need to be added to the classroom
training, or that were covered but misunderstood by one or more of the
participants. Areas to be reinforced during DRVSM training include:

-   The importance of sector-to-sector coordination to APREQ a
    non-approved aircraft. In particular, situations in which the
    controller needs to APREQ the non-approved aircraft with multiple
    sectors should be highlighted and clearly demonstrated (e.g., when a
    non-approved aircraft is at an altitude that happens to be at the
    base or ceiling altitude of the next sector).

-   APREQ requirements apply to ***all*** non-RVSM-approved aircraft.
    Some controllers mistakenly thought APREQ requirements applied only
    to non-approved aircraft transitioning through RVSM airspace to get
    above FL410 or below FL290, and not to military, lifeguard, or ferry
    flights that had been pre-coordinated for accommodation, traffic
    permitting, at a level RVSM altitude.

-   The upper and lower bounds of RVSM airspace. Specifically, clearly
    demonstrate the following cases:

    -   FL420 is not a valid final altitude for any aircraft, regardless
        of its RVSM status.

    -   If there is a non-RVSM-approved aircraft at FL280, FL290 is an
        available altitude for a RVSM-approved aircraft. If, however,
        the non-approved aircraft is at FL290, FL300 is not available.

-   DRVSM remains exclusionary airspace during slow traffic periods when
    workload could otherwise permit accommodation. An aircraft that is
    non-RVSM-approved, is not one of the types of flights to be
    accommodated with prior approval (e.g., military, lifeguard, or
    ferry flights), and is not capable of flying at FL430 is precluded
    from FL290 -- FL410. A bad ride, for example, does not justify
    giving a non-RVSM-approved aircraft an RVSM altitude. The only valid
    exception is a declared emergency.

-   The rules for when the non-RVSM-approved indicator is displayed or
    not displayed below FL290 are complex and need to be clearly
    illustrated during training. For example, when does the controller
    need to actively look for the non-approval status ***before***
    giving a clearance to climb?

-   Procedures for aircraft that lose their ability to maintain RVSM.
    Clearly demonstrate that such situations are not emergencies, per
    se. The goal is to achieve 2000-foot vertical separation as quickly
    as possible, even if this means climbing the aircraft temporarily to
    a higher RVSM altitude). Once traffic permits, remove the aircraft
    from RVSM airspace. Removal from RVSM airspace may mean descending
    the aircraft to FL280 ***or*** climbing to FL430.

-   Initially, controllers should expect to see errors in the equipage
    suffix filed in flight plans. Some pilots may initially forget to
    file a "/W" or "/Q" suffix (which indicates the aircraft is
    RVSM-approved), and, therefore, the aircraft may erroneously be
    designated as non-approved. If the controller verifies the aircraft
    is RVSM-approved, the equipage suffix needs to be corrected in Host.

## Verbal Feedback for Supervisory Training Emphasis

Some of the training feedback provided may be best incorporated into
supervisory personnel training. This includes:

-   The presence of a non-RVSM-approved aircraft increases workload and
    complexity, and may require the addition of an RA-side at a sector
    that would not otherwise need one. For this reason, supervisory
    awareness of the presence of a non-approved aircraft in RVSM
    airspace is critical. Training should emphasize coordination and
    notification procedures for non-RVSM-approved aircraft.

-   The fact that DRVSM airspace is exclusionary may impact traffic
    density and flows at some sectors by causing aircraft that would
    otherwise fly at higher altitudes into airspace below FL290. Special
    care should be used, especially during the early stages of DRVSM
    until the changes in traffic flows becomes routine.

-   Some participants felt there is the potential with DRVSM to reduce
    the buffer between "very busy, but okay" and "down the tubes".
    Today, one indication that the sector team is reaching workload
    saturation is when all the altitudes are in use. Since there are six
    more altitudes with DRVSM, a sector may reach saturation before all
    the altitudes are used. Consequently, the controller may not fully
    realize how busy he/she is becoming. Supervisory training should
    stress the need for vigilance on monitoring sector workload and
    staffing.

## Controller Perceptions of DRVSM Skills

To gain additional insights into training needs, ten distinct skills or
operational changes associated with DRVSM operations were identified.
They are:

1.  Issuing altitude clearances to the newly added RVSM altitudes, e.g.,
    "USA123 climb and maintain FL320".

2.  Monitoring and applying the revised appropriate altitudes for
    direction of flight.

3.  Using a RVSM altitude to resolve a confliction (e.g., instead of
    vectoring).

4.  Monitoring separation and identifying/resolving conflictions, given
    the changed separation criteria.

5.  Scanning the situation display to identify non-RVSM-approved
    aircraft.

6.  Maintaining separation for non-RVSM-approved aircraft in RVSM
    airspace.

7.  New requirements for coordination and pointouts.

8.  Procedural changes.

9.  Phraseology changes.

10. Changes to sector stratifications.

Controllers were asked to rate on a scale of 1 to 3, each of these ten
skills/changes against two distinct criteria described in Table 4-1.
Each participant rated the training relevance and ease of training of
each of the ten skills/changes prior to their first DRVSM run to
document their initial perceptions, and then after each of the six DRVSM
runs to capture the changes in their views as they gained DRVSM
experience.

TABLE 4-1. DRVSM Skills/Changes Criteria and Rating Scales

+----------------+-------------------------+--------------------------+
|                | **Criteria**            | **Rating Scale**         |
+----------------+-------------------------+--------------------------+
| **Training     | Degree of relevance of  | 1 -- Not at all relevant |
| Relevance**    | the skill or change to  |                          |
|                | adapting to (i.e.,      | 2 -- Somewhat relevant   |
|                | becoming comfortable    |                          |
|                | with) RVSM operations   | 3 -- Very relevant       |
+----------------+-------------------------+--------------------------+
| **Ease of      | Ease of training the    | 1- Minimal simulation    |
| Training**     | skill or change as      | training required for    |
|                | measured by the amount  | proficiency (e.g., 1     |
|                | of simulation training  | hour)                    |
|                | required to achieve     |                          |
|                | proficiency in the      | 2- Moderate simulation   |
|                | skill                   | training required for    |
|                |                         | proficiency (e.g., 3-4   |
|                |                         | hours)                   |
|                |                         |                          |
|                |                         | 3- Significant           |
|                |                         | simulation training      |
|                |                         | required for proficiency |
|                |                         | (e.g., more than 6       |
|                |                         | hours)                   |
+----------------+-------------------------+--------------------------+

An understanding of which new skills or changes are most important to a
controller's ability to adapt to DRVSM operations and the amount of
simulation time needed to develop each skill will help in structuring
the DRVSM training. An analysis of the training relevance ratings per
DRVSM run indicates that the ratings tended to stabilize after the first
three DRVSM runs. Further, the ratings in the earlier runs sometimes
differed substantively from those in the later runs, after the
participants had the benefit of a few hours of DRVSM experience. For
this reason, the average ratings for the last three runs are considered
good indicators for the actual relevance or importance of each skill to
adapting to DRVSM. Table 4-2 provides the training relevance ratings for
the ten skills/changes in decreasing order of importance, as well as a
comparison with the controllers initial perception of how relevant the
skill /change would be. A similar analysis was performed for the ease of
training ratings. These results are shown in Table 4-3.

TABLE 4-2. DRVSM Skills/Changes Training Relevance Ratings

+--------------------------+-------------+-------------+-------------+
| **Skill/Factor**         | **Avg.      | *           | **Average   |
|                          | Rating for  | *Experience | Rating      |
|                          | DRVSM**     | vs. Initial | Prior to    |
|                          |             | Perception  | First DRVSM |
|                          | **Runs 4, 5 | (Less,      | Run**       |
|                          | and 6**     | Same, or    |             |
|                          |             | More)**     | **(Pe       |
|                          |             |             | rception)** |
+--------------------------+-------------+-------------+-------------+
| Maintaining separation   | 2.5         | Less        | 2.8         |
| for non-RVSM-approved    |             |             |             |
| aircraft in RVSM         |             |             |             |
| airspace                 |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Using a RVSM altitude to | 2.5         | More        | 1.8         |
| resolve a confliction    |             |             |             |
| (e.g., instead of        |             |             |             |
| vectoring)               |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Scanning the situation   | 2.2         | Less        | 2.8         |
| display to identify      |             |             |             |
| non-RVSM-approved        |             |             |             |
| aircraft                 |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Monitoring separation    | 2.1         | Less        | 2.3         |
| and                      |             |             |             |
| identifying/resolving    |             |             |             |
| conflictions, given the  |             |             |             |
| changed separation       |             |             |             |
| criteria                 |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Procedural changes       | 2           | Less        | 2.3         |
+--------------------------+-------------+-------------+-------------+
| Changes to sector        | 1.9         | Less        | 2.3         |
| stratifications          |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Monitoring and applying  | 1.9         | Less        | 2.2         |
| the revised appropriate  |             |             |             |
| altitudes for direction  |             |             |             |
| of flight                |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Issuing altitude         | 1.8         | More        | 1.7         |
| clearances to the newly  |             |             |             |
| added RVSM altitudes,    |             |             |             |
| e.g., "USA123 climb and  |             |             |             |
| maintain FL320"          |             |             |             |
+--------------------------+-------------+-------------+-------------+
| New requirements for     | 1.7         | Less        | 2.2         |
| coordination and         |             |             |             |
| pointouts                |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Phraseology Changes      | 1.4         | Less        | 1.7         |
+--------------------------+-------------+-------------+-------------+

TABLE 4-3. DRVSM Skills/Changes Ease of Training Ratings

+--------------------------+-------------+-------------+-------------+
| **Skill/Factor**         | **Avg.      | *           | **Average   |
|                          | Rating for  | *Experience | Rating      |
|                          | DRVSM**     | vs. Initial | Prior to    |
|                          |             | Perception  | First DRVSM |
|                          | **Runs 4, 5 | (Easier,    | Run**       |
|                          | and 6**     | Same, or    |             |
|                          |             | Harder)**   | **(Pe       |
|                          |             |             | rception)** |
+--------------------------+-------------+-------------+-------------+
| Maintaining separation   | 2.2         | Easier      | 2.3         |
| for non-RVSM-approved    |             |             |             |
| aircraft in RVSM         |             |             |             |
| airspace                 |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Scanning the situation   | 2           | Same        | 2           |
| display to identify      |             |             |             |
| non-RVSM-approved        |             |             |             |
| aircraft                 |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Monitoring separation    | 1.8         | Easier      | 2           |
| and                      |             |             |             |
| identifying/resolving    |             |             |             |
| conflictions, given the  |             |             |             |
| changed separation       |             |             |             |
| criteria                 |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Monitoring and applying  | 1.7         | Harder      | 1.3         |
| the revised appropriate  |             |             |             |
| altitudes for direction  |             |             |             |
| of flight                |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Using a RVSM altitude to | 1.6         | Harder      | 1.5         |
| resolve a confliction    |             |             |             |
| (e.g., instead of        |             |             |             |
| vectoring)               |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Procedural changes       | 1.6         | Easier      | 1.8         |
+--------------------------+-------------+-------------+-------------+
| Changes to sector        | 1.6         | Harder      | 1.3         |
| stratifications          |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Issuing altitude         | 1.6         | Harder      | 1.3         |
| clearances to the newly  |             |             |             |
| added RVSM altitudes,    |             |             |             |
| e.g., "USA123 climb and  |             |             |             |
| maintain FL320"          |             |             |             |
+--------------------------+-------------+-------------+-------------+
| New requirements for     | 1.4         | Easier      | 1.7         |
| coordination and         |             |             |             |
| pointouts                |             |             |             |
+--------------------------+-------------+-------------+-------------+
| Phraseology Changes      | 1.2         | Same        | 1.2         |
+--------------------------+-------------+-------------+-------------+

Not surprisingly, the rank ordering of the skills under both rating
criteria are fairly similar, with skills associated with identifying and
separating non-RVSM-approved aircraft in RVSM airspace topping both
lists, and phraseology, coordination, and pointout changes at the
bottom. This information will be used in structuring specific goals for
training simulation problems.

## Analysis of Training Duration Ratings

The questionnaires for the third DRVSM simulation were also designed to
obtain insight into the appropriate duration of DRVSM training,
particularly simulation training. Prior to the first DRVSM run,
participants were asked the following question: Which of the following
statements **best** describes your view of RVSM training at this time?

  --- -------------------------------------------------------------------
  1   I feel well prepared to begin live RVSM operations now.

  2   RVSM training should include approximately 1 hour of classroom
      training, and 3 - 4 simulation (DYSIM) problems, approximately 8
      hours of training total.

  3   RVSM is a significant change to ATC operations and preparing the
      workforce will require significantly more than 8 hours of training.
  --- -------------------------------------------------------------------

To assess how their views changed as they gained experience with DRVSM,
the participants were asked the following question after each of the six
DRVSM simulations: Which of the following statement **best** describes
your view of RVSM training at this time?

  --- -------------------------------------------------------------------
  1   I feel well prepared to begin live RVSM operations now.

  2   Although I feel prepared to begin live RVSM operations now,
      additional simulation practice or experience would be beneficial.

  3   I feel additional simulation experience is needed prior to the
      start of live RVSM operations.
  --- -------------------------------------------------------------------

Table 4-4 provides the results of the ratings, averaged over the six
participants. An average rating of 2.2 prior to the first DRVSM run
indicates that the participants tended to feel that more than eight
hours of training would be required for DRVSM. As they became more
experienced, however, their estimate of the amount of training required
tended to decrease.

TABLE 4-4. Training Duration Ratings

  ---------------- ----------- --------- --------- --------- --------- --------- ---------
                   **Prior to  **After 1 **After 2 **After 3 **After 4 **After 5 **After 6
                   First DRVSM DRVSM     DRVSM     DRVSM     DRVSM     DRVSM     DRVSM
                   Run**       Run**     Runs**    Runs**    Runs**    Runs**    Runs**

  **Cumulative     1 Hour      2 Hours   3 Hours   4 Hours   5 Hours   6 Hours   7 Hours
  Training**[^3]                                                                 

  **Avg. Rating**  2.2         2.2       2.2       1.8       1.5       1.7       1.5
  ---------------- ----------- --------- --------- --------- --------- --------- ---------

In addition to the average rating, it is pertinent to look at how many
of the participants felt well prepared to begin live DRVSM operations
after each run, and how many felt they needed more simulation time prior
to starting live operations.

TABLE 4-5. Distribution of Training Duration Ratings

  ---------------- ---------- --------- --------- --------- --------- --------- ---------
                   **Prior to **After 1 **After 2 **After 3 **After 4 **After 5 **After 6
                   First      DRVSM     DRVSM     DRVSM     DRVSM     DRVSM     DRVSM
                   DRVSM      Run**     Runs**    Runs**    Runs**    Runs**    Runs**
                   Run**                                                        

  Number of "1"    2          2         3         4         4         3         4
  responses (i.e.,                                                              
  well prepared to                                                              
  start live DRVSM                                                              
  ops)                                                                          

  Number of "3"    3          3         2         1         1         1         1
  responses (i.e.,                                                              
  need additional                                                               
  simulation                                                                    
  before live                                                                   
  DRVSM ops)                                                                    
  ---------------- ---------- --------- --------- --------- --------- --------- ---------

At least for this small sample, the data provided in Tables 4-4 and 4-5
suggest that 3 to 4 hours of simulation time produced approximately the
same degree of comfort with DRVSM operations as 6 hours of simulation
time.

# RESULTS FOR Objective 3: Assess the impacts of using RVSM altitudes tactically in CVS environment

The FAA is considering whether to implement "tactical" use of RVSM
altitudes prior to full DRVSM implementation in January 2005. Under
tactical use, controllers would have the opportunity to use RVSM
standards between two RVSM-approved aircraft in their own sector. Prior
to handoff to the next sector, the controller would return the aircraft
to conventional separation or coordinate otherwise. If adopted, tactical
use would be optional, providing a tool available for use at the sector
team's discretion.

From experience in other locales, it is known that tactical use is
effective only if the percentage of RVSM-approved aircraft is large,
e.g., 60 percent or higher. Consequently, the earliest likely date for
instituting tactical use is June 2004, six months prior to the full
implementation of DRVSM. The FAA is investigating the viability, cost,
and benefits of instituting tactical use for this relatively short
period of time.

To obtain field controller input on the operational benefits, the first
two runs of the third DRVSM simulation were designed to assess tactical
use of RVSM altitudes in a CVS environment. The percentage of
RVSM-approved aircraft varied between 60 and 70 percent in the two
tactical runs. Prior to the first run, the participants were given an
introductory familiarization briefing on tactical use.

## 5.1 Overview of Tactical Use Results {#overview-of-tactical-use-results .unnumbered}

All of the participants indicated they found tactical use to be
beneficial to the sector team (and five out of six participants felt the
benefit should be significant), ***provided*** there is a means of
distinguishing RVSM-approved aircraft from non-approved aircraft on the
radar scope. Key benefits cited included a reduction of vectoring (e.g.,
a 1000-foot altitude change for one aircraft can eliminate the need to
vector several aircraft), less potential for operational errors, and
facilitating the transition to full DRVSM operations.

Having an indicator for non-RVSM-approved aircraft on the radar scope
was considered essential for tactical use by all of the participants,
yet this capability is not planned to be available until full DRVSM
implementation. During the tactical use simulation runs, the only means
of discerning whether an aircraft was RVSM-approved was searching for
the aircraft's equipage suffix on the URET aircraft list. Since the
sectors were staffed with two controllers during the tactical runs, the
RA-side was able to perform the required monitoring and searching of the
equipage field to identify opportunities for the R-side to use 1000-foot
separation. The participants felt that in a one-person sector, or a busy
two-person sector, the lack of an indication of RVSM status in the data
block would effectively eliminate their ability to take advantage of
tactical use.

The other automation concern associated with tactical use is the fact
that conflict alert is triggered each time a controller uses RVSM
separation. This was considered distracting by all the participants and
very distracting by some. The fact that conflict alert will sometimes be
displayed at adjacent sectors, not just the sector applying RVSM
separation, could be particularly problematic given that controllers who
chose not to use RVSM tactically will, nonetheless, be affected.
Moreover, since use would be optional during the tactical phase,
changing conflict alert parameters for RVSM-approved aircraft pairs is
not viable.

## Use of RVSM Altitudes Tactically

Controllers were asked to rate the extent to which there would be
opportunities to use RVSM altitudes tactically when considering the full
range of operations found in their areas of specialization. On a scale
of 1 to 5, with 1 meaning "none", 3 "moderate" and 5 "significant", the
average rating was 3.7. Individual responses were spread across the full
range of ratings. For example, while one controller indicated he/she
foresees no situations in which tactical use of RVSM altitudes would be
the best solution, others felt there would be significant opportunity to
use it. When asked separately to rate the opportunities to use RVSM
altitudes tactically during the simulation runs, the results were
identical.

While the participants in the simulation represent a very small sample
size, it is likely that some proportion of the controller population
will choose to **not** use RVSM altitudes tactically. Since the
participants found that the tactical use runs served as effective
training for the subsequent DRVSM runs, the fact that the use of RVSM
altitudes tactically would be optional could actually complicate the
training program for full DRVSM implementation.

## Impact of Tactical Use

The participants were asked to rate the impact of tactical use on
workload, sector complexity, and potential for error on a scale of 1 to
5, with 1 indicating a "significant reduction", 3 "no impact", and 5
"significant increase". The average ratings for both tactical use runs
are shown in

Table 5-1.

TABLE 5-1. Impact of Tactical Use

  --------------------- ---------------- ---------------- ----------------
                        **Workload**     **Sector         **Potential for
                                         Complexity**     Error**

  Average Rating        3 (See below)    2.5              2.3
  --------------------- ---------------- ---------------- ----------------

Analysis of the individual participants' ratings and written comments
indicate that there was a general consensus that tactical use did not
negatively affect sector complexity or the potential for error during
the two tactical use runs. This finding is consistent with the average
ratings of 2.5 and 2.3 for sector complexity and potential for error,
respectively. Some participants felt there was no impact, while others
felt there was either a moderate or significant reduction in sector
complexity and potential for error due to tactical use.

While the average rating for impact on workload was 3, this statistic is
misleading, since every participant felt tactical use had *some* impact,
and there was significant disagreement as to the nature of the impact.
Three participants felt sector workload was reduced, (one indicated a
significant reduction, and two a slight reduction), and the remaining
three felt it was increased, with one feeling it was significantly
increased. For those indicating workload increased, the comments
indicated the increase was due to the efforts required to ascertain
aircraft eligibility for 1000-foot separation and to assess whether
conflict alert warnings represented a real confliction or a valid
tactical use of RVSM altitudes.

## Benefits and Risks for Tactical Use

Participants' input on the benefits and risks of implementing tactical
use was collected via written questionnaires and group debrief sessions.
The following benefits were identified:

-   Reduction of vectoring (e.g., a 1000-foot altitude change for one
    aircraft can eliminate the need to vector several aircraft). This
    would be particularly helpful in narrow sectors with little room to
    vector.

-   Less potential for operational errors. Not only does tactical use
    provide another option to resolve a potential confliction, altitude
    separation can be achieved more quickly when only 1000 feet is
    required.

-   In sectors in which there are major streams of crossing traffic and
    altitude congestion under CVS, controllers may be able to solve a
    complex situation by moving a single aircraft.

-   Experience in using RVSM altitudes tactically probably reduces the
    amount of training needed to transition to full DRVSM (but only for
    those controllers who opt to use RVSM altitudes tactically).

The risks associated with tactical use include those resulting from the
automation issues identified in Section 5.1, the general risk associated
with any significant operational change, and the implications of
optional use. Some controllers were concerned that the workforce could
be lulled into a false sense of security given that controllers at the
next sector may not accept a handoff for aircraft separated using
tactical RVSM. Others felt this risk is one that they deal with every
day, since there are many reasons a handoff request may be denied.

## Overall Utility of Implementing Tactical Use

Participants were asked to rate the overall utility of implementing
tactical use by considering factors such as benefits, impacts, costs,
and the fact that it would be in effect for no more than six months
prior to full DRVSM implementation. Costs were not quantified, but
described only as broad categories of effort (such as workforce training
for tactical use, then retraining for full DRVSM deployment, and
procedures development). The following 5-point rating scale was used:

  --- -------------------------------------------------------------------
  1   Not at all useful in my area of specialization

  3   No impact (positive or negative) to my area of specialization

  5   Extremely useful in my area of specialization
  --- -------------------------------------------------------------------

The average rating was 4. Four of the six participants felt it would be
useful (a rating of 4) or extremely useful (a rating of 5), and two were
neutral (a rating of 3).

# Conclusions

The findings of the third DRVSM simulation serve to validate and enhance
the draft DRVSM procedures, support the development of an effective
DRVSM training plan, and provide the FAA with information to support a
decision on tactical use of RVSM altitudes prior to full DRVSM
deployment.

With two exceptions, the preliminary procedures developed for the
simulation were determined to be complete and effective. The
participants suggested the procedure for controllers to notify
supervisory personnel of the presence of a non-RVSM-approved aircraft be
streamlined to take advantage of the fact that military, lifeguard, and
ferry flights requesting to fly at a RVSM altitude will be coordinated
with traffic flow management in advance. With pre-coordination between
traffic flow and supervisory personnel, controllers should learn of
non-RVSM-approved aircraft to be accommodated at a RVSM altitude in
their sector from their operations supervisor rather than the reverse.
Secondly, participants suggested an additional procedure to help ensure
continual situational awareness of a non-RVSM-approved aircraft
operating in RVSM airspace. Specifically, a requirement will be added
for pilots of non-RVSM-approved aircraft to state their flight is
non-RVSM-approved on initial radio contact at each sector traversed
while operating in RVSM airspace. These changes are being incorporated
into the proposed DRVSM revisions to FAA Handbook 7110.65 and FAA Order
7210.3.

Several findings of the third DRVSM simulation will be incorporated into
the DRVSM training plan for en route controllers and supervisors.
Meaningful data was obtained to answer the single largest unknown for
the training plan, namely the duration of simulation training needed to
adapt to DRVSM operations. From the subjective data collected, it
appears that a total simulation time of three to four hours per
controller will suffice. Based on suggestions from the field
participants, the training plan will consider using six to eight
simulation problems of 20 to 30 minutes in duration each, instead of 45
to 60 minute problems, and that each problem will have specific,
well-defined training objectives. Based on the simulation findings, the
individual skills most relevant to adapting to DRVSM operations are
better understood, as is the amount of simulation training required to
acquire each skill. Moreover, input gained will assist in developing
more effective classroom training and CBI materials.

Lastly, the issues and risks of implementing tactical use of RVSM
altitudes prior to the full implementation of DRVSM in January 2005 are
now better understood. A key finding of the simulation is that the plan
to display the non-RVSM indicator in data blocks only for full DRVSM
operations, and not for tactical use, is not viable. Lack of a visual
indication in the data block would effectively preclude the ability of
the sector to apply reduced vertical separation tactically during
single-person sector operations or during heavy traffic periods. The
participants found the manual search and analysis of flight data to
identify RVSM-approved aircraft sufficiently workload intensive that it
would be feasible only in two-person sector operations and only then
with moderate traffic densities. This effectively eliminates many of the
benefits of tactical use. The participants also found it distracting to
receive conflict alert notifications each time tactical use was applied
between two RVSM-approved aircraft, particularly at a sector adjacent to
the sector applying tactical use. While the first issue may be able to
be resolved if it is feasible to display the non-RVSM-approval indicator
during the tactical use timeframe, there is no viable solution for the
second issue. Conflict alert parameters cannot be modified to RVSM
standards during the tactical use timeframe since use of RVSM altitudes
tactically is optional. Finally, the optional nature of tactical use
would significantly complicate the training program for full DRVSM
implementation by effectively necessitating one training program for
those who use RVSM tactically on a routine basis (and who would have
already developed many of the necessary skills associated with adapting
to DRVSM) and those who do not. Given these factors and the additional
costs associated with implementing tactical use for a six-month period
at best, the negatives associated with tactical use appear to outweigh
the positives.

**APPENDIX A**

**DRAFT CONTROLLER AND PILOT PHRASEOLOGY**

# Accommodation of non-RVSM approved military or lifeguard aircraft {#accommodation-of-non-rvsm-approved-military-or-lifeguard-aircraft-1 .unnumbered}

[Phraseology]{.underline}: For pilot/controller communication

+-----------------------------------+-----------------------------------+
| # Circums                         | # Phras                           |
| tance {#circumstance .unnumbered} | eology {#phraseology .unnumbered} |
+-----------------------------------+-----------------------------------+
| ATC wish to know RVSM status of   | ## Confirm RVSM Approved {#c      |
| flight                            | onfirm-rvsm-approved .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot indication that flight is   | ## Affir                          |
| RVSM approved                     | m RVSM {#affirm-rvsm .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot indication that flight is   | ## Negative                       |
| non-RVSM approved                 | RVSM {#negative-rvsm .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot of State aircraft           | ## Negati                         |
| indicating that flight is         | ve RVSM State Aircraft {#negative |
| non-RVSM approved.                | -rvsm-state-aircraft .unnumbered} |
+-----------------------------------+-----------------------------------+
| ATC denial of clearance into RVSM | **Unable clearance into RVSM      |
| airspace                          | airspace, maintain (or descend    |
|                                   | to, or climb to) FL**             |
+-----------------------------------+-----------------------------------+

[Phraseology]{.underline}: For coordination between ATC personnel/units

+-----------------------------------+-----------------------------------+
| # Message {#message .unnumbered}  | # Phraseo                         |
|                                   | logy {#phraseology-1 .unnumbered} |
+-----------------------------------+-----------------------------------+
| To verbally supplement an         | **Negative RVSM** or **Negative   |
| automated estimate message        | RVSM State Aircraft** (as         |
| exchange which does not           | applicable)                       |
| automatically transfer approval   |                                   |
| status                            |                                   |
+-----------------------------------+-----------------------------------+
| To verbally supplement estimate   | **Negative RVSM or Negative RVSM  |
| messages of non-RVSM approved     | State Aircraft** (as applicable)  |
| aircraft                          |                                   |
+-----------------------------------+-----------------------------------+

# Non-RVSM approved aircraft transitioning through RVSM airspace {#non-rvsm-approved-aircraft-transitioning-through-rvsm-airspace-1 .unnumbered}

[Phraseology]{.underline}: For pilot/controller communication

+-----------------------------------+-----------------------------------+
| # Circumsta                       | # Phraseo                         |
| nce {#circumstance-1 .unnumbered} | logy {#phraseology-2 .unnumbered} |
+-----------------------------------+-----------------------------------+
| ATC wish to know RVSM status of   | ## Confirm RVSM Approved {#con    |
| flight                            | firm-rvsm-approved-1 .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot indication that flight is   | ## Affirm                         |
| RVSM approved                     | RVSM {#affirm-rvsm-1 .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot indication that flight is   | ## Negative RV                    |
| non-RVSM approved                 | SM {#negative-rvsm-1 .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot of State aircraft           | ## Negative                       |
| indicating that flight is         |  RVSM State Aircraft {#negative-r |
| non-RVSM approved.                | vsm-state-aircraft-1 .unnumbered} |
+-----------------------------------+-----------------------------------+
| ATC denial of clearance into RVSM | **Unable clearance into RVSM      |
| airspace                          | airspace, maintain (or descend    |
|                                   | to, or climb to) FL**             |
+-----------------------------------+-----------------------------------+
| Pilot requesting clearance        | **(call sign) request climb to FL |
| through RVSM airspace and is      | \_\_\_, negative RVSM**           |
| non-RVSM approved.                |                                   |
+-----------------------------------+-----------------------------------+

[Phraseology]{.underline}: For coordination between ATC personnel/units

+-----------------------------------+-----------------------------------+
| #                                 | # Phraseo                         |
|  Message {#message-1 .unnumbered} | logy {#phraseology-3 .unnumbered} |
+-----------------------------------+-----------------------------------+
| To verbally supplement an         | **Negative RVSM** or **Negative   |
| automated estimate message        | RVSM State Aircraft** (as         |
| exchange which does not           | applicable)                       |
| automatically transfer approval   |                                   |
| status                            |                                   |
+-----------------------------------+-----------------------------------+
| To verbally supplement estimate   | **Negative RVSM or Negative RVSM  |
| messages of non-RVSM approved     | State Aircraft** (as applicable)  |
| aircraft                          |                                   |
+-----------------------------------+-----------------------------------+

**Base/ceiling coordination requirements with abutting sectors**

[Phraseology]{.underline}: For coordination between ATC personnel/units

+-----------------------------------+-----------------------------------+
| #                                 | # Phraseo                         |
|  Message {#message-2 .unnumbered} | logy {#phraseology-4 .unnumbered} |
+-----------------------------------+-----------------------------------+
| To verbally supplement an         | **Negative RVSM** or **Negative   |
| automated estimate message        | RVSM State Aircraft** (as         |
| exchange which does not           | applicable)                       |
| automatically transfer approval   |                                   |
| status                            |                                   |
+-----------------------------------+-----------------------------------+
| To verbally supplement estimate   | **Negative RVSM or Negative RVSM  |
| messages of non-RVSM approved     | State Aircraft** (as applicable)  |
| aircraft                          |                                   |
+-----------------------------------+-----------------------------------+

# Aircraft loss of equipment required for RVSM operation {#aircraft-loss-of-equipment-required-for-rvsm-operation-2 .unnumbered}

[Phraseology]{.underline}: For pilot/controller communications

+-----------------------------------+-----------------------------------+
| # Circumsta                       | # Phraseo                         |
| nce {#circumstance-2 .unnumbered} | logy {#phraseology-5 .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot reporting equipment         | ## Unable                         |
| degraded below RVSM requirements  |  RVSM due to Equipment {#unable-r |
|                                   | vsm-due-to-equipment .unnumbered} |
+-----------------------------------+-----------------------------------+
| ATC requesting pilot to report    | ## Re                             |
| when able to resume RVSM.         | port able to resume RVSM {#report |
|                                   | -able-to-resume-rvsm .unnumbered} |
+-----------------------------------+-----------------------------------+
| Pilot ready to resume RVSM after  | ## Ready to resume RVSM {#        |
| equipment/weather contingency.    | ready-to-resume-rvsm .unnumbered} |
+-----------------------------------+-----------------------------------+

# Aircraft experiencing Wake Turbulence in RVSM airspace {#aircraft-experiencing-wake-turbulence-in-rvsm-airspace-1 .unnumbered}

[Phraseology]{.underline}: Use existing phraseology associated with
turbulence instances.

[^1]: No separation violations occurred during the third DRVSM
    simulation.

[^2]: The traffic scenarios were specifically designed to permit
    accommodation of the non-RVSM-approved aircraft, so that the
    accommodation and transition procedures could be exercised and
    assessed.

[^3]: Cumulative training includes classroom and simulation training.
    Classroom training was provided prior to the first DRVSM run and
    lasted approximately one hour. Each DRVSM simulation run was
    essentially an hour.