**Terrain Display Alternatives:**

**Assessment of Information Density and Alerting Strategies**

Revision: 4/6/98

Bud, Melissa J.; Stearns, Mary; Mengert, Peter

U.S. Department of Transportation

Research and Special Programs Administration

John A. Volpe National Transportation Systems Center

Cambridge, MA 02142

**^PREFACE^**

^Electronic\ display\ technology\ is\ a\ recent\ development\ offering\ many\ possibilities\ for\ visual\ representations\ of\ terrain\ information.\ Electronic\ displays\ of\ terrain\ information\ have\ the\ potential\ to\ increase\ pilot\ positional\ awareness\ and\ to\ aid\ in\ terrain\ avoidance.\ This\ report\ describes\ a\ series\ of\ four\ experiments\ conducted\ to\ identify\ human\ factors\ issues\ associated\ with\ presenting\ terrain\ information\ on\ electronic\ displays.\ The\ experimental\ results\ provide\ information\ to\ pilots,\ aviation\ regulators,\ and\ display\ designers\ to\ help\ them\ ensure\ that\ this\ new\ technology\ is\ used\ effectively\ to\ improve\ aviation\ safety.^

^The\ experimental\ research\ was\ performed\ in\ the\ Cockpit\ Human\ Factors\ laboratory\ located\ at\ the\ Volpe\ National\ Transportation\ Systems\ Center\ (Volpe\ Center).\ In\ the\ experiments,\ researchers\ examined\ electronic\ display\ presentations,\ positional\ awareness,\ terrain\ alerts,\ and\ terrain\ avoidance.^

^The\ authors\ would\ like\ to\ thank\ the\ many\ people\ who\ were\ integral\ to\ the\ completion\ and\ success\ of\ this\ sequence\ of\ experiments:\ Dr.\ Daniel\ J.\ Hannon\ designed\ the\ experiments\ and\ provided\ guidance\ during\ experimentation\ and\ analyses;\ Stephen\ Ransom,\ EG&G\ Services,\ and\ Jonathan\ Baxter\ contributed\ to\ the\ design,\ testing,\ and\ analyses;\ and\ Frank\ Sheelen,\ W.T.\ Chen,\ designed\ the\ software\ used\ in\ these\ experiments\ and\ provided\ technical\ and\ computer\ support\ throughout\ the\ experimental\ process.^

^The\ authors\ would\ also\ like\ to\ acknowledge\ their\ FAA\ sponsors:\ Dr.\ Maureen\ Pettitt,\ Chief\ Scientific\ and\ Technical\ Advisor\ for\ Human\ Factors\ and\ Dr.\ Eleana\ Edens,\ manager\ of\ the\ Cockpit\ Human\ Factors\ Program,\ AAR-100,\ for\ their\ guidance\ and\ support\ of\ this\ work.^

**^TABLE\ OF\ CONTENTS^**

^Appendices^

^Appendix\ A:\ Research\ Tools^

^A-1.\ Frasca\ 242\ Simulator\ 25^

^A-2.\ Simulator\ Flight\ Instrument\ Panel\ 26^

^Appendix\ B:\ Materials\ and\ Displays\ -\ The\ Influence\ of\ Terrain\ Depictions\ on\ Navigational\ Accuracy\ (Experiment\ 1)^

^B-1.\ Electronic\ Approach\ Display\ 27^

^B-2.\ Paper\ Approach\ Chart\ 28^

^B-3.\ Flight\ Checklist\ 29^

^B-4.\ Post-Flight\ Questionnaire\ 30^

^Appendix\ C:\ Materials\ and\ Displays\ -\ Detecting\ and\ Employing\ Positional\ Information\ on\ a\ Supplementary\ Display\ (Experiment\ 2)^

^C-1.\ Electronic\ Display\ for\ Map\ Condition\ 2:\ Navigation\ and\ Terrain\ Information\ 31^

^C-2.\ Electronic\ Display\ for\ Map\ Condition\ 3:\ Terrain\ Information\ 31^

^C-3.\ Chart\ of\ Flight\ Path\ 32^

^C-4.\ Script\ for\ Experiment\ 2\ (Albany\ Experiment)\ 33^

^Appendix\ D:\ Materials\ and\ Displays\ -\ Pilot\ Preferences\ for\ Information\ Presentation\ on\ Electronic\ Displays\ for\ Terrain\ Avoidance\ (Experiment\ 3)^

^D-1.\ Static\ Electronic\ Display\ Format:\ 500-ft\ Contours,\ Filled\ Contours,\ and\ Navigation\ Information\ 34^

**^TABLE\ OF\ CONTENTS\ (CONTINUED)^**

**[Section]{.underline}** **[Page]{.underline}**

^D-2.\ Static\ Electronic\ Display\ Format:\ 1000-ft\ Contours,\ Filled\ Contours,\ and\ Navigation\ Information\ 35^

^D-3.\ Static\ Electronic\ Display\ Format:\ 2000-ft\ Contours,\ Unfilled\ Contours,\ and\ Navigation\ Information\ 36^

^D-4.\ Electronic\ Display\ Format\ Preference\ Task\ Rating\ Criteria\ 37^

^D-5.\ Static\ Electronic\ Display\ Format\ Comparisons\ 38^

^D-6.\ Display\ Rating\ Questionnaire\ 39^

^Appendix\ E:\ Materials\ and\ Displays\ -\ Value\ of\ Alerts\ for\ Enhancing\ Interpretability\ of\ Electronic\ Displays\ for\ Terrain\ Avoidance\ (Experiment\ 4)^

^E-1.\ Low-Density\ Display\ Format\ 40^

^E-2.\ Medium-Density\ Display\ Format\ 40^

^E-3.\ High-Density\ Display\ Format\ 41\ ^

^E-4.\ Paper\ Chart\ Including\ Flight\ Paths\ for\ the\ Practice\ and\ Experimental\ Runs\ 42^

^E-5.\ Example\ of\ Flight\ Script\ (for\ Experiment\ 4)\ 43^

^E-6.\ Post-Experiment\ Questionnaire\ 44^

**^LIST\ OF\ FIGURES^**

**^[Figure]{.underline}^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^\ ^^[Page]{.underline}^**

^4-1.\ Flight\ cross\ track\ error\ (XTE)\ at\ each\ obstacle\ point\ across\ trials\ 7^

^6-1.\ Preferences\ by\ contour\ increment\ level\ 15^

**^LIST\ OF\ TABLES^**

**[Table]{.underline} [Page]{.underline}**

^4-1.\ Sequence\ of\ trial\ presentation\ for\ approaches\ by\ information\ on\ the\ electronic\ display\ 6^

^5-1.\ Participants'\ flight\ experience\ 9^

^5-2.\ Participants'\ age\ and\ flight\ hours\ by\ map\ condition\ 9^

^5-3.\ Distribution\ of\ participants\ by\ whether\ or\ not\ terrain\ conflict\ was\ detected\ 11^

^6-1.\ Participants'\ flight\ experience\ 13^

^6-2.\ Electronic\ display\ map\ formats\ 14^

^6-3.\ Paired\ comparison\ ratings\ for\ each\ display\ format\ 15^

^7-1.\ Participants'\ flight\ experience\ 17^

^7-2.\ Mean\ reaction\ times\ to\ alerts\ for\ display\ formats\ 19^

^7-3.\ Distribution\ of\ responses\ to\ alerts\ by\ correct\ and\ actual\ (pilot)\ response\ 19^

^7-4.\ Percentages\ of\ correct\ and\ incorrect\ responses\ across\ alert\ points\ 20^

^7-5.\ Invalid\ response\ categories\ used\ to\ code\ incorrect\ responses\ 20^

# ^[EXECUTIVE\ SUMMARY]{#_Toc398435611}^

Current technology makes navigation and terrain information available on
electronic display screens in the cockpit. This information must be
presented clearly for pilots to maintain positional awareness and to
avoid collision with terrain. However, there are few recommendations or
guidelines as to how electronic displays of information should be
designed or evaluated.

Electronic displays present enhanced information to the pilot which may
help to reduce the accident rate in General Aviation (GA) flight. For
example, in the United States from 1983 to 1994,
Controlled-Flight-Into-Terrain (CFIT) incidents accounted for 32% of the
GA accidents in instrument weather conditions.[^1] Enhanced real-time
positional information on electronic displays might help to prevent
collisions with terrain.

The Volpe National Transportation Systems Center (Volpe Center), under
the sponsorship of the Federal Aviation Administration's (FAA's) Office
of the Chief Scientific and Technical Advisor for Human Factors,
AAR-100, conducted a series of interrelated experiments to explore the
human factors issues in depicting terrain on electronic displays. The
series of experiments examined instrument-rated GA pilots' ability to
interpret terrain depicted on electronic plan view displays using a
flight simulator.

Because the resolution of electronic display screens causes a
disproportionate relationship between the airplane symbol and the
terrain features, the initial experiment examined the utility of
displaying nonthreatening terrain to pilots during approaches.
Participants flew the approaches accurately and did not veer off course
despite the disproportionate size of the airplane symbol relative to
terrain features.

Terrain elevation information, presented on a supplemental plan view
display, might prove useful to detect and avoid dangerous terrain. In
the second experiment, pilots failed to make use of the terrain
information presented on an electronic display. When they were presented
with a display showing only terrain features, pilots showed heightened
awareness to terrain but not enough to take corrective action. When the
display presented showed navigation and terrain information, pilots
failed to detect the dangerous terrain.

The third experiment measured pilots' preferences for information
density on plan view electronic displays. Despite findings that pilots
may better recognize potential obstacles using displays with a lower
density of information, participants preferred the displays with higher
content levels.

Visual alerts may enhance the interpretability of electronic plan view
displays. In the final experiment, pilots used an electronic display
paired with a visual terrain alert. Using alternate map formats, the
pilots were measured to see how well they interpreted these displays to
obtain terrain elevation information when there was a potential problem.
The map formats varied in terms of density of terrain and navigation
information. The pilots were able to interpret the displays with equal
facility regardless of the map format.

This series of experiments addressed several human factors issues
associated with presenting terrain information on electronic displays.
The summation of these points is that terrain information on electronic
displays helps pilots to avoid terrain if combined with a terrain
alerting system, regardless of the information density of the map. These
results suggest that designs for display formats should incorporate
alerts to make pilots aware of danger and reorient them quickly. Display
formats that incorporate these recommendations should contribute to a
reduction in the number of accidents which result from a loss of
positional awareness, such as CFIT accidents. Further research into
information presentation options and systems and the benefit of training
is needed.

# ^[1.\ INTRODUCTION]{#_Toc398435612}^

Current technology consolidates navigation and safety information, such
as weather and traffic, on electronic display screens in the cockpit.
The information displayed must be clear to allow pilots to maintain
positional awareness and avoid collision with terrain. However, there
are few recommendations or guidelines available on how to design or
evaluate the acceptable display configurations.

Electronic displays also incorporate additional information that, until
recently, could not be presented. This increased information available
to the pilot may contribute to reducing the General Aviation (GA)
accident rate. Because research has shown the unchanged incidence of
Controlled-Flight-Into-Terrain (CFIT) accidents in GA operations (32% in
instrument conditions from 1983 to 1994[^2]), the Volpe Center and the
Federal Aviation Administration's (FAA's) Office of the Chief Scientific
and Technical Advisor for Human Factors, AAR-100, conducted experiments
to examine the potential value of, and issues in, consolidating and
depicting additional information using electronic displays.

# ^[2.\ PURPOSE]{#_Toc398435613}^

^Technological\ development\ in\ avionics,\ specifically\ in\ electronic\ display\ screens,\ computer\ technology,\ and\ memory\ processors,\ is\ expanding\ rapidly\ and\ the\ human\ factors\ issues\ that\ are\ inherent\ in\ these\ technologies\ warrant\ examination.\ The\ Volpe\ Center,\ under\ the\ sponsorship\ of\ the\ FAA's\ AAR-100,\ conducted\ a\ series\ of\ experiments\ to\ examine\ how\ quickly\ and\ accurately\ pilots\ are\ able\ to\ interpret\ alternative\ presentations\ of\ information\ on\ electronic\ display\ screens.^

# ^[3.\ PROCEDURES]{#_Toc398435614}^

## ^[3.1\ PARTICIPANTS]{#_Toc398435615}^

Participants in the experiments were general aviation (GA) pilots with
multi-engine and instrument ratings. The pilots were recruited from the
Subject Database maintained at the Center for Human Factors Research at
the Volpe National Transportation Systems Center (Volpe Center) and from
fliers posted at local airports.[^3]

## ^[3.2\ APPARATUS]{#_Toc398435616}^

The Center for Human Factors Research at the Volpe Center maintains a
Frasca 242 light twin engine instrument flight simulator which was used
for these experiments. This simulator is a fixed-base training device
reconfigured and instrumented for data collection. It did not have an
outside the window (OTW) display at the time of these experiments.

The flight dynamics of the simulator were designed to resemble a Piper
Aztec. (Appendix A-1 shows the Frasca 242 simulator.) There is a picture
window in the wall behind the simulator which permitted the researchers
to observe the pilot during testing. Researchers also observed the pilot
on a television monitor located outside the simulator room using a
closed-circuit television. During the experiments, researchers
communicated with the pilots using a standard aircraft intercom system
with headsets.

Flight instruments in the Frasca are arranged in the standard T
formation in front of the pilot's seat. The avionics stack is directly
in front of the copilot's seat. The electronic display screen is in the
position of the center avionics stack in a Piper Aztec. The electronic
display screen measures 8.5-in wide by 6.5-in high with multi-functional
buttons along the bottom and right-hand side of the screen. A diagram of
the flight instrument panel, including the electronic display, is
presented in Appendix A-2.

The Volpe Center developed software programs to simulate flight and to
produce the maps presented on the electronic display screen. The maps
were shown on a plan view display in the upper left hand corner of the
screen, measuring approximately 4.5-in wide by 5-in high.

## ^[3.3\ ELECTRONIC\ DISPLAY\ FORMATS]{#_Toc398435617}^

The electronic display formats used in this series of experiments were
consistent in size, color attributes, and symbology. Terrain was
displayed in one of two ways: as green contour lines of uniform color on
a black background with green elevation labels on the lines or as solid,
stacked polygons in a range of greens---dark greens for the lowest
elevations to light greens or yellow for the highest elevations.
Elevation labels were placed within the solid polygon.

Both terrain presentations used similar conventions to describe
elevation. For the contour lines, the maximum elevation, above mean sea
level (MSL), for the area between two bounding lines, was described by
the lower of the two elevation labels. For the solid contour shapes, the
maximum elevation, above mean sea level, for an area of a certain color,
was described by the elevation label within that polygon.

Navigation information on the electronic display screen employed formats
consistent with previous research using the Frasca simulator.[^4] Purple
lines indicated airways. Global Positioning System (GPS) waypoints,
VORs, and intersections were presented in white and labeled with the
appropriate identifiers. Runways were indicated with a yellow line that
was thicker than the airway lines. Labels were shown in capital letters.

The airplane shape, shown in yellow, was the same size in all
experiments. Information on the screen moved relative to the airplane's
course changes. For north-up oriented maps, the airplane symbol moved
along the map until it reach an edge of the screen. The map would update
and the airplane would appear in the middle of the screen with
corresponding terrain and navigation information. For track-up
(inside-out) oriented maps, the airplane symbol remained heading up and,
in a constant position, near the bottom of the screen, and centered
horizontally.

## ^[3.4\ PROTOCOL]{#_Toc398435618}^

^Procedural\ protocols\ were\ reviewed\ and\ approved\ by\ the\ MIT\ Committee\ on\ the\ Use\ of\ Human\ Subjects.\ Accordingly,\ pilots\ signed\ an\ informed\ consent\ form\ and\ filled\ out\ an\ information\ sheet\ prior\ to\ research\ and\ were\ debriefed\ at\ completion\ of\ the\ study.^

# ^[4.\ TERRAIN\ DEPICTIONS\ AND\ NAVIGATIONAL\ ACCURACY]{#_Toc398435619}^

^Electronic\ displays\ with\ terrain\ depictions\ could\ be\ useful\ for\ navigation\ when\ there\ is\ potentially\ dangerous\ terrain\ and,\ consequently,\ a\ requirement\ for\ a\ high\ degree\ of\ flight\ accuracy\ on\ approach.\ The\ purpose\ of\ the\ first\ experiment\ was\ to\ determine\ whether\ electronic\ displays\ of\ terrain\ affect\ the\ pilot's\ navigational\ performance\ during\ instrument\ approaches.\ Previous\ research\ has\ suggested\ that\ providing\ aircraft\ position\ information,\ relative\ to\ the\ terrain,\ increases\ pilot\ accuracy\ when\ compared\ to\ a\ display\ with\ no\ position\ information.^[^5]^\ However,\ if\ the\ pilot\ does\ not\ recognize\ that,\ based\ on\ space\ and\ pixel\ limitations\ of\ the\ display,\ the\ scale\ of\ terrain\ depictions,\ relative\ to\ other\ symbols,\ i.e.,\ airplane\ symbol,\ is\ disproportionate,\ a\ pilot\ might\ tend\ to\ overcorrect\ his\ or\ her\ track\ to\ avoid\ terrain.^

^The\ experimental\ question\ was,\ is\ pilots'\ cross\ track\ error\ (XTE)\ more\ likely\ to\ deviate\ from\ the\ flight\ path\ when\ they\ use\ maps\ with\ terrain\ features\ as\ compared\ with\ maps\ without\ terrain\ features?\ Variation\ in\ XTE\ would\ indicate\ that\ pilots\ deviated\ from\ their\ intended\ course\ due\ to\ terrain\ representations\ which\ are\ not\ in\ scale\ with\ the\ aircraft\ symbol.\ (XTE\ is\ measured\ as\ the\ perpendicular\ lateral\ distance\ of\ the\ airplane's\ actual\ location\ to\ the\ course\ line.)\ ^

## ^[4.1\ PROCEDURE]{#_Toc398435620}^

Participants in this experiment flew three fictional GPS approaches, two
of which had corresponding electronic displays. Appendix B-1 shows an
example of the electronic approach display used in this experiment. The
participants were aided by maps on the electronic display screen and
corresponding paper instrument approach charts when they flew these
approaches. The paper instrument approach charts created for this
experiment were similar in format and information to National Oceanic
and Atmospheric Administration (NOAA) or Jeppesen approach charts
(Appendix B-2 shows a paper approach chart used in this experiment).

The three approaches had similar geometry (turn angles, rate of descent,
rate of ascent) but varied in headings and elevations. These approaches
showed mountain peaks at four relevant positions: on an initial approach
turn, adjacent to the final approach leg, adjacent to the missed
approach leg, and on the missed approach turn. The approach used as a
baseline, to establish flight technical ability, only differed in that
it did not present terrain information or use an electronic display.

The four target terrain features on each of the two experimental
approach were positioned close to the course line and in conformance
with Terminal Instrument Procedures (TERPS) minimum clearances. The
electronic charts were drawn such that terrain contours were close to,
but did not cross, the flight path. The terrain features had a lower
elevation than the closest anticipated point on the flight path.
Position of the terrain, relative to the course line (left or right),
varied between the two approaches.

Participants flew all approaches from the initial approach fix to the
end of the published missed approach procedure under IFR conditions and
were instructed to maintain certain aircraft performance measures and to
perform the normal activities of an approach procedure. (Participants
used the Flight Checklist shown in Appendix B-3.) Participants were
given unlimited practice flight time to become familiar with the
simulator.

When the participants felt comfortable, they flew six trials (i.e., a
practice approach, two baseline approaches, and three experimental
approaches). Table 4-1 shows the sequence of trials as well as the
navigation and terrain information shown on the electronic display.
Participants' XTE was measured for all approaches.

**^Table\ 4-1.\ Sequence\ of\ trial\ presentation\ for\ approaches\ by\ information\ on\ the\ electronic\ display.^**

  ------------------ ----------------------------- ----------------------------- --------------------------------------------- ----- ---------------------- ---------------------- -------- ----- ----------------------------- ----------------------------- ----------- ----- ----------------------------- ----------------------------- ---------------------------- ----- ----------------------------- ----------------------------- ----------------------------
  **Trial Number**   **Approach**                  **Approach**                  **Information Shown on Electronic Display**                                                                                                                                                                                                                                                                                                               
                     ^Subject\ 1^                  ^Subject\ 2^                                                                                                                                                                                                                                                                                                                                                                            
  ^Practice^         ^Experimental\ Approach\ 1^   ^Experimental\ Approach\ 2^   ^None^                                        ^1^   ^Baseline\ Approach^   ^Baseline\ Approach^   ^None^   ^2^   ^Experimental\ Approach\ 2^   ^Experimental\ Approach\ 1^   ^Terrain^   ^3^   ^Experimental\ Approach\ 1^   ^Experimental\ Approach\ 2^   ^Terrain\ and\ navigation^   ^4^   ^Experimental\ Approach\ 2^   ^Experimental\ Approach\ 1^   ^Terrain\ and\ navigation^
  ^5^                ^Baseline\ Approach^          ^Baseline\ Approach^          ^None^                                                                                                                                                                                                                                                                                                                                                    
  ------------------ ----------------------------- ----------------------------- --------------------------------------------- ----- ---------------------- ---------------------- -------- ----- ----------------------------- ----------------------------- ----------- ----- ----------------------------- ----------------------------- ---------------------------- ----- ----------------------------- ----------------------------- ----------------------------

^The\ participants\ were\ debriefed\ after\ completing\ six\ trials\ and\ were\ asked\ for\ their\ reactions\ to\ the\ electronic\ charts.\ The\ Post-Flight\ Questionnaire\ used\ in\ the\ debriefing\ is\ in\ Appendix\ B-4.^

## ^[4.2\ RESULTS]{#_Toc398435621}^

The mean XTE for each participant's baseline approach flights was within
requirements for flight competency. The maximum baseline XTE occurrence
was 0.5 miles, but XTE was most often less than 0.1 miles. For the
experimental approaches, the maximum occurrence of XTE was 0.745 miles,
which is also within requirements for flight competency. Figure 4-1
shows a scatter plot of the XTE at the obstacle points (labeled A, B, C,
and D) for participants flying the experimental approaches. Participants
had the opportunity to use an electronic moving map display when flying
these approaches.

**^Figure\ 4-1.\ Flight\ cross\ track\ error\ (XTE)\ at\ each\ obstacle\ point\ across\ trials.\ (Obstacle\ points\ located\ at\ A,\ B,\ C,\ and\ D)^**

The XTE at the obstacle points reveals that the participants were as
likely to err toward the terrain as away from it (24 incidents; 12
toward, 12 away). The XTE is evenly distributed about the flight path
with no skew. In debriefing, participants reported in debriefing that
they did not find the terrain information on the display useful for
making the approaches. The debriefing results corroborate the XTE
results. The participants used the map display as a navigation aid for
heading, elevation, and distance information, or as a reference for
position relative to course line.

## ^[4.3\ DISCUSSION]{#_Toc398435622}^

Use of terrain depictions on electronic displays was not associated with
flight performance in this experiment. Participants did not alter their
flight path systematically when the electronic display showed their
airplane symbol flying near nonthreatening terrain depictions. Since the
safe terrain did not negatively impact navigation, further research is
warranted to examine issues related to avoidance of dangerous terrain.

# ^[5.\ DETECTING\ AND\ USING\ POSITION\ INFORMATION]{#_Toc398435623}^

^Before\ advocating\ the\ use\ of\ electronic\ displays\ for\ terrain\ avoidance,\ it\ is\ necessary\ to\ know\ how\ readily\ pilots\ can\ interpret\ terrain\ information\ from\ supplemental\ electronic\ displays.\ In\ this\ experiment,\ participants\ were\ routed\ off\ their\ original\ airway\ toward\ terrain\ that\ was\ at\ a\ higher\ elevation\ than\ the\ aircraft.\ The\ experimental\ scenario\ made\ it\ difficult\ for\ participants\ to\ avoid\ terrain\ impact\ if\ they\ used\ only\ the\ standard\ flight\ instruments\ and\ paper\ charts.\ The\ supplementary\ electronic\ display\ presented\ relative\ position\ information\ that\ made\ it\ easier\ to\ detect\ the\ dangerous\ terrain.\ If\ the\ pilots\ did\ not\ use\ the\ terrain\ and\ position\ information\ on\ the\ electronic\ display,\ the\ rerouting\ in\ the\ scenario\ would\ cause\ them\ to\ impact\ terrain.^

## ^[5.1\ PROCEDURE]{#_Toc398435624}^

In addition to holding instrument and multi-engine ratings, more than
half of the twelve participants held Commercial and ATP ratings and
three were Certified Flight Instructors (CFI). The mean age of the
eleven men and one woman participants was 52 years (SD = 17). Table 5-1
shows participants' flight experience.

**^Table\ 5-1.\ Participants^^'^^\ flight\ experience.^**

+----------------------+----------------------+----------------------+
| **Total Flight       | **Instrument Flight  | **Time Since Last    |
| Hours**              | Hours **             | Flight**             |
|                      |                      |                      |
| **(in hours)**       | **(in hours)**       | **(in days)**        |
+----------------------+----------------------+----------------------+
| ^R                   | ^                    | ^Median\ =\ 6^       |
| ange\ =\ 586-17,000^ | Range\ =\ 70-10,000^ |                      |
|                      |                      |                      |
| ^Mean\ =\ 3,120^     | ^Mean\ =\ 1,625^     |                      |
|                      |                      |                      |
| ^(SD\ =\ 5,290)^     | ^(SD\ =\ 2,964)^     |                      |
+----------------------+----------------------+----------------------+

^Participants\ were\ assigned\ to\ one\ of\ three\ Map\ Conditions\ and\ asked\ to\ carry\ out\ a\ simulated\ flight\ to\ Albany,\ NY.\ Four\ pilots\ were\ randomly\ assigned\ to\ each\ of\ the\ three\ Map\ Conditions.\ The\ descriptive\ statistics\ for\ the\ participants\ in\ each\ of\ the\ three\ Map\ Conditions\ are\ show\ in\ Table\ 5-2.^

**^Table\ 5-2.\ Participants^^'^^\ age\ and\ flight\ hours\ by\ Map\ Condition.^**

+----------------+----------------+----------------+----------------+
| **Map          | **Age**        | **Total Flight | **Instrument   |
| Conditions**   |                | Hours**        | Flight Hours** |
|                | **(in years)** |                |                |
|                |                | **(in hours)** | **(in hours)** |
+----------------+----------------+----------------+----------------+
| ^Map\          | ^Mean\ =\ 56^  | ^M             | ^M             |
| Condition\ 1:^ |                | ean\ =\ 7,550^ | ean\ =\ 1,713^ |
|                | ^(SD\ =\ 13)^  |                |                |
| ^\ N           |                | ^(             | ^(             |
| o\ Display,\ ^ |                | SD\ =\ 5,204)^ | SD\ =\ 2,209)^ |
|                |                |                |                |
| ^\ Paper       |                |                |                |
| \ Chart\ Only^ |                |                |                |
+----------------+----------------+----------------+----------------+
| ^Map\          | ^Mean\ =\ 62^  | ^M             | ^M             |
| Condition\ 2:^ |                | ean\ =\ 5,810^ | ean\ =\ 2,719^ |
|                | ^(SD\ =\ 8.8)^ |                |                |
| ^\             |                | ^(             | ^(             |
| Display\ with\ |                | SD\ =\ 7,511)^ | SD\ =\ 4,856)^ |
|  Terrain\ and^ |                |                |                |
|                |                |                |                |
| ^\ Navigation  |                |                |                |
| \ Information^ |                |                |                |
+----------------+----------------+----------------+----------------+
| ^Map\          | ^Mean\ =\ 38^  | ^M             | ^Mean\ =\ 443^ |
| Condition\ 3:^ |                | ean\ =\ 2,559^ |                |
|                | ^(SD\ =\ 20)^  |                | ^(SD\ =\ 526)^ |
| ^\ Display\ wi |                | ^(             |                |
| th\ Terrain\ ^ |                | SD\ =\ 1,413)^ |                |
|                |                |                |                |
| ^\ Info        |                |                |                |
| rmation\ Only^ |                |                |                |
+----------------+----------------+----------------+----------------+

^The\ Map\ Condition\ assignment\ determined\ the\ type\ of\ electronic\ map\ display,\ if\ any,\ provided\ to\ the\ participants\ on\ the\ supplemental\ display\ screen.\ Participants\ in\ Map\ Condition\ 1\ used\ a\ paper\ chart\ and\ were\ not\ provided\ an\ electronic\ display.\ Participants\ in\ Map\ Condition\ 2\ were\ given\ a\ paper\ chart\ and\ an\ electronic\ display\ with\ terrain\ and\ navigation\ information.\ Participants\ in\ Map\ Condition\ 3\ were\ provided\ with\ a\ paper\ chart\ and\ an\ electronic\ display\ of\ terrain\ information\ only.^

^The\ participants\ in\ Map\ Conditions\ 2\ and\ 3\ used\ a\ Volpe-designed\ electronic\ moving\ map\ of\ the\ area\ between\ Poughkeepsie\ and\ Albany\ (approximately\ 40\ miles)\ which\ had\ 1000\ foot\ terrain\ contour\ lines,\ navigation\ information,\ and\ a\ north-up\ orientation.\ The\ navigation\ information,\ only\ available\ to\ participants\ in\ Map\ Condition\ 2,\ included\ airways\ labeled\ with\ Victor\ airways\ numbers,\ VORs,\ and\ intersections.\ Appendices\ C-1\ and\ C-2\ show\ the\ displays\ used\ in\ Map\ Conditions\ 2\ and\ 3.^

^Prior\ to\ flight,\ the\ researcher\ reviewed\ the\ Frasca\ instruments,\ radio\ stack,\ and\ the\ electronic\ map\ symbols\ with\ the\ participant,\ where\ appropriate,\ without\ revealing\ the\ study's\ goal\ of\ examining\ terrain\ avoidance\ displays.\ The\ Frasca\ simulator\ generated\ moderate\ turbulence\ during\ the\ entire\ experiment\ to\ increase\ workload.^

^First,\ the\ participant\ flew\ a\ practice\ flight\ and\ was\ instructed\ to\ perform\ normal\ activities\ of\ en\ route\ flight\ while\ becoming\ familiar\ with\ the\ simulator.\ The\ participant\ used\ a\ NOAA\ sectional\ chart\ of\ the\ area\ around\ Allentown\ for\ navigation\ and\ dialed\ up\ the\ Allentown\ VOR\ for\ familiarization.\ The\ researcher\ answered\ the\ participants'\ questions\ over\ the\ headset.\ Participants\ were\ given\ unlimited\ practice\ time,\ but\ most\ spent\ approximately\ 20\ minutes.^

^After\ the\ practice\ flight,\ but\ prior\ to\ the\ experimental\ trial,\ the\ participant\ completed\ an\ en\ route\ paper\ chart\ familiarization\ task,\ which\ required\ him\ or\ her\ to\ identify\ ten\ items\ circled\ in\ red\ on\ a\ photocopy\ of\ a\ NOAA\ sectional\ chart\ for\ another\ area.\ All\ participants\ completed\ this\ task\ accurately.^

^On\ the\ experimental\ flight,\ the\ participants\ were\ instructed\ to\ fly\ to\ Albany.\ They\ used\ a\ sectional\ chart\ of\ the\ area\ around\ Albany\ and\ approach\ plates\ for\ both\ Albany\ and\ Poughkeepsie.\ The\ airplane\ started\ in\ level\ flight\ at\ 3000\ ft\ above\ sea\ level,\ which\ was\ the\ Minimum\ En\ Route\ Altitude\ (MEA)\ on\ the\ originating\ airway\ (V123).\ The\ researcher,\ as\ Air\ Traffic\ Control\ (ATC),\ communicated\ with\ the\ pilot\ using\ the\ headset\ and\ vectored\ the\ participant\ around\ weather\ off\ the\ original\ airway\ with\ no\ altitude\ change.\ Shortly\ before\ the\ pilot\ reached\ a\ new\ airway,\ V213,\ the\ researcher\ reported\ loss\ of\ radar\ contact\ and\ cleared\ the\ pilot\ direct\ to\ Albany.\ The\ participant\ was\ still\ flying\ at\ an\ altitude\ of\ 3000\ ft,\ although\ the\ new\ airway\ had\ a\ Minimum\ Obstacle\ Clearance\ Altitude\ (MOCA)\ of\ 5500\ ft.\ The\ chart\ of\ this\ flight\ path\ is\ presented\ in\ Appendix\ C-5\ and\ the\ ATC\ script\ is\ presented\ in\ Appendix\ C-4.^

^If\ the\ participant\ noticed\ the\ terrain\ and\ questioned\ the\ current\ altitude,\ ATC\ issued\ a\ higher\ clearance\ and\ the\ trial\ ended.\ If\ the\ participant\ did\ not\ question\ ATC\ immediately\ after\ radar\ contact\ was\ lost,\ the\ participant\ flew\ for\ approximately\ ten\ more\ minutes\ to\ allow\ time\ to\ notice\ the\ terrain\ conflict,\ albeit\ too\ late\ to\ take\ corrective\ action.\ If\ the\ participant\ still\ did\ not\ notice\ the\ higher\ terrain,\ the\ trial\ was\ terminated.^

^During\ debriefing,\ the\ researchers\ explained\ that\ the\ experimental\ goal\ was\ to\ see\ how\ effective\ the\ electronic\ display\ was\ in\ improving\ pilot\ ability\ to\ avoid\ terrain.\ The\ researchers\ emphasized\ that\ the\ study\ was\ not\ a\ test\ of\ piloting\ skills,\ but\ was\ intended\ to\ demonstrate\ the\ potential\ usefulness\ of\ the\ display.^

## ^[5.2\ RESULTS]{#_Toc398435625}^

As shown in table 5-3, 83% of the participants failed to detect the
terrain conflict. Half of the participants (two out of four) in Map
Condition 3, electronic display with terrain information only, detected
the higher terrain. Neither participant, however, caught the error in
time to avoid impact with the terrain. One of the participants in Map
Condition 3 that did not detect the terrain reported, in debriefing,
that he had noticed the terrain elevation but did not, and would not,
question ATC. The remaining eight participants in the other two Map
Conditions did not recognize the terrain conflict. One participant
avoided the terrain due to tuning the wrong VOR.

**^Table\ 5-3.\ Distribution\ of\ participants\ by\ whether\ or\ not\ terrain\ conflict\ was\ detected.^**

<table>
<tbody>
<tr class="odd">
<td><p><strong>Map Conditions</strong></p></td>
<td><p><strong>Detected</strong></p>
<p><strong>Terrain Conflict</strong></p></td>
<td><p><strong>Did Not Detect</strong></p>
<p><strong>Terrain Conflict</strong></p></td>
<td><p><strong>% of Total</strong></p></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><p><strong><sup>Map Condition 1:</sup></strong></p>
<p><strong><sup> No Display,</sup></strong></p>
<p><strong><sup> Paper Chart</sup></strong></p></td>
<td><p><sup>0</sup></p></td>
<td><p><sup>4</sup></p></td>
<td><p><sup>33%</sup></p></td>
<td><p><strong><sup>Map Condition 2:</sup></strong></p>
<p><strong><sup> Display with Terrain and </sup></strong></p>
<p><strong><sup> Navigation Information</sup></strong></p></td>
<td><p><sup>0</sup></p></td>
<td><p><sup>4</sup></p></td>
<td><p><sup>33%</sup></p></td>
<td><p><strong><sup>Map Condition 3:</sup></strong></p>
<p><strong><sup> Display with Only Terrain </sup></strong></p>
<p><strong><sup> Information</sup></strong></p></td>
<td><p><sup>2</sup></p></td>
<td><p><sup>2</sup></p></td>
<td><p><sup>33%</sup></p></td>
</tr>
<tr class="odd">
<td><p><strong><sup>% of Total</sup></strong></p></td>
<td><p><sup>17%</sup></p></td>
<td><p><sup>83%</sup></p></td>
<td><p><sup>100%</sup></p></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>

## ^[5.3\ DISCUSSION]{#_Toc398435626}^

These results suggest that the participants focused on terrain elevation
when it was highlighted as in Map Condition 3, where, for example, the
participants used a display that presented only terrain information.
This interpretation was supported by the comments during debriefing from
the participants who detected the terrain conflict. These participants
reported that they detected the higher elevation terrain because they
had been thinking about the reasons for presenting a terrain-only
electronic display. If they did not have this terrain-only display,
these participants reported that they would not have thought about
terrain during the scenario.

Participants' comments supported the interpretation that pilots do not
tend to think of terrain as an issue when flying en route. A few
participants mentioned that they might have been more alert to
potentially dangerous terrain if they had been flying in the mountainous
western United States. One third of the participants reported that they
depend on ATC for altitude guidance and were trained not to question
ATC.

Kuchar and Hansman also demonstrated this high level of trust in ATC. In
their experiment, pilots were vectored into terrain by ATC and had a
very low initial terrain conflict recognition rate, regardless of the
electronic display presentation, contours or spot elevations. However,
once the pilots recognized that ATC might not give accurate vectors,
terrain conflict recognition increased dramatically when the pilot was
provided with a display with terrain contour information.[^6]

In the debriefing, participants mentioned that they were not comfortable
trusting the electronic display, which was unfamiliar, when it
contradicted ATC and their training. Three participants cited the high
workload (i.e., the turbulence and resulting difficulty in maintaining
straight and level flight) as too distracting to permit them to include
the display into their scan. One-third of the participants said that if
they had bought the map for their own airplane, understood it was for
terrain alert, and had practice using it, they would have noticed the
terrain conflict.

Participants presented with a terrain only-display noticed the terrain
but participants with the terrain and navigation display did not. This
suggests that the additional clutter of airways may have reduced their
ability to focus on terrain. However, since the participants in Map
Condition 1, without an electronic display, failed to give attention to
terrain issues, the underlying issue may be *attention*. These results
proved to be a turning point in the conceptualization of this series of
experiments. The crucial issue emerging that drawing the pilot's
attention to the display was, possibly, more important than the
information presentation. The responses from the participants suggest
that pilots will use an electronic display for terrain avoidance if
their attention is drawn to the display for a terrain-related reason,
and are presented with clear, informative terrain information.

# ^[6.\ PREFERENCES\ FOR\ INFORMATION\ DENSITY\ ON\ ELECTRONIC\ DISPLAYS\ USED\ FOR\ TERRAIN\ AVOIDANCE]{#_Toc398435627}^

This experiment examined pilots' preferences for density of information
on electronic displays. Although the previous experiment suggested that
minimal information on an electronic display creates the clearest
presentation, other studies have indicated that pilots prefer maps with
higher information density.[^7] In this experiment, participants were
asked what map features would have been useful to detect and avoid
terrain in the earlier experiment. The participants were presented
alternative electronic displays, with different terrain and navigation
features. Their preferences were recorded to see if they agreed with the
previous results.

## ^[6.1\ PROCEDURE]{#_Toc398435628}^

The eight participants had participated in the previous experiment. They
held instrument and multi-engine ratings; half held ATP ratings, and two
were also CFIs. The average participant age (seven men and one woman)
was 55 years (SD = 17 years). Table 6-1 shows participants' flight
experience.

**^Table\ 6-1.\ Participants^^'^^\ flight\ experience.^**

+----------------------+----------------------+----------------------+
| **Total Flight       | **Instrument Flight  | **Time Since Last    |
| Hours**              | Hours**              | Flight**             |
|                      |                      |                      |
| ** (in hours)**      | **(in hours)**       | **(in days)**        |
+----------------------+----------------------+----------------------+
| ^R                   | ^R                   | ^Median\ =\ 7.5^     |
| ange\ =\ 586-17,000^ | ange\ =\ 102-10,000^ |                      |
|                      |                      |                      |
| ^Mean\ =\ 5,372^     | ^Mean\ =\ 1,747^     |                      |
|                      |                      |                      |
| ^(SD\ =\ 5,412)^     | ^(SD\ =\ 3,354)^     |                      |
+----------------------+----------------------+----------------------+

^Participants\ viewed\ twelve\ alternative\ electronic\ display\ presentations\ on\ a\ desktop\ computer\ and\ on\ the\ electronic\ display\ screen\ in\ the\ Frasca\ 242\ simulator.\ They\ evaluated\ twenty-four\ static\ electronic\ maps\ each\ of\ which\ had\ a\ standard\ yellow\ aircraft\ symbol\ in\ the\ center\ for\ reference.\ The\ airways\ were\ labeled\ with\ MOCAs\ rather\ than\ Victor\ airway\ numbers.\ Participants\ rated\ the\ displays\ using\ paired\ comparisons\ and\ individual\ evaluations.^

^Each\ electronic\ display\ showed\ three\ types\ of\ information;\ contour\ type,\ contour\ increment\ level,\ and\ navigation\ information\ available.\ Appendices\ D-1\ through\ D-3\ show\ three\ representative\ display\ alternatives.\ There\ were\ two\ terrain\ layouts\ for\ each\ terrain\ increment\ level.^

^Each\ participant\ was\ shown\ twelve\ maps,\ one\ from\ each\ cell\ in\ the\ design.\ Table\ 6-2\ has\ twelve\ cells,\ each\ of\ which\ has\ a\ unique\ combination\ of\ alternatives.\ The\ participants\ viewed\ all\ the\ display\ formats\ on\ a\ desktop\ computer\ prior\ to\ evaluating\ them.\ Participants\ were\ instructed\ to\ look\ at\ the\ displays\ and\ develop\ their\ own\ evaluation\ criteria,\ with\ terrain\ avoidance\ as\ the\ main\ focus.\ Participants\ received\ an\ instruction\ sheet\ which\ included\ suggestions\ for\ rating\ criteria\ (see\ Appendix\ D-4\ for\ a\ copy\ of\ this\ form).\ Each\ participant\ was\ asked\ to\ evaluate\ how\ useful\ each\ display\ format\ was\ for\ alerting\ a\ pilot\ to\ a\ terrain\ conflict\ in\ a\ situation,\ similar\ to\ the\ previous\ experiment,\ where\ a\ pilot\ is\ vectored\ off\ course\ and\ not\ given\ a\ new\ clearance.^

**^Table\ 6-2.\ Electronic\ display\ map\ formats.\ ^**^\ ^

^(Note:\ two\ terrain\ layouts\ were\ created\ for\ each\ terrain\ increment\ level.)^

The participants rated the displays using the paired comparison
technique.[^8] Participants compared each of the twelve alternative
formats to each of the eleven others, for a total of 66 trials. The
participants were asked to think carefully about each comparison and
were given unlimited time to complete the task. In each trial, two
static charts were presented on a screen side by side with a 17-point
radio button rating system located underneath the charts at the bottom
of the screen. The rating system ranged from "absolutely better" for one
chart to "same" in the center (between the two) to "absolutely better"
for the other chart (see Appendix D-5 for an image of the paired
comparison program). Participants were instructed to select the radio
button on the rating scale that corresponded with their rating for that
pair of charts. They could change their responses at any time, including
responses to previous trials.

For the paired comparisons, the twelve design alternatives received
scores based on the participant's selection. The scores ranged from 1 to
17, where 1 indicated a rating of most preferred, 9 indicated a rating
of equal preference and 17 indicated a rating of least preferred. When a
participant indicated a preference for one chart, that chart was given a
low rating, between 1 and 9, that corresponded with the gradation of
preference. The nonpreferred chart was assigned a rating of 18, minus
the preferred chart's rating.

The participants then viewed, and evaluated, each display format using
the electronic display in the Frasca simulator. The Frasca simulator
display gave the participants a more realistic context for evaluating
the charts' function. The participants evaluated each format using a
display rating questionnaire (shown in Appendix D-6). Following this,
participants were debriefed on their opinions and suggestions.

## ^[6.2\ RESULTS]{#_Toc398435629}^

### ^6.2.1[\ Paired\ Comparison\ Task]{.underline}^

The mean rating scores for the two terrain layouts within every cell of
the design were analyzed using Wilcoxon Rank Sums. Since there were no
significant differences between the ratings for terrain layout, the
rating scores for terrain layouts in each cell were pooled and
descriptive statistics calculated. These results are presented in table
6-3.

**^Table\ 6-3.\ Paired\ comparison\ ratings\ for\ each\ display\ format.^**

^(1\ =\ most\ preferred\ format,\ 17\ =\ least\ preferred\ format).^

<table>
<tbody>
<tr class="odd">
<td></td>
<td><p><em>Solid color contours and Navigation information</em></p></td>
<td><p><em>Solid color contours and</em></p>
<p><em>no Navigation information</em></p></td>
<td><p><em>Contour rings </em></p>
<p><em>and Navigation information</em></p></td>
<td><p><em>Contour rings </em></p>
<p><em>and no Navigation information</em></p></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><p><em><sup>500</sup><sup>’</sup></em></p>
<p><em><sup>increment</sup></em></p>
<p><em><sup>level </sup></em></p></td>
<td><p><strong><sup>Mean = 3.65</sup></strong></p>
<p><strong><sup>(SD = 3.00)</sup></strong></p></td>
<td><p><sup>Mean = 8.28</sup></p>
<p><sup>(SD = 4.68)</sup></p></td>
<td><p><sup>Mean = 7.24</sup></p>
<p><sup>(SD = 4.46)</sup></p></td>
<td><p><sup>Mean = 11.05</sup></p>
<p><sup>(SD = 4.13)</sup></p></td>
<td><p><em><sup>1000</sup><sup>’</sup><sup> increment</sup></em></p>
<p><em><sup>level</sup></em></p></td>
<td><p><sup>Mean = 5.84</sup></p>
<p><sup>(SD = 4.05)</sup></p></td>
<td><p><sup>Mean = 9.88</sup></p>
<p><sup>(SD = 4.59)</sup></p></td>
<td><p><sup>Mean = 8.20</sup></p>
<p><sup>(SD = 4.52)</sup></p></td>
<td><p><sup>Mean = 12.25</sup></p>
<p><sup>(SD = 3.40)</sup></p></td>
</tr>
<tr class="odd">
<td><p><em><sup>2000</sup><sup>’</sup></em></p>
<p><em><sup>increment</sup></em></p>
<p><em><sup>level</sup></em></p></td>
<td><p><sup>Mean = 7.34</sup></p>
<p><sup>(SD = 4.31)</sup></p></td>
<td><p><sup>Mean = 11.58</sup></p>
<p><sup>(SD = 4.04)</sup></p></td>
<td><p><sup>Mean = 9.35</sup></p>
<p><sup>(SD = 4.63)</sup></p></td>
<td><p><strong><sup>Mean = 13.34</sup></strong></p>
<p><strong><sup>(SD = 2.88)</sup></strong></p></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>

^Table\ 6-3\ shows\ that\ the\ most\ preferred\ display\ format\ (rating\ =\ 3.65,\ SD\ =\ 3.00)\ had\ solid\ color\ contours,\ navigation\ information,\ and\ 500'\ contour\ increments.\ The\ least\ preferred\ display\ (rating\ =\ 13.34,\ SD\ =\ 2.88)\ had\ contour\ rings,\ no\ navigation\ information,\ and\ 2000'\ contour\ increments.^

^The\ graph\ in\ figure\ 6-1\ shows\ that\ participants\ preferred\ more\ detail,\ i.e.,\ navigation\ information,\ finer\ grained\ contour\ increments,\ and\ solid\ color\ contours.^

**^Figure\ 6-1.\ Preferences\ by\ contour\ increment\ level.\ ^**

^(Note:\ rating\ score\ inversely\ related\ to\ preference.)^

### ^[6.2.2]{#_Toc398435630}[\ Individual\ Evaluation\ Task]{.underline}^

Evaluation of alternative display formats showed that participants
preferred a higher density of information, as indicated in the paired
comparison results. Generally, they preferred finer grained contour
increment levels (500 ft or 1000 ft). Most of the participants preferred
the denser information alternative, especially the positional
orientation provided by navigation and terrain information, and the
solid color contours.

This preference for solid color contours reflected participants'
preference for color to change relative to elevation, which allows
graphical information to substitute for digital. Participants also
reported that the solid color contours gave them a better three
dimensional perception of elevation. Only two participants said they
disliked the solid color contours because the colors cluttered the
display. Three participants suggested that unfilled contours could
create visual space for more and different information.

Participants suggested the following ways to improve the displays:

·
^Reduce\ clutter\ by\ providing\ fewer\ elevation\ labels\ and\ allowing\ color\ gradation\ to\ describe\ the\ curves.\ ^

·
^Place\ an\ altitude\ tag\ on\ the\ airplane\ to\ provide\ a\ clearer\ sense\ of\ the\ relationship\ between\ the\ airplane\ and\ the\ terrain.^

·
^Restrict\ navigation\ information\ to\ the\ current\ flight\ path.\ (It\ is\ not\ necessary\ to\ include\ all\ the\ airways.)^

·
^Include\ more\ airway\ information\ such\ as\ VOR\ frequencies,\ DMEs,\ and\ assigned\ altitudes.^

·
^Eliminate\ en\ route\ MOCAs\ when\ terrain\ contours\ and\ elevation\ labels\ are\ present.\ MOCAs\ should\ be\ replaced\ by\ airway\ numbers\ (this\ was\ the\ format\ in\ the\ previous\ experiment).^

· ^Add\ range\ information.^

· ^Add\ distance\ and\ heading\ to\ nearest\ airport.^

· ^Indicate\ terrain\ higher\ than\ the\ airplane\ in\ red.^

· ^Use\ track-up\ display\ presentations.^

## ^[6.3\ DISCUSSION]{#_Toc398435631}^

Participants preferred to use electronic displays with dense information
to improve their terrain awareness. They preferred graphical
presentations to reduce redundancy and saw graphical presentations as
freeing space for additional information. These results are consistent
with previous research.[^9] However, the previous experiment indicated
that a high density of information on the displays was not the best way
to foster positional and terrain awareness. It appears that *how* the
information was presented was secondary to getting the pilots to look at
the display. Unless the display captures the pilot's attention, it
cannot be useful for terrain avoidance. With these results in mind, the
final experiment was designed to explore the usability of display
formats paired with visual alerts.

# ^[7.\ VISUAL\ ALERTS\ AND\ THE\ INTERPRETABILITY\ OF\ ]{#_Toc398435632}^

# ^ELECTRONIC\ DISPLAYS\ OF\ TERRAIN^

Since the display is not in the primary field of view, the three prior
experiments explored how pilots incorporate into their instrument scan a
supplemental electronic display which did not have warning cues. This
final experiment measured the effectiveness of a visual alert presented
together with an electronic display for terrain avoidance. If the visual
alert presented in this experiment highlights potential problems, this
alert could help pilots to better interpret a range of display formats,
including those with dense information.

## ^[7.1\ PROCEDURE]{#_Toc398435633}^

Six instrument- and multi-engine-rated pilots participated in this
experiment; four of the six held Commercial and ATP ratings, and four
held CFI ratings. All the participants in this test were male with an
average age of 42.5 years and a standard deviation (SD) of 14 years.
Table 7-1 shows participants' flight experience.

**^Table\ 7-1.\ Participants^^'^^\ flight\ experience.^**

+----------------------+----------------------+----------------------+
| **Total Flight       | **Instrument Flight  | **Time Since Last    |
| Hours**              | Hours**              | Flight**             |
|                      |                      |                      |
| ** (in hours)**      | **(in hours)**       | **(in days)**        |
+----------------------+----------------------+----------------------+
| ^R                   | ^Range\ =\ 60-2,800^ | ^Median\ =\ 8.5^     |
| ange\ =\ 593-28,000^ |                      |                      |
|                      | ^Mean\ =\ 779^       |                      |
| ^Mean\ =\ 5,740^     |                      |                      |
|                      | ^(SD\ =\ 1,096)^     |                      |
| ^(SD\ =\ 10,922)^    |                      |                      |
+----------------------+----------------------+----------------------+

^The\ participants\ flew\ the\ Frasca\ 242\ simulator\ and\ were\ required\ to\ react\ to\ the\ display\ screen\ by\ pressing\ one\ of\ the\ six\ buttons\ located\ below\ the\ screen.\ There\ was\ a\ clearly\ marked\ label\ above\ each\ button.\ The\ labels,\ from\ left\ to\ right,\ were,\ "No\ Action,"\ "Left,"\ "Climbing\ Left,"\ "Climb,"\ "Climbing\ Right,"\ and\ "Right".^

^The\ electronic\ display\ formats\ varied\ on\ three\ dimensions:\ map\ density\ (high,\ medium,\ and\ low),\ airplane\ orientation\ (track-up\ and\ north-up),\ and\ alert\ type\ (true\ alarm\ and\ false\ alarm).\ One\ thousand\ foot\ contour\ increments\ were\ present\ on\ all\ the\ maps.\ The\ high\ density\ map\ had\ navigation\ information\ (airways,\ VORTACs,\ and\ intersections)\ and\ solid\ contour\ shapes.\ The\ medium\ density\ map\ had\ solid\ contour\ shapes\ but\ no\ navigation\ information.\ The\ low\ density\ map\ had\ contour\ rings\ and\ also\ lacked\ navigation\ information.\ Appendices\ E-1\ through\ E-3\ show\ examples\ of\ the\ three\ display\ formats.^

^Each\ participant\ flew\ two\ practice\ flights\ and\ four\ experimental\ flights.\ The\ practice\ flight\ was\ 25\ miles\ long\ and\ included\ procedures\ similar\ to\ the\ experimental\ runs,\ but\ only\ presented\ six\ alerts,\ one\ for\ each\ electronic\ display\ in\ each\ of\ the\ two\ airplane\ orientations:\ north-up\ and\ track-up.\ The\ experimental\ flights,\ 100\ miles\ in\ length\ and\ 40\ to\ 45\ minutes\ long,\ were\ identical\ to\ one\ another\ for\ navigational\ purposes.\ The\ four\ experimental\ flights\ differed\ in\ terrain\ layout\ and\ sequence\ of\ alerts\ (i.e.,\ independent\ variables\ manipulated).\ Alerts\ were\ presented\ twelve\ times\ on\ each\ experimental\ flight.^

^The\ researcher\ briefed\ each\ participant\ at\ a\ computer\ terminal\ prior\ to\ the\ experiment.\ The\ researcher\ showed\ the\ participant\ six\ electronic\ displays,\ each\ of\ the\ six\ formats\ presented,\ and\ explained\ how\ to\ interpret\ the\ electronic\ displays.\ The\ participant\ was\ given\ the\ paper\ chart\ (see\ Appendix\ E-4\ for\ a\ copy\ of\ the\ chart\ with\ the\ routes\ flown)\ and\ told\ to\ fly\ the\ assigned\ route\ at\ the\ assigned\ altitude.\ The\ paper\ chart\ included\ information\ present\ on\ standard\ charts\ such\ as\ headings,\ VORTACs,\ and\ airways,\ but\ did\ not\ provide\ terrain\ elevation\ information.\ The\ participant\ was\ told\ to\ respond\ to\ ATC\ commands\ (as\ issued\ by\ the\ researcher)\ such\ as\ changes\ in\ altimeter\ setting,\ communication\ frequencies,\ and\ transponder\ squawk\ codes,\ and\ to\ react\ to\ the\ alert\ maps\ as\ accurately\ and\ quickly\ as\ possible.\ (See\ Appendix\ E-5\ for\ a\ flight\ script).\ The\ participant\ was\ informed\ that\ the\ assigned\ route\ was\ within\ restricted\ airspace\ and\ he\ or\ she\ should\ not\ deviate\ from\ course\ unless\ necessary.^

^The\ first\ practice\ flight\ familiarized\ the\ participants\ with\ the\ flight\ dynamics\ and\ the\ moderately\ high\ turbulence\ present\ throughout\ the\ experiment\ without\ having\ to\ use\ the\ electronic\ display.\ There\ was\ no\ time\ limit\ set\ for\ this\ practice\ flight\ but\ most\ participants\ flew\ for\ approximately\ ten\ to\ fifteen\ minutes.^

^The\ participant\ flew\ a\ second\ practice\ flight\ using\ an\ assigned\ route\ that\ included\ alert\ triggers\ to\ become\ familiar\ with\ the\ electronic\ display\ and\ to\ practice\ responding\ to\ alerts.\ During\ this\ flight\ and\ the\ subsequent\ experimental\ flights,\ the\ participant\ was\ required\ to\ maintain\ 5,000\ feet\ and\ 170\ to\ 180\ knots\ airspeed.\ The\ electronic\ display\ was\ blank\ and\ navigation\ was\ by\ VORs.\ The\ participant\ was\ not\ permitted\ to\ use\ the\ autopilot.^

^During\ the\ flights,\ the\ researcher\ would\ trigger\ an\ alert\ every\ two\ to\ five\ minutes,\ presenting\ the\ participant\ with\ a\ potential\ terrain\ conflict.\ During\ an\ alert,\ the\ airplane\ would\ continue\ flying,\ the\ screen\ would\ flash\ twice,\ displaying\ a\ red\ background\ with\ the\ word\ "alert"\ in\ black\ letters,\ and\ then\ the\ map\ display\ would\ appear.\ The\ participant\ was\ expected\ to\ assess\ the\ situation\ and\ determine\ what\ action,\ if\ any,\ was\ necessary\ to\ maintain\ terrain\ separation.\ The\ participant\ was\ instructed\ to\ keep\ the\ airplane\ straight\ and\ level\ and\ to\ respond\ to\ the\ alert\ by\ pressing\ a\ button\ located\ below\ the\ screen.\ Pressing\ a\ button\ blanked\ the\ screen.\ The\ participant\ could\ not\ change\ his\ mind\ after\ making\ a\ selection.\ Reaction\ time\ (in\ seconds)\ was\ measured\ from\ the\ time\ an\ alert\ appeared\ on\ the\ screen\ to\ the\ time\ the\ pilot\ pressed\ a\ button\ (made\ a\ decision).\ Percentage\ correct\ responses\ at\ each\ alert\ point\ were\ recorded\ for\ every\ flight\ in\ addition\ to\ flight\ data\ such\ as\ plane\ orientation,\ speed\ and\ pitch.^

^After\ the\ participant\ completed\ four\ experimental\ flights,\ he\ or\ she\ completed\ a\ questionnaire\ (see\ Appendix\ E-6)\ to\ obtain\ subjective\ responses\ to\ the\ experiment.\ The\ researcher\ used\ the\ questionnaire\ as\ the\ starting\ point\ to\ discuss\ the\ session\ with\ the\ participant.^

## ^[7.2\ RESULTS]{#_Toc398435634}^

The mean overall reaction time to alerts was 5.5 seconds with a standard
deviation of 4.18 seconds. The median reaction time was 4.6 seconds.
(See table 7-2 for the mean reaction times for each display format
combination.) There were no significant differences in reaction time by
map density, airplane orientation, or alert type using factorial
Analysis of Variance (ANOVA).

**^\ ^^\ Table\ 7-2.\ Mean\ reaction\ times\ to\ alerts\ for\ display\ formats.^**

The percentage of correct responses was 81%. A correct response was
defined as responding with any of the five action types when an action
was required or with "No Action" when no action was required. Error was
divided into two categories: "False Positive" errors resulted from
participants responding with one of the five types of actions if no
action was required, and "False Negative" errors which resulted from the
participants responding "No Action" when there was an action required.
No determination was made in advance as to what type of action was the
most appropriate for each "action" trial, and consequently, type of
action was not analyzed for accuracy. Table 7-3 shows accuracy of
responses to alerts.

**^\ Table\ 7-3.\ Distribution\ of\ responses\ to\ alerts\ by\ correct\ and\ actual\ (pilot)\ response.^**

+-------+-------+-------+-------+-------+-------+-------+-------+
|       | [Co   | [Co   | [     |       |       |       |       |
|       | rrect | rrect | Total |       |       |       |       |
|       | Re    | Re    | Re    |       |       |       |       |
|       | spons | spons | spons |       |       |       |       |
|       | e]{.u | e]{.u | es]{. |       |       |       |       |
|       | nderl | nderl | under |       |       |       |       |
|       | ine}: | ine}: | line} |       |       |       |       |
|       |       |       |       |       |       |       |       |
|       | A     | No    |       |       |       |       |       |
|       | ction | A     |       |       |       |       |       |
|       |       | ction |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
| ^[A   | ^4    | **^11 | ^5    | ^[A   | *     | ^3    | ^4    |
| ctual | 3.1%^ | .8%^* | 4.9%^ | ctual | *^6.9 | 8.2%^ | 5.1%^ |
| \ Res |       | *^\*^ |       | \ Res | %^**^ |       |       |
| ponse |       |       |       | ponse | \*\*^ |       |       |
| ]{.un |       |       |       | ]{.un |       |       |       |
| derli |       |       |       | derli |       |       |       |
| ne}:^ |       |       |       | ne}:^ |       |       |       |
|       |       |       |       |       |       |       |       |
| ^Ac   |       |       |       | ^N    |       |       |       |
| tion^ |       |       |       | o\ Ac |       |       |       |
|       |       |       |       | tion^ |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+
| ^[    | ^50%^ | ^50%^ | ^     |       |       |       |       |
| Total |       |       | 100%^ |       |       |       |       |
| \ Res |       |       |       |       |       |       |       |
| ponse |       |       |       |       |       |       |       |
| s]{.u |       |       |       |       |       |       |       |
| nderl |       |       |       |       |       |       |       |
| ine}^ |       |       |       |       |       |       |       |
+-------+-------+-------+-------+-------+-------+-------+-------+

^\*\ These\ errors\ are\ referred\ to\ as\ False\ Positive\ because\ they\ occurred\ when\ no\ action\ was\ necessary\ and\ the\ participant\ responded\ with\ an\ action.^

^\*\*\ These\ errors\ are\ referred\ to\ as\ False\ Negative\ because\ they\ occurred\ when\ an\ action\ was\ necessary\ and\ the\ participant\ responded\ with\ no\ action.^

^The\ nearly\ 19%\ error\ rate\ led\ to\ the\ question:\ Were\ there\ alert\ points\ (trials)\ along\ the\ flight\ path\ that\ produced\ a\ significantly\ higher\ number\ of\ errors\ than\ other\ alert\ points?\ Error\ rate\ was\ examined\ at\ each\ alert\ point.\ Percentages\ of\ correct\ and\ incorrect\ responses\ across\ trials\ are\ shown\ in\ table\ 7-4.\ Four\ points\ along\ the\ flight\ path\ had\ a\ total\ error\ rate\ of\ 25%\ or\ higher\ (trials\ 3,\ 7,\ 10,\ and\ 11).^

**^Table\ 7-4.\ Percentages\ of\ correct\ and\ incorrect\ responses\ across\ alert\ points.^**^\ (Bold\ type\ indicates\ the\ four\ points\ where\ the\ error\ was\ greater\ than\ 25%\ of\ the\ responses.)^

The higher incorrect response rate at some alert points was of concern.
Perhaps participants did not respond to the alerts in the manner that
was intended at these points or participants may not have been oriented
appropriately when the alert was triggered. Using a program to examine
map type and airplane position and heading during each alert trigger,
incorrect responses (false positives and false negatives) were
subcategorized. Each incorrect response, whether it was a false positive
or a false negative, was classified as either a "real error," or as one
four types of "invalid response," outlined in table 7-5.

**^Table\ 7-5.\ Invalid\ response\ categories\ used\ to\ code\ incorrect\ responses.^**

+----------------------------------+----------------------------------+
| **INVALID RESPONSE CATEGORIES**  |                                  |
+----------------------------------+----------------------------------+
| ·                                |                                  |
+----------------------------------+----------------------------------+
| 0.  ^Participant\ responded      | 0.                               |
| \ based\ on\ an\ anticipated\ na |  ^Participant\ was\ provided\ wi |
| vigational\ maneuver\ instead\ o | th\ more\ look\ ahead\ than\ was |
| f\ based\ on\ current\ heading.^ | \ intended\ and\ the\ participan |
|                                  | t\ responded\ to\ higher\ terrai |
| ·                                | n\ ahead\ that\ was\ not\ suppos |
|                                  | ed\ to\ be\ seen.\ (This\ happen |
|                                  | ed\ primarily\ on\ track-up\ map |
|                                  | s\ because\ the\ airplane\ was\  |
|                                  | often\ placed\ at\ the\ bottom\  |
|                                  | of\ the\ screen\ with\ maximum\  |
|                                  | look\ ahead\ instead\ of\ in\ th |
|                                  | e\ center\ on\ north-up\ maps.)^ |
|                                  |                                  |
|                                  | ·                                |
+----------------------------------+----------------------------------+
| 0.  ^Pa                          |                                  |
| rticipant\ was\ not\ on\ the\ ex |                                  |
| pected\ flight\ path\ when\ the\ |                                  |
|  alert\ was\ triggered\ and\ was |                                  |
| \ therefore\ oriented\ incorrect |                                  |
| ly\ to\ the\ terrain\ features.^ |                                  |
|                                  |                                  |
| ·                                |                                  |
+----------------------------------+----------------------------------+

0.  ^Computer\ glitches\ prevented\ the\ researcher\ from\ triggering\ the\ alert\ at\ the\ appropriate\ time\ and\ therefore\ the\ participant\ was\ oriented\ incorrectly\ to\ the\ terrain\ features\ (this\ was\ rare).^

^Half\ of\ the\ incorrect\ responses,\ 19%\ of\ the\ total\ responses,\ were\ coded\ as\ invalid\ responses.\ A\ new\ data\ set\ was\ created\ for\ additional\ analyses\ excluding\ the\ invalid\ responses,\ approximately\ 9%\ of\ the\ data\ points.\ An\ ANOVA\ for\ reaction\ time\ and\ two\ chi-squares\ were\ performed\ on\ the\ reduced\ data\ set;\ these\ results\ were\ not\ significant.^

## ^[7.3\ DISCUSSION]{#_Toc398435635}^

There were no significant differences in interpretability for any of the
displays. If terrain elevations were clearly marked, pilots were able to
quickly interpret displays with varying information densities. These
results open a range of electronic display possibilities. However, if
the goal of this type of research is to create a display that produces
little to no error, then error rate in this experiment indicated that
the format presented was not ideal. Further studies to assess
generalizability of these results and explore ways to increase accurate
interpretability are needed.

# ^[8.\ OVERVIEW]{#_Toc398435636}^

Terrain depictions on maps can be useful, but the depictions must
present information in a clear manner. It is apparent that there needs
to be a mechanism to draw the pilot's attention to the display. This
could be a minimal display (such as the terrain-only map in chapter 5)
or an alerting system used in conjunction with the display (visual alert
in chapter 7).

This research also supports previous work that indicates pilots prefer
displays with higher densities of information. Electronic displays with
terrain depictions, of varying information densities, combined with a
terrain alerting system, will allow pilots to quickly and accurately
interpret information useful for terrain avoidance.

The major issues yet to be addressed which were raised by this series of
experiments were: familiarity with electronic displays (amount of
training), clarity and interpretability of the display (placement and
presentation of elevation information), and airplane relative position
on the map. Each of these issues need to be addressed. To further
investigate the human factors aspects of electronic displays, it is
necessary to modify training practices in experimentation, including
training to proficiency. One possibility for modifying elevation
information is to label elevations of a safety floor and only present
the terrain that is relevant to the pilot (terrain at his altitude or
higher). The position of the airplane could be modified to provide
maximum look ahead, instead of in the middle of the screen, as it was in
the last experiment (chapter 7).

Overall, this series of experiments addressed important issues for
designing electronic display terrain avoidance systems and of interest
for future research.

# ^[9.\ REFERENCES]{#_Toc398435637}^

^Amar,\ M.J.,\ R.J.\ Hansman,\ Jr.,\ D.J.\ Hannon,\ T.W.\ Vaneck,\ and\ A.I.\ Chaudhry.\ (1995).\ A\ Human\ Subject\ Evaluation\ of\ Airport\ Surface\ Situational\ Awareness\ Using\ Prototypical\ Flight\ Deck\ Electronic\ Taxi\ Chart\ Displays.\ Final\ Report\ DOT-VNTSC-FAA-95-36\ (DOT/FAA/AR-95/103).^

^Bud,\ M.,\ P.\ Mengert,\ S.\ Ransom,\ and\ M.D.\ Stearns.\ (1997).\ General\ Aviation\ Accidents,\ 1983-1994:\ Identification\ of\ Factors\ Related\ to\ Controlled-Flight-Into-Terrain\ (CFIT)\ Accidents.\ Final\ Report\ DOT-VNTSC-FAA-97-8.^

^Hannon,\ D.J.\ and\ M.S.\ Huntley,\ Jr.\ (1995).\ Electronic\ Depiction\ of\ Instrument\ Approach\ Procedure\ (IAP)\ Charts.\ Phase\ I:\ Development\ and\ Preliminary\ Evaluation.\ Final\ Report\ DOT-VNTSC-FAA-95-14.^

^Kuchar,\ J.K.\ and\ R.J.\ Hansman,\ Jr.\ (1993).\ An\ exploratory\ study\ of\ plan-view\ terrain\ displays\ for\ air\ carrier\ operations.\ ^*^International\ Journal\ of\ Aviation\ Psychology^*^,\ 3(1),\ 39-54.^

^Kuchar,\ J.K.\ and\ R.J.\ Hansman,\ Jr.\ (1993).\ Results\ of\ the\ terrain\ database\ resolution\ requirements\ study\ (Report\ ASL-93-4).\ Cambridge,\ MA:\ MIT\ Aeronautical\ Systems\ Laboratory.^

^Kuchar,\ J.K.\ and\ R.J.\ Hansman,\ Jr.\ (1991).\ Advanced\ terrain\ displays\ for\ transport\ category\ aircraft\ (Report\ ASL-91-3).\ Cambridge,\ MA:\ MIT\ Aeronautical\ Systems\ Laboratory.^

# ^Appendix\ A-1.\ Frasca\ 242\ Simulator^

# ^Appendix\ A-2.\ Simulator\ Flight\ Instrument\ PanelAppendix\ B-1.\ Electronic\ Approach\ DisplayAppendix\ B-2.\ Paper\ Approach\ Chart^

*For Research Purposes Only*

# ^Appendix\ B-3.\ Flight\ Checklist^

Approach number:

[0]{.underline} [1]{.underline} [2]{.underline} [3]{.underline}
[4]{.underline} [5]{.underline}

Called in Final Approach \_ \_ \_ \_ \_ \_

Applied carb heat (on) \_ \_ \_ \_ \_ \_

Boost pumps (on) \_ \_ \_ \_ \_ \_

Wing Flaps (down) \_ \_ \_ \_ \_ \_

Landing gear (down) \_ \_ \_ \_ \_ \_

Called in Missed Approach \_ \_ \_ \_ \_ \_

Applied carb heat (off) \_ \_ \_ \_ \_ \_

Boost pumps (off) \_ \_ \_ \_ \_ \_

Wing Flaps (up) \_ \_ \_ \_ \_ \_

Cowl Flaps (open) \_ \_ \_ \_ \_ \_

Landing gear (up) \_ \_ \_ \_ \_ \_

Maintained speed: +/- 5 kts

Maintained altitude: +/- 100 ft.

Responded to ATC when flight

corrections requested

# ^Appendix\ B-4.\ Post-Flight\ Questionnaire^

^1.\ What\ kind\ of\ information\ was\ most\ helpful\ to\ you\ on\ the\ electronic\ map?^

^2.\ In\ particular,\ was\ the\ terrain\ depiction\ useful\ information?^

^3.\ Were\ there\ specific\ legs\ of\ the\ flight\ where\ the\ electronic\ map\ facilitated\ flying?^

^4.\ At\ Waypoint\ turns,\ did\ you\ fly\ through\ them\ or\ anticipate\ them?^

^\ In\ either\ case,\ did\ you\ use\ the\ moving\ map\ or\ the\ needle\ for\ the\ turn?^

^5.\ Which\ information\ on\ the\ map\ did\ you\ find\ unnecessary?^

^6.\ Would\ you\ find\ this\ map\ useful\ in\ VFR\ conditions\ as\ well?^

^7.\ Any\ other\ suggestions\ or\ comments\ on\ the\ map\ display?^

^8.\ Any\ other\ suggestions\ or\ comments\ on\ the\ flight\ testing\ procedure?^

# ^Appendix\ C-1.\ Electronic\ Display\ for\ Map\ Condition\ 2:\ Navigation\ and\ Terrain\ Information^

# ^Appendix\ C-2.\ Electronic\ Display\ for\ Map\ Condition\ 3:\ Terrain\ Information[\ ]{.underline}Appendix\ C-3.\ Chart\ of\ Flight\ Path^

# ^Appendix\ C-4.\ Script\ for\ Experiment\ 2\ (Albany\ Experiment)^

**Pilot Briefing:** Airplane starts at TRESA at 3000' on V123 direct to
Albany.

Pilot is talking to NY approach control.

ATIS at Albany is not functioning.

Autopilot is NOT to be used.

**Flight:**

^ATC:\ "N123SH\ This\ is\ New\ York\ approach.\ Maintain\ 3000"^

^(after\ a\ few\ minutes\ of\ flight)^

^ATC:\ "N123SH\ contact\ Albany\ approach\ control\ on\ 122.4"^

^\ (pilot\ contacts\ Albany)^

^Alb.\ ATC:\ "N123SH\ Albany\ altimeter\ setting\ is\ two-niner-niner-seven"^

^(at\ WIGAN)^

^ATC:\ "N123SH\ this\ is\ a\ vector\ around\ weather\ along\ Victor\ 123.\ Turn\ Left\ to\ heading\ of\ two-niner-zero\ 290.\ Expect\ 15\ miles."^

^(after\ new\ heading\ is\ established)^

^ATC:\ "N123SH\ contact\ Albany\ on\ 125.025"^

^(pilot\ notices\ error)^

^ATC:\ "Roger,\ N123SH,\ make\ that\ 122.6"^

^ATC:\ "N123SH\ your\ transponder\ is\ intermittent,\ squawk\ code\ 2526\ and\ ident"^

^(before\ intersection\ with\ V213\ and\ terrain)^

^ATC:\ "N123SH,\ radar\ contact\ has\ been\ lost,\ you\ are\ cleared\ direct\ to\ Albany"^

^(if\ pilot\ questions\ altitude)^

^ATC:\ "Roger,\ expect\ higher\ in\ 2\ miles"^

^ATC:\ "N123SH\ climb\ and\ maintain\ 7000."^

# ^Appendix\ D-1.\ Static\ Electronic\ Display\ Format:\ 500-ft\ Contours,\ Filled\ Contours,\ and\ Navigation\ InformationAppendix\ D-2.\ Static\ Electronic\ Display\ Format:\ 1000-ft\ Contours,\ Filled\ Contours,\ and\ Navigation\ InformationAppendix\ D-3.\ Static\ Electronic\ Display\ Format:\ 2000-ft\ Contours,\ Unfilled\ Contours,\ and\ Navigation\ Information^

# ^Appendix\ D-4.\ Electronic\ Display\ Format\ Preference\ Task\ Rating\ Criteria^

For this part of the study we will ask you to look at a series of two
chart comparisons. There will be 66 comparisons in all.

Your task will be to rate the maps based on the scenario you just flew.

Ask yourself the following question:

**^Which\ of\ the\ two\ maps\ would\ have\ been\ more\ useful\ to\ me\ during\ the\ flight\ to\ notice\ and\ avoid\ terrain?^**

^Before\ you\ begin,\ you\ will\ have\ a\ chance\ to\ look\ at\ each\ map\ individually\ at\ your\ own\ pace.\ During\ this\ time,\ think\ about\ criteria\ you\ will\ use\ for\ evaluation.\ Please\ try\ to\ maintain\ consistent\ criteria\ throughout\ the\ rating\ task.^

^The\ following\ are\ a\ few\ suggestions\ for\ setting\ criteria.\ Please\ note\ they\ are\ only\ [suggestions]{.underline}:\ ^**^You\ are\ not\ limited\ to\ these\ nor\ are\ you\ required\ to\ use\ them.^**

^-\ Is\ the\ level\ of\ clutter\ useful?^

^-\ Is\ the\ level\ of\ detail\ useful?^

^-\ Is\ the\ text\ easy\ to\ interpret?^

^-\ Is\ the\ terrain\ presentation\ clear?^

^-\ Is\ the\ navigation\ information\ clear\ and/or\ necessary?^

^You\ will\ be\ able\ to\ pace\ yourself\ through\ the\ comparisons.\ Click\ on\ the\ rating\ circle\ that\ most\ appropriately\ matches\ your\ evaluation\ of\ the\ maps.\ Choosing\ a\ rating\ will\ not\ advance\ the\ computer\ to\ the\ next\ comparison.\ Only\ when\ you\ click\ on\ "^**^Next^**^"\ will\ the\ computer\ advance.\ You\ may\ also\ go\ back\ to\ previous\ comparisons\ if\ you\ want\ to\ change\ your\ mind.^

# ^Appendix\ D-5.\ Static\ Electronic\ Display\ Format\ Comparisons^

# ^Appendix\ D-6.\ Display\ Rating\ Questionnaire^

^I.\ Can\ the\ map\ be\ easily\ interpreted?\ Explain.^

^II.\ What\ suggestions\ for\ improvement\ do\ you\ have?^

^A.\ What\ information\ should\ or\ shouldn't\ be\ included?^

^B.\ What\ are\ the\ best\ and\ worst\ features\ of\ the\ map?^

^C.\ In\ what\ other\ ways\ could\ information\ be\ presented?^

^III.\ Is\ the\ terrain\ depiction\ clear\ and\ easy\ to\ understand?\ Explain.^

^A.\ Does\ it\ have\ enough\ detail?\ If\ no,\ what\ else\ should\ be\ added?\ ^

^B.\ Does\ it\ have\ unnecessary\ detail?\ If\ yes,\ what\ should\ be\ removed?^

^IV.\ If\ the\ map\ contains\ navigation\ information:^

^A.\ Is\ the\ depiction\ clear\ and\ easy\ to\ understand?\ If\ no,\ why?^

^B.\ Is\ there\ enough\ information?\ Should\ anything\ be\ added\ or\ removed?^

# ^Appendix\ E-1.\ Low-Density\ Display\ Format^

**Appendix E-2. Medium-Density Display FormatAppendix E-3. High-Density
Display Format** **Appendix E-4. Paper Chart Including Flight Paths for
the **

**Practice and Experimental Runs**

*practice flight indicated with dotted line and experimental flight
indicated with straight line*

# ^Appendix\ E-5.\ Example\ of\ Flight\ Script\ (for\ Experiment\ 4)^

^Talking\ to\ ATC\ at\ Poreman\ Airport\ on\ 122.4^

^\ ATC:\ "N123SH\ this\ is\ Poreman\ Approach.\ Maintain\ 5000\ feet"^

^1\ (phase\ 1:\ first\ alert:\ alarm)^

^2\ (phase\ 1:\ second\ alert:\ alarm)^

^3\ (phase\ 1:\ third\ alert:\ false)^

^\ ATC:\ ^**^"^^N123SH\ altimeter\ is\ two-niner-niner-four^^"^**

^4\ (phase\ 2:\ first\ alert:\ alarm)^

^\ ATC:\ ^**^"^^N123SH\ transponder\ is\ intermittent,\ squawk\ code\ 4021\ and\ ident^^"^**

^5\ (phase\ 2:\ second\ alert:\ false)^

^\ ATC:\ ^**^"^^N123SH\ contact\ Poreman\ on\ 131.25^^"^**

^6\ (phase\ 2:\ third\ alert:\ alarm)^

^\ ATC:\ ^***^"^**^N123SH\ Report\ crossing\ AYEST^**^"^***

^7\ (phase\ 3:\ first\ alert:\ alarm)^

^\ ATC:\ ^***^"^**^N123SH\ contact\ Poreman\ on\ 122.75^**^"^***

^\ ATC:\ ^**^"^^N123SH\ altimeter\ is\ 29.99^^"^**

^8\ (phase\ 3:\ second\ alert:\ false)^

^\ ATC:\ ^**^"^^N123SH\ transponder\ is\ intermittent\ squawk\ code\ 2040\ and\ ident^^"^**

^9\ (phase\ 3:\ third\ alert:\ false)^

^\ ATC:\ ^**^"^^N123SH\ contact\ Poreman\ on\ 130.5^^"^**

^\ ATC:\ ^***^\ ^*^"^^N123SH\ transponder\ is\ intermittent,\ squawk\ code\ 3231\ and\ ident^^"^**

^10\ (phase\ 4:\ first\ alert:\ false)^

^\ ATC:\ ^**^"^^N123SH\ altimeter\ is\ 29.98^^"^**

^11\ (phase\ 4:\ second\ alert:\ false)^

^\ ATC:^**^\ ^^"^^N123SH\ contact\ Poreman\ approach\ on\ 128.75^^"^**

^12\ (phase\ 4:\ third\ alert:\ alarm)^

# ^Appendix\ E-6.\ Post-Experiment\ Questionnaire^

1\. ^Which\ map(s)\ did\ you\ find\ the\ easiest\ to\ interpret?\ Why?^

2\.
^Which\ map(s)\ did\ you\ find\ the\ most\ difficult\ to\ interpret?\ Why?^

3\.
^In\ what\ other\ ways\ could\ this\ alert\ information\ be\ presented?^

4\.
^Should\ more\ detail\ be\ added\ to\ any\ of\ the\ maps?\ If\ yes,\ what\ else\ should\ be\ added?\ ^

5\.
^Did\ any\ of\ the\ maps\ have\ unnecessary\ detail?\ If\ yes,\ which\ maps\ and\ what\ should\ be\ removed?^

6\.
^Which\ orientation\ of\ the\ map\ (north\ up\ or\ track\ up)\ did\ you\ prefer?\ Why?^

7\. ^Any\ other\ suggestions/comments:^

8\.
^Have\ you\ had\ any\ flight\ experience\ in\ mountainous\ areas?\ If\ so,\ how\ much?^

[^1]: ^\ Bud,\ M.,\ Mengert,\ P.,\ Ransom,\ S.,\ and\ Stearns,\ M.D.\ General\ Aviation\ Accidents,\ 1983-1994:\ Identification\ of\ Factors\ Related\ to\ Controlled-Flight-Into-Terrain\ (CFIT)\ Accidents.\ Final\ Report\ DOT-VNTSC-FAA-97-8.\ May\ 1997.^

[^2]: ^\ Bud,\ M.,\ Mengert,\ P.,\ Ransom,\ S.,\ and\ Stearns,\ M.D.\ General\ Aviation\ Accidents,\ 1983-1994:\ Identification\ of\ Factors\ Related\ to\ Controlled-Flight-Into-Terrain\ (CFIT)\ Accidents.\ Final\ Report\ DOT-VNTSC-FAA-97-1.\ May\ 1997.^

[^3]: ^\ Due\ to\ the\ time\ (a\ minimum\ of\ four\ hours\ per\ participant)\ and\ cost\ requirements\ with\ pilot\ participants,\ sample\ sizes\ were\ kept\ small.\ Although\ these\ sample\ sizes\ were\ not\ large\ enough\ to\ provide\ significant\ statistical\ power,\ they\ were\ considered\ adequate\ for\ identifying\ trends\ in\ the\ data.^

[^4]: ^\ Hannon,\ D.J.,\ and\ Huntley,\ M.S.,\ Jr.\ Electronic\ Depiction\ of\ Instrument\ Approach\ Procedure\ (IAP)\ Charts.\ Phase\ I:\ Development\ and\ Preliminary\ Evaluation.\ Final\ Report\ DOT-VNTSC-FAA-95-14.\ May\ 1995.^

[^5]: ^\ Amar,\ M.J.,\ Hansman,\ Jr.,\ R.J.,\ Hannon,\ D.J.,\ Vaneck,\ T.W.,\ and\ Chaudhry,\ A.I.\ A\ Human\ Subject\ Evaluation\ of\ Airport\ Surface\ Situational\ Awareness\ Using\ Prototypical\ Flight\ Deck\ Electronic\ Taxi\ Chart\ Displays.\ Final\ Report\ DOT-VNTSC-FAA-95-36\ (DOT/FAA/AR-95/103).\ November\ 1995.^

[^6]: ^\ Kuchar,\ J.K.\ and\ R.J.\ Hansman\ Jr.\ (1991).\ Advanced\ terrain\ displays\ for\ transport\ category\ aircraft\ (Report\ ASL-91-3).\ Cambridge,\ MA:\ MIT\ Aeronautical\ Systems\ Laboratory.^

    ^Kuchar,\ J.K.\ and\ R.J.\ Hansman\ Jr.\ (1993).\ An\ exploratory\ study\ of\ plan-view\ terrain\ displays\ for\ air\ carrier\ operations.\ ^*^International\ Journal\ of\ Aviation\ Psychology^*^,\ 3(1),\ 39-54.^

[^7]: ^\ Kuchar,\ J.K.\ and\ R.J.\ Hansman\ Jr.\ (1993).\ Results\ of\ the\ terrain\ database\ resolution\ requirements\ study\ (Report\ ASL-93-4).\ Cambridge,\ MA:\ MIT\ Aeronautical\ Systems\ Laboratory.^

[^8]: ^\ This\ technique\ was\ based\ on\ the\ one\ used\ in\ Amar,\ M.J.,\ et\ al.\ A\ Human\ Subject\ Evaluation\ of\ Airport\ Surface\ Situational\ Awareness\ Using\ Prototypical\ Flight\ Deck\ Electronic\ Taxi\ Chart\ Displays.\ Final\ Report\ DOT-VNTSC-FAA-95-36\ (DOT/FAA/AR-95/103).\ November\ 1995.^

[^9]: ^\ Kuchar,\ J.K.\ and\ R.J.\ Hansman\ Jr.\ (1993).\ Results\ of\ the\ terrain\ database\ resolution\ requirements\ study\ (Report\ ASL-93-4).\ Cambridge,\ MA:\ MIT\ Aeronautical\ Systems\ Laboratory.^