By following sound human factors design prin-
ciples, many of the difficulties that the subjects en-
countered during their interactions with the GPS unit
could have been avoided. It was found that the number
of button presses per task, on average, was positively
correlated with the length of time users spent focused
on the GPS unit during a task. This finding empha-
sizes that a focus should be placed on ease-of-use,
including the reduction of button presses necessary
for task completion and the minimization of head-
down time with regard to cockpit-based devices.

One salient example of how head-down glance
times can affect air safety pertains to scanning the
environment for other aircraft. If two aircraft are ona
collision course with a combined closing velocity of
300 knots and a separation of 1 nautical mile, the
aircraft will collide in 12 seconds. Therefore, if the
attention of the pilots is simultaneously diverted
inside their cockpits for 10 seconds, as was observed
in this study (see Figure 3), the pilots of the two
aircraft will have only 2 seconds in which to detect and
avoid a collision.

CONCLUSIONS AND SUGGESTIONS FOR
CHANGE

The findings of this usability study were very infor-
mative and provided a good indication of the design
principles that should be applied to enhance ease-of-
use of the evaluated GPS unit. These results may be
used in conjunction with the findings from subse-
quent studies concerning display design to develop
general performance-based guidelines. In the follow-
ing section are listed some basic human factors design
principles that were found to be lacking with regard to
the design of the evaluated GPS unit. Application of
these principles is necessary to achieve ease-of-use
which, in turn, will minimize head-down time and
increase safety.

Function allocation and terminology
*If the methods for activating various functions are
included in a help information section at the bottom
ofeach screen, all pertinent functions should be given
in that section, It may appear that the function does
not exist if it is not intuitive and is not referred to in
the help section.

14

* A given function should be assigned to one but-
ton, not different buttons in different areas of the
interface.

Terms should be used consistently throughout the
system.

*The term “escape” has no inherent meaning to a
user unfamiliar with computer jargon and should
be avoided for this reason.

Button identification

*Shape coding of buttons should reflect function
when possible to minimize head-down time. Tac-
tile coding, such as raised bumps or edges, would
also be useful for non-visual control identifica-
tion. Push-button labels should reflect the func-
tion of the button.

Feedback

Feedback should give the user an indication of
what type of error has occurred.

* Auditory tones should have some meaning (for
instance, a user should know the distinction be-
tween the meaning of a two-beep and three-beep
error message).

*Textual feedback would be very useful to the
casual user or novice user for relaying the cause or
type of error.

General

+ A button which, when pressed, brings the user back
toacentral location in the menu structure or the most
frequently used portion of the software interface
would save button presses and time.

*The implementation of an “undo” function would
save time and keystrokes —especially if the user is
unaware of how the present state of operation was
reached or is unaware of how to exit the present state.

*Features/functions should be documented in a
manual, help screen, or other source. Functions that
are not documented are not expected by the user and
can interfere with efficient use of the system.

+ An editing function should be incorporated into the
design to allow alteration of erroneously entered text.
This is important in the flight environment, as turbu-
lence or distractions could cause the entering of an
incorrect waypoint.

* Deactivating the flight plan before altering it may
have caused more problems (extra button presses and
increased head-down time) than it prevented.

¢ Functions that are infrequently used should be de-
signed intuitively to avoid use of the owner’s manual.


Button Press Count

34

32

30

28

26

24

22

20

18

16

14

12

10

123.4 5 6 7 8 9 1011 1213 1415 1617 18 19 20 21 22 23 24 25 27 28 29 30 31 32 33 34 35 36 37
Task Number

@ Excess Button Presses
Minimum Button Presses

Figure 2. Minimum Button Presses Needed to Accomplish Tasks and
Button Press Count Exceeding Minimum for Each Task.




FOREWORD

A series of studies is being conducted as part of the Civil Aeromedical Institute’s (CAMI’s) General
Aviation Human Factors Research program, which incorporates both near-term and far-term objectives.
The program focuses on the integration of instrumentation into the general aviation cockpit. Because
many companies are designing and developing general aviation in-cockpit devices, there is an increasing
need to determine the degree to which sound human factors principles are being incorporated. The
program involves the use of systematic usability testing procedures at the Institute’s Usability Testing
Laboratory to aid in identifying areas of design that may either hinder or enhance pilot performance and
flight safety. The objective of this program is to generate comprehensive guidelines for the design and
implementation of displays for current and future generation general aviation aircraft.

The Flight Standards Service (AFS-1) sponsored the study; the General Aviation & Commercial
Division (AFS-800) provided project oversight. Dr. Thomas McCloy and Mr. Ronald Simmons,
Human Factors Division (AAR-100), provided program coordination.

iii


REFERENCES

AOPA Air Safety Foundation. (1995). GPS/Loran:
A guide to modern navigation. Frederick, MD:
Author.

Beringer, D.B. (1996). Use of off-the-shelf pe-based flight
simulators for aviation human factors research (Tech-
nical Report DOT/FAA/AM-96/15). Washing-
ton, DC; Federal Aviation Administration. Avail-
able from: National Technical Information Ser-
vice, Springfield, VA 22161. Order No.
ADA309237.

Huntley, M.S., Turner, J.W., Donovan, C.S., and
Madigan, E. (1995). FAA aircraft certification:
Human factors and operations checklist for standalone
GPS receivers (DOT-VNTSC-FAA-95-12). Wash-
ington, DC: Federal Aviation Administration.

Landauer, T.K., and Nachbar, D.W. (1995). Selection
from alphabetic and numeric menu trees using a
touch screen: Breadth, depth and width. In: ACM
CHI ‘85 Conference Proceedings (pp. 73-8). San
Francisco, CA.

15

Mayhew, D. J. (1992). Principles and guidelines in
software user interface design. Englewood Cliffs,
NJ: Prentice-Hall.

Miller, D.P. (1981). The depth/breadth tradeoff in
hierarchical computer menus. In: Proceedings of
the Human Factors Society 25th Annual Meeting
(pp. 296-300). Santa Monica, CA: Human Fac-
tors Society.

Nendick, M.D., and St. George, R. (1995). Human
factors aspects of global positioning systems (GPS)
equipment: A study with New Zealand pilots. Pre-
sented at the Eighth International Symposium on
Aviation Psychology, Columbus, OH.

U.S. Department of Transportation, Federal Aviation
Administration. (1983). Pilots’role in collision avoid-
ance (Advisory Circular 90-48C).


sweat rey suy suuy) sup) sun 9sino)
TROL, quaosoq feng, AyD072A, opmunty 2ourIsiq] oumexoduay, powololg po doais, wsenuoy
Aeydsiq. sdD aousTaJoy
ezientuy indyng weg 99INOG XL woyney penuaiayiq quON smeig SdD
sean :
sowey an TINMa SIOARY smog speoey ue Mane skemysty
suonomsqQ, OOLaV seary seary qnokey]
Teonso,, vaow Pala poomsoy payjonuoy | | HOH aaN YOA Kemuny suodary
Poy
<
sinoquod sommpeay sply
wadog SUONOENSSY Teiseop uonesiaen,
reorpvee,| | soe, | | Smuaueem | | Sepumog | [ ssumas | | sfumes | | sfumes | Trusoy awa] | 2280 | {reusog aun
sBumog sSumes suonoun,| suonouny
uoneyyeisuy dey Suraopy owt], xi



APPENDIX A

Menu Flow Diagram

Al


Average Glance-Down Time (Seconds)

75

70

65

55

50

45

40

35

30

25

20

15

10

5 6 7 8 9 1011 12 13 14 15 16 17 18 19 20 21 22 23 24 25 27 28 29 30 31 32 33
Task Number

Figure 4. Average Cumulative Head-Down Time for Tasks in Flight.



Table 1. GPS Unit Buttons and Associated Functions _

¢ Turns POWER On / Off.

¢ Brings up the WAYPOINT menu in the Moving Map display.

* Brings up the NEAREST menu from Moving Map, Navigation, and Location
screens.

* Cannot be accessed from Calculator, Flight Plan, Database and Setup Menu
Screens, any or their sub-options, or while in the Waypoint menu.

* Plots a DIRECT course to selected navaid or destination while in the Moving
Map and Database screens.
* Cannot be accessed from any other location.

Toggles between Main Menu and Setup Menu while in either respective
location.

* Used to escape/exit current menu location and backtrack to a higher level
menu.

* Locks in selection (ENTER).

* Activates option.

© Switches to next flight plan while changing flight plans in Waypoint meni.

¢ Removes (takes OUT) user defined waypoints in the Database.

+ “Flips” through information pages when in Moving Map display.

* Activates flight plan while in the Flight Plan screen when cursor is located on
a waypoint. (undocumented function).

+ Toggles between “Position” and “Cursor” modes in the Moving Map display.

* Zooms IN while in Moving Map display.
¢ Zooms IN while in user waypoint viewing screen of Database.
¢ Inserts waypoints in Flight Plan screen.

* Zooms OUT while in Moving Map display.
* Zooms OUT while in user waypoint viewing screen of Database.
¢ Removes (takes OUT) waypoints in Flight Plan screen.

* Moving Map cursor position UP.
* Moves UP to previous menu option.

¢ “Flips” to next information page when in Database.
* Scrolls UP through alphabet when inputting alphanumeric text.

* Moving Map cursor position DOWN.
* Moves DOWN to next menu option.

+ “Flips” to previous information page while in the Database.
¢ Scrolls DOWN through alphabet when inputting alphanumeric text.

* Moving Map cursor position LEFT.
* Move to character position to the LEFT when inputting alphanumeric text.
© Switches to previous flight plan while in Flight Plan screen.

Ooo°0 ceo 86 @6eee

* Moving Map cursor position RIGHT.
* Moves to character position to the RIGHT when inputting alphanumeric text.
¢ Switches to next flight plan while in Flight Plan screen.

(Note: Variations in fonts will be referred to in the Results section of this paper. [Also take
special note of the bold, bold-underlined, and bold-italic lines, which emphasize some of the
inconsistencies in function allocation.])





COCKPIT INTEGRATION OF GPS: INITIAL ASSESSMENT —
MENU FORMATS AND PROCEDURES

INTRODUCTION

Justification for Current Testing

There has been a dramatic increase in the use of the
global positioning system (GPS) by general aviation
pilots that has led to a plethora of manufacturers
producing GPS receivers. Unfortunately, there is no
standard for data entry and retrieval, display type, or
placement within the cockpit. General aviation pilots
often use hand-held or add-on GPS units because they
are relatively inexpensive and can be placed within the
cockpit where space is available. While the use of GPS
is a significant aid to navigation, it may distract the
pilot from the primary tasks of flying and visual
scanning. This is especially true when the receivers are
not integrated into the instrument panel and do not
utilize optimal software interface design. For instance,
a poorly designed GPS menu structure may be such
that it unnecessarily complicates the navigation task.

It should be noted that the potential problems seen
with GPS units relative to the pilot-computer inter-
face are not unique to these devices. This is merely the
latest round of design efforts attempting to incorpo-
rate new technologies into the cockpit, with the usual
difficulties associated with rapidly bringing a product
to market. Many of the “smaller” companies (as com-
pared with the large airframe manufacturers) involved
in the production of these devices do not have any in-
house human factors support, and much of the result-
ing design derives from engineering and marketing
concerns, not necessarily human performance con-
cerns. Thus, we see the same kinds of problems with
these devices as have been seen in previous installa-
tions of computer-based cockpit aids to navigation. In
particular, the familiar problem areas are once again
the physical (hardware) and logical (software) inter-
faces between the pilot and the system. The research
represented in this study was an initial effort to
examine a typical (representative) portable GPS de-
vice and to determine what types of problems might
be associated with its operation and, hence, might
reduce its usability to the pilot.

Research addressing the human factors aspects of
GPS controls, displays, and user attitudes was carried
out by Nendick and St. George (1995). Data from

172 questionnaires filled out by New Zealand pilots
who had utilized GPS were used in the analyses. It was
found that errors included “transposing coordinates;
transcription errors from maps when entering data
into the GPS; hitting the wrong key; forgetting the
keying sequence to obtain the correct information;
and inadvertently pressing a key twice in turbulence,
resulting in a change of mode or number or letter.”
Twenty percent of the questionnaires analyzed by
Nendick and St. George reported misreading of dis-
played information.

Because all of the information needed by a pilot for
GPS navigation cannot fit into the display window at
once, the information is separated into various cat-
egories. Each category may be several pages deep.
Ideally, the information needed most often should be
on a single page, with the secondary information on
adjoining pages. However, individual manufacturers
have their own ideas about which items should go
together (AOPA Air Safety Foundation, 1995). Miller
(1981) employed an on-screen, word-search task for a
depth/breadth tradeoff study and found that a two-
level depth with eight-choice breadth at each level
produced the fastest acquisition times and the fewest
errors (compared with six-level depth/two-choice
breadth, three-level depth/four-choice breadth, and
one-level depth/sixty-four-choice breadth). An in-
crease in depth and breadth increased acquisition
time. However, Miller suggests that an expansion in
breadth is preferred over an expansion in depth.
Landauer and Nachbar (1985) came up with a similar
conclusion when employing touch screens. The opti-
mization of depth and breadth is an important design
consideration for tasks requiring speed and accuracy.
Miller (1981) concluded that, for systems of moder-
ate size, the number of hierarchical levels should be
minimized but not at the expense of display crowding.

Sometimes technical constraints will lead to a less
than optimal interface design. Many systems are well
organized in a technical sense, and focus on features
and functions, rather than the overall purpose of the
device and interface. The organization of a system
should be designed with a thorough analysis and
understanding of the primary tasks of the intended
users (Mayhew, 1992).




with the work required to accomplish a given task and
indicate how engrossed subjects became with the GPS
unit during certain tasks.

An Advisory Circular (90-48C, 1983) was dissemi-
nated by the Federal Aviation Administration to make
pilots aware of the actions they should perform to
reduce midair collisions and near misses. Besides
scanning for other aircraft, pilots must be aware of
nearby obstacles, changes in terrain, etc. A valuable
tool in preventing mishaps and promoting safety is
visual scanning. Another important task that pilots
must undertake is scanning flight instruments. Time
away from scanning the outside environment and the
aircraft instruments reduces situational awareness;
therefore, head-down time should be minimized.

DISCUSSION AND RECOMMENDATIONS

It is suggested that in cases where some form of
menu is deemed necessary to accomplish specific
tasks, the focus should be on simplifying the menu
structure and unit manipulation so as to allow the
pilot the opportunity to attend to visual scanning
tasks. The observed head-down time is cause for some
concern because pilots who are not scanning the
environment or aircraft instruments for relatively
long periods of time may be at risk of becoming a
safety hazard to themselves or others. The number of
button presses necessary to accomplish certain tasks,
and the number of button presses that the GPS users
executed for certain tasks, in general, appear excessive
and should be reduced through redesign efforts.

There are several primary factors that may contrib-
ute to the observed button-press count and head-
down time in this study, including: (1) requirements
for moving through the menu structure, (2) users’
level of understanding of buttons, (3) ease of recovery
from erroneous inputs, (4) quality and usefulness of
feedback, and (5) the level of consistency within the
GPS unit interface.

Ease of moving through the menu structure was
rated rather positively by subjects after using the GPS
unit. However, more efficient means of travel through
the menu system would be advantageous. For in-
stance, pressing a single button to return to a main
menu or “central location” would be useful, as well
as “direct” access to frequently used functions.

A list of the nearest airports, intersections, etc.,
could be accessed with a single button press allowing
the pilot to choose a destination and plot a direct

13

course to that site. However, this function could not
be accessed from every location in the menu structure.
It is suggested that this function be available from
anywhere in the system so that a pilot under stress
because of mechanical problems, for instance, can easily
use the navigation aid and attend to flying the aircraft.

The subjects’ understanding of the push buttons
was occasionally less than desirable. This problem was
due, in part, to: (1) multi-function buttons, (2) non-
intuitive labeling, and (3) the lack of appropriate
“help messages.”

Recovery from erroneous inputs was often time
consuming, occasionally frustrating, and very diffi-
cult for a subject. An “undo” function would have
been useful in several cases, especially following the
accidental deletion of a leg. Also, there were minimal
editing functions in the flight plan screen; therefore,
if a waypoint was inadvertently “locked in,” the user
would have to delete the unwanted waypoint, and
enter the correct waypoint from the beginning, in-
stead of editing the initial waypoint.

Feedback was only in the form of auditory tones,
except for one location in the software interface in
which a textual feedback message was observed. Al-
though the auditory-error indication feedback was
useful for informing users that an error had occurred,
no indication of the nature of the error was given. It
is suggested that textual feedback be given if an error
occurs. This would prevent errors from inducing
confusing cycles as seen in the “double errors” de-
scribed earlier. Also, casual users may re-familiarize
themselves with the system more quickly with infor-
mative feedback. The presently incorporated feed-
back tones were not tested at actual aircraft noise
levels, so it is not known if the tones could be heard in
an actual aircraft environment.

Inconsistency of push-button function appeared in
several areas of the GPS unit interface. The same
button should not be used for different tasks and
different buttons used for the same task. For instance,
it is advisable to consistently assign the ENT button
to the function of entering data, options, etc. How-
ever, the ENT button was used for scrolling through
flight plans, entering, deleting, and “flipping” through
waypoint information pages. The OUT button was
used for functions pertaining to zooming out and
taking out waypoints. However, the ENT button also
served to take out waypoints in one area of the system.


Data aan Antenna J. ack
Cz
Display
Screen \
G

Figure 1. GPS Unit Button Configuration and Labeling (not to scale).


“pakeydstp

SE JoqRy fem “yred
soffOy 01 passaid 9q,
asnur uonng saveo1puy

ammyesodwiay, poodsiry
" epmn < Burpea)
woy uasou aqanp pmaly ALY epising poworpuy IPeOH
you uondg ‘said
woung £q pareanioe
nuoul sareoipuy ...

end aaa ados op | | osnee | | aLepmery | | opmniy

oasosou aiqereay wing jong 7830

uonduinsuod uonesiaeN:
Teng PULA TeOHIOA,

Aeidsiq Keidsia Keidsiq

aan NI YOA wasn uvig 3 uonesoy woneStAeN Aeidsiq dew
|

o asequieg eid 143i soeNgeD uoKe0077 uonesIARN, dey Suraopy

t i f i f

nua Ue

JoUITEPOSIG.

t
Sd vo uNy,



DOT/FAA/AM-98/9

Office of Aviation Medicine
Washington, D.C. 20591

Cockpit Integration of GPS:

Initial Assessment—Menu
Formats and Procedures

Steven S. Wreggit
Delbert K Marsh, II

Civil Aeromedical Institute
Federal Aviation Administration
Oklahoma City, OK 73125

March 1998

Final Report

This document is available to the public
through the National Technical Information
Service, Springfield, Virginia 22161.

@

U.S. Department
of Transportation

Federal Aviation
Administration


Table 2. Instructions Given to Subjects Doing Usability Testing

. In the flight plan screen set a waypoint at Wichita Mid-Continent

. Set a second waypoint at Newton City Airport, identified KEWK.

. Activate the flight plan.

RLOlN

. In the moving map display, determine the tower frequency of the airport you are

presently at, KICT, and read it back to me.

wn

. In the Calculator menu, input the information necessary to calculate wind speed and

direction.

. Access the Flight Plan screen.

. Delete only the second waypoint which is the Newton City Airport, identified KEWK

. Add a waypoint in its place at the STONS intersection.

. Add a third waypoint at the Shroeder Field Airport, identified as M66.

. Set the flight plan up to fly and go to the moving map display.

=] DS] 0] 00] IID

. In the waypoint menu in the moving map display, change the flight plan by deleting the

STONS leg, then return to flight.

. In the waypoint menu, alter the flight plan by adding a waypoint at McPherson City.

13.

Place a second waypoint near the airport 47K.

14.

Place a third waypoint at Sedgwick.

15.

Set the unit up to fly and return to the moving map display.

16.

in the waypoint menu, alter the flight plan by adding a leg from the M66 waypoint to the
McPherson City waypoint.

17.

Place a second leg from the McPherson City waypoint to the waypoint near 47K

18.

Place a third leg from the waypoint near the airport 47K to the Sedgwick City waypoint.

19.

Set the unit up to fly and return to the moving map display.

20.

In the Flight Plan screen, alter the flight plan by removing only WPTO01.

21.

Set the unit up to fly and return to the moving map display.

22.

In the waypoint menu, delete the waypoint located at McPherson City.

23.

In the waypoint menu, switch to flight plan ten, set it up to fly, and return to the moving
map display.

. Switch back to flight plan one, set it up to fly, and return to the moving map display.

25.

In the Calculator menu, input the information necessary to calculate fuel consumption.

26.

Report the fuel consumption for the KICT to M66 leg.

27.

Alter the flight plan by adding the Newton City Airport, KEWK, to the end of the
itinerary and set it up to fly.

28.

Alter the flight plan in the waypoint menu to delete the Sedgwick City waypoint, set it up
to fly, and return to the moving map display.

29.

Access the airport information screen of the Database.

30.

Find the Wichita Mid-Continent Airport, located in Wichita, identified KICT.

31.

Report the field elevation, and the length and width of Runway O1L.

32.

Create a direct course to the third nearest airport.

33.

Place the cursor on Newton City Airport, identified KEWK, and create a direct course.

34.

Use the Flight Plan screen and clear out flight plan one.

35.

Use the Database to delete all the user defined waypoints.

36.

Clear the flight track.

37.

Return to the moving map display.



a light source placed directly in front of one of the
camera lenses. The light source, an infrared bulb, was
not visible to the subjects.

Questionnaires. The participants in this study
completed a pre-flight participant experience ques-
tionnaire and a post-flight ease-of-use questionnaire.
The pre-flight questionnaire consisted of basic ques-
tions pertaining to age, gender, flying experience,
GPS experience, and personal computer experience.

The post-flight questionnaire consisted of various
open-ended, free-response questions and horizon-
tally-oriented descriptive graphic rating scales with
anchors of “strongly disagree,” “neither disagree or
agree,” and “strongly agree.” Participants were in-
structed to place a single vertical line on several scales
to indicate the level ofagreement with each correspond-
ing statement. Each question pertained to the use of the
GPS unit.

Procedure

There were three main phases in this study. In
Phase 1, each subject completed a pre-test screening
questionnaire and was trained to use the GPS unit and
BGARS. The functions of the GPS controls were first
demonstrated by the experimenter as the subject ob-
served. Following the brief overview, a flow diagram
of the GPS menu structure was used to graphically
highlight the GPS features and organization of the
menu system. Following the overview of controls,
features, and menu structure, the experimenter dem-
onstrated menu and control functions while using the
GPS unit. The subject was then allowed to interact
with the GPS unit and “explore” the system. Each
subject was given a sheet containing various tasks to
accomplish while using the GPS unit. Subjects were
instructed to question the experimenter if confused
about any aspect pertaining to the use of the GPS unit.
Once the subjects felt comfortable using the GPS
unit, the experimenter tested their knowledge of the
unit by asking them to perform tasks similar to those
that would be in the forthcoming usability test. The
participants were tested until they could demonstrate
proficiency by accomplishing all tasks given to them
by the experimenter. The subjects were then familiar-
ized with the BGARS controls and displays and al-
lowed to fly a short course to become acquainted with
the flight characteristics of the simulator. This train-
ing and familiarization phase involved approximately
three hours of participation.

In Phase 2, the GPS-usability test took place.
Subjects accomplished routine flight tasks in the
BGARS and performed 37 GPS-related tasks requir-
ing waypoint setting, GPS navigation, and general
GPS data entry and GPS data retrieval (see Table 2).
Each GPS-related task was given verbally by the ex-
perimenter (located at the experimenter station) to the
subject in BGARS, through an aviation headset. A
second experimenter sat slightly behind and to the
side of the subject in BGARS to observe and take
notes. This testing phase involved approximately one
hour of flying and interacting with the GPS unit.

Phase 3 was the post-flight debriefing. Following
the flight, the participants completed a post-flight
questionnaire and were debriefed by the experimenter.
During the debriefing session, both experimenters
asked questions of the subjects concerning apparent
problems the subjects had experienced when interact-
ing with the GPS interface and responses on the post-
flight questionnaire. The debriefing was kept rather
informal so that the subjects felt comfortable offering
information and responding readily to open-ended
questions from the experimenters.

RESULTS

The collected data were in written and videotape
form to be used for subsequent analyses. This section
is organized so that it will be clear where subjects made
errors when interacting with the GPS interface and
why they made those errors.

A-Priori Human Factors Evaluation: Physical
Layout of the GPS Unit

An evaluation of the GPS hardware and software
configuration was carried out prior to the usability
study. The purpose of this evaluation was to deter-
mine if any of the characteristics of the GPS unit
would inhibit the system’s efficiency-of-use. If hard-
ware interface problems were discovered before sub-
jects were run, it would indicate that subsequent
usability test findings may be caused, in part, by these
problems.

The push buttons on the GPS unit were found to be
adequate. Diameter, displacement, and center-to-cen-
ter spacing were appropriate. The push-button labels
were located on the buttons and contrasted with the
equipment background. The push-button labeling
was backlit so that it was easily discernible in darkness.


Questions related to the usability of computer-
based menu structures have been examined for some
time now. However, these studies have, for the most
part, focused on personal computer (PC) use. Though
some GPS menu systems are similar to those seen on
desk-top or lap-top computers, there are factors that
make them unique. For instance, GPS display win-
dows are generally smaller than computer monitors.
Also, general aviation GPS users usually fly an aircraft
while interacting with a GPS unit. Therefore, data
were collected concerning the usability of a GPS unit
while in simulated flight.

METHOD

The main purpose of the current study was to
collect usability data from volunteer pilot participants
as they interacted with a GPS unit during a flight in an
aircraft simulator. Prior to data collection, a human
factors evaluation of the GPS unit was carried out
addressing control configuration, control labeling,
display labeling, and menu structure. The results of
this evaluation were examined, along with corre-
sponding subsequent usability data.

Subjects

Nine volunteers were recruited from the local general-
aviation pilot community. Each subject was fully in-
formed of pertinent details of the study. All subjects were
current private pilots. Subjects were screened through
the use ofa questionnaire to determine flight experience,
GPS-use experience, and personal-computer experience,
as well as other general questions relevant to the study.
Novice GPS users were employed as test subjects to
protect against negative transfer of training from previ-
ous use and familiarization with another GPS unit of
different design.

Apparatus

Basic General Aviation Research Simulator
(BGARS). The BGARS is a medium-fidelity, fixed-
base, computer-controlled flight simulator (see
Beringer, 1996). The controls and displays used in the
BGARS simulate those of a Beech Sundowner. Con-
trol inputs are provided by analog controls, including
a damped and self-centering yoke, navigation radio
frequency selection module, rudder pedals, throttle,
flap control, and trim control. Instruments are dis-
played on a cathode ray tube (CRT) and react in real

time to all control inputs and aircraft conditions. The
external views consist of a 50-degree forward-pro-
jected view and two smaller left-view CRTs.

GPS device. The GPS device evaluated in this study
was the Magellan EC-10X, an off-the-shelf unit with a
moving map display. This unit can be fastened to a
pilot’s upper leg with an elastic strap or mounted to the
yoke. For this study, upper-leg placement was used
because it represents the worst visual angle. The LCD
display dimensions are 6.0" vertically, by 4.5" horizon-
tally, and can be backlit with varying degrees of intensity.
The Magellan has 12 buttons, which access and control
the various features of the unit (see Figure 1). Some
buttons are multi-functional, allowing the user to per-
form different tasks using the same button depending on
which mode is operational. See Table 1 for a list of
button functions. The GPS unit was interfaced with and
driven by the BGARS software. The menu structure is
graphically depicted in a flow-chart format in Appendix A.

Video recording equipment. Two color video cam-
eras were positioned in and around BGARS to record
subjects’ hand and eye movements. The two video
signals were converted to a single composite video
image (split screen). The composite image was time
stamped and recorded on videotape for subsequent
examination. A high-resolution color monitor, lo-
cated at an experimenter’s station, was used for view-
ing the time-stamped composite video in real time
and for review of the videotapes. Two microphones
were used to pick up verbal instructions from the
experimenter’s station, as well as utterances from the
subject in the flight simulator. The audio signals from
both microphones were fed directly to the video
recorder. See Appendix B for the audio/video equip-
ment setup.

Event Timer/Recorder. A PC-based timing pro-
gram was developed to capture task completion time
as the usability test was being conducted. After the
experimenter read a task instruction to a subject, the
experimenter activated a computer timer with a key
press. Upon task completion by the subject, the ex-
perimenter manually stopped the timer with another
key press. Task start time, completion time, and
elapsed time were written to the computer hard drive.

Event Marker. An event marker was developed to
be used in conjunction with the event timer. With this
device the experimenter had the ability to visually
mark task-start times and task-completion times on
videotape. The event-marker signal was emitted from








Seven of nine subjects did not deactivate the flight
plan before attempting to delete a given leg from a
flight plan when using the waypoint menu of the
moving map display. Most subjects recovered from
the error, quickly deactivated the flight plan, and then
followed the instructions to delete the leg from the
flight plan. One subject initially did not deactivate the
flight plan, thus could not delete the leg. In an attempt
to determine a correct course of action, the subject
exited the menu containing flight plan manipulation
options and unknowingly moved the cursor in the
moving map display over a waypoint (bringing up
information about the waypoint-an intersection). At
that point, the subject wanted to return to the flight
plan manipulation menu, but was not able to access it
until the cursor was moved off of the intersection on
the moving map display. There was no indication
given to the subject that placing the cursor over a
waypoint in the moving map display would “lock out”
the menu containing flight plan manipulation op-
tions. The subject’s actions (not deactivating flight
plan and moving the cursor over an intersection) were
cycled through twice, creating much confusion as to
how to accomplish the task of deleting a leg from an
active flight plan.

Deleting a waypoint from an active flight plan is very
similar to deleting a leg from an active flight plan.
However, the subject must first deactivate the flight plan
and then delete any associated legs of the flight plan
before deleting the waypoint. Three subjects attempted
to delete a given waypoint without first deleting the
corresponding leg from the flight plan. One subject
attempted this twice, even though the first attempt did
not succeed. One subject deleted the waypoint success-
fully but then became “lost” in another mode of the
moving map display after pressing the ENT burton (this
action brought the user to the “cursor mode” of the
moving map). Normally, the subject would press ENT
again to return to the “position mode” of the moving
map. However, the cursor was positioned over a way-
point, thus the subject could not return to the appropri-
ate mode as expected (similar to the subject being locked
out of the flight plan manipulation menu, described
above). There was no indication that placing the cursor
over a waypoint in the moving map display would “lock
out” access to the other map mode. Another subject
accidentally deleted an incorrect leg from the flight plan.
At that point, the subject realized the error but could not

determine how to “undo” the action (there is no “undo”
option). The subject attempted to re-insert the leg but
did not do so successfully.

Error Messages (Feedback)

Textual error messages appeared only in the “calcu-
lator menu” of the graphic interface of the evaluated
GPS unit. A message simply appears in a box stating
that the user must first activate the flight plan before
using the calculator functions. This message is quite
helpful, because it stops any attempt to use the calcu-
lator functions with an inappropriate setup.

A short, single beep is given as auditory feedback
when a button is pressed to let users know that their
action was registered. Auditory error messages are in the
form of two or three short beeps in quick succession.
These two series of beeps indicate that an erroneous
button press has occurred, but they do not convey
information concerning the nature of the error. In the
post-flight questionnaire, the statement, “If I had done
something incorrectly I was given enough information
to let me know what I had done wrong” was rated by
subjects, resulting in an average sore of 15.57 ona scale
of 0 to 100 (zero being “Strongly Disagree”).

A series of two beeps occurred when subjects at-
tempted to place user-defined waypoints over pre-
existing waypoints or landmarks (i.e., airports,
intersections, etc.). A series of three beeps occurred
more frequently than the double beeps and were
usually associated with the most troublesome interac-
tions with the GPS interface. For example, triple
beeps were heard during the “double errors” discussed
above. More specifically, if subjects pressed the OUT
button (the correct button for this task) to remove a
waypoint, and the flight plan was active, three beeps
would be heard. If the subject pressed the ENT button
(the incorrect button for this task) to remove a
waypoint, three beeps would be heard. This leaves the
subject wondering which action was incorrect since
both seem incorrect. Both are incorrect at the time
because the flight plan has not been deactivated.
However, once the flight plan has been deactivated,
the subject may be unsure if the ENT button or the
OUT button should be pressed. Unfortunately, the
ENT button will reactivate the flight plan, and the
cycle will start over again. Several subjects, on more
than one occasion, became trapped in this cycle of
three-beep auditory error “messages.”


Average Glance-Down Time (Seconds)

22

20

18

16

14].

12

10

5 6 7 8 9 1011 12 13 14 15 16 17 18 19 20 21 22 23 24 25 27 28 29 30 31 32 33
Task Number

Figure 3. Average Head-Down Glance Time for Tasks in Flight.

11



Technical Report Documentation Page

2. Govemment Accession No.

T. Report No.
DOT/FAA/AM-98/9

3, Recipients Catalog No.

4, Title and Subtitle

Cockpit Integration of GPS: Initial Assessment—Menu Formats
and Procedures

5. Report Date
March 1998

6. Performing Organization Code

7. Author(s)
Wreggit, S.S., and Marsh, D.K., II

8. Performing Organization Report No.

9, Performing Organization Name and Address
FAA Civil Aeromedical Institute

P.O. Box 25082
Oklahoma City, OK 73125

10. Work Unit No. (TRAIS)

11. Contract or Grant No.

72, Sponsoring Agency name and Address
Office of Aviation Medicine
Federal Aviation Administration
800 Independence Avenue, S. W.
Washington, DC 20591

73. Type of Report and Period Covered

4. Sponsoring Agency Code

15. Supplemental Notes

16. Abstract

A popular portable Global Positioning System (GPS) unit (Megellan EC-10X), representative of this class of devices, was
examined for its usability by general aviation pilots, Nine private pilots participated in the experimentation, which was
accomplished in three phases: familiarization and training, usability testing, and post-experiment debriefing. During
familiarization and training, participants were asked to study flow diagrams representing GPS interface logic, observed a
demonstration of the unit's features and procedures, and then were allowed to practice with the unit until they could
demonstrate proficiency. During the usability testing phase, participants performed 37 GPS-related tasks requiring
waypoint setting, GPS navigation, and general GPS-data entry and retrieval. Findings from videotape, questionnaire,
and debriefing data indicated that a number of menu structures interfered with the pilots’ successful entry of data,
editing of stored data, and activation of functions. For example, one source of confusion resulting in excess button
presses was the need to deactivate the flight plan before any editing could be done. Recommendations are made for
defining the form of the interface structure in this class of devices, including: A given function should be consistently
assigned to one button, feedback should be consistent and meaningful, and an “undo” or “back” function would be a
very useful way to decrease the number of button presses required by this interface.

17, Key Words 18. Distibution Statement

Document is available to the public through
the National Technical Information Service,
Springfield, VA 22161

Aviation Research, Cockpit Displays, Control-Display
Units, Flight Simulation, Global Positioning System,
Human-Machine Interface, Menus, Navigation, User-
Computer Interface

19. Security Classi. (ofthis report)
UNCLASSIFIED

20. Security Classit. (of this page)
UNCLASSIFIED

21. No. of Pages
23

22. Price

Form DOT F 1700.7 (8-72)

Reproduction of completed page authorized



dV

aN

Aejdstp deut uo
woneoo] parisep

<_—_
Je ST Josind way
.LN@ sassasd Jos)
uondo
<—_______ bow oyeanoe 01
.LN@ sassaid Jos)
(da 28ueyD) OT di
XX dd aa7aa oalddv aiataa dam adv | | govasurv

t

i

i

f

f

A4


NOTICE

This document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of
information exchange. The United States Government
assumes no liability for the contents or use thereof.


APPENDIX B

Video/Audio Equipment Flow Diagram

Video Camera #1 Video Camera #2 Microphone

Video Effects
Machine

Video Track (Composite Picture)

Audio Track

Time Encoder Box

Video with Time Stamp

S-VHS VCR

Monitor

Bl



Control labeling did not appear to be intuitive in all
cases. For example, pressing the button labeled MNU
did not take the user specifically to a menu but
generally to a higher level of the menu structure or
“out” ofacertain mode. See Huntley, et al. (1995) for
acomprehensive human factors and operations check-
list for standalone GPS receivers.

Button Presses to Accomplish GPS Tasks

One clear indication of the work required to ac-
complish a given task when using an interface is the
number of keystrokes/button presses necessary for
task completion. In Figure 2 a graph with the mini-
mum number of button presses needed for the comple-
tion of each of 36 GPS-related tasks, and the average
excess button presses executed by subjects to accom-
plish a given GPS-related task is shown (task 26 was
excluded because no button presses were needed for
this task). An important finding is that the total
number of button presses per task is significantly
correlated with the average head-down time per
task (r = 0.8163; p<.01).

Excess button presses are often a function of the
method by which subjects chose to enter alphanu-
meric information. For example, when a subject se-
lected an alphanumeric string to be altered, the cursor
would default on the right-most character. Some
subjects chose to move the cursor to the left-most
character before entering the necessary information.
This action is thought to be a natural tendency that
results from reading left to right. Subjects were also
observed moving the cursor through more menu se-
lections than necessary to reach their desired selec-
tion. For example, when the cursor was located at the
top of a list in a menu, some subjects were observed
moving the cursor from top to bottom through the
list. However, the cursor could have been moved “up”
from the top of the menu, and the cursor would have
appeared at the bottom of the menu.

Excess button presses were caused, in large part, by
subject confusion concerning the function of the
OUT button and the ENT button and also because a
flight plan had to be deactivated before it could be
altered (Tasks 7 and 20). A flight plan was usually
deactivated by subjects prior to any attempt to alter it.
However, following the deactivation of the flight
plan, some subjects would unknowingly reactivate the
flight plan by erroneously pressing the ENT button
(instead of the OUT button to “take out” a waypoint).
Because the reactivation of the flight plan was not an

obvious result of pressing the ENT button, most
subjects did not notice that the flight plan had been
reactivated. Therefore, subjects simply assumed that
pressing the ENT button was an incorrect action and
pressed the OUT button (the correct action initially).
However, previously pressing the ENT button had
reactivated the flight plan, thus no result occurred
when the OUT button was pressed. Some subjects
then pressed the ENT button again. The experiment-
ers referred to this type of erroneous action as a
“double error” because subjects were pressing an in-
correct button while the flight plan was still activated.

Some subjects initially pressed the OUT button
but did not deactivate the flight plan; therefore, no
result occurred from pressing the OUT button. Sev-
eral subjects then deactivated the flight plan and
completed the task. However, other subjects assumed
that pressing the OUT button was an incorrect action
and pressed the ENT button (double error). Pressing
the ENT button did not accomplish the task either,
therefore the subjects were confused as to which
button would remove a waypoint. These subjects
eventually deactivated the flight plan, but occasion-
ally pressed the ENT button again, thereby reactivat-
ing it. Most subjects did not notice the result of their
action because the reactivation of the flight plan was
not an obvious result of pressing the ENT button. As
can be seen, the double-error event caused confusion
and slowed the performance ofa relatively simple task.

Excess button presses were observed when two
subjects accidentally “locked in” an incorrect waypoint
by prematurely pressing the ENT button prior to
completely entering the desired waypoint ID. Thus,
an unwanted waypoint with a similar ID to the desired
waypoint was locked in. There is no “undo” function
or editing function available. Therefore, ifa waypoint
is locked in accidentally, the user must delete the
unwanted waypoint and re-enter, from the beginning,
the desired waypoint.

Excess button presses were seen in the case of one
subject when confusion arose concerning the words
“activate” and “deactivate” with regard to the flight plan
(i.e., flight plan must always be deactivated before it can
be altered) (Task 15). When a flight plan is active the
word “deactivate” appears as a menu option, indicating
that the flight plan is active and can be deactivated by
selecting the menu option. It appeared that the subject
assumed that the words indicated current status (flight
plan deactivated or activated) instead of an action.


Level of Consistency

Consistency within the GPS menu structure and
function allocation were another focus of this study.
Observed inconsistencies either hampered or pre-
vented the completion of some intended tasks. In
Table 1, take special note of the bold, bold-under-
lined, and bold-italic lines, which emphasize some of
the inconsistencies in function allocation.

Different buttons — same function. While in the
database screen and the moving map display, a user is
able to retrieve site information (i.e., airport, way-
point, etc.). This information is typically too lengthy
for all of it to fit on a single screen at one time,
therefore a page metaphor is employed. In other
words, a user is able to “flip” through various informa-
tion pages, similar to those of a spiral notebook. To
flip to the next or previous page in the database, the up
and down arrow buttons must be pressed. However,
in the moving map display, pressing the up and down
arrow buttons will move the map cursor, and the user
will lose the displayed site information. The user must
then place the cursor back onto the prior location
before the information will again be displayed. To flip
through the information pages in the moving map
display, the user must repeatedly press the ENT but-
ton while the cursor is located over the desired site on
the displayed map. Pressing the ENT button in the
database will “lock in” the presently displayed site so
that information can be viewed. Therefore, using a
button sequence that is appropriate on one screen may
result in the loss of information on another.

While interacting with the flight plan screen, users
can switch from one flight plan to another by pressing
the left and right arrow buttons to proceed to a lower
or higher flight plan number. However, in the way-
point menu of the moving map display, the same task
is accomplished by repeatedly pressing the ENT but-
ton. Therefore, the user can only ascend to higher-
numbered flight plans. This cycling through the flight
plans increases the overall number of button presses
required to reach the desired flight plan and the time
to complete the task. If the desired flight plan number
was inadvertently missed, as was observed in the
study, it was necessary for the subject to toggle through
all of the flight plan numbers again.

Other inconsistencies were observed when com-
paring the task of deleting user waypoints in the
database screen with the same task as performed in the
flight plan screen. In the latter, the user presses the
OUT button to remove a waypoint and the ENT

button to confirm the action. In the database display,
the user removes a user-defined waypoint by pressing
and holding the ENT button for five seconds. This
problem is aggravated because the ENT button is
typically associated with selecting or entering an op-
tion. Also, in the database screen, where the ENT
button is used to delete a waypoint, there is no “help”
information at the bottom of the display.

Feedback. Inconsistent feedback from the unit is
heard when a user attempts to change a flight plan
while it is still active. While in the flight plan screen,
a user will receive a double beep if an attempt to
change flight plans is made, but while in the waypoint
menu of the moving map display, a triple beep will
result. Textual feedback was located on only one
screen within the software interface.

On-screen help. Another inconsistency exists in
the terminology used within the “help” information
at the bottom of most screens. This help information
serves as a reminder to the user concerning the func-
tion of various buttons fora specific screen. At roughly
half the help-information locations, the message “Press
MNU to Exit” would be displayed, while the message
“Press MNU to Escape” would be displayed at other
areas of the interface. Both “escape” and “exit” re-
ferred to the same action: to backtrack to a higher-
level menu location. Inconsistencies with terminology
may lead to confusion and should be avoided. To
further the inconsistency problem with regard to on-
screen help, not all screens contain help information.

Head-Down Time Resulting From Interaction
With the GPS Interface

Head-down time is associated with the number of
button presses made, error messages received by the
user, and inconsistencies experienced when using the
GPS unit. The average head-down glance time is de-
fined as the amount of time that subjects glanced
down at the GPS unit without looking at the flight
instruments or environment. In Figure 3, the average
head-down glance time across subjects was commonly
10 seconds or greater. In other words, the subjects, on
average, spent 10 seconds or more without glancing at
the aircraft instruments or the outside environment.
Performance of tasks that elicited the greatest average
head-down glance times, for the most part, involved
altering flight plans while in flight. Cumulative head-
down time is defined as the sum of the mean head-
down glance times per task, averaged across nine
subjects (Figure 4). These measures are associated