Developing the Personal Minimums Tool

To test the preliminary intervention, seven indi-
viduals from our research group participated in a
ninety minute program conducted by a pilot/flight
instructor facilitator using the preliminary instructor's
guide. Six of the seven were pilots. Among them were
aisline transport, instructor, commercial, private, and
student pilot certificate holders, One rated pilot was
no longer actively flying. The student pilot had so-
loed. Pilot ages ranged from approximately twenty to
fifty. All participants were college graduates.

The results of this test were useful to the further
development of the program confirming several ofour
ideas concerning the approach taken. Pilots responded
favorably to the concept of personal minimums. They
pasticipated actively in the discussions, completed
each exercise, and offered numerous comments dur-
ing the evaluation following the training seminar.
They recognized the benefits of establishing personal
minimums as illustrated by statements such as, “stan-
dard operating procedures are not enough by them-
selves”, “knowing your own limits is important”, and
“admitting that one has limits takes away some anxi-
ety of not knowing where those limits end.” A com-
ment was also made that, like the FARs, personal
minimums are intended to “protect us rather than
festrict us.”

The pilots participating were able to identify fac-
tors influencing preflight decisions. When asked for
examples, the pilots were able to provide them orally
in the discussions and written in the exercises. How-
ever, the number of factors they could generate in the
time provided (approximately 5-10 minutes) was lim-
ited. More experienced pilots tended to have more
factors than did those wich less experience. Group
listing and discussion of risk factors tended to gener-
ate interest and additional factors.

Pilots wete able to write guidelines that pertained
to their routine flight activities. However, the list
generated by each pilot consisted of three to five
guidelines. During the portion of this activity when
pilots wrote sample guidelines, the list of factors
provided as samples was used extensively. This would
indicate that pilots need to be prompted as to which
factors are involved in their activities. The list was
helpful in expanding the number of guidelines they
might be able to produce under time pressure. The list

14

of risk factors could be used as prompts for pilots in
training seminars to generate their own guidelines.
These individually produced guidelines generated both
additional examples, as well as modified existing ones
during the group discussion.

The evaluation of the serninar consisted of a group
discussion of the pros and cons of this approach.
Generally, che comments were favorable and expressed
enthusiasm for an approach to generate personal mini-
mums, The ficxibility of the format to generate a
useable set of personal minimums was favored because
the opportunity co add or delete items gave the indi-
vidual pilot “control.” General encouragement was
offered to try the intervention on a wider group of
participants. The case history used was well received
because it gave a related problem to solve and gener-
ated ideas for risk factors. The pilot in the case history
had a difficult go/no-go decision to make. The pilots
studying the case history wanted to know “what they
could do to help” the troubled pilot in the case history
to more casily make his preflight go/no-go decision.

The evaluation of a sample set of getsonal mini-
mums illustrated the potential problems. A sample
guideline for thunderstorm clearance contained a
non-standard number for the distance from thunder-
storms that the pilot would maintain. Questions were
asked, where did you get that number and how do you
know ifit is right. Since the sample was only adummy
one to generate discussion, no one’s feelings were dam-
aged and the discussion could continue. If the guideline
had been identifiable with someone in the seminar, such
questions could have made the individual defensive or
ended discussion in general so as to avoid embarrass-
ment. A method to generate discussion is important to
increasing the learning potential, but embarrassment in
front a peer should be avoided or minimized.

During the initial test, the piloc and the aircraft
categories tended to follow 2 conventional line of
thoughr. The pilot category attracted guidelines that
addressed the experience, capabilities, physiological
health, psychological health, and so forth of che pilot.
Similarly, the aircraft category contained conventional
preflight icems such as fuel load, performance character-
istics, number of engines, maintenance conditions, equip-
ment, and so forth. The scmaining three categories
tended to consist of less defined items, and the same


Developing rhe Personal Minimums Tool

Table 3. Example of REARM risk value matrices (Shively, 1988),

Weather Risk Value Matrix
Ceiling/Visibitity
Wind Velocity
(Knots)
Clear YFR (>1000-3) Minimums
30 3 4 §
>20 2 3 4
>10 4 2 3
<10 3 4 5

Table 4. SAFE crew factors as rank ordered by expert panel (Shively, 1930).

Crew Facior Variables
(As rank ordered by subject matter experts from mosi to ieast risky)

4) EMS experience of pilot
2) IFR currancy of pilot

3) Crew rest

4) Pilot's familiarity with area

a) en route
b) at site

8) Pilot's familiarity with aircraft
c) Number of pilots/aircraft

7) Hours since last meal

8) Commercial rating



Developing the Personal Minimums Too!

This definition of preflight decision making in-
cludes many factors (perhaps considered months ahead
of the flight) often overlooked in a pilor’s preparation
for flight, including, for example:

¢ theinitial motivation to fly the family to the beach for
vacation;

a decision to forego maintenance items on the air-
craft;

*a decision to take the BFR or instrument compe-
tency;

¢a decision to install navigation, communication,
anti-icing, or weather detection equipment.

This definition also includes short-term factors
such as:

eassuring the fuel reserve is sufficient;

* concluding that you have to get the Soss/family there
on time;

* recognizing I might disappoint them by not going
now.

It should be noted that this definition excludes
some decisions that are usually considered go/no-go
decisions (e.g., the rejected takeoff decision).

Importance of Preflight Decisions

The importance of decisions made prior to flight is
underscored by studies that have shown that a signifi-
cant number of pilot decisions (or lack of decisions)
made during the preflight phase led directly to acci-
dents and incidents much later in flight (McEthatton
& Drew, 1993). Of the 125 Aviation Safety Reporting
System (ASRS) incident reports reviewed in their
study, ninety percent of all time-relaced human errors
occurred in the preflight or taxi-out phase of opera-

tion (See Figure 2). General aviation examples of such
errors might include:

* Knowingly overloading the airplane and failing to
clear an obstacle at the end of the runway on takeoff.

«Running out of fuel enroute because of poor flight
planning (e.g., insufficient fuel reserve).

* Deciding that the high ranking passenger (boss or
family member) must get there despite the weather,
resulting in premature contact with the ground on
approach.

¢ Flying with a known medical condition then becom-
ing disoriented (e.g., in haze).

On a fundamental level, a flight is a sequence of
events that are related and influenced by the previous
events. So it should be no surprise that preflight

Figure 2. Operational phase vs. error and incident occurrence (after McElhatton and Drew, 1993).

18


TABLE OF CONTENTS

OVERVIEW AND INTRODUCTION ---scecessssesnsssssssecceeeseen eesecsencnenseeneenee veces}

BACKGROUND ---

Literature --

Intervention Development -------------n-re---ravennencceerenenenerenneenenenenarneesnteenentananennanecentnn il
THE PERSONAL MINIMUMS TRAINING PRODUCT DESCRIPTION -----------e-eee--o-n-nenererenere 17

Stage 1: Introducing Personal Minimums --------s--------secsncnateennrsnenetencnasensnrannenntnsnnnscnsnne 17

Stage 2: Identifying Risk Factors ------------------000- oneeeecaecenecnerenenenceccenctencnenenneenecennnnne 20

Stage 3: Composing Guidelines ---------------------00--0ce sre-cereneeenerennenrenenansnucsnanarstsnnscnecnnee 21

Stage 4: Evaluating Personal Minimums ------------------------—-n-n-nnnennnnnrnenrneeesceceeeenceecteeceenen 26

Concluding Summary ---- aonenecnanesenens:

sovevsncanecneaeneenneenen 26

‘ FUTURE DEVELOPMENT

Field Testing and Refinement -----------------------r--------n-e ree nnen nner eennacnncneeneeateerenmnncnnenne 27

Experiments ---

Expected Results -------------~---------n+nn00nnennarernecenenncnnnennensecnnnnnnnnnneeceenenasnnenecarerrecerenees 2B

Further Research Questions ---

panenenceencenecenenenereeentenccesarennenenenenencee 29)

REFERENCES ---------------eeenvnen-nvornaeencnnrneeney

Appendix A: Preliminary Instructor's Guide -------------------r---e-nerentnensteeancccerenneescerenenenes An]



Safety Program Managers, Safety Counselors, and
flight instructors to obtain feedback from them on the
concepts, methods, materials, and evaluation tech-
niques that work for them individually and their pilot
learners in actual field settings. Such insights will
contribute to the successful implementation of the
personal minimums program.

Experimental Results

We expect that both personal minimums and im-
posed minimums will achieve positive results because
both offer an effective refresher on risk factor aware-
ness. However, we believe that the personal mini-
mums approach will be significantly more effective in
changing behavior, especially in the long term. SOPs
are effective, but often they are opposed by other
organizational or group policies (such as, always ac-
complishing the mission) or in economic terms, Pilots
may be inclined co balance one organizational or
group policy against another. Furthermore, some
people are not highly motivated enough to follow the
restrictions set down by others. Personal minimums,
on the other hand, have a persona] commitment
attached. These commitments atc powerful psycho-
logical fosces that have been proven effective in decid-
ing human behavior in many settings. We believe that
they will be effective in aviation as well.

Experiments in determining the capabilities of pi-
lots to assign relative risk values are expected to show
that some pilots have significantly better capabilities
than other pilors. However, the number of factors that a
pilot can integrate into such an evaluation is expected to
be small and on the order of seven factors, plus or
minus two. If such limits exist, chen the risk manage-
ment approach also must include a risk assessment
aspect to assist the pilot in identifying the key factors
involved in an intended flight, An additional result of
these experiments is expected to be the identification
of factors pilors identify as key in selected missions.
Such missions may include VER flight in marginal
VMC (visual meteorological conditions), flight in
known icing conditions, flight in areas of known
windshear, flight in mountainous terrain, extended
Aight over open water, or others.

29

Developing the Personal Minimums Tool

FURTHER RESEARCH QUESTTONS

Severai research questions have been identified to
guide the development of risk management approaches
in preflight go/no-go decisions. Answers to these
questions are expected to lead to a better understand-
ing and to refinements in such approaches:

1) What types (self-generated or imposed) of preflight
minimums are pilots more likely to follow when
making actual preflight decisions?

2) Are techniques that rate risk levels (i.e., SAFE, RE-
ARM, ALERT, etc.) effective for positively influenc-
ing a pilot’s go/no-go decisions?

3) How effective is increasing the pilot's knowledge of
risk in a genera] aviation operation in causing pilots
to act on that knowledge?

4) What risk levels are acceptable to pilots in various
situations and how do pilots decide the acceptability
of chat risk?

5) How far can we push pilots into risk assessment and
management? Can they be taught to make assess-
ments of relative risk levels?

REFERENCES

Adams, R. J. & Payne, B. (1992). Administrative risk
management for helicopter operations. The Inter
national Journal of Aviation Psychology, 2 (1), 39-
52.

Alkov, R.A. (1991), U.S. Navy aircrew coordination
taining: A progress report. In R.S. Jensen, Pro-
ceedings of the Sixth Internarional Symposium on
Aviation Psychology, Columbus, OH: OSU De-
pastment of Aviation.

Beach, L. R. & Lipshiez, R. (1993). Why classical
decision theory is an inappropriate standard for
evaluating and aiding most human decision mak-
ing. In G. A. Kfein, J. Orasanu, R. Calderwood,
and C, E. Zsambok (Eds.), Decision making in
action: Models and methods (pp. 21-35). Norwood,
NJ: Ablex Publishing Corporation.

Boley, K. O. (1985, September), ALERT: Aviation
litmus evaluation risk test. U. $. Army Digest, p.
17-19.


guidelines we: . often included in various categories
by different pilots. The environmental category usu-
ally contained the meteorological guidelines. Addi-
tional items included airport lighting, navigational
aids, terrain featurcs, and others. Operational/mis-
sion categories tended to include not only the types of
flight operations such as nighe, over-water, personal,
fun flight, training, but other items including towing
gliders and banners, aerobatics, aerial applications,
emergency medical flights, business meetings, family
trip, high altitude flight, and so forth. The organiza-
tional/sociological category tended to include items
that characterized the pressures that are placed on the
pilot-in-command and alternate ways to manage such
pressures. The examples included the need to get toa
meeting or home, how to handle a higher ranking or
executive passenger, pilot briefing statements con-
cerning alternate plans, and so forth.

General comments suggested thar an initial intro-
duction of the concept of personal limits was neces-
sary, but it should be shorter, faster moving, and
harder hitting. Pilots suggested the need for a short
video of slide presentation as opposed to an oral
presentation. This comment was also emphasized
during program management review of the approach
at the FAA. Participants also wanted to have an
opportunity to prioritize the risks so that the most
threatening could be addressed. Further consider-
ation should be given to this question. ‘The interac-
tion of two or more factors also needs to be considered.
Interaction effects will be more complicated to address.

Intervention Design Concepts

The design concepts for the preliminary training
seminar developed for using personal minimums as a
risk management tool during preflight decisions are

based on four major concept features. These features
include:

1) _ aiding pilots in the identification of risk factors,

2) building commitment to support use of personal
minimums,

3) providing credible examples to a wide range of
pilots, and

4) providing an easy-to-use format for generating

personal minimums.

Developing the Personal Minimums Toot

Identifying risk factors. Risk factors form the basis
for personal minimum development as an approach to
managing tisk associated with flight. Three reasons
support this approach. First, the approach allows
pilots to identify and highlight major risk factors that
many pilots recognize and might classify as “high risk”
or “killer” factors. These include thunderstorms, ic-
ing, lack of currency, poor preflight planning, and so
forth. The intervention uses these factors as a way for
pilots to describe actions they deen: useful and are
willing to follow to prevent or reduce the effects these
factors could potentially have on their flight. From
these sisk factors flow guidelines for personal mini-
mums. For example, pilots may state (as their personal
minimum) that they will do a complete preflight prior
to each cross-country flight. They may define what
their complete planning entails as well.

Second, the training intervention offers group dis-
cussions and examples that widen the range of factors
that an individual pilot might consider for her or his
own personal minimums. These discussions provide
exposuse to new factors as well as expand the pilot’s
knowledge base. For example, 2 pilot may not file «
flight plan because no immediate safety benefit is
recognized. However, during the group discussion,
benefits may be recognized that go beyond providing
a place for rescuers to start the search for a downed
aircraft. Hence, filing che flight plan may take on
more practical meaning and importance to the pilot
{e.g., flight planning). As 2 result, che pilot may do
both the flight planning and filing of a flight plan in
the future as her or his minimums.

Third, the identification of key factors allows the
pilot to customize or individualize the risk factors
inco guidelines developed for the types of flying that
are doneona regular basis. By excluding items that are
irrelevant to the pilot, time and effort are not ex-
pended on unimportant items and the time can be
focused on those of optimum importance. If a large
general list of personal minimums is generated, many
pilots would be expected either to nor use the list or to
modify it by eliminating those items that do not
pertain to them.

Building Commitment. Building commitment is
another major feature of the preliminary intervention.
It involves the development of an individual pilot's



Students are asked to relate examples from their
own experiences in which they decided not to make a
flight for safery reasons, and examples of situations in
which they decided to go but later regretted the
decision or had co abort the flight, The instructor
draws parallels between the examples offered and the
three points listed above. In the rare case in which the
pilots cannot provide examples, the instructor should
use her or his own experiences. Instructors must be
careful not to dominate the seminar with their own
“war stories.” Instead, they should make the examples
real for the participants using their experiences.

A Case Study

To stimulate thinking about risk factors and how
they apply to preflight decisions the instructor can
offer one or more case studies in the fosm of “trigger
tapes” of situations developed in the FAA’s “Back to
Basics” program or stories found elsewhere. One ex-
cellent example was published in Flying Magazine’s, “I
Learned Abour Flying From That” (McCutcheon,
1991). In this case, a pilot feels pressure to make a
medical evacuation flight in a Cessna 210 with inop-
erative radios, in questionable weather, with night
approaching. After arriving for the pick up, he finds
thar there is both a child and a mental patient to
transport, plus families and other cargo, causing the
plane to be overloaded. Despite the chiid’s condition,
because of the numerous risk factors, he decides not to
takeoff. As students read th’s story, they are asked to
identify the risk factors facing the pilot including the
subtle psychological factors such as the condition of
the child. When all have listed these factors individu-
ally, the class is opened for discussion, and all are
invited to share their discoveries. The same could be
done with any aviation preflight scenario, including
those from individuals in the class, presented cither in
front of the whole class or in small groups.

Case histories make identifying risk factors seem
meaningful because they are placed in the context of
real-world situations. The use of specific case histories
helps to add flexibility to the program by allowing
various audiences to be addressed with the same basic
program. For example, if the students are balloonists,
balloon case histories mean much more to them than
airplane examples.

2

Developing the Personal Minimums Toot

Students are asked to divide into small groups and
gencrate a more comprehensive list of preflight risk
factors. Aftcr generating the list, the students arc
asked to organize them into a taxonomy with six
suggested categories: pilot, aircraft, envitonment,
operation/ mission, organization/social, and miscella-
neous. A format for the students to use {Table 5) is
provided as a handout, The first three categories are
those normally used to represent the pilot's world.
The operation/mission category is added as a place to
put factors regarding the operarional aspects of that
particular flight. The organization/social category is
included as a place to put risk factors that organiza-
tions can add to the pilor’s decision process including
subtle pressures to complete flights on schedule. Fi-
nally, the miscellaneous category is included ro un-
derscore the emphasis on personal freedom in the
construction of this tool in anticipation that some
may not wish to identify a particular risk factor with
one of the given categories, The structure presented is
offered as a starting point, not a required set of
categories, What is placed under any category will be
a function of each pilot’s mental model of how these
factors are classified.

By grouping risk factors into these categories they
are more easily remembered and hence applied to the
decision process than they would be if unorganized.
This categorizing process also helps to establish deci-
sion rules or guidelines for future use. After compler-
ing these tasks, the students will have the skills necessary
to identify risk factors that pertain to preflight deci-
sions and to place them into categories that have the
most self-meaning. Students will also become aware
of the general methods thar other pilots use to make
decisions and the risk factors involved for them.

STAGE 3: COMPOSING
GUIDELINES

The third stage is the most important because
students develop their own personal minimum guide-
lines into an easy-to-use checklist format. The three
activities of this stage are:

1) defining guidelines,
2) generating examples, and
3) generating guidelines on < checklist form.


A long list of guidelines could be difficult to sort
cheough before preflight planning ot duting use. To
provide an organizing principle, a set of six categories
was suggested, but not dictated for organizing the
guidelines. However, pilots are encouraged to define
and use these categories in ways that have meaning to
them, as opposed to rigid definitions. The six sug-
gested categories of guidelines are, pilot, aircraft,
environmenr, operation’ mission, of ganizational/so-
ciological, and miscellaneous.

THE PERSONAL MINIMUMS
TRAINING PRODUCT

The primary product of this initial research pro-
gram is apreliminary ceaining incecvention that asstecs
pilots in constructing a personal minimums checklist
tool for managing risks in their preflight go/no-go
decisions, An instructor's guide, entitled “Using Per-
sonal Minimums as a Risk Management Tool for
Preflight Decisions” is provided in Appendix A. The
guide is designed to be used by the instructor in
preparation fot conducting the seminar. The seminar
is designed, not only to assist in risk assessment, but
students are encouraged to learn from each other
through sharing personal flying experiences. Instruc-
tors/facilirators are expected to use their own experi-
ences to stimulate and motivate pilots to set and
follow personal minimums.

The seminar is divided into four stages. In che firse
stage students are given an introduction to personal
minimums presenting the basic concepts and showing
the need for pilots to use them. The second stage
provides a knowledge base of preflight risk factors for
GA pilots. In the third stage, students actually com-
pose personal minimum guidelines with assistance
from the facilitator. In the fourch stage, students
evaluate their product through review and sharing of
ideas with each other. The following is a detailed
description of the instructor guide stage by stage.

17

Developing the Personal Miximums Tool

STAGE 1: INTRODUCING
PERSONAL MINIMUMS

The introductory section highlights the reasons for
using personal minimums as a too! for managing the
risks associated with preflight go/no-go decisions.
The instructor must use this time to provide back-
ground concepts, instill motivation for the pilots to
learn, and show what level of participation will be
expected of them in the seminar. Finally, the instruc-
ror should point out chat some behavior change is
expected following the seminar regarding their use of
petsonal minimums in preflight decision making.
This introductory stage focuses on five basic questions:

1} What are preflight decisions?

2) Why are these preflight decisions important?

3) What are personal minimums?

4) Why are personal minimums impostant?

5) Why should an individual pilot establish personal
minimums?

Defining Preflight Decisions

When asked about preflight decisions, pilots often
jimi their answers to inspection of the aircraft, weather
evaluation, and flight navigation planning. While
preflight decisions include all of these activities, other
factors such as experience and social pressures may
contribute to possible errors in judgmenc. For our
purposes, preflight decisions are defined as: “Any and
all decisions made prios to taxiing the airplane onto a
runway with the intent to takeoff.” In aircraft other
than airplanes, preflight decisions are any and all
decisions made prior to the first task to make the
aircraft airborne (i.e., gliders—giving the initiate/
launch signal; balloons—giving the command to
launch; helicopters—advancing the throttle). Under
this broad definition, preflight starts with che first
idea that a pilot generates about a possible flight until
the pilot iniviates the first action to make the aircraft
airborne or until thar possibility no longer exists.


Developing the Personal Minimsame Tool

concern was that the inrerview was a trick to get them
to admit to violating FARs and that enforcement
action would follow. The interviewer's response was
that the research was in support of a FAA funded
project. They were told that the interviewer did not
represent the FAA and that any information collected
could not be traced to the pilot. That trust was
maintained and no data were recorded that could be
linked to the pifor.

Questions concerning the pilox’s preflight actions
produced frequent mention of the aircraft preflight
and the weather. Preflight actions also included fuel
and oil checks, draining the sumps, and checking
instruments and avionics. Some pilots reported that
working with the weather information caused prob-
lems. One pilot complained of “too much informa-
tion” and that the DUAT (Direct User Access
Terminal) system was “confusing”. He preferred to
talk with someone. Another stated chat “FSS (Flight
Service Station) doesn’t want us to fly, there is always
bad news from flight service.” Most of the pilots
interviewed used both che FSS and the DUAT system
as a routine part of all their cross-country flights.
However, one pilot did state that he often would “go
up to see how it is, then come back” ifhe did not like
the weather.

When asked about the guidelines or limits that they
placed on themselves, some provided good suggcs-
tions while others were reluctant to make such state-
ments of to put risk values on situations. Marginal
visual flight rule (MVFR) conditions (ceiling 1000-
3000 AGL, visibility 3-5 stature sniles) rended co be
the point at which pilots started to draw the line on
visuai flight rules (VFR) cross-councry flights. Par-
ticular concern was noted as the ceilings approached
1000 feet above ground level (AGL). One instrument
rated pilot did not fly in what he termed hard IMC
(instrument meteorological conditions) and described
as ceilings below 1000 feet, any icing, or thunder-
storms. This pilot offered the comment that flight
following was an “excuse to violate the rules.” Another
rule offered was that VFR over-the-top was “nor smart.”

All pilots were asked the question, “in the context
of piloting an airplane, what does ‘risk’ mean to you.”
While no good definitions were provided, every pilot
made some statement acknowledging that risk existed

in aviation. One pilot put it simply, you are “crazy if
you don’t think it (risk) exists.” Another added that
there is “nevera time of no risk.” Most pilots however,
offered situations that they consider risky. These
included cross country flights, quickly deteriorating
weather, being low on fuel, rain showers, lowering
ceilings, turbulence that is uncomfortable, mountain-
ous terrain, extended over-water flights, single engine
flight at night, being hurried to be on time, being too
optimistic about one’s pilot capabilities, and lack of
proficiency. A few pilots made comparisons to other
activities. One compared single-engine flying to the
tisk of riding motorcycles, which he also enjoyed.
Another stated that the risk in flying general aviation
aircraft was “less than driving a car.”

To assist in generating risk factors and guidelines
used to manage them, pilots were asked about situa-
tions in the past that concerned them in any way.
Althougin some of the pilots were initially reluctant to
talk about their own situations, all eventually re-
counted a specific situation. Two such examples were
given by two pilots who had each accidentally run a
fuel tank dry causing the engine to quit. Onc of these
was as a student pilot with an instructor on-board. A
second pilot had run a tank dry on a high performance
aircraft. He said it took 15-20 seconds to get the
engine restarted. The same pilot had a retractable gear
stick in the “up” position. Another recounted a strong,
gust that resulted in a go-around in California. An
eastern U.S. pilortold of trying to get home froma fly-
in by flying through a valley with the tops of the
mountains obscured. Flight service had said the weather
would be “OK” but rain showers quickly developed
and visibiliry dropped. The only escape had been to
turn around and follow a set of railroad tracks our of
the valley. The factors that had generated concern in
chese past situations represent possible learning expe-
ticrces for pilots about preflight decisions.

When asked to supply suggestions about training
pilotsto reduce risk, several examples were offered. Some
simply stated that a pilot must maintain proficiency for
whatever type of flying she or he planned (such as
taildraggers, instrument conditions, or in marginal
weather). One pilot recommended extensive reading,
then went on to provide a short list of specific things
to read that included some popular aviation magazine


Developing the Personal Minimums Tool

PART II: MOTIVATIONAL JUDGMENT

The motivation to choose and execute a suitable course of
action within the available time frame.

The first part (rational judgment) is the mental
ability of the pilot to detect, recognize, and diagnose
problems, to establish available alternatives, and to
determine the risk associated with each alternative.
This part is purely rational, and if it could be used
alone (which is not possible), would allow problem
solving using mathematical functions in much the
same manner as a computer. This does not mean it
would be error free; it uses information that is proba-
bilistic and therefore, predicts outcomes that are not
certain. In addition, rational judgment depends upon
the amount, type, and accuracy of the information
stored in the pilot’s memory as well as his or her
learned capabilities to retrieve and process informa-
tion. To optimize rational judgment requires high
leveis of knowledge, experience, organized mental
structures, and. systematic computational and prob-
lem solving abilities.

The second part is motivational judgment or the
bias aspect of judgment. The emphasis is on the
directional rather than the aspects of motivation deal-
ing with intensity. This part of judgment says that
humans (and pilots) base their decisions, in part,
upon bias factors or cendencies to use less than purely
rational (as defined by society) information. These
factors include immediate gratification such as ego,
adventure, commitment, duty, social pressure, and
emotional arousal in the form of worry, fear, stress,
anxiety, and euphoria, as well as more long term biases
such as risk-taking attitudes, and personality factors,
e.g., fear of failure and defensivencss. Optimizing
motivational judgment requires both 1) an awareness
of biasing factors and 2) a will (motivation) to sup-
press these error producing factors so that decisions
can be made on the basis of relevant safery factors
from the physical world.

At this time, the rational aspece ~f pilor iudgment
has received very little attention in aviator sesearch.
However, there is much in the literature outside of
aviation, including studies related to stock brokers,

livestock judges, and medical diagnesticians, indicat-
ing that this aspect of judgment, can be taught. In
each of the areas studied, judgmenial training occurs
over a fairly long apprenticeship program in which the
trainee observes the expert make decisions and learns
by observation. Bill Rouse and his colleagues have
performed a series of experiments to develop fault
diagnosis training systems to be administered on
computer (Rouse, 1979). One demonstration study at
Ohio State (Jensen, Adrion, and Maresh, 1986) has
shown the effectiveness of the DECIDE model in
teaching rational judgment to pilots.

On the other hand, che mictivational aspect of pilot
judgment has received the bulk of research. Early
efforts following the Jensen and Bene! study focusing
on this part of the mode] have shown that m:otiva-
tional training can be effective. The model used in all
of these studies may be called the attitude model or
five hazardous attitudes, Anti-Authority, Impulsiv-
ity, Invulnerability, Macho, and Resignation. An
awareness of these attitudes, that are found to some
extent in everyone, can help to develop a more positive
and rational approach toward flying decisions. Train-
ing studies using this model have demonstrated that
pilot decision making improves anywhere from 13 to
100 percent as a result of attitude training (Buch and
Dich}, 1982; Telfer, 1989; Telfer, 1987; Dichl, 1987;
Dieh!, 1992; Fox, 1991; Alkov, 1991}. Even more
impressive results have been provided by two helicop-
ter operational training studies. Petroleum Helicop-
ter Inc. (PHI) and Bell Helicopter have both offered
the attitude method of judgment training to large
numbers of helicopter pilots. PHI has reported a 54
percent reduction in accidents after giving this train-
ing to their pilots. Belt Helicopter in two studies
reported a 36 percent decrease and a 48 percent
decrease in accident rates after the training. Both
organizations point to the judgment training as the
most important cool now available to improve safety
in helicopter flying. However, attitude training, as it
has been tested in each of these researca programs, has
negative connotations and its benefits seem to have
reached a plateau. Perhaps, greater emphasis needs to
be placed on the rational side of the model emphasiz-
ing information processing.


FUTURE DEVELOPMENT

This program produced a new concept for aiding
pilots in generating personal minimums to manage
flight risks during preflight go/no-go decisions. It is
believed that this approach to personal minimums
training for preflighc decisions has a great deal of face
validity; however, itis a new idea that has a certain risk
of failure if it is not fully developed and presented to
pilots properly. Training programs often fail if their
approach is not fully developed. Furthermore, strate-
gies for the assessment of the approach need to be
developed to show its effectiveness. The authors think
that this tool to aid pilots in preflight decision making
has se much potential for improving aviation safety
that it deserves to be fully developed and implemented
with a proper assessment tool.

It is recommended that further efforts to support
the program’s use proceed in two directions simulta-
neously. The first uses field testing to refine the
training intervention into a turn-key training tool for
use by Aviation Safety Program Managers, Safety
Counselors, and flight instructors. The second com-
pletes the study of the feasibility and effectiveness of
personal minimumsasa tisk management tool through
laboratory studies. The research team sees these two
approaches as mutually supporting and occurring
simultaneously.

FIELD TESTING AND TRAINING
REFINEMENT

To refine adequately the preliminary personal mini-
mums training intervention program into a turn-key
training tool, the design team recommends a series of
field tests and instructional system design activities.
The field cests provide both a subject pool for the
experiments described subsequently and evaluation
opportunities for the training intervention refine-
ment. The field cest series is expected to be offered to
the following audiences:

27

Developing the Personal Minitmaums Tool

1) University aviation students,

2) Local flying clubs,

3) Local Aviation Safery Seminars taught by the inves-
tigators,

4) Regional Safery Seminars using Aviation Safety Pro-
gram Managers and research team members as in-
structor teams,

5) Regional Safety Seminars using Aviation Safety Pro-
gram Managers as instructors,

6) Various other target communities of pilots represent-
ing a diverse sample of pilots from across the country.
(e.g., pilots in mountainous terrain, high-traffic
areas, routine over-water operation areas, Alaska, etc.).

In these field tests, an evaluation program will be
administered at three levels: reaction, learning, and
behavior. At the reaction level, each participant will
complete an evaluation questionnaire at the end of the
training seminar to provide immediate feedback on
the acceptability of the training program. During the
training seminars, the research team will collect obser-
vational data and sample participant responses to
assess learning or knowledge acquisition during the
seminar. The behavioral level assessment will occur
approximately six weeks following each training semi-
nar to assess use of the training in actual flight opera-
tions. This assessment will use a short questionnaire,
administered by mail, and selected telephone inter-
views for non-respondents.

These evaluations will provide both formative and
summative evaluations of the training materials at
each level of the field rests as described above. The
objectives are to make the training seminar and its
support materials useful and effective to participants
while providing an acceptable and efficient program
to expected instructors. Additionally, the program
evaluation will address the applicability of this ap-
proach to pilots of various skill. experience, and certifi-
cation levels operating in a variety of flight operations,
special environments, and target populations.


Developing the Personal Minimums Tool

Apayenb wiy Malnay

“aye ‘Suey

@UIN-OUO “SWAY BSN POY! ‘UOISINEI Pou Je
Pappe dq 0} SWA}! ‘@PN|oU) Suid: SROBUR_OSHy
SNOsUE)EOSKN

Wty Anjunoo 68030 yORS 40} SAAN

40 $S4 Woy Buyoug soyjeom puepuRs @ aH

aiqewene
JUAUND JSOW—S}SBOBIO) 9 Spode, JONES

“yueucduioo puymssous yesue
Wnwixew UBY) S8oq UoUOdWICD puYAssosy)

“ju Aq suuoysiopumg we piony

“squBty Anunoo
S029 40} ‘1 ueUy JayeauB—AypaisiA WUBIY UIA

sqy6iy Anunos
88010 10) TOV 'Y__ UeYD seJeeuB—Buynod IA

quewuodAug

q46y pouvesd eBueyo Aew yoRpA sypuy BuuesU
s10)02) esoig Ayeinoied—sampecoid Bujeiado
WINUUFUI 6 39U9}S)Xa UO pajouq suaBusssed WY
SO|mY Gry YM eoUeLdWOD Uj

saned poyoeye geo
0) JIQeHEAG Sraquinw auOYd % PHzd YpAID audUd

uopeugsep yeas 0) aiqe Bujaq jou Jo yeawe
peyedxa ueYy 4918) 40) equyeAe suBid aeWIE}Yy

feuopezjuebigneo}60}0190¢

ISHS TTY asp
feuoneredo s0AI@081 YOA
Ssiy —elueser fony WNL

shep
“TM ysB} UIUAEA SauNpacod AquaBieuta pemainay

shep ~ untqua sGuipuey
pus syooye; — —edAj u) sousuadxe jus9ay

wursaiy

uogebiaeu 10) pasn speyo jBuoqoas wWaLND

uaBidxo roy ISWY aaoge WG ON

sw6y wGu
{te 104 yeuoesedo UONBUNSap 36 BuyyGy Asaunry

Deyeyo SWEION
syy6iy Anjunoo ssoro we 40) pay verd yub44

_ Sage} HOd Woy
Paujwuayap uELy s919q % ~~ Y)BUa Aemunys

‘Syuowaunbar y4A (eulGeuws
vay s012016 0} payoujses suojesedo WYBIN
leuoges3dQ

sunoy omy uetg seu0) yufay

40 OY Udy YRIUIE Ul aQqeyeae JayeM Bunun
Say URpM yUUP Cu—~eay foYooTY

aay Sruppioneripayy

aay ssauy

feasayuy jeuOU ye Uae

paysar WOM

shep ~ unyyea

sGuypue; 2 8yoe4e) ~ —eouapedxe yuauna
Ud

“sopoejeo xis uy seujepInG suunuyujw jeuosued so sejdwiexy “9 e1Ge 1


DEVELOPING THE PERSONAL MINIMUMS TOOL FOR MANAGING RISK
Durinc Prerticnt Go/No-Go DEcIsions

INTRODUCTION AND OVERVIEW

As commonly employed, the term “personal mini-
mums” represents an individual's set of operating
criteria, procedures, rules, or guidelines used to assist
that individual in making personal decisions. In this
report, “personal minimums” pertains to the guide-
lines used by a pilot in making the preflight decision
to go of to postpone the trip. The popular aviation
literacure contains many discussions of the benefits of
personal minimums as a risk management tool to aid
pilots in making this preflight go/no-go decision (e.g.,
Clausing, 1990). This literature offers anecdotal and
narrative examples that suggest appropriate guide-
lines to minimize risks associated with aircraft flight.
However, no accepted, structured method exists for
compiling the guidelines into a usable tool. This
report presents the results of a study to determine
feasibility of providing such a tool. The results in-
clade a preliminary training program thar provides
pilots with a structured approach to generating their
own personal minimums.

To introduce pilots to the idea of personal mini-
mums as a risk management tool, this new approach
emphasizes creating a set of self-selected, self-imposed
minimums to guide the individual in making pre-
flight decisions as opposed to relying solely on mini-
mums imposed by others. The simplified training
approach developed in this program uses a list of
known risk factors to prompt pilots in selecting and
writing guidelines that they expect will assist them in
making successful future risk management decisions.
To provide organization to the guidelines, 2 suggested
taxonomy further aids pilots in placing their guide-
lines into an easy-to-use checklist so that each pilot
may reference them quickly during subsequent use.

To introduce pilots to this new approach of self-
imposed personal minimums and to provide an initial
test bed to further evaluate the concep:s incorporated,
the research team designed a preliminary 90-minute

training seminar. The seminar uses a flexible lecture-
discussion format and incorporates both extensive
examples as well as a case history to relate the program
te real-world pilot decisions, Preliminary testing of
the approach within the research group as well as
discussions with potential users resulte? in positive
initial reactions.

To complete the training seminar design and to
assure success in field introduction, the research team
anticipates further testing and evaluation as prompted
by four program goals. First, pilots with various flight
experience, certifications, and flying time in various
operational and geographic sercings must perceive the
approach as useful. Second, the Aviation Safety Pro-
gram Managers, Safery Counselors, and other poten-
tial program instructors must input their program
requirements to assure chat this approach supports
their efforts. Such instructor support will include
providing easily used intervention materials as well as
helping to reinforce the more general program efforts.
Third, the preliminary nature of chis seminar design
requires further development., including input from
pilots to assure successful implementation. Additional
tisk factors need to be identified, guideline examples
written, and case histories found that are pertinent to
a wider range of pilots. Fourth, formal evaluation will
provide an avenue for enhancing and monitoring the
success of this program. The evaluation process will
provide useful dara to modify the successful program.

This report is divided into three sections: back-
ground, product description, and future develop-
ment. The background section describes the literature
review that supports the development of personal
minimums as a risk management decision supporc
tool. Additionally, the background section contains
the results and discussion of preliminary research
tasks completed during the first year. This section
may only be of value co teachers and researchers whose



Figure 1

Figure 2.

Figure 3.

Table 1.

Table 2.
Table 3.
Table 4.
Table 5.

Table 6.

LIST OF FIGURES

. Anexample of an organizational go/no-go checklist used to guide pilots in
making preflight decisions —--~-~--------n0r-ne2-n--nceewentnnneeecceconrecevencenneenencnenceenvmwwenne ennee 3
. Human error and incident occurrence related to flight operation phase --------------------------- 18
. Sample personal minimums checklist with categories wwonn= 25
LIST OF TABLES
Fatal and Non-Fatal Accident Proportions in Percent of Total for U.S. Civil Aviation
Pilot Caused Accidents from 1970-1974 --------0----+. a. eaneceseececuee aonee. 2
Examples of ALERT risk value matrices ---2-------s-rseceernennneneeeneennnnerececescrucsensesecnnencccece 7
Example of REARM tisk value matrices -- - - 8
SAFE crew factors as rank ordered by expert panel ------------------nn--------eeecenerenceeeeeeenenennee &
Examples of preflight risk factors ----------------c+e--eeececnneseeenemnnecssaneecnneseneeenmnnonmvnarenecen 2)
Examples of Personal minimums guidelines in six categories - -- 24

vi


to read that included some popular aviation magazine
columns, training manuals, and some classic piloting
texts such as Stick and Rudder (Langewiesche, 1944).
A unique suggestion was a game in which several
pilots went on board the sameairplane. After they had
flown out of the local area, one pilot was placed under
the hood for several minutes and given vectors to fly
by another pilot. The pilot under the hood then took
it off and started to find his exact location while still
flying the aircraft. The pilots took turns. The winner
was the one who found his location in the shortest
cime. They varied the rules on what methods could be
used to identify their location.

These pilot interviews produced several findings
useful to our approach to assisting pilots in making
preflight decisions. (1) Pilot awareness of the range of
decisions needed during preflight should be increased.
(2) This awareness should expand the number of
factors considered, the extent of risk involved in these
factors, and provide guidelines for evaluating the
factors that apply to their individual flight operations.
(3) Pilots need to underscand that risk exists and have
reasonable ideas for managing that risk. (4) Pilots
need help using current forms of weather information
to make decisions. (5) Pilots are interested in training
interventions that assist them in making decisions, if
they meee their learning necds and are easily available
to them.

Guideline Development and Intervention
Testing

Following che interviews, the project moved into
ewo areas: {1) identifying guidelines thar pilots use in
making their decisions, and 2) intervention initial
testing. The guidelines developed are not an exhaus-
tive set, but examples of guidelines that pilots con-
sider usable in their environment. The training
intervention product is expected to be a test bed for
evaluating the feasibility of using a flexible strategy for
helping pilots to establish preflight personal minimums.

Guideline Development. The review of the litera-
ture, the pilot interviews, and the experience of the
project staff produced numerous examples of guide-
lines used by pilots when making their preflight deci-
sions. This preliminary list of guidelines is intended
co show the range and types of examples of guidelines

13

Developing the Personal Minimums Tool

that pilots use in making preflight decisions. Ir is not
an exhaustive list, bur it does contain key elements to
help stimulate discussion and gencrate guidelines ap-
plicable to particular pilots.

For purposes of the pilot interviews, “guideline”
was defined as, “A statement, such asa policy or rule,
for conduct.” Guidelines include acronyms, rules of
thumb, regulations, standard operating procedures,
and so forth that assist pilots in preflight decisions.

During this task, many pilots often agreed on the
general nature of a guideline, but they often disagreed
over the specific quantities or qualities that should be
placed in any particular one. An example would be a
guideline to define minimum currency. Some pilots
stated that the minimum FAR requirement of three
takeoffs and landings in the last ninety days was
sufficient. Another suggested that if a pilot with only
afew hundred houss total time, did not fly at least fifty
hours per year, she or he might not be safe, bur ifa
pilot had five thousand hours total time, then five
hours a year might be sufficient.

These discussions highlight the fact that pilots
consider similar guidelines but often disagree on the
specifics within a guideline. Of particularly concera
to the pilots is whether they set their own standards or
have limits imposed on them. However, some pilots,
usually the less experienced ones, tended to prefer
more assistance from instructors in setting their own
limits. The more experienced pilots preferred to set
their own jimits. These results could be interpreted to
mean that, with varying amounts of assistance, most
pilots who set their own limits would be more likely
to comply voluntarily with those self-set limits than
those whose limits are imposed on them,

Intervention Initial Testing. A preliminary training
intervention was then developed to teach pilots ro set
personal minimums. The instructor’s guide for this
training intervention is presented in Appendix A. The
research team wanted answers to several questions
concerning the intervention. How would pilots re-
spond to the personal minimums concept? Could
pilots identify factors influencing preflight decisions?
Could pilots write guidelines that pertained to their
routine flight activities. Could pilots evaluate guide-
lines written by other pilots? Finally, how would
pilots evaluate the training intervention?


Developing the Personal Minimums: Too!

GO/NO-GO CHECKLIST

If any one of the following situations is present cancel the trip. If aiready in flight ask to land until conditions
warrant continuing.

oe)

11.

12,

Visibility and Ceiling Requirements.

AIRPLANE—Two miles visibility and five hundred foot ceiling.

HELICOPTER—One-half mile visibility and five hundred foot ceiling.

You must always have these minimums.

Foggy Weather—Beware of fog, make certain ceiling and visibiliry minimums are present and you won't get
caught in the fog.

Snow Squaiis—-Snow squalls develop very quickly in the spring and fall months. Maintain ceiling and visibility
minimums and remember they can deteriorate extremely fast.

Wind—Thirty knots of wind speed are the maximum allowable because of emergency water landing
conditions and turbulence.

Mountain Passes—-If passes are partially obscured and appear marginal do not enter them for investigation. It
may be too late to rurn around.

Pilots—Watch for indications that the pilot's mental and physical condition are not conductive to safe flying,
ie., anges, tiredness, nervousness, or inattention.

Aircraft—Be concerned if you observe damage, dirt, fue! or oil leaks. Report it to the Aviation Officer.

In Flight Communications--Make sure radio communications are maintained with dispatcher oc FAA.
Loose Cargo and Qverloading-—-Never averload an aircraft. Make certain the pilot has all cargo secuce,
Passenger Briefing—The pilot must give you a briefing before departure on where the emergency equipment
is and how to use it.

Peseonal Protection Equipment—You must wear a FS inflatable vest with a survival kit in a helicopter over
water. Although you don’t have to wear them in an airplane they must be made available to you. We strongly
urge you to wear them. A flight helmet, fire resistant coveralls, and gloves will be furnished to you and must be
worn on all helicopter flights.

Helicopter Foreman——A qualified helicopter foreman is required to supervise each helicopter and is respon-
sible for your safety around helicoprers.

Figure 1. An example of an organizational go/no-go checklist used to guide pilots in making
preflight decisions (United States Department of Agriculture, Forest Service, Alaska Region, 1987).


decisions affect subsequent flight phases. Once the
commitment to flight is made, a strong force exists in
the pilot’s mind to continue to the intended destina-
tion. Social or peer pressure may make admitting an
error difficult or seemingly impossible and time pres-
sures may seem insurmountable. Furthermere, once

option to add more fuel. Recognizing and deaiing
with factors such as these could improve the quality of
preflight decisions and improve safety.

In preflight decision making, the Federal Aviation
Regulations (FARs) require a pilot to “become famil-
iax with ALL available information concerning the
flight” (FAR 91.103) and the pilot is “directly respon-
sible for” and “the final authority as to the operation
of theaircraft” (FAR 91.32). Furthermore, FAR 91.13
requires that the aircraft be operated in a way that does
not endanger the life or property of others. Neverthe-
less, in setting these limits, the FAA still places most
of the responsibility for risk management in the hands
of the individual pilot. The FAA limits are based on
the capabilities of ideal pilots and equipment because,
in addition to regulating for safety, they are tasked to
promote aviation. Before each flight the individual
pilot must determine whether his or her currency and
aireraf meet this ideal level of skill and capability. If
ideal levels are not met, pilot judgment is expected to
produce more conservative operational minimums
for that situation.

Personal Minimums

Pilots possess a wide range of experience, capabili-
ties, and training that makes each one unique. Physi-
ological and psychological states vary from one day ro
the next adding ro the uniqueness of each flight. On
a particular day, or even at various times during the
day, a pilot may be tired or rested, stressed or un-
stressed about a personal or professional situation, or
any combination of other possibilities. These sources
of variability suggest the need to conduct an analysis
of the risks that individuals and situations bring into
a proposed flight prior to the decision to takeoff.

Personal minimums are defined as an individual-
ized set of decision criteria (standards) to which the
pilot is committed as an aid to preflight decisions.

Developing the Personal Misimcms Teol

Theories in psychology suggest that people are more
likely co follow standards that they have made them-
selves than those imposed upon them by someone else
(Festinger, 1957). Therefore, the emphasis should be
placed on personally set minimums that may differ
from those offered by someone else in authority such
as the FARs or SOPs (Standard Operating Proce-
dures). Personal minimums have two characteristics
that should be recognized: 1) they are unique to the
individual pilot and 2) they can be changed with time
and situation.

Importance of Personal Minimums to GA Pilots

There are three reasons why petsonal minimums
are important in general aviation: 1) there is a mini-
mum level of outs:“2 supervision in most flights, 2)
many pilots have very little awareness of risk factors,
and 3) general aviation safety depends upon voluntary
compliance. Robb (1984) states an old adage that the
“less the suzervision, the less the safety” (p. 120).
NTSB accident data combined with FAA activity
estimates clearly show that the accident rate for plea-
sure flying has remained consistently about twice that
of general aviation as a whole, ten times that of
corporate or executive aviation, and about one and a
half times that of aerial application (Robb, 1984, p.
120). The most prevalent cause of pleasure flying
accidents is inadequate preflight preparation o plan-
ning (McElhatton & Drew, 1993; Robb, 1984)!

While some pilot operations take place under the
direct supervision of regulators, management, super-
visors, or others in authority, many flights in generai
aviation remain unsupervised. Pilots know that com-
pliance with regulations, SOPs, or rules (and even
personal minimums) depend upon voluntary obser-
vance by che pilor. Flying is nor unique as an enter-
prise of trust. In automobile driving, the actions of the
driver are not observed by the police at every corner.
Instead, the safety of the driver mainly depends upon
voluntary compliance with the ‘rules of the road’ and
setting some personal standards which vary depend-
ing on the situation.

In business aviation, commercial flight operations,
flying clubs, and other situations, operational limira-
tions or sets of rules define what the operational


DOT/FAA/AM-96/19

Office of Aviation Medicine

‘Washington, D.C. 20591

Developing the Personal

Minimums Tool for Managing
Risk During Preflight
Go/Nc -Go Decisions

Larry A. Kirkbride
Richard S, Jensen

Gerald P. Chubb

The Ohio State University
Columbus, Ohio 43210

David R. Hunter
Office of Aviation Medicine

Federal Aviation Administration
Washington, DC 20510

July 1996

Final Report

19960827 137.

This document is available to the public
through the National Technical Information
Service, Springfield, Virginia 22161.

@

U.S. Department
of Transportation

Federal Aviation
Administration



Developing she Personal Minimums Toot

operations such as emergency medical evacuation or
rescue where aircraft accident rates have been higher

than desired. The usefulness of such methods have *

been shown to be highly dependent upon the type of
operations or missions flown (Shively, 1990). A bsief
description of three such assessment tools follows.

The U.S. Acmy’s ALERT (Aviation Litmus Evalu-
ation Risk Test) is a paper and pencil based evaluation
process intended to assist aviation operations com-
maaders in assessing risk (Boley, 1985). ALERT gives
its users a method to evaluate the probability and
magnitude of flight risks for che following six ele-
ments: supervision, planning, crew selection, crew
endurance, weather, and mission complexity. A set of
mattices assigned a numeric risk value to each element
(see Table 2). The linear element design of this ap-
proach produced a sum value for each clement. The
total was used to assign a qualitative risk value which
was then rated as low risk (0-12), caution (13-23), or
high risk (24-30) for each operation. The unit com-
mander would then use the ratings to assist in pre-
flight decisions concerning risk management. To
decrease risk, the commander may change any of the
six elements in order to achieve an appropriate or
acceptable risk level.

The U.S. Coast Guard's REARM (Risk Evaluation
and Aviation Resource Management) system consists
ofamodified and extended version of ALERT (Shively,
1988). Coast Guard applications were for helicopter
medical evacuation operations. While the basic struc-
ture of ALERT is used in REARM, attributes and
verbal descriptions of the variables were changed to
accommodate representative Coast Guard missions
(mission type was changed from “support”, “nap of
the earth”, and “night vision goggles” to “site”, “hos-
pital”, and “scene” mission) and zwo factors (crew
experience and weather) were expanded to permit a
further differentiation of variable efiects (see Table 3).
The risk value for wind in excess of thirty knots was
increased by 1 for each 10 knots. Risk values for night
were multiplied by 2.5. As with ALERT, REARM’s
values were summed to predict a relative level of
mission tisk. The scale was divided into four parts:
normal (6-16), caution (17-25), coordination required
(26-35), and danger (36-40). The matrices total

provided the pilot with a risk assessment and recom-
mendation of the level of supervision that should
make the go/no-go decision.

The NASA Ames Research Center developed a risk
assessment tool called, SAFE (Safety Assessment for
Flight Evacuation). The SAFE approach is similar to
ALERT and REARM but it uses a personal computer
based expert sysrem (EXSYS) model to predict risk
profiles on civilian emergency medical services (EMS)
helicopter operations (Shively, 1990), The computer
increased speed, accuracy, and efficiency of the risk
assessment. This approach permits factor interaction
to be incorporated and variable weighting of factors.
Five types of factors were investigated: mission, crew,
organizational, environmental, and aircraft. These
factors were further divided into subdivisions to allow
rank ordering as co their influence on risk. The subdi-
visions for the crew factor are shown in Table 4. For
example, weather received a higher weighting due to
its greater correlation with EMS accidents. Interac-
tion may be shown between level of supervision,
weather, and crew experience. With pilor use, SAFE’s
automatic data collection feature aids the programs
evaluation as well as its continuing research.

The extent to which any of the mentioned models
are in routine use, as well as their acceptability to
pilots and management, would provide guidance as to
the efficacy of the approach and its potential applica-
tions in general aviation. Only limited field test data
are available in the open literature. Further testing is
needed.

Training Adults

It is important to consider two basic educational
issues in our discussion of the preflight situation. The
first issue concerns the most effective ways to train
adults in applying advanced skills such as problem
solving and decision making (Means, er al., 1993).
Due to sociological and organizational factors, adules
differ from children and adolescents in ow and why
they learn, The second issue involves the study of
decision making training in real-world decision mak-
ing situations (Klein & Calderwood, 1991). These
two issues are briefly reviewed to provide background



Classical versus Natural Decision Making. The ratio-
nal part of the judgment model is closely related to
what has been called classical decision making (CDM).
CDM is characterized by utility theory, Baysian ap-
proaches, and other normative models of human
decision making. Beach and Lipshitz (1993) provide
an extensive list of reasons why this approach may be
inappropriate in natural settings, like flying. Orasanu
and Connelly (1993) suggest naturalistic decision
making (NDM) as an alternate conceptual frame-
work. While CDM can prescribe what to do and help
explain why, NDM is basically descriptive, not pre-
scriptive. CDM recognizes that cven a completely
rational approach, even one aided by “perfect” math-
ematical models run on computers, would no: fz
in error free flight since the information that pilots
must use is probabilistic and contains errors. NDM
includes aspects of motivational judgment mentioned
in the model of pilot judgment. For a much more
detailed description of CDM and NDMas they apply
to aviator decision making, please see our sister report
to the FAA (Jensen, et al., in preparation).

Risk Assessment and Managemen: Livera-cre

Recognizing that risks exist, pilors and managers of
flight operations implement a variety of methods to
assess and manage the risk. A common method in-
volves providing a written set of SOPs. Many opera-
tions also use systemns-based approaches including
pilot selection methods, on-going safety programs,
training of pilor and non-flying personnel, and other
management support methods. None of these meth-
ods alone can be expected to climinace accidents, but
together, the more comprehensive systems-based ap-
ptoaches such as these improve safety significantly
{Adams & Payne, 1992}.

Risk and Uncertainty. Rowe (1977) defines risk as
“the potential for realization of unwanted, negative
consequences ofan event” (p. 24). Arrhe general level,
Rowe further represents risk as involving two major
components: (1) existence of a loss or unwanted
consequence, and (2) uncercainty in the accurrence of
che consequence expressed as a probability. Boch campo-
nents are important in risk assessment. In the absence of
loss or unwanced consequence, there is no risk. And in
the absence of uncertainty, risk docs not exist.

Developing the Personal Minimums Taol

Risk assessments can be performed using either the
CDM or the NDM approach. Most of the risk assess-
ment literature adopts the CDM perspective, Classic
decision theory assumes that there is an optimal or
best choice to be made in any decision and that models
can be used in many situations to analyze and deter-
mine which alternative is best. Such an approach has
been taken in military command and control, busi-
ness management, and medical diagnosis. The second
is an evaluation of risk at the time some situation
warrants: In real-time risk assessment the NDM ap-
proach may be more relevant. In our context, both
approaches seem to be relevant. While the CDM
tational approach is likely to work best in establishing
personal minimums in a setting apart from actual
flight, the NDM approach may become more relevant
when the pilot is attempting to apply the personal
minimums in the context of actual flight preparation.

Uncertainty exists in the absence of information
about past, present, or future events, values, or condi-
tions (Rowe, 1977, p. 17-18). Two major types of
uncertainty are involved: (1) descriptive uncertainty—
absence of information relating to the identity of the
variables that explicitly define a system (inability to
describe the degrees of freedom of the system), and (2)
measurement uncertainty—absence of information re-
lacing to the specific value of each variable in a system.
This distinction suggests two ways to help pilots: a)
identify relevant risk dimensions and b) determine the
associated values for cost and probability.

Developing Takeoff Guidelines. Even when the rel-
evant dimensions (factors or variables) are known,
pilots rarely have cools that allow them to assess the
probabilicy of occurrence, to weigh the existence of
unwanted consequences, of to manage uncertainty.
Rarely do general aviation organizations offer risk
assessment models to assist pilots in their analysis of
preflight go/no-go decisions. Several organizations
have approached the problem of making rational
preflight go/no-go decisions by providing simple pre-
scriptive decision tools. Generally these tools are
checklists or operation manuals thar say, “if these
conditions exist, thea do aot fly.” Other aviation
organizations have approached the problem by apply-
ing more complex risk assessment techniques. The
best examples have typically been for helicopter


Developing the Personal Mintmums Tool

STAGE 4: EVALUATING AND BUILD-
ING ON PERSONAL MINIMUMS

The fourth stage consists of the evaluation of guide-
lines written in Stage 3 for the purpose of further
developing personal commitment and to gain addi-
tional ideas for building and modifying the checklist
over the long term. This evaluation stage can be highly
rewarding (i.e., changing behavior) because it offers
opportunities for verbal expression and public com-
mitment. However, the instruczor/facilicator must be
very sensitive to students who may not wish to share
their ideas, or those who might consider them too
personal to share. Suggested steps in this evaluation are:

1) Check for missing, unclear, and duplicate items.

2) Ask general questions such as:

Are the personal minimums flexible?

Do they cover the range of intended activities?

Do they address flying currency questions?

Do they cover the range of equipment and

aircraft routinely used?

3) Will the pilot understand the intended puxposes for
each itern when used at a later date?

4) Isa general stacement included about how the pilot
will approach “non-routine” activities notcovered by
this set of minimums?

Seminar leaders should consider encouraging stu-
dents co suggest other evaluation methods. These
inputs can be used as examples in furure seminars.

Due to time limitations, most pilots will not com-
plete a set of personal minimums during the training
period. In addition, critical evaluation of an incom-
plete personal minimums checklist by another pilor
within the class could potentially embarrass some
pilots and thus discourage them from completing and
using the method. To avoid this problem, the instruc-
tor should providea sample personal minimums check-
list (such as Table 2) for each individual to review
critically and to discuss.

26

The instructor may use the discussion summary to
reinforce the requirement for review and revision of
any personal minimums checklist. This includes the
regular inspection/revision of one’s checklist during
biennial flight reviews, instrument currency checks,
safety program participation, of other recurrent train-
ing. A personal minimums checklist should be consid-
ered a flexible document that develops and changes
with time and experience.

CONCLUDING SUMMARY

In summary, the instructor reviews the three pri-
mary learning activities and reinforces the specific
learning elements which occurred in each. 1) The
pilots find and lise important preflight factors associ-
ated with risk, 2) each pilot transforms risk factors
into guidelines to place in a usabie checklist format, 3)
each pilot generates a preliminary set of evaluation
criteria with which to judge any self-generated personal
minimums or to review those generated by a peer.

In closing, the instructor reminds each pilot that
the personal minimums checklist is unique to her or
him. Other pilots’ minimums will be different. The
checklist need only have meaning to its user. Further-
more, it represents only a starting point for pilots to
consider. As new tisk factors are discovered, each pilot
must consider how they might by incorporated into
his or her minimums checklist. Finally, the checklist
must continuously be reviewed and changed as re-
quired to accommodate the pilor’s changing flight
activities and capabilities. To provide each pilot with
further sources of information, additional references
are listed. Presently identified references include, ad-
visory cizculars, magazines, books, newsletter (IFR
Refresher, FAA Safery Review, etc.), and standard pilot
references (Airman’s Information Manual, FARs, etc.).


Developing the Perzonal Minimums Tool

‘sau0Bezeo Ui 38142043 SUINUZUIW ‘eUOssed ajdUuES “¢ aunB}4

paubis

“PUeWWOD Ul JOHd ey} SB JUsLUBpNI Seq

Aus yosyer spuepue}s asayy yU6Iy papuayul
yoea uo pajaye aq ued JEU] SjenpIAIPU

We JO Waueq ay) 10) yerNIe a1y Jo UOReIedo
@yeS By) 10, SauapiNG se swunwWIUIWy
jeuonesedo esey) yes OAey | <asoding

"YUM pamarnen
‘pasioy
20ltd

SUINUHUIW (eUCnZedO

Qo a

o

a

“Oy ‘Swiay oWwn-avO
‘suey @sn paywuy ‘uoHSnal prou ye poppe
69 0) SUD}! ‘opnjoy SUlaY SNOBUB_SISHY

oodcooodd

yuan AUT

El dacdda NOAA

25


Developing the Personal Minimums Tool

interest is in the program development process and
rationale. Theproduct description section describes the
preliminary training interventior, its design, use, and
intent. This section provides a detailed description of
the training intervention and will be of interest to
potential users such as pilots, instructors, Aviation
Safety Program Managers, and Safety Counselors.
The future development section outlines the next steps
required to complete the development of the personal
minimums concept and turn it into a program useful
to pilots.

BACKGROUND

While the popular aviation literature has discussed
personal minimums for some time, no rational, defen-
sible method to generate these minimums has been
established. In a search of the licerature, no examples
were found of a particular pilot’s writcen personal
minimums, On the other hand, several examples of
organizational decision aids or job aids, such as the
U.S. Forest Service Aviation’s Go-No-Go Checklist
(Figure 1) were found. Also, there are numerous
standard operating procedures (SOPs) that can be
found in most large flying organizations.

Three literature areas/sources were examined to
provide the background for developing a personal
minimums tool to manage risk during preflight. First,
the basic aviation decision-making lite:ature was ex-
amined for examples of ways to assist pilots in deci-
sion making tasks. Second, the general risk assessment
and management literature was cxarnined to provide

Table 1.

the basis for teaching this aspect of preflight decision
making. Third, because the target learners will be
adults, an examination of the literature on the unique
aspect of adult learning was examined for theory and
approaches to teaching this population.

Pilot Decision Making Literature

The need for pilot judgment training was made
clear in a taxonomy of pilot e~rors presented by Jensen
and Benel (1977). Three broad categories of behav-
ioral activities were identified: procedural, percep-
tual-motor, and decisional activities, Table I is
summary of fatal and non-fatal procedural, percep-
tual-motor, and decisional error proportions of pilot-
caused accidents that occurred during the time period
from 1970 through 1974 in U.S. Civil aviation. This
finding has been reinforced in a recent study of ASRS
incidents reported in airline operations in which cog-
nitive errors accounted for almost half of the aviation
incidents (McElhatron & Drew, 1993). It seems clear
that pilot decision making errors result in aircraft
accidents and incidents.

A practical model of pilot judgment was presented
in Jensen and Benel (1977) with two parts: 1) an
ability and 2) a motivation as shown below:

PART I: RATIONAL JUDGMENT

The ability to discover and establish the relevance of all
available information relating to problems of flight, to
diagnose these problems, to specify alternative courses of
action, and to assess the risk associated with each alternative,

Fatal and Non-Fatal Accident Proportions in Percent of Total for U.S. Civil Aviation

Pliot Caused Accidents from 1970-1974 (Jense:i and Benel, 1977).

Pitot Behavioral Activity Fatal Non-Fatal
Procedural 4é 8.6
Perceptuai-Motor 43.8 56.3
Decisional 51.6 35.1



Developing che Personal Minimums Tool

commitment to the use of personal minimums as a
tisk management tool. Three mental forces are at
work to bring about commitment: voluntary genera-
tion of personal minimums, generation of personal
minimums in written form, and peer reinforcement
for personal minimums use. The key is to gain com-
mitment that is sufficiently strong that it has a posi-
tive effect in unsupervised flights.

The preliminary training intervention starts with a
blank form upon which a pilot can produce his or her
own set of guidelines. There is no preset lise of im-
posed guidelines. There is instead a wide range of
examples that pilots can use to assist them in writing
their own set. The examples they do generate ate their
own and while the pilots may be asked to voluntarily
share them with the group, they are never forced to let
another pilot see thern or evaluate them. Results of the
initial field test seinforced this aspect. Pilots were
reluctance to share any but their best or least controver-
sial guidelines.

Writing down a set of personal minimums in-
creases the commitment of an individual to his ox her
own set of minimums. When pilots are asked if they
have personal minimums, most will answer that they
do, but many have difficulty listing more than a few
such limits. Generating a list rends ro develop com-
mitment to that list. The review, updating, and main-
tenance of alisc is cxapccted to further thatcommitment.
To assist in generating an initial list, the intervention
provides a simple place te stare a list of personal
minimums. Additionally, the pilots are specifically
asked ro make a commitment to these personal mini-
mums in the form of a signed statement that the
personal minimums reflect their best judgment as a
pilot.

Peer pressure is a strong influence on any indi-
vidual operating within a group. This is especially true
among pilots. This pressure can be both negarive and
positive. As previously discussed, being embarrassed
in the training seminar would be an undesirable
example of the negative effects of peer pressure. In the
intervention, positive pressure is used to build contin-
ued support for personal minimums. During the
intervention the facilitator provides positive rcin-
forcement throughout the program. Additionally,

pilots are asked to have the personal minimums they
develop reviewed by a trusted peer, confidant, or
favorite flightinstructor. The basic concept is that, for
personal minimums to be an accepted practice within
the pilot community, generating and using personal
minimums depends upon the development of positive
reinforcement by the community.

Identifying Credible Examples. Utilizing examples is
a fundamental approach in learning. The preliminary
intervention depends heavily upon providing and
generating a wide range of useful examples of personal
minimum guidelines for pilots. The range of experi-
ence levels and types of flight operations in which
pilots participate are extensive for general aviation
pilots. Creating a generic set of minimums applicable
to all is impossible. However, the general aviation
population provides a rich set of examples for an
individual pilot co review when generating her or his
own personal minimums.

Inexperienced pilots, particularly student pilots
may not have sufficient knowledge or experience to
develop their own personal minimums. During the
preliminary field testing of this intervention, student
pilots expressed confusion and lack of confidence in
the personal minimums they developed. They sug-
gested that more gencric examples of minimums are
necessary for the various stages of their learning.
Specifically, they suggested that these examples be
provided in a form chat they could review and revise
with their instructors during their training. Future
developments in personal minimums interventions
should directly address the specific needs of students
by providing more specific examples for them to use
with their instructors.

Ease of Use. The program is designed to be easy to
use and to gain early and Jong term acceptance within
the pilot community. Minimums must be easy not
only to generate, but to use as well. Checklists are
readily accepted and used by the pilot community.
The checklist format promotes the collection of
memory aids (GUMP, I AM SAFE, etc.), rules of
thumb, and other materials into one place where they
can be maintained in a logbook or flight bag and
accessed easily. The checklist was designed for casy
updating and revision.


Developing the Personal Minimums Tool

suogeiedo jebe)
SOUBIGUIOD UW
SJUREASUOS (BIOURULS
UOINSOd/AWOIURS WOH
syuawyuKe Anes

. seyu Gnd
souyepind yeuoneedO
1@Ag] UOIsHuednS
aouryoduy uonEUysEg
‘SYUERSUOD SUL

feuoneZjueiug
neoBojo190s

ayqejene (se)yreorddy
‘sfeyop umouy
vodie ajewayy
owes Si vkemny
AixaWuioo acedsay
vor. Supeg
(s)Aemuny,
Sunybrn
swajshs poduy
SOUBUOPOd “NEY
i Burjues 19401
sueGuessed
s1ap6 Buyro)
suauueq Guwro,
SpNHNe YBIVMOT
suogesado yenods
Wein
ue,

Worssyyuonesedo

Awmqeyene Yodas soeo\A
ues Guizeoypuey
SOABM UBIUNOW
sommpesadwiay 219095
asnpj604

SouSINGIN |
SuuO}sapUNY |

Bun)

pusnssoig

Aypoyaa puny

ywbroy Buyes
AuHaISIA

uawuowjAUy

ssepjAoBayeg
Bugy6r)

adh 1809,
ropdony

qwouldinbe uogoayep s8yjeOy

snyeys yyBayosd yRuny

sanuadeosip aoueuaqwey

(s)weyshs voRBa}UNURLOD
swaysks yuepunpoy
Aymquyene jon4
sebieyooqiny
uoyeZzunssasduaBAxCE
(s)uayshs jeuopetinen
quouxdinde Gustopnuy
edojanus soueuuoped
Ayxorduuce swaiség
paads

souereg pue j6IEN
Jequinu ourug
4anodasi0H

yey

SU01S190q OD-ON/OY YB Youd exEjdiy U) StOJOR4

UoyHPUOS feDsAU
uouUINN,

yoy
s6napjoyoory
ssajg

UOREIPOW

‘sSsauy]

Ws

aBpeyouy
Suruey juaunoay
oun WyBiy 1270)

1 61y 40 Aous0ey
FaAdy ayeoyIaD

ltd

“8203903 ASH WBiyeud jo sedwexy “Gg ENGEL

22


ACKNOWLEDGMENTS

The authors thank for his assistance and support during this program:

Roger M. Baker, Jr.
Aviation Safety National Program Manager
Flight Standards Service

The authors also thank the following individuals within The Ohio State University for
contributing to this program:

Lori Rakovan
James Guilkey
John Fisher
David Fournier
Janeen Adrion. Kochan
Joshua Downs

The authors also thank numerous unnamed pilots from across the United States for their
suggestions and candid discussions.

iii


for the design of an intervention program te train
pilots in making preflight go/no-go decisions and risk
assessments.

Darkenwald and Merriam (1982) describe adult
learning as a “complex phenomenon” that has com-
monalties with childhood learning, but also substan-
tial differences. Among the differences, an adult has
an independent self-concept, has the ability to be a
self-directed learner, exhibits readiness to learn, and
has 2 weil established orientation to learning. Adults
become more differentiated from each other as they
age. Their psychosocial and physical development
differ. They possess differing abilities to employ com-
plex problem-solving strategies. They possess 2 rich
and varied set of life experiences upon which to draw.
As «result, “an adult’s ability to acquire new informa-
tion may have more to do with lifestyle, social roles,
and attitudes than with innate ability to learn”
(Darkenwald & Merriam, 1982, p. 75). Furthermore,
all these factors combine to challenge any individual
who is attempting to develop training for adults.

Knowles’ (1980) concept of “andragogy”, the art
and science of helping adults learn, cffers four points
that have influenced many successful adult training
programs. First, most adults have moved from being
a dependent personality to that of a self-directed
individual. As a result, adults tend co want to influ-
ence what they learn and how chey learn it. Forcing an
unwanted or unacceptable approach on adules may
not work well. Second, adultes have 2 growing wealth
of experiences that is a rich resousce for learning.
These experiences provide self-identity and are highly
valued. The challenge is to use this experience without
negatively challenging an individual's attitude, social
role, or lifestyle. Third, an adule’s readiness to learn is
closely related co the tasks or social roles in which the
adult participates. Adults with children may be inter-
ested in patenting classes, Pilots may be interested in
aeronautical decision making (ADM) interventions.
But pilocs without children are unlikely to be inter-
ested in parenting, while parents who are not pilots are
less likely to be interested in ADM. Thus, successful
training must be pertinent to how they see their roles.
Fourth, time perspectives change in adults. Adults cake
an immediacy of application perspective. This means
what they learn muse have an immediate application or

Developing the Personal Minimums Toot

they are untikely to pursue the learning activity. Addi-
tionally, adults take a problem-censered approach. They
tend to seek learning experiences chat offer solutions to
what they see as their immediate or anticipated problems.
The implications of these approaches to adult train-
ing are that interventions must address all of these
issues to be effective. They must account for a wide
range of experience and multiple social roles. A pro-
gram should actively involve the adult learners in the
training program at manv Jevels. That is, from plan-
ning to participation. i.earning should draw upon the
experience of the participants. Such learning is often
implemented in the form of group discussion so that
experiences can be shared and valued. Finally, the
training must be clearly useful and applicable to the
anticipated tasks or social roles of the participants.

Real-World Decision Making Training

The liserature is quite clear that in designing a
decision making training program, one must give
careful consideration to the real-world context in
which the decision are being made. Klein and
Calderwood (1991) reviewed field research and labo-
ratory studies which show that people, especially
experts in a domain, do not conform to the nozmative
decision making model in natural settings. Actually,
normative models more accurately describe what be-
ginners do (Ocasanu and Connelly, 1993). Beach and
Lipshitz (1993) argue that the classical decision theory
is an inappropriate standard for evaluating and aiding
most decision making and cite four points. First, they
argue that real-life decision tasks often differ mark-
edly from those for which classic decision theory was
designed. Second, professional decision makers sel-
dom rely upon classical theory to make their deci-
sions. Third, classical decision theory's reliance upon
gambles as an all-purpose analogy for decisions does not
apply in many natural settings due to the fundamental
tole of control. Fourth, the classical decision theory
assumption (that use will improve decision success) is
empirically unproved and questionable.

Means, Salas, Crandall, & Jacobs (1993) argue that
decision making training needs “to take into account the
specific characteristics of the task and of the social and
organizational context within which it is performed”
(p. 326). Orasanu and Connolly (1993) characterize


program implementation. An efficient, effective train-
ing program requires appropriate evaluation of both
the process and its products.

A fundamental and useful evaluation approach
commonly used in training divides evaluation of
eifectiveness into four basic levels (Kirkpatrick, 1959).
These levels are reaction, learning, behavior, and
results. In practice, these levels are described as:

1) Reaction Did the learners like the training?

2) Learning Did the learners learn the objectives?

3) Behavior Did the learner’s behavior change on
the job as a result of che training?

4) Results Did the change in behavior make any

difference in job performance?

Evaluation on these levels provide useful data to
training program developers, instructors, sponsors,
and others. Results for the first three levels could be
expected for most programs addressing personal mini-
mums. However, data on the fourth level, the results
level, may be difficult and expensive to obtain.

INTERVENTION DEVELOPMENT

In the development of a preliminary methodology
for teaching pilots to develop personalized minimums
for preflighe decisions, three primary tasks were ac-
complished, 1) a list of risk factors for preflight
decisions were generated, 2) pilots were interviewed
to develop prototype guidelines for personalized mini-
mums, and 3) a preliminary training program to assist
pilots in creating personal minimums was developed
and tested. The following is a brief description of
these tasks and their results.

Checklist of Preflight Factors

To compile a taxonomy of preflight go/no-go risk
factors that contribute to flight safety for routine
general aviation activities, che scientific and popular
sviation Jirerature relating to decision making, risk
management, and related aviation areas was reviewed.
Particular emphasis was placed on pilor characteris-
tics, aircraft characteristics, environmental conditions,
and operational activities. Some examples of risk

i

Developing the Personal Minimums Tool

factors identified by these sources are visibility, cloud
ceiling, cro. -wind component, precipitation, day-
light, night, topograpty, and total and recent flight.

Given the list of risk factors, the question remains,
can pilots use chem co assess risk levels for any parcicu-
lar flight? The literature offers mixed findings on pilot
abilities to identify risk factors and assign priorities,
particularly multiple factors using multiple criteria.
Research shows thar pilots can assign values to risk
factors (Flathers, Giffin, & Rockwell, 1981; Curry,
1976), when making flight related decisions such as
diverting from a flight plan or in landing situations.
However, these decisions are rclated to technical flight
decisions, not preflight choices to minimize risk.

On the other hand, some of the literature suggests
thar pilots do not analyze risks well. Collins (1986),
using informal interviews concluded that “the risks
perceived by the five pilots queried turn out to be not
significant” (p. 67) when compared to accident statis-
tics. Collins further states that “pilots insulate them-
selves from reality, imagining risks where none exist
and ignoring those chat are real.” Robb (1984) sug-
gests that any such approach must provide guidance
on key factors and on the appropriate weights given to
those factors.

Initial Pilot Interviews

To obtain information from pilots on preflight
decisions, an interview guide was developed and ad-
ministered to six pilots ata large southern fly-in. Most
of those interviewed routinely flew single-engine air-
planes, and two of those interviewed had instrument
satings. Thesc semi-structured interviews consisted of
general questions concerning preflight actions, guide-
lines or rules used to make the go/no-go decision, risk
factors in aviation, and examples of risky situations
from their past flying. The interviewer identified
himself as a researcher studying pilot decision mak-
ing, particularly go/no-go decisions. At no time was
the individual pilor’s name requested, but most
interviewees offered at least their first name.

An attempt was made at the start of each interview
to establish crust berween the interviewer and the
pilot. The most common initial question asked by the
interviewee was, “Do you represent the FAA?” The


Technical Documentation Page_

1. Report No.
DOT/FAA/AM-9G/19

2. Government Accession No,

3. Recipient's Catclog No.

4. Title and Subtitie
Developing the Personal Minimums Tool for Managing Risk During Preflight
Go/No-Go Decisions

5. Report Date
July 1996

6. Performing Organization Code

7. Author(s)
Larry A. Kirkbride, Richard S. Jensen, Gerald P. Chubb, and David R.
Hunter

8. Performing Organization Report No.

9. Performing Organization Name and Address
The Ohio State University
Coiumbus, Ohio 43210

10, Werk Unit No. (TRAIS}

11. Contract or Grant No.
DTAFA01-92-10204

12, Sponsoring Agency Narne and Address
Offfice of Aviation Medicine

Federal Aviation Administration

800 Independence Avenue, S. W.
Washingron, DC 20591

13. Type of Report and Period Coverec:

15. Supplementary Notes
‘Work was accomplished under approved task AM-A-94-189.

16, Abstract

This report describes the preliminary design of a structured method for pilots to construct personal minimums
related to safety for use during preflight decision making. The design uses a three-step process by which
individual pilots prepare personal minimums that apply to the scope of their routine flight activities. The
approach encourages voluntary compliance with the self-generated personal minimums guidelines prepared in

checklist form.
17. Key Words 18. Distribution Stetement
Personal Minimums, Risk, Preflight Decisions, Document is available to the public through the
Decision Making, Decision Aids, National Technical Information Service,
Aeronautical Decision Making, Pilor Training Springfield, VA 22161
19, Securtty Classif. (of this report) 20. Security Closstf. (of this page) 21. No. of Pages 22. Price
Unclassified Unclassified 49

Form DOT F 1700.7 (8-72)

Reproduction of completed page authenzed



Developing the Personal Minimums Teol

minimutns include. In these types of operations, pi-
lots with less experience must operate with higher
ceilings, higher approach minimums, lignicr winds,
and lowes crosswind components. In airline and mili-
tazy operations, an extensive set of SOPs provide tools
to assist pilots in making critical preflight go/no-go
decisions. Flights may be canceled by dispatchers,
chief pilots, or supervisory personnel even before the
pilot has an opporcunity to address the go/no-go
decision. Such supervision remains effective in im-
proving safety if it is maintained over an extended
period of time and appears to be one reason fora better
safety record.

Personal assignment of minimums requires an
awareness of the risk factors involved before one can
attempt to manage them. Risk awareness is an effec-
tive tool to increase safety, but requires experience and
knowledge. Insurance companies demand higher rates
frose lower time pilots, pilots with limited experience
in high petformance or complex aircraft, and from
pilors without upgrade training in advanced aircraft.
Some pilots may think chat their responsibility ends
with understanding their own safety risk. “Allowable
tis” understood by passengers may be quite different
from that of the pilot. The pilot has a responsibility to
be sensitive to the level of understood risk acceptable
to his or her passengers and company. Therefore,
some study may be required by the pilots to establish
both their personal risk factor knowledge base and
that of others who may be affected by the flight.

The popular flying literature has discussed the idea
of personal minimums for years. Numerous articles
describe accidents, incidents, and near misses in at-
tempts to offer an awareness of these issues to pilots
without exposure to the consequences. The following
reasons are often offered for the establishment of a
written set of personal minimums:

* Promotes safer flight operations

¢Heightcns awareness of risk factors and provides
techniques to manage risk

Highlights the effects of pressure (commitment, peer,
supervision, passenger, time, destination, financial,
etc.)

20

* Fosters “good practice”

Encourages competent or professional image and
behavior

+ Assists in legal requirement compliance

* Provides a rational, defensible method to support a
“no-go” decision to passengers, employers, and others

Most pilots, especiclly more experienced and well
trained pilots, limit the conditions under which they
will fly. Thus most already possess a preliminary set of
personal minimums. However, few have a written set
that they can reference. A personally developed and
written set of minimums can carry the psychological
force needed for a pilot to resist a tempting risk in
unsupervised aviation when there may be pressures to
take it.

STAGE 2: IDENTIFYING RISK
FACTORS

The objective of the second stage is to develop in
the students a knowledge base of risk factor awareness.
The activities in this stage include a short facilitator
discussion of risks, a case study with the whoie class,
and a longer smail group exercise to generate a tax-
onomy of preflight risk factors.

Risk factors are defined as, “something that ac-
tively contributes to the production of a negative
resulc” (in this casc, with regard to the expected
flight). Numerous examples of preflight risk factors
can be offered including, icing, thunderstorms, a
questionable magneto drop, an inoperative commu-
nication radio, no flight in type for more chan 30 days,
family pressure to get home, or getting only four hours
of sleep in the last 24. Research has shown that when
people make decisions, they usually consider only a
very small number of factors and they tend to make
these decisions based on a few basic rules discovered
in previous experiences, and/or on their “gut feeling”
about the situation. This activity stage is designed to
expand the pilor’s knowledge base of risk factors that
go into preflight decision making and to classify them
into categories thar ate easily recalled wnen they are
needed.


NOTICE

This document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of
information exchange. The United States Government
assumes no liability for the contents or use thereof.


Developing the Personal Minimums Toot

EXPERIMENTS

To investigate further the feasibility and effective-
ness of personal minimums in risk management, a
series of experiments is recommended. The objective
of these experiments is to compare the approaches of
self-generated, voluntary minimums (such as personal
minimums) to those of imposed minimums (i.e.,
SOPs, rules, regulations, etc.) as tools in changing
pilot risk management behavior and to determine
whether pilots can be taught effectively to assess
relative risk. In these experiments, two types of vari-
ables would be tested, (1) types of training (personal
minimums, imposed minimums, or no training) and
(2) experience and certification levels. Experimental
subjects will come from those individuals who partici-
pate in field testing of the intervention as described
previously, Control subjects will come from an avail-
able pool of pilots similarly qualified, but untrained
in risk assessment and management using personal
minimums. The experimental protocol will use
ground-based realistic flight scenarios to collect data.
Verbal protocols will be used to collect data during the
preflight planning and go/no-go decision making
stages. Dara analysis will be by standard verbal proto-
col analysis techniques. The analysis will focus on (1)
awareness of risk factors available during preflight
activities, (2) methods used to manage recognized
tisks, and (3) outcome decisions made about the
proposed flights.

Finally, a test is recommended of whether or not
pilots trained in preflight decision making can make
better relacive risk assessments than those not compa-
rably trained. If the results are positive, we may have
avalid tool to offer. Controversy exists on the ability
of pilots ro assess relative risk. If pilots can assess
relative risk, then they may be able to use that knowl-
edge to implement appropriate risk management tech-
niques. If experienced pilots can make better relative
risk assessrnents, then the possibility exists that train-
ing may offer a surrogate for experience in developing
relative risk assessment and management techniques
beyond making the simple go/no-go decision. For
example, can a change of aircraft, avionics, autopilot,
destination airpore, or other factors, implemented
singularly or in combination reduce the relative risk

28

so the flight can be made at an accepeable risk level.
However, if experienced pilots are not better able to
assign relative risk values, then simple training inter-
ventions may not be appropriate due to strongly-held
but erroneous individualistic opinions or other causes.
As a starting point, two primary questions should be
addressed. (1) Can a group of general aviation pilots
be identified that possess significantly better abilities
to assign relative risk values to those risk factors that
can be evaluated at the preflight phase? (2) If such a
pilot group can be identified, then is it possible to
identify the characteristics of that group chat conerib-
ute to such a capability? Suggested characteristics
offered include: flight time, initial training, recurrent
training, education, or other life experiences.

EXPECTED RESULTS

Two types of zesults are expected. The field testing
is expected to result in a refined, production quality
persona! minimums training program that is proven
successful with both pilots and potential instructor
personnel. The experimental results are expected to
provide a scientific basis for using personal mini-
mums and new ideas for the development of the
product.

Field Testing Results

The series of field tests and a structured instruc-
tional systems design approach will result in a refined
production quality personal minimums training pro-
gram for pilots to use as a risk management tool
during preflight takeoff decisions. The program pro-
vided will have been proven effective and can be
handed over to flight instructors and the FAA's Avia-
tion Safety Program Managers and Safery Counselors
with confidence that it will succeed. It will have been
evaluated using assessment tals, someof which could
be used continuously in the training setting to provide
continued assessment data during the life of the train-
ing seminar. The field testing also will serve to in-
struct some of the Aviation Safety Program Managers
and Safery Counselors in how to facilitate this program.

Throughout both the field testing end experimen-
tal programs, the research team expects to introduce
the ideas of personal minimums to the FAA Aviation


Guideline Definition

The dictionary definition of a guideline is “aline or
rule by which one is guided: as an indication of or
outline of policy or conduct.” Intended for use in a
checklist, guidelines are simple rules of thumb, say-
ings. if-thea statements, or memory aids. Guidelines
are used frequently in aviation to help remember
checklist icems and to cause you to think before acting
impulsively. For example some guidelines used by GA
pilots are:

* “Dead foot, dead engine”

* “Never fly single-engine at night in the mountains”

* “GUMP: Gas, Undercarriage, Mixture, Prop”

* “On final approach, always check first and then say
aloud, ‘Gear indicates down and locked”

¢ “The three most useless items in aviation: The runway
behind you, the altitude above you, and the fuel
that’s not in your tank”

A fonger list of guideline examples is shown in
Table 6. Care should be taken to provide samples of
risk factors and guidelines that match the cettifica-
tion levels, flight activities, and other vasiables of the
students in thatclass. Particular items on the checklist
can be used or changed to reinforce subjects from
other training efforts or the newest “hot” topic in
aviation safety. The facilitator may request examples
of guidelines from the students to be sure they under-
stand the concept.

23

Developing the Personal Minimums Tool

Composing Personal Minimum Guidelines

After the presentation of the guideline definition
and examples, the students are asked to work indi-
vidually on their own guidelines using the blank
checklist shown in Figure 3. The taxonomy headings
or categories on the checklist provide an initial orga-
nizing structure that the pilot is encouraged to change
to fit his or her individual way of organizing the
guidelines. Each pilot is given sufficient time to gen-
erate at least one guideline in each category. The most
important category and the most difficult to handle is
the “Pilot.” Special attention may be needed in that
category to be sure the students provide several items
before setting it aside. Students should also be encour-
aged co consider the sisk factors identified in their
previous exercise in the development of guidelines.

This exercise can be used to reinforce risk factors
and direc: discussion toward voluntary exchange of
ideas about personal minimums. The success of the
personal minimums approach depends upon building
a personal commitment to writing and to the use of
these minimums in preflight decision situations. The
final step in this stage is to ask students to commit to
the process by signing a statement of commitment to
use minimums, also provided in Figure 3.



Developing the Personal Minimures Tool

Table 2. Exampies of ALERT risk value matrices (Boley, #985).

Weather Risk Value Matrix
Ceiling/Visibitity
Wind Velocity
(Knots) Clear YER (>1000-3} Minimums

30 3 4 5
20 2 3 4

10 1 2 3

Crew Endurance Risk Value Matrix
Length of Rest
Quality of Rest
Optimum Adequate Minimal
Tactical 3 4 5
Training 2 3 4
Unit 1 2 3
Supervision Risk Value Matrix
Mission
Command Control
Support NOE* NVG**
OPCON 3 4 6
Ground 2 3 4
Aviation 1 2 3
* Nap of the earth
“ Night vision goggles


Developing the Personal Minimums Too!

decision making in veal-noskd os naturalistic serrings
as including some or all of the following features: (1)
ill-steuctured problems, (2) uncertain dynamic envi-
ronmenis, (3) ill-defined, competing, or shifting goals,
(4) action/feedback loops, (5) time stress, (6) high
stakes. (7) multiple players, and (8) organizational
goals and norms. Such characteristics exist in pre-
Digha decisions. :

Means et al. (1993) offer three reasons why ignor-
ing context creates problems, The first basic argument
is that, “decision training based on classical decision
theory has not been shown to transfer to natural tasks
outside the classroom” (p. 326). Studies “suggest that
formal models taught in professional programs do noz
get used on Yne job by ‘business managers, hnancial
analysts, of medical diagnosticians” (p. 307). This
training approach is not effective because the circum-
stances under which the model is tzught and prac-
ticed, and the model's view of decision making are
inconsistent with real-world decision making. Incon-
sistency may be attributed to ill-defined problems,
oformanion Yor Tess quandification, o: decision
tasks that are embedded in other activities, goals, or
emotions, Furthermore, when decisions must be made
under time pressure, normative models require too
much tine- Heuristic choices, such as using rules of
thumb, may be more accurate than a truncated nor-
mative model in a particular domain. Finally, many
formar models sequine mort eost than they ase worth.
Some are 00 complex, difficult, of time consuming.
Therefore, under time pressure, the added cognitive
processing load may be too much.

The first argument saised by Means et al. (1992)
supports che distinction made between analysis and
evaluation as different kinds of risk assessments that
arent likey weed vo wriline difesene approaches. The
second argument they raise, points to the fact chat
evaluation must always consider time constraints, but
analysis can occur in a more relaxed environment.
However, their third argument suggests that pilors
should not be turned into amateur decision analysts.
If decision analysis supports risk assessment, pilots
sherds be shown she sesules, sather than expecting
them to perform any analysis themselves. Decision
analysis would require an extensive knowledge of the
probability theory chat few pilots have, and it is

10

unteasonable to expect they would want to learn
about decision theory just so they can assess risk on a
logical basis.

An atgument for including decisional characteris-
tics and context is that bias training transfers only
marginally to natural settings (Means et al-, 1993, p.
326). Bias reduction or elimination training is aimed
at biases such as overconfidence, representativeness,
hindsight, and confirmation. Such designs try to
inhibit natural or intuitive ways of thinking. Inhibit-
ing said ways is difficult to do in a single seminar or
short-time-frame training program. The “five hazard-
ous thoughts” is a form of bias training which may be
the reason such training may have reached a plateau.
Means et al. (1995) recognized that bias reduction
training has some reduction effect within a task but
that limited generalization of the bias extends outside
the task. They conclude that with only minimal ef-
fects being shown in an experimental evaluation of
these training programs, it is unlikely that the real-
world effects would be long lasting.

Another basic asgement for including decisional
characteristics and context is that, “in teal world
settings, experts and novices differ in how they use
their domain knowledge, not in their ability to use
particular problem-solving methods or decision rules”
(Means et al., 1993, p. 326). As individuals know
more, they begin to understand things differently and
often have the ability to frame the problem better.
They develop abilities to reorganize information, to
recognize familiar patterns, and to attend to critical
indicators. These skilis are domain specific and the
expert’s knowledge is highly proceduralized. These
factors require differentiating what experts know from
how experts use their knowledge. The critical point is
thatin many realaworld decision making asks, cogni-
tive overload separates the novice from the expert.

Fundamental Trajning Evaluation

To measure training effectiveness in real-world
environments, both formative (process) and summative
(product) evaluation have value. Formative evalua-
tion provides feedvack to those developing training
programs during the development process, Summative
evaluation provides additional feedback ar the comple-
tion of the development and following training