include items which if not properly checked,
could pose imminent danger to aircraft, crew,
or passengers, as well as damage to persons or
property on the ground during takeoff or land-
ing. Examples of these would include fuel
quantity and flaps on the “BEFORE TAKE-
OFF” checklist and flaps and gear on the
“BEFORE LANDING” checklist

“Checklists are like things-to-do lists. They’re
only helpful if you remember to look at them.
Checklists get forgotten in entirety. If a key-
board response was required for each item on
a ‘BEFORE START” checklist before the
engine start valve would open, that checklist
could not be forgotten, etc.”

“We have to ‘sell’ the average line pilot that it
is professional as well as ‘cool/manly, etc.’, to

accomplish each checklist thoroughly every .

time! We have to show how it will help the
flight crewmember himself to do the check-
list.”

ct f the design of the checkl

‘That's where either the checklist or the proce-
dure should be changed.” (emphasis added)

“My company management pilots need to
more strongly endorse checklist importance
and standardization.”

“Our airline has excellent checklists and pro-
cedures which are carefully followed by crews.
Errors still creep in.”

“We must expect errors, and plan and design.
knowing there will be errors.”

“We don't need another gadget to check T/O
waming systems. A specific ‘Killer Item’
recheck is appropriate.”

“Checklists must cover a dead tired crew.”

“Brevity and simplicity.”

3.4.8 Summary oF Finpincs

From this survey, we may draw some conclusions

regarding checklists in everyday use.

+ Larger print and/or better letter spacing on

checklists would be desirable.

19

« The small sample of respondents who use

CRTs for checklists find them preferable to
other types of checklists. They all feel that the
CRT checklists are easier to use over all cock-
pit lighting conditions; that they are easier to
get at; that they are easier to use over all
Operating conditions; that they facilitate quicker
use; and that if items are skipped, they can be
retumed to more easily than with a paper
checklist.

Our discussions with some corporate users of
electronic checklists revealed a negative side
tothese devices. They indicate that CRT check-
lists can be more difficult to use; that they can
Tequire a great deal of heads-down time; and
that it is cumbersome to return to skipped
items.

Pilots felt thatthe creation ofa “core” checklist
across industry lines would only meet the
“Jowestcommondenominator” andthus would
penalize the innovators and the conscientious.

Color-coding for easy recognition of check-
lists was reported to be desirable and is already
being used by some operators. This takes
different forms, from colored borders on check-
list cards, to solid colored cards, to colored
folders to hold the cards. Variations of all of
these are being used by airlines at present.

There are many sources of interruption to
checklists. Some, such as multiple ATC com-
munications at inappropriate times, are re-
ported as causing distractions and increasing
workloads.

Mostof the airlines which were covered in this
survey were reported to have a policy for the
use of checklists which the crews followed.
However, 1/2 of the respondents stated that
individuals in the cockpit influenced whether
checklists were done correctly, or at all. This
indicates a lack of compliance which should
be addressed by the airlines.

The survey questions concerning procedures
forusing checklists verify ourconcems that, in
fact, checklists are used in an environment that
prevents crews from dedicating predictable
chunks of their attention to the completion of
these lists, and that they accomplish these lists
under conditions that are ideal for causing



AIRCRAFT OCCURRENCE
TYPE

75.

76.

77.

78.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

MLG

LRG

MLG

SMA

MLG

MDT

LIT

MLG

MLG

LRG

SMT

Aircraft aborted T/O due to high
wind noise around Capt.’s window

Didn’t make required log book
entries

Gear doors didn’t retract on raising
the gear, damage to doors on
subsequent landing

Aircraft unable to pressurize,
descended with special handling
a

Emergency descent due to loss of
‘zation :
Aircraft landed gear up

Cabin altitude horn sounded, unable
to control cabin altitude, emergency
descent with altitude overshoot

Aircraft took off with cockpit door
open and flight attendant still stowing
baggage

Aircraft lost right engine cowling and
had Tight engine failure at 1,000’ in
climl

Aircraft had smoke in the cockpit and
pressurization problems, descended
and continued to destination

Go-around due to no gear extension
and GPWS warning

Aircraft landed with the cabin not
secured and with flight attendants not
in assigned landing positions

Possible health hazard to ground
personnel from operating radar

Aircraft aborted takeoff from 40’ in
the air resulting in aircraft damage

C7

CAUSE
Window design such that the handle appeared
properly in place but the securing dogs weren’t

properly in place, window is not a checklist item
or it might have been noticed

Had an asymmetric flap procedure on landing, used
abnormal list and normal, during the confusion and
subsequent relief of being on the ground, they

Crew did the checklists required for unretracted
gear doors, used all published procedures

Switch not in proper position to allow
pressurization, was answered for on the before-
taxi checklist but not properly checked

Failure of door seal, used all appropriate checklists
and landed without incident

Busy watching traffic ahead on final, didn’t extend
gear or do GUMP check

Improper altitude put in altitude reminder while
F/O was busy trying to do the checklists and talk
with ATC

Flight attendant supposed to close cockpit door,
inadequate flight attendant training, cockpit door
not on any checklist

Latches to the cowl are supposed to be checked on
preflight, pilot claims he did, all emergency
P res followed, uneventful landing

Did the electrical smoke or fire checklist, isolated
the problem, continued to destination and landed
with the emergency equipment standing by on the
ground

Crew got behind the program with an approach in
the weather and a change of runways during
approach, missed the geur on the checklist

Checklist still reflects the use of a call button to
alert the flight attendants at the time the no-smoke
sign was turned on - with the new smoking regs,
the no-smoke sign is on all the time for this airline
- checklist or operating policy should be revised

After a demanding flight the crew did the
checklists and thought they had turned the radar to
standby - radar had different switching than what
they were used to and may not have been tumed to
standby

Pilot took off with the control lock on the yoke -
didn’t use checklist to back up flow pattern



15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

26.

AIRCRAFT

LRG

SMA

SMA

SMA

LRG

MLG

MLG

OCCURRENCE

Hydraulic problem after takeoff,
dumped fuel, declared an emergency

Aircraft lost partial power on takeoff,

hit powerline and made gear-up
landing on grass area of airport

Aborted takeoff due to engine
disintegration with associated fire
warning

Gear retraction during takeoff roll,
aircraft dropped to runway

Gear up landing

Runway incursion on rollout causing
ted. takeoff by a MLG

Pilot not flying shut down both
engines in improper response to a
warning light, aircraft was between
1,200” and 1,500’ AGL after take-
off, able to sade engines and
continue

Altitude overshoot on departure

Aircraft returned to land, nose gear
pin installed

Altitude alert activated in cruise,
descent begun and oxygen masks

Landed wrong runway from an ILS
approach

Unable to control cabin altitude,
made a descent to control it

C3

CAUSE

"A" system hydraulic failure on takeoff, subsequent
multiple abnormals due to air conditioning
problems, emergency declared with return to
airport, equipment standing by and tow to the gate

No time for emergency checklists, cause of loss of
power under investigation

Aborted, performed emergency checklist, checked
by fire crew, taxiing to gate crew noticed
further engine fire which they extinguished, taxied
to the gate

Failure to follow proper checklist, instructor giving

dual instruction gave pilot improper instructions
regarding a short field takeoff and the proper
positioning of the gear handle

Pilot extended flaps on final instead of gear and
didn’t use a checklist to assure gear down, ignored
warning horn assuming it was a stall warning near
the ground and of no consequence

Called for after landing checklist on rollout,
misunderstood “hold short” instructions which had
been acknowledged by the F/O, started across
runway, too much confusion

No use of checklist, highly experienced Capt. tried
to do an abnormal procedure without reference to
the checklist and without coordinating with the F/O
who was flying

PIC flying, check-pilot in the right seat acting as

F/O and known for not encouraging checklist use

or altitude callouts, aircraft sometimes flown as a

single pilot operation, poor coordination and no

clear direction from the PIC és to procedure to be
lowe

Nose gear pin installed during tow to gate, during
checklist the crew checked for gear pins, felt two
and thought it was three

Crew did not turn on the pressurization switches
when doing the checklist, thought they had but
missed them

Crew busy changing frequencies, doing checklists,
etc., aircraft had been flown fully automated, on.

Capt. called for correction on ILS, took over
aircraft and landed on the wrong runway in poor
visibility

Found air conditioning pack switches off, the rest

of the checklist had been performed properly but
those had been missed


45.

47.

48.

49.

50.

Sl.

52.

53.

55.

356.

357.

58.

AIRCRAFT OCCURRENCE

MLG

MLG

MLG

MLG

SMT

MLG

LRG

MLG

MLG

MLG

After liftoff a door light came on and
aircraft could not be pressurized,
returned to land

Minor overshoot on descent

Altitude overshoot on descent

Altitude overshoot and excessive
speed

Altitude undershoot on climb and
missed altimeter on approach

Possible near miss

Left engine running after the securing
checklist and leaving the aircraft

Altitude overshoot, possible conflict
with other traffic

Aircraft landed with considerable fuel
imbalance

Aircraft overshot +h course,
corrected for normal approach and
landing

Altitude overshoot on descent

Flight departed with less than planned
fet ioed P

Altitude overshoot on descent for ILS

Aircraft would not pressurize in
climb

Partial hydraulic loss, manual gear
extension

c-12

CAUSE

On door light checks on the checklists on the
ground the door light was not illuminated

Contributing factors were preoccupation with
checklist and PA

New capt. ing line operating experience, doing
checklist, changing frequencies, getting ATIS, de-
icing airplane, autopilot did not capture properly,

also no altitude alert on this type of aircraft when
it is on all the rest of the fleet, nonstandardization

Light aircraft with a fast climb, crew busy doing
checklists, frequency changes, etc., got way behind
the airplane, attempting mixed use of autothrottle
and manual control unsuccessfully

Sloppy use of checklists

Pilot had been in contact with approach, had been
given a discrete code and cleared below the 1-AX
TCA, approach did not pass on info to LAX,
passed near inbounds to LAX that apparently did
not see him

Did not physically check that fuel control switches
were in cutoff, fuel control switch positions easily
confused

Crew busy doing arrival prep such as PA, ATIS,
checklists, etc., misunderstood altitude cleared to
and descended too low

Crossfeeding taking place, did not reinstate proper
fuel pump confi; ion before landing, should be
an item on the ‘klist for fuel pump
configuration

Unintelligible controller instructions, interruptions
of checklist, missed p: inbound course setting
on resumption of ist

Descent on autopilot, checklists in progress,
autopilot failed to capture altitude, recovered
manually

Aircraft not fueled, did not properly check the fuel
load on the pre-engine start checklist

Aircraft programmed for automatic ILS approach
capture, while crew was busy doing the before
landing checklist the FMS intercepted the localizer
and began a premature descent, corrected manually

Cabin altitude control lever in the wrong position,
missed on checklist

Used appropriate abnormal hydraulic checklist


mechanical checklist for “BEFORE TAKE-
OFF” and “BEFORE LANDING,” although
they used paper checklists for all other ““Nor-
mal” checklists.

With one exception, “Abnormal” and “Emer-
gency” checklists were kept in manuals, many
of which were not tabbed for quick reference.
The carrier that was the exception used paper
cards in color-coded folders kept in the cock-
pit.

COLOR CODING

Only five ofthe carriersused any colorcoding,
despite the fact that it could facilitate location
of acritical checklist. Carriers cite costas their
reason for not using color coding.

MEMORY ITEMS

Most carriers studied used memory items in
“Emergency” checklists. One Part 121 carrier
had reduced them to one item on each of three
checklists, and one Part 135 operator had no
memory items.

CONTENTS AND ORGANIZATION OF
MANUALS AND CHECKLISTS

Manwals and checklists for aircraft produced
outside the United States often have problems
with language, they lack tabs, there is insuffi-
Cient detail, they contain too many modifica-
tions and changes, and have a classification of
checklists different from what is normally
found in the United States. In addition, opera-
tors report that changes are very difficult to get
approved by the Administrator.

There were a number of instances of missing
items on checklists, and groups of checklists
not carried from the AFMs to the operating
checklists.

Also, a number of things which could create
confusion forthe crewsusingthem were noted.
In some cases the order in which checklists
were listed differed from the sequence in
which the actions should be taken, thereby
making them more difficult to use. Inconsis-
tent policy statements onthe handling ofemer-
gencies were seen. And there was one instance
of opposing actions being prescribed by the

13

AFM and the operating checklist on one “Ab-
normal” checklist item.

The manuals and checklists of the Part 121
carriers are generally better than those of the
Part 135 carriers, but they could still be im-
proved and standardized. There are, however,
major Part 121 carriers that are worse in this
respect than some Part 135 carriers, so itis not
possible to judge quality only by the size and
prominence of the carrier. AFMs for aircraft
flown by regional carriers, whether produced
by foreign manufacturers or in the U.S., were
often not of the quality of content of those
produced by the large U.S. manufacturers.

Frequently, there were large discrepancies
between the contentof the AFM and what was
included in the Company Operating Manuals
and checklists. Yet, there were instances where
the abbreviated checklists, although lacking
parts, were stamped “FAA APPROVAL” and
signed off by a POI. This would seem to
demand more cautious and knowledgeable
surveillance.

3.4 ALPASURVEY

3.4.1 InrropucTION

A survey of airline pilots was done by the Air Line
Pilots Association (ALPA) to obtain opinions on the
design and use of checklists from those who use
them on a daily basis. Surveys were mailed by
ALPA to the Central AirSafety Chairmen and Local
Air Safety Chairmen of eight airlines, for distribu-
tion to “pilots in different crew positions and flying
different aircraft, if possible.” Survey questions
ranged from the subject of pilots’ use of checklists
to the design of checklists. ALPA promised ano-
nymity and requested a return within a one-month
period. Eighty survey forms were sent out and
returned. (A copy of the survey, including important
results, is attached as Appendix D.)

3.4.2, RESPONDENT CHARACTERISTICS
« Thenumberoftypes of transport aircraft flown.

ranged from 1 to over 10 per individual, with
an average of 3.83 types.


5-1 Do you have a personal “must check" list that
you check regardless of how the formal checklists
are accomplished (such as the old "GUMP" list)?

© When do you use it?

5.2 Do you on feel this sort of list would be useful to
all -end crews?

5.3 Ro Jou have specific checklists to cover undone items
(such as for starting engines after a single-engine taxi)?

5.4: If 5.3 is "No," what do you use for memory jogs to assure
completion of checklist items?

© Coffee handle
©@ Checklist

©@ Go through the lt ng

e Other (ples epi rt

5.5 ee Tot checklist procedures such that you find
yourself reading checklists during periods of
otherwise high workload (i.c., taxiing in ORD,
Biven a runway change inthe middle ofa tight
‘approach, etc.

5.6 If 5.5 is "Yes," do you
e Stop the list until it becomes less busy?

© Press on and hope that nothing gets missed?
5.7 aries the different sirora types in

sirtine's inventory follow

procedures for Checkin weet om

e Under what conditions do they not?



1.4 The following questions pertain only to those who have used electronic (CRT) checklists and paper
checklists and will attempt to ascertain the relative advantages and disadvantages of the two types. Please

a. Easier to use in all conditions of cockpit

illumination & CRT paper 0
b. Greater susceptibility to skipping items 1CRT paper 5
c. Easier to get at and use S CRT paper 1
d. Ease of use in different operating conditions
© Stationary on the ground SCRT paper 1
. © Moving on the ground SCRT paper 1
® Airborne CRT paper 1
€, More heads-down time required CRT Paper 2
f. Quicker to use S CRT paper 1
8. If items are skipped and returned to (such as
in taxiing without all engines Operating),
which is easier to use? 4CRT paper 2
1.5 If a checklist response is written "as required" do you answer with
a. A known value (i.e. - flaps...15°)? Yes 83 No 4
b. "As required"? Yes 15 No 72
1.6 Please indicate your feelings on the design of checklists you currently use.
a. List is too long Yes 19 No 69
b. List doesn’t cover enough Yes 10 No 76
c. Print is too small Yes 5 No $4
d. Easy to skip items unintentionally Yes 35 No 54
¢. Dimensions of list are too large Yes 10 No 78
f. Convenient to use Yes 70 No 16
8. Easy to use at night Yes 58 No 31
© Is there sufficient supplemen
lighting to make it readily visible? Yes 67 No 14
h. Organized in a manner that promotes a smooth
flow pattern Yes 70 No 20
i, Organized in a manner that reflects standard
operating procedure for the company Yes 86 No 3

j. Convenient place to stow the lists Yes 73 No 17
Kk. Easy to locate “emergency” lists when needed Yes 56 No 31
1. Do you feel that the checklist workload is

equally distributed among all crewmembers? Yes 70 No 19
m. Any other comments

2. 0 RR’ ‘0 CHECKLIST USE

2.1 Please indicate on a scale of 1 to 10 (with 10 being the highest), which of the following activities tend
most to disrupt good checklist procedures, If they are particularly disruptive at one or another phase of
operation, please indicate at which phase(s) - (i.e., ground, climb, cruise, ce or approach and landing).

score
a. Ground personnel communications 3.05(2)
b. Company radio 3.06(7) |
‘c. Flight attendant requests 4.4 3) 7
d. ATC communications 3.4 (1)
. Crew conversations 2.4 (9)
f. Navigation requirements 2.4 (9)
g. External taxiing distractions 4.25(4)
h. Configuring aircraft for departure 2.09(10)
i, Extemal inflight distractions 2.82(8)
j. Configuring aircraft for approach 3.27(6)
k. Aircraft abnormalities 4.065)
1. Any others

D-3


AIRCRAFT OCCURRENCE
TYPE

127.

128.

129.

130.

131.

132.

133.

134,

135.

WDB

MLG

WDB

MLG

WDB

WDB

MLG

Inaccurate navigation, deviation from
assigned track

Altitude overshoot during descent

Both engines shut down at 1500” in
climb, restarted and continued flight

Altitude overshoot during approach

Engine failure in cruise, | declared
emergency, landed at nearest
suitable airport

Cargo compartment fire, emergency
not declared since aircraft was on

final for landing, did declare an
emergency on the ground with a
passenger evacuation

Engine flame-out when throttles were
retarded for descent

Unable to control cabin altitude,
descended to control

Aircraft with incorrect fuel
load, had to make a fuel stop

C-18

CAUSE

FMS programmed. improperly, should have been
caught on review of programming for checklist

Crew busy handling communications with company
and ATC, doing PA announcements, running
checklists, set wrong altitude into the altitude
reminder

did not use the checklist for an abnormal
annunciator light, used the wrong switches to solve
the problem, no crew coordination

Controller cleared the aircraft to 3000’, thought he
had cleared them to 4000’, they got busy doing
checklists and other duties and descended to 2600”

Shutdown due to high EGT and low EPR, used
appropriate checklists

Illegally shipped hazardous cargo, crew indicated
that with a two-man crew in this of situation,
trying to fly the aircraft, do checklists and
everything else, one person is “out of the loop”
trying to get information on the problem and the
other person is left to do everything else

Proper checklists used including restart checklist,
successful restart, problem caused by bad bleed
valve which is in the process of modification
fleetwide

Engine start switch in the wrong position for pack
operation, should have been caught on the after
starting checklist

During predeparture checklists the crew was
distracted by on board FAA inspectors, didn’t
check fuel properly


Ina drill, at an informal meeting with the NTSB, a
DHC-8 Captain was asked to locate the “ENGINE
FIRE” checklist in the Horizon booklet. He was
unable to do so in a reasonable amount of time. This
inability to locate critical checklists is perhaps one
Teason why the “ENGINE FIRE” checklist was
never completed in the Horizon accident.

3.5.6 SUMMARY oF FinpINGs

Apart from paper and laminated card, no checklist
devices were found which were easily adaptable to
all aircraft types. And, one respondent to the ALPA
survey commented that the aircraft he flew didn’t
even have a place to stow them.

As far as we can see, no manual device currently in
use has the potential, by itself, to entirely eliminate
pilot error in the use of checklists,

4. SUMMARY AND RECOMMENDA-
TIONS

This includes a summary of the data gathered and
recommendations for improving checklists.

41 FINDINGS

4.1.1 ConFoRMANCE

Twenty of 21 NTSB reports illustrate that lack of
conformance with standard operating procedures
may be as big a problem as checklist layout and
design, if not bigger. Forty-three percent of the
ASRS reports indicate thata lack of training contrib-
uted to this lack of conformance.’ Comments by
ALPA support this indication, We observed an
instance of this during one of our jumpseat rides
where the crew did not read their“BEFORE LAND-
ING” checklist.

The inconsistent application of policies and proce-
dures for checklist use may also adversely affect
conformity. Some operators were very specific in
the guidance they gave their crews, others gave no
direction on either policy or procedures for checklist
use. The latter were frequently vague as to who
challenges, who responds, and when.

4.1.2 InrerRurrions

Fifty-eight percent of the ASRS reports mentioned
interruptions as being the cause of problems in
checklist use. The interruptions fall into two catego-
Ties:

+ Extemal interruptions to the crew during their

Ly

use of a checklist.

+ Interruptions to operational tasks caused by
using a checklist.

The ALPA survey confirmed the disrupted
and disrupting aspects of checklist use and its
implications for flight safety.

We also observed that operations activities
often led to checklists being done from
memory, responses being given without the
corresponding action being taken, and check-
list items being missed. Similarly, ourcockpit
observations revealed that diligentuseofcheck-
lists by flight crews while taxiing could easily
detract from the safe operation of the aircraft
on the ground.

4.1.3 Cxeckuist AND MANUAL DEsIGN, ORGANIZA-
TION, AND CONTENTS

Missing, inconsistent, and incorrect procedures were
said to contribute to 20% of the problems in the
ASRS reports. In fact, we found many of these
problems in our review of Part 121 and Part 135
Operators’ manuals and checklists. And many of
these manuals and checklists also lacked organiza-
tion and the completeness needed to support in-
formed use by flight crews. The manuals and check-
lists provided by large U.S. manufacturers were
usually more organized and easier to use than those
from foreign or small U.S. manufacturers, The lack
of organization and clarity in the manuals and check-
lists from the smaller and foreign manufacturers
often presented a problem for regional carriers fly-
ing the smaller, commuter-type aircraft. However,
even the manuals and checklists from large U.S.
manvfacturers suffered at times from changes made
by the operators. This resulted in an end product that
was no better, and occasionally worse, than what
was available to small carrier crews.

Examples of the problem found included the follow-
ing:

* checklist procedures not in the order in which
they should be used;

+ items missing from checklists and/or not car-
tied over from the AFM;

* procedures specified in the Airplane Flight
Manuals (AFMs) inconsistent with actions
Prescribed in the operating checklists;

23



AIRCRAFT OCCURRENCE
TYPE
79. LRG Aircraft ized requiring
of rapid depressurization and
explosive depressurization checklists
and diversion to a nearby field

Aircraft declared an
climbout and returned to

use

cy on

81. MLG In climb the aft cargo door light
illuminated, unable to pressurize,
continued to destination and landed
82. MLG Aircraft unable fod control
pressurization, hom sounded, masks
dropped, emergency declared

Didn’t control cabin altitude, got
passenger oxygen masks, recovered

ization, continued to
destination climbing above 25,000’
illegally (due to no availability of
automatic oxygen mask presentation)
to avoid weather

83. MLG

84. LIT Near mid-air collision, took evasive

action

85. SMA Aircraft landed gear up after an

aborted landing and go-around

86. MLG Loss of pressurization and emergency
descent

87. MLG Jetway shifted causing minor aircraft

damage, blamed on aircraft rolling

88. LRG Near overtemp on starting engine #1

89. MLG Aircraft rolled forward on engine

start, brakes applied suddenly causing

flight attendants to fall with two
sustaining minor injuries

Damage to aircraft tow bar during
pushback

CAUSE

Cracks in the cabin in the wheel well area probably
due to aircraft age

Engine loss on climbout with use of emergency and
normal checklists

Cargo door light not noticed during ff
checklists, continued due to below ‘gas ing
minimums at departure point

Loss of pressurization, cause unknown, used
emergency checklists and lures, continued to
destination at lower altit

Bleed switches not on and not noticed out of the
proper position on the checklist

Busy doing checklist for descent and both had
inside the cockpit, although under positive
control, the controller didn’t point out the traffic

Too much float on a hot day, went around. Didn’t
put down for second approach, did a GUMP
check and missed the gear, gear hom didn’t work
because of high approach power setting

Lost both packs simultaneously, used emergency
checklists and descent, donned oxygen masks, both
packs came back on the line, continued to
destination, cause unknown

Brakes were set per the securing checklist

At a stop on a through flight maintenance had been
working on a thrust reverser problem, start levers
had been left in idle rather than cutoff during the
work, this was not caught prior to start since "start
levers to cutoff” is not on the before start checklist
on a through flight

Brakes not set during checklist, chocks pulled by
ground crew without informing cockpit crew, non-
standard procedure for use of parking brakes prior
to engine start

Abnormal start due to APU electrics inop, no
specific checklist to cover, used normal flow
pattern during an abnormal start

SECOND GROUP OF REPORTS FOLLOWS ON PAGE C-9

C8


3.2 ASRS REPORTS SUMMARY

ASRS reports provide a rich source of information
regarding problems in aviation. They are submitted
ona voluntary basis by pilots, controllers, and others
in the operational side of the industry. Because
submissions are voluntary, the contents of this data-
base should not be considered representative enough
for use in describing all errors and problems that
occur in the cockpit. The crews report the problems
that they want to report. Nevertheless, there is no
reason to doubt that the problems that are reported
did in fact occur.

Those submitting reports are asked to identify them-
selves for purposes of phone contact by ASRS for
amplifying information; however, all reports are
deidentified shortly after being received. The re-
Ports are available for research on specific subjects.
We requested reports on any occurrences involving
checklists over the past five years. We received
summaries of 195 reports that were relevant to our
study. A summary of each of those is included in
Appendix C. The following "shows categories of
errors made and gives examples of each.

* Sixty-five were cases of checklist items being
missed or incorrectly performed by the crew:

- Engine flamed out at altitude from fuel
exhaustion. Declared anemergency.Crew
had not tumed on all boost Pumps as
instructed in the checklist.

- Control lock still installed on the yoke
during takeoff. Aborted flight 40’ inthe air
after noticing lack of control response.

- Altimeter mis-set by 1", not checked by
Crew, altitude overshoot on short final,
wamed by the GPWS.

* Tenhadnothingonthe “before landing” check-
list to accomplish the required action:

- Aircraft landed with fuel badly out of
balance limitations, no item on the check-
list to check fuel pump configuration.

- Altitude undershoot in climb. The resetof
the altimeter at 18,000' to QNE (the set-
ting of altimeters to 29.92 at 18,000 feet
and above) was not on the checklist.

+ Eleveninvolved poorly designed checklists or

manuals:

- Checklist called for throttle to be pulled
out 1/2" on start, whether engine was hot
ornot. On start, the pilot could not control
the plane and hit the fuel pump (the throttle
Should be closed for hot-engine starts).

- Altitude overshoot on climbout. Check-
listprocedure hasaltimeters reset at 10,000
in the climb - far too late when assigned
altitude is below that.

* Six had no checklist to use:

- Aircraft failed to pressurize because nei-
ther air conditioning pack was function-
ing. No abnormal checklist was available
to cover that condition (this wason a wide
body airplane).

- Aircraft landed gearup. No checklist, and
the pilot didn’t use a GUMP check.

¢ Twenty indicated that the appropriate check-

list was not used by the crew:

- At 1,500'inclimb, an experienced Captain
Cut the fuel to both engines (two-engine
aircraft) in response to an annunciator
light for right engine EEC. Copilot (PF)
reported that the Capt. did not refer to the
abnormal checklist or coordinate with
him prior to the action.

- Crewlooking forunfamiliar airport, didn’t
do the final checklist, and landed gear up.
Waming hom didn’t sound until the flare
~ too late.

* Seventy-four showed poor crew coordination

in the use of a checklist:

- Engine shut down needlessly in flight
during performance of electrical abnor-
mal checklist procedures. First Officer
started APU for backup - Captain saw the
low oil pressure light at APU start and
mistook it for an engine low oil pressure
light, shutting down the engine. First Of-
ficer didn’t inform Captain of starting the
APU, and Captain didn’t confirm engine
low oil pressure with First Officer before
shutting down the engine. Emergency



CRT

Scroll

- eve Ny me

« Positive check on checklist gress for
those lists on the mechani portion

» The lists on the mechanical device can
be interrupted without losing track of
Progress

- Easy to use and move around as the
checklists are done

- Easy to stow
Inexpensive to reproduce

Inexpensive to update

+ Tough and hard to destroy

. Difficult to mark on and mess up

. Fairly easy to stow

. Remains legible longer than paper
checklists

- Can’t lose checklists

- Can present systems schematics in the

case of “Abnormal” or "Emergency"
checklists

. Color-coded for ease of use
. No stowage problem

+ Permanent fixture - can’t get lost

- Promotes "heads-up" posture

. Relatively easy to make changes to
checklists

. Stows out of the way on the glare shield
5 Easy to mark progress

+ Groups all checklists together - including

the "Abnormal" and "Emergency"
checklists

- If properly tabbed, makes it easy to find
any needed checklist

P wn

2.

. Slide or switch/light combination takes up

Disadvantages

- Necessitates the use of two sets of lists

cockpit real estate

. Easy to mark on and mess up
. Becomes worn easily
. Eas} to misplace or remove from the

airplane

. May be difficult to use under Poor lighting

conditions

+ More expensive to produce than paper lists
. Bulky in comparison to a folded paper

checklist

. May displace another display such as radar
- Requires a lot of "heads-down" time

. Takes up cockpit real estate

. Can be cumbersome to find a list or go

back to a point in a list

+ Can be hard to read (size of print and

distance from the viewer, and some are not
lighted at night)

Difficult to go back to a prior item on a
checklist

. Can be bulky on aircraft with a large
‘number of lengthy checklists


mistakes. Rather than dedicating chunks of
time to checklist use, Many crews perform
these lists concurrently with other flight tasks,
About 1/3 of those who Tesponded that they
found themselves doing checklists at times of
otherwise heavy workload said that they con-
tinued with the checklist as they did other
tasks, completing checklistitemsasthey found
time.

+ Emergency checklists are often not easily
located when needed. It was suggested that it
be made mandatory for them to be carried ina
Teadily accessible place in the cockpit, rather
than within a manual in a flight bag.

3.5 OTHER SOURCES OF INFORMATION

3.5.1 NTSB anp RELatep Meerincs

We participated indiscussions with an investigator
for the NTSB and representatives of a tegional Part
121 carrier who were developing anew checklist for
a foreign manufactured aircraft that they had in
service. The carrier’s People expressed their con-
cems with the manuals and checklists that are avail-
able for use with the . foreign manufactured aircraft
that they are operating. We subsequently reviewed
the AFMs and checklists for those aircraft.

One aircraft type had an AFM that covered the
information required by the FARs; e.g., Limita-
tions, Emergencies, and Performance (the greater
Part of the manual was devoted to performance).
There was also a Normal section which encom-
passed “Normal” and “Abnormal” checklists, No
systems descriptions were included. Other concems
and problems that this operator expressed included
the following:

* One AFM contained 82 checklists for abnor-
mal and emergency situations, Of the 82, 39
were “Emergency” checklists. Many of the
39, such as “UNPRESSURIZED FLIGHT,”
would not have been classified “Emergency”
checklists by many U.S. manufacturers or
airlines, However, the operators are constrained
touse these checklists as they stand, with their
‘Tnultiple memory items, which put a heavy
memory load on their sometimes low-experi-
ence-level crews, We quote from an Advisory
Notice from the manufacturer pertaining to
these checklists:

“Operators are reminded that abbreviated

20

checklists (as opposed to lengthy, detailed
expanded checklists) are not published by
as a document approved by an
Airworthiness Authority and, if they are to be
used,.they must comply at all times with
Current procedures as set forth in the latest
Tevision of the Approved Flight Manual.”

FAR 125.75 states that“...the certificate holder
dures and modified performance data presen-
tation are approved by the Administrator.”
This regional operator told us, however, that
they had little luck trying to modify these
manuals and checklists. Whether due to poor
Operator modifications or reluctance on the
Part of the POI to allow change, we don't
know.

. This aircraft, since itsmanufacture (4+ years),
has had an average of 300 modifications per
year. Some of these modifications involve
major hardware changes or procedural changes
that necessitate checklist changes. Because of
the volume of changes, the operator has found
it difficult to modify the aircraft, keep their
crews adequately informed, and make timely
changes to manuals and checklists which then
must undergo POI approval.

3.5.2 Am Transport AssociATION (ATA) Fucar
Crew Cuecx.ist Workinc Group MEETINGS

The ATA hosted a working group on checklist and
manual design to work with the FAA in developing
guidelines for use by POIs in evaluating Part 121
and Part 135 manuals and checklists. This group
was assembled to provide the FAA with industry
input for the checklist and manual section of the
Draft Inspectors’ Handbook. We were invited to
Participate,

Prior to the two meetings that we attended, we met
with the FAA member responsible for writing this
section of the Handbook. We Provided him with
data we had found on recent MIL SPECS which
Provided guidance in manual and checklist con-
struction (MIL-M-7700C, 18 May 1989, MIL-C-
81222C [AS], 22 Feb. 1978, MIL-C-27278B, 5 July
1973). In addition, we advised him of checklist and
manual problems that we had encountered in meet-
ings and discussions with airlines, He, in tum,
provided us with the results of the first Flight Crew
Checklist Working Group meeting, which we had
missed. This included the Progress to date on the



pit. In the absence of cockpit research
dealing specifically with this issue, we
recommend, in “Guidelines” (, i:
A) that the checklist body be 10-point
type, boldface, all caps, and that the check-
list title be 12-point type, boldface, all
caps.

- To the greatest degree possible, should
have no greater number of items than can
be presented on a single checklist card and
can be easily read and stowed ina readily
accessible place in the cockpit.

. “Emergency” checklists should be quick to
access and easy to use under stressful condi-
tions. They:

- Should be quickly accessible in the cock-
pit by both the Captain and First Officer,

- Should be available on a card (on the
reverse of the “Normal” checklist card if
possible) as well as in the manual.

- Should be in a standard format. The order
in which the emergencies are Presented
on the card should be standardized. This
should cover all aircraft types in a
company’s fleet, and should take a form
such as all engine problems first, or all
fires first, etc., (to be decided by each
company). In this manner, a crew flying
fora Particularcompany will know where
to look for individual checklists regard-
less of what aircraft they are flying. In
addition, the order in which the Proce-
dures are presented for each emergency
should be standardized to the greatest
degree practical, Particularly within type.

- Should have a clearly defined start and
finish with a title set off by type two sizes
larger than that of the text, boldfaced, and
all caps, Each list of procedures should be
Clearly separated from other lists. This
should facilitate quick identification un-
der conditions of stress and low illumina-
tion,

- Should be composed of type no smaller
than that of well-designed “Normal”
checklists, and if space permits, larger.
“Emergency” checklists are often used

undercircumstancesofenvironmental and
Psychological stress, and consequently
should be as readable as possible,

- Should be easy to understand and execute,
Each “Emergency” checklist should be
composed of only those items needed to
combat the emergency. They should be
listed in the order in which they are to be
performed. They should be stated in com-
mon tenninology, in a positive manner,
and in as few words as can be used to
convey the action.

Subsequent procedures which must be
performed as a result of the emi
Procedure, (e.g., “SINGLE GENERA-
TOR” procedure after a generator loss
due to shutting down an engine as a result
ofanengine fire), should be covered inthe
expanded checklists in the manual.

42.2 MAnuats
° Procedures for checklist use:

- Should be clearly defined in the manual,
This should include clear direction as to
which flight officer reads what
and which responds, and should Specify
this for each phase of Operation; ie., air-
Plane stationary, airplane taxiing, airplane
in the air.

- Should require quantitative or
i iating responses forall appropriate
checklist challenges, Whenever possible,
responses should specify position or
quantity; _e.g.,  FLAPS....20,
FUEL.....48,000#, etc. The answer “AS
REQUIRED” should not be allowed.

- Should limit dual response items to the
highest priority safety critical items,

- Should require that checklists wom to the
Point of reduced readability be immedi-
ately replaced. No Minimum Equipment
List (MEL) delay should be allowed on
this item.

* Format requirements:

- Should specify a standardized table of



were told that they deal with “Abnormal” proce-
dures instinctively, from an ingrained habit, and
then refer to the Operations Manual kept in the
aircraft. Checklists are also not used intwo pilotIFR
flights, where each pilot knows the Standard Oper-
ating Procedure and follows it when underway.
Although we anticipated that we might find ex-
amples of the latest technology in checklists in these
visits, we did not. As noted above there was some
interest in automated checklists on CRTs, but forthe
most part more conventional types were the stan-
dard.

3.5.5 Cocxrrr Devices In Usk

In order to determine whether there was some new
technology available which could be easily adapted
to general use, and could help to eliminate checklist
errors, we did a small survey of what was available.
From the results of this survey, we have listed
advantages and disadvantages of the various kinds
surveyed (see Appendix B).

The automated checklist on a CRT is liked by many
of those who use it. Some who use it on a regular
basis and report favorably onit also report that it can
take more heads-down time if anything unplanned
or out of the ordinary occurs. Others report it as too
cumbersome and use paper or laminated checklists
instead, even when the other technology is avail-
able. In some cases, it usurps the radar CRT. Many
aircraft would require a very costly retrofitto enable
the use of this technology.

The checklist on a scroll has been around for many
years, and is still used enthusiastically by many,
including crews of some Air Force planes in the
Current inventory. It can be cumbersome to use if
one needs to retum to a priorportion of the checklist.
It also takes up cockpit space, which is in short
supply in many aircraft. In addition, itneeds a paper
checklist backup in case of mechanical failure. One
Corporation we visited used scroll checklists that
were generated on a personal computer with a dot
matrix printer — not the best combination for leg-
ibility. Their checklists did not require approval
from a POI since corporations operate underPart91,
and this allowed them to make changes as they saw
fit. Their preflight checklist contained 129 items,
and other checklists also seemed excessively long.

By far the most prevalent types of checklists are
Paper or laminated paper. They come in various
sizes and shapes, some big and unwieldy, some so
small as to be unreadable except in perfect condi-

22

tions. One major problem with these is the ease with
which you can lose your place through interrup-
tions,

We observed that mechanical checklists are used for
“BEFORE TAKEOFF” and “BEFORE LAND-
ING.” Theirusers like them since they are a positive
measure of checklist progress. The other “Normal”
checklists that the crews use are paper or laminated
cards,

We have seen one example of a unit which reads the
checklists to the user in a synthesized voice. It will
restate missed items until they are complete, if
Programmed to do so. As far as we know, it is
currently only in limited use, with some corporate
Part 91 operators. One major airline is considering
doing an evaluation of this technology with an eye
to possible use. One drawback that we can foresee is
the addition of another noise in cockpits which are
already noisy enough.

Some users kept all checklists in booklets in the
cockpit. Some checklists were partially laminated
throughout, some were in plastic sleeves. Those that
were well tabbed and indexed were easy to use. One
of the best examples of these was the checklist
booklet from Flight Safety Canada, Inc., for use in
the Canadair Challenger 601. This included color-
coded, laminated tabs, well-indexed “Abnormal”
and “Emergency” sections, and heavy, hard-fin-
ished paper pages with 10-point type or larger. It
was easy to use and very legible. Moreover, the
aircraft for which it was designed had a convenient
Storage slot for it; its compactness would make it
easy to adapt other aircraft to accommodate it.

‘The worst example we saw was that of the checklist
booklet from the Horizon DHC-8 involved in an
accident at the Seattle-Tacoma Intemational Air-
Port, on 4/15/88. It was printed in eight-point type,
mixed case (sometimes all lowercase), and not good
quality of print. The tabbing can best be explained
by quoting from the NTSB “Human Performance
Investigator’s Factual Report” of the accident:

“Locating a specific checklist requires the user
to identify the desired checklist in the table of
contents, note the number of the divider at
which the checklist is filed, and tum to the
desired checklist whichis inserted before (for-
ward of) the numbered divider.”

4+

ee



10.

11.

12.

13.

14.

AIRCRAFT OCCURRENCE

TYPE

LRG No nose wheel steering, had to be
towed off runway

SMA Gear up landing

SMT Unauthorized entry onto runway

MLG Unauthorized runway crossing

MLG Possible traffic conflict, early turn to
SID heading

LIT Aborted takeoff

LRG Departed 10,000 Ibs. light on fuel,
retumed to airport

MLG Unable to pressurize after takeoff,
emergency declared

WDB Altitude excursion and request for

immediate turnaway from weather
because of loss of F/O altimeter,
flight instruments, and radar

WDB Deviation from assigned SID, started
to fly the wrong SID

LRG Crossed hold-short line but didn’t
quite have a runway incursion

MLG Abnormal lights on takeoff, engine
fire warning after takeoff, crew
continued to destination

MDT Altitude overshoot in emergency

LIT Total electrical failure with
emergency battery activation, spoilers
were deployed and would not retract,
diverted to longer runway for landing
and blew main gear tires on landing

CAUSE

Use of emergency and normal checklists - missed
one item on the “descent” checklist

No written checklist available - interruption from
pilot-passenger

Busy finishing checklists and misheard "clearance
on request” for “cleared on course”

Busy running checklists, poor crew coordination
Reading checklist instead of paying attention to
SID, poor crew coordination

Didn’t turn on water injection system for takeoff,
poorly designed checklist item, lack of
understanding of standard procedures

Busy doing checklists and no one verified the
proper fuel loading - lack of clear procedures for
lers to use and crews to verify proper fueling

Pack switches not on, checklist item not
accomplished, also not caught by the F/O on the
quick check prior to declaring an emergency, found
subsequently

F/O flying, Capt. and S/O doing an abnormal
electrical checklist, one part of the procedure
knocked off the F/O instruments and radar at the
time they were to penetrate a line of weather

Confusion during time of reading checklists prior
to takeoff and receiving runway and SID
sesignment changes without programming in the
FM!

Too busy with short taxi distance, unfamiliarity
with taxi route, and amount of checklist to be
accomplished

Engine fire bell went out and all engine indications
normal, had been prior work on and abnormal
lights for bleed air problems, did “air cond. supply
temp hi" checklist, later maintenance found a 1"
hole in the engine due to starter reengaging

Loss of pressurization, emergency descent, trying
to control cabin altitude and do emergency and
abnormal checklists and get clearance from center,
"1,000 ft. above" didn’t get called

Bad freon air-conditioner installation resulting in
power loss, used emergency procedures


Clearly, the use of checklists in the cock-
pit is required for safe operations. Just as
Clearly, they must be used in an environ-
ment thatis disruptive and promotes error
in their use. At the same time, checklist
use is an important contributor to cockpit
workload. Checklists that are easy to read
and use will be more resistant to error and
will contribute less to cockpit workload
than those that are not.

3.4.5 CoMPLIANCE, CREWMEMBER VARIATIONS, AND
Cockprr Resource MANAGEMENT (CRM)

One issue that surfaced during the survey was that of
crew compliance. One respondent commented:

“Checklists are not that important. A bad crew
can screw up a good checklist. A good crew
can work safely with any checklist.”

Other comments included were:

+ “Checklists are mandatory for safety. How-
ever, they are only as good as the persons

+ “Personal discipline seems to be the major
variant.”

+ “Don’t give into complacency - it’s our big-
gest foe.”

Though the overwhelming majority indicated that
their airlines prescribed methods of checklist use
and their crews adhered to them, 72.6% also felt that
individual crewmembers influenced the manner in
which checklists were performed. Sixty point five
percent felt that this resulted in variations in check-
list performance, and 43.6% felt that this meant
checklists were done in a nonprescribed way, or
were not done. There appears to be an inconsistency
in these responses. Although stating that most crews
followed prescribed procedure, they also felt that
individuals had a great influence on the manner of
checklist performance. The following comments
shed light on the state of cockpit resource manage-
ment and crew coordination:

* “This (the lack of standard use or nonuse) will
be difficultto correct until the attitude of those
individuals is changed.”

* “Ourcaptains are so nonstandard that the First
Officer’s job is much more difficult. Our air-

17

line provides us with basically good proce-
dures and checklists, but the captains (particu-
larly the older pilots) refuse to use them.”

+ “Some captains continually fail to call for
checklists, leaving it up to the other
crewmembers to be a little aggressive and ask
if they’re ready for it (the checklist).”

When asked if their airline had a policy of Cockpit
Resource Management (CRM), 73% of the respon-
dents indicated that their airline had a definite policy.
The following comments are representative, al-
though contradictory.

+ “Most ‘old heads’ don’t even understand the
concepts in CRM, they are from the school of
Zeus.”

+ “Our airline has a very good standard operat-
ing procedure. Even though the Captain has
the ultimate authority, all crewmembers are
encouraged to actively participate in cockpit
Operations and not hesitate to voice their con-
cers regarding irregularities or any sort of
‘judgment’ call.”

These two respondents are apparently from differ-
ent airlines, which espouse different philosophies
on CRM. One appears to have a strong, definite
policy which has been impressed on the crews, the
other either no CRM policy, ora policy which is not
being followed.

3.4.6 CHECKLIST ACCESSIBILITY

When queried about the checklists they currently
use, 31 (35.6% of those who answered the question)
felt that their “Emergency” checklists were not easy
to locate when needed.

+ “I would have to dig into my flight bag for
emergency checklist handbook.”

« “Emergency checklists should be red for all
fleets/airlines (color coding) and should be

mequired by FAA io be readily accessible
(emphasis added) - not in binders in flight
bags.”

« “BAe-146 needs a place to stow both ‘Nor-
mal’ & ‘Emergency’ checklists.”

+ “Abnormal/Emergency inmanuals... difficult
to find.”



ported on very favorably by the pilots using them
during our cockpit observation on that airline.

One Part 121 airline used paper checklist cards for
“Normal,” “Abnormal,” and “Emergency” check-
lists, and stowed them all in the cockpit. The size of
the paper checklist cards studied varied, and is
important only in that itmust be large enough to hold
legible checklists, and small enough to be stowed
readily in some location in the cockpit.

Those studied ranged from a fourfold 10 7/8" x 5 1/
2" to ano-fold 8 1/2" x 11.” The former was very
crowded and difficult to read, whereas the latter was
very legible. In some cases, the large cards designed
to be no-fold were observed to have been folded by
the crews, presumably for convenience.

Most of the carriers kept their “Abnormal” and
“Emergency” checklists in manuals or booklets of
some sort. All of the Part 135 manuals studied, and
some of the Part 121 manuals, lacked tabbing for
quick reference and easy identification. This lack of
tabbing could provide an added impediment to a
crew at a time when they are already dealing with a
situation other than normal. The use of a booklet,
capable of being stowed in the cockpit, is preferable
to a manual stowed in a flight bag from the stand-
point of accessibility. Handier yet would be a sepa-
rate card of “Emergency” checklists stowed in the
cockpit.

If a booklet or a manual is to be used, it should be
properly tabbed for quick reference. Each major
section should be tabbed with the name of the
section, and each subject within a section tabbed to
correspond with the appropriate subject shown in
the section index. The section index should be on the
first page of each section, following the tab. If the
manual contains a section on aircraft systems, there
should be a tabbed subsection for each individual
system, (¢.g., engines, flight controls, etc).

3.3.4 Cotor Coping

Two of the Part 121 carriers, and three of the Part
135 carriers used color coding for easy identifica-
tion of “Abnormal” and “Emergency” checklists.
There have been instances cited in ASRS reports in
which crews have had difficulty in locating “Emer-
gency” checklists. Human factors research indi-
cates that color coding can be effective in helping to
identify emergency checklists. Advisory Circular
25-11, dated 7/16/87 recommends red be used for
the most serious conditions, and yellow be used for

abnormal conditions of a less immediate nature.

3.3.5 MEMory IEMs

Memory items on “Emergency” checklists have
been a point of difference in corporate philosophies
for years. Of the Part 121 “Emergency” checklists
reviewed, all had some form of memory items; those
items which the crew must commit to memory for
performance in an emergency situation, to bring the
emergency under control before referring to the
checklist. One major carrier, which was not in-
cluded in our study, has adopted the philosophy that
memory items are not only not necessary, but may
precipitate a mistake through too much haste. They
have eliminated memory items from their “Emer-
gency” checklists, and instead use them as lists from
which to work. This is not the case with most
carriers. They range from having memory items for
all the initial steps in all the “Emergency” checklists
toa very limited number of items on a small number
of checklists. The former is more common. The
latter is represented by one of the Part 121 operators
in our sample. Only three of their “Emergency”
checklists contained memory items: “ENGINE
FAILURE,” “ENGINE FIRE,” and “ENGINE
TAILPIPE FIRE,” and each list contained only one
memory item. In all three cases the item was the
same, “THROTTLE,CLOSE. CLOSE.”

The Part 135 carriers were apparently not much
different from the Part 121 carriers in this regard. Of
the 10 studied, eight used memory items. One did
not require them, and the tenth provided no “Emer-
gency” checklists for study.

3.3.6 MANUAL AND CHECKLIsT CONTENTS AND ORGA-
NIZATION

The Part 121 carriers generally exhibit more legible
and professional-looking checklists and manuals
than their Part 135 counterparts. However, there is
still room for standardization and improvement.
Despite the generally high quality of professional
standards and performance of Part 121 scheduled
carrier pilot groups, there have been many instances
of lapses in checklist use, some with catastrophic
results. If minimum standards for legibility, acces-
sibility, and quick recognition were adopted, the

availability of a checklist easy to read and use would

discourage checklist misuse, whereas lack of stan-
dards in the past has contributed to this misuse.
From that point it would become a question of
airline training and discipline, and individual pro-
fessionalism.


Included in this standard terminology should be a
dictionary of abbreviations to be used whenever
abbreviations are needed. To quote MIL-M-7700C:
“The glossary of each manual shall contain a list of
the abbreviations used in the manual, except for
normally accepted and understood abbreviations
such as ac, dc, and rpm.” Although the MIL SPEC
mentions “manual” specifically, the same would
apply to checklists, since they derive from the flight
manuals. In MIL-M-7700C there is a list of ap-
proved abbreviations, and MIL-STD-12D is dedi-
cated to abbreviations. Some of them are different
from those used in civilian aviation, but a lexicon for
standardization would resolve these differences and
create a set of abbreviations, with a basis in the MIL
SPECS, for industry use.

We feel that in the interest of standardization, and to
ease Crew transition from one aircraft type to an-
other, alexicon of common tems and abbreviations
must be developed.

CLARIFICATION OF “NORMAL," “ABNOR-
MAL,” AND “EMERGENCY”

There must be clear definitions of what are to be
regarded as “NORMAL,” “ABNORMAL,” and
“EMERGENCY.” The manufacturer of one im-
Ported aircraft flown by the regional airlines in-
cludes 39 “EMERGENCY” checklists out of a total
of 82 checklists. An example of one checklist clas-
sified improperly as an “EMERGENCY,” in our
opinion, is “UNPRESSURIZED FLIGHT.”

One set of definitions of “ABNORMAL” and
“EMERGENCY” has been created by Flight Safety
Canada, Ltd.

“EMERGENCY PROCEDURES’—“This
section deals with foreseeable but unusual
situations in which immediate and precise
action may be required by the crew.”

“ABNORMAL PROCEDURES”—“Proce-
dures in this section address foreseeable situ-
ations involving failures, in which the system’s
redundancy or selection of an alternate system
will maintain an acceptable level of airworthi-
ness.”

In MIL-M-7700C there are definitions for“WARN-
INGS” and “CAUTIONS” which could be bor-
rowed for “ABNORMALS” and “EMERGEN-
CIES.”

“WARNING” —“Operating procedures, tech-

niques, etc., which could result in personal
injury or loss of life if not carefully followed.”

“CAUTION” —“Operating procedures, tech-
niques, etc., which could result in damage to
equipment if not carefully followed.” To the
latter, we would add, “and if not carefully
followed, could eventually lead to personal
injury or loss of life.”

The Flight Safety definitions are not as strongly
worded as the ones in the MIL SPEC, but do convey
the sense of urgency, nonetheless. A combination of
these definitions would satisfy the need to provide
Strict guidelines for use by aircraft manufacturers
and airlines in the preparation of aircraft flight
manuals and checklists.

MANAGEABILITY OF CHECKLISTS

Paper checklists should be of an easily used and
stowed size. We recommend in “PRINT SIZE and
STYLE” that card checklists be 8 1/2" x 11,” either
laminated or trifold, We also recommend, if pos-
sible in keeping with the recommendations on print
size and style, that there be a combination on one
card of “Normal” and “Emergency” checklists, One
group on one side of the card, one on the other. One
airline uses this combination, The combinationmakes
the task of location of needed checklists far easier.
However, in this case, the recommendations for
print size and style are not met.

To retain the recommended size of print we Tecom-
mend that there be two cards, one for “Normal”
checklists, and one for “Emergency” checklists —
color-coded for easy identification. These should
both be kept in the same, easily accessible place in
the cockpit. These two groups of checklists are the
ones that should allow ready access, The “Normal”
checklists are used all the time in daily operation.
“Emergency” checklists will not be needed on a
Steady basis, but should be immediately available
when they are needed.

It is normal practice with many airlines to keep
“Abnormal” checklists in the flight manual. Since
they are not needed on an immediate basis, this
access is adequate.

We recognize that these guidelines do not address
the concem of the proper use of checklists by pilots.
However, we feel strongly that if easily usable,
readable checklists are available to pilots, the ten-
dency to neglect or to misuse checklists may be
reduced,


FIGURE A-1. EXTREMES OF PRINT SIZE AND STYLE

GEFORE TAKEOFF (FIMAL ITEMS)
Te Vindshret ig eat - OF
- Pilot Mest - On
- Tran - a
+ O11 Cooler Flaps - CLOSED/TEMPS MORAL
= Ligne - AS REQUIRED

BER nun
nl
B
E
8

BEFORE TAKEOFF {FIRAL ITS)
. ate

e
2. Pilot Meat - ON
3. Transponder - ON
4. O41 Cooler Flaps - CLOSED/TEMPS MOSAL
5. Lights - AS REQUIRED
CR6. Ice Protection - AS REQUIRED
CR7. Flow Selectors - OFF
CRB. Speed Levers - 100% WHEN CLEARED

A3



randomly selected and so are not assumed to be
representative of what is used in the industry. They
are, however, examples of materials in daily use by
major carriers.

3.3.1 Poxicy anp Procepurss ror Cuecxuist Use

Allofthe Part 121 operators studied specified some
policy regarding the use of checklists for their crews
to follow. Some had very specific guidelines regard-
ing who was to read each checklist, by what phase of
flight it was to be accomplished, in what manner it
should be read (e.g., challenge/response or silent),
whether with single or dual response, and what
responses should be given in lieu of “CHECKED”
or “AS REQUIRED.” Others only used phrases
such as “Checklist use is mandatory.”, and “Safe
operating procedures are not overlooked while giv-
ing attention to the checklist.” Still others merely
Specified who should read each checklist and at
what phases of flight they should read it. One
example of this is the airline specifying that the First
Officer should read all “Normal” checklists while
the aircraft is stationary, and the pilot not flying
should read all “Normal” checklists while the air.
craft is in motion.

Of the Part 135 operators, only one did not have
some sortof policy forthe crews to follow. Theother
Policies ranged from numbered notations on each
checklist margin as to who should answer each
challenge, to the very detailed and explicit direc-
tions from one of the carriers to their crews. Their
Policy statements were as good as some of the larger
carriers, and better than others.

One carrier was unique among all the carriers stud-
ied in that it specified that its “Normal” checklists
were to be used as “work” lists rather than “done”
lists. Rather than the items being accomplished and
then checked for completion by the use of the
checklist, it specified that the challenge be read, the
item be accomplished, and then the Tesponse be
given, indicating accomplishment. While this is
Sometimes the case with “Emergency” checklists,
and often the case with “Abnormal” checklists, it is
not usual with “Normal” checklists.

Three issues arise with policy and procedures for
checklist use. They are:

* When should checklists be used?

The time achecklist is to be usedis spelled out,
in part, in the name of the checklist; eg,

“BEFORE TAXI,” “BEFORE LANDING,”
etc. Some of the carriers in their policy state-
ments are even more specific; prescribing in
what phase of flight, and at what point in the
phase of flight a checklist is to be read. In a
number of the cases we studied, however, this
was left to the pilot.

Who should read/respond to the checklist
items?

This was handled by the airlines ina multitude
of ways. Some addressed the issue with a
detailed policy statement stating which pilot
should read which checklist and which pilot
should respond. Others made a margin nota-
tion oneach checklist with anumber designat-
ing which pilot was to respond. Others did not
address the issue.

Another point in this issue is that of dual
Tesponse. This involves items which must be
checked and responded to by at least two
crewmembers, frequently at busy phases of
flight, some airlines have items to which all
members of a three-person crew must re-
spond. This creates a division of attention for
the pilot flying. Of the Part 121 carriers stud-
ied, most used some dual response items in all
“Normal” checklists, whereas, of the Part 135
carriers, only one did. One of the Part 121
carriers limited dual response items to““GEAR”
and “FLAPS,” and then only on two checklist
Procedures; “FLAPS” on the “TAX?” proce-
dures list, and “GEAR” and “FLAPS” on the
“LANDING” procedures list. Limiting dual
Tesponse requirements to, one or two items
Teduces the amount of time when both
crewmembers have theirheads down, yet pro-
vides an additional level of attention to ensure
that the gear and flaps are positioned properly
for high-risk phases of flight,

How should the checklists be used?

This issue was not addressed by many of the
airlines. And those that did address it were not
always consistent, As an example, let us use
the checklistresponse““ASREQUIRED.” One
carrier did not use any “AS REQUIRED”
Tesponses on some of its aircraft, but did on
others.



declared with unscheduled landing.

- Aircraft taxied across an active runway
after instructions to hold short. First Offi-
cer got instructions, assumed Captain had
heard them and started doing the check-
list, heads-down.

- Early turn to a SID (Standard Instrument
Departure) heading with traffic conflict.
Crew busy reading the checklist and not
backing each other up.

¢ Eighteen involved the use of an incorrect or

incomplete procedure as prescribed by the
checklist:

- Aircraft departed 10,000 1bs. light on fuel.
New fueling procedure provided no clear
means of fuel load verification for fuelers
or crews.

- First Officer lost his instruments and the
radar as he was about to penetrate a line of
cells. Captain and Second Officer were
doing an electrical abnormal checklist
which knocked off the instruments and
radar.

* One-hundred thirteen involved an interrup-
tion or distraction, either from the use of a
checklist, from operational matters, or from
some extraneous event:

- Overshotaltitude by several thousand feet,
inexperienced crew busy doing the check-
list and working ATC radios.

- Altitude overshot on descent. Between
FL310 and FL180, crew had five speed
quently they had threemore speed changes,
two more heading changes, and three
runway changes - the last occurring at
400’ on final. The altimeter of the pilot
flying did not get set properly.

- Aircraft almost departed with a spoiler
extended. Crew taxiing with one engine
shutdown. Controller advanced theirtake-
off position. Rushing to complete every-
thing and missed the annunciator light for
the extended spoiler. Caught by crew ina
following aircraft.

(The percentages add up to more than 100% because
many samples involved multiple considerations.)

Since these reports are provided to NASA/ASRS on
a voluntary basis, information which would not
otherwise be available is provided about problems
in aviation. Although they may not be completely
Tepresentative of the industry, these findings help to
point out the variety of the problems encountered
with regard to checklist misuse.

SUMMARY OF FINDINGS
* CHECKLIST USE

In 43% of the reports studied the crew had
either not used the checklist at all, or had
missed important items on the checklist.

+ CHECKLIST AND MANUAL DESIGN

These factors accounted for 20% of the re-
ports, Design problems included items miss-
ing from checklists and inaccurate or incom-
plete procedures which could lead to poten-
tially dangerous practices.

* TRAINING

Thirty-eight percent of the reports involved
inadequate crew coordination. This could in-
dicate an absence of instructions in the AFM
or inadequate training in checklist use.

« INTERRUPTIONS

Interruptions accounted for 58% of the re-
ports. There was about an even division of the
following two types of disruptions:

- events, suchas ATC calls, interrupting the
crew’s use of checklists;

-  thenecessity to read a checklist interrupt-
ing an operational task, such as maintain-
ing a position in a departure queue.

3.3. PART 121 AND PART 135 CHECKLIST
AND MANUAL REVIEW

We reviewed six Part 121 operators’ and nine Part
135 operators’ manuals and checklists as one means
of identifying good and bad aspects of current air
carrier checklist practices. These materials were not


+ “Twould like to see a card(s) with the immedi-
ate action emergency procedures with their
none-memory [sic] reference actions in the
cockpit, so we wouldn’thave to be finding itin
a book at a critical, busy moment.”

3.4.7 Orxer OBSERVATIONS

Although almost 70% said that they had a personal
“must check” list which they used in addition to the
formal checklists, only about 1/2 felt this would be
useful to all front-end crews. Whether this indicated
that they felt this “must check” wouldn’t work with
others, or were reluctant to suggest imposing some-
thing else on other crews, was not clear.

Anumber used some form of memory jog to remind
them to complete some items ona checklist (such as
when taxiing with fewer than all engines operating).
Examples of this are a coffee cup inverted over the
flap handle, the checklist between the throttles, or a
“post-it” note on the windshield. However, 62%
said they just repeat the entire list. From the perspec-
tive of 21.5 years in airline cockpits, the writer finds
this difficult to believe. We think 20% would be
closer to the actual number.

When asked if their procedures were such that they
found themselves reading checklists during periods
of high workload, 62.5% said “Yes.” The manner in
which they coped with this is cause for alarm. While
many said they stopped the checklist until they had
more time, 30% said they “press on and hope that
nothing gets missed.” To again quote John Lauber
in his Flight Safety Foundation address - “Another
step involves the question of handling disruptions or
distractions, some of which are not under the control
of the crew, and others of which are. It must be
recognized that any disruption or interruption of
sequentially dependent tasks is associated with a
high probability that some or alll of the elements of
these tasks may be missed entirely, especially if a
significant amount of time passes during the period
of interruption. Thus, operating procedures should
explicitly state that any interruption to an ongoing
sequence of activities, especially running check-
lists, will automatically trigger a restart of the pro-
cess which was interrupted. Obviously, this has to
be done in a reasonable manner, but it should be the
dominant mode of operation for all pilots.”

Responses to one survey question indicate that most
crews follow the standard company procedures for
checklist use. However, when asked later whether
individual crewmembers influence the manner in

18

which checklists are used, a majority of the respon-
dents responded affirmatively. The following com-
ment is a case in point:

“Some two-man crews tend to abbreviate or
use silent checklists during high workload
times.”

Our own cockpit experience reflects the fact that
two-man crews tend to be less formal operationally
than three-man crews, and the above comment sup-
ports this.

The suggestion of a core checklist with allowable
variations for aircraft type and operating environ-
mentelicited mixed responses. The comments ranged
from negative, to advisory, to positive. Some com-
ments were:

+ “A large group of pilots will never agree on
anything.”

+ “Anind dard checklist will .
(This ties in with an ASRS report received
which cites a fleet with generic checklists. The
writer complained of illogical flow patterns
resulting from an attempt to accommodate
different aircraft types, and of PA announce-

ments on final approach.)

* “Would allow less confusion when moving to
different aircraft.”

* “This should be done with much input from
line pilots. Not supervisory types and inspec-
tors who do not have the experience. I’ve been
in both situations.”

The section requesting suggestions from the respon-
dents to improve checklists elicited many com-
ments, The following representative comments are
quoted as received.

+ “Keep them as brief and simple as possible.”

* “State of the art - electronic checklists with
throttle interlock (for critical items such as
gearand flaps) for T/O (takeoff) and landing.”
(Four of the respondents suggested some ver-
sion of this.)

* “Lastitemsonpre-takeoff: killer items double-
checked. Pan Am uses this.” These would



FIGURE 3-1. TYPEFACE SAMPLES

BEFORE STARTING ENGINES

1 CHECKED MD-80

EXTERNAL ELECTRIC & PMELMUTIC SOURCE - START

PRECMUTIC PREssums (25 1 Pal MUN)
COMPLETE - BEFORE START CHECKLIST

AFTER ENGINES STABILIZED
PREUATIC X-FELOS

FLT INSTR/SWITCHES/BUGS.

e KEK

gases

FUEL PANEL/QUANTITY AND

“SEV __ 4 cam
++ CKD
6 SET
* ‘ESET
* + OM
+ OM
. oN
cw
* oo
* (AS REQ)
* ar het rtd GUPPLY SvITCHES coats
: i saath wana SO Sa
* SEAT BELT SIGNS (5 Minutes es Prior To Departure) 6POINT
PRIOR TO ENG START C OR PUSH-OUT
ALLEY POWER...
ENGINE IGNITION
FUEL PUMPS .... .
ANTI-COLLISION iOR REQUIRED
ANTI-COLLISION/EXTERIOR LIGHTS...... ON/AS
DOOR ANNUNCIATORS ... 899 SPEEDS-KIAS
AIR CONDITIONING SUPPLY SWITCHES OE Fine on Raps _sserge
a Weight Va Vio | Va Vio. Vxse Vvse| Vaer
vome wo [10s 100 1 | or
10 POINT ‘wewlsn wil Ss
g000 82 oe | os 80 Ss 105, so
C99 SPEEDS-KIAS.
(20% Flaps 0% Flaps 100% Flape
vile Wen Ta Vos Var
‘rao we v6 [ar uss a1 18 | or
Nao wor isles aes te |
tooo wor ieee ats tne fa
own vo) r/o swe mt woe | Soe
ton vw! tesla eo we |

5 POINT



+ whole sets of procedures not carried over from
the AFM to the operating checklists;

* incomplete procedures;

+ checklists difficult to locate in manuals either

because of poor tabbing, poor indexing, or
poor titles.

4.1.4 READABILITY

The typography of manuals and checklists varied
widely, from five-point type to 10-point type or
larger, the smaller type being difficult to read. Often
print was blurred, and contrast of print to back-
ground poor, despite the obvious fact thatif manuals
and checklists are difficult to read, they will be
difficult to use, The Air Carrier Operations Bulletin
Part 135 No. 88-5 - Flight Crew Checklists (NTSB
Safety Recommendation A-88-72.) says:

a. “The National Transportation Safety Board
(NTSB) in their investigation of a commuter
air carrier accident discovered that the flight
crew checklist was not constructed in such a
manner that would provide adequate legibility
in normal or emergency conditions. NTSB
believes that under operational circumstances,
adeficiency in legibility and size of printcould
compromise the intended use of this device.

b. Principal operations inspectors should take
appropriate actions during the course of rou-
tine air carrier surveillance, inspections, or
flight checks of their assigned operators for
review of current checklist format. Flightcrew
checklists used by air carriers should include
the appropriate actions necessary for normal
and emergency procedures, printed in clear,
concise, and legible form.”

Although directed at Part 135 operators, this applies
to all operators. The regulations should be changed
to reflect the same standards for Parts 121 and 135
operators. The current regulations reflect a lack of
clear and consistent direction for manufacturers,
operators, and POIs alike. The manufacturers should
have clear guidelines to follow in producing usable
manvyals and checklists for new aircraft. Opera-
tors should have clear manuals and checklists for
their crews. And the POIs and evaluation groups
should be given unambiguous guidance on what
standards to apply to the design of manuals and
checklists.

4.1.5 Cotor Copinc
Color coding of checklists and manuals is used
little, although it could facilitate locationof acriti
checklist. The airlines usually cite cost as the
for not using color coding.

4.1.6 INCONSISTENCY

Often there was a lack of consistency
AFMs and checklists. Insome cases, checklist itet
and even some procedures were not carried
from the AFMs to the operating checklists.

4.1.7 DEFINITION OF “ABNORMAL” AND “EMERGENCY” 4
The use of the terms “ABNORMAL” and “EMER- :
GENCY” were inconsistent among manufacturers

and operators and from aircraft type to aircraft type

within the same operator's fleet. The use of “NOR-

MAL,” “ABNORMAL,” and “EMERGENCY” is

sometimes inconsistent throughout a fleet. The terms

themselves vary, with the terms “NON NORMAL”

and “IRREGULAR” used somewhat interchange-

ably with ABNORMAL” and “EMERGENCY,”

but there are also differences in meaning.

The lack of a standard definition for “emergency”
has created particular problems for checklist design.
Excessive numbers of emergencies result in emer-
gency checklists of extreme length, excessive num-
bers of memory items, and inconsistent responses to
teal emergencies that are not always so labeled, e.g.,
loss of all generators. One foreign aircraft that had
39 sets of “Emergency” procedures, many of which
would have been classified “Abnormal” by major
U.S. manufacturers. Inflight events that are classi-
fied as emergencies (for example, low-level
unpressurized flight) in one aircraft type but not
another in the same fleet reduces the flight crews’
respect for the term and contributes to their confu-
sion regarding their priorities for action.

4.1.8 EmercEncy CHECKLISTS

“Emergency” checklists are sometimes difficult to
locate when needed. They are often in manuals
stowed in flight bags and are reported to be difficult
to retrieve.

In some cases in our study, we encountered groups
of “Emergency” checklists that had an excessive
number of checklists (39 in one case). This made the
checklists cumbersome to use and made it more
difficult to find a single checklist.

4.1.9 Heaps-Down Tie
The use of CRT-presented rather than hand-held



i
The Use and Design of
Flightcrew Checklists —
and Manuals

Mepeen, D.C. 20591

John W. Turner
EG&G Dynatrend

M. Stephen Huntley, Jr.
U.S. Department of Transportation

Research and Special Programs Administration -
John A: Volpe

National Transporation Systems Center
‘ Sacnbsidge,b MA 02142

Final Report

This document is available to the public through
the National Technical Infarmation Service,
Springfield, Virginia 22161.

hn aw ad



73. MI

74,

15.

76.

78.

81.

82.

83.

 __.

Altitude overshot in climbout

MLG Altitude overshoot in climbout,
aircraft would not pressurize

WDB Wild aut oscillations in flight,

yy going to manual control

MLG Passed hold short point on a taxiway
cutting off another aircraft

MLG Altitude overshoot in climbout

MLG Altitude overshoot in climbout, not
caught by controller

WDB Altitude overshoot in descent

MLG Possible missed crossing restriction
on both altitude and speed

MLG Altitude overshoot in climbout

MLG Altitude undershoot in climbout,
missed crossing restriction

MLG Complaint concerning close parallel
approaches

WDB Aircraft experienced tiple
emergency ae landed short tof
destination

MLG Runway incursion

C-14

CAUSE

Pilot not flying busy doing checklist during a high
rate climb at low level, altitude alert nonstandard
from other aircraft in the fleet, pilot flying

distracted temporarily

ight and checklist use didn’t catch
open o1 valves, aircraft wouldn’t
and momentarily distracted crew
attention from the altitude

at

Crew didn’t turn on pitot heat, didn’t catch it on

resulting in rudder for lowe
in r inputs for lower
aircraft was at high speed

‘Two-man crew busy doing checklists and working
and company radio, . ai
taxi instructions and F/O art monitor closely
enough because of other duties

Pilot not flyin; ying reading the checklist, failed to call

1000’ before altitude, ACARS message came

seross a the same time as they hit the assigned
ti

Crew busy doing checklist and other duties, wrong
altitude set in the altitude reminder, overshot and
in the overshoot received a clearance to higher

Two-man crew busy in arrival procedures in busy
area, , copilot busy doing comm, etc.,
capt. flying aircraft, Programming the computer
and doing checklists, missed altimeter reset at
18,000"

Aircraft developed a Pressurization, problem in
descent, crew busy doing abnormal procedure and
flying aircraft missed crossing restrictions, but at
the same time the controller gave them new altitude
and heading which cancelled prior restrictions

Lower altitude assigned than original clearance
when aircraft was almost at the new assigned and
at a high climb rate, also distracted doing the
checklist and altimeter didn’t get reset

Changes in altitude clearance by departure, crew
busy ¢ doing checklist and other departure duties and

tumed prematurely resulting in lower altitude at
crossing point

Reporter ests staggering aircraft, in addition to
being alarming to passengers it distracts from
checklist and other duties

Proper use of abnormal, emergency and normal
checklists

Aircraft had been cleared to hold short, F/O busy
doing checklist and not listening, capt.
misunderstood clearance


“ENGINE OVERSPEED”

“PROP OVERSPEED”

“FUSELAGE SMOKE OR FIRE”
“DOUBLE GENERATOR FAILURE”
“BATTERY OVERHEAT”

The “Emergency” checklists of another car-
rieralsolacked many operationally significant

procedures which were in the AFM. Among
these were:

- “PROP MALFUNCTION — OVER-
SPEED”

- “FUSELAGE FIRE”

- “TOTAL ELECTRICAL FAILURE”

- “LOSS OF ALL SYSTEM FLUID”

Manufacturers as well as operators were re-
miss. An example can be shown from the
AFM of one Part 135 aircraft. It lacks proce-
dures or checklists to deal with problems such.
as “LOSS OF ALL GENERATORS.”

Procedures were not presented in the order in
which they should be accomplished. One Part
135 carrier’s “Normal” checklisthad “SHUT-
DOWN?” following “BEFORE TAKEOFF.”
Nomnally “SHUTDOWN?” is the last of the
“Normal” procedures. Procedures should be
presented in chronological order.

Intemal inconsistencies were also found. These
concemed a variety of issues such as:

- Crew size. One operator’s “Emergency”
section preface contained the following
statement:

“Emergency procedures have been for-
mulated based on single-pilot operation
of the airplane.”

However, throughout the section of the

“ Company Aircraft Operating Manual de-
voted to Flight Operations, there are many
references to “Pilots” (plural) and “Crew
Coordination.” Although the aircraft can
be flown single-pilot, it was obvious that
the company intends it to be flown as a
two- pilot operation at least part of the
time. Yet, nowhere was it addressed how
emergencies were to be handled during
two-pilot operation.

i

- Aircraftequipment. Anotherexampleof
confusion in a Part 135 carrier AFM
concemed the response to a waming
light. The instructions were as follows:
“Any illumination (or flicker) of either
CHIP DETECT annunciator light (if
installed) requires immediate shutdown
of the affected engine.”

It is strange that an annunciator light so
important that its illumination requires
immediate shutdown of an engine could
be placed on the list of options for an
aircraft, and not be required equipment.

- Procedures. Another carrier exhibited
confusion between the AFM and the op-
erational checklist. In the “AIR START
— NO STARTER ASSIST” checklist,
one item in the AFM called for “PROP

«FULL FORWARD.”
in the checklist from the
“PROP LEVER...

The same it
CAOM said

opposite actions, we wonder which is
correct.

If flight crews are to be expected to have
confidence in and use checklists, the pro-
cedures that the lists describe must be
correct and must be consistent with the
procedures described in the associated
manuals.

¢ Alackofclarity ofpurposeof the checklist and

the AFM. An AFM is designed to present
Specific information to an operator’s person-
nel, including flight crews, about the opera-
tions of the aircraft. It is not, nor is it intended
to be, a training manual. This is also the case
with a checklist, which is to be used to assure
proper completion of items necessary for safe
operation of the aircraft. Despite this, some
operators use AFMs and checklists for con-
veying messages which should be given in
taining. Examples of this are illustrated from
these instances in one carrier’s checklists and
another’s AFM.
- “Immediately prior to touchdown, lower
up-wind wing and align the fuselage with
the runway by use of the rudder.”

- “Piloting with an engine inop.” - “Use


The material from the regional Part 121 carrier
studied illustrated some of the shortcomings found
in the manuals and checklists of smaller carriers,
especially the Part 135 carriers, many of which fly
airplanes produced outside the United States, Al-
though the manuals and checklists of U.S. aircraft
manufactured for the regional and Part 135 market
don’t generally come up to the standards of those
produced by the U.S. manufacturers of large air-
craft, the problems seem to be even worse in manu-
als and checklists for aircraft of foreign manufac-
ture, Part of this is a problem of language and
terminology. Part of it seems to arise from the fact
that the manual and checklist material from foreign
manufacturers is approved by their equivalent of the
FAA under the bilateral agreement. Problems in-
clude:

+ Lack of tabs in the manuals, which makes it
more difficult to find important information
quickly. One manual was tabbed but most of
the tabbed sections were not numbered, even
though references were made to those sections
by number.

* Accessibility of important information. One
AFM had no systems descriptions of. any sort,
Another, in its “Abnormal” and “Emergency”
sections, frequently made references to fig-
ures and paragraphs in otherparts of themanual
rather than supplying the needed information
at that point. These characteristics decrease
the value of the manual as a reference in
addressing abnormal and emergency situa-
tions.

. Anexcessive number of “Emergency” check-
lists, and a classification of “EMERGENCY”
which wasnotconsistent with general usage in
the United States. The AFM for one foreign
airplane contained 82 “Abnormal” and““Emer-
gency” checklists, of which 39 were classified
“Emergency.” Many of the 39 would nothave
been classified “Emergency” by most U.S.
standards.

+ Anexcessive numberofmemory items. These
checklists were for an airplane operated bya
regional carrier, sometimes flown by low-
experience-level crews. This combination of
an overwhelming number of memory items
and low-time crews is conducive to errors in
emergencies,

10

* Missing items on checklists, Exampies of this
are seen in the following.

- Carrier B

No mention of “GEAR” on the “BE-
FORE STARTING” checklist, and no
mention of “FLAPS” on any checklist
prior to takeoff, -

- Carrier E

Onall three groups of checklists — “Nor-
mal,” “Abnommal,” and “Emergency” —
there appear challenges withoutresponses,
as in “EXCESSIVE LOADMETER

FAILURE,” “BATTERY. +” 0
Tesponse).
> CarrierG
Operationally important items not carried
Over to the checklists from the AFM in-
cluded:
+ From “ENGINE FIRE OR SEVERE
DAMAGE,” “FUEL CROSS-

* From “ELECTRICAL SMOKE OR
FIRE,” “RECIRCFAN. ...esssssss..OFF.”

* Insome cases, “Emergency” checklists were
notcarried over from the AFM to the operating
checklists. FAR 125.71 states that “Each cer-
tificate holder shall Prepare and keep currenta
manual. A copy of the manual... shall be
.-fumished to - (1) Its flight crewmembers.”
FAR 125.73 says “The manual must
include...(m)procedures for ‘ensuring compli-
ance with emergency procedures,...” FAR
25.1581 states “An Airplane Flight Manual
must be furnished with each airplane, and it
must contain the following: ...(1) Information
required by 25.1583 through 25.1587,”
25.1585,”Operating Procedures,” includes
emergency operation of the systems. One car-
tier was using checklists that did not include
11 “Emergency” checklists that were in the
AFM. This certainly circumvents the intent of
the FARs. Among the checklist procedures
that were missing were the following:



16. MLGA

17.

18.

19.

21.

23.

24,

26.

27.

28.

29.

AIRCRAFT OCCURRENCE
TYPE

MLG

MLG

LRG

MLG

MLG

MLG

MLG

MLG

MLG

MLG

Aircraft undershot crossing altitude

Altitude overshoot on descent,
aircraft on autopilot

Altitude overshoot on climb

Altitude overshoot on climb

Altitude overshoot in climb

Aircraft missed crossing restriction

Momentary application of heavy auto
brake on landing, resulted in a very
noticeable lurch during rollout

Aircraft several thousand feet high on
crossing restriction

Probable needless engine shutdown in
flight, emergency declared with a
precautionary landing short of the
destination

Altitude overshoot on climbout

Altitude overshoot on climbout
Altitude overshoot on descent
Altitude overshoot on descent
Speed deviation on STAR

Altitude undershoot in climb

c-10

CAUSE

Crew busy getting ATIS, working radio, doing
checklists, tuned wrong VOR frequency, and didn’t
make crossing restriction

Captain busy with checklist, F/O programmed the
autopilot wrong and knocked off altitude hold

Maximum performance climb, light aircraft, tired
crew, busy doing checklist and working radio,
didn’t reset altimeter soon enough and went
through the assigned altitude

Late at night, long flight sequence, light, fast
climbing aircraft, multiple frequency changes,
doing checklist, didn’t catch it

Crew didn’t reset altimeters to 29.92" at 18,000’,
distracted from the checklist by turbulence

Due to multiple frequency changes and looking for
traffic climb checklist was never done, and
altimeters weren’t reset

While doing the landing checklist the F/O
inadvertently programmed the auto brake for T/O,
due to darkness and having to do a 360 degree turn
on final, the error was not caught

Poor crew coordination, inexperience on the
aircraft and that portion of the route structure for
the captain, running the checklist

While performing the checklist for an electrical
abnormal, captain mistook an APU low oil
pressure light for an engine low oil pressure light
and shut down the engine, poor crew coordination
while doing electrical abnormal and F/O was
starting the APU

Captain had called 1000’ before the altitude and
got busy doing something else, F/O looked away to
do something that wasn’t called for on the checklist
at that point and went through the altitude

Very short flight, frequency changes (both
company and ATC), auto throttles not operating,
doing checklists, overloaded two-man crew

Busy two-man crew, set improper altimeter and
overshot by 1000’

Two-man crew doing checklists and other duties on
descent for landing, altitude capture not set on
autopilot, no altitude warning on the aircraft,
caught by the crew after they had overshot

Captain handflying aircraft for practice, F/O doing
checklists, handling radio, etc., both missed the
speed restriction on the STAR

Crew neglected to reset altimeters to 29.92" at
18,000’, missed it on the checklist



MD-80

EXTERWAL ELECTRIC & PMELOATIC SOURCE - START
PMRUMATIC K-FEEDS
eed

pemumare
Peumatic ease ai P81 KW)
‘COMPLETE. 1aRT COLCELIST

CLES STABILIZED

58

58d9e ese

FIGURE A-2. MD-80 CHECKLIST

Ta

‘**(SETTING)
‘**(SCTTING)
ie}

mH
re ainspece ws «
ANTS. :

APU AIR...
ATR CONDITIONING SUPPLY SVITCUES

ENGINE ANTI-ICE & IRL BEAT
PLUMATIC X-FEEDS.
anu...

RADAR & TRasronnen
aPu QI & ELECTRIC Pvt |

PARKING
atm CONDITIONING SUPPLY surroans .
PeuuTic X-Petae



86.

87.

88.

89.

90.

91.

93.
94.

97.

98.

AIR
AIRCRAFT OCCURRENCE

MLG Engine oil leak caused further engine
problems resulting in shutdown, other
generator didn’t pick up the lost load

WDB Aircraft had to return to land due to
two cargo doors open.

MLG Altitude undershoot on climb

WDB Aircraft took off over weight on a
limited runway with antiskid inop

MLG Cabin altitude climbed above 10,000’
with no altitude warning horn,
Passenger oxygen masks deployed,
returned to departure point

MLG Aircraft overshot altitude on profile
descent

MLG Altitude undershoot in climb

MLG Altitude overshoot during STAR
MLG Unauthorized landing

WDB Aircraft took off with gear pins
installed and had to return to land

MLG Aircraft took off with nose gear pin
installed and had to return to land

LRG Aircraft overshot altitude in climb

MLG Altitude overshoot during descent,
less than standard separation with
other aircraft

C15

CAUSE

Confusion in the cockpit due to nonstandardization

of fleet, compounding problems, controller queries
during a busy time, ‘FIC IG
THE EMERGENCY CHECKLIST

Glass cockpit airplane, CRT wiped clean during the
fire teat in before starting engines checklist, F
misconception from training concerning recall of
items to the CRT after start led to not seeing doors
open light (crewmember had been led to believe
that int tion was automatically displayed on
power change over after start when it had to be
recalled manually)

Reset of altimeter at 18,000’ is not on the checklist
and the crew forgot it

Rushed after maintenance delay working
on antiskid, very short taxi with rushed checklists
and engine start, message on weights to check

i tcher for reduced V1 speed, dispatcher
Tel them to manuals, manuals poorly set up to
get info, two-man crew in busy environment unable
to find info readily

Proper use of riate checklists, inop cabin
altitude warning horn and auto pressurization

Aircraft on sutopilot with altitude hold engaged,
pilot not flying doing checklist, altitude warning
horn did not sound and autopilot did not capture
altitude

Altimeters not reset, didn’t catch it in the checklist,
low flight crew experience level, fleet
nonstandardization

Flight crew distracted doing checklist

Crew given poor vectors to final and then tumed
on for a short, steep descent for landing, thought
they heard a clearance which was for another
aircraft - this aircraft uses a mechanical checklist
with two blanks for “cleared for the approach” and
“cleared to land" - thinking he had heard that, the
copilot moved the slides indicating to the capt. that
clearance was received

Crew distracted by maintenance while reading the
checklist and missed the gear, pins

F/O distracted on walkaround by new hire
accompanying him, missed nose gear, PIC can’t
see gear pins in the cockpit as on other aircraft in
the fleet, missed on the checklist

Aircraft in heavy weather, pilot flying called for
the climb check, aircraft sustained a lightning
strike, misread autopilot annunciators, and changed
autopilot settings resulting in an overshoot
Two-man crew in busy environment, running
checklists, etc., and altitude alert didn’t sound



normal, yellow for abnormal, and red for
emergency. In the interests of economy, the
users may wish to stay with black lettering on
a white background, however, the white on a
dark we have found to be easily
read under all light conditions and we recom-
mend it.

2. CHECKLIST TEXT — 10-point (0.1") type,
all caps, boldface, in a typeface equivalent to
those recommended in the MIL SPECS. This
should be black lettering on a white back-
ground.

3. NOTES — 10-point (0.1") type, initial caps,
lowercase following, in a typeface equivalent
to those recommended in the MIL SPECS.
This should be black lettering on a white
background.

If space and economy permit, we recommend mov-
ing up to 14-point type (0.15") for checklist head-
ings and 12- point type (0.125") for checklist text
and notes. Flight Safety has done thisin their Canadair
checklists and it produces superior readability.

FORMAT

We recommend a format of challenge and response
—consisting of the query to the left margin, fol-
lowed by a dotted separation, followed by the re-
quired response (to be right justified). This is the
specified format in MIL-C-81222C, is quite com-
mon in industry use, and is illustrated in Figures A-
2 and A-3,

COLOR CODING

Throughout the industry the use of color-coded
annunciator lights is standard — red indicates
“WARNING” or danger, yellow indicates “CAU-
TION,” green indicates safety. Flight Safety Canada,
Lid. and some air carriers have carried this color
coding through in checklist use. “Abnormal” check-
lists are identified by headings of yellow, and“Emer-
gency” checklists by headings of red, with the
“IMMEDIATE ACTION” items boxed in red.

We recognize that to do this is more costly, but we
recommend it strongly. Color coding such as the
above lends itself to ready identification, and hence
ease of use.

OVERALL COLOR USE

The MIL SPECS previously quoted specify the use
of black type on white paper, with the exception of
the checklist headings recommended to be white

A-4

print on a dark background. However, a limited
study done by the head of the Publication Depart-
ment of a regional carrier, in conjunction with an
optometrist, indicates that better readability is at-
tained under normal variations of ambient cockpit
lighting by the use of black type on a bright lemon
yellow background. This would appear to be bome
out somewhat by the study done a number of years
ago by big city fire departments which led to new
equipment being delivered with bright yellow paint.
They found that the equipment was more visible to
other drivers with that paint scheme than with the
standard fire-engine red. Once again, economics
entered the picture, and most fire equipment is still
red.

We have seen the results of the regional carrierstudy
and agree that it promotes better readability under a
variety of ambient cockpit lighting conditions.

BREVITY AND CLARITY

The following is a. quote from MIL-C-81222C:
“...procedures shall be presented in checklist form,
abbreviated from the amplified checklist or proce-
dures inthe NATOPS Flight Manual. This abbrevia-
tion is to be accomplished by omitting explanatory
material and reducing the check item to the mini-
mum necessary to describe the required action. For
example, the step ‘Reduce airspeed to 130 knots
TAS for best glide’ can be abbreviated ‘Airspeed, -
130 KIAS Glide’.” MIL-C-27278B says: “The
procedures of the checklist shall be derived by
abbreviating the procedures and eliminating the
amplifications of the procedures in the procedure
sections of the parent manual...”

As indicated by the above, no ambiguity or excess
verbiage should be allowed in checklists. The re-
quired items and no more should be covered. One
checklist studied had 139 items on the “AIRPLANE
ACCEPTANCE” checklist. Thisis excessive. These
items should be checked on a defined preflight, but
to cover every item on a preflight in a checklist is to
court checklist neglect by crews.

LEXICON

Standardized terminology, consisting of common
aeronautical terms, should be used in all cases. MIL-
M-7700C says: “Standard terminology. In most
cases, use the terminology for equipment that is
consistent with the intended operator’s standard
usage and is preferable to some of the more techni-
cally descriptive nomenclature [sic]. Some examples
are: “throttle’ vs. “power control lever’, ‘circuit
breaker’ vs. “fault circuit detector’...”


rn

The general issue of requiring a specific re-
sponse inlieu of the““AS REQUIRED” shown
on a checklist was addressed. The request for
aspecific response requires that the crew look
at the item being checked in order to give that
response. The discretion to answer “AS RE-
QUIRED” permits careless checking and poor
checklist habits. Six of the Part 135 carriers
allowed the use of the “AS REQUIRED”
response, as did two of the Part 121 carriers.
The handbooks of three of the Part 121 carriers
stated that a specific answer should be substi-
tuted for“AS REQUIRED,” and one Part 135
carrier very specifically disallowed “AS RE-
QUIRED” and specified precise responses.

les of this would be “12 QUARTS,”
“ON,” etc. One major carrier eliminated the
problem by not having “AS REQUIRED” as
a checklist response.

3.3.2 ALPHANUMERICS

The comparison of print size and letter case used in
the text of the checklists revealed a number of
problems. This was true of both the Part 121 carriers
and the Part 135 carriers.

“Normal” checklists for all but one of the Part 121
carriers and 50% of the Part 135s were in 10-point
type, and usually in all caps (see Figure 3-1). This
was normally quite legible, but in some cases, the
quality of print was poor and that affected the
legibility considerably. MIL SPECS (MIL-C-
81222C and MIL:-C-38778A) recommend the use
of 12-point type for the body of the text. One of the
Part 121 carriers used six-point type, mixed case
(see Figure 3-1), their checklists were difficult to
tead, and it would have been easy to lose one’s place
if distracted by other operational requirements. In
the Part 135 checklists, of the 50% that did not use
10-point type, the type size varied down to seven-
point, mixed case, and was not very legible. One set
of regional checklists incorporated a V speed table
in five-point type (see Figure 3-1), and the numbers
were almost illegible.

“Abnomal” and “Emergency” checklists showed
even greater inconsistency in alphanumeric sizes
than the “Normal” checklists. One major carrier in
their “Normal” checklist used 10-point type, all
caps. Yet their“Abnormal” checklist, although kept
in a well-tabbed pilots’ handbook and easy to find,
was in six-point type and mixed case, and difficult
to read. Their “Emergency” checklists were pre-
sented on a color-coded paper card with one side in

10-point type, the other side in eight-point type.
Both sides were in all caps. The eight-point was
slightly less legible than the 10. It appears that this
combination of type was used in order to include all
the checklist items on a single card. Another Part
121 carrier, although using legible 10-point type in
their “Normal” checklist, used eight-point type and
all capitals with the letters spaced closely together
for their other checklists.

Among the Part 135 checklists, the same sorts of
problems, but more pronounced, were often seen.
One of the regionals used legible 10-point type for
the “Normals” and then reduced to seven-point type
for their “Emergency” checklists. The reverse was
found in another case, with the “Normal” checklists
in the small, difficult-to-read print.

The practice of using smaller, less legible type for
“Abnormal” and “Emergency” checklists than for
“Normal” checklists was found amongst both major
and regional carriers. Since these are checklists
which are used under conditions of stress, and often
with poor illumination, they should be as legible as
possible, and surely not smaller than the “Normal”
checklists.

Clear, 10-point type presents alegible checklist, and
isused by anumber of the major carriers we studied.
However, with type larger than 10-point, as is rec-
ommended by the aforementioned MIL SPECS and
by the Human Engineering Guide to Equipment
Design, the checklist page becomes larger, or
morepages are necessary, and checklist stowage and
handling becomes more of a problem. ©

3.3.3 METHOD oF PRESENTATION

All of the Part 121 carriers studied used paper
checklists for at least the bulk of their “Normal”
checklists. By contrast, only 50% (five) of the Part
135 operators did this. One Part 135 carrier had its
“Normal” checklists on a laminated card, and the
other four were in either a manual or a separate
checklist booklet.

One of the major carriers studied used paper check-
list cards for all but the “BEFORE TAKEOFF” and
“BEFORE LANDING” checklists. These were
mechanical, in either a lighted slide or a lighted
toggle switch configuration, depending on the air-
plane type. They did have a printed backup in the
Operating Manual to cover the possibility of a
mechanical checklist malfunction. The use of these
mechanical checklists for this limited use was re-



writing of the Handbook. Also included was written
input he had solicited from the airline representa-
tives regarding their positions onmanuals and check-
lists, and input for possible use in the Handbook.

Since this section of the Draft Inspectors’ Hand-
book was something which would govem their
manuals and checklists for the foreseeable future,
the airlines participated actively. Their views were
understandably quite parochial, and included much
debate on semantics, to eliminate, as far as possible,
any but very narrow interpretations by POIs. There
was general agreement among the airlines that if it
were not necessary to mention a specific point in the
handbook, it should be left out completely, rather
than having a general statement subject to varying
interpretations.

3.5.3 Jumpseat OpservaTion RIDEs

We took jumpseat observation rides on seven occa-
sions, on four different airlines. We did this to see
how checklists were actually being used in flight.
The aircraft flown included two DC-9s, a MD-80, a
DC-10,aL-1011, a B-727, and a Saab-340. None of
the aircraft used a computerized checklistona CRT.

All used paper “Normal” checklist cards in varying -

sizes. On three aircraft, a mechanical checklist was
used for the “BEFORE TAKEOFF” and “BEFORE
LANDING” checklists. The crews using these me-
chanical checklists were highly in favor of them.

The manner in which the checklists were performed
varied widely. Three crews from the same airline
performed inauniform manner, indicating thorough,
standardized training. Two crews of another airline
performed in a loose manner — sufficiently loose
that one of them never ran the “BEFORE
LANDING?” checklist.

It appeared, from these jumpseat rides, that the
performance of checklists in an airline that has a
strong emphasis on training and standardization
will be more likely to be uniform. Where less
emphasis is placed on those factors, and less disci-
pline prevails, checklist use will be correspondingly
more variable,

3.5.4 Corporate On-Site Visrrs

Corporate aviation often makes use of the latest
technology before the airlines, since corporations
are not subject to the economic constraints imposed
by a large fleet. They also frequently carry execu-
tives whose loss to the company in an accident could
be critical. We believe this colors their thinking

21

regarding technology vs. cost decisions. Interested
in this tendency to use the newest equipment, we
made on-site visits to two corporate aviation depart-
ments to assess their current checklist technology. A
peculiarity of corporate aviation departments is that
they can change their checklists whenever they
want, as they see fit, and without prior approval,
since they operate under Part 91.

One corporation flew two Canadairs and one
Westwind. All three aircraft, at the time of our visit,
used a backlit, fold down, scroll checklist for all
“Normal” checklists. This was mounted in the cen-
ter of the glare shield. The pilots reported that they
liked it, as they always knew where they were in the
checklists, regardless of interruptions. “Emergency”
and “Abnormal” checklists were carried in the cock-
pit, in a laminated, color-coded, well-tabbed book-
let prepared by Flight Safety Canada, Inc. This
booklet also contained backup “Normal” checklists
foruse ifthe scroll was inoperative. These “Normal”
checklists were not as comprehensive as the
corporation’s own, used on the scroll. All three
aircraft have the capability of upgrading to auto-
mated checklists on CRTs, and the corporation
stated their intent to do this in the near future. Since
the checklists would usurp the radar presentation, in
bad weather the crew would revert to the scrolls.

The other corporation had a larger aviation depart-
ment encompassing a Gulfstream G-4, a Westwind
1 and 2, a Beech King Air, and a number of Bell Jet
Ranger and Bell 222 helicopters. The fixed wing
aircraft all require two pilots. The only case where
a rotary wing aircraft requires two pilots is the 222
in IFR weather.

All their aircraft used laminated card checklists,
despite the fact that the Westwind 2 had checklists
available on the radar CRT. The reason given by the
chief pilot was standardization. He also felt that the
CRT checklists were more cumbersome to use, and
took more time.

The G-4 will have the automated checklists installed
in its Sperry, all-glass cockpit this year. It will have
a dedicated CRT. Whether that installation will
supplant the laminated cards remains to be seen.

The rotary wing aircraft crews did not use available
checklists when underway. The only check nor-
mally done when underway is an engine gauge
check on descent. During an engine loss or tail rotor
failure, the crew is too busy to read a checklist. We



39,

41.

42.

43.

45.

47.

48,

49.

50.

SL.

AIRCRAFT OCCURRENCE
TYPE

MLG

MLG

LRG

MLG

MLG

. SMT

LIT

SMA

MLG

SMT

SMT

Aircraft departed on wrong runway

Incorrect V speeds set and not caught
until during the T/O roll

Aborted T/O due to flaps not set

Altitude overshoot on SID

Aircraft took off with gear pin
installed, returned to land

Aircraft landed gear up

Overweight landing

Aircraft landed gear up

Aborted T/O, flaps not set for takeoff

Engine failure and separation during
climbout

Gear not down for landing, minor
from runway contact during
a successful go-around

Aircraft off course by 20 miles or so

Red gear warning light on approach

CAUSE

Ui ted aircraft with subsequent rushin;
and baif-done job Of choi ir crew .
coordination, itearing clearance but not monitoring
Capt.’s taxiing, Capt, late starting second engine
after single engine taxi with rushed and incomplete
checklist and subsequent confusion

Operating rushed, late at night and fatigued and
gave standard checklist response rather than
thorough check

Had read checklists and responded but the flaps
weren’t set, disrupted diumal rhythm - crew had
flown late sequences all month and this trip had all
early checkins

During abnormal start procedure premature pulling
of extemal electrical power caused automatic bug

and altitude reminder resets, improper bug set was
caught on the checklist, altitude reminder was not

Gear pin flag removed and stowed in cockpit by
contract ground personnel, pin still remained
installed, crew on doing checklist counted three red
flags but didn’t check to make sure that a pin was

connected to each

Crew preoccupied with approach to unfamiliar
airport, didn’t do final check, gear horn sounded
just at the flair with power reduction

Crew fatigued and rushed, it t fueling not
caught prior to departure, ino mention of fuel Toad
on any of the checklists

Only used checklist partially, checklist difficult to
read at night, busy monitoring traffic at busy
airport, neither pilot nor instructor caught the error

Fatigued crew with other distractions neglected to
extend flaps and didn’t read the taxi checklist

Cause unknown at present, emergency checklist
performed, emergency declared, landing without
further incident

Pilot had down early in the approach, raised it
because of windshear encounter, with bad weather
and other distractions, did not extend gear again,
poor instrument scan, lack of checklist or GUMP
use

Using automated systems and Omega, both FMS
and Omega had gross errors, both systems
previously written up in the log for maintenance
action

Unable to extend gear normally, used emergency
procedure and checklist


52.

53.

54,

55.

56.

57.

58.

59.

61.

62.

63.

MLG

MDT

MDT

MLG

SMT

LIT

MDT

SMT

SMA

MLG

LIT

MLG

AIRCRAFT OCCURRENCE
TYPE

Failure to shut down right engine
prior to leaving aircraft

Flaps not fully retracted after

i » flaps damaged by passenger
bus driving under the wing on the

ramp

Aircraft had to level during climb
due to cabin altitude warning hom to
allow cabin to catch up and to
Pressurize

Engine fire with return to dey
point and emergency declere

Aircraft left with less than required
fuel, no serious consequences

Aircraft landed gear up
Aircraft made go-around during an

ILS approach, anomalies in
instrument readings

Aircraft departed with incorrect fuel
load, had to divert to alternate to get
fuel

Aircraft landed gear up

Aircraft landed gear up

Complaint of passengers smoking in
the aisles and it sign off prior
to completion of flight

Inflight engine shutdown due to loss

of oil pressure and quantity,
emergency declared

Altitude excursion on final approach

CAUSE

Crew claims to have used shutdown checklist, also
weat to belly baggage bin before leaving and didn’t
notice engine running

High demands on crew by ATC on rollout to clear
the runway quickly, during after landing checklist
the F/O was interrupted many times and didn’t
retract flaps fully, SILENT checklist without other
crew monitoring

Too short a time period during taxi to accomplish
all items satisfactorily, including checklist, missed
the air conditioning pack switches, should have
delayed to lish everything

Used engine fire emergency checklist, looked for
single engine landing checklist and couldn’t find,
checklists in the process of revision with conflicts
between some lists, FAA aware of the problems
but no action to date

Distracted attention in the cockpit during the
reading of checklist

No checklist, gear warning horn did not operate

Crew fatigue, missed proper settings on nav
receivers, no items on checklist to cover this

Distraction in the cockpit at the time the checklist
was being read, holding for fuel to be loaded, rush
to make schedule, fuel last item on the crew
acceptance checklist and not on any other checklist
for a crosscheck

No checklist, task saturation at low level, gear
handle used but gear didn’t extend, gear warning
horn inop, didn’t confirm gear green lights

Pilot monitoring hot air balloons and other traffic,
sun in his eyes, lowered flaps instead of gear,
didn’t get warming hom due to high manifold
pressure because of ATC-requested high speed on
approach

Crew not using checklist correctly and not
monitoring passenger conduct

Crew had a low oil pressure warning and ignored
it because of Previous transducer failures on this
aircraft type, low oil quantity and pressure caused
a flame-out, did emergency checklist

Aircraft stall warnings systems activated, crew
followed stall procedures including lowering the
nose to pick up speed for configuration, system had
failed, aircraft was not in a stall


2.2 Do you feel there are times when the use of a
checklist is disruptive to good operating procedures? Yes 37 No 58

(if "Yes," please explain)

2.3 What percent of the time is the "Sterile Cockpit" concept, below 10,000 ft., adhered to by your airline’s
crews?

a. 100% of the time 21
b. 75% of the time i
c. 50% of the time
d, less than 50% of the time 10
3. DEGREE TO WHICH PROCEDURES ARE DEFINED IN PILOT HANDBOOKS

3.1 Is @ standardized method for the use of
checklists spelled out in your company
operating manual? Yes 88 No 6

3.2. If so, do most of the crews adhere to

the prescribed method? Yes 85 No 7
3.3 Do you think the prescribed method could
be improved upon? Yes 42 No 44
@ How?
4. vy. (CKLIST US! LE TO Cl CTERI.

4.1 Do the individual crewmembers have any influence
on the manner in which a checklist is performed? © Yes 69 No 26

4.2 If so, does this result in variations, from one
crew to another, in the way in which the checklists
are performed? Yes 52 No 34

4.3 Does the influence of the individual crewmembers
sometimes ett i ne fered in othes the a
performed, or being performed in other than the
prescribed manner? Yes 41 No 53

4.4 Any comments



contents, including clear reference to the

- Should include tabbed dividers for sec-
tions thatmay have tobe accessed quickly.
For checklists, these should include stan-
dandized, color-coded tabs, by section
C‘Nonmal,” “Abnormal,” and “Emer-
gency”) and appropriately labeled tabs
within each section. Each section should
beginafter the tab with the first page being
a clear, alphabetized index.

Establish quantitative and behavioral criteria
for checklist accessibility and readability.
Developaprototype checklist foruse by safety
inspectors for evaluating air carrier checklists
and flight manuals.

Develop and evaluate the usefulness of a stan-
dard format organization, and table of con-
tents for aircraft flight manuals.

Evaluate the use of all caps vs. mixed case

4.23 CHECKLIST TRAINING me lettering in checklist design.
The required training curriculum for each aitli .
should incorporate checklist training, including: ' ¢ Develop and evaluate the use of a standard
\ terminology forcontrols, displays, and inflight
* Proper use of checklists. operations in checklists and flight manuals.
* Crew coordination in the use of checklists. + Evaluate the utility, safety benefits, and limits

of audio checklists, checklists on CRTs, and
-2 The necessity for compliance with checklists, checklists with artificial intelligence features,
42.4 Review or FARS

both in a laboratory setting and in an opera-
tional context. (There is currently an audio
This review should be conducted to determine the Checklist design available from Heads-Up
need for: Technology that will be the subject of a study
by a major airline.)
¢ A clear definition of “NORMAL,”

“ABNORMAL,” and “EMERGENCY.” If
not accomplished by FAR change, this should
be specified in an Advisory Circular. This will
standardize the use of these terms for both
manufacturers and airlines, and should provide
the means to design “Emergency” checklists
which are similar in length and content. At
present, some manufacturers include in their
“Emergency” checklists many checklists that
would be considered “Abnormal” by others.
This has resulted in some “Emergency”
checklists of excessive length.

A rewrite of the FARs, or an Advisory Circu-
lar, to indicate that manuals and checklists for
Part 121 and Part 135 operators have essen-
tially the same, well-defined basic require-
ments. This should include all stages from
initial approval to operator requested changes.
‘Those parts not required by the scope of opera-
tion of smaller Part 135 carriers could be
eliminated.

4.2.5 RESEARCH AND DEVELOPMENT
Research and development should be conducted to:

27

Evaluate the benefits of color coding and
different font styles on checklist readability
for electronic as well as paper checklists.

Evaluate the operational feasibility of safety
Critical checklist item interlocks that would
prevent aircraft takeoff without completion of
safety critical items.

Evaluate the utility, safety benefits, and limits
ofmechanical checklists such as those used by
a major airline for “BEFORE TAKEOFF”
and “BEFORE LANDING.”

Developand evaluate a prototype checklist for
Parts 135 and 121 use. This list would be
developed as an example of how human fac-
tors principles in the use of formatting, font
size, and color coding can be applied to check-
list design.

Determine the influence of memory items on
emergency checklists on the speed and accu-
racy with which emergency procedures are
performed.



7. If you have say suggestions or comments for i improving checklist presentation, or a means of assuring that
checklists are done in their entirety, please explain ther

8. BACKGROUND INFORMATION (Average data shown)

The following information will be used anonymously to help the survey team evaluate the data received. .
8.1. Experience flying transport aircraft

a. T ypes
b. Hours in type

c Seats flown.

8.2 Experience flying other sophisticated aircraft
a, T
b. Hours in type.

© Seats flown,
8.3Hours in each seat collectively

b Fee Ottcer 3570

c. Second Officer 2910 (of these, 22 had no 2nd officer time.)
8.4 Aircraft and seat currently flown
8.5 Age  _45,78 (ranged from 31-66)
8.6 Sex Male 94 Female 1 (32 yr. old DC-9 Capt.)
8.7 Visual correction

a. None Yes__ No
b. Nearsighted Yes__ No
c. Farsi; Yes__ No.
7 ‘arsighted s__ No__
¢. Do you use corrective lenses while
flyii Yes 36 No $1
© single focal Yes No___
© bifocal Yes__ No__
© trifocal Yes__ No
© top-and-bottom focal Yes~~ No__

8.8 Does company have a ific poli
cockpit resource management? re ves 6 No 3

8.9 If 80, do most of the Captains

adhere to the policy? Yes 2 No 12
© If not, do they basically adhere
to Captain’s autonomy? Yes 32 No 3
YOU ‘OUR TIM:

D7 A°U.8. GOVERNMENT PRINTING OFFICE: 1991 - $61-020/21851



empty auxiliary fuel tanks, plenty of fuel in
main tanks - checklist not followed.

Grand Island, NE, 6/29/84 - Pioneer Airways,
Inc. Swearingen SA 227-AC - minor aircraft
damage - loss of control on takeoff roll, struck.
tunway light - left prop on start locks - check-
list not followed.

Selawik, AK, 10/16/84 - Ryan Air Service,
Inc. Beech 3NM - substantial aircraft damage
- gear up landing - checklist not followed.

San Antonio, TX, 12/24/84 - K. E, Cohlima
Beech 95-CS5 - substantial aircraft damage -
gear up landing - checklist not followed.

Holly Springs, MO, 2/8/85 - Professional
Aviation Beech 58 - substantial aircraft dam-
age - gear up landing - couldn’t lower gear
manually because the pilot couldn't unstow
the crank - checklist not followed.

Berkeley, MO, 2/13/85 - Britt Airways, Inc.
Swearingen SA 226-TC - both engines quit on
final due to ice ingestion - plane landed with-
outdamage- nothing onthe checklistconcem-
ing the use of auto-ignition in freezing outside
air temperatures.

Williston, ND, 4/7/85 - Pioneer Airlines, Inc.
Swearingen SA 227-AC - substantial aircraft
damage - landed gear up - improper use of
checklist.

Potsdam, NY, 5/17/85 - Sair Aviation Piper
PA-31-350- substantial aircraftdamage- gear
up landing - checklist not followed.

Atlanta, GA, 5/19/85 - Basil Aircraft Services
Embraer EMB-110-P1 - substantial aircraft
damage - collision with parked aircraft on
rollout - insufficient hydraulic brake pressure
due to incorrect monitoring of waming annun-
ciator light and use of incorrect procedure -
checklist not used.

Nashville, TN, 5/31/85 - General Aviation,
Inc. Gulfstream G-159 - aircraft destroyed -
two crew fatalities -loss of control afterengine
Joss on takeoff, prop didn’t feather - H. P. cock
levers not in “cruise lockout” position - item
not done on checklist before takeoff.

¢ Dallas, TX, 8/7/85 - Air Midwest, Inc.
Fairchild/Swearingen SA 226-TC - substan-
tial aircraft damage - gear up landing - could
have manually extended gear - didn’t use
checklist.

* Orlando, FL, 4/22/86 - Craig Air Center Beech
95-BSS - substantial aircraft damage - gear up
landing - late extension of gear, aircraft landed
on gear doors - checklist not followed.

¢ Indianapolis, IN, 7/9/86 - PDQ Air Service
Beech BE-58 - substantial aircraft damage -
gear up landing - checklist not used.

* Jacksonville, FL, 10/7/86 - Top Flight, Inc.
Ted Smith Aerostar 600 - substantial aircraft
damage - gear up landing - checklist not used.

¢ Santa Barbara, CA, 10/30/86 - Wings West
Airlines, Inc. Fairchild/Swearingen SA-226-
TC - substantial aircraft damage - one serious
injury, two minor injuries - gear up landing -
prop fragmented and punctured passenger
compartment - gear waming hom circuit
breaker deliberately pulled and gear called for
but not extended - checklist not followed.

* Florence, SC, 2/5/87 - Atlantis Leasing, Inc.
Swearingen SA-226-TC - substantial aircraft
damage - gear up landing - checklist not fol-
lowed.

In one of these cases, the incident was directly
attributable to the use of an inadequate
manufacturer's checklist. In another case, inflight
distractions contributed to a lack of conformity to
checklist procedures. One report cited excessive
workload as a factor. In another case, the NTSB
cited the company management for“improperemer-
gency procedures training” of its pilots.

Of the 21 cases reviewed, 20 involved lack of
conformance with the FARs regarding checklist
use. In the cases not involving extenuating circum-
stances, it is not possible to ascertain the reason for
nonconfomnity from the information we have. But,
the large proportion of instances of nonconformity
indicates that this problem may be as great a prob-
lem as is checklist design, if not greater.


59.

61.

62.

63.

64,

65.

66.

67.

68.

69.

70.

71.

72.

AIRCRAFT OCCURRENCE

MLG

LRG

MLG

MLG

MLG

MLG

LIT

MLG

MLG

MLG

MLG

Altitude overshoot of 1000’ in
descent

Altitude overshoot on climbout

Altitude overshoot on descent

Aircraft declared an emergen
smoke in the cockpit, diverted to land
short of destination

Aircraft lost comm on an active
runway, caused a go-around

Altitude overshoot in descent

Aircraft landed without clearance

Aircraft landed without clearance

Altitude overshoot on descent

Near collision on a runway, aircraft
cleared into position to hold on a
runway where another aircraft had
been cleared for T/O

Altitude undershoot in climbout

Altitude undershoot in descent, went
below crossing restriction

Aircraft didn’t make crossing
restriction

Altitude undershoot in descent,
missed crossing restriction

CAUSE

Altimeter set incorrectly by 1°, not caught on two
checklists yy

Distracted by radio, setting instruments, and
checklists, didn’t make 1000" before altitude
callout, altitude reminder sounded

Crew busy getting ATIS, doing descent and

checklist, set altimeter improperly,
itimeter setting not checked with ch that
ATC

Various annunciator warnings, smoke in the
cockpit, used oxygen masks, ran normal checklists
but no emergency checklists were mentioned

Crew busy doing checklist and final items for T/O,

didn’t notice a comm switch in the off position

Fatigue, descending in bright sunlight, hydraulic
activation caused a voltage spike knocking
autopilot altitude hold, also making PA
oe cei crew did not notice autopilot not
checklist

engaged when running

Approach during rough weather, crew busy
controlling aircraft and checklist, dialed in
wrong frequency and didn’t catch it until on the

Approach control didn’t switch the flight over to
tower, crew busy running checklist, etc., didn’t
catch it until on the ground

Doing checklist, reset altimeter for local pressure
when only cleared to 18,000’, altitude alert is only
triggered by captain’s altimeter, not both, so didn’t

Crew busy doing checklist but did hold short to
check runway as everyone should, saw other
aircraft rolling and held short

Altimeter not reset, crew busy running checklists
and handling aircraft in bad weather, NEW
;CKLIST PROCED' S$ ALTIMETERS
RESET FROM _OFE TO ONH AT 10,000’ - TOO
LATE FOR ACCURATE USE WHEN
ASSIGNED ALTITUDES BELOW 10,000"

New capt., low light level, high workload
including running checklists, misread DME for
prossing testriction, other pilot did not recheck on
his cl

Two-man aircraft, high work load including
checklists, controller confusion as to a prior
restriction

Pilot flying busy with aircraft in turbulence and
icing conditions, non- crossing restriction,
pilot not flying out of the loop doing the checklist

C-13 and didn’t catch the error



35.

36.

37.

38.

39.

41.

42.

43.

WDB

MLG

MLG

MLG

MLG

MLG

MLG

FT OCCURRENCE

Near mid-air collision on arrival
route, took evasive action

Altitude overshoot in climb

Altitude overshoot on descent

Altitude overshoot on descent

Altitude overshoot on descent

Aircraft almost aligned with the
wrong runway for landing, FAA
check airman on board made no
comment, caught the error in time

Aircraft almost on a runway
which was too short for their weight,
caught by the company and relayed
by tower controller

Crew returned to to have an
extended spoiler fixed. spotted by
crew of a following aircraft

Aircraft almost departed with seat
belt sign off and correct takeoff
power settings

Altitude overshoot in descent

Aircraft crossed runway hold line
during taxi after instructions to hold
short, potential conflict

Aircraft landed without clearance
from the tower

Altitude overshoot on climbout

CAUSE

Crew doing checklists and crosschecking settin;

on instruments as per company policy, just tnissed
other aircraft crossing the arrival route, no warning
from the controller

New capt., new copilot, new airplane, new airport,
very rushed, rushed the checklists (missing an
item), unfamiliarity with autopilot resulted in
overshoot

Forgot to reset altimeter leaving 18,000’ in the
descent

Read in range checklist completely at 24,000 and
missed the altimeter reset at 18,000’, premature
completion of the list

Two-man crew, between 310 and 180 had five
speed changes and two hdg. changes, one altimeter
got reset, the one of the pilot flying did not; in
addition, after the overshoot there were three more
speed changes, two more hdg. changes and three
runway changes (the last one taking place at 400’
on final), THIS IS RIDICULOUS

Capt. busy looking for airport, running checklists
and helping recent upgrade copilot

Runway was the longer of the two and into the
wind, but had a terrain restriction, crew was busy
doing checklists and tending to a passenger
problem and didn’t actually check the performance
charts for the runway

Taxiing with one engine shut down, holding off on
checklist, takeoff position advanced by controller,

rushed to complete ev ing and missed indicator
light for partially extended spoiler
Rushed tumaround, trying to beat a curfew, rushed

checldists and missed items, caught on the taxi for

Training flight, instructor busy doing checklists and
instructing, autopilot lost the altitude hold and
neither pilot caught it until after the overshoot

Two-man crew doing challenge and response
checklists and required PA announcements and
missed holding short

Heavy traffic, a great deal of maneuvering close
in, busy doing checklists, didn’t switch over from
approach to tower

Pilot flying new on the aircraft, pilot not flying
busy with communications, traffic watch and
checklists, pilot flying did not reset altimeter and it
‘was not caught on the checklist



The average hours in each seat were:

Captain 4,140

First Officer 5,570

Second Officer 2,910

(22 had no S/O time)

‘The lowest hours in each seat were:

Captain 0

First Officer 3,000

Second Officer 2,000

The highest hours in each seat were:
Captain 20,000

First Officer 10,000

Second Officer 5,000
Age ranged from 31 to 66 (the oldest being a
Tetread Captain retuming as Second Officer)
with an average age of 45.78 years,

Forty-one percent wore corrective lenses to
fly. :

3.4.3. Cueckuist Layout, DESIGN, AND UsE

ILICY FOR CHECKLIST USE

Ninety-three point six Percent responded that

theirairlines spelled outa standardized method

forthe use of checklists. (This is considerably

more than we found in our review of Parts 121
135 carriers.) Almost as many felt that the

crews followed the prescribed method. How-

ever, when asked if the prescribed method
could be improved upon, almost half said

“Yes.” Some of the pertinent suggestions

dardization.

- “Simplified (checklists) to prevent ‘crews
Not using prescribed method’, and use
enforced by all levels of administration
and training.”

: “Responses from aircraft (type) to aircraft
(type) should be the same.”

(One problem with this is that the manu-
facturers can't agree on what the name for

14

anobjectis—i..,\powerlever’/ throttle,”
etc,,and many checklist responses are tied
to placards on cockpit panels or aircraft
manual terminology.)

- “Do not require dual response by the pilot
flying the aircraft.”

- “On two-man crews, checklists are too
long, especially final items before take-
off. And I feel the F/O (First Officer)
should read the challenge and respond
while on the ground.” (The respondent
wants the F/O to be responsible for all
aspects of the checklists on the
freeing the Captain for operational du-
ties.)

+ ALPHANUMERICS

Thirty-nine percent felt it was easy, with cur-
Tent checklist typography and designs, to skip
items unintentionally. Although 94.5% indi-
Cated that print size was adequate, when asked
later in the survey if they felt that larger print
would be an improvement, 75% said “Yes.”
The fact that 41% of those responding wear
corrective lenses to fly may be pertinent here.

* METHOD OF PRESENTATION

- LAMINATED CARDS

Of those responding, 66% are currently
using laminated cards, either for their
“Normal” checklists or for all checklists,
Of these, 20% use another form of check-
list in addition (such as “Emergency” and
“Abnormal” checklists ‘kept in a manual),
Eighty-cight felt that it was not advanta-
geous to use a mix or combination of
checklist types, such as Paper and me-
chanical checklists.

- ELECTRONIC CHECKLISTS

‘The small number (six) of Tespondents
using electronic checklists on CRTs felt
the CRT was superior to the paper check-
listexcepton “heads-downtime” required.
On that, three felt the CRT took more
“heads-down time,” two felt the Paper
checklist did, one declined to answer the
question. They all feltthat the CRT check-



[ONS FOR 1VEM!

ERGELDURES AND USE Please check “Yes” or "No.” Your added comments below each section would be
6.1 sao cya coe ches, to be wed

industrywide, with variations by aircraft

type and operating environment Yes 39 No 53
6.2 Use of automated checklists wherever possible Yes 44 No 31
6.3 No use of checklists on the ground when

the aircraft is moving Yes 27 No 68
6.4 Use of color coding for easy

identification of checklists Yes 77 No 15
6.5 On paper checklists, use larger print

or better letter spacing, or both Yes 69 No 23
6.6 Use a mechanical marker to mark

checklist progress Yes 34 No 55

D6


transmissions while ‘setting-up’ bugs and
radios for two different approaches, and
being assigned to side-step to land on a
third runway. Usually flight crew cannot
Tespond as controller goes from one trans-
mission to another in steady stream of
clearances and modifications to clear-
ances.”

* Ground personnel communications

Respondents identified conversations ‘with gate
agents, fuelers, push-back crews, mechanics,
etc., as disruptive of checklist operations prior
to taxi.

+ Flight attendant requests

One respondent felt so strongly about this
source of interruption that he scored it 11 ona
scale of 1 to 10, and most felt that this was a
Problem in at least some phases of operation.
‘There was no agreement on which phase was
most affected. One respondent said:

“Interruptions aremy’ big deal. F/As(flight
attendants) who eitherdon’tknow ordon’t
Care what you're doing, ATC, etc. How
do you stop that?”

+ Extemal taxiing distractions

Thiscoveredeverything from complex airport
layouts, to poorly marked taxi- and Tunways,
to other airport traffic. Amajorcontribution to
this problem is ground vehicles which do not
give way to aircraft, and over which ground
Controllers claim to have no authority,

It has been suggested from time to time that
taxiing distractions could be eliminated by
complete. When queried about this, about
72% said “No.” The following comments are
typical:

- “Very difficult to stop and run takeoff
check at most airports,”

~ “Not practical.”

- “Checklists can be distracting when taxi-
ing, but can be managed safely,”

16

- “Pilots are capable of responding while
jing”

- “We can walk and chew gum.”
The consensus seems to be that they can
handle the distractions. However, ASRS and
NTSB data indicate that distractions may be
more disruptive than many pilots are willing to
admit.

This last category, “Extemal taxiing distrac-
tions,” also contains elements of the second
type of interruption — that of the checklist
becoming an interruption to operational tasks,

Asked ifthey felt “there are times whenthe use
of a checklist creates an interruption to good
Operating procedures?”, 39% said“‘Yes,” One
felt that during an Abnormal/Emergency situ-
ationhe should handle the problem and use the
checklist when and if he had time. Another
Said the problem was worse during taxi out.

“While checktist is being run it is easy to
miss radio calls. It is better without so
much dual response.”

A report from the All Nippon Airways Flight
Standards Committee quotes the 1979 NASA
ASRS9th Quarterly report, conceming check-
lists becoming an interruption to operational
Procedure. And an analysis from that 9th
Quarterly report of ASRS air carrier distrac-
tion reports associated with checklists, found
two characteristics common to all the Teports,

1. “Every report indicated that checklist ac-
complishment received cockpit Priority
over ATC requirements, Every incident
ended in a potential or actual violation of
ATC rules or regulations,”

2. “The checklistactivity was almost always
going on at the same time other cockpit
tasks were being performed; radar moni-
toring, minor malfunctions, System op-
eration, traffic watch, etc. Checklist ac-
complishment became a cause of distrac-
tion, not by itself but as a part of cockpit
workload. In the incident(s) reported, the
workload became ‘excessive’ and ‘time
Tan out’ before all tasks could be com-
pleted.”



2.2 STUDY OF PARTS 121 AND 135 OPERA-
TOR INFORMATION

A sample of checklists cards and expanded check-
lists in handbooks from prominent Parts 121 and
13S air carriers were examined:

* To identify design and implementation prac-
tices that should be promoted;

* Todetermine if there was a need for guidance
in the design and implementation of check-
lists;

* To identify design and implementation issues
that should be addressed by research, regula-
tions, or recommendations to the industry.

2.3 ALPA SURVEY

The Airline Pilots Association (ALPA) surveyed
line pilots to request their experiences and opinions
concerning the checklists they use. It was expected
that the information provided by this survey would
indicate the operational significance of various char-
acteristics of checklist design and design options,
serve to identify safety issues that we may have
missed in our analyses, and identify differences in
pilot opinion regarding checklist issues.

2.4 ‘ADDITIONAL SOURCES OF INFORMA-

TION
+ Discussions with an NTSB investigator and

representatives of two regional carriers.

* Meetings of the ATA Flight Crew Checklist
Working Group. This group was convened to
Provide a forum between the FAA group
responsible for writing the manual and check-
list guidelines for the Draft Inspectors’ Hand-
book and industry representatives.

+ Jumpseat rides on regional and major carriers
to observe use of checklists by crews, and to
ascertain conditions that interfere with check-
list use.

* Visits to two corporate aviation departments
to discuss checklist technology used in corpo-
rate cockpits, and to elicit opinions on that
technology.

¢ Examination of guidelines for manual and
checklist construction in human factors hand-
books and military specifications (MIL
SPECS).

3. FINDINGS AND DISCUSSION

3.1 NTSB REPORTS SUMMARY

From the beginning of 1983 to 10/7/86, there were
21 accidents/ incidents (involving multi-engine air-
planes) investigated by the NTSB, in which the
improper use of a checklist or a defective checklist
was suspected. In 24% (five) of these, the checklist
was notused at all. Of the remainder, amanufacturer’s
checklist was found to be inadequate in one case,
and in the other cases the checklists were not prop-
erly followed.

The danger of checklist misuse is seen in the results
of the accidents, 81% (17) of which resulted in
substantial damage or destruction of the aircraft. A
brief summary of the NTSB investigations follows.

+ Detroit, MI, 1/11/83 - United Airlines DC-8-
54F - aircraft destroyed - three crew fatalities
- improper trim setting caused loss of aircraft
control - might have been compounded by
unqualified 2nd officer occupying 1st officer
position during takeoff - checklist not fol-
lowed.

+ Bryce, UT, 4/2/83 - Republic DC-9-82 - both
engines flamed out due to fuel starvation -
emergency declared -engines restarted -check-
list not followed due to distraction.

+ Little Rock, AR, 4/13/83 - Central Flying
Service Beech BE-58 - substantial aircraft
damage - gear up landing excessive workload
and checklist not used.

* Luke AFB, AZ, 5/28/83 - Republic DC-9-31
- forced landing caused by engine flameout
due to fuel exhaustion - a tripped fuel quantity
circuit breaker was not noticed during the
preflight checklist - checklist not followed.

+ Blountville, TN, 10/28/83 - Atlantic South-
east Embraer EMB 110-P1 - substantial air-
craft damage - 16 minor injuries - aircraft
landed gear up due to indication of one gear
not down and locked - no confirmation made
on indication problem - checklist not fol-
lowed.

* Longview, TX, 2/29/84 - Mid America Air-
ways, Inc. Beech E-5S5 - substantial aircraft
damage - two minor injuries - total loss of
power, forced landing - took off on almost



27.

29.

31.

32.

33.

35.

36.

37.

38.

AIRCRAFT = OCCURRENCE

WDB Initiated a go-around at 500’ AGL
because of gear not down

MLG Poorly desi; and potentially

SMA Aircraft moved forward after start
and hit the nearby fuel pump

MLG Go-around due to GPWS activation at
500°

SMA Gear up landing

SMA Gear up landing

SMT Misuse of code

+ misleading center controller with

possible altitude conflict

LRG Altitude overshoot

LRG False fire warning, causing use of

and evacuation

of ai after landing with minor
injury to passenger

MLT Aircraft made inadvertent slats
extended and up
T/O, no serious consequences

SMA Aircraft landed gear up

SMA Aircraft landed gear up

C4

"Generic" checklist used for an eatire fleet, has no
logical flow pattern and requires a PA
announcement on final in contravention of the FAR
sterile cockpit rule, has been approved by the POI

Pilot used aircraft checklist which called for
throttle to be pulled out 1/2" on start, regardless of
whether warm or not, aircraft parked close to fuel
Pump, unable to control

Cockpit confusion due to monitoring close traffic
on parallel approaches, gear handle not fully in
down detent, when fully in detent GPWS continued
to sound, turned off pax O2 instead of GPWS
because of proximity of switches in nonstandard
cockpit igurations of the same model aircraft

Gear was not down and locked despite the use of a
checklist, pilot also did not utilize his normal
GUMPS k

Used checklist but missed the gear, CFI in the
aircraft didn’t GUMP the aircraft, but owner
claimed to have done that twice

Sloppy use of the checklist in entering transponder
code

Poor crew ordingtion. disregard of CRM and

Proper procedures by Capt. (on one takeoff the

pots was just finished about 10 kts. prior to
Vn)

After checking, there was no apparent fire, crew
had used emergency checklist and fought supposed
fire, declared an emergency and evacuated te
aircraft

Flaps had been programmed when checklists were
done, flaps raised when taxiing in proximity of a
large pile of dirt, flaps never extended, T/O
warning horn not programmed to sound without
flaps since flaps retracted-slats extended T/O is one
configuration for that aircraft

Pilot forgot to extend gear, didn’t use normal
checklist procedure with a GUMP backup due to
fatigue, inop circuit breaker for gear warning horn

Pilot didn’t do. GUMP check, inop gear hom,
distraction in the pattern



checklists may be expected to increase flight crew
heads-down time. This, coupled with the amount of
heads-down time necessary for reprogramming com-
puters when changes of routing are received, could
cause important decreases in the capability of the
crew to concentrate on other duties such as monitor-
ing traffic.

4.1.10 Summary or Factors DETRACTING FROM
Goop Cueckuist Drsicn AND Use

Flight deck observations, pilot reports, relevant avia-
tion safety databases, and our review of checklists
and handbooks currently in use by some air carriers
indicate:

* Operational conditions and priorities limit the
time available to flight crews for examining
checklist items.

¢ Use of checklists involves flight crew heads-
downtime that can be dangerous during termi-
nal operations.

+ Some flight crews only use checklists when it
does not slow down other aircraft operations.

* Regardless of time available, some crews do
not use checklists during some operations for
which lists are provided.

* The print on some checklists is difficult to
read under poor lighting.

+ Responsibility of individual crewmembers
conceming checklist use is not always clear or
well defined.

* Thetypes of items included on checklists vary
among carriers.

+ Someinflightevents are considered emergen-
cies by some carriers but not by others.

° Emergency checklists and handbooks are not
always quickly accessible to the flight crew.

It is difficult to quickly locate emergency
procedures insome checklists and handbooks.

* Procedures indicated on some checklists are
inconsistent with those described in the com-
panion flight manual.

25

* Some checklists do not include procedures for
all common emergencies.

« In some cases, the size and formatting of
emergency checklists makes them more diffi-
cult to read than normal checklists.

42 RECOMMENDATIONS

We did not collect sufficient data to determine if
poor checklist design and poor habits in the use of
checklists were widespread throughout the indus-
try. However, our data do support the conclusion
that there are Parts 135 and 121 carriers who are
operating with poorly designed checklists and manu-
als, and who have flight crews who are not well
trained in the use of these aids and who admit to not
using them when they are expected to.

Accordingly, we make the following recommenda-
tions regarding the design and use of checklist and
manuals. We also recommend supporting research
and development activities.

42.1 CHECKLISTS

+ “Normal” Checklists should be short and easy
to use. They:

- Should include only those items that are
pertinent to the safety and control of the
aircraft.

- Should be listed in an order that mini-
mizes heads-down time and the attention
of more than one crewmember at a time.

- Sublists, e.g., “BEFORE TAXT” check-
list and “AFTER TAKEOFF” checklist,
should appear on the checklist card in the
order in which they will be used.

- Should have selected safety critical items
such as gear and flaps as final items on
“BEFORE TAKEOFF” and “BEFORE
LANDING” checklists, even if this re-
peats an earlier item in the checklist. This

. will facilitate quick and last-minute refer-
ence to these items.

- Should have alphanumerics of sufficient
size, clarity of print, and contrast, to be
easily read under any illumination condi-
tions likely to be encountered in the cock-


1 INTRODUCTION

Checklists have been used, in one form or another,
since the beginning of manned flight, and certainly
since the inception of the airline industry. Even the
most rudimentary reminders to assure aircraft readi-
ness were an early form of checklist. With the
increasing complexity of aircraft, the ability of the
pilot(s) to accomplish all the items necessary for
safety without some type of checklist was dimin-
ished, and with the advent of larger and multi-
engine aircraft, a more formal checklist became
necessary to assure completion of the multitude of
items to be checked. However, as aircraft grew
larger and more complex, as checklists grew in size,
and as traffic increased, interferences to checklist
use also increased, with resultant increases in the
Probability that errors would be made in the use of
checklists and checklist-driven procedures. ASRS],
reports, data in NTSB files, pilot reports, and direct
cockpit observations indicate that checklists can be
misused easily and are sometimes even ignored.
There is much concem throughout the industry and
some empirical support that such misuse or lack of
use has contributed to the occurrence and severity of
aircraft accidents.

1.1 REASON FOR THE STUDY

Following its investigation of the crash of North-
west Flight 255 in Detroit, in August 1987, The
National Transportation Safety Board concluded
that “...the flight crew did not perform the checklist
Procedures in the manner prescribedin the company’s
Airplane Pilot’ s Handbook.” They noted that train-
ing and checking practices currently in use by the
airlines do not promote effective use of checklists.

Although it is not clear that checklist design was an
important contributor to the Flight 255 crash, the
NTSB did include among the seven recommenda-
tons produced by their investigation, the Class II
priority Action (A-88-68) that the FAA take steps
“to determine if there is any type or method of
presenting checklists that produces better perfor-
mance on the part of user personnel.”

The objectives of this study were: a) to identify
conditions that interfere with cockpit crews execut-
ing or verifying normal and abnormal cockpit pro-
cedures through the use of checklists; b) to deter-
mine the need and nature of FAA action to Promote

Tue Use AND DesiGn oF FLuiGHTCREW CHECKLISTS AND MANUALS

good checklist practices; and c) to determine Te-
quirements for research on the design and use of
cockpit checklists.

12 APPROACH
The following processes were used to accomplish
the objectives of the study:

+ Determine the contents and readability of cur-
rent checklists and handbooks;

* Identify operational conditions that interfere
with checklist use;

+ Identify flight crew practices that interfere
with checklist use;

* Identify design, procedural, operational, and
flight crew characteristics that Promote good -
checklist use.

13 PRODUCTS
* Specification and discussion of conditions
that interfere with good checklist practices,

* Guidelines for checklist design and evalua-
tion.

. Recommendations for furtherstudy in areas of
checklist design where more information is
required.

¢ Recommendations for changes in FARs to
Promote improved use and design of check-
lists.

2. METHODS

We used the following means of gathering informa-
tion for this study,

21 NTSB AND ASRSREPORT SUMMARIES
Relevant NTSB and ASRS accident/incident re-
Ports were reviewed to identify conditions that could
Promote the misuse of checklists, and to identify
Operational errors that may have resulted from check-
list misuse.


100.

101.

102.

103.

104.

105.

106.

107.

108.

109.

110.

11.

112.

LIT

MLG

MLG

MLG

MLG

LRG

MLG

LRG

MLG

MLG

MLG

MLG

Aircraft landed without clearance

Altitude overshoot in descent

Aircraft landed on the wrong runway

Aircraft flew wrong radial on
departure
Aircraft missed crossing restriction

Altitude overshoot of 1100’ on
climbout

Aircraft missed crossing restriction

Altitude overshoot on short final

Altitude overshoot on climbout

Altitude undershoot at top of climb
and in cruise, not noticed until
descent for landing, controller didn’t
cat

Aircraft experienced loss of
pressurization, made emergency
descent and declared an emergency
Engine flame-out at altitude from fuel
exhaustion, declared, got
engine relight at lower altitude

Altitude deviation during approach

Altitude overshoot on climbout

C-16

CAUSE
airport, crew monitorin in close
imity for the parallel runway, ing checklist,

didn’t contact tower

Crew didn’t reset altimeter at 18,000’, cau;
when they ran the checklist after the over

Being vectored for one runway, confusion over
controller comments concerning another, busy
ing .

later

Not set properly in nav instruments prior to
departure and not caught on checklist

Concer over airport below minimums, discussing
alternate plans, busy running checklist

Automated cockpit set to altitude capture with
autothrottles set, crew doing checklist, autopilot did
not capture

Crew busy doing checklist items, clearance
misunderstood by the pilot flying and not caught in
time by the other pilot

ing checklist in turbulence, pilot flying altimeter
set off 1", multiple approach control course and
speed changes, mistake not caught until GPWS
sounded and approach control altitude alert sounded

Crew busy looking for traffic and doing checklist,
new crew to aircraft in both seats, high
performance climb with a 2000’ assigned altitude

‘Crew new to the airplane, both used to three-man
crew, now on a two-man aircraft, missed setting
altimeters at 18,000’ and didn’t catch it on the
checklist

Appropriate checklists used

Ran the main tanks dry with a lot of fuel in the
center tank, didn’t have all the boost pumps on and.
didn’t catch it on the checklist

Two-man crew, very busy environment with many
heading and changes, frequency changes,
ATIS, reading the checklist - one pilot thought he
heard a clearance and started down, clearance not
confirmed because of frequency congestion

Due to loss of partial aircraft systems and transfer
of aircraft control and subsequent abnormal
checklists altimeter was not reset at 18,000’, the
transition level altimeter reset is not on a checklist


FIGURE A-3. DC-9 CHECKLIST

—_——DG@-9 NORMAL PROCEDURES CHECKLIST

BEFORE STARTING ENGINES
LOG BOOKS AND SEL..........sesssssssesereesetesned CHECKED

# RUDDER PEDALS AND
SEATS... ADJUSTED AND LOCKED

.. CLOSED AND LOCKED
2 PANELS/MASKS/INTERPHONE/
GOGGLES
EMERGENCY LIGHTS.
%* PROBE HEAT...

% WINDSHIELD ANTHICE.
ANTI-SKID...

PRESSURIZATION
AIR COND SHUTOFF..
FLIGHT GUIDANCE PANE!
FLT INSTR/SWITCHES/BUG:

FUEL PANELQUANTITY AND

DISTRIBUTION. SET/___ LBS AND CHECKED
GEAR HANDLE AND

LIGHTS.........
TRANSPONDER.

e KEE

He
an
g
&
g
q
m
a
Z
=

HEHEHE

SEAT BELT SIGNS (5 Minutes Prior To Departure)

PRIOR TO ENG START OR PUSH-OUT

GALUZY POWER..

ENGINE IGNITION.
FUEL PUMPS.....
AUX HYDRAULIC PUMP

ANTI-SKID (After Leaving Ramp Area).
RENG (One Engine Taxi)
FLIGHT CONTROLS.

BEFORE TAKE-OFF
Use Mechanical Checklist

AFTER TAKE-OFF — CLIMB
After Airplane Clean Up When Workload Permits.
«UP AND NO LIGHTS

UPINO LIGHTS
PRESSURIZATION AND AIR COND..............-.:0++4 CHECKED
10,000 Ft, MSL
ENGINE IGNITION. AS REQUIRED

FUEL SYSTEM.
STERILE COCKPIT.

ALTIMETERS . FROSSCHECKED
HYDRAULIC PUM! . LOW/OFF
18,000 Ft, MSL,

EXTERIOR LIGHTS AS REQUIRED
ALTIMETERS

(Outside Continental U.S., Reset At The Specified
Transition Altitude Obtained From Charts Or ATC.) .
——

AT


10.

11.

12.

13.

14,

1S.

AIRCRAFT
TYPE

MLG

MLG

MLG

MLG

MLG
MLG

MLG
MLG

MLG
MLG

MLG

WDB

WDB

OCCURRENCE

Aircraft landed without clearance
from the tower

Aircraft overshot altitude in descent,
‘on autopilot

Aircraft overshot altitude in climb

Aircraft overshot altitude on SID

Aircraft overshot altitude on descent

Aircraft emergency evacuation
leaving the ramp

Aircraft overshot altitude in climb

Runway incursion during taxi
Altitude excursion, aircraft on
autopilot

Near mid-air collision, took evasive
action

Emergency descent made and
emergency declared, couldn’t control
cabin altitude

Near mid-air collision, no time for
evasive action

Aircraft overshot turn to final

Aircraft aborted T/O.

Questionable descent clearance

co

_ CAUSE

Two-man crew, very busy trying to locate an
unfamiliar airport, doing checklists, etc., didn’t
switch frequencies

Autopilot sensing taken off F/O altimeter which
was set 1 inch too high (30.79° vs. 29.79")
Aircraft on test flight, two-man crew, pilot flying
new on aircraft, pilot not flying overly busy with
extensive test flight checklist and didn’t call 1000°
before the altitu

Preoccupation with the checklist and no call for
1000’ before the altitude

‘Two-man crew fairly new to the airplane, busy
running checklists and other duties, knocked off
altitude hold by mistake and didn’t catch it until
after descent below assigned altitude

Alleged right engine fire, ran emergency checklists
and did emergency evacuation

Didn't reset altimeters at 18,000’ and didn’t catch
it on the checklist

Crew busy doing checklists and briefing

Crew busy doing checklists and other duties, did
not catch the fact that the autopilot had gone to
another mode and started to climb

Aircraft level, crew busy changing radio and doing
checklist, looked up to see small aircraft very close
at the same altitude, no mention by the controller

Did emergency checklists, auto pressurization lost,
regained control with manual pressurization,
continued to destination

Aircraft in level flight under positive control, did
outside check, dropped eyes to checklist, looked
back up to see an aircraft within 150’ crossing at
the same altitude, no mention by the controller
although the controller did say afterwards he had
the aircraft on radar

Crew busy programming the FMC and doing
checklist, got behind the airplane and didn’t get
into the slot until 1000”

F/O sliding window came open on T/O, not
latched properly, item not on checklist for positive
check

Crew busy doing checklists, handling multiple
radios, etc., got a descent clearance from one
controller, a frequency change, and the following
controller questioned the altitude



NE |

CHECKLIST GUIDELINES

The need fora set of standards to guide manufactur-
ers and airlines in developing manuals and check-
lists is becoming more and more apparent. Any
Proposed guidelines would have to encompass a
number of areas, such as print size and style, format,
color coding, overall color use, brevity, clarity, etc.
Another area of concem is readability under all
Conditions of cockpit lighting, from bright sunlight
cruising at altitude to night flight with low ambient
cockpit lighting. Although supplementary lighting
would normally be used in the latter case, too much
white light will temporarily destroy night vision.

Bearing these points in mind, the following set of
guidelines are proposed as the first step in the final
development of a set of standards for industry use.

PRINT SIZE AND STYLE

Figure A-1 shows two extremes of print size and
style. The first is a copy of the actual checklist on a
Jetstream 31 involved in an accident in New Orleans
in 1987. It is representative of the size and style of
print used in the checklists of some smaller carriers
and is clearly too small (0.075") and tightly spaced
for adequate legibility under the range of lighting
conditions which an aircrew will normally encoun-
ter. Figure A-2 is a copy of the actual checklist on an
MD-80 that was involved in an accident in Detroit in
1987. The print is the same size as that of the
Jetstream 31 checklist, and although it is formatted
better, we still find it too small for easy readability
in all lighting conditions. The second example in
Figure A-1 demonstrates the recommendation made
in the Human Engineering Guide to Equipment
Design, for use if any lighting conditions less than
one-foot candle can be expected. Although highly
legible, the letters are too large (0.20") for practical
use.

What we recommend is between the extremes cited
above and finds its basis in MIL SPEC recommen-
dations and current applications by a number of
major airlines, An example is shown in the DC-9
checklist in Figure A-3. In that example, the print
sizeis0.15" (14 point) for the primary heading (DC-
9 NORMAL...); 0.125" (12 point) for the checklist
names (i.e., BEFORE STARTING ENGINES); and
0.1" (10 point) for the checklist text. It is also done
in all caps, boldface type, with the exception of the
notes, which are in initial caps with lowercase
following. MIL-C-81222C and MIL-C-38778A
specify the use of 14-point (0.15") type for checklist
headings, and the use of 12-point 0.125") type for

the body of the checklist. Both of these are slightly
larger than that used in the DC-9 checklist and
appear to represent a good compromise between
legibility and practicality.

From the practical standpoint, the use of 12-point
type (0.125") throughout the text of a document
Tesults in 54 lines of type, with 1" margins top and
bottom, onan 8 1/2"x 11" page (i.e., the size used in
this report). The size shown in example two of
Figure 1 (0.2") results in 29 lines on an 8 1/2" x 11"
page with less than 1" margins top and bottom. Since
many checklists contain more than 29 items, this
would result in an increase in the number of pages
required to accomplish a checklist. We feel that
normal checklists should be kept to no more than
one 8 1/2" x 11" page — either laminated or trifold
— if a card checklist is to be used. The reasons for
that are as follows:

a. Many pilots clip the checklists to the yoke or
parts of the window apparatus for use. This is
easy with one page — more than one page
becomes too bulky.

b Having to flip through more than one page to
read normal checklists in amultiple-leg day is
cumbersome.

c. A checklist of one page can be found more
easily and quickly.

d. A single-page checklist is easier to stow and
retrieve when needed.

¢. We feel that anything that promotes ease of use
with a checklist will discourage misuse, or
neglect, of checklists,

Based on the above, our recommendations for print
size and style are as follows:

1. CHECKLIST HEADINGS — 12-point
(0.125") type, all caps, boldface, in a typeface
equivalent to those recommended in the MIL
SPECS. These should be black type ona white
background, or white lettering on a dark back-
ground. The latter is recommended in MIL-C-
1472C, in “Human Engineering Guide to
Equipment Design,” and is currently in use by
Flight Safety Canada, Ltd. in their Canadair
checklists. Flight Safety varies the
according to the type of checklist: white for


rudder and control wheel to control air-
craftheading, maintaining aircraft wings
essentially leveled.”

- The“SYNPHR (synchrophaser) FAIL”
checklist gives a procedure for eliminat-
ing the beat between the engines if the
synchrophaser is inoperative.

Pilots at the career stage of flying for an airline
should not need basic flying lessons. If they
are not aware of the proper techniques by this
time, training would seem a more appropriate
means for correcting this than a checklist.
Including training information in AFMs and
checklists only increases their size and detail,
and makes them more difficult to use for their
intended purpose.

The format and content of a number of the regional
carrier AFMs, Company Operating Manuals, and
checklists that we reviewed indicated a need for
standards and careful oversite concerning their de-
sign and publication. While some carriers provide
their crews with manuals and checklists that are
accurate and easy to use, others do not appear to
recognize the importance of these documents to
flight safety. One of the worst examples was seen in
the “Emergency” checklist of one Part 135 Carrier.
These had been stamped “FAA APPROVAL” and
signed off by a POI (even though not required for a
Part 135 operation) but lacked procedures for 11
“Emergencies” that were in the AFM. There were
several carriers using checklists that were missing
procedures that were specified in their AFMs; a
number of these involving operationally significant
items. Some of these omissions are in violation of
FAR 135.83 (c). This may be symptomatic of the
regional Part 121 and the Part 135 operators, and the
surveillance given them. The interpretation of the
FARs by POIs is sometimes inconsistent, and vari-
able enforcement may result from this. This leads to
practices in the use and design of manuals and to
checklists which are questionable, and which at
times detract from the safety standards intended to
be provided by these documents.

3.3.7 Summary oF Finpines

+ POLICY AND PROCEDURES FOR
CHECKLIST USE

All of the carriers had some direction for the
use of checklists by their crews. The policies

12

varied widely from carrier to carrier, though
not necessarily differing according to the
carrier's size. Some were very detailed poli-
cies, spelled out in operating manuals, cover-
ing all aspects of checklist use, and some were
only notations in the margin of a checklist
noting who was to respond to a challenged
item.

Several NTSB and ASRS reports identified
poorcrew coordination in the use of checklists
as a likely contributor to aircraft accidents.
The absence of detailed policies and proce-
dures concems the responsibilities of indi-
vidual crewmembers in the use of checklists
increases the possibility of poor crew coordi-
nation during safety-critical activities involv-

Dual responses to checklist items were used
by most Part 121 carriers, but by only one Part
135 operator. Many pilots consider multiple
Tesponses to checklist items to reduce safety.
Checklists are frequently done on the roll.
When the heads of both pilots go down, even
for a moment, safety is compromised.

The response “AS REQUIRED” was allowed
by two of the six Part 121 carriers and six of the
nine Part 135 carriers. Many required a spe-
cific response of a quantity or setting in place
of “AS REQUIRED.”

ALPHANUMERICS

The bodies of the checklists varied from clear,
legible 10-point type, all caps, with good print
quality, to six-point type, mixed case, difficult
to read. In some cases, the type size used on
“Emergency” lists was smaller than that used
on the “Normals.” Closely packed six-point
type is difficult to read quickly under any
conditions. Itis easily misread under the stress
of emergencies and/or under low cockpit illu-
mination. The size and resources of the carrier
had no apparent bearing on the legibility of the
checklist: a major carrier had one of the most
illegible checklists examined.

CHECKLIST PRESENTATION
Paper checklists were most commonly used

for“‘Normal” checklists, although one carrier
used laminated cards. Another carrier used a



113.

114,
15.
116.

117.

118.

119.
120.
121.

122.
123.
124.

125.

126.

AIRCRAFT OCCURRENCE

TYPE

WDB Aircraft failed to pressurize, returned
to point of departure

WDB Aircraft landed without clearance

MLG Engine flamed out, single attempt at
restart unsuccessful, landed short of
destination

MLG Aircraft taxied into position on an
active runway, possibly without
clearance

MLG Near mid-air collision

LRG Aircraft landed without clearance

LIT Aircraft crossed an active runway
after instructed to hold short

MLG Aircraft filled with smoke at 37,000’,
declared an emergency and landed
short of destination

WDB Partial runway incursion, caused a
g0-

WDB Deviation from assigned SID

MDT Altitude overshoot on climbout

MLG Aircraft had abnormal lights prior to
vi, continued Tio had engine fire
warning at V2, lights went out
they continued to destination

WDB Engine disintegrated at about V1,
crew aborted, residual fire put out by
emergency crew

MLG Engine not shut down prior to exiting

aircraft

C-17

CAUSE

Neither air conditioning pack was operating, no
checklist for that abnormal procedure, returned and
found a start arm switch in the wrong position,
didn’t catch it on the checklist after

engines, the only checklist for packs inop 1s found
under the expanded checklist for rapid
decompression (77?)

Crew busy with tight approach and doing checklist,

didn't contact tower until after rollout, Mower didn’t
even know they had landed

Used all appropriate checklists, abnormal,
emergency, and normal

Confusion as to controller instructions, capt. called
for last items on the before takeoff checklist which
are normally done only when cleared into position

Aircraft on approach, on autopilot and
autothrottles, crew was busy changing frequencies
and doing the checklist, when looked up the
other aircraft was crossing 300’ above and al
700 out

Making

ed approach for an autoland, doing
ee rhe

forgot to shift frequencies

Copilot got instructions, assumed captain had them,
started to do the checklist heads down and didn’t
catch the crossing, poor crew coordination

Used appropriate checklists and procedures

Crew busy doing checklist, misunderstood
clearance to taxi up to and hold short, taxied
beyond the hold short point

During taxi aircraft received runway changes,
chan; ed SID in FMS, runway reassigned, in doing
the checklist and other duties, SID didn’t get

changed again

Crew busy dodging thunderstorms on departure,
changing frequencies, flying the aircraft, doing
checklist, no altitude waming on the MDT when
Capt. had been flying an airplane that had one

Poor procedures, did an abnormal checklist for an
air conditioning supply temp high, when
maintenance checked the aircraft they found a 1"
hole in the engine where the starter reengaged

Crew followed proper procedure and used
appropriate checklists

Stressful flight, stress resulting from merger, ir
crew coordination, lack of use. of checklig Poo’



CHECKLIST SURVEY (95 returns for 80 mailed)
The reasons for the survey are fivefold. Each reason will have its own set of questions. The reasons are as
follows:
1) Identify layout and other design characteristics of checklists that inhibit or promote easy use;

2) Determine what of flight operations interfere with checklist use, and identify the phases of
flight during which these distractions are most likely to occur;

3) Determine the degree to which checklist procedures are defined in the pilot handbook;
4) Identify variations in checklist use that can be attributed to crewmember characteristics;
5) Identify procedures or design changes that could be used to promote error-free checklist use.
1. LAYOUT AND DESIGN OF CHECKLISTS
1.1 Types of checklists you have used (please check types used and circle type currently used)...
currently used

a. Paper checklist 25 Yes 74 No 21
b. Laminated card(s) 52 Yes $2 No 43
c. Electronic (CRT) 1 Yes 8 No
© Does the display replace
another display, such as
weather radar Yes 4 No 1
d. Mechanical scroll 1 Yes No =
e. Mechanical pointer Yes § No
f. Mechanical slide Yes No 86
g. Toggle switch/annunciator light
combination Yes 9 No9l
h. Have you used, or do you now use, .

a mix of the above (i.e., - paper
checklist & mechanical slide) Yes 19 No 74
@ If "yes," are the "normal"
checklists segregated from the
"emergency" and "abnormal" lists Yes 16 No 6

(please explain in what way).

i. Do you see an advantage to a mix

of checklist types? Yes 11 No 77
(please explain)
1.2 Does the “silent” checklist have a
place in airline cockpits? Yes 71 No 23
1.3 Of the following checklists, which do you feel should be "challenge/response" and which should be
"silent"?
challenge/response silent
@ Airplane acceptance 32 38
© Before start 71 3
© Before taxi ol pty
© Before takeoff 16 9
© Climb 16 o
© Crise 12 8
© Descent/In range aB
@ Before landing B 3
© After landing 21 3S
e i i pz}

D2


lists were easier to use in all cockpit
lighting conditions; that they were easier
to get at; that they were easier to use in all
operating conditions; that they facilitated
quickeruse; and, thatifitems were skipped,
they could be more easily retumed to than
with a paper checklist.

The suggestion of using automated (elec-
tronic) checklists wherever possible met
with a positive response. Fifty-eight point
six percent of the respondents felt it would
be helpful, but the following qualifica-
tions are typical:

“No matter how they are presented, au-
tomated orclay tablet, they must be read.
and followed.”

(This indicates that at least one of the
respondents is doubtful that reading and
following checklists is done consistently
and uniformly.)

“I don’t like the idea of automated or
mechanical lists because of the frequent
changes to our checklists. The cost of
changing these would make it harder to
get the company to make changes.”

- MECHANICAL MARKERS

The suggestion to “use a mechanical
marker to mark checklist progress” met
with little enthusiasm. Many felt it was an
archaic concept. One said he already used

- “called a finger.” However, in
jumpseat observation rides we had the
opportunity to watch a crew using a me-
chanical slide checklist for “BEFORE
TAKEOFF” and “BEFORELANDING.”
They were enthusiastic aboutit, felt that it
provided a positive indication of checklist
progress, and eliminated the problem of
losing one’s place in interrupted check-
lists.

+ COLOR CODING

When asked if they felt “use of color coding
for easy identification of checklists” was a
good idea, 83.7% said “Yes.” This is used by
some airlines, both Part 121 and Part 135.
Some of the comments elicited were:

15

- “Our current procedure.”
- “For Emergency checklist at least.”

3.4.4 CHECKLIST INTERRUPTIONS
Checklist interruptions come in two varieties:

Interruptions to checklist use.
+ Interruption of operational tasks by checklist

use, such as can occur during a busy approach
or an emergency.

While most of the respondents felt that inter- -

Tuptions were a problem, noteveryone agreed.
One sheltered soul said:

mnised by interruptions, J have never seen
6 m dave t

He was, however, adefinite minority ofone, in
that respect, as the following survey results
regarding interruptions will show.

The respondents were questioned about the
importance of potential interruptions to check-
list use, and asked to rate them on a scale of 1
to 10, with 10indicating very important. While
afewscored some of those listed very high, the
average scores were middle of the scale. The
top-ranked four were as follows:

« ATC communications

“ATC should be educated/indoctrinated
to the-hazard(s) associated with multiple
frequency changes (which takes attention
from the checklists/lookout doctrine/navi-
gating, etc.) during descent/approach
(VFR and in the weather). This also re-
Moyes the pilot not flying from the ‘net-
work’ at a critical time. Frequency chang-
ing requires intense attention inside the
cockpit...”

Others voiced similar sentiments:

“Most disruptive area of operation and
checklist interruption: ATC transmission
in initial approach area. Try and read a
checklist between CIVET (52.4 miles NE
ofLAX) and LAX ona VFR day. Typical
to have six frequency changes, a dozen