# Presentation: 659767

## When Things Go Wrong:Flight Crews and Emergencies

**Barbara Burian, Ph.D., FRAeS**
- NASA Ames Research Center

- The Emergency and Abnormal Situations study was funded through NASA’s former Aviation Safety and Security Program.

## Emergency and Abnormal Checklists

- Boeing 777 ECL

**Notes:**
Checklists are essential tools that crews use to help them respond appropriately to emergency and abnormal situations.  Two examples of checklists – B777 ECL and a paper checklist (paper checklists can look very different from this one – this is just one example and I used it because it is already in the public domain (included in an NTSB accident report). 

## Checklists and Procedures

*Checklists and Procedures*

- Checklist Design

- Checklist Content

## Checklist Design Factors 

- (Paper, Electronic, EFB)

**Physical Properties and Interface           -   size, weight, materials, integration w/displays****	**

**Typography and use of Symbology        -   font, font size, boldface, intuitive symbology **

**Layout, Format, Organization****	**** -   visual look, arrangement, “white” space**

**Purpose                                                     -   fix, troubleshoot, manage situation, guide CRM **

**Objective **(of checklist item)**                      -   direct action, inform, assess, make decision**

**Length and Workload                               -   physical length, timing length, workload**

**Nomenclature & Abbreviations****	****   -   terms, labels, abbreviations**

**Language, Grammar, & Wording            -   English?, verb tense, reading difficulty, clarity, ****	****       ****	****   ****	****                       orientation/perspective, directiveness**

**Level of Detail  ****	****                                   -   amount of information provided**

**Engineering Completeness                     -   all necessary steps included **

**Engineering Coherence                           -   order of steps/timing makes “sense” to aircraft**

**Logical Coherence                                   -   order of actions makes sense to the pilot **
**	****	****	****	****       ****and make “sense” operationally**

**Progression & Checklist Navigation      -   movement within & between checklists/manuals**

**Access                                                      -   finding correct checklist, prime real estate pgs.**

**Notes:**
List of checklist design and content areas

## Checklists and Procedures

*Checklists and Procedures*

- Checklist Design

- Checklist Content

**Notes:**
Paying attention to design and content isn’t enough – attention to the context in which emergency and abnormal situations occur is essential

## Pack Trip

*Pack Trip*

- Master Caution Alert sounds
- Crew Identifies that Pack has tripped off
- Crew completes 4 step procedure
- Flight proceeds normally

**Notes:**
Illustration of range of situations – a pack trip is generally pretty benign and easily handled – not much increase in stress or workload – some pack trip checklists may have more than 4 items

## ValueJet 592 - In-flight Fire Florida Everglades

*May 11, 1996*

- 14:04:09    Takeoff
- 14:10:03     [chirping sound] “’bout to
-                    lose a bus”
- 14:10:15    “Got an electrical problem”
- 14:10:20    “We’re losing everything”
- 14:10:22    “We need to go back to
-                    Miami
- 14:10:25    “Fire, fire, fire, fire!” (from
-                    cabin)
- 14:10:32    “Uh, 592 needs immediate
-                    return to Miami” (to ATC)
- 14:13:43     CVR stops recording
- 14:14          ValuJet 592 disappears
-                    from radar

**Notes:**
Illustration of situation at the other end of the continuum.  Note how much time elapsed between the time they knew that had an electrical problem and when they knew they were on fire.

## Requirements of the Emergency or Abnormal Situation

## Requirements of the Emergency or Abnormal Situation

- Operational Requirements

**Notes:**
But context includes other issues beyond the specifics of a given situations – see this slide and then next two.  Checklists need to be written to address all of these different contextual issues.

## Requirements of the Emergency or Abnormal Situation

- Operational Requirements

- Aircraft and Systems

## Requirements of the Emergency or Abnormal Situation

- Operational Requirements

- Aircraft and Systems

- Philosophies and  Economic Constraints

- Manufacturer philosophies
- Company philosophies and policies
- Types of checklists available (electronic, paper)
- Philosophy of checklist use and functionality of electronic checklists
- Cost-benefit tradeoffs, e.g., diversions
- Updates and revisions

## Checklists and Procedures

*Checklists and Procedures*

- Checklist Design

- Checklist Content

- Context

**Notes:**
Addressing checklist design, content, and contextual issues is still not sufficient when developing effective emergency and abnormal checklists and procedures.  Human performance capabilities and limitations under the high stress and high workload that characterize emergency and abnormal situations must also be addressed.

## Requirements of the Emergency or Abnormal Situation

- Operational Requirements

- Human Performance Capabilities and Limitations under High Stress and Workload

- Aircraft and Systems

- Philosophies and  Economic Constraints

**Notes:**
What is meant by human performance capabilities and limitations? – three domains: cognitive, motor, affective (emotional)

## Human Performance under Stress 

*Human Performance under Stress *

- Well-learned motor skills
- remain robust and relatively unaffected by stress

- Our simulator training really paid off.  This was my first engine shutdown in 20 years of flying and it felt like I had done it a thousand times before!
-                                        (ASRS Report, Accession #466167)

**Notes:**
The good news – well-learned motor skills are relatively unaffected by stress (quote from an Aviation Safety Reporting System (ASRS) Incident report)

## Human Performance under Stress 

*Human Performance under Stress *

- Tunneling
- narrowing of human attention
- restricts scanning of environmental cues
- narrow focus on most salient or threatening cues
- yields poor differential diagnosis of situation
- Working Memory
- capacity and length of time information can be held decreases
- when exceeded – difficulty performing mental calculations, problem solving, making sense of disparate pieces of information, shifting mental sets (concurrent task management)
- Tendency to Rush

**Notes:**
The bad news – cognitive performance is negatively affected by high stress and workload.

## Human Performance under Stress 

*Human Performance under Stress *

- We did find communication difficult and the use of oxygen masks, intercom, trying to talk to ATC was a handful.
- At night made it that much harder to read/accomplish checklist items.  Turning cockpit lights on sooner would have helped.
-                                        (ASRS Report, Accession #472755)

**Notes:**
This is one of my favorite ASRS quotes.  Seems so simple – turn on the lights to see the checklist more easily – but under high stress and workload – even the “obvious” is not so obvious.

## How Checklists are Used

*Checklists and Procedures*

- Checklist Design

- Checklist Content

- Context

- Human Performance Capabilities

## Checklists and Procedures: 

- Example: Saudi Arabian Flight 163, August 19, 1980

- Crew searched unsuccessfully for several minutes for a cargo fire checklist in the Abnormal section of the QRH.
- The checklist they were looking for, but never found, was in the Emergency section of the QRH instead.

*Checklists and Procedures: *
*Context and Human Performance Considerations*

**Notes:**
Beginning with this slide, most of the rest (up to the countermeasures at the end) are examples of how emergency and abnormal checklists have failed to accommodate human performance limitations under stress and/or the demands of the context in which emergencies occur.  QRH stands for Quick Reference Handbook – a manual of emergency and abnormal checklists – somewhat similar to the pocket checklists.

## Checklists and Procedures: 

- Example: Six Different Checklist Titles for the Same Condition –
-                 B737 no flap, no slat

| All Flaps Up Landing |
| --- |
| Flaps – All Flap and LED Up Landing |
| No Flap/No Slat Landing |
| Symmetrical Non-Normal Trailing Edge Flaps or No Flaps  |
| No Trailing Edge Flaps  |
| Alternate Flaps Operation  |

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Checklists and Procedures: 

- Example: Pilots routinely make errors in correctly recalling “memory items”

| Air Carrier/Manuf. | N of CL with MI | Total
N of MI | Action Item MI | Conditional MI | Note MI | Other MI |
| --- | --- | --- | --- | --- | --- | --- |
| A  Classic  | 23 | 120 | 93 | 21 | 3 | 3 |
| B  Classic  | 4 | 15 | 13 | 1 | 0 | 1 |
| C  Classic  | 16 | 112 | 73 | 16 | 21 | 2 |
| D  Classic  | 5 | 17 | 15 | 2 | 0 | 0 |
| Boeing Classic | 16 | 113 | 73 | 16 | 22 | 2 |
| E  NG  | 9 | 20 | 17 | 3 | 0 | 0 |
| F  NG  | 3 | 11 | 10 | 1 | 0 | 0 |
| G  NG  | 12 | 45 | 37 | 5 | 2 | 1 |
| H  NG  | 10 | 44 | 35 | 5 | 2 | 2 |
| Boeing NG | 18 | 129 | 83 | 19 | 24 | 3 |
| Boe. NG – Rev. | 13 | 77 | 52 | 10 | 14 | 1 |

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Checklists and Procedures: 

- Example: Applying multipliers to landing distances

-  **1.7**

**1.55**

-  **1.8**

**1.35**

**1.15**

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Checklists and Procedures: 

- Example: Information missing from checklist – Valujet 558, Jan. 7, 1996

**The missing information **
**was included in the AOM expanded checklists but was never transferred to the QRH checklists**.

**AOM**

*Checklists and Procedures: *
*Context and Human Performance Considerations*

**Notes:**
Crew reset the circuit breakers on very short final (around 100 ft AGL) – put the aircraft immediately in ground mode – spoilers deployed and the aircraft immediately lost lift, slammed down onto the ground, lost its nosewheel and bounced back up into the ground – Captain performed a go-around and landed (sans nosewheel) on a different runway – closed the airport because of debris all over one runway and a huge gouge cut down the middle of the other

## Checklists and Procedures: 

- Example: Confusing and complex wording

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Checklists and Procedures: 

- Example: Crew confusion – FedEx 1406, September 5, 1996

- FE was confused by step 5 and did not complete steps 6 and 7

- Items Pertaining to Adjusting Cabin Altitude or Flight Level

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Example: Complex Navigation

**Notes:**
Orange circles denote decision points (i.e., conditionals), arrows represent progression through the checklist based upon which conditionals are true.  Note the green arrow coming from the first decision point (Enroute terrain clearance is a consideration) it skips you past several items, down to the bottom and then takes you back up – big green loop’s purpose is to bypass the three items at the top of the page (see top magenta arrow) – not a very elegant solution – imagine getting interrupted in the middle of the loop and trying to figure out where you left off when you return to the checklist.  Second magenta arrow point to a note that will always be skipped over and actually pertains to the decision made in the third orange circle.

## Accomplishing the [immediate action] item for cockpit/cabin smoke on the ground in the XXX aircraft induces the abnormal procedure of equipment overheat due to the step of the turning off left and right recirculation fans, the left recirculation fan being the primary equipment cooling on the ground. 

- Accomplishing the [immediate action] item for cockpit/cabin smoke on the ground in the XXX aircraft induces the abnormal procedure of equipment overheat due to the step of the turning off left and right recirculation fans, the left recirculation fan being the primary equipment cooling on the ground.
-                                    (ASRS Report, Accession #473359)

*Checklists and Procedures: *
*Context and Human Performance Considerations*

- Example: Checklist actions cause another abnormal situation to
-                 occur

## Hydraulic caution light illuminated while taxiing...I completed the QRH checklist...We rolled to a stop in the grass...A very poorly written QRH emergency checklist. CALLBACK: ...The checklist is for use in-flight, not on the ground. 

- Hydraulic caution light illuminated while taxiing...I completed the QRH checklist...We rolled to a stop in the grass...A very poorly written QRH emergency checklist. CALLBACK: ...The checklist is for use in-flight, not on the ground.
-                                        (ASRS Report, Accession #437817)

*Checklists and Procedures: *
*Context and Human Performance Considerations*

- Example: Checklist actions not appropriate for situation

## Checklists and Procedures: 

- Example: Swissair 111, Sept. 2, 1998

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Checklists and Procedures: 

- Example: Swissair 111, September 2, 1998 - continued

- Time from first abnormal odor until Swissair 111 crashed in the ocean:
-       20 minutes, 40 seconds

*Checklists and Procedures: *
*Context and Human Performance Considerations*

## Human Performance and Context in 

*Human Performance and Context in *
*Emergency and Abnormal Checklists *

- can be written to address the multiple contexts in which emergency and abnormal situations occur
- can be designed to accommodate many human performance limitations that occur under high stress and high workload

- The Good News: Emergency and Abnormal Checklists and Procedures

## To Ensure that Context is Addressed:

*Addressing Context in*
*Emergency and Abnormal Checklists*

- Make sure checklists are designed for full range of the scenarios for which they will be used (e.g., pressurization: slow leak – to – explosive/rapid decompression)
- Make sure checklists are appropriate for all phases of flight and aircraft configurations for which they may be used (e.g., on ground, in-flight, throttles at idle)
- Make sure checklists address relevant environmental conditions (e.g., icing) and loss of pertinent equipment that has been MEL’ed.
- Conduct a **realistic**** **timing assessment of crucial checklists and procedures (length of completion).
- Conduct a **realistic** workload assessment of checklists and procedures, especially when conducted during various phases of flight.

## To Ensure that Tunneling is Addressed:

- Draw crew attention to environmental or situational cues that support diagnosis
- Draw crew attention to environmental or situational cues that contraindicate diagnosis
- Cues specified must be ones that flight crews are able to assess

*Addressing Human Performance under Stress in Emergency and Abnormal Checklists*

## To Ensure that Working Memory Limitations are Addressed:

- Provide information or resources (e.g., EFB) to diminish/eliminate memory load and need to perform mental calculations
- Place remaining “memory items” on quick reference cards/QRH cover (paper and unannunciated electronic checklists)
- Integrate all needed information (tables, normal checklists) with emergency and abnormal checklists – “get in, stay in”
- Make sure all checklists are complete and ramification of crucial steps is provided (before the step is to be carried out)
- Provide “purpose of item” and “purpose of checklist” statements
- Provide information describing aircraft performance limitations
- Provide information describing remaining aircraft capabilities
- However, don’t go overboard with providing so much information that checklists take forever to go through/read

*Addressing Human Performance under Stress in Emergency and Abnormal Checklists*

## Barbara Burian, Ph.D.

**bburian@mail.arc.nasa.gov**
**Emergency and Abnormal Situations Study**
**http://human-factors.arc.nasa.gov/eas**

## The Context in Which Emergency and Abnormal Situations Occur

- JetBlue 292

- Aloha 243

- Air Canada 797

- Explosive Decompression

- Misaligned Nosegear

- In-flight Fire

**Notes:**
Illustrations of what is meant by context – different types of situations with different levels of time criticality and lethality levels

## In-flight Smoke, Fire, Fumes Integrated Checklist

- Courtesy of United Airlines