# Presentation: 363464

## Applying Error and Threat Management Concepts to Soaring Operations 

- Key Dismukes
- Chief Scientist for Aerospace Human Factors
- NASA Ames Research Center
* *

**Notes:**
(These are informal notes to accompany my power-point presentation.  I will be working these notes up for a formal paper later.)

	If i don’t change how you think about flight safety today then i’ve failed in my job and you should fire me.  I do not claim to be the best soaring pilot around or the safest—plenty of people in this room are better pilots than i.  I’m here because of my professional experience—i lead a nasa research team that conducts research on crew performance, human error, and aviation safety.  

## Most Aviation Accidents Attributed to Pilot Error

- What does this mean?
- Lack of skill, vigilance, or conscientiousness?
***OR***
- Inherent vulnerability to error?
- Substantial body of scientific research on pilot performance, error, and safety
- Mostly directed to airline ops
- Lack database of human factors in soaring accidents (NTSB and ASRS reports)
- Many principles from airlines and military flight safety can be applied to soaring
- Will illustrate with soaring examples

**Notes:**
I think you all know that most aviation accidents in all domains are attributed to pilot error rather than equipment failure.  But what does this mean?  Every pilot i’ve every met has opinions on aviation accidents and safety.  It may surprise you to know that there is a substantial body of scientific research on pilot performance, error, and safety.  Mostly that research has been directed at the issues faced by airline pilots—we collaborate extensively with airline training and operations departments.  Unfortunately hardly any research has been directed to the issues of soaring safety.  When an airliner crashes the ntsb conducts an extensive investigation that examines crew performance issues in great detail.  Increasingly the ntsb is drawing upon scientific research in human performance issues, and we in turn use ntsb reports as one of our databases.  Unfortunately no database of this sort exists to help the soaring community.  Furthermore soaring pilots—unlike airline and GA pilots-- rarely contribute to the asrs, a massive database of incidents that but for the grace of god could have become accidents.  It is hard to make any sort of meaningful analysis without data—opinions unsupported by data do little to improve safety.  I urge the soaring community to find ways to develop databases of human factors and situation factors involved in sailplane accidents and incidents.

	Fortunately i can still find something to talk about today because many of the principles that have emerged from airline flight safety do apply to any type of flight operation.  Today i will talk about some of those principles and try to illustrate them with examples from my own soaring experience. 

## Dispense with Fallacies

**Fallacy:****	**Pilots who have accidents lack “the right 			stuff”
**Truth:**	- 	Not supported by data
- 		- 	With increasing skill pilots take on greater 			challenges
- 		- 	Experts in all professions make errors
**Fallacy:****	**A flight operation is either “safe” or “unsafe”
**Truth:**	- 	Every operation has finite degree of risk
- 		- 	Identify and assess risks and develop plan of 		action

**Notes:**
But first let me mention some commonly held fallacies we should dispense with:
	1)  Pilots who have accidents lack “the right stuff”, either they lack skill or they were goofing off.  But the actual accident data do not support this contention.  For one thing, in soaring especially, as pilots develop skill they take on more challenging flying.  Their skill works in their favor, but challenging flights expose them to greater risk.  Most important, from a scientific perspective, experts in all professions make errors, not because they are goofing off, but because error is a normal manifestation of the way the human cognitive system works, which i will say more about later.
	2)  A flight operation is either ‘safe’ or ‘unsafe’.  But that attitude about safety is itself unsafe.  Professionals recognize that every operation has a degree of risk, which means there is always some finite probability of an accident.  The professional approach is identify and assess the risks and to develop a plan of action to minimize the risks.  More about this later.

## Line-Oriented Safety Audits (LOSA)

**Airline crews typically make one or more errors on most flights**
- In spite of training, skill, and frequent practice
**Humans perform tasks far beyond capabilities of computers**
- Work with incomplete and ambiguous data, interpret diverse data sets, project downstream consequences
- Humans are in airline cockpits to deal with the unexpected
**Cognitive features that enable unique human abilities also produce vulnerability to characteristic forms of error**
-  No matter how skillful and conscientious -- error rate is never zero

**Notes:**
Pilots in major U.S. airlines are highly trained and quite skillful, they get a lot of practice in normal operations.  Yet recent observational studies reveal that crews typically make one or more errors on every flight, and more on challenging flights.  They make these errors in tasks they perform day in and day out and even though they are motivated not to make mistakes, in part because they know they are being observed.  Why does this happen?  To answer that you have to know something about the human cognitive system.  Humans can perform tasks far beyond the reach of any computer.  For example, humans are able to work with incomplete and conflicting information, humans can understand the implications of apparently unrelated sets of information, and humans can project the downstream consequences of current events.  It is now technically possible to build aircraft that can conduct the entire flight without human intervention.  However we keep humans in the cockpit because only humans can deal with the unexpected.  But this extraordinary ability of humans also means that we cannot completely avoid making errors.  No matter how skillful we are at a given task, we will always make errors on some trials—the error rate may be very low but it can never be zero.  That’s the way our brains are wired.

## Research on Two Major Domains of Pilot Error

- 1)**	****	****Judgment and Decision-making**
- 2)**	****	****Prospective memory - remembering ****	****	****	****	****to perform intended actions**
**Talk will be technical in places**
- Illustrate with real-life soaring situations
- Groundwork for practical countermeasures
**Address mistakes of experts, not novices**
- Level of expertise of private pilot rating in gliders
- How cognitive processes operate at this level of expertise

**Notes:**
I’m going to talk about two major domains of pilot performance that are vulnerable to errors that often have serious consequences:  1) judgment & decision making and 2) prospective memory—remembering to perform intended actions.  To try to explain why we are vulnerable to error in these domains i am going to have to talk about some technical aspects of human cognition, however i will try to illustrate these technical abstractions with real-life examples from soaring.  If you get lost in the technical part, do not dispair, all i am doing is laying groundwork for the latter part of the my talk in which i will identify specific practical ways you can reduce your vulnerability to error when flying.

	Let me emphasize that i will be talking about the mistakes made by experts, not the mistakes by novices in the early stages of learning tasks—that’s the subject of a different talk.  I know many of us often feel more like novices than experts when we go cross-country, but from a cognitive science perspective, being an expert does not require willing a national soaring contest.  Anyone with a private pilot rating has substantial domain expertise, and i am going to be talking about the cognitive processes that make people with that or any level of expertise vulnerable to characteristic forms of error.

## Judgment and Decision-Making

**Most common human factor cited by NTSB**
- But this displays hindsight bias
**Experts in their training make decisions in ways other than formal analyses**
- Formal analysis =	1) listing and evaluating all relevant aspects of situation
- 		2) identifying all relevant options
- 		3) assessing pros and cons of each option
- 		4) assigning weighted score to each option
**Gary Klein et al. studied firefighter scene commanders “Recognition-primed decision-making”**
- Quickly scanned scene
- Identified situation in terms of prototypes from memory
- Appropriate solution pops into mind
**Naturalistic decision-making**
- Demonstrated in work of professionals in many fields
- Not something we choose -- happens automatically as we develop experience

**Notes:**
The most common factor in aircraft accidents cited by the ntsb  is judgment and decision-making.  However, simply saying that the crews made the wrong decision, leading to an accident, is misleading and displays what cognitive scientists call ‘hindsight bias’.  After the accident we of course know that the decisions made were wrong, but at the time in the situation the crew faced, the decisions they made often seemed quite appropriate to the situation.  

	There is now a substantial body of research showing that  all experts working in their professional domains—not just pilots—make decisions in ways very different from formal analysis.  By formal analysis i mean listing and evaluating all relevant aspects of a situation, identifying all relevant decision options, assessing the pros and cons of each option, assigning a value to each option based on the potential outcomes, and choosing the option with the highest value summed across all aspects.  For example, in a seminal research study Gary Klein and his colleagues found that firefighter scene commanders when arriving at the scene of a fire, do not systematically list all the attributes of the fire, 1, 2, 3,.;4,...identify all possible ways of managing the fire, A,B, C, D... and then rate the pros and cons of each management method in terms of each attribute of the fire and the come up with a composite score for each management method.  If they followed such an analytical process the building would probably burn down before they could make a decision.  Rather what these expert firefighters did was the quickly scan the fire scene.  Immediately and automatically they identified what type of situation they faced in terms of prototypical situations they had previously dealt with.  At the same time the most appropriate method of managing the fire also popped automatically into mind.    Klein called this process ‘recognition-primed decision making’, which is part of a larger field of study called naturalistic decision making to distinguish it from the kind of formal analysis that economists traditionally assumed that humans make to arrive at rational and optimal decisions.  Naturalistic decision-making has now been demonstrated in the work of professionals in many fields, including pilots, and in the lives of individuals in their everyday activities.  Naturalistic decision-making is not something we choose to do—it happens automatically as we develop experience and skill.  

## Comparison

**Formal problem-solving methods**
- Slow
- Serial processing
- Require substantial mental effort
- Difficult to use under time pressure, stress
- 	(But can be used to uncover hidden aspects, downstream consequences)
**Naturalistic decision-making**** **
- Automatic
- Fast
- Parallel processing
- Less mental effort
- (But subject to characteristic biases that can cause errors)

**Notes:**
Naturalistic decision-making has certain advantages over formal analytical decision making.  Formal methods are slow, involve serial processing, and require considerable mental effort.  They are quite difficult to use under time pressure and stress.  In contrast  naturalistic decision making is automatic, fast, involves parallel processing, and requires much less mental effort.  However naturalistic decision making is also subject to systematic biases that can lead to unwitting errors.

## Cognitive Biases and Heuristics

**Anchoring and adjustment**
**Assimilation bias**
**Availability heuristic**
**Base rate fallacy**
**Confirmation bias**
**Conjunction fallacy**
**Conservatism**
**Endowment effect ****(aka Status quo bias)**
**Escalation of commitment**
**Estimation overconfidence**
**Expectation bias**

**Familiarity bias ****(similar to Assimilation bias)**
**Favor of causal explanations**
**Framing effect**
**Fundamental attribution bias**
**Future discounting**
**Omission bias**
**Prior hypothesis bias**
**Reasoning by analogy**
**Recency bias**
**Representativeness heuristic**

**Notes:**
.  This is a list of systematic biases that naturalistic decision-making is subject to.  I won’t take time to explain each of these terms to you, but i am going to give you two real-world personal soaring experiences that will illustrate how some of these biases influence decision-making.

## Aero-retrieve from Indian Valley Airporta personal experience

**Notes:**
Last fall i flew in the annual Octoberfest mini-contest at Williams.  I will tell you in advance that what happened on this flight was not my finest hour.  I escaped mishap more by luck than skill.  Soaring conditions were quite weak and i ended up landing out at Indian Valley airport, which is not actively maintained by the owner.  I landed without difficulty, assessed the field for aero retrieve, and then called on the radio for Kenny to come tow me out.  Overall the field looked fairly good to me:  the runway is long and there are no obstacles, and i would have places to land in case of a rope-break.  The wind was calm.  I did notice two potential threats.  There was a herd of horses in the field, however i shooed them off and they kept their distance.  More important, much of the runway was thinly covered in with something like thistleweed, one to three feet high.  The weed was dry and brittle and easy to push down.  The weeds did not cause any problem on the landing, but because i would not have a wingrunner i was a bit apprehensive that the down wing would snag in the weeds and pull me off the runway.  The runway is maybe 60 feet wide, with shallow drainage ditches running parallel along both sides of the runway.  

Kenny landed with the towplane and we put the sailplane in the part of the runway where the weeds were lowest and thinist.  I stamped down the weeds for several yards in front of the wing and propped the right wing up partway on some dried horse dung to reduce the ground drag. I had experienced no problem with weeds dragging the wing on the landing so i assumed that once i got the wing up i would have no problem.

Kenny pointed out that the horses’ hooves had cut the runway surface up into loose dirt, which might cause a dust problem on takeoff.  We decided to cock the towplane to the side so the propwash would not come straight down the runway.  We set everything up and arranged our radio signals.  Kenny powered up the towplane and we started rolling.  I held hard left rudder and left stick to get the wing up as quickly as possible.  The wing did in fact come up quickly.  What i had not thought of when we cocked the towplane was that this put the propwash over just the right wing, not the left wing.  The combination of propwash with my control inputs slammed the left wing down on the ground before i could react and my efforts to raise it with opposite controls had no immediate effect—the wing was down in the weeds.  At the same time i went IMC—the towplane and the landscape  disappeared in a cloud of dust.  I was able to keep my orientation by looking over the canopy rail at the ground and i saw that i was about to go in the ditch.  I yanked the flaps down and pulled back on the stick and to my relief the ASW 20 immediately broke ground and i was out of the cloud. 

## Cognitive Factors Contributing to My Flawed Judgment and Decision

**Representativeness bias**
**Situation framed by matching similar prototypes in memory**
**Fair amount of experiences in off-field landings (50-60?)**
- Appeared to be good match to previous retrieve
**Statistical sampling problem**
- Previous successes may not give true risk probability (wrong in either direction)
**Recognition-primed decision-making**
- Does not inform individual of downstream consequences of novel actions: cocking the tow plane
- Does not anticipate consequences of novel combinations of factors: 	loose dirt, cocking the tow plane, weeds, narrow runway, gullies
- Sunk costs: invested time in aero retrieve
**Framing of situation and goal: “make aero retrieve work”**
**Wishful thinking: more attractive option (aero retrieve) may have biased perception of risks**

**Notes:**
Obviously my judgment and decision-making in this episode were badly flawed.  But simply telling myself that what i did was dumb and resolving never to do that again does nothing to protect me from making dumb decisions in situations that are not identical to this situation.  However by identifying the cognitive biases that undermined my decision-making, we can identify some general principles that may help me and you protect ourselves in  wide range of decision-making situations.  So i’m going to go back through this incident and point out some of the ways naturalistic decision-making may have blinded sided me.

When individuals encounter a familiar situation their brain automatically compares the situation with previous situations and generates a mental perspective of the situation framed in terms of previous experience—this is called the representative-ness heuristic—my brain is matching the current situation to prototypical representations of previous experience in memory.  I used to fly 1-26s, which is an excellent way of meeting farmers.  In 35 years of flying sailplanes i may have made 50 or 60 off-field landings and a fair number of aero retrieves.  The scene at Indian Valley airport generally matched my memory prototype of a field from which an aero retrieve can be made safely.  But there is a statistical sampling problem here.  Let’s say that the reality were that there is a average 1% change of an accident in the aero retrieve situations i lumped together in my mind.  I might make 50 aero retrieves with no problem but if i keep doing it long enough something bad will happen.  So recognition-primed decision-making often fails to give an accurate picture of the degree of risk.  And this might work in either direction.  If i had had an accident on of those earlier aero retrieves than my mental representation of risk might be too large rather than too small.

Another shortcoming of recognition-primed decision-making is that it does not inform us of downstream consequences of our actions if those actions are novel.  I had never cockped the towplane sideways before and thus did not anticipate that the propwash hitting only the right wing might cause trouble.  A related  shortcoming is that recognition-primed decision-making does not anticipate the consequences of novel combinations of factors.  In this incident, as in most accidents, several factors conspired to work against me:  the loose dirt on the runway, cocking the towplane, the weeds, the narrowness of the runway, and the gullies along the runway.  

Still other cognitive biases may have been at play that afternoon.  You may have heard of the concept of ‘sunk costs’ the proclivity people have, once they have committed themselves to a project to keep investing in that project even after it starts failing and it would be better to cut one’s losses and get out.  After i called Kenny to come get me i was psychologically invested in making the aero retrieve work and may have been unconsciously reluctant to send him home and call for a ground retrieve.  Another aspect is that i framed the situation confronting me as how to make the aero retrieve work, rather than framing the situation more broadly as finding the best way  to get home.  Finally, there may have been an aspect of wishful thinking:  aero retrieve was a much more desirable option because a ground retrieve would have taken several hours and my friends and i would have missed the BBQ (assuming i had any friends that devoted).  Conceivably the much greater attractiveness of the aero retrieve unconsciously biased my perception of the risks.

## Implications of this Episode

**Natural decision methods:**
- Fail to alert us to hidden dangers
- Sometimes bias perception of degree of risk of options
**Judith Orasanu: “Plan continuation error”**
- Failure to revise original plan when circumstances change
- One of most common forms of error in airline accidents

**Notes:**
To pull it all together, the way we typically and naturally make decisions may fail to alert us to hidden dangers lurking in a familiar situation and may bias our perception of the degree of risk involved in the options available to us.  A colleague of mine, Judith Orasanu, has been studying decision-making among airline pilots and has found that one of the most common forms of decision error in accidents is plan continuation error.  In these situations crews start out with a plan that is perfectly appropriate to the situation, however as the flight progresses the situation changes making the original plan not so good.  In plan continuation error the crew fails to recognize the need to change the original plan even though they encounter evidence that conditions have changed.  

## Ways to Reduce Vulnerability to Decision Biases

**Be aware of limitations and biases inherent in normal cognitive processes**
**Explicitly identify your assumptions**
- Example: wind direction for off-field landing
- Voicing assumptions may prompt pilot to search for contrary evidence
- Share assumptions with other personnel
**Ask: “What if ...?”**
**Ask: “Is anything different today from previous encounters?”**
**If several unusual aspects present, think how they might interact to produce downstream consequences**
- Individual factors may be benign but combine to produce threat
**Always have a back door**

**Notes:**
Is there a way we could reduce our vulnerability to these types of decision bias?  I suggest we could protect ourselves in several ways:
1)  Be aware of the limitations of natural decision-making processes.  This will help protect you from overconfidence and encourage you to search for hidden dangers.
2)  Explicitly identify your assumptions.  For example, if you are setting up an off-field landing you might say to yourself:  i assume the wind is out of the north because it was out of the north when i took off.  Voicing your assumptions may prompt you to search for contrary evidence.  If another person is with you, for example, the tow pilot, speak your assumptions out loud—the other person may have made different assumptions.
3)  Ask:  What if...?  What if the wind is not out of the north—do i have enough room to stop before the fence?
4)  Ask:  Is anything different today from my previous encounters?  Maybe i have landed out in this field before and know it is a good field though narrow and forget that the previous time i was in a 15 meter configuration but today i have my 18 meter tips on.        
5)  If several unusual aspects are present, think especially about how those unusual aspects might work together and produce downstream consequences.  The loose dirt, the cocked towplane,  the weeds, and the ditch by themselves were not a major threat, but together they were.
6)  Always have a back door.  If things don’t work out the way you expect, it may be too late to start developing a new plan—have something ready.

## Prospective Memory (PM)

**Remembering to perform intended actions**
**Common in everyday life - in aviation can be fatal**
- Several major airline accidents
- Several friends killed or injured because failed to hook up a control rod during assembly
**Will discuss prototypical PM situations in soaring**
- Errors most likely in presence of interruptions, distractions, or concurrent task demands

**Notes:**
Let’s move on to another major source of pilot error that my research group is studying:  prospective memory—remembering to perform intended actions.  In everyday life we often forget to do something we intended, and usually this is just a frustration.  But in aviation forgetting to do something can be fatal.  There are have several major airline accidents in which the pilots or a controller forget to perform a normal procedural step.  In soaring i have had several friends killed or injured because they failed to hook up a control rod while assembling their sailplane.   

Our research shows that there are three prototypical situations in which pilots, like all other individuals, are vulnerable to forgetting to perform an intended task, and that these errors are most likely in the presence of interruptions, distractions, or concurrent task demands.

## 1.  Performing Habitual Tasks from Memory

**Habitual tasks deeply encoded as procedural memory**
- Not likely to forget how to perform task
- Vulnerable to omitting a step when interrupted or distracted
**Habitual tasks (e.g., assembling frequently used sailplane)**
- Require minimal mental effort: each step of task “pops” into mind automatically
- Executing each step automatically triggers retrieval of next step from memory
- Environmental stimuli (e.g., parts of sailplane) help prompt retrieval of each step
- Sequential execution of steps and environmental stimuli work together

***Prospective memory***
***  ******Prototypical situations***

**Notes:**
1)  Performing a habitual task from memory, e.g. assembling the sailplane.  When we perform a task frequently enough that it becomes habitual, the steps of the task are encoded in memory quite robustly—we are not likely to forget how to perform the task, but we are still vulnerable to omitting a step when interrupted or distracted.  Here’s why.  The steps of an habitual task are encoded in memory in such a way that we don’t have to think hard about how to perform the task.  When we assemble a sailplane that we fly frequently each step of the assembly process pops into mind automatically.  The way this works is that executing each step of the assembly automatically triggers retrieval from memory of the next step:  step one triggers step two, which triggers step three, and so on.  Environmental stimuli also support this process of retrieving each step of the assembly.  What in the world do i mean by environmental stimuli?  The parts of the glider.  These two things work together—the sight of the glider parts and each step triggering memory of what to do next.  

## Opportunities for Error While Performing Habitual Tasks

**Example (1): Interruptions while assembling sailplane**
- ASW 20 horizontal stab held to vertical stab by bolt
- Interruption after finger-tightening bolt
- Breaks chain of one step triggering next step
- Removes environmental stimulus (view of bolt)
- Distinct chance will jump to next step, forgetting to cinch down the bolt
**Example (2): Landing with gear retracted**
- Several scenarios, for example:
- Develop habit of extending gear on downwind
- Visual scene provides cues, stimuli that trigger retrieval from memory
- Straight-in approach removes normal visual cues
- Interruptions, distractions, or high workload can have the same effect

***Prospective memory***

**Notes:**
Let’s say i am assembling my asw20.  The horizontal stab is held to the vertical stab by a bolt, which requires a special wrench to cinch down.  So let’s say that after i mount the horizontal and screw the bolt down with my fingers i discover that i forgot to put the wrench in my pocket, so i’m forced to interrupt the assembly and walk over to my trailer to get the wrench.  While i’m getting the wrench my neighbor asks if i can hold his wing for a moment, which i do.  But look what’s happened to my memory retrieval process:  the interruption has broken the chain by which the step of sliding the horizontal stab in place triggers me to insert the bolt, which in turn triggers me to tighten the bolt with the wrench.  Furthermore because i am no longer standing at the back of the plane looking at the horizontal the bolt no longer serves as a stimulus to help me remember what i was doing.  In this situation there is a distinct possibility that i will jump to the next step and forget to finish tightening the bolt.

Here is another example:  some of us have discovered that a great way to make a short field landing is to leave the landing gear retracted.  Why would we forget to perform a task that we have successfully performed hundreds of times before?  There are several factors—here is one scenario.  You develop the habit of lowering the landing gear as you turn downwind.  The act of turning downwind and the sight picture of the airport provides what we call a cue, a visual stimulus that is linked in procedural memory to pulling the landing gear lever and which automatically triggers retrieval of that item from memory.  But one day you get low and need to make a straight-in approach—guess what, the normal visual cue is missing and you may not remember to lower the gear.  Interruptions and distractions on the downwind leg can have the same effect.

## 2. Intending to Perform a Non-habitual Task that must be Deferred

**Example: During assembly you notice main gear tire is low**
- Decide to fill with air after pushing to launch area
- Get busy with launch preparations and forget to fill tire
**Why do we forget?**
- Brain has no special mechanisms for these situations
**How do we (sometimes) remember to perform deferred intentions?****	**
- Depends on noticing environmental cues linked in memory to the intention: cues trigger retrieval from memory
- Happenstance process
- Example: Might notice air bottle by hangar

***Prospective memory***
***  ******Prototypical situations***

**Notes:**
2) Intending to perform a non-habitual task that must be deferred.   For example, while assembling your sailplane you notice that the main gear tire is quite low.  You decide to fill it up after you push the plane down to the launch area because that is where the air bottle is.  But when you get to the launch area your buddies are already launching, you hurry to complete your cockpit preparations and you forget that you intended to fill the tire.  Why do we forget these intentions so often.  From our research we have concluded that the brain has no special mechanism for remembering to perform deferred intentions.  Well how do we ever remember to perform deferred intentions?  The most widely accepted theory is that we depend on noticing cues in our environment that have some link in our memory to the intention stored in memory and noticing such cues sometimes triggers retrieval of the memory.  Notice i said ‘sometimes’—it seems to be a rather happenstance process.  For example, while i am pushing the sailplane out i may notice the air bottle by the hanger and that visual cue may remind me of my intention.  But i may not notice the air bottle and may not think about it even if i do.

## 3. Interleaving Several Tasks Concurrently

**Example: Entering data or commands in flight computer**
- Going head down interrupts outside visual scan
- Similar to using cell phone while driving
- Mentally engaging tasks fully occupy consciousness (focal attention) 	  	momentarily forgetting outside scan
- Eyes stay down longer than intended

***Prospective memory***
***  ******Prototypical situations***

**Notes:**
3)  Interleaving several tasks concurrently.   For example, when i go head down to punch info into my flight computer, obviously i cannot simultaneously look out the cockpit for conflicting traffic.  This situation is closely similar to using a cell phone while driving your car.  We like to tell ourselves that we are careful and only look down for a second, but driving accident statistics and research data both show that people often lose track of where their car is going while talking on the cell phone.  The problem is that when we engage in a mentally demanding task, such as making entries in our flight computers or talking on our cell phones, our that task fully occupies our minds for a moment and in that moment we cannot think about looking outside and for a moment longer, or several moments, we forget the outside task.  You may think you only look down for a second but i’m willing to bet that if we hooked an eye-tracking system up to you we would see that your eyes are sometimes down much longer than your realize.  I wonder how long it will be before we have a mid-air caused by a glider fooling with his computer in flight?

## Ways to Reduce Vulnerability to Prospective Memory Errors

- 1)**  Try to avoid interrupting critical tasks**
- If interrupted, minimize time eyes are away from tasks
- 2)**  If interrupted, create salient cue to remind yourself**
- 	 Example: When interrupted during assembly, put hands in pockets
- 3)**  Use checklist for “killer” items**
- Know difference between “do” list and checklist
- Checklist for killer items: short as possible
- Checklists provide redundancy
- 4)**  When deferring tasks:**
- Create salient cue
- Identify explicitly where and when you intend to perform deferred tasks
- Example: put tape on canopy to remind about low tire
- 5)**  When going head down:**
- Develop/maintain habit of performing _only_ one step at a time
- Scan horizon between each step

**Notes:**
Can we prevent all prospective memory errors?  No.  Can we reduce our vulnerability to these errors?  Yes.  Here are some suggestions:  
1) When performing a critical task, such as assembling a sailplane, try to not let anything interrupt you.  For example, if someone comes up to talk with you, tell them you will talk with them when you are through.  Try to avoid turning your eyes from the task at hand.  But even telling them not to interrupt is itself a brief interruption.  
2)  If interrupted when performing a critical task, create some salient cue that will forcefully remind you.  For example, if interrupted during assembly, put one hand in your pocket and do not remove your hand until you have resumed the task and have positively verified where you left off.  Or, alternately, start the task over from the beginning.
3)  Use checklists for killer items.  Know the difference between a checklist and a do list.  A checklist, as opposed to a do list, is used after you have completed a procedure to check that you have completed the killer items on the procedure.  Unlike a do list that lists all items of a procedure, a checklist should be as short as possible, focusing your attention on those items that will get you in deep trouble if omitted.  Checklists provide redundancy, an important safeguard that I will discuss later.
4)  When deferring an important, but not life-threatening task, also try to create a cue that you will likely notice when the time comes to perform the task  For example i might put a piece of tape on the front of the canopy to remind me of the low tire.  Also, when you form the intention initially, identify specifically when and where you want to perform the deferred task and visualize yourself performing it.  Also you might ask someone else to remind you, which should double your chances of success.
5)  When you go head down in flight to make computer entries, or do anything else (like filling your pee bag), try to develop and maintain a habit of performing only one step at a time and then looking back up and sweeping your gaze across the horizon.  For computer programming, what i mean by ‘one step’ is a single keypad entry.  For filling your pee bag, i will let you work it out for yourself.

I will have some more suggestions for error countermeasures at the end of my talk. Obviously decision making and prospective memory are not the only areas in which we make mistakes when flying, but they are two of the most dangerous areas, and i hope i have illustrated why we must examine cognitive processes to come up with effective ways to reduce our vulnerability to error.

## Error and Threat Management (ETM)

- Latest development in Crew Resource Management (CRM)
- CRM development started in late ‘70s (airlines, USAF, & NASA)
- Accidents caused by poor communications, failing to grasp all aspects of situation, failing to manage workload effectively, and failing to develop appropriate plans
- CRM originally focussed on preventing errors_x0008_
- Workload management, communications, situation awareness, decision-making, leadership/followership, and automation management
- ETM emphasizes detecting and managing errors and threats
- Instead of criticizing crews for making mistakes, instructors should reward crews for catching/managing errors
- ETM in early stages of development

**Notes:**
I am going to spend the last part of my talk discussing something the airlines have been developing recently:  Error and threat management.  It is a concept that may be useful in soaring operations.  ETM is the latest development in the field of crew resource management—CRM.  I know some of you are quite familiar with CRM but some of you may never have heard of it.  I do not have time to really get into this broad area but will say a few words.  Back in the late 70s the airlines and the USAF realized that they were having accidents with fully functional aircraft flown by highly competent crews.  They and nasa got together and realized that the problems in these accidents stemmed from poor crew communication, failing to grasp all aspects of the situation,  failing to manage workload effectively, and failing to develop appropriate plans of action.  CRM was developed as a countermeasure.  CRM is a set of high level concepts that help individuals deal with workload, communication, situation awareness, decision making, leadership and followership, and more recently automation management.  

As originally conceived, CRM was about preventing errors and it has certainly helped reduce error rates and accident rates.  But more recently the airlines have come to realize that no matter how good pilots are they will make mistakes.  So the emphasis in this latest generation of CRM—ETM—is on recognizing errors and threats and finding ways to manage them so they don’t get out of hand.  Traditionally instructors have criticized pilots, even highly experienced pilots, for making errors.  Under ETM. instructors are supposed to reward crews for catching their own mistakes and finding ways to manage the consequences of those mistakes.  

The concept of ETM is still being developed within the airline industry. 

## Applying ETM to Soaring

**Robert Sumwalt - USAirways captain and sailplane pilot**
- Extensive experience in aviation safety
- Collaborates with airlines/ALPA/NASA
**Remainder of talk combines my ideas with Robert’s ideas on adopting ETM to soaring operations**

**Notes:**
.  Captain Robert Sumwalt is a USAirways captain and a convert to soaring.  He has done a lot of work in airline safety and recently has been working up ideas on applying ETM to soaring.  In the remainder of this talk i am going to meld some of Robert’s ideas with my own ideas about ETM—they say the sincerest form of flattery is to steal someone’s ideas.  

## Principles of ETM

**Recognize vulnerability to errors**
- Especially decision-making and prospective memory
** **
** **
** **
** **
** **
** **

**Notes:**
I’ve been talking about the importance of recognizing our vulnerability to errors and recognizing situations in which errors are most likely to occur. 

## Principles of ETM

**Recognize vulnerability to errors**
** ****Identify threats**
**Three domains of threat:**
- (1)  Those present of every flight (e.g., rope break)
- (2)  Threats present only on a particular flight (e.g., my Indian 	     Valley escapade)
- (3)  Threats specific to certain situations (e.g., Carl Herald’s topic    today)
**Before takeoff and before each phase of flight:**
- Identify threats, ask “what if...”, and develop plan
**Presence of multiple threats is especially hazardous**
- Vulnerability to error goes way up
- Treat as red warning flag

**Notes:**
Now add to that the concept of recognizing threats before they eat you.  What do i mean by threat?  Anything aspect of the situation that might endanger the flight.  Obviously our inherent vulnerability to error is a large category of threat.  What are other threats?  I think of threats as falling into two domains—those that are present in every flight and those that are unique to a particular flight.  Premature termination of tow is a threat present on every glider flight.  In my Indian Valley escapade, threats specific to  that aero retrieve were loose dirt, weeds, narrow runway, gullies, and my owned flawed analysis of the situation.  

I suggest that we would all benefit from periodically reviewing the threats present on every flight and also the threats present in specific situations—as Carl Herold is doing today.  It would also be very useful before you take off and before each new phase of flight to mentally identify any unusual threats that may be involved in what you are about to do.  By identifying the unusual threats you can ask ‘what if’ and come up a plan of action.  

## Principles of ETM

**Recognize vulnerability to errors**
** ****Identify threats**
**Treat interruptions, distractions, and deferred tasks as red warning flags**
**Cannot identify threat in advance**

**Notes:**
Some threats we cannot identify in advance.  Interruptions, distractions, and needing to defer tasks temporarily are all unpredictable.  So whenever you find yourself being interrupted or distracted, treat it as a big red warning flag.  Another situation that should raise a red warning flag occurs whenever you find yourself dealing with multiple threats that may interact—as I did at Indian Valley.  Error rates are higher when pilots must deal with multiple threats—perhaps in part because of the additional workload. 

## Principles of ETM

**Recognize vulnerability to errors**
** ****Identify threats**
**Treat interruptions, distractions, and deferred tasks as red warning flags**
**Redundancy: develop multiple layers of defense**
**Defenses (countermeasures) are essential**
**No defense is perfect (e.g., checklists)**
**Thus need defense in depth**
**Threats and error much less likely to penetrate multiple layers**
**Be sure layers are independent of each other**

**Notes:**
The concept of redundancy is central to ETM.  Because threats are always present and errors are inevitable, we must have defenses—countermeasures.  But no defense is perfect—all defenses have holes and sometimes fail.  For example, checklists are good defenses against prospective memory errors, but sometimes we forget to run the checklist and sometimes we do check an item and fail to recognize that it has not been performed correctly.  Thus this concept is about defense in depth.  A threat or error is much less likely to get through several independent defenses than through just one defense.    But be careful—those defenses must be truly independent of each other, otherwise a single error could remove them all, like dominos falling.  

## Layers of Defense to prevent assembly errors

***Adapted from ***
***Robert Sumwalt***

**Self-connecting**
**controls**

**Pilot assembles glider**

**Positive**
**control check**

**Independent critical**
**assembly check**

**Wing runner**
**check**

**Notes:**
This slide illustrates how we can use layers of defense to drastically reduce the danger of getting airborne with a control disconnected.  After assembling the glider, we can use a checklist that guides us to check that the killer items of assembly have been performed.  The positive control check is the next layer of defense.  And recently the SSA has suggested that we initial our wingtape when we run the PCC and have wing runners check that the tape has been initialed.  

## Principles of ETM

**Recognize vulnerability to errors**
** ****Identify threats**
**Treat interruptions, distractions, and deferred tasks as red warning flags**
**Redundancy: develop multiple layers of defense**
**Communicate your perceptions of threats and risk to fellow pilots**
**Soaring mostly single pilot, but always a team operation**
- Other pilots may not have noticed the threat
- Help each other remember critical items to perform

**Notes:**
In soaring we mostly fly single-pilot, but not always.  And we always interact with fellow pilots when assembling our gliders, getting ready to launch, and flying the two.  So in that sense, soaring is always a multi-crew operation.  This next principle says to communicate your perceptions of threats to your fellow pilots and crew members.  They may not have noticed the same threats you have and they not have recognized the risks you noted.  And if you see  pilots who appear to have forgotten a critical safety item do not assume that they remember it by themselves—maybe they will and maybe they won’t.  

## Principles of ETM

**Recognize vulnerability to errors**
** ****Identify threats**
**Treat interruptions, distractions, and deferred tasks as red warning flags**
**Redundancy: develop multiple layers of defense**
**Communicate your perceptions of threats and risk to fellow pilots**
**Standardize critical procedures**
**Airlines and military rely on SOP to reduce errors and accidents**
**Soaring ops are different situation, **_**but**_** ...**
**Each pilot should:**
**Work out explicit procedures for critical tasks**
**Develop strong habits to standardize execution of tasks**
**Standardizing your procedures reduces vulnerability to forgetting items or performing incorrectly**
**Use checklists **

**Notes:**
The airlines and military place heavy reliance on using standard operating procedures for every aspect of flight operations.  They do this for a reason—standardizing procedures has been shown to substantially reduce errors and accidents.  Standardizing procedures basically means finding the best way to perform a procedure and then following that procedure every time.  Now, soaring operations are very different from airline and military ops—in our domain the issue is not getting all glider pilots to do everything the same way but it does make sense for each glider pilot to work out explicit procedures for certain critical tasks and to develop strong habits to standardize how he or she performs those tasks.  Assembling the glider, preflight inspection of the glider, pre-launch checks, and pre-landing checks are obvious tasks for which you can develop a standardized procedure.  Standardizing your procedure reduces your vulnerability to forgetting to perform critical items or performing them incorrectly.  And using a checklist after completing the procedure further reduces your vulnerability to error.

## Integrated Suggestions for Reducing Vulnerability to Error and Managing Threats

-  1)**   Recognize areas of vulnerability to errors**
-  2)**   Identify and voice your assumptions**
-  3)**   Identify threats**
-  4)**   Ask “What if ...”**
-  5)**   Communicate your perceptions of threats to fellow pilots**
-  6)**   Always have a back door**
-  7)**   Treat interruptions, distractions, and deferred tasks as red   flags:**
**Minimize interruptions during critical tasks**
**Create salient cues as reminders**
-  8)**   Break head-down tasks into small steps and interleave with  scanning the horizon**
-  9)**   Standardize critical procedures**
- 10)**  Use checklists**

**Notes:**
I realize that i now have given you three different sets of suggestions for dealing with the challenges of decision-making, for dealing with vulnerability to PM errors, and for ETM.  In this last slide i have integrated these into a single set of suggestions.  I caution you that these are not the be all and end all.  These represent issues that are active areas of research.  These suggestions will certainly not prevent you from making errors, but they may help you  reduce the number of errors you make and they should help you detect errors and threats and they should help you reduce the possibility that errors and threats will turn into an accident.