# Presentation: 209526

## A Human Factors Review of General Aviation Accidents in Alaska Versus the Rest of the U.S. 

**A Human Factors Review of General Aviation Accidents in Alaska Versus the Rest of the U.S. **

**Notes:**
Scott Shappell, Ph.D.
FAA/Civil Aerospace Medical Institute
AAM-510
Oklahoma City, OK 73125
(405)954-4082

Cristy Detwiler, B.A.
FAA/Civil Aerospace Medical Institute
AAM-510
Oklahoma City, OK  73125
(405)954-2438

Douglas Wiegmann, Ph.D.
University of Illinois at Urbana Champaign
Institute of Aviation
Savoy, Illinois  61874
(217)244-8637

## Introduction

**Introduction**

**Aviation in Alaska poses a unique challenge to those that frequent the Alaskan skies.**
**Geographically, Alaska spans some 591,000 square miles.**
**Mountains, glaciers, lakes and long coastlines are just some of the features that accentuate the terrain.**
**Due to location, size, and other factors, climactic changes often occur.**
**Because of the geography of Alaska, many people have resorted to flying as a means of transportation.**
**With the combination of terrain and climate, aviators in Alaska are faced with highly demanding situations.**

**Notes:**
Aviation in Alaska poses a unique challenge to those that frequent the Alaskan skies, particularly when compared with the rest of the United States. Geographically, the Alaska land mass spans some 591,000 square miles. Mountains, glaciers, lakes, and long coastlines are just a few of the features that accentuate this terrain, offering a veritable obstacle course when traversing from point A to point B in this great state. It is because of this geography that many people have resorted to flying as a means of transportation. There is not enough stable land mass to travel by car and it is essentially less efficient to travel any other way. In addition, climactic changes often occur due to the location, size, and other factors that affect the land mass. 
With the combination of terrain and climate, aviators in Alaska are faced with highly demanding situations. It is imperative that the aviation community, which chooses to fly in Alaska, be “on top of their game” when behind the yoke. Not only is it necessary that these aviators be highly familiar with the path of flight, but they must be ready for any situation that arises. Aviators must know, for example, that the pass they are flying through has mountain peaks that are frequently obscured by clouds and be prepared to successfully navigate through that. Tragically, this is not always the case.

## Introduction (continued)

**To date, the majority of research has focused on the demographics of aviation accidents (e.g. – age, gender, experience, medical condition, etc.).**
**Very few studies have examined the human error component ... at least that is, until recently. **
**The Human Factors Analysis and Classification System (HFACS), has provided an organized way of investigating human error associated with accidents and incidents so that they can be either prevented or their consequences mitigated.**

**Notes:**
To date, the majority of research has focused on the demographics of aviation crashes (e.g. – age, gender, experience, medical condition, etc...), but until recently, few have focused on the human human component. The Human Factors Analysis and Classification System (HFACS), developed by Scott Shappell, Ph.D. (AAM-510) and Douglas Wiegmann, Ph.D. (University of Illinois) while they were in the U.S. Navy, has provided an organized way of investigating human error associated with accidents and incidents so they can be prevented or their consequences mitigated.
HFACS is currently used to investigate accidents/incidents in several aviation and industrial settings throughout the world. In the U.S., the U.S. Navy/Marine Corps, U.S. Army, and U.S. Air Force all use HFACS as either a primary investigative tool or as a means to analyze existing accident/incident data. The FAA (CAMI), in cooperation with NASA and the University of Illinois, has been employing HFACS with both commercial and general aviation accidents for the last 5 years.
This presentation is a small part of a much larger effort that has revealed the “face of human error” within civilian aviation. For other presentations and publications please see:
Wiegmann, D. and Shappell, S. (2003). A human error approach to aviation accident analysis: The human factors analysis and classification system. Ashgate Publishing Company: Aldershot, Great Britain.
Shappell, S. and Wiegmann, D. (2003). A human error analysis of general aviation controlled flight into terrain (CFIT) accidents occurring between 1990-1998.  Office of Aerospace Medicine Technical Report No. DOT/FAA/AM-03/4. Civil Aerospace Medical Institute, Oklahoma City, OK 73125.
Shappell, S. and Wiegmann, D. (2003). Reshaping the way we look at general aviation accidents using the human factors analysis and classification system. Proceedings of the Twelfth International Symposium on Aviation Psychology, Dayton, Ohio.
Wiegmann, D. and Shappell, S. (2001). Human error analysis of commercial aviation accidents: Application of the Human Factors Analysis and Classification System (HFACS). Aviation, Space and Environmental Medicine, 72, 1006-1016.
Pape, A., Wiegmann, D., and Shappell, S. (2001). Air traffic control and aviation accidents: A human factors analysis. Proceedings of the Eleventh International Symposium on Aviation Psychology, Ohio State University.
Wiegmann, D. and Shappell, S. (2001). A human error analysis of commercial aviation accidents using the Human Factors Analysis and Classification System (HFACS). Office of Aviation Medicine Technical Report No. DOT/FAA/AM-01/3. Civil Aeromedical Institute, Oklahoma City, OK 73125.

## “Swiss Cheese” Model of Human Error

**Latent Conditions**

**Latent Conditions**

**Failed or**
**Absent Defenses**

**Unsafe**
**Supervision**

**Preconditions**
**for**
** ****Unsafe Acts**

**Unsafe**
**Acts**

**Adapted from Reason (1990)**

**Accident & Injury**

**Active Conditions**

**“****Swiss Cheese” Model of Human Error**

**Notes:**
HFACS incorporates Reason’s (1990) “Swiss cheese” model of how humans contribute to the breakdown of safe flight operations. In this model, system failures are classified as either active or latent conditions. However, the exact nature of these failures or “holes” in the cheese have yet to identified and described. In this section, we provide a framework or taxonomy for identifying, classifying, and organizing active and latent failures with in the system. The framework describes four levels of failure within the system which include: (a) organizational factors, (b) unsafe supervisory practices, (c) unsafe conditions of operators, and (d) the unsafe acts operators commit. Each level is described in detail, beginning with the level most closely tied to the accident itself, unsafe acts.

## Slide 5

**Notes:**
HFACS
This figure provides an overview of the HFACS framework. Briefly, HFACS describes four levels of failure within the system which include: 
 Organizational Influences - resource management, organizational climate, and operational process
 Unsafe Supervision - inadequate supervision, planned inappropriate operations, failed to correct known problem, and supervisory violations
 Preconditions for Unsafe Acts - adverse mental states, adverse physiological states, physical/mental limitations, physical environment, technological environment, crew resource management, and personal readiness
 Unsafe Acts of Operators - decision errors, skill-based errors, perceptual errors, and violations (routine & exceptional)
So, using the HFACS framework, we set out to examine the role human error plays in general aviation accidents that have occurred in Alaska as compared to the rest of the United States.

A more complete description of HFACS can be found in:
Shappell, S. and Wiegmann, D. (2001). Beyond Reason: Defining the holes in the Swiss Cheese.  Human Factors in Aviation Safety, 1(1), 59-86.
Shappell, S. and Wiegmann, D. (2000). The Human Factors Analysis and Classification System – HFACS.  Office of Aviation Medicine Technical Report No. DOT/FAA/AM-00/7.  Civil Aerospace Medical Institute, Oklahoma City, OK 73125, 2000.

## Methods

**Analyzed **_**all**_** FAR Part 91 - GA accidents occurring between 1990 and 2000 and identified those associated with aircrew error (n= 15,526).**
**The 15,526 accidents were associated with nearly 30,000 human causal factors, as reported by the National Transportation Safety Board (NTSB).**
**The NTSB human causal factors were classified into HFACS causal categories independently by seven GA pilots.**
** ****All were certified flight instructors**
** ****Mean flight hours = 3,530**

**Notes:**
Data
GA accident data from calendar years 1990-2000 was obtained from databases maintained by the NTSB and the FAA’s National Aviation Safety Data Analysis Center (NASDAC). In total, nearly 21,000 GA accidents were extracted for analysis (this analysis only included those accidents where the investigation was considered final). These so-called “GA” accidents actually included a variety of aircraft being flown under several different operating rules: 1) 14 CFR Part 91 – Civil aircraft other than moored balloons, kites, unmanned rockets, and unmanned free balloons; 2) 14 CFR Part 91F – Large and turbine-powered multiengine airplanes; 3) 14 CFR Part 103 – Ultralight vehicles; 4) 14 CFR Part 125 – Airplanes with seating capacity of 20 or more passengers or a maximum payload capacity of 6,000 pounds or more; 5) 14 CFR Part 133 – Rotorcraft external-load operations; 6) 14 CFR Part 137 – Agricultural aircraft operations. In addition, the database contained several accidents involving public use aircraft (i.e., law enforcement, state owned aircraft, etc.) and a few midair accidents involving military aircraft.
Although all of the accidents obtained can be found within the NTSB under the heading of “general aviation,” we were only interested in those accidents involving aircraft operating under 14 CFR Part 91. After all, it is difficult to envision that large commercial aircraft being ferried from one airport to the next (operating under 14 CFR Part 91F) or aircraft being used to spread chemicals on a field (operating under 14 CFR Part 137) can be equated with small private aircraft being flown for personal or recreational purposes (operating under 14 CFR Part 91). This left us with roughly 19,000 accidents in the database.
For this analysis we were primarily concerned with powered aircraft and therefore conducted another reduction of the data to include only accidents involving powered fixed-wing aircraft (i.e., no gliders, ultra-lights, balloons, or blimps), helicopters, and gyrocopters. The remaining accidents were then examined for aircrew-related causal factors. Since we were only interested in those involving aircrew error, not those accidents that were purely mechanical in nature or those solely attributable to other human involvement, a final reduction of the data was conducted. Note, this does not mean that mechanical failures or other sources of human error did not exist in the final database, only that some form of aircrew error was also involved in each of the accidents included. In the end, 15,526 accidents were included in the database and submitted to further analyses using the HFACS framework.
Causal Factor Classification using HFACS
Five GA pilots were originally recruited from the Oklahoma City area as subject matter experts and received roughly 16 hours of training on the HFACS framework. All five were certified flight instructors with a minimum of 1,000 flight hours in GA aircraft (mean = 3,530 flight hours) as of June 1999 when the study began. Note that since the study began, two more pilot subject matter experts have been added due to attrition from the original five GA pilots. They were also certified flight instructors with comparable flight hours.

## In general aviation, only the bottom two tiers of HFACS will likely be populated.

**Today however, we will only focus on the bottom tier.**

**Notes:**
In using HFACS to examine the general aviation accident data, the majority of the accidents are coded as either a precondition for unsafe acts or an unsafe act. This is due primarily to the fact that there is not much of an organizational structure or supervisory influence on those aviators who are leisurely flying, as compared to their counterparts who are on agriculture assignments or those who are running charter flights. 
With few exceptions (e.g., flight instructors and flight training institutions), the top two tiers of HFACS (Unsafe Supervision and Organizational Influences) remained sparsely populated when examining the General Aviation accidents leaving the majority of causal factors within the bottom two tiers of HFACS. However, in the interest of time and for the purposes of this initial presentation, we have restricted our analyses to the unsafe acts of aircrew - the bottom tier of the framework. 

## FAR Part 91 - General Aviation:Overall

**Notes:**
Previous research using HFACS performed at both the University of Illinois and the Civil Aerospace Medical Institute (CAMI) has shown that HFACS can be reliably used to analyze the underlying human factors causes of both commercial and GA accidents (Wiegmann & Shappell, 2001, 2003; Shappell & Wiegmann, 2003). Furthermore, these analyses have helped identify general trends in the types of human error that have contributed to civil aviation accidents. 
When the GA accidents between 1990-2000 were examined using the HFACS framework; several heretofore unknown facts regarding GA aviation safety were revealed. For instance, it appears that safety efforts over the last several years have had little impact (flat trend lines) on any specific type of human error associated with GA accidents. If anything, they have had a ubiquitous impact – albeit unlikely. Equally noteworthy, skill-based errors have contributed to GA accidents more than any other error form (roughly 80% of all GA accidents examined). Given that most of these skill-based errors were technique (stick-and-rudder) errors, it would seem to indicate that there may be a problem associated with current training and/or pilot currency/proficiency.

(Note that most accidents are associated with multiple causal factors. Therefore, these percentages will not necessarily add up to 100%).

## AK vs RoUS – Human Errors

- **

- **

- **

- **

- **  Significant at the .001 level

**Notes:**
Aviation accidents in Alaska have been of particular interest to the Administrator and AFS-800. This figure depicts the percentage of accidents associated with at least one instance of each unsafe act (i.e. DE - Decision Error; SBE - Skill-based Error; PE - Perceptual Error, and V – Violations) for accidents in Alaska (red) versus the rest of the U.S. (blue). On the surface, there were small differences in the pattern of human error associated with accidents occurring in Alaska and the rest of the U.S. Specifically, accidents in Alaska appear to be associated with slightly more decision errors, but fewer skill-based errors, perceptual errors and violations than accidents occurring in the rest of the U.S. Note however, that although the differences were statistically significant (p<.001), the differences were small.
If anything, these data suggest that while pilots involved in accidents in Alaska may, on average, make slightly more bad decisions. However, their overall skill appears to be better than their counterparts in the rest of the U.S., they appear to be less prone to accidents due to spatial disorientation, visual illusions, etc, and they are less likely to violate rules regulating the safe conduct of flight.

(Note that most accidents are associated with multiple causal factors. However, the occurrence of a given unsafe act (e.g., DE, SBE, PE, or V) was only counted once per accident for this analysis. As before, these percentages will not necessarily add up to 100%).

## FAR Part 91 - General Aviation:

**Alaska vs. Rest of the U.S.**

**Notes:**
It is interesting to note that when accidents associated with each of the unsafe acts (decision errors, skill-based errors, perceptual errors, and violations) are graphed annually for Alaska (open circles) and the rest of the U.S. (closed circles), that the differences that do exist (albeit small), have remained relatively constant over the years. While this is interesting, the larger question is what differences, if any, exist in the types of these errors and violations committed in Alaska versus the rest of the U.S. Presumably, by knowing what types of unsafe acts are made, intervention/mitigation strategies can be developed to address them.

(Note that most accidents are associated with multiple causal factors. However, the occurrence of a decision error was only counted once per accident for this analysis.)

## Decision Errors

**Decision Errors**

| Error | Alaska | RoUS | Total |
| --- | --- | --- | --- |
| In-flight Planning/Decision | 58    (14.1%) | 857  (18.8%) | 915  (18.4%) |
| Unsuitable Terrain | 159  (38.8%) | 221  (4.9%) | 380  (7.7%) |
| Refueling | 13    (3.2%) | 321  (7%) | 334  (6.7%) |
| Planning/Decision | 20    (4.9%) | 302  (6.6%) | 322  (6.5%) |
| Go-around | 16    (3.9%) | 295  (6.5%) | 311  (6.3%) |
| Remedial Action | 12    (2.9%) | 280  (6.1%) | 292  (5.9%) |
| Accidents with at least one Decision Error | 411  (100%) | 4560 (100%) | 4971 (100%) |

**Notes:**
This table lists the top five decision errors committed by GA pilots involved in accidents. As can be seen from the totals to the left, poor in-flight planning/decision errors were associated with a little over 18% of the accidents we examined. These types of errors were followed by decisions to takeoff or land on unsuitable terrain, refueling decisions, planning/decision errors committed while on the ground, decisions to go-around, and remedial actions.
With regard to accidents in Alaska versus the rest of the U.S., it appears that the only real difference was that GA accidents in Alaska were 12 times more likely to be associated with takeoff and landing from/on unsuitable terrain (odds ratio = 12.423, M-H = 607.722, p <.001). No other differences were significant.

(Note that most accidents are associated with multiple causal factors. As such, more than one decision error can be associated with a given accidents. Therefore, these percentages will not necessarily add up to 100%).

## Skill-Based Errors

**Skill-Based Errors**

| Error | Alaska | RoUS | Total |
| --- | --- | --- | --- |
| Directional Control | 187  (18.8%) | 1806  (11.6%) | 1993  (12.1%) |
| Airspeed | 71    (7.2%) | 1685  (10.9%) | 1756  (10.6%) |
| Aircraft Control | 51    (5.1%) | 1227  (7.9%) | 1278  (7.7%) |
| Compensation for Wind Conditions | 145  (14.6%) | 886    (5.7%) | 1031  (6.2%) |
| Accidents with at least one Skill-Based Error | 993  (100%) | 15512 (100%) | 16505 (100%) |

**Notes:**
While continuing the deeper inspection of the data, the top four types of skill-based errors identified within the GA accident data all involve basic stick-and-rudder skills. The most common skill-based error involves maintaining directional control followed by errors involving airspeed, control of the aircraft, and compensation for winds.
With regard to differences between accidents in Alaska and those in the rest of the U.S., it appears that those flying in Alaska are almost twice as likely to experience a loss of directional control (odds ratio = 1.761, M-H = 44.755, p <.000) and almost three times as likely to inadequately compensate for wind conditions (odds ratio = 2.823, M-H = 124.437, p<.000).

(Note that most accidents are associated with multiple causal factors. As such, more than one skill-based error can be associated with a given accidents. Therefore, these percentages will not necessarily add up to 100%).

## Violations

**Violations**

| Violation | Alaska | RoUS | Total |
| --- | --- | --- | --- |
| VFR Flight into IMC | 27  (25%) | 304  (13.9%) | 331  (14.4%) |
| Procedures/Directives | 14  (13%) | 244  (11.2%) | 258  (11.2%) |
| Operating with Known Deficiencies | 6    (5.6%) | 234  (10.7%) | 240  (10.4%) |
| Flight into Adverse Weather | 10  (9.3%) | 186  (8.5%) | 196  (8.5%) |
| Fuel System | 4    (3.7%) | 165  (7.5%) | 169  (7.4%) |
| Aircraft Weight and Balance | 12  (11.1%) | 127  (5.8%) | 139  (6.1%) |
| Design Stress Limits of Aircraft  | 3    (2.8%) | 118  (5.4%) | 121  (5.3%) |
| Accidents with at least one Violation | 108 (100%) | 2187 (100%) | 2295 (100%) |

**Notes:**
Recall that violations are the “willful” disregard for the rules and regulations governing safe flight. Consequently, our pilot-raters were told only to code a violation if there was sufficient evidence to classify the cause factor as a “willful” disregard for the rules. That conservative approach meant that accident causes like “inadvertent” VFR flight into IMC” was not considered a violation.
When the data were classified in this manner, not surprising, VFR flight into IMC was the most common violation contributing to GA accidents (associated with just over 14% of the accidents we examined). This was followed by not adhering to procedures or directives, operating with known deficiencies, flight into adverse weather (not IMC), issues associated with the fuel and weight & balance, and exceeding the design limits of the aircraft. 
With regard to differences between accidents in Alaska and those occurring in the rest of the U.S., although not statistically significant in accordance with the conservative significance level that we have chosen, the decision to continue VFR flight into IMC was two times more likely to occur in Alaska than in the rest of the U.S. (odds ratio = 2.065, M-H = 9.390, p = .002). No other differences were noted. 

(Note that most accidents are associated with multiple causal factors. As such, more than one violation can be associated with a given accidents. Therefore, these percentages will not necessarily add up to 100%).

## Discussion

**No major differences between Alaska and Rest of U.S. with regard to overall pattern of human error**
**If anything, slightly more decision errors associated with Alaska accidents and fewer skill-based errors, perceptual errors, and violations**

**Notes:**
Ultimately there were not major differences between Alaska and the rest of the US with regard to the overall pattern of human error. If anything, there were slightly more decision errors associated with accidents occurring in Alaska and fewer skill-based errors, perceptual errors, and violations. 

## Discussion

**No major differences between Alaska and Rest of U.S. with regard to overall pattern of human error**
**If anything, slightly more decision errors were associated with Alaska accidents along with fewer skill-based errors, perceptual errors, and violations**
**However, when looking deeper, differences were revealed in the specific types of errors and violations committed by those flying in Alaska**
**12 times more likely to take-off or land on unsuitable terrain **
**Almost two times as likely to experience a loss of directional control and almost three times as likely to inadequately compensate for wind conditions.**
**Almost twice as likely to continue VFR flight into IMC weather**

**Notes:**
When further examination of the data was performed, the specific types of errors were shown, some of which support prior research in this area. 
However, when looking deeper, differences were revealed in the specific types of errors and violations committed by those flying in Alaska
12 times more likely to take-off or land on unsuitable terrain 
Two times as likely to experience a loss of directional control
Three times as likely to inadequately compensate for wind conditions.
Twice as likely to continue VFR flight into IMC weather

## What does this all mean?

**Although  there were significant differences in the overarching human error categories, the presence of very different types of errors and violations need to be addressed.**
**Intervention strategies developed for pilots flying in Alaska should address the following issues:**
** ****Takeoff and landing from/on unsuitable terrain**
** ****Directional control**
** ****Adverse weather conditions such as:**
**Compensation for Winds**
**VFR flight into IMC**

**Notes:**
In sum, although there were significant differences in the overarching human error categories, the presence of very different types of errors and violations need to be addressed. It would appear from our comprehensive human error analysis that intervention strategies developed for pilots flying in Alaska should address the following issues:
Takeoff and landing from/on unsuitable terrain
Directional control
Compensation for weather conditions and winds
VFR flight into IMC

## What can we do?Intervention Matrix

**What can we do?****Intervention Matrix**

**Notes:**
CAMI and the University of Illinois (Drs. Shappell and Wiegmann) have developed a intervention matrix based on the HFACS framework. As can be seen, the unsafe acts are depicted down the left hand side and the various human error perspectives as described in their book, A human error approach to aviation accident analysis: The human factors analysis and classification system. 
Recall that the major differences between accidents in Alaska and those in the rest of the U.S. involved Decision Errors (the decision to takeoff or land from/on unsuitable terrain), Skill-based Errors (Maintaining directional control, Compensation for winds), and Violations (intentional/continued VFR flight into IMC). Any interventions to address issues in Alaska should therefore focus on those areas.

## Alaska CapstoneIntervention Matrix

**Alaska Capstone****Intervention Matrix**

**CAPSTONE**

**CAPSTONE**

**CAPSTONE**

**Notes:**
In many ways, the current effort to reduce aviation accidents in Alaska (known as the Alaska Capstone Program) is focused on improving aviation safety through improved technology. Specifically, efforts have been directed at improving avionics by taking advantage of Automated Dependent Surveillance – Broadcast (ADS-B), Flight Information Services – Broadcast (FIS-B) and GPS to provide pilots information regarding traffic and graphical weather information in the cockpit. Terrain displays and mountain pass weather cameras are also providing information to pilots, critical to flying in IMC while in the air and during the mission planning phase on the ground. The question remains however, whether these interventions will impact those areas identified by the previous accident analysis.
Certainly, an argument can be made that improved weather, terrain, and traffic displays will impact accidents due to spatial disorientation, and controlled flight into terrain (see the solid blue block). Unfortunately, these interventions will do little to address major differences between accidents in Alaska and the rest of the U.S. The only exception may be VFR flight into IMC, but there is some evidence that improved weather depictions will actually encourage pilots to “shoot the gap” rather than avoid weather altogether (see recent CAMI research by Dr. Williams and Dr. Beringer in this area).
Frankly, technology based intervention strategies will do little to address issues such as taking off and landing on unsuitable terrain, or the ability of pilot to compensate for winds and control their aircraft. Perhaps what is needed is thinking “out of the technology box” and in areas that we didn’t even know were issues.

## CAMI/Univ. of IllinoisThe Year Ahead

**CAMI/Univ. of Illinois****The Year Ahead**

**Notes:**
In FY04, we will be addressing a number of issues as identified by AFS-800 including:
What are the exact types of errors committed within each error category?
How do the different error types relate to one another, or with other HFACS variables?
What can be done to intervene given the information that is now available, and what more might be done with the additional refined data (developing the Intervention Matrix)?
What are the human factors associated with air tour/sightseeing accidents and how do they compare with other GA and commercial aviation accidents?
What are the specific types of human error associated with EMS helicopter accidents and how do they compare with other fixed- and rotary wing operations.
Transition HFACS database to field (with ASY-1 and NASDAC)

Are there other areas we should be focusing our efforts at????

## Scott Shappell, Ph.D.

**Scott Shappell, Ph.D.**
- FAA/Civil Aerospace Medical Institute
- AAM-510
- Oklahoma City, OK 73125
- (405)954-4082
- Scott.Shappell@faa.gov
**Douglas Wiegmann, Ph.D.**
- University of Illinois at Urbana Champaign
- Institute of Aviation
- Savoy, Illinois  61874
- (217)244-8637
- dwiegman@uiuc.edu