# Presentation: 095263

## HFACS ANALYSIS OF GENERAL AVIATION ACCIDENT DATA: IMPLICATIONS FOR AERONAUTICAL DECISION-MAKING

**Douglas Wiegmann, Ph.D.**
**University of Illinois**

**HFACS ANALYSIS OF GENERAL AVIATION ACCIDENT DATA: IMPLICATIONS FOR AERONAUTICAL DECISION-MAKING**

**Scott A. Shappell, Ph.D.**
**Civil Aeromedical Institute**

**FAA-99-G-006**

**Notes:**
ASSESSING THE RELIABILITY OF THE HUMAN FACTORS ANALYSIS AND CLASSIFICAITON SYSTEM (HFACS) WITHIN THE CONTEXT OF GENERAL AVIATION
D. A. WIEGMANN1 AND S. A. SHAPPELL2
1University of Illinois at Urbana-Champaign, Savoy, IL, 2Civil Aeromedical Institute, Oklahoma City, OK
Purpose. The Human Factors Analysis and Classification System (HFACS) is a general human error framework for analyzing aviation accidents. Previous validation research has shown that HFACS produces reliable results when applied to both military and commercial aviation accident data. To date, however, the reliability of the system within the context of general aviation (GA) has yet to be examined. The purpose of the present study, therefore, was to assess the reliability of HFACS when applied to the analysis of GA accidents in an effort to further evaluate empirically the utility of the framework.  Method.  A comprehensive review of fatal GA airplane accidents  (FAR Part 91) that occurred between 1995 and 1998 was conducted using databases maintained by the NTSB and FAA. All pilot-related causal factors associated with these accidents were independently coded by two certified pilots using the HFACS framework. Results.  The reliability of the HFACS system, as indicated by the level of agreement between investigators, was assessed using Cohen’s Kappa. The Kappa indices obtained fell within the “Good” to “Excellent” range using conventional standards. Discussion.  These findings corroborate previous research demonstrating the reliability of HFACS and underscore its utility as a general human error analysis tool across all categories of aviation operations. 

## Slide 2

## Slide 3

**Notes:**
With few exceptions (e.g., flight instructors and flight training institutions), the top two tiers of HFACS (Unsafe Supervision and Organizational Influences) will not be populated when examining general aviation accidents.  Consequently, only the bottom two tiers of HFACS were examined here.  A more complete description of the data will be available at a later date.

## Method

- A total of 2,391 accidents was identified for further analysis
- Only those accidents in which the accident investigation was completed and causal factors determined were included in this analysis.

**Method**

**Notes:**
FAR Part 91 General Aviation Accidents
The purpose of the final study was to explore the reliability of HFACS in classifying general aviation accidents. Albeit, the upper supervisory and organizational factors may be non-existent in many GA accidents, however, the lower tiers of the framework – preconditions and unsafe acts, are likely to be the same.  A comprehensive review of fatal FAR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA.  A total of 2,696 accidents were identified for further analysis.  The 6,098 causal factors associated with these accidents were independently coded by five general aviation pilots, all certified flight instructors with flight hours ranging from 1,250 to 11,000.

## Method

- This HFACS coding focused solely on the causal factors identified by the NTSB during the original accident investigation.
- No new cause factors were created during this error-coding process.

**Method**

**Notes:**
FAR Part 91 General Aviation Accidents
The purpose of the final study was to explore the reliability of HFACS in classifying general aviation accidents. Albeit, the upper supervisory and organizational factors may be non-existent in many GA accidents, however, the lower tiers of the framework – preconditions and unsafe acts, are likely to be the same.  A comprehensive review of fatal FAR Part 91 accidents between January 1990 and December 1998 was conducted using database records maintained by the NTSB and the FAA.  A total of 2,696 accidents were identified for further analysis.  The 6,098 causal factors associated with these accidents were independently coded by five general aviation pilots, all certified flight instructors with flight hours ranging from 1,250 to 11,000.

## Method

-  Five general aviation pilots/certified flight instructors
-  Flight hours ranged from 1,250 to 11,000
- Training:
- Four-hour workshop on HFACS
- Practice coding 20 accidents as a group
- Practice coding 50 accidents independently, followed by a review/consensus meeting

**Method**

**Notes:**
Notes same as slide.

## Sample of the Types of Human Error Typically Found

**Sample of the Types of Human Error Typically Found**

## Procedure

- Independently coded NTSB cause factors (no new ones created)
- Randomly paired with a second pilot who coded the same set of accidents.
- Pilots met to compare codes and achieve consensus
- They were then assigned another 1/3 of the accidents for a particular year and randomly paired with another pilot.
- This process continued until all the accidents had been coded.

**Procedure**

**Notes:**
Notes same as slide.

## Results

- On average, pilot agreed 79% of the time on how the causal factors should be coded using HFACS.
- These percentages varied only slight across the years of data analyzed in this study (range was 77% to 83% agreement).
- When overall agreement was corrected for chance using Cohen’s Kappa, the resulting index was .722, which is considered “good” by conventional standards.

**Results**

**Notes:**
Notes same as slide.

## Percentage of Accidents

**Notes:**
The percentage of accidents associated with each form of unsafe act (skill-based errors, decision errors, perceptual errors, and violations) are presented for each year of the study (1990-1998).  Note that because each accident can have multiple causal factors associated with it, the percentages of accidents associated with each unsafe act will not equal 100%.  Furthermore, because the percentage of one error form goes down does not necessarily mean that another must go up.  In effect, they are independent of each other.
An inspection of the figure illustrates several heretofore, unknown facts regarding fatal GA accidents.  First, there has been little impact of efforts to date on specific types of human error associated with fatal GA accidents (i.e., the lines are essentially flat).  This is in direct contrast to what has been reported previously in military and commercial aviation using HFACS.  Second, skill-based errors have been associated with 4 out of every 5 accidents (80%) since 1990.  These skill-based errors are primarily technique (stick-and-rudder) type errors indicating failures associated with training and currency/proficiency.  Third, nearly 40% of all fatal GA accidents are associated with violations of the rules, and are typically the result of “continuing” flight into instrument meteorological conditions when authorized visual flight rules only.  It is important to point out that these violations are “willful” departures from the rules and not simply inadvertent flight into the weather (classified as a decision error).  Like violations, decision errors were also associated with nearly 40% of all fatal accidents, but perceptual errors (often due to visual illusions and spatial disorientation) were associated with less than 15% of all fatal accidents.  It should be pointed out that many of our current intervention strategies and research efforts have been aimed at these last two error forms.

## Results:

**Category of Cause-factors (%)**

- Unsafe Acts (73.5%)

- Other (26.5%)

**Results:**

**Notes:**
Conclusion
Within the context of military, commercial, and general aviation, HFACS appears to be reliable.  However, there is room for improvement. These improvements to the HFACS coding process could involve refinement of terms and methods for coding factors.  Also of concern is the lack of detailed accident data contained in civilian databases, unlike that of military accident reports. Such conditions will limit the reliability of any system.
The data reported here also highlight the importance of reassessing the reliability and other validation criteria of error frameworks when the type or category of data (military vs. commercial operations vs. GA) is changed.
Finally, additional efforts are being made to validate the HFACS framework outside the flightdeck, including areas of aviation maintenance and air traffic control.

## Results:

**Types of Unsafe Acts (%)**

- Type of Unsafe Act

- % of Cause Factors

- N = 2597

- N = 912

- N = 714

- N = 109

**Results:**

**Notes:**
Conclusion
Within the context of military, commercial, and general aviation, HFACS appears to be reliable.  However, there is room for improvement. These improvements to the HFACS coding process could involve refinement of terms and methods for coding factors.  Also of concern is the lack of detailed accident data contained in civilian databases, unlike that of military accident reports. Such conditions will limit the reliability of any system.
The data reported here also highlight the importance of reassessing the reliability and other validation criteria of error frameworks when the type or category of data (military vs. commercial operations vs. GA) is changed.
Finally, additional efforts are being made to validate the HFACS framework outside the flightdeck, including areas of aviation maintenance and air traffic control.

## Percentage of Cause Factors

- Year

- Skill-based Errors

- Perceptual Errors

- Decision Errors

- Violations

**Notes:**
Notes same as slide.

## Variety of Decision Codes (N = 185)

- altitude inadequate
- judgment poor
- inflight planning decision poor
- inflight planning decision inadequate
- planning decision improper
- refueling not performed
- aborted take off not performed
- low altitude flight maneuver performed
- remedial action delayed
- aborted landing delayed
- wrong runway selected
- all available runway not used
- weather evaluation inadequate
- go around delayed
- preflight briefing service disregarded
- altitude improper
- aerobatics performed
- unsuitable terrain for take off landing selected
- preflight briefing service not used
- hazardous weather advisory disregarded
- procedure directives not followed
- vfr into imc inadvertent
- lift off premature
- aborted take off delayed

- preflight briefing service not obtained
- weather forecast disregarded
- go around not performed
- altitude low
- inflight planning decision delayed
- anti ice de ice system not used
- proper altitude not selected
- flight to alternate destination not performed
- weather evaluation misjudged
- weather evaluation improper
- weather evaluation poor
- missed approach not performed
- emergency procedure simulated
- vfr into imc attempted
- pull up excessive
- pull up delayed
- planned approach improper
- planned approach poor
- taxi speed excessive
- ostentatious display
- fuel supply inadequate
- carburetor heat improper use of
- flight into adverse weather continued
- flight into adverse weather inadvertent
- low altitude flight maneuver intentional

- flight into adverse weather initiated
- emergency procedure delayed
- flight advisory disregarded
- weather forecast not obtained
- maneuver excessive
- remedial action inadequate
- stall spin initiated
- missed approach delayed
- gear retraction not performed
- low pass performed
- planning decision poor
- carburetor heat not used
- flight to alternate destination delayed
- updating of recorded weather info not obtained
- pull up performed
- go around attempted
- emergency procedure not performed
- altitude clearance inadequate
- improper decision
- stall spin performed
- maneuver performed
- remedial action not performed
- refueling improper
- ac handling improper
- proper altitude not maintained

**Variety of Decision Codes (N = 185)**

## Top 10 Decision Errors

| Decision Error | Frequency |
| --- | --- |
| Poor In-flight Planning | 184 (25.8%) |
| Wrong Altitude Selection | 78 (10.9%) |
| Over Commitment to Plan | 50 (7.0%) |
| Poor Judgment | 50 (7.0%) |
| Poor Flight Planning | 36 (5.0%) |
| Inaccurate Weather Evaluation | 22 (3.1%) |
| Refueling Not Performed | 21 (2.9%) |
| Inadequate Preflight Briefing | 20 (2.8%) |
| Improper Remedial Action | 16 (2.2%) |
| Wrong Maneuver | 15 (2.1%) |

## Variety of Violation Codes (N = 115)

- ac control not maintained
- ac unattended engine running intentional
- ac weight balance continued
- ac weight balance disregarded
- ac weight balance exceeded
- ac weight balance excessive
- ac weight balance improper
- act clearance not complied
- aerobatics attempted
- aerobatics improper
- aerobatics initiated
- aerobatics intentional
- aerobatics performed
- air speed exceeded
- altitude clearance inadequate
- altitude disregarded
- altitude inadequate
- altitude low
- ATC clearance not followed
- attitude indicator not available
- buzzing intentional
- buzzing performed
- certification improper for flight
- decision height disregarded

- decision height not complied with
- decision height not maintained
- decision height not used
- descent height disregarded
- design stress limits of ac exceeded
- dispatch procedures not followed other govt personnel
- external navigation lights not used
- flight into adverse weather
- flight into adverse weather attempted
- flight into adverse weather continued
- flight into adverse weather improper
- flight into adverse weather inadvertent
- flight into adverse weather initiated
- flight into adverse weather intentional
- flight into adverse weather performed
- flight into adverse weather selected
- flight manuals disregarded
- flight navigation instruments inadequate
- fuel supply inadequate
- hazardous weather advisory disregarded
- ice frost removal from ac improper
- ifr procedure improper
- ifr procedure not followed
- impairment alcohol

- impairment drugs
- inflight briefing service not used
- inflight planning decision improper
- information insufficient designated examiner
- information insufficient pic
- loading of cargo improper
- low altitude flight maneuver attempted
- low altitude flight maneuver intentional
- low altitude flight maneuver performed
- low pass intentional
- low pass performed
- maintenance annual inspection not complied with
- maintenance annual inspection not performed
- maintenance design changes improper pic
- maintenance installation improper
- maintenance major alteration improper pic
- maintenance major repair improper pic
- maintenance service bulletin not complied with
- maneuver attempted
- minimum descent altitude below
- minimum descent altitude disregarded
- minimum descent altitude not complied with
- minimum descent altitude not maintained
- minimum descent altitude not obtained/maintained

**Variety of Violation Codes (N = 115)**

## Top 10 Violations

| Violation | Frequency |
| --- | --- |
| VFR into IMC | 262 (28.7%) |
| Flight into Adverse Weather | 157 (17.2%) |
| Stress Limits Exceeded | 87 (9.5%) |
| IFR Procedure Not Followed | 53 (5.8%) |
| Weight and Balance Exceeded | 49 (5.4%) |
| Aerobatics Performed | 48 (5.3%) |
| Ostentatious Display/Buzzing | 48 (5.3%) |
| Procedure/Directives Not Followed | 46 (5.0%) |
| Operating With Known Deficiencies | 40 (4.4%) |
| Min. Descent Altitude not Complied with | 26 (2.9%) |

## Top 5 Decision Errors/Violations

| Causal Factor | Frequency |
| --- | --- |
| VFR into IMC | 262 (16.1%) |
| Poor In-flight Planning | 184 (11.3%) |
| Flight into Adverse Weather | 157 (9.7%) |
| Stress Limits Exceeded | 87 (5.4%) |
| Wrong Altitude Selection | 78 (4.8%) |

## Conclusions

- These factors have been explore by the Wx JSAT
- FAA is sponsoring research to empirically explore these factors (go beyond “expert opinion”)
- When decision errors are addressed separately from violations:
- Flight planning and management become the top issues
- This may be more inline with the traditional conceptualization of ADM

**Conclusions**

**Notes:**
Notes same as slide.