# Presentation: 107138

## Introduction

**Naval Safety Center**
**School Of Aviation Safety**

**Introduction**
** ****to**
**Maintenance Error Analysis**

**Notes:**
Presentation time: 1 hour
Recommend: Display Motivational Posters/Statements/Articles in the room to support training points and hold students’ interest.

Photo: Aloha Airlines 1988 Accident with one fatality and eight injuries (NTSB archive)

INTRODUCTION:
The Naval Safety Center and the Naval Postgraduate School’s “School of Aviation Safety”, in cooperation with the FAA and NASA, offer this presentation as an overview of human factors in aviation accidents with an introduction to the U.S. Navy’s HFACS-ME model (Human Factors Analysis and Classification System – Maintenance Extension). The goal of this program is the reduction of aviation-related accidents and incidents, in particular, those related to maintenance and ramp activities. HFACS-ME enables organizations to classify and  develop meaningful intervention strategies to prevent similar accidents.

Most investigators can usually identify the major events that cause an accident.  Unfortunately, many of the contributing cause factors are often missed or overlooked.  To prevent a similar accident, however, an investigation must uncover ALL of the cause factors, and more importantly, determine “why” those factors occurred. To date, there is no standardized system that offers a reliable method for determining, coding, and analyzing the “whys” in aviation maintenance error.  The HFACS-ME program is one initiative to correct this deficiency.

## Worldwide Commercial Jet Accident Rates 1965-1999

**Aircraft Design**
**        ****Airway System**
**	****FAA Initiatives**
**	****       ****Technology ( Radar, ILS,GPWS)**
**	****	****    ****Qualifications/Certifications**
**	****	****	****Crew Resource Management**
**	****	****          ****	**

**Accident Rate (accidents per million departures)**

**Notes:**
Commercial aviation has truly become one of the safest forms of transportation, but there is still a need for improvement.

This chart shows the significant reductions in commercial jet aviation accident rates (per million departures) since 1965. 

Improvements in the aviation industry, including (but not limited to) aircraft design, the airway system, FAA initiatives, technology improvements, the industry-wide qualification, certification, and licensing process, as well as, Crew Resource Management have brought the accident rates to the low levels shown on the chart.

HOWEVER, several major accidents still occur each year and often result in hundreds of fatalities.  So, again, improvements are necessary.

(Boeing 1999 Statistical Summary, June 2000)

## Worldwide Departures 1965-1999

**Worldwide Departures 1965-1999**

**Notes:**
To make matters worse, the NUMBER (not rate) of accidents per year are going to increase.  Why?

Although the accident rate per million departures has stabilized (as seen in the previous slide), the number of annual departures continues to grow.  This translates to a greater NUMBER of accidents each year, even if the RATE remains stabilized at its present low level.  

There were 18.68 million departures worldwide in 1999.

Note: The increased number of departures is possible because of the greater number of aircraft and the efficiency of operations.  The number of certified jet aircraft greater than 60,000 pounds (non-military) increased from below 2000 in 1965 to 14,358 in 1999.

(Boeing 1999 Statistical Summary, June 2000)

## Primary Cause of Hull Loss Accidents

**(Boeing, 2000)**

**Primary Cause of Hull Loss Accidents**
**Worldwide Commercial Jet Fleet (1990-1999)**

**Notes:**
To further reduce the accident rate, we must identify and correct the causes of accidents.
This information was charted using 135 total KNOWN cause hull loss accidents during 1990-1999.
(There were an additional 65 UNKNOWN or awaiting reports during this period for a total of 200 hull loss accidents!)

As you can see, flight crew errors comprised the largest percentage of primary cause factors, but maintenance factors were a very considerable 6 percent.
IMPORTANT:  Because this chart shows ONLY the Primary Factors, in DOES NOT INCLUDE contributing factors which involve more than one category.

Primary Factor    Number of Accidents       	%
Flight Crew   	  91 	67
Airplane                     	  15                     	11
Maintenance                 	    8                     	  6
Weather                     	  10                      	  7
Airport/ATC             	    5                       	  4
Misc./Other                 	    6                      	  4
Total    KNOWN        	135                   	99.XX
Unknown                    	  65		
TOT HULL LOSS        	200
(Boeing 1999 Statistical Summary, June 2000)

## How Significant are Maintenance Factors?

_**Studies found that:**_
** ****Maintenance contributed to 15% of commercial **
**   ****jet accidents ****(Boeing, 1995)**
**Maintenance was the 2nd greatest contributor to     **
**  ****fatalities, following CFIT ****(CAA, 1992)**

## The Heinrich Ratio

**The Heinrich Ratio**

**Fatal Accident 1**

**Non-fatal accidents 10**

**Reportable incidents 30**

**Unsafe acts 600**

## Additional Maintenance/Ramp Error Data

**20-30% of engine in-flight shutdowns and 50% of engine-related flight delays/cancellations are caused by maintenance error ****(Boeing, 1997)**
**48,800 non-airworthy aircraft are dispatched per year as a result of maintenance error ****(Marx, 1998)**
**Ramp accidents cost $2-2.5 Billion annually ****(**_**Ramp Safety**_**, Vol. 11:3)**

## Maintenance Accident Costs

- • **Avg. cost of an in-flight engine shutdown is $500,000 **
**• ****Avg. cost of a flight cancellation is $50,000 **
**• ****Avg. cost of a return to gate is $15,000**
**• ****Avg. ground damage incident costs $70,000**
**• ****One airline estimates $75-$100 million/year is lost **
**• ****Airline Transport Association estimates that ground ****	****damage costs $850 million/yr**
**	****	****	****	****	****	****(Source: hfskyway.faa.gov)**

## Watershed Event

**Paradigm Shift: Aloha Airlines, 1988 **

**Watershed Event**

**Brainstorming**

**Notes:**
OK, so we must investigate all mishaps.   Are there better or different methods?
Occasionally, an accident is so unique and overwhelming that it sends shock waves through the industry.  These “watershed events” provide a change in focus and are major catalysts for change.
One of the more famous “watershed events” occurred in 1988, when an Aloha Airlines B737-200 suffered structural fuselage failure and subsequent decompression.  The National Transportation Safety Board (NTSB) conducted an investigation of this accident and cited several human factors issues associated with older aircraft.1  As a direct result of the Aloha Airlines accident, the FAA convened an International Conference on Aging Aircraft in June, 1988. After the second such conference, a leading FAA researcher noted that "the more we looked at problems in maintenance operations, and particularly those of aging aircraft, the more we saw human factors as some part of the problem.“2
NTSB synopsis: FLT 243 EXPERIENCED AN EXPLOSIVE DECOMPRESSION AND STRUCTURAL FAILURE AT FL240 WHILE ENROUTE FROM HILO, HI, TO HONOLULU, HI. APRX 18 FT OF CABIN SKIN AND STRUCTURE AFT OF THE CABIN ENTRANCE DOOR AND ABOVE THE PASSENGER FLOORLINE SEPARATED FROM THE ACFT. ONE FLT ATTENDANT WHO WAS STANDING IN THE AISLE WAS SWEPT OVERBOARD. THE FLT DIVERTED TO MAUI AND A LANDING WAS ACCOMPLISHED. EXAMINATION OF THE ACFT REVEALED DISBONDING AND FATIGUE DAMAGE WHICH LED TO THE FAILURE OF THE LAP JOINT AT S-10L AND THE SEPARATION OF THE FUSELAGE UPPER SKIN BETWEEN STATIONS 360 AND 540. (SEE NTSB/AAR-89/03) 
Probable Cause:Fuselage,attachment..Fatigue & Separation
Contributing Factors:Maintenance,inspection of aircraft..Improper..Company maintenance personnelSupervision..Inadequate..Company/operator managementInadequate surveillance of operation..FAA(organization)Aircraft/equipment, inadequate airframe..Manufacturer 

 1 National Transportation Safety Board (1989). Aircraft accident report: Aloha Airlines Flight 243, Boeing 737-200, N73711, near Maui Hawaii, April 28, 1988. (NTSB/AAR - 89-03). Washington, DC: US Government Printing Office. 
  2Shepherd, W.T. (1990). Meeting objectives. In J.F. Parker, Jr. (Ed.) Final Report-Proceedings of the Second Federal Aviation Administration Meeting on Human Factors Issues in Aircraft Maintenance and Inspection Information Exchange and Communications (pp 6-10). Falls Church, VA: BioTechnology, Inc
NTSB Photo/narrative: Retrieved from hfskyway.faa.gov website: “Human Factors guide for Aviation Maintenance” (1998)

## Maintenance Resource Management Programs

**Maintenance Resource Management Programs**

**Notes:**
The Dupont Model (Gordon Dupont, Transport Canada)
Organizations cannot hope to achieve their accident prevention goals by simply focusing on improved investigations, or by reinforcing any single safety program.  Their efforts (and solutions) must instead have an organizational scope.

The puzzle pieces represent the “complete picture” of programs necessary for the maximum reduction of maintenance error.  

Many companies have implemented portions of this “puzzle”, but without all of the pieces, their total maintenance error prevention programs will be much less effective.

## Error, Incident, Hazard, & Accident Investigation/Reporting

**Error, Incident, Hazard, & Accident Investigation/Reporting**
**Determine Cause Factors (& Targets for Subsequent    **
**    ****Intervention)**
**Provide Information for Suggesting Intervention Development**

**Notes:**
Human Factors Incident Investigation

Aircraft component problems are investigated and recorded as a matter of routine.  Unfortunately, human errors are not investigated with the same diligence.

Dupont recommends that trained investigators conduct human factors investigations for every incident (that meets a determined threshold).  
-The investigator should be a trusted member of the organization.  
-The determined causes should be acted upon as required and the information gained should be widely disseminated.

## Identify Causal Factor Patterns/Trends

**Identify Causal Factor Patterns/Trends**
**Assess Hazards/Risks to Prioritize Potential Targets for **
**    ****Intervention**
**Provide Means to Forecast Potential Impact of Identified **
**   ****Interventions (ROI)**

**Notes:**
Incident Data Analysis

An accurate database of human factor incidents and accidents is necessary to identify hazardous trends and track the effectiveness of corrective actions.

Database commonality between organizations is also desired to maximize the useful sharing of information.

## Establish Metrics for Evaluating Intervention Effectiveness  

**Establish Metrics for Evaluating Intervention Effectiveness  **
**   ****(ROI)**
**Provide Lessons Learned for Specific Operations/Locations**
**Permit Team/Individual Participation in the Development of **
**   ****Interventions**

**Notes:**
Feedback

Feedback is a critical element of any communication process.  Feedback on maintenance error can be generated through the use of direct verbal communication, newsletters, statistics, etc.

## Reporting Errors, Incidents, Hazards, & Accidents

**Reporting Errors, Incidents, Hazards, & Accidents**
**Active Participation in Intervention Development **
**Proactive Identification of Hazards/Errors**
**Top Management Support & Individual Buy-In**

**Notes:**
Company Culture

Company Culture is comprised of an organization’s norms which may either increase or decrease the likelihood of maintenance error.  Companies should develop strong safety cultures, backed by a CEO approved Safety Policy, to reduce maintenance error.

## Individual Worker - Awareness & Skill Development (Performance

_**Individual Worker**_** -** Awareness & Skill Development (Performance
-    Optimization, Personal Safety, Team Synergy, & Proactive
-    Improvement)
_**Line Supervisor**_** -** Awareness, Skill (listed above), Implementation,
-    & Investigation/Reporting
_**Upper Management**_** -** Awareness, Overview of Skills,
-    Investigation/Reporting, Implementation, & Metrics
_**Safety/QA**_** -** Awareness, Skill Development (listed above),
-    Implementation, Investigation/Reporting, & Metrics

**Notes:**
Human Factors Training

And finally, we have Human Factors Training.  It is at the center of this puzzle because it is critical to the success of the other programs. 

Human Factors Training can directly benefit the individual maintainer in the prevention of shop floor errors, but without the success of the other programs, the latent conditions throughout the organization will make an accident much more likely.

## Slide 16

**Notes:**
The Complete Picture

To have a successful maintenance resource management program, all program components must work effectively together. 

 The  “complete picture” offers the greatest potential for reduction of maintenance error!

Because Incident Investigations and Incident Data Analysis are so important to this puzzle, we will now examine the evolution of investigation theories and the development of HFACS-ME.  (go to next slide)

## “Edward’s

**“****Edward’s**
**Shell Model”**

**Damage/Injury**

**Mishap Event**

**Unsafe Act**

**Unsafe**** ****Condition**

**Supervision/Organization**

**“****Heinrich’s**
**Domino Theory”**

## Reason’s “Swiss Cheese” Model

**Accident**

**Reason’s “Swiss Cheese” Model**

**Notes:**
A third model, Reason’s Swiss Cheese Model (1990), has since become the industry standard.
Noting the interrelationships discussed in the previous models, it follows that organizational defenses can stop an accident’s chain of events.  
All defenses have one or more of the following functions:
	- To create understanding and awareness of local hazards
	- To provide alarms and warnings when danger is imminent
	- To restore the system to a safe state
	- To interpose safety barriers between the hazards and the potential losses
	- To contain and eliminate the hazards should they escape this barrier
	- To provide the means of escape and rescue should hazard containment fail
Unfortunately, there are “holes” in those defenses that can line up and allow the chain of events to continue. The  “holes” are also always changing (not static) with new equipment, personnel, procedures, and working conditions.  
 The “holes” are caused by two basic kinds of human errors: active and latent failures:
	Active Failure - an action that has an immediate effect: operator actions
	Latent Failure - an action that has a delayed effect (possibly years): poor design, gaps in supervision, undetected 
		       defects, shortfalls in training, inadequate tools
There will always be “holes” in defenses and humans will always make errors, the key to accident reduction is to identify and repair the individual “holes” before the chain of events is complete.

Reason provides an excellent theoretical  model to describe system and organizational errors; but, how do we proceed to identify the “holes”?  One method of identifying the “holes” is by using the Human Factors Analysis and Classification System (HFACS) developed by the Naval Safety Center.  HFACS incorporated the features of all three models for operational use in aircrew accidents, and it has now been adapted for maintenance (HFACS-ME).

Sources:  	
	Reason, J. (1990). Human Error, Cambridge, UK: Cambridge Press, and
	Reason, J. (1997) Managing the Risks of Organizational Accidents, Aldershot, UK: Ashgate Publishing

## HFACS- Maintenance Extension

**Aberdeen, South Dakota**
**(Payne Stewart accident) **
**October 25, 1999**

## HFACS-ME in Commercial Aviation

**The  FAA’s  Office of Aviation Medicine requested that the US Navy’s School of Aviation Safety apply HFACS-ME to commercial airline accidents**
**15 NTSB Maintenance related accident reports were analyzed from provided sets**
**HFACS-ME was successfully used to code existing NTSB reports**

**Notes:**
HFACS, as previously stated, has been used effectively within the Navy, but it has also proven effective in the analysis of commercial aviation accidents!

Without reopening an investigation, HFACS was effectively applied to existing NTSB reports.
The NTSB/FAA Maintenance Accident Report InfoBase constructed by Galaxy Scientific Corporation for the FAA Office of Aviation Medicine was utilized.  It offers full-text search and hyper linking capabilities which are invaluable tools for researchers and users to review past mishaps. The reports were provided by the Honorable John Goglia, NTSB Member.

Procedure. Each mishap case was independently reviewed and the HFACS-ME codes for each case were entered into a spreadsheet for subsequent tabulation.  Each causal factor was assigned only one HFACS-ME code, and codes were only assigned to issues clearly identified as having had contributed to the mishap. 

Analysis.  15 (63%) reports were selected as clearly having maintenance as a contributing causal factor.  Those excluded involved an in-flight lavatory fire, a lightning strike followed by a fuel cell explosion, a fatality from malfunctioning in-flight service equipment, incorrect take-off/approach procedures, and catastrophic engine failures.  The mishaps were coded independently by the two judges and achieved an “excellent” level of agreement between the two raters.

And what were the results? (Quickly go to next slide)

## HFACS-ME Results of NTSB Reports

**HFACS-ME Results of NTSB Reports**

***NOTE: Similar proportions were found in US Navy accidents!**

**Notes:**
Results:
The major primary and contributing  factors observed include inadequate supervision, failed communications, attention/memory errors, and procedural violations. 
(A total of 36 primary and contributing factors were clearly listed in summary and discussion of  the 15 sample NTSB maintenance related cases that were coded. Some cases had more than one of the same factors present. In a subsequent analysis of the entire case report, arguments could be made for 150 potential latent conditions and 52 active failures)
Based on the original findings, the error sources can be prioritized and then targeted for intervention:
Management Conditions – 60.0% of the 15 NTSB/FAA mishaps reported Supervisory Conditions, whereas 27% had Organizational Conditions (not shown). The majority of management issues involved inadequate procedures, supervision, and training.
Maintainer Conditions – 20.0% of the 15 NTSB/FAA mishaps reported Crew Coordination. Note: Maintainer Conditions were under reported, more are likely present and have an effect. Pass down of information within work groups and from the company to the employees was listed.
Working Conditions – 13% of the 15 NTSB/FAA mishaps reported Environment and Workspace Conditions. Note: Workspace Conditions were under reported, more are likely present and have an effect. Lighting and confined workspace were mentioned as factors.
Maintainer Acts – 47% of the 15 NTSB/FAA mishaps reported Maintainer Errors, whereas 47% had Violations. Most errors entailed omissions or incomplete procedures, whereas violations involved not following procedures. 

Conclusions:
The HFACS-ME was effective in capturing the nature of, and relationships among, latent conditions and active failures present in NTSB/FAA mishaps. The insights gained provide a solid perspective for the development of potential intervention strategies.

## HFACS-ME Summary

**(1)**** HFACS-ME is effective in classifying maintenance errors.**
**(2)**** HFACS-ME may be utilized on both major and minor accidents to fully capture maintenance errors.**
**(3)**** HFACS-ME enables organizations to develop successful intervention strategies.**

**Notes:**
Presentation Summary:

The HFACS-ME model, as you have just seen, has demonstrated effectiveness. 

With HFACS-ME, a safety organization can:
           - effectively classify (code) maintenance related accident causal factors
           - develop specific intervention strategies to mitigate causal factors

NOTE: Because most major accidents occur during flight operations, it is essential to also evaluate the “minor” accidents and incidents that occur on the ramp and in the hangar. Such mishaps involve activities that can lead directly to damage to the aircraft or injury to the maintainer.

## Questions?

**Ramp & Maintenance **
**Incentive for Safety Improvement**

**Questions?**

**Notes:**

Photo: “The good news...we only had one piece of FOD, the bad news....”  (Container ingested in engine)

Questions?
If not, this concludes this presentation on aviation maintenance error and the development of the HFACS-ME model.