Memorandum

U.S. Department of Transportation

Federal Aviation Administration

Subject: INFORMATION: Risk Assessment for Reciprocating Engine

Airworthiness Directives

Date: 5-24-99

From: Manager, Engine and Propeller Standards Staff, ANE-110 Reply to

Attn. of:

Mark Rumizen,

ANE-110

781-238-7113

FAX 781-238-

7199

To: Manager, Aircraft Manufacturing Division, AIR-200

Manager, Brussels Aircraft Certification Staff, AEU-100

Manager, Engine Certification Office, ANE-140

Manager, Engine Certification Branch, ANE-141

Manager, Engine Certification Branch, ANE-142

Manager, Boston Aircraft Certification Office, ANE-150

Manager, New York Aircraft Certification Office, ANE-170

Manager, Airframe and Propulsion Branch, ANE-171

Manager, Rotorcraft Directorate, ASW-100

Manager, Rotorcraft Standards Staff, ASW-110

Manager, Airplane Certification Office, ASW-150

Manager, Rotorcraft Certification Office, ASW-170

Manager, Special Certification Office, ASW-190

Manager, Small Airplane Directorate, ACE-100

Manager, Small Airplane Standards Office, ACE-110

Manager, Atlanta Aircraft Certification Office, ACE-115A

Manager, Propulsion Branch, ACE-140A

Manager, Chicago Aircraft Certification Office, ACE-115C

Manager, Propulsion Branch, ACE-118C

Manager, Wichita Aircraft Certification Office, ACE-115W

Manager, Propulsion Branch, ACE-140W

Manager, Anchorage Aircraft Certification Office, ACE-115N

Manager, Transport Airplane Directorate, ANM-100

Manager, Transport Standards Staff, ANM-110

Manager, Airframe and Propulsion Branch, ANM-112

Manager, Seattle Aircraft Certification Office, ANM-100S

Manager, Propulsion Branch, ANM-140S

Manager, Denver Aircraft Certification Office, ANM-100D

Manager, Los Angeles Aircraft Certification Office, ANM-100L

1. INTRODUCTION

This memo provides guidance for Aircraft Certification Offices (ACOs) to use when evaluating

reciprocating engine service problems for determination of appropriate FAA action. Airworthiness

Directives (AD’s) are required for unsafe conditions, but the determination of which types of engine

service problems should be considered unsafe conditions is dependent upon the type of airplane in which

the engine is installed. Reciprocating engines are typically installed in small airplanes intended for
personal use, and the regulations governing the design and operation of these airplanes incorporate

"mitigating features" to lessen the criticality of the engine. These mitigating features include low stall

speeds, handling and stability criteria, emergency landing procedures, crashworthiness, and pilot training.

These mitigating factors don’t guarantee safety when an engine service problem occurs, but instead

provide a level of assurance that a pilot can reasonably fly the airplane to a safe landing. Using loss of

engine power as measure of an airplane’s ability to accommodate engine failures, actual service data

indicates that total aircraft power losses on turbine powered transport aircraft are ten times more likely to

result in fatalities than on small piston powered GA aircraft. Therefore, it can be substantiated that

General Aviation (GA) aircraft equipped with reciprocating engines differ from turbine powered

transports relative to the criticality of the engine.

This uniqueness of the GA fleet has resulted in inconsistent bases for issuance of ADs related to

reciprocating engine service problems. In some cases, ADs have been issued where other, less

burdensome forms of FAA action would have been more appropriate. And, conversely, in some cases

where no FAA action was taken, an AD was warranted based on the potential safety risk. The FAA and

the turbine engine industry have addressed similar continued airworthiness inconsistencies by instituting

formalized, quantitatively-based risk assessment methodologies for evaluation of service problems. Risk

assessment methodologies can also be applied to the GA reciprocating engine fleet, but must be modified

to accommodate the less sophisticated technical resources and the incompleteness and inaccuracies of

service data that is typical of the GA industry. The risk assessment methodology presented below should

be considered a general guideline, rather than a specific procedure, to use for the evaluation of GA

reciprocating engine service problems. It must be emphasized that, because each service problem presents

its own unique set of circumstances, the risk assessment methodology will need to be customized to

accommodate each analysis.

2. RISK ASSESSMENT METHODOLOGY

A risk analysis utilizes data and information on a service problem to quantify the expected number of

future events over a specified time period. The risk analysis should consider the consequences of the

service problem relative to safety of flight, the probability of that service problem occurring, and the

exposure of the current GA fleet to the problem. The following procedure is provided to assist in

development of a risk analysis for a GA engine service problem. Because the particular details of any

given service problem vary, this procedure can only be considered a starting point; evaluation methods

will likely require customization to fit the specific data. It should also be noted that in many cases, all of

the necessary data may not be available, and estimates must be used in place of the actual data. If

necessary, engineers or flight test pilots can be consulted regarding the characteristics of airplane

response to a given engine problem.

An example based on an actual service problem will be provided to parallel each step of the following

risk assessment process. Each subparagraph will contain its respective step from the example at the end of

the descriptive text. The example will be based on the service problem evaluated for issuance of recent

AD, which addressed failures of engine crankshafts.

a. Consequences of the Engine Service Problem

The first step in the process involves evaluation of the engine service problem to determine the potential

effect on flight safety. For the purpose of this Guidance Memo, engine service problems that are being

considered for AD action can typically be grouped in one of the three following hazard levels:

1. Hazardous: Engine service problems that cause fire, uncontainment or other problems that

could result in immediate collateral damage to the aircraft. These require minimal evaluation
as they represent a direct safety hazard to the aircraft and they should be considered an unsafe

condition that warrants an AD. However, a risk analysis should still be performed to help

determine compliance times for the AD.

2. Major: Engine Service Problems that cause a significant power loss. These events pose an

indirect hazard to the aircraft and do not necessarily require an AD. As discussed above, the

design of GA airplanes incorporate mitigating features that contribute to lessening the severity

of an engine service problem. Other factors, such as probability of the event occurring and

fleet exposure, need to be considered for these service problems before initiating an AD.

3. Minor: Other types of service problems that do not result in a significant power loss, such as a

partial power loss, rough running, pre-ignition, backfire, single magneto failures. These are

potential AD candidates only if the probability of the event is very high.

Information on the consequences of the service problem should be obtained from the production approval

holder (PAH), which includes the engine manufacturer, STC holder, or PMA holder.

EXAMPLE: Manufacturing defects in a certain population of engine crankshafts had experienced

numerous failures resulting in 13 accidents over a six year time period. Failure of the crankshaft resulted

in immediate engine shutdown, but did not result in uncontained engine destruction, failure of the engine

mounting system, fire, or other collateral damage. Therefore, the failure mode posed an indirect hazard

to the airplane and was classified as "major".

b. Identification of Suspect Population

The suspect population consists of all engines on which the service problem might occur. This could

include the entire fleet of a particular engine model, or a subset of that fleet. For example, a quality escape

might only impact a range of engine serial numbers shipped over a certain time period. Identification of

the suspect lot requires input from the PAH. The suspect population can be defined in the following

terms:

• Direct Population: this represents the engines that are confirmed to have the suspect part or

condition and on which the service problem might occur. The direct population can be defined

only if records exist that specifically define engine serial numbers, or a range of engine serial

numbers, on which the risk of the service problem exists. However, the number of engines in the

direct population can be determined based on the number of parts shipped. The conversion of the

number of suspect spare parts to an equivalent number of engines must take a conservative

approach, and assume that a minimum number of the suspect parts were installed in each engine.

• Indirect Population: this represents the engines that require further inspection or maintenance

action to determine if they have the suspect part or condition. This would apply if, for example, a

suspect lot of spare parts were shipped to various third party repair facilities, and records are not

available to identify which engine serial numbers the parts were installed in. Or, if the failure

condition results from an improper repair or maintenance procedure, and it is not known which

engines underwent the repair or action, then all engines of the particular model must be

considered suspect. Determination of the total number of engines of a particular model that are

currently in service can be obtained from the engine manufacturer, or from the FAA aircraft

registry in Oklahoma City.
EXAMPLE: Data from the engine manufacturer and from the FAA indicates that the suspect crankshafts

are installed on approximately 10,100 engines.

• Because the FAA/APO GA Survey presents operating hours for airplanes, not engines, the

number of equivalent airplanes needs to be calculated:

o assume 13% aircraft are twin engine (FAA/APO GA Survey)

o 10,100 engines = 87% N + 2 x (13% N), where N = total no. of airplanes

o N = 8938 airplanes, (1162 twins + 7775 singles)

• this is the direct population because this is an estimate of the number of engines equipped with

the suspect crankshafts

c. Event Rate

The event rate is expressed as the number of service problem events per operating hour. The rate can be

based on actual service experience, test data, or analysis. The rate may change with time; for example, for

a fatigue-related problem, the rate may increase as a part or engine accumulates more total time.

In many cases, only data on the number of accidents is available, not the number of total events. The

event rate will then need to be estimated from the available data. To accomplish this, the following

relationship between shutdowns, accidents and fatal accidents was derived from an analysis of FAA SDR

data and NTSB accident data:

• Shutdowns/power losses:>1 every 10,000 hours

• Accidents: 1 every 100,000 hours

• Fatal Accidents: 1 every 1,000,000 hour

For the purpose of the risk assessment, the event rate is assumed to be equivalent to the shutdown/power

loss rate. The following formulas can then be used to estimate the number of events from available

accident data:

• No. of events = (No. of accidents) X 10, or

• No. of events = (No. of fatal accidents) X 100

EXAMPLE:

• NTSB accident data indicated 13 accidents due to failures of engine crankshafts over the period

from 1986 to 1992

• The event rate needs to estimated from the accident rate

• It is assumed that the event rate will not change in the future.

• estimate applicable airplane flight hours over relevant time period

• piston fleet est’d at 198,335 aircraft (FAA/APO GA Survey)

• applicable airplanes estimated as 8938 (step b above)

• applicable airplanes as % of piston fleet = 8938/198335 = 4.5% of fleet

• 189,947,000 hours for total fleet over ‘86-’92 time period

• 4.5% of total fleet hrs for applicable population = 8,559,036 aircraft hours
• calculate event rate

• 13 accidents/incidents over ‘86-’92 time period

• 13 accidents/ 8,559,036 hrs = 1.52 x 10-6 accident rate

• 10 x (accident rate) = 15.2 x 10-6 event rate

d. Exposure to Failure Condition

The exposure to the service problem is a function of the suspect population, and the number of hours

those engines can be expected to operate over a specified time period.

• Determination of the appropriate time period to use for the analysis depends on the characteristics

of the service problem. In some cases, for high utilization aircraft, it may be appropriate to use the

overhaul period and assume that maintenance is not performed between overhauls. A one year

specified time period may be used if no other basis exists for the estimate.

• The number of hours per engine must be estimated. Manufacturer’s data can be used, or the

General Aviation and Air Taxi Activity Survey, published by the FAA Office of Aviation Policy

and Plans, provides GA fleet utilization hours to estimate the number of hours the suspect

population of engines are operated.

• The total hours of exposure of the suspect population can then be found by multiplying the direct

population by the number of hours per engine per year, multiplied by the specified time period.

EXAMPLE: A one-year time period was chosen for this analysis and the utilization rate was estimated as

130 hour/airplane/year (based on FAA/APO GA Survey).

• Exposure = (130 hrs/airplane/yr) x (8938 airplanes) = 1.16x106 hours

e. Expected Events

The expected number of events can then be found by multiplying the event rate by the number of hours of

exposure over the specified time period. The expected number of events can then be compared to

historical data or FAA safety objectives for the respective event criticality level (hazardous, major or

minor) to determine the appropriate form of FAA action, if any. However, for small populations of at-risk

engines, the risk exposure may be unacceptable even if the analysis forecasts a low number of expected

events. In those cases, further analysis may be required.

The following table illustrates possible alternative courses of FAA action based on the risk assessment

results. It is provided as a recommended guideline, and as previously stated, each service problem will

have unique aspects that may require modifications to this process.
Recommended FAA Action1

Expected Number

of Events2

Minor Failure

Consequences

Major Failure

Consequences

Hazardous Failure

Consequences

Low None

ANPRM

GA Alert

AC 43-16

Or

SAIB

Airworthiness Directive

(AD)

Medium GA Alert

AC 43-16

Or

SAIB3

Airworthiness

Directive (AD)

(EXAMPLE)

Airworthiness Directive

(AD)

High Airworthiness

Directive (AD)

Airworthiness

Directive (AD)

Airworthiness

Directive(AD)

1. This assumes that company actions such as Service Letters, Service Bulletins, and Type Club

or other association publications will be taken. If not, then FAA action may be required to

compensate for the lack of company action.

EXAMPLE:

• Expected events = (event rate) x (exposure)

= (15.2 x 10-6 events/hour) x (1.16x106 hours) = 18 expected events

• For the purposes of the table shown below, 18 expected events are assumed to represent a

"medium" value, and for a major failure condition, an AD is recommended.

f. Other Considerations

The following additional factors should be considered when evaluating the need to issue an AD:

• If the suspect parts are installed on an identifiable group of engines (i.e., by engine serial

number), or if only a small fleet of the suspect engine model exists, then the per flight risk, or risk

exposure of any individual aircraft, to the service problem is higher for a given event probability.

In these cases, an AD would be more likely to be required.

• Service problem occurrence rates that change over time must be considered in the analysis. These

service problems are typically fatigue-related and are more likely to occur as the part or engine

accumulates more operating hours. Additional data is often required to properly assess these

conditions.

• In some instances, where the indirect population greatly exceeds the direct population (those

engines with the suspect part), the number of expected events will be low relative to the size of

the fleet. If an AD is required, the AD compliance section should structured to limit the burden on

the indirect population of engines.
• Other sources of data that can be used to support the risk analysis include FAA Service Difficulty

Report (SDR) and Accident/Incident data, and data from GA organizations such as Airplane

Owners and Operators Association (AOPA) or Aeronautical Repair Station Association (ARSA).

These organizations can conduct surveys of their members to obtain specific information.

Original signed by:

Thomas A. Boudreau, Manager

ANE-110