# Federal Aviation Administration

Small Airplane Directorate



# Small Airplane Risk Analysis

(SARA)

HandbookTable of Contents
Section PAGE

1. Purpose.............................................................................. 1

2. Related Publications ...................................................... 1

3. Superseded Documents ...................................................... 1

4. Definitions ........................................................................ 1
CHAPTER 1. INTRODUCTION ...................................................... 4

1-1 Replacing the Airworthiness Concer ns Process Risk Assessment ......... 4

1-2 Use of Risk Methods ...................................................... 5

1-3 SARA Process ............................................................... 5

1-4 SAD Product Range ...................................................... 8

1-5 SAD COS Website ...................................................... 9

1-6 Small Airplane Fleet Demographics .................................... 9

1-7 Airworthiness Concerns Sheet ............................................. 9
CHAPTER 2. SARA PROCESS FOR AVIONICS AND AIRPLANE SYSTEMS AND EQUIPMENT ...................................................... 10

2-1 Description of Included Systems ............................................. 10

2-2 Definition of Hazard Levels for Avi onics and Airplane Systems and Equipment Installations ...................................................... 10

2-3 SARA Overview for Avionics and Airplane Systems and Equipment ........................................................................ 13

2-4 Example Problem ............................................................... 24

2-5 The Large Fleet Problem ...................................................... 29
CHAPTER 3. SARA PROCESS FOR PO WERPLANT, PROPELLER, AND APU INSTALLATIONS ...................................................... 31

3-1 Description of Included Systems ............................................. 31

3-2 Evaluation Methods ............................................................... 31

3-3 Working with the Engine and Propeller Directorate .................. 32

3-4 Hazard Ratios and Risk Guidelines .................................... 32

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Table of Contents (continued)
Section PAGE
CHAPTER 4. SARA PROCESS FOR AIRFRAME AND OCCUPANT PROTECTION SYSTEMS ...................................................... 35

4-1 Airframe Risk Analysis ...................................................... 35

4-2 Hazard Level Definitions for Airf rame – Primary Structure .................. 35

4-3 Investigation of Airworthiness Issue .................................... 36

4-4 Quantitative Analysis ...................................................... 36

4-5 Quality Escapes in Primary Structure .................................... 37

4-6 Corrosion in Primary Structure ............................................. 39

4-7 Metal Fatigue in Primary Structure .................................... 41

4-8 Composite Materials – Primary Structure .................................... 49

4-9 Other Structure – Control Surfaces, Tabs, and High Lift Devices ......... 50

4-10 Other Structure – Secondary Structure .................................... 50

4-11 Other Structure – Landing Gear ............................................. 50

4-12 Other Structure – Occupant Protection .................................... 51
APPENDIX 1. WORKSHEET FOR AVI ONICS, SYSTEMS, AND POWERPLANT INSTALLATIONS ............................................. A1-1
APPENDIX 2. WORKSHEET FOR AIRFRAME AND OCCUPANT PROTECTION ........................................................................ A2-1
APPENDIX 3. AIRWORTHINESS CONCERNS SHEET .................. A3-1

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010List of Acronyms

Acronym Definition

ACO Aircraft Certification Office
ACP Airworthiness Concerns Process
ACS Airworthiness Concern Sheet

AD Airworthiness Directive

AIR Aircraft Certification Service
AMC Acceptable Means of Compliance
AOPA Airplane Owners and Pilots Association
APU Auxiliary Power Unit
ASE Aircraft Safety Engineer

BVID Barely Visible Impact Damage
CARB Corrective Action review Board
COS Continued Operational Safety

EAA Experimental Aircraft Association
ECS Environmental Control Systems

EPD Engine and Propeller Directorate

FMP Fatigue Management Plan
GM Guidance Material

IAR Immediately Adopted Rule
IDG Integrated Drive Generator
MSAD Monitor Safety Analyze Safety

NPRM Notice of Proposed Rule Making
NTSB National Transportation Safety Board
RAS Risk Analysis Specification
SAD Small Airplane Directorate
SAIB Special Airworthiness Information Bulletin
SARA Small Airplane Risk Analysis
SDR Service Difficulty Report
SMS Safety Management System
Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010List of Acronyms, continued

Acronym Definition

STC Supplemental Type Certificate
TIS Time in Service

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010List of Figures

Figure Number Title

1 Overview of the Small Airplane Risk Analysis (SARA) Process 2 Event Guidelines for Class I Airplanes and Other Aircraft 3 Event Guidelines for Class II Airplanes 4 Event Guidelines for Class III Airplanes 5 Event Guidelines for Class IV Airplanes 6 Example Problem for Avionics and Airplane Systems and Equipment 7 Effect of Fleet Size on Expected Number of Events 8 Example Quality Escape Scenario 9 Life Analysis Example 10 Life Analysis Flowchart 11 Control Program Based on Component Replacement or Modification 12 Control Programs Based on Damage Tolerance Inspections

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 List of Tables

Table Number Title

1 Aircraft Class Definitions

2 Required Data for Analysis of Avionics and Airplane Systems and Equipment
3 Risk Calculations
4 Recommended Avionics a nd Airplane Systems and Equipment Hazard Ratios

5 Estimating Number of Fatal Accidents for Avionics and Airplane Systems and Equipment 6 Estimated Number of Fatal Accidents for Example Problem 7 Effect of Fleet Size on Expected Number of Events

8 Recommended Hazard Ratios for Single-Engine Powerplant, Propeller, and Auxiliary Power Unit Installations

9 Recommended Hazard Ratios for Multi-Engine Powerplant, Propeller, and Auxiliary Power Unit Installations
10 Structural Hazard Level Classifications 11 Qualitative Hazard Level Descriptions for Corrosion 12 Qualitative Hazard Level Descriptions for Fatigue 13 Adjustments to Time-in-Service for Reported Crack Size1

1. Purpose
The Sm all Airplane Directorate’s (SAD’s) Sma ll Airplane Risk Analysis (SARA) process provides Aviation Safety Engineers (ASEs) with guidance for estimating the risk associated with airworthiness concerns. SA RA also provides guidance on how ASEs can use estimated risk as a consideration in making unsafe condition determinations and in evaluating corrective actions.

2. Related Publications The following publications provide the ASE with additional inform ation on the subject of risk analysis. a) FAA Order 8110.107 Monitor Safety/Analyze Data, March 12, 2010. b) FAA IR-M-8040.1C Airworthiness Directives Manual, May 17, 2010. c) Risk Analysis Specification, October 11, 2006. (FAA Document) d) Engine and Propeller Directorate Con tinued Airworthiness Assessment Process Handbook, Draft. e) AC 39-8 Continued Airworthiness Assessmen ts of Powerplant and Auxiliary Power Unit Installations of Transport Category Airplanes, September 8, 2003. f) Transport Airplane Directorate Ri sk Assessment Methodology (TAD RAM) Handbook, Draft. g) Rotorcraft Risk Analysis Handbook, Draft. h) European Aviation Safety Agency (EASA) Part 21 Acceptable Means of Compliance (AMC) and Guidance Material (GM), October 17, 2003. i) AC 91-82 Fatigue Management Programs for Airplanes with Demonstrated Risk of Catastrophic Failure Due to Fatigue. j) AC 23-1309-1D System Safety Analysis and Assessment for Part 23 Airplanes. k) AC 43-16A General Aviation Maintenance Alerts

3. Superseded Documents
The Sm all Airplane Directorate’s Airworth iness Concerns Process (ACP) Guide is superseded by the FAA Order 8110.107 and this handbook.

4. Definitions The following definitio ns apply to the info rmation provided in this handbook. Do not assume these definitions should apply to the sa me or similar terms used in other federal documents, regulations or ACs. a) CARB. Corrective Action Review Board (see FAA Order 8110.107).

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20102

b) Compliance schedule. A tim etable for performing speci fied corrective actions (e.g., maintenance, inspections, part replacements, a lterations, etc.) to alleviate an identified unsafe condition. c) Continued airworthiness. The ongoing activities associated with ensuring a product remains in compliance with the intent of the applicable Product Design Standards. d) Control program. The combination of compliance sc hedule and corrective actions needed to alleviate an identified unsafe c ondition. Note that the condition is no longer considered to be unsafe while it is being addressed by a control program. e) Event. The occurrence of a failure or othe r condition. Events typically trigger investigations that seek causes of a safety i ssue. The safety issue (or condition) is then evaluated for safety implications. f) Fatal Accident. An occurrence associated with th e operation of an aircraft which takes place between the time any person boards th e aircraft with the intention of flight and until such time as all such persons have disembarked, and in which any person (including those on the ground injured as a re sult of the essen tial operation of the flight) suffers a fatal injury. This definition excludes maintenance, ramp positioning and ground personnel movement, unless injuries are a direct result of the aircraft’s action or malfunction. g) Fatal accident ratio. The conditional probability that an outcome of a specific severity level will result in a fatal accident. h) Fatal injury. An injury that results in death within 30 days of the accident. i) Fleet Risk. A quantitative assessment output equal to the average number of future events expected to occur within a given time. Fleet risk can be differentiated into two types (see below), and typically cover the time required for problem resolution. However, in the case of uncorrected fleet risk and control program fleet risk for control programs that do not incorporat e final corrective actions (e.g., recurring inspections), fleet risk usually cover a 20-year period or shorter interval corresponding to the exp ected life of the fleet. 1) Uncorrected fleet risk. The forecasted number of future events expected to occur in the fleet (or, if applicable, the rele vant affected subfl eet) if no corrective actions are incorporated. 2) Control program fleet risk. The forecasted number of future events expected to occur in the fleet (or, if applicable, th e relevant affected subfleet) during the control program. j) Hazard level. Level of event outcome, as define d by its effect on the aircraft, passengers, and crew. k) Hazard ratio. The conditional probability that a particular failure mode will result in an event of a specific hazard level. l) Individual Risk. This is the risk per flight-hour orper flight cycle. The per flighthour (or per flight cycle) gui delines apply to the average of the fleet or subfleet suspected of having the unsafe condition. The terminology of individual risk is not

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20103

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 meant to imply that the risk to an indivi dual person is calculate d. Rather, individual risk is normally calculated by dividing the fleet risk by the tota l number of flighthours (or flight cycles) within the assumed exposure period. 1) Uncorrected individual risk. The forecasted number of future events per flighthour (or per flight cycle) e xpected to occur in the fleet (or, if a pplicable, the relevant affected subfleet) if no corrective actions are incorporated. 2) Control program individual risk. The forecasted number of future events per flight-hour (or per flight cycle) expected to occur in the fleet (or, if applicable, the relevant affected subfleet) during the control program. m) MSAD. Monitor Safety/Analyze Data (see FAA Order 8110.107). n) Safety Issue. Cause(s), contributing factor(s), or finding(s) that led to, or could lead to, an unsafe outcome. Safety decisions are rendered on issues/causes, not events. For example, investigation of an uncomma nded flight control surface movement--an event--might reveal that the cause was a ci rcuit failure in the autopilot’s computer. Circuit failure is the safety issue/cause to evaluate for safety implications, and to take corrective action against.4

Chapter 1. Introduction

1-1 Replacing the Airworthiness Concerns Process Risk Assessment
The Aircraft Certifica tion Service (AIR) de veloped a Risk Analysis Specification (RAS) as part of the ongoing development of a Safety Management System (SMS). The RAS requires risk analysis to be data driven, objective, and co mparable across AIR’s product range. The RAS clarifies thes e requirements as follows:

 Be quantitative (statistical); e.g., have quantified values for probability and severity.

 Recognize and address ‘conditional proba bilities’ in quan titative terms.

 Define and use a standardized safety-rela ted severity, and relate it to ‘fatal accident(s)’.

 Provide means for ongoing data colle ction and analysis to support quantitative risk analysis. Aviation Safety Engineers (ASEs) have used the method described in the Small Airplane Directorate’s (SAD’s) Airworthiness Concerns Process (ACP) Guide to complete risk assessments of airworthiness concerns on SAD products. The ACP risk assessment relied heavily on the ASE’s qualitative assessment of the severity of the airworthiness concern. The ACP method placed less emphasis on the fr equency of failures or defects and how often these failures or defects actually resu lted in outcomes of a given severity. The potential existed in the ACP risk assessment that failures or defects that rarely result in catastrophic outcomes (for example) could be rated as highly as failures that more frequently result in a ca tastrophic outcome. To address the potential shortcomings in the ACP risk assessment and to comply with the RAS, SAD has developed the Small Airplane Ri sk Analysis (SARA) process. SARA uses quantitative risk measures including the fre quency and expected number of events that result in a given outcome severity. The SARA process is similar to the risk analysis methods described in AC 39-8, Engine and Propeller Directorate Continued Airworthiness Assessment Process Handbook, a nd EASA’s Part 21 Acceptable Means of Compliance (AMC) and Gu idance Material (GM).
To avoid confusion with the traditional se verity terms used in type certification (catastrophic, hazardous, etc), SARA define s severity using ‘Levels’. SARA makes continued airworthiness recommendations base d on the frequency and number of events expected to result in Level 5 (the most se vere), Level 4, and Level 3 outcomes. Other, lower hazard levels exist, specifically Le vels 0, 1, and 2. These h azard levels correspond to hazard descriptions used during type certification, incl uding ‘no safety effect’ and ‘minor safety effect’. We will not manage continued airworthiness based on severity levels below Level 3. Levels 3, 4, and 5 are defined in appropriate sections of this document.
Consistent with the well known ‘Swiss cheese ’ analogy to risk management, using three separate levels of severity provides us with multiple opportunities to correct
Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20105

airworth iness issues before they can resu lt in the most severe outcome. By basing decisions on three different severity levels, we can minimize the impact on the risk analysis accuracy of uncertainti es in the input data, includ ing the hazard ratios and the frequency of the underlying defect or failure.

1-2 Use of Risk Methods We have historically used probabilistic methods in type certification to define levels of safety for systems and avionics design (see AC 23.1309-1D) . It is a natural extension to apply a sim ilar approach to the continued ope rational safety for systems and avionics. We can also apply these methods to powerplant, propeller, and APU installations (see AC 39- 8 and Engine and Propeller Directorate Conti nued Airworthiness Assessm ent Process Handbook, Draft ). A probabilistic approach is not always a ppropriate for m aking decisions on whether airworthiness action is necessary. For example, the majority of primary structure in small airplanes is single-load path structure designed without the be nefit of fail-safe or damage tolerance principles and maintained without the benefit of directed inspection programs. Also, airplane components intended to provi de occupant protection must function as intended in a survivable incident or accide nt. Using a probabilistic approach in these types of situations is not appropriate for making decisions on whether airworthiness action is necessary. However, probabilistic methods can help us determine how quickly we should take an airworthin ess action and how effective a proposed airworthiness action may be in reducing the risk associat ed with an airworthiness concern. SARA recognizes the inherent differences in achieving the required level of safety for each engineering discipline. SARA provides risk guidelines for system s, avionics, and powerplant installations in terms of the freque ncy of events of different severity levels. SARA uses an additional measure, the expect ed number of events over a one year period, as a risk guideline for whether ASEs should consider immediate action. For airframe and occupant protection, SARA provides r isk guidelines in terms of the severity a defect or failure will have on th e airframe or occupant protection component. ASEs should consider airworthin ess action in these cases if in vestigation reveals that the defect or failure may occur in other products of the same type design.

1-3 SARA Process The flowchart in Figure 1 provides an overv iew of the SARA process. F igure 1 shows a clear delineation between the avionics, systems and propulsion philosophy and the airframe and occupant protection approach. We anticipate airworth iness concerns that may not fall within the scope of avionics , systems, and propulsion installations or airframe and occupant protection. Some airw orthiness concerns may also overlap more than one engineering discipline. In these cases, ASEs should contact SAD to determine which risk analysis method is most appropriate for their situation.

a. Avionics, Airplane Systems and Equipment. In the avionics and airplane systems and equipment approach, ASEs will need to determine the hazard ratios appropriate for their airworthiness concern. Hazard ratios are ‘conditional probabilities’

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20106

that transform the frequency of defects or failures into the frequency of outcom es of different severities, Level 3, 4, or 5.
ASEs determine the hazard ratios from service h istory if adequate service history is available for a specific safety concern on a sp ecific make/model or group of airplanes. Collaboration with subject matter experts, flight test pilots, and huma n factors specialists is critical to developing appropriate hazar d ratios. As a process check, ASEs should compare calculated hazard ratios against the h azard ratios provided in Chapter 2 of this handbook. ASEs include the hazard ratios in their presentation to the CARB and provide SAD with descriptions of how the hazar d ratios were developed and any unique characteristics of the installation.
In situations where insufficient service histor y is available, A SEs should use the hazard ratios provided in Chapter 2 of this handbook. ASEs may adjust these hazard ratios based on specifics of a particular instal lation and on engineering judgment.
ASEs use the hazard ratios to determine the un co rrected individual risk, expressed as the frequency of outcomes of a given severity level. ASEs will also determine the uncorrected fleet risk, the to tal number of expected outcomes of a given severity level over a fixed period of time. These risk values represent an estimate of future outcomes if no corrective actions are taken. ASEs will determine the uncorrected fleet risk for two time periods, ‘long-term’ exposure (used to determine whether mandatory action is needed) and ‘short-term’ exposure (used to determine whether immediate action is needed). ASEs will make recommendations for airworthiness actions by comparing these risk values against the risk guideline s provided in Chapter 2 of this handbook. If mandatory action is considered, ASEs shoul d work with the design approval holder or type club to develop candidate control program s. ASEs shoul d evaluate the effectiveness of the candidate programs by evaluating the ab ility of the control programs to reduce the individual and fleet risk to acceptable leve ls. We do not necessarily require the most effective control program, but any control pr ogram should reduce the individual and fleet risk to acceptable levels.

b. Propeller, Powerplant, and APU Installations. The SARA process for propeller, powerplant, and APU installa tions is the same pr ocess used for avionics and airplane systems and equipment. However, th e hazard ratios presented in Chapter 3 of this handbook take into account the significan t differences in outcomes resulting from powerplant failures in single-engine and multi- engine airplanes. Coordination with other Directorates, especially the Engine and Propell er Directorate, may be helpful in resolving propeller, powerplant, and APU inst allation airworthiness concerns.

c. Airframe and Occupant Protection. In the airfram e approach, ASEs first make a determination of whether the defect or fa ilure may occur on other products of the same type design. AC 91-82 describes how ASEs can make these de terminations for instances of fatigue cracking. If the de fect or failure may occur again in the fleet, ASEs make airworthiness recommendations based on the e ffect on the airframe’s strength, fatigue characteristics, or stiffness. If an ASE determines that ai rwor thiness actio n is necessary, the ASE works with the DAH and type club to develop candidate control programs.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20107

Figure 1: O verview of the Small Airp lane Risk Analysis (SARA) Process

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20108

d. Fatal Accid ents. ASEs will need to determine the num ber of fatal accidents that may result if no corrective acti on is taken. The number of fata l accidents is not used to make recommendations on airworthiness act ion but may be used by management to prioritize resources.

e. Worksheets. SAD has developed worksheets for the avionics, systems, and powerplant installations approach an d the airf rame and occupant protection approach to assist ASEs in completing the risk analysis . These worksheets are included in Appendix 1 and 2 and are also available in electronic format on SAD’s COS website.
f. Voluntary Actions. ASEs should consider taking voluntary actions, including SAIBs, service information, or an article in AC 43-16A General Aviation Maintenance Alerts when mandatory ac tion is not appropriate.

1-4 SAD Product Range SAD’s range of products is unique, including m anned free balloons, airships, sailplanes, single-engine and twin-engine airplanes, busin ess jets, and airplane s used in scheduled commuter service. We have applied different exp ectations for levels of safety to each of these products in both ty pe certification (see AC 23.1309-1D) and continued airworthiness. We will continue to recognize the need for diffe rent levels of safety for different SAD products by adjusting the risk guidelines for each type of product. SARA provides separate guidelines for each class of aircraft defined in Table 1. These aircraft classes are similar to those used in AC 23.1309-1D.
Table 1: Aircraft Class Definitions

Class Description Single Reciprocating Engi ne Airplanes < 6,000 lbs. Restricted Category Agricultural Airplanes Sailplanes Normal Category Airships Class I

Manned Free Balloons Multiple Reciprocating Engine Airplanes < 6,000 lbs. Single Turbine Engine Airplanes < 6,000 lbs. Class II

Multiple Turbine Engine Airplanes < 6,000 lbs. Single Reciprocating Engi ne Airplanes > 6,000 lbs. Multiple Reciprocating Engine Airplanes > 6,000 lbs. Single Turbine Engine Airplanes > 6,000 lbs. Class III

Multiple Turbine Engine Airplanes > 6,000 lbs. Class IV Commuter Category Airplanes

Note: Restricted category airpla nes, other than agricultural airplanes, should be evaluate d using the class definition most appropriate for their engine configuration and maximum weight.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 20109

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 1-5 SAD COS Website We recognize the need to provide ASEs with a library of reference m aterials to facilitate risk analysis. ASEs will need access to the late st revision to this document, a catalog of risk analysis examples, hazard ratios, co mputational tools, and fleet demographic information. The Directorate has establis hed a continued opera tional safety (COS) website to provide ASEs access to this info rmation. ASEs can access the website at the following address: https://intranet.faa.gov/FAAEmployees/org/lin ebusiness/avs/offices/air/div_dir/ace/cosm/

1-6 Small Airplane Fleet Demographics
Each risk an alysis will require informati on on the total number of airplanes in the affected fleet and the number of hours on thes e airplanes. To assist ASEs, the Directorate will develop a ‘fleet demographics snapshot’ for each type certificated airplane. The snapshot will include the number of airp lanes of a specific model, their year of manufacture, average hours per year on a speci fic model, an estimate of the total fleet hours, and how many airplanes of a specific mo del are used in commercial service. We will provide this data as a ‘snapshot’ for a specific point in time. We will then update the snapshot on a periodic basis. If a snapshot is not available for your airplane, you can request SAD to develop one for you.

1-7 Airworthiness Concerns Sheet The SARA process retains the Airw orthine ss Concerns Sheet (ACS) as a means of soliciting information from the aviation community. The ACS is not part of any rulemaking activity; its purpose is to solicit technical and econo mic input. If one year or more passes between when the ACS was transmitted and an action (a Notice of Proposed Rulemaking (NPRM), for example) is consid ered, the ASE should evaluate the need for seeking the aviation communities’ input again with a new ACS. The ACS is used in the same way the ACS was used in the ACP. An ACS should be transmitted to the aviation community and fee dback received before an airworthiness action is initiated. ASEs should complete an ACS for Emerge ncy Airworthiness Directive (AD) and Immediately Adopted Rule (IAR) actions. However, unlike the routine NPRM process, the ACS is developed concurrently with the Emergency AD or IAR. In these cases, ASEs should complete and transmit the ACS at the same time they initiate the AD worksheet. The response time for this type of ACS shoul d be consistent with the urgency of the concern, typically 10 days or less. It ma y be necessary for the ASE to communicate directly with user groups to ensure responses are collected in time to support the urgency of the action. The Emergency AD or IAR action will be prepared as the ACS is processed by the user groups. This enhances the deci sion making process by helping gather all available service information prior to issu ing the Emergency AD or Adopted Rule AD. Later, if new information becomes available, the ASE working with the Small Airplane Directorate AD group may initiate further rulemaking to adjust the regulatory impact as appropriate (increase or decrease the AD requirements, for instance).10

Chapter 2. SARA Process for Avionics and Airplane Systems and Equipment

2-1 Description of Included Systems This section describes the SARA process for airplane avionics and system s and equipment typically certificated under part 23 subpart F (or predecessor regulations). ASEs should use the SARA process for avioni cs and airplane systems and equipment, including, but not limited to, the following airplane components.

 pneumatic systems,

 fluid systems (excluding fuel and engine oil systems),

 flight control systems,

 electrical and elec tronic systems,

 avionics systems, and

 mechanical systems and equipment.

2-2 Definition of Hazard Levels for Avionics and Airplane Systems and Equipment ASEs should use the following defin itions of hazard level when completing the SARA process on avionics and airplane systems and equipment. These definitions are similar to those in AC 39-8 with some changes to better reflect small airplane design and operating environment.

a. Level 5 Event: Defect or failure conditions th at are expected to result in:

 Occupant fatalities or incapacitation or fatal injury to the pilot(s), normally with the loss of the airpla ne. These events typically involve loss of survivable cabin volume or environment and uncontrolled or high speed, high angle impacts with terrain.

 Notes:

o Fatal Injury. The NTSB defines “fatal in jury” as an injury that results in death within 30 days of the accident.

b. Level 4 Event: Defect or failure conditions that are expected to result in:

 Serious injuries including life-threatening injuries. Some fatalities may occur in these events even though survivable cabin volume or environment is usually maintaine d. These events may involve high speed, low angle impacts or low speed impacts. These events may also result in hull loss.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201011

 Notes:

o Serious Injuries. The NTSB de finition of "serious injury" means any injury that:

 Requires hospitalization fo r more than 48 hours, commencing within seven days from the date the injury was received,

 results in the fracture of any bone (except simple fractures of fingers, toes or nose),

 involves lacerations that cause severe hemorrhages, nerve, muscle or tendon damage,

 involves injury to any internal organ, or

 involves second or third de gree burns or any burns affecting more than five percent of the body surface.

o Forced landing. For Class IV airplanes (typically commuter category airplanes), ASEs should consider any forced landing as a Level 4 event. Forced landing is defined as the inability to continue flight due to the consequences of damage, uncontrolled fire or thrust lo ss where imminent landing is obvious but aircraft controllability is not necessarily lost (i.e., total power loss due to fuel exhaustion will result in a "forced landing”). The term "emergency la nding" may also be used to mean a forced landing if there is an urgent requirement to land, but declaration of an emergency does not necessarily imply that a forced landing is imminent. An air turn back or diversion due to a malfunction is not a forced landing, since there is a lack of urgency and the crew ha s the ability to select where they will perform the landing. However, off-airport landings are almost always forced landings.

o Uncontained (uncontrolled) fire s in a typical single-engine airplane configuration. ASEs should consider fires or smoke that impinge on or enter the airplane cabin as Level 4 events.

c. Level 3 Event: Defect or failure conditions that are expected to result in:

 Substantial damage to the aircraft or second unrelated system. Non-life threatening serious injuries (for example, simple fractures).

 Notes:

o Substantial damage to the aircraft or second unrelated system. "Substantial damage" mean s damage or structural failure that adversely affects th e limit loads capability of a Primary Structural Element or the performance or flight

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201012

characteristics of the aircraft, and that would norm ally require major repair or replacement of the affected components. Typically not considered “subs tantial damage” (because they do not generally result in the effects delineated above) are:

o engine failure damage limited to the engine or its mounts,

o bent fairings or cowlings,

o dented skin,

o small puncture holes in the skin or fabric,

o damage to landing gear associated with runway departures,

o wheel, tires,

o flaps,

o engine accessories on failed engine,

o brakes or

o wing tips. Damage to a second unrelated system must impact the ability to continue safe flight and landing. Coordination and agreement between the engine /propeller/APU manufacturer and the airframe manufacturer may be required to properly categorize events related to second system damage. In general, aircraft are designed to be di spatched with one part of a redundant system inoperative with no effect on flight safety. Therefore, an uncontained rotor event which severed an unrelated hydraulic system line without significantly degrading the ability to continue safe fli ght should not be considered a level 3 event.

o Small penetrations of aircraft fu el lines or aircraft fuel tanks that lead to fuel exhausti on are a level 3 classification. Assistance of the airframe manu facturer should be sought when questions arise.

o Damage to a second engine that results in a significant loss of thrust or an operational probl em requiring pilot action to reduce power is a level 3 event. Minor damage which was not observed by the crew during flight and which did not affect the ability of the engine to continue safe operation for the rest of the flight is should not be considered a level 3 event.

o Uncontained (uncontrolled) fires th at do not result in a Level 4 event. The concern is impinging flames onto the wing/fuselage or acting as an ignition s ource for flammable material

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201013

antic ipated to be present. Localized fires with very limited fuel sources (such as gearbox and IDG fires) may exit the fire zone in locations remote from th e wing, fuselage and airplane structure or flammable material anticipated to be present and thus not be deemed level 3 since they do not present the above concern. Fires inside the fire z one are not level 3 unless they
escape.

o Rapid depressurization of the cabin is a level 3 event.

o For multi-engine airplanes, permanent loss of thrust or power greater than one propulsion system (inflight) is a level 3 event.

o Temporary or permanent inability to climb and fly 1000 feet above terrain (increased threat from terrain, inclement weather, etc.) along the intended route that results in restricted capability (i.e., multiple propulsion system malfunctions or single propulsion system malfunctions a nd/or other aircraft system malfunction or crew error. ) is a level 3 event.

o Any temporary or permanent impairment of aircraft controllability caused by, for instance, propulsion system malfunction, thrust reverser in flight deployment, propeller control malfunction, or propulsion system malfun ction coupled with aircraft control system malfunction, abnormal aircraft vibration, or crew erro r is a level 3 event.

o Malfunctions or failures resul ting in smoke or other fumes delivered through the ECS system causing serious impairment is a level 3 event. Serious impairment includes the loss of crew’s ability to see flight d eck instrumentation or perform expected flight duties. Purely psychological aspects of the concern of odors, etc., are not to be included, nor are concerns about long-term exposure.

2-3 SARA Overview for Avionics and Ai rplane Systems and Equipment. ASEs should perform the steps outlines in Figure 1 when completing the SARA process for defects and failures relate d to avionics and airplane systems and equipment. A description of these steps is prov ided in the following paragraphs.

a. Collecting Data for Avionics and Airplane Systems and Equipment. To complete the SARA process for avionics a nd airplane systems and equipment, an ASE will need to collect the data summarized in Table 2.

b. Risk Parameter Calculation. After collecting the data in Table 2, the ASE will perform the calculations summarized in Table 3.

c. Frequency of Failures and Defects. ASEs should select service data over an appropriate historical time frame to determin e the frequency of failures and defects. In some cases, service difficulty reports may ex tend over several years. In other case,

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201014

reports m ay have only been received over the pa st few years. For example, if ten reports have been received over the past five years, and eleven reports over the past ten years, a frequency based on the last five years of data may give a better estimation of future events. (In some instances, ASEs should consider using life analysis methods similar to the methods used for airframe fatigue, as described in Chapter 4.) ASEs should use caution when evaluating serv ice reports for a safety issue on which we have previously taken action. Partially eff ective mitigation actions may result in a low frequency of recent past events and ma y bias the decision process towards a nonconservative action to address future events. Table 2: Required Data for Analysis of Avionics and Airplane Systems and Equipment

Input Data Source

Number of Airplanes Demogr aphic snapshot on SAD COS website, FAA Registry, Manufacturer’s records. (Note: If more than one suspect component is installed in an airplane, then risk calculations should consider the number of installed components.)

Flight cycles per year per airplane, Flight hours per year per airplane Demographic snapshot on SAD COS website, Aircraft Bluebook.

Long-term Exposure Assume 20 years for risk calculations

Short-term Exposure Assume 1 year for risk calculations

Frequency of failure or defect Service history in MSAD, Manufacturer’s data

Hazard Ratios ASEs should determine hazard ratios from service history if adequate service history is available for a specific safety concern on a specific make/model or group of airplanes. Calculated hazard ratios can be compared against the values in Table 4 as a check.
In situations where insufficient service history is available, ASEs should use the

recomm ended hazard ratios in Table 4 and adjust these hazard ratios based on specifics of a particular installation and on engineering judgment.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201015

Table 3: Risk Calculations

Risk Parameter Equation Frequency of Failure or Defect (per hour or per flight cycle) Total Number of Defects or Failures / Affected Fleet Hours or Flight Cycles (Calculation based on in-service data) Level 3, 4, and 5 Hazard Ratios Fraction of defects or failu res resulting in Level 3, 4, and 5 events. (Note: Level 3 events in clude Level 4 and Level 5 events. Similarly Level 4 events include Level 5 events.) See Table 4. Uncorrected Individual Risk, Frequency of Level 3, 4, and 5 Events (per hour or per flight cycle) Frequency of Defect or Failure x Level 3, 4, and 5 Hazard Ratios

Long-Term Exposure (hours or flight cycles) Number of Airplanes x Average Flight Hours (Flight Cycles) per year per airplane x Remaining Service Life of Fleet (20 years default) Short-Term Exposure (hours or flight cycles) Number of Airplanes x Average Flight Hours (Flight Cycles) per year per airplane x 1 year Uncorrected Fleet Risk, Short-Term Expected Number of Level 3, 4, and 5 Events Frequency of Level 3, 4, and 5 Events x Short Term Exposure

Uncorrected Fleet Risk, Long-Term Expected Number of Level 3, 4, and 5 Events Frequency of Level 3, 4, and 5 Events x LongTerm Exposure

Control Program Risk Frequency of Level 3, 4, and 5 Events x Control Program Exposure

d. Random and Wearout Failures. We have traditionally treated failures in avionics and airplane systems and equipment as random failures, with a constant rate of failure over time. The rate of failure may act ually increase over time for many failure scenarios. These types of failures are cal led wearout failures. Wearout is a common failure mode in airframe failures, particularly in metal fatigue. ASEs should consider the possibility of w earout failure when evaluating airworthiness concerns in avionics and ai rplane systems and equipment. ASEs should suspect wearout failures when the number of failure reports appears to increase over time. A typical example would be five SDRs over the last three years, two SDRs in the three to six year time frame, and no SDRs older than six year s. ASEs should review the guidance on life analysis in paragraph 4-6 for met hods of evaluating wearout failures.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201016

e. Haz ard Ratios for Avionics and Airplane Systems and Equipment. ASEs should determ ine hazard ratios from service history if adequate service history is available for a specific safety concern on a sp ecific make/model or group of airplanes. ASEs should calculate the data described in Table 2 and de termine the hazard ratios as described in Table 3. ASEs should collaborate with subject matter e xperts, flight test pilots, and human factors sp ecialists when developing h azard ratios. ASEs should compare calculated hazard ratios against the values in Table 4 as a process check. ASEs should include the hazard ratios in their pr esentation to the CARB and provide SAD with descriptions of how the hazard ratios were developed and any unique characteristics of the installation. In developing hazard ratios based on service history, ASEs should assume that the next failure or defect will result in the lowest severity with no reported outcomes. For example, an airworthiness concern has result ed in 10 Level 3 outcomes and no Level 4 or Level 5 outcomes. In computing the hazard ra tios, ASEs should assume that the next event will result in a Level 4 outcome. In th is case, an ASE would calculate the hazard ratios based on 11 Level 3 events, 1 Level 4 ev ent, and no Level 5 events. This approach will result in a reasonably conservative interpretation of service history.
In situations where insufficient service hi story is available, A SEs should use the recommended hazard ratios in Table 4. SAD has developed the hazard ratios in Table 4 for a num ber of common defects and failure s in avionics and airplane systems and equipment. ASEs may adjust these hazard ratios based on specifics of a p articular installation and on engineering judgment. If ASEs adjust these hazard ratios, ASEs should include the hazard ratios in their pr esentation to the CARB and provide SAD with descriptions of how the hazard ratios were ad justed and any unique characteristics of the installation. We recognize that these hazard ratios may not cover every defect or failure an ASE will encounter. SAD is developing methods to evalua te failures in avionics, airplane systems, and powerplant installations that may be modeled using methods similar to the life analysis used for fatigue in metal structures.
f. Risk Guidelines for Avionics, Airplane Systems and Equipment. Once the ASE has calculated the risk parameters in Table 3, the ASE then compares the risk parameters against the recommended risk guide lines included in Figures 2, 3, 4, and 5 to make recommendations on airworthiness actions. The risk guidelines are presented for the four classes of aircraft described in Table 1. ASEs should consider airworthiness action if any if the risk v alu es exceed the risk guidelines.

 High Risk. If the risk parameters fall within an area of the guidelines labeled ‘High Risk’, ASEs should consider taking immediate action (Immediately Adopted Rule or Emergency AD). The goal of the immediate action is to reduce the risk to at least the level of ‘Excessive Risk’. If the immediate action reduces risk only to the level of ‘Excessive Risk’ ASEs should consider additional action to further reduce th e risk. The additional action would typically be an NPRM.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201017

 Excessive Risk . If the risk param eters fall w ithin an area of the guidelines labeled ‘Excessive Risk’, ASEs shoul d consider developing a routine NPRM action.

 Reasonable Risk. Mandatory airworthine ss action is not gene rally indicated if the r isk parameters fall into an ar ea of the guidelines labeled ‘Reasonable Risk’. In these cases, ASEs may cons ider voluntary actions including SAIBs and service information.

g. Fatal Accidents. If the ASE determines that ma ndatory action is necessary, the ASE should estimate the number of fatal accide nts that may result if no corrective action is taken. Table 5 provides ASEs with the calculations necessa ry to estimate the number of fatal accidents. ASEs may adjust the recomm ended fatal accident ratios as necessary based on the circumstances of the airw orthiness concern and service history.

h. Control Programs. If an ASE determines that ma ndatory action is necessary, the ASE should work with the type design holder to develop a control program. Acceptable control programs must reduce the long term individual risk for Level 3, 4, and 5 events below the long term individual risk guideline, into the area labeled ‘Reasonable Risk’ in Figures 2, 3, 4, and 5. Control programs must also reduce the risk while the control program is implemented. The control program risk must be within the area on the ri sk guidelines corresponding to ‘excessive risk’ or ‘reasonable risk’. The control program ri sk ensures that the compliance timing in control programs provides an acceptable level of safety. ASEs should evaluate the control program fleet risk over the entire period of tim e allowed for compliance. For example, if a c ontrol program requires replacing a specific component within the next three years, ASEs would estimate how many airplanes comply over each of the next three years and the num ber of airplanes not yet in compliance in each of the next three years.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201018

Table 4: Recommended Avionics and Airpla ne S ystems and Equipment Hazard Ratios

Failure Condition Hazard Ratios Level 5 Level 4 Level 3 Flight Control System Failure .15 .3 1.0 FC_LOSSOFCONTROL SMA_MISRIGGED SMA_FAILURE_CONTROL-CABLE Flight Controls (un-commanded movement ) .05 .1 1.0 FC_UNCOMMANDEDMANUVEUR FC_UNCOMMANDEDPITCH FC_HARDOVER Brake System Failure (loss of braking) .1 .3 1.0 MS_BRAKE_FAILURE FC_LOST_BRAKES Hydraulic System Failure (multiple systems) .05 .25 .7 MS_MULTIPLE_HYDRAULIC_FAILURE Oxygen system Failure .1 .3 1.0 MS_OXYGEN_SYSTEM_FAILURE Environmental Control Failures .1 .3 1.0 EC_LOSS_CABIN_PRESSURE SMA_FAILURE_PRESSURIZATION EC_ICE_PROTECTION Cabin Safety, Smoke and Fire .3 .7 1.0 CS_SMOKE_AND_FIRE CS_SMOKE_AND_FIRE_CABIN CS_SMOKE_AND_FIRE_CARGO Landing Gear System Failure 0 0 .5 SMA_FAILURE_LDNG_GEAR_EXTEND Crew Seat failure .05 .25 .8 CS_CREW_SEAT_SLIPPED Avionics and Navigation Failure .1 .3 1.0 AV_LOSS_OF_AIRSPEED AV_MISLEADING_AIRSPEED AV_LOST_DISPLAYS AV_MISLEADING_ALTITUDE AV_MISLEADING_ATTITUDE AV_STALL_WARNING Electrical Power Distribution Failure .2 .5 1.0 PD_BAD_WIRE_SMOKE_FIRE PD_LOST_ELECTRIC_POWER

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201019

Table 5: Estim ating Number of Fatal Accide nts for Avionics and Airplane Systems and Equipment

Hazard

Level Historical
Fatal Accident Ratio Estimated Number of Fatal Accidents

Level 5 1.00 Level 5 Fatal Accidents =
Level 5 Fatal Accident Ratio x
20 Year Fleet Risk for Level 5 Outcomes

Level 4 0.10 Level 4 Fatal Accidents =
Level 4 Fatal Accident Ratio x ( 20 Year Fleet Risk for Level 4 Outcomes – 20 Year Fleet Risk for Level 5 Outcomes )

Level 3 0.00 Level 3 Fatal Accidents =
Level 3 Fatal Accident Ratio x ( 20 Year Fleet Risk for Level 3 Outcomes –
20 Year Fleet Risk for Level 4 Outcomes – 20 Year Fleet Risk for Level 5 Outcomes )

Total Estimated Fatal Accidents Level 5 + Level 4 + Level 3 Fatal Accidents

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201020

Figure 2: E vent Guidelines for Cl ass I Airplanes and Other Aircraft

0.7

O ut c om

e s

7

. 0 Outcomes

7 0 O

u t comes

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201021

Figure 3: E vent Guidelines for Class II AirplanesUncorrected Individual RiskOf Level 5 Outcomes(Frequency of Level 5 Outcomes)Number of Outcomes per Hour(per Flight) Event Guidelines for AC 23.1309-1D Class II Airplanes (Multi-Engine Reciprocating, Single-Engine Turbine, or Multi-engine Turbine Airplanes < 6,000 lbs.)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 Exposure, Fleet Hours(Cycles) 10 -3

10 -7 10 -6 10 -5 10 -4

10 -8 0.5 Outco

m e s High Risk

Reasonable Risk Excessive Risk High Risk (if more than 0.5 outcomes expected in next 1 year period)Uncorrected Individual RiskOf Level 4 Outcomes(Frequency of Level 4 Outcomes)Number of Outcomesper Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -2

10 -6 10 -5 10 -4 10 -3

10 -7 5 .0

O u tc omes High Risk

Reasonable Risk Excessive Risk High Risk (if more than 5.0 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)Uncorrected Individual RiskOf Level 3 Outcomes(Frequency of Level 3 Outcomes)Number of Outcomesper Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -1

10 -5 10 -4 10 -3 10 -2

10 -6 50 Outc

o m e s High Risk

Reasonable Risk Excessive Risk High Risk (if more than 50 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201022

Figure 4: E vent Guidelines for Class III AirplanesUncorrected Individual RiskOf Level 5 Outcomes(Frequency of Level 5 Outcomes)Number of Outcomes per Hour(per Flight) Event Guidelines for AC 23.1309-1D Class III Airplanes (Single-Engine Recipr ocating, Multi-Engi ne Reciprocating, Single-Engine Turbine, or Multi-Engine Turbine Airplanes > 6,000 lbs)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 Exposure, Fleet Hours(Cycles) 10 -4

10 -8 10 -7 10 -6 10 -5

10 -9 0

. 3 Outc

o m e s High Risk

Reasonable Risk Excessive Risk High Risk (if more than 0.3 outcomes expected in next 1 year period)Uncorrected Individual RiskOf Level 4 Outcomes(Frequency of Level 4 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -3

10 -7 10 -6 10 -5 10 -4

10 -8 3. 0 O ut co

m e s High Risk

Reasonable Risk Excessive Risk High Risk (if more than 3.0 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)Uncorrected Individual RiskOf Level 3 Outcomes(Frequency of Level 3 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -1

10 -5 10 -4 10 -3 10 -2

10 -6 3 0 O ut c omes High Risk

Reasonable Risk Excessive Risk High Risk (if more than 30 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201023

Figure 5: E vent Guidelines for Class IV Airplanes

0. 1 Out

c o m e s

1 .0 Outcomes

10

O u tc omes

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201024

2-4 Example Problem

a. Data Collec tion. You receive reports of complete loss of braking in Model X airplanes due to ruptured br ake lines in the wheel well. A check of MSAD safety information database shows that over the last 20 years there are 255 reports of the same or similar failures resulting in loss of braking in Model X airplanes. The aircraft registry lists approximately 820 active Model X airplane s. Historical data indicates that the majority of these aircraft have been active ov er the last 20 years. Since the Model X is a single engine general aviati on aircraft under 6000 pounds MTOW (Class I), it is assumed to have an average of 125 flight hours pe r year, and each flight cycle lasts 0.61 hours.

b. Failure Mode. In this example, we have assume d that the reported failures occur randomly over the past 20 years. ASEs shoul d evaluate the available failure data for evidence of a wearout failure mode.

c. Worksheet. ASEs should use the worksheet incl uded in Appendix 1 to complete the following calculations.

d. Hazard Ratios. Review of the 255 reported fail ures resulted in the following assignments of severity to each of the repor ted failures. These assignments were made using engineering judgment and by collaborating with subject ma tter experts, flight test pilots, and human factors specialists.

 25 of the failures resulted in Level 5 outcomes.

 76 of the failures resulted in Level 4 outcomes. (Note: Level 4 outcomes include those failures that resu lted in Level 5 outcomes.)

 255 of the failures resulted in Level 3 outcomes. (Note: Level 3 outcomes include those failures that resu lted in Level 4 outcomes.)

 Level 5 Hazard Ratio = Number of fa ilures resulting in Level 5 outcomes / Total number of failures = 25 / 255 = 0.098  0.1

 Level 4 Hazard Ratio = 76 / 255 = 0.298  0.3

 Level 3 Hazard Ratio = 255 / 255 = 1.0
These calculated hazard ratios corre spond to the recommended hazard ratios for loss of braking included in Table 4.

e. Uncorrected Individual Risk. Failures in the braki ng system, and, m ore importantly, the hazardous outcomes due to failures in the braking system, are independent of the length of flight or tota l flight hours and are a function of when the brakes are used in landing (and to a lesser ex tent taxi and takeoff) . In this case, the frequency of the brake system failure should be on a per flight cycl e basis and not a per flight hour basis.

 Frequency of braking system failure = ( 255 reported failures / ( 820 airplanes x 20 years ) ) x ( 1 year / 125 hours per airplane ) x ( 0.61 hours / flight ) = 7.59 x 10 -5 per flight cycle

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201025

 Frequency of Level 5 outcom es = 0.10 x 7.59 x 10 -5 per flight = 7.59 x 10 -6 per flight cycle

 Frequency of Level 4 outcomes = 0.30 x 7.59 x 10 -5 per flight = 2.28 x 10 -5 per flight cycle

 Frequency of Level 3 outcomes = 1.00 x 7.59 x 10 -5 per flight = 7.59 x 10 -5 per flight cycle
f. Uncorrected Fleet Risk.

 Short-term Exposure = 1 year x 820 airplanes x 125 hours per airplane per year x 1 flight / 0.61 hours = 1.68 x 10 5 flight cycles

 Short-term Level 5 Outcomes = = 7.59 x 10 -6 x 1.68 x 10 5 = 1.28 Level 5 outcomes

 Short-term Level 4 Outcomes = = 2.28 x 10 -5 x 1.68 x 10 5 = 3.83 Level 4 outcomes

 Short-term Level 3 Outcomes = = 7.59 x 10 -5 x 1.68 x 10 5 = 12.75 Level 3 outcomes

 Long-term Exposure = 20 x Short-term Exposure = 3.36 x 10 6 flight cycles

 Long-term Level 5 Outcomes = = 7.59 x 10 -6 x 3.36 x 10 6 = 25.5 Level 5 outcomes

 Long-term Level 4 Outcomes = = 2.28 x 10 -5 x 3.36 x 10 6 = 76.6 Level 4 outcomes

 Long-term Level 3 Outcomes = = 7.59 x 10 -5 x 3.36 x 10 6 = 255 Level 3 outcomes

g. Risk Guidelines. We have plotted the uncorrected i ndividual risk a nd fleet risk in Figure 6.

 For Level 5 outcomes:

o The uncorrected individual risk is above the risk guideline for mandatory airworthiness action (NPRM) (7.59 x 10 -6 > 1 x 10 -6 ) but below the risk guideline fo r immediate action (7.59 x 10 -6 < 5 x 10 -4 ).

o The short-term fleet ri sk is above the risk guideline for recommending immediate action (1.28 > 0.7).

 For Level 4 outcomes:

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201026

o The uncorrected individual risk is above the risk guideline for

m andatory airworthiness action (NPRM) (2.28 x 10 -5 > 1 x 10 -5 ) but below the risk guideline fo r immediate action (2.28 x 10 -5 < 5 x 10 -3 ).

o The short-term fleet risk is just below the risk guideline for immediate action (3.83 < 7.0).

 For Level 3 outcomes:

o The uncorrected individual risk is just below the risk guideline for recommending an NPRM (7.59 x 10 -5 < 1 x 10 -4 ). (Note that FAA Order 8110.107 directs ASEs to pres ent findings to the CARB if the uncorrected individual risk is within two-thirds of the risk guideline.)

o The short-term fleet risk below the risk guideli ne for immediate action. (12.75 < 70.0).
An ASE should consider mandato ry airworthiness acti on if any of the risk values at any severity level exceeds the risk guidelines. In th is example, the risk values for Level 4 and Level 5 result in a recommendation for at leas t an NPRM. The short-term fleet risk for Level 5 results in a recommendation for immediate action.

h. Fatal Accidents. Table 6 provides an estimate of the number of fatal accidents that might occur if this airworthiness c oncern is left uncorrected. (See Table 5 for description of method.) Again, ASEs will not use these values for making recommendations for airworthiness action. Management may use these values for resource allocation.
Table 6: Estimated Number of Fa tal Accidents for Example Problem

Hazard

Level Recommended
Fatal Accident Ratio Estimated Number of Fatal Accidents

Level 5 1.00 Level 5 Fatal Accidents =

1.00 x 25.5 = 25.5

Level 4 0.10 Level 4 Fatal Accidents = 0.10 x ( 76.6 – 25.5 ) = 5.1

Level 3 0.00 Level 3 Fatal Accidents = 0.00 x ( 255 – 76.6 – 25.5 ) = 0.0

Total Estimated Fatal Accidents 30.6

i. Control Program Risk. Any control program in this example must be capable of reducing the individual risk below the long-term individual risk guidelines shown in Figure 6. Because of the high level of the fl eet risk for Level 5 events (1.28 Level 5 events over the next year) any control program must also consider the control program fleet risk guideline for Level 5 events, no more than 0.7 Level 5 even ts while the control

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201027

program is implemented across the fleet. If we expect 1.28 events to occur in just one year, any control program in this example w ould have to quickly remove and replace the affected components across the fleet.

 Candidate Control Program: Fleetwide replacement of affected component within a six month period.

o Control program fleet risk (Level 5) = 7.59 x 10 -6 per flight cycle x 0.5 year x 820 airplanes x 125 hours per airplane per year x 1 flight / 0.61 hours = 0.64 Level 5 Events

o The control program fleet risk is below the control program risk guideline ( 0.64 Level 5 Events < 0.70 Level 5 Events)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201028

Figure 6: E xample Problem for Avionics , Airplane Systems and Equipment, and Powerplant, Propeller and APU InstallationsUncorrected Individual RiskOf Level 5 Outcomes(Frequency of Level 5 Outcomes)Number of Outcomes per Hour(per Flight) Event Guidelines for AC 23.1309-1D Class I Airplanes (Single Reciprocating Engine Airplanes < 6,000 lbs. Use also for Restricted Cate gory Agricultural Airplanes, Normal Category Airships, and Sailplanes)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 Exposure, Fleet Hours(Cycles) 10 -2

10 -6 10 -5 10 -4 10 -3

10 -7 0

. 7

O u tc o m e s High Risk

Reasonable Risk Excessive Risk

High Risk (if more than 0.7 outcomes expected in next 1 year period)Uncorrected Individual RiskOf Level 4 Outcomes(Frequency of Level 4 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -1

10 -5 10 -4 10 -3 10 -2

10 -6 7. 0 O

u tc o m e s High Risk

Reasonable Risk Excessive Risk High Risk (if more than 7.0 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)Uncorrected Individual RiskOf Level 3 Outcomes(Frequency of Level 3 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 0

10 -4 10 -3 10 -2 10 -1

10 -5 70 Out

c omes High Risk

Reasonable Risk Excessive Risk High Risk (if more than 70 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles) X X Uncorrected Individual RiskShort-Term ExposureLong-Term Exposure Short-Term Uncorrected Fleet Risk

Long-Term Uncorrected Fleet Risk

X X Uncorrected Individual Risk

X X

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201029

2-5 The Large Fleet Problem

a. Problem Statement. O ne consequence of relatively low values for the acceptable uncorrected risk guidelines for Class I airplane s is that a particular failure and outcome can occur at a rate below the uncorrected indivi dual risk guideline but still have relatively high uncorrected fleet risk. A failure can occu r in a large fleet of airplanes at a rate acceptable to the individual risk guidelines (and to type certification requirements) but we expect a substantial number of Level 5 or Level 4 outcomes to occur.

b. Example.

 Fleet size: 25,000 airplane s (Cessna 172s currently on the FAA registry)

 Long-term (20 years) Exposure: 25,000 airplanes x 125 hours/ye ar/airplane x 20 years = 6.25 x 10 7 fleet flight hours.

 Uncorrected Individual Risk for Level 5 outcomes: 1 x 10 -7 Level 5 outcomes per flight hour (for example)

 Uncorrected Long-term Fleet Risk: 6.25 x 10 7 fleet flight hours x 1 x 10 -7 Level 5 events / flight hour = 6.25 Level 5 events In this example, the uncorrect ed individual risk is below th e risk guidelines for Level 5 outcomes in Class I airplanes. However, becau se of the large fleet size, we expect a significant number of Level 5 ev ents over the next 20 years.

c. Fleet Size. We have depicted this result on th e Level 5 risk guideline shown in Figure 7 along with the expected number of Level 5 events fo r a range of fleet sizes. We have also summarized the Figure 7 data in the following table. Table 7: Effect of Fleet Size on Expected Number of Events

Affected Fleet Size Hours per year per airplane 20-Year Fleet Exposure (hours) Uncorrected Individual Risk (Level 5 outcomes

per flight hour) Uncorrected Fleet Risk (Level 5 outcomes)

1,000 125 2.50 x 10 6 1 x 10 -7 0.25 4,000 125 1.00 x 10 7 1 x 10 -7 1.00 25,000 125 6.25 x 10 7 1 x 10 -7 6.25

d. Level of Safety Provided by Type Certification. In general, we cannot require a higher level of safety for continued operat ional safety than we require for type certification. The corresp onding certification requirement 14 CFR § 23.1309 (and AC 23.1309-1D) was written with the in tent of reducing the acceptab le level of certitude for complex systems that provide an overall incr ease in the level of safety by improving pilot situational awareness. However, many simple systems in small airplanes were type certificated without consideration of their failu re rates. In cases of simple systems or systems with a long history of satisfactory se rvice history in othe r airplanes, we may require higher levels of safety (acceptabl e uncorrected individua l risk) than AC 23.1309- 1D recommends.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201030

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 e. Recommendations. In situations where the uncorrected indivi dual risk guideline is not exceeded, the ASE and CARB should st ill consider airworth iness action if we expect one or more Level 5 events or 10 or more Level 4 events over a 20 year period. The ASE and the CARB should consider:

 The number of outcomes expected over 20 years (long-term fleet risk).

 How close the uncorrected individual risk is to the risk guidelines.

 How much confidence we have in the unc orrected individual risk including hazard ratios and the frequenc y of the failure or defect.

 The reliability of similar systems installed on other airplanes.

 Input from the design approval holder, i ndustry associations, and type clubs. Figure 7: Effect of Fleet Size on Expected Number of EventsUncorrected Individual RiskOf Level 5 Outcomes(Frequency of Level 5 Outcomes)Number of Outcomes per Hour(per Flight)

0 .7 O

u tc om

e s1,0004,00025,00031

Chapter 3. SARA Process for Powerplant , Propeller, and APU Installations

3-1 Description of Included Systems This section describes the SARA proce ss for powerplant, propeller, and A PU installations typically certificated under pa rt 23 subpart E (or predecessor regulations). ASEs should use the SARA process for powerplant, propeller, and APU installations, including, but not limited to, the following airplane components.

a. The airplane powerplant installation, including:

 Main engines and propellers,

 Engine and propeller accessories in cluding engine oil systems, engine bleed systems, gear boxes, etc.,

 Engine and propeller co ntrols and indications,

 Engine and propeller protection systems, including engine fire protection systems, engine over-speed protection systems, engine icing protection systems, etc.,

 Engine nacelles and cowling, includi ng inlets, exhaust nozzles, core cowls, fan cowls, thrust reversers, etc.,

 Engine struts and pylons, and

 Fuel systems, including fuel feed sy stems, refuel and de-fuel systems, fuel transfer systems, fuel syst em controls and indications, etc.

b. The auxiliary power unit (i f installed), including:

 The APU,

 APU accessories,

 APU inlet and exhaust systems,

 APU control systems,

 APU indicating systems,

 APU fire protection systems, and

 APU bleed air systems, etc.

3-2 Evaluation Methods The risk evaluation for powerplant, propeller, and APU installations is the same as that described in Chapter 2 for avionics and ai rplane systems and equipment. ASEs should use the SARA process when the airworthiness concern involves issues associated with the airplane design. For issues concerning the powerplant, propeller, or APU design,

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201032

ASEs should refer to the Engine and Prope ller Directorate Continued Airworthiness Assessm ent Process Handbook for powerplant design issues.

3-3 Working With the Engine and Propeller Directorate Safety concerns involving powerplant, propelle r , or APU installations in small airplanes will often require close coordination between the ACO, SAD, and E&PD. For safety concerns resulting from shortcomings in th e airplane design, excl usive of the engine (propeller or APU) design, SAD will be responsible for making control program decisions. Similarly, for safety concerns cau sed primarily by shortcomings within the engine type design, the E&PD is responsib le for making control program decisions. Different risk management objectives may l ead to differences between Directorates on the appropriate actions necessa ry to address identified unsafe conditions, or on the timing of those actions. Due to the inter-involvemen t of the Directorates with propulsion related safety issues, especially those that are installation dependent, close coordination is recommended. However, if differences persis t after coordination and discussion of the assumptions, available data, and objectives, the final control program decision will rest with the Directorate having airworthiness responsibility for the associated product. The E&PD has developed the Engine and Prope ller Directorate Continued Airworthiness Assessment Process Handbook (Draft) to provide ASEs with guidance specific to evaluating safety concerns resulting prim arily from the powerplant design. ASEs should refer to the E&PD guidance fo r powerplant design issues.

3-4 Hazard Ratios and Risk Guidelines The risk analysis for safety concerns resu lting prim arily from shortcomings in the airplane design uses the same SARA methods described in Chapter 2 for avionics and airplane systems and equipment. ASEs should refer to the guidance and methods included in Chapter 2 of this handbook wh en evaluating safety concerns involving powerplant, propeller, or APU inst allations in small airplanes.

a. Hazard ratios. ASEs may use the recommended h azard ratios in Table 8 and Table 9 when evaluating safety concerns resu lting primarily from shortcomings in the airplane design. If the safety concern result s from the engine design, ASEs should use the guidance provided in the Engine and Propell er Directorate Continued Airworthiness Assessment Process Handbook.

b. Risk guidelines. ASEs should use the risk guidel ines in Figures 2, 3, 4, and 5 when evaluating safety concerns involving pow erplant, propeller, or APU installations in small airplanes in which the design of the airplane is the primary shortcoming. Otherwise, ASEs should use the guidance provided in the Engine and Propeller Directorate Continued Airworth iness Assessment Process Handbook.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201033

Table 8: Recommended Hazard Ratios for Single-Engine Po werplant, Propeller, and Auxiliary Power Unit Installations (Use only when the safety concern involves shortcomings in the airplane design. Othe rwise, refer to the Engine and Propeller Directorate Continued Airworth iness Assessment Process Handbook)

Failure Condition Hazard Ratios Level 5 Level 4 Level 3 Propeller Separation .05 .12 1.0 PROP_FAILURE PROP_SEPARATION PROP_HUB PROP_TIP_SEPARATION Propeller Debris .01 .02 .05 PROP_COUNTERWEIGHT PROP_EROSION_STRIP_SEPARATION Control System .01 .03 1.0 PROP_ENG_OVERSPEED PROP_FLAT_PITCH PROP_GOVERNOR PROP_LOW_PITCH PROP_RUNAWAY PROP_NOT_FEATHER Engine Shutdown .01 .1 1.0 ENGINE_SHUTDOWN Engine Fire (reference AC 39-8) ENGINE_FIRE_IN_ENGINE ENGINE_APU_FIRE Engine Uncontained (reference AC39-8) ENGINE_UNCONTAINED_FAILURE ENGINE_APU_UNCONTAINED

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201034

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 Table 9: Recommended Hazard Ratios for Multi-Engine Powerplant, Propeller, and Auxiliary Power Unit Installations (Use only when the safety concern involves shortcomings in the airplane design. Othe rwise, refer to the Engine and Propeller Directorate Continued Airworth iness Assessment Process Handbook)

Failure Condition Hazard Ratios Level 5 Level 4 Level 3 Propeller Separation .3 .45 1.0 PROP_FAILURE PROP_SEPARATION PROP_HUB PROP_TIP_SEPARATION Propeller Debris .2 .2 1.0 PROP_COUNTERWEIGHT PROP_EROSION_STRIP_SEPARATION Control System .01 .03 1.0 PROP_ENG_OVERSPEED PROP_FLAT_PITCH PROP_GOVERNOR PROP_LOW_PITCH PROP_RUNAWAY PROP_NOT_FEATHER Engine Shutdown .01 .1 1.0 ENGINE_SHUTDOWN Engine Fire (reference AC 39-8) ENGINE_FIRE_IN_ENGINE ENGINE_APU_FIRE Engine Uncontained (reference AC39 -8) ENGINE_UNCONTAINED_FAILURE ENGINE_APU_UNCONTAINED35

Chapter 4. SARA Process for Airframe and Occupant Protection

4-1 Airframe Risk Analysis Approxim ately seventy percent of all airframe airworthiness directives address metal fatigue and twenty-five percent address co rrosion. A quantitative, probabilistic risk analysis capable of predicting when a structural component would fail and cause an accident requires a probabilistic damage tolerance analysis. The vast majority of the small airplane fleet received type certification prior to the regulations that required fati gue evaluation. Only a small perc entage of the small airplane fleet has damage tolerance information readily available. It is impractical to expect type certificate holders to develop th e required information in a ti mely manner to support COS investigations. The SARA process therefore ba ses COS decisions on whether fatigue cracks or corrosion are likely to occur on other airplanes in th e fleet and whether these cracks or corrosion will reduce the strength of the structure to below certification basis levels.

4-2 Hazard Level Definitions for Airframe – Primary Structure A defect or failure in prim ary structure that reduces margins of safety below that required by the type design certificati on basis should be considered for airworthine ss action. An ASE can assign a hazard level to these defects or failures following the gui dance in Table 10.
Table 10: S tructural H azard Level Classifications

Hazard Level Residual Strength Residual Stiffness (Flutter Speed)

Level 5 ≤ 100% Limit Load ≤ 1.0 V D

Level 4 ≤ 130% Limit Load ≤ 1.1 V D

Level 3 < 150% Limit Load < 1.2 V D Examples of such defects or failu res include, but are not limited to:

 Fatigue cracks and stress corrosion cracking in metallic structure,

 Corrosion in metallic structure that re duces the strength of the structure by reducing effective net section or that causes corrosion pi tting and cracking,

 Quality control escapes in metallic structure resulting in under-strength parts or components. Examples would include incorrect alloy, incorrect heat treatment, missing or incorrect fasten ers, inadequate edge margins, or misdrilled holes.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201036

 Delam inations, disbonds, or susceptibility to barely visible impact damage (BVID) in composite structure.

 Quality control escapes in composite structure including incomplete bonding, voids, foreign material inclusion, pr ocess errors, missing or incorrect fasteners, inadequate edge margins, or misdrilled holes.

 Defects or failures that alter the stiffn ess or mass distribution of the structure resulting in a reduction in the flutter ma rgins below that required by the type design certification basis. Stiffness ma y be affected by the same defects or failures as strength, including fatigue cracks, stress corrosion cracking, corrosion, delaminations and disbonds, a nd quality control escapes. The mass distribution of the structure could be e ffected by conditions such as moisture ingress in honeycomb or composite structures.

 Defects or failures that reduce the fa tigue life of life limited components, reduce the inspection threshold or insp ection interval of damage tolerant components (reduced residual strength capability), or that require more effective inspection methods of damage tolerant components. These defects or failures could include quality control escap es resulting in use of material with inferior fatigue, crack growth, or fracture properties.

4-3 Investigation of Airworthiness Issue ASEs should investigate the cause of an airw orthiness concern to determ ine whether the defect or failure may occur on other products of the same type design. Airworthiness concerns that are not expected to occur el sewhere in the fleet, for example, corrosion caused by accidental spillage of a corros ive fluid, are generally not candidates for mandatory airworthiness action. AC 91-82 provides a discussion of investigating instances of fatigue cracking. If the ASE’s investigation sh ows that the airworthiness concern may occur on other products of the same type design, the ASE should make recommendations on airworthiness action based on the defect’s or failure’s effect on the structure as defined in Table 10.

4-4 Quantitative Analysis
ASEs should m ake airworthiness recomme ndations based on the results of their investigation of the airworthiness issue a nd their quantitative assessment of the hazard level of the airworthiness concern. ASEs do not need to complete a probabilistic analysis to determine whether an airworthin ess concern is an unsafe condition. The majority of prim ary structure in small airplanes is singl e-load path structure designed without the benefit of fail-safe or damage tolerance prin ciples and maintained without the benefit of directed inspection programs. A probabilisti c approach for these structures is not appropriate for making decisions on whet her airworthiness action is necessary. However, an ASE should complete a quantit ative risk analysis to determine the effectiveness of candidate control programs. To complete this analysis, the ASE needs to determine the expected number of airplanes or components with the de fect or failure and

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201037

the effectiveness of the candidate control progr am s. We discuss the us ual types of defects or failures affecting primary structure, the control programs we typically use to address these issues, and how to evaluate candidate control programs in the following paragraphs. We have developed the worksheets provided in Appendix 2 to assist ASEs in evaluating airframe concerns. These worksheets are av ailable on the Directorate’s COS website.

4-5 Quality Escapes in Primary Structure (Fixed Number of Affected Components)

a. Need for Corrective A ction. An ASE should base a recommendation on corrective action largely on the quality escape’ s hazard level, as defined in Table 10. Quality escapes may include engineering erro rs, manufacturing and assembly errors, or inspection errors.

b. Uncorrected Individual and Fleet Risk. In a quality escape situation, the uncorrected fleet risk (both s hort-term and long-term) is equa l to the number of airplanes manufactured with the quality escape. Th e uncorrected individual risk for level 3 outcomes is 1.0 per flight or per hour. This va lue for uncorrected individual risk is above the risk guidelines used in the avionics, airp lane systems and equipment, and powerplant, propeller and APU installation appr oach for taking immediate action.

c. Immediate Action. We typically addre ss quality escapes in primary structure using a two step approach. The first is to take immediate action (Emergency AD or IAR) placing temporary operating limitations on the affected airplanes. For example, quality escapes that reduce the structure’s static stre ngth can be mitigated in the short-term by reducing the airplane’s maximum weight, lim it load factor, and operating airspeeds. Other immediate actions may include inspecti on of the airplane to determine if the quality escape exists on the airplane

d. Control Programs. The immediate action described above may be able to mitigate the risk associated with continued operation of quality escapes. However, we do not allow continued reliance on operating limita tions as a permanent fix for a defect in primary structure when modifications or re pairs can eliminate th e cause of the unsafe condition. (See FAA IR-M-8040.1C Airworthine ss Directives Manual. ) We therefore follow the immediate action with an NPRM to require repair , modification, or replacement of the affected components. A typical quality escape scenario is de picted in Figure 8. We know, from the manufacturer, that a small number of airpla nes were manufactured with the defect, 17 airplanes in this example. If we did not intervene with a control program, the 17 airplanes would continue to operate with the defect. (Of course, the compa ny would issue service information and some owners might voluntar ily comply with the service information. However, we would not accept voluntary comp liance as an acceptable control program for Level 3 or higher defects in primary structure.) The short-term control program mitigates th e risk until we can implement the long-term control program. The elapsed calendar time (and fleet exposure) to implement the longterm program is depicted in Figure 8. This de lay corresponds to the time to complete the NPRM process and the compliance time allowed in the final rule. The long-term plan
Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201038

replaces, m odifies, or repairs as quickly as possible. The long-term control program should be capable of replacing or modifying all of the quality escape components in a reasonable period of time. Factors ASEs s hould consider in determining a compliance schedule include the availability of replacement components, maintenance resources, and the hazard level (Level 3/4/5) of the quality escape.

e. Control Program Risk. The short-term control program should reduce the future risk of components with below required strength to zero so that the control program risk is zero. It is possible that ope rating restrictions may not be 100% effective in reducing the risk. A pilot may intentionally or inadvertently exceed the reduced limitations. An ASE may consider the possibility of airplanes exceeding the reduced limitations in determining the control program risk. This may be an important consideration in instances of large affected fl eet sizes or the types of ope ration. The long-term control program should be capable of replacing or modifying all of the quality escape components in a reasonable period of time.
f. Fatal Accidents. ASEs should use the values in Table 5 for estimating the number of fatal accidents resulting from long- term fleet exposure. ASEs should consider adjusting the fatal accident ratio for Level 5 outcomes when the structure’s strength is reduced below limit load or the flutter speed is reduced to within the normal operating envelope. For example, if the structure is only capable of 2g, we would expect nearly 100% of the quality control defects to result in fatal accidents. Figure 8: Example Quality Escape Scenario

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201039

4-6 Corrosion in Primary Structure (C onstant Defect or Failure Rate)

a. Need for Corrective A ction. An ASE should base a corrective action recommendation on the corrosion hazard level, de fined in Table 10, and the root cause of the corrosion finding. In making a recommenda tion on the need for corrective action, an ASE should understand that, left uncorrecte d, an instance of corrosion with Level 3 effects may progress in hazard level to Level 4 and Level 5. An ASE should consider not only the current hazard level of a corrosion finding, but th e potential for the finding to progress in hazard level. An ASE may not always have adequate da ta from the corrosion finding or from the manufacturer to make a quantitative determin ation of hazard level. In these cases, an ASE should use engineering judgment to make a qualitative determination of hazard level. Table 11 provides guidance an ASE should consider in making a qualitative determination of hazard level. Table 11: Qualitative Hazard Level Descri ptions for Corrosion (Use only if residual strength and stiffness da ta is not available.)

Hazard Level Description of Corrosion Corrosion through 25% or more of the component cross sectional area. Level 5

Corrosion that is a potential ly urgent airworthiness concern requiring immediate action. Level 4 Corrosion through up to 25% of the component cross sectional area.
Corrosion that requires rework / blend-out that exceeds allowable limits, requiring a repa ir or complete or partial replacement of a principal st ructural element (PSE) as defined by the original equipment manufacturer’s structural repair manual. Corrosion that is widespread and requires blend-out approaching allowable rework limits. Level 3

Corrosion through up to 10% of the component thickness.

b. Uncorrected Individual and Fleet Risk. Estimating the unco rrected individual and fleet risk for corrosion issues has to cons ider that instances of corrosion that start with Level 3 consequences will likely progre ss in hazard level to Level 4 and Level 5 consequences. Estimating when corrosion re aches Level 4 or Level 5 would require knowledge of the rate at which the structural component is corroded away. In general, we do not have data or a statisti cal model that provides us with this knowledge. However, we do know that we expect all inst ances of Level 3 corrosion to progress to Level 4 and then Level 5 with an associated time lag between th e different hazard levels. In estimating the uncorrected risk, it is sufficient to set the risk of Level 4 and Level 5 outcomes to the same value as for Level 3, recognizing that each level is reached at a different point in
Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201040

tim e. In estimating the uncorre cted individual risk and flee t risk, ASEs should consider that corrosion may be significantly under-r eported in the service history. ASEs may increase the number of corrosion findings used in the risk analysis to account for the effect of under-reporting.

c. Immediate Action. Immediate action is sometimes necessary in addressing corrosion issues. An ASE should consider im mediate action in the following situations:

 The airworthiness concern is the re sult of finding Level 4 or Level 5 corrosion in a Class IV airplane.

 The airworthiness concern is the resu lt of finding Level 5 corrosion in a Class I, II, or III airplane.

 An inspection program required by an existing AD was not adequate to detect corrosion before it reached Level 5 consequences.

d. Control Programs. We typically address unsafe c onditions due to corrosion with inspections and on-condition repa irs, part replacement, or modifications. Some corrosion airworthiness actions require applicat ion of corrosion prevention treatments. Airworthiness actions should co rrect root causes of corrosion, for example, plugged drain holes, when possible. An ASE may predict the expected number of fu ture corrosion events (with at least Level 3 effects) assuming a constant defect or fa ilure rate, based on service history. An ASE may adjust the estimated failure rate to account for under-reporting of corrosion.Instances of minor corrosion, including Level 3 corrosion, may especially be underreported in the available databases. ASEs shoul d consider alternate measures of exposure, such as calendar years rather than the usual measure of flight cycl es or flight hours. (More advanced statistical methods for pr edicting the expected number of corrosion events often do not give more accurate or reli able predictions due to the variables that effect corrosion initiation.) ASEs do not need to make separate predicti ons of the expected number of Level 4 or Level 5 events. Making these detailed pred ictions would require data on how fast corrosion progresses through a structural component.

e. Control Program Risk. In the absence of termina ting action or addressing the cause of the corrosion, we expect instances of corrosion to continue to occur and progress in hazard level throughout the fleet. In these cases, a corro sion control program should be capable of detecting corrosion before the hazard level progresses beyond Level 3. The control program risk for Level 3 outcome s may therefore remain relatively high. However, the control program risk for Le vel 4 or Level 5 outcomes should be zero. ASEs should monitor the in-service perfor mance of the control program. Mandatory inspection finding reports incl uded in the airworthiness action can assist ASEs in collecting data on the control pr ogram effectiveness. ASEs should consider more frequent inspections or additional actions, incl uding application of corrosion prevention treatments, for any control program that does no t detect corrosion befo re it progresses in hazard level to Level 4 or Level 5.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201041

f. Fatal Accid ents. A review of past airworthiness action suggests that about 1 in 20 corrosion ADs is a direct result of a fatal accident. The remaining ADs are the result of corrosion findings that were exte rnally visible or detectable (fuel leak or wavy or bubbled skin) or found at annual inspection. Based on this data, an estimate of the number of fatal accidents would be five percent of all occurrences of corrosion.

4-7 Metal Fatigue in Primary Structure

a. Need for Corrective A ction. Metal fatigue is of part icular concern as an airworthiness issue. We expect the number of cracks in the fleet to increase exponentially as the fleet ages, with a significant percen tage, if not the enti re fleet, eventually experiencing the same fatigue cracking scenar io. We also expect a crack with Level 3 consequences to grow in size and progress in hazard level to Level 4 and Level 5 consequences. An ASE should recommend corrective action for reports of fatigue cracking that result or will result in at least Level 3 consequences. An ASE may not always have adequate data from the fatigue crack finding or from the manufacturer to make a quantitative determinatio n of the hazard level. In these cases, an ASE should use engineering judgment to make a qualitative determination of the hazard level. Table 12 provides guidance an AS E should consider in making qualitative determinations. Table 12: Qualitative Hazard Level Descriptions for Fatigue Cracks (Use only if residual strength and residual stiffne ss data is not available.)

Hazard Level Description of Fatigue Crack Level 5 Fatigue crack through 25% of the cross sectional area up to and including total component failure. Level 4 Fatigue crack at least 0.10 inch in maximum dimension up to 25% of the cross sectional area. Level 3 Fatigue crack less than 0.10 inch in maximum dimension.

b. Uncorrected Individual and Fleet Risk. Metal fatigue usually presents itself as cracks found in service. Other scenarios may in clude data provided by the type certificate holder showing that cracks are likely to fo rm or that existing life limits should be shortened. In each of these scenarios ASEs should estimate the uncorrected individual and fleet risk by estimating the number of cracks currently present in the fleet and the number of cracks likely to de velop in the future. ASEs can determine these values using the life analysis methods described below. Estimating the uncorrected individual and fleet risk for fatigue issues is similar to corrosion in that fatigue that starts with Level 3 consequences will likely progress in Hazard level to Level 4 and Level 5 consequences. Estimating when a fatigue crack reaches Level 4 or Level 5 would require knowle dge of the crack growth rate. This data is not available for the majority of the small airplane fleet. However, we do know that we expect all instances of Level 3 fatigue crack ing to progress to Level 4 and then Level 5 with an associated time lag between the different hazard levels. In estimating the uncorrected risk, it is sufficient to set the risk of Level 4 and Level 5 outcomes to the

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201042

sam e value as for Level 3, recognizing that each level is reached at a different point in time.

c. Immediate Action. Immediate airworthiness action is sometimes necessary in addressing fatigue issues. An ASE should consider immedi ate action in the following situations:

 The airworthiness concern is the result of finding a Level 4 or Level 5 fatigue crack in a Class IV airplane.

 The airworthiness concern is the result of finding a Level 5 fatigue crack in a Class I, II, or III airplane.

 An inspection program required by an ex isting AD was not adequate to detect a Level 5 crack.

d. Control Programs. AC 91-82 describes how an app licant can develop a control program based on inspections or component re placement/modification to address fatigue cracking. We expect control programs base d on replacement/modification to provide a higher degree of risk mitigation than control programs based on inspections alone. Removing the source of the unsafe condition w ith a design change clearly reduces the risk to the least practical amount. Control programs based on damage tolerance- based inspections with on-condition repair or modification may be capable of managing th e risk associated with fatigue cracking. Without changes to the design, we expect fatig ue cracks to continue to initiate throughout the fleet and any cracks will progress in h azard level. Any inspection-based control program should be capable of detecting instan ces of fatigue cracking before they progress in hazard level beyond Level 3. An ASE shoul d consider a fatigue crack finding with Level 4 or Level 5 consequences during a re petitive inspection as evidence that the control program is not capable of adequately mitigating the risk. In these cases, the FAA may require a control program based on fleet-wide component replacement/modification. In evaluating control programs for fatigue, ASEs will need to complete a life analysis to determine:

 The time-in-service (TIS) corresponding to a cumulative probability of 1-in1,000 (1/1,000) of a crack initiating in a component. (For the purposes of life analysis, ‘crack initiation’ corresponds to the existence of a ‘small’ fatigue crack. This may be the smallest, reliabl y detectable crack size or a crack approximately 0.10 inches in maximum dimension.) Components at or above this TIS require a control program. Co mponents below this TIS can continue to operate without restrictions.

 The number of cracked components we exp ect at any point in the service life of the fleet. This information, combined with crack growth data, allows us to estimate the number of Level 3, Level 4, and Level 5 cracking events at any point in the fleet’s service life.

e. Life Analysis. A life analysis uses statistical distributions (typically a two parameter Weibull distribution) to estimate the fleet-wide distribution of time-in-service (TIS) to the initiation of fati gue cracking. Figure 9 presents a typical life analysis using

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201043

Figure 9: L ife Analysis Example

The Directorate encourages ASEs to become familiar with life analysis methods. We realize that many ASEs may have infrequent opportunities to co mplete a life analysis and use available software tools. In these circumst ances, the Directorate will assist the ASE in completing the analysis. The Directorate’s COS SharePoint site provides points of contact for assistance in life analysis. ASEs completing a life analysis should use the following standard methodology:

 Time-in-Service (TIS): ASEs should de termine the TIS for crack findings and negative inspection results (suspensions) from the TI S included in inspection reports. If this data is not available, an ASE can estimate the TIS based on fleet demographic snapshots provide d on the SAD COS SharePoint site.

 Suspensions: Suspensions should be incl uded in the analysis to account for the remainder of the fleet without kn own crack detections. Suspensions are usually components known not to be cr acked after a directed inspection. We will assume that components that have not yet been inspected are also suspensions.

 Components per Airplane: ASEs shoul d be careful to properly count the number of components in the fleet. Fo r example, an airplane will have
Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201044

identical wing-to-fuselage attachm ent jo ints on both sides of the airplane. One airplane will thus have tw o components. Both or ne ither components could be cracked, or one could be cracked and th e other not cracked. ASEs should base the life analysis on the number of components in the fleet, not the number of airplanes.

 Reported Crack Size: Reported fatigue cracks may include cracks ranging in size from small cracks to cracks that ha ve grown to the point of component failure (complete element fracture). Th e TIS for larger cracks will include a significant portion of time spent growi ng the crack after crack initiation. The discrepancy between the TIS reported for small cracks and large cracks can significantly change the re sults of the life analysis, since we are trying to estimate the TIS to crack initiation (assu med to be the presence of a ‘small’ crack). An ASE can adjust the TIS for a crack of a given size to the TIS corresponding to crack initiation usi ng crack growth data supplied by the manufacturer. If this data is not availa ble, an ASE can adjust the TIS based on the ‘2/3’ rule of thumb. (If the TIS at component failure due to fatigue was 15,000 hours, the TIS at crack initiati on was approximately 10,000 hours.) If crack growth data is not available, an ASE may use the guidance in Table 13 to adjust the TIS for various crack sizes. Table 13: Adjustments to Time-in-Service for Reported Crack Size (Use only if crack growth data is not available.) Reported Crack Size Approximate TIS at Crack Initiation Crack through entire cross section, component failure = Reported TIS / 1.5

Crack 25% through the part thickness = Reported TIS / 1.3

Small crack (approximately 0.10 inches) = Reported TIS / 1.0

 If the life analysis is based on only on e reported fatigue crack, the WeiBayes distribution (WinSmith/SuperSmith) is th e preferred distribution. The Weibull slope parameter, β , should be set to 4.5 for aluminum wing and empennage structures and to 3.8 fo r steel wing structures.

 If the life analysis is based on fewer than 21 reported cracks, the ASE should complete both a WeiBayes analysis a nd a two-parameter Weibull distribution using rank regression (this is th e default analysis method in WinSmith/SuperSmith). The ASE should then use engineering judgment to decide which analysis appears to bett er fit the reported crack findings. An ASE should use caution when five or fewe r reported cracks are included in the analysis. Justifying use of a distribut ion other than a WeiBayes may be difficult in these circumstances.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201045

Figure 10: Life An alysis Flowchart

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201046

 If the life analysis is based on 21 or more reported cracks, the ASE should use the two-pa rameter Weibull distri bution using rank regression. ASEs can use other distributions, including the three-parameter Weibull and log-normal distributions, if these distribut ions provide a significantly be tter fit to the data. ASEs should be careful if confronted with reported cracks that appear to fit a three-parameter or ‘curved’ Weibull distributions. These cracki ng scenarios may actually be a mixture of more than one underlying distribution represen ting different cracking scenarios, including severe usage and manufacturing issues. AS Es should consult with the Directorate whenever the data appears to ‘curve’ or if the ASE uses a distribution other than the twoparameter Weibull distri bution in the analysis. Life analysis software packages, incl uding WinSmith/SuperSmith, are capable of producing many of the risk analysis values we need, including the TIS corresponding to a cumulative probability of 1/1,000 and the expect ed number of cracks at any point in the service life of the fleet. ASEs conducting a life analysis can use the software’s capabilities to compute these numbers or they can use a spreadsheet developed by the Directorate to compute these numbers. The spreadsheet, called FatigueLife, is available on the Directorate’s COS SharePoint site . An ASE may also develop their own spreadsheet or software code for compu ting these numbers, using for example, the methods in The New Weibull Handook, 4 th Edition by Dr. Robert B. Abernethy. Figure 10 provides a flowchart of the life analysis process. The flowchart incorporates the standard methodology outlined above and is based on many of the best practices recommended The New Weibull Handook.
f. Time-in-Service for Control Program. The Directorate requires a control program to address fatigue cracking when a component reaches the TIS corresponding to a 1-in-1,000 (1/1,000) cumulative probability of fatigue crack initiation. An ASE can use the WinSmith/SuperSmith software or the FatigueLife spreadsheet to determine the B.1 life (1/1,000). Once a component (airplane) reaches the B.1 life, the component must have a fatigue management program (FMP). AC 91-82 provides additional information on FMPs.

g. Expected Number of Cracks. An ASE can use the WinSmith/SuperSmith software or the FatigueLife spreadsheet to determine the expected number of cracks in the fleet. At a minimum, ASEs should determ ine the expected number of cracks at the present time and at one, three, five, ten, and twenty years in the future. ASEs use this information to determine the expediency required for control programs relying on component replacement or modification and to evaluate the effectiveness of inspectionbased control programs.

h. Calibrating Life Analysis to Reported Crack Findings. ASEs should calibrate the life analysis against the number of cracks reported from the field. ASEs should complete this analysis only af ter the majority (if not all) of the components in the fleet have been inspected. It is possible, even after updating with the most recent inspection results, that the life analysis will over-predict or under-predict the number of cracks in the fleet.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201047

ASEs should com pare the expected number of cracks at the present time against the number of cracks reported in the field. If the life analysis over-p redicts the number of cracks, the Weibull characteristic life parameter, η , needs to be adjusted to a higher value. Conversely, if the life analys is under-predicts the number of cracks, the value of η needs to be adjusted to a lower value. ASEs should use judgment when making these adjustments. For example, if the life analysis predicts 20 cracks and the field re ports 19, no adjustment to the analysis is necessary. However, ASEs should adjust the an alysis if 20 cracks are predicted and only 15 are reported in the field. ASEs can use the FatigueLife spreadsheet to adjust the value of η until there is a reasonable agreement between the analysis and inspection results.
i. Control Programs Based on Component Replacement or Modification. A control program based on component replacement or modification is si milar to the ‘safelife’ approach used in type certification. Co mponents above the TIS at which we require a control program must be removed from se rvice, replaced, or modified. Components with less than the TIS requiring a control program can continue to operate without restriction until they reach the TIS. The difficulty with the ‘safe-lif e’ approach in continued opera tional safety is that several in-service components will likely be well a bove the TIS required for a control program. The options available for these components include immediately removing them from service (including potential grounding of th e airplane) or immediately replacing or modifying the component. In many cases, repl acement or modification parts may not be available or maintenance resource s may have limited availability. In these cases, the Directorat e and the responsible ACO have worked with the design approval holder, STC applicants, and airplane owners to develop a staged compliance schedule that maintains an acceptable level of safety, maximizes use of available replacement or modification hardware and maintenance resources, and minimizes the effect on airplane owners. These complian ce schedules require t hose components with the highest risk of developing a fatigue crack (high time components) to be replaced or modified first, followed by components with lower TIS. A control program based on component replaceme nt or modification is depicted in Figure 11. Control program 1 requires fleet-wide repl acement of the affected component over a two year compliance time. This control program reduces the expected number of cracks in the fleet to the least amount. However, th is control program may have the unintended consequence of grounding airplanes if sufficie nt hardware or maintenance resources are not available. Control program 2 includes a staged compliance schedule over a four year period that addresses high time component s first followed by components with lower TIS. Control program 2, although resulting in more cracks in the fleet, may provide an acceptable level of safety when combined with additional inspections for components waiting for replacement or modification. Marv Nuss presented a conference paper on a staged compliance program developed for the Cessna 402. A copy of his paper and pres entation are available on the Directorate’s COS website.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201048

Figure 11: Control Program Based on Co mponent Replacement or Modification

j. Control Programs Based on Damage Tolerance-Based Inspections. Control programs based on damage tolerance-based inspections with on-condition repair or modification may be capable of managing the risk associated with fatigue cracking. These control programs may be capable of main taining an acceptable level of safety if they detect all incidences of fatigue cracki ng before the crack reaches Level 4 or Level 5. Figure 12 depicts the effectiveness of a c ontrol program based on damage tolerance inspections. An effective inspection-base d control program should be capable of detecting instances of fatigue cracking befo re they progress in hazard level beyond Level 3. An ASE should consider a fatigue crack findi ng with Level 4 or Level 5 consequences during a repetitive inspection as evidence that the control program is not capable of adequately mitigating the risk. In these cases, the FAA may require a control program based on fleet-wide component replacement/modification. Inspection programs in which the detectable crack size results in Level 4 or Level 5 hazards are not acceptable for a long-term control program. In these cases, the FAA will require a control program based on fleet-w ide component replacement modification. These inspections can be used as short-term me asures to mitigate the risk associated with continued operation until part repl acement or modification can occur.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201049

Figure 12: Control Program s Base d on Damage Tolerance Inspections

k. Control Program Risk. Control programs based on fleet-wide component replacement or modification should be capable of replacing or modifying all components in a reasonable period of time. Factors ASEs should consider in determining a compliance schedule include the fleet-wide dist ribution of TIS and the risk associated with continued operation, availability of replacement components, and availability of maintenance resources. Control programs based on inspections should be capable of detecting fatigue cracks before they progress to Level 4 or Level 5 hazards. Mandatory reporting requirements included in the airworthiness action can provide ASEs with the data necessary to monitor the effectiveness of the inspection program. ASEs should consider more frequent inspections or terminating actions if servic e history shows the insp ection program is not capable of preventing Level 4 or Level 5 fatigue cracks.

l. Fatal Accidents. A review of past airworthiness action suggests that about 1 in 20 fatigue ADs is a direct result of a fatal acc ident. The remaining ADs are the result of findings that were easily detectable (fuel leaks) or found at annual inspection. Based on this data, an estimate of the number of fa tal accidents would be five percent of all occurrences of fatigue.

4-8 Composite Materials – Primary Structure

For com posite primary structure, ASEs shoul d use the most appropriate method presented for metallic primary structure to determine hazard level and the expected number of future events. For defects or failures that w ould be obvious and would be repaired before further flight, the residual st rength and residual stiffness capab ility of the structure does

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201050

not need to account for environm ental factors. If the defect or failure would not be detected and repaired before further flight , the residual strength a nd residual stiffness must be calculated using appropr iate environmental factors.

4-9 Other Structure - Control Surfaces, Tabs, and High Lift Devices

a. Flutter. An ASE should base a recommendation on airworthiness action using the

m ethods described for primary structure if the defect or failure in the control surface can reduce the flutter margins below that re quired by the type certification basis.

b. Separation of Control Surface. An ASE should base a recommendation on airworthiness action using the methods described for primar y structure if the defect or failure may cause the control surface to separa te from the airplane and damage vital parts of the airplane or cause serious or fa tal injuries to persons on the ground.

c. Reduced Controllability. An ASE should base a recommendation on airworthiness action usin g the methods described for mechan ical system risk analysis if the defect or failure may reduce the controllab ility of the airplane in any axis or reduce the effectiveness of high lift devices.

4-10 Other Structure - Secondary Structure

a. Separation of Secondary Structure. An ASE should base a recommendation on airworthiness action using the methods described for primar y structure if the defect or failure may cause the secondary structure to separate from th e airplane and damage vital parts of the airplane or ca use serious or fatal injuries to persons on the ground,

b. Reduced Controllability. An ASE should base a recommendation on airworthiness action using the methods described for system risk analysis if the defect or failure may reduce the controllability of the airplane in any axis or reduce the effectiveness of high lift devices.

4-11 Other Structure - Landing Gear Landing gear collapse and failure of the p ilo t to lower the landing gear are common occurrences in small airplanes. The Director ate completed a detailed study (available on the SAD COS website) of gear up and collapsed gear incidents in Class I airplanes (single reciprocating engine, less than 6,000 lbs). The study showed that no serious or fatal injuries have occurred as a result of thes e incidents. In fact, th e highest hazard level resulting from these incidents was Level 3, whic h occurred about 50% of the time. At this time, the Directorate believes that ASEs should recommend only voluntary action in Class I airplanes to address defects or failures in the landing gear that will result only in gear collapse. ASEs may consider mandatory ac tion for airplanes in other classes, due to the higher landing speeds and poten tial for the airplane to be used for hire or in commercial operations.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 201051

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 4-12 Other Structure – Occupant Protection
An ASE should consid er corrective action for any defect or failur e in a design feature intended to improve survivability in accide nts. Examples would include defects or failures in safety equipment, including seats, seat belts, shoulder harn esses, or inflatable restraints, that lower the strength of the safe ty equipment below that required by static or dynamic occupant protection requirements. Additional examples include interior structure intended to retain items of mass or to separate cargo space from the passenger compartment.Appendix 1 A1-1

Small Airp lane Risk Analysis (SARA) Page 1 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations Worksheet

Make: Model/Series: Serial Numbers: Date:

ASE: Office: Telephone: Electronic Mail: Reason for Airworthiness Concern:

Frequency of Failure (Rando m or Constant Failure Rate)

Service history, total number of failures during sel ected service history time frame: (input)

Fleet exposure, selected se rvice history time frame, years: (input)

Fleet exposure, fleet size during selected service hist ory time frame: (input)

Fleet exposure, average annual usage, flight hours per airplane per year or flights per airplane per year: (input)

Frequency of failure = total number of failures during selected time frame = (calculated) selected time frame x fleet size x average annual usage

Haz ard Ratios

Service history, number of failures duri ng selected service history time frame that resulted in Level 5 outcomes: (input)

Service history, number of failures duri ng selected service history time frame that resulted in Level 4 outcomes: (includes Level 4 and Level 5 outcomes) (input)

Service history, number of failures duri ng selected service history time frame that resulted in Level 3 outcomes: (includes Level 3, 4, and 5 outcomes) (input)

Hazard Ratio, Level 5 = Number of fail ures that resulted in Level 5 outcomes = (calculated) Total number of failures during selected time frame

Hazard Ratio, Level 4 = Number of failur es that resulted in Level 4 outcomes = (calculated) Total number of failures during selected time frame

Hazard Ratio, Level 3 = Number of failur es that resulted in Level 3 outcomes = (calculated) Total number of failures during selected time frame

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 1 A1-2

Small Airp lane Risk Analysis (SARA) Page 2 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations Worksheet

Uncorrected Fleet Risk - Long-term Fleet Risk:

Number of Level 5 Outcomes = Frequency of Level 5 Outcomes x Long-term Exposure: (calculated)

Number of Level 4 Outcomes = Frequency of Level 4 Outcomes x Long-term Exposure: (calculated)

Number of Level 3 Outcomes = Frequency of Level 3 Outcomes x Long-term Exposure: (calculated) Uncorrected Individual Risk

Frequency of Level 5 Outcomes = Frequency of Failu re x Level 5 Hazard Ratio: (calculated)

Frequency of Level 4 Outcomes = Frequency of Failu re x Level 4 Hazard Ratio: (calculated)

Frequency of Level 3 Outcomes = Frequency of Failu re x Level 3 Hazard Ratio: (calculated)

Fleet Exposure (Future)

Short-term Exposure, expected number of airplanes in fleet over the next one year period: (input)

Short-term Exposure, average annual usage Flight hours per year per airplane or flights per year per airplane: (input)

Short-term Exposure = expected number of airplanes in fleet over next one year period x Average annual usage x 1 year: (calculated)

Long-term Exposure, expected numbe r of airplanes in fleet over the next 20 year period: (input)

Long-term Exposure, average annual usage Flight hours per year per airplane or flights per year per airplane: (input)

Long-term Exposure = expected number of airplanes in fleet over next 20 year period x Average annual usage x 20 years: (calculated)

Uncorrected Fleet Risk - Short-term Fleet Risk:

Number of Level 5 Outcomes = Frequency of Level 5 Outcomes x Short-term Exposure: (calculated)

Number of Level 4 Outcomes = Frequency of Level 4 Outcomes x Short-term Exposure: (calculated)

Number of Level 3 Outcomes = Frequency of Level 3 Outcomes x Short-term Exposure: (calculated)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 1 A1-3

Small Airp lane Risk Analysis (SARA) Page 3 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations Worksheet

Plot uncorrected individual ri sk against short-term and long-term exposure on the appropriate risk guidelines given on pages 4, 5, 6, and 7 of this worksheet.

The plotted information provides the ASE with a re commendation for appropriat e airworthiness action.

Comments: (input)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 1 A1-4

Small Airp lane Risk Analysis (SARA) Page 4 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations Worksheet

0.7 Outcomes

7.0

O u t co

mes

7 0 O u t co

mes

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 1 A1-5

Small Airp lane Risk Analysis (SARA) Page 5 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations WorksheetUncorrected Individual RiskOf Level 5 Outcomes(Frequency of Level 5 Outcomes)Number of Outcomes per Hour(per Flight) Event Guidelines for AC 23.1309-1D Class II Airplanes (Multi-Engine Reciprocating, Single-Engine Turbine, or Multi-engine Turbine Airplanes < 6,000 lbs.)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 Exposure, Fleet Hours(Cycles) 10 -3

10 -7 10 -6 10 -5 10 -4

10 -8 0. 5 Out

c om

e s High Risk

Reasonable Risk Excessive Risk High Risk (if more than 0.5 outcomes expected in next 1 year period)Uncorrected Individual RiskOf Level 4 Outcomes(Frequency of Level 4 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -2

10 -6 10 -5 10 -4 10 -3

10 -7 5.0 Outcomes High Risk

Reasonable Risk Excessive Risk High Risk (if more than 5.0 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)Uncorrected Individual RiskOf Level 3 Outcomes(Frequency of Level 3 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -1

10 -5 10 -4 10 -3 10 -2

10 -6 50 Outcomes High Risk

Reasonable Risk Excessive Risk High Risk (if more than 50 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 1 A1-6

Small Airp lane Risk Analysis (SARA) Page 6 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations WorksheetUncorrected Individual RiskOf Level 5 Outcomes(Frequency of Level 5 Outcomes)Number of Outcomes per Hour(per Flight) Event Guidelines for AC 23.1309-1D Class III Airplanes (Single-Engine Reciprocating, Multi-Engine Reciprocating, Single-Engine Turbine, or Multi-Engine Turbine Airplanes > 6,000 lbs)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 Exposure, Fleet Hours(Cycles) 10 -4

10 -8 10 -7 10 -6 10 -5

10 -9 0.3 Ou

t co

m es High Risk

Reasonable Risk Excessive Risk High Risk (if more than 0.3 outcomes expected in next 1 year period)Uncorrected Individual RiskOf Level 4 Outcomes(Frequency of Level 4 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -3

10 -7 10 -6 10 -5 10 -4

10 -8 3.0 Outco

m es High Risk

Reasonable Risk Excessive Risk High Risk (if more than 3.0 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)Uncorrected Individual RiskOf Level 3 Outcomes(Frequency of Level 3 Outcomes)Number of Outcomes per Hour(per Flight)

10 0 10 4 10 3 10 2 10 1 10 8 10 7 10 6 10 5 10 -1

10 -5 10 -4 10 -3 10 -2

10 -6 30 Outcomes High Risk

Reasonable Risk Excessive Risk High Risk (if more than 30 outcomes expected in next 1 year period)

Exposure, Fleet Hours(Cycles)

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 1 A1-7

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 Small Airplane Risk Analysis (SARA) Page 7 of 7 Avionics, Airplane Systems and Equipment, and Powerplant, Propeller and APU Installations Worksheet

0. 1 Out

c omes

1.0 Out
co

m es

10

O utcom

e sAppendix 2 A2-1

Small Airp lane Risk Analysis (SARA) Page 1 of 4 Airframe and Occupant Protection Worksheet

Date:

ASE: Office: Telephone: Electronic Mail: Make: Model/Series: Serial Numbers:

Reason for Airworthiness Concern:

Primary Structure – Fatigue (Wearout Failure Rate)

Material

Aluminum Steel Other: (Input)

Component

Wing Pressurized Fuselage Empennage Other: (Input)

Analysis Type

Weibayes Two-Parameter Weibull Other: (Input)

TIS Adjusted for Crack Size

Yes No

Crack Reports

Total Number of Suspensions (Input)

Total Number of Reported Cracks (Input)

Number of Level 5 cracks (Input)

Number of Level 4 cracks (Input)

Number of Level 3 cracks (Input) Analysis Parameters

Hours Flights

η = (Input)

β = (Input)

B.1 Life = (Input)

Expected Number of Cracks (Calculated) (Example Values)

Fleet Hours Expected Number of Cracks 0 5 62500 7 187500 11 312500 15 625000 30 1250000 70

Expected Number of Cracks

0 20 40 60 80

0 500000 1000000 1500000Num ber of Cracks

Fleet Hours

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 2 A2-2

Small Airp lane Risk Analysis (SARA) Page 2 of 4 Airframe and Occupant Protection Worksheet

Primary Structure – Corrosion (Ran dom or Constant Failure Rate)

Component Corrosion Reports

Wing, Outboard Time Period of Fi ndings, years: (Input) Wing, Center Section Fuselage Total Number of Corrosion Findings: (Input) Pressurized Fuselage Empennage Total Number of Level 5 Findings: (Input) Engine Compartment Other: (Input) Total Number of Level 4 Findings: (Input)

Total Number of Level 3 Findings: (Input)

Estimated Rate of Under-reporting: (Input)

Rate of Corrosion (Calculated) (Example Values)

4

1.8

2.2

3

6.7

Year

0 20 48.4 12.0 145.3 Expected Number of Corrosion Events Underreported Events

4.0 Underreported Rate of Corrosion, Events per year

Expected Number of Corrosion Events

Reported Events Number of Corrosion Reports

Time Frame, Years

Rate of Corrosion, Events per year

Underreporting Factor

Expected Number of Corrosion Events

0.0 20.0 40.0 60.0 80.0 100.0 120.0 140.0 160.0

0 5 10 15 20 YearsCorrosion Events

Reported

Underreported

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 2 A2-3

Small Airp lane Risk Analysis (SARA) Page 3 of 4 Airframe and Occupant Protection Worksheet

Primary Structure – Quality Escape (Fixed Number of Defects or Failures

Number of Effected Components: (Input)

Number of Effected Airplanes: (Input)

Severity

Level 3 Level 4 Level 5 (Example Values)

18

9

0 18 9 20 18 9 Number of Effected Components

Number of Effected Airplanes

Number of Quality Escapes

0 5 10 15 20

0 5 10 15 20 YearsNumber of Escapes

Comonents Airplanes

Primary Structure – Composites: Risk analysis for composite structures will most likely follow a constant failure rate similar to the analysis for corrosion in metals. ASEs should consult the SARA Handbook for information on risk analysis of composite primary structure.

Other Structure – Control Surfaces, Tabs, High Lift Devices, and S econdary Structure: If the hazards involved in the airworthiness concern are flutter or se paration of a surface which may damage vital parts of the airplane or cause serious or fatal injuries to persons on the ground, ASEs should select the most appropriate primary structure risk analysis method. If reduced controllability is the hazard, ASEs should use the SARA proces s for avionics, airplane systems and equipment, and powerplant, propeller, and APU installations.

Occupant Protection: ASEs should select the most appropriat e primary structure risk analysis method for any components intended to provide occupant protection. These components include seats, seat belts, shoulder harnesses, and interior structure intended to retain items of mass or to separate cargo space from the passenger compartment.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 2 A2-4

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 Small Airplane Risk Analysis (SARA) Page 4 of 4 Airframe and Occupant Protection Worksheet

Comments: (Input)Appendix 3 A3-1

ACS Completion Instructions
The Sm all Airplane Directorate has developed a template for the ACS. A representation of the template is given on the following page. The format of the template is fixed, with the user able to input information in the gr ey blocks. The user can move from block to block using either the computer mouse to clic k on each block, the ‘Tab’ key, or the arrow keys on the computer keyboard.

Block 1: Date: (Number of characters allowed: 20)

Input into Block 1 the date you complete th e ACS. The block length provides sufficient space to spell out the month, for example: ‘September 28, 2007’.

Block 2: Name: (Number of characters allowed: 35) Title: (Number of characters allowed: 35), (e.g., Aerospace Engineer, Program Manager, etc.) Office: (Number of characters allowe d: 35), (e.g.,Atlanta ACO) Department: (Number of characters allowed: 35), (e.g., ACE-118W) Street Address: (Number of characte rs allowed: 35) City, State, ZIP: (Number of characte rs allowed: 35) Telephone: (Number of characters allowed: 35) Electronic Mail: (Number of characters allowed: 35)

Complete Block 2 with your contact information.

Block 3: Make: (Number of characters allowed: 40), (e.g., Cessna) Model / Series: (Number of characters allo wed: 40), (e.g., 172 Series, or 402C ) Serial Numbers: (Number of characters allowed: 100)

Complete Block 3 with as much airplane make, model, series, and serial number information as possible. The ASE should use the information provided on the Type Certificate Data Sheet to ensure accurate and consistent identification of the make, model, series, and serial number informa tion. A complete description of the make, model, and serial numbers is often needed to determine the scope of the airworthiness concern. If available, the manufacturers’ Service Bulletin(s) usually contain this information. If the series or serial number range is unclear or unknown, then this should be explained and requested as part of the information the owners / operators may be able to provide. (This information is require d on AD worksheets, so it will be needed in the event the airworthiness concern evolves into a proposed AD action.)

Block 4: Reason for Airworthiness Concern: (Number of charac ters allowed: 220)

Complete Block 4 with a short synopsis of the airworthiness concern. For example:

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 3 A3-2

U.S. Department Of Transportation

Federal Aviation Administration 

# Airworthiness Concern Sheet

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010

Date: (Block 1)

Reply to: Make: Name: Model / Series: (Block 3) Title: Serial Numbers: Office: Department: (Block 2) Reason for Airworthiness Concern: (Block 4) Street Address: City, State, ZIP: Telephone: Electronic Mail:
Federal Aviation Administration (FAA) Description of Airw orthiness Concern

(Block 5)

Request for Information (For example: Proposed alternate inspection or repair procedures, cost impact, etc. Your comments or replies to the FAA need to be as specific as possible. Please provide specific examples to illustrate your comments or conce rns.)

(Block 6)

This Airworthiness Concern Sheet (ACS) is intended as a means for FAA Aviation Safety Engineers to coordinate airworthiness concerns with aircraft owners/operators th rough associations and type clubs. At this time, the FAA has not made a determination on what type of corrective action (if any) s hould be taken. The resolution of this airw orthiness concern could involve Airworthine ss Directive (AD) action or a Special Airworthiness Information Bu lletin (SAIB), or the FAA could determine that no action is need ed at this time. The FAA’s final determination will depend in part on the information received in response to this ACS.

The FAA endorses dissemination of this tec hnical information to all manufacturers and requests association and type club commen ts.

Attachments: (Block 7) Transmittal: (B lock 8) Response Requested Service Difficulty Report by: (Block 9) Accident/Incident Data System Federal Aviation Administration Service Letter / Bulletin Airplane Owners and Pilots Date: Special Airworthiness Information Association Bulletin Experimental Aircraft Emergency Federal Aviation Administration or Association (10 days) National Transportation Safety Board Type Club Alert Safety Recommendation Type Certificate Holder (30 days) Airworthiness Directive Other: Information Alternate Means of Compliance (90 days) Other :Appendix 3 A3-3

‘There have been several cases of m ain wheel bearing failures reported to the FAA within the past six months. These failures have resulted in wheel loss during takeoff.’

‘In the past year, the FAA has received 33 Service Difficulty Reports (SDRs) of cracks and corrosion in Cessna 190/195 magne sium aileron hinge brackets, P/N 0322709-1.’

Block 5: Federal Aviation Administration (FAA) Description of Airworthiness Concern: (Number of characters allowed: 1,200)

Complete Block 5 with a succinct, complete description of the airworthiness concern. Under no circumstances should your description of the airworthiness concern (or any other part of the ACS) state that the FAA ‘proposes to mandate’, ‘is considering an AD’, or otherwise suggest th at rulemaking is imminent. The purpose of an ACS is to collect information from the aviation community prior to making a decision on an appropriate couse of action to resolve an airworthiness c oncern. The ACS must be kept separate from the rulemaking process, and must not violate ‘Ex Parte’ requirements.

Examples:

‘On January 1, 2001, the manufacturer publ ished service bulletin SB01-01. The service bulletin recommended eddy current in spections for fatigue cracks in the lower caps of the wing main spars. The manufacturer has received service bulletin inspection reports of five instances of either cracke d or failed spars on in-service airplanes. Fortunately, the inspections found the cracked spars before a wing separation occurred.

The manufacturer has recently completed an engineering evaluati on that shows that the eddy current inspection pro cedure may not be able to always detect cracks before the cracks reach the critical crack length. The engineering evaluation also shows that the wing structure does not have fail-safe capability.

The manufacturer has developed a new se rvice bulletin, SB07-01, and a service kit, K07-01, that define a strap modification to the wing. The manufacturer believes that the strap modification is needed to assure continued airworthiness. The manufacturer estimates that 500 man-hours would be require d to install the modification. The cost of service kit, K07-01, is $2,662.’

And,

‘The manufacturer has notified the FAA that defective bushings have been installed on four models of airplanes. The defective bushings are installed on the horizontal stabilizer hinge on the vertical fin. The de fective bushings did not receive a required thermal process during manufacture of the bushings. The manufacturer’s service bulletin, SB 04-03, provides instructions on replacement of the defective bushings.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 3 A3-4

The affected airp lane serial numbers are given in the ‘Make/Model Series/ Serial Number’ block above.’

Block 6: Request for Information: (Number of characters allowed: 300)

Complete Block 6 with a request for info rmation from the aviation community. The purpose of the request is to gather informa tion that could provide additional insight or otherwise help resolve the airworthiness concern.

Examples:

‘The FAA requests information regarding this airworthiness concern. In particular, the FAA requests information and supporting data regarding an NDI procedure that can detect the described cracks before they reach their critical length, alternate modification designs, and cost information.’

‘The FAA requests your comments on the maintenance history of the parts involved. Proposals for inspection or repa ir procedures should include the estimated costs of the procedure. Also give any other co mments that you feel are necessary.’

Block 7: Attachments

Use the computer mouse to ‘check’ or ‘ un-check’ the appropriat e boxes in Block 7.

 Never include or att ach proprietary data.

 Do not include the risk analysis. The ri sk analysis may sugge st to the aviation community that FAA action is imminent, regardless of the ACS process.

 FAA Safety Recommendations are for FAA internal use only. However, after consultation with the Office of A ccident Investigation, AAI-200, you may share selected information with the public. NTSB Safety Recommendations are a matter of public record and you may share them with the public in their entirety.

Block 8: Notification

Use the computer mouse to ‘check’ or ‘un- check’ the appropriate boxes in Block 8. We have provided an ‘Other’ check box in Bloc k 8 that allows you to input (up to 80 characters) a list of other organizations you have transmitted the ACS to.

 Always notify the FAA’s Small Airplane Directorate Continued Operational Safety Branch, ACE-113.

 Always notify the type certificate holder.

 Always notify the Airplane Owners and Pilots Association (AOPA). AOPA is responsible for disseminating the informa tion to the appropriate user groups.

 Notify the Experimental Aircraft Associ ation (EAA), if the concern involves an experimental or amateur built airplane.

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010Appendix 3 A3-5

Small Airplane Risk Analysis (S ARA) Handbook Revision: Original Date: September 30, 2010 The Aircraft Owners and Pilots Association (AOPA) has agreed to forward the ACS to the appropriate type clubs or organizations. Therefore, the ASE will send AOPA the completed ACS. ASEs should send the doc ument to AOPA electronically, providing AOPA with the most efficient means for forwarding the document to type clubs and expediting the data gathering. The type cl ubs will respond to AOPA with technical and economic information pertinent to the airwor thiness concern. AOPA will consolidate the feedback into one response.

Although AOPA is the primary conduit to th e type clubs, this should not preclude sending the ACS directly to type clubs. If communication between the FAA and a type club are strong, notifying the t ype club directly, in addition to AOPA, can expedite the data gathering process.

Block 9: Response Requested by: (Number of characters allowed: 20)

Complete Block 9 by inputting a date (based on calendar days from the date shown in Block 1 of the ACS) by which responses to the ACS should be r eceived. Based on the initial risk analysis, your choices of ‘Response Requested by’ dates include:

 ‘Emergency (10 days or less)’. Use of this date indicates an urgent safety concern. The airworthiness concern involves potential catastrophic failure / loss of life. Expect minimal owner / ope rator response by th e requested date.

 ‘Alert (30 days)’. Use of this date indi cates a high level of concern. Encourage associations and type clubs to use electronic and facsimile media to expedite the response process. Expect fewer owner / operator responses by the requested date.

 ‘Information (90 days)’. Use of this da te indicates a ‘non-emergency’ concern. This date allows associations and type clubs time to utilize print media, mass mailings, etc, maximizing the number and quality of owner / operator
comments.