An Assessment of the Effectiveness of
Public Law 111

- 216 in Reducing Accident Risk 1

Office of Accident Investigation and Prevention
Federal Aviation Administration
November 9 , 2012

1 The original assessment was completed by the Office of Accident Investigation and Prevention, Federal Aviation Administration, November 22, 2010.Assessment of PL 111

- 216

2

This document presents the results of a safety analysis of prop osed rulemaking by the Federal Aviation Administration (FAA) on Pilot Certification and Qualification Requirements for Air Carrier Operations, pursuant to Public Law 111

- 216, The Fiscal Year 2010 Federal Aviation
Administration Extension Act . Section 209( b) of the Act mandates that FAA convene a panel of experts to establi sh the best methods and required training time, plus recurrent training requirements, to ensure that flight crewmembers of Part 1 21 air carriers and Part 1 35 air carriers

“ master aircraft systems, maneuvers, procedures, takeoffs and landings, and crew coordination .“

Section 216 of the Act directs FAA to undertake rulemaking to ensure that Part 1 21 air carriers

“ develop and implement means and methods for ensuring that flight crewmembers h ave proper qualifications and experience. ” Section 216 a lso requires the rulemaking to ensure at least the following.

Comprehensive pre

- employment screening of prospective flight crewmembers

, “including an assessment of the skills, aptitudes, airmanship, and suitability of each applicant” to function effectively in an air carr ier environme nt.

All flight crewmembers must obtain an airline transport pilot (ATP) certificate and have “appropriate multi

- engine aircraft experience.”

Section 217 of the Act r equires rulemaking to amend minimal requirements under P art 61 of the Federal Aviation Regulations for obtaining an ATP to ensure that a pilot can “function effectively in an air carrier operational environment. Amendments to Part 61 must ensure that appl icants for an ATP have received flight training, academic training, or operational experience that effectively prepares them for a multi

- pilot environment, adverse weather conditions, including icing conditions, high altitude operations, and adherence “to the highest professional standards.” Finally, Section 217 retains the required minimum of 1,500 hours before a pilot can qualify for an ATP, including “sufficient flight hours” in difficult operating conditions.

Based on a detailed review of 417 acciden ts over 10 years, the analysis found that the rulemaking would be most effective against accidents involving turbojets in on

- demand Part 1 35 operations and against twin

- engine aircraft accidents in Part 1 35 commuter operations. The rulemaking would have a significant effect against Part 1 21 accidents, but less so than against

Part 1 35 accidents . The rulemaking would have had little to no effect on the small number of

Part 1 25 accidents that occurred over the 10

- year study period.



### Anticipated Rulemaki ng

An ATP license is currently required to operate as a pilot

- in

- command (PIC) during Part 121 operations. The Act and the rulemaking would retain the minimum of 1,500 hours to qualify for an ATP. In addition, a s directed by the Act , the rulemaking will require an ATP and an aircraft type rating for all pilots in Part 1 21 operations, and will require that at least 50 of the minimum 1,500 hours to qualify for an ATP be in multi

- engine aircraft. The rulemaking also will include a restricted

- privileges ATP which allows credit for appropriate academic and classroom training and credit for military flight experience. For example, the rule may require 1,000 hours total time as a pilot for a graduate of an accredited aviation degree program, or a minimum of 75 0Assessment of PL 111

- 216

3

hours total time for a military pilot. The restricted

- privileges ATP also would have to be a minimum age o f 21.

The rule also will require time in a flight simulation training device to obtain an ATP as part of the effort to ensure “sufficient flight hours” in difficult operati ng conditions and to incorporate the additional experience requirements defined in Section 217 of the Act.

In addition, the proposed rule will require either a straightforward minimum of 2,500 hours of pilot experience to qua lify as a PIC in Part 1 21 operations, or the rule may alter that by requiring
that a prospective PIC has at least 1,000 hours of experience in air carrier operations , plus the 1,500

- hour minimum to qualify for an ATP

. Though the net effect of these 2 varia tions on PIC eligibility is not exactly synonymous, the effect on accident reduction is essentially the same.



### Safety Analysis

Analytical Approach

The analysis assesses the likely capacity of the proposed rules to reduce the total number of
accidents as well as fatal accidents in the future. It will address all operations in which the PIC

must have an ATP or in which the rulemak ing will require an ATP of the f irst officer (FO)

. Therefore the analysis will address all Part 1 21 and Part 1 25 operations; al l operations involving
twin

- engine aircraft as Part 1 35 commuters; and all on

- demand Part 1 35 or Part 91K operations involving turbojets or other airplanes configured with 10 or more seats. The analy sis i s based on

a ll accidents that meet the above criter ia from fiscal year (FY) 2001 through and including FY
2010 as follows:

Part 1 21 Operations 354 accidents and 370 aircraft

Part 1 25 Operations 5 accidents and 5 aircraft

Part 1 35 Commuter 1 4 accidents and 1 4 aircraft

Part 1 35 On

- Demand 44 accidents and 44 aircraft

The preferred approach would be to assess each element of a ny rulemaking as a discrete item. However, the statutory directions include elements that are difficult to isolate. Where possible the elements are addressed separately, but they ar e not assigned discrete scores. Instead score s are assigned for the entire rulemaking as a single proposal.

Scores represent the likelihood that the propos ed rules could have prevented the accident had the rules been in place at the time of each accide nt as a result of the rule’s capacity to have precluded
or corrected a contributing condition. The rule was first scored against each accident qualitatively with the following categories of effectiveness : none; low; moderate; moderate

- to

high; high; and v ery high. Those categories correspond to ranges in the following order: 0%

, 15%

, 35 % , 55 %, 75 % and 90%. The end result was a quantified set of benefits in which, for example, a score of 35 % against an accident would have reduced the risk by a theoretical

. 35
fatal accidents and a theoretical 35 fatalities. All scores were reviewed by a 3

- person panel.Assessment of PL 111

- 216

4

Note that the mere presence of some c ondition such as a pilot with less than the proposed flight
hours did not necessarily mean that the rulemaking would ha ve been effective against a particular accident scenario. Many accidents simply d o not reflect at all on pilot performance

. Where a pertinent issue was influential in an accident, its score depended on an assessment both of the degree to which the factor influenced the accident and the degree to which a new rule realistically could be expected to have changed the outcome of a particular accident, based on the details of each accident flight. Finally, as a matter of analytical principle, no accident r ecei ved a score higher than 0.9 based on the assumption that we can never be certain that any intervention
would eliminate all risk in a particular scenario . With these caveats, the analysis focused on the following factors:

Part 1 21 PIC with less than 2

, 500 hours

Part 1 21 FO with less than 1

, 500 h ours, 1

, 000 hours, and 750 hours

Part 1 21 FO operating without an ATP

Part 1 21 FO operating without a rat ing in the aircraft being flown Crew Resource Management
Sterile Co ckpit Procedures
Professionalism Weather ( thunderstorm, icing, turbulence, gusty crosswinds etc.)
Stall

- upset or unusual attitude

Part 1 21 operators had 354 accidents involving 370 Part 1 21 aircraft in the 10

- year study period. The majority of the 354 Part 1 21 accidents involved low

- level outcom es or isolated risk that did

not a ffect most aircraft occupants. For example, turbulence accounted for 102 of the 354 accidents, all but a few of which involved single injuries, while gate

- ramp accidents accounted
for 88 accidents. M ost of those accident s did not reflect on crew performance. Similarly,

m iscellaneous on

- board events and deer strikes on runways accounted for another 10 accidents, and 12 more events qualified as accidents only because an injury oc curred during a precautionary
evacuation. T hese types of events account for 60 percent of the total accident set for Part 1 21 operators, while still others involved maintenance and other issues not related to crew performan ce. Many of the remainder involved relatively low

- level outcomes, though wi th varying levels of risk , while others involved high

- time PICs with high

- time FOs with ATP ratings who likely would not be reached by this rulemaking. The point here is not to dismiss these events but to suggest that benefits, as measured by avoided acci dents, fatalities, and serious injuries, often will be more modest than advocates might a nticipate

.

Data Sources and Data Quality

Accident reports and structured accident data from the National Transportation Safety Board (NTSB) w ere the primary data sou rce, supplemented where necessary and possible with data from FAA. Accident data will be limited to 10 years due to the poor quality of data available in
either NTSB o r FAA reports on pilot experience for earlier accidents . FY 2001 was selected as
the st arting point because that was the point at which NTSB data on pilot flight hours began to
capture co

- pilots at least somewhat systematically.Assessment of PL 111

- 216

5

However, several issues limit the applicability of some accidents even within this 10

- year dataset. In the cas e of accidents involving U.S. operators in foreign countries, foreign governments do not always release the factual information, probable cause statement, and contributing factors. I nf

ormation that is available through the NTSB for these accidents rarely

i nclude pilot data

. Pilot data is also limited for domestic accidents that ha ve relatively low

- level outcomes, such as turbulence or ramp

- gate accidents. In addition, other records randomly
involve blank data cells for PIC or FO flight hours and FO certif icates and type ratings

.

Where any of these problems existed, the text of each NTSB accident report w as reviewed, as

were files in the FAA’s Office of Accident Investiga tion and Prevention. That exercise added or corrected data for some or all pertinen t fields for more than 2 0 0 pilots , but some or all pertinent
data on some pilots and causal factors is simply not available.

U seful flight

- hour data was identified for 33 2 PICs among the 370 acc ident aircraft. All but 4 of
the 33 2 PICs had more than 2,50 0 hours. Of the other 38 PICs, damage to 2 aircraft was found during maintenance, and the respective air carriers could not associate either accident with specific crews

. Consistent with comments above about the frequent absence of flight

- hour data
in c ertain categories of accidents, 24 of the remaining 3 6 accidents for which PIC flight

- hour data was not available did not reflect on pilot performance. Those 24 accidents involved maintenance issues, ramp events caused by ground staff, and turbulence inju ries in clear air turbulence or after adequate warnings were ignored. Eight of the other 12 accidents involved large jets operating international flights and 3 involved large jets on domestic flights by major air carriers. Most or all of those captains w ould have had significant flight experience. Consequently, the absence of data on those PICs does not affect the findings in this report.

Useful d ata on flight hours or ratings was available for 326 FOs (8 8 %) . The 45 cases for which
no data was availabl e included the s ame 38 cases noted above. Consequently, again, the absence of data in the 45 records will not affect the findings of this report.



### Results of the Analysis, Part 1 21 Operations

All but 4 PICs exceeded the straightforward 2,500

- hour minimum

. The alternate approach of requiring an ATP plus at least 1,000 hours of experience in air carrier operations likely would have affected the eligibility of a fifth PIC , whose total flight time marginally exceeded 2,500
hours. All 5 of those PICs operate d with FOs who had commercial ratings rather than ATPs. Crew performance was a primary issue in 4 of those 5 accidents. The fifth accident was caused by a tug operator who struck a stationary regional jet at the gate. Though the rule would have
affected the eligibility of both pilots to operate that flight, the rule would not have influenced the
accident.

The proposed PIC minim um hours and the required ATP certificate to operate as an FO in Part 121 operations would have affected the eligibility of 83 accident crews in the 10

- year accident set. However, crew performance was an issue in just 34 of those cases, and the outcome of some of those 34 would not have been affected by this rulemaking. For example, 2 fatal accidents in
which FOs did not have A TP ratings illustrate how the rule could have affected the eligibility of the FOs without influencing the outcome: Air Midwest Flight 5481 (21 fatal) crashedAssessment of PL 111

- 216

6

immediately after lift off from Charlotte, NC due to an improperly

- rigged elevator; and Chalks

Oce an Airways Flight 101, (20 fatal) crashed into the Atlantic Ocean when the wing failed in normal flight. Though both FO s would have been affected by the requirement to have an ATP,
the rule would not have prevented the accidents, and they would have score d a “ zero

. ”

Conversely, some of the other elements in the rulemaking , such as screening pilot applicants

, preparing pilots better for the complex environment of air carrier operations, better training in stall recognition, and the importance of profess ionalism w ould have had some minimal chance of influencing additional accident outcomes.

The rulemaking was found to be effective in varying degrees against just 31 acciden ts , and in
most cases the scores were “low” to “moderate

- to

- high ” categories. Th ere were no accidents
that received scores of “very high. ” Appendix 1 shows the scoring for each of the 3 1 Part 121
accidents against which the rulemaking received positive scores

. Table 1, summarizes the scoring of the complete rulemaking against all 354 Part 121 accidents from FY 2001 through FY

2010.

Table 1: Scoring Summary in Part 1 21 Operations FY 2001 Through FY 2010

Qualitative
Effectiveness Score Quantified

Score Number of

Accidents Accidents

Avoided

None 0 323 0

Low 0.15 17 2.55

Moderate 0. 35 6 2.1

Moderate

- to

- High 0.55 7 3.85
High 0.75 1 0.75
Very High 0.90 0 0

Total 354 9.25

In the accidents that received a “high” score (75% reduction in the likelihood of the accident) or
“moderate

- to

- high” (55%)

, crew performance was a significant role in the accident

. Th ese accident s are summarized below :

High (75%)

On October 16, 2001

, an Embraer ERJ

- 145 operated by Mesa Airlines landed
hard in Roanoke , VA . The PIC briefed a visual approach to runway 33 and
indicated to the FO that a go

- aro und was not an option due to surrounding terrain

. The crew n oted a quartering cross wind at 15 mph gusting to 21 mph . The FO said
the approach was normal until 300 above the ground, whe re he called "one dot high" and Vr ef +5. Airspeed dropped abruptly, the stick shaker activated

, and the PIC added power

, but the aircraft landed hard before the engines could spool up. Damage included broken and cracked frames and stringers, popped rivets, and skin worn through in the lower aft pressure vessel, plus scraped s kin on the lowerAssessment of PL 111

- 216

7

aft fuselage. The NTSB identified the probable cause as the PIC 's failure to maintain airspeed. Factors included the failure of both pilots to follow company

CRM and flight manual procedures and the PIC's improper approach briefing.

Both pilots were hired during a period of rapid expansion for the carrier and moved through what the NTSB report described as a “rushed’ upgrade program.

The PIC had 2

, 548 hours with 200 in make

- model. The commercially

- rated FO had 1, 850 hours with 90 hours in make

- model

. The PIC was hired 21 months prior to the accident and had been flying as an active flight instructor

. The PIC
had accrued 600 hours as an FO in the BE

- 1900 and qualified for upgrade to PIC

in the ERJ based on seniority. The FO was hired wi th 400 hours, flew as an FO on
the Beech 1900

, then upgraded to the ERJ as an FO based o n seniority.

Neither pilot had any prior experience in swept

- wing aircraft nor had the carrier provided any jet indoctrination or training on swept wing aerodynamics. The rule’s emphasis on screening pilot applicants, professionalism, preparing pilots better for Part 121 operating environments, and more training in stall recognition
would have likely prevented this accident.

Moderate

- to

- High (55%)

1. On August 13, 2 004, a Convair 580 cargo flight operated by Air Tahoma
crashed on approach in to Cincinnati , OH

. The FO was killed, the PIC was seriously injured, a nd the aircraft was destroyed. During the flight from Memphis to Cincinnati, the FO was the flying pilot. Ap proximately 20 minutes before the accident, he noticed that there was a problem with the handling characteri stics of the airplane. In a six

- minute span, he commented five times that he was experiencing abnormal forces on the control yoke. He said, “Feels l ike I need a lot of force. It is pushing to the right for some reason. I don't know why...I don't know what's going on .” With the runway in sight, the FO again noted “What is wrong with this plane? It ’ s really funny. I got something all messed up here."

The PIC, who was preoccupied for most of the flight with reconciling his pre

flight weight and balance miscalculations, eventually advised the FO that they had a fuel imbalance because he, as the non

- flying pilot, had erroneously left the

cross

- feed open. This oversight caused a prolonged and unintended fuel transfer that was prohibited in the Aircraft Flight Manual (AFM) and the company Flight Operations Manual and was placarded in the cockpit. The FO then stated that,

"W e're going to flame out

. " However, the PIC responded, "I got the cross

- feed

open. Just keep the power on." Despite having adequate total fuel on board, the aircraft lost power due to fuel starvation and crashed one mile short of the

runway.

The PIC had 2,500 hours of flight time (not incl uding the hours accumulated from 20 years of experience as a flight engineer on the Lockheed Electra, Douglas DC

8, and Lockheed L

- 1011). The commercially rated FO had 2,488 total flightAssessment of PL 111

- 216

8

hours, which included 1,564 hours as a Boeing 727 flight engineer an d 924 hours as either PIC or FO of mostly single

- engine airplanes. He had 3 months of experience as an air carrier pilot. The rule would have required more experience and seasoning for both pilots, especially for the FO , which might have enabled
him to ide ntify that a rather large fuel imbalance between the left and right wing
fuel tanks was the cause of his lateral control problem. Additional experience may have also led the PIC to reconcile his weight and balance prior to departure, complete his checklis ts at the times prescribed in the operations manual, and respond with more urgency when he finally recognized the fuel imbalance.

2. On July 6, 2008 a McDonnell

- Douglas (Boeing) DC

- 9, operated by USA Jet on a cargo flight, crashed in heavy fog while on a pproach in mountainous terrain to Saltillo, Mexico. The captain, who was the flying pilot, was killed, and the first officer sustained serious injuries. The flight data recorder shows the aircraft alternating above and below the glide slope and left and ri ght of the localizer throughout the approach. The aircraft ultimately stalled and crashed 1,000 feet short and 500 feet right of the extended runway centerline.

The captain had 6,630 total flight hours and 2,111 in the DC

- 9. The FO on that flight was an experienced flight engineer but had just 830 flight hours and 75 hours as a part 121 pilot. The deficiencies in piloting skill, decision

- making, CRM, and operations in a multi

- crew environment that contributed to this accident are addressed in the propose d rulemaking. Had the first officer been more experienced and better trained, he would have been more likely to have intervened or called for the go around as the approach became increasingly unstable. Either form of intervention would have possibly averte d this accident.

3. On February 10, 2001, a Beechcraft BE

- 1900D operated by Great Lakes Aviation landed unintentionally gear

- up on runway 4R at Chicago’s O’Hare International Airport. Seven of the 15 passengers and two pilots received minor injuries, an d the aircraft was substantially damaged. According to the PIC, the crew was executing the “Flaps

- Up Landing Checklist” abnormal procedure because the wing flap system was inoperative. Neither the PIC nor the FO reported hearing the landing gear warning ho rn , and the FO indicated that, “No gear horn sounded nor did we ever think the landing gear were not down.”

When the aircraft was craned off the runway and electrical power applied, the landing gear warning horn sounded, a red light illuminated in t he gear handle, and all three landing gear position lights illuminated red. The NTSB determined that the probable cause of the accident was the flight crew not lowering the landing gear or verifying its position with three separate checklists. Contributing to the accident was the flight crew’s failure to comply with the minimum equipment list placard that prohibited the silencing of the landing gear warning horn.

The PIC held an ATP certificate with 2,400 flight hours with 600 hours in the B E

1900D, 580 of which were PIC. The FO held a commercial license with 1449Assessment of PL 111

- 216

9

flight hours, 191 of which were SIC. With the additional experience required by the proposed rulemaking and the emphasis on crew coordination, leadership, and pilot professionalism, it is possible that the flight crew would have been more vigilant in their completion of checklists especially when flying with inoperative equipment.

4

. On June 20, 2007, a Beech craft BE

- 1900D operated by Great Lakes Aviation

l anded long and bounced in visual conditi ons at Laramie , WY . The crew tried to
slow the aircraft but turned onto a taxiway at high speed , where the right prop eller
struck an electrical box . A p rop eller blade then broke off and struck the right side
of the fuselage. The PIC later noted that the approach was about 14 knots above the targeted speed

, and the airplane was one dot high on the glide slope. No injuries occurred to the 2 pilots and 9 passengers, but the aircraft was substantially damaged. The NTSB determined the probable cause of the accident was t he
PIC’s improper decision making

, his misjudgment of speed and distance, and his
failure to go around during an unstable approach. Contributing to the accident

w as improper crew coordination

, a nd the FO 's failure to intervene during the

uns table approach and landing

.

The PIC had 2,250 hours of pilot experience and the commercially

- rated FO had 774 hours. The rule would have required more experience and seasoning for both pilots in an air carrier environment which may well have led the PIC to go around upon recognizing an unstable approach or might have compelled the FO to
intervene.

5. On January 6, 2003, an Embraer EMB

- 145LR operated by ExpressJet Airlines was substantially damaged when it overran the end of a snow

- covered runway during landing in Cleveland, OH. The captain, who was the flying pilot, allowed the aircraft to float excessively and was unable to stop the aircraft on the remaining runway after touchdown. There were no reports of injuries to the crew, the flight attendant, or the passengers.

The flight left the departure airport with a computerized flight plan, weather, and the Notices to Airman (NOTAMs) for their route of flight. While being vectored for the approach, the controller informed the flight crew that the runway visual range (RVR) was had decreased from 4,500 feet to 4,000 feet, which was the minimum listed on the approach plate used by the flight crew. However, the FAA had released a NOTAM a month prior which had raised the landing minimums to 5,000 feet. This N OTAM was not received by the airline from their weather service provider and was consequently not issued to the flight crew. The approach controller was in possession of this NOTAM, but was not required to inform the flight crew that the approach was below landing minimums.

The flight crew conducted an instrument approach briefing, but did not discuss the runway conditions or the amount of runway required to land (nor was it required). They were, however, in possession of a local NOTAM that describedAssessment of PL 111

- 216

10

the landing runway as “covered with thin wet snow.” Landing distance charts

indicate that EMB

- 145 flight crews should add 20 percent to the normal landing distance when using a wet runway. The captain reported that he used a rule of thumb computation to deter mine the landing distance because he did not have enough time to ask the first officer to compute it.

The NTSB determined the probable cause of the accident was the captain's failure to attain a proper touchdown on runway, and his subsequent failure to p erform a

go

- around, both of which resulted in a runway overrun. Factors were the company's inadequate dispatch procedures with their failure to provide current NOTAMs, and the snow covered runway.

The captain held an Airline Transport Pilot certificate wi th 3,765 hours and 661 in
the EMB

- 145. The first officer held a commercial pilot certificate with 2,550 hours and 804 in make and model. While there were several contributing factors to this accident beyond the flight crew’s control, they would have benefi tted from enhanced CRM and a better understanding of transport airplane performance (specifically contaminated runway considerations for takeoff and landing). Had they been exposed to the rigors of the training outlined in the proposed rulemaking they woul d have been more likely to consider all factors in their approach planning and more likely to call for and execute a go around.

6 . On August 14, 2008, the tail section of an Embraer EMB

- 145LR operated by Trans States Airlines struck the runway during lan ding in St. Louis , MO . There were no reported injuries to the crew or the passengers , and the aircraft was substantially damaged

.

According to the accident report, the first officer indicated during the en

- route portion of the flight that his landings wi th the flaps set at 45

- degrees were, “a little
extreme...a little non

- standard, but way more fun.” As the aircraft entered the landing flare, approximately 34 feet above the touchdown zone, the flight data recorder indicated that the pitch angle during the l anding flare increased to 13 degrees and the angle of attac k to 17 degrees. Moments later the stick shaker/stall warning system activated at which time the ta il section impacted the runway.

The NTSB determined that the probable cause of the accident was the FO ’s excessive angle of attack during the landing flare resulting in abnormal runway contact. Both pilots held ATP certificates. The captain had 13,000 total hours and 4,000 in the EMB

- 145. The first officer had 1,800 total hours and 1,000 in type.

Des pite his certification and hours, the first officer’s statements reflect a lack of understanding of aerodynamics, stalls, and professionalism that is expected of air carrier pilots. His lack of flying skills and professionalism could have been offset by a captain who heeded the warnings and exercised the kind of enhanced leadership and CRM skills outlined in the proposed rulemaking. This additional training and experience would have increased the performance of both the captain
and the first officer and mig ht have helped to prevent this accident.Assessment of PL 111

- 216

11

7. On November 20, 2000, an ATR

- 72 operated by Atlantic Southeast Airlines (ASA) Inc., encountered moderate turbulence while maneuvering for landing at 10,000 feet above mean sea level. The fasten seat belt sign w as “ON,” and the two flight attendants were preparing the cabin for landing. During the encounter, one of the two flight attendants was seriously injured. No other injuries were reported by the flight crew, the other flight attendant, or the 36 passengers on board.

Prior to the flight, hazardous weather advisories (SIGMET and AIRMET) were issued calling for large areas of moderate turbulence along the route of flight. The captain was in possession of these weather reports and was required by company poli cy to brief the entire flight crew on the expected weather along the route and any forecasts of turbulence. According to the accident report, however, the captain briefed only one of the two flight attendants at which time he related that,
“it would be a l ittle rough getting out of Atlanta.”

The captain held an ATP certificate with 4,234 flight hours and 2,510 in the ATR

72, and the first officer held a commercial pilot’s license. The NTSB determined that the probable cause of this accident was an in fli ght encounter with forecasted turbulence which resulted in serious injury to a flight attendant. Contributing to the accident was the captain’s inadequate briefing to the cabin crew, which was required by the airline’s Standard Practice manual. Had the cap tain and the first officer been exposed to the enhanced meteorology, CRM, and leadership training of the proposed rulemaking, they likely would have taken the forecasted turbulence listed in the hazardous weather reports more seriously and ensured that
bot h flight attendants were briefed accordingly prior to the flight. These actions could have prevented the injuries sustained by the flight attendant.

The remaining 23 accidents in the 10

- year accident set against which the rulemaking scored positively we re limited to “low” and “ moderate ” scores. They included 2 fatal accidents (Colgan

in Buffalo , NY and Corporate Airlines in Kirksville

, MO ), plus 3 seriou s injuries and hull losses

to 3 turboprops and 1 cargo jet. The two fatal accidents received a score of “moderate” (35%

reduction in the likelihood of the accident) and ar e summarized below:

1. On February 12, 2009, a Bombardier DHC

- 8

- 400, operated by Colgan Air,

Inc

., was on an instrument approach to Buffalo

- Niagara International Airport, New York, when it crashed into a residence about five miles short of the runway
after the PIC lost control of the aircraft killing the PIC, FO , two flight attendants,
and all 45 passengers. O ne person on the ground was also killed

, and the airplane was destroyed.

The NTSB determined that the probable cause of the accident was the PIC’s “inappropriate response to activation of the stick shaker, which led to an aerodynamic stall from which the airplane did not recover.” The PIC’s response was inappropriate because he pul led back on the control column rather than pushing it forward to reduce the angle of attack . As a result, the airplane’s pitchAssessment of PL 111

- 216

12

increased and its airspeed decreased resulting in the stall. One contributing factor was both pilots’ failure to monitor airspeed via their primary flight displays (PFDs)

, and thus their failure to recognize the impending stick shaker onset as airspeed fell and pitch increased. The NTSB noted that the “failure of both pilots to detect this situation was the result of a significant b reakdown in their monitoring responsibilities and workload management.” The PIC’s poor response suggests he was surprised by stick shaker activation . Another contributing factor was the crew’s lack of professionalism in failing to follow FAA regulations fo r sterile cockpit rules below 10 ,000 feet.

The crash might have been prevented by this rule’s emphasis on foundational training in pilot monitoring responsibilities, in particular, as well as foundational training in CRM, stall recognition and recovery, m ulti

- crew operations, and professionalism.

2. On October 19, 2004, a BAE Systems BAE

- J3201, operated by Corporate
Airlines struck trees on final approach and crashed short of the runway in Kirksville, Missouri. The PIC

, SIC , and 11 of the 13 passengers we re killed. The two surviving passengers received serious injuries

, and the aircraft was destroyed.

The airplane struck trees short of the runway threshold after the flight crew
continued an unst ab ilized approach below the m inimum d escent a ltitude (MDA)
wi thout adequate visual reference of the runway environment . The crew failed to follow FAA regulations for non

- precision approach procedures and failed to implement proper division of duties between the flying pilot and the monitoring pilot . The NTSB determi ned from the cockpit voice recorder (CVR) that b oth pilots were looking outside the cockpit window for the runway when they reached the MDA, which meant that no one was monitoring the airplane’s altitude. It was also determined that the crew engaged in unp rofessional, non

- essential conversation during the approach, which was a violation of regulations that prohibit such activity during flight below 1 0,000 feet

.

The NTSB determined that the probable cause of the accident was the pilots’

failure to follow e stablished procedures, properly conduct a non

- precision approach, and their failure to adhere to established division of duties between the flying and non

- flying pilot. Contributing factors in the accident were: t he pilots ’
failure to make standard callout s ; failure to establish a professional demeanor and adhere to sterile cockpit rules ; and fatigue.

The captain held an ATP certificate with 4,234 total flight hours and 2,510 in the

BAE

- J3201. The co

- pilot held a commercial pilot certificate with 2,856 tot al flight hours and 107 hours in the BAE

- J3201. Had the crew benefitted from the rulemaking’s emphasis on foundational training in multi

- crew operations CRM, energy management, knowledge

- based decision

- making, leadership, and professionalism, it is possibl e that the accident might have been prevented.Assessment of PL 111

- 216

13

In sum, the rulemaking would require a minimum of 2 ,500 hours for Part 1 21 captains , an ATP for Part 121 FOs, and an emphasis on screening pilot applicants, preparing pilots better for Part 121 operating env ironments, professionalism, and more training in stall recognition. These provisions would have had some degree of influence on 31 of the 354 accidents in the Part 121 data set (8.8%), but likely would have reduced the risk of accidents by a theoretical s core of just

9. 2 5 accidents among 354 accidents over 10 years (2. 6 %). The rulemaking also would have
averted a theoretical 23. 1 fatalities (5. 3 %), 3. 7 serious injuries ( 1.9 %), and 11 minor injuries ( 2.9%) over the 10

- year accident set.

Part 1 2 5 Operation s

.

A PIC on a Part 1 25 flight must have an ATP. The rulemaking would not require FO s to have an

ATP. Therefore, the net effect of the rulemaking would be limited to the various emphasis areas that the rule specifies.

Part 1 25 operators had 5 accidents in the 10

- year study period. They resulted in 4 hull losses, 1 fatality and 1 serious injury. All 5 captains had total flight time in excess of the proposed 1 ,500

- hour minimum to obtain an ATP. Fo u r of the 5 captains had more than 9,000 hours.

T wo of t he four hull losses were caused by uncontrollable engine fires in flight , and a third was caused by an engine failure that prevented the prop from feathering. One of these accidents led to an emergency landing and post

- crash fire on snow

- covered tundra in Alaska, while the other
two led to int entional ditching. In the fourth hull

- loss accident, the crew undershot the runway while landing to deliver fuel at a remote mining site in Alaska. The DC

- 4 had been modified
with a raised cockpit above fuselage to accommodate an upward swinging , modified nose door

, which “presented a different sight picture” than a normal DC

- 4. The FO had only a commercial certificate but the captain, who was the pilot flying, had nearly 22,000 hours and 13,600 in make

model. T he r ulemaking would no t have influenced these scenario s

.

The fifth accident involved wing

- to

- wing contact with a parked aircraft on a general aviation

ramp in Newark , NJ . The NTSB cited the Part 1 25 crew’s visual misjudgment of clearing
distance between airc raft. The accident report does not identify which pilot was flying the Part

1 25 Boeing 737

- 700, but t he PIC had nearly 13,000 hours and 1,245 in type, and the FO held an

ATP. The rulemaking is unlikely to affect visual perception of wing clearance. In s um, the proposed rulemaking would have had no effect on the 5 Part 1 25 accidents over the past 10

years.

Part 1 35 Commuters.

For Part 1 35 commuters, the rulemaking would affect only the requirements to obtain an ATP for pilots

- in

- command of multi

- engine airplanes. Part 1 35 commuters had 1 4 accidents involving
multi

- engine airplanes over the 10

- year study period. They resulted in 1 fatal accident with 2 fatalities, plus 4 hull losses. Just 2 of the 1 4 crews included FO ’ s, one of whom was an ATP.
Total flight hours for these PIC s ranged from 2,942 to nearly 25,000 hours.Assessment of PL 111

- 216

14

Of the 14 accidents in the past 10

- year dataset involving multi

- engine commuter aircraft , 3 were caused by faulty maintenance and a fourth accident was caused by unforecast and unexpect ed turbulence, which was the only accident in which an FO was flying. Among the remaining 1 0 accidents, the broader elements in the rulemaking may have influenced 8 accidents, which included 1 hull loss and 7 aircraft with substantial damage. However, th e rule scored highly against just one accident. The emphasis on professionalism, particularly the need to complete checklists and to follow procedures, plus the emphasis on weather were the primary sources of
effectiveness in these accidents.

Overall, th e rulemaking would have averted a theoretical 1.8 of the 14 accidents in 10 years. It
also would have avoided a theoretical 0.35 fatali ties and 0.45 serious injuries

. Appendix 2 shows the scoring for the 8 commuter accidents against which the rulemaking received a positive score.

Part 1 35 On

- Demand

In principle, the rulemaking would affect all PICs operating on

- demand Part 135 services involving either turbojet airplanes or airplanes configured with 10 or more passenger seats. However, the latte r category was not involved in any on

- demand Part 1 35 accidents in the 10

- year study period. Consequently, the on

- demand accident set was limited to 41 of 44 accidents involving turbojet airplanes. (Three of the accidents occurred outside the United State s and

a ccident data was unavailable. Useful data was available on 38 of the 4 1 PIC s and 37 of the 4 1
FOs.

As with other segments of commercial aviation outlined above, many a ccidents did not reflect on

crew performance . The rulemaking received positive scores against 19 of the 44 accidents

. There were 8 accidents that received a score of “moderate” (35%) and 11 accidents that received a score of “low” (15%). S cores were mostly a result of the targeted emphasis on professionalism, weather, and the value of recognizing go

- around gates during unstable approaches (airmanship). The most significant accident to receive a “moderate” score involved an approach

- to

- landing accident in Aspen, CO. The accident is summarized below:

On March 29, 2001, N303GA, a Gulfs tream III operated as a part 135 on

- demand flight by the Avjet Corporation, crashed while on final approach into Asp en, CO

. The airplane crashed 2,400 feet short of the runway threshold after the flight crew
continued an unst ab ilized approach below the m in imum d escent a ltitude without having adequate visual reference to the runway

. The two pilots

, one flight attendant, and all 15 passengers were killed

, and the aircraft was destroyed.

T he flight was late departing Los Angeles, CA and was in jeopardy of mi ssing the
noise restriction curfew that was in effect at the time of the accident. The weather
brief which the co

- pilot received during preflight as well as the weather obtained

en

- route indicated that a visual approach would likely be available at the
des tination. When t he aircraft neared Aspen, however, ATC informed the flight
crew that three of the preceding arrivals had executed missed approaches due to deteriorating visibility at the end of the runway. Despite this knowledge, the first
officer call ed f

or, and the captain brief

ed a visual approach to the landing runway.Assessment of PL 111

- 216

15

According to the accident report

, the flight crew made numerous procedural errors, demonstrated poor crew coordination, and the darkness and snow showers adversely affected the crew’s a bility to avoid terrain. The NTSB also con cluded
that the presence of a passenger in the flight deck jump seat may have increased
the p ressure on the captain to complete the flight.

The NTSB determined that the probabl e cause of th is accident was the pi lots’ operation of the aircraft below the minimum descent altitude without having visual reference with the runway. Contributing to the accident was the vague wording of a NOTAM regarding the limitations on circling approaches at night in Aspen , the deteri orating weather, and the pressure on the captain to complete the flight.

The captain held and ATP certificate with 9,900 hours of total flight time and 1,475 in the Gulfstream III. The first officer held an ATP certificate with 5,500
total hours and 913 hours in the Gulfstream II and III. In spite of the considerable flight time and experience that both these pilots had, it is possible that the enhanced training requirements would have been effective in improving the performance and situational awareness of the flight crew during this accident.

In sum, the rulemaking would have avoided a theoretical 4.45 accidents ( 1 0.9 % ), 11 of a total of

32 fatalities ( 3 4

. 3 % )

, and 2.85 of 11 serious injuries ( 25.9 % )

. Appendix 3 shows the scoring for
the 19 on

- demand ac cidents against which the rulemaking received a positive score.



### Summary

Based on a detailed review of 417 accidents over 10 years, the analysis found that the rulemaking would be most effective against accidents involving turbojets in on

- demand Part 1 35 operations and against twin

- engine aircraft accidents in Part 1 35 commuter operations.

However

, the rulemaking would have had a significant effect against only a small number of Part

1 21 accidents, and the rulemaking would have had no effect against the s mall number of Part

1 25 accidents that occurred over the 10

- year study period.

T he analysis also noted that a large percentage of accidents had no reflection on crew performance , due to the prevalence of ramp accidents caused by others, turbulence acc idents that involve either passenger behavior or conditions that could not be anticipated, and several types
of miscellaneous events

. Nevertheless, the rulemaking could save a total of about 1. 55 accidents per year, plus 3. 4 fatalities and 0.7 serious inj uries per year in Part 121 and Part 135 operations

.Assessment of PL 111

- 216

16

Appendix 1: Scoring Against 31 Part 121 Accidents in Which t he Rulemaking Received Positive Scores, FY 2001

– FY 2010

NTSB # Dam age High Injury Date Carrier Location Aircraft Service Fatal Score

NYC02 LA013 S ubstantial M inor /None 10/16/2001 Mesa Roanoke, VA ERJ

- 145 P assenger 0 0.75

DEN07LA101 Substantial Minor/None 6/20/2007 Grea t Lakes Laramie, W Y BE

- 1900D Passenger 0 0.55

DCA04MA068 Destroyed Fatal 8/13/2004 Air Tahoma Florence, KY CV

- 580 C argo 1 0 .55

CHI01FA084 Substantial Minor/None 2/10/2001 Great Lakes Chicago, IL BE

- 1900 Passenger 0 0.55

NYC03FA035 Substantial Minor/None 1/6/2003 ExpressJet Cleveland, OH ERJ

- 145LR Passenger 0 0.55

MIA01LA030 N one Serious 11/20/2000 Atlantic Southeast A shville, NC ATR

- 72 Passenger 0 0.55

DCA08RA077 Destroyed Fatal 7/6/2008 USA Jet Saltillo, Mex ico DC

- 9

- 15F Cargo 1 0.55

DFW08CA215 Substantial Minor/None 8/14/2008 Trans States St. Louis, MO ERJ

- 145LR Passenger 0 0.55

FTW04LA225 Substantial Minor/None 8/ 30/2004 Custom Air Transport El Paso , TX B727 Cargo 0 0.35

DCA04MA045 Substantial Minor/None 5/9/2004 American Eagle San Juan, PR ATR

- 72 Passenger 0 0.35

ANC01FA011 Substantial Minor/None 10/22/2000 Frontier Flying
Service Bethel, AK BE

- 1900 Passenger 0 0.35

DCA08WA071 Substantial Minor/None 5/16/2008 Asia Pacific Airlines Phonpei,
Micronesia B727

- 200F Cargo 0 0.35

DCA09MA027 Destroyed Fatal 2/12/2009 Colgan Air Buffalo, NY DHC

- Q400 Passenger 50 0.35

DCA05MA004 Destroyed Fatal 10/19/2004 Corporate Airl ines Kirksville, MO BAE

- 32 Passenger 13 0.35

DCA07FA037 Substantial Minor/None 4/12/2007 Pinnacle Traverse City, MI CL

- 600

- 2B19 Passenger 0 0.15

NYC06LA191 Substantial Minor/None 7/24/2006 American Eagle Newark, NJ ERJ

- 135LR Passenger 0 0.15

DCA05MA09 9 Substantial Minor/None 9/18/2005 Spirit Fort Lauderdale,

FL A321

- 231 Passenger 0 0.15

NYC05FA094 Substantial Minor/None 6/7/2005 UPS Louisville , KY MD

- 11F Cargo 0 0.15

DCA08FA018 Substantial Minor/None 12/16/2007 Air Wisconsin Providence , RI CL600

- 2B19 Passenger 0 0.15

DCA04MA082 Substantial Minor/None 9/19/2004 FedEx Memphis, TN MD

- 11 Cargo 0 0.15

DCA09FA048 Substantial Serious 5/6/2009 World Airways B altimore, MD DC

- 10

- 30 Passenger 0 0.15

FTW03MA160 Substantial Minor/None 5/24/2003 Southwest Amaril lo, TX B737

- 300 Passenger 0 0.15

CHI02LA170 Substantial Minor/None 6/16/2002 Vanguard Kansas City, MO MD

- 82 Passenger 0 0.15

LAX04LA050 Substantial Minor/None 11/14/2003 United San Francisco, CA B747

- 400 Passenger 0 0.15

CEN10LA363 None Serious 6/28 /2010 American Eagle Longview, TX ERJ

- 145LR Passenger 0 0.15

DCA05MA071 None Serious 5/31/2005 Air Wisconsin Chicago, IL CL

- 600

- 2B19 Passenger 0 0.15

DFW05LA112 None Serious 4/29/2005 Southwest Little Rock, AR B737

- 700 Passenger 0 0.15

ERA09LA488 None Serious 8/27/2009 Airtran Inverness, FL B717

- 200 Passenger 0 0.15

CHI02LA111 None Serious 4/18/2002 ATA Springfield, MO B757 Passenger 0 0.15

CEN09MA142 D estroyed Serious 1/27/2009 Empire Lubbock, TX ATR

- 42

- 300 Cargo 0 0.15

DCA04WA043 Destroyed Minor/N one 4/28/2004 Centurion Air Cargo Bogota, Colombia DC

- 10

- 30F Cargo 0 0.15Assessment of PL 111

- 216

17

Appendix 2 : Effectiveness of the Rulemaking Against 8 Part 135 Commuter Accidents in Which the Rulemaking Received Positive Scores, FY 2001

– FY 2010

Date Operator Damage Fatal Se rious Location Aircraft Service Event Score

4/4/2004 Peninsula Airways Substantial 0 0 Unalaska, AK G

- 21 Commuter Lost Control on Landing 0.75

3/13/2002 Warbelows Air

Ventures Substantial 0 0 Fairbanks, AK PA

- 31 Commuter Landed Gear Up 0.15

8/24/2002 Ca pe Smythe Air Substantial 0 0 Nuiqsut, AK PA

- 31 Commuter Landed Gear Up 0.15

7/13/2003 Air Sunshine Substantial 2 0 Treasure Cay,
Bahamas CE

- 402C Commuter Ditched; Engine & Pilot Response 0.15

9/17/2004 Everts Air Alaska Substantial 0 0 Telida, AK PA

- 31

- 350 Commuter Landed Gear Up 0.15

2/19/2009 Frontier Flying

Service Substantial 0 1 Nome, AK PA

- 31

- 350 Commuter CFIT, Flat Terrain 0.15

2/1/2010 Cape Air Substantial 0 0 Watertown, NY CE

- 402C Commuter Overran on Landing 0.15

3/15/2010 Servant Air Destro yed 0 2 Kodiak, AK BN

- 21

- 21 Commuter LOC on Climbout 0.15

Appendix 3 : Effectiveness of the Rulemaking Against 19 Part 135 On

- Demand Accidents in

Which the Rulemaking Received Positive Scores, FY 2001

– FY 2010

Date Operator Damage Fatal Serious Location Aircraft Service Event Sco

re

12/20/2000 Million Air Dallas Substantial 0 0 Jackson, WY HS

- 125

- 700 Passenger App & Landing 0.15

3/29/2001 Avjet Destroyed 18 0 Aspen, CO Gulfstream

III Passenger CFIT 0.35

8/28/2001 Grand Aire Express Substantial 0 0 Detr oit Falcon Cargo RE

- Landing 0.35

3/25/2002 Corporate Flight

Management Substantial 0 0 Anderson, IN MU

- 300 Passenger RE

– Landing 0.15

8/13/2002 Corporate Flight

International Destroyed 0 0 Big Bear City, CA CE

- S550 Passenger RE

– Landing 0.35

8/30/2002 Care Flight

International Destroyed 1 4 Lexington, KY Lear 25C Med ical RE

– Landing 0.15

4/8/2003 Grand Aire Express Destroyed 0 2 St Louis, MO Falcon Cargo Ditched, Fuel 0.35

11/11/2003 West Coast

Charters Substantial 0 0 Wheeling, IL CE

- 560XL Pa ssenger RTO Overrun 0.15

2/20/2004 Aztec Captital

Corporation Substantial 0 1 Ft. Lauderdale Lear 25B Passenger RE

– Landing 0.35

3/19/2004 Airnet Systems Substantial 0 0 Utica, NY Lear 35A Passenger Hard Landing 0.15

11/28/2004 Air Castle

Corporation D estroyed 3 3 Montrose, CO CL

- 600 Passenger LOC

– Climbout 0. 3 5

7/15/2005 Aspen Base

Operations Substantial 0 1 Eagle, CO Lear 35A Passenger Hard Landing 0.15

1/5/2006 Netjets as

Executive Substantial 0 0 Woodruff, WI CE

- 560 Passenger RE

– Landing 0.15

6 /2/2006 International Jet

Charter Destroyed 2 0 Gro ton, CT Lear 35A Passenger CFIT

– Water 0.35

1/10/2007 Airnet Systems Substantial 0 0 Columbus, OH Lear 35 Cargo LOC

– Flight 0.15

3/17/2007 Air Trek Substantial 0 0 Beverly, MA CE

- 500 Med App & Landing 0.15

8/23/2007 Jet Share US Substantial 0 0 Westhampton, NY Lear 60 Passenger App & Landing 0.15

7/31/2008 East Coast Jets Destroyed 8 0 Owatonna, MN HS

- 125

- 800 Passenger App & Landing 0.35

2/24/2009 Bankair Substantial 0 0 Kansas City, MO Lear 35A Carg o App & Landing 0.15