### FINAL REGULATORY EVALUATION

FUNCTION AND RELIABILITY FLIGHT TESTING FOR TURBINE-POWERED AIRPLANES WEIGHING 6,000 POUNDS OR LESS

(14 CFR, Section 21.35)

OFFICE OF AVIATION POLICY AND PLANS AIRCRAFT REGULATORY DIVISON (APO-300) Daniel F. Leach May 2, 2011

## I. INTRODUCTION

For Title 14, Code of Federal Re gulations (14 CFR), part 23 airc raft certifications, function and reliability (F&R) flight testing is required by 14 CF R § 21.35(b)(2) for all airplanes weighing more than 6,000 lbs. maximum certificated weight. F&R flight testing is not required for gliders or for part 23 airplanes weighing 6,000 lbs. or less. This Fina l Regulatory Evaluation examines the impact of this Federal Aviation Administration (FAA) final rule, which will extend F&R flight testing to part 23 turbine-powered airplanes weighing 6,000 lbs. or l ess. The exception for gliders and for reciprocatingengine-powered airplanes weighing 6, 000 lbs. or less will be retained.

The FAA has determined that this change is necessary to promote safety. This safety determination is based on the new light-weight turbine-powered airplanes having design features and performance characteristics similar to larger airplanes. We expect that adoption of this final rule will enhance safety and reduce costs by substantially reducing the numbe r of service difficulties and airworthiness directives experienced post-certification.



## II. GUIDE TO ACRONYMS USED IN THIS DOCUMENT AD—Airworthiness Directive CAB—Civil Aeronautics Board CAR—Civil Air Regulations CFR—Code of Federal Regulations FAA—Federal Aviation Administration F&R—Function and Reliability SCR—Special Certification Review SDR—Service Difficulty Report VLJ—Very Light Jets



## III. BACKGROUND

The requirement for F&R flight testing originates with the Civil Aeronautics Board (CAB) imposing a “service test” requirement for all U.S. aircraft in 1947 (12 FR 2086, March 29, 1947). The purpose of

this test was to “ascertain whether there is reasona ble assurance that the airplane, its components, and equipment are reliable and function properly.” A pr eceding rulemaking (12 FR 1028, February, 13, 1947) referred to a study showing extensive maintenance issu es and a greater likelihood of accidents in the early

life of new model aircraft. The conclusion was draw n that the accident rate would be significantly reduced if before type certification, the Civil Aer onautics Board (CAB) required aircraft to undergo tests specifically designed to ascertain their reliability and proper functioning. In 1950, the CAB amended the airworthiness standards to exclude “... smaller airplanes, specifically those of 6,000 pounds maximum weight or less” from the service test requirement (15 FR 8899, December 15, 1950). The introductory text of the revision explains that most of the significant changes in



### 2the amendment “stem from the desire for simplification of the rules in this Part with respect to the smaller airplanes, specifically those of 6,000 pounds maximu m weight or less, which would be expected to be used mainly as personal airplanes.” The introductory text explains that the service test requirement was removed for airplanes of 6,000 pounds or less maxi mum weight because “experience seems to indicate that this rule imposes a burden upon the manufactur ers not commensurate with the safety gained.” The requirement for F&R flight testing, and the exception for part 23 airplanes of 6,000 pounds or less maximum weight, is now found in 14 CFR part 21 (§ 21.35(b)(2)). The decision to exclude part 23 airplanes of 6,000 pounds or less maximum weight from F&R flight testing was based on the state of technology existing in 1950. These airplanes were also expected to be used mainly as personal airplanes. Civil aircraft produced in the period 1945-1955 were typically single piston-engine airplanes weighing less than 3,000 pounds with engine output less than 300 horsepower.



## IV. DISCUSSION OF COSTS AND BENEFITS

A. Total Costs and Benefits of this Final Rule We expect that the typical certification project for an airplane subject to the final rule will be for a new airplane design with a turbine engine type previously used in a type certificated aircraft requiring 165

hours 1 of F&R flight testing at a total cost of about $317,000. In the case of a new airplane design and an engine type not previously used on a type certificated airplane, we estimate that double the hours (330) will be required, so the total cost will double to $634,000. We expect that this final rule will enhance safety and reduce costs by substantially reducing the number of safety incidents and Airworthiness Directives experienced post-certification. A partial estim ate of the expected costs that will be avoided for a single new airplane design amounts to $1.8 million, with a present value of $1.6 million. These avoided costs are approximately six times the costs of our 165-hour “typical” estimate and approximately triple the higher 330-hour estimate. Consequently, the expected benefits of this final rule greatly exceed its modest expected costs. For additional detail, see th e separate sections on costs and benefits below.

1 See the separate cost section below for the reason we increased the number of hours from 150 (the minimum required by § 21.35(f)) to 165.



### 3Response to Cessna Comment

In a comment on the NPRM Cessna stated that the proposed NPRM has the potential to impose “ burden upon the manufacturer not commensurate with safety gained .” For example, if a simple, well-developed airplane were modified with a reliable, well-developed turb ine engine, it is not likely that F&R testing would discover issues not encountered during properly conducted certification testing. In this case, the 150 hour F&R testing proposed by th e NPRM would be an undue burden on the manufacturer ....

The rule addresses applicability only and does not change the minimum number of hours required by § 21.35(f) (that is, 150 or 300 hours, whichever appropriate). Some simu ltaneous flight testing performed

as part of the certification process may also be counted toward F&R flight testing if the conditions of

§ 21.35(a) are met at the time the testing was performed (for example, testing related to Type Inspection

Authorization). Given the substantial cost-beneficia l nature of the rule for our case of a newly type

certificated airplane with an engine previously used on a type certificated airplane, we believe the rule

will remain cost-beneficial.



### B. Who is Affected by this Rule? Manufacturers of part 23 turbine-powered airpla nes weighing 6,000 lbs or less are affected.



### C. Assumptions and Sources of Information

 We use a two-year period of analysis, as we find this period sufficient to show the cost-beneficial nature of this final rule. We use the period from th e beginning of 2007 to the end of 2008, as the data used in the analysis are from this period. The short period of analysis reflects the inherent nature of F&R flight testing, designed as it is to uncover design or system reliability flaws that otherwise would reveal themselves in the very early life of an aircraft.

 Discount rate is 7 percent (Office of Manageme nt & Budget, Circular A-94, “Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs,” October 29, 1992, p. 8).

 Data on costs of compliance with this proposal we re obtained from a part 23 airplane manufacturer

and FAA estimates.



### D. Costs of this Final Rule

We estimate the costs of this final rule based on the F&R costs incurred by a part 23 airplane manufacturer for a turbojet airplane with a maximum weight greater than 6,000 lbs. The turbine-powered Eclipse EA-500, with maximum weight less than 6,000 lbs, was recently type certified under a program in



### 4which it voluntarily undertook a reduced 200-hour F&R program, 100 hours pre-certification and 100 hours post-certification. 2 The F&R costs for the Eclipse EA-500 would be difficult, if not impossible, to obtain, however, as Eclipse Aircraft has been li quidated under Chapter 7 bankruptcy and a new firm, Eclipse Aerospace, has been formed to take over the Eclipse assets. We believe our use of more readily obtainable data for a part 23 turbojet weighing somewh at more than 6,000 lbs. is adequate. Moreover, as we will see below, the ad-hoc nature of Eclipse’ s voluntary F&R program appears to have limited the appropriateness of the Eclipse F&R cost data even if available. We may overestimate the cost of F&R flight testing by our use of costs for an airplane we ighing more than airplanes affected by the final rule. We estimate F&R cost per hour in order to more e asily incorporate different estimates of total F&R flight hours. Test pilot and flight test engineer costs are FAA estimates. All other cost estimates were provided by a part 23 airplane manufacturer. The estimates are shown in the Table 1 below:

2 Pratt & Whitney Canada developed a new PW610F engine for the Eclipse EA-500.



### 5

## Table 1. Estimated Costs of F&R Flight Testing

Cost Item F&R Cost/hr Total Item Cost (150 hours) Hours Cost to replace two engines Cost to replace one engine

Fuel ($4.10 per US gal) $ 377.20 $ 56,580 1100 $ 464,890.80 $ 232,445.40 Labor (Mechanic) $ 51.50 $ 7,725 150 $ 63,394.20 Parts $ - 1 $ 422.63 Engine reserves $ 422.63 $ 63,394

Total "direct" cost $ 127,699 Hours Test AP Refurbishment

Indirect factor 1 0.35 $ 44,695 1100 $ 315,839.18 150 $ 43,068.98

Refurbishment of test airplane 2 $ 287.13 $ 43,069 1 $ 287.13

Discount allowance 3 $ 240.00 $ 36,000

Test pilot costs 4 $ 168.27 $ 25,240 Hours 2 Test Pilots 1 Pilot (year) Flight test engineer costs 4 $ 76.92 $ 11,538 2080 $ 350,000.00 $ 175,000 F&R Cost (150 hrs) $ 288,242 150 $ 25,240.38 F&R Cost (165 hrs) $ 317,066 1 $ 168.27 F&R Cost (330 hrs) $ 634,132

Hours Flt Test Engineer 2080 $ 160,000.00 150 $ 11,538.46

1 $ 76.92

Table notes: (1) Pre & post fl ight, trip expenses and other administrative costs (2) Refurbishment of the test airplane after completion of all type certification prior to airplane sale (3) $240 is per hour estimated reduction in airp lane resale value ow ing to used status. (4) Test pilot and flight test engineer costs are FAA estimates. All other cost estimates were provided by a part 23 airplane manufacturer.

Cost Discussion For aircraft subject to F&R flight testing under 14 CFR § 21.35(b)(2), § 21.35(f) requires at least 300 hours of F&R flight testing “with a full complement of engines” for aircraft “incorporating turbine engines of a type not previously used in a type certif icated aircraft,” and at least 150 hours for all other aircraft. As most proposed very light jet (VLJ) type certification projects appear to be based on the type certificated Williams FJ-33 engine or other previously type certificated engines, we expect this minimum requirement to hold for the typical project subject to this final rule. F&R flight testing for 150 hours was required for the airplane whose data we use here, so one of our cost estimates assumes 150 hours of F&R flight testing. Sometimes, to fulfill the requirements of F&R flight testing, more than the minimum number of f light hours are necessary. For the purposes of this cost analysis, we used an average extension of 10%, or 15 hours, so our “typical” estimate assumes 165 hours



### 6of F&R flight testing. We double that estimate to al so provide an estimate for a new airplane design with a new engine design. We now discuss the detailed cost items in Table 1. We make no changes to the Fuel cost estimate provided to us, as the price of fuel assumed is close to the current price. The cost of Parts is minimal as parts are typically provided on consignment. Other majo r cost categories reflect user cost, or wear and tear, of the test airplane. After testing the test airplane may be sold. Buye rs understandably will be reluctant to buy the airplane with its heavily tested engines of uncerta in value, so the used engines are returned to the vendor and the cost of two new engi nes is allocated to the entire 1100 hours used in the sample certification testing program. (We were unabl e to ascertain whether the used engines had been provided on consignment, or whether the airplane manufacturer had received a discount on the new engines upon returning the used engines.) The engine co st of one hour of F&R flight testing is 1/1100 of the total cost so the cost of a 150 F&R flight tes ting program is 150 times that. These costs are labeled in

the table as Engine reserves . The Total “direct” cost of $127,699 is the sum of fuel costs, mechanic costs, and engine reserves. Total “direct” cost is multiplied by an Indirect factor of 0.35 to account for pre- and post-flight costs, trip expenses, and other administra tive costs. As the cost of engines has been accounted for, Refurbishment of test airplane essentially reflects user cost of the airframe. The Discount allowance of $240 is the per hour estimated reduction in the resale value of the airplane owing to its used status. The cost estimates provided to us did not include crew costs, so these costs are FAA estimates. We estimate the hourly costs of two test pilots and one flight test engineer using fully-burdened annual salaries and an engineering labor year (2080 hours). Our final figures are $288,242 for a 150-hour F&R flight testing program, $317,066 for a 165-hour program, and $634,132 for a 330-hour program.



### E. Benefits of this Final Rule We expect that adoption of this final rule w ill enhance safety and reduce costs by substantially reducing the number of service difficulties experienced post-certification. This expectation is supported by evidence from the service experience of the Eclipse EA-500. The Eclipse Special Certification

Review 3 (SCR) team looked at 85 Eclipse Service Difficulty Reports (SDRs), submitted between July 29, 2007, and May 13, 2008. The Eclipse SCR team "conc luded the majority of the SDRs resulted from reliability issues separate from compliance with th e minimum FAA standards" (Eclipse SCR, Executive Summary). There also were six Eclipse-related ADs i ssued in the one-year period between November 2007 and November 2008, and the Ec lipse SCR also reports several instances of undocumented problems.

3 Special Certification Review: Eclipse Aviation Corpor ation Model EA-500 Airplane. Prepared for the Federal Aviation Administration Associate Administrato r for Aviation Safety, September 12, 2008.



### 7Eclipse voluntarily conducted its own limited F&R flight testing. However, the FAA team did not view F&R flight testing as a requirement for Eclipse to rece ive its type certificate (Eclipse SCR, p. 28). This deficiency in Eclipse’s volunteer F&R flight testing program provides direct empirical evidence for the benefits of F&R flight testing. The general evid ence from SDRs and ADs has already been noted. More specifically, the number of SDRs and ADs experienced by the Eclipse EA-500 appears to have been high. Table 2 below compares the SDR/AD experience of the Eclipse EA-500 compared to the Cessna 510. These airplanes are highly comparable, both being twin-engine VLJs and close competitors. Moreover, both models were type certificated in the same m onth (September 2006) and began delivery within one month of each other (Cessna: Nove mber 2006, Eclipse: December 2006). 4 Table 2 shows that while there are 60% more Eclipse EA-500s, the number of SDRs i ssued against the Eclipse EA-500 is almost triple the number issued against the Cessna 510. 5 Of more concern, there have been six ADs issued against the Eclipse EA-500 and none against the Cessna 510.



## Table 2. Eclipse EA-500 vs. Cessna 510 Eclipse EA-500 Cessna 510 No. of US-Registered Airplanes 264 163 No. of SDRs (part 91 operations only) 1 

### 28 10 No. of ADs 6 0 Table note: (1) Excludes 85 Eclipse SDRs written by DayJet, under part 135 operations. General Note: Figures in table as of December 1, 2009.

Listed in Table 3 below are the main specific problem s encountered by the Eclipse EA-500, ranked by the estimated likelihood these problems would have been uncovered by a mandatory F&R flight testing program. The cost of the modification to solve th e pitot/angle-of-attack (AOA) problem—and hence the

avoidance of which would be a benefit of F&R f light testing--has been estimated in AD 2008-02-04.

While there are costs associated with the other problems (and corresponding benefits from avoiding them), they are difficult, if not impossible, to estimate. For example, AD 2008-24-07 addresses the problem of engine surges by limiting the Eclipse EA- 500 to a maximum operating altitude of 37,000 feet,

4 We should note, however, that (1) the Eclipse EA-500 was a new aircraft design, while the Cessna 510 was a derivate design and (2) Eclipse Aviation had no previous experience certifying an aircraft, while Cessna Aircraft is a well-established firm with decades of certification experience.

5 In order not to distort the comparison between the Eclip se EA-500 and the Cessna 510, 85 SDRs written by DayJet for its Eclipse fleet under part 135 operations were excluded, as SDRs are required for part 135 operations, but voluntary for part 91 operations (other than subpart K operations).



### 8

### 9 

## Table 3. Major Eclipse Problems

Eclipse problem Reference Likelihood these problems will be revealed by a mandatory F&R flight testing program Trapping of moisture and freezing condensation in the pitot/angle-of-attack (AOA) system, causing erroneous AOA and airspeed information, with consequent possible loss of control. SCR, p. 25, AD 2008-02-04 Likely (70-75% chance) Engine surges caused by hard carbon buildup on the static vane, which could lead to loss of engine power. AD 2008-24-07 Probable (50-60% chance) Pitch & rudder trim problems. SCR, p. 17 Probable (50-60% chance) Brake problems. (Eclipse brakes are mechanical.) SCR, p. 26 Measurably increase likelihood (40-50% chance) Tire problems SCR, p. 26 Measurably increase likelihood (40-50% chance) Autopilot turbulence sensitivity, often disengaging and not easily reengaging. SCR, p. 26 Small (10% chance) Throttle position exceeding its maximum range, which could cause loss of control of both engines. AD 2008-16-15 Unlikely (1-2% chance)

so the cost of this AD is the loss in benefits from not being able to operate at the maximum altitude for the airplane. As we show below, however, the bene fit from avoiding just the pitot/AOA problem greatly exceeds our estimated cost of F&R flight testing, even for a full 330-hour testing program.

Direct Costs Per Airplane of Pitot/AOA Problem AD 2008-02-04 addresses the Eclipse pitot/AOA problem with a design modification. The AD estimates the parts cost of the modification to be $7,000 and that the modifi cation will require 30 workhours at a billable wage rate of $80 an hour, for a to tal wage cost of $2,400 per airplane. So the total direct cost per airplane is estimated to be $9, 400 per airplane. The AD does not estimate out-of-service cost, which we now estimate.

Out-of-Service Cost Per Airplane of Pitot/AOA problem We estimate out-of-service cost as the opportunity co st of capital, the capital in this case being the airplanes themselves. Conceptually , we measure the opportunity cost of capital as the cash flow forgone because the airplane is out of service. We measure that cash flow by lease rates. As a percentage of airplane value, lease rates for the Eclipse EA-500 range from 0.4% to 0.6% per month. During the period 2007-2008 airplane value is approximated by new airp lane selling price of approximately $1.6 million. 6

Based on an estimated 30 hours of labor, we use a four-day out-of-service time and estimate out-ofservice cost per airplane as $1.6 million x .5% x 4/30 = $1,067.

6 Robert Zuskin, Jet Perspectives, Inc., Sterling, Va. Zusk in reports that the selling pric e of the Eclipse EA-500 was heavily discounted from its list price.Total Costs of Pitot/AOA Problem We present our total estimated costs of the Pitot/AOA problem in Table 4 below:



## Table 4. Cost of Design Modificati on to the Eclipse Pitot/AOA System [Estimated Avoided Costs Owing to F&R flight testing]

Labor cost $ 2,400 30 work-hours x $80/hr Parts cost $ 7,000 Total direct cost per airplane $ 9,400 Total direct cost on U.S. operators (264 APs) $ 2,481,600 Out-of-service costs per airplane $ 1,067 Total out-of-service costs (64 airplanes) $ 68,267 Total Cost on U.S. operators (264 airplanes) $ 2,549,867 Expected Benefit (Total Cost [avoided] x 0.7 probability) $ 1,784,907 Present Value Benefit (Expected Benefit discounted 1.5 years) $ 1,612,649

The design modification of the pitot/AOA system was required by AD 2008-02-04 for the existing fleet of 64 Eclipse EA-500 airplanes at the time of AD issuance, January 9, 2008 (Published in Federal Register , January 22, 2008 (73 FR 3618)). The effective date of the AD was February 26, 2008, with a compliance period of 60 days. For the remaining airplanes produced by Eclipse Aircraft, the modification was incorporated during production. Accordingly, the cost of the design modification applies to all 264 U.S-registered model EA-500 airplanes (of 267) produced by Eclipse Aviation until it ceased production in October, 2008. Out-of-service costs, however, apply only for the 64 airplanes directly affected by the

AD.

As shown in the table we estimate the total cost of the design modification for the Eclipse fleet to be $2.5 million. As discussed above, however, we assess the probability of F&R flight testing uncovering the pitot/AOA problem to be approximately 0.7 to 0.75. A ccordingly, using the lower figure, we calculate the Expected Benefit as the Total Cost (avoided) times 0.7, or $1.8 million. As the FAA issued a type certificate for the Eclipse EA-500 on September 30, 2006, approximately 1.5 years prior to the compliance date for this AD, we discount the Expected Benefit 1.5 years to find a Present Value Benefit of $1.6 million.



## V. SUMMARY

This Final Regulatory Evaluation examines the impact of this final rule, which will extend F&R flight testing to part 23 turbine-powered airplanes weighing 6, 000 lbs. or less. The exception for gliders and for reciprocating-engine-powered airplanes weighing 6, 000 lbs. or less will be retained. The FAA has determined that this rule change is necessary to pr omote safety. This determination is based on the new



### 10

### 11 light-weight turbine-powered airplanes having design features and performance characteristics similar to larger airplanes. We expect that this final rule will enhance safety and reduce costs by substantially reducing the number of service difficulties experienced post-certification. The evidence for this proposition is provided by the experience of the Eclipse EA-500, which appears to have had an unusually high number, per airpla ne, of post-certification SDRs and ADs. Eclipse did voluntarily conduct its own limited F&R flight testing. However, the FAA team did not view F&R flight testing as a requirement for Eclipse to receive its type certificate. Consequently, we find that there were a number of problems experienced by the EA-500 post-cer tification with significant likelihood of being uncovered by a mandatory F&R flight testing program. We deem one of these problems—the problem with the pitot/AOA system--as “likely” to have been uncovered by a mandatory F&R flight testing program. We estimate the expected total costs of fi xing this one problem--and hence the avoidance of which would be a benefit of F&R flight testing— to be $1.8 million, almost six times our estimated $317,000 cost of F&R flight testing. The benefits of F&R flight testing greatly exceed its modest costs.