# The Economic an d Safety Impacts of Performance

- Based Regulation and Consensus Standards

Catherine L. Taylor Don Dinh

U.S. Department of Transportation Office of the Secretary of Transportation John A. Volpe National Transportation Systems Center



## March 12 , 2015

Prepared for:
FAA Office of Aviation Policy and Plans

## No t i ce

T h i s docu m e n t i s d isse m i n a t e d u nd e r t h e s po n s o r s h i p o f t h e D e p ar t m e n t o f T ra n s po r t a ti o n i n t h e i n t e r e s t o f i n f o r m a t i o n e x ch a n g e

. T h e U n it e d S t a t e s Go v e r n m e n t a ss u m e s n o li a b i l it y f o r the

con t e n t s o r u s e t h e r e o f

.
The United States Government does not endorse produc ts or manufacturers. Trade or manufacturers’ names appear herein solely because they are considered essential to the objective of this report

.

## REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704

- 0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden esti mate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate f or Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 222 02

- 4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704

- 0188), Washington, DC 20503. 1. AGENCY USE ONLY (Leave blank) 2. REPORT DATE February 2015 3. REPORT TYPE AND DATES COVERED Final Report

4. TITLE AND SUBTITLE

The Economic and Safety Impact s of Performance Based Regulation and Industry Consensus Standards 5a. FUNDING NUMBERS FA4SBN NY103

6. AUTHOR(S)

Catherine L. Taylor and Don Dinh 5b. CONTRACT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

U.S. Department of Transportation John A Volpe National Transportation Systems Center 55 Broadway Cambridge, MA 02142

- 1093 8. PERFORMING ORGANIZATION REPORT
NUMBER DOT

- VNTSC

- FAA

- 15

- 06

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

Federal Aviation Administration Office of Aviation Policy and Plans Federal Aviation Administration 800 Independence Ave, SW Washington, DC 20591 10. SPONSORING/MONITORING
AGENCY REPORT NUMBER

11. SUPPLEMENTARY NOTES

12a. DISTRIBUTION/AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE

13. ABSTRACT (Maximum 200 words) In response to Public Law 113

- 53, the “Small Aircraft Revitalization Act of 2013,” the Federal Aviation Administration (FAA) is conducting a rule

making to reorganize part 23 of the Federal Aviation Regulations (FAR) related to airworthiness standards for small aircraft. The reorganization of part 23 will involve replacing current prescriptive requirements with performance

- based regulations and streamlining the approval of safety advancements by using consensus standards to clarify how the objectives in part 23 can be met using specific designs and technologies. The text of the Small Aircraft Revitalization Act mentions several motivations for the changes including: improving safety, reduc ing regulator y compliance costs, spurring innovation and technology adoption. Further, as the title of the law suggests, it is hoped that th ese changes will revitalize the small aircraft manufacturing industry. This report investigates whether performance

- based regulat ion (PBR) or streamlining of regulatory approval via use of industry consensus standards in other sectors has resulted in the positive outcomes anticipate d in relation to the part 23 reorganization.

14. SUBJECT TERMS

Part 23 Reorganization, Performance

- B ased Regulation, PBR, Performance Standards 15. NUMBER OF PA GES 30

16. PRICE CODE

17. SECURITY CLASSIFICATION
OF REPORT Unclassified 18. SECURITY CLASSIFICATION
OF THIS PAGE Unclassified 19. SECURITY CLASSIFICATION
OF ABSTRACT Unclassified 20. LIMITATION OF ABSTRACT
Unlimited

NSN 7540

- 01

- 280

- 5500 Standard Form 298 (Rev. 2

- 89) Prescribed by ANSI Std. 239

- 18
298

- 102i 

# Contents

L i s t of Abbreviations

................................ ................................ ................................ .............. ii
Executive Summary

................................ ................................ ................................ ............... 1

1. Introduction

................................ ................................ ................................ ................... 3

2. Background

................................ ................................ ................................ .................... 4

2.1 Performance

- Based Regulation

................................ ................................ ................................ .. 4

2.2 Variations Within Performance Based

- Regulatory Regimes

................................ ...................... 5

2.3 Consensus Standards

................................ ................................ ................................ .................. 6



### 3. Impacts of Performance

- Based Regulation

................................ ................................ ...... 9

3.1 Cost Savings and Efficiency

................................ ................................ ................................ ......... 9

3.1.1 Magnitude of the cost savings

................................ ................................ ..................... 11

3.2 Innovation and Safety Improvements

................................ ................................ ...................... 12

3.2.1 Removing Impediments to Meeting Consumer Demands

................................ ........... 13

3.2.2 Spurring Technology Diffusion

................................ ................................ ..................... 14

3.2.3 Creating Radical Innovation

................................ ................................ ......................... 16

3.3 Industry Revitalization

................................ ................................ ................................ .............. 16

3.4 Limiting Factors

................................ ................................ ................................ ......................... 17



### 4. Industry Consensus Standards

................................ ................................ ...................... 19

4.1 Innovation

................................ ................................ ................................ ................................ . 20

4.2 Product Costs

................................ ................................ ................................ ............................ 20

4.3 Safety

................................ ................................ ................................ ................................ ........ 21

4.4 Industry Revitalization

................................ ................................ ................................ .............. 22

4.5 Caveats

................................ ................................ ................................ ................................ ...... 22



### 5. Other Experiences with PBR and Other Regulatory Reforms

................................ .......... 23

5.1 HACCP in Food Safety

................................ ................................ ................................ ............... 23

5.2 New Zealand Leaky Building Crisis

................................ ................................ ............................ 24



### 6. Conclusions

................................ ................................ ................................ .................. 26

7. References

................................ ................................ ................................ ................... 27ii 

# L i s t of Abbreviations

Abbreviation Term

APO Office of Aviation Policy and Plans
ASTM American Society for Testing and Materials
BIA Building Industry Authority
BSEE Bureau of Safety and Environmental Enforcement
CAC Command

- and

- Control Regulations
CAFE Corporate Average Fuel Economy
CCP Critical Control Point
COTS Commercial

- off

- the

- Shelf components
CPSC Consumer Product Safety Commission
EPA Environmental Protection Agency
FAA Federal Aviation Administration
FAR Federal Aviation Regulations
FSIS Food Safety Inspection Service
GA General Aviation
GAO US General Accounting Office
HACCP Hazard Analysis Control Point
LSA Light

- Sport Aircraft
NAHB National Association of Home Builders
NHTSA National Highway Traffic Safety Association
NRC Nuclear Regulatory Commission
OMB Office of Management and Budget
OSHA Occupational Safety and Health Administration
PBR Performance

- based Regulations
PHMSA Pipeline and Hazardous Material Safety Administration
PIO Pilot Induced Oscillation
R

- LOC Runway Loss of Control
SAE Society of Automotive Engineers
S

- LSA Special Light Sport Aircraft
SMS Safety Management Systems
USCG United States Coast Guard
USDA U.S. Department of Agriculture1 

# Ex ecutive Summary

In response to Public Law 113

- 53, the “Small Aircraft Revitalization Act of 2013,” the Federal Aviation Administration ( FAA ) is conducting a rule

- making to reorganize part 23 of the Federal Aviation Regulations (FAR) related to airworthiness standards for small aircraft. The reorganization of part 23 will involve replacing current prescriptive requi rements with performance

- based regulations and streamlining the approval of safety advancements by using consensus standards to clarify how the objectives in part 23 can be met using specific designs and technologies. The text of the Small Aircraft Revital ization Act mentions several motivations for the changes including: improving safety, reducing regulatory compliance costs, spurring innovation and technology adoption. Further, as the title of the law suggests, it is hoped that these changes will revitali ze the small aircraft manufacturing industry.

The FAA Office of Aviation Policy and Plans (APO) has asked the Volpe Center to investigate whether performance

- based regulation (PBR) or streamlining of regulatory approval via use of industry consensus stand ards in other sectors has resulted in the positive outcomes anticipated in relation to the part 23 reorganization.

Regarding the use of PBR, this research finds the following:

• The available literature is practically unanimous in the assertion that PBR wou ld result in lower production costs for industry , i.e. higher production efficiency. The basis for th is assertion is that firms will naturally seek cost savings efficiencies in an effort to maximize profits. The argument is that p rescriptive regulations im pede firms from adopting least cost processes , while PBR affords the flexibility to meet regulatory goals using a least cost option that is tailored to their
own circumstances.

• Performance

- based standards will not by themselves spur innovation and progres s toward new safety features. PBR will not necessarily provide an incentive to go beyond the standard set by the regulatory agency. Therefore, the advantage of PBR regarding fostering innovation related to safety and other product features is that the flex ibility afforded by PBR will not impede the adoption of the innovations.

• If the regulating agency wishes to promote diffusion of certain technologies, PBR is capable of serving that function. By carefully crafting performance based standards and by tighte ning those requirements over time as new technology becomes available, PBR would be able to speed the diffusion of technological improvements.

• To the extent that design standards inhibit competition in a market, moving to performance standards may increase competition and foster growth in the market. In addition, the use of PBR can facilitate the use of cost saving measures and remove barriers to innovation. In turn, lower prices and innovative product features can spur demand and revitalize an industry.2 • Based on the experience for the LSA segment, it appears that using consensus standards as a way of streamlining the aircraft certification process has resulted in lower costs of certification and facilitated the introduction of new and innovative designs.

Based on these findings, it appears likely that adopting PBR as a method of safety regulation for part 23 small general aviation aircraft will result in many of the positive impacts mentioned as objectives of the “Small Aircraft Revitalization Act of 2013 .” However, the magnitude of the positive impacts will be determined by how much freedom and flexibility the new performance standards actually provide. Providing guidance on “means of compliance” through consensus standards acts will assist manufacturers in developing methods for meeting the performance standards. However, it is possible that those consensus standards may turn out to be de facto design standards and erode the potential benefits of adopting PBR.3 

# 1. In troduc tion

In response to Public Law 113

- 53, the “Small Aircraft Revitalization Act of 2013 ,” the FAA is conducting a rule

- making to reorganize part 23 of the Federal Aviation Regulations (FAR) related to airworthiness standards for small aircraft. The reorganization of part 23 will involve repla cing current prescriptive requirements with performance

- based regulations and streamlining the approval of safety advancements by using consensus standards to clarify how the objectives in part 23 can be met using specific designs and technologies. The tex t of the Small Aircraft Revitalization Act mentions several motivations for the changes including: improving safety, reducing regulatory compliance costs, spurring innovation and technology adoption. Further, as the title of the law suggests, it is hoped t hat these changes will revitalize the small aircraft manufacturing industry.

The FAA Office of Aviation Policy and Plans (APO) has asked the Volpe Center to investigate whether performance

- based regulation (PBR) or streamlining of regulatory approval via use of c onsensus standards in other sectors has resulted in the positive outcomes anticipated in relation to the part 23 reorganization. Specifically, APO asked the following questions:

• Did PBR or use of industry consensus standards lead to lower product c osts? If so, what was the magnitude of the cost reductions?

• Did PBR or use of industry consensus standards spur innovation, lead to wider spread adoption of new technology , or foster faster introduction of safety improvements?

• Did PB R or use of industry co nsensus standards lead to higher numbers of products being sold or another form of industry revitalization?

The document presents the results of the Volpe Center’ s research into these topics

. The remainder of this report is organized as follows:

• Section 2 provides background information on PBR and use of consensus standards

• Section 3 discusses the economic and safety impacts of PBR

• Section 4 discusses the economic and safety impacts o f the use of consensus standards

• Section 5 provides additional experience with PBR and other regulatory r eforms

• Section 6 pro vides conclusions of the research

.4 

# 2. Background

2.1 Performance

- Based Regulation

The available literature is remarkably consistent in its description of PBR. The basic idea of PBR is to use performance standards or goals to define the regulatory objective and allow the regulated entity freedom and flexibility to determine the best way to meet the standard.

• “[ Performance

- based regulations ] stipulate a performance goal but allow the firms flexibility in determining how best to meet that goal [1] .”

• “When a regulation sets performance goals, and allows individuals and firms to choose how to meet them, it is called a performance

- based regulation [2]

. ”

• “ Perfo rmance based regulation can be defined as regulation that specifies required outputs, rather than inputs and thus provides a degree of freedom to the regulated to determine how they will achieve compliance [3]

. ”

• “In contrast to design standards, performance standards set a general attainment target but leave firms free to decide how to meet the goal [4] .”

• “Performance

- based models, often referred to as “outcome

- based” or “market

- based” regulation, specify an outcome to be achieved without prescribing the specific requirements to reach that outcome [ 5] .”

• “ Performance

- based approaches focus primarily on results. They can improve the objectivity and transparency of [U.S. Nuclear Regulatory C ommission ( NRC )] decision

- making, promote flexibili ty that can reduce licensee burden, and promote safety by focusing on safety

- successful

outcomes [6] .”

• “A performance based regulatory standard is a requirement, r egulation or rule that identifies the desired outcome of the system but gives the developer the discretion in how they achieve the outcome [7] .”

• “ Performance based legislation means regulation which prescribes the outcomes to be achieved rather than focusing on the step by step processes to which businesses must comply. This allows industries as well as individual firms to take different (and optimal) approaches to achieving outcomes or performance targets [8]

. ”

• "Performance standard" is a standard as defined above that states requirements in terms of required results with c riteria for verifying compliance but without stating the methods for5 achieving required results. A performance standard may define the functional requirements for the item, operational requirements, and/or interface and interchangeability characteristics. A performance standard may be viewed in juxtaposition to a prescriptive standard which may specify design requirements, such as materials to be used, how a requirement is to be achieved, or how an item is to be fabricated or constructed [9]

. ”

The counterpart of performance

- based standards is prescriptive standards. Prescriptive standards are a variety of command

- and

- c ontrol (CAC) regulations that specify exactly how to achieve compliance by dictat ing technologies or processes.

Another alternative to CAC regulations that is often mentioned in the context o f PBR is management

based regulation . Management

- based regulation “directs regulated organizati ons to engage in a planning process that aims toward the achievement of public goals, offering firms flexibility in how they achieve public goals [10] .” Examples of management

- based regulations are requirements for Safety Management Systems (SMS) required by several US DOT modal agencies, certain requirements imposed on nuclear power facilities by the US Nuclear Regulatory Agency (NRC), and food safety processes

requ ired by USDA and FDA. In general, these management

- based regulations apply to entities conducting ongoing repeated processes that require continual monitoring for safety. As such, results of this type of regulation are of limited applicability to the part 23 experience since the part 23 airworthiness certification process for the most part requires a one

- time review of an aircraft design. However, in the food safety context, the management

- based regulation is paired with a performance standard. The use of p erformance standards in food safety is discussed in Section 5.1 below.

Market

- based regulation is also widely mentioned in the context of PBR. Market

- based regulat ion uses prices and other economic incentives to promote certain policy goals; they are most often used in environmental regulation. Prominent examples of market

- based regulations are carbon taxes and emissions trading permits wh ere the use of economic mec hanisms cause firms to internalize the external costs they would otherwise not pay directly ( often called social costs , or simply externalities) of their activities. The issue of externalities is of limited relevance to the part 23 reorganization, so whil e there is significant literature on the experienced impacts from market

- based regulations, that literature has been largely set aside for the purposes of this report.



## 2.2 Variations Within Performance Based

- Regulatory Regimes

There are several variations in performance

- based regulation in practice . A full taxonomy of PBR applications has not yet been developed but a discussion of a few key areas of variation is useful.

Performance standards can be either tightly or loosely specified. More loosely specified s tandards tend to use qualitative standards such as “sufficient” or “reasonable.” The lack of specificity require s regulators to make judgments when assessing whether the standard has been met. As such, those types of standards are not very precise and the ultimate goals of the regulation might not be achieved. A more6 tightly specified standard would use a quantitative measure to determine compliance. Further those quantitative measure s may either be directly observed or they might be based on predictions or simulations. For example, t he quantitative environmental standard for a coal

- fired power plant might involve directly measuring emissions from a smoke stack while the safety standard at a nuclear power plant might be based on computer simulations that run a system through a variety of high

- risk scenarios [2]

.
An additional distinguishing factor among different PBR regimes is the degree to which prescriptive require ments are also included ; f ew regulatory regimes are pure PBR . Often the regulations are a hybrid of prescriptive requirements with additional performance standards. One approach is to have design standards as the regulation but allow exemptions if a petiti oner can demonstrate an equivalent level of performance . Alternatively the rules may be performance

- based but may offer strong guidance involving acceptable designs that would meet the performance

- standards.

The spectrum of possible performance

- based stand ards can also be related to the distance the standard is from the ultimate goal of the regulation. In relation to the p art 23 reorganization, the ultimate goal of the regulation is the safety of the aircraft occupants , those on the ground , and occupants of other aircraft sharing the airspace . Potentially, the performance standard could relate to an allowable number of fatalities or injuries per unit (say, hours flown, or per flight)

. However, such a performance standard is not practical or advisable for a n umber of reasons. First, the number of fatalities resulting from a certain aircraft model depends on many factors outside the manufacturer’s control such as the weather, airport configuration, human error, maintenance, etc. In addition, the general public would likely find it highly objectionable for a safety

- focused agency to describe as acceptable any number of fatalities or injuries other than zero.

A step further from th at purest form of performance standards would be performance standards related to each individual safety system on the aircraft. The performance standards for that level might be relate to a standard for the minimum power of an engine, or the reliability of a navigation system, etc. Further down the spectrum might be a performance standard for individual part of the aircraft, such as a strength test for a certain welding seam.



## 2.3 Consensus Standards

OMB C ircular 119 requires a ll federal agencies to use “voluntar y consensus standards” instead of government

- unique standards in their procurement and regulatory activities, except where inconsistent with law or otherwise impractical. One of the stated goals of the government for requiring the use of voluntary census s tandards is to “reduce the burden of complying with agency regulation.” The definition of “ v oluntary c onsensus s tandards” from OMB Circular 119 is provided in Figure 1 , below. The definition emphasizes that the process by which the standards are developed should be characterized by : o penness, b alance of interest, due process, an appeals process and consensus (not unanimity).7 Usually consensus standards are used either to specify a level of performance or to produce harmonization or interoperability among products developed by different manufacturers. In some cases
F ederal regulatory agencies incorporate consensus standards by reference within the regulat ions. For instance, the National Highway Transportation Safety Administration incorporates standards developed by the Society of Automotive Engineers (SAE) for items such as automotive safety glass, headlamps, window defrosting systems, etc. 1

The role of c on sensus standards for the part 23 reorganization is bit different than for other contexts. In the case of part 23, the FAA regulations will specify the performance standards. They will not be developed through a consensus standards body. However ASTM, the international standards organization
, will develop a set of consensus standards that will provide guidance on “means of compliance” with the FAA regulations. The FAA will still conduct an assessment to make sure that the aircraft design meets the performa nce standards in the regulations but the process for that review will be streamlined. In addition, once a design feature has been approved by FAA, it can be referenced by other designers and manufacturers that wish to include that feature. This change to p rocess is expected to result in significant cost savings related to aircraft certification.

Use of international consensus bodies like ASTM may also produce harmonization of the general aviation regulations among different countries. Such harmonization wou ld have advantages to general aviation ( GA ) manufacturers who could then sell the same model aircraft in multiple countries. As a result of larger markets for each model of aircraft, manufacturers can increase their production of each model and perhaps rea lize cost savings due to economies of scale.

1 For an example, see 49 CFR 571.5

.8 Figure 1 . Definition of Voluntary Consensus Standards from OMB Circular 119

4. What Are Voluntary, Consensus Standards?
a. For purposes of this policy, "voluntary consensus standards " are standards developed or adopted by voluntary consensus standards bodies, both domestic and international. These standards include provisions requiring that owners of relevant intellectual property have agreed to make that intellectual property availab le on a non

- discriminatory, royalty

- free or reasonable royalty basis to all interested parties. For purposes of this Circular, "technical standards that are developed or adopted by voluntary consensus standard bodies" is an equivalent term.
(1) "Voluntary consensus standards bodies" are domestic or international organizations which plan, develop, establish, or coordinate voluntary consensus standards using agreed

- upon procedures. For purposes of this Circular, "voluntary, private sector, consensus standard s bodies," as cited in Act, is an equivalent term. The Act and the Circular encourage the participation of federal representatives in these bodies to increase the likelihood that the standards they develop will meet both public and private sector needs. A voluntary consensus standards body is defined by the following attributes:
(i) Openness. (ii) Balance of interest. (iii) Due process. (vi) An appeals process. (v) Consensus, which is defined as general agreement, but not necessarily unanimity, and includes a process for attempting to resolve objections by interested parties, as long as all comments have been fairly considered, each objector is advised of the dis position of his or her objection(s) and the reasons why, and the consensus body members are given an opportunity to change their votes after reviewing the comments.
b. Other types of standards, which are distinct from voluntary consensus standards, are th e following:
(1) "Non

- consensus standards," "Industry standards," "Company standards," or "de facto standards," which are developed in the private sector but not in the full consensus process. (2) "Government

- unique standards," which are developed by the government for its own uses. (3) Standards mandated by law, such as those contained in the United States Pharmacopeia and the National Formulary, as referenced in 21 U.S.C. 351.9 

# 3. Impacts of Performance

- Based Regulation

The FA A directed this research to investigate the economic and safety impacts of PBR that have been experienced in other contexts. Unfortunately, after a careful and thorough review of the available literature, we found only minimal empirical research related to measured impacts from real world applications of PBR.

In 2002 , the Regulatory Policy Program at the Center of Business and Government at Harvard University’s Kennedy School of Government convened a workshop to “see what could be learned from agencies’ experience w ith performance standards and to begin to identify the likely conditions for the effective use of performance standards.” Representatives from se veral U.S. regulatory agencies as well

as several academics participated. The report on the proceedings of that workshop was very useful resource for this paper and it is cited in several places. However, an important conclusion from that workshop is that there has been little empirical ev idence of the effectiveness of PBR works in practice [2]

.
More recently, in the wake of the Deepwater Horizon oil spill, The Department of Labor, Occupational Safety and Health Administration (OSHA); Department of Interior, Bureau of Safet y and Environmental Enforcement (BSEE); Department of Homeland Security, United States Coast Guard (USCG); Environmental Protection Agency (EPA); and Department of Transportation, Pipeline and Hazardous Materials Safety Administration (PHMSA) co

- sponsored a stakeholder meeting on “ the use and implementation of performance

- based regulatory models for enhanced safety and environmental performance in the United States oil and gas industry [5] .” The materials discussed at this more recent meeting did not provide new empirical information related to assessing the effectiveness PBR.

Nonetheless, this document provides an overview of the conceptual literature related to im pacts of PBR and
, where available
, qualitative discussion of the impacts from instances where PBR has been implemented.



## 3.1 Cost Savings and Efficiency

The FAA asked whether PBR would result in lower product cost to the consumer . The literature d oes not address that specific question but is practically unanimous in the assertion that PBR would result in lower production costs, i.e. higher production efficiency. The basis for that assertion is that firms will naturally seek cost savings efficiencie s in an effort to maximize profits. CAC regulations impede firms from adopting least cost processes
, but PBR affords the flexibility to meet regulatory goals using a least cost option that is tailored to their own circumstances

.

• “[ An advantage of PBR is ] that it uses market forces to encourage firms to find low

- cost10 solutions to meet a given standards [1] .”

• “By focusing on outcomes... performance standards give fir ms flexibility to seek the lowest

- cost means for them to achieve the stated

- level of performance [2] .”

• “Setting performance standards and allowing choice of produ ction methods and, over time, innovation to meet standards should allow greater efficiency in meeting a particular public health goal [11] .”

• “Performance standards leave firms free to choose or invent least

- cost solutions to given regulatory objectives. They foster innovation , impede competition less, and can produce more flexible, results

- oriented policy than design standards [4] .”

• “ The performance

- based approach is appealing because it holds the prospects for increased flexibility in how regulated entities meet regulatory objectives. This, in turn, may promote more
cost

- effective and innovative solutions. Because particular methods or technologies are not prescribed, performance

- based approaches provide a more level playing field in the choice of products or technologies. This means particular suppliers or producers are not favored over others [12] .”

• “ Industry is constantly searching for ways of cutting costs and creating extra revenue. It is often industry that is best suited to identify the most cost effective solutions using innovative technologies, best practices and Commercial

- off

- the Shelf (COTS) components. The opportunities afforded by the introduction of performance based regulation can help to satisfy th ese needs. The freedom and choice passed to industry by identifying ‘what’ is required and not ‘how’ it is to be achieved should benefit all parties. The supplier is to be given the freedom to choose what standards they will follow in developing solutions to meet the requirements of the regulator. In turn they must be willing to shoulder some of the risk and responsibilities associated with the operation of their delivered product in service [13]

. ”

• “With design standards there is little incentive for industry to develop a cheaper method to achieve a regulatory goal in question, because the new approach would not be consistent with the specified design. Design standards ignore the particular circumstances of individual firms or processes. At best, this is costly to firms; at worst, it makes compliance impossible [4]

. ”

• “ As we have achieved success, we have learned a great deal about the limitations of ‘command

and

- control.’ Prescriptive regulations can be inflexible, resulting in costly actions that defy common sense by requiring greater costs for smaller returns. This ap proach can discourage technological innovation that can lower the costs of regulation or achieve environmental benefits beyond compliance [14] .”

Circling back to F AA’s original question, w hether those production cost savings resulting from a change to PBR are passed along to consumers would be determined by a variety of other factors related to the11 market for small general aviation aircraft such as amount of competi tion among suppliers to the
in dustry and the availability of substitute products

. Regardless of whether those cost savings are passed on to the consumer, doing the same work with fewer resources creates value for society.



### 3.1.1 Magnitude of the cost savings

Whil e the literature was unanimous that cost savings should be expected from PBR relative to prescriptive standards, there is only very limited information available regarding what the magnitude of those cost savings might be.

Figure 2 shows a comparison of CAC costs compared to least cost solution for a variety of air pollution control ini ti atives. The table shows that the cost of CAC solution in the available studies ranges from being seven percent higher than the least cost solution to 22 times the least cost solution [15]

. However, it must be noted that the studies underlying the cost estimates reported in the table are all from 1984 or earlier and the least cost solution may not be related to PBR, but instead might be a form of market

based regulation or a nother type of policy instrument

. For instance, the 198 3 study related airport noise control compared the costs associated with mandatory retrofitting of airplane engines with quieter technology to the use of noise charges. The table below is provided as c ontext as to the magnitude of what the cost savings might be related to a move away from prescriptive standards but don’t necessarily map to the cost savings resulting from PBR.

Figure 2

. Empirical Studies of Air Pollution Control Costs

Pollutants covered Year of Study Ratio of CAC Cost to Least Cost

Sulphates standards 1982 1.07
Airport Noise 1983 1.72
Sulphur Dioxide 1984 1.78

CFC emissions from non

- aerosol applications 1980 1.96
Hydrocarbons 1984 4.15

Particulates 1984 4.18
Sulphur Dioxide 1981 4.25
Nitrogen Dioxide Regulations 1983 5.96

Particulates 1974 6.00

Particulates 1984 22.00

Another piece of information related to the costs of overly prescriptive regulations comes from a 1998 survey conducted by the National Association of Home Builders (NAHB). That survey found that about12 ten percent of the cost of building a typical new home can be attributed to “ unnecessary regulation and regulatory delays, fees associated with building, plumbing, electrical, and tree removal permits, disposal of construction wastes [16] .” This example may be interpreted as an upper bound on the potential cost savings due to streamlining the aircraft certification process under part 23 . First, it is derived from survey data based on a builder’s interpretation of “unnecessary ” which may not be an accurate accounting of which regulations are actually necessary. Second, the ten percent estimate includes the costs of permits which are probably used to cover the costs of site visits by inspectors . The cost of site visits by inspectors is generally incurred for each housing unit built whereas for p art 23 the aircraft design would undergo one airworthiness review for the design and then unlimited numbers of units could be built without additional in spections related to the design. Therefore, the cost of the initial review could be spread out over many aircraft

.

Another cost savings estimate comes from a case study discussing the move by the Department of Health and Human Services to change to a perf ormance approach to hospital fire safety . The change was estimated to save “ one half of the costs of compliance [4]

. ” The estimate is potentially useful because it is derived from an instance where the regulating agency changed from prescriptive regulation to performance standards. In addition, setting fire safety standards for the design of a hospital is roughly analogous to the setting safety standards for the des ign of an airplane. They both have multiple systems that interact in order to both prevent and mitigate safety problems

. The hospital fire safety systems include sprinkler systems, fire extinguishers, fire walls, emergency exits, fire retardant materials, etc. Airplanes have navigation systems, power systems, braking systems, etc. However, it i s not clear what “the cost of compliance” refers to. Second, this estimate comes from an overview document which is over 30 years old and the basis for estimate is no t provided. This estimate of cost savings may not be grounded in empirical evidence

.



## 3.2 Innovation and Safety Improvements

Promoting innovation is often cited a reason to adopt PBR. As is described in Section 3.1 , m uch of the discussion of the i nnovation benefits of PBR pertain to innovations related to cost savings . However, a move to PBR may have impacts on other types of innovation as well. This se ction discusses the potential impact of PBR on innovation of new product features de sired by consumers , particularly innovation related to new safety features.

The literature reviewed as part of this effort did not contain any quantitative assessment s of safety impacts. At the OSHA sponsore d meeting regarding Performance

- Based Regulations in the Oil and Gas Industry, Dr. M. Sam Mannan discussed his research on the pros and cons of performance

- based regulatory models . H e found that he was not able to answer the question of whether prescriptive or performance

- based regulations produced better safety outcomes. He states that the reason for knowledge gap is that no one is tracking the right data [5]

.13 

### 3.2.1 Removing Impediments to Meeting Consumer Demands

A significant concern related to CAC design standards is that “they fail to provide long

- term dynamic incentives that induce innovative behavior [1] .” CAC design standards may indeed be state

- of

- art when they are developed but overtime they act to “fossilize” that stage of technological innovate and impede the further improvements.

• “A s ignificant advantage of performance standards

– particularly over the long run

– is that they enhance innovation. A shift to performance standards can release the imagination of those closest to the ultimate regulatory impacts

– e.g., plant engineers

– and put private ingenuity to work to find more cost effective solutions to public policy problems [4] .”

• “Design standards can constrain creativity, reduce the feasibl e set of solutions to a problem, and freeze technology. Performance standards remove these constraints [4] .”

P rescriptive approach es to safety are also criticized because they are slow to respond to new technology :

• “Best practices identified in prescriptive standards are often out of date with those required to support evolving technologies. This can restrict the technologies available for use and could prevent ind ustry from adopting best practices available at the time [13] .”

• “Performance standards can also accommodate technological change and the emergence of new hazards in ways that prescriptive technology

- based standards generally cannot [2]

. ”

In addition, prescriptive regulations tend to be responses to past experience and as such may not anticipate new dangers or risks associated with an evolving industry [13]

. 2

However, performance

- based standards will not by themselves spur innovation and progress toward new safety features. PBR will not necessarily provide an incentive to go beyond the standard set by the regulatory agency [2]

. Therefore, t he a dvantage of PBR regarding fostering innovation related to safety and ot her product feat ures is that the flexibility aff orded by PBR will not impede the adoption of the
innovations.

In contrast, a disadvantage of design standards is that by prescribing a set design, the regulator in effect disallows better technologies from being used instead. In the case of consumer goods, manufacturers may more easily meet latent consumer demand for even higher s tandards in the areas of say, safety, and not be hampered by the prescriptive regulations of the regulator. That is, in contrast to CAC, PBR

2 Some of the prescriptive rules that currently exist in part 23 are based on lessons learned from accident history and those prescriptive rules have been effective in avoiding repetitive accidents. If the industry standards and other accepted means of compliance do not capture and maintain the corrective practices in these lessons learned, new airplane developers may not be aware of them

. Therefore it is important to ensur e that the industry standards capture the lessons learned and corrective practices embodied in the current prescriptive rules.14 can more easily accommodate technological innovation [2]

. Therefore the answer to whether the change to PBR would result in new safety features or other innovative features being introduced into the market depends on whether consumers in the small general aviation aircraft mark et desire and would pay for additional safety features or other technological innovations

.
Although current part 23 rules don’t explicitly disallow new safety technologies such as angle of attack indicators, terrain displays, or touchscreen controlled devi ces , the current rules discourage their use because they require a special condition action or equivalent level of safety determination which typically take a number of month to complete which in turn may result in certification delays . The process for an equivalent level of safety determination also involves a high degree of uncertainty as to whether the new design proposed by the manufacturer would be found acceptable by FAA. The FAA knows of one recent case where a manufacturer decided against incorporat ing a new safety feature precisely because of the uncertainty around the equivalent level of safety determination and the understanding that including the new feature would have added approximately six months to the project. Breyer notes this issue as well :

• “. . . design standards limit the firm’s flexibility. A firm that finds a cheaper or more effective way of achieving the regulation’s objective must undertake the heavy burden of forcing a change in the standard before it can use its new method. For the same reason, a design standard tends to freeze existing technology and to favor those firms already equipped with that technology over potentially innovative new competitors. A performance standard permits flexibility and change. It is directly addressed t o the problem that must be solved. And since the agency must, in any event, consider the comparative performance of different machines in order to write a design standard, it may be easy for the agency to write its standard directly in terms of performance goals such as cleaner air or fewer injuries [17] ”

By clearly stating the performance standards that will be used to judge the airworthiness of proposed aircraft design, the part 23 re organization can be expected to reduce the uncertainty associated with introducing new designs. A logical consequence of ma king it easier to introduce new designs should be

an increase in the amount of new designs introduced into the market place. T he use of industry consensus standards may also be instrumental in reducing the cost of introducing new designs. Section s 2.3 and 4.1 contain more discussion on the cost implications of the use of cons ensus standards.



### 3.2.2 Spurring Technology Diffusion

There is a distinction as to whether a certain regulatory scheme spurs radical new innovation or whether it causes the wider diffusion of existing innovation. Many discussions in the literature do not differen tiate between the two concepts but both issues are of interest related to the FAA’s questions for this report.

If the regulating agency wishes to promote diffusion of certain technologies, PBR is capable of serving that function. By carefully crafting per formance based standards and by tightening those requirements15 over time as new technology becomes available, PBR would be able to speed the diffusion of technological improvements.

The Consumer Product Safety Commission (CPSC) is responsible for regulating the safety of any consumer product not specifically assigned to another federal agency. Its authorizing legislation requires the CPSC to use performance

- based standards for its regulations. For example, in its regulations related to making medicine bottle s “child

- proof”, the performance measure is a test in which children actually try to open the bottles after passively watching an adult open a bottle (i.e., no special instructions are provided to the children). A product passes the test if fewer than 20 percent of four year old children can open the bottle during the test [4]

.
However, we did not find evidence that the CPSC typically requires safety standards high er than those available in the market today. Presumably, it would be politically difficult to require something beyond what is currently achievable. However, the CPSC potentially has the power to spur increased diffusion of existing technology through its performance standards. For instance, the CPSC is currently considering requiring for newly manufactured table saws a system that detects contact between a person and a saw blade and automatically stops the saw blade. Such a requirement is only being consid ered because such a system has already been developed. The “SawStop” system is a patented automatic braking system that using measurements of electrical conductivity of the saw blade immediately stops the table saws if the blade comes in contact with the human skin (contact with the human body changes the conductivity of the saw blade
, so when a change is detected the saw blade is stopped). The CSPC gave a Safety Commendation Award to SawStop in 2001. This example shows how through the use of performance

s tandards and by raising those performance standards in response to safety innovations, the CPSC has the potential to increase diffusion of a safety innovation.

Another example of how performance

- based regulations can stimulate technology diffusion is the
C orporate Average Fuel Economy (CAFE) performance standards. These standards call for automobile manufacturers to improve the average fuel economies of their fleet to prescribed target. The standards are developed based on the technologies that are currentl y available today or are in advance d stages of development such that they would be available very soon
, and the levels are set such that existing technologies must be adopted more widely throughout a manufacturer’s model fleet. The benefit of stating the s tandard and not the exact technologies is that the manufacturer can mix and match available technologies to reduce costs and meet heterogeneous customer needs and preferences.

As a result, the CAFE standards certainly spurred diffusion of technological innovations in the automobile sector, including downsizing gasoline engines, more frequent use of turbo boosting, more diesel

- powered light duty vehicles, the development of more hybrid electric vehicles and battery electric vehicles, and the investment in hydrogen fuel cell vehicles [18]

.
Even though the CAFE standards are developed such that they can be met using existing technologies, there is also evidence that they have spurred radical innovation as well. A representative of the steelmaki n g industry states that the CAFE standards forced them to develop high strength but lighter

weight steel products to meet the standards.16 • "The (Corporate Average Fuel Economy) s tandards were a kick in the teeth

. We've exponentially increased the rate at which we're developing new products"

- John Farris, vice president and general manager of Nucor Steel

- Texas [19]

.



### 3.2.3 Creating Radical Innovation

In some cases the structur e of the regulations can incentivize firms to innovate in order to achieve results even better than the standards developed by the regulator. However, the most compelling

exa mples relate to instances of market

- based environmental regulation such as tradable permits for emissions or pricing/taxing schemes related for emissions [20]

. 3 M arket

- based mechanisms provide incentives for regulated entities to continually improve their performance (beyond any government set amount) because there are market incentives to do so. For example, prior to the Clean Air Act of 1990 new coal plants were required to use scrubbers which were the best available technology at the time. During this period patents awarded to electric power utilities generally related to reducing the operating costs of scrubbers. The 1990 Clean Air Act enacted a market

- based tra ding regime with tradable permits. Electric utilities that produced fewer emissions than they were allowed by their permits could sell those additional permits and thereby generate revenue. The patents awarded to electric power utilities in the subsequent period continued to relate to reducing operating costs but also related to improving the performance of the scrubbers [21]

.
In the part 23 context, the regulator d oes not need to create artificial market

- based incentives to produce superior outcomes because the safety benefits or other innovative benefits can be assessed and valued in the actual market. Therefore, if there is latent consumer demand for even higher l evels of safety for part 23, the part 23 reorganization’s use of performance standards should allow manufacturers to fulfill that latent consumer demand. In fact, there is evidence of consumer demand for new safety features in the light sport aircraft (LSA ) category. New safety features found in LSA include parachutes, auto pilot, and air bags, among others.



## 3.3 Industry Revitalization

The available literature does not directly comment on the likelihood that streamlining regulations or adopting PBR would revita lize an industry. However, the literature does provide commentary on some complementary impacts.

3 Kemp and Pontoglio, while acknowledging that theoretical literature sup ports the idea that market

- based regulation should be the preferred regulatory instrument for spurring eco

- innovation, find through empirical research that market

- based environmental regulatory schemes produce only marginal innovations and that traditional regulation has been responsible for radical innovation in some environmental contexts. Kemp and Pontoglio also emphasis the importance of the specific regulatory context and thus applicability of this finding to the part 23 context is not clear [20]

.17 A few authors state that the use of design standards can inhibit competition in a market. For example, a guidance document on the use of performance standards notes the following: “Performance standards are more compatible with competition. Design standards may inhibit competition and effectively serve as a barrier to entry into the marketplace for firms that do not possess the particular technology required by a design standard. Performance standards avoid these anticompetitive effects [4] .” For example, some local building codes prevented using plastic pipes for water sy stems in residential buildings. As a consequence plastic pipe manufacturers complained that the codes prevented them from competing even though their products performed just as well. More competition in a market benefit s consumers because the competition i ncreases pressure to provide lower prices and improved products. When competition in an industry is limited, the small number of manufacturers may restrict supply in order to keep prices high. Therefore opening the markets to more competition may cause the industry

to grow.

As discussed in Sections 3.1 and 3.2 , PBR can facili tate cost

- saving efficiencies and remove barriers to new product innovations. To the extent that products can be offered at lower prices and with new features , it can also be expected to stimulate demand for the product and thereby help to revitalize the
i ndustry.

Finally, a literature review of the macro

- level economic impacts of regulatory policy concluded that there is evidence that poorly designed regulations can stifle economic activities and ultimately reduce economic growth. The literature review als o finds that regulation can reduce the entry of new firms into markets. Therefore, based on the empirical studies reviewed, one could conclude that better crafted regulations would lead to economic stimulation and growth. However, the authors of the literature review emphasi ze that the effects of regulation are context specific [22]

.



## 3.4 Limiting Factors

Many authors warn that PBR may not be appropriate in all cont exts. In addition, it can be difficult to craft the standards themselves in a way that provides enough specificity for them to be adequately enforced and ensure compliance. As mentioned in Section 2.2 , loosely specified standards using terms like “sufficient” or “reasonable” are difficult to interpret and enforce and as a result the safety objectives of the standards may not be met.

The magnitude of the cost savings , innovation, and/or revitalization will depend on whether the PBR standards truly allow for freedom and flexibility. For instance, as discussed in 2.2 , the closer the performance standards are to the ultimate goal of the regulation (safety) the more freedom and flexibility they will provide. In contrast, where a performance standard is narrowly defined, such as specifying the performance of a pump in an industrial process, the performance standard may actually offer very little flexibility to the regulated firm [2] .In relation to the part 23 reorganization, a n aircraft perfor mance standard could be that the seats and seatbelts in the plane be able withstand a certain G

force as an crash

- worthiness factor. But such a factor may be limiting if the same level of crash

-18 worthiness protection could be accomplished by an entirely dif ferent means, such as use of an airbag.

S ome note that PBR may impose higher costs on small business which may find it easier to comply if they are simply told what processes to follow rather than searching for a way to meet standards [2] . To compensate for this possibility, in some cases, “non

- binding codes of practices” have been developed by government, or industry associations to provide practical guidance on how to meet the standards. The part 23 reorganization effort intends to pair performance standards with direction on means of compliance provided by consensus standards developed by ASTM. Some authors warn that such “codes of practice” may in effect act as pr escriptive standards that would erode the potential benefits of the
PBR.

W hile the costs to the regulated entities are generally thought be lower under PBR, the costs to the regulatory agency may be higher to develop and enforce the PBR [23] .However, there may be agency costs savings in the form of less time and resources needed to provide variances and exemptions to design standards [4]

.
Grandfathering is another issue that may limit the impacts of the change to PBR. Grandfathering is intended to eas e the burden of transitioning to new regulations for industry and manufacturers. Grandfathering provisions exempt certain pre

- reform era members from select parts, or in some circumstances all, of the new qualifications or standards. Grandfathering provisio ns tend to have provisions that state an entity can continue to operate under the previous regulatory regime until such time as the entity does an upgrade. For example, changes in building codes don’t usually require immediate compliance of all existing bu ildings but when building undergoes a significant rebuild or remodel, the entire building must be brought “up to code.” The unintended consequence of such grandfathering provisions is that they act to delay and postpone those upgrades by adding additional cost to the upgrades. As a result they slow down adoption of new technologies beyond the ones that are subject to the regulations. To continue the building code example, the additional costs of bringing building “up to code” not only delay the adoption of the electrical or fire safety regulations in the new building code but also delay the adoption of more energy efficient building materials as well because of the second order effects.

The issue of grandfathering in the US aviation regulations is not unique to the part 23 reorganization . An aircraft model that receives an airworthiness certificate can continue to be manufactured even if the regulations change over time. Small change s to the initial design can be approved in isolation but if there are signifi cant number of changes made to the model the entire design will require a complete costly and time consuming airworthiness certification process. Therefor e even with the additional flexibility allowed by PBR, some manufacturers may wish to continue produci ng their existing models because developing a new model with innovative new features would still incur some amount of cost due to review by FAA.19 

# 4. Industry Consensus Standards

The literature related to the use of industry consensus standards as a form of safety regulation is not as well developed as the literature related to PBR. However, there is at least one very relevant example that we rely on as a case study related to consensus standards: the relatively new Light Sport Aircraft (LSA) c ategory.

In 2 004, The Federal Aviation Administration created a new segment in the general aviation industry for sport pilots, light sport aircraft, and light sport aircraft repairmen under FAA Order 8130.2G

. The rule includes a special airworthiness certificate that i s issued to an aircraft that meets the def inition of LSA

. 4

T o receive the special airworthiness certificate , the manufacturer must provide a statement of the aircraft’s compliance with an identified consensus standard

. In addition, t he rule has provisions for obtaining sport pilot certifications, flight instructor certifications, and repairman certifications. One of

t he most notable feature s of the rule allows sport pilots to utilize a U.S. driver’s license in lieu of a medic al certificate, an attempt to allow more pilots in the industry t o fly a wider range of aircraft. This feature was aimed at pilots who are otherwise healthy, but who choose not to renew their third class medical certificate (perhaps because of the expense) , to be able to continue flying in LSA’s. However, if a pilot has previously applied for a medical certificate and was turned down, the FAA forbids that pilot from becoming a Sport Pilot indefinitely. Furthermore, Sport Pilots cannot fly a t night, on instruments, for pro fit , and they are limited on the types of controlled airspace they may enter.

The experience regarding LSA is informative to the part 23 reorganization because both involve the use of consensus standards. However, as noted above the LSA rulemaking involves several additional aspects so the results of may not be completely applicable to the part 23 reorganization

.

4 1 4 CFR Part 1.1 defines light sport aircraft to be any aircraft, other than a helicopter or powered

- lift that, since its original certification, has continued to meet the following: (1) A maximum takeoff weight of not more than

— (i) 1,320 pounds (600 kilogr ams) for aircraft not intended for operation on water; or (ii) 1,430 pounds (650 kilograms) for an aircraft intended for operation on water. (2) A maximum airspeed in level flight with maximum continuous power (VH) of not more than 120 knots CAS under
stan dard atmospheric conditions at sea level. (3) A maximum never

- exceed speed (VNE) of not more than 120 knots CAS for a glider. (4) A maximum stalling speed or minimum steady flight speed without the use of lift

- enhancing devices (VS1) of not more than 45 kn ots CAS at the aircraft's maximum certificated takeoff weight and most critical center of gravity. (5) A maximum seating capacity of no more than two persons, including the pilot. (6) A single, reciprocating engine, if powered. (7) A fixed or ground

- adjust able propeller if a powered aircraft other than a powered glider. (8) A fixed or feathering propeller system if a powered glider. (9) A fixed

- pitch, semi

- rigid, teetering, two

- blade rotor system, if a gyroplane. (10) A non

- pressurized cabin, if equipped wi th a cabin. (11) Fixed landing gear, except for an aircraft intended for operation on water or a glider (12) Fixed or retractable landing gear, or a hull, for an aircraft intended for operation on water. (13) Fixed or retractable landing gear for a glider.20 

## 4.1 Innovation

The substitution of consensus standards for formal FAA review has a pparently reduced the cost of introducing new designs . AST M International is the body responsible for developing the LSA consensus standards . In a paper available on the ASTM website, ASTM touts the success of its LSA consensus standards. The authors estimate the approval process to cost $10,000 to $20,000 and ma ke a
compar ison to $20 million e stimated cost to certify a new aircraft model under part 21 [24]

. The comparison to a part 21 certification is perhaps a stretch giv en the great disparity in size and complexity of the two types of aircraft, but the larger point that the LSA certification costs are quite low still stands.

S everal observers of the new LSA segment have stated that the newly created category resulted in many new aircraft designs and innovations :

• “There’s no argument that LSA has ignited a bushel of new design s [25] ”

• “In terms of sales, the segment is a success. According to a presentation given by the FAA at the Friedrichshafen (Germany) airshow, the total LSA fleet size is now nearing 10,000 airplanes . Of those, around 6,700 are experimental

- LSAs and around 1,300 are special

- LSAs. There are more than 100 approved LSA designs, a number that might not decrease over time, since the cost of admission to the industry is very low. And there seems to be an

en thusiastic market for LSAs, much more so than many observers predicted [26] .”

• One decade into the new SP/LSA rule, “a large number of new LSA models have arrived o n the market [27] .”

• “Many customers chose to equip their LSA with loads of features. No one anticipated autopilots on LSA in 2003, yet today most of the higher en d models are so equipped. When something like an autopilot

– or digital glass panels or angle of attack indicators or airframe parachute systems, and more

– can be added to an LSA without the high cost of conventional certification, these improvements prol iferate [27] .”

In 2008, ASTM reported that 76 new aircraft designs were developed as a result of the ASTM design standards [24]

. Currently there are over 130 approved LSA designs suggesting that over the last six years, on average nine new designs per year have been introduced

. 5



## 4.2 Product Costs

Despite the observation that th ere has been impressive number of new designs created for the LSA segment, there is evidence that some manufacturers believe that the 1,320 pound maximum weight

5 See http://www.faa.gov/aircraft/gen_av/light_sport/media/ExistingModels.pdf for a list of the current models.21 regulation is impractical for light sport aircraft. Piper and Cirrus originally entered the mar ket with rebranded European models but then quickly exited. Cessna developed a brand new model, the Skycatcher. The model had some design issues during testing phase, and the initial design was then modified to correct it. Cessna abruptly pulled the Skycat cher in 2013 [29]

.
One reason pointed to for the big American manufacture r s pulling out of the LSA segment is the difficult y of designing aircraft under 1 ,320 pounds without being very costly

. However, t he st rict weight and size requirements on these aircraft have resulted in LSAs becoming expensive to develop for the manufacturers, and consequently they are often costly for consumers as well [29]

. The original price for the Skycatcher was $110,000 but then increased to $150,000 and the price of other LSA models has also increased “dramatically” overtime [29]

. ASTM reports that the price of typical LSA is $130,000 which is considerably cheaper than the retail price of $240,000 for a part 23 GA aircraft. However, that price tag is higher than people originally estimated for new LSA segment. ASTM by spreading the initial cost of the certification over the number of units built, estimates that only about $1,000 of the LSA price is due to cost of the approval process [24]

.



## 4.3 Safety

Early research from 2009 into the safety of LSAs found that one out of every 100 special LSAs had been involved a fatal accident and that the fatal accident rate per 100,000 hours flown was just over 5, twice the rate for personal part 23 aircraft [26]

.

In a more recent 2012 article for The Aviation Consumer
, Paul Bertorelli calculates various LSA accident rates for the top 10 LSA manufacturers for the period 2005 to 2012 [25] . This sample selection yields 1,440 aircraft and 960,000 flight hours. He found that the LSA fatal accident rate of 1.4 per 100,000 flight hours is a bit higher than the GA average 1.2, but given the small sample si ze of available flight hours for LSA, the difference may not be statistically significant. He also states that the overall accident rate of LSA is substantially higher than the accident rate for GA. The GA accident rate is reported to be 6.3 per 100,000 fl ight hours. An overall average LSA accident rate is not provided, but accident rates for each LSA manufacturer are provided showing that only Cessna and Jabiru had accident rates lower than the GA accident rate. The other eight manufacturers had accidents exceeding the GA average, often by several fold. The majority of LSA accidents are runway loss of control (R

- LOC) which are rarely fatal. Bertolli notes that the lighter weight of the LSA aircraft makes them it more difficult to land, more susceptible to P IOs, and more easily disturbed by crosswinds and gusts.

Both reports mention that experienced pilots are the ones who are actually getting into LSA accidents

more frequently than pilots who are learning to fly in an LSA and one report provides a quote to that

e ffect from FAA safety manager Harris Hooper: “ pilots transitioning from conventional GA aircraft to LSAs are involved in more fatal LSA accidents than are newly trained sport pilots [26] .” Experienced pilots who do not want to deal with the hassle of the medical requirements are abandoning their GA22 planes and hopping into an LSA without expecting them to handle much differently. Yet it may be unreasonable to as sume that a 1,320 pound airplane will handle similarly to one that weighs 500 pounds more. Flight instructor Earl Kesser, of Carson City NV, affirms that “it takes a finer touch to fly an LSA [25] .” Jerry Eichenberger, a flight instructor who operates a school out of Ohio believes that LSA’s “take a significant amount of transition training [25] .” Many flight instructors believe that it is often easier to teach a novice how to safely fly an LSA airplane rather than teach an experienced GA pilot since the two aircraft can handle differently, due to the restrictive weight regula tions. However, there still is not enough data to conclusively determine what may be causing the differences because the LSA category is still a new one

. Bertorelli believes “it is too soon to draw blanket conclusions about LSA fleet safety in general. We simply need more flight hours.”

Therefore, it seems that observers of the LSA segment are linking higher overall accident rates to factors other than the use of consensus standards. And the available information suggests that there is little to no differe nce in fatal accident rates between LSA and GA. One observer calls the ASTM consensus standards “quite high [29] .”



## 4.4 Industry Revitalization

D espite the absence of t he major US manufacturers, there are still several European firms providing aircraft in the segment and new models are regularly introduced at trade shows. One observer states “the innovations keep coming” and the market is “thriving” even though the new s egment has “not quite become the chicken in every pot

– an airplane in every hanger

– scenario GA hoped for [29] .”



## 4.5 Caveats

In a 2010 assessment, the FAA visited 30 LSA manufacturers in the United States to conduct random evaluations. Inspectors discovered that many safety standards were being violated. Many manufacturers did not keep records of planes or aircraft owners to ensure quality and control. Additionally, th e FAA said it found irregularities in how sport aircraft companies imported aircraft parts. A report published by the FAA stated that it:

“ F ound irregularities in how U.S. firms imported aircraft or plane parts. FAA regulations require that aircraft manufactured outside the U.S. be assembled in countries that have agreed to follow U.S. standards. In some cases, all or most of the construction had been done in other nations, even when U.S. firms told the FAA that they were the manufacturer [30]

. ”

Dan Johnson of the Light Aircraft Manufacturers Association believes the FAA audits do not currently threaten the LSA industry because growing pains are exp ected when new regulations are introduced. Johnson stated that many of the violations uncovered “ involved a failure to keep up paperwork ” but that “missing a document doesn’t mean an airplane wasn’t build right, it just means you can’t prove it was built a certain way [30] .”23 

# 5. Other Experiences with PBR and Other Regulatory Reforms

This section discusses the experience of regulatory reform in a few cases where more in

- depth information was available. Keep in mind that every regulatory context is different and the experience from one industry may not be relevant to the part 23 reorganization.

In the course of our research, we observe that other researchers in this are a tend to be drawn to investigating instances where there were observed regulatory failures. The researchers tended to not find fault with PBR itself, but with crafting the standards, compliance
, or enforcement of the regulations

. Thus on e conclusion stemming from this research is that to experience the conceptual advantages discussed in this document, the PBR must be enacted appropriately to ensure compliance and the performance standards themselves must be carefully crafted.

As shown in the New Zealand building code case study below, one attribute of a well

- crafted performance standards is that they cover all relevant areas. But more pragmatically, the process of crafting performance standards may be one of trial and error. For that reaso n it is important the regulatory agency monitor the results of the rules and make refinements and adjustments to deal with shortcomings as they become apparent. For example, NHTSA has revised its performance standards related to airbags twice since they we re first create d in 1991. NHTSA observed that in some cases they airbags were doing more harm than good (especially for smaller occupants) and in 1997 altered the regulations to allow depowered airbags. In 2001 the standards were again revised to recapture increasing injuries & fatalities in high severity crashes that result ed from the depowered airbags . The newest standards call for air bags to be tested with female and child crash dummies and have features that allow for differential airbag deployment, de pending on the characteristics of the occupant.



## 5.1 HACCP in Food Safety

The Hazard Analysis Control Point (HACCP) system for food safety involves establishing process control by identifying points in the production process that are most critical to monitor an d control. During the
1990s HA C CP was mandated for seafood, fruit juice, meat, and poultry. Under HACCP, the processors themselves identify critical control points (CCPs) in their own plants, assess the risks at CCPs, and conduct their own monitoring and t esting. The regulatory agency then monitor s that the plant is following the program. As such, the HACCP is a form of “management

- based” regulation [10]

. However,
U SDA also utilizes a performance

- based standard for allowable levels of S almonella and the USDA conducts the testing itself [23]

. Testing for E. Coli is somewhat different. The standards established are not enforceable as regulatory standards because a scientific basis for those standards has not been established and th e plant conducts the testing while the inspectors monitor that the plants are actually conducting the testing as part of the HACCP plan [23]

.24 Because the HACCP invo lves continuous monitoring a continuous process
, and airworthiness certification is a one

- time approval process, the HACCP experience may be of limited usefulness. However, this regulatory context involves public safety and utilizes PBR, so the experience of food safety regulation is perhaps worth examining

.
The literature regarding HACCP and PBR in food safety suggest that the industry prefers the flexibility afforded by HACCP and would not like to return the previous “poke and sniff” regime [23]

.
Unnevehr and Jensen provide a discussion of the economics of HACCP [11] . They n ote that HACCP offers firms a great of flexibility to develop processes to fit circumstances , and that it reduces costs of regulatory enforcement. They find that the higher upfront costs of developing a HACCP system result in economies of scale and disadva ntage small businesses and also not e that the system provides incentives for vertical integration of production to better control the entire food supply system and identify CCPs. The concern that performance

- based regulations may favor larger firms over small firms was mentioned
in the conceptual literature as well.

However, May (2004) in an overview of several PBR case studies finds that there have been “spectacular lapses in the quality of meat production leading to massive recalls since the HACCP system has been implemented [23] .”However, the author does not indicate if such problems exi sted in the previous regime or if the other changes in the industry since that time may have created those issues. The issues may also be relate d to deficiencies in compliance/enforcement . A 2002 GAO report found that USDA’s Food Safety Inspection Service (FSIS) was “not ensuring that all p lants’ HACCP plans meet regulatory requirements and, as a result, consumers may be unnecessarily exposed to unsafe foods that can cause foodborne illnesses [31] .”



## 5.2 New Zealand Leaky Building Crisis

In a series of papers, Peter May provides details of the so called “Leaky Building Crisis” in New Zealand

.

The crisis occurred subsequent to an overhaul in the New Zealand building codes resulting from the Building Act 1991 . The Building Act incorporated PBR with broad objectives relating to protecting people, their health and safety, and the environment and sub

- objectives related to averting potential injuries, fire safety, protection fr om hazardous materials, protecting property from damage, accessibility, and energy efficiency. The Building Industry Authority (BIA) was created as a new Crown agency concerned with enacting the Act, but building regulation was delegated to local authoriti es. A building owner could either have the building certified by the local authority or by third

- party certifiers, thereby incorporating some market

- mechanisms of competitive pressure into the building inspection
services sector. The third party certifiers were to be certified by the BIA [23]

.
The “crisis” emerged due to lack of weathertightness in a particular type of exterior cladding used in condominiums and a ty pe of stucco used in high

- end residences. The problem affected up to 18,000 homes and numerous multi

- family buildings.25 However the link between the subsequent crisis and PBR is not clear. First, the same issues related to those building products appeared in the U

. S

. and Vancouver
, Canada which ha d fairly prescriptive building codes

. Therefore, one cannot claim the PBR “caused” the crisis. However, May suggests that the
pa rticulars of the PBR regime in New Zealand allowed the problem to remain unresolved for a longer period than would have bee n the case under a different regulatory regime

.

In addition, two investigations conducted into the crisis supported the basic concep ts of PBR but found that the issues related to compliance, the design of the system of procedural and technical controls, responsibility, and accountability. The Hunn Report states “The Building Act has clearly succeeded in providing the building industry with the scope to develop innovate and cheaper building solutions. However, hand

- in

- hand with the service or product provider being given the ability to determine and provide design and construction solutions must go a responsibility and accountability to guarantee their performance against the Building Code’s requirements. This has not happened [32]

. ” May observes that the lack of accountability of key players was a key factor undermining the effectiveness of the performance

- based regulatory regime [12]

.
In the wake of the crisis there was no effort to return to the building code to its pre

- PBR state. Rather the Building Act was modified to correct the issues uncovered as a result of the crisis. For instance, there was not a performance objective related to adequate provision of shelter which might have addressed the weatherti ghtness issue.

Interestingly, May also found that local authorities were hesitant to use a heavy hand in enforcing building codes because the locality feared that they might lose the development opportunity to a more lax locality. So
, the lesson for the part 23 reorganization may be that care should be taken to ensure that the desire to revitalize an industry does not undermine the safety objectives of the FAA. This sounds obvious, but the New Zealand experience suggests otherwise.26 

# 6. Conclusions

Regarding the use of PBR, this research finds the following:

• The available literature is practically unanimous in the assertion that PBR would result in lower production costs, i.e. higher production efficiency. The basis for that assertion is that firms will naturally seek cost savings efficiencies in an effort to maximize profits. Prescriptive regulations impede firms from adopting least cost processes but PBR affords the flexibility to meet regulatory goals using a least cost option that is tailored to their own circumstances.

• Performance

- based standards will not by themselves spur innovation and progress toward new safety features. PBR will not necessarily provide an incentive to go beyond the standard set by the regulatory agency. Therefore, the advan tage of PBR regarding fostering innovation related to safety and other product features is that the flexibility afforded by PBR will not impede the adoption of the innovations.

• If the regulating agency wishes to promote diffusion of certain technologies, PBR is capable of serving that function. By carefully crafting performance based standards and by tightening those requirements over time as new technology becomes available, the PBR would be able to speed the diffusion of technological improvements.

• To th e extent that design standards inhibit competition in a market, moving to performance standards may increase competition and foster growth in the market. In addition, the use PBR can facilitate the use of cost saving measures and remove barriers to innovat ion. In turn, lower prices and innovative product features can spur demand and revitalize an industry.

• Based on the experience for the LSA segment, it appears that using consensus standards as a way of streamlining the aircraft certification process has re sulted in lower costs of certification and facilitated the introduction of new and innovative designs.

Based on these findings, it appears likely that adopting PBR as a method of safety regulation for part 23 small general aviation aircraft will result in many of the positive impacts mentioned as objectives of the “Small Aircraft Revitalization Act of 2013.” However, the magnitude of the positive impacts will be determined by how much freedom a nd flexibility the new performance standards actually provide. Providing guidance on “means of compliance” through consensus standards acts will assist manufacturers in developing methods for meeting the performance standards. However, it is possible that those consensus standards may turn out to be de facto design st andards and erode the potential benefits of adopting PBR.27 

# 7. References

[1] Economic Report of the President, 2003.

[2] Coglianese, Cary, Jennifer Nash, and Todd Olmstead. “Performance

- Based Regulation: Prospects and Limitation in Health, Safety, and Environmental Protection,” Regulatory Policy Program Report No. RPP

- 03 (2002).

[3] Deighton

- Smith, Rex. 2008. “Process and performance

- based regulation: Challenges for regulatory governance and regulatory reform” in Peter Carroll, Rex Deighton

- Smith, Helen Silver, and Chris Walker (eds), Minding the Gap: Revisiting the promise and Performance of Regulatory Reform in Australia, Canberra: ANU E Press, chapter 7.

[4] Project on Alternative Regulatory Approaches, “Performance Standards: A Practical Guide to the Use of Performance St andards as a Regulatory Alternative” September 1981.

[5] Occupational Safety and Health Administration (OSHA) 2012. “Expert Forum on the use of Performance

- Based Regulatory Models in the U.S. Oil and Gas Industry, Offshore and Onshore. Action: Notice of Stake holder meeting” Docket Number OSHA

- 2012

- 0033.

[6] U.S. Nuclear Regulatory Commission, 2002. “Guidance for Performance

- Based Regulation.”

[7] Viscusi, K., 1983. “Risk by Choice: Regulating Health & Safety in the Workplace,” Cambridge, MA: Harvard University Press

[8] Q ueensland Government, Department of State Development. 2006. “Guidelines on Alternatives to Prescriptive Regulations.

[9] OMB Circular 119 (Revised), February 10, 1998

[10] Coglianese, Cary and David Lazer, 2003. “Management

- based regulation: prescribing private management to achieve public goals.” Computer Information Science Faculty Publications. Paper 3.

[11] Unnevehr, Laurian J. and Helen H. Jensen, 1998. “The Economic Implications of Using HACCP as a Food Safety Regulatory Standard,” Working Paper 99

- WP 228.

[12] May, Peter J. “Performance

– Based Regulation” Jerusalem Papers in Regulation and Governance, Working Paper No. 2. April 2010.

[13] Moore, Keith. 2005. “The Move to Performance Based Standards

– Managing Uncertainty” University of York, Safety Critical Systems Engin eering MSc Dissertation.28 [14] Clinton, Bill and Al Gore. 1995. “Reinventing Environmental Regulation,” National Performance
Review.

[15] Tietenberg, T.H., 1990. “Economic Instruments for Environmental Regulation” Oxford Review of Economic Policy, Vol 6, No. 1.

[16] U.S. House of Representatives. Committee on Small Business. Regulatory Reform Initiatives and Their Impact on Small Business. Hearing before the Committee, Serial No. 106

- 60. Washington DC: US Government Printing Office, 2000.

[17] Breyer, S. 1982. “Regulation and I ts Reform,” Harvard University Press. Quoted by David Besanko (1987) “Performance versus design standards in the regulation of pollution,” Journal of Public Economics 34. 19

- 44.

[18] Eichenberger, John. “Regulating Innovation,” NACS magazine, August 2014, pp. 5 4

- 55.

[19] Pete, Joseph S. “Fuel Efficiency Standards Have Driven Innovation in Steelmaking” NWI.com, May 10, 2014.

[20] Kemp, Rene and Serena Pontoglo, 2011. “The Innovation Effects of Environmental Policy
Instruments

– A Typical Case of the Blind Men and the Eleph ant?” Ecological Economics, 72 (2011) 28

- 36.

[21] Popp, David. 2001. “Pollution Control Innovations and the Clean Air Act of 1990,” NBER Working Paper 8593.

[22] Parker, David and Colin Kirkpatrick, “The Economic Impact of Regulatory Policy: A Literature Review of Quantitative Evidence,” OECD, Expert Paper No. 3, August 2012.

[23] May, Peter. 2004. “Performance

- based Regulation and Regulatory Regimes,” 13th World Conference on Earthquake Engineering, Vancouver, BC August 1

- 6, 2004, Paper No. 3254.

[24] Stephenson, James and D eborah Stephenson, “Light Sport Aircraft Industry Takes Flight on ASTM International Standards,” ASTM Standardization News November/December 2008.

[25] Bertorelli, Paul. “LSA Accident Survey: Low Fatals, High Overall,” The Aviation Consumer, November 2012.

[26] Goye r, Robert. “LSA Safety Picture Emerging,” Flying Magazine, August 19, 2009.

[27] Johnson, Dan, “The Next Decade of LSA Innovation,” General Aviation News, September 3,
2014.

[28] Stephenson, James and Deborah Stephenson, “Light Sport Aircraft Industry Takes Flight o n ASTM International Standards,” ASTM Standardization News November/December 200829 [29] Cipriani, Rocco. “Are LSAs Too Light Weight for American Companies?” Aviation Marketing Consulting, March 19, 2014.

[30] Levin, Alan. “Sport Plane Makers Warned by FAA to Stop Eva ding Rules”. Bloomberg News, June 29, 2012.

[31] U.S. General Accounting Office, 2002. “Meat and Poultry: Better USDA Oversight and Enforcement of Safety Rules Needed to Reduce Risk of Foodborne Illnesses.” GAO

- 02

- 902.

[32] The Hunn Report. Report of the Overview Gr oup on the Weathertightness of Buildings on the Building Industry Authority. October 2002.