GAO

United States General Accounting Office

Report to the Subcommittee on Aviation,

Committee on Transportation and

Infrastructure,

House of Representatives

June 2000 AVIATION SAFETY

Safer Skies Initiative

Has Taken Initial Steps

to Reduce Accident

Rates by 2007

GAO/RCED-00-111

Page 1 GAO/RCED-00-111 Safer Skies Initiative

Contents

Letter 5

Executive Summary 8

Principal Findings 13

Chapter 1

Background

20

FAA Expects Continued Growth in Aviation 20

Fatal Accident Rates Have Decreased for U.S. Aviation 22

The Safer Skies Initiative Continued Ongoing Efforts to Use

Data Analysis to Address Safety Problems 29

Objectives, Scope, and Methodology 32

Chapter 2

The Safer Skies

Initiative Should Help

Improve Aviation

Safety

34

The Safer Skies Initiative Addresses Major Safety Problems in

Commercial Aviation 34

Accident Data and Other Resources Were Used to Identify Safety

Problems That Caused Many Fatal Accidents in General Aviation 44

Improving Cabin Safety Is Important but Will Have

Little Impact on Lowering the Fatal Accident Rate 48

Additional Work on Future Hazards Could

Help Anticipate and Prevent Fatal Accidents 49

Conclusions 50

Recommendations 51

Agency Comments 52

Chapter 3

The Safer Skies

Initiative Has Made

Progress in Selecting

and Implementing

Interventions

53

The Safer Skies Methodology Is Based on Previous Efforts to

Identify Safety Problems 54

The Safer Skies Initiative Has Made the Most

Progress With Problems Studied Previously 57

Early Experience Indicates That Future

Problems Will Require More Analysis 61

Effective Implementation Is Critical Next Step in Making

Progress Toward the Goals Set for Reducing Fatal Accidents 61

Conclusions 68

Recommendations 68

Agency Comments 69
Contents

Page 2 GAO/RCED-00-111 Safer Skies Initiative

Chapter 4

The Safer Skies

Initiative Has Not Yet

Developed

Performance Measures

to Evaluate the

Effectiveness of Most

Interventions

71

Federal Law Requires the Development of Performance

Measures as Part of the Budget Process 71

Determining the Effectiveness of Safer Skies’ Initiatives Will

Require the Development of Additional Performance Measures 73

Most Safer Skies’ Interventions Are Being Implemented

Without Determining How to Evaluate Their Effectiveness 74

Conclusions 84

Recommendations 85

Agency Comments 85

Chapter 5

Coordination Has Been

Extensive but Needs

Improvement for the

Safer Skies Initiative to

Succeed

86

The Safer Skies Initiative Involves an Unprecedented Level

of Coordination Between Industry and Government 86

The Funding, Prioritization, and Evaluation of Safer Skies

Interventions Could Be Better Coordinated 87

Conclusions 94

Recommendations 95

Agency Comments 95

Appendixes Appendix I: GAO Contacts and Staff Acknowledgments 98

Key Aviation Studies 99

Tables Table 1: Number of Fatal Accidents and Deaths by Type of

Aviation Operation, 1988-97 10

Table 2: Fatal Accidents and Deaths by Type of Aviation

Operations, 1988-97 22

Table 3: Projected Numbers of Fatal Accidents by Type of

Operation in 2007 26

Table 4: The 16 Safety Problems Addressed by Safer Skies 31

Table 5: Numbers of Fatal Accidents and Fatalities in Commercial

Aviation 1988-97 41

Table 6: Number of Past and Projected Fatal Accidents by Type

of Aviation Operation 44
Contents

Page 3 GAO/RCED-00-111 Safer Skies Initiative

Table 7: Status of Safer Skies’ Activities for Commercial and

General Aviation 58

Table 8: Baselines and Goals for Reducing the Occurrence of

Uncontained Engine Failure in Commercial Aviation 76

Figures Figure 1: Commercial Aviation and General Aviation Flight

Hours, 1982-99 21

Figure 2: Fatal Accident Rates for Commercial and General

Aviation, 1982-99 25

Figure 3: Safety Problems That Caused Fatal Accidents in

U.S. Commercial Aviation, 1988-97 37

Figure 4: Fatalities in U.S. Commercial Aviation by Safety

Problem, 1988-97 43

Abbreviations

CFIT Controlled Flight Into Terrain

DOT Department of Transportation

FAA Federal Aviation Administration

GAO General Accounting Office

NTSB National Transportation Safety Board

PICS Partners in Cabin Safety
Page 4 GAO/RCED-00-111 Safer Skies Initiative
United States General Accounting Office

Washington, D.C. 20548

Page 5 GAO/RCED-00-111 Safer Skies Initiative

Resources, Community, and

Economic Development Division

B-283391 Letr

June 28, 2000

The Honorable John J. Duncan, Jr.

Chairman

The Honorable William O. Lipinski

Ranking Democratic Member

Subcommittee on Aviation

Committee on Transportation

and Infrastructure

House of Representatives

As requested, we are reporting on the actions taken by the Federal Aviation Administration (FAA),

through the Safer Skies initiative, towards the goal of reducing the nation’s fatal aviation accident

rates by 2007. Our report contains recommendations designed to ensure that the implementation of

interventions approved by FAA and the Safer Skies steering committees is tracked and that the

interventions are evaluated for their effectiveness in meeting the goal.

As arranged with your offices, unless you publicly announce its contents earlier, we plan no further

distribution of this report until 7 days after the date of this letter. At that time, we will send copies to o

the appropriate congressional committees; the Honorable Rodney E. Slater, Secretary of

Transportation; the Honorable Jane F. Garvey, Administrator, FAA; and the Honorable Jacob Lew,

Director, Office of Management and Budget. We will also make copies available to others upon

request.
B-283391

Page 6 GAO/RCED-00-111 Safer Skies Initiative

If you have any questions about this report please contact me or Robert White at (202) 512-2834.

Other key contributors to this report are listed in appendix I.

Sincerely yours,

Gerald L. Dillingham, Ph.D.

Associate Director, Transportation

Issues
Page 7 GAO/RCED-00-111 Safer Skies Initiative
Page 8 GAO/RCED-00-111 Safer Skies Initiative

Executive Summary

Purpose The continued growth forecast for U.S. aviation in the coming decade will

likely bring a rise in fatal accidents if the current accident rate is not

reduced.1 Commercial aviation, used by most Americans when they fly,

experienced an average of 6 fatal accidents a year in the United States in

1994-96; general aviation experienced an average of 380 a year.2 If the

projected growth in flight hours occurs and the fatal accident rate is not

reduced, GAO estimates in this report that the number of fatal commercial

aviation accidents could rise to 9 per year and the number of fatal general

aviation accidents to 484 by 2007. The Federal Aviation Administration

(FAA), the Congress, and the aviation industry have acknowledged this

potential danger and have recommended ways to address it. In 1997, two

major commissions on aviation safety recommended reducing the nation’s

aviation accident rate by 80 percent by 2007. To meet this challenging goal,

both the White House Commission on Aviation Safety and Security and the

congressionally mandated National Civil Aviation Review Commission

recommended that FAA and the aviation industry work together to identify

and address the causes of fatal accidents. To unify government and

industry efforts to reduce the accident rate by addressing the greatest

threats to aviation safety, FAA announced the Safer Skies initiative in April

1998 with the broad initial goal of reducing the number of fatal accidents

per million flight hours by 80 percent by 2007.

Aviation experts from FAA, the aviation industry, and other government

agencies formed three steering committees to oversee the initiative’s work

in three broad areas: commercial aviation, general aviation, and cabin

safety. The steering committees will analyze data to identify the most

serious threats to safety, to find the root causes of accidents, and then to

determine the best actions to break the chain of events that lead to

accidents. Resources will be directed first to implementing those best

actions, referred to as interventions. The steering committees have

identified 16 specific safety problems—6 related to commercial aviation, 6

to general aviation, and 4 to cabin safety. These problems will be addressed

1

The fatal accident rate is calculated by dividing the number of fatal accidents by a measure

of aviation activity such as the number of aircraft hours flown.

2

Commercial aviation includes both large air carrier operations and smaller commuter

operations. General aviation includes a wide variety of aircraft, ranging from corporate jets

to small piston-engine aircraft as well as helicopters, gliders, and aircraft used in operations

such as firefighting and agricultural spraying. In establishing accident reduction goals, FAA

and Safer Skies steering committees used 1994-96 as the baseline years for commercial

aviation and 1996-98 as the baseline years for general aviation.
Executive Summary

Page 9 GAO/RCED-00-111 Safer Skies Initiative

by teams of aviation experts who can recommend one or many

interventions for the safety problems they are addressing. Some of the

safety problems, such as weather, will be addressed by both commercial

aviation and general aviation teams because their causes and interventions

may differ for these types of operations.

In light of the critical importance of the Safer Skies initiative in improving

aviation safety, the Chairman and Ranking Democratic Member of the

Subcommittee on Aviation, House Committee on Transportation and

Infrastructure, asked GAO to review the implementation of this initiative.

Specifically, they asked GAO to determine (1) to what extent addressing

the safety problems selected by the Safer Skies initiative will help reduce

the fatal accident rate; (2) what progress the initiative has made in

identifying and implementing interventions to address each of these safety

problems; (3) what progress has been made in assessing the effectiveness

of those interventions; and (4) how FAA is coordinating the Safer Skies

initiative with other safety activities conducted throughout the agency, in

partnership with the aviation industry, and by other federal agencies.

Background Since 1982, air travel in the United States has increased dramatically, and

flight safety has improved. The number of hours flown by commercial

aircraft more than doubled from 8 million hours in 1982 to nearly 18 million

hours in 1999. FAA estimates that commercial aviation aircraft will fly more

than 24 million hours in 2007, an increase of 37 percent from 1999. Growth

in general aviation has been less consistent, but FAA estimates that general

aviation flight hours will increase to about 36 million hours in 2007, a

growth of nearly 19 percent over 1999. Although the accident rates for both

types of operations are low, both the number and the frequency of aviation

deaths will likely increase if these rates are not reduced as the growth in air

travel continues. In the 10-year period from 1988 through 1997, the United

States had 4,471 fatal aviation accidents that resulted in a total of 9,802

deaths. Commercial aviation accounted for only 2 percent of the fatal

accidents, while general aviation accounted for 98 percent. (See table 1.)
Executive Summary

Page 10 GAO/RCED-00-111 Safer Skies Initiative

Table 1: Number of Fatal Accidents and Deaths by Type of Aviation Operation,

1988-97

Source: GAO’s analysis of data from the National Transportation Safety Board.

Results in Brief The Safer Skies initiative addresses the safety problems that have

contributed to fatal accidents in the past, and in conjunction with other

safety programs, it can be expected to reduce the fatal accident rate and

thus enhance the safety of the nation’s air passengers. In commercial

aviation, the initiative addresses safety problems that accounted for over

three-quarters of the fatal accidents in those operations in 1988-97. Other

FAA initiatives are addressing additional safety problems, which should

complement Safer Skies’ efforts to meet the goal of an 80-percent reduction

in the fatal accident rate for this segment of the aviation industry. In

general aviation, the Safer Skies initiative plans to address safety problems

that appear to be the most common causes of fatal accidents. The initiative

has adopted a less aggressive goal in general aviation of reducing the

number of fatal accidents to 350 in 2007, which represents about a 20-

percent reduction. Finally, the initiative addressed four safety problems in

cabin safety. Improving cabin safety will have little impact on lowering the

fatal accident rate because cabin safety accounted for only two U.S.

commercial aviation fatalities in 1988-97. No quantitative goal was set for

safety improvements in cabin safety. To date, safety improvement efforts by

FAA and the initiative have focused on reducing the causes of past

accidents and incidents, which may not be entirely predictive of future

ones. Studying growth and technological changes in the aviation industry

can help anticipate and prevent the safety problems and accidents that are

likely to arise from such changes. An international work group has been

formed to address future hazards, and a number of FAA staff participate in

this work group. Coordinating these efforts with Safer Skies’ work should

enhance the initiative’s efforts to reduce the fatal accident rate.

As of April 1, 2000, Safer Skies teams had started work on 13 of the 16

safety problems and had begun implementing interventions for 5 of these—

Fatal accidents Deaths

Type of operation Number Percentage Number Percentage

Commercial aviation 85 2 1,756 18

General aviation 4,386 98 8,046 82

Total 4,471 100 9,802 100
Executive Summary

Page 11 GAO/RCED-00-111 Safer Skies Initiative

2 in commercial aviation and 3 in cabin safety. Teams have made the most

progress in selecting interventions to address safety problems when they

have been able to build on previous studies for which widely supported

recommendations already existed. Since its inception in April 1998, the

Safer Skies initiative has evolved as new safety problems have been

addressed. For example, the process used to analyze safety problems and

select interventions has been modified as Safer Skies teams have begun to

address safety problems that have received less extensive study. Because

many of these safety problems are long-standing ones that have not been

fully resolved by prior efforts, progress will depend on effectively

implementing the chosen interventions. The initiative has developed a

process for tracking the implementation of interventions to improve safety

in commercial aviation. However, the implementation of Safer Skies’

interventions is not assured because the tracking system for commercial

aviation is not sufficiently detailed to assess progress in implementing

interventions, the system for general aviation is still in development, and no

system exists for tracking recommended interventions in cabin safety.

Without such systems, the Safer Skies initiative cannot ensure that all of

the interventions approved to increase aviation safety will be put into

action.

Since most of the interventions developed under the Safer Skies initiative

are in early implementation stages, little progress has been made in

evaluating their effectiveness. Of the five Safer Skies teams that have begun

implementing interventions, only one has developed a performance

measure to evaluate whether the interventions it has selected are helping to

reduce the safety problems that cause fatal accidents and are worth what

they cost. Such evaluation requires that performance measures be

developed to serve as the yardsticks for measuring the progress made

toward the program goals—a process required for federal programs by the

Government Performance and Results Act of 1993. The initiative has

established safety goals based on reducing the fatal accident rate for

commercial aviation by 80 percent and reducing the number of fatal

general aviation accidents to 350 by 2007. It plans to measure annual

progress toward meeting those goals. Fatal accidents occur too rarely,

especially in commercial aviation, to serve as measures of the effectiveness

of specific interventions. Other indicators exist or can be developed to

measure the unique effect of individual interventions.

FAA has coordinated extensively with aviation experts from industry, other

federal government agencies, and its own staff, but GAO’s review identified

three coordination problems that could undermine the implementation and
Executive Summary

Page 12 GAO/RCED-00-111 Safer Skies Initiative

evaluation of Safer Skies’ interventions. First, although FAA officials have

repeatedly committed to funding interventions agreed upon by all parties

working on the initiative, skepticism still exists among some participants

as to whether this commitment can or will be honored. This is particularly

true in general aviation. Furthermore, if funding is limited, it remains

unclear what process will be used to reprioritize available resources to

ensure funding for interventions that emerge later but have greater

potential for reducing the fatal accident rate. Finally, the Safer Skies

initiative, FAA, and the Department of Transportation (DOT) have not

agreed on how they will measure progress in achieving the accident

reduction goal for commercial aviation.

We presented a draft of this report to DOT and FAA for comment. DOT and

FAA officials characterized the report as fair and reasonable and provided

technical clarifications, which were made as appropriate. The officials

concurred with most of our recommendations. However, DOT and FAA

officials disagreed with our recommendation that some basic criteria and

processes should be established for reprioritizing interventions if funding is

limited. Because we believe that such criteria and processes would be

useful in assessing the potential impact of safety interventions, we did not

modify or delete our recommendation. The officials also disagreed with

two of our recommendations calling for an analyses of safety problems that

have not been studied previously and of problems that may arise in the

future. The officials provided information showing that FAA is taking these

actions, so we withdrew these recommendations.
Executive Summary

Page 13 GAO/RCED-00-111 Safer Skies Initiative

Principal Findings

The Safer Skies Initiative

Should Help Improve

Aviation Safety

The six safety problems that Safer Skies addresses in commercial aviation

accounted for about 79 percent of the fatal accidents in commercial

aviation in 1988-97. Three of these safety problems accounted for 58 of the

85 accidents during this period: pilots’ losing control of their aircraft, pilots’

flying otherwise controllable aircraft into the ground or water, and

accidents during approach and landing.3 Commercial aviation teams will

develop interventions for these safety problems in large aircraft, while

accidents that involved smaller commuter aircraft were referred to the

general aviation steering committee for review. To further reduce the fatal

accident rate for commercial aviation, the commercial aviation teams will

address three other safety problems4 that resulted in fewer fatal accidents

but have the potential to cause many more in the future. FAA also has

ongoing initiatives to address the causes of several additional safety

problems that caused four fatal accidents, including sabotage, fuel tank

explosions, and structural problems.

In general aviation, the Safer Skies initiative also addresses major safety

problems, but the goal chosen does not encourage aggressive steps to

decrease general aviation accidents. Although the data available on general

aviation accidents are less detailed than those available on commercial

aviation accidents, the problems to be addressed in general aviation

include ones identified in past safety reports and in the National

Transportation Safety Board’s (NTSB) accident reports as major causes of

3

Loss of control, which refers to accidents in which the pilot should have maintained or

regained control of the aircraft but did not, is the largest cause of fatal large air carrier

accidents in the United States. Controlled flight into terrain, which refers to flying an

otherwise controllable aircraft into the ground or water, is the leading cause of fatal large air

carrier accidents worldwide. Approach and landing refers to situations in which a crash

occurs during the approach to the airport or attempt to land when the pilot might have been

able to land safely but did not.

4

These problems, which resulted in a total of nine fatal accidents, include weather,

uncontained engine failure, and runway incursions. Uncontained engine failure occurs when

a heavy engine part rotating at high speed cracks and breaks out of the engine housing. In

two U.S. accidents, engine parts have breached the body of the aircraft resulting in fatalities.

Runway incursions are occurrences at a towered or nontowered airport, involving an

aircraft, vehicle, or pedestrian within the runway safety area, that creates a real or potential

collision hazard with an aircraft taking off, intending to take off, or landing or intentding to

land.
Executive Summary

Page 14 GAO/RCED-00-111 Safer Skies Initiative

fatal accidents. These include, for example, weather, loss of control, and

runway incursions. In establishing a goal for general aviation, the initiative

did not adopt the 80-precent goal proposed by the two aviation safety

commissions. The initiative chose a goal of 350 fatal general aviation

accidents in 2007. This represents a 20-percent reduction in the number of

fatal accidents projected for that year given expected growth. The Safer

Skies initiative also set an interim goal of 379 fatal accidents for each of the

next 3 years. This interim goal represents a 7-percent increase over the

number of fatal accidents in 1999 and does not challenge the general

aviation community to continue the kinds of safety improvements that

helped reduce such accidents to 354 last year.

Initially, the Safer Skies initiative focused on addressing the safety

problems that caused past fatal accidents and serious incidents. Aviation’s

significant growth and changes in the industry’s operations are likely to

lead to types of accidents that differ from those in the past. To anticipate

and prevent accidents that could result from such changes, the Joint Safety

Strategy Initiative in Europe5 has formed a work group to develop a method

for examining future hazards. Representatives from FAA associated with

the Safer Skies initiative participate in this work group to help coordinate

the initiative’s work on past accidents and incidents with the international

work on future hazards. This work on future hazards is preliminary and is

currently focused on developing a methodology for identifying and

addressing these potential safety problems. As this work group’s efforts

progress, coordinating these two efforts should help avoid duplication of

effort and foster awareness of and solutions to these potential problems

internationally.

The Safer Skies Initiative

Has Made Progress

In Selecting and

Implementing Interventions

As of April 1, 2000, Safer Skies teams had identified a number of

interventions, and efforts were being implemented to address 5 of the 16

safety problems; study is under way on an additional 8; and 3 have not yet

been addressed. The teams dealing with well-studied safety problems have

made the most progress in selecting and implementing interventions. These

include uncontained engine failure, controlled flight into terrain, and

weather in commercial aviation and controlled flight into terrain and

weather in general aviation. Progress occurred because these teams were

able to take advantage of existing studies and safety recommendations. For

5

The Joint Safety Strategy Initiative includes members from European aviation

manufacturers, associations, and regulators.
Executive Summary

Page 15 GAO/RCED-00-111 Safer Skies Initiative

example, the team reviewing uncontained engine failure has completed its

work; the more extensive engine inspections it recommended are under

way. In addition, air carriers had taken action on some high-priority

recommendations before FAA issued a final rule or the Safer Skies team

issued its final report. Specifically, air carriers began installing the

enhanced navigational equipment in their aircraft to prevent accidents

from controlled flight into terrain before FAA issued its final rule in March

2000 or the commercial aviation team working on controlled flight into

terrain issued its final report in June 2000. Educational information has

also been made available to improve cabin safety by publicizing the safety

benefits of greater passenger use of seat belts and child restraint systems

and issues associated with the stowage of carry-on baggage.

GAO found that the process Safer Skies teams have been using to choose

interventions and implement them is reasonable and has allowed FAA and

industry groups to reach consensus on how to address safety problems.

The first Safer Skies teams that used this process were able to compare

their results with those of prior FAA and industry studies. The process for

analyzing data and selecting interventions has been modified by Safer Skies

teams to accommodate differences in available data on other safety

problems. For example, the runway incursion team analyzed incidents

because so few fatal accidents result from this safety problem. Such

evolution in the process will be critical when addressing safety problems

that have not been studied previously.

The interventions selected by Safer Skies teams can reduce the fatal

accident rate only if they are implemented effectively. GAO’s past work has

shown that FAA does not consistently follow through on implementing key

safety recommendations.6 The initiative has developed a system for

tracking the implementation of interventions to improve safety in

commercial aviation. However, the implementation of Safer Skies’

interventions is not assured because the tracking system for commercial

aviation is not sufficiently detailed to assess progress in implementing

interventions. Furthermore, although the general aviation steering

committee is approaching final approval on interventions to address two

safety problems, it is still developing a tracking system, and no system was

developed to track interventions implemented in cabin safety.

6

Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing Safety

Recommendations (GAO/RCED-96-193, Sept. 23, 1996).
Executive Summary

Page 16 GAO/RCED-00-111 Safer Skies Initiative

The Safer Skies Initiative

Has Yet to Develop Ways to

Evaluate the Effectiveness

of Most Interventions Under

Way

The Safer Skies initiative has developed a way to evaluate the effectiveness

of one intervention it has under way to reduce the fatal accident rate.

Performance measures are needed both to fulfill the mandate of the

congressional commission that such performance measures be established

and to meet the requirements of federal law, which requires federal

departments requesting funding to evaluate the effectiveness of their

programs. Since fatal aviation accidents are infrequent, the effectiveness of

Safer Skies’ interventions must be measured using information about other

kinds of events, such as incidents that typically precede accidents. The

Safer Skies steering committees recognized early on that alternative

measures would be needed to measure the unique effect of individual

interventions. Thus far, however, only the uncontained engine failure team

has developed a quantifiable performance measure. In contrast, the two

general aviation teams that have completed their work included no

specific, quantifiable measures for evaluating the effectiveness of their

interventions. For example, one team recommended better marking of

towers and power wires to prevent fatal accidents that result when lowflying aircraft strike these objects. The team’s performance measure was a

decrease in the number of accidents involving wires and towers. The

effectiveness of this intervention cannot be measured without specific,

quantified baseline information on how many of these accidents occurred

in the past, what growth is expected in general aviation, and what

reduction the team hopes to achieve with this intervention. No

performance measures were developed to evaluate the educational

interventions implemented for the four cabin safety problems.

Coordination Has Been

Extensive but Could Be

Improved to Enhance the

Impact of Safer Skies’

Interventions

Although Safer Skies steering committees and teams included many

government and industry aviation experts, three aspects of coordination

could be improved to enhance the chances of implementing and evaluating

the initiative’s safety interventions. First, the steering committees for

commercial aviation and general aviation have both sought the

commitment of all participants to implementing and funding interventions

before giving final approval to move forward. However, FAA’s commitment

has come at different points in the approval process for interventions

recommended by these steering committees. FAA’s commitment to the

general aviation interventions is still uncertain even after some participants

believed that the steering committee had granted its final approval. As a

consequence, general aviation participants were more skeptical about

whether FAA would implement or fund their safety interventions. In

October 1999, FAA formed an executive council to help coordinate the
Executive Summary

Page 17 GAO/RCED-00-111 Safer Skies Initiative

implementation of the agency’s safety agenda, but this council has not yet

documented its process for approving and funding interventions.

Second, if funding is limited, it remains unclear what process will be used

to reprioritize available resources to ensure funding for interventions that

emerge later but have greater potential for reducing the fatal accident rate.

A Safer Skies team has just begun work on loss of control—which caused

the greatest number of fatal accidents in commercial aviation in 1988-97.

Interventions to address loss of control are thus likely to be critical for

reducing the fatal accident rate. If funding is limited, some resources may

have to be shifted from existing programs and safety initiatives. The Safer

Skies initiative and FAA’s executive council have not yet established any

process for reprioritizing interventions if funding is limited.

Finally, coordination among Safer Skies steering committees, FAA, and

DOT needs to improve to ensure the effective evaluation of Safer Skies

interventions. DOT is responsible for setting safety goals for all modes of

transportation under its authority, including aviation. Each of its agencies,

including FAA, provides more detailed information on how it will achieve

those safety goals. Currently, DOT and FAA measure progress toward the

goal of an 80-percent reduction in the fatal accident rate for commercial

aviation in different ways. Specifically, DOT’s performance plan measures

progress using a fatal accident rate based on flight hours, while FAA’s

strategic plan and the Safer Skies initiative use an accident rate based on

aircraft departures as the measure of activity. Since the ultimate mission of

all three groups is to reduce the fatal accident rate, using the same activity

measure to calculate that rate would make sense. Because most

commercial aviation accidents occur during takeoff and landing, GAO

believes that using departures better measures passengers’ exposure to

risk.

Recommendations To improve FAA’s safety agenda for decreasing fatal aviation accidents,

GAO makes a number of recommendations in this report to the Secretary

of Transportation to direct the FAA Administrator in her work with the

Safer Skies steering committees. These recommendations focus on

establishing a more challenging goal for reducing fatal accidents in general

aviation and improving the implementation and evaluation of the Safer

Skies initiative.
Executive Summary

Page 18 GAO/RCED-00-111 Safer Skies Initiative

Agency Comments and

Our Evaluation

GAO provided copies of a draft of this report to the Department of

Transportation and FAA for their review and comment. GAO met with FAA

officials, including the Deputy Associate Administrator for Regulation and

Certification and the Director of Aircraft Certification. The FAA officials

concurred with the majority of our recommendations and characterized the

Safer Skies report as generally “fair and reasonable.” They informed GAO

of actions taken by the agency since GAO completed its audit work in

March. This information has been incorporated as appropriate. FAA

concurred with the need to set more challenging interim and long-term

goals for general aviation and plans to do so in the future. FAA officials

agreed that improvements were needed in how the Safer Skies initiative

tracks the implementation of interventions, although they disagreed with

the level of detail suggested by GAO. They also agreed with GAO’s

recommendations to improve the baseline data and performance measures

used to evaluate the impact of these interventions.

The Safer Skies initiative has taken steps to expand and improve its

tracking of interventions, but the system still does not clearly identify and

specify time frames for major commitments and deliverables for each

approved intervention. Without a reliable tracking system, FAA and Safer

Skies will not be in a position to ensure that recommended interventions

are implemented to improve aviation safety.
Page 19 GAO/RCED-00-111 Safer Skies Initiative
Page 20 GAO/RCED-00-111 Safer Skies Initiative

Chapter 1

Background Chapte1r

The Federal Aviation Administration (FAA) has forecast continued growth

for commercial and general aviation over the next decade. 1 Growth over

the past few decades brought innovations to improve flight safety that

contributed to a dramatic lowering of the accident rate by the mid-1970s.

Further reductions in the accident rate have, however, remained elusive.

Unless the current accident rate can be reduced, the number of fatal

accidents is likely to increase as aviation operations continue to grow.

During the 1990s, FAA, the aviation industry, and the Congress all

acknowledged and studied this potential danger. They set ambitious targets

for reducing the accident rate, made over a thousand recommendations for

improving aviation safety, and implemented a number of safety initiatives.

In spite of these efforts, the accident rate, which is already low, has

remained fairly steady. The FAA Administrator, White House and

congressional task forces, and aviation industry groups have concluded

that FAA and the aviation industry must coordinate their efforts to

prioritize safety recommendations and focus resources on those with the

most potential to decrease the accident rate. In 1998, the FAA

Administrator announced the Safer Skies initiative, a joint governmentindustry effort to identify and address the greatest threats to aviation safety

in order to reduce the fatal accident rate by 80 percent by the year 2007.

FAA Expects

Continued Growth in

Aviation

Over the past several decades, aviation has grown substantially in the

United States, and FAA expects this growth to continue into the next

century. Commercial aviation has grown consistently since 1982, while

growth in general aviation has been less consistent. One key measure of

aviation activity shows that the number of flight hours for commercial

aircraft more than doubled from 8 million hours in 1982 to nearly 18 million

hours in 1999. In contrast, general aviation activity dropped fairly steadily

from the early 1980s until 1995. While general aviation has grown since

1995, it has not yet returned to 1990 levels. The number of general aviation

1

Commercial aviation includes both large air carriers and commuter air carriers.

Specifically, commercial aviation includes all air carriers offering scheduled and

nonscheduled service by major air carriers flying under 14 C.F.R. part 121 and all air carriers

operating scheduled service under 14 C.F.R. part 135. General aviation aircraft include all

U.S.-registered civil aircraft not operated under 14 C.F.R. part 121 or part 135. General

aviation includes a wide variety of aircraft, ranging from corporate jets to small pistonengine aircraft as well as helicopters, gliders, and aircraft used in operations such as

logging, firefighting, and agricultural spraying. General aviation also includes on-demand air

carriers that operate nonscheduled service under 14 C.F.R. part 135.
Chapter 1

Background

Page 21 GAO/RCED-00-111 Safer Skies Initiative

flight hours decreased by nearly 9 percent from 32.6 million hours in 1982

to 29.9 million hours in 1999. (See fig. 1.)

Figure 1: Commercial Aviation and General Aviation Flight Hours, 1982-99

Source: GAO’s analysis of data from the National Transportation Safety Board.

FAA has forecast continued growth for commercial aviation as well as for

general aviation into the next century. The number of planes will increase,

and these aircraft will fly more miles, spend more hours in the air, and carry

more people. For example, FAA estimates that commercial aviation flight

hours will grow to 24 million hours in 2007—an increase of 37 percent from

1999. In commercial aviation, FAA projects that the use of large air carriers

will grow at an annual rate of 4 percent, while the use of commuter air

carriers will grow at 3 percent per year. Although growth has been more

erratic in general aviation than in commercial aviation, FAA projects an

annual growth rate of 2.2 percent for general aviation into the next century.

FAA estimates that general aviation flight hours will increase to about 36

million hours in 2007, a growth of nearly 19 percent over 1999.

In millions

0

5

10

15

20

25

30

35

1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999

General aviation 

Commercial aviation
Chapter 1

Background

Page 22 GAO/RCED-00-111 Safer Skies Initiative

Fatal Accident Rates

Have Decreased for

U.S. Aviation

Even with the growth in aviation, fatal accidents remain relatively rare,

especially in commercial aviation. Fatal accident rates for U.S. aviation are

low and have decreased over the past decades for both commercial and

general aviation. The fatal accident rate can be calculated as the number of

accidents with one or more fatalities divided by a measure of aviation

activity, such as the number of aircraft miles flown, aircraft hours flown, or

departures.

More Fatal Accidents Occur

in General Aviation, but

Commercial Aviation

Accidents Can Be

Catastrophic

In the 10-year period preceding the initiative, 4,471 fatal aviation accidents

occurred in the United States, resulting in a total of 9,802 deaths. Table 2

shows the distribution of accidents and deaths for commercial aviation,

which includes large and commuter air carriers, and general aviation,

which includes on-demand air taxis. General aviation accounted for the

largest number of fatal accidents and deaths in 1988-97. The initiative

addresses both commercial and general aviation, but increased attention is

focused on further improving the safety of commercial aviation because

large and commuter air carriers are the primary forms of air transportation

for most Americans. While fatal commercial aviation accidents are rare,

large airplane accidents can cause more deaths in an instant than most

events, other than wars or natural disasters. They consequently raise

concerns with both the public and the media, and commercial aviation is

held to a higher standard of safety than other forms of transportation. With

commercial aviation expected to grow steadily into the next century,

aviation accidents will occur with a frequency that will be unacceptable to

the public unless steps are taken to decrease the fatal accident rate. While

such accidents remain rare, FAA recognizes that the public demands a high

standard of safety and expects continued improvement.

Table 2: Fatal Accidents and Deaths by Type of Aviation Operations, 1988-97

Source: GAO’s analysis of data from the National Transportation Safety Board.

Fatal accidents Deaths

Type of operation Number Percentage Number Percentage

Commercial aviation 85 2 1,756 18

General aviation 4,386 98 8,046 82

Total 4,471 100 9,802 100
Chapter 1

Background

Page 23 GAO/RCED-00-111 Safer Skies Initiative

How Fatal Accident Rates

Are Calculated

FAA tracks the number of passenger fatalities for various types of aviation

operations and calculates accident rates. Basically, the rates are calculated

by dividing the number of accidents with one or more fatalities by one of

the various measures of aviation activity. For example, the fatal accident

rate for commercial aviation for 1988-97 is 0.058 per 100,000 flight hours,

which was calculated by dividing the number of fatal accidents (85)2 by the

number of flight hours (151 million). This translates into about one fatal

accident for every 2 million hours flown. The three activity measures

generally used to calculate fatal accident rates are the number of individual

flights (referred to as departures), aircraft miles flown, and aircraft hours

flown. Each activity measure reflects different exposures to the risks

associated with flying. For example, most commercial aviation accidents

occur during takeoff or landing, rather than during the cruise phase, which

constitutes the largest part of the total mileage and hours flown. For this

reason, we believe that departures are usually the best measure of

exposure to risk.

For large and commuter air carriers, all three fatal accident rates are

tracked. But for general aviation, the only measure of exposure is the

number of flight hours estimated from survey data. Thus, fatal accident

rates for commercial aviation (large and commuter air carriers) are

generally expressed in terms of the number of fatal accidents per 100,000

departures, while fatal accident rates for general aviation are expressed as

the number of fatal accidents per 100,000 flight hours as estimated by FAA’s

annual survey.3 General aviation flight hours are not sufficiently reliable for

use in calculating a fatal accident rate for general aviation because they are

estimated from a voluntary survey, according to FAA.

2

We used the 85 fatal accidents in commercial aviation for the purpose of illustration. When

calculating fatal accident rates in aviation, the National Transportation Safety Board (NTSB)

excludes accidents that resulted from sabotage and hijacking. In 1988-97, one fatal U.S.

accident resulted from sabotage and would thus be excluded. The number of accidents that

NTSB would use to calculate the fatal accident rate for the period is 84 accidents.

3

FAA uses an annual survey to estimate flight hours because it does not require general

aviation operators to report such key measures as the number of hours flown or the number

of takeoffs. The General Aviation and Air Taxi Activity Survey provides FAA with

information on the operations of these aircraft.
Chapter 1

Background

Page 24 GAO/RCED-00-111 Safer Skies Initiative

Fatal Accident Rates Have

Decreased

Over the past few decades, the annual rate of fatal aviation accidents has

decreased significantly for both commercial and general aviation. While the

accident rates are low, they have shown little improvement recently. For

large commercial air carriers, the U.S. accident rate was 26 fatal accidents

per million departures in 1959. Following the advent of large jet aircraft in

the 1960s, the rate fell to one or fewer fatal accidents per million

departures and has remained fairly steady for three decades. The fatal

accident rate for commuter aircraft has also fallen over the last several

decades. The accident rate for commuter air carriers fell from about 2 fatal

accidents per million departures in 1982 to 3 per 10 million departures in

1996. While there were no fatal commuter accidents in 1998, the five fatal

accidents in 1999 resulted in a fatal accident rate of nine per million

departures. This increase in the fatal accident rate reflects a 1997

narrowing in the definition of commuter air carrier to include only small

aircraft with nine or fewer seats.4 Similarly, the accident rate for general

aviation aircraft has dropped since 1960. The fatal accident rate of six per

100,000 flight hours in 1960 fell to less than two by the early 1980s. The fatal

accident rate for general aviation continued to decrease fairly steadily

through the 1980s, increased slightly in the early 1990s, and has dropped

steadily since 1995. In 1999, the fatal accident rate for general aviation was

1.2 fatal accidents per 100,000 flight hours. (See fig. 2.)

4

Since March 20, 1997, aircraft with 10 or more seats formerly operating scheduled service

under 14 C.F.R. part 135 have been required to follow the more stringent safety requirements

that apply to larger aircraft under 14 C.F.R. part 121.
Chapter 1

Background

Page 25 GAO/RCED-00-111 Safer Skies Initiative

Figure 2: Fatal Accident Rates for Commercial and General Aviation, 1982-99

Source: GAO’s analysis of NTSB data.

The reductions in the fatal accident rates resulted from a combination of

technological advances that improved safety. In commercial aviation, these

advances included the replacement of large, piston-engine aircraft with jet

aircraft with far more reliable engines, the development of navigational

equipment to warn pilots of impending crashes, better ground navigation

aids, improved aircraft instrumentation, and increased air traffic radar

coverage. Some of these improvements have also benefited smaller

commuter and general aviation aircraft. As commuter air carriers switched

from small aircraft to sophisticated turboprop aircraft, the accident rate

among the larger commuter aircraft became comparable to that of large air

carriers.

0

0.5

1.0

1.5

2.0

1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999

General aviation

Commercial aviation

Per 100,000 flight hours
Chapter 1

Background

Page 26 GAO/RCED-00-111 Safer Skies Initiative

If Greater Numbers of

Fatalities Are to Be Avoided,

the Fatal Accident Rate

Must Be Reduced

If the current fatal accident rate holds steady and aviation activity grows as

FAA has projected, the increased air traffic will result in greater numbers of

crashes and fatalities. We estimate that the average of six fatal commercial

aviation accidents per year in 1994-96 will likely rise to nine per year by

2007. Similarly, the fatal accidents for general aviation will probably mount

from an average of 380 in 1996-98 to 484 in 2007.5 Table 3 shows our

projections of the number of fatal accidents in 2007 calculated from FAA’s

growth estimates and the current fatal accident rate for each type of

aviation operation.

Table 3: Projected Numbers of Fatal Accidents by Type of Operation in 2007

a

The annual average is for the baseline years chosen by FAA and Safer Skies steering

committees in establishing their goals: 1994-96 for commercial aviation and 1996-98 for

general aviation.

Source: GAO’s analysis of data from FAA and NTSB.

The prospect of more accidents and deaths is unacceptable to the public,

FAA, and the aviation industry. Avoiding that outcome means reducing the

fatal accident rate significantly. The final report of the National Civil

Aviation Review Commission concluded in 1997 that the “anticipated

growth in aviation between now and the first quarter of the next century

will almost certainly lead to an occurrence of aviation accidents with a

frequency that will be wholly unacceptable to the public.” The Commission

called for a joint industry-government effort to reduce the accident rate

substantially.

5

FAA and Safer Skies steering committees have chosen different baseline years for

commercial aviation (1994-96) and general aviation (1996-98). We have used those years in

table 3 projecting the number of fatal aviation accidents and in subsequent tables in this

report comparing the accident reduction goals chosen by FAA and Safer Skies steering

committees with the 80-percent goal set forth by the White House and congressional

commissions on aviation safety.

Number of fatal accidents

Type of operation Annual averagea Projected for 2007

Commercial aviation 6 9

General aviation 380 484
Chapter 1

Background

Page 27 GAO/RCED-00-111 Safer Skies Initiative

FAA and the Aviation

Industry Made Previous

Efforts to Reduce the Fatal

Accident Rate

During the 1990s, FAA and aviation industry groups had separate and joint

efforts under way to use available data to identify and address the major

causes of accidents. A series of fatal crashes and concern that the number

of accidents and fatalities will increase as air traffic increases prompted

these efforts to reduce the accident rate. Many of the reports that resulted

from these efforts set specific goals and included recommendations for

decreasing aviation accidents. Although FAA and the aviation industry

acted on some of these recommendations, the fatal accident rate has

remained fairly stable but low.

The effectiveness of previous efforts to reduce the fatal accident rate is

believed to have been undercut by their limited scope and a lack of

coordination between government and industry groups. Many of the

studies issued during the 1990s were under the leadership of either FAA or

a particular segment of the aviation industry. For example, FAA, on its own,

studied controlled flight into terrain (CFIT)6 accidents and runway

incursions. Separately, the Flight Safety Foundation brought together

participants from many segments of the aviation industry to study CFIT

and approach and landing but initially had only limited FAA involvement.

The Aerospace Industries Association initiated an extensive study on the

causes of safety-related problems in aircraft engines, including

uncontained engine failure.7 (For a list of key aviation studies and our

related reports, see app. II.) According to FAA and industry officials we

interviewed, efforts to address specific safety issues were generally

unsuccessful when one group failed to coordinate its work with that of

other groups that had important roles in aviation safety.

Many of these reports issued during the 1990s set specific goals for

reducing the overall fatal accident rate or for addressing specific aviation

safety problems that result most often in fatalities. They also included

numerous specific recommendations to FAA and the aviation industry to

help meet these goals. Among the key reports were the following:

6

CFIT is flying an otherwise controllable aircraft into the ground or water.

7

Uncontained engine failure occurs when a heavy engine part rotating at high speed cracks

and breaks out of the engine housing. In two U.S. accidents, engine parts breached the body

of the aircraft. One accident on takeoff resulted in the death of 2 passengers, while the other

accident crippled key aircraft systems in flight, resulting ultimately in a crash that killed 111

passengers and crew.
Chapter 1

Background

Page 28 GAO/RCED-00-111 Safer Skies Initiative

• In 1993, the Flight Safety Foundation led an international task force on

CFIT, the leading cause of fatal commercial aviation accidents

worldwide. The task force provided specific recommendations and

training aids aimed at reducing CFIT accidents. The task force set a goal

of reducing these accidents 50 percent worldwide by 1998 and other

goals targeting improvements in the regions of the world with the

highest CFIT rates.8

• In January 1995, over 1,000 government, industry, and union officials

attended an FAA-sponsored safety conference. The officials agreed that

they shared responsibility for pursuing a goal of zero accidents. Their

report identified 173 high-priority safety initiatives in the areas of crew

training, air traffic control and weather, safety data collection and use,

applications of emerging technologies, aircraft maintenance procedures

and inspections, and development of flight operating procedures.9

• Following the May 1996 ValuJet crash, an FAA task force recommended

in September 1996 that FAA target agency resources to safety risks,

improve the certification and oversight of new air carriers, and address

concerns about inspector guidance and resources.10

• In February 1997, the White House Commission on Aviation Safety and

Security recommended that the government and the aviation industry

establish a national goal to reduce the aviation fatal accident rate by a

factor of five (meaning 80 percent) within 10 years. To achieve that goal,

the Commission made specific recommendations for reengineering

FAA’s regulatory and certification programs.11 The Commission did not

explicitly state whether the national goal should apply to all types of

aviation operations.

8

Flight Safety Foundation, Controlled Flight Into Terrain: Education and Training Aid

(Disseminated under the sponsorship of the U.S. Department of Transportation.).

9

Zero Accidents ... A Shared Responsibility, U.S. Dept. of Transportation, FAA, Feb. 9, 1995.

10FAA 90 Day Safety Review, U.S. Dept. of Transportation, FAA, Sept. 16, 1996.

11Final Report to President Clinton, White House Commission on Aviation Safety and

Security, Feb. 12, 1997.
Chapter 1

Background

Page 29 GAO/RCED-00-111 Safer Skies Initiative

• In December 1997, the National Civil Aviation Review Commission

recommended that the government and the aviation industry work

together to achieve the White House Commission’s goal of an 80-percent

reduction in the accident rate over the next 10 years and recommended

specific safety improvements for achieving that goal.12 While the

Commission did not explicitly state whether the 80-percent goal should

apply to all types of aviation operations, the Commission specifically

discussed the accident rates for large jets, commuter air carriers,

general aviation operations, and air taxis.

Both the White House Commission on Aviation Safety and Security and the

National Civil Aviation Review Commission called for FAA and the aviation

industry to work together on aviation safety issues.

The Safer Skies

Initiative Continued

Ongoing Efforts to Use

Data Analysis to

Address Safety

Problems

On April 14, 1998, the Vice President, the Secretary of Transportation, and

the FAA Administrator announced the Safer Skies initiative, a new aviation

safety program committed to reducing the fatal accident rate by 80 percent

by 2007. Under the initiative, experts from FAA, the aviation industry, and

other government agencies with responsibility for aviation are to jointly

analyze U.S. and global data to identify the most serious threats to aviation

and to find the root causes of accidents. They will then determine the best

actions to break the chain of events that lead to accidents and direct

resources first to those actions. These actions are also referred to as

interventions.

FAA Invited Members of

Ongoing Industry and

Government Safety Groups

to Join the Safer Skies

Initiative

When FAA announced the Safer Skies initiative, the agency invited

participants from a number of ongoing industry and government safety

groups to join in creating a unified safety agenda. In establishing the

agenda for the initiative, the commercial and general aviation steering

committees joined with and expanded the preexisting efforts. To develop

the unified agenda, key government and industry aviation officials are to

conduct data analyses to identify the causes of fatal accidents and

determine what interventions are needed to prevent them.

12Avoiding Aviation Gridlock and Reducing the Accident Rate: A Consensus for Change,

National Civil Aviation Review Commission, Dec. 11, 1997.
Chapter 1

Background

Page 30 GAO/RCED-00-111 Safer Skies Initiative

Several of the preexisting safety groups were already using data-driven

approaches to study aviation safety issues. Representatives of air carriers,

aircraft and engine manufacturers, and related associations had

established a commercial aviation group in January 1997 to analyze fatal

commercial aviation accidents and to recommend ways to prevent them.

Before joining the initiative, this group had outlined a process for obtaining

accident data from U.S. and international sources and for reaching

consensus on the safety problems to be addressed. Another industry group

analyzing data on uncontained engine failure had developed a process for

analyzing safety data, using case studies to identify root causes, and

evaluating the feasibility of proposed interventions. A third group that

represented a cross-section of various general aviation constituencies,

such as pilots and small aircraft manufacturers, was addressing the causes

of fatal general aviation accidents.13 A joint government-industry group

sponsored by FAA was continuing work on issues pertaining to the safety

of passengers and crew members in the aircraft cabin that had been started

separately by FAA, industry associations, and unions representing flight

attendants.14 FAA invited members from all four of these groups to

participate in the initiative.

Steering Committees

Selected 16 Safety

Problems for the Safer Skies

Initiative to Address

Safer Skies formed steering committees of safety experts from government

and industry to lead the work in each of its three agenda areas: commercial

aviation, general aviation, and cabin safety. Each steering committee has

co-chairs and participants from both FAA and industry. The commercial

and general aviation steering committees used available data to select the

safety problems to be addressed in their respective agenda areas. In

contrast, the cabin safety steering committee continued the work on safety

problems that had already been under way as a joint FAA-industry effort

that preceded Safer Skies. The three Safer Skies steering committees

ultimately chose to address 16 safety problems: 6 in commercial aviation, 6

in general aviation, and 4 in cabin safety. The commercial aviation and

general aviation steering committees selected several of the same safety

problems, including weather and loss of control over the aircraft. Because

safety problems can affect large and small aircraft differently, the

commercial and general aviation steering committees planned to have

13Prior to joining the Safer Skies initiative, the Joint Safety Coalition was known as the

General Aviation Coalition.

14This group was called Partners in Cabin Safety.
Chapter 1

Background

Page 31 GAO/RCED-00-111 Safer Skies Initiative

separate teams study each safety problem with one exception. A joint team

will study runway incursions because commercial and general aviation

aircraft often share the same runways and accidents have occurred

involving both types of aircraft. Table 4 lists and briefly explains each

safety problem.

Table 4: The 16 Safety Problems Addressed by Safer Skies

Source: FAA.

Problem identified Definition of problem

Commercial aviation

Controlled flight into terrain Flying an otherwise controllable aircraft into terrain.

Loss of control Accidents resulting from situations in which the pilot should have maintained or

regained aircraft control but did not.

Uncontained engine failure Small cracks in high-speed rotating parts that, if left undetected, can propagate until

failure.

Runway incursion An occurrence at a towered or nontowered airport, involving an aircraft, vehicle, or

pedestrian within the runway safety area, that creates a real or potential collision

hazard with an aircraft taking off, intending to take off, or landing or intending to land.

Approach and landing Situations in which a pilot may have been able to land an aircraft safely, but did not.

Weather Meteorological conditions (icing, turbulence, etc.) that adversely affect aircraft

performance.

General aviation

Controlled flight into terrain Flying an otherwise controllable aircraft into terrain.

Loss of control Accidents resulting from situations in which the pilot should have maintained or

regained aircraft control but did not.

Runway incursion An occurrence at a towered or nontowered airport, involving an aircraft, vehicle, or

pedestrian within the runway safety area, that creates a real or potential collision

hazard with an aircraft taking off, intending to take off, or landing or intending to land.

Aeronautical decision-making Often a fundamental element in accident causal chains, where a pilot did not make the

best safety decision about a flying or nonflying situation.

Weather Meteorological conditions (icing, turbulence, etc.) that adversely affect aircraft

performance.

Survivability Safety research and initiatives that would reduce fatalities.

Cabin safety

Passenger interference Passengers who hinder crew members in performing their duties.

Passenger seat belt use Injuries sustained by passengers who are not wearing their seat belts when

encountering unexpected air turbulence.

Carry-on baggage Articles brought into the airplane cabin by passengers.

Child safety restraints Safety issues associated with the commercial aviation transportation of children under

the age of 2 years old.
Chapter 1

Background

Page 32 GAO/RCED-00-111 Safer Skies Initiative

Objectives, Scope, and

Methodology

At the request of the Chairman and Ranking Democratic Member of the

Subcommittee on Aviation, House Committee on Transportation and

Infrastructure, we reviewed the design and implementation of the Safer

Skies initiative. Specifically, they asked us to determine (1) to what extent

addressing the safety problems selected by the Safer Skies initiative will

help reduce the fatal accident rate; (2) what progress the initiative has

made in identifying and implementing interventions to address each of

these safety problems; (3) what progress the Safer Skies initiative has made

in assessing the effectiveness of those interventions; and (4) how FAA is

coordinating the Safer Skies initiative with other safety activities

conducted throughout the agency, in partnership with the aviation industry,

and by other federal agencies.

Because Safer Skies is a 10-year project that hopes to reach its goals in

2007, we analyzed domestic flight operations and accident data for the

decade that preceded the 1998 announcement of Safer Skies and for the

decade to come. We examined data on fatal accidents and their causes for

all types of aviation operations in the United States from 1988 through

1997. We also examined projected data for aviation operations and

accidents through 2007.

To determine whether addressing the safety problems chosen by Safer

Skies will help reduce the fatal accident rate, we interviewed FAA officials

responsible for overseeing Safer Skies, officials at the Department of

Defense and the National Aeronautics and Space Administration involved

in aviation safety, and the chairs and many members of the steering

committees for commercial aviation, general aviation, and cabin safety. We

reviewed documents related to each of these steering committees as well

as data used by these groups in choosing the problems on which Safer

Skies would focus. We also discussed the problems Safer Skies selected as

priorities with staff at the National Transportation Safety Board, the Flight

Safety Foundation, and other aviation safety groups.

To determine what progress has been made in identifying, developing, and

implementing intervention strategies for the Safer Skies initiative, we

interviewed the FAA and industry chairs of the teams formed to address

each problem under study. We obtained and reviewed team reports

completed for each safety problem to understand the analysis process,

modifications made to it by successive work groups, and actions planned

to improve aviation safety in each problem area.
Chapter 1

Background

Page 33 GAO/RCED-00-111 Safer Skies Initiative

To determine what progress has been made to date in assessing the

effectiveness of its actions to improve aviation safety, we reviewed

implementation plans to determine whether schedules were being met and

whether ways had been chosen to measure the success of such actions. We

also reviewed available team reports and relevant data to determine

whether sufficient data were available to measure Safer Skies’ progress in

improving aviation safety.

To determine how FAA coordinated the Safer Skies initiative with safety

activities conducted throughout FAA and in partnership with the aviation

industry, we reviewed information on related industry and government

safety activities. Specifically, we sought information on activities under the

auspices of FAA, the Department of Defense, the National Aeronautics and

Space Administration, the National Transportation Safety Board, selected

engine and aircraft manufacturers, several major air carriers, the Air

Transport Association, and the Aircraft Owners and Pilots Association.

During our interviews with members of the Safer Skies steering

committees and teams, we discussed efforts to coordinate their work with

other government and industry safety activities. We also reviewed the

reports from each Safer Skies team for safety problems where coordination

would be appropriate. We discussed the budgetary implications of the Safer

Skies initiatives and the criteria for prioritizing resources with FAA officials

and steering committee members.

We conducted our work from August 1999 through June 2000 in accordance

with generally accepted government auditing standards.
Page 34 GAO/RCED-00-111 Safer Skies Initiative

Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety Chapte2r

Addressing the 16 safety problems chosen by the Safer Skies initiative

should help reduce the nation’s fatal accident rate. In commercial aviation,

eliminating the six safety problems to be addressed by the initiative would

approach the 80-percent goal. Other FAA initiatives are addressing

additional safety problems in commercial aviation, which should

complement efforts under the Safer Skies initiative. In general aviation, the

initiative will address six problems that appear to be among the most

common causes of fatal accidents for this type of operation, according to

available accident data. While the initiative has adopted the 80-percent goal

in commercial aviation, which transports most passengers who fly in the

United States, the initiative adopted a less aggressive goal for general

aviation, which accounted for the vast majority of the fatal aviation

accidents. The goal in general aviation is to reduce the number of fatal

accidents to 350 in 2007, which represents about a 20-percent reduction.

Finally, the initiative addressed four problems in cabin safety. Improving

cabin safety will have little impact on lowering the fatal accident rate

because cabin safety accounted for only two U.S. fatalities in commercial

aviation in 1988-97. No quantitative goal was set for safety improvements in

cabin safety. To date, safety improvement efforts by FAA and the Safer

Skies initiative have focused on past accidents and incidents, which may

not be entirely predictive of future ones. Studying growth and

technological changes in the aviation industry can help anticipate and

prevent the safety problems and accidents that are likely to arise from such

changes. Preliminary international efforts have been initiated to address

future hazards, and coordinating these efforts with Safer Skies work could

enhance the initiative’s efforts to reduce the fatal accident rate.

The Safer Skies

Initiative Addresses

Major Safety Problems

in Commercial Aviation

The Safer Skies initiative plans to address six safety problems that

accounted for 79 percent of the fatal accidents in commercial aviation in

1988-97. If past accident causes continue, completely eliminating these six

safety problems might approach the 80-percent goal. FAA also has safety

initiatives under way to address several of the safety problems in

commercial aviation not addressed by the initiative. These include

sabotage, fuel tank explosions, and structural problems. In combination

with the Safer Skies initiative, FAA’s safety initiatives have potential for

reducing the fatal accident rate in commercial aviation. For commercial

aviation, the Safer Skies initiative has established a goal of reducing the

fatal accident rate by 80 percent in 2007 in accordance with the goal

envisioned by the White House and congressional commissions on aviation

safety.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 35 GAO/RCED-00-111 Safer Skies Initiative

The Safer Skies Initiative

Identified Six Major Safety

Problems

The Safer Skies initiative will address six safety problems that accounted

for 79 percent of the fatal commercial aviation accidents in 1988-97. Three

of these safety problems were major ones both worldwide and in the

United States: pilots’ losing control of their aircraft, pilots’ flying otherwise

controllable aircraft into the ground or water (CFIT), and accidents during

approach and landing. These three safety problems accounted for 58 of the

85 fatal accidents in U.S. commercial aviation during this period. The

commercial aviation teams are examining 34 of these accidents, which

involved larger aircraft. The commercial aviation steering committee

referred the remaining 24 fatal accidents to the general aviation steering

committee for review because they involved small commuter aircraft with

nine or fewer seats that operated scheduled commercial service. This was

done because (1) the aircraft involved are more similar to general aviation

aircraft than to larger commercial aircraft, (2) the types of operating

environments and safety problems that caused the accidents more closely

resemble those of general aviation than those of commercial aviation, and

(3) the interventions to address safety problems in general aviation are

more likely to correct these safety problems than interventions designed

for large commercial aircraft. We reviewed the National Transportation

Safety Board’s (NTSB) reports for the 24 small commuter accidents and

found that most of the accidents happened in Alaska when pilots flew into

mountains after deteriorating weather reduced visibility. On the basis of

our review, we concur with the commercial aviation steering committee’s

assessment that these accidents more closely resemble general aviation

accidents and would likely benefit from the interventions that emerge to

address these safety problems in general aviation aircraft.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 36 GAO/RCED-00-111 Safer Skies Initiative

The potential for improving safety in these smaller commuter aircraft

exists with a number of the interventions proposed by the general aviation

teams working on weather and CFIT. It is unclear whether the initiative or

FAA has mechanisms in place to ensure that small commuter operators will

benefit from the interventions developed. For example, many of the

interventions involve providing additional training to pilots on weather

conditions and assessing the risk factors associated with each flight.

Because the initiative plans to deliver much of this training jointly with the

Aircraft Owners and Pilots Association, it is essential that notification

about this training also be provided to small commuter operators and pilots

who could benefit from this training but may not be members. Although

members of several other organizations participate in the general aviation

steering committee and study teams,1 neither the initiative nor FAA has

made specific provisions to ensure that such interventions are also directed

at small commuter aircraft operators and pilots, as well as at general

aviation pilots. Because small commuter accidents accounted for 28

percent of the 85 fatal accidents in commercial aviation in 1988-97,

reducing the fatal accident rate by 80 percent depends on addressing these

safety problems in small commuter aircraft, as well as in large commercial

aircraft.

To further reduce the fatal accident rate for commercial aviation, the

initiative will address three additional safety problems that have resulted in

fewer fatal accidents in the United States from 1988 through 1997. The

steering committee chose runway incursions, uncontained engine failure,

and weather, each of which resulted in from two to four fatal accidents.

These three safety problems accounted for an additional 9 accidents, or 11

percent of the 85 fatal accidents. The committee selected these problems

because they caused past fatal accidents or serious incidents that could

have cost many lives. These areas were also included because each

occurred with greater frequency in the United States than worldwide and

because FAA or the aviation industry had already begun work on these

safety problems. (See fig. 3.)

1

Other participating groups include, for example, National Air Transportation Association,

the Helicopter Association International, and the National Business Aviation Association.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 37 GAO/RCED-00-111 Safer Skies Initiative

Figure 3: Safety Problems That Caused Fatal Accidents in U.S. Commercial Aviation,

1988-97

Source: FAA.

Our analysis of aviation data and review of safety reports confirmed that

the initiative is addressing three major safety problems that caused fatal

accidents in commercial aviation, as well three other safety problems that

have the potential to cause accidents with large numbers of fatalities.

Reducing or eliminating safety problems resulting from CFIT, loss of

control, approach and landing, runway incursions, weather, and

uncontained engine failure should help lower the fatal accident rate. Safer

Skies participants, FAA officials, and industry aviation experts whom we

interviewed also believe that the initiative is addressing the most important

aviation safety problems. Most of these aviation experts indicated strong

Referred to Safer Skies

general aviation 

steering committee

	 l	CFIT 16

	 l	Loss of control 7

l	Approach and landing 1

Handled by FAA

	 l	Structure 2

l	Fuel tank explosion 1

l	Sabotage 1

Gate/ramp

Other 2

Handled by Safer Skies

commercial aviation teams

	 l	Loss of control 19

l	Approach and landing 9

l	CFIT 6

l	Runway incursion 4

l	Weather 3

l	Uncontained engine failure 2

•

43

12

4

Commercial aviation safety problems handled by the Safer Skies commercial aviation team

Commercial aviation safety problems referred to the general aviation steering committee

Commercial aviation safety problems not addressed by Safer Skies

24

•

•

•

•

•
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 38 GAO/RCED-00-111 Safer Skies Initiative

support for addressing such major safety concerns as CFIT, approach and

landing, and loss of control. Furthermore, because of the increasingly

global nature of commercial aviation, addressing these safety problems

means that many of the interventions recommended by the initiative might

have applicability worldwide, as well as in the United States. Many aviation

experts we interviewed also supported the inclusion of safety problems

with fewer fatalities but with a high potential for fatalities, such as runway

incursions and uncontained engine failure. They agreed that reducing or

eliminating these safety problems should help reduce the fatal accident

rate.

Addressing Additional

Safety Problems Could

Further Reduce the Fatal

Accident Rate

In addition to successfully addressing the major safety problems discussed

above, addressing additional safety problems could further reduce the fatal

accident rate in commercial aviation. FAA has a number of aviation safety

initiatives under way that potentially can contribute to improvements in the

safety of smaller commuter aircraft sometimes used in commercial

aviation. For example, FAA’s Capstone Project focuses on improving

general aviation safety in Alaska by providing additional navigational aids

but also has potential application for addressing the safety problems of

small commuter aircraft used elsewhere. FAA also has ongoing initiatives

to address the causes of 4 of the 18 commercial aviation accidents not

being addressed by Safer Skies teams. These include programs overseen by

the agency’s Office of Civil Aviation Security to reduce the threat of

sabotage, hijacking, and the transportation of hazardous cargo. Other FAA

initiatives are under way to address the structural problems of aging

aircraft and fuel tank explosions. FAA has, for example, published a notice

proposing requirements for design reviews and mandatory maintenance

actions for fuel tank systems on large transport aircraft.

Of the remaining fatal accidents in commercial aviation that the initiative is

not addressing, 12 were on-ground fatalities, and 2 resulted from other

causes.2 The on-ground accidents each involved the death of a single

worker or unauthorized individual at the airport. Most of these accidents

occurred near the boarding gate or ramp. For example, several employees

were fatally injured when struck by an aircraft’s propeller or nose gear

2

In one of these two accidents, a private general aviation aircraft collided in flight with a

commuter aircraft, which landed safely. The other accident involved the on-board fatality of

a pilot who entered an unpressurized area of a cargo aircraft. The co-pilot landed the

aircraft without further incident.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 39 GAO/RCED-00-111 Safer Skies Initiative

during the course of their work. Of the on-ground fatalities, two resulted

from individuals gaining unauthorized access to airport areas that should

have been secured, nine involved various airline or airport employees who

sustained injuries in the workplace, and one involved a passenger who fell

out of an aircraft catering door and onto the ground. Because on-ground

accidents accounted for 14 percent of the 85 fatal accidents in commercial

aviation in 1988-97, reducing the fatal accident rate by 80 percent by 2007

will be difficult if these safety problems are not addressed.

FAA has initiatives to address some of the safety problems that caused onground fatalities, but it is unclear how systematically these problems are

being addressed. Specifically, FAA’s Office of Civil Aviation Security

oversees airline and airport programs to limit access to secure areas to

authorized individuals. The status of FAA’s efforts to address workplace

safety issues that resulted in on-ground fatalities is less clear. FAA is

responsible for regulating the safety and health aspects of the work

environment of aircraft crew members when the aircraft is in operation.

However, FAA has not promulgated specific regulations that address all

employee safety and health issues associated with working conditions on

aircraft. FAA held a public meeting in December 1999 to gather information

on issues associated with working conditions on and around aircraft and to

determine whether additional regulations should be proposed. However,

FAA does not currently have a group addressing workplace safety issues

and could not identify any regulations, guidance, or other initiatives that

have been developed to address the types of workplace safety problems

that caused most of the on-ground fatalities.

Improving Commercial

Aviation Safety Involves

Considering More Than

Reducing the Fatal Accident

Rate

Looking at the number of fatalities associated with various safety

problems, as well as their contribution to the fatal accident rate, provides

additional perspective on Safer Skies’ commercial aviation agenda.

Reductions in the fatal accident rate are closely linked to reductions in the

number of fatal accidents. Following this logic, the greatest reductions in

the fatal accident rate can be achieved by eliminating the safety problems

that caused the greatest number of accidents with one or more fatalities.

However, strict adherence to the goal of reducing the fatal accident rate

could result in focusing attention and resources on the causes of accidents

that resulted in single fatalities, rather than on those causes that result in

multiple fatalities, as well as multiple accidents. In choosing which safety

problems to address, the commercial aviation steering committee selected

safety problems that will help reduce fatalities, as well as the fatal accident

rate.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 40 GAO/RCED-00-111 Safer Skies Initiative

The fatal accident rate in commercial aviation can most quickly be reduced

by addressing the three safety problems that form the core of the Safer

Skies agenda in commercial aviation: CFIT, loss of control, and approach

and landing. These three safety problems accounted for 34 fatal accidents

involving larger aircraft that commercial aviation teams are handling and

24 additional small commuter accidents that general aviation teams are

handling. (See table 5.) If the initiative is successful in developing and

implementing interventions to eliminate these three safety problems for

both large aircraft and small commuter aircraft, it would make progress

toward preventing the kinds of safety problems that caused 68 percent of

the fatal accidents in 1988-97. If the initiative could successfully eliminate

all six safety problems on its agenda for commercial aviation, it would

approach the goal of an 80-percent reduction in the fatal accident rate.

However, other safety problems actually resulted in more fatal accidents

and thus could reduce the fatal accident rate more quickly if eliminated.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 41 GAO/RCED-00-111 Safer Skies Initiative

Table 5: Numbers of Fatal Accidents and Fatalities in Commercial Aviation 1988-97

a

Totals do not add to 100 because of rounding.

Source: GAO’s analysis of data from FAA and NTSB.

The initiative could approach the 80-percent goal more quickly by

eliminating on-ground accidents, which caused more fatal accidents in

commercial aviation than all other safety problems except loss of control

and CFIT. On-ground accidents caused 12 fatal accidents in commercial

aviation—14 percent of the total. While the safety problems that caused onFatal accidents Fatalities

Number Percentagea Number Percentagea

Commercial aviation accidents to be

addressed by Safer Skies commercial

aviation teams

Loss of control 19 22.4 530 30.2

Approach and landing 9 10.6 128 7.3

CFIT 6 7.1 339 19.3

Runway incursion 4 4.7 25 1.4

Weather 3 3.5 16 0.9

Uncontained engine failure 2 2.4 113 6.4

(Subtotal) 43 50.6 1,151 65.5

Commercial aviation accidents involving

small commuter aircraft to be addressed

by Safer Skies general aviation teams

CFIT 16 18.8 38 2.2

Loss of control 7 8.2 37 2.1

Approach and landing 1 1.2 5 0.3

(Subtotal) 24 28.2 80 4.6

Commercial aviation accidents to be

addressed by other FAA safety initiatives

Structural problems 2 2.4 10 0.6

Fuel tank explosion 1 1.2 230 13.1

Sabotage 1 1.2 270 15.4

(subtotal) 4 4.8 510 29.1

Gate/ramp 12 14.1 12 0.7

Other 2 2.4 3 0.2

Total 85 100 1,756 100
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 42 GAO/RCED-00-111 Safer Skies Initiative

ground accidents merit addressing, the safety problems that Safer Skies’

commercial aviation team has chosen to address resulted in multiple fatal

accidents and many more fatalities. For this reason, the initiative will

probably have more impact on improving the safety of air transportation

for the majority of the nation’s passengers than addressing other safety

problems, such as on-ground accidents, whose elimination could reduce

the fatal accident rate more but would save fewer lives.

While focusing on reducing the fatal accident rate by addressing the safety

problems that caused the most commercial aviation accidents, the

approach taken by the initiative also resulted in choices that recognized

where the greatest number of fatalities have occurred or could occur. The

three major problems addressed by the initiative’s commercial aviation

teams (CFIT, loss of control, and approach and landing) accounted for 57

percent of the 1,756 fatalities in 1988-97. This rises to 66 percent when all

six safety problems on the commercial aviation agenda are considered. The

additional small commuter accidents that are to be addressed by general

aviation teams account for another 5 percent of the total fatalities. The only

other safety problems that resulted in hundreds of fatalities were sabotage

and fuel tank explosions. The initiative did not focus on these problems for

two reasons. First, only one fatal accident resulted from each of these

safety problems in 1988-97. Second, FAA already has initiatives under way

to address both sabotage and fuel tank explosions. The initiative did

include two safety problems on its commercial aviation agenda that each

accounted for only about 1 percent of the fatalities in U. S. commercial

aviation during this period. However, the initiative recognized the potential

of runway incursions, which accounted for 25 U.S. fatalities, to result in

hundreds of fatalities. While weather resulted in few commercial aviation

accidents and 16 fatalities, the commercial aviation steering committee

members felt that the problems of turbulence and icing merited attention.

In contrast, the 12 on-ground accidents each resulted in a single fatality

that together accounted for fewer than 1 percent of the nation’s

commercial aviation fatalities. Eliminating the safety problems that caused

on-ground fatalities could reduce the fatal accident rate more quickly than

eliminating either CFIT or approach and landing accidents that involved

large commercial aircraft. The commercial aviation steering committee has

selected safety problems that will help reduce fatalities, as well as the fatal

accident rate. (See fig. 4.)
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 43 GAO/RCED-00-111 Safer Skies Initiative

Figure 4: Fatalities in U.S. Commercial Aviation by Safety Problem, 1988-97

Source: FAA.

The Safer Skies Initiative

Has Adopted the 80-Percent

Goal for Commercial

Aviation

The Safer Skies initiative and FAA have adopted the goal of reducing the

fatal accident rate for commercial aviation by 80 percent by 2007.

Specifically, the goal is to reduce the fatal accident rate for commercial

aviation from a 1994-96 baseline of 0.037 fatal accidents per 100,000 flight

hours to 0.007 fatal accidents per 100,000 flight hours in 2007. The meaning

of this goal can be more readily understood by considering the current

number of fatal commercial aviation accidents and the number of accidents

projected for 2007 if further safety improvements are not undertaken. In

1994-96, the United States averaged six fatal commercial aviation accidents

each year. Given the projected growth of commercial aviation, we estimate

that this number could increase to nine in 2007 if safety is not improved. If

the initiative achieves the goal of an 80-percent reduction in the fatal

accident rate for commercial aviation, we estimate that the number of fatal

accidents expected in 2007 would drop to two. (See table 6.)

Fatalities

0

100

200

300

400

500

600

Loss of 

Control

CFIT

Approach 

and landing

Uncontained 

engine failure

Runway 

incursion

Weather

Commercial aviation teams

CFIT

Loss of 

control

Approach 

and landing

Referred to the general 

aviation steering committee

Sabotage

Fuel tank 

explosion

Other

Other FAA efforts Other

Gate/ramp

Structure
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 44 GAO/RCED-00-111 Safer Skies Initiative

Table 6: Number of Past and Projected Fatal Accidents by Type of Aviation

Operation

a

The annual average is for the baseline years chosen by FAA and Safer Skies steering

committees in establishing their goals: 1994-96 for commercial aviation and 1996-98 for

general aviation.

b

To project the number of fatal accidents likely in 2007, we used FAA’s most recent projected

growth rates for flight hours in commercial aviation and general aviation, which were

released in March 2000. The updated growth rate for general aviation was larger than the

earlier growth rate available to the general aviation steering committee, which projected 437

fatal accidents in general aviation for 2007.

c

Given FAA’s most recent projected growth rates, the Safer Skies goal for commercial

aviation of an 80-percent reduction in the fatal accident rate would likely result in two fatal

accidents in 2007. The general aviation steering committee set a goal of reducing the

number of fatal general aviation accidents to 350 in 2007.

d

For commercial aviation, we computed the number of fatal accidents that would result

from decreasing the fatal accident rate in 2007 by 80 percent. For general aviation, we

computed the number of fatal accidents that would result from decreasing by 80 percent the

number of such accidents projected for 2007.

Source: GAO’s analysis of data from FAA and NTSB.

Accident Data and

Other Resources Were

Used to Identify Safety

Problems That Caused

Many Fatal Accidents

in General Aviation

The general aviation steering committee used available accident data,

safety reports, and professional expertise in aviation to identify safety

problems that caused many of the fatal accidents in general aviation. The

six safety problems chosen were controlled flight into terrain, loss of

control, aeronautical decision-making, runway incursions, weather, and

survivability. Steering committee members told us that they selected these

safety problems after reviewing the available data on general aviation

accidents and past industry and government-sponsored safety reports on

general aviation. They said that the NTSB accident reports were

challenging to analyze because many lacked the detail needed to determine

the root causes of accidents. They noted, for example, that most general

aviation aircraft are not equipped with such key equipment as flight data

Number of fatal accidents

Type of operation

Annual

averagea

Projected

for 2007b

Safer Skies

goal for 2007c

80-percent

reductiond

Commercial

aviation

6 922

General aviation 380 484 350 97

Total 386 493 352 99
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 45 GAO/RCED-00-111 Safer Skies Initiative

recorders that would help identify the safety problems that caused the

accidents. To meet these additional challenges, FAA developed a training

course tailored to the needs of those responsible for analyzing general

aviation accidents. Both FAA and industry members attended this course

before starting the analysis phase. The general aviation analysis reports on

CFIT and weather also included recommendations to address problems

with the quality of the data on general aviation accidents. In response to

these recommendations, the general aviation steering committee chartered

a team in April 2000 to develop strategies to (1) provide increased detail

about factors that have contributed to or caused general aviation accidents

and incidents and (2) improve the quality and timeliness of estimates of

general aviation activity. Members of the steering committee told us that, in

cases where the safety problems that caused the fatal accidents were

unclear, they used their experience as either pilots or experts in general

aviation to determine the possible causal factors involved in the accidents.

Members of the steering committee also examined past industry and

government reports on the causes of general aviation accidents. One key

report was the Nall Report, a report on general aviation accident trends and

factors published annually by the Aircraft Owners and Pilots Association’s

Air Safety Foundation. According to the 1998 Nall Report, the major causes

of fatal general aviation accidents were weather, loss of control or other

errors during flights in which the pilot was maneuvering the plane, and

accidents on approach to the airport. Another key report was FAA’s study

of the causes of general aviation CFIT accidents.3 FAA’s study concluded

that CFIT accidents accounted for 17 percent of the general aviation

fatalities and 32 percent of general aviation accidents in weather conditions

requiring pilots to have instrument ratings to fly.

Steering committee members also told us that several reports indicated

growing problems with runway incursions involving general aviation

aircraft. For example, a study by DOT’s Office of Inspector General showed

that general aviation pilots caused the majority of runway incursions

attributable to pilot error in 1990-96.4 Members of the steering committee

told us that they also decided to address survivability in an effort to

3

General Aviation Accidents, 1983-1994: Identification of Factors Related to ControlledFlight-Into Terrain (CFIT) Accidents, U.S. DOT, FAA (DOT/FAA/ARR-100-97-2, July 1997).

4

Runway Incursion Program, DOT, FAA (AV-1998-075, Feb. 9, 1998).
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 46 GAO/RCED-00-111 Safer Skies Initiative

decrease the number of fatalities among those who survive the impact of a

crash but ultimately die from their injuries.

Although the data available on general aviation accidents are less detailed

than those available on commercial aviation accidents, the general aviation

problems the initiative plans to address represent reasonable choices. Most

of the safety problems chosen have been identified in past safety reports

and NTSB accident reports as major causes of fatal accidents in general

aviation. These include weather, loss of control, CFIT, and runway

incursions. Aeronautical decision-making has also been cited repeatedly as

a factor in such safety problems as weather, when pilots exercise judgment

about whether to depart or turn back when faced with potential danger. In

addition, aeronautical decision-making includes those decisions made

relating to aircraft maintenance. Most of the Safer Skies participants, FAA

officials, and aviation experts we interviewed concurred that the six

general aviation safety problems to be addressed by the initiative are

reasonable ones that will help to reduce the fatal accident rate.

The Safer Skies Initiative

Adopted a Goal of Reducing

Fatal Accidents in General

Aviation to 350 in 2007

Although both the White House and congressional commissions on aviation

safety called for an 80-percent reduction in the nation’s fatal accident rate,

FAA and the Safer Skies initiative applied this goal only to commercial

aviation and adopted a less aggressive accident reduction goal for general

aviation. The goal is to reduce the number of fatal general aviation

accidents to 350 in 2007. This represents a 20-percent reduction in the

number of fatal accidents that would likely result from projected growth in

general aviation. Because general aviation accounted for 98 percent of U.S.

fatal accidents in 1988-97, the goal of an 80-percent reduction in the

nation’s fatal accident rate set forth by the two major aviation commissions

is unreachable if these fatal accidents are not greatly reduced. The

congressionally mandated commission on aviation safety discussed the

fatal accident rates for all kinds of aviation operations, including general

aviation. Because this commission did not explicitly apply the 80-percent

goal to general aviation, it remains unclear whether it intended the goal to

apply to general aviation as well as commercial aviation. The goal

adopted—350 fatal accidents—contrasts sharply with the 97 fatal accidents

that would likely result if the 80-percent goal were achieved. (See table 6.)

The steering committee did not adopt the 80-percent goal for general

aviation because of strong objections from the general aviation community.

Representatives of the general aviation community argued that, given the

varied experience levels of its pilots, reducing fatal accidents by 80 percent
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 47 GAO/RCED-00-111 Safer Skies Initiative

would be impossible without grounding the fleet. One general aviation

representative said that there was a prevailing concern in the general

aviation community that any agreement on a solid goal would lead to more

regulation and less growth. In addition, these representatives objected to

establishing a goal that involved a fatal accident rate. The fatal accident

rate for general aviation is calculated by dividing the number of fatal

accidents by the number of flight hours. Data on general aviation flight

hours are estimated using an annual survey of general aviation operators

conducted by FAA. Response to the survey is voluntary. Because the flight

hours are estimated on the basis of this survey, representatives of the

general aviation community questioned the reliability of these data and

expressed concern about using flight hours to calculate past and future

fatal accident rates. As a result, the Safer Skies steering committee for

general aviation agreed not to use the survey data on flight hours to

calculate a fatal accident rate until the data are more reliable. Instead, the

accident reduction goal for general aviation was expressed in terms of the

number of fatal accidents, rather than the fatal accident rate.

To set its goal of reducing fatal accidents to 350, the general aviation

steering committee reviewed available data on fatal accidents. The steering

committee found the number had declined fairly steadily since 1990 in

response to past initiatives to improve safety. The data used by the steering

committee showed that, in 1996-98, an average of 379 fatal general aviation

accidents occurred each year.5 The steering committee used this average

and the 1.6-percent annual growth expected in general aviation to project

that 437 accidents would occur in 2007 if additional safety initiatives were

not undertaken. They agreed that a reduction to 350 fatal accidents would

be achievable. This represents a 20-percent reduction in the number of fatal

accidents that they estimated would occur without additional safety

initiatives (437). According to projections by the steering committee and

the general aviation community, a reduction of this magnitude would

prevent 363 accidents from 2000 through 2007.

The goal of reducing the number of fatal accidents to 350 in 2007 is

probably achievable, but this goal is not likely to push the general aviation

community toward more safety improvement as aggressively as it could.

5

The general aviation steering committee used preliminary NTSB data to compute the

average number of fatal accidents for 1996-98, which resulted in an annual average of 379. In

table 6 and throughout our report, we used the official NTSB accident statistics for 1999,

which were released after the steering committee’s projections. This resulted in a slightly

higher annual average of 380 fatal accidents in general aviation.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 48 GAO/RCED-00-111 Safer Skies Initiative

We believe that this goal is achievable for two reasons. First, although the

level of general aviation activity has increased, the number of fatal

accidents decreased to 354 in 1999, a decrease of 17 percent since 1994.

Both FAA and industry officials attributed this decrease in part to ongoing

safety initiatives. The goal of 379 accidents established for each of the next

3 years represents a 7-percent growth in the current number of fatal

accidents. Second, the goal of 350 accidents set for 2007 is only 4 fewer

fatal accidents than occurred last year. Hence, the long-term goal is

achievable if the general aviation community is able to hold its number of

fatal accidents steady as its air traffic grows by an expected 2.2 percent per

year in the coming decade.

We recognize that an 80-percent reduction in fatal accidents is probably not

achievable in general aviation at this time because of the diversity in pilots’

experience levels, aircraft types, and operating environments. However, we

believe that a more aggressive goal would encourage greater efforts by

general aviation operators, manufacturers, associations, and FAA to make

safety improvements in general aviation operations that could save lives.

Improving Cabin Safety

Is Important but Will

Have

Little Impact on

Lowering the Fatal

Accident Rate

Improving cabin safety is unlikely to have much impact on reducing the

overall fatal accident rate. In contrast to the safety problems addressed by

the commercial and general aviation steering committees, the safety

problems addressed by the cabin safety steering committee have not

resulted in numerous fatalities, and few data are available on any injuries

that result from these problems. The Safer Skies initiative identified only

two fatalities in U.S. commercial aviation in 1988-97 related to cabin safety

problems.6 While passengers and crew have been injured in the cabin

environment, few data exist on these incidents because air carriers are not

required to report such incidents unless they involve a serious injury or

fatality. The study of cabin safety problems thus relies more on information

shared by flight attendants and air carriers than on analysis of the limited

data available. Because cabin safety resulted in few fatalities and affords

few data for analysis, it is arguable whether it was appropriate to include

cabin safety issues in an initiative directed at reducing the fatal accident

rate through a data-driven analysis of safety problems.

6

Both accidents resulted in the death of a passenger who was not secured by a seat belt

when the aircraft encountered turbulence. The commercial aviation steering committee lists

weather as the safety problem that caused these two accidents.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 49 GAO/RCED-00-111 Safer Skies Initiative

Although not appropriate for Safer Skies’ focus on the safety problems that

caused fatal aviation accidents, cabin safety issues are an appropriate topic

for FAA to address jointly with the aviation industry. NTSB and flight crews

have raised concerns about the potential for injuries and fatalities in the

cabin. FAA and industry were jointly studying cabin safety problems before

the initiative was announced. The safety problems under study included

those involving child restraint systems, passenger seatbelt use, passenger

interference with crew, and carry-on baggage. Concerns about these safety

problems are not new. For example, NTSB has long advocated FAA’s

requiring the use of child restraints for passengers under the age of 2. NTSB

was concerned enough about the use of child restraints to launch a

campaign aimed at making parents aware of the benefit of putting children

in approved child restraint systems and to declare 1999 as the “year of child

transportation safety.” Similarly, representatives of air carrier crews have

expressed concern that the incidents of passengers interfering with crew

members are increasing.

Additional Work on

Future Hazards Could

Help Anticipate and

Prevent Fatal

Accidents

In December 1997, the congressionally mandated commission on aviation

safety recommended that FAA and the aviation industry jointly develop a

strategic plan to improve aviation safety and that the process “begin with

analysis of both previous and potential failures to meet safety

expectations.” These failures include accidents, incidents, insight from

flight operational data, and aviation system changes. The analysis of the

causes of past accidents provides insights into safety problems that exist

within the current aviation system, while the analysis of aviation system

changes can help anticipate future hazards that may arise from such

changes as growth and technological advances (e.g., vertical takeoff and

landing by aircraft). The approaches to the analysis of past safety problems

and future hazards are distinct. A data-driven approach is particularly

useful for analyzing the safety problems that caused past fatal accidents.

Data on nonfatal accidents and incidents can also be used to identify and

address safety problems that did not result in fatalities but could have. The

data-driven approach is based on the assumption that identifying a problem

is possible where historical data are available. While this approach can be

used to address the safety problems in the current operating environment,

other types of analyses may be more useful for anticipating and preventing

the safety problems that could result in new types of fatal accidents. For

example, the anticipated growth in air traffic will lead to more congestion

around airports, increasing the possibility of runway incursions and midair

collisions near airports. Anticipating how changes in the aviation industry
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 50 GAO/RCED-00-111 Safer Skies Initiative

may increase existing safety problems or bring about new ones can better

position both FAA and the aviation industry to prevent accidents.

While FAA, Safer Skies, and industry groups have made progress in the

analysis of the causes of past accidents and incidents, efforts to analyze

and anticipate future hazards are more preliminary. The Joint Safety

Strategy Initiative in Europe7 has formed a work group to develop a method

for examining future hazards. A number of FAA staff participate in this

work group, which should facilitate the cooperative exchange of ideas and

information on this topic. As of April 2000, the Safer Skies initiative had not

established a process for analyzing future hazards. A systematic analysis of

the changes occurring in the aviation industry could enhance Safer Skies’

ongoing efforts to reduce the fatal accident rate. Several of the aviation

experts interviewed suggested that the initiative could benefit from going

beyond the analysis of data on past accidents to consider safety problems

that may arise from rapid changes in the aviation operating environment.

Participants on Safer Skies’ commercial aviation steering committee also

indicated that while data-driven approaches are helpful, it is also important

to consider future hazards. FAA’s Director of the Aircraft Certification

Service8 said that the initiative’s first priority was to understand and

eliminate the safety problems that caused past accidents but that the

commercial aviation steering committee also plans to address future

hazards and recently added this topic to its agenda for consideration.

Because work on future hazards could help anticipate and prevent fatal

accidents, this topic is important for the Safer Skies steering committees to

address, especially as it applies to commercial aviation. Coordinating this

effort with the work initiated by European and FAA staff on future hazards

should help avoid duplication of effort and foster awareness of and

solutions to these potential problems internationally.

Conclusions The premise of both the White House and congressional commissions on

aviation safety was that data on past and possible future causes of

accidents could be used to focus resources on substantially reducing the

fatal accident rate. While the Safer Skies initiative has made significant

7

The Joint Safety Strategy Initiative includes members from European aviation

manufacturers, associations, and regulators.

8

The Director of the Aircraft Certification Service co-chairs the commercial aviation

steering committee.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 51 GAO/RCED-00-111 Safer Skies Initiative

strides, it has not yet carried out this mandate as fully as it could. The six

safety problems that the initiative is addressing accounted for almost 80

percent of the fatal accidents in commercial aviation in 1988-97. Our review

showed that the initiative and FAA have work under way to address these

and other safety problems in commercial aviation. However, the initiative

has not challenged all sectors of the aviation community to push

aggressively for safety improvements. Although the initiative has adopted

the challenging goal of reducing the fatal accident rate for commercial

aviation by 80 percent by 2007, general aviation is not being asked to set a

similarly challenging goal. While an 80-percent reduction in fatal accidents

is probably not achievable in general aviation at this time, the goal adopted

by the initiative does not push the general aviation community toward

implementing the kinds of interventions that could substantially lower the

fatal accident rate. A more rigorous goal would encourage greater efforts

by general aviation operators, manufacturers, associations, and FAA to

make needed safety improvements. In addition, many of the interventions

developed to improve general aviation safety could also benefit small

commuter operators and pilots, but this benefit will not be realized without

a systematic way of ensuring that training and other interventions are also

directed at small commercial aviation operations. Finally, the Safer Skies

initiative and most aviation safety studies to date have focused on the

causes of past accidents. While analyses of accident data are useful for

determining the causes of past accidents, reducing fatal accidents during a

period of rapid growth in aviation will probably require the analysis of the

changing aviation environment to anticipate future safety problems.

Preliminary international efforts have been initiated to consider future

hazards, and integrating these efforts with Safer Skies’ work would

enhance the initiative’s efforts to reduce the fatal accident rate.

Recommendations To further reduce the nation’s fatal accident rate and save lives in the type

of aviation operation that causes the most fatal accidents and fatalities, we

recommend that the Secretary of Transportation direct the FAA

Administrator to work with the general aviation community to

• set a more challenging goal for reducing the number of fatal general

aviation accidents by 2007,

• set interim goals to assess progress toward this new goal, and

• ensure that training and other interventions that emerge from general

aviation teams are communicated to small commuter operators and

pilots who may benefit from them.
Chapter 2

The Safer Skies Initiative Should Help

Improve Aviation Safety

Page 52 GAO/RCED-00-111 Safer Skies Initiative

Agency Comments DOT and FAA officials concurred with our recommendations aimed at

setting a more challenging interim goal and long-term goals for general

aviation and said that they planned to do so in the future. However, the

officials noted that existing general aviation accident data are too

inaccurate to be used as the basis for setting an accident reduction goal.

The general aviation steering committee has established a work group to

recommend ways to improve the quality of general aviation data. The

officials stated that FAA and the general aviation community would review

the accident reduction goal when the quality of the data improves.

DOT and FAA officials disagreed with our recommendation aimed at

ensuring that training and other interventions emerging from general

aviation teams are communicated to small commuter operators because

they believe that mechanisms already exist to do this. The officials

explained that a number of associations representing smaller commuter

aircraft participate on the general aviation steering committee and on its

analysis and implementation teams. These organizations provide conduits

for transmitting interventions developed by the general aviation teams to

small commuter operators. We agree that these organizations may facilitate

the transfer of safety interventions developed by the general aviation teams

to small commuter air carriers. However, it will be difficult to achieve the

mandated 80-percent reduction in commercial aviation fatalities without

systematic improvements in the safety record of small commuter air

carriers, which accounted for 28 percent of fatal commercial aviation

accidents. We believe that Safer Skies would benefit from a systematic plan

for ensuring that interventions developed by general aviation teams are

communicated to and implemented by small commuter operators. For this

reason, we did not modify or delete our recommendation.

DOT and FAA officials disagreed with our recommendation calling for an

analysis of future safety problems arising from the rapid growth and

changes in aviation. The officials noted that efforts involving FAA, Safer

Skies, and the European aviation industry are already under way to address

future hazards in aviation. On the basis of the information presented by

DOT and FAA officials, we withdrew this recommendation.
Page 53 GAO/RCED-00-111 Safer Skies Initiative

Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions Chapte3r

Joint FAA and industry teams have started work on 13 of the 16 problems

being addressed by the initiative. A two-part process has been developed

for use by these teams to first analyze accident and incident data and then

to use that analysis to identify, select, and implement safety interventions

to help prevent accidents in the future. That process is reasonable and has

allowed FAA and industry groups to reach consensus on how to address

safety problems identified under the initiative. This process was not used

to address cabin safety problems because the cabin safety steering

committee had already begun its work before the process was developed.

The Safer Skies teams have made progress primarily in those areas that had

been studied extensively in the past for which widely supported

recommendations already existed. The interventions recommended for five

problems are now being implemented: uncontained engine failure and

CFIT in commercial aviation and passenger seatbelt use, child restraint

systems, and carry-on baggage in cabin safety. The process being used will

require more extensive analysis in the future as teams begin to address

safety problems that have not been studied previously. Finally, the success

of the interventions that the Safer Skies teams have chosen to address

these long-standing safety problems depends in part on effective

implementation. Our past work has shown that FAA does not consistently

follow through on implementing key safety recommendations.

Furthermore, FAA and the aviation industry began implementing some of

the Safer Skies safety interventions before having a process in place to

track their progress. The initiative has developed a process for tracking the

implementation of interventions to improve safety in commercial aviation.

However, the implementation of Safer Skies’ interventions is not assured

because the tracking system for commercial aviation is not sufficiently

detailed to assess progress in implementing interventions. Furthermore,

the cabin safety steering committee implemented its interventions without

having a tracking process in place, and the general aviation steering

committee is working toward the final approval of interventions to address

two safety problems without having a tracking process. Without a complete

tracking process, FAA and the industry cannot ensure that the initiative will

improve aviation safety in each of these areas.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 54 GAO/RCED-00-111 Safer Skies Initiative

The Safer Skies

Methodology Is Based

on Previous Efforts to

Identify Safety

Problems

For the Safer Skies initiative, FAA and the aviation industry jointly

developed a two-part process to analyze accident data and then to choose

from among the possible interventions. This process grew out of a previous

FAA effort that used a data-driven approach to identify threats to aviation

safety and develop interventions to address those threats. During the first

part of this process, an analysis team reviews accident data to determine

what went wrong, why it went wrong, and what interventions might be the

most effective in preventing similar accidents in the future. The second

part of the process involves another team that assesses the feasibility of

each potential intervention, prioritizes the interventions on the basis of

their effectiveness and feasibility, and submits plans for implementing

projects to the steering committee for approval. However, as we discuss

later in this chapter, the steering committee addressing cabin safety

problems did not use this process.

The Initiative Uses a TwoPart Process to Analyze

Data and Identify

Interventions

The initiative uses a two-part process to analyze data and identify

interventions to address safety problems in commercial aviation and

general aviation. This process is modeled on an analysis of the most

significant threats to aviation safety conducted in 1997 by staff from FAA’s

Aircraft Certification Service. The two-part process was developed for use

by the teams addressing safety problems in commercial aviation but has

also been used by the general aviation teams with some modifications.

Under the process, the steering committee forms an analysis team for each

aviation safety problem. The team, which includes members from FAA and

the aviation industry, reviews accident data, determines accident causes,

and identifies possible interventions to prevent future accidents. For

selected accidents, the team develops a detailed sequence of events that

includes the actions by pilots and air traffic controllers as well as any

system or equipment failure. The team determines what went wrong and

why and then considers various interventions that could have prevented

the accident. In its final report, the analysis team ranks all of the identified

interventions by their effectiveness in preventing similar accidents and

presents them to the steering committee for further action.

Once the analysis team completes its work on a safety problem, the

steering committee forms a second team to assess the feasibility of

implementing the interventions suggested by the analysis team. The

implementation team assesses feasibility in six areas: the cost of the

intervention; the time needed to implement it; whether it requires

regulatory changes; technical feasibility; the practicality of the project
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 55 GAO/RCED-00-111 Safer Skies Initiative

within the operating environment or the nationwide aviation system; and

political feasibility. The implementation team prioritizes the interventions

by both effectiveness and feasibility and then presents the resulting

prioritized list to the steering committee. Once the steering committee

initially approves an intervention, the implementation team develops a

detailed project plan for implementation that is sent to the steering

committee for final approval. Once detailed plans are approved, the

interventions are then implemented by the responsible organizations.

The general aviation teams have made some modifications to the analysis

process initially developed for use by the commercial aviation teams.

Although the first commercial aviation analysis team considered feasibility

as well as effectiveness, the two-part process ultimately approved for

commercial aviation teams considers only effectiveness at the analysis

stage. Any consideration of such matters as cost and the need for

developing new regulations is left to the implementation team. In contrast,

the general aviation analysis teams consider both effectiveness and

feasibility. Our review of the general aviation analysis team’s reports for

CFIT and weather confirmed that such feasibility criteria as cost and the

need for new regulations have been considered far earlier in the

assessment of general aviation interventions than in the process now used

by commercial aviation teams. While other feasibility factors are also

considered, cost, the avoidance of interventions that would require new

regulations, and acceptability to the general aviation community have

weighed heavily in the choice of interventions to address general aviation

safety problems. In emphasizing cost and acceptability to the aviation

community, the general aviation teams have selected training and other

interventions that will be more affordable to general aviation pilots.

The Cabin Safety Team Used

a Different Approach

While the initiative is using a systematic, defined approach to consider

ways to address safety problems in commercial and general aviation, a

different approach was used to address cabin safety problems. Several

months before the announcement of the Safer Skies initiative, FAA

established the Partners in Cabin Safety (PICS) team to provide

information to the public about four cabin safety problems: passenger

interference with flight crews, the safety benefits of greater use of seat

belts by passengers, the safety benefits of child safety restraints, and

potential safety issues arising from the stowage of carry-on baggage.

According to PICS team members, FAA identified these problems before

assigning them as tasks to the team in January 1998. Team members

discussed such additional issues as in-flight medical emergencies and cabin
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 56 GAO/RCED-00-111 Safer Skies Initiative

air quality but settled on the four that were eventually included. Unlike the

commercial and general aviation teams, the PICS team limited the possible

interventions to ones that did not require that FAA create new regulations,

a process that was viewed by some participants as too slow and unlikely to

result in consensus among various industry and government participants.

Consequently, the PICS team focused on interventions that involved

educating passengers.

The four cabin safety problems addressed differ in several important ways

from those safety problems addressed by the Safer Skies teams in

commercial and general aviation. First, the cabin safety problems resulted

in only two fatalities in U.S. commercial aviation from 1988 through 1997,

both involving passengers not using their seat belts when the aircraft

encountered turbulence. In contrast, during the same period of time, there

were more than 9,800 fatalities in all commercial and general aviation

accidents. Second, air carriers are not required to maintain or submit data

on cabin safety incidents unless they involve fatalities or serious injuries.

Since only limited historical data on cabin safety accidents and injuries

were available for analysis, the PICS team did not conduct a causal analysis

as has been done by the analysis and implementation teams for both

commercial and general aviation.

The PICS team disbanded in January 1999 after it completed the

development of passenger education materials. As part of its passenger

education efforts, the team distributed brochures on child restraint

systems from a previous campaign by FAA. In addition, the Luggage and

Leather Goods Manufacturers of America, along with FAA, developed a

brochure addressing carry-on baggage concerns, which the team members

were asked to distribute to airlines, luggage stores, and airports. It was also

put on FAA’s World Wide Web site for further distribution by interested

parties. Steering committee members and FAA officials also told us that the

team worked with air carriers to develop additional cabin announcements

for the stowage of carry-on baggage and the importance of seat belt usage.

Finally, the PICS team developed a passenger safety checklist for

publication on FAA’s Web site, which addressed passenger interference

with flight crews, seat belt usage, child restraint systems, and carry-on

baggage. This checklist, however, is not currently available on FAA’s Web

site. According to an official at FAA’s Flight Standards Service, the

passenger safety checklist project is on hold until the agency appoints a

new national resource specialist for cabin safety who will review the

document before it is made available to the public.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 57 GAO/RCED-00-111 Safer Skies Initiative

The Safer Skies

Initiative Has Made the

Most

Progress With

Problems Studied

Previously

Since the FAA Administrator announced the Safer Skies initiative in April

1998, work has started on 9 of the 12 safety problems to be addressed in the

commercial and general aviation safety areas. Teams have made the most

progress in selecting interventions for safety problems when they could

build on previous studies for which widely supported recommendations

exist. The commercial aviation steering committee plans to have work

started on all of the identified problems before the end of fiscal year 2000,

but the general aviation steering committee has not yet determined when

work on three of its six problems will begin. Table 7 shows the status of the

work on each of the 12 safety problems to be addressed in commercial and

general aviation as of April 1, 2000.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 58 GAO/RCED-00-111 Safer Skies Initiative

Table 7: Status of Safer Skies’ Activities for Commercial and General Aviation

a

The commercial aviation steering committee combined the implementation teams for CFIT

and approach and landing accidents because many of the interventions chosen by the

analysis teams for these two safety problems overlapped.

b

The joint FAA/industry team working on uncontained engine failure developed the

prototype process used by the Safer Skies analysis and implementation teams.

c

The analysis activities for runway incursions include both commercial and general aviation

accidents reviewed by a joint team.

d

The commercial aviation steering committee will form separate teams to address two

weather issues—turbulence and icing.

e

The general aviation CFIT and weather reports were presented to us as final reports.

However, in responding to our draft report, FAA told us that these reports had not received

final approval.

Source: GAO ‘s analysis of Safer Skies’ data.

Analysis team Implementation team

Team

formed

Final report

issued

Team

formed

Final report

issued

Commercial aviation

CFIT X XXa X

Loss of control X

Uncontained engine failureb X XX X

Runway incursionsc X

Approach and landing X X Xa X

Weatherd

Turbulence X

Icing

General aviation

CFIT X XX e

Loss of control

Runway incursions X

Aeronautical decision-making

Survivability

Weather X X X e
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 59 GAO/RCED-00-111 Safer Skies Initiative

The steering committees charged with developing interventions for each of

the safety problems first formed analysis teams to work on problems for

which major studies had already been done or were under way. The

ongoing and completed studies conducted by FAA and the industry

provided information the analysis teams could use to identify the causes of

accidents and potential interventions. For example, the Flight Safety

Foundation had completed an extensive study on CFIT, examining over 250

accidents and incidents worldwide. The foundation had also developed

training materials for pilots and made other recommendations to prevent

CFIT accidents. In another instance, the team analyzing weather-related

accidents involving general aviation aircraft identified 11 safety studies that

had preceded its efforts, all of which recommended interventions similar to

the ones the team ultimately identified. FAA participants on the Safer Skies

commercial aviation steering committee told us that beginning with

previously studied safety problems helped team members make progress in

developing the team’s two-part process for analyzing data and identifying

interventions and become comfortable with the analysis and selection

process before moving onto more complex issues that may involve original

research and analysis. However, this approach meant that work on another

area that is important for reducing the fatal accident rate in commercial

aviation did not start work until September 1999—17 months after the

initiative’s announcement. FAA identified loss of control as the single

largest cause of fatal commercial aviation accidents involving U.S.

operators.

To date, the implementation of interventions has concentrated mostly in

areas for which analysis and implementation were well under way or

complete when the initiative began. The first of the interventions to be

implemented addressed uncontained engine failure: FAA issued a series of

airworthiness directives requiring enhanced inspections of high-speed

rotating parts in certain jet engines.1 The directives require industry

maintenance personnel to perform additional, more detailed inspections to

check for cracks and other signs of irregularities whenever an engine is

disassembled for overhaul or maintenance. According to staff at FAA’s

Engine and Propeller Directorate, these directives affect more than 90

percent of the jet engines that U.S. airlines currently use. FAA and the

1

FAA issues airworthiness directives to address unsafe mechanical conditions that surface

after an aircraft has been certified and in use. The directives contain FAA’s requirements for

airlines to correct unsafe aircraft conditions that have occurred or are likely to occur in

aircraft of the same design.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 60 GAO/RCED-00-111 Safer Skies Initiative

industry are also taking steps to implement an intervention endorsed by the

commercial aviation team examining CFIT. The team has recommended

that enhanced navigational equipment be installed on new and existing

aircraft to warn pilots of impending crashes. Air carriers began installing

the enhanced navigational equipment to prevent CFIT accidents in their

aircraft before FAA issued the final rule requiring that the equipment be

installed in the commercial fleet. Specifically, air carriers began installing

the enhanced navigational equipment to prevent CFIT accidents in their

aircraft before FAA issued its final rule in March 2000 and before the

commercial aviation team working on CFIT issued its final report in June

2000. This equipment is now being included on some new aircraft, and

airlines had equipped about 4,000 aircraft already in service with the new

technology by December 31, 1999.

The timetable for analysis and implementation teams addressing the

problems included under the initiative has changed since the initiative was

announced in April 1998. According to the chairs of the Safer Skies steering

committees, some of these schedule changes occurred because the

analysis process took longer than anticipated. In other cases, changes to

the analysis approach required rescheduling Safer Skies’ efforts. For

example, FAA officials explained that the final report date for the

commercial aviation CFIT implementation team was rescheduled after the

steering committee decided that combining the CFIT and approach and

landing teams for the implementation analysis made sense because of

overlap in the interventions they had identified. Several high-priority

interventions to address CFIT accidents in commercial aviation were,

however, forwarded to the steering committee for final approval and

implementation without waiting for the implementation team’s final report.

An FAA co-chair of the general aviation steering committee told us that

they changed the start dates for several of the general aviation teams

because general aviation accidents are more numerous than commercial

aviation accidents and analyzing them proved more time-consuming than

anticipated. This FAA official also said that some general aviation groups

participating in the initiative do not have enough people or resources to

serve on multiple teams simultaneously. While we believe that these

decisions were justified, they also effectively mean that the interventions to

resolve some key safety problems will not be identified or implemented

until later than originally anticipated.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 61 GAO/RCED-00-111 Safer Skies Initiative

Early Experience

Indicates That Future

Problems Will Require

More Analysis

Additional analysis will be needed to identify interventions to address

current and future safety problems for which few or no previous studies

exist. Safer Skies teams relied initially on a limited number of case study

analyses to identify the causes of accidents and incidents, as well as the

interventions that could prevent them in the future. The teams compared

the results of these case studies with the causes and interventions

identified by previous studies to determine whether they are consistent.

For example, the team working on CFIT in commercial aviation completed

detailed event sequences for 10 accidents and found that the causes

identified and the interventions it recommended were similar to those of

prior studies. Safer Skies teams working on approach and landing

accidents in commercial aviation and weather-related accidents in general

aviation also compared the results of their analyses with those of prior

studies.

Along with other changes as the Safer Skies initiative has evolved, this

approach has been modified as teams addressed additional safety

problems. For example, the runway incursion analysis team expanded its

case studies to include incidents because there were so few fatal accidents

involving runway incursions. Similarly, the analysis team now working on

loss of control in commercial aviation has selected a larger number of case

studies because this safety problem has not been the subject of extensive

prior analysis.

Effective

Implementation Is

Critical Next Step in

Making Progress

Toward the Goals Set

for Reducing Fatal

Accidents

The Safer Skies initiative has identified the major safety problems to be

addressed, has made progress in identifying their root causes, and has

developed interventions to address some of them. Reducing fatal accidents

depends in part on the effective implementation of these interventions. As

discussed in chapter one, however, many of these safety problems are longstanding ones that have persisted in spite of previous studies and

recommendations. In addition, FAA has not consistently followed through

on implementing safety recommendations in the past. The Safer Skies

initiative does not yet have in place a process to track the implementation

of these interventions that is sufficiently detailed and covers interventions

chosen to improve safety in commercial aviation, general aviation, and

cabin safety.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 62 GAO/RCED-00-111 Safer Skies Initiative

The Success of Safer Skies

Interventions Depends on

Effective Implementation

Reducing the fatal accident rate in commercial aviation and the number of

general aviation accidents will depend in part on effective implementation

of the interventions chosen by the Safer Skies teams. Many of the safety

problems that the initiative addresses are long-standing ones that have

been studied extensively in the past. Actually resolving these problems has

proven difficult in the past and remains very challenging. Similar

interventions have been recommended, but the desired reductions in fatal

accident rates have not been achieved. For example, extensive prior

studies of CFIT and approach and landing accidents in commercial aviation

recommended many of the same interventions that are now being

implemented by the Safer Skies commercial aviation steering committee.

Furthermore, reaching the 80-percent goal in commercial aviation will

depend heavily on the successful implementation of interventions to

address the safety problems that caused the most fatal accidents: loss of

control, CFIT, and approach and landing. To reach the goal in commercial

aviation, interventions must be effectively implemented for both small

commuter aircraft and large commercial air carriers. Even after safety

interventions have been identified, implementing them has proven

challenging.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 63 GAO/RCED-00-111 Safer Skies Initiative

As DOT’s Inspector General and we have reported previously, FAA does not

consistently follow through on implementing safety recommendations. Our

review showed that FAA usually agreed with the recommendations on

aviation safety made by GAO, NTSB, and DOT’s Inspector General. FAA

had implemented 64 percent of the 256 recommendations that we

reviewed; however, FAA had not completed actions to implement the

remaining 36 percent of the recommendations.2 We found that FAA

sometimes did not establish time frames for implementing the

recommendations or did not meet established times for implementing

them. Similarly, DOT’s Inspector General found that of the 23 near-term

actions FAA planned for addressing runway incursions in its 1998 Action

Plan, 15 had not been completed on time.3 We found that even safety

recommendations that received specialized attention, intensive follow-up,

and heightened awareness among industry, the Congress, and the public

have not been fully implemented. For example, NTSB considered runway

incursions so serious that it repeatedly placed this safety problem on its

lists of critical safety recommendations in the early 1990s. Although FAA

concurred with NTSB’s recommendations, our review found that several of

the corrective actions needed had not been implemented, including actions

to improve (1) visibility at airports; (2) runway lighting, signage, and

surface markings; and (3) radar and related equipment to alert air traffic

controllers to impending runway incursions. FAA developed several plans

in the 1990s to decrease runway incursions. In spite of these programs, the

actual number of runway incursions has increased. DOT’s Inspector

General noted in 1999 that the number of runway incursions had increased

from 292 in 1997 to 325 in 1998, in part because FAA had not set aside the

funds needed to support the initiatives and projects in the runway

incursion action plan.4 As a result, FAA has made limited progress in

implementing its plan, and milestones have been missed and extended.

DOT’s latest performance report for fiscal year 1999 shows continuing

problems in this area. The actual number of runway incursions (321) was

19 percent higher than the goal of 270 established in DOT’s performance

plan.5

2

Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing Safety

Recommendations (GAO/RCED-96-193, Sept. 23, 1996).

3

Follow-up Review of FAA’s Runway Safety Program, DOT, FAA, (AV-1999-114, July 21, 1999).

4

Federal Aviation Administration: Aviation Safety, DOT Inspector General Report No. AV1999-069 (Statement of Alexis M. Stefani, Deputy Assistant Inspector General for Aviation,

U.S. Department of Transportation before the Subcommittee on Transportation and Related

Agencies, Committee on Appropriations, U.S. House of Representatives, Mar. 10, 1999).
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 64 GAO/RCED-00-111 Safer Skies Initiative

Industry participants in the Safer Skies initiative have voiced concern that

some interventions may not be implemented promptly or at all. Some of the

same Safer Skies participants questioned whether enough resources would

be available to complete the implementation of the selected interventions.

Without assurance of adequate resources, it is likely that the choice of

interventions by Safer Skies teams will be constrained by cost

considerations and the implementation of recommended interventions will

be incomplete. Effective implementation will also depend on having a

process for tracking the implementation of interventions to be carried out

by all Safer Skies participants, including FAA; other government agencies;

manufacturers; airlines; and other industry participants.

The Steering Committees

Have Not Yet Developed

Effective Processes for

Tracking the

Implementation of

Interventions

FAA and the aviation industry began implementing some of the Safer Skies

safety interventions before developing a systematic way of tracking the

progress being made. This occurred in part because the steering

committees incorporated some safety initiatives already under way and

endorsed the resulting interventions before they developed a systematic

tracking process. In addition, Safer Skies teams have recommended that a

few high-priority safety initiatives be started before final implementation

reports are issued. While moving forward on important safety initiatives

makes sense, ensuring their successful implementation depends on

effective tracking. Interventions have been implemented in both

commercial aviation and cabin safety with no tracking process in place.

The general aviation steering committee is moving toward approval of

interventions for CFIT and weather but has not yet developed a tracking

process. Several of the Safer Skies participants we interviewed voiced

some concerns about whether all the interventions being identified would

eventually be implemented, given FAA’s past problems in implementing

recommended safety improvements.

Tracking Has Been Limited

and Not Systematic

In its December 1997 report, the congressionally mandated commission on

aviation safety recommended that FAA’s and the industry’s strategic plan

include milestones for accomplishing specific tasks. The commission noted

that the plan should be detailed enough that milestones for accomplishing

specific tasks can be readily recognized by agency management and the

industry, as well as the public. In addition, the commission directed FAA to

5

DOT FY 2001 Performance Plan: FY 1999 Performance Report.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 65 GAO/RCED-00-111 Safer Skies Initiative

report periodically on where initiatives stand, why any delays are

occurring, and whether and why changes are being made to the plan. These

recommendations are in accordance with sound internal controls for

program management.

The Safer Skies initiative, which was announced in April 1998,

implemented a number of interventions without first developing a process

for tracking their progress. In some cases, these were interventions that

were developed by teams whose work was incorporated into the Safer

Skies effort. In commercial aviation, for example, FAA, relying on the work

of the uncontained engine failure team, published airworthiness directives

beginning in April 1999 to require more extensive inspections of aircraft

engines. The commercial aviation team working on CFIT also implemented

several interventions in or before September 1999. These included

interventions to verify the operational status of radar equipment to provide

minimum safe altitude warnings to pilots and to develop a template for

standard operating procedures to be used by airlines in training their pilots

in techniques to avoid CFIT accidents.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 66 GAO/RCED-00-111 Safer Skies Initiative

In September 1999, the commercial aviation steering committee recognized

the need for the systematic tracking of interventions and directed a work

group to develop a proposal. At the commercial aviation steering

committee’s meeting in January 2000, the work group presented its

proposal for a Joint Implementation Measurement Team. The team

designed the tracking process to provide a high-level report on whether

each intervention is being implemented as planned. Specifically, this team’s

responsibilities will include tracking whether the implementation of

approved interventions complies with the implementation plans and their

milestones; helping to predict the potential effectiveness of the proposed

interventions; and identifying ways of measuring whether the intervention

is achieving the desired risk reduction. The team will also provide a brief

explanation of what is causing noncompliance with the plan and whether a

solution has been found to resolve the problem. As conceived, the tracking

report is to be a high-level progress report that does not intrude on the

internal planning of the organizations responsible for carrying out the

interventions. The tracking report thus does not provide detailed

information on interim and long-term milestones or identify individuals

responsible for implementing the plan and preparing progress reports for

the tracking committee. Without more detailed information than is

currently provided in the proposed tracking report, it may be difficult for

the steering committee to assess progress in implementing interventions.

For example, the tracking team’s January report notes that FAA has

completed a plan for implementing two programs critical to gaining access

to safety data6 and that other industry and government groups have plans in

development. However, the tracking report provides no information about

the milestones established by FAA’s plan for establishing these key

programs, both of which have experienced delays in the past. After we

identified concerns about the tracking system, the commercial aviation

steering committee agreed that improvements are needed, and it is working

on revisions. A draft version provided for our review in June 2000 still

lacked key information about major commitments, deliverables, and

milestones.

Tracking implementation is even more critical for the more complex

initiatives whose success depends on coordinated efforts by both FAA and

the aviation industry. For example, successful implementation of the

6

These two programs are the Flight Operational Quality Assurance program and the Aviation

Safety Action Program. Both programs facilitate the collection, protection, and analysis of

safety data voluntarily submitted by airlines or pilots.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 67 GAO/RCED-00-111 Safer Skies Initiative

highest-priority intervention to prevent CFIT accidents in commercial

aircraft—the installation of enhanced aircraft navigational equipment to

warn pilots of impending crashes—requires coordination among many

parties:

• FAA must certify that the equipment works, issue technical standards

for manufacturers, and issue a final regulation to require that the

equipment be installed on new and existing aircraft.

• Aircraft manufacturers need to make the equipment standard on new

aircraft and retrofit it in older aircraft.

• Air carriers need to incorporate the appropriate procedures for

maintaining and using this equipment into maintenance and flight

manuals and to train pilots in its use.

• FAA needs to update its guidance to its inspectors so that they can

ensure that air carriers properly carry out their responsibilities for

training, maintenance, and use of the equipment.

Without a tracking system that provides more detailed information on the

implementation of complex interventions, the commercial aviation steering

committee will not have the information needed to ensure that they are

fully implemented in accordance with planned milestones.

The implementation of interventions to improve cabin safety has also not

been adequately tracked. The cabin safety steering committee, which

completed the development of passenger education materials before it

disbanded in January 1999, carried out most of its interventions with no

Safer Skies tracking process in place. However, we found that educational

materials related to passenger interference with crew had not been

distributed or made available on FAA’s Web page as of April 2000.

Furthermore, according to a member of the cabin safety steering

committee, the distribution of other cabin safety brochures was, in some

instances, never completed. The absence of a systematic process for

tracking Safer Skies interventions may have contributed to inaccuracies in

reporting on the status of cabin safety interventions. Specifically, the DOT

FY 2001 Performance Plan and FY 1999 Performance Report states that all

initiatives relating to cabin safety were completed as planned. However,

planned actions to include material on passenger interference with crew

had not been completed as of April 2000.

Finally, although the general aviation steering committee is reviewing draft

implementation team reports that recommend interventions to address

CFIT and weather, it has no process in place to track the implementation of
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 68 GAO/RCED-00-111 Safer Skies Initiative

interventions once they are approved. According to the FAA co-chairs of

the general aviation steering committee, this group has committed to track

the interventions selected but has not yet developed a process for doing

that and plans to discuss this issue at a future meeting. Without

coordinated, detailed implementation plans that assign responsibilities,

FAA and the Safer Skies steering committees will not be able to ensure that

all parties complete their portion of the plan and that implementation

occurs on time. In addition, as part of the Safer Skies process, FAA and the

general aviation community identified efforts that could be accomplished

in the short term or were already under way to address the safety areas to

be addressed by the initiative. FAA and the industry implemented a number

of these short-term initiatives, such as the development and distribution of

various safety videos and training aids. However, Safer Skies did not track

the implementation of these interventions or evaluate their effectiveness.

Conclusions The progress made by the initiative to date has resulted in the

implementation of interventions for five safety problems—two in

commercial aviation and three in cabin safety. However, a coordinated,

centralized method of tracking will be necessary to ensure full

implementation of these and future interventions. In the past, FAA has

developed plans to make safety improvements but has not consistently

implemented them successfully. An effective tracking system would

provide for identifying the individuals or entities responsible for

implementation, setting milestones, establishing resource estimates, and

preparing progress reports. Without a systematic tracking mechanism,

there is no assurance that any of the selected interventions will be fully

implemented. While the commercial aviation steering committee has

developed a system to track the implementation of the interventions it

approves, this system is not sufficiently detailed to ensure their

implementation. The general aviation steering committee, which is nearing

final approval on interventions to address safety problems related to

weather and controlled flight into terrain, is only now developing a tracking

system modeled after the one used by the commercial aviation steering

committee. Finally, nothing comparable has been developed to track

interventions recommended by the cabin safety teams.

Recommendations To ensure that interventions are implemented and that effective and

feasible interventions are identified in the future for issues that the

initiative has yet to address, we recommend that the Secretary of
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 69 GAO/RCED-00-111 Safer Skies Initiative

Transportation direct the FAA Administrator to advise the Safer Skies

steering committees to take the following actions:

• Develop a systematic way of tracking the implementation of

interventions approved by all Safer Skies steering committees. This

tracking system should include the identification of responsibility for

implementation, the establishment of short- and long-term milestones

and resource estimates, and the preparation of progress reports. The

progress reports should provide information on the detailed steps to be

taken by all government and industry participants to ensure the

successful implementation of each intervention. Progress reports

should highlight and explain any delays in meeting the milestones. This

system should be shared with the relevant Safer Skies steering

committees and FAA’s focal point for the initiative as well as with the

team that recommended the intervention.

Agency Comments DOT and FAA officials concurred with our recommendation on the need to

track the implementation of interventions to achieve results, but they

disagreed with the level of detail we advised. The officials stated that the

commercial aviation steering committee’s draft revised tracking system

provides better information for tracking the major commitments and

deliverables. The expectation is that more detailed implementation plans

will be maintained within each implementing organization. The officials do

not believe that it is realistic for steering committees to review the details

of every organization’s action plan. They also noted that the general

aviation steering committee is developing a tracking system similar to that

used to track commercial aviation interventions. We agree that the Safer

Skies initiative has taken steps to improve its tracking system for

commercial aviation and to work toward the development of a similar

system for general aviation. However, the revised tracking system provided

for our review in June 2000 did not clearly identify and include time frames

for major commitments and deliverables for each of the interventions

approved by the commercial aviation steering committee. We agree that

individual FAA and industry organizations responsible for implementing

Safer Skies interventions would logically have far more detailed systems

for tracking implementation than the steering committees. However,

without a reliable tracking system in place that contains basic information

on major deliverables, responsibilities, and time frames, FAA and Safer

Skies will not be in a position to ensure that recommended interventions

are implemented to improve aviation safety.
Chapter 3

The Safer Skies Initiative Has Made Progress

in Selecting and Implementing Interventions

Page 70 GAO/RCED-00-111 Safer Skies Initiative

DOT and FAA officials disagreed with our recommendation that FAA and

the Safer Skies steering committees should analyze a sample of safety

problems that were not studied previously. The officials presented

information that showed some Safer Skies’ work groups were using or

would be using a sample of previously unexamined safety problems in their

work. For this reason, we withdrew the recommendation.
Page 71 GAO/RCED-00-111 Safer Skies Initiative

Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to Evaluate

the Effectiveness of Most Interventions Chapte4r

Of the five Safer Skies teams that have begun implementing interventions,

only one has developed a performance measure to evaluate whether the

interventions it has selected are helping to reduce the safety problems that

cause fatal accidents and are worth what they cost. Such evaluations

depend on performance measures that serve as the yardsticks for

measuring progress toward program goals. The initiative’s ultimate goal is

saving lives by reducing fatal accidents. Federal law requires that federal

departments evaluate the effectiveness of the program activities for which

they request funding. FAA will evaluate progress toward its broad goals for

aviation safety using performance measures based on reducing the fatal

accident rate for commercial aviation and the number of fatal accidents in

general aviation. However, additional performance measures will be

needed for evaluating the effectiveness of the interventions selected by the

teams working on each of the safety problems. Most teams are still

analyzing data on safety problems and selecting safety interventions and

thus have not yet determined how to evaluate the effectiveness of

interventions selected. Although teams working on 5 of the 16 safety

problems have recommended interventions that are being implemented,

only one of these teams developed an adequate performance measure

before its interventions were implemented.

Federal Law Requires

the Development of

Performance Measures

as Part of the Budget

Process

To ensure that programs achieve their objectives and that funds are

expended wisely, federal law requires that each department develop

performance measures as part of its budget request. Performance

measures are the yardsticks used to evaluate the effectiveness of the

activities undertaken as part of federal programs. The initiative plans to

develop performance measures to evaluate the effectiveness of the

interventions it recommends to save lives by addressing the safety

problems that cause fatal accidents. However, developing good

performance measures can be difficult. While it is useful to establish a

baseline of information about past fatal accidents, they occur too rarely to

serve as performance measures to evaluate the effectiveness of

interventions. Years may elapse between specific types of fatal accidents,

such as uncontained engine failure, making it difficult to see trends or

evaluate the effectiveness of interventions. Instead, the initiative must

develop performance measures based on events that occur more frequently

and that can be linked closely to interventions.

A congressional mandate exists for the measurement and evaluation of all

federal programs. Performance measurement is a central premise of the

Government Performance and Results Act of 1993 (Results Act). This act
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 72 GAO/RCED-00-111 Safer Skies Initiative

requires annual performance plans to cover each program activity set out

in a federal agency’s budget. Among other requirements, performance plans

are to (1) establish performance indicators to be used in measuring or

assessing the outcomes of each program activity, (2) determine how to

compare actual results with the performance goals, and (3) describe the

means to verify and validate information used to report on performance. In

accordance with this law, DOT develops annual plans that include

performance measures for specific programs and activities. Agencies under

DOT, such as FAA, develop more detailed plans and performance measures

for each program activity.

Because of its impact on FAA’s programmatic and budgeting activities, the

Safer Skies initiative falls under the Results Act’s requirement to evaluate

program performance. Moreover, it was developed in response to the

National Civil Aviation Review Commission’s report, which specifically

directed FAA and the aviation industry to establish performance measures

and milestones to assess the initiative’s progress in meeting safety goals, to

review priorities periodically, and to monitor progress. The Safer Skies

initiative incorporates the idea of establishing performance measures to

evaluate progress toward safety goals. As a result, the Safer Skies teams

that recommend interventions are tasked with developing the performance

measures for those interventions approved by the steering committees.

For a performance measure to be useful, a baseline must be established

against which to measure the effect of the intervention. Good evaluation

criteria include (1) definitions of baseline information on the extent of the

safety problem over a particular period prior to the implementation of the

intervention and (2) timeframes for evaluating changes using the

performance measure. Goals and time frames must also be established to

determine what the program is expected to achieve and by when. For the

initiative, appropriate baseline information includes both the total number

of fatal accidents and the number of fatal accidents caused by each safety

problem within each type of aviation operation (i.e., commercial aviation

and general aviation).

Good performance measures have several key features: the event to be

measured (e.g., a runway incursion) or desired outcome (a reduction in the

number of runway incursions) is measurable; data on the event are or

could be collected; and the event occurs with sufficient frequency between

evaluations for progress to be measurable. The performance measures

under development to evaluate Safer Skies’ initiatives can be assessed

against these criteria.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 73 GAO/RCED-00-111 Safer Skies Initiative

Determining the

Effectiveness of Safer

Skies’ Initiatives Will

Require the

Development of

Additional

Performance Measures

Determining the effectiveness of Safer Skies interventions will require the

development of performance measures other than the overall goals set for

commercial and general aviation. Fatal aviation accidents occur so

infrequently that their usefulness is limited as a measure of the success of

Safer Skies’ interventions. This is especially true for commercial aviation,

which had a total of 85 fatal accidents in the United States from 1988

through 1997. The fact that a particular type of accident has not occurred

for several years does not mean that the underlying safety problem has

been successfully addressed. Furthermore, for several reasons it may be

difficult or impossible to match a specific implementation plan to a

numerical reduction in fatal accidents overall or attributable to a specific

safety problem. For example, in general aviation the lack of detail in

accident reporting makes it difficult to determine specific accident causes;

the lack of pilot profiles makes it difficult to evaluate the effectiveness of

pilot training strategies; and it is hard to predict how many aircraft owners

will install new safety equipment in the future. Thus, to determine to what

extent an intervention is reducing fatal accidents attributed to a specific

safety problem, teams will need to develop additional performance

measures. The commercial aviation steering committee recognized early

on this need to develop interim measures to evaluate the unique effect of

individual interventions.

Even if a team identifies suitable performance measures for a specific

safety problem, it may be difficult to determine whether a particular

intervention, cluster of interventions, or other outside factors influenced

changes in the performance being measured. This is especially true for

situations in which teams choose numerous interventions to address a

safety problem. While the uncontained engine failure team developed a

single primary intervention, the team working on CFIT in commercial

aviation has already initiated several interventions and is contemplating

dozens more. Similarly, the general aviation team working on weather

recommended 17 interventions. Without some way to independently

evaluate the effectiveness of individual interventions or clusters of

interventions, the initiative will have little way of knowing whether

particular interventions save lives and are thus worth the time or money

being expended on them.

In developing performance measures, one option involves using the

precursors to accidents as proxies for the likelihood of fatal accidents.

Precursors are events that, although they typically precede a particular

type of fatal accident, often occur without culminating in a crash. For
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 74 GAO/RCED-00-111 Safer Skies Initiative

example, approach and landing accidents are almost always preceded by

unstable approaches to the airport, but many unstable approaches may

culminate in a hard or late landing that does not result in injuries or a crash.

Performance measures based on precursors have been developed to

evaluate initiatives for one of the safety problems the initiative is

addressing, uncontained engine failure. The success of this approach

depends on identifying appropriate accident precursors that can serve as

proxies for the specific safety problem the team is addressing. Precursors

are most useful when they follow the criteria for good performance

measures: they are measurable, relevant data on them are available, and

they occur with sufficient frequency.

Most Safer Skies’

Interventions Are

Being Implemented

Without Determining

How to Evaluate Their

Effectiveness

Of the 16 Safer Skies teams, 8 have recommended safety interventions for

implementation; interventions from 5 of these teams have been or are being

implemented; but only one has developed a performance measure that can

show whether the intervention is effective at saving lives. Most Safer Skies

teams are still analyzing data on safety problems and selecting

interventions and have not yet determined how to evaluate the

effectiveness of interventions selected. Of the five teams whose

recommendations are being implemented, three have developed some

performance measures. Only the uncontained engine failure team has

developed two quantifiable performance measures that are based on

accident precursors. In contrast, the general aviation teams working on

CFIT and weather developed some general performance measures for

reducing accidents resulting from these safety problems but did not

quantify these measures. No performance measures were developed to

evaluate the educational interventions implemented to address the four

cabin safety problems. Finally, the team working on CFIT accidents in

commercial aviation has implemented one intervention in advance of the

team’s final report. While this team has not yet developed a performance

measure for this intervention, it is considering using an accident precursor.

Performance measures based on accident precursors have potential for use

in evaluating the effectiveness of additional interventions being considered

to address CFIT and other safety problems. FAA does not presently collect

data on some accident precursors that could be used to evaluate the

effectiveness of Safer Skies interventions and faces significant barriers to

collecting such data.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 75 GAO/RCED-00-111 Safer Skies Initiative

The Uncontained Engine

Failure Team Has Chosen

Two Accident Precursors as

Performance Measures

The Safer Skies team working on uncontained engine failure chose two

accident precursors as performance measures for evaluating the

effectiveness of the intervention it recommended: more extensive engine

inspections. Because uncontained engine failure caused just two fatal

accidents in the United States in 1988-97, fatal accidents are too infrequent

to serve as a performance measure. But well-established trend data show

that the safety problem occurs much more frequently, resulting not in fatal

accidents but in incidents with severe or serious consequences on an

average of about 1.5 times a year.1 The team chose the rate of these

incidents as the primary performance measure for its recommended

intervention. The team also chose another accident precursor as a second

performance measure: the number of cracks detected in engine disks when

engines are overhauled.2 Data analysis identified cracked disks as the

primary cause of uncontained engine failure. According to staff at FAA’s

Engine and Propeller Directorate, each crack detected during inspections

probably avoids an uncontained engine failure that could have had severe

or serious consequences.

Both accident precursors chosen—the rate of uncontained engine failure

with severe or serious consequences and the detection of cracks in engine

disks—have some of the attributes of a good performance measure. Both

can be counted, and reporting mechanisms are in place for collecting the

key data needed for both measures. Hence, it will be possible to evaluate

whether the more extensive engine inspections lead to the detection of

more cracks and fewer instances of uncontained engine failure with severe

or serious consequences. However, good performance measures track

events that occur often enough between evaluations to show whether

progress is being made. Uncontained engine failure with severe or serious

consequences occurs from one to three times a year, according to data

from 1992-98, while cracks in engine disks are likely to be discovered about

once in 25,000 inspections, according to staff at FAA’s Engine and Propeller

1

Severe consequences (level 4 events) include fatal or serious injury, loss of the aircraft hull,

and forced landing of the aircraft. Serious consequences (level 3 events) include substantial

damage to the aircraft or an unrelated system, uncontrolled fire, rapid cabin

depressurization, temporary or permanent inability to climb or fly the aircraft 1,000 feet

above terrain, and temporary or permanent impairment of the aircraft’s controllability.

2

Disks are heavy, high-speed rotating parts inside an engine with attached fan blades that

produce thrust. Undetected manufacturing flaws or contaminants can undermine a disk’s

structural integrity, allowing a crack to occur. If a crack causes a disk to fail and break apart,

fast-moving fragments of the disk can disable or damage the airplane and may have

catastrophic results.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 76 GAO/RCED-00-111 Safer Skies Initiative

Directorate. Hence, 2 to 5 years may elapse before the effectiveness of the

more extensive engine inspections can be judged. Nonetheless, tracking

both measures should provide sufficient data for reasonable interim and

final performance measures, and the enhanced inspections provide an

opportunity to avert potentially catastrophic accidents.

The uncontained engine failure team established much of the information

needed to use its performance measures to evaluate the effectiveness of

enhanced engine inspections. During our review, we worked with FAA staff

on the team to develop additional information to provide a more complete

context for how that intervention relates to the overall Safer Skies effort

and to the fatal accident rate in commercial aviation. We then developed a

template for this information that can serve as a model for other Safer

Skies implementation teams. (See table 8.) The template displays the data

critical for understanding the extent of the safety problem and the baseline

for measuring progress in addressing it, including the frequency of the

problem’s occurrence in 1988-97 and projections of its occurrence with and

without the recommended intervention by 2007, the target year for Safer

Skies to achieve an 80-percent reduction in the overall fatal accident rate.

The template reflects the team’s goal of reducing the rate and projected

number of uncontained engine failures with severe or serious

consequences by 50 percent by 2007.

Table 8: Baselines and Goals for Reducing the Occurrence of Uncontained Engine Failure in Commercial Aviation

Baseline information on uncontained engine failure

Cause of fatal accident Baseline (1988-97 Annual average

Projected for 2007

without intervention d

80-percent

reduction

All causes 85 6 9 2

Uncontained engine failure 2 cc c

Baseline information on intervention and performance measures

Performance measure Baseline Annual average Projected for 2007

without intervention e

Projected for 2007

with intervention

Rate of detection of cracks

(estimated)

a aa 1 per 25,000

inspections

Rate of uncontained engine failure

events with severe or serious

consequences

0.9 per 10 million

departuresb

0.9 per 10 million

departures

0.9 per 10 million

departures

0.45 per 10 million

departures

Number of uncontained engine

failure events with severe and

serious consequences

11b 1.5 2 1
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 77 GAO/RCED-00-111 Safer Skies Initiative

a

Past data on the number of disk inspections are not available. Thus, data on the rate of

detection of cracks under the prior requirements for engine inspections are not available.

b

Established using available baseline data from 1992-98.

c

Because uncontained engine failure occurs so infrequently, both the annual average of 0.2

and any projections are unreliable.

d

Our projection uses FAA’s projected annual growth rates of 4.0 percent for large air carriers

and 3.0 percent for commuter air carriers.

e

FAA’s projection is based on Boeing’s data showing growth in annual aircraft departures

from 16 million in 1998 to 22 million in 2007.

f

The annual average is for the baseline years chosen by FAA and the Safer Skies steering

committees in establishing their goal for commercial aviation: 1994-96.

Source: GAO’s analysis of data from Safer Skies, FAA, and NTSB.

General Aviation Teams Did

Not Develop Quantified,

Specific Performance

Measures

The general aviation implementation teams for CFIT and weather have

completed their draft reports but did not develop quantified, specific

performance measures to evaluate the effectiveness of the interventions

they recommended. The general aviation CFIT team recommended 5

interventions subdivided into 22 distinct subinterventions. None of the 22

subinterventions included specific, quantified performance measures. For

example, the CFIT team recommended developing criteria for

standardizing the marking of wires, towers, and support structures to help

decrease the number of CFIT accidents that occur when pilots of low-flying

aircraft, such as helicopters and small planes, fly into these obstacles. As

one measure of effectiveness, the team chose a decrease in the number of

CFIT accidents involving wires or towers. However, the team did not

provide any baseline information about the number of past CFIT accidents

that involved wires or towers or the types of aircraft involved. To the extent

that such baseline information is available, it provides a yardstick against

which to measure progress in reducing these accidents. Furthermore, the

team did not provide any specific interim or long-term accident reduction

goals for the number of accidents or the percentage of the fleet affected.

Without such information, it will be impossible to determine whether or by

how much CFIT accidents involving wires or towers have decreased. Other

performance measures for general aviation CFIT initiatives share this lack

of quantification and specificity. Without baseline information on the

occurrence of the problem prior to the implementation of the intervention

and specific quantified goals, it will be impossible to evaluate the

effectiveness of the interventions implemented.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 78 GAO/RCED-00-111 Safer Skies Initiative

The general aviation team working on weather experienced similar

problems in setting performance measures for its interventions. The team’s

final report recommended 17 interventions subdivided into 49 distinct

subinterventions. Of the 49 subinterventions, only 1 included a quantified,

specific performance measure. The rest had either no performance

measures or performance measures that were not quantified or specific.

Some of the interventions for which no performance measures were

established involve research that is still ongoing to develop the technology

suggested in the intervention. For example, NASA has the lead in

developing equipment to sense turbulence and warn flight crews so that

they can avoid or reduce the dangers associated with turbulence. Because

research on this technology is preliminary, the performance measures are

described broadly as reducing fatalities and injuries. It is likely too early to

establish performance measures for these interventions. The performance

measures included for many other subinterventions were too broad to

allow actual evaluation of their effectiveness. The performance measure

for most of these was a “decrease in the number of weather-related

accidents.” These performance measures are neither quantified nor linked

in any specific way to the interventions, which makes it impossible to

determine what portion of the reduction, if any, is attributable to individual

interventions or clusters of interventions. Of the performance measures

developed, several measure progress in implementing training

interventions, rather than the effectiveness of the training in reducing

safety threats. For example, one intervention involves training Flight

Service Station specialists and supervisors on in-depth weather analysis

and interpretation to improve the weather briefings given to general

aviation pilots. The associated performance measure involves training all

of these FAA staff by 2002, rather than measuring the effectiveness of that

training. In other cases, the team did not include a performance measure

when one could have been developed. For example, one intervention

involves conducting a refresher clinic for flight instructors to update them

about current weather information and provide appropriate training

materials for them to use with general aviation pilots. No performance goal

was specified for this intervention. To measure how well this intervention

has been implemented, it is possible to determine the number of flight

instructors, to establish a goal for how many attend this training each year,

and to have them provide information on how many pilots they

subsequently train using the information. To determine whether the

intervention is effective, the pilots who receive the training could later be

surveyed to determine whether they had used the weather information

provided or their safety records could be compared with the records of

pilots who did not have the training. The link between accident reduction
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 79 GAO/RCED-00-111 Safer Skies Initiative

and such training is more tenuous than the link between crack detection

and the prevention of uncontained engine failure, but it is possible to gain

at least some information about the effectiveness of the training. Without

such feedback, it is difficult to determine whether the training is effective

and should be continued.

Without more specific baseline information on these performance

measures prior to the implementation of the interventions and interim and

long-term goals for progress, the initiative will not be able to evaluate the

impact of these interventions. In responding to our draft report, FAA noted

that the implementation teams for CFIT and weather relied on the

expertise of team members, following analysis of the root causes of

accidents, to determine the probable effectiveness of the interventions.

Safer Skies analysis and implementation reports described problems with

the quantity, quality, and type of data currently available about general

aviation. These problems include shortcomings in the data for the types

and numbers of operations and in the level of detail of the actual accident

investigations. FAA concluded that the problems with general aviation data

make it difficult to measure the effectiveness of individual intervention

strategies by the traditional approach of how they affect accident rates.

While we acknowledge the need to improve general aviation data, we also

believe that such data can provide some indication of the relative frequency

and importance of the causes of fatal accidents. Such information is also

important for making decisions about which interventions to fund and

expedite, considering their potential effectiveness and the number of fatal

accidents that their use might prevent. While it may not be possible to

develop quantitative performance measures for all interventions proposed

by the implementation teams, good performance measures depend on

having measurable events, a way to collect data on those events, and an

event that occurs with sufficient frequency between evaluations for

progress to be measurable. The performance measures for both general

aviation weather and CFIT could be improved where possible by

identifying and quantifying baseline information, ensuring that a means

exists for collecting data on the performance measure, and setting interim

and long-term goals against which to measure progress in implementing the

intervention.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 80 GAO/RCED-00-111 Safer Skies Initiative

The Safer Skies Initiative

Did Not Develop a Strategy

for Evaluating Cabin Safety

Interventions

The Safer Skies cabin safety steering committee completed work on four

safety problems and implemented most interventions without developing

any strategy for evaluating the interventions. Although the steering

committee completed its work in January 1999, it did not develop

performance measures for the interventions it selected. While the

initiative’s broad goal is reducing the fatal accident rate, the broad goal for

cabin safety is educating the flying public about four areas: passenger

interference with flight crews, passenger use of seat belts, child restraint

systems, and carry-on baggage. The steering committee distributed

brochures about carry-on baggage and the importance of child restraint

systems and worked with air carriers to develop additional cabin

announcements to remind passengers to use their seat belts. The team did

not, however, set up any evaluation to show whether the public’s

knowledge about these issues improved as a result of these interventions

and whether that improved knowledge results in fewer fatalities.

While useful performance measures could be defined in each of the four

cabin safety areas, the steering committee did not develop a strategy for

evaluating the impact of its educational initiatives. For example, the

steering committee did not plan or track the distribution of the flyers it

issued about carry-on baggage or child restraint systems, and it developed

no performance measures for evaluating the effectiveness of these

initiatives to educate the public. Furthermore, FAA does not have a

mechanism for consistently collecting data about any of these areas.

Airlines are required to report information related to cabin safety only if

something happens in the cabin that results in serious injuries or death. As

a consequence, the agency does not have baseline data for measuring

improvements that may result from its initiatives. Thus, the Safer Skies

initiative has no way of measuring the effectiveness of its educational

efforts in the cabin safety area.

Precursors of Accidents

Have Potential for Use as

Performance Measures in

Other Safer Skies Areas

Precursors to accidents have the potential for use as performance

measures for evaluating interventions to address at least three other Safer

Skies safety problems: CFIT, runway incursions, and approach and landing.

Precursors are needed because fatal aviation accidents caused by all three

safety problems occur rarely. The precursors for each safety problem have

at least some of the attributes of good performance measures.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 81 GAO/RCED-00-111 Safer Skies Initiative

Navigational Alerts Could Serve

as a Performance Measure for

One CFIT Intervention

The Safer Skies team working on CFIT accidents in commercial aviation is

considering using an accident precursor to evaluate the effectiveness of

one of its interventions: the installation of enhanced navigational

equipment on aircraft that sounds alerts to warn pilots of impending

crashes. The equipment tracks data on the frequency of the alerts and the

situations in which they occur. Although these data are not currently

collected by FAA, they could be used to develop a performance measure

based on the alerts sounded as precursors to CFIT accidents. The

performance measure of alerts sounded could indicate the number of

dangerous situations avoided.

Alerts sounded by this navigational equipment have several features of a

good performance measure. First, the alerts can be measured. Second, the

equipment itself tracks such warnings. Finally, the alerts are sounded with

sufficient frequency to be useful as a performance measure. According to

the manufacturer, enhanced navigational equipment was installed in over

4,000 aircraft from March 1996 through December 1999. In 14 instances, the

alerts enabled pilots to recover from impending crashes.

Runway Incursion Incidents

Could Serve as a Performance

Measure

Runway incursion incidents that do not result in accidents provide another

useful performance measure and are being used as such by FAA. From 1988

through 1997, 2,345 runway incursions resulted in five fatal accidents and

59 fatalities in the United States.3 However, runway incursions have the

potential to cause much greater numbers of fatalities; the collision of two

large aircraft on the ground in the Canary Islands in 1977 resulted from a

runway incursion and took more than 580 lives. Because runway incursion

incidents are increasing in the United States and have the potential to lead

to fatal accidents, FAA’s Performance Plan for FY 2000 has used these

incidents to establish a performance measure for a series of safety

recommendations designed to reduce accidents caused by runway

incursions. The Safer Skies team addressing runway incursions has not yet

identified interventions, but FAA’s ongoing work offers some useful

performance measures for measuring progress in addressing this safety

problem.

3

Five runway incursions involving commercial aircraft occurred in the United States during

this period. In classifying fatal accidents, the commercial aviation steering committee

classified one of these accidents as an approach and landing accident for purposes of

analysis because one plane landed on top of another plane.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 82 GAO/RCED-00-111 Safer Skies Initiative

Runway incursion incidents have all three features of a good performance

measure. First, the incidents can be counted. Second, the data can be

collected because FAA already has a mechanism for reporting runway

incursions.4 Moreover, FAA has collected data on them for years, and

therefore has historical data that can be used to establish baselines against

which the effectiveness of interventions intended to reduce runway

incursions can be measured. For example, one intervention now in use by

FAA involves deploying action teams to airports that have experienced high

numbers of runway incursion incidents to determine the causes and

develop action plans to resolve them. Data on runway incursion incidents

can be used to determine whether the use of action teams reduces such

incidents at the airports in question. Finally, runway incursion incidents

occur with sufficient frequency to make it possible to measure progress

between evaluations. Several hundred runway incursion incidents have

been reported each year this decade.

Unstable Approaches Could

Serve as a Performance Measure

for Approach and Landing

The Safer Skies team working on approach and landing accidents in

commercial aviation is considering using an accident precursor to evaluate

the effectiveness of training and other related interventions. The team

determined that unstable aircraft approaches to airports were clearly

precursors to many approach and landing accidents.5 Several problems can

contribute to unstable approaches, including excess speed on approach,

aircraft flaps not in position, and an approach that is too steep or too

shallow. Data on each of these key aspects are recorded on an aircraft’s

flight data recorder. Thus, the team has an opportunity to develop a

performance measure based on reducing the number of unstable

approaches.

Unstable approaches have some features of good performance measures.

First, they are measurable. Second, data on them can be obtained from

flight data recorders. However, there are barriers to obtaining these data

that must be overcome before unstable approaches can be used as a

4

Nonetheless, runway incursions are now underreported because FAA does not keep

statistics on runway incursions at airports without towers because there are no air traffic

controllers present to report the incidents.

5

Unstable approaches can lead to loss of control, landing short of the runway, and

overrunning the runway, among other problems. An approach can become unstable for any

of the following reasons: late air traffic control clearance to descend, late notification of the

landing runway, late selection of the landing configuration, rapidly changing weather, poor

prior planning by the crew, or a pilot’s misjudgment of the circumstances.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 83 GAO/RCED-00-111 Safer Skies Initiative

performance measure for approach and landing interventions. Finally,

unstable approaches occur frequently enough to measure progress

resulting from interventions.

Potential Barriers Exist to

the Use of Some

Accident Precursors as

Performance Measures

Barriers exist to using some accident precursors as performance measures.

For example, the use of unstable approaches as a performance measure

depends on access to information from aircraft flight data recorders. While

some airlines use data from flight recorders to analyze the causes of safety

problems on routine flights, there are barriers to sharing this information

with FAA or with other airlines. Logistical barriers include (1) the limited

information tracked by older flight data recorders still in use and (2)

differences in the ways that air carriers have programmed flight data

recorders to track key information. Because of these differences, the kinds

of data items needed to track unstable approaches are not being captured

with enough consistency for this measure to be a good indication of

performance throughout commercial aviation.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 84 GAO/RCED-00-111 Safer Skies Initiative

Other potential barriers also prevent the use of unstable approaches as a

performance measure. Among these barriers are the ongoing debate about

how data from flight recorders are to be shared, who should have access to

these data, and whether legal enforcement cases can be initiated on the

basis of these data. Numerous major aviation safety reports in this decade

have advocated a program that would gather and analyze information from

flight data recorders about routine flights. FAA has for years promised to

establish such a program.6 However, the inability of FAA, the aviation

industry, and other federal agencies to reach consensus on key aspects of

this program has delayed its finalization. While shared data can move

safety forward, concerns about potential litigation, criminal indictments,

and the violation of an air carrier employee’s privacy have served as

barriers to the establishment of the program. Such concerns have also

delayed the finalization of other programs to enhance the sharing of

aviation safety data. For example, safety reports have for years

recommended the establishment of Aviation Safety Action Programs to

encourage voluntary self-reporting of safety violations by pilots; FAA

issued an advisory circular providing guidance for these programs on

March 17, 2000.7

Conclusions Most Safer Skies teams have not finished analyzing the causes of the safety

problems they are working on and have not yet selected interventions to

prevent the problems. Thus, these teams have not developed methods to

evaluate the effectiveness of their interventions. But when interventions

have been selected, most have been implemented without first determining

how to evaluate their effectiveness. Neither FAA nor the aviation industry

will have the information that will be critical in determining whether the

interventions have made progress in resolving the safety problems until

appropriate performance measures are developed. Evaluating the impact

of safety interventions depends on having good baseline data on the extent

of the problem prior to the implementation of the intervention, explicit

6

The program that FAA has promised to implement, the Flight Operational Quality

Assurance program, uses flight data to detect technical flaws, unsafe practices, or

conditions outside of desired operating procedures early enough to allow timely

intervention to avert accidents or incidents.

7

An Aviation Safety Action Program is a partnership program between FAA and the aviation

industry to encourage voluntary self-reporting by pilots of safety violations. The program

provides some protection from enforcement actions for inadvertent violations in return for

valuable data that can be used to analyze safety problems.
Chapter 4

The Safer Skies Initiative Has Not Yet

Developed Performance Measures to

Evaluate the Effectiveness of Most

Interventions

Page 85 GAO/RCED-00-111 Safer Skies Initiative

short- and long-term goals against which to measure progress, and

performance measures that are clearly linked to the safety problem being

addressed. In addition, as Safer Skies teams select interventions to address

the safety problems that caused fatal aviation accidents, it would be useful

to identify clearly any existing barriers to the development of performance

measures. These barriers include differences in aircraft equipment and the

absence of needed data. Once such problems are clearly identified, FAA

and the aviation industry can work jointly to resolve them.

Recommendations To improve the ability to determine the effectiveness of Safer Skies

interventions, we recommend that the Secretary of Transportation direct

the FAA Administrator to work with the Safer Skies steering committees to

direct the teams to identify the extent of fatal accidents resulting from the

safety problems they are working on. If possible, data should be developed

to establish a consistent baseline against which to measure the progress

that results from the Safer Skies initiative. If an analysis team has already

completed its report, the implementation team working on the same safety

problem should develop these baseline data. More specifically, to better

measure progress toward overall safety goals, we recommend that the FAA

Administrator work with the Safer Skies steering committees to revise the

implementation guidance to (1) develop an overall performance measure

or measures to determine progress toward eliminating the safety problem

the team is addressing; (2) consider using accident precursors as

performance measures for the safety problem in question; and (3) identify

any barriers that may impede the implementation of performance

measures.

Agency Comments DOT and FAA officials agree in principle with the need for baseline data on

the extent of each safety problem and performance measures to determine

progress toward overall safety goals. They concur with the potential of

accident precursors as possible performance measures and with the

importance of identifying any barriers that may impede the implementation

of performance measures.
Page 86 GAO/RCED-00-111 Safer Skies Initiative

Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed Chapte5r

FAA coordinated extensively with numerous representatives from the

aviation industry, other federal agencies involved in aviation safety, and its

own staff on the identification of safety problems and the selection of

interventions. However, efforts to prioritize, fund, and evaluate Safer Skies

initiatives could be better coordinated with industry and within FAA and

the Department of Transportation (DOT). Joint government-industry

efforts to improve safety are not new, but participants noted that the

initiative was more inclusive than prior joint efforts. This inclusive

approach should help FAA gain consensus on which interventions will best

address aviation safety problems. However, our review identified three

coordination problems that could undermine the implementation and

evaluation of Safer Skies interventions. First, although FAA officials have

repeatedly committed to funding interventions agreed upon by all parties

working on the initiative, skepticism still exists among some participants

as to whether this commitment can or will be honored. This is particularly

true in general aviation. It also remains unclear what process will be used,

if funding is limited, to reprioritize available resources to ensure funding

for interventions that emerge later but have greater potential for reducing

the fatal accident rate. Finally, Safer Skies steering committees, FAA, and

DOT have not coordinated how they will measure progress in achieving the

accident reduction goal for commercial aviation.

The Safer Skies

Initiative Involves an

Unprecedented Level

of Coordination

Between Industry and

Government

FAA included aviation experts from a wide range of government and

industry organizations on the Safer Skies steering committees and the

teams working on the 16 safety problems. Many participants represent

groups that are directly responsible for the nation’s aviation safety, such as

the air carriers and the manufacturers of aircraft and engines. Other

participants come from trade associations that represent various aviation

groups or from federal agencies that share responsibility for aviation safety,

including the Department of Defense and the National Aeronautics and

Space Administration. In addition, while giving priority initially to reducing

the U.S. accident rate, the initiative recognized the increasingly global

nature of aviation. In an effort to address both domestic and worldwide

aviation safety problems, the commercial aviation steering committee

included representatives from two international aviation authorities, the

Joint Aviation Authorities and the International Civil Aviation Organization.

Joint efforts between industry and government officials to study aviation

safety problems are not new. In prior years, government and industry

convened various joint teams to review aviation safety issues and make

recommendations; however, according to Safer Skies participants, those
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 87 GAO/RCED-00-111 Safer Skies Initiative

earlier teams did not always include representatives from major

organizations who were responsible for aviation safety. As a result, FAA

was not always successful in obtaining consensus on the safety

interventions that those teams recommended. Safer Skies participants

noted that the level of participation and cooperation for this initiative is

unprecedented among the major groups responsible for aviation safety and

should enhance FAA’s chances of implementing the safety interventions

made by the various teams.

Moreover, the initiative coordinated ongoing aviation safety work that was

being conducted independently by FAA, industry, and other federal

agencies. For example, aircraft manufacturers had initiated an exhaustive

study on ways to prevent uncontained engine failure. FAA eventually joined

the aircraft manufacturers in this study, and it subsequently became part of

the Safer Skies agenda. In addition, the industry and FAA had been

conducting independent studies on runway incursions and CFIT. Under the

initiative, representatives from the aircraft manufacturers, airline industry,

and government are members of the teams studying 16 safety problems,

and together they will decide on the strategies to address them.

The Funding,

Prioritization, and

Evaluation of Safer

Skies Interventions

Could Be Better

Coordinated

While coordination between government and industry organizations

participating in the initiative has been extensive, we identified three areas

in which coordination could be improved. First, although FAA has

committed to funding interventions approved by the Safer Skies steering

committees, uncertainty remains about the agency’s ability to fund these

safety interventions. The steering committees for commercial aviation and

general aviation have both sought commitment to the implementation and

funding of interventions before giving final approval to move forward.

However, FAA’s commitment has come at different points in the approval

process for interventions recommended by these steering committees, and

FAA’s commitment to the general aviation interventions was still uncertain

even after some industry and FAA officials believed the steering committee

had given its final approval. As a consequence, general aviation participants

were more skeptical about whether FAA would implement or fund their

safety interventions. Second, it remains unclear what process will be used

to reprioritize available resources if funding is limited. Finally, Safer Skies

steering committees, FAA, and DOT have not coordinated how they will

measure Safer Skies’ progress in achieving the goal of reducing the fatal

accident rate in commercial aviation by 80 percent by 2007.
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 88 GAO/RCED-00-111 Safer Skies Initiative

Skepticism Persists About

FAA’s Ability to Fund Safety

Interventions

Skepticism persists about whether FAA can or will be able to honor its

commitments to fund the interventions approved by the Safer Skies

steering committees to reduce the fatal accident rate. This is especially true

in the general aviation community. This skepticism results partly because

the process for approving and funding Safer Skies interventions has

worked differently for general aviation than it has for commercial aviation

thus far. This has contributed to differing perceptions about the likelihood

of the funding and implementation of interventions. These perceptions

have resulted in part from the different processes used by the two steering

committees to seek approval and funding from participating organizations,

from the way interventions have moved forward within these two Safer

Skies committees, and from FAA’s handling of the interventions

recommended by them.

The Process for Final Approval

of Interventions Has Worked

Differently in the Two Steering

Committees

The final approval of recommended safety interventions has worked

differently in the commercial aviation and general aviation steering

committees. The commercial aviation steering committee has documented

its process for approving interventions, which involves members’ gaining

the approval of their respective organizations for both implementation and

funding. This approval comes in two stages. First, steering committee

members brief their respective organizations on the general concept of

each intervention under consideration and seek preliminary approval of

each intervention. Changes and modifications may be suggested by the

organizations. For organizations that will be involved in the

implementation of an intervention, the preliminary approval also involves a

tentative commitment to fund the cost of implementing any interventions

for which they are responsible. Once members grant preliminary approval,

the steering committee asks the team to draw up detailed implementation

plans for each intervention. These implementation plans are then

submitted to the steering committee for the next level of approval.

Members subsequently seek final approval of these plans from the

organizations they represent, including firm resource and funding

commitments if appropriate. When participating organizations concur with

the detailed implementation plans, the steering committee grants final

approval. To date, most of the commercial aviation teams have forwarded a

few interventions at a time for final approval by the steering committee,

rather than complete lists of interventions to address multiple aspects of

complex safety problems, such as CFIT. Thus, when the commercial

aviation steering committee has given its final approval for an intervention,

members interviewed told us they assumed that the intervention had a high

priority and that implementation would take place because the
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 89 GAO/RCED-00-111 Safer Skies Initiative

organizations responsible for implementation had already committed both

the staff and funding needed.

In contrast, the general aviation steering committee had not documented

its process for approving interventions at the time of our review, although it

recently developed draft procedures, according to FAA’s response to our

draft report.1 Furthermore, commitment to provide resources for them is

still pending, although some members of both FAA and industry who serve

on the steering committee understood that final approval had been given to

the interventions chosen to address CFIT and weather. Once these two

implementation teams submitted their draft reports to the general aviation

steering committee, the steering committee asked members to have their

organizations review and comment on each intervention. This process

resulted in preliminary approval or disapproval of the concept of each

intervention, in some cases after the intervention was modified.

Organizations responsible for the implementation of interventions also

were expected to give a tentative commitment to fund the cost of their

implementation. The steering committee then asked the teams to develop

detailed implementation plans for each intervention and to submit those

for its final approval. These two teams recommended and developed plans

for a total of 17 interventions, many of which involve subinterventions and

will require substantial resources either in the form of staff or funding from

FAA. Because of the number and potential cost of interventions contained

in the two general aviation reports, FAA requested that the general aviation

steering committee prioritize the interventions. The general aviation

steering committee prioritized the interventions in the letter that

transmitted the final CFIT and weather reports to the FAA Administrator in

March 2000. Unlike the commercial aviation teams, which have presented

one intervention at a time to the steering committee, the general aviation

teams have presented their complete series of interventions for each safety

problem.

As the general aviation CFIT and weather reports moved toward final

approval, however, confusion arose. Some industry and FAA participants

believed that these reports had received final approval. This perception is

supported by a March 22, 2000, letter from the industry and FAA co-chairs

of the general aviation steering committee transmitting to the FAA

Administrator the final CFIT and weather implementation reports with

1

We did not have an opportunity to review the general aviation steering committee’s draft

procedures.
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 90 GAO/RCED-00-111 Safer Skies Initiative

their detailed implementation plans. The letter and accompanying reports

identified high-priority interventions for immediate implementation. These

participants were concerned because FAA was still undecided which

interventions would actually be implemented and funded. In contrast,

FAA’s informal written comments in response to our draft report state that

final approval has not been given to either implementation report and

depends on the completion of detailed implementation plans by the FAA

offices responsible for carrying out the implementation. According to the

Director of Aircraft Certification, confusion arose because some members

of the steering committee had “misperceptions” about what levels of

approval had been agreed to.

FAA’s Internal Review and

Funding Process for Safer Skies

Interventions Has Led to Some

Uncertainty About Whether

Some Interventions Will Be

Funded

FAA’s internal review and funding process for Safer Skies interventions has

led to uncertainty about whether some interventions will be funded, in part

because interventions forwarded by the commercial aviation and general

aviation steering committees have been handled somewhat differently thus

far. Like the other organizations participating in the initiative, FAA must

commit its own resources to the interventions that it is responsible for

implementing. In October 1999, FAA formed an executive council to help

coordinate the implementation of the agency’s safety agenda, including

how to provide funding and staff resources for Safer Skies interventions.

The executive council includes the heads of each of FAA’s major program

offices, its general counsel, and a regional administrator. The executive

council has not yet documented its process for approving and funding

interventions, however, and it remains unclear at what point FAA is

committing resources to implement Safer Skies interventions. This

uncertainty has led to different perceptions on the part of some FAA and

industry participants about the likelihood that interventions will be

implemented and funded.

FAA staff working on the initiative described differences in the way the

executive council has handled interventions proposed by the two steering

committees. These differences have resulted in a clear indication of

funding for commercial aviation interventions before that steering

committee’s final approval is given, while the general aviation steering

committee’s final approval was given on a series of weather and CFIT

interventions that have yet to be approved and funded by FAA. When

proposed Safer Skies interventions are under serious consideration by the

steering committees, they are also presented to the executive council for

discussion of their possible impact on workload and budget, according to

FAA staff who serve as co-chairs of the two steering committees. The

executive council provides feedback to the steering committees before
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 91 GAO/RCED-00-111 Safer Skies Initiative

interventions are approved. FAA staff serving on Safer Skies committees

presented conflicting views, however, of when FAA commits to funding

interventions. Several of the FAA staff interviewed said that FAA’s

commitment of staffing and funding to commercial aviation interventions

occurs before that steering committee gives its final approval to

interventions.

However, the Director of FAA’s Aircraft Certification Service, who serves as

co-chair of the commercial aviation steering committee, described the

executive council’s role as having more room for interpretation of the

intervention and a subsequent determination of whether funding is

available. She said that, once the intervention is approved, the executive

council again discusses it, determines whether to accept it as stated or to

modify it, assigns it to an FAA office for implementation, and determines

how it fits in with the office’s existing priorities. The program office then

reviews the intervention, can suggest modifications that will achieve the

same goal, and determines whether the intervention can be accomplished

with existing resources or requires a request for additional funding. She

said that the executive council could also request that the steering

committee modify or prioritize interventions. For example, she said that

FAA agreed to implement the commercial aviation CFIT team’s

recommendation to develop precisionlike airport approaches,2 concluded

that the agency’s resources would not permit the completion of approaches

for all airports in the time frame envisioned by the intervention, and is now

working with the steering committee to identify which airports present the

greatest risks and should be completed first. Similarly, she said that the

council asked that the general aviation steering committee approve a

different way to accomplish one intervention without hiring additional staff

and prioritize its list of CFIT and weather interventions according to which

ones will have the most impact on improving safety and reducing fatalities.

Because the executive council’s role is new and its procedures remain

undocumented, confusion persists about when FAA commits its resources

to implementing the safety interventions approved by the steering

committees. For example, although FAA’s executive safety council had

agreed in principle to the highest priority interventions to address general

aviation safety problems caused by weather and CFIT, FAA’s response to

our draft report indicated that final approval and funding depend on the

completion of detailed implementation plans. As a consequence, several

2

Precisionlike approaches improve aviation safety by enabling all flight crews and aircraft to

fly a stabilized vertical path to the runway end for all instrument approach procedures.
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 92 GAO/RCED-00-111 Safer Skies Initiative

Safer Skies participants from FAA and industry, especially those working

on general aviation issues, expressed some concern about whether the

recommended interventions would be funded or implemented.

These concerns stem partly from FAA’s past record for implementing safety

recommendations. FAA’s budget does not specifically identify and commit

resources to implementing Safer Skies interventions. For example, FAA has

no funds set aside in its budgets for fiscal years 2000 or 2001 for general

aviation interventions. However, FAA’s Deputy Associate Administrator for

Regulation and Certification said that the agency’s approach to budgeting is

to retain flexibility by not identifying specific budget amounts for such

efforts as the Safer Skies initiative. While we do not advocate including

specific Safer Skies line items in FAA’s budget, the uncertainty about

funding and implementation also exists because FAA has either not fully

funded or not implemented some safety recommendations in the past.

Several industry participants in the initiative specifically mentioned

concerns about FAA’s lack of follow through on safety recommendations to

decrease the number of runway incursions. Although FAA has received

many recommendations for reducing runway incursions, continuing

problems in this area have been partially attributable to insufficient funding

of the safety plans FAA developed, according to DOT’s Inspector General.3

Additionally, after initially planning to fund the agency’s new inspection

system,4 FAA has still not provided funding to hire analysts to review

inspection data on the nation’s 10 major airlines for possible safety

concerns. While FAA has implemented many safety recommendations over

the years, concerns still persist about the agency’s ability to fund new

safety initiatives. Greater assurance about the implementation of Safer

Skies interventions could be provided in two ways. First, as mentioned in

chapter 3, stronger mechanisms for tracking the implementation of

interventions from all three steering committees need to be established.

Second, clarifying FAA’s process for committing resources for

implementing interventions would provide greater assurance of their

3

Federal Aviation Administration: Aviation Safety, DOT Inspector General Report No. AV1999-069 (Statement of Alexis M. Stefani, Deputy Assistant Inspector General for Aviation,

U.S. Department of Transportation before the Subcommittee on Transportation and Related

Agencies, Committee on Appropriations, U.S. House of Representatives, Mar. 10, 1999).

4

We presented our findings and recommendations about the new inspection system, the Air

Transportation Oversight System, in Aviation Safety: FAA’s New Inspection System Offers

Promise, but Problems Need to Be Addressed (GAO/RCED-99-183, June 28, 1999).
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 93 GAO/RCED-00-111 Safer Skies Initiative

implementation. Both of these steps would improve coordination between

FAA and other Safer Skies participants.

Thus far, the interventions approved by steering committees have not

required a major commitment of time and resources by either FAA or

industry groups. But future interventions may require substantial resources

not included in FAA’s current budget, and choices may have to be made

about which interventions to fund. Furthermore, FAA addresses and funds

many issues beyond those on the Safer Skies agenda, including security

issues and improvements to the air traffic control and airport

infrastructure. FAA’s executive council provides a forum for agency

managers to discuss and prioritize program and resource needs. However,

without clear priorities and a unified aviation safety agenda that also takes

such issues into account, FAA will continue to address aviation piecemeal,

rather than as an integrated system. While the Safer Skies initiative

represents a major step in the direction of coordinating the nation’s

aviation safety agenda, a more far-reaching effort has not yet been

undertaken to coordinate the nation’s complete aviation agenda.

The Initiative Does Not

Have A Process for

Prioritizing Interventions to

Ensure the Implementation

of Those With the Greatest

Potential to Reduce the

Fatal Accident Rate

The initiative has not developed a process for prioritizing interventions to

ensure the implementation of those with the greatest potential to reduce

the fatal accident rate if funding is limited. The initiative has involved

prioritization at several points thus far. First, the teams addressing safety

problems in commercial aviation and general aviation have prioritized the

interventions they considered. For example, the general aviation weather

team considered numerous possible safety interventions and eventually

developed a list of 17 that it presented in order of priority. The steering

committees have also prioritized interventions. For example, the

commercial aviation steering committee has moved quickly on several

interventions that the CFIT implementation team considered as having a

high priority and potential for effectiveness. At the request of the executive

council, the general aviation team created a unified list to prioritize its

CFIT and weather interventions. Given the constraints of FAA’s budget,

such prioritization is critical to ensuring that funds are expended on the

interventions that will be most effective in reducing the fatal accident rate.

The ability to reprioritize resources for Safer Skies interventions and other

aviation work may also become critical. The Safer Skies team has just

begun work on loss of control—the safety problem that caused the greatest

number of fatal accidents in commercial aviation in 1988-97. Interventions

to address loss of control are thus likely to be critical for reducing the fatal
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 94 GAO/RCED-00-111 Safer Skies Initiative

accident rate. If funding is limited, this may mean reprioritizing funding

from existing programs and Safer Skies interventions that have already

been approved to those with more potential to reduce the fatal accident

rate and save lives. The initiative’s success will depend in part on its ability

to identify those interventions with the most potential impact and to

prioritize their implementation and funding. Safer Skies steering

committees and FAA’s executive council have not yet established any

process for reprioritizing interventions if funding is limited.

Safer Skies Steering

Committees, FAA, and DOT

Have Different Ways of

Measuring Progress in

Reducing Commercial

Aviation’s Fatal Accident

Rate

A lack of coordination among Safer Skies steering committees, FAA, and

DOT has resulted in their having different ways of measuring whether the

goal of reducing the fatal accident rate for commercial aviation by 80

percent is achievable by 2007. DOT is responsible for setting safety goals

for all modes of transportation under its authority, including aviation.

Generally, FAA and other agencies under DOT have established specific

goals and use measurements that evaluate their progress in meeting those

goals that are in line with those set by DOT. But currently, DOT and FAA

measure progress toward the goal of an 80-percent reduction in the fatal

accident rate for commercial aviation in different ways. DOT’s

Performance Plan for fiscal year 2001 establishes goals for reducing the

fatal accident rate in commercial aviation that rely on the Safer Skies

initiatives. To determine the progress made in reducing the rate, DOT’s plan

uses aircraft flight hours as the activity measure. In contrast, the

commercial aviation steering committee and FAA use aircraft departures as

the measure of aviation activity. Because DOT, FAA, and Safer Skies all

share a common goal of reducing the fatal accident rate, consistency would

be desirable in the aviation activity measure they use to calculate the

progress being made toward that goal. Since most commercial aviation

accidents occur during takeoff and landing, we believe that using

departures would better measure the effectiveness of the Safer Skies

interventions for commercial aviation.

Conclusions Additional steps need to be taken to ensure that those safety interventions

most critical to reducing the nation’s fatal accident rate are given top

priority and funding. If FAA’s process for prioritizing and funding Safer

Skies interventions is not clarified, there is no assurance that the agency

will be able to implement these interventions. If funding is limited, a

process may well be needed for reprioritizing available staffing and funding

to ensure that the interventions with the greatest potential for reducing the

nation’s fatal accident rate and saving lives are implemented first. Even if
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 95 GAO/RCED-00-111 Safer Skies Initiative

Safer Skies steering committees and FAA agree on the priorities for the

nation’s safety agenda, these priorities will continue to compete for

resources with other aviation needs until FAA develops a unified aviation

agenda. Finally, FAA, the Safer Skies commercial aviation steering

committee, and DOT are not using the same aviation activity measure to

calculate the progress of Safer Skies interventions in reducing the fatal

accident rate for commercial aviation. Consequently, they may reach

different conclusions about the effectiveness of the Safer Skies

interventions in achieving the goal of reducing the fatal commercial

aviation accident rate by 80 percent by 2007.

Recommendations To ensure the implementation of the Safer Skies safety interventions, we

recommend that the Secretary of Transportation direct the FAA

Administrator to clarify the executive council’s process for committing to

the funding and implementation of interventions and coordinate with the

Safer Skies steering committees about the meaning and timing of this

commitment. To ensure that the interventions with the greatest potential

for reducing the fatal accident rate and improving aviation safety receive

needed resources, we recommend that the Secretary of Transportation

direct the FAA Administrator to ensure that the executive council has a

process in place for reprioritizing interventions if funding is limited.

To ensure that the extent of progress toward reducing the fatal accident

rate for commercial aviation is measured consistently, we recommend that

the Secretary of Transportation ensure that DOT, FAA, and the Safer Skies

commercial aviation steering committee all use departures as the activity

measure for calculating the rate.

Agency Comments DOT and FAA officials concurred with our recommendations to clarify the

executive council’s process for committing to the funding and

implementation of interventions and to use departures as the activity

measure for calculating the fatal accident rate in commercial aviation. They

disagreed with our recommendation that FAA’s executive council should

develop a process for reprioritizing interventions if funding is limited. The

officials said that such reprioritization falls under the agency’s normal

processes for reprogramming funding. However, the role of the executive

council is to help coordinate the implementation of the agency’s safety

agenda—including how to provide funding and staff resources for Safer

Skies interventions. We believe that it would be useful for the executive
Chapter 5

Coordination Has Been Extensive but Needs

Improvement for the Safer Skies Initiative to

Succeed

Page 96 GAO/RCED-00-111 Safer Skies Initiative

council to establish some basic criteria and processes for evaluating and

comparing the potential impact of existing and emerging safety

interventions. For this reason, we did not modify or withdraw our

recommendation.
Page 97 GAO/RCED-00-111 Safer Skies Initiative
Page 98 GAO/RCED-00-111 Safer Skies Initiative

Appendix I

GAO Contacts and Staff Acknowledgments ApApenednxidexiIs

GAO Contacts Gerald L. Dillingham, (202) 512-2834

Robert White, (202) 512-2834

Acknowledgments In addition to those named above, Leslie Albin, Bonnie A. Beckett, Fran

Featherston, David K. Hooper, Christopher M. Jones, and Phillis L. Riley

made key contributions to this report.
Page 99 GAO/RCED-00-111 Safer Skies Initiative

Key Aviation Studies

AOPA Air Safety Foundation. 1998 Nall Report: Accident Trends and

Factors for 1997.

Frederick, MD.: 1998.

Boeing Commercial Airplane Group. Statistical Summary of Commercial

Jet Airplane Accidents: Worldwide Operations, 1959-1991. Seattle: June,

1992.

Federal Aviation Administration. Zero Accidents...A Shared Responsibility.

Feb. 9, 1995.

Federal Aviation Administration and The Aerospace Industries Association.

Technical Report on Propulsion System and Auxiliary Power Unit (APU)

Related Safety Hazards.

June 25, 1999.

Federal Aviation Administration. FAA 90 Day Safety Review. Sept. 16, 1996.

Federal Aviation Administration, Office of Aviation Research. General

Aviation Accidents, 1983-1994: Identification of Factors Related to

Controlled-Flight-Into-Terrain (CFIT) Accidents. July 1997.

Flight Safety Foundation. Controlled Flight Into Terrain Education and

Training Aid, Vols. 1 and 2.

Flight Safety Foundation. Killers in Aviation: FSF Task Force Presents

Facts About Approach-and-Landing and Controlled-Flight-Into-Terrain

Accidents. Nov.-Dec. 1998, Jan.-Feb. 1999.

The MITRE Corporation. Reports by Airline Pilots on Airport Surface

Operations: Part 1. Identified Problems and Proposed Solutions for Surface

Navigation and Communications. McLean, VA.: May 1994.

The MITRE Corporation. Reports by Airline Pilots on Airport Surface

Operations: Part 2: Identified Problems and Proposed Solutions for Surface

Operational Procedures and Factors Affecting Pilot Performance. McLean,

VA.: Mar. 1996.

The MITRE Corporation. Reports by Airport Traffic Control Tower

Controllers on Airport Surface Operations: The Causes and Prevention of

Runway Incursions. McLean, VA.: Sept. 1998.
Key Aviation Studies

Page 100 GAO/RCED-00-111 Safer Skies Initiative

National Civil Aviation Review Commission. Avoiding Aviation Gridlock

and Reducing the Accident Rate: A Consensus for Change. Dec. 11, 1997.

U.S. Department of Transportation, Office of Inspector General. Federal

Aviation Administration’s Runway Incursion Program (AV-1998-015).

Washington, D.C.: Dec. 8, 1997.

U.S. Department of Transportation, Office of Inspector General. Runway

Incursion Program, Federal Aviation Administration (AV-1998-075).

Washington, D.C.: Feb. 9, 1998.

U.S. Department of Transportation, Office of Inspector General. Follow-Up

Review of FAA’s Runway Safety Program (AV-1999-114). Washington, D.C.:

July 21, 1999.

U.S. Department of Transportation, Research and Special Programs

Administration. Investigation of Controlled Flight Into Terrain, Vols. 1 and

2. Mar. 1996.

U.S. Department of Transportation, Research and Special Programs

Administration. Investigation of Controlled Flight Into Terrain,

Descriptions of Flight Paths for Selected Controlled Flight into Terrain

(CFIT) Aircraft Accidents, 1985-1997. Mar. 1999.

White House Commission on Aviation Safety and Security. Final Report of

the White House Commission on Aviation Safety and Security. Feb. 12,

1997.

Related GAO Products Aviation Safety: FAA’s New Inspection System Offers Promise, but

Problems Need to Be Addressed (GAO/RCED-99-183, June 28, 1999).

Aviation Safety: FAA Has Not Fully Implemented Weather-Related

Recommendations (GAO/RCED-98-130, June 2, 1998).

Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA in

Identifying and Responding to Risks (GAO/RCED-98-6, Feb. 27, 1998).

Aviation Safety: FAA Oversight of Repair Stations Needs Improvements

(GAO/RCED-98-21, Oct. 24, 1997).
Key Aviation Studies

Page 101 GAO/RCED-00-111 Safer Skies Initiative

Aviation Safety: Efforts to Implement Flight Operational Quality Assurance

Programs (GAO/RCED-98-10, Dec. 2, 1997).

Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing

Safety Recommendations (GAO/RCED-96-193, Sept. 23, 1996).

(348180) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00