General Aviation Safety: Additional FAA Efforts Could Help Identify and Mitigate Safety Risks
Released: 2012-10-04T00:00:00

Letter

Background

General aviation is characterized by a diverse fleet of aircraft flown for a variety of purposes. In 2010, FAA estimated that there were more than 220,000 aircraft in the active general aviation fleet, comprising more than 90 percent of the U.S. civil aircraft fleet. Included among these aircraft are airplanes, balloons, unmanned aircraft systems, gliders, and helicopters. (See fig. 1.) Airplanes comprise the vast majority—almost 80 percent—of the general aviation fleet. According to a 2009 FAA study, general aviation airplanes have an average age of 40 years. In addition, most are single-engine piston, such as the Beechcraft Bonanza, Cessna 172, and Piper Archer.

FAA designates a small, but growing, portion of the general aviation fleet as “experimental.” These include aircraft used for racing and research as well as exhibition aircraft, such as former military aircraft known as warbirds. The largest group of experimental aircraft—and the fastest growing segment of the general aviation fleet, according to FAA—is defined by FAA as “experimental-amateur built” (E-AB). Individuals build E-AB aircraft either from kits sold by manufacturers or from their own designs. E-AB aircraft can contain previously untested systems, including engines not designed for aircraft use, and modifications of airframes, controls, and instrumentation. The E-AB fleet is diverse, ranging from open-framework designs with no cabin structure to small, pressurized airplanes able to fly long distances. The majority are simple craft used primarily for short personal flights. The expertise of the builders varies, as does the experience of the pilots and the availability of training for transitioning to the aircraft. Following a successful inspection of the aircraft and documentation review, FAA issues a special airworthiness certificate in the experimental category to the aircraft’s builder and assigns operating limitations in two phases specifying how and where the aircraft can be flown.the builder determines the aircraft’s airspeed and altitude capabilities and develops a flight manual. Phase II refers to normal operations after the flight testing is completed.

Phase I is the required flight test period, in which General aviation aircraft can be used for a wide variety of operations, although about 78 percent of general aviation operations fall under one of four types: personal (e.g., a pilot taking his family on a sightseeing trip); business (e.g., a pilot flying herself to a meeting); corporate (e.g., a professionally-piloted aircraft transporting corporate employees around the globe); and instructional (e.g., a student flying with a certified flight instructor).

These operations are conducted from the more than 2,950 public use general aviation airports (which primarily serve general aviation aircraft) as well as from thousands of other airports (including those that support commercial air service) and landing facilities (e.g., heliports). General aviation flights operate under various federal aviation regulations. For purposes of this report, our definition of general aviation includes flights operated under part 91 general operation and flight rules.

GAO-12-117. Our definition of flight-instructor based schools includes individual flight instructors. more information about the estimated number of active airplane pilots and selected pilot certificate requirements and limitations.

Various offices within FAA are responsible for ensuring general aviation safety, most notably the Flight Standards Service, Aircraft Certification Service, Office of Accident Investigation and Prevention, and Office of Runway Safety. According to FAA, the agency’s fiscal year 2011 budget submission included nearly $203 million for activities within the Aviation Safety organization related to the top priority of reducing the general aviation fatal accident rate. FAA’s responsibilities include administering aircraft and pilot certification, conducting safety oversight of pilot training and general aviation operations, and taking enforcement actions against pilots and others who violate federal aviation regulations and safety standards. FAA also collects general aviation fleet and flight activity data through an annual survey and supports the NTSB by gathering information about general aviation accidents. According to NTSB officials, FAA collects information on the vast majority of general aviation accidents.

NTSB is responsible for all aviation accident investigations—using the information gathered by FAA and its own investigators—and for determining the probable cause of accidents. NTSB uses a coding system of aircraft accident categories and associated phases of flight that are useful in describing the characteristics and circumstances of aviation accidents. For ease of interpretation and to categorize similar events, NTSB identifies one event as the “defining event” of the accident, which generally describes the type of accident that occurred—hard landing, midair collision, or fuel exhaustion, for example. In addition, NTSB identifies the causes of an accident and the contributing factors, which describe situations or circumstances central to the accident cause. Just as accidents often include a series of events, the reason those events led to an accident may reflect a combination of multiple causes and contributing factors. For this reason, a single accident report can include multiple cause and contributing factor codes. NTSB also collects descriptive information about the environmental conditions, aircraft, and people involved in aviation accidents. It captures its findings and descriptive information in its Aviation Accident Database. NTSB calculates general aviation accident and fatality rates, which it does using its own accident data and FAA’s annual estimates of general aviation flight activity. NTSB may also recommend regulatory and other changes to FAA and the aviation industry based on the results of its investigations and any studies it conducts.

The U.S. general aviation industry includes a number of trade groups, “type clubs,” and other organizations that actively promote the importance of safety and, in many cases, offer educational opportunities to pilots. Many of the groups also work with FAA on advisory and rulemaking committees. Prominent trade organizations include the Aircraft Owners and Pilots Association (AOPA), the Experimental Aircraft Association (EAA), the General Aviation Manufacturers Association (GAMA), and the National Business Aviation Association (NBAA). The Society of Aviation and Flight Educators (SAFE) and the National Association of Flight Instructors represent certified flight instructors and other aviation educators. The American Bonanza Society (ABS), the Cirrus Owners and Pilots Association, and the Lancair Owners and Builders Organization are examples of the several general aviation type clubs.

General Aviation Accidents Decreased, but Some Segments Had Disproportionate Shares of Accidents

Our analysis of NTSB accident data showed that the annual number of general aviation accidents generally decreased for 1999 through 2011. We also identified several characteristics of accidents with respect to the types of operations and the causes of the accidents. These characteristics were largely consistent with observations made during our last review of general aviation safety in 2001. To better understand these characteristics, where possible, we sought to measure their occurrence in numbers of accidents in relation to their overall occurrence in, for instance, total flight hours or pilot certifications as estimated by FAA. In doing so, we identified some accident characteristics that, based on our analysis, appear to occur disproportionately. However, we also identified methodological and conceptual limitations with the activity data—particularly the General Aviation and Part 135 Activity Survey that FAA uses to estimate annual flight hours and the number of active aircraft—that we discuss later in this section. See table 2 for a summary of the characteristics of general aviation accidents according to our analysis of the NTSB accident data.

From 1999 through 2011, nonfatal accidents involving general aviation airplanes generally decreased, falling 29 percent, from 1,265 in 1999 to 902 in 2011. Fatal accidents generally decreased as well, falling 24 percent. Figure 2 indicates the number of fatal and nonfatal accidents for each year we reviewed. During this period of time, though the majority (approximately 56 percent) of all accidents resulted in no injuries, there were more than 200 fatal accidents each year.

From 1999 through 2011, personal operations accounted for 73 percent of airplanes in nonfatal general aviation accidents and 77 percent of airplanes in fatal general aviation accidents. (See fig. 3.) This is not a new phenomenon. As we reported about accidents occurring in 1998, personal operations accounted for more than 75 percent of fatal general aviation accidents.

From 1999 through 2011, airplanes flying instructional operations were the second most often involved in accidents. However, instructional operations were also the operation with the smallest proportion of fatal accidents. According to our analysis, almost 38 percent of accidents that occurred during instructional flying involved hard landings or loss of control while the aircraft was on the ground. These types of events are less likely to cause fatalities than other types of events. It is also possible that the presence of a certified flight instructor onboard to share the management of the cockpit and other tasks may have contributed to the lower fatality rate for instructional operations.

Corporate operations, in which a professional pilot flies an aircraft owned by a business or corporation, was the least common type of operation to be involved in general aviation accidents. Corporate operations accounted for less than 1 percent of fatal general aviation accidents and less than 0.5 percent of nonfatal accidents. From 2008 through 2011, there were no fatal accidents involving corporate airplanes, giving corporate operations an accident record similar to that of commercial air carriers. Again, this is not a new phenomenon. As we reported in 2001, the low number of accidents involving corporate operations is attributable to a number of factors, including the pilot’s training, experience, and participation in ongoing training to maintain and improve their skills, as well as the safety equipment that is typically installed on corporate aircraft.

According to a representative of the NBAA, an organization representing companies that rely on general aviation aircraft to conduct business, most corporate operations also benefit from advanced technologies, including avionics that provide synthetic vision and terrain displays; auto-throttle, which helps maintain airspeed; and fuel gauges that are built to the standards required for commercial airliners. Further, airplanes used for corporate purposes are often powered by turbine engines and may be subject to additional safety requirements. Flying for corporate purposes can also differ from other types of flying. Whereas a pilot flying for fun may perform several take-offs and landings and practice maneuvers, a corporate flight likely includes a single take off and landing, with the majority of time spent en route—one of the phases of flight when the fewest fatal accidents occur.

Regarding the type of aircraft involved in general aviation accidents, single-engine piston airplanes accounted for almost 76 percent of airplanes in nonfatal general aviation accidents and 60 percent of airplanes in fatal accidents. Single-engine piston airplanes are the most common type of aircraft in the general aviation fleet and, according to stakeholders, the type of aircraft most commonly flown by pilots holding private pilot certifications and flying for personal reasons. According to AOPA, mechanical failures cause relatively few accidents, indicating that the frequency with which single-engine piston airplanes are in accidents is not necessarily a reflection of the safety of the aircraft.

E-ABs were the second most common airplane involved in general aviation accidents. From 1999 through 2011, E-AB aircraft accounted for 14 percent of airplanes in nonfatal general aviation accidents and approximately 21 percent in fatal accidents. According to EAA, the organization that represents experimental and amateur-built aircraft owners, E-AB airplanes were also the fastest growing type of aircraft in the general aviation fleet in recent years. In 2011, there were approximately 33,000 registered E-AB aircraft, a 10 percent increase from 3 years earlier. AOPA’s 2010 Nall Report—an annual safety report that provides perspectives on the previous year’s general aviation accidents— indicated that the physical characteristics and the manner in which these aircraft are used expose E-AB aircraft pilots to greater risk and make accidents less survivable.

In 2012, NTSB completed a safety study of E-AB aircraft that included the use of an EAA survey of E-AB pilots. Among other findings, NTSB concluded that the flight test period—the first 50 hours of flight—is uniquely challenging for most E-AB pilots because they must learn to manage the handling characteristics of an unfamiliar aircraft while also managing the challenges of the flight test environment, including instrumentation that is not yet calibrated, controls that may need adjustments, and possible malfunctions or adverse handling characteristics. NTSB added that the E-AB safety record could be improved by providing pilots with additional training resources and, accordingly, made several recommendations to FAA and EAA regarding flight training and testing.

Some Industry Segments Experienced Fatal Accidents Disproportionately to Their Estimated Annual Flight Hours

To better understand the above observations about the airplanes involved in and the types of operations flown during general aviation accidents, we compared the proportions of fatal accidents by airplane category and operation type to their shares of FAA estimated flight hours for 1999 through 2010. For this analysis, we considered 5 airplane categories: (1) non-E-AB, single-engine piston; (2) non-E-AB, multi-engine piston; (3) non-E-AB, turbine engine; (4) E-ABs, regardless of engine type; and (5) others. As designated, there is no overlap in this categorization. If there were no relationship between accidents and airplane category, then we would expect each airplane category to be involved in accidents in proportion to its share of overall flight activity; for example, we would expect an airplane category that comprised 50 percent of general aviation flight hours to also comprise 50 percent of accidents. We found this to be the case with the single-engine piston airplane. Though the single-engine piston airplane is most often involved in fatal general aviation accidents, its share of fatal accidents (60 percent) was slightly less than its share of general aviation flight hours (65 percent). By comparison, E-ABs comprised 21 percent of fatal accidents, but only 4 percent of estimated flight hours. With regard to type of operation, we found that 77 percent of fatal accidents occurred during personal operations, but only 40 percent of the estimated flight hours involved personal operations. (See table 3.)

Loss of control in flight—the unintended departure of an aircraft from controlled flight, airspeed, or altitude—was the most common defining event in fatal general aviation accidents. Loss of control can occur because of aircraft malfunction, human performance, and other causes. During the period we examined, 1,036 fatal accidents (31 percent) were categorized as loss of control in flight. This was the most common event in a fatal accident for 3 of the 4 types of general aviation operations— personal, instructional, and business operations—and for all types of airplanes. FAA and the industry recently completed a review of a subgroup of fatal loss of control accidents and will be developing detailed implementation plans for the intervention strategies.

Pilot Error Was a Cause of Most Accidents, but Targeting Mitigations Is Difficult because of a Lack of Pilot Data

According to our analysis of NTSB data, the pilot was a cause in more than 60 percent of the general aviation accidents from 2008 through 2010. The pilot’s actions, decision making, or cockpit management was a cause for 70 percent of the airplanes in fatal accidents and 59 percent in nonfatal accidents. NTSB and other experts view aviation accidents as a sequence of events with multiple causes and contributing factors. Of the 2,801 general aviation accidents that occurred from 2008 through 2010 for which a causal determination was made, 71 percent were determined to have multiple causes. In approximately 34 percent of fatal accidents involving airplanes, the cause was a combination of the pilot’s actions and the failure to properly attain or maintain a performance parameter—e.g., airspeed and altitude.

In some instances, there was more than one pilot associated with an airplane. Since we were unable to determine from the data which pilot was in control of the aircraft at the time of the incident, we included data on all pilots involved in the accident in this and subsequent analyses regarding pilot characteristics or experience. percent of all pilots had fewer than 100 hours in any given airplane make and model, then we could expect the results of the above analysis even if pilot flight hours in the airplane make and model had no relation to accidents. We discuss the implications of the lack of this and other data later in this section.

To further explore the relationship between pilot flight hours and accidents, we looked at the portion of pilots with fewer than 100 hours in the accident airplane make and model where the pilot was determined to be a cause of the accident and compared it to the portion of pilots with more than 100 hours in the accident airplane make and model. We then did the same using pilot certification levels. Our analysis of accidents from 2008 through 2010 found that private pilots with fewer than 100 hours of experience in the accident airplane make and model were a cause of fatal and nonfatal general aviation accidents at similar rates as pilots with more than 100 hours of experience and with higher pilot certifications. For fatal accidents, 73 percent of pilots with fewer than 100 hours of experience in the accident airplane make and model were a cause as compared to 76 percent of pilots with more than 100 hours of experience. In nonfatal accidents, those portions were 63 and 64 percent, respectively. With regard to pilot certification levels, we found that in nonfatal accidents, private pilots were a cause more often (68 percent) than other types of pilots (percentages ranging from 52 to 58 percent); but in fatal accidents, similar proportions of private and commercial pilots were found to be a cause (75 percent and 80 percent, respectively).

Although some experts may believe that lack of experience can contribute to pilot error and accidents, the above suggests that this might not necessarily be the case. However, we do not have enough information to draw any real conclusions because FAA lacks certain key information about pilots that could help identify the root causes of accidents and, thus, risk mitigation opportunities. First, FAA’s estimate of the number of active pilots is an imperfect measure because, according to FAA’s definition, an active pilot is a certificated pilot who holds a valid medical certificate. However, depending on the type of operation the pilot is flying and the pilot’s certification level, age, and health condition, the medical certificate is valid for between 6 and 60 months. The designation as active is also not an indication of whether the pilot has actually flown in the previous year. Second, though pilots report total flight hours as part of their medical certificate application, a pilot’s experience in different makes and models of aircraft—which is not collected—is also relevant as there are risks associated with operating an unfamiliar airplane. As described above, this information would be necessary to draw conclusions about the effect of pilot flight hours on accidents. Third, though pilot flight hours are to be reported as part of the accident report, investigators are not always able to obtain this information for accident pilots as the logbooks in which it is recorded are sometimes destroyed in accidents. Of the 3,257 pilots involved in an accident from 2008 through 2010, pilot flight hours in the accident airplane make and model was missing for 514, or 16 percent of them. Missing data can compromise the validity of analyses that seek to examine the relationship between pilot experience and the causes of general aviation accidents.

In addition, FAA does not maintain information about where pilots were trained or whether noncommercial pilots participate in any recurrent training programs other than its WINGS pilot proficiency program— information that would facilitate analyses of the relationship between pilot training and the causes of general aviation accidents and that could help identify shortcomings in current pilot training programs. Private pilots are not required to participate in recurrent training, though they must successfully complete a biennial review of their skills and knowledge by a designated pilot examiner or a certified flight instructor. In recent years, as pilot training has been identified as a contributing factor in high profile accidents, there has been a renewed focus on the sources and amount of pilot training and on altering the training paradigm. FAA has been required to take steps to maintain qualification and performance data on airline pilots, but there has been no decision about whether recurrent training will be included in the database, and no such effort has been undertaken with regard to the remaining pilot population. Without more information about the training of general aviation pilots—and not just those who are in accidents—FAA’s efforts to identify and target risk areas and populations is impeded.

Flight Activity Data Limitations Impede FAA’s Ability to Assess General Aviation Safety and Target Risk Mitigation Efforts

FAA estimates of general aviation annual flight hours—a measure key to NTSB’s calculation of general aviation accident and fatality rates and NTSB’s and FAA’s assessments of the safety of general aviation—may not be reliable because of methodological and conceptual limitations with the survey used to gather flight activity data. Since 1978, FAA has used a survey of aircraft owners to estimate annual general aviation flight hours. The survey was redesigned in 1999, and FAA has modified it since then, on its own volition and in response to NTSB recommendations, to improve the survey’s ability to capture activity trends. Changes include sampling 100 percent of certain subpopulations of general aviation aircraft owners who were previously underrepresented in the random sample response—such as owners of turbine engine, rotorcraft, and Alaska-based aircraft—and revising the process for collecting information from owners of multiple aircraft. FAA and NTSB believe these changes have improved the reliability of the survey’s estimates, but some conceptual and methodological limitations persist.

First, as with all surveys that rely on self-reported data, there is the risk that respondents will not be able to accurately recall and report information, introducing error and perhaps bias into the survey’s estimates. The general aviation survey, which is usually open from March through August each year, asks respondents to estimate the number of hours flown during the previous calendar year. Depending on funding availability, the survey has opened later or for shorter periods of time. This year, because of contracting-related delays in bringing the survey consultant on board, aircraft owners did not receive the first request for information about 2011 flight hours until August 2012. According to NTSB, accuracy depends on the record-keeping habits and memories of aircraft owners, and in some cases, the aircraft owners’ ability to obtain needed information from pilots who fly their aircraft. Though some portion of aircraft owners may record each flight in their logbooks, to which they can refer to complete the survey, logging each flight is not mandatory. To the extent aircraft owners rely on their recollection of flight hours flown in the previous year, long delays such as the one occurring this year are likely to further degrade the resulting information.

Second, the survey has long suffered from low response rates, and this shortcoming, combined with limited information about the population, can call into question any estimates based on the survey’s results. Since the current method for calculating the response rate was implemented in 2004, the overall response rate has ranged from 43 and 47 percent annually through 2010. The primary problem with low response rates is that they can lead to biased estimates if survey respondents and nonrespondents differ with regard to the variables of interest—in this case, annual flight hours. According to guidance from the Office of Management and Budget, agencies should plan to conduct item-level bias analyses if the expected response rate of the survey is below 70 percent and to consider the anticipated response rate in the decision to proceed with the survey. In 2011, the survey contractor completed a nonresponse analysis and concluded that there was no evidence of significant bias. However, relatively little is known about the aircraft owners who do not respond and, as a result, the contractor and we concluded that the sample is not rich enough in information to understand the differences between the two groups. For instance, there may be certain characteristics of owners that are associated with flying habits, such as the owner’s age or certification level. Though a low response rate does not necessarily imply bias, it does raise the possibility for it. Further, the ability to detect any such bias is limited by what is known about those who do not respond. Given these conditions, bias remains a serious concern.

An alternative data collection method implemented in 2004 for owners of multiple aircraft may also introduce bias to the survey’s flight-hour estimates. In an effort to improve response rates among owners of multiple aircraft who were less likely to respond because of the burden of multiple forms, the survey administrators developed a modified data collection procedure for these owners. This includes sending out a form and calling these owners to verify receipt of the survey and encouraging participation. Survey staff also collect essential data—including the number of hours flown—during these phone calls. This alternative method accounted for data for approximately 23 percent of the aircraft owners responding to the survey that estimated 2010 flight hours. These efforts may have improved response rates, but these owners, the aircraft they own, and their use of the aircraft likely differ from owners of a single aircraft. By encouraging responses from a particular set of owners, survey estimates may be biased.

Flight hours account for what stakeholders refer to as “exposure” or how often particular types of operations or aircraft are flown. FAA’s flight hour estimates can provide a general sense of the relationship between hours and accidents. However, the methodological and conceptual limitations we have identified call the estimates’ precision into question. As a result, these estimates may not be sufficient for drawing conclusions about small changes in accident rates over time—including FAA’s progress toward its goal to reduce the fatal general aviation accident rate per 100,000 flight hours by 10 percent over 10 years. Implementing alternative means of collecting flight hour data, such as requiring the reporting of aircraft engine-revolution or run-time data, could supplement or replace the data generated through the survey and add rigor to FAA’s flight-hour estimates. Moreover, more precise flight-hour data could allow FAA to better target its safety efforts at subpopulations within the general aviation community. This could include reviewing an industry segment’s characteristics, such as the number of fatal accidents relative to its portion of estimated flight hours and setting a measurable goal for improving safety within that segment. Though FAA has attempted to address the disproportionate number of fatalities within the E-AB community by developing an advisory circular to encourage transition training for pilots, it has not set a specific goal for reducing fatal accidents in that segment.

FAA and NTSB, to their credit, have recognized that flight-hour estimates derived from the general aviation survey are imperfect. FAA has discussed ways to improve its flight-hour data, including requiring general aviation owners to report flight hours (in the form of engine-revolution or run-time data) directly to FAA during aircraft registration renewals or at the annual aircraft maintenance check. However, collecting data from these alternative sources has not progressed beyond internal discussions. In addition, organizations representing pilots have generally been opposed to suggestions for increased data collection, which they view as potential impediments to flying. According to these groups, general aviation pilots typically would prefer to avoid additional regulation or federal involvement.

In 2005, NTSB explored using alternative approaches to determining annual general aviation activity, approaches that involved using other measures as proxies for hours flown—including the number of active pilots and fuel consumption. However, there are shortcomings to each of these options. As discussed previously, active pilots are defined as those who have current medical certifications; this is not related to whether the pilot actually flew in a given year. And while aviation gas consumption could be a proxy measure for piston engine aircraft activity, some piston- engine aircraft are used for operations other than general aviation. Further, jet fuel consumption cannot reasonably be used as a proxy for the general aviation activity of turbine engine aircraft because of the many types of operations (e.g., air taxi, air ambulance, etc.) flown by these aircraft.

FAA’s Singular Goal to Reduce the Fatal Accident Rate May Mask Problems in Certain Segments of General Aviation

In 2008, FAA set a goal to reduce the fatal general aviation accident rate by 10 percent—from a baseline of 1.12 fatal accidents per 100,000 flight hours to 1 fatal accident per 100,000 flight hours—over 10 years, from 2009 to 2018. This single long-term safety goal may mask problems in certain segments of the community. The goal stemmed from FAA’s desire to have a target for its general aviation safety improvement efforts that accounted for changes in flight activity over time. According to FAA officials, they were looking for a goal that was achievable and represented an improved level of safety. FAA did not meet the annual targets for the goal in 2009 and 2010 and, according to projections of flight activity, it does not appear FAA will meet its target in 2011.

This singular goal is applied to an industry that is diverse in aircraft types and operations—some of which experience accidents at a higher rate than others. General aviation airplanes differ significantly in size and performance, ranging from single-seat E-AB airplanes to large corporate jets. The types of flying and pilot experience also vary by segment. Some private pilots may only fly a few times each year, while some corporate pilots may keep a schedule similar to that of a commercial airline pilot. In addition, given the expense of flying and maintaining an airplane, downturns in the economy can decrease activity in some segments of general aviation. Changes in flight activity in certain segments of the industry could mask or minimize problems in others and contribute to a rate that does not accurately reflect the trends in the individual segments. (See fig. 4.) For instance, total general aviation flight hours have decreased since the most recent recession, but some segments have declined at a faster rate than others. Personal flying hours in 2010 were 4 percent lower than they were in 2008; corporate flying hours, by comparison, were almost 15 percent lower in 2010 than in 2008. Historically, corporate flying has been one of the safest types of general aviation operations. From 1999 through 2010, corporate airplane operations accounted for just 1 percent of fatal general aviation accidents but 14 percent of flight hours. And from 2008 through 2011, there were no fatal accidents involving corporate airplane operations. As a result, changes in corporate flight activity could result in changes in the overall fatal accident rate that are not necessarily a reflection of changes in safety but rather a reflection of the changing composition of general aviation flight activity. In addition, as previously discussed, the rate is based on estimates of annual general aviation flight hours that may not be reliable.

There has been some discussion within FAA and industry about implementing separate goals for each segment of general aviation. According to one stakeholder we interviewed, the types of operations— even among fixed-wing aircraft—differ enough to warrant such a disaggregation. He explained that an hour flown during a corporate operation, during which an advanced aircraft flies from point to point with a significant portion of the time spent en route, is quite different from a pilot flying for pleasure and practicing maneuvers and take-offs and landings—the phase of flight when most accidents occur. However, other stakeholders we interviewed maintained that they all fly under the same operating rules, so it is proper to consider the safety of general aviation as a whole. Given the significant dissimilarities among the various general aviation sectors, along with the varied accident and fatality rates, setting separate safety improvement goals would allow FAA to take a more risk- based approach and target its resources and safety improvement efforts to the unique characteristics of and risks posed by each sector.

FAA Has Key Initiatives Under Way to Improve General Aviation Safety, but One Has Several Shortcomings

FAA has embarked on key initiatives to achieve its goal of a 10-percent reduction in the fatal general aviation accident rate per 100,000 flight hours by 2018. One is the long-standing General Aviation Joint Steering Committee (GAJSC), which is led by the Office of Accident Investigation and Prevention. More recently, FAA announced a 5-year strategy to improve general aviation safety that was developed by the General Aviation and Commercial Division of the Flight Standards Service. Although both initiatives work toward the overall goal of reducing general aviation fatalities, the GAJSC is using a data-driven approach to identify risks in general aviation operations and propose mitigations, while the 5- year strategy is composed of a wide variety of activities under four focus areas.

FAA Renewed the GAJSC in Early 2011

In January 2011, FAA renewed the GAJSC, a joint FAA effort with the general aviation industry, the National Aeronautics and Space Administration (NASA), and NTSB that in 1998 was part of the Safer Skies Initiative. Utilizing the model of the Commercial Aviation Safety Team (CAST), the GAJSC’s goal is to focus limited government and industry resources on data-driven risk reductions and solutions to general aviation safety issues. The GAJSC consists of a steering committee that provides, among other things, strategic guidance and membership outreach. It also consists of a safety analysis team (SAT), which determines future areas of study and charters safety studies, among other things. GAJSC officials indicated that they would charter working groups as issues for study were identified.

The first working group of the renewed GAJSC focused on loss of control in approach and landing accidents. This area was selected because, according to analyses of NTSB accident data for fatal airplane accidents that occurred from 2001 through 2011and for which NTSB had completed its investigation, loss of control was the number one causal factor. The working group divided into three subgroups—reciprocating non-E-AB aircraft, turbine engine aircraft, and E-AB aircraft—and agreed upon a sample of 30 accidents to be analyzed by each. Despite issues such as a lack of data and the consistency of member participation, the working group developed 83 intervention strategies. These strategies were used to develop the 27 safety enhancements that were presented to the GAJSC for approval. The GAJSC approved 23 of the safety enhancements. The next steps will include developing detailed implementation plans for each of the strategies, with the SAT conducting resource/benefit evaluations of each plan. The SAT then will determine which are the most effective solutions, draft a master strategic plan, and submit the plan to the GAJSC for approval. Implementation is expected to begin upon approval. During implementation, the SAT will be responsible for tracking implementation schedules and levels, tracking the effectiveness of the intervention strategies, and recommending areas for future study. We believe that with the GAJSC’s renewal and adoption of CAST-like methods, it has the potential to contribute to a reduction in general aviation accidents and fatalities over the long term.

FAA Launched a 5-Year Strategy in 2011 to Help Reduce the Fatal General Aviation Accident Rate

In March 2011, FAA announced its 5-year strategy to improve general aviation safety. This initiative is a complementary effort to the work of the GAJSC. FAA described the strategy as a nonregulatory approach conducted in partnership with the general aviation community and coordinated across FAA lines of business. The strategy has four focus areas—(1) risk management, (2) safety promotion, (3) outreach and engagement, and (4) training—and includes a 2-year review and the development of validation metrics as each phase of the plan is implemented.

FAA initially planned to concentrate its risk management efforts in three areas: (1) the top 10 causes and contributing factors in fatal general aviation accidents—initiated in coordination with the GAJSC, (2) E-AB aircraft, and (3) agricultural operations, which comprise one segment of the general aviation sector. To begin this effort, an FAA team identified the top ten causes of fatal general aviation accidents as well as the leading contributing factors, and provided the information to the GAJSC. The GAJSC, as previously discussed, is using the results of the data analysis to focus its efforts on loss-of-control accidents during approach and landing.

For the safety promotion aspect of its 5-year strategy, FAA relies on the FAA Safety Team (FAASTeam). Created in September 2004 as the the FAASTeam consists of 154 education and outreach arm of FAA, FAA employees in eight regional field offices, along with 32 groups and 2,500 individual members from the general aviation industry. In 2011, FAA refocused the FAASTeam—from national and international activities—to promote general aviation safety and technical proficiency through a host of nationwide seminars and contact with pilots at airports. A significant part of the FAASTeam’s new focus is the annual FAA safety standdown—a series of nationwide meetings that highlight issues of concern for general aviation and include industry and GAJSC member participation. The 2012 standdown focused on loss of control, the focus of a GAJSC working group, from three different perspectives: (1) preflight mistakes, (2) aeronautical decision making, and (3) handling a loss of control. In addition, the FAASTeam is conducting workshops for certified flight instructors to increase the quality of training offered to general aviation pilots. The FAASTeam has also been examining intervention strategies by working directly with designated pilot examiners to promote its educational opportunities to all applicants for practical tests.

In its outreach and engagement efforts for the 5-year strategy, FAA has briefed aviation associations, type clubs, and flight instructors, and, with the assistance of the Aviation Accreditation Board International, held a symposium on flight training with academia in July 2011. FAA has also reached out to major aviation insurance providers. As a result of these and other efforts, FAA reports that it has strengthened its links with aviation associations while also improving its outreach efforts to type clubs.

The training portion of FAA’s 5-year strategy includes chartering an aviation rulemaking committeeon pilot testing standards and training, expanding its focus on certified flight instructors, and revamping the WINGS pilot proficiency program. In September 2011, FAA announced the establishment of an aviation rulemaking committee to address concerns from AOPA, SAFE, and others about the testing and training standards for pilots.flight instructor, private pilot, instrument rating, and commercial pilot certificates. It made nine recommendations to FAA to enhance the pilot- testing and pilot-training processes. The recommendations included establishing a stakeholder body to assist in the development of knowledge test questions and handbook content as well as transitioning to a single testing standard document for the knowledge test. FAA concurred with most of the rulemaking committee’s recommendations.

The rulemaking committee focused on the certified To increase its focus on certified flight instructors, FAA is reviewing certified flight instructor recurrent training and renewal requirements. FAA also updated the advisory circular on flight instructor courses and published it in September 2011.

The FAASTeam’s voluntary WINGS pilot proficiency program is being revamped to encourage more participation. An FAA-established industry group has been surveying pilots to determine what changes need to be made to the WINGS program. Once the survey is completed, the resulting data will be analyzed and recommendations for changes will be made by the end of fiscal year 2012. FAA officials anticipate implementing changes to the program as funding becomes available in fiscal year 2013.

The 5-Year Strategy Has Significant Shortcomings

FAA’s 5-year strategy to improve general aviation safety suffers from several shortcomings that hinder its potential for success. First, senior FAA officials acknowledged that there are no specific performance goals or measures for the activities under the 5-year strategy. The officials said that because the goal of the initiative, as a whole, is to change general aviation culture, the strategy’s success will be measured through changes in the general aviation fatal accident rate. They also indicated that they are developing validation metrics as each phase of the plan is implemented. However, successful results-oriented organizations measure their performance at each organizational level by developing performance measures. Without performance goals or measures for the individual initiatives implemented under the 5-year strategy, FAA will not be able to evaluate the success or failure of those activities, regardless of whether the fatal accident rate is reduced. Further, FAA has yet to meet its annual target for the general aviation fatal accident rate goal and may not meet the overall goal by 2018. Therefore, it is even more crucial that FAA determine whether these activities have been successful.

Second, the strategy was developed without the initial input of significant stakeholders—the GAJSC and the general aviation industry. Successful agencies we have studied based their strategic planning, to a large extent, on the interests and expectations of their stakeholders, and stakeholder involvement is important to ensure that agencies’ efforts and resources are targeted at the highest priorities. According to officials from the GAJSC and the general aviation industry groups we contacted, although they were briefed on the strategy, they were not consulted in its development and were surprised by the announcement of the strategy. General aviation industry trade groups, type clubs, and other organizations are active in promoting a safety culture and continuous education among their members. For example, AOPA offers numerous seminars each year to educate the pilot community, and EAA offers advisory programs for experimental aircraft builders and pilots. Further, many initiatives are joint efforts of FAA and the industry. Involving stakeholders in strategic planning efforts can help create a basic understanding among the stakeholders of the competing demands that confront most agencies, the limited resources available to them, and how those demands and resources require careful and continuous balancing. FAA officials have indicated that their initial publication of the strategy served as a “straw man” for obtaining industry’s input and that there has been industry acceptance of the strategy as demonstrated by various industry groups’ development of plans and programs supporting the strategy. However, a lack of industry input into the development and announcement of the strategy jeopardizes its prospects for acceptance and success. This may be indicated in the current perspective of two industry groups—which is that the best use of industry resources to improve general aviation safety is through the work of the GAJSC.

Third, the FAASTeam, which will be the main vehicle for promoting the 5- year strategy to the industry, lacks the confidence of two significant general aviation industry stakeholders we interviewed, and its reorganization has not been completed. These industry stakeholders indicated that there is inconsistency in the focus of the FAASTeam. One stakeholder noted that industry “struggles to understand the role of the FAASTeam,” and the other stated that the FAASTeam is “well intentioned, but unfocused.” In addition, FAA initially planned to reorganize the FAASTeam to reduce the number of volunteers to a strong core group and to include a national FAASTeam located in Washington, D.C. However, a senior FAA official recently indicated that the restructuring of the FAASTeam is in flux and that the plan to reduce the number of volunteers to a strong core group does not begin until 2013. We believe that until there is a strong performance management structure, input and buy-in from industry, and a respected and organized FAASTeam, the effectiveness of the 5-year strategy will be in jeopardy.

FAA Has Other Initiatives Under Way That Could Also Contribute to Improved General Aviation Safety

Formed a rulemaking committee to recommend revisions to the small airplane airworthiness standards: In August 2011, FAA chartered a rulemaking committee to reorganize part 23—which promulgates airworthiness standards for small airplanes—according to airplane performance and complexity criteria as opposed to the traditional criteria of airplane weight and propulsion. The goals of this rulemaking committee include increasing safety and decreasing certification costs. Co-chaired by the manager of FAA’s Small Airplane Directorate, the rulemaking committee includes members representing other sections of the Aircraft Certification and Flight Standards Services as well as members from industry groups, manufacturers, and foreign aviation authorities. The committee is expected to complete its work by the summer of 2013.

Encouraging adoption of a safety management system (SMS): In guidance issued in April 2011, FAA encouraged general aviation business and corporate operators to develop and implement SMS.

Visual flight rules govern the procedures for conducting flight under visual conditions, as opposed to instrument flight rules, which govern the procedures for conducting flights using instruments. year 2007, the Weather Camera Program has funded the procurement and installation of 182 weather camera sites in Alaska. The cameras provide near real time video images of sky conditions at airports, mountain passes, and strategic VFR locations, such as high- use air routes, to enhance pilots’ situational awareness. According to FAA, this new capability is providing measurable reductions in weather-related VFR accidents in Alaska. FAA’s goal is to install a total of 221 weather camera sites.

Technology and Equipment May Also Help Improve General Aviation Safety

According to FAA, new technologies such as inflatable restraints (air bags), ballistic parachutes, weather in the cockpit, angle-of-attack indicators, and terrain avoidance equipment could significantly reduce general aviation fatalities. Angle of attack indicators and inflatable restraints have the greatest likelihood of significantly improving safety. Angle-of-attack indicators provide the pilot with a visual aid to prevent loss of control of the aircraft. Previously, cost and complexity of indicators limited their use to the military and commercial aircraft. FAA has streamlined the approval of angle-of-attack indicators for general aviation aircraft and is working to promote the retrofit of the existing fleet. FAA is also streamlining the certification and installation of inflatable restraints with the goal of making all general aviation aircraft eligible for installation. Further, FAA is working with manufacturers to define equipage requirements and support the Next Generation Air Transportation System (NextGen)—a new satellite-based air traffic management system that by 2025 will replace the current radar-based system—by streamlining the certification and installation of NextGen technologies. Some industry experts told us, however, that there might not be future opportunities to significantly improve general aviation safety with the aid of technology since most accidents are still attributed to pilot error.

Conclusions

To further reduce the number of fatal general aviation accidents, FAA needs to effectively target its accident mitigations, as it is attempting to do through the GAJSC. The agency’s ability to do so, however, is limited by a lack of pilot data. For instance, FAA does not maintain certain key information about general aviation pilots, including how many are actively flying each year and whether they participate in recurrent training other than FAA’s own WINGS program. Without this information, FAA cannot determine the potential effect of the various sources and types of training on pilot behavior, competency, and the likelihood of an accident. The lack of pilot data also makes it difficult to identify the root causes of accidents attributed to pilot error and determine appropriate risk mitigation opportunities.

The annual survey FAA uses for collecting general aviation flight-activity data suffers from significant limitations—limitations that call into question the resulting activity estimates FAA produces as well as the accident rates calculated by NTSB. Though FAA has improved the survey over the years, our concerns remain because the survey continues to experience response rates below 50 percent and relies on the record-keeping habits and memories of survey respondents who sometimes have to recall details that occurred more than 12 months earlier. Further, other methods for obtaining general aviation flight-activity data have encountered resistance from the industry. Without a more accurate reporting of general aviation flight activity, such as requiring the reporting of flight hours at certain intervals—e.g., during registration renewals or annual maintenance inspections—FAA lacks assurance that it is basing its policy decisions on a true measure of general aviation trends, and NTSB lacks assurance that its calculations of accident and fatality rates accurately represent the state of general aviation safety.

Given the diversity of the general aviation community—illustrated, for example by the wide variety of aircraft in the fleet and the varying nonfatal and fatal accident rates among the general aviation segments, the adoption of a singular agency goal–-a 10 percent reduction in the general aviation fatal accident rate per 100,000 flight hours by 2018 is not the most effective risk-based tool for achieving general aviation safety gains. The goal does not take into account the variety of general aviation operations or the risks associated with each. For example, one hour flown during a personal operation is not the same as one hour flown during a corporate operation. Also, economic conditions affect each segment differently, making it difficult to discern if a change in the accident rate is an indication of a change in the safety of the industry. If the goal is reached, the overall success might mask ongoing safety issues in one or more segments of the community.

FAA officials have indicated that the success of the 5-year strategy— which is composed of numerous initiatives—will be measured through changes in the general aviation fatal accident rate. However, successful results-oriented organizations measure their performance at each organizational level by developing performance measures. For this reason, we think it is important for FAA to develop performance measures for the significant initiatives underlying the 5-year strategy. This is important because if FAA does not measure the performance of the significant underlying initiatives, it will not be able to determine whether the initiatives were effective in their own right. In addition, in order for the FAASTeam to be successful in its promotion of the 5-year strategy, it must be well respected within the general aviation community. We are not making a recommendation regarding the FAASTeam at this time since plans for restructuring it are in flux and its volunteer force realignment is not scheduled to begin until 2013.

Recommendations for Executive Action

To enhance FAA’s efforts to improve general aviation safety, we recommend that the Secretary of Transportation direct the FAA Administrator to take the following four actions:

To expand the data available for root cause analyses of general aviation accidents and other purposes, collect and maintain data on each certificated pilot’s recurrent training, and update the data at regular intervals.

Improve measures of general aviation activity by requiring the collection of the number of hours that general aviation aircraft fly over a period of time (flight hours). FAA should explore ways to do this that minimize the impact on the general aviation community, such as by collecting the data at regular events (e.g., during registration renewals or at annual maintenance inspections) that are already required.

To ensure that ongoing safety issues are addressed, set specific general aviation safety improvement goals—such as targets for fatal accident reductions—for individual industry segments using a data- driven, risk management approach.

To determine whether the programs and activities underlying the 5- year strategy are successful and if additional actions are needed, develop performance measures for each significant program and activity underlying the 5-year strategy.

Agency Comments

We provided the Department of Transportation (DOT) with a draft of this report for review and comment. DOT officials agreed to consider our recommendations and provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Transportation, the Chairman of NTSB, and interested parties. In addition, this report is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix II.

Appendix I: Scope and Methodology

Our objective was to conduct a comprehensive review of general aviation safety. To do so, we addressed the following questions: (1) what are the characteristics and trends in general aviation accidents from 1999 to 2011 and (2) what actions have been taken by the Federal Aviation Administration (FAA) to improve general aviation safety?

To identify the characteristics of and trends in general aviation accidents, we conducted a data analysis using the National Transportation Safety Board’s (NTSB) Aviation Accident Database. We limited our analysis to accidents involving airplanes operating under Part 91 of the Federal Aviation Regulations that occurred from January 1, 1999, through December 31, 2011, in the U.S. We excluded accidents that occurred in U.S. territories, possessions, and international waters. To assess the reliability of the NTSB data, we reviewed documentation on data collection efforts and quality assurance processes, talked to knowledgeable NTSB officials about the data, and checked the data for completeness and reasonableness. We determined that these data were sufficiently reliable for the descriptive and comparative analyses used in this report. To supplement our analysis of the NTSB accident data, we also analyzed FAA’s general aviation flight-hour estimates for 1999 through 2010 and estimated active pilot data for 2011. To assess the reliability of these data, we reviewed documentation on data collection efforts and quality assurance processes and talked to knowledgeable FAA officials. In assessing the reliability of the flight-hour estimates, we also spoke with the contractors responsible for executing the survey that yielded these estimates, the General Aviation and Part 135 Survey. We determined that the flight-hour data and the active pilot data were sufficiently reliable for the purposes of this engagement. Specifically, these data elements were sufficiently reliable to provide meaningful context for the numbers and characteristics of accidents that we report. However, we also determined that because of the methodological limitations identified—a low response rate and the potential for nonresponse bias—the flight-hour estimates developed from the General Aviation and Part 135 Survey may not have the precision necessary to measure small changes in the general aviation accident rate over time.

To identify actions FAA and others have taken to improve general aviation safety, we reviewed our prior reports as well as documents and reports from FAA, NTSB, NASA, and general aviation industry trade and other groups, including the Aircraft Owners and Pilots Association (AOPA), the Experimental Aircraft Association (EAA), and the Society of Aviation and Flight Educators (SAFE); FAA orders, notices, advisory circulars; and applicable laws and regulations. We also determined the roles and responsibilities of FAA and NTSB in collecting and reporting general aviation safety data. In addition to interviewing officials from the various FAA offices and divisions responsible for general aviation safety, we interviewed aviation experts affiliated with various aviation industry organizations. (See table 4.)

To obtain additional insight into the general aviation industry, we attended the September 2011 AOPA Aviation Summit in Hartford, Connecticut; the March 2012 Annual FAA Aviation Forecast Conference in Washington, D.C.; the February 2012 Northwest Aviation Conference in Puyallup, Washington; and the June 2012 NTSB General Aviation Forum in Washington, D.C.

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, the following individuals made important contributions to this report: H. Brandon Haller, Assistant Director; Pamela Vines; Jessica Wintfeld; Russ Burnett; Bert Japikse; Delwen Jones; Josh Ormond; and Jeff Tessin.