General Aviation Safety: Additional FAA Efforts Could Help Identify and Mitigate Safety Risks
Published: 2012-10-04T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Although the U.S. aviation system is one of the safest in the world, hundreds of fatalities occur each year in general aviationwhich includes all forms of aviation except commercial and military. The general aviation industry is composed of a diverse fleet of over 220,000 aircraft that conduct a wide variety of operationsfrom personal pleasure flights in small, piston aircraft to worldwide professionally piloted corporate flights in turbine-powered aircraft. According to 2011 National Transportation Safety Board (NTSB) data, 92 percent of that years fatal accidents occurred in general aviation. The majority of general aviation accidents are attributed to pilot error. GAO was asked to examine the (1) characteristics of and trends in general 2011 and (2) recent actions taken by FAA to improve general aviation safety. GAO analyzed NTSB accident data, reviewed government and industry studies and other documents, and interviewed FAA and NTSB officials and industry stakeholders.
section_title: What GAO Found
paragraphs: The number of nonfatal and fatal general aviation accidents decreased from 1999 through 2011; more than 200 fatal accidents occurred in each of those years. Airplanesparticularly single-engine piston airplanesflying personal operations were most often involved in accidents. Most general aviation accidents are attributed to pilot error and involved a loss of aircraft control. Some segments of the industry experienced accidents disproportionately to their total estimated annual flight hours. For example, among the airplane categories we reviewed, experimental amateur-built airplanes were involved in 21 percent of the fatal accidents but accounted for only 4 percent of the estimated annual flight hours. In another example, corporate operations were involved in about 1 percent of fatal accidents while accounting for 14 percent of estimated annual flight hours. We can draw some conclusions about general aviation accident characteristics, but limitations in flight activity and other data preclude a confident assessment of general aviation safety. The Federal Aviation Administrations (FAA) survey of general aviation operators, on which the agency bases its annual flight-hour estimates, continues to suffer from methodological and conceptual limitations, even with FAAs efforts to improve it over the years. To obtain more reliable data, FAA has discussed requiring that flight-hour data be reported, such as during annual aircraft maintenance inspections. FAA has set a goal to reduce the fatal general aviation accident rate per 100,000 flight hours by 10 percent from 2009 to 2018. However, given the diversity of the industry and shortcomings in the flight activity data, this goal is not sufficient for achieving reductions in fatality rates among the riskier segments of general aviation. Further, achieving the goal could mask continuing safety issues in segments of the community.
FAA has embarked on several initiatives to meet its goal of reducing the fatal general aviation accident rate by 2018. These include the renewal of the General Aviation Joint Steering Committee (GAJSC) with a data-driven approach and the implementation of the Flight Standards Services 5-year strategy. The GAJSC, a government-industry partnership, focuses on analyzing general aviation accident data to develop effective intervention strategies. The 5-year strategy involves numerous initiatives under four focus areas: (1) risk management, (2) outreach which is composed of FAA staff and industry volunteers, will be responsible for carrying out significant portions of the strategy. While the GAJSCs efforts are modeled on an approach deemed successful in contributing to a reduction in fatal jeopardize its potential for success. For example, the strategy lacks performance measures for the significant activities that comprise it. Without a strong performance management structure, FAA will not be able to determine the success or failure of the significant activities that underlie the 5-year strategy.
section_title: What GAO Recommends
paragraphs: GAO recommends, among other things, that FAA require the collection of general aviation aircraft flight-hour data in ways that minimize the impact on the general aviation community, set safety improvement goals for individual general aviation industry segments, and develop performance measures for the significant activities underlying the 5-year strategy. Department of Transportation officials agreed to consider GAOs recommendations and provided technical comments, which GAO incorporated as appropriate.

section_title: Letter
paragraphs: The U.S. aviation system is one of the safest in the world and a significant  contributor to the nation’s economy. However, hundreds of fatalities occur  each year in the sector known as general aviation, which includes all  forms of aviation except commercial and military. According to National  Transportation Safety Board (NTSB) data, 92 percent of all fatal aviation  accidents in 2011 occurred in general aviation. Each fatal general  aviation accident typically involves a small number of casualties because  of the smaller aircraft that are usually flown in general aviation.  Nevertheless, these accidents can profoundly affect communities—as  with the November 23, 2011, crash near Apache Junction, Arizona, that  killed six people, including a father and his three children who were going  to the father’s home for Thanksgiving—and the nation—as with the  August 9, 2010, crash near Aleknagik, Alaska, in which former Senator  Ted Stevens and four others perished and several others were seriously  injured. According to National Transportation Safety Board (NTSB) data,  the majority of general aviation accidents occurs because of an error on  the part of the pilot and often involves causes similar to or the same as  those identified in prior accidents. In June 2011, NTSB added “improve  general aviation safety” to its most wanted list of 10 critical changes  needed to reduce transportation accidents and save lives.
You asked us to examine general aviation safety in the U.S. This report  discusses the (1) characteristics of and trends in general aviation  accidents from 1999 through 2011 and (2) recent actions taken by the  Federal Aviation Administration (FAA) to improve general aviation safety.  To address our objectives, we analyzed NTSB accident data, consulted  our prior work on general aviation safety trends as well as other related  work, reviewed other government and industry documents, and  interviewed FAA and NTSB officials and industry stakeholders. We  conducted this performance audit from June 2011 to October 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives. See appendix I for more  information about our scope and methodology.
subsections: section_title: Background
paragraphs: General aviation is characterized by a diverse fleet of aircraft flown for a  variety of purposes. In 2010, FAA estimated that there were more than  220,000 aircraft in the active general aviation fleet, comprising more than  90 percent of the U.S. civil aircraft fleet. Included among these aircraft are  airplanes, balloons, unmanned aircraft systems, gliders, and helicopters.  (See fig. 1.) Airplanes comprise the vast majority—almost 80 percent—of  the general aviation fleet. According to a 2009 FAA study, general  aviation airplanes have an average age of 40 years. In addition, most are  single-engine piston, such as the Beechcraft Bonanza, Cessna 172, and  Piper Archer.
FAA designates a small, but growing, portion of the general aviation fleet  as “experimental.” These include aircraft used for racing and research as  well as exhibition aircraft, such as former military aircraft known as  warbirds. The largest group of experimental aircraft—and the fastest  growing segment of the general aviation fleet, according to FAA—is  defined by FAA as “experimental-amateur built” (E-AB). Individuals build  E-AB aircraft either from kits sold by manufacturers or from their own  designs. E-AB aircraft can contain previously untested systems, including  engines not designed for aircraft use, and modifications of airframes,  controls, and instrumentation. The E-AB fleet is diverse, ranging from  open-framework designs with no cabin structure to small, pressurized  airplanes able to fly long distances. The majority are simple craft used  primarily for short personal flights. The expertise of the builders varies, as  does the experience of the pilots and the availability of training for  transitioning to the aircraft. Following a successful inspection of the  aircraft and documentation review, FAA issues a special airworthiness  certificate in the experimental category to the aircraft’s builder and  assigns operating limitations in two phases specifying how and where the  aircraft can be flown.the builder determines the aircraft’s airspeed and altitude capabilities and  develops a flight manual. Phase II refers to normal operations after the  flight testing is completed.
Phase I is the required flight test period, in which  General aviation aircraft can be used for a wide variety of operations,  although about 78 percent of general aviation operations fall under one of  four types: personal (e.g., a pilot taking his family on a sightseeing trip); business (e.g., a pilot flying herself to a meeting); corporate (e.g., a professionally-piloted aircraft transporting corporate  employees around the globe); and  instructional (e.g., a student flying with a certified flight instructor).
These operations are conducted from the more than 2,950 public use  general aviation airports (which primarily serve general aviation aircraft)  as well as from thousands of other airports (including those that support  commercial air service) and landing facilities (e.g., heliports). General  aviation flights operate under various federal aviation regulations. For  purposes of this report, our definition of general aviation includes flights  operated under part 91 general operation and flight rules.
GAO-12-117. Our definition of flight-instructor based schools includes individual flight  instructors. more information about the estimated number of active airplane pilots and  selected pilot certificate requirements and limitations.
Various offices within FAA are responsible for ensuring general aviation  safety, most notably the Flight Standards Service, Aircraft Certification  Service, Office of Accident Investigation and Prevention, and Office of  Runway Safety. According to FAA, the agency’s fiscal year 2011 budget  submission included nearly $203 million for activities within the Aviation  Safety organization related to the top priority of reducing the general  aviation fatal accident rate. FAA’s responsibilities include administering  aircraft and pilot certification, conducting safety oversight of pilot training  and general aviation operations, and taking enforcement actions against  pilots and others who violate federal aviation regulations and safety  standards. FAA also collects general aviation fleet and flight activity data  through an annual survey and supports the NTSB by gathering  information about general aviation accidents. According to NTSB officials,  FAA collects information on the vast majority of general aviation  accidents.
NTSB is responsible for all aviation accident investigations—using the  information gathered by FAA and its own investigators—and for  determining the probable cause of accidents. NTSB uses a coding  system of aircraft accident categories and associated phases of flight that  are useful in describing the characteristics and circumstances of aviation  accidents. For ease of interpretation and to categorize similar events,  NTSB identifies one event as the “defining event” of the accident, which  generally describes the type of accident that occurred—hard landing,  midair collision, or fuel exhaustion, for example. In addition, NTSB  identifies the causes of an accident and the contributing factors, which  describe situations or circumstances central to the accident cause. Just  as accidents often include a series of events, the reason those events led  to an accident may reflect a combination of multiple causes and  contributing factors. For this reason, a single accident report can include  multiple cause and contributing factor codes. NTSB also collects  descriptive information about the environmental conditions, aircraft, and  people involved in aviation accidents. It captures its findings and  descriptive information in its Aviation Accident Database. NTSB  calculates general aviation accident and fatality rates, which it does using  its own accident data and FAA’s annual estimates of general aviation  flight activity. NTSB may also recommend regulatory and other changes  to FAA and the aviation industry based on the results of its investigations  and any studies it conducts.
The U.S. general aviation industry includes a number of trade groups,  “type clubs,” and other organizations that actively promote the  importance of safety and, in many cases, offer educational opportunities  to pilots. Many of the groups also work with FAA on advisory and  rulemaking committees. Prominent trade organizations include the  Aircraft Owners and Pilots Association (AOPA), the Experimental Aircraft  Association (EAA), the General Aviation Manufacturers Association  (GAMA), and the National Business Aviation Association (NBAA). The  Society of Aviation and Flight Educators (SAFE) and the National  Association of Flight Instructors represent certified flight instructors and  other aviation educators. The American Bonanza Society (ABS), the  Cirrus Owners and Pilots Association, and the Lancair Owners and  Builders Organization are examples of the several general aviation type  clubs.
section_title: General Aviation Accidents Decreased, but Some Segments Had Disproportionate Shares of Accidents
paragraphs: Our analysis of NTSB accident data showed that the annual number of  general aviation accidents generally decreased for 1999 through 2011.  We also identified several characteristics of accidents with respect to the  types of operations and the causes of the accidents. These  characteristics were largely consistent with observations made during our  last review of general aviation safety in 2001. To better understand  these characteristics, where possible, we sought to measure their  occurrence in numbers of accidents in relation to their overall occurrence  in, for instance, total flight hours or pilot certifications as estimated by  FAA. In doing so, we identified some accident characteristics that, based  on our analysis, appear to occur disproportionately. However, we also  identified methodological and conceptual limitations with the activity  data—particularly the General Aviation and Part 135 Activity Survey that  FAA uses to estimate annual flight hours and the number of active  aircraft—that we discuss later in this section. See table 2 for a summary  of the characteristics of general aviation accidents according to our  analysis of the NTSB accident data.
From 1999 through 2011, nonfatal accidents involving general aviation  airplanes generally decreased, falling 29 percent, from 1,265 in 1999 to  902 in 2011. Fatal accidents generally decreased as well, falling 24  percent. Figure 2 indicates the number of fatal and nonfatal accidents for  each year we reviewed. During this period of time, though the majority  (approximately 56 percent) of all accidents resulted in no injuries, there  were more than 200 fatal accidents each year.
From 1999 through 2011, personal operations accounted for 73 percent  of airplanes in nonfatal general aviation accidents and 77 percent of  airplanes in fatal general aviation accidents. (See fig. 3.) This is not a new  phenomenon. As we reported about accidents occurring in 1998,  personal operations accounted for more than 75 percent of fatal general  aviation accidents.
From 1999 through 2011, airplanes flying instructional operations were  the second most often involved in accidents. However, instructional  operations were also the operation with the smallest proportion of fatal  accidents. According to our analysis, almost 38 percent of accidents that  occurred during instructional flying involved hard landings or loss of  control while the aircraft was on the ground. These types of events are  less likely to cause fatalities than other types of events. It is also possible  that the presence of a certified flight instructor onboard to share the  management of the cockpit and other tasks may have contributed to the  lower fatality rate for instructional operations.
Corporate operations, in which a professional pilot flies an aircraft owned  by a business or corporation, was the least common type of operation to  be involved in general aviation accidents. Corporate operations  accounted for less than 1 percent of fatal general aviation accidents and  less than 0.5 percent of nonfatal accidents. From 2008 through 2011,  there were no fatal accidents involving corporate airplanes, giving  corporate operations an accident record similar to that of commercial air  carriers. Again, this is not a new phenomenon. As we reported in 2001,  the low number of accidents involving corporate operations is attributable  to a number of factors, including the pilot’s training, experience, and  participation in ongoing training to maintain and improve their skills, as  well as the safety equipment that is typically installed on corporate  aircraft.
According to a representative of the NBAA, an organization representing  companies that rely on general aviation aircraft to conduct business, most  corporate operations also benefit from advanced technologies, including  avionics that provide synthetic vision and terrain displays; auto-throttle,  which helps maintain airspeed; and fuel gauges that are built to the  standards required for commercial airliners. Further, airplanes used for  corporate purposes are often powered by turbine engines and may be  subject to additional safety requirements. Flying for corporate purposes  can also differ from other types of flying. Whereas a pilot flying for fun  may perform several take-offs and landings and practice maneuvers, a  corporate flight likely includes a single take off and landing, with the  majority of time spent en route—one of the phases of flight when the  fewest fatal accidents occur.
Regarding the type of aircraft involved in general aviation accidents,  single-engine piston airplanes accounted for almost 76 percent of  airplanes in nonfatal general aviation accidents and 60 percent of  airplanes in fatal accidents. Single-engine piston airplanes are the most common type of aircraft in the general aviation fleet and, according to  stakeholders, the type of aircraft most commonly flown by pilots holding  private pilot certifications and flying for personal reasons. According to  AOPA, mechanical failures cause relatively few accidents, indicating that  the frequency with which single-engine piston airplanes are in accidents  is not necessarily a reflection of the safety of the aircraft.
E-ABs were the second most common airplane involved in general  aviation accidents. From 1999 through 2011, E-AB aircraft accounted for  14 percent of airplanes in nonfatal general aviation accidents and  approximately 21 percent in fatal accidents. According to EAA, the  organization that represents experimental and amateur-built aircraft  owners, E-AB airplanes were also the fastest growing type of aircraft in  the general aviation fleet in recent years. In 2011, there were  approximately 33,000 registered E-AB aircraft, a 10 percent increase from  3 years earlier. AOPA’s 2010 Nall Report—an annual safety report that  provides perspectives on the previous year’s general aviation accidents— indicated that the physical characteristics and the manner in which these  aircraft are used expose E-AB aircraft pilots to greater risk and make  accidents less survivable.
In 2012, NTSB completed a safety study of E-AB aircraft that included the  use of an EAA survey of E-AB pilots. Among other findings, NTSB  concluded that the flight test period—the first 50 hours of flight—is  uniquely challenging for most E-AB pilots because they must learn to  manage the handling characteristics of an unfamiliar aircraft while also  managing the challenges of the flight test environment, including  instrumentation that is not yet calibrated, controls that may need  adjustments, and possible malfunctions or adverse handling  characteristics. NTSB added that the E-AB safety record could be  improved by providing pilots with additional training resources and,  accordingly, made several recommendations to FAA and EAA regarding  flight training and testing.
subsections: section_title: Some Industry Segments Experienced Fatal Accidents Disproportionately to Their Estimated Annual Flight Hours
paragraphs: To better understand the above observations about the airplanes involved  in and the types of operations flown during general aviation accidents, we  compared the proportions of fatal accidents by airplane category and  operation type to their shares of FAA estimated flight hours for 1999  through 2010. For this analysis, we considered 5 airplane categories:  (1) non-E-AB, single-engine piston; (2) non-E-AB, multi-engine piston; (3)  non-E-AB, turbine engine; (4) E-ABs, regardless of engine type; and (5)  others. As designated, there is no overlap in this categorization. If there  were no relationship between accidents and airplane category, then we  would expect each airplane category to be involved in accidents in  proportion to its share of overall flight activity; for example, we would  expect an airplane category that comprised 50 percent of general aviation  flight hours to also comprise 50 percent of accidents. We found this to be  the case with the single-engine piston airplane. Though the single-engine  piston airplane is most often involved in fatal general aviation accidents,  its share of fatal accidents (60 percent) was slightly less than its share of  general aviation flight hours (65 percent). By comparison, E-ABs  comprised 21 percent of fatal accidents, but only 4 percent of estimated  flight hours. With regard to type of operation, we found that 77 percent of  fatal accidents occurred during personal operations, but only 40 percent  of the estimated flight hours involved personal operations. (See table 3.)
Loss of control in flight—the unintended departure of an aircraft from  controlled flight, airspeed, or altitude—was the most common defining  event in fatal general aviation accidents. Loss of control can occur  because of aircraft malfunction, human performance, and other causes.  During the period we examined, 1,036 fatal accidents (31 percent) were  categorized as loss of control in flight. This was the most common event  in a fatal accident for 3 of the 4 types of general aviation operations— personal, instructional, and business operations—and for all types of  airplanes. FAA and the industry recently completed a review of a  subgroup of fatal loss of control accidents and will be developing detailed  implementation plans for the intervention strategies.
section_title: Pilot Error Was a Cause of Most Accidents, but Targeting Mitigations Is Difficult because of a Lack of Pilot Data
paragraphs: According to our analysis of NTSB data, the pilot was a cause in more  than 60 percent of the general aviation accidents from 2008 through  2010. The pilot’s actions, decision making, or cockpit management was  a cause for 70 percent of the airplanes in fatal accidents and 59 percent  in nonfatal accidents. NTSB and other experts view aviation accidents as  a sequence of events with multiple causes and contributing factors. Of the  2,801 general aviation accidents that occurred from 2008 through 2010  for which a causal determination was made, 71 percent were determined  to have multiple causes. In approximately 34 percent of fatal accidents  involving airplanes, the cause was a combination of the pilot’s actions and  the failure to properly attain or maintain a performance parameter—e.g.,  airspeed and altitude.
In some instances, there was more than one pilot associated with an airplane. Since we  were unable to determine from the data which pilot was in control of the aircraft at the time  of the incident, we included data on all pilots involved in the accident in this and  subsequent analyses regarding pilot characteristics or experience. percent of all pilots had fewer than 100 hours in any given airplane make  and model, then we could expect the results of the above analysis even if  pilot flight hours in the airplane make and model had no relation to  accidents. We discuss the implications of the lack of this and other data  later in this section.
To further explore the relationship between pilot flight hours and  accidents, we looked at the portion of pilots with fewer than 100 hours in  the accident airplane make and model where the pilot was determined to  be a cause of the accident and compared it to the portion of pilots with  more than 100 hours in the accident airplane make and model. We then  did the same using pilot certification levels. Our analysis of accidents from  2008 through 2010 found that private pilots with fewer than 100 hours of  experience in the accident airplane make and model were a cause of fatal  and nonfatal general aviation accidents at similar rates as pilots with more  than 100 hours of experience and with higher pilot certifications. For fatal  accidents, 73 percent of pilots with fewer than 100 hours of experience in  the accident airplane make and model were a cause as compared to 76  percent of pilots with more than 100 hours of experience. In nonfatal  accidents, those portions were 63 and 64 percent, respectively. With  regard to pilot certification levels, we found that in nonfatal accidents,  private pilots were a cause more often (68 percent) than other types of  pilots (percentages ranging from 52 to 58 percent); but in fatal accidents,  similar proportions of private and commercial pilots were found to be a  cause (75 percent and 80 percent, respectively).
Although some experts may believe that lack of experience can contribute  to pilot error and accidents, the above suggests that this might not  necessarily be the case. However, we do not have enough information to  draw any real conclusions because FAA lacks certain key information  about pilots that could help identify the root causes of accidents and,  thus, risk mitigation opportunities. First, FAA’s estimate of the number of  active pilots is an imperfect measure because, according to FAA’s  definition, an active pilot is a certificated pilot who holds a valid medical  certificate. However, depending on the type of operation the pilot is flying  and the pilot’s certification level, age, and health condition, the medical  certificate is valid for between 6 and 60 months. The designation as  active is also not an indication of whether the pilot has actually flown in  the previous year. Second, though pilots report total flight hours as part of  their medical certificate application, a pilot’s experience in different makes  and models of aircraft—which is not collected—is also relevant as there  are risks associated with operating an unfamiliar airplane. As described  above, this information would be necessary to draw conclusions about the  effect of pilot flight hours on accidents. Third, though pilot flight hours are  to be reported as part of the accident report, investigators are not always  able to obtain this information for accident pilots as the logbooks in which  it is recorded are sometimes destroyed in accidents. Of the 3,257 pilots  involved in an accident from 2008 through 2010, pilot flight hours in the  accident airplane make and model was missing for 514, or 16 percent of  them. Missing data can compromise the validity of analyses that seek to  examine the relationship between pilot experience and the causes of  general aviation accidents.
In addition, FAA does not maintain information about where pilots were  trained or whether noncommercial pilots participate in any recurrent  training programs other than its WINGS pilot proficiency program— information that would facilitate analyses of the relationship between pilot  training and the causes of general aviation accidents and that could help  identify shortcomings in current pilot training programs. Private pilots are  not required to participate in recurrent training, though they must  successfully complete a biennial review of their skills and knowledge by a  designated pilot examiner or a certified flight instructor. In recent years,  as pilot training has been identified as a contributing factor in high profile  accidents, there has been a renewed focus on the sources and amount  of pilot training and on altering the training paradigm. FAA has been  required to take steps to maintain qualification and performance data on  airline pilots, but there has been no decision about whether recurrent  training will be included in the database, and no such effort has been  undertaken with regard to the remaining pilot population. Without more  information about the training of general aviation pilots—and not just  those who are in accidents—FAA’s efforts to identify and target risk areas  and populations is impeded.
section_title: Flight Activity Data Limitations Impede FAA’s Ability to Assess General Aviation Safety and Target Risk Mitigation Efforts
paragraphs: FAA estimates of general aviation annual flight hours—a measure key to  NTSB’s calculation of general aviation accident and fatality rates and  NTSB’s and FAA’s assessments of the safety of general aviation—may  not be reliable because of methodological and conceptual limitations with  the survey used to gather flight activity data. Since 1978, FAA has used a  survey of aircraft owners to estimate annual general aviation flight hours.  The survey was redesigned in 1999, and FAA has modified it since then,  on its own volition and in response to NTSB recommendations, to  improve the survey’s ability to capture activity trends. Changes include  sampling 100 percent of certain subpopulations of general aviation  aircraft owners who were previously underrepresented in the random  sample response—such as owners of turbine engine, rotorcraft, and  Alaska-based aircraft—and revising the process for collecting information  from owners of multiple aircraft. FAA and NTSB believe these changes  have improved the reliability of the survey’s estimates, but some  conceptual and methodological limitations persist.
First, as with all surveys that rely on self-reported data, there is the risk  that respondents will not be able to accurately recall and report  information, introducing error and perhaps bias into the survey’s  estimates. The general aviation survey, which is usually open from March  through August each year, asks respondents to estimate the number of  hours flown during the previous calendar year. Depending on funding  availability, the survey has opened later or for shorter periods of time.  This year, because of contracting-related delays in bringing the survey  consultant on board, aircraft owners did not receive the first request for  information about 2011 flight hours until August 2012. According to  NTSB, accuracy depends on the record-keeping habits and memories of  aircraft owners, and in some cases, the aircraft owners’ ability to obtain  needed information from pilots who fly their aircraft. Though some portion  of aircraft owners may record each flight in their logbooks, to which they  can refer to complete the survey, logging each flight is not mandatory. To  the extent aircraft owners rely on their recollection of flight hours flown in  the previous year, long delays such as the one occurring this year are  likely to further degrade the resulting information.
Second, the survey has long suffered from low response rates, and this  shortcoming, combined with limited information about the population, can  call into question any estimates based on the survey’s results. Since the  current method for calculating the response rate was implemented in  2004, the overall response rate has ranged from 43 and 47 percent  annually through 2010. The primary problem with low response rates is  that they can lead to biased estimates if survey respondents and  nonrespondents differ with regard to the variables of interest—in this  case, annual flight hours. According to guidance from the Office of  Management and Budget, agencies should plan to conduct item-level  bias analyses if the expected response rate of the survey is below 70  percent and to consider the anticipated response rate in the decision to  proceed with the survey. In 2011, the survey contractor completed a  nonresponse analysis and concluded that there was no evidence of  significant bias. However, relatively little is known about the aircraft  owners who do not respond and, as a result, the contractor and we  concluded that the sample is not rich enough in information to understand  the differences between the two groups.  For instance, there may be  certain characteristics of owners that are associated with flying habits,  such as the owner’s age or certification level. Though a low response  rate does not necessarily imply bias, it does raise the possibility for it.  Further, the ability to detect any such bias is limited by what is known  about those who do not respond.  Given these conditions, bias remains a  serious concern.
An alternative data collection method implemented in 2004 for owners of  multiple aircraft may also introduce bias to the survey’s flight-hour  estimates. In an effort to improve response rates among owners of  multiple aircraft who were less likely to respond because of the burden of  multiple forms, the survey administrators developed a modified data  collection procedure for these owners. This includes sending out a form  and calling these owners to verify receipt of the survey and encouraging  participation. Survey staff also collect essential data—including the  number of hours flown—during these phone calls. This alternative method  accounted for data for approximately 23 percent of the aircraft owners  responding to the survey that estimated 2010 flight hours. These efforts  may have improved response rates, but these owners, the aircraft they  own, and their use of the aircraft likely differ from owners of a single  aircraft. By encouraging responses from a particular set of owners, survey  estimates may be biased.
Flight hours account for what stakeholders refer to as “exposure” or how  often particular types of operations or aircraft are flown. FAA’s flight hour  estimates can provide a general sense of the relationship between hours  and accidents. However, the methodological and conceptual limitations  we have identified call the estimates’ precision into question. As a result,  these estimates may not be sufficient for drawing conclusions about small  changes in accident rates over time—including FAA’s progress toward its  goal to reduce the fatal general aviation accident rate per 100,000 flight  hours by 10 percent over 10 years. Implementing alternative means of  collecting flight hour data, such as requiring the reporting of aircraft  engine-revolution or run-time data, could supplement or replace the data  generated through the survey and add rigor to FAA’s flight-hour  estimates. Moreover, more precise flight-hour data could allow FAA to  better target its safety efforts at subpopulations within the general aviation  community. This could include reviewing an industry segment’s  characteristics, such as the number of fatal accidents relative to its  portion of estimated flight hours and setting a measurable goal for  improving safety within that segment. Though FAA has attempted to  address the disproportionate number of fatalities within the E-AB  community by developing an advisory circular to encourage transition  training for pilots, it has not set a specific goal for reducing fatal accidents  in that segment.
FAA and NTSB, to their credit, have recognized that flight-hour estimates  derived from the general aviation survey are imperfect. FAA has  discussed ways to improve its flight-hour data, including requiring general  aviation owners to report flight hours (in the form of engine-revolution or  run-time data) directly to FAA during aircraft registration renewals or at  the annual aircraft maintenance check. However, collecting data from  these alternative sources has not progressed beyond internal  discussions. In addition, organizations representing pilots have generally  been opposed to suggestions for increased data collection, which they  view as potential impediments to flying. According to these groups,  general aviation pilots typically would prefer to avoid additional regulation  or federal involvement.
In 2005, NTSB explored using alternative approaches to determining  annual general aviation activity, approaches that involved using other  measures as proxies for hours flown—including the number of active  pilots and fuel consumption. However, there are shortcomings to each of  these options. As discussed previously, active pilots are defined as those  who have current medical certifications; this is not related to whether the  pilot actually flew in a given year. And while aviation gas consumption  could be a proxy measure for piston engine aircraft activity, some piston- engine aircraft are used for operations other than general aviation.  Further, jet fuel consumption cannot reasonably be used as a proxy for  the general aviation activity of turbine engine aircraft because of the many  types of operations (e.g., air taxi, air ambulance, etc.) flown by these  aircraft.
section_title: FAA’s Singular Goal to Reduce the Fatal Accident Rate May Mask Problems in Certain Segments of General Aviation
paragraphs: In 2008, FAA set a goal to reduce the fatal general aviation accident rate  by 10 percent—from a baseline of 1.12 fatal accidents per 100,000 flight  hours to 1 fatal accident per 100,000 flight hours—over 10 years, from  2009 to 2018. This single long-term safety goal may mask problems in  certain segments of the community. The goal stemmed from FAA’s desire  to have a target for its general aviation safety improvement efforts that  accounted for changes in flight activity over time. According to FAA  officials, they were looking for a goal that was achievable and  represented an improved level of safety. FAA did not meet the annual  targets for the goal in 2009 and 2010 and, according to projections of  flight activity, it does not appear FAA will meet its target in 2011.
This singular goal is applied to an industry that is diverse in aircraft types  and operations—some of which experience accidents at a higher rate  than others. General aviation airplanes differ significantly in size and  performance, ranging from single-seat E-AB airplanes to large corporate  jets. The types of flying and pilot experience also vary by segment. Some  private pilots may only fly a few times each year, while some corporate  pilots may keep a schedule similar to that of a commercial airline pilot. In  addition, given the expense of flying and maintaining an airplane,  downturns in the economy can decrease activity in some segments of  general aviation. Changes in flight activity in certain segments of the  industry could mask or minimize problems in others and contribute to a  rate that does not accurately reflect the trends in the individual segments.  (See fig. 4.) For instance, total general aviation flight hours have  decreased since the most recent recession, but some segments have  declined at a faster rate than others. Personal flying hours in 2010 were 4  percent lower than they were in 2008; corporate flying hours, by  comparison, were almost 15 percent lower in 2010 than in 2008.  Historically, corporate flying has been one of the safest types of general  aviation operations. From 1999 through 2010, corporate airplane  operations accounted for just 1 percent of fatal general aviation accidents  but 14 percent of flight hours. And from 2008 through 2011, there were no  fatal accidents involving corporate airplane operations. As a result,  changes in corporate flight activity could result in changes in the overall  fatal accident rate that are not necessarily a reflection of changes in  safety but rather a reflection of the changing composition of general  aviation flight activity. In addition, as previously discussed, the rate is  based on estimates of annual general aviation flight hours that may not  be reliable.
There has been some discussion within FAA and industry about  implementing separate goals for each segment of general aviation.  According to one stakeholder we interviewed, the types of operations— even among fixed-wing aircraft—differ enough to warrant such a  disaggregation. He explained that an hour flown during a corporate  operation, during which an advanced aircraft flies from point to point with  a significant portion of the time spent en route, is quite different from a  pilot flying for pleasure and practicing maneuvers and take-offs and  landings—the phase of flight when most accidents occur. However, other  stakeholders we interviewed maintained that they all fly under the same  operating rules, so it is proper to consider the safety of general aviation  as a whole. Given the significant dissimilarities among the various general  aviation sectors, along with the varied accident and fatality rates, setting  separate safety improvement goals would allow FAA to take a more risk- based approach and target its resources and safety improvement efforts  to the unique characteristics of and risks posed by each sector.
section_title: FAA Has Key Initiatives Under Way to Improve General Aviation Safety, but One Has Several Shortcomings
paragraphs: FAA has embarked on key initiatives to achieve its goal of a 10-percent  reduction in the fatal general aviation accident rate per 100,000 flight  hours by 2018. One is the long-standing General Aviation Joint Steering  Committee (GAJSC), which is led by the Office of Accident Investigation  and Prevention. More recently, FAA announced a 5-year strategy to  improve general aviation safety that was developed by the General  Aviation and Commercial Division of the Flight Standards Service.  Although both initiatives work toward the overall goal of reducing general  aviation fatalities, the GAJSC is using a data-driven approach to identify  risks in general aviation operations and propose mitigations, while the 5- year strategy is composed of a wide variety of activities under four focus  areas.
subsections: section_title: FAA Renewed the GAJSC in Early 2011
paragraphs: In January 2011, FAA renewed the GAJSC, a joint FAA effort with the  general aviation industry, the National Aeronautics and Space  Administration (NASA), and NTSB that in 1998 was part of the Safer  Skies Initiative. Utilizing the model of the Commercial Aviation Safety  Team (CAST), the GAJSC’s goal is to focus limited government and  industry resources on data-driven risk reductions and solutions to general  aviation safety issues. The GAJSC consists of a steering committee that  provides, among other things, strategic guidance and membership  outreach. It also consists of a safety analysis team (SAT), which  determines future areas of study and charters safety studies, among  other things. GAJSC officials indicated that they would charter working  groups as issues for study were identified.
The first working group of the renewed GAJSC focused on loss of control  in approach and landing accidents. This area was selected because,  according to analyses of NTSB accident data for fatal airplane accidents  that occurred from 2001 through 2011and for which NTSB had completed  its investigation, loss of control was the number one causal factor. The  working group divided into three subgroups—reciprocating non-E-AB  aircraft, turbine engine aircraft, and E-AB aircraft—and agreed upon a  sample of 30 accidents to be analyzed by each. Despite issues such as  a lack of data and the consistency of member participation, the working  group developed 83 intervention strategies. These strategies were used  to develop the 27 safety enhancements that were presented to the  GAJSC for approval. The GAJSC approved 23 of the safety  enhancements. The next steps will include developing detailed  implementation plans for each of the strategies, with the SAT conducting  resource/benefit evaluations of each plan. The SAT then will determine  which are the most effective solutions, draft a master strategic plan, and  submit the plan to the GAJSC for approval. Implementation is expected to  begin upon approval. During implementation, the SAT will be responsible  for tracking implementation schedules and levels, tracking the  effectiveness of the intervention strategies, and recommending areas for  future study. We believe that with the GAJSC’s renewal and adoption of  CAST-like methods, it has the potential to contribute to a reduction in  general aviation accidents and fatalities over the long term.
section_title: FAA Launched a 5-Year Strategy in 2011 to Help Reduce the Fatal General Aviation Accident Rate
paragraphs: In March 2011, FAA announced its 5-year strategy to improve general  aviation safety. This initiative is a complementary effort to the work of the  GAJSC. FAA described the strategy as a nonregulatory approach  conducted in partnership with the general aviation community and  coordinated across FAA lines of business. The strategy has four focus  areas—(1) risk management, (2) safety promotion, (3) outreach and  engagement, and (4) training—and includes a 2-year review and the  development of validation metrics as each phase of the plan is  implemented.
FAA initially planned to concentrate its risk management efforts in three  areas: (1) the top 10 causes and contributing factors in fatal general  aviation accidents—initiated in coordination with the GAJSC, (2) E-AB  aircraft, and (3) agricultural operations, which comprise one segment of  the general aviation sector. To begin this effort, an FAA team identified  the top ten causes of fatal general aviation accidents as well as the  leading contributing factors, and provided the information to the GAJSC.  The GAJSC, as previously discussed, is using the results of the data  analysis to focus its efforts on loss-of-control accidents during approach  and landing.
For the safety promotion aspect of its 5-year strategy, FAA relies on the  FAA Safety Team (FAASTeam). Created in September 2004 as the    the FAASTeam consists of 154 education and outreach arm of FAA, FAA employees in eight regional field offices, along with 32 groups and  2,500 individual members from the general aviation industry. In 2011,  FAA refocused the FAASTeam—from national and international  activities—to promote general aviation safety and technical proficiency  through a host of nationwide seminars and contact with pilots at airports.  A significant part of the FAASTeam’s new focus is the annual FAA safety  standdown—a series of nationwide meetings that highlight issues of  concern for general aviation and include industry and GAJSC member  participation. The 2012 standdown focused on loss of control, the focus of  a GAJSC working group, from three different perspectives: (1) preflight  mistakes, (2) aeronautical decision making, and (3) handling a loss of  control. In addition, the FAASTeam is conducting workshops for certified  flight instructors to increase the quality of training offered to general  aviation pilots. The FAASTeam has also been examining intervention  strategies by working directly with designated pilot examiners to promote  its educational opportunities to all applicants for practical tests.
In its outreach and engagement efforts for the 5-year strategy, FAA has  briefed aviation associations, type clubs, and flight instructors, and, with  the assistance of the Aviation Accreditation Board International, held a  symposium on flight training with academia in July 2011. FAA has also  reached out to major aviation insurance providers. As a result of these  and other efforts, FAA reports that it has strengthened its links with  aviation associations while also improving its outreach efforts to type  clubs.
The training portion of FAA’s 5-year strategy includes chartering an  aviation rulemaking committeeon pilot testing standards and training,  expanding its focus on certified flight instructors, and revamping the  WINGS pilot proficiency program. In September 2011, FAA announced  the establishment of an aviation rulemaking committee to address  concerns from AOPA, SAFE, and others about the testing and training  standards for pilots.flight instructor, private pilot, instrument rating, and commercial pilot  certificates. It made nine recommendations to FAA to enhance the pilot- testing and pilot-training processes. The recommendations included  establishing a stakeholder body to assist in the development of  knowledge test questions and handbook content as well as transitioning  to a single testing standard document for the knowledge test. FAA  concurred with most of the rulemaking committee’s recommendations.
The rulemaking committee focused on the certified  To increase its focus on certified flight instructors, FAA is reviewing  certified flight instructor recurrent training and renewal requirements. FAA  also updated the advisory circular on flight instructor courses and  published it in September 2011.
The FAASTeam’s voluntary WINGS pilot proficiency program is being  revamped to encourage more participation. An FAA-established industry group has been surveying pilots to determine what changes need to be  made to the WINGS program. Once the survey is completed, the resulting  data will be analyzed and recommendations for changes will be made by  the end of fiscal year 2012. FAA officials anticipate implementing changes  to the program as funding becomes available in fiscal year 2013.
section_title: The 5-Year Strategy Has Significant Shortcomings
paragraphs: FAA’s 5-year strategy to improve general aviation safety suffers from  several shortcomings that hinder its potential for success. First, senior  FAA officials acknowledged that there are no specific performance goals  or measures for the activities under the 5-year strategy. The officials  said that because the goal of the initiative, as a whole, is to change  general aviation culture, the strategy’s success will be measured through  changes in the general aviation fatal accident rate. They also indicated  that they are developing validation metrics as each phase of the plan is  implemented. However, successful results-oriented organizations  measure their performance at each organizational level by developing  performance measures. Without performance goals or measures for the  individual initiatives implemented under the 5-year strategy, FAA will not  be able to evaluate the success or failure of those activities, regardless of  whether the fatal accident rate is reduced. Further, FAA has yet to meet  its annual target for the general aviation fatal accident rate goal and may  not meet the overall goal by 2018. Therefore, it is even more crucial that  FAA determine whether these activities have been successful.
Second, the strategy was developed without the initial input of significant  stakeholders—the GAJSC and the general aviation industry. Successful  agencies we have studied based their strategic planning, to a large  extent, on the interests and expectations of their stakeholders, and  stakeholder involvement is important to ensure that agencies’ efforts and  resources are targeted at the highest priorities. According to officials  from the GAJSC and the general aviation industry groups we contacted,  although they were briefed on the strategy, they were not consulted in its  development and were surprised by the announcement of the strategy.  General aviation industry trade groups, type clubs, and other  organizations are active in promoting a safety culture and continuous  education among their members. For example, AOPA offers numerous  seminars each year to educate the pilot community, and EAA offers  advisory programs for experimental aircraft builders and pilots. Further,  many initiatives are joint efforts of FAA and the industry. Involving  stakeholders in strategic planning efforts can help create a basic  understanding among the stakeholders of the competing demands that  confront most agencies, the limited resources available to them, and how  those demands and resources require careful and continuous balancing.  FAA officials have indicated that their initial publication of the strategy  served as a “straw man” for obtaining industry’s input and that there has  been industry acceptance of the strategy as demonstrated by various  industry groups’ development of plans and programs supporting the  strategy. However, a lack of industry input into the development and  announcement of the strategy jeopardizes its prospects for acceptance  and success. This may be indicated in the current perspective of two  industry groups—which is that the best use of industry resources to  improve general aviation safety is through the work of the GAJSC.
Third, the FAASTeam, which will be the main vehicle for promoting the 5- year strategy to the industry, lacks the confidence of two significant  general aviation industry stakeholders we interviewed, and its  reorganization has not been completed. These industry stakeholders  indicated that there is inconsistency in the focus of the FAASTeam. One  stakeholder noted that industry “struggles to understand the role of the  FAASTeam,” and the other stated that the FAASTeam is “well  intentioned, but unfocused.” In addition, FAA initially planned to  reorganize the FAASTeam to reduce the number of volunteers to a strong  core group and to include a national FAASTeam located in Washington,  D.C. However, a senior FAA official recently indicated that the  restructuring of the FAASTeam is in flux and that the plan to reduce the  number of volunteers to a strong core group does not begin until 2013.  We believe that until there is a strong performance management  structure, input and buy-in from industry, and a respected and organized  FAASTeam, the effectiveness of the 5-year strategy will be in jeopardy.
section_title: FAA Has Other Initiatives Under Way That Could Also Contribute to Improved General Aviation Safety
paragraphs: Formed a rulemaking committee to recommend revisions to the small  airplane airworthiness standards: In August 2011, FAA chartered a  rulemaking committee to reorganize part 23—which promulgates  airworthiness standards for small airplanes—according to airplane  performance and complexity criteria as opposed to the traditional  criteria of airplane weight and propulsion. The goals of this rulemaking  committee include increasing safety and decreasing certification  costs. Co-chaired by the manager of FAA’s Small Airplane  Directorate, the rulemaking committee includes members  representing other sections of the Aircraft Certification and Flight  Standards Services as well as members from industry groups,  manufacturers, and foreign aviation authorities. The committee is  expected to complete its work by the summer of 2013.
Encouraging adoption of a safety management system (SMS): In  guidance issued in April 2011, FAA encouraged general aviation  business and corporate operators to develop and implement SMS.
Visual flight rules govern the procedures for conducting flight under visual conditions, as  opposed to instrument flight rules, which govern the procedures for conducting flights  using instruments. year 2007, the Weather Camera Program has funded the  procurement and installation of 182 weather camera sites in Alaska.  The cameras provide near real time video images of sky conditions at  airports, mountain passes, and strategic VFR locations, such as high- use air routes, to enhance pilots’ situational awareness. According to  FAA, this new capability is providing measurable reductions in  weather-related VFR accidents in Alaska. FAA’s goal is to install a  total of 221 weather camera sites.
section_title: Technology and Equipment May Also Help Improve General Aviation Safety
paragraphs: According to FAA, new technologies such as inflatable restraints (air  bags), ballistic parachutes, weather in the cockpit, angle-of-attack  indicators, and terrain avoidance equipment could significantly reduce  general aviation fatalities. Angle of attack indicators and inflatable  restraints have the greatest likelihood of significantly improving safety.  Angle-of-attack indicators provide the pilot with a visual aid to prevent  loss of control of the aircraft. Previously, cost and complexity of indicators  limited their use to the military and commercial aircraft. FAA has  streamlined the approval of angle-of-attack indicators for general aviation  aircraft and is working to promote the retrofit of the existing fleet. FAA is  also streamlining the certification and installation of inflatable restraints  with the goal of making all general aviation aircraft eligible for  installation. Further, FAA is working with manufacturers to define  equipage requirements and support the Next Generation Air  Transportation System (NextGen)—a new satellite-based air traffic  management system that by 2025 will replace the current radar-based  system—by streamlining the certification and installation of NextGen  technologies. Some industry experts told us, however, that there might  not be future opportunities to significantly improve general aviation safety  with the aid of technology since most accidents are still attributed to pilot  error.
section_title: Conclusions
paragraphs: To further reduce the number of fatal general aviation accidents, FAA  needs to effectively target its accident mitigations, as it is attempting to do  through the GAJSC. The agency’s ability to do so, however, is limited by  a lack of pilot data. For instance, FAA does not maintain certain key  information about general aviation pilots, including how many are actively  flying each year and whether they participate in recurrent training other  than FAA’s own WINGS program. Without this information, FAA cannot  determine the potential effect of the various sources and types of training  on pilot behavior, competency, and the likelihood of an accident. The lack  of pilot data also makes it difficult to identify the root causes of accidents  attributed to pilot error and determine appropriate risk mitigation  opportunities.
The annual survey FAA uses for collecting general aviation flight-activity  data suffers from significant limitations—limitations that call into question  the resulting activity estimates FAA produces as well as the accident  rates calculated by NTSB. Though FAA has improved the survey over the  years, our concerns remain because the survey continues to experience  response rates below 50 percent and relies on the record-keeping habits  and memories of survey respondents who sometimes have to recall  details that occurred more than 12 months earlier. Further, other methods  for obtaining general aviation flight-activity data have encountered  resistance from the industry. Without a more accurate reporting of general  aviation flight activity, such as requiring the reporting of flight hours at  certain intervals—e.g., during registration renewals or annual  maintenance inspections—FAA lacks assurance that it is basing its policy  decisions on a true measure of general aviation trends, and NTSB lacks  assurance that its calculations of accident and fatality rates accurately  represent the state of general aviation safety.
Given the diversity of the general aviation community—illustrated, for  example by the wide variety of aircraft in the fleet and the varying nonfatal  and fatal accident rates among the general aviation segments, the  adoption of a singular agency goal–-a 10 percent reduction in the general  aviation fatal accident rate per 100,000 flight hours by 2018 is not the  most effective risk-based tool for achieving general aviation safety gains.  The goal does not take into account the variety of general aviation  operations or the risks associated with each. For example, one hour flown  during a personal operation is not the same as one hour flown during a  corporate operation. Also, economic conditions affect each segment  differently, making it difficult to discern if a change in the accident rate is  an indication of a change in the safety of the industry. If the goal is  reached, the overall success might mask ongoing safety issues in one or  more segments of the community.
FAA officials have indicated that the success of the 5-year strategy— which is composed of numerous initiatives—will be measured through  changes in the general aviation fatal accident rate. However, successful  results-oriented organizations measure their performance at each  organizational level by developing performance measures. For this  reason, we think it is important for FAA to develop performance measures  for the significant initiatives underlying the 5-year strategy. This is  important because if FAA does not measure the performance of the  significant underlying initiatives, it will not be able to determine whether  the initiatives were effective in their own right. In addition, in order for the  FAASTeam to be successful in its promotion of the 5-year strategy, it  must be well respected within the general aviation community. We are not  making a recommendation regarding the FAASTeam at this time since  plans for restructuring it are in flux and its volunteer force realignment is  not scheduled to begin until 2013.
section_title: Recommendations for Executive Action
paragraphs: To enhance FAA’s efforts to improve general aviation safety, we  recommend that the Secretary of Transportation direct the FAA  Administrator to take the following four actions:
To expand the data available for root cause analyses of general  aviation accidents and other purposes, collect and maintain data on  each certificated pilot’s recurrent training, and update the data at  regular intervals.
Improve measures of general aviation activity by requiring the  collection of the number of hours that general aviation aircraft fly over  a period of time (flight hours). FAA should explore ways to do this that  minimize the impact on the general aviation community, such as by  collecting the data at regular events (e.g., during registration renewals  or at annual maintenance inspections) that are already required.
To ensure that ongoing safety issues are addressed, set specific  general aviation safety improvement goals—such as targets for fatal  accident reductions—for individual industry segments using a data- driven, risk management approach.
To determine whether the programs and activities underlying the 5- year strategy are successful and if additional actions are needed,  develop performance measures for each significant program and  activity underlying the 5-year strategy.
section_title: Agency Comments
paragraphs: We provided the Department of Transportation (DOT) with a draft of this  report for review and comment. DOT officials agreed to consider our  recommendations and provided technical comments, which we  incorporated as appropriate.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Transportation, the Chairman of NTSB, and  interested parties. In addition, this report is available at no charge on the  GAO Web site at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. Key contributors to this report are  listed in appendix II.
section_title: Appendix I: Scope and Methodology
paragraphs: Our objective was to conduct a comprehensive review of general aviation  safety. To do so, we addressed the following questions: (1) what are the  characteristics and trends in general aviation accidents from 1999 to 2011  and (2) what actions have been taken by the Federal Aviation  Administration (FAA) to improve general aviation safety?
To identify the characteristics of and trends in general aviation accidents,  we conducted a data analysis using the National Transportation Safety  Board’s (NTSB) Aviation Accident Database. We limited our analysis to  accidents involving airplanes operating under Part 91 of the Federal  Aviation Regulations that occurred from January 1, 1999, through  December 31, 2011, in the U.S. We excluded accidents that occurred in  U.S. territories, possessions, and international waters. To assess the  reliability of the NTSB data, we reviewed documentation on data  collection efforts and quality assurance processes, talked to  knowledgeable NTSB officials about the data, and checked the data for  completeness and reasonableness. We determined that these data were  sufficiently reliable for the descriptive and comparative analyses used in  this report. To supplement our analysis of the NTSB accident data, we  also analyzed FAA’s general aviation flight-hour estimates for 1999  through 2010 and estimated active pilot data for 2011. To assess the  reliability of these data, we reviewed documentation on data collection  efforts and quality assurance processes and talked to knowledgeable  FAA officials. In assessing the reliability of the flight-hour estimates, we  also spoke with the contractors responsible for executing the survey that  yielded these estimates, the General Aviation and Part 135 Survey. We  determined that the flight-hour data and the active pilot data were  sufficiently reliable for the purposes of this engagement. Specifically,  these data elements were sufficiently reliable to provide meaningful  context for the numbers and characteristics of accidents that we report.  However, we also determined that because of the methodological  limitations identified—a low response rate and the potential for  nonresponse bias—the flight-hour estimates developed from the General  Aviation and Part 135 Survey may not have the precision necessary to  measure small changes in the general aviation accident rate over time.
To identify actions FAA and others have taken to improve general  aviation safety, we reviewed our prior reports as well as documents and  reports from FAA, NTSB, NASA, and general aviation industry trade and  other groups, including the Aircraft Owners and Pilots Association  (AOPA), the Experimental Aircraft Association (EAA), and the Society of  Aviation and Flight Educators (SAFE); FAA orders, notices, advisory  circulars; and applicable laws and regulations. We also determined the  roles and responsibilities of FAA and NTSB in collecting and reporting  general aviation safety data. In addition to interviewing officials from the  various FAA offices and divisions responsible for general aviation safety,  we interviewed aviation experts affiliated with various aviation industry  organizations. (See table 4.)
To obtain additional insight into the general aviation industry, we attended  the September 2011 AOPA Aviation Summit in Hartford, Connecticut; the  March 2012 Annual FAA Aviation Forecast Conference in Washington,  D.C.; the February 2012 Northwest Aviation Conference in Puyallup,  Washington; and the June 2012 NTSB General Aviation Forum in  Washington, D.C.
section_title: Appendix II: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, the following individuals made  important contributions to this report: H. Brandon Haller, Assistant  Director; Pamela Vines; Jessica Wintfeld; Russ Burnett; Bert Japikse;  Delwen Jones; Josh Ormond; and Jeff Tessin.