Aviation Safety: FAA Efforts Have Improved Safety, but Challenges Remain in Key Areas
Published: 2013-04-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Even with nearly 80,000 flights each day within the national airspace system, there has not been a fatal commercial aviation accident in more than 4 years. The U.S. airspace system is arguably one of the safest in the world, with key aviation stakeholdersthe FAA, airlines, airports, aircraft manufacturers, and the National Transportation Safety Board (NTSB)working together to ensure these results.
As the federal agency responsible for regulating the safety of civil aviation in the United States, FAA is responsible for, among other things: setting aircraft certification standards, collecting fleet and flight activity data, conducting safety oversight of pilot training and general aviation operations, and safely integrating aircraft into the national airspace. As the aviation industry evolves, FAA must remain diligent in its efforts to ensure the continued safety of aviation. In 2010, Congress passed the Airline Safety and Federal Aviation Administration Extension Act, which, in part, called for FAA to better manage safety risks.
This testimony focuses on (1) FAAs aircraft certification process and (2) FAAs use of data to enhance safety and improve aviation oversight. The testimony is based on GAOs previous work and updated with industry reports and information provided by FAA officials.
GAO has previously recommended that FAA address several data quality weaknesses. FAA concurred with most of these recommendations and has taken steps toward addressing some.
section_title: What GAO Found
paragraphs: The Federal Aviation Administration (FAA) is responsible for approving the design and airworthiness of new aircraft and equipment before they are introduced into service. FAA approves changes to aircraft and equipment based on evaluation of industry submissions against standards set forth in federal aviation regulations and related guidance documents. In September 2011, we reported that, overall, FAA did a good job following its certification processes in assessing the composite fuselage and wings of Boeing's 787 against its airworthiness standards. However, the approval process--referred to as certification--presents challenges for FAA in terms of resources and maintaining up-to-date knowledge of industry practices, two issues that may hinder FAA's efforts to conduct certifications in an efficient and timely manner. FAA is currently assessing its certification process and identifying opportunities to streamline it.
FAA plans to continue analyzing data reactively to understand the causes of accidents and incidents, and to augment this approach through implementation of a safety management system (SMS). SMS is a proactive approach that includes continually monitoring all aspects of aviation operations and collecting and analyzing appropriate data to identify emerging safety problems before they result in death, injury, or significant property damage. FAA has put in place various quality controls for its data; however, GAO has identified a number of areas where FAA does not have comprehensive risk-based data or methods of reporting that capture all incidents. The following are among the key areas GAO identified as needing improved data collection and analysis.
Runway and ramp safety . Additional information about surface incidents could help improve safety in the airport terminal area, as data collection is currently limited to certain types of incidents, notably runway incursions, which involve the incorrect presence of an aircraft, vehicle, or person on a runway and certain airborne incidents, and does not include runway overruns, which occur when an aircraft veers off a runway or incidents in ramp areas, which can involve aircraft and airport vehicles.
Airborne operational errors . FAA's metric for airborne losses of separation--a type of operational error--is too narrow to account for all potential risk.
General aviation . FAA estimates of annual flight hours for the general aviation sector, which includes all forms of aviation except commercial and military, may not be reliable.
Pilot training . FAA does not have a comprehensive system in place to measure its performance in meeting its annual pilot school inspection requirements.
FAA has taken steps to address safety oversight issues and data challenges in many of these areas. For example, FAA is planning to develop a program to collect and analyze data on runway overruns, but it will be several years before FAA has obtained enough information about these incidents to assess risks. Sustained attention to these data collection and analysis issues will be necessary to ensure that FAA can more comprehensively and accurately assess and manage risk.

section_title: Letter
paragraphs: I appreciate the opportunity to testify today on the Federal Aviation  Administration’s (FAA) efforts to oversee aviation safety. Even with nearly  80,000 flights each day within the national airspace system (NAS), there  has not been a fatal commercial aviation accident in more than 4 years,  and although hundreds of fatalities continue to occur each year in general  aviation, the number of overall general aviation accidents has trended  downward. The U.S. airspace system is arguably one of the safest in the  world, with key aviation stakeholders—FAA, the airlines and other aircraft  operators, airports, aircraft manufacturers, and the National  Transportation Safety Board (NTSB)—working together to achieve these  results. Nevertheless, we must not become complacent because of the  extraordinary safety record that has been achieved to date. Congress,  FAA, and other stakeholders must remain diligent in their oversight of  aviation safety.
Pub. L. No. 111-216, 124 Stat. 2348 (2010). management systems (SMS). This risk-based oversight approach is  becoming the standard throughout the global aviation industry and is  recognized by aviation leaders, such as the International Civil Aviation  Organization (ICAO), as the next step in the evolution of safety. In  addition to SMS, FAA’s certification process attempts to ensure that  safety is built into the aircraft and equipment used in the NAS.agency has taken steps to improve its oversight approach, challenges  remain in key areas.
My statement today highlights two areas that are important to FAA’s  safety efforts: the certification process and the collection and analysis of  risk-based data as part of SMS. This statement is drawn from a body of  work that we have completed from June 2009 to October 2012 regarding  FAA’s safety oversight efforts. We have updated this information through  a review of FAA documents and interviews with FAA officials. A list of  related GAO products is included at the end of this statement, along with  footnoted references to these products throughout the statement. The  reports and testimonies cited in this statement contain more detailed  explanations of the methods used to conduct our work. This body of work  was conducted in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Certification is a Key Component of FAA’s Aviation Safety Oversight
paragraphs: Among its responsibilities for aviation safety, FAA issues certificates that  approve the design and production of new aircraft and equipment before  they are introduced into service; these certificates demonstrate that the  aircraft and equipment meet FAA’s airworthiness requirements. FAA also  grants approvals for such things as changes to air operations and  equipment. Certificates indicate that the aircraft, equipment, and new air  operators are safe for use or flight in the NAS. While industry  stakeholders have expressed concerns about variation in FAA’s  interpretation of standards for certification and approval decisions,  stakeholders and experts that we interviewed for our 2010 report  indicated that serious problems occur infrequently. In addition, in  September 2011 we reported that FAA did a good job following its  certification processes in assessing the composite fuselage and wings of  Boeing’s 787 against its airworthiness standards.
The certification process also provides an example of how FAA is  attempting to use a more proactive approach in finding solutions to a  potential problem. In the case of flammability regulations that govern  transport type aircraft, FAA has primarily developed its regulations on a  reactive basis. That is, as accidents and incidents have occurred, their  causes have been investigated, and the findings used to develop  regulations designed to prevent the future occurrence of similar incidents  or accidents. To supplement this oversight method, FAA has proposed a  new, threat-based approach for flammability regulations that will base the  flammability performance for different parts of the aircraft upon realistic  threats that could occur in-flight or in a post-crash environment.
FAA recognizes the value of certification as a safety tool, however the  agency faces some significant challenges, including resources and  maintaining up-to-date knowledge of industry changes. According to a  report from the Aircraft Certification Process Review and Reform Aviation  Rulemaking Committee, these certification challenges will become  increasingly difficult to overcome, as industry activity is expected to  continue growing and government spending for certification resources  remains relatively flat. As one means of responding to its certification  workload, FAA relies on designees, however, our prior work has shown  that there are concerns that designee oversight is lacking, particularly  with the new organizational designation authorities in which companies  rather than individuals are granted designee status. There are also  concerns that, when faced with certification of new aircraft or equipment,  FAA staff have not been able to keep pace with industry changes and,  thus, may struggle to understand the aircraft or equipment they are  tasked with certificating. SMS implementation within FAA should reduce  certification delays and increase available resources to facilitate the  introduction of advanced technologies. In response to a provision in the  2012 FAA Reauthorization, FAA is assessing the certification process and  identifying opportunities to streamline the process.
section_title: Better Quality and More Complete Data Could Help FAA Further Improve Safety Oversight
paragraphs: As we stated above, FAA plans to continue using data reactively to  understand the causes of accidents and incidents, and is implementing a  proactive approach—called an SMS approach—in which it analyzes data  to identify and mitigate risks before they result in accidents. FAA is also  overseeing SMS implementation throughout the aviation industry. Safety  management systems are intended to continually monitor all aspects of  aviation operations and collect appropriate data to identify emerging  safety problems before they result in death, injury, or significant property  damage. Under SMS, which FAA began implementing in 2005, the  agency will analyze the aviation safety data it collects to identify  conditions that could lead to aviation accidents or incidents and to  address such conditions through changes to FAA’s organization,  processes, management, and culture. As we reported in September  2012, according to FAA, the overarching goal of SMS is to improve safety  by helping ensure that the outcomes of any management or system  activity incorporate informed, risk-based decision making. FAA’s business  lines, such as the Air Traffic Organization and the Aviation Safety  Organization, are currently at different stages of SMS implementation and  it is likely that full SMS implementation will take many more years.
SMS relies heavily on data analysis and, while FAA has put in place  various data quality controls, it continues to experience data challenges  including limitations with some of its analyses and limitations to or the  absence of data in some areas. Data limitations and the lack of data  may inhibit FAA’s ability to manage safety risks. For example, we found  that some FAA data used in risk assessments may not be complete,  meaningful, or available to decision makers. We have also reported that  the agency currently does not have comprehensive risk-based data,  sophisticated databases to perform queries and model data, methods of  reporting that capture all incidents, or a level of coordination that  facilitates the comparison of incidents across data systems. Furthermore,  technologies aimed at improving reporting have not been fully  implemented. As a result, aviation officials managing risk using SMS  have limited access to robust FAA incident data. Implementing systems  and processes that capture accurate and complete data are critical for  FAA to determine the magnitude of safety issues, assess their potential  impacts, identify their root causes, and effectively address and mitigate  them.
Our recent work on aviation safety and FAA oversight issues has  identified a number of specific areas where FAA’s risk-based oversight  could be improved through improved data collection and analysis,  including: runway and ramp safety, airborne operational errors, general  aviation, pilot training, unmanned aircraft systems, and commercial  space. FAA has taken steps to address safety oversight issues in many of  these areas, including making changes to or committing to make changes  to its data collection practices in response to our recommendations in  most of these areas. Nonetheless, sustained FAA attention will be  necessary to ensure that the agency’s ability to comprehensively and  accurately assess and manage risk is not impaired.
Runway and ramp safety. Takeoffs, landings, and movement around  the surface areas of airports (the terminal area) are critical to the safe  and efficient movement of air traffic. In a June 2011 incident at John  F. Kennedy International Airport in New York, for example, a jumbo jet  carrying 286 passengers and crew almost collided with another jumbo  jet, which reportedly missed a turn and failed to stop where it should  have to avoid the occupied runway. Safety in the terminal area could  be improved by additional information about surface incidents, which  is currently limited to certain types of incidents, notably runway  incursions and certain airborne incidents, but does not include runway  overruns or incidents in ramp areas. Without a process to track and  assess these overruns or ramp area incidents, FAA cannot assess  trends in those areas and the risks posed to aircraft or passengers in  the terminal area. FAA is planning to develop a program to collect and  analyze data on runway overruns, something we recommended in  2011, but it will be several years before FAA has obtained sufficient  information about these incidents to be able to assess risks.still collects no comprehensive data on ramp area incidents and NTSB  does not routinely collect data on ramp accidents unless they result in  serious injury or substantial aircraft damage. In 2011, we  recommended that FAA extend its oversight to ramp safety and FAA  concurred.
Airborne operational errors. Operational errors –also referred to as  losses of separation—occur when two aircraft fly closer together than  safety standards permit due to an air traffic controller error. We  reported that FAA’s risk-based process for assessing airborne losses  of separation is too narrow to account for all potential risk and  changes in how errors are reported affect FAA’s ability to identify  trends. For example, FAA’s current process for analyzing losses of  separation assesses only those incidents that occur between two or  more radar-tracked aircraft. By excluding incidents such as those that  occur between aircraft and terrain or aircraft and protected airspace,  FAA is not considering the systemic risks that may be associated with  many other airborne incidents. FAA has stated that it is planning to  include these incidents in its risk assessment process before the end  of 2013, something we recommended in 2011.changes to reporting policies affect its ability to accurately determine  safety trends. For instance, we reported in October 2011 that the rate  and number of reported airborne operational errors in the terminal   In addition, FAA’s  area increased considerably since 2007.to reporting policies and processes in 2009 and 2010 make it difficult  to know the extent to which the recent increases in reported  operational errors are due to more accurate data, an actual increase  in the occurrence of incidents, or both.
General aviation. General aviation is characterized by a diverse fleet  of aircraft flown for a variety of purposes. In 2010, FAA estimated that  there were more than 220,000 aircraft in the active general aviation  fleet, comprising more than 90 percent of the U.S. civil aircraft fleet.  The number of nonfatal and fatal general aviation accidents  decreased from 1999 through 2011; however, more than 200 fatal  accidents occurred in each of those years. In October 2012, we  reported that general aviation flight activity data limitations impede  FAA’s ability to assess general aviation safety and thereby target risk  mitigation efforts.  For example, FAA estimates of annual general  aviation flight hours may not be reliable because of methodological  and conceptual limitations with the survey upon which flight activity  estimates are based. These limitations include survey response rates  below 50 percent. Without more comprehensive reporting of general  aviation flight activity, such as requiring the reporting of flight hours at  certain intervals, FAA lacks assurance that it is basing its policy  decisions on an accurate measure of general aviation trends, and  NTSB lacks assurance that its calculations of accident and fatality  rates accurately represent the state of general aviation safety.
Lack of comprehensive flight hour data is an issue we have also  identified in other segments of the aviation industry, including  helicopter emergency medical services (HEMS) and air cargo  transportation. We recommended in 2007 and 2009 respectively that  FAA take action to collect comprehensive and accurate data for  HEMS and general aviation operations. In 2011, we confirmed that  FAA now annually surveys all helicopter operators and requests,  among other things, information on the total flying hours and the  percentage of hours that were flown in air ambulance operations. Our  recommendations to FAA for air cargo and general aviation data  remain unaddressed.
See GAO, Initial Pilot Training: Better Management Controls Are Needed to Improve  FAA Oversight, GAO-12-117 (Washington, D.C.: November 4, 2011). data found that, while FAA requires its inspectors to conduct on-site  inspections of each of these schools and their pilot examiners at least  once per year, the agency does not have a comprehensive system in  place to adequately measure its performance in meeting its annual  inspection requirements. Without complete data on active pilot  schools and pilot examiners, it is difficult to ensure that regulatory  compliance and safety standards are being met. In addition, it is  unclear whether required inspections for pilot examiners were  completed because FAA’s data system lacks historical information.  One potential implication is the quality of training that recreational pilot  candidates receive, which could contribute to the many general  aviation accidents in which pilot error is cited as a contributing  factor. In 2011, we recommended that FAA develop a  comprehensive system to measure performance of pilot school  inspections and noted that this recommendation may require  modifying or improving existing data systems. In responding to our  recommendation, FAA officials said they agreed that improvements in  oversight data were needed and indicated that they believe efforts  already in existence or under way address our recommendations.
Unmanned aircraft systems (UAS). FAA and the National Aeronautics  and Space Administration (NASA) are taking steps to ensure the  reliability of both small and large UAS by working on certification  standards specific to UAS and undertaking research and development  efforts to mitigate obstacles to the safe and routine integration of UAS  into the national airspace. Some of these obstacles include  vulnerabilities in UAS operation that will require technical solutions.  However, we found that these research and development efforts  related to overcoming these obstacles cannot be completed and  validated without safety, reliability, and performance standards for  UAS operations, which FAA has not developed due to data  limitations. Standards for UAS operations are a key step in the  process of safely integrating regular UAS operations into the national  airspace. Once standards are developed, FAA has indicated that it  will begin to use them in UAS regulations; until then, UAS will  continue to operate as exceptions to the regulatory framework rather  than being governed by it.
Commercial space. FAA also oversees the safety of commercial  space launches that can carry cargo and eventually humans into  space. FAA is responsible for licensing and monitoring the safety of  such launches and of spaceports (sites for launching spacecraft).   However, FAA is prohibited by statute from regulating commercial  space crew and passenger safety before 2015 except in response to  a serious injury or fatality or an event that poses a high risk of causing  a serious injury or fatality. FAA has interpreted this limited authority  as allowing it to regulate crew safety in certain circumstances and has  been proactive in issuing a regulation concerning emergency training  for crews and passengers. However, FAA has not identified data that  would allow it to monitor the safety of the developing space tourism  sector and determine when to regulate human space flight. To allow  the agency to be proactive about safety, rather than responding only  after a fatality or serious incident occurs, we recommended in 2006  that FAA identify and continually monitor indicators of space tourism  industry safety that might trigger the need to regulate crew and  passenger safety before 2015 and use it to determine if the  regulations should be revised.working with its industry advisory group, the Commercial Space  Transportation Advisory Committee, to develop guidelines for human  spaceflight.
According to agency officials, FAA is  Chairman Rockefeller, Ranking Member Thune, and Members of the  Committee, this concludes my written statement. I would be pleased to  answer any questions that you may have at this time.
section_title: GAO Contact and Acknowledgments
paragraphs: For further information on this statement, please contact Gerald L.  Dillingham, Ph.D., at (202) 512-2834 or by email at dillinghamg@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this statement. GAO staff who  made key contributions to this testimony include Heather MacLeod,  Assistant Director; Brooke Leary; Paul Aussendorf; Russell Burnett;  Vashun Cole; Laura Erion; Brandon Haller; Dave Hooper; Dan Hoy;  Delwen Jones; Maureen Luna-Long; Teresa Spisak; Pam Vines; and  Jessica Wintfeld.
section_title: Related GAO Products
paragraphs: Unmanned Aircraft Systems: Continued Coordination, Operational Data,  and Performance Standards Needed to Guide Research and  Development. GAO-13-346T. Washington, D.C.: February 15, 2013.
General Aviation Safety: Additional FAA Efforts Could Help Identify and  Mitigate Safety Risks GAO-13-36. Washington, D.C.: Oct 4, 2012.
Unmanned Aircraft Systems: Measuring Progress and Addressing  Potential Privacy Concerns Would Facilitate Integration into the National  Airspace System. GAO-12-981. Washington, D.C.: September 14, 2012.
Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk  Management.GAO-12-898. Washington, D.C.: Sep 12, 2012.
Aviation Safety: FAA Is Taking Steps to Improve Data, but Challenges for  Managing Safety Risks Remain.GAO-12-660T. Washington, D.C.: Apr 25,  2012.
Aviation Safety: FAA Has An Opportunity to Enhance Safety and Improve  Oversight of Initial Pilot Training. GAO-12-537T. Washington, D.C.: Mar  20, 2012.
Initial Pilot Training: Better Management Controls Are Needed to Improve  FAA Oversight. GAO-12-117. Washington, D.C.: November 4, 2011.
Aviation Safety: Enhanced Oversight and Improved Availability of Risk-  Based Data Could Further Improve Safety. GAO-12-24. Washington,  D.C.: Oct 5, 2011.
Aviation Safety: Status of FAA’s Actions to Oversee the Safety of  Composite Airplanes. GAO-11-849. Washington, D.C.: Sep 21, 2011.
Aviation Safety: Certification and Approval Processes Are Generally  Viewed as Working Well, but Better Evaluative Information Needed to  Improve Efficiency. GAO-11-14. Washington, D.C.: October 7, 2010.
Air Ambulance: Effects of Industry Changes on Services Are Unclear.  GAO-10-907. Washington, D.C. September 30, 2010.
Aviation Safety: Improved Data Quality and Analysis Capabilities Are  Needed as FAA Plans a Risk-Based Approach to Safety Oversight.  GAO-10-414. Washington, D.C.: May 6, 2010.
Aviation Safety: Better Data and Targeted FAA Efforts Needed to Identify  and Address Safety Issues of Small Air Cargo Carriers. GAO-09-614.  Washington, D.C.: June 24, 2009.
Aviation Safety: Potential Strategies to Address Air Ambulance Safety  Concerns. GAO-09-627T. Washington, D.C.: April 22, 2009.
Aviation Safety: Improved Data Collection Needed for Effective Oversight  of Air Ambulance Industry. GAO-07-353. Washington, D.C.: February 21,  2007.
Commercial Space Launches: FAA Needs Continued Planning and  Monitoring to Oversee the Safety of the Emerging Space Tourism  Industry. GAO-07-16. Washington, D.C.: October 2006.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.