Aviation Safety: Enhanced Oversight and Improved Availability of Risk- Based Data Could Further Improve Safety
Published: 2011-10-05T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Takeoffs, landings, and movement around the surface areas of airports (the terminal area) are critical to the safe and efficient movement of air traffic. The nation's aviation system is arguably the safest in the world, but close calls involving aircraft or other vehicles at or near airports are common, occurring almost daily. The Federal Aviation Administration (FAA) provides oversight of the terminal area and has taken action to improve safety, but has been called upon by the National Transportation Safety Board (NTSB) and others to take additional steps to improve its oversight. As requested, this report addresses (1) recent actions FAA has taken to improve safety in the terminal area, (2) recent trends in terminal area safety and factors contributing to those trends, and (3) any additional actions FAA could take to improve safety in the terminal area. To address these issues, GAO analyzed data from FAA data; reviewed reports and FAA documents; and interviewed federal and industry officials.
section_title: What GAO Found
paragraphs: Since 2007, FAA has taken several steps to further improve safety at and around airports, including implementing procedural and technological changes to improve runway safety, proposing a rule that would require airports to establish risk-management plans that include the ramp areas where aircraft are serviced, collecting more data on safety incidents, and shifting toward risk-based analysis of airborne aviation safety information. Several of these initiatives are intended to better identify systemic issues in air traffic safety. Rates of reported safety incidents in the terminal area continue to increase. FAA met its interim goals toward reducing the total number of runway incursions--the unauthorized presence of an airplane, vehicle, or person on the runway--in 2009 and 2010, but the overall rate of incursions at towered airports has trended steadily upward. In fiscal year 2004, there were 11 incursions per million operations at these airports; by fiscal year 2010, the rate increased to 18 incursions per million operations. The rate and number of airborne operational errors--errors made by air traffic controllers--have increased considerably in recent years, with the rate nearly doubling from the second quarter of fiscal 2008 to the same period of 2011. FAA has not met its related performance goals. Comprehensive data are not available for some safety incidents, including runway overruns or incidents in ramp areas. Recent increases in reported runway incursions and airborne operational errors can be somewhat attributed to several changes in reporting policies and procedures at FAA; however, trends may also indicate an increase in the actual occurrence of incidents. Enhanced oversight and additional information about surface and airborne incidents could help improve safety in the terminal area. FAA oversight in the terminal area is currently limited to certain types of incidents, notably runway incursions and certain airborne incidents, and does not include runway overruns or incidents in ramp areas. In addition, the agency lacks data collection processes, risk-based metrics, and assessment frameworks for analyzing other safety incidents such as runway overruns, incidents in ramp areas, or a wider range of airborne errors. Further, changes to reporting processes and procedures make it difficult to assess safety trends, and existing data may not be readily available to decision makers, including those at the regional and local levels. As a result, FAA may have difficulty assessing recent trends in safety incidents, the risks posed to aircraft or passengers in the terminal area, and the impact of the agency's efforts to improve safety.
section_title: What GAO Recommends
paragraphs: GAO recommends that FAA (1) extend oversight of terminal area safety to include runway overruns and ramp areas, (2) develop risk-based measures for runway safety incidents, and (3) improve information sharing about incidents. The Department of Transportation agreed to consider the recommendations and provided clarifying information about efforts made to improve runway safety, which GAO incorporated.

section_title: Letter
paragraphs: The U.S. airspace system is one of the safest in the world, but mishaps  and close calls involving aircraft or other vehicles at airports or in the  airspace around airports are common, occurring almost daily. While few  of these incidents result in accidents or the loss of life, the risk of  catastrophic accidents in the terminal area remains, particularly during  takeoff and landing, which are considered the most critical phases of flight  given the higher level of risk associated with them. In a June 2011  incident at John F. Kennedy International Airport in New York, for  example, a jumbo jet carrying 286 passengers and crew almost collided  with another jumbo jet, which reportedly missed a turn and failed to stop  where it should have to avoid the occupied runway. The Federal Aviation  Administration (FAA) focuses its terminal area safety oversight on  runways and taxiways and on aircraft in the air near airports. By  contrast, safety oversight of operations in ramp areas, which include  areas of airports where aircraft are prepared for arrival and departu historically been handled primarily by airlines and airports.
You asked us to look at aviation safety and update our prior work,  including our 2007 report on runway and ramp safety. To do so, we  addressed the following questions: (1) What actions has FAA taken to  improve safety in the terminal area since 2007? (2) What are the trends in  terminal area safety and the factors contributing to these trends? and (3)  What additional actions could FAA take to improve terminal area safety?
To answer these questions we reviewed our prior work and other reports  and literature on safety in the terminal area and analyzed FAA, National  Transportation Safety Board (NTSB), and Occupational Safety and Health  Administration (OSHA) data on surface and airborne incidents in the  terminal area. In order to ensure that FAA data were sufficiently reliable  for our purposes, we reviewed agency documentation and interviewed  agency officials. We also obtained information about data from OSHA and  NTSB to assure data reliability. We interviewed FAA officials, as well as  industry experts and representatives from key industry associations. In  addition, in order to obtain background information on safety in the  terminal area, we interviewed facility managers and airport officials and  observed facility operations and the application of key technologies at  four air traffic control facilities and one airport, which were located near  GAO offices. (See app. I for more information on our objectives, scope,  and methodology.)
We conducted this performance audit from February 2011 to October  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The terminal area is the area around an airport extending from the airfield  or surface to about 10,000 feet vertically and out to about 40 miles in any  direction. The terminal area includes airport surface areas such as  runways, taxiways, and ramps, as well as the airspace covered by air  traffic control towers—typically within 5 miles of a towered airport—and by  terminal radar approach control (TRACON) facilities, which typically  handle air traffic to within about 40 miles of an airport (see fig. 1).
Terminal area safety incidents can occur on the surface at airports or in  the airspace around them. Surface incidents may threaten the safety of  aircraft, passengers, and airport workers, among others. Terminal area  safety incidents that happen on runways and taxiways include incursions  and excursions. Runway incursions typically involve the incorrect  presence of an aircraft, vehicle, or person on a runway, and runway  excursions generally occur when an aircraft veers off or overruns a  runway (see fig. 2). Ramp incidents can involve aircraft or airport  vehicles, such as baggage carts or ground handling vehicles, as well as  airline and airport employees and others.
Airborne safety incidents in the terminal area often involve a loss of the  minimum required distance between aircraft—as airplanes fly too close to  each other—or as individual aircraft fly too close to terrain or obstructions.
These incidents are called “losses of separation,” because there is a  violation of FAA separation standards that ensure established distances  are maintained between aircraft or other obstacles while under the con of air traffic controllers. Generally, air traffic controllers must maintain  either vertical or horizontal separation between aircraft (see fig. 3), and  losses of separation occur when both of these measures are violated,  based on phase of flight and size of the aircraft.
Safety in the terminal area is a shared responsibility among FAA, airlines,  pilots, and airports. FAA air traffic controllers oversee activity on runways  and taxiways, and airlines and airports provide primary safety oversight in  ramp areas. Several FAA offices have a role in terminal area safety  including:    The Office of Runway Safety (Runway Safety) within the Air Traffic  Organization (ATO) Safety Office was established in 1999 and le and coordinates the agency’s runway safety efforts. Its primary  mission is to improve runway safety by decreasing the number a severity of runway incursions. Runway Safety is responsible for  nd  developing a national runway safety plan and performance measures for runway safety and evaluating the effectiveness of runway safety  activities. The office currently has an acting director.  Other FAA  offices, including the Office of Aviation Safety, the Office of Airports,  other components of ATO, and regional offices sup port Runway  Safety’s work to identify hazards and analyze risk.    ATO manages air tr affic control and develops and maintains runway  safety technology.    The Office of Aviation Safety and Flight Standards Service (Flight  Standards) within it conduct safety inspections of airlines, audit air  traffic safety issues, and administer a program to obtain information  about safety incidents involving pilots.    The Office of Airports oversees airport-related safety, including airpo infrastructure. This includes issuing airport operating certificates to  commercial service airports, establishing airport design and safety  standards, and inspecting certificated airports. The Office of Airports  also provides Airport Improvement Program (AIP) grants to airports to  help support safety improvements.
Airlines and airports typically oversee the safety of operations in ra areas. Ramp areas are typically small, congested areas in which  departing and arriving aircraft are serviced by ramp workers, who include  baggage, catering, and fueling personnel. These areas can be dange for ground workers and passengers. As noted in our 2007 report on  runway and ramp safety, FAA’s oversight of ramp areas is generally  provided indirectly through its certification of airlines and airports. Bot NTSB and OSHA investigate accidents in the ramp area. Thus, NTSB  investigates ramp accidents—and other accidents involving aircraft—that  occur from the time any person boards an aircraft with the intention t o fly  until the time the last person disembarks the aircraft, if the accident  h  results in serious or fatal injury or substantial aircraft damage. OSHA  can conduct an inspection in response to a fatality, injuries, or a  complaint, unless it is preempted by an exercise of statutory authority by  FAA.
FAA collects and analyzes information about various safety incidents in  the terminal area in order to track incidents, identify their causes, and  assign severity levels. Currently data are collected at towered airports for  runway incursions, some other surface incidents, and for airborne  incidents. By contrast, no complete data are collected for incidents in  ramp areas. FAA categorizes incidents according to the actions or  inactions of air traffic controllers, pilots, or others, such as pedestrians or  vehicle operators. Table 1 provides hypothetical examples of each type of  incident. Depending on the type of incident identified—air traffic control  surface event, operational error or deviation, pilot deviation, or  pedestrian/vehicle deviation—different offices within FAA are responsible  for investigating individual incidents.
FAA is in the process of implementing a data-driven, risk-based approach  to safety oversight that FAA expects will help it continuously improve  safety by identifying hazards, assessing and mitigating risk, and  measuring performance. For decades, the aviation industry and FAA  have used data reactively to identify the causes of aviation accidents and  incidents and take actions to prevent their recurrence. Using a safety  management system approach, the agency plans to use aviation safety  data to identify conditions that could lead to incidents, allowing it to  address risks proactively. FAA’s current approach for analyzing  information about safety in the terminal area includes separate  approaches for surface and airborne incidents.    Surface incidents. For runway incursions, Runway Safety collects  information from the Administrator’s Daily Bulletin and the Air Traffic  Quality Assurance database (ATQA)—a mandatory reporting system  with incident information recorded by FAA air traffic controller  supervisors, support specialists, and managers—and other sources  such as incident investigations. Runway Safety determines how an  event will be categorized (e.g., air traffic control, pilot, or  vehicle/pedestrian deviation, etc.), and the runway incursion severity  classification team, which consists of representatives from the Office  of Airports, Flight Standards, and ATO Terminal Services, determines  the severity of the incursion.    Airborne incidents. For terminal area incidents that occur in the air,  the primary source of information is ATQA. FAA recently adopted a  new process for analyzing these incident data and has taken steps to  increase the amount and quality of information collected. FAA officials  stated that, prior to this change, data were limited to the information  collected in ATQA from FAA managers and supervisors, with limited  input from individual controllers through controller statements  gathered during incident investigations. We will discuss the new  system in more detail in the following section.
section_title: FAA Has Taken Actions to Reduce Risk in the Terminal Area
subsections: section_title: Procedural and Technological Changes to Improve Runway Safety
paragraphs: FAA has taken several steps since 2007 to further improve surface safety  at airports, focusing most notably on efforts to reduce the number and  severity of runway incursions—the agency’s key performance measures  for this area. (See fig. 4.) As part of its 2007 Call to Action Plan, the  agency implemented new safety approaches and developed milestones  for the implementation of various mid- and long-term initiatives, such as  conducting safety reviews of 20 airports where incursions were of  greatest concern, upgrading airport markings at airports, and reviewing  cockpit and air traffic procedures. Additionally, FAA’s 2009–2011 National  Runway Safety Plan establishes priorities for each FAA office involved in  reducing incursion risks and identifies performance targets for reducing  the risk of runway incursions, including an overall goal to reduce total  runway incursions by 10 percent from 1,009 in fiscal year 2008 to 908  incursions by the end of fiscal year 2013. In 2010, FAA issued an order  that further strengthened the role of Runway Safety as the agency’s focus  for addressing incursions and improving runway safety. FAA has also  proposed new rules related to airport safety management systems that  address ramp areas.
Additionally, FAA established local and regional runway safety action  teams that assess runway safety issues at particular airports, formulate  runway safety action plans to address these concerns, and execute their  runway safety programs. FAA also established the Runway Safety  Council (Council) with aviation industry stakeholders to develop a  systemic approach to improving runway safety. The Council’s Root  Cause Analysis Team—comprised of representatives from FAA and  airlines—investigates severe runway incursions to determine root causes  in order to identify systemic risks. The Root Cause Analysis Team  presents recommendations to the Council, which in turn, assigns  accepted recommendations to FAA or the aviation industry, based on  which is best able to address root causes and prevent further incursions.  The Council is responsible for tracking recommendations and ensuring  that they get implemented.
FAA’s layered approach to addressing runway safety includes a range of  actions, such as encouraging airport improvements, including improving  runway safety areas; changes to airport layout and runway markings,  signage, and lighting; providing training for pilots and air traffic controllers;  mitigating wildlife hazards; and researching, testing, and deploying new  technology. According to its 2009–2011 National Runway Safety Pla annual safety reports, FAA’s efforts to decrease the risk of surface  incidents include:  Improving runway safety areas. In order to reduce fatalities and  injuries from runway excursions, the Office of Airports has provide between $200 and $300 million annually since 2000 through AIP  grants to improve runway safety areas, which are unobstructed areas  surrounding runways.    Outreach to general aviation pilots. Regional runway safety action  team meetings, briefings, and clinics for general aviation pilots and  flight instructors discuss the importance of runway safety and how to avoid incursions. FAA also provided training, printed materials, and  electronic media such as DVDs explaining runway safety.    New terminology. FAA adopted international air traffic co terminology for taxi clearance instructions to help avoid  miscommunication between pilots on the taxiway and runway a traffic controllers. These included new mandatory detailed taxi  instructions, including directional turns, fo taxiing to and from ramps and runways.  r all aircraft and vehicles    Upgraded markings. Markings—such as enhanced centerlines drawn on taxiways and runways—wer commercial airports in 2010. e installed at all 549 FAA-certificated    Hot spot identification. Hot spots—locations on runways or taxiways  with a history of collisions or incursions or the heightened potentia such incidents—have been identified on airport diagrams to alert  pilots of complex locations on runways and taxiways.    Airport layout. Some airports have relocated taxiways, allowing pilots to avoid crossing active runways during the taxi phase. These “end  around taxiways” facilitate ground movement and minimize conflict with aircraft operating on runways. The Office of Airports has alsoreleased guidance on the design of taxiways and aprons to help  prevent runway incursions.    Training. FAA has developed video programs, training modules, and best practices for pilots, controllers, and airport personnel aimed at  heightening awareness of situations that could lead to incursions. F now also requires that runway incursion prevention be included in ning for controllers, pilots, and all certificated airport  refresher trai employees.    Research and development of best practices and other useful  information. Runway Safety’s Web site has resources, best prac and statistics on runway safety. Moreover, Runway Safety has  produced DVDs and Pilot’s Guide brochures, as well as runway  incursion safety alerts for airport operators.    Wildlife Hazard Mitigation. In addition to an active research program  for developing practical techniques for mitigating bird strikes, F encouraged all certificated airports to conduct wildlife hazard  assessments and is pursuing rulemaking to make it mandatory for  certificated airports to do so. FAA currently provides AIP funds to  hire qualified wildlife biologists to develop assessments and mitigation  plans, as needed.
A number of available technological systems are intended to help re the number and severity of runway incursions, and FAA has made  progress installing several of these systems since 2007. For example order to prevent collisions, FAA completed installation of the Airport  Surface Detection Equipment, Model X (ASDE-X) system at 35 major  airports, which provides air traffic controllers a visual representation of  traffic on runways and taxiways (see fig. 5). Other systems that will  provide safety information directly to pilots are being installed or tested.  For example, runway status lights, an automatic series of lights that giv pilots a visible warning when runways are not clear to enter, cross, or  depart on, are planned to be installed at 23 airports by August 2016.
See  appendix II for more information on safety.
To date, runway excursions have not received the same level of attention from FAA as incursions. However, excursions can be as dangerous as  incursions; according to research by the Flight Safety Foundation,  excursions have resulted in more fatalities than incursions globally. FAA  is now planning efforts to track and assess excursions as well. According  to FAA officials, in response to recommendations that we and others hav made, Runway Safety will begin overseeing runway excursions on  October 1, 2011. Specific responsibilities include collecting and analyzing  data to develop steps to reduce the risk of such incidents. According to  FAA officials, the office plans to develop an official definition of an  excursion, develop a data collection instrument and performance metrics  e  that would enable it to collect and evaluate excursion data, and develop  training and steps to help mitigate excursions. According to a timeline  from Runway Safety, it will be several years before this program is totally  implemented and FAA has detailed information about excursions.
FAA recently issued two proposed rules for airports under the agency’s  authority to issue airport operating certificates. The first proposed rule,  issued in October 2010, would require airports to establish safety  management systems for ramps areas, as well as other parts of the  airfield, including runways and taxiways. As previously noted, FAA  historically has not primarily overseen safety in ramp areas, which are  typically controlled by airlines or—to a lesser extent—airports using their  own practices. FAA’s proposal would require airports to establish safety  management systems for the entire airfield environment in order to  ensure that  individuals are trained on the  surface of the terminal area;  safety implications of working on the    hazards are identified proactively, and analysis systems are in place;    data analysis, tracking, and reporting syste ms are available for trend  analysis and to gain lessons learned; and  there is timely communication of safety issues to all stakeholders.
A second proposed rule, issued in February 2011, would establish  minimum training standards for all personnel who access ramp areas.
Required training would occur at least yearly and include familiarization  with airport markings, signs, and lighting, as well as procedures for  operating in the nonmovement (ramp) area. The public comment period  for these proposed rules closed during July 2011. FAA has not indicated  when the rules would be finalized.
We reported in 2007 that FAA lacked ground handling standards for ramp  areas. In the absence of agency standards, other organizations have  developed tools to improve ramp safety. For example, the Flight Safety  Foundation has collected best practices and developed a template of  standard operating procedures to assist ramp supervisors in developing  their own procedures. The guidelines are wide ranging and include the  reporting of safety information, ramp safety rules, the positioning of  equipment and safety cones, refueling, and caring for passengers, among  other areas. In addition to the Flight Safety Foundation guidelines, the  International Air Transport Association, an international airline  association, has developed a safety audit program for ground handling  companies aimed at improving safety and cutting airline costs by  drastically reducing ground accidents and injuries. The program is  available to all ground service providers, who, after successfully  completing the audit, are placed on a registry. As of August 2011, Seattle- Tacoma International Airport is the only domestic airport participating in  the program.
section_title: Data Collection and Risk- Based Analysis of Airborne Aviation Safety Information
paragraphs: Controllers are required to report any occurrence that may be an operational deviation,  operational error, proximity event, or air traffic incident if the reported issue is known only  to the employee and occurs while the employee is directly providing air traffic services to  aircraft or vehicles or first level watch supervision. concerns have been addressed through informal discussions between  ATSAP officials and FAA facilities.
In other efforts to obtain more safety data, FAA has taken other steps to  make incident reporting less punitive. For example, in July 2009, FAA  changed its incident reporting policy such that individually identifying  information, such as air traffic controller names and performance records,  is no longer associated with specific incidents in ATQA, the central FAA  database used by air traffic control managers or supervisors to report  incidents. In addition, in July 2010, FAA also stopped issuing incident “not  to exceed” targets to individual air traffic control facilities (e.g., towers,  TRACONS, or en route facilities). According to officials, these targets  created an incentive for underreporting of less serious incidents by  supervisors at the facility level, and the targets were discontinued in order  to encourage increased reporting at the agency.
FAA is also implementing new technologies, specifically, the Traffic  Analysis and Review Program (TARP), an error detection system that can  be used to automatically capture losses of separation that occur while  aircraft are under the control of air traffic control towers and TRACONs.  Historically, FAA relied on air traffic controllers and their supervisors to  manually report on operational errors, something we have noted in the  past may negatively impact data quality and completeness. TARP  automatically captures data on all airborne losses of separation, which,  according to officials, will increase the volume of data FAA gathers on air  traffic safety incidents and enable FAA to obtain a more complete picture  of potential safety hazards. According to the fiscal year 2010 FAA  Performance and Accountability Report, FAA has deployed TARP at 150  air traffic control tower and TRACON facilities. According to FAA officials,  TARP is currently being used as an audit tool for approximately 2 hours  per month at some facilities—in lieu of full-time use at all facilities—but  further implementation of the system has been delayed as the agency  evaluates the impact of the system on controllers and determines how the  system will be used and how to handle the additional workload that will be  created as more incidents are captured and require investigation.  Following the completion of these steps, FAA will take 210 days to fully  deploy TARP. Currently, incidents identified through TARP are being  included in official incident counts.
FAA is also shifting to a new, risk-based process for assessing a select  category of airborne losses of separation. FAA began using the Risk  Analysis Process (RAP)—which is adapted from a similar process used  by the European Organization for the Safety of Air Navigation  (Eurocontrol)—in fiscal year 2010. While the new process is being  established, RAP will be used in tandem with the existing system. RAP is  currently limited to Losses of Standard Separation (LoSS). This subset  of airborne incidents includes those in which the separation maintained is  less than 66 percent of the minimum separation standard for the planes  involved. Under RAP, FAA determines both the severity and the  repeatability of selected LoSS events (that is, how likely a certain LoSS  event will occur again at any airport under similar circumstances based  on a number of factors). These factors include proximity of planes to one  another at the time of the event, rate of closure between planes, controller  and pilot recovery, and whether or not Traffic Collision Avoidance System  (TCAS) technology is triggered by the incident. Prior to the development  of RAP, FAA categorized losses of separation based on proximity alone:  the greater the loss of separation between two planes, the greater the  severity of the incident in question. Operational errors were then rated  on an A–C scale, with those that retained more than 90 percent of  required minimum separation categorized as proximity events. (Fig. 7  compares the threshold for review of incidents in each system.) Officials  stated that RAP is more robust than the previous system because it is  able to take numerous factors into account when determining event  severity, as well as overall risk to air traffic safety. In addition, the RAP  will assess risk for LoSS events that were not assigned a severity  category under the old system. As a result, they said the agency will be  better equipped to identify systemic issues in air traffic safety and to issue  related corrective action requests. Based on analysis of systemic issues  identified across incidents, RAP released its first five corrective action  requests on July 19, 2011, which were developed to mitigate specific  hazards that contribute to what RAP has identified as the highest risk  events.
We, the Department of Transportation Inspector General (IG), and NTSB  have raised concerns about terminal area safety. For example, we  recommended, in 2007, that FAA take several steps to enhance runway  and ramp safety, such as updating its national runway safety plan,  collecting data on runway excursions, and working with OSHA and  industry to collect and analyze better information on ramp accidents. In  2007, FAA put in place a Director for Runway Safety and issued a Call to  Action aimed at reducing the risk of incursions following several high- profile incidents (see table 2 for select recommendations to FAA). The IG  also made recommendations to FAA about runway safety issues and  recommended that FAA take several steps to reduce the risk of airborne  incidents and improve oversight of this area. For example, the IG  recommended that FAA clearly document the severity ratings used by  FAA for runway incursions, revise the national plan for runway safety, and  realign the Office of Runway Safety. With regard to airborne incidents,  the IG recommended establishing a process to rate the severity of pilot  deviations and corresponding performance goals, developing milestones  for implementing TARP, and assuring that Flight Standards works with  ATO Safety Services to determine whether losses of separation are pilot  or controller errors, among other recommendations. Further, NTSB  continues to include runway safety, safety management systems, and  pilot and air traffic controller professionalism issues on its list of most  wanted safety improvements.
section_title: Reported Surface and Airborne Incidents Have Increased, and Several Key Factors Likely Contribute to Trends
subsections: section_title: Rate of Reported Runway Incursions Has Increased since 2004, but Serious Incidents Have Significantly Declined
paragraphs: In fiscal years 2009 and 2010, the agency met its interim goals toward  reducing the total number of runway incursions at towered airports, but  the rate of incursions per million operations continued to increase (see   fig. 8). As noted in our 2007 report, both the number and rate of  incursions reached a peak in fiscal year 2001, prompting FAA to focus on  runway safety. The number and rate of incursions at towered airports  decreased dramatically for a few years thereafter, though the impact of  FAA’s efforts on these outcomes is uncertain. Beginning in 2004,  however, both the number and rate of incursions began increasing again.  For example, in fiscal year 2004, there were 733 incursions at a rate of  11.4 incursions per 1 million tower operations, compared with fiscal year  2010, when there were 966 incursions at a rate of 17.8 incursions per 1  million such operations. Although the rate of incursions at towered  airports continues to increase, the number of incursions at these airports  peaked in fiscal year 2008.
The most serious runway incursions at towered airports—where collisions  are narrowly avoided—decreased by a large amount from fiscal year  2001 to 2010, and FAA met or exceeded its goals for reducing the rate of  these incidents. FAA classifies the severity of runway incursions into  four categories—A through D—and its performance targets call for the  reduction of the most severe incursions (category A and B) to a rate of no  more than 0.45 per million air traffic control tower operations by fiscal  year 2010 and for the rate to remain at or below that level through fiscal  year 2013. The number of the most severe incidents at towered airports  also dropped from fiscal year 2001 to 2010. Thus, category A and B  incursions at these airports decreased from 53 to just 6 during that time,  with category A incursions decreasing from 20 to 4, and category B  incursions decreasing from 33 to 2 (see fig. 9).
In fiscal year 2010, the majority of incursions at towered airports were  classified as pilot deviations (65 percent), followed by vehicle/pedestrian  deviations (19 percent), and operational errors and deviations by air traffic  controllers (16 percent) (see fig. 10). Further, for every year since 2001,  pilot deviations comprised the majority of runway incursions at these  airports, and the proportion involving these errors steadily increased from  about 55 percent of all incursions in fiscal year 2001 to 65 percent in  fiscal year 2010.
We previously reported that most runway incursions at towered airports  involved general aviation aircraft and that trend continues. General  aviation aircraft make up nearly a third of total operations at towered  airports but have consistently accounted for about 60 percent of incidents  each year since 2001. More specifically, the rate of incursions per million  tower operations involving at least one general aviation aircraft is higher  than the rate of incursions not involving general aviation aircraft, and the  rate has increased every year since fiscal year 2004 (see fig. 11). Further,  general aviation aircraft were involved in over 70 percent of the most  serious—category A and B—incursions from fiscal year 2001 through the  second quarter of fiscal year 2011. According to FAA officials, general  aviation pilots may be more susceptible to incursions and other incidents  because of their varying degrees of experience and frequency of flying.  Furthermore, general aviation pilots do not generally undergo the same  training as commercial airline pilots do.
With regard to runway excursions, our review of NTSB data found that  general aviation aircraft are also involved in most runway excursions.  Although FAA does not yet formally collect information on runway  excursions, NTSB provided us with accident investigation reports on 493  accidents that involved runway overruns or excursions since 2008. Seven  of these accidents were fatal, resulting in 14 fatalities. Our review of  these reports found that 97 percent of the accidents involving excursions  referred to the involvement of at least one general aviation aircraft.
In our 2007 report, we found that efforts to address the occurrence of  safety incidents in ramp areas were hindered by the lack of data on the  nature, extent, and cost of ramp incidents and accidents and by the  absence of industrywide ground handling standards. As discussed above,  FAA collects no comprehensive data on incidents in the ramp area, and  NTSB does not routinely collect data on ramp incidents that do not result in  injury or aircraft damage. Likewise, as mentioned above, OSHA, the  primary source of ramp fatality data, does not collect data on incidents that  do not result in at least three serious injuries or fatalities. In the ramp area,  OSHA data on worker fatalities show the number of deaths in the ramp  area to have varied between 3 and 11 from 2000 to 2010. The rate  remained constant—between 4 and 6 deaths per year—from 2008 to 2010.
section_title: Rate and Number of Reported Airborne Operational Errors Increased since 2007, Including the Most Serious Incidents
paragraphs: The rate of reported airborne operational errors in the terminal area  increased considerably in recent years. From the second quarter of fiscal  year 2008 to the second quarter of fiscal year 2011, the rate and number  of reported airborne operational errors increased significantly. During this  time period, the rate of reported airborne operational errors in the terminal  area nearly doubled, increasing 97 percent, and the number of reported  airborne operational errors increased from 220 to 378. The rate of  incidents began a notable climb in the fourth quarter of fiscal year 2009,  peaked in the second quarter of fiscal year 2010, and remained at rates  higher than the historical average through the second quarter of 2011  (see fig. 12). FAA officials attributed at least some portion of the spike in  reported incidents during the second quarter of fiscal year 2010 to  approximately 150 events that occurred as the result of the  misinterpretation of an arrival waiver at one TRACON facility.
While the rate of airborne operational errors has increased over time in  both the TRACON and tower environments, the rate of errors in the  TRACON environment has increased more. Between the second quarters  of fiscal years 2008 and 2011, the rate of operational errors in the  TRACON environment increased from 8.5 to 22.6 operational errors per  million air traffic control operations—a 166 percent increase (see fig. 13).  In comparison, operational errors increased by 53 percent in the tower  environment.
Overall, the rate of the most severe airborne operational errors more than  doubled between the second quarter of fiscal year 2008 and the second  quarter of 2011. While the least severe (category C) incidents are more  numerous than the most severe, the most severe (category A) incidents  increased from 5 in the second quarter of 2008 to 14 in the second quarter  of 2011. In comparison, category C operational errors increased by 135  percent, and category B operational errors decreased by 5 percent.
These incident rates do not meet FAA goals under both the prior severity  system and using new risk assessment measures. In fiscal year 2010,  FAA reported 3.32 category A and B operational errors per million air  traffic control operations, significantly exceeding its targeted rate for fiscal  year 2010 of 2.05 per million operations. In fiscal year 2011, FAA  replaced its operational error measure with a new measure—the System  Risk Event Rate (SRER)—a 12-month rolling rate of the most serious  LoSS events per thousand such events. The rate of high-risk events  also increased using this measure. According to data provided by FAA,  the number of the most serious LoSS events—called high-risk events in  the new risk assessment process—spiked from 9 events in December  2010 to 16 events in January 2011 but has since decreased. However,  the overall SRER increased significantly between December 2010 and  February 2011 (from 21.9 to 29.9 high-risk LoSS events per 1000 LoSS  events) and remains significantly elevated above FAA’s target of 20  serious LoSS events for every thousand such events. The SRER for the  12-month period ending in April 2011 was 28.97. According to FAA  officials, the agency’s target of 20 LoSS events per thousand LoSS  incidents represents the system performance baseline gathered using  human reporting and may therefore be an unrealistic target as the agency  moves to gathering data electronically. FAA plans to continue to collect  data on and categorize events using both the old and new systems. Once  FAA has completed a 2-year baseline period, it has committed to conduct  an independent review of both metrics to determine whether any  improvements are needed.
section_title: Several Key Factors Likely Contribute to Trends in Runway Incursions and Airborne Operational Errors
paragraphs: Several factors have likely contributed to recent trends in runway  incursions and airborne operational errors. The agency has noted that  recent increases in runway incursions and airborne operational errors are  primarily attributable to changes in FAA’s reporting practices, which  encourage increased reporting of incidents. We found evidence to  suggest that changes to reporting policies and processes have likely  contributed to the increased number of incidents reported—both into  ATQA, the official database for incidents—and into ATSAP, the  nonpunitive reporting system for air traffic controllers. In addition, the  implementation of new technologies and procedures in the terminal area  also likely contributed to an increase in the number of reported airborne  incidents and runway incursions. FAA has carried out changes aimed at  increasing reporting, and each of these factors may have contributed to  an increase in the number of reported incidents. That said, it is possible  that the increase in safety incidents in the terminal area may also reflect  some real increase in the occurrence of safety incidents. As a  comparison, we looked at the rate of en route operational errors, which  are captured automatically by airplane tracking technology and would  therefore not be expected to substantially increase by a change in  reporting practices or procedures at the agency. We found that the  average rate of en route operational errors in fiscal year 2010 was 38  percent higher than the year before, and that the overall rate increased 13  percent from the second quarter of fiscal year 2008 to the same quarter in  2011. According to FAA officials, some of the increase in reported en  route errors may be attributable to increased confidence in the  nonpunitive nature of the system—reflected by a decrease in the number  of requests for reclassification of incidents from en route facilities.
Changes to reporting processes and policies at FAA may explain in part  the recent upward trend in reported runway incursions and airborne  operational errors. Since operational errors and other losses of separation  in both the tower and TRACON environments are currently reported  manually by FAA supervisors and quality assurance staff into ATQA,  changes in reported error rates may be partially attributable to changes  made to encourage more comprehensive reporting of incidents. Most  notably, as previously discussed, FAA changed its incident reporting  policy in July 2009 such that individually identifying information, such as  air traffic controller names and performance records, are no longer  associated with specific events in the ATQA database. According to  officials, this change may encourage controllers to share more  information about incidents with supervisors and quality assurance  officers. In addition, in fiscal year 2010, FAA stopped issuing incident “not  to exceed” targets to individual facilities. According to officials, these  targets may have led supervisors in the past to underreport less serious  incidents in order to meet these targets. These policy changes may have  increased reporting to an extent that these effects are apparent in incident  rates. (See fig. 15 for recent FAA changes to reporting practices overlaid  on report operational errors.)
Implementation of a nonpunitive, confidential, system of reporting for air  traffic controllers may have also contributed to the real increase in the  occurrence of operational errors, according to FAA officials. While the  implementation of ATSAP may affect reporting rates—either by  increasing reporting or by lowering the number of reports to supervisors  given that the system satisfies reporting requirements—officials told us  that it could also inadvertently lead to an actual increase in the  occurrence of operational errors or deviations. According to these  officials, the reduced personal accountability ATSAP provides may make  some air traffic controllers less risk averse in certain situations. In  addition, officials also noted that ATSAP may present a barrier to  managerial efforts to directly manage controller performance. For  example, if a report is filed into ATSAP, a supervisor may have limited  options to assign training or take other corrective actions in response to  an incident, even if he or she is aware that an error was made by an  individual air traffic controller, presuming the incident did not involve  alcohol or drug use or other such violations.
The implementation of ATSAP may have resulted in increased reporting  of incidents, although reporting into this system does not directly affect  official trends in operational errors. According to FAA officials, the  confidential, nonpunitive nature of ATSAP has contributed to a positive  change in the reporting at FAA. As a result, errors that previously may  have gone unreported by air traffic controllers are now being reported to  ATSAP. However, data entered into ATSAP are not directly available to  FAA and do not feed into ATQA (see fig. 16). In addition, it is possible  that some incidents that would have previously been reported to FAA are  now being reported only to ATSAP, thus decreasing the number of  incidents reported to FAA. According to ATSAP data, approximately 35  percent of all incidents reported to ATSAP in 2010 were “known” to  FAA—meaning that the incident was reported into ATQA by a supervisor  or manager, as well as into ATSAP by an air traffic controller—while the  other 65 percent were not official reported to FAA.
Implementation of new technologies in the terminal area may also impact  recent trends in surface incidents and airborne losses of separation. For  example, since FAA’s ongoing implementation of TARP will allow FAA to  automatically capture losses of separation in the tower and TRACON  environments, it will also likely increase the number of reported losses of  separation. According to officials, during its limited testing at facilities,  TARP has already captured errors that were not being reported by air  traffic controllers. For surface incidents, the ASDE-X system alerts  controllers when aircraft or vehicles are at risk of colliding on runways,  resulting in the identification of incidents that controllers might otherwise  not be aware of. Designed as a surface surveillance system, ASDE-X  helps to prevent collisions by raising alarms when aircraft appear to be at  risk of colliding. As these alerts draw attention to near misses or potential  collisions, they also serve to notify personnel of possible incursions and  thus may have contributed to an increase in reported events, even as  they may have prevented accidents. We analyzed the number of reported  incursions at airports with ASDE-X and found that, at many of these  airports, the number of reported incursions actually increased after their  ASDE-X systems became operational. (For more information about our  analysis of how the number of runway incursions changed after the  installation of ASDE-X, see app. III.) Officials with the Sensis Corporation,  the developer of ASDE-X, acknowledged to us that this may be a side  effect of the deployment of the system.
section_title: Enhanced Oversight and Additional Information about Incidents Could Help Improve Safety in the Terminal Area
paragraphs: FAA has taken steps to improve safety in the terminal area since 2007  and has both reduced the number of serious incursions and undertaken  successful efforts to increase reporting of incidents, but we identified two  areas in which FAA could further improve management of data and  technology in order to take a more proactive, systemic approach to  improving terminal area safety. These areas include: (1) enhanced  oversight of terminal area safety, including the management of runway  excursions and ramp areas, and (2) assurance that data for risk  assessments are complete, meaningful, and available to decision makers.
subsections: section_title: Federal Oversight of Terminal Areas Could Be Enhanced
subsections: section_title: Runway Safety Oversight Remains Limited
paragraphs: Stakeholders we spoke with generally lauded Runway Safety’s efforts on  incursions, but FAA could do more to expand to other aspects of runway  safety—notably runway excursions—as well as playing a more active  oversight role in ramp areas. As we noted earlier, FAA is rolling out a new  program to gather and analyze data on excursions, which should allow  the agency to better understand why excursions happen and develop  programs to mitigate risk. FAA is exercising some additional authority  over ramps by proposing rules to address airport safety management  systems and training for personnel accessing ramp areas, but these  efforts are limited and involve requiring airports to develop and implement  their own safety guidelines. In 2007, we reported that the lack of  standards for ramp operations hindered safety, and an upcoming report  by the Airport Cooperative Research Program (ACRP) continues to find  that no comprehensive standards exist with regard to ramp area  markings, ground operations, or safety training. The two proposed rules  by FAA on airport safety management systems and training establish  some standards for the ramp area, but proposed federal oversight would  still be limited. The proposed rule implementing safety management  systems for airports would require airports to develop plans to identify and  address hazards in the ramp area and on the airfield, in addition to  ensuring that all employees with access to runways, taxiways, and ramps  receive training on operational safety and on the airport’s safety  management system. Other aspects of ground handling, such as surface  marking and ground operations, would continue to largely be overseen by  airlines and the ground handling companies that are contracted by them.
The placement of the Runway Safety within the ATO Safety Office may  limit its ability to serve as an effective focal point for runway and terminal  area safety, given that aspects of runway and terminal area safety fall  under the purview of several parts of FAA, including ATO, the Office of  Airports, and the Office of Aviation Safety. In 2010, the IG recommended  that the placement of Runway Safety within ATO be reconsidered, because  the office may be limited in its ability to carry out cross-agency risk  management efforts. Subsequently, the IG determined that Runway Safety  had demonstrated effectiveness within ATO, but pointed to a need to  periodically review organizational structures and processes to ensure that it  continues to be effective. Runway Safety oversees data, assessments, and  performance measures across a number of safety areas—air traffic control,  pilot actions and training, outreach to general aviation, airport infrastructure,  and technologies, among others—which are under the purview of different  offices within FAA. As a result, Runway Safety has the potential to serve as  the focal point for risk management in the terminal area.
section_title: FAA Data for Risk Assessment May Not Be Complete, Meaningful, or Available to Decision Makers
subsections: section_title: Impact of Changes in Reporting Policies and Processes on Measures of Incidents and Risk Is Unclear
paragraphs: Multiple changes to reporting policies and processes in recent years make  it difficult to know the extent to which the recent increases in some terminal  area incidents are due to more accurate reporting or an increase in the  occurrence of safety incidents or both. For example, FAA officials have  specifically attributed the increase in airborne operational errors to changes  in reporting practices following the implementation of ATSAP, but, as  previously mentioned, the relationship between the implementation of  ATSAP and an increase in errors is uncertain. Likewise, other changes to  performance measures and internal reporting policies, such as removal of  individually identifying information from ATQA, further obscure the source  of recent trends. Without accurate and consistent measures of safety  outcomes, FAA cannot assess the risks posed to aircraft or passengers  over time or the impacts of its efforts to improve safety.
As we noted in a 2010 report, FAA has embarked on a data-driven, risk- based safety oversight approach. As part of this effort, FAA has  established a new, risk-based measure to track losses of separation, but  measures for runway incursions are not risk based, reflecting instead a  simple count of incidents. Thus, FAA currently rates the severity of  incursions based on proximity and the response time to avoid a collision  and does not differentiate between types of aircraft—or the number of  lives put at risk—as part of its severity calculation. While any loss of life is  catastrophic, the impact of an accident involving a commercial aircraft  carrying hundreds of passengers can have different implications than an  accident involving smaller aircraft. According to industry stakeholders, the  use of proximity as the main criterion for severity of incursions is overly  simplistic. As a result, FAA may be unable to use incursion data to  identify the most serious systemic safety issues. Similarly, the application  of risk assessment to measures of runway safety could allow FAA to  focus individually on the risk posed by incursions by large commercial  aircraft, as well as the risk posed by an ever-increasing incursion rate  among general aviation operations. Additional attention to the root causes  of incidents involving general aviation could potentially identify strategies  addressing this ongoing challenge, which may include the installation of  low-cost ground surveillance systems. While FAA officials did not detail  immediate plans to alter the measure for incursions, officials did state that  the agency plans to introduce surface incidents into RAP at the beginning  of fiscal year 2012. The joint FAA-aviation industry Runway Safety  Council is a first step toward the effort to reduce incursion risks. By  identifying causes and making recommendations that could help  determine what changes would be needed to make measures more risk- based, the Council’s Root Cause Analysis Team can help reduce  incursion risks. Further, according to FAA officials, the new measure  being developed for excursions will be risk-based; however, this measure  will not be fully in use until 2014.
By contrast, FAA has taken steps to improve its ability to assess the risk  of airborne operational errors and to collect more information. However,  under the new risk assessment system, far fewer incidents are subject to  analysis than were included in previous, nonrisk-based iterations, and the  measure may therefore not account for all potential risk. For instance,  RAP does not yet have procedures to assess losses of separation with  terrain and with airspace boundaries. Currently, LoSS events in which at  least 66 percent of minimum separation is maintained between aircraft  are not assessed through FAA’s recently launched RAP. According to  FAA officials, the 66 percent threshold for inclusion in RAP was adopted  in recognition of the resources required for the enhanced risk-analysis  process. This initial threshold is not based on specific risk-based criteria.  Furthermore, losses of separation eligible for inclusion in RAP are  currently limited to those that occur between two or more radar-tracked  aircraft. As a result, many incidents—such as those that occur between  aircraft and terrain or aircraft and protected airspace—are currently  excluded from FAA’s process for assessing systemic risk. According to  FAA officials, this exclusion is in part because there is no system in place  through which the current RAP proximity inclusion threshold could be  applied to these types of incidents, although FAA officials stated that an  effort is under way to expand RAP to include these areas. In addition,  FAA’s new measure for assessing air traffic risk levels—the SRER—does  not include many losses of separation that were tracked under the old  measure, although it does expand the assessment process to include  some errors caused by pilots. Further, while the technology has been  developed to collect data automatically for potential operational errors  involving losses of separation, FAA has delayed the full implementation of  TARP in air traffic control towers and TRACONS. According to FAA  officials, the implementation of TARP may create workload challenges for  quality assurance staff, as the technology is likely to capture hundreds of  potential losses of separation that were not previously being reported  through existing channels. In 2009, the IG recommended that FAA  establish firm timelines for the full implementation of TARP.
Data collected through ATSAP, the nonpunitive reporting system, have  limitations. There is the potential for serious events to be reported only  to ATSAP and to therefore not be included in the official ATQA database  or in RAP. Such events are referred to as “unknown” events. In 2010, 65  percent of incidents reported to ATSAP were unknown to FAA. (See fig.  17.) FAA officials acknowledged that there are a large proportion of  unknown incidents but stated that these incidents are likely to be minor. In  addition, some information about incidents reported to ATSAP is available  for analysis by FAA and the aviation industry via the Aviation Safety  Information Analysis and Sharing (ASIAS) program, as de-identified  ATSAP data are shared with ASIAS. Further, they noted, ATSAP  reports may be procedural, rather than reports of incidents or operational  errors. Available data from the ATSAP program office indicates, however,  that some of the “unknown” reports in the system were potentially serious  events. For example, between the program’s launch in July 2008 and  June 2011, 74 out of 253 ATSAP reports that were classified as  potentially hazardous did not appear in ATQA, accounting for 29 percent  of these reports. In June 2011—the most recent month for which data  are available—approximately one third of all ATSAP reports classified as  potentially major events, and 42 percent of those classified as hazardous,  did not appear in ATQA. According to officials, ATSAP allows controllers  to report incidents that may have otherwise gone unreported, and the  program facilitates early detection and improved awareness of  operational deficiencies and adverse trends. These unknown events, FAA  officials point out, would not likely have been reported into ATQA before  the implementation of ATSAP.
Information sharing challenges may impact the ability of FAA to analyze  safety data and understand safety trends. Multiple FAA programs and  data systems assign contributing factors to incidents, but factors are not  coordinated across programs. For instance, both ATSAP and RAP have  developed sets of factors that are identified as contributing to incidents  during the incident investigation process. However, despite the fact that  these two programs look at some of the same type of incidents (airborne  losses of separation), program officials have not coordinated their  development of the categories used to describe incidents. As a result,  officials we interviewed stated that it is difficult to compare data across  systems. For example, both ATSAP and RAP issued internal reports  identifying top factors contributing to reported incidents, but there is no  apparent overlap between the two lists. In addition, while the ATQA  database contains more than 50 contributing factors for operational  errors, FAA and the ATSAP program office do not use these data to  identify systemic safety issues (see table 3). According to FAA officials,  FAA is currently developing a common set of contributing factors for  ATSAP and RAP, as well as a translation capability that will allow for the  inclusion of historical data on contributing factors in future analyses. The  IG raised concerns about the quality of ATQA data on contributing factors  in a 2009 report, noting that FAA does not consistently include fatigue  issues in contributing factor data it collects on operational errors. FAA  has added contributing factors related to fatigue to ATSAP and is  exploring ways to gather objective shift, schedule, and related resource  management data to support enhanced fatigue analysis.
Regional and local access to and awareness of data related to both  individual incidents and incident trends may be limited. According to FAA  officials we interviewed at the regional level, it is difficult for supervisors at  the regional and facility levels to obtain information on incident trends  specific to their area of supervision in part because key databases, such  as ATQA, do not have the capability to allow regional supervisors to run  region- or facility-specific data queries. In addition, while multiple data  resources may be available, officials stated that information on incidents  is scattered, and no central source exists where employees can identify  available data resources. While FAA has made advances in the quantity  and comprehensiveness of the data it collects on incidents in the terminal  area, officials stated that the agency faces difficulty in developing  sophisticated databases with which to perform queries and modeling of  the data. According to FAA officials, the full implementation of CEDAR will  address many of the deficiencies identified by regional and local offices.
The nature and scope of ramp accidents are still unknown, just as they  were when we reported in 2007, and we were told by officials with Airports  Council International that it can be difficult to for airports to get data on  incidents in the ramp area—areas typically overseen by airlines. This will  pose a challenge as airports move to implement safety management  systems and seek to identify and mitigate hazards. As one aviation expert  explained, even if data are available locally—which they may not be—the  number of incidents at individual airports can be too few to allow for the  identification of root causes or the proactive identification of risk.
section_title: Conclusions
paragraphs: The Office of Runway Safety focuses on improving safety by reducing the  number and severity of runway incursions. However, risk management in  terminal areas involves more than just incursions—notably runway  excursions and incidents in ramp areas. Runway Safety plans to start  tracking runway excursions in October 2011, but it will take several years  to develop processes for identifying and tracking incidents, identifying and  mitigating risks, and measuring outcomes. Likewise, FAA does not track  incidents in ramp areas, although we previously recommended that FAA  work with the aviation industry and OSHA to develop a mechanism to  collect and analyze data on ramp accidents. Airports implementing plans  for safety management systems under FAA’s proposed rule will need  data that are useful, complete, and meaningful in order to accurately  assess risk and plan for safety, but FAA cannot yet provide meaningful  data for the assessment and management of risks posed by runway  excursions or ramp areas. Without information on incidents in these  areas, FAA and its safety partners are hampered in their ability to identify  risk, develop mitigation strategies, and track outcomes.
FAA addresses runway incursions as a specific type of incident and does  not distinguish between commercial and general aviation in its  performance measures. However, risks posed by runway safety incidents  to passengers and aircraft in the national airspace system are different for  commercial aircraft and general aviation. FAA performance measures for  runway incursions—including the number, rate, and severity—do not  reflect differences between commercial and general aviation and are not  risk-based. The agency has installed risk-reduction technologies at larger  commercial service airports, for example, but in the absence of risk-based  performance measures, it lacks the ability to prioritize projects or measure  effectiveness. With regard to general aviation, this traffic currently  accounts for about a third of total tower operations, but 60 percent of  runway incursions involve these aircraft. While Runway Safety has  acknowledged that general aviation has caused more runway incursions,  without performance measures that reflect risk, FAA may not be able  identify appropriate mitigation strategies to address the large—and  growing—proportion of runway incidents—including both incursions and  excursions—involving general aviation aircraft. Strategies to decrease the  risk posed by safety incidents involving general aviation could include  additional outreach to these pilots, increased remediation following pilot  errors, or the installation of technologies such as low-cost ground  surveillance at airports serving general aviation traffic.
Safety in the terminal area is a shared responsibility among FAA, airlines,  pilots, and airports, and there are a number of FAA offices that either  collect or analyze terminal area incident data, but useful access to  complete and meaningful data is limited. The agency currently does not  have comprehensive risk-based data, sophisticated databases to perform  queries and model data, methods of reporting that capture all incidents, or  a level of coordination that would facilitate the comparison of incidents  across systems. Technologies aimed at improving reporting have not  been fully implemented. As a result, aviation officials managing risk using  safety management systems, including local and regional decision  makers, have limited—if any—access to FAA incident data. For example,  FAA’s official database for air traffic safety does not allow local or regional  FAA safety officials to run region- or facility-specific data queries. Further,  under the new risk assessment process used for losses of separation,  fewer incidents are assessed and accounted for in performance  measures—such as losses of separation between aircraft and terrain or  aircraft and protected airspace—which may distort risk assessment  processes. Finally, according to FAA officials, one reason the agency has  not fully implemented TARP is that implementation of TARP may create  workload challenges for FAA quality assurance staff, as the technology is  likely to capture hundreds of potential losses of separation that were not  previously being reported through existing channels. FAA offices and  others using a safety management system approach to manage risk  should have access to complete and meaningful data to allow for hazard  identification and risk management. The ability of FAA and airport  officials—and the local Runway Safety Action Teams that they serve on— to identify safety risks, develop mitigation strategies, and measure  outcomes is hindered by limited access to complete and meaningful data.
section_title: Recommendations for Executive Action
paragraphs: To enhance oversight of terminal area safety to include the range of  incidents that pose risks to aircraft and passengers, we recommend that  the Secretary of Transportation direct the FAA Administrator to take the  following three actions:    develop and implement plans to track and assess runway excursions  and extend oversight to ramp safety;    develop separate risk-based assessment processes, measures, and  performance goals for runway safety incidents (including both  incursions and excursions) involving commercial aircraft and general  aviation and expand the existing risk-based process for assessing  airborne losses of separation to include incidents beyond those that  occur between two or more radar-tracked aircraft; and    develop plans to ensure that information about terminal area safety  incidents, causes, and risk assessment is meaningful, complete, and  available to appropriate decision makers.
section_title: Agency Comments
paragraphs: We provided the Departments of Transportation and Labor, NTSB, and  the National Aeronautics and Space Administration (NASA) with a draft of  this report for review and comment. The Department of Transportation  agreed to consider our recommendations and provided clarifying  information about efforts made to improve runway safety, which we  incorporated. The Department of Labor, NTSB, and NASA provided  technical corrections, which we also incorporated.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 7 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretaries of Transportation and Labor,  NTSB, the Administrator of NASA, and interested parties. In addition, this  report will be available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. Key contributors to this report are  listed in appendix IV.
Gerald L. Dillingham, P Director, Physical Infrastructure Issues  h.D.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objective was to review aviation safety and update our 2007 report on  runway and ramp safety. To do so, we addressed the following  questions: (1) What actions has the Federal Aviation Administration (FAA)  taken to improve safety in the terminal area since 2007? (2) What are the  trends in terminal area safety and the factors contributing to these trends?  and (3) What additional actions could FAA take to improve terminal area  safety?
To identify actions FAA has taken since 2007 to improve safety in the  terminal area and to identify additional actions FAA could take to improve  safety, we reviewed our prior reports, as well as documents and reports  from FAA, the Department of Transportation Inspector General (IG), the  National Transportation Safety Board (NTSB), the International Civil  Aviation Organization (ICAO), and others; FAA orders, advisory circulars,  and regulations; and applicable laws. We also determined the roles and  responsibilities of FAA, NTSB, the Occupational Safety and Health  Administration (OSHA), the National Aeronautics and Space  Administration (NASA), airports, and airlines involving runway, terminal  airborne, and ramp safety. In addition to interviewing officials from FAA,  IG, and NTSB, we interviewed aviation experts affiliated with the Air Line  Pilots Association, Airports Council International, Air Transportation  Association, the Flight Safety Foundation, and the National Air Traffic  Controllers Association about terminal area safety practices and  technologies. We also interviewed researchers from the Air Cooperative  Research Program (ACRP) of the Transportation Research Board and  experts affiliated with various aviation technology companies. To obtain  information about air traffic control operations, observe the application of  key technologies, and interview facility managers, we visited four FAA  facilities that were near our GAO offices: the Potomac Consolidated  Terminal Radar Approach Control facility, the Washington Air Route  Traffic Control Center, and the air traffic control towers at Ronald Reagan  Washington National Airport and the Seattle-Tacoma International Airport.  We also interviewed airport officials with the Port of Seattle at Seattle- Tacoma International Airport. To obtain information about the Air Traffic  Safety Action Program (ATSAP), we interviewed officials with the ATSAP  program office and attended an Event Review Committee meeting in  Renton, Washington. We also reviewed FAA’s progress in addressing  recommendations that we, IG, and NTSB have made in previous years  and reviewed the processes that FAA uses to collect and assess runway  and air traffic safety data.
To identify and describe recent trends in terminal area safety and the  factors contributing to these trends, we obtained and analyzed data from  FAA, NTSB, and OSHA on safety incidents in the terminal area. We  analyzed FAA runway incursion data collected from fiscal year 2001  through the second quarter of fiscal year 2011, as well as FAA data on  airborne operational errors from the Air Traffic Quality Assurance  database (ATQA) from the third quarter of fiscal year 2007 through the  second quarter of fiscal year 2011. We limited our analysis to airborne  operational errors in order to avoid double counting of surface operationa errors that are included in our counts of runway incursions. We used  Operations Network data from FAA to determine rates of incursions and  airborne operational errors per million operations. Rates of incursions  were calculated per million tower operations, and rates of airborne  operational errors were calculated per million operations perfo traffic control towers, terminal radar approach control (TRACON) facilitie s,  and en route facilities on a quarterly basis. We also reviewed NTSB data  involving runway incursions and excursions from 2008 through June 2011  and summarized OSHA data on fatalities in the ramp area from 2001 through 2010. We used statistical models to assess the association  between safety incidents and the concentration of general aviatio operations and the implementation of the Airport Surface Detection  Equipment, Model X (ASDE-X) surface surveillance system from fiscal  year 2001 through April 2011. These models estimated how the number  of incursions changed after airports installed ASDE-X or increased the  proportion of operations involving general aviation. The models accoun for other factors that may contribute to incursions, such as long-term  weather patterns, runway layouts, as well as controller and pilot  experience. See appendix III for more information about the methods and  results of these analyses. To assess the reliability of FAA data, we (1)  reviewed internal FAA documents about its collection, entry, and  maintenance of the data and (2) interviewed FAA officials who were  knowledgeable about the content and limitations of these data. Both  NTSB and OSHA provided information about the reliability of their  excursion and fatality data, respectively. We determined that these data  t  were sufficiently reliable for the descriptive and comparative analyses  used in this report.
We conducted this performance audit from February 2011 to October  2011 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the aud it to  obtain sufficient, appropriate evidence to provide a reasonable basis  for  our findings and conclusions based on our audit objectives. We believe  the evidence obtained provides a reasonable basis for our findings  and  conclusions based on our audit objectives.
section_title: Appendix II: Technologies to Improve Runway Safety
paragraphs: Researching, testing, and deploying new technology is a major part of  FAA’s risk-reduction strategy. A number of available technological  systems are intended to help reduce the number and severity of runway  incursions. For example, to give air traffic controllers better visibility of  activity on the airfield and help prevent collisions, FAA has installed the  ASDE-X system at 35 major airports, while the Airport Surface Detection  Equipment, Model 3 (ASDE-3) radar and the Airport Movement Area  Safety System (AMASS) provide surface surveillance at 9 additional  airports. Runway status lights, which will be installed at 23 airports, are a  fully automatic series of lights that give pilots a visible warning when  runways are not clear to enter, cross, or depart on. To mitigate the risks  posed by runway excursions, FAA conducted research that led to the  development of the Engineered Materials Arresting System (EMAS), a  bed of crushable concrete designed to stop aircraft from overrunning  runway areas. As of July 2011, EMAS has been installed at 52 runways  at 36 airports, and there are plans to install 11 EMAS systems at 7 others.  According to FAA officials, EMAS has successfully arrested seven  overrunning aircraft with no fatalities or serious injuries and little damage  to the aircraft to date. (See table 4 for a brief description of technologies  designed to improve runway safety.)
section_title: Appendix III: Technical Appendix
paragraphs: This appendix summarizes our statistical analysis of the  between the ASDE-X airfield surface surveillance system and runway  ine focused on ASDE-X, among many other runway safety cursions. W programs, because of its potential for persuasive impact evaluation. We  describe how the process FAA used to install ASDE-X created a “quasi- experiment,” which allows u as to compare how incursions changed at  sirports that received the technology relative to airports that kept the  cursions. W rograms, because of its potential for persuasive impact evaluation. We  escribe how the process FAA used to install ASDE-X created a “quasi- xperiment,” which allows u s to compare how incursions changed at  irports that received the technology relative to airports that kept the  tatus quo.
subsections: section_title: Evalu ASDE-X on Runway Incursions
paragraphs: A a s lo c m p th  key goal of ASDE-X is to make air traffic controllers more aware of  ctivities on taxiways and runways in order to avoid collisions. The  ystem consists of airfield radar and sensors that collect data on the  cation of aircraft and vehicles. Computers transform these data in ontinuously updated maps of the airfield, which are displayed on color  onitors in air traffic control towers. The system warns controllers of  otential collisions—which may draw attention to possible incursions— rough visual and audible alarms.
AA used a selective and staggered process to install ASDE-X at 34  irports from 2003 through 2011 (of 35 airports slated to receive this  ystem). The variation among airports receiving the technology, as well  F a s as the times when they received i evaluation. This type of analysis compares the change in incursions over  time at airports that installed ASDE- nX with the change at airports that did  t, allows for a quasi-experimental  valuation. This type of analysis compares the change in incursions over  me at airports that installed ASDE- X with the change at airports that did  ot receive the technology, also known as a “difference-in-difference.” alysis of site-specific safety and efficiency benefits as compared to site-specific costs. airports provide a plausible comparison group for analysis, but we use variety of other comparison groups to ensure that our findings are robu st.
Many factors that may contribute to incursions are controlled in our  analysis here. We control for variation among airports in runway and  taxiway layouts, markings, and lighting, in addition to long-term variati in weather, air traffic, and pilot and controller skills. The staggered  installation of ASDE-X makes bias due to short-term weather conditions  or pilot and controller experience unlikely, because these factors would  need to be correlated with 34 installation times throughout the country. In  addition, the staggered installation lets us control for factors that affect all  airports equally, such as changes in training and procedures made  throughout the country at the same time.
section_title: Data Sources
paragraphs: The time period of our analysis spans fiscal years 2001 through April  2011. We assembled data on the number of incursions that occurred pe month at each FAA-towered airport  in this period, along with data on air  traffic control tower operations. The latter data included the number of  monthly tower operations at each airport, as well as the mixture of  commercial and general aviation operations. The operations data ide the population of interest, including the many smaller airports with no  incursions that do not appear in the incursion data.
FAA provided the installation dates and locations for the ASDE-X, runway status lights, FAROS, and low-cost ground surveillance systems. We  used these data to identify whether each technology was installed for  each airport and month between fiscal year 2001 and April 2011.
section_title: Statistical Modeling
paragraphs: We used a statistical model to estimate the association between ASDE-X and the number of incursions for airport i and month t. The model took the  form of  E(Yit | a,p,tit,xit) = a exp(δp + atit + xitβ),  where Yit randomly varies according to the Poisson distribution, a is vector of airport fixed effects, p  is a vector of year-month fixed effects, tit indicates whether ASDE-X was operational at airport i in month t, xit are  other time-varying covariates, and δ, α, and β are vectors of parameters.  We estimated the change in incursions after the installation of ASDE-X  using one contemporaneous, before-and-after parameter, α, becaus ASDE-X likely has an immediate effect on incursions once it has been  installed that does not change over time. ,4 The covariates xit  included the number of air traffic control tower  operations (to measure exposure and variation in the nature of activity  across airports), indicators for having 25 to 60 percent and greater than  60 percent of operations involving general aviation (excluding 0 to 25  percent), and indicators for having the runway status lights, FAROS, or  low-cost ground surveillance systems installed at airport i and time t.
T airport. The 485 airports in the panel ensure that these estimates will be accurate  approximations, even if incursions are not Poisson-distributed. As Wooldridge 2003 (674- 675) notes, conditional ML estimators consistently estimate the parameters of a fixed  effects model, even with arbitrary forms of over- and under-dispersion, heteroskedasticity,  and serial correlation. As a result, we can safely use the Poisson conditional likelihood to  estimate the parameters while using cluster-robust standard errors.
The models used several groups to compare the change in incursions  before and after ASDE-X was installed, in order to assess the sensitivity  of our results to plausible alternatives. The groups included  1.  all FAA-towered airports;  2.  airports that were included in the FAA benefit-cost analysis above b did not receive ASDE-X;  3.  the top 100 airports in tower operations from fiscal year 2001 through  4.  airports that had a similar ground surveillance system, ASDE- 3/AMASS, installed prior to the first installation of ASDE-X (baseline 5.  airports that did not have ASDE-3/AMASS installed at baseline; and  6.  airports that did not have ASDE-3/AMASS installed at baseline and  that were among the top 100 airports in tower operations from fiscal  year 2001 through April 2011.
section_title: Runway Incursions Rise after the Installation of ASDE-X, but Better Reporting May Explain the Change
paragraphs: Figure 18 plots the average monthly incursion rate for airports that did  and did not receive ASDE-X, rescaled to a ratio of the over-time mean to  better express the trends. The smooth lines summarize the average  incursion rate for each group and month using nonparametric locally  weighted regression models. The vertical lines show the ASDE-X  installation times for each airport.
Prior to the first installation of ASDE-X, the incursion rate changed in  roughly the same ways for the ASDE-X and comparison airports. As FAA  began to install the system in late 2003, and the incursion rate began to  increase for the ASDE-X airports, but it decreased and then increased at  a slower rate for the comparison airports. Substituting the other  comparison groups in these plots produces similar patterns.  Consequently, the raw data suggest that reported incursions increased at  airports that received ASDE-X, as compared to the change at airports t did not receive the system.
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Heather MacLeod, Assistant  Director; Russ Burnett; Martha Chow; Dave Hooper; Delwen Jones; Molly  Laster; Brooke Leary; Josh Ormond; and Jeff Tessin made key  contributions to this report.