For decades, FAA, other federal regulators, and the aviation industry have used data in a reactive fashion—that is, to identify the causes of aviation accidents and incidents and take actions to prevent their recurrence. Aviation accident data are collected by NTSB, but FAA also collects some accident data and uses various databases and voluntary reporting programs to collect incident data, such as for runway incursions—the unauthorized presence of an aircraft, vehicle, or person on a runway. FAA also gathers and analyzes data through its inspection and certification programs to ensure industry compliance with safety regulations. (App. I provides more information on the databases discussed in this statement.) Since 1998, FAA has partnered with the airline industry through the Commercial Aviation Safety Team (CAST) to identify precursors and contributing factors and ensure that efforts to improve safety focus on the most prevalent categories of accidents. CAST has reduced the risk in commercial aviation by focusing on areas such as controlled flight into terrain, loss of control, and runway incursions. CAST analyzes accident and incident data to identify precipitating conditions and causes, and then formulates an intervention strategy designed to reduce the likelihood of a recurrence. According to CAST, its work—along with new aircraft, regulations, and other activities—reduced the commercial aviation fatal accident rate by 83 percent from 1998 to 2008 and is an important aspect of FAA’s efforts to improve aviation safety by sharing and analyzing data. (Fig. 1 illustrates the number of fatal and nonfatal commercial air carrier accidents from 1998 through 2010.) Similarly, FAA analyzes data on incidents, such as those where there is a risk of a catastrophic accident. For example, data on runway incursions, other surface incidents, and airborne incidents are collected at airports that have air traffic control towers. FAA analyzes those data to categorize incidents according to the actions or inactions of air traffic controllers, pilots, or others, such as pedestrians or vehicle operators, and determines the severity of those incidents. These data are then used to assess the root causes of incidents to identify potential remedies. Using this process, FAA has taken steps to improve safety in the terminal area since 2007 and has both reduced the number of serious runway incursions—where collisions are narrowly avoided or where there was a significant potential for a collision—and undertaken successful efforts to increase the reporting of incidents. Although FAA will continue using data in a reactive manner to understand the causes of accidents and incidents, it is shifting emphasis to a proactive approach in which it analyzes data to identify and mitigate risks to prevent future accidents as part of its adoption of SMS. As a result, data that can be used to help identify accident and incident precursors— such as data on an incident that is voluntarily reported by pilots, air traffic controllers, or others to FAA or air carriers—have become more critical for accident prevention, according to the independent review of FAA’s safety oversight in 2008. CAST is also now moving beyond the forensic approach of examining past accident data to a more proactive approach that will focus on risk prediction and mitigation strategies and aims to reduce the U.S. commercial fatality risk by 50 percent from 2010 to 2025. FAA’s effort to integrate aviation safety data—the Aviation Safety Information Analysis and Sharing (ASIAS) system, which connects 46 safety databases across the industry and has 45 participating airlines—is integrated into the CAST process. ASIAS enables better safety information management and data sharing as it proactively extracts from public and non-public data sources, including accidents, incidents, and voluntary reporting. FAA has demonstrated the potential of using integrated safety data to better understand the causes of certain safety events and identify mitigating strategies. For example, FAA fused data from government and industry sources to identify underlying factors contributing to numerous nuisance warnings pilots were receiving from their terrain awareness warning systems (TAWS). By combining all of the data, FAA was able to identify needed changes in the way it handles air traffic as well as improvements in the design of the TAWS software. FAA also plans to use data proactively to model the impact of the Next Generation Air Transportation System (NextGen) on the safety of the national airspace system, to proactively identify risks that might emerge with the introduction of NextGen changes. Figure 2 illustrates the type of transition FAA plans as the agency shifts its emphasis to a proactive assessment of emerging safety risks. As part of its oversight system for commercial air carriers, FAA collects and analyzes data to ensure that the industry complies with safety regulations. FAA uses the Air Transportation Oversight System (ATOS), a risk-based data-driven system, to oversee maintenance and operations at all air carriers. Under the ATOS concept, FAA inspectors use data analysis to focus their inspections on areas that pose the greatest risk. ATOS also permits inspectors to shift the focus of their inspections in response to changing conditions within air carriers’ operations. In contrast, FAA’s oversight program for the remaining operators (i.e., air taxi, general aviation, etc.) focuses on inspectors completing a prescribed number of inspection activities annually and is primarily based on checking operator compliance with regulations. Implementing systems and processes that capture accurate and complete data is critical for FAA to determine the magnitude of safety issues, assess their potential impacts, identify their root causes, and effectively address and mitigate them. As such, FAA has various processes in place to help ensure data quality and is taking steps to address remaining weaknesses. For example, FAA established an agency-wide order on data management that specifies the roles and associated responsibilities for data management within the agency. This order applies to all sharable information from FAA and other sources used to perform the agency’s mission. In accordance with the data management order, FAA’s Office of Aviation Safety established a data management framework that includes a four- step process for importing data from other FAA offices and external sources. This process includes data acquisition—obtaining information from various data owners, data standardization—validating data by comparing a new data set with previous data sets to identify inconsistencies, data integration—translating data values into plain English and correcting data errors, and data loading—importing data into the agency’s own systems. FAA has furthermore put in place data quality controls that we consider good practices for handling data, although weaknesses remain in some areas. For example, FAA has developed training for users on data systems and restricted access to the data. The FAA databases we reviewed in 2010 also had at least some controls in place to ensure that erroneous data are identified, reported, and corrected. However, several of the databases lacked an important control in that managers do not review the data prior to entry into the system. This quality control is important because it could affect accuracy and completeness.controls in place and is taking steps to address its data weaknesses; FAA has however, vulnerabilities remain that potentially limit the usefulness of FAA’s data for some of the safety analyses planned to support SMS. In 2010, we made several recommendations to FAA to help improve and expand its capability to use data for aviation safety oversight. For example, we recommended that FAA extend standard quality controls, as appropriate, to the databases that support aviation safety oversight. Although FAA concurred with our recommendations, it has not fully implemented them. FAA has put in place various quality controls for its data, but it continues to experience data challenges—including limitations with the analysis it conducts and data it collects, as well as the absence of data in some areas. Some of these limitations hinder the agency’s ability to comprehensively and accurately assess and manage risk, as seen in the following examples: FAA’s changes to reporting policies impact its ability to accurately determine operational error trends. The rate and number of airborne operational errors—errors made by air traffic controllers—have increased considerably in recent years, with the rate nearly doubling for errors in the terminal area from 2008 to 2011. Multiple changes to reporting policies and processes during this time make it difficult to know the extent to which the recent increases in operational errors are due to more accurate reporting, an increase in the occurrence of safety incidents, or both. For example, FAA removed air traffic controller names from reports in the Air Traffic Quality Assurance (ATQA) database, which may encourage controllers to share more information about incidents. (See fig. 3.) Without determining the potential impact of these policy changes, FAA cannot ensure accurate and consistent measures of operational errors and cannot assess the risks posed over time. FAA believes that these changes likely caused the increases in operational errors but has not yet conducted an analysis to validate the linkage. Incomplete data on operational errors are assessed, making it difficult to account for all potential risk. Operational errors can be captured in multiple reporting systems. For instance, an air traffic controller’s failure to maintain minimum separation between two aircraft—a loss of separation—could be reported to the ATQA database by a supervisor and will also be captured automatically by airplane tracking technology—the Traffic Analysis and Review Program (TARP)—if it is in use at the relevant facility. However, FAA’s current process for analyzing data on losses of separation captured by these systems only assesses those incidents that occur between two or more radar-tracked aircraft. By excluding incidents such as those that occur between the aircraft and terrain or aircraft and protected airspace, FAA is not considering the systemic risks associated with many other airborne incidents. We recommended last year that FAA expand its current risk assessment process,that it will include these incidents in its risk assessment process before the end of 2013. Lack of coordination among data systems may affect FAA’s ability to conduct comprehensive data analyses. As previously mentioned, multiple programs and systems capture safety data. Some of these programs—including the Air Traffic Safety Action Program (ATSAP), ATQA, and the Risk Analysis Process (RAP) that considers ATQA and TARP data—also assign contributing factors to the incidents they review. (See fig. 4.)