Aviation Safety: FAA Has Begun Efforts to Make Data More Publicly Available
Released: 1997-05-27T00:00:00

Letter

Background

FAA is responsible for setting standards, assessing compliance, and taking enforcement actions to ensure that the airlines meet safety standards. To carry out this responsibility, FAA monitors the airlines’ compliance with the Federal Aviation Regulations through periodic inspections. Those regulations set the standards for the airlines’ operations and maintenance functions.

A number of possible indicators of aviation safety exist. In a 1988 report, we identified and assessed potential ways of measuring the airlines’ performance in areas important to safety. The accident rate is a widely recognized measure of overall aviation safety. However, because accidents occur so infrequently, there are no statistically significant differences in the accident rates among similar airlines. Also, because accident rates reflect what has already happened, their relevance to accident prediction or prevention can be limited. Among the other measures discussed in that report were information on inspection results, unsafe incidents, airlines’ financial condition, pilots’ competence, and maintenance quality.

Safety-related aviation information varies in the extent to which it is available to the public. In general, “availability” indicates whether or not information is protected from dissemination by federal law. For example, the National Transportation Safety Board, the official source of information on airline accidents, routinely publishes information on aviation accidents. On the other hand, the public can obtain some other information only after making a request through the Freedom of Information Act (FOIA). According to FAA, information on the enforcement actions against regulated entities (i.e., air carriers, airports, manufacturers, schools, or repair stations) has generally been available to the public only through FOIA requests, or when FAA elects, on a case-by-case basis, to publicize an enforcement action.

FAA’s Actions to Provide More Aviation Safety Information to the Public

FAA began to take a number of actions to provide aviation safety-related information to the public in July 1996. The Administrator asked FAA’s Office of System Safety to assemble a working group of senior-level officials to determine how the FAA could most efficiently and effectively accomplish this task. In addition to FAA’s then-Deputy Administrator, the group included representatives from FAA’s offices of Regulation and Certification, Chief Counsel, Government and Industry Affairs, Civil Aviation Security, and Public Affairs.

FAA solicited comments from the public and from the aviation community on how best to educate the public about, and make information available on, commercial aviation safety. FAA contracted with a consultant to generate a discussion of and obtain feedback on the types of aviation safety data that FAA might make available to the public, the means by which such information might be distributed, and the issues and considerations that arise in the distribution of these data. The contractor’s draft report was made available for public comment through the Federal Register on November 13, 1996.

According to senior FAA officials, in deciding what means the agency would use to provide greater information to the public, FAA recognized the challenges of availability and accessibility. FAA noted the growing use of the Internet as an expedient and cost-effective means to provide information, especially to those in government, the aviation industry, academia, and the media. As a result, FAA announced on January 29, 1997, that it would use the Internet to pursue all three of its information strategies: establishing an aviation safety information web site linked to FAA’s Internet web site, publicizing significant enforcement actions, and undertaking a public education campaign on aviation safety.

However, because broad sections of the general public may not have access to the Internet, FAA recognized that it might need to distribute safety information through some other supplementary means. FAA considered using toll-free telephone numbers to provide the public with certain safety information. However, on the basis of the experience of the National Highway Traffic Safety Administration, FAA decided that it lacked the staff resources to answer the large number of calls that it might receive. FAA subsequently decided to provide information, at least initially, through other public channels. As the Internet information effort develops, FAA expects to reassess the need for toll-free telephone access.

FAA announced that beginning on February 1, 1997, it would issue press releases on newly issued enforcement actions concerning significant cases against regulated aviation entities that involve safety and security issues, including cases seeking civil penalties of $50,000 or more. As of April 16, FAA had issued press releases about three enforcement actions involving civil penalties, along with three instances in which it has revoked air carriers’ operating certificates. In addition to its normal procedures for issuing press releases, FAA has included them on its aviation safety information web site. FAA’s homepage is pictured in figure 1.

FAA began its public education campaign about aviation safety on April 2, 1997. On the basis of the public comments received on the consultant’s draft report, FAA determined that it needed to explore more effective ways of communicating with consumers about aviation safety. To complement its information-sharing efforts, FAA’s public education campaign is designed to help the public better understand the safety of the overall system. FAA prepared a short overview of the aviation safety system and included it on its aviation safety information web site.

In addition to the press release and public education information, the aviation safety information web site includes a link to a web site maintained by the FAA’s Office of System Safety, where the public can access and search several of the principal sources of aviation safety data and information that are used by the federal government. It also includes an explanation of how to use the data and cautions about how those calculations should and should not be interpreted. Figure 2 shows the information presented on the web site on aviation safety data.

FAA plans to make public various aviation safety-related databases over time. When it was first made available to the public, the web site included three aviation safety databases:

The NTSB Aviation Accident/Incident Database, which is the official repository of aviation accident data and causal factors. NTSB generally defines an “accident” as an occurrence associated with the operation of an aircraft in which individuals are killed or suffer serious injury, or the aircraft is substantially damaged. An NTSB-defined incident is an occurrence, other than an accident, associated with the operation of an aircraft that affects or could affect the safety of operations. The NTSB database contains only selected incident reports. As of April 9, 1997, this database included a total of 37,696 records of aviation accidents and incidents, dating back to 1983. By far, the vast majority (34,073, or approximately 90.4 percent) concerned general aviation aircraft accidents and incidents; 3,623 records (9.6 percent) concerned large or commuter air carriers’ accidents and incidents.

The NTSB’s safety recommendations to FAA with FAA’s responses. NTSB uses information it gathers during accident investigations and the determination of probable cause to make safety recommendations to all elements of the transportation industry. The recipient of a recommendation must respond formally to the recommendation and specify what action is or is not being taken and why. This database includes the 3,471 recommendations made by NTSB to FAA since 1963, along with FAA’s responses.

The FAA’s Incident Data System, which contains a more extensive collection of records of aviation incidents—potentially hazardous events that do not meet the aircraft damage or personal injury thresholds contained in NTSB’s definition of an accident. As of April 9, 1997, this database included a total of 67,057 records of aviation incidents, dating back to 1978. As with the NTSB’s Aviation Accident/Incident Database, a relatively small percentage (28.0 percent) of the total number of records concerned incidents experienced by large or commuter air carriers.

Users cannot readily retrieve complete copies of these three databases. Rather, users may browse (i.e., look at) individual records, count records (e.g., all accidents involving commuter air carriers during a given time period), or select particular reports on the basis of user-supplied words or phrases (e.g., smoke) and/or user-selected criteria, such as the aircraft’s category of operation.

FAA added another database on March 31, 1997, that provides the means by which the accident and incident information can be put into some context. FAA extracted this database—Airline Traffic Statistics—from information gathered by the Bureau of Transportation Statistics (BTS). It contains three selected measures of individual airlines’ operations: the number of departures, hours flown, and miles flown, by year, in domestic commercial service during the 5-year period from 1991 through 1995. Those statistics are the activity measures most frequently used to calculate accident and incident rates for the airlines. Unlike the first databases that FAA included on its web site, users cannot search the data on traffic statistics on the Internet. Users can, however, obtain a copy of this complete database from FAA’s web site, for use on their own computers.

FAA includes warnings and disclaimers to explain the limitations of the databases it includes on its web sites. In general, these warnings and disclaimers state that the contents of the web sites are unofficial. FAA notes that the databases may not be complete and makes no certification about the accuracy of the data.

Early Data Indicate That Public Demand for Aviation Safety Information on the Internet Has Grown

Since FAA first established its aviation safety web site on the Internet, it has seen an approximately fourfold increase in the number of users who have accessed the safety data web site each week. FAA’s computers measure usage in several ways, and each indicates that usage of FAA’s site has grown since it was made public. The best measure of web site usage, according to FAA officials, is the number of users who have accessed the site. Although FAA cannot identify every individual user who accesses its site, it does count the number of users that access the web site over a period of time using a measure called a “user session.” In mid-January, before FAA publicly announced the availability of the web site, it averaged about 2,000 user sessions per week, even though the web site consisted mainly of a page explaining that the data will be available at a later date. After media attention about the availability of the web site in late January, the usage that week grew to almost 9,000 user sessions. After declining over several weeks, usage again grew after FAA added the searchable safety data to the site on February 28. FAA hosted about 8,200 user sessions during the last week of March. Figure 3 illustrates the number of user sessions per week for the safety data web site.

In addition to an increase in the number of users, FAA’s data indicate that the public is utilizing the safety data web site more often than when it was first made available. First, FAA tracks the average time of each user session. The length of the average user session had grown to about 12 minutes in early April. Also, FAA tracks the number of times each user requests a file from FAA’s computers—called a “hit.” The average number of hits generated during each user session has also grown, from 18.6 in early January to as high as 31.9 in early April. (These data on the number of hits per user session are displayed in table 1.) According to FAA officials, the increases in both of these statistics indicate that users are finding the safety data more useful, possibly for research, than in the past. They added, however, that it is too early to tell if these trends will continue.

Finally, FAA’s computers also keep track of the host computer of each user who accesses the safety data site. The user’s host computer is operated by the organization that provides access to the Internet, whether that organization is an Internet service provider (such as America Online, Compuserve, or Netcom, that mainly serve the public) or another organization, such as Boeing. These data indicate that many of the host computers that access FAA’s site most frequently are operated by Internet service providers. Other frequent users are the Air Force and airlines such as Delta, which operate host computers that are generally available only to their employees. These same data indicate that about 10 percent of those who access FAA’s site are doing so from a computer located outside the United States—mostly from Germany and Canada.

Because FAA made the safety education material available only recently (on April 2, 1997), it has only limited information on the number of user sessions for that web site: FAA recorded 313 user sessions on that web site for the week ending April 5 and 515 for the week ending April 12. For the press releases on enforcement actions, however, FAA’s statistics indicate that weekly usage has generally fallen since FAA first made those press releases available, and fewer users have accessed this page than have accessed the safety data web site. Figure 4 shows the change in the number of weekly uses of the press release information since early February 1997.

FAA’s Plans to Expand Information About Aviation Safety on Its Web Site

FAA plans to add other safety-related information to the web site gradually over time. By May 31, 1997, FAA plans to add data from the FAA National Airspace Incident Monitoring System, which includes information on near mid-air collisions. On June 1, 1997, FAA expects to make available a quarterly report of enforcement actions in the safety and security areas against aviation entities. This report, which describes enforcement actions closed with a civil penalty or orders of certificate suspension or revocation, will cover the first quarter of 1997. Thereafter, FAA expects to issue its quarterly enforcement reports about 30 days after the end of each quarter.

FAA has also indicated that it will expand the available information on airline traffic statistics in two ways. First, it will add data for 1996 as soon as it receives them from BTS in June or July. In addition, FAA expects to add traffic statistics for commuter airlines. At present, the traffic statistics that FAA has posted are limited to ones on domestic operations by large air carriers (i.e., generally those that operate aircraft with more than 60 seats). In addition, by the end of September 1997, FAA will develop a new database that will provide certain basic information about each air carrier, such as the number of specific makes, the models, and the ages of the aircraft flown by the carrier and the date when the carrier was certificated by the Department of Transportation (DOT) and FAA to operate. According to FAA officials, the agency has not yet decided how much data should be provided from the existing FAA databases or whether some information could be better provided by the individual air carriers, perhaps in conjunction with their trade associations, through direct links between their respective Internet web sites and FAA’s.

According to FAA officials, the agency also intends to evaluate its efforts to provide safety information to the public, but not until March 1998, after the web site has been in operation for approximately 1 year.

GAO’s Observations on FAA’s Progress to Date

In an October 1996 report on aviation safety, we concluded that the time had come for FAA to begin the process that can lead to publishing airline-specific safety data. The report recommended that the Secretary of Transportation instruct the Administrator of FAA to study the feasibility of developing measurable criteria for what constitutes aviation safety, including those airline-specific, safety-related performance measures that could be published for use by the traveling public. DOT concurred with that recommendation.

FAA’s Internet web site represents a good first step toward providing the public with some aviation safety information. Providing the information in which FAA has the greatest confidence—NTSB’s accident/incident data, FAA’s incident data, and BTS’ traffic data—seems to be a reasonable approach. The early data on the usage of the web site indicate that the public has an interest in aviation safety data.

FAA has said that evaluating its efforts will be an important aspect of its overall strategy of providing more information to the traveling public. We agree. Such an evaluation could help FAA determine whether it is meeting the needs of the traveling public and whether it should improve, refine, or expand its safety information, as well as improve the quality of the underlying data. It might also incorporate considerations of the extent to which the public finds these data easily usable, in view of the complexity and size of the posted databases. While it is too early to conduct an evaluation, FAA could begin the planning necessary to ensure that its evaluation produces meaningful results.

Agency Comments

We provided DOT and FAA with copies of a draft of this report. We met with DOT and FAA officials, including the Manager of FAA’s Safety Data Services Division, acting on behalf of the Deputy Assistant Administrator for System Safety. DOT and FAA officials agreed with the draft report’s overall message and provided editorial and technical comments that we incorporated as appropriate.

The information in this report was developed through discussions with officials at FAA and analysis of data on the usage of FAA’s web site over time. We also reviewed previously issued GAO products, pertinent federal regulations, and FAA’s Internet web sites. We did not independently assess the quality of the data that FAA includes on its Internet web sites. We performed our review from March through mid-April 1997 in accordance with generally accepted government auditing standards.

As you requested, unless you publicly announce its contents earlier, we plan no further distribution of this report for 30 days. We will then send copies to the Secretary of Transportation; the Director, Office of Management and Budget; and other interested parties. We will also make copies available to others upon request.

Major contributors to this report were Thomas Kai; Steve Martin; and James Sweetman, Jr. Please call me at (202) 512-3650 if you or your staff have further questions.

Gerald L. Dillingham Associate Director, Transportation Issues The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.