Aviation Safety: Better Data and Targeted FAA Efforts Needed to Identify and Address Safety Issues of Small Air Cargo Carriers
Released: 2009-06-24T00:00:00

Letter

Background

The U.S. economy depends on the air cargo industry for the delivery of small, time-sensitive packages under 100 pounds, freight of 100 pounds or more, and mail. Air cargo carriers fall into two distinct categories: (1) all- cargo carriers that transport only cargo and (2) passenger carriers that transport cargo as belly freight in passenger aircraft. For the most part, all- cargo carriers can be categorized according to three business models: (1) large carriers, such as United Parcel Service (UPS) and FedEx, which operate large narrow-body and wide-body aircraft under part 121 of federal aviation regulations; (2) feeder carriers, which operate midsize and small aircraft (e.g., Cessna Caravans, Mitsubishi MU-2B-60s) under part 135 or part 121 on regularly scheduled flights in support of large cargo carriers; and (3) ad hoc carriers, which operate small aircraft (e.g., Cessna 401s, Beech Bonanzas) under part 135 and are individually contracted to haul cargo out of smaller airports while not necessarily operating on a regular schedule. Throughout this report, we use the term “small carriers” when referring to both feeder carriers and ad hoc carriers. Some carriers operate under both part 121 and part 135, and one large carrier leases aircraft to small carriers to provide feeder operations. (See fig. 1 for an illustration of a large carrier feeder-ad hoc relationship.) FAA estimated that as of May 6, 2009, the large all-cargo fleet contained 471 narrow-body and 593 wide-body aircraft and the small carrier all-cargo fleet contained 1,515 aircraft.

Several federal transportation agencies play significant roles in air cargo safety. These agencies are FAA, the Department of Transportation’s (DOT) Pipeline and Hazardous Materials Safety Administration (PHMSA), and NTSB.

Two FAA offices in particular have important responsibility for the oversight of air cargo carriers. First, FAA’s Flight Standards Service oversees cargo carrier operations conducted under parts 121 and 135. For each carrier, Flight Standards assembles a team of inspectors (known as a “certificate management team”) led by principal inspectors who focus on avionics, maintenance, or operations. For large carriers, dedicated teams of inspectors use the risk-based Air Transport Oversight System (ATOS) to carry out their duties. Under ATOS, inspectors develop surveillance plans for each carrier based on data analysis and risk assessment, and adjust the plans periodically in accordance with inspection results. For feeder and ad hoc carriers operating under part 135, inspectors—who unlike ATOS’s dedicated inspection teams may be assigned to multiple air carrier and other certificates—use the National Flight Standards Work Program Guidelines (NPG) to ensure that carriers comply with safety regulations. For NPG, Flight Standards annually identifies a minimum set of required inspections to be undertaken. In addition, individual inspectors determine annual sets of planned inspections based on their knowledge of and experience with the carriers they oversee. Second, FAA’s Office of Security and Hazardous Materials enforces hazardous materials (hazmat) safety policies and conducts annual inspections of cargo and passenger carriers. Inspectors in this office work exclusively on issues related to compliance with hazmat requirements. When violations of statutory and regulatory requirements are identified, FAA has a variety of enforcement tools at its disposal with which to respond, including administrative and legal sanctions.

In addition, PHMSA ensures the safe transport of hazmat by air and other modes. PHMSA promulgates regulations concerning the types and amounts of hazmat that can or cannot be transported by air—often differentiating between what hazmat can be carried on all-cargo versus passenger aircraft—and maintains its own database of hazmat incidents as well as a portal that pulls together hazmat data from other databases.

NTSB investigates and determines a probable cause for each U.S. aviation accident, which is defined as “an occurrence associated with the operation of an aircraft which takes place between the time any person boards the aircraft with the intention of flight and all such persons have disembarked, and in which any person suffers death or serious injury, or in which the aircraft receives substantial damage.” NTSB makes transportation safety recommendations to federal, state, and local agencies and private organizations to reduce the likelihood of recurrences of transportation accidents but has no authority to enforce its recommendations. NTSB also conducts annual reviews of aircraft accident data and determines U.S. aviation accident and fatal accident rates. NTSB also periodically holds public hearings and forums, and issues special studies on various transportation safety topics.

Air Cargo Accidents and Fatal Accidents Have Decreased, but Small Carriers Have Accounted for Higher Proportions of Both

From 1997 through 2008, air cargo accidents and fatal accidents each declined by about two-thirds. Despite this decline, small cargo carriers consistently experienced the largest shares of accidents and, especially, fatal accidents.

Air Cargo Accidents and Fatal Accidents Declined from 1997 through 2008

Annually, air cargo accidents decreased 63 percent, from 62 in 1997 to 23 in 2008. Average annual air cargo accidents declined from the first to the second half of our review period, from an average of 45 accidents per year from 1997 through 2002 to an average of 28 accidents per year from 2003 through 2008. Fatal air cargo accidents also decreased over our 12-year review period, falling from 12 in 1997 to 4 in 2008. In addition, from the first to the second half of our review period, fatal cargo accidents dropped from an average of 10 per year to an average of 6 per year. The fluctuation in annual air cargo accidents could be the result of a number of factors, including the general decline in aviation activity after September 11, 2001; a fluctuation in overall U.S. aviation accidents; and other factors. (See fig. 2.)

Ad hoc carriers experienced the largest decline in accidents, with 28 fewer accidents—dropping from 36 to 8—followed by large carriers, with 9 fewer accidents—dropping from 14 to 5 from 1997 through 2008. Feeder carrier accidents fluctuated during this period, reaching a high of 17 accidents in 2003 compared to a low of 7 accidents both in 1999 and in 2007. We do not know why the spike occurred in 2003. Large carriers had 3 fatal accidents during our review period, which occurred in 1997, 2000, and 2004. (See fig. 3.) Without actual data on the number of flight hours, however, we cannot determine an accident rate, and thus we do not know if the decline in ad hoc carrier accidents represents a better safety record for that sector of the air cargo industry.

Small Cargo Carriers Had Higher Proportions of Accidents and Nearly All Fatal Accidents

The small carriers (feeders and ad hoc) in our review experienced 79 percent of the air cargo accidents. Ad hoc carriers accounted for about half of accidents while the feeders were involved in over a quarter of them. (See fig. 4.) Feeder and ad hoc carriers averaged 29 accidents per year while large carriers averaged 8 accidents each year.

Small air cargo carriers accounted for 96 percent of the fatal air cargo accidents that occurred from 1997 through 2008. Ad hoc carriers accounted for the majority of fatal accidents and feeders for over one-third of fatal accidents. (See fig. 5.) Together, feeder and ad hoc carriers averaged 8 fatal accidents per year while large cargo carriers experienced a total of 3 fatal accidents from 1997 through 2008.

Accident Rates Have Fluctuated for Large Cargo Carriers but Are Not Available for Small On- demand Cargo Carriers

The accident rate per departure for large air cargo carriers has fluctuated over the last 25 years, but the overall trend has been downward, and in 2007 was roughly the same as for passenger carriers. It is possible to calculate these rates because FAA requires those carriers to report operational data (e.g., flight hours or departures).

However, FAA does not require small on-demand carriers operating under part 135 to report operational information, and the majority of feeder and ad hoc cargo carriers fall into this group. In 2003, NTSB recommended that FAA collect this type of data and, according to NTSB, FAA is still reviewing the costs and benefits as well as options for collecting and processing the data. However, the lack of data about the flight hours for small on-demand carriers precludes calculation of the industry’s current accident or fatality rates or changes in the rates over time, making it difficult to determine whether the industry is becoming more or less prone to accidents. Instead, FAA relies on an annual survey of aircraft owners to form the basis for estimates of small carrier operations, but this survey does not distinguish between passenger and cargo operations, making it impossible to use the survey estimates to calculate cargo or passenger accident rates for on- demand operations or for the cargo industry as a whole.

Even though operational data are not available for small air cargo carriers, their fatal accident rates would exceed those of large carriers for the latter part of our review period. From 2005 through 2008, there were no fatal accidents among large cargo carriers, so their fatal accident rate for those years was zero. However, there were 17 fatal feeder and ad hoc accidents over the same period, meaning that their fatal accident rate, if it could be determined, would be higher than zero—though it is unclear how much higher. This logic would not hold for air cargo accidents in general because large carriers had accidents in each year from 1997 through 2008, though they had fewer accidents than the feeder and ad hoc carriers. The lack of data makes it difficult for FAA and industry to target further improvements to the areas with the highest risk.

Pilot Performance, Accumulated Risk, and Other Factors Have Contributed to Air Cargo Accidents

Our review of NTSB and FAA air cargo accident and incident data as well as our interviews with industry officials and analyses of industry documents revealed that pilot performance was a prominent factor in air cargo accidents. Additionally, we concluded that accumulated risk, challenging operating conditions in Alaska, and undeclared hazmat were also prominent contributors to air cargo accidents.

Pilot Performance Was a Major Contributor to Air Cargo Accidents from 1997 through 2008

Our review of NTSB reports for 417 completed air cargo accident investigations found that pilot performance was cited as the probable cause for about 59 percent of them. Specifically, we found that NTSB cited pilot performance as the probable cause for about 53 percent of non- fatal and about 80 percent of fatal air cargo accidents. (See fig. 6.) Examples of pilot performance issues in these accidents included the pilot’s failure to maintain control of the aircraft or to execute the appropriate procedure. Our review determined that the second most prominent cause of air cargo accidents was some type of equipment equipment failure or malfunction. failure or malfunction.

Pilots of small cargo aircraft have fewer human and other resources available to them to help avoid mistakes or recover from unexpected circumstances. Typically, there is no second pilot to share in the pilot’s many duties and help respond to emergencies. Eighty-one percent of the fatal air cargo accidents from 1997 through 2008 were single-pilot flights. The lack of a second pilot coupled with the many duties of a single pilot merits a mention of the issue of pilot fatigue. Although NTSB indicated fatigue as a contributing factor—not a probable cause—in just 4 of the 443 accidents in our data, 12 of 27 experts we surveyed ranked pilot fatigue as one of the three most serious challenges to safe air cargo operations. The view of the experts is not out of line with the accident record to the extent that the concern about pilot fatigue has led to vigilance in identifying and addressing fatigue issues.

Further compounding the lack of pilot resources, cargo aircraft operated under part 135 are not required to have on-board safety technology such as a traffic collision and avoidance system, a terrain awareness and warning system, or an autopilot, which could aid a single pilot in monitoring the environment or responding to changing weather conditions. Most of these systems are required for small passenger aircraft that also operate under part 135.

Additionally, small cargo aircraft may fly into airports where FAA does not provide air traffic control services at all hours and the airports offer fewer services than might be required for passenger operations. For example, at the Bethel, Alaska, airport—the transportation hub for the remote villages in the area and the third-busiest airport in Alaska—FAA provides air traffic control services from 7 a.m. to 8 p.m. from November to March and 2 hours later in other months, and the airport clears its runway of snow and staffs its aircraft rescue and fire-fighting equipment only during operations of passenger aircraft with more than 30 passenger seats.

Our Analysis Indicates That Most Fatal Accidents Involved Multiple Risk Factors

We analyzed NTSB reports of the 93 fatal air cargo accidents that occurred from 1997 through 2008 using FAA’s Flight Risk Assessment Tool and identified three or more risk factors in 63 of the accidents and four or more risk factors in 41 accidents. FAA’s tool is located in appendix III and includes 38 risk factors— in the areas of pilot qualifications and experience, operating environment, and equipment—each with an assigned value ranging from 2 to 5, with 5 indicating the highest risk. While we do not know how the presence of these risk factors differs from their occurrence during normal operations, the experts told us that the unrecognized accumulation of multiple risk factors can create a potentially dangerous situation. One 1997 fatal accident, which NTSB attributed to the pilot’s disregard of the preflight weather briefing for severe weather, involved six risk factors. The pilot had not flown a minimum number of hours during the previous 90 days, had not accumulated a minimum amount of experience flying the aircraft type involved in the accident, and was flying solo. Additionally, the pilot encountered severe turbulence and icing during the night flight. Table 1 lists the five most common risk factors we identified in the 93 fatal air cargo flights.

Alaska’s Challenging Operating Conditions Factored Prominently in Air Cargo Accidents

With 18 fatal air cargo accidents from 1997 through 2008, Alaska led all states in this statistic because aviation operations in that state face several unique challenges. Alaska is more dependent on aviation for the transport of goods and people than other states because it lacks a comprehensive road system. Less than 10 percent of the state is accessible by road. Therefore, goods must be transported to remote villages via air or barge, and barge transport is not an option during the winter months. These factors make Alaska highly dependent on cargo aircraft, which often fly into poorly maintained airports that often do not meet FAA standards. Consequently, most of the accidents in Alaska involved small aircraft. Alaska is also subject to unusual weather conditions. Taken together, we believe these challenges render Alaska more susceptible to aviation accidents and fatal accidents than other states.

Of the 27 experts we surveyed, 5 ranked operating conditions in Alaska as one of the top three challenges to the safe operation of cargo flights; in addition, 12 experts in our panel indicated that Alaskan operating conditions do pose at least moderate challenges to safety, whereas 2 experts (a pilot of large aircraft and a government official) said these conditions were not a challenge. Seven experts said they did not have enough specific knowledge to judge the degree of challenge that Alaskan operating conditions pose to safety.

Some Experts Indicated that One of the Most Serious Challenges to Safe Air Cargo Operations Is Undeclared Hazmat, but Few Accidents Have Been Attributed to Hazardous Cargo

Very few of the cargo accidents occurring from year to year were conclusively caused by hazmat. However, 11 experts on our panel ranked undeclared hazmat—materials not noted as hazardous in shipping documents and/or labeled as such on their packaging—-among the greatest challenges to safe cargo operations, second only to pilot fatigue. According to our review of the NTSB accident data, only three cargo accidents involved hazmat. They occurred in 1997, 1998, and 2006 to large carriers, and none resulted in fatalities. The 2006 accident resulted in an NTSB hearing and recommendations that we discuss later in this report.

The problem of undeclared hazmat was cited primarily by government and large carrier experts in our survey. Specifically, three of the four government experts and two of the three large carrier experts cited it as the most serious challenge to air cargo safety. These opinions may stem from the previously cited fires and the relative rarity of a destroyed aircraft among large carriers, as well as government concerns about the transport of lithium batteries on aircraft.

FAA hazmat officials told us that undeclared shipments of hazardous materials represent the biggest challenge they face and that lithium batteries are the most challenging type of hazmat in air transportation. We reported in January 2003 that FAA, in the early 1990s, identified a number of incidents associated with batteries, particularly lithium batteries, aboard aircraft in which the batteries caused fires, smoke, or extreme heat. In response to these and other concerns, DOT took a number of actions designed to strengthen the regulations for the transportation of lithium batteries. In January 2008, NTSB noted that lithium batteries had been involved in at least 9 aviation incidents, and both primary and secondary lithium batteries are regulated as hazardous materials for the purposes of transportation. In December 2007, NTSB made six recommendations to PHMSA following a UPS aircraft fire at Philadelphia International Airport in February 2006, in which a number of secondary lithium batteries were found in the accident debris. The recommendations included requirements for transporting primary lithium batteries in fire-resistant containers and stowing cargo containing secondary lithium batteries in crew-accessible locations so that any fire hazards can be quickly addressed. These recommendations remain open with acceptable responses from PHMSA because they have not yet been implemented, but according to NTSB, actions are planned that, if satisfactorily completed, may comply with the safety recommendations. FAA and PHMSA have embarked on a lithium battery action plan, which aims to reduce the risk associated with the transport of batteries on aircraft by passengers and as cargo. The primary focus of this plan is all types of lithium batteries. According to DOT, PHMSA and FAA have also initiated a rule- making project to consider additional measures to enhance the safety of lithium battery shipments such as packaging, hazard communication, and stowage requirements. PHMSA plans to publish a notice of proposed rulemaking by December 2009. FAA, the Air Line Pilots Association, and PHMSA also issued safety alerts or advisories in 2007 that addressed smoke and fire hazards, recommended crew actions in the event of a battery fire, noted the availability of guidance for the safe transport of batteries and battery-powered devices on board aircraft, and provided information on proper packing and handling procedures for these batteries.

Large Numbers of Air Cargo Incidents May Be Precursors to Accidents

Although our analysis for this study included 443 air cargo accidents, cargo carriers were involved in more than twice as many incidents during the first 11 years of our review period, and FAA and others recognize that incidents are potential precursors to more serious accidents. In an analysis of air cargo data for 1997 through 2007 from FAA’s Accident/Incident Data System, we identified over 900 air cargo incidents. These incidents covered a broad set of events, such as an engine losing power at 7,000 feet; a cargo door opening in flight; and an aircraft engine coming into contact with a fuel truck. FAA does not use incident data to identify precursors to aviation accidents, because the data were not developed for this purpose. However, the agency is moving toward using data to better identify precursors to accidents, but until it does so, it may be missing opportunities to make air cargo operations and aviation, in general, safer. For example, from 2000 to 2007, one ad hoc cargo carrier was listed in FAA’s database 10 times with incidents that resulted in varying degrees of damage to its aircraft, from none to substantial, and in NTSB’s database with one non-fatal accident. This carrier subsequently experienced a fatal accident in 2008. Had this carrier’s incident data been used to identify accident precursors, inspectors might have been alerted to underlying problems that might have been addressed, potentially preventing the subsequent fatal accident. In addition, NTSB’s accident database also does not track incidents in a way that would allow empirical analysis.

The notion that incidents can be precursors to more serious accidents is accepted both inside and outside aviation. NASA’s Aviation Safety Reporting System collects, analyzes, and responds to aviation safety incident reports voluntarily submitted by pilots and others to lessen the likelihood of aviation accidents. In its 2005 Safety Management Manual, the International Civil Aviation Organization noted that for accidents, there are precursors evident before the accident, and focusing solely on instances of serious injury or significant damage is a wasted opportunity, since the factors contributing to such accidents may be present in hundreds of incidents. The International Civil Aviation Organization further noted that “effective safety management requires that staff and management identify and analyze hazards before they result in accidents,” particularly since there is the opportunity “to identify why the incidents occurred and, equally, how the defenses in place prevented them from becoming accidents.” The National Academy of Engineering undertook an accident precursor project in February 2003, which culminated in a report that included aviation accident precursor analysis and management. The report concluded that existing initiatives are not as effective as they could be and encourages government agencies that regulate high-hazard industries to increase their support of research into methods for effectively analyzing and managing precursors.

FAA Safety Improvement Efforts Focus Primarily on Large Carriers and, According to Experts, Vary in Effectiveness

Many government and industry efforts to improve safety focus primarily on large carriers. Such efforts include programs in which carriers and employees voluntarily disclose potential safety issues, attempts by carriers to institutionalize their safety procedures through safety management systems (SMS)—a proactive, risk-based approach to addressing potential hazards—and FAA’s ATOS oversight program even though there is nothing intrinsic to preclude these concepts from being implemented among small cargo carriers. Cargo experts view voluntary disclosure programs, efforts by associations to improve their members’ safety procedures, and carrier- implemented SMSs as the most effective current safety programs affecting air cargo (see fig. 7).

Experts Rate Voluntary Disclosure Programs as One of the Most Effective Cargo Safety Programs, but Primarily Large Carriers Participate

The intent of voluntary disclosure programs is to identify and correct safety problems in a nonpunitive way and to provide additional safety information to FAA. Our panel of experts ranked FAA’s voluntary disclosure programs as the most effective current program for improving air cargo safety. Specifically, 16 of 27 experts ranked FAA’s voluntary disclosure programs as one of the most effective current efforts to improve air cargo safety, and all experts able to judge indicated that the programs were effective on some level. FAA operates multiple voluntary disclosure programs, which use different data sources to help identify safety deficiencies. The three major ones are Flight Operations Quality Assurance (FOQA), the Aviation Safety Action Program (ASAP), and the Voluntary Disclosure Reporting Program (VDRP). At the current time, FOQA is used only by large carriers because of the level of technology required and ASAP is not typically used by small carriers.

FOQA collects and makes available for analysis digital flight data generated during the normal operations of the 23 participating carriers. As of January 2008, the only cargo carriers participating in FOQA were large carriers. Participating carriers pay for the special flight data recorders that can record FOQA data; these recorders cost approximately $20,000 each. Although such an investment can be expensive for some air carriers, some aircraft models come with the data recorder already built in. However, smaller carriers tend to operate older aircraft, which lack the data recorder equipment.

ASAP encourages industry employees to report safety information that may be critical in identifying potential precursors to accidents. Under this program, employees of air cargo carriers and other participating entities report safety events, which a committee that includes the carrier, the employee labor group, and FAA then reviews and determines appropriate corrective actions, such as remedial training. FAA agrees not to pursue enforcement actions for safety violations reported exclusively under this program. As of December 2008, 73 carriers participated in ASAP, including 8 cargo carriers. Seven of these 8 cargo carriers were large carriers, possibly because large carriers are more likely than small carriers to have the time and resources required for participation. Officials from an ad hoc carrier we interviewed said it was not practical for their carrier to enter into an agreement with FAA and then organize meetings to discuss disclosures when the carrier could operate an informal safety issue disclosure program internally.

VDRP encourages regulated entities, such as air carriers or repair stations, to voluntarily report instances of regulatory noncompliance. FAA does not take legal action on VDRP disclosures, but a violation with the same root cause can be reported only once by a carrier. Cargo carriers of all types we interviewed indicated that they have participated or would participate in VDRP. However, a 2008 DOT Blue Ribbon Panel found FAA does not routinely analyze VDRP data to identify trends and patterns that could indicate safety risks. FAA noted in commenting on a draft of this report that it began conducting regular analysis of VDRP data in January 2009 and that it modified the VDRP data software system and associated guidance to enable the identification of national trends of disclosures that represent the highest risk to safety.

Numerous Other Initiatives Including Industry Association Efforts Are Aimed at Improving Safety for Carriers of All Sizes; Some Experts Believe These Are among the Most Effective at Improving Air Cargo Safety

Numerous industry efforts to improve safety are aimed at different sectors of the air cargo industry. These include efforts by membership associations to improve safety among their members, which 14 of the 27 experts on our panel ranked among the top three most effective current efforts to improve air cargo safety—second only to voluntary disclosure. In addition, all experts in a position to comment indicated that these association efforts were at least slightly effective, and 11 of those experts indicated that they were greatly effective. We were unable to identify a central clearinghouse for these association efforts, but the ones we did identify fell into the following six general categories: establishing SMSs, providing fatigue awareness training, providing pilot skills training, adding on-board safety systems, improving flight risk assessments, and providing cargo-specific aircraft rescue and fire-fighting training. One expert said that membership-based efforts are often the most effective because they directly reflect the voluntary priorities of the membership organization and are often directly tailored to the group’s specific needs. Examples of industry efforts, including some by industry associations and one joint industry-federal effort, follow.

The Regional Air Cargo Carriers Association—an association of primarily feeder cargo carriers—has developed an SMS template tailored to the needs of smaller, feeder cargo carriers. A Regional Air Cargo Carriers Association official said that the organization’s members found FAA’s SMS guidance appropriate for large carriers with safety departments, but less useful for feeder or ad hoc carriers that might have only a few employees. SMSs are considered in the international community to be an important way to improve safety in aviation operations and are required by the International Civil Aviation Authority. In commenting on a draft of this report, FAA indicated that it has not endorsed the Regional Air Cargo Carriers Association’s SMS program, nor does FAA believe it is consistent with FAA training and program guidelines for SMS. SMS is discussed in more detail later in this report.

The National Air Transportation Association (NATA)—an association primarily of general aviation service companies whose members include some feeder and ad hoc carriers—has tailored flight risk assessments to the needs of its members by automating much of the assessment process. NATA officials said that feeder or ad hoc cargo carriers, like many general aviation operators, do not have the support personnel that larger carriers have to help with preflight checklists and other tasks, and that risk assessments would just add to those tasks unless they were largely automated. NTSB has recommended that a segment of the part 135 community—emergency medical services—utilize flight risk assessments before accepting flights.

In some cases, large carriers act as membership associations by helping their feeder network acquire safety enhancements. For example, the Federal Express feeder program, which helps finance on-board safety enhancements on feeder aircraft, has reduced the number of accidents among its feeder network according to Federal Express officials.

Since 2002, Alaska’s Medallion Foundation—a federally-funded, safety promotion organization that is overseen by FAA—offers training for part 135 pilots and has developed safety audits that can lead to carrier certifications in various areas, such as operational risk management, maintenance and ground service, and internal evaluation, to improve air transportation safety in Alaska. According to FAA, it has also approved a modified protocol for ASAP administered centrally by the Medallion Foundation in order to increase the feasibility of ASAP for small operators in Alaska. Meetings with FAA personnel to review reports submitted under the program are conducted by telephone, and all reports are tracked in a central database according to the agency. DOT said that the modified ASAP protocol with the Medallion Foundation has worked well for enabling small, remotely situated operators in Alaska to participate in the program.

The Commercial Aviation Safety Team, a joint FAA-industry effort, has developed an integrated, data-driven strategy to reduce the commercial aviation fatality risk in the United States and promote new government and industry safety initiatives throughout the world. The Team has completed work on 40 of its 65 safety enhancements aimed at eliminating accident causes. Six of the safety enhancements specifically target cargo operations, and each of them is still under way as of the Team’s most recent update in May 2007.

The Dallas-Fort Worth International Airport is developing a curriculum and a cargo-specific aircraft rescue and fire-fighting training course. Airport officials said that the course will focus on the unique challenges and approaches to fighting a cargo fire. For example, airport officials said that many aircraft rescue and fire-fighting teams treat passenger/cargo fires and cargo fires similarly when they should be treated differently, because passenger/cargo fires can involve hundreds of people whereas cargo fires typically only endanger the flight crew. The course will also provide hands-on training in the use of cargo-specific fire-fighting tools, such as hull-penetrating tools or devices for locating hot spots—tools that most fire fighters rarely use.

Carrier-Based SMSs Are Viewed as Effective by Experts but Are Currently Geared toward Large Carriers

Thirteen cargo experts ranked carrier SMSs as one of the most effective current efforts to improve air cargo safety, making this the third most frequently cited current safety effort. SMSs can differ in their specifics, but FAA defines SMS as a proactive, risk-based approach to addressing potential hazards by categorizing the risk level and taking appropriate mitigating actions to reduce the risk to an acceptable level. Some countries, such as Canada, require carriers to implement an SMS, but the United States does not yet require domestic carriers to do so. FAA and industry officials agreed that FAA will require part 121 carriers to implement SMSs in the next few years. In addition, FAA has issued guidance on developing an SMS, but none of the air cargo carriers we interviewed have implemented one. However, several of the air cargo carriers we interviewed said they had safety programs that are similar to SMSs. An expert in aviation safety said that effective SMSs are good for institutionalizing safety improvements and taking proactive steps to reduce the number of accidents. He further noted that larger companies with more airplanes and more resources are better positioned to do this. By contrast, companies with one airplane and one pilot will not have enough staff time to submit the paper work. As a result, the Regional Air Cargo Carriers Association—a membership organization for feeder carriers—has developed simplified SMS guidance specifically for part 135 cargo carriers. Some airports are also implementing SMSs. Most experts in our panel did not rank airport SMSs among the most effective current efforts, possibly because cargo airports have not implemented them nationwide.

FAA Uses Airworthiness Directives and Operations Specifications to Improve Aviation Safety

FAA uses airworthiness directives (AD) and operations specifications to improve aviation safety. An AD is a notification to owners and operators of aircraft that a particular model of aircraft, engine, avionics, or other system has a known safety deficiency that must be corrected. Carriers are prohibited from operating any aircraft that is out of compliance with any applicable AD. Operations specifications are specific limits and requirements developed for individual operators, such as the specific aircraft the carrier is allowed to operate. Ten experts on our panel ranked ADs and operations specifications among the three most effective current efforts to improve air cargo safety. Our interviews with carriers showed that some carriers depend on ADs and operations specifications to learn about safety issues that other carriers have discovered. An official from one feeder carrier said that ADs are like product recalls, and without them, she would never know that there was a problem until there was an accident. FAA also uses other methods for communicating with carriers, such as informational messages, alerts, advisory circulars, and seminars. However, none of the 27 experts on our panel indicated that FAA informational materials and seminars were any more than moderately effective at improving the safety of air cargo operations.

FAA Uses Risk-Based Inspection System for Large Carriers Only, and Experts Did Not Rate FAA’s Oversight as One of the Most Effective Efforts to Improve Air Cargo Safety

FAA oversees the compliance with safety regulations of all 58 part 121 cargo large carriers by using ATOS, which applies a risk-based inspection system tailored to each carrier regulated under part 121. For example, under ATOS, principal inspectors develop surveillance plans for each airline based on data analysis and risk assessment, and adjust the plans periodically to reflect inspection results. Under ATOS, principal inspectors are assigned to just one part 121 carrier. Our interviews with part 121 carriers and FAA inspectors revealed mixed opinions about ATOS. Some of the carriers, particularly the smaller part 121 carriers, indicated that transitioning to ATOS was too complicated and costly and that its focus on administrative reviews has reduced the number of on-site FAA inspections they receive. In addition, some FAA inspectors said that the ATOS paperwork is time consuming and can have the effect of tethering them to their computers. For example, officials from a small part 121 air carrier said that they had to hire a full-time person to work on implementing ATOS as well as spend over $500,000 to hire a company to help revise the carrier’s manuals to satisfy FAA requirements under ATOS. However, officials from a large carrier indicated that ATOS is a more effective oversight system than NPG once it is fully implemented. In addition, some officials from large carriers said that the bureaucratic nature of ATOS limited the amount of direct oversight they receive. FAA officials said that ATOS does not impose new requirements on carriers, and FAA does not require carriers to set up an ATOS program. FAA officials also said that because ATOS is more robust than the oversight system it replaced— NPG—inspectors may find omissions in manuals that were overlooked before. Carriers may then be required to correct these deficiencies to meet regulatory requirements. In addition, FAA officials said they are exploring options for reducing the time inspectors spend at their computers and increasing the time they spend doing hands-on inspections. For example, FAA officials said that FAA is reducing the number of ATOS program elements in order to make inspection planning and management easier.

The 303 part 135 carriers remain under the NPG system, which requires all active carriers to be inspected at least once a year and sets the numbers of required inspections nationally and planned inspections at the local level. Some FAA inspectors we interviewed who use the NPG system said they do not base their planned inspections on risk factors but, rather, on what was done the previous year or what they have time to do (NPG inspectors typically oversee several carriers). FAA officials said that FAA is moving toward a risk-based oversight “Safety Assurance System” for part 135 carriers. FAA has completed a gap analysis that compared the existing part 135 oversight system to the system requirements of the new system, and it plans to have the risk-based system developed for part 135 carriers by 2013.

The number of inspections that part 135 carriers receive each year varies greatly under NPG. For fiscal years 2004 through 2007, we found 18 part 135 carriers received one inspection in a year, while 6 part 135 carriers received hundreds of inspections in a year. An FAA official said that variation should be expected. For example, carriers with more airplanes, airplanes of more types, and routes to more regions will receive more required inspections under NPG.

The number of part 135 carriers that each cargo inspector oversees also varies greatly. On the low end (bottom 10 percent), part 135 cargo inspectors oversee 5 passenger and cargo carriers on average, and on the high end (top 10 percent), inspectors oversee 34 passenger and cargo carriers. While some variation is unavoidable, FAA officials said that FAA has not established guidelines to ensure that the workload is balanced among inspectors. The National Academy of Sciences also found FAA had inadequate staffing standards for its safety inspectors.

The majority of the experts on our panel did not rank FAA oversight and inspections among the most effective current efforts to improve air cargo safety. Although 5 of the 27 experts rated FAA oversight and inspections among the three most effective current efforts to improve air cargo safety, none of the experts representing carriers’ perspectives listed FAA oversight among the most effective current efforts. Officials from an ad hoc cargo carrier said that FAA inspectors do not have enough specific knowledge of cargo operations to effectively oversee cargo operations. Two FAA cargo inspectors that also oversee passenger carriers said that their formal cargo oversight training consisted of a 2-hour online course. FAA officials said part 135 does not differentiate between passenger and cargo operations. However, FAA officials recognized that inspectors that oversee part 135 ad hoc operations may benefit from additional training and are revising an existing multiday course for maintenance inspectors to address cargo operations. Our analysis of oversight data for part 135 carriers showed that cargo inspectors also oversee passenger carriers, often in larger numbers. This could limit inspectors’ ability to focus on cargo-specific issues. Officials from several cargo carriers of different types said that FAA inspectors do not do enough on-site inspections to effectively find and correct safety problems. For example, FAA inspectors and carrier officials said that part 135 inspectors often focus on administrative reviews. FAA officials said there needs to be an appropriate balance of on-site inspections and administrative reviews, both of which are important for determining carrier compliance and ensuring safe operations. Ultimately, however, FAA officials said that regulatory compliance is an air carrier responsibility; FAA is responsible for ensuring that air carriers are capable of complying and, in some cases, administrative reviews may be the best way for FAA to do that.

FAA Rarely Responds to Cargo Carrier Regulatory Violations with Legal Action or Fines

FAA’s oversight includes enforcement efforts, which are designed to promote compliance with statutory and regulatory requirements for aviation safety. When violations are identified, an FAA order calls for inspectors to take the actions most appropriate to achieve future compliance. These actions range from educational and remedial efforts, to administrative actions (such as warning notices), to punitive legal sanctions (such as fines or loss of operation certificate). Violations can be identified by FAA inspectors or by others, such as air traffic controllers or state or local government officials. The relevant FAA inspector prepares a report and recommends an enforcement action. That report and proposed enforcement action are then reviewed and possibly changed at various levels depending on the nature of the recommended enforcement action.

FAA closes most air cargo regulatory violations with administrative action, such as a warning notice, or without taking any action. For 1997 through 2008, over half (56 percent) of the 6,564 enforcement actions against cargo carriers were administrative, and another 17 percent involved no action. These were very similar to the percentages for passenger carriers. Within cargo, the ad hoc sector had the lowest percentage of legal actions. Fourteen percent of ad hoc carrier enforcement actions were legal, compared with 24 percent of larger carrier enforcement actions and 16 percent of feeder carrier enforcement actions. Ad hoc cargo carriers also had the largest reductions, on average, among cargo carrier types, with the reduced fines 64 percent below the initially recommended fines. Our previous work found that FAA reduced legal actions for several reasons, including proof that the violator took corrective action to prevent a reoccurrence of the violation or economic hardship that might accrue to the entity that caused the violation.

The percentages of cargo carrier violation cases closed with administrative or no action represent a continuation of trends that we observed in our last work on FAA enforcement in 2004. At that time, we found that FAA generally closes cases against passenger and cargo carriers with administrative actions, and reducing the amounts of fines may reduce the deterrent effects of those actions. Since then, our analysis of FAA’s Enforcement Information System data shows that the share of violations resolved using administrative or no action has increased slightly. Moreover, FAA still lacks information on how these actions have influenced the effectiveness of the enforcement actions, and the recommendation we made in that report—that FAA develop a process for measuring the performance and effectiveness of its enforcement actions— remains open.

Some NTSB Recommendations for Improving Cargo Safety Have Not Been Implemented, and the Experts Did Not Consider These Recommendations among the Most Effective Measures

NTSB investigates transportation accidents, including air cargo accidents, and makes recommendations to improve safety. NTSB made numerous recommendations based on air cargo accidents but not all of those were related specifically to cargo issues. For example, after an air cargo accident in 1997, NTSB recommended that FAA require all part 121 air carriers to include additional information and training to flight crews in order to avoid that type of accident in the future. Other NTSB recommendations, however, are cargo-specific recommendations, and all of them remain open. Table 2 summarizes the status of NTSB’s cargo- specific recommendations.

Most experts did not rank NTSB recommendations among the top efforts to improve air cargo safety. One expert on our panel described NTSB recommendations as the most effective effort to improve air cargo safety, and another 8 of the experts in our panel ranked NTSB recommendations as one of the top three efforts, but 18 experts did not include them among the top three efforts. According to our interviews, NTSB recommendations related to cargo operations are not always practical to implement. For example, one carrier noted that NTSB’s recommendation related to aircraft fire-fighting information (cited above) is challenging to implement because there are many cargo aircraft configurations, and even knowledge of the configuration involved in the incident that prompted the recommendation would not have helped firefighters respond to the incident. Despite these reservations, the carrier indicated that it is helping to implement the recommendation because NTSB believes it would improve safety.

NTSB also holds public hearings and meetings on topics of particular interest to transportation safety personnel. For example, it held a forum on air cargo safety in 2004, but no explicit recommendations emerged from the forum. Five experts on our panel ranked NTSB’s meetings as one of the three most effective current air cargo safety efforts and none of the experts ranked the meetings as the most effective effort.

Experts Believe Additional Improvements to Air Cargo Safety Could Be Achieved through Better Technology, Data, and Standards

The 27 experts in our panel commented on numerous additional steps that could further improve air cargo safety. Experts in our panel ranked installing state-of-the-art on-board safety systems on all cargo aircraft and tracking part 135 operations as the potential measures that would most improve air cargo safety (see fig. 8).

Experts Believe Improved On-Board Safety Technology Would Most Improve the Safety of Air Cargo Operations

Adding state-of-the-art on-board safety technology was the potential measure to improve air cargo safety endorsed by the most experts in our panel. Better on-board technology, particularly for smaller aircraft, could provide additional tools to better inform pilots’ judgment and decision making. One expert said that the type of technology needed depends on the type of carrier. He noted that large carriers often already have state-of- the-art technology, and that keeping pace with new technologies as they emerge is the challenge for them. However, he said that feeder and ad hoc air cargo carriers are not required to have certain on-board technologies, and that they would benefit from installing better situational awareness technologies, like Traffic Alert and Collision Avoidance Systems (TCAS) or Automatic Dependant Surveillance Broadcast (ADS-B), on their aircraft. TCAS monitors warn the pilot of potential collision dangers, and ADS-B uses satellite-based technology to broadcast aircraft identification, position, and speed with once-per-second updates. Other experts on the panel also indicated that TCAS and ADS-B would most improve safety on the smaller aircraft that lack it.

FAA’s Capstone Program in Alaska has shown that better technology can reduce aircraft accidents. As described earlier, Alaska’s challenging operating conditions factored prominently in air cargo accidents over the last decade. The Capstone Program funded technology upgrades that provide pilots with information on terrain, weather, and air traffic. FAA’s goal was to reduce Alaska’s higher-than-average aviation accident rate. FAA has stated that an independent study found that, from 2000 through 2004, accidents for Capstone-equipped aircraft were reduced by 47 percent.

However, the experts on our panel were not unanimous about the potential for improving safety through better on-board technologies. The experts who represent the part 135 perspective did not rate improving on- board safety systems as highly as the other experts did. Only one of the six part 135 pilots or carriers we surveyed ranked this as one of the top three potential improvements. The other five indicated that improving on-board safety systems is not feasible or would only slightly improve air cargo safety. For example, some part 135 ad hoc and feeder carriers we interviewed indicated that state-of-the-art on-board safety systems are not affordable relative to the value of the aircraft. Specifically, officials from one ad hoc cargo carrier said that traffic collision avoidance systems (such as TCAS or ADS-B) installed on a Cessna, valued at $100,000 to $200,000, would make the biggest improvement in safety, but such a system would cost about $25,000 to install on each aircraft, which the officials said was not practical.

Tracking Part 135 Cargo Operations Ranked Second among Top Steps toward Improving Air Cargo Safety among Experts

As stated earlier, we were unable to determine accident rates for small feeder or ad hoc cargo carriers because FAA does not track part 135 operations. However, operational data—such as flight hours or landings— for on-demand part 135 cargo operations would allow analysts to determine accident rates for all cargo carriers as they currently can do for all part 121 (large) carriers. This is important because a higher proportion of air cargo accidents and nearly all fatalities occur to part 135 (small) carriers. Many experts who responded to our survey indicated that better data on part 135 cargo flight and operations could improve air cargo safety. Specifically, tracking these data was ranked among the top measures with the greatest potential by the second largest share of experts in our panel.

Industry experts and officials we spoke with said there is a need for FAA to have this type of data. For example, NTSB has recommended in 2003 that FAA collect additional operational data from small air carriers in order to generate accident and incident rate information for all sectors of commercial aviation, including air cargo, but the recommendation remains open because an FAA official said that FAA chose not to collect the information. In addition, numerous industry stakeholders told us that not having these data precludes FAA from effectively targeting its safety initiatives. One of the experts on our panel said that people might assume that the part 135 carriers with the most accidents are the ones with the poorest safety records, but that may not be the case when the number of operations is considered.

One official from a part 135 carrier said that the industry is generally against greater reporting because it would increase workload. However, he stated that if FAA begins requiring such data, companies will find a way to comply because they already collect the data. We interviewed part 135 carriers of various sizes, and they all indicated that they already track operational data internally and could report these data to FAA without a substantial additional effort, but this was not designed to be a representative sample.

Pilot and Carrier Experts Are Divided on How Aligning Cargo Regulations with Passenger Regulations Would Affect Air Cargo Safety

As discussed earlier in this report, the regulations under which most large cargo carriers typically operate (supplemental) differ from the regulations under which most passenger carriers operate (domestic or flag). Although part 121 pilot experts on our panel indicated that aligning cargo regulations with passenger carrier regulations would improve safety, carrier experts generally disagreed. All three pilots, who fly under part 121, ranked alignment of regulations as the top potential measure to improve air cargo safety. As an example, the Air Line Pilots Association, the employee organization for most commercial U.S. pilots, supports the alignment of duty time regulations for all part 121 carriers. The Association believes that longer flight times can increase pilot fatigue and thus increase mistakes and accidents, and, as stated earlier in this report, the experts rated pilot fatigue as a serious challenge to safe cargo operations. On the other hand, none of the seven carrier experts ranked aligning cargo regulations with passenger regulations among their top three potential measures for improving air cargo safety. Five of them indicated that aligning regulations would have a slight or no improvement, and the other two indicated that aligning regulations would not be feasible.

Although not specifically related to cargo operations, NTSB has also recommended that FAA revisit its time and duty regulations as they may be related to fatigue. For example, in 2008, it recommended that FAA develop guidance for operators to use in establishing fatigue management systems and then continually assess the effectiveness of the systems. It also recommended in 1995 that FAA review its flight and duty time regulations to include the findings of fatigue and sleep research. These recommendations remain open because FAA has not completed its actions related to these issues. In commenting on a draft of this report, FAA said that it convened a committee to address pilot fatigue issues and that the data it collects may be used in future rule-making efforts regarding pilot flight time, duty, and rest regulations.

Conducting Flight Risk Assessments before a Flight Can Help Reduce Accumulated Risk and, According to Experts, Improve Safety

As described earlier, many air cargo accidents over the last 10 years occurred in conditions of accumulated risk—when several risk elements were present, but none was individually significant enough to result in the flight’s cancellation. FAA, the Flight Safety Foundation, and NATA have each developed tools that pilots or carriers could voluntarily use to assess accumulated risk factors and determine if the flight should go forward. These tools assign values to various risk elements, such as single pilot operations, night flights, and flights into areas without accurate weather reports. See appendix III for FAA’s sample flight risk assessment tool.

Eight of the 27 experts on our cargo safety panel ranked flight risk assessment as one of the three potential efforts that could most improve safety. Additionally, 18 experts indicated that incorporating flight risk assessment checklists into air cargo daily operations would have a moderate or great effect on improving the safety of their operations. One expert on our panel said that flight risk assessment cannot prevent all accidents and that not all flights with multiple accumulated risk factors have accidents, but assessing the risk factors may help pilots reduce the number of cargo accidents by recognizing when the accumulated risks become unacceptably high and at which point pilots would either find ways to mitigate those risk factors or delay the flight.

Despite the high level of support among our expert panelists for using flight risk assessment checklists, only 1 of the 10 carriers we interviewed used them in their daily operations. Officials from one part 135 carrier said that carriers are busy enough doing all the things that FAA requires to worry too much about ideas that might be very productive but are nonetheless optional. Other experts pointed out that all of the flight risk assessment checklists currently available were designed for passenger operations and that cargo carriers would have to tailor the tools to their needs—potentially a critical obstacle to implementation.

Conclusions

Aviation in the United States remains safe and air cargo accidents have declined over the last 10 years, although fatal accidents do occur every year. Most of those accidents and nearly all of the fatal accidents in the last decade have happened to feeder and ad hoc carriers. However, the lack of operational data for part 135 carriers, which make up the bulk of the feeder and ad hoc carriers, makes it impossible to determine accident and fatality rates for small carriers or to track cargo-wide accident or fatality rates over time. FAA’s information on small carrier operations is based on its annual survey of aircraft owners, which does not differentiate between passenger and cargo operations, making it impossible to use the survey results for cargo operators. While the numbers of accidents suggest that the fatality rates for feeder and ad hoc carriers are higher than the rates for large carriers, it is impossible to know how much higher the fatality rates are for feeder and ad hoc carriers without data on the numbers of operations for all types of cargo aircraft. It is also difficult for FAA and industry to target further safety improvements to the areas with the highest risk.

Despite the higher numbers of accidents and fatal accidents among small cargo carriers, FAA’s safety programs have focused primarily on large cargo carriers, the industry segment in which accidents and accident rates have steadily declined. While it makes sense to focus first on large carriers, which operate larger aircraft with larger crews and cargo holds, the safety of the smaller aircraft is also important. There is nothing intrinsic to small carriers that precludes risk-based oversight, voluntary disclosure programs, or the use of SMSs, but these efforts are usually targeted toward, or at least primarily used by, the large cargo carriers. However, cost is a concern for carriers, and poor economic conditions throughout the air cargo sector may mean that few funds will be available in the near term for new safety initiatives.

In addition, FAA has increasingly focused on potential accident precursors that would reduce the risk of accidents before a related accident even occurs. However, neither FAA nor NTSB systematically tracks incidents in a way that would allow empirical analysis, even though incidents are widely viewed as accident precursors. Over half of the fatal air cargo accidents since 1997 had multiple risk factors. However, preflight risk assessment checklists are not required to be used within the cargo industry. The concept of assessing and recognizing accumulated risk through flight risk assessment presents an additional low-cost opportunity for identifying and reducing the risk associated with some cargo flights that might otherwise go unnoticed.

Recommendations

To help FAA improve the data on and the safety of air cargo operations, we recommend that the Secretary of Transportation direct the FAA Administrator to take the following four actions: Gather comprehensive and accurate data on all part 135 cargo operations to gain a better understanding of air cargo accident rates and better target safety initiatives. This can be done by separating out cargo activity in FAA’s annual survey of aircraft owners or by requiring all part 135 cargo carriers to report operational data as part 121 carriers currently do.

Promote the increased use of safety programs by small (feeder and ad hoc) cargo carriers that use the principles underpinning SMS and voluntary self-disclosure programs.

Evaluate the likelihood that cargo incidents could be precursors to accidents and, if FAA determines they are, create a process for capturing incidents that would allow in-depth analysis of incidents to identify accident precursors related to specific carriers, locations, operations, and equipment.

Create incentives for cargo carriers to use flight risk assessment checklists in their daily operations, including tailoring a sample flight risk assessment checklist for part 135 cargo carriers.

Agency Comments

We provided copies of a draft of this report to DOT and NTSB for their review and comment. Both agencies provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to interested congressional committees, the Secretary of Transportation, and the Chairman of the National Transportation Safety Board. We are also making copies available to others on request. In addition, this report is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-2834 or dillinghamg@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Objective, Scope, and Methodology

Our objective in conducting this study was to review the nature and extent of safety issues in the air cargo industry and what the Federal Aviation Administration (FAA) and others are doing and could do to address them. To accomplish this objective, we established the following research questions: (1) What have been recent trends in air cargo safety? (2) What factors have contributed to air cargo accidents in recent years? (3) What have FAA and the industry done to improve air cargo safety, and how do experts view the effectiveness of these efforts? (4) What do experts say FAA and industry could do to further improve air cargo safety?

To determine trends in air cargo safety, we obtained and analyzed accident and incident data for calendar years 1997 through 2008.

From the National Transportation Safety Board (NTSB), we obtained accident data for part 121 and part 135 all-cargo and mail operations that occurred from January 1, 1997, through December 31, 2008. To capture the full extent of cargo operations in Alaska, however, we also included passenger/cargo accidents because Alaska’s by-pass mail system, which requires carriers to have a certain share of the passenger market to obtain a by-pass mail contract, resulted in fewer all-cargo carriers in that state. From these data we identified 443 fixed-wing aircraft accidents, including 93 fatal accidents. Six of the accidents involved 2 aircraft, so our data included a total of 449 accident aircraft.

From FAA, we obtained data on part 121 and part 135 fixed-wing all-cargo accidents and incidents that occurred from January 1, 1997, through December 31, 2007. From these data, we eliminated accidents that were also included in the NTSB data, for a total of 937 accidents and incidents. To avoid confusion when discussing the two data sets, we refer to the FAA data as “incidents” and the NTSB data as “accidents.”

Because we are familiar with and have previously determined that these data were sufficiently reliable for the nationwide descriptive and comparative analyses used in this report, we interviewed agency officials knowledgeable about the databases from which the data were derived to determine that the accident and incident data used in this report continue to be sufficiently reliable for the types of analyses we performed. We also obtained information on industry trends by conducting a literature search and reviewing the resulting documents, conducting a survey of air cargo experts, and interviewing officials and reviewing relevant documents from FAA, the Pipeline and Hazardous Materials Safety Administration (PHMSA), air cargo industry associations, air cargo carriers, airports, an employee group, and others.

We also conducted site visits to Alaska, Ohio, and Texas. Those locations were selected to be geographically diverse and as the states with the largest number of air cargo accidents or because of the relatively large number of air cargo carriers of various sizes located there.

To assess what factors have contributed to air cargo accidents in recent years, we conducted several analyses. First, to determine prominent accident causes, we analyzed data on probable causes and contributing factors from completed NTSB investigations of 417 air cargo accidents. Second, to assess accumulated risk, we applied FAA’s proposed flight risk assessment tool to NTSB’s reports on the 93 fatal cargo accidents that occurred during our review period. To do this, we searched each accident report for the 38 risk factors in the tool, such as “pilot flight time less than 100 hours in the last 90 days.” For each factor found, we noted its corresponding risk value on an Excel spreadsheet and tabulated the total score as well as the total number of risk factors for each fatal accident. Third, for indications of other factors contributing to air cargo accidents, we surveyed a panel of air cargo experts, which is discussed in more detail later below; analyzed documents and interviewed officials from FAA, PHMSA, NTSB, air cargo industry associations, air cargo carriers, airports, an employee group, and others; and conducted site visits to Alaska, Ohio, and Texas (see the previous paragraph).

To determine what FAA and the air cargo industry have done to improve safety, we interviewed FAA and industry officials, reviewed key documents, and analyzed FAA’s oversight and enforcement data for all- cargo carriers. We interviewed officials and tested the data and found it sufficiently reliable for our purposes. To obtain experts’ opinions about how FAA and the air cargo industry could further air cargo safety, we surveyed a panel of 27 air cargo safety experts. The experts rated and provided relative rankings on the effectiveness of current efforts to improve air cargo safety, the severity of safety challenges faced by the air cargo sector of aviation, and the potential improvements that additional efforts could have on air cargo safety. We selected the panel of experts with the assistance of the National Academy of Sciences to represent the perspectives of a cross-section of air cargo stakeholders. The specific experts, their affiliation, and their expert perspectives are listed below.

To develop our survey of air cargo experts, we reviewed existing studies about air cargo safety, including previous and ongoing GAO work, and interviewed air cargo safety stakeholders. GAO subject matter experts designed draft questionnaires in close collaboration with a social science survey specialist. We conducted pretests with four people knowledgeable in the field of air cargo (representatives from air carriers, airports, and air transportation associations) to help further refine our questions, develop new questions, clarify any ambiguous portions of the survey, and identify any potentially biased questions. These pretests were conducted in-person and by telephone. We worked with the National Academy of Sciences and internally to develop the panel of experts and obtain contact information.

We launched our Web-based survey on August 18, 2008, and received all responses by November 5, 2008. Log-in information to the Web-based survey was e-mailed to participants. We sent one follow-up e-mail message to all nonrespondents a week later, and contacted by telephone all those who had not completed the questionnaire within 3 weeks. We received responses from all 27 of our selected experts.

Because our survey was not a sample survey, there are no sampling errors; however, the practical difficulties of conducting any survey may introduce nonsampling errors. For example, differences in how a particular question is interpreted, the sources of information available to respondents, or the types of people who do not respond can introduce unwanted variability into the survey results. We included steps in both the data collection and data analysis stages for the purpose of minimizing such nonsampling errors. As indicated above, GAO subject matter experts collaborated with a social science survey specialist to design draft questionnaires, and versions of the questionnaire were pretested with four knowledgeable people in the air cargo field. From these pretests, we made revisions as necessary. We examined the survey results and performed computer analyses to identify inconsistencies and other indications of error. A second, independent analyst checked the accuracy of all computer analyses.

Appendix II: Expert Responses to GAO Survey

We worked with the National Academy of Sciences to identify air cargo experts that included carrier, pilot, airport, aircraft manufacturer, government, and human factors and safety performance perspectives. We sent our Web-based survey to 27 air cargo experts and received responses from all 27 experts. Our survey was composed of closed- and open-ended questions. In this appendix, we include all the survey questions and aggregate results of responses to the closed-ended questions; we do not provide information on responses provided to the open-ended questions. For a more detailed discussion of our survey methodology, see appendix I.

Q1. In your opinion, how effective, if at all, is each of the following current efforts to improve the safety of cargo-only flights?

Q2. Considering the above list of efforts (question 1 a-j), which three do you believe are the most effective in improving air cargo safety?

Q3. Besides the efforts listed above, do you know of any other current significant efforts to improve cargo-only aviation safety? If so, please explain.

Q4. In your opinion, how much of a challenge, if any, does each of the following issues pose to safely operating cargo-only flights?

Q5. Considering the above list of challenges (question 4 a-n), which three do you believe are the greatest challenges to air cargo safety? a. Pilot fatigue related to nighttime flying, ineffective rest periods, or commuting b. Carrier policies that, by their nature, give pilots economic incentive to fly in less-than-ideal conditions c. Flights scheduled with less than 4 hours of notice given to the crew d. Low piloting experience overall, as well as low piloting experience in the specific types of cargo aircraft operated f. Variation within the cargo-only sector regarding the priority given to safety g. Availability of aircraft rescue and fire-fighting services provided by persons with cargo-specific knowledge and training h. Difficult cargo-only flight and operating conditions (e.g., airports with limited technology, mountainous terrain, nighttime operations) i. Alaska's aviation and operating j.

Q6. Besides the challenges listed above, do you know of any other significant challenges to safe cargo-only air operations? If so, please explain.

Q7. In your opinion, how much improvement, if any, in the safety of cargo- only air operations would be provided if each of the following measures were implemented?

Q8. Considering the above list of possible measures (question 7 a-k), which three do you believe would provide the greatest improvement to air cargo safety? a. FAA increasing the amount of on-site inspections that it conducts training and knowledge of cargo-only air operations c. Carriers and flight schools providing better training for cargo-only pilots Transportation (DOT) or Transportation Security Administration (TSA) increasing shipper knowledge and declaration of hazardous materials e. DOT or TSA promoting better shipper compliance with rules for the handling and transport of declared hazardous materials standards for ramp employees involved in loading and unloading cargo g.

Industry setting uniform standards for ramp employees involved in loading and unloading cargo 3 incorporating flight risk assessment checklists into their daily operations i.

FAA collecting and using part 135 and part 91 cargo flight and operations data to better target safety efforts j.

Q9. Besides the possible measures listed above, do you have any other suggestions for significantly improving cargo-only aviation safety? If so, please explain.

Q10. Please provide any other comments that you have regarding cargo- only safety.

Appendix III: FAA’s Sample Flight Risk Assessment Tool

Appendix IV: GAO Contact and Staff Acknowledgments

Staff Acknowledgments

In addition to those named above, Teresa Spisak (Assistant Director), Richard D. Brown, Keith Cunningham, Elizabeth Eisenstadt, Michele Fejfar, David Hooper, Mitchell Karpman, Valerie Kasindi, Sara Ann Moessbauer, Susie Sachs, Christine San, Pamela Vines, and Crystal Wesco made key contributions to this report.