United States General Accounting Office

GAO Report to Congressional Requesters

December 1997 AVIATION SAFETY

Efforts to Implement

Flight Operational

Quality Assurance

Programs

GAO/RCED-98-10

GAO United States

General Accounting Office

Washington, D.C. 20548

Resources, Community, and

Economic Development Division

B-275990 

December 2, 1997

The Honorable Wendell H. Ford

The Honorable Ron Wyden

United States Senate

The analysis of aircraft data recorded during flight has played a crucial

role in determining the causes of crashes. Recently, however, some U.S.

airlines have begun to analyze flight data from uneventful airline flights to

identify potential problems and correct them before they lead to accidents.

In your letter of December 2, 1996, you asked us to examine efforts by the

Federal Aviation Administration (FAA) and U.S. airlines to implement

Flight Operational Quality Assurance (FOQA) programs. The objective of a

FOQA program is to use flight data to detect technical flaws, unsafe

practices, or conditions outside of desired operating procedures early

enough to allow timely intervention to avert accidents or incidents. These

programs are voluntary efforts by airlines that involve equipping aircraft

with specialized devices to continuously record up to hundreds of

different flight data parameters from aircraft systems and sensors,

analyzing the data, identifying trends, and taking action to correct

potential problems. The analysis of flight data allows airlines to

reconstruct entire flights on the basis of the values over time of flight data

parameters, such as heading, altitude, throttle settings, ground speed, and

many others. Currently, about 33 foreign airlines and 4 U.S.

airlines—United, US Airways, Continental, and Alaska—have implemented

FOQA or FOQA-type programs.

You requested that we determine (1) how FOQA programs will enhance

aviation safety, (2) the costs and benefits of such programs, and (3) the

factors that could impede their full implementation and actions that could

be taken to overcome any impediments.

Results in Brief The early experience of domestic airlines with established Flight

Operational Quality Assurance programs, as well as the testimony of

foreign airlines with extensive experience in this area, attests to the

potential of such programs to enhance aviation safety by identifying

possible safety problems that could lead to accidents. Airlines have used

Flight Operational Quality Assurance programs to identify potential

problems that were previously unknown or only suspected. Where

potential problems were already known, airlines have used these

programs to confirm and quantify the extent of the problems. And most

Page 1 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

important, on the basis of analyses of flight data, airlines have taken

actions to correct problems and enhance aviation safety.

The costs associated with implementing a Flight Operational Quality

Assurance program depend upon a large number of factors, including the

technology used to capture flight data, the number and types of aircraft to

be equipped with this technology, and personnel costs. Although the

program is primarily viewed as a safety program, U.S. and foreign airlines

have reported financial benefits as well. With additional data on aircraft

systems and engine conditions, airlines are better able to achieve optimum

fuel consumption and avoid unneeded engine maintenance. Although more

difficult to quantify, enhanced safety should result in lower costs over time

as a result of accidents avoided and lower insurance premiums. FAA’s

preliminary estimates place the annual cost of a program with 50 aircraft

at approximately $760,000. Savings from reduced expenditures for fuel,

engine maintenance, and accident costs for a 50-aircraft program are

estimated at $1.65 million per year. FAA’s estimates suggest a net savings

from 50 aircraft of $892,000 per year.

The primary factor impeding the implementation of Flight Operational

Quality Assurance programs among the major domestic carriers is the

resolution of data protection issues. Airline managers and pilots raise

three significant data protection concerns: (1) use of the data for

enforcement/disciplinary purposes; (2) disclosure to the media and the

public under the provisions of the Freedom of Information Act; and

(3) disclosure through the civil litigation discovery process. FAA has taken

a number of actions that may resolve these issues, although it is not clear

whether the aviation community will be satisfied with FAA’s actions. First,

FAA has begun work on a rulemaking procedure to establish what

protections from enforcement actions, if any, will apply to information

submitted to FAA under a Flight Operational Quality Assurance program.

Second, on October 9, 1996, the Congress enacted legislation and FAA has

begun work on a rulemaking procedure that would prohibit the

Administrator from disclosing voluntarily submitted safety information

under certain circumstances. These actions may ameliorate concerns

about the Freedom of Information Act. And third, airlines currently seek to

protect voluntarily collected safety information from disclosure in civil

litigation on a case-by-case basis.

Background Modern commercial aircraft contain sophisticated electronic systems that

gather, process, and manage digital data on many aspects of flight. These

Page 2 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

data originate from various systems and sensors throughout the aircraft.

The data range from pilot operations to the outputs of sensors and

systems. Some of these data are continuously recorded by the aircraft’s

digital flight data recorder to help investigators understand what happened

if the aircraft is involved in an accident or a serious incident.1

 Designed to

survive crashes, flight data recorders typically retain the data recorded

during the last 25 hours of flight.

Rather than analyzing flight data only after an incident or accident, some

airlines routinely analyze the flight data from regular flights. Their aim is

to identify problems that occur in routine operations and to correct these

problems before they become accidents or incidents. In its 1992 study for

FAA,

2

 the Flight Safety Foundation coined the term “Flight Operational

Quality Assurance” to describe this function. The Foundation defined FOQA

as “a program for obtaining and analyzing data recorded in flight to

improve flight crew performance, air carrier training programs and

operating procedures, air traffic control procedures, airport maintenance

and design, and aircraft operations and design.”

FOQA has its origin in the use of flight data recorders3 as mandated by the

Civil Aeronautics Administration in 1958. Although the first flight data

recorders captured only six parameters,4

 they were a valuable tool for

reconstructing what had occurred preceding a crash. In addition to

recording data to assist in crash investigations, some airlines began to

monitor data recorded on routine flights. Initially, the monitoring systems

captured airworthiness data, but over time they have expanded to include

operational data. FOQA programs were first established in Europe and Asia,

and only within the past few years have some U.S. airlines begun adopting

such a system on a trial basis. At present, about 33 foreign airlines and 4

U.S. airlines—United, US Airways, Continental, and Alaska—have

implemented FOQA or FOQA-type programs. (See app. I for more detailed

background information on FOQA and U.S. airlines’ experience with these

programs; see app. II for a list of airlines worldwide that have

implemented FOQA programs.)

1

The National Transportation Safety Board, the official source of information on airline accidents,

defines accidents as events in which individuals are killed or suffer serious injury, or the aircraft is

substantially damaged. Incidents are defined as occurrences other than accidents associated with the

operation of an aircraft that affect or could affect the safety of operations. 49 C.F.R. 830.2.

2

Flight Safety Foundation, Air Carrier Voluntary Flight Operational Quality Assurance Program (1992).

3

The flight data recorder is commonly referred to as the “black box.”

4

The six required parameters were time, airspeed, heading, altitude, vertical acceleration, and time of

radio transmission.

Page 3 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

As part of FAA’s strategy to achieve significant reductions in aviation

accident rates despite the rapid increase in air travel anticipated over the

next decade, in 1995 the agency initiated a FOQA demonstration project to

promote the voluntary implementation of FOQA programs by U.S. airlines.

The objective of such a program is to use flight data to detect technical

flaws, unsafe practices, or conditions outside of desired operating

procedures early enough to allow timely intervention to avert accidents or

incidents. For example, identifying repeated instances of unstabilized

approaches to a particular airport could help to define a new approach

pattern less likely to lead to an accident under adverse conditions, or to

improved pilot training. Such a system has potentially broad application to

flight crews’ performance and training, aircraft operating procedures, air

traffic control procedures, aircraft maintenance, and airport design and

maintenance. Major airlines in Europe and Asia, as well as the U.S. airlines

that have FOQA programs, are uniform in their support of the program.

How FOQA Works. FOQA involves (1) capturing and analyzing flight data to

determine if the pilot, the aircraft’s systems, or the aircraft itself deviated

from typical operating norms; (2) identifying trends; and (3) taking action

to correct potential problems. Airlines with FOQA programs typically use a

device called a quick access recorder to capture flight data onto a

removable optical disk that facilitates the data’s frequent removal from the

aircraft.5

 Periodically, the optical disks are removed from the aircraft, and

the flight data are analyzed by the ground analysis system at a centralized

location. The data are analyzed by a computer system that evaluates about

40 to 80 predefined events for deviations from the airline’s specified

tolerance thresholds. For example, an event might be the descent rate

during approach. Deviations of more than certain predetermined values,

called “exceedances,” are flagged and evaluated by a monitoring team.

After investigating these exceedances to determine their validity and

analyzing them to understand possible causes, the monitoring team will

propose and evaluate corrective actions. Periodically, airlines aggregate

exceedances over time to determine and monitor trends. (For a more

complete discussion of FOQA operations, see app. I.)

The FOQA Demonstration Project. In July 1995, FAA initiated a 3-year,

$5.5 million demonstration project to facilitate the start-up of voluntary

airline FOQA programs and to assess the costs, benefits, and safety

enhancement associated with such programs. FAA provided hardware and

software to each of the three airlines—United, US Airways, and

5

These data typically include the parameters required to be collected on the aircraft’s flight data

recorder plus many more parameters. See app. I for more information on quick access recorders and

flight data recorders.

Page 4 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

Continental—that have implemented FOQA programs according to the

demonstration project’s requirements. FAA purchased quick access

recorders to equip 15 Boeing 737 aircraft at each of the three airlines. FAA

also purchased a ground analysis system—the computer hardware and

software for analyzing and visualizing FOQA data—for US Airways and

Continental. Because United already had purchased a ground analysis

system that analyzes these data for other types of aircraft, FAA purchased

for the airline the additional software needed to analyze FOQA data from

737s. For their part, these airlines funded the cost of obtaining

supplemental type certification6

 of the airborne equipment, the costs of

installation and maintenance, and the cost of personnel to run and monitor

the program. Alaska Airlines is the fourth U.S. airline to have begun a FOQA

program, but it has only recently met the demonstration project’s

requirement for an agreement on FOQA by the airline’s pilot union.

Consequently, the project has not yet provided any equipment to the

airline. Alaska Airlines, however, received quick access recorders and a

ground analysis system from the FAA Structural Loads Program and uses

this equipment to operate its FOQA program. (See app. III for more

information on the Structural Loads Program.) Other airlines that are

participating in the demonstration project and are considering the

implementation of a FOQA program are America West, Delta, Northwest,

Trans World, Southwest, Continental Express, and United Parcel Service.7

(See app. I for a detailed description of the FAA demonstration project.) As

a research and development effort of the FOQA initiative, FAA is developing

the Aviation Performance Measuring System, an advanced system for

conducting automated analysis and research on FOQA data. (See app. III for

a description of this system and FAA’s other related technical programs.)

Rather than requiring airlines to implement FOQA, FAA has chosen to

promote the initiative through a cooperative demonstration project in

partnership with the industry. According to the demonstration project’s

program manager, it would be premature for FAA to mandate FOQA at this

time because U.S. aviation is still in the early stages of developing FOQA

and is primarily in a learning mode. The program manager contends that a

mandated program would stifle innovation, encounter substantial

resistance from airlines and pilots, and most likely result in minimal

compliance. Thus, at present, FAA is working with the industry to raise

interest in the concept, facilitate the design and implementation of

6

An FAA type certificate is issued when an aircraft, aircraft engine, propeller, or appliance is properly

designed and manufactured, performs properly, and meets the regulations and minimum standards

prescribed by the Administrator. An FAA supplemental type certificate is required when there is a

change to an aircraft, aircraft engine, propeller, or appliance. 49 U.S.C. 44704.

7

Although not a participant in the demonstration project, American Airlines is considering the

implementation of an internal FOQA-type program.

Page 5 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

voluntary FOQA programs, provide financial and technical assistance, and

foster innovation.

FOQA Identifies

Potential Safety

Problems

The primary characteristic that distinguishes FOQA from other safety

reporting programs, such as the Aviation Safety Reporting Program or

Aviation Safety Action Programs,8

 is that FOQA provides objective,

quantitative data on what occurs during flights rather than what is

subjectively reported by individuals. Instead of needing to rely on

perceived problems or risks, FOQA yields precise information on many

aspects of flight operations, and this information can be used to help

objectively evaluate a wide range of safety-related issues.

U.S. and foreign airlines have reported on previously unknown or

suspected problems for which FOQA has provided objective information

that resulted in corrective actions. One airline found through its FOQA

program that more exceedances occurred during visual flying than during

instrument flying. This finding prompted the airline’s flight-training

managers to rethink the relative emphasis given visual and instrument

flying in the airline’s training programs. Another airline’s FOQA analysis

confirmed that the incidence of descent rate exceedances during

approaches was significantly higher at a particular runway at a U.S. airport

than at other runways. After investigating the problem, the airline

concluded that the air traffic control approach had been set too high,

requiring pilots to descend more steeply than usual during their final

approach. When the airline shared its findings with FAA management, the

approach was modified to correct this potential problem.

For landings, some airports’ air traffic control procedures require pilots to

approach high and fast and then descend steeply. These approaches can

result from a number of factors, including noise abatement rules, traffic

volume, terrain, or weather conditions. Although airline managers know

about the situations from pilots’ reports, FOQA gives them the quantitative

information to demonstrate the extent of this problem at the various

airports. With these data in hand, managers can be more effective in

addressing the problem and taking action to mitigate or eliminate risks.

FOQA can also help airlines determine the frequency of certain occurrences

rather than having to rely on human judgment, particularly for the level of

maintenance required. Two examples of these types of occurrences are

hard landings and exceedances in engine temperatures. Prior to FOQA,

8

See app. IV for a description of these programs.

Page 6 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

airlines generally relied on pilots’ judgment of the necessity for corrective

action if a hard landing occurred or an engine overheated. FOQA, however,

can provide better information on the amount of force the aircraft

experienced during a hard landing. Similarly, FOQA gives more data on the

engines’ temperatures and the duration of overheating in some aircraft

than were previously available without FOQA. With these data, managers

can make more informed decisions about whether the aircraft needs to be

inspected to check for structural damage or whether an engine needs to be

overhauled.

U.S. and foreign airlines have reported that they have used FOQA analysis to

identify a variety of potential safety problems and take corrective action to

resolve or mitigate them. These have included steep takeoffs, which can

damage the aircraft’s tail; approaches that are outside the prescribed

procedures for a “stabilized” approach; descent rates or bank angles that

are considered excessive; high taxi speeds; hard landings; wind shear

occurrences; ground proximity warnings; and engine malfunctions.

Corrective action can include notifying pilots of a change in standard

operating procedures or restating and emphasizing them, correcting an

equipment problem, or providing additional training. The continued

monitoring of trends will tell the airline if the corrective action has been

effective or whether additional measures are needed.

A number of airlines plan to complement the use of FOQA data with

information from safety reporting systems, such as Aviation Safety Action

Programs or internal pilot reporting systems. FOQA data, originating from

aircraft sensors and systems, tell “what” happened to the aircraft. Internal

safety reporting systems, based on reports of pilots, flight crews, and other

persons, are more likely to tell “why” something happened. Together,

information from FOQA and internal reporting systems can provide valuable

insight into current and emerging problems.

FOQA’s Potential

Costs and Benefits

Based on preliminary estimates from an ongoing cost-benefit study by

Universal Technical Resource Services, Inc. (UTRS), an FAA contractor,

table 1 summarizes the estimated annual costs for airlines to equip 15, 50,

and 100 aircraft with quick access recorders, purchase a ground analysis

system, and pay FOQA-related salaries.9

9

Because FAA’s cost-benefit study is in progress, we were not able to verify FAA’s estimates of FOQA

costs and savings. The cost and savings figures are preliminary and may change as more data are

gathered.

Page 7 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

Table 1: Estimated Total Annual Costs,

by Fleet Size 15 aircraft 50 aircraft 100 aircraft

Equipment costs $98,500 $259,000 $492,000

Personnel costs 385,000 500,000 775,000

Total annual costs $483,500 $759,000 $1,267,000

Note: Equipment costs are based on the invoice price paid to vendors in the FOQA

demonstration project. To annualize the figures, the equipment purchase costs have been spread

over a 5-year period. Personnel costs are based primarily on estimates of FOQA management,

analysis, monitoring, and engineering costs from an airline participating in the demonstration

project.

Source: UTRS.

The cost-benefit study estimates that airlines will reduce their

expenditures for fuel and maintenance as well as reduce the number of

accidents and incidents over time, avoiding their associated costs.

Because FOQA programs analyze additional data on aircraft systems and

engine conditions, airlines are better able to achieve optimum fuel

consumption and avoid unneeded engine maintenance. Although more

difficult to quantify and directly relate to a FOQA program, enhanced safety

should result in lower costs over time as a result of accidents avoided and

lower insurance premiums. Table 2 summarizes the estimated annual

savings for fleet sizes of 15, 50, and 100 aircraft. Fuel savings and engine

savings figures are based on estimates of a 0.5-percent reduction in fuel

consumption and a 1-percent reduction in engine maintenance costs. The

safety savings figure is based on a hypothetical 1-percent reduction in the

annual costs incurred from accidents. FAA’s contractor based its safety

savings calculation on a current loss rate of 2 aircraft per million

departures at a cost of $150 million for each loss.

Page 8 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

Table 2: Estimated Total Annual

Savings, by Fleet Size 15 aircraft 50 aircraft 100 aircraft

Fuel savings $145,800 $486,000 $972,000

Engine savings 300,000 1,000,000 2,000,000

Safety Savings 49,500 165,000 330,000

Total annual savings $495,300 $1,651,000 $3,302,000

Note: Fuel and engine savings were estimated on the basis of discussions with an airline

participating in the FOQA demonstration project. Safety savings were estimated on the basis of

information from a European airline with a long-term FOQA program. Savings estimates were also

based on an assumption of 3,000 flight hours per aircraft per year.

Source: UTRS.

According to these annual cost and savings estimates, FOQA would result in

net annual savings of $11,800 for 15 aircraft, $892,000 for 50 aircraft, and

$2,035,000 for 100 aircraft. See table 3.

Table 3: Estimated Net Annual

Savings, by Fleet Size 15 aircraft 50 aircraft 100 aircraft

Total annual costs $483,500 $759,000 $1,267,000

Total annual savings 495,300 1,651,000 3,302,000

Net annual savings $11,800 $892,000 $2,035,000

Source: UTRS.

Factors Impeding

Implementation and

Actions to Overcome

Impediments

Although airline officials, pilot organizations, and FAA officials recognize

the potential for improving safety and operations through FOQA programs,

airline officials and representatives of the pilot organizations were

unanimous in their view that data protection issues need to be resolved.

Both airline officials and pilots’ representatives stated that the lack of

protections for FOQA data has been a major contributor to pilot unions’

reluctance to sign FOQA agreements with airlines and airlines’ reluctance to

implement FOQA programs.

According to the Flight Safety Foundation’s 1992 report, the greatest

impediment to the implementation of FOQA in the United States is

associated with the “protection of data from use for other than safety and

operational improvement purposes.” Basically, airline managers and pilots

have three concerns: (1) that the information may be used in

enforcement/discipline actions, (2) that such data in the possession of the

federal government may be obtained by the public and the media through

Page 9 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

the provisions of the Freedom of Information Act (FOIA), and (3) that the

information may be obtained in civil litigation through the discovery

process. Similar concerns have been expressed in connection with other

programs under which information is submitted voluntarily to FAA.

Enforcement Representatives from each of the major airlines as well as the unions that

represent pilots from the major airlines—the Air Line Pilots Association,

the Allied Pilots Association, the Independent Association of Continental

Pilots, and the Southwest Airlines Pilot Association—told us that the

airlines and pilots fear the possibility that FOQA data might be used against

them in FAA enforcement proceedings. In addition to these concerns,

pilots’ representatives were concerned that airline managers could use

FOQA data to punish or discipline pilots.

FAA Enforcement. Many U.S. airlines and their pilots appear frustrated with

FAA’s delay in issuing a regulation implementing the nonenforcement

policy articulated in a February 1995 policy letter from the Administrator

to the Air Line Pilots Association and the Air Transport Association. FAA’s

letter said that no enforcement action will be taken on the basis of the

information gained through FOQA. Specifically the letter stated:

“The FAA commits that it will not use information collected by a carrier in an FOQA program

to undertake any certificate or other enforcement action against an air carrier participating

in such a program or one of its individual employees. Notwithstanding, the FAA reserves its

rights to use, for any other purpose, information obtained from sources other than FOQA,

including flight-recorder parameters specifically required by the Federal Aviation

Regulations. The limitation on the use of information applies only to information collected

specifically in an FOQA program.”

In an April 1997 letter to the Air Transport Association’s FOQA Steering

Committee, the Director of FAA’s Flight Standards Service said that the

1995 policy letter will remain in effect until the regulation on enforcement

is issued. The letter stated that a proposed rulemaking setting forth FAA’s

enforcement protection policy should be ready by the end of 1997.

According to airline officials and a pilot union’s representative, FAA’s delay

in promulgating an enforcement regulation has hampered efforts to reach

agreement with some pilot unions and threatens the continuance of

agreements already reached. One of the issues facing FAA is how broad the

enforcement protection should be. FAA attorneys have concluded that it is

beyond the scope of FAA’s authority and in violation of its statutory duties

Page 10 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

to issue a regulation that precludes the agency from taking action if FOQA

data reveal that an airplane was not in a condition for safe flight or that a

pilot lacked qualifications. Pilots’ representatives, however, have cited the

precedent of FAA’s cockpit voice recorder regulation that prohibits the

agency from using the record in enforcement actions without exceptions.10

FAA officials told us that the agency is trying to find the proper balance

between carrying out its enforcement responsibilities and providing

incentives for implementing safety programs and sharing information with

FAA. In similar programs, such as the Aviation Safety Reporting Program,

Air Carrier Voluntary Disclosure Reporting Procedures, and Aviation

Safety Action Programs,11 under which safety information is voluntarily

submitted, the agency has a policy of addressing alleged violations through

administrative actions or forgoing and/or waiving the imposition of any

legal enforcement if certain qualifying criteria are met. These programs are

intended to encourage prompt reporting of violations, sharing of important

safety information, and pilot training to enhance future compliance. While

the qualifying criteria differ for each program, these programs exclude

actions that are deliberate or demonstrate or raise questions of

qualifications. Generally, the parameters of the programs, including the

qualifying criteria, are spelled out in the governing advisory circular. It is

FAA’s belief that by offering incentives, such as forgoing legal enforcement

actions under certain conditions, more problems may be reported and

ultimately corrected than could be discovered through other means, such

as inspections.

Airline Enforcement. Airline managers are working with their respective

pilot unions to enter into data-use agreements that include individual

protection provisions. According to the Flight Safety Foundation study,

data-use agreements with pilot associations have existed since flight data

recorders were first required in the late 1950s. Having such an agreement

is a precursor to becoming a full partner in the FOQA demonstration

project. Generally, these agreements provide, among other things, the

company’s assurance not to use the recorded flight data for punitive or

10The cockpit voice recorder regulation provides that: “The Administrator does not use the cockpit

voice recorder record in any civil penalty or certificate action.” 14 C.F.R. 91.609(g). FAA’s regulations

also provide enforcement protection with some qualifications to information collected under the

Aviation Safety Reporting Program. Specifically, the regulation provides that “The Administrator of the

FAA will not use reports submitted to the National Aeronautics and Space Administration under the

Aviation Safety Reporting Program (or information derived therefrom) in any enforcement action

except information concerning accidents or criminal offenses which are wholly excluded from the

Program.” 14 C.F.R. 91.25.

11See app. IV for a description of these programs.

Page 11 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

disciplinary action against a crew member, or as evidence in any

proceeding. Also, to ensure the protection of the company’s employees,

the data-use agreements generally provide for the de-identification of the

information as soon as possible, usually within 7 days. This practice

ensures the confidentiality and anonymity of the flight crew members

participating in the program.

Freedom of Information

Act Requests

Both airlines and pilots are concerned that FOQA data could become public

and available to the media through the federal FOIA, if such data are

provided directly to FAA.

12 The federal FOIA sets forth a policy of broad

disclosure of government documents to ensure “an informed citizenry,

vital to the functioning of a democratic society.” NLRB v. Robbins Tire &

Rubber Co., 437 U.S. 214, 242 (1978). The Congress understood, however,

that “legitimate governmental and private interest could be harmed by

release of certain types of information.” FBI v. Abramson, 456 U.S. 615, 621

(1982). Accordingly, the act provides for nine categorical exemptions.

In the past, safety information voluntarily submitted to FAA, for example

under Air Carrier Voluntary Disclosure Reporting Procedures, has been

protected under exemption 4 of FOIA. Exemption 4 protects trade secrets

and commercial or financial information obtained from a person that is

privileged or confidential. Airline officials and pilots’ representatives

expressed concern that FOQA data may not be protectable under this

exemption.

Recently, the Congress enacted the Federal Aviation Reauthorization Act

of 1996, which contains a provision that protects voluntarily submitted

information under certain circumstances. Specifically, under the

provision, notwithstanding any other provision of law, the Administrator is

barred from disclosing voluntarily provided safety- or security-related

information if the Administrator finds that

“(1) the disclosure of the information would inhibit the voluntary provision of that type of

information and that the receipt of that type of information aids in fulfilling the

Administrator’s safety and security responsibilities; and (2) withholding such information

from disclosure would be consistent with the Administrator’s safety and security

responsibilities.” 49 U.S.C. 40123.13

12Currently, airlines provide no FOQA data to FAA. Rather, FAA reviews aggregated trend information

on the airlines’ premises.

13A similar provision was included in the National Transportation Safety Board Amendments of 1996 to

protect information that is voluntarily submitted to the Board.

Page 12 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

The provision also requires the Administrator to issue regulations to

implement the section.

The House report accompanying this legislation noted with approval the

data-sharing programs such as FOQA and the Committee’s intent to

encourage and promote these sorts of innovative safety programs. The

report provides that information submitted under these programs would

arguably be protected from release under exemption 4 of FOIA; however,

the report notes that such a decision to withhold the information would be

discretionary with the agency. The report states that to provide assurance

that such information is not publicly released, the legislation would

prohibit FAA from disclosing voluntarily submitted safety information.

According to the report, this protection should “alleviate the aviation

community’s concerns and allow the data-sharing safety programs to move

forward.” Moreover, the report noted that the provision would not reduce

the information available to the public, since the public does not receive

the data. Rather, the report states that public safety will be enhanced by

the increase in FAA’s understanding of ongoing trends in operations and

technologies.14

FAA is currently working on a rulemaking procedure that will prohibit the

release of voluntarily submitted safety data through FOIA.

15 It is expected

that the rulemaking will provide the procedures that the agency will use in

making the required determinations. It is also expected that FOQA data will

be proposed as qualifying for the protection. According to an FAA attorney,

the determinations for the FOQA program may be included in the notice of

proposed rulemaking on the FOQA nonenforcement policy. The anticipated

FOIA rulemaking and the subsequent findings to include the FOQA program

within the protection should help mitigate or resolve the industry’s fears

about the possible disclosure of FOQA data through FOIA requests if FOQA

data are provided directly to FAA.

Discovery Process in Civil

Litigation

Some airline officials have told us that although they want to improve

aviation safety by implementing a FOQA program, the voluntary collection

14H.R. Rep. No. 104-714 pt. 1 at pp. 40-41 (1996).

15In the Final Report of the White House Commission on Aviation Safety and Security, dated Feb. 12,

1997, a recommendation was made that FAA should work with the aviation community to develop and

protect the integrity of standard safety databases that can be shared in accident prevention programs.

The report stated that FAA needed to expeditiously complete rulemaking to implement the voluntary

disclosure protection provision and that the agency should assess the adequacy of the new legislative

authority and implementing regulation 1 year after the regulations take effect. The report further

stated that any necessary regulatory or legislative modifications identified at that time should be

promptly addressed.

Page 13 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

of data may potentially expose airlines to greater liability in civil litigation.

FOQA data may indicate conditions outside of desired operating

procedures. Airline officials and pilot representatives told us that they are

concerned that through broad discovery rules, FOQA data could be

inappropriately used or disclosed to the public. The general purpose of

discovery is to remove surprise from trial preparation so that parties may

obtain the evidence necessary to evaluate and resolve their dispute.16

Since FOQA data are retained at the airlines and are not currently provided

directly to FAA, the focus has been on the airlines’ ability to protect the

information.

Under federal rules, parties in litigation in federal court are authorized to

obtain discovery of any matter, not privileged, which is relevant to the

subject matter involved in the pending action, whether it relates to the

claim or defense of the party seeking discovery or to the claim or defense

of any other party. Generally, privileges are narrowly construed and in

some cases are qualified. However, even in the absence of a privilege, a

district court has broad discretion under the federal rules to issue an order

to protect a person from annoyance, embarrassment, oppression, or undue

burden or expense if there is a good cause for issuance of the order.

Courts generally invoke a balancing test to decide when a protective order

is appropriate and how it is to be applied.17

In two recent cases, the airlines have tried to convince federal courts that

voluntarily collected safety data similar to FOQA data should be protected

from discovery or, at the very least, covered under a protective order.18 In

both cases, the courts sought to achieve a balance between the airlines’

desire to protect the information and the plaintiffs’ right to a fair trial. In

the first case, the court rejected a claim that the information should be

protected under the self-critical evaluation privilege but limited the

possible uses of the documents it ordered to be produced.19 This

determination was effected through a protective order. In the other case,

166 Moore’s Federal Practice, section 26.02 (Matthew-Bender 3d ed.).

176 Moore’s Federal Practice, ch. 26 (Matthew-Bender 3d ed.).

18Court Order of Oct. 26, 1995, In re: Air Crash at Charlotte, North Carolina, on July 2, 1994, MDL

Docket No. 1041 (D.S.C. 1995) (the court rejected the claim of self-critical evaluation privilege); but

see, Court Order of Nov. 14, 1995, In re: Air Crash at Charlotte, North Carolina, on July 2 1994, MDL

Docket No. 1041 (D.S.C. 1995) (the court issued a protective order); and In re: Air Crash Near Cali,

Columbia, on Dec. 20, 1995, 959 F. Supp. 1529 (S.D. Fla. 1997) (the court rejected the claim of

self-critical evaluation privilege but recognized a new qualified privilege for the American Airlines

Safety Action Program). For a more detailed discussion of these court cases, see app. V.

19The self-critical evaluation privilege, when recognized, protects documents that reflect an internal

self-analysis.

Page 14 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

the court also rejected the claim of self-critical evaluation privilege but at

the same time recognized a new qualified privilege for information

collected under a partnership program with FAA, the American Airlines

Safety Action Program.20

Although airlines are generally pleased with the court’s decision to grant a

qualified privilege to Aviation Safety Action Program materials, it is not

clear whether other courts will recognize this new privilege or extend it to

other safety and security information that has been voluntarily collected.

Nor is there a guarantee that FOQA data or other similar information, if

found not to be privileged, would be covered under a protective order.

However, we found no instances to date in which FOQA data have been

subject to a discovery request. This situation may result from the fact that

airlines are just beginning to institute FOQA programs. However, some of

the pilot union officials we spoke with noted that discovery is a concern

because of the potentially large amounts of data that will be collected.

While some in the aviation community believe that one way to ensure

protection would be through legislation, there does not appear to be a

consensus to seek legislation at this time.21 Concern has been expressed

that the failure of a legislative effort may adversely affect how courts treat

voluntarily collected safety information.

In the event that FAA does receive FOQA data directly, according to FAA

attorneys, it has provisions in place for dealing with requests from private

litigants for documents in the agency’s possession. FAA attorneys noted

that a request for records from a private litigant, when the agency is not a

party to the action, will generally be treated as a FOIA request (see 49

C.F.R. 9.13). If the agency is a party to the litigation, FAA will seek to

protect the information, if appropriate, under a claim of government

privilege and, if that fails, to release the information under a protective

order.

Agency Comments We provided copies of a draft of this report to the Department of

Transportation and FAA for their review and comment. We met with

officials, including FAA’s Deputy Associate Administrator for Regulation

and Certification and the demonstration project’s program manager. They

20Thus, the court provided that the plaintiff could come forward with a persuasive showing of need and

hardship. In such case, the court would review the voluntarily collected information in camera and

evaluate whether the plaintiff’s interests overcome the powerful interest that weighs in favor of

preserving the confidentiality of the information. No such showing was made in this case.

21Limited legislative protection has been provided for cockpit voice recorders, 49 U.S.C. 1154.

Page 15 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

agreed with the report and provided several technical corrections, which

were incorporated into the report.

Scope and

Methodology

To obtain the information in this report, we reviewed FAA’s FOQA

demonstration project’s requirements, policies, and plans to assist airlines

in implementing FOQA programs. We discussed specific details of the

project with FAA’s Deputy Associate Administrator for Regulation and

Certification as well as the demonstration project’s program manager and

contractor. We conducted interviews with FAA and National Aeronautics

and Space Administration officials responsible for developing the Aviation

Performance Measuring System. We discussed FOQA issues with the

National Transportation Safety Board. We interviewed representatives of

each of the 10 largest passenger airlines: Alaska, America West, American,

Continental, Delta, Northwest, Southwest, Trans World, United, and US

Airways; representatives of each of the four unions—the Air Line Pilots

Association, the Allied Pilots Association, the Independent Association of

Continental Pilots, and Southwest Airlines Pilot Association—representing

the pilots of these airlines; and United Parcel Service. We also conducted

interviews with the Air Transport Association, the Flight Safety

Foundation, and the vendors providing hardware and software for the

demonstration project. Last, we interviewed and collected information

from foreign airlines and Britain’s Civil Aviation Authority on their

respective FOQA efforts.

We performed our work primarily at FAA headquarters in Washington, D.C.,

and conducted our evaluation from January through October 1997 in

accordance with generally accepted government auditing standards.

As requested, we plan no further distribution of this report until 30 days

after the date of this letter unless you publicly announce the report’s

contents earlier. At that time, we will send copies to the appropriate

congressional committees and the Department of Transportation and FAA.

Page 16 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990 

We will also make copies available to others upon request. Please call me

at (202) 512-2834 if you have any questions about this report. Major

contributors to this report are listed in appendix VI.

John H. Anderson, Jr.

Director, Transportation Issues

Page 17 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Contents

Letter 1

Appendix I 

The FOQA Concept

and Its

Implementation in the

United States

20

Appendix II 

U.S. and Foreign

Airlines With FOQA or

FOQA-TYPE

Programs

30

Appendix III 

FAA’s Related

Technical Programs

31

Appendix IV 

FAA’s Voluntary Safety

Reporting: Selected

Programs

33

Appendix V 

Discovery-Related

Court Actions

35

Appendix VI 

Major Contributors to

This Report

38

Tables Table 1: Estimated Total Annual Costs, by Fleet Size 8

 Table 2: Estimated Total Annual Savings, by Fleet Size 9

Page 18 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Contents

 Table 3: Estimated Net Annual Savings, by Fleet Size 9

Abbreviations

APMS Aviation Performance Measuring System

AQP Advanced Qualification Program

ASAP Aviation Safety Action Programs

ASRP Aviation Safety Reporting Program

ASRS Aviation Safety Reporting System

BA British Airways

DEMOPROJ FOQA demonstration project

DFDAU digital flight data acquisition unit

FAA Federal Aviation Administration

FAR Federal Aviation Regulation

FOIA Freedom of Information Act

FOQA Flight Operational Quality Assurance

FSF Flight Safety Foundation

GAIN Global Analysis and Information Network

KLM Royal Dutch Airlines

NASA National Aeronautics and Space Administration

PCMCIA Personal Computer Memory Card International

Association

QAR quick access recorder

UTRS Universal Technical Resource Services, Inc.

Page 19 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

FOQA’s Background Flight Operational Quality Assurance (FOQA) had its origin in the use of

flight data recorders as mandated by the Civil Aeronautics Administration

in 1958. Although the first flight data recorders captured only six

parameters—time, airspeed, heading, altitude, vertical acceleration, and

time of radio transmission—they were a valuable tool for reconstructing

what had occurred before and during accidents. By the 1960s, airlines had

begun to monitor data on routine flights. Initially, the monitoring systems

captured airworthiness data, but over time they have expanded to include

operational data. In the late 1960s, Trans World Airlines began a program

to monitor a limited number of parameters related to approaches and

landings as flight data recorders received periodic maintenance.

At least eight foreign airlines have had FOQA-type programs in operation for

over 25 years. A program using data from flight data recorders was begun

by British Airways (BA) in 1962 to validate airworthiness criteria. Although

limited by today’s standards, BA’s program contained the seeds of a

modern, safety-oriented FOQA program. Currently, BA analyzes the flight

data from all of the aircraft in its fleet through its Special Events Search

and Master Analysis program. Over the years, the number of foreign

airlines that have implemented a FOQA-type program has steadily risen.

Japan Airlines’ FOQA program of over 15 years includes a printer in the

cockpit so that pilots can monitor their own performance. All Nippon

Airways began a program to analyze flight data in 1974. Other foreign

airlines with established FOQA programs include Scandinavian Airlines

System, Royal Dutch Airlines (KLM), and Lufthansa. Many of these airlines

are convinced that FOQA is a critical component in their safety efforts and

that the program has paid valuable safety dividends over the years.

Currently, about 33 foreign airlines and 4 U.S. airlines—United, US

Airways, Continental, and Alaska—have implemented FOQA programs (see

app. II for the complete list).

Recognizing the value of operational flight data and the critical nature of

flight crews’ performance in accidents, the Flight Safety Foundation (FSF)

proposed and was selected by the Federal Aviation Administration (FAA) in

1991 to study FOQA. In its 1992 report on FOQA, FSF said that

“The proposal was based on FSF’s conviction, formed by the positive experiences of its

international member airlines using FOQA, that the appropriate use of FOQA data by airlines,

pilot associations and aircraft and equipment manufacturers would result in a significant

improvement of flight safety by identifying operational irregularities that can foreshadow

accidents and incidents.”

Page 20 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

The FSF study concluded that FOQA must proceed in the United States and

that the implementation of FOQA by U.S. airlines would have a more

positive impact on Part 121 operational safety than any other human

factors program included in FAA’s research and development plans. FSF

recommended that FAA promote voluntary FOQA programs by instituting a

demonstration program in partnership with industry. In 1992, FAA’s Flight

Standards Service proposed funding for a demonstration program. On

February 9, 1995, FAA announced its plans for an FAA-industry

demonstration project, and the Administrator sent a policy letter to the Air

Transport Association and the Air Line Pilots Association stating that FAA

would not use FOQA data for enforcement purposes, provided that the

airlines met certain requirements.

How FOQA Works At a minimum, FOQA involves the analysis of flight data on a routine basis

to reveal situations requiring corrective actions before problems occur. To

institute such a program, airlines need methods to capture flight data,

transform the data into the appropriate format for analysis, and generate

reports and visualizations to assist personnel in analyzing the data.

Although different methods are available, the following describes how a

representative FOQA program operates; the descriptions are based on the

experience of the four U.S. airlines that have implemented FOQA.

Management. A typical program is managed and operated by a FOQA

manager, one or more analysts, and a FOQA monitoring team (sometimes

referred to as the exceedance guidance team) made up of airline pilots

who work on FOQA on a part-time basis. Generally, the majority of the

monitoring team’s pilots are also representatives of the pilot union. These

individuals manage the FOQA program in strict adherence to the

agreements made with the pilot union, most notably on ensuring the

confidentiality of pilots’ identities. This group is responsible for defining

and refining exceedances and parameters, reviewing and analyzing data,

and determining and monitoring corrective actions.

Data capture. The first step is the capture of data over the duration of the

flight. Flight data comprise snapshots of values or measurements from

various aircraft systems. Each data item represents information from a

discrete source, such as an instrument or sensor. Generally, these data

items are referred to as “parameters.” Examples of parameters are

“altitude” or “landing gear position.” Recording rates vary, depending on

the parameter, ranging from many times per second to about once per

minute.

Page 21 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

Although flight data recorders continuously record, at a minimum,

FAA-mandated parameters during every flight, they typically are not

designed to provide frequent access to their data but rather to survive the

extreme conditions during and after crashes to preserve flight data for

accident investigations. These devices are housed in crash-resistant,

sealed containers designed to withstand high “g” forces, submersion in

water, and fire. Obtaining frequent access to flight data recorders for FOQA

purposes, however, would produce increased wear on internal

mechanisms and result in shortened mechanical life and increased

expense for a very specialized device.22 Also, flight data recorders may not

capture a sufficient number of parameters to be useful for FOQA purposes.

Currently, FAA requires from 16 to 29 parameters to be recorded on flight

data recorders in transport aircraft;23 a FOQA program, however, would

likely capture many more parameters. Typically, the 200-500 parameters

available on modern digital aircraft allow a more comprehensive set of

conditions to be monitored. Finally, flight data recorders hold about 25

hours of flight data, a relatively short time period. Instead, some U.S.

airlines use a device called a quick access recorder (QAR) to record FOQA

data to a removable optical disk or Personal Computer Memory Card

International Association (PCMCIA) card.24 QARs record flight data that are

output from the aircraft’s digital flight data acquisition unit (DFDAU), the

same device that feeds parameters to the flight data recorder. On average,

QARs hold from 100 to 200 hours of flight data.

Data transfer. As aircraft receive periodic servicing, the medium (optical

disk or PCMCIA card) containing flight data is removed from the QAR and

sent to a central location for analysis. A new disk or card is inserted into

the QAR for the next round of flights. Airlines retrieve the data on

schedules ranging from 3 to 20 days.

An alternative to physical recording media is the use of datalink systems to

transmit information directly to the ground-based system, eliminating the

need to retrieve data from the aircraft. Two participating airlines are

investigating the use of automatic wireless data transfer upon landing at

specially equipped airports. Data would be transmitted on a radio

22The newer solid-state flight data recorders, however, have no moving parts and would not experience

wear problems. Transferring data from these devices takes several minutes to perform.

23Under a recently issued rule, FAA requires the recording of 16 to 29 parameters by the flight data

recorder on all existing transport aircraft, depending on the aircraft model, its internal systems, and its

date of manufacture. Aircraft manufactured after the rule, however, will be required to record 88

parameters within 5 years. 62 Fed. Reg. 38362 (July 17, 1997).

24Other airborne data collection systems in use around the world include QARs using tape cartridges

and solid-state devices.

Page 22 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

frequency link from the aircraft to a receiving station after the aircraft

lands. In turn, a local area network would transfer the data to the ground

analysis station. Data encryption and other methods would be used to

ensure the security of the transmitted FOQA data.

Data processing and analysis. Each airline has a ground analysis system

where airborne collected data are processed and analyzed. The ground

analysis system transforms the raw digital flight records into usable form

for processing, analyzes the flight information, and generates information

on any detected exceedances that represent deviations from normal

operating practices or exceptional conditions.

The flight data analysis component of the ground analysis system

categorizes operational events to be flagged by defining a set of

parameters that indicate normal operating envelopes. The associated

thresholds for these parameters vary by the type of aircraft and associated

operating limits, accepted practices for safe operations, the phase of flight,

and the duration of any irregularity. For example, the threshold of selected

parameters may be defined for various altitudes, e.g., 1,000, 500, 250, and

100 feet, during landing mode events. Typically, 40 to 80 events are defined

and analyzed for a particular aircraft. For example, events might be the

ground speed during taxi or the descent rate during approach. The

analysis software will track the descent over time to calculate a rate in

terms of feet per minute. Depending on the aircraft’s altitude, a descent

rate in excess of specified thresholds will trigger an exceedance. Various

categorization schemes are used to classify the seriousness of the

exceedance. U.S. airlines use two or three categories to describe the

seriousness of exceedances, ranging from minor deviations to major

deviations. Exceedances are typically specified on the basis of a strategy

for identifying those that have the greatest potential for safety and

performance considerations. Once the initial exceedance categories and

associated parameters have been defined and utilized, they are subject to

an ongoing evaluation and refinement process.

The ground analysis software also validates the quality and integrity of the

collected data and filters out any marginal or transitory irregularities.

Ground analysis systems also include protective mechanisms, such as the

de-identification of pilot and specific flight information and user access

privileges based on assigned passwords. As the data are processed, the

flight number and day of the month are removed and saved into a separate

controlled file. This step “de-identifies” the FOQA data.

Page 23 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

The FOQA monitoring team investigates each exceedance to determine

what occurred and the severity of the exceedance. An analyst will review

the parameter values surrounding the event and other information to

determine if the exceedance was valid or if the exceedance was based on

bad data, a faulty sensor, or some other invalidating factor. For example,

one flight had excessive rudder input on landing that correctly registered

as an exceedance. On closer examination, it was determined that because

the aircraft was making a cross-wind landing, the use of large rudder input

was justified. In this case, the exceedance was deemed invalid and was

removed from the exceedance database.

Depending on the particular circumstances of the exceedance, the pilot

association’s representative may contact the flight crew to gather more

information. After reviewing the situation to determine the exceedance’s

cause, the FOQA monitoring team and pilot association’s representative will

determine any necessary corrective action. Corrective action can range

from additional flight crew training, to revisions of the operating

procedures, to redesigns of equipment.

Trend analysis. On a periodic basis, airlines aggregate and analyze

exceedances over time—for example, the number of unstabilized

approaches at a particular airport per month, over the last 12 months. This

type of analysis provides valuable information to the airline, especially in

terms of whether the airline’s performance is improving, holding steady, or

deteriorating. This look at aggregate exceedances over time provides

airline managers with a new perspective on potential problems that would

not be visible otherwise. On the basis of the trend analysis, airline

managers can take corrective action to reduce or eliminate these

exceedances by focusing on the root causes and making or recommending

changes.

Data retention. Detailed FOQA data, including exceedances, are destroyed

in 30 days or less by three of the four U.S. airlines with FOQA programs.

Trend data, however, are kept indefinitely.

Aircraft equipping decisions. The U.S. airlines with active FOQA programs

have each equipped a portion of their available fleets with QARs. They

began their programs by equipping their more modern, technically

advanced aircraft with QARs—late-generation aircraft already contain the

sensors and advanced digital systems that acquire and control many more

flight data parameters than earlier-generation aircraft. Generally, these

airlines do not plan to equip any of their older, analog-based aircraft, such

Page 24 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

as Lockheed L-1011, McDonnell Douglas DC-9 and DC-10, and Boeing 727,

737-100, and 737-200, with QARs to record flight data because these aircraft

would be expensive to retrofit and because the airlines plan to retire many

of them in the near future.

Several U.S. airlines plan to equip all new aircraft with QARs or other

technology to capture FOQA data. Some new aircraft, for example, are

delivered with QARs as standard equipment. Airlines cited several

advantages in having new aircraft delivered with factory-equipped QARs.

One advantage is that aircraft are not taken out of service to be retrofitted

with equipment. Another advantage is that the additional cost of a QAR can

be spread over the finance period of the new aircraft.

Depending on the specific goals of a FOQA program, an airline may wish to

equip some or all of its fleet to collect flight data. If a program’s goal is to

identify broad trends in flight operations and safety, the airline may

choose to equip only a portion of its fleet. If a program’s goal, however, is

to more closely monitor the flight operations and performance of

individual aircraft, the airline may want to equip more or all of its fleet.

For an airline that begins by equipping only a portion of its fleet, more

aircraft will likely be added to the program so that these data can be

monitored as its FOQA program matures and efficiency and maintenance

functions are added to the program. Some U.S. airlines, for example, are

planning to use FOQA data to cut aircraft maintenance costs by more

closely monitoring engine conditions and fuel consumption.

The FOQA Demonstration

Project

On July 11, 1995, FAA awarded a 2-year contract25 to execute a FOQA

demonstration project, referred to as DEMOPROJ by FAA, to Universal

Technical Resource Services, Inc. (UTRS), of Cherry Hill, New Jersey. The

contract stated that

“The goal of DEMOPROJ is to facilitate the start-up of the FOQA initiative and to

comprehensively assess the cost-benefits and safety enhancement effectiveness of an

implemented FOQA program in which airlines voluntarily employ in-flight recorded data to

routinely monitor their flight operations.”

UTRS facilitated the establishment of collaborative partnerships between

FAA, UTRS, and interested airlines. Airlines may participate in DEMOPROJ at

one of three levels within the project, ranging from attending meetings and

expressing interest to a full partnership with FAA. Level 3 participation

25The term of the contract was later revised to 3 years.

Page 25 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

refers to the airlines that have not yet established an official FOQA program

but attend meetings to learn about FOQA. At Level 2, the airlines already

have their own equipment or will acquire equipment using airline funding,

but they allow UTRS to monitor and document their program. Level 1

describes a full partnership in which equipment and software are provided

through DEMOPROJ. Currently, 11 airlines are participating in DEMOPROJ. The

airlines participating at Level 1 are United, US Airways, and Continental.

All other participating airlines in DEMOPROJ are at Level 3: Alaska, America

West, Delta, Northwest, Trans World, Southwest, Continental Express, and

United Parcel Service.

The airline participants were selected on the basis of a number of

characteristics, including financial stability, management commitment,

resource commitment, fleet characteristics, fleet size, aircraft availability,

and an approved implementation and operation plan. Additionally, airlines

are required to sign nondisclosure and cooperation agreements that define

the treatment of confidential and proprietary information, enumerate data

access control and security provisions, and specify the responsibilities and

contributions of each party. Participating airlines also had to secure

agreements with their pilot associations for the collection and analysis of

flight data. These airlines made the commitment to record and process

FOQA data on all scheduled flights that are equipped with FAA-supplied

equipment, participate in periodic project reviews, and allow UTRS to

interview airline personnel during the project to document procedures,

problems, issues, and solutions.

UTRS assisted airlines in determining the equipment best suited to their

needs, acquiring the equipment, and delivering it for installation by the

airlines. Hardware and software were selected from commercially

available, off-the-shelf sources. As part of this effort, the contractor

developed an Equipment Overview to facilitate the airlines’ analysis and

selection of available equipment.

UTRS also monitors and documents the airlines’ FOQA demonstration

programs’ policies, procedures, usage, and effectiveness. The contractor is

collecting and analyzing information on how each airline is implementing

FOQA, including data processing and analysis; the retention of detail and

trend data; the selection of flight data parameters; and the adjustment of

threshold values, system effectiveness, technical problems, and resource

information for establishing and maintaining a FOQA program. These

findings are integrated and disseminated among participants throughout

the study. UTRS is also collecting information about the projects’ costs and

Page 26 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

anticipated benefits. The contractor is determining how each airline

transforms FOQA data into information and how this information is used in

the airline’s decision-making. UTRS holds periodic meetings for all partners

to promote the sharing of information and lessons learned.

UTRS, with airlines’ and pilot associations’ involvement, is developing a

FOQA advisory circular to provide information and guidance to airlines on

how to design, implement, and maintain a FOQA program. This document is

scheduled to be issued approximately 90 days after FAA issues its proposed

rulemaking on enforcement policy in connection with FOQA. UTRS is also

developing a cost-benefit analysis that will provide estimates of (1) the

costs that an airline would incur when starting and maintaining a FOQA

program and (2) potential savings. The cost-benefit study is scheduled to

be completed in January 1998.

UTRS will issue a technical report and a set of FOQA guidelines in June 1998.

The technical report will be an overall description of the technical effort to

implement FOQA, summarizing the airlines’ experiences with commercially

available equipment and systems. The FOQA guidelines will synthesize the

airlines’ experiences in implementing FOQA with a view toward helping

new airlines learn from the airlines that have implemented a FOQA

program. The guidelines will include information on (1) designing a FOQA

program; (2) the start-up and initial operation of a system; (3) the use of

FOQA for trend analysis, knowledge building, and decision-making; and

(4) critical success factors for implementing a FOQA program.

In fiscal years 1995 through 1997, according to the FAA FOQA program

manager, FAA allocated $5.5 million for DEMOPROJ. The manager stated that,

as of September 26, 1997, DEMOPROJ had expended $2.1 million, including

$1.1 million for the purchase of hardware and software for the three Level

1 airline participants. FAA plans to pursue follow-on development focused

on the acquisition and use of FOQA information by FAA for safety monitoring

purposes.

U.S. Airlines With Active

FOQA Programs

Currently, four U.S. airlines have active FOQA programs: United Airlines,

US Airways, Continental Airlines, and Alaska Airlines. These airlines have

equipped a number of their aircraft with QARs, from 7 aircraft at Alaska

Airlines to 52 aircraft at United Airlines. The number of parameters

continuously recorded on the QARs range from about 38 to over 300,

depending on the airline and the type of aircraft.

Page 27 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

United Airlines. United Airlines has the largest and longest-running FOQA

program of any U.S. airline, begun in 1995. As of August 1997, United had

52 aircraft equipped with QARs and had collected FOQA data on over 25,000

flights. The aircraft currently equipped include Boeing 737-500s and 777s

and Airbus 319s and 320s. United plans to equip over 120 aircraft by 1999,

including all new aircraft currently on order. DEMOPROJ has funded the

purchase of QARs to equip 15 Boeing 737-500s and additional data analysis

packages and computer equipment to run on systems that United had

already established. The remainder of the hardware and software was

purchased by United, which has been tracking and correcting exceedance

events for more than a year. United has identified and taken corrective

action to reduce the incidence of a number of safety- and

maintenance-related exceedances.

US Airways. US Airways has 23 QAR-equipped aircraft. Its program, begun

in September 1996, has collected FOQA data on over 18,000 flights to date.

Aircraft equipped include Boeing 737-400s and 767s. US Airways, however,

characterizes its program as being in the data collection and

trouble-shooting phase and just beginning the data analysis and trending

phase. DEMOPROJ has funded the purchase of QARs to equip 15 Boeing

737-400s and a ground analysis system. Six additional 737-400s have been

equipped with QARs paid for by a separate FAA program, the Structural

Loads Program (see app. III). In addition to these aircraft, US Airways is in

the process of purchasing QARs and equipping 12 Boeing 767s. Data from

all QARs are being accessed by both programs. DEMOPROJ has also funded a

trial program of a wireless ground datalink system with five specially

equipped Boeing 757s.

Continental Airlines. Continental has equipped 15 Boeing 737-500s with

QARs. In addition, Continental plans to equip with QARs all new aircraft on

order. These include Boeing 737-500s, –600s, –700s, and –800s as well as a

number of 757s. Begun in December 1996, Continental’s program has

analyzed the flight data from over 11,000 flights to date. According to the

program manager, this program is in the data collection phase and will

soon be making the transition to the data analysis and trending phase.

Alaska Airlines. Alaska Airlines has equipped six McDonnell Douglas

MD-80s and one Boeing 737-400 with QARs. In addition, Alaska has

equipped a flight simulator with equipment to record hundreds of flight

parameters. Begun in July 1996, the program has analyzed the flight data

from over 5,000 flights to date. Still in the early stages of the program,

Page 28 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I 

The FOQA Concept and Its Implementation

in the United States

Alaska plans to “go slow” and refine its program. Alaska’s FOQA manager

said that the airline may eventually equip every aircraft in its fleet.

Unlike United, US Airways, and Continental, which are Level 1

participants in DEMOPROJ, Alaska is a not yet a full partner in DEMOPROJ

because it has only recently secured the required agreement with its pilot

union on FOQA. The airline, however, has received six QARs and a ground

analysis system from FAA’s Structural Loads Program (see app. III). Alaska

uses the equipment and analysis system for both the Structural Loads

Program and FOQA.

Page 29 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix II 

U.S. and Foreign Airlines With FOQA or

FOQA-TYPE Programs

Adria Airways

Aeroflot

Air Afrique

Air France

Air Inter

Air Liberte

Alaska Airlines

All Nippon Airways

Asiana Airlines

Balkan Airlines

Britannia

British Airways

British Midland

Cathay Pacific Airways

China Airlines

China Southern Airlines

China South West Airlines

Continental Airlines

Emirates

Ethiopian Airlines

EVA Airways

Garuda Indonesia

GB Airways

Gulf Air

Japan Airlines

Royal Dutch Airlines (KLM)

Kuwait Airways

Lufthansa

Qantas Airways

Saudia Arabian Airlines

Scandanavian Airline System (SAS)

Singapore Airlines

TAP Air Portugal

Thai Airways International

United Airlines

US Airways

Wideroe26

Source: GAO and The Flight Data Company, Ltd.

26The list may not be comprehensive.

Page 30 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix III 

FAA’s Related Technical Programs

Aviation Performance Measuring System. In 1993, FAA contracted with the

National Aeronautics and Space Administration (NASA) to establish and

demonstrate the feasibility of developing an Aviation Performance

Measuring System (APMS). The objective of the APMS effort is to develop

tools and methodologies to allow large quantities of flight data to be

processed in a highly automated manner to address questions relating to

operational performance and safety. APMS is concerned with converting

flight data into useful safety information in support of the national air

transport system, airlines, and air crews. Although concerned with all

aspects of flight operations, APMS primarily will develop an objective

method for continuously evaluating air crews’ technical performance in

support of FOQA and the Advanced Qualification Program (discussed

below).

Current FOQA programs focus on exceedances; APMS, however, will expand

FOQA’s scope by utilizing all flight data. The tools will facilitate multiple

functions, including the acquisition of flight data, their storage in a

database management system, the study of statistical characteristics and

trends, the development of “data mining” techniques, and better methods

of visualizing flight data. APMS will also investigate flight animation

capabilities to assist flight crews in replaying and understanding

exceedances. Finally, APMS will facilitate the sharing of data among

databases, products, and interested parties. According to NASA officials,

one of the most important components to be developed by APMS is a risk

assessment tool to measure how much risk is associated with certain

activities, for example, the riskiness of flights to/from certain airports.

After APMS began in 1993, the project documented the status of the

technologies, systems, and software used by foreign airlines with FOQA

programs. According to the NASA project manager, the project has

conducted user needs studies at Alaska Airlines, United Airlines, and US

Airways and has commitments to conduct user needs studies at America

West, Trans World Airlines, Comair, and United Parcel Service. The APMS

team is also building prototype systems at several airlines. Alaska Airlines

is now in its third prototype APMS system. The project was scheduled to

begin building the initial prototype system at United Airlines on

November 1, 1997. Eventually the developed technology will be

transferred to industry so that a relatively low-cost system will be

commercially available. APMS management hopes to initiate the transfer of

this technology to commercial vendors in 12 to 18 months.

Page 31 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix III 

FAA’s Related Technical Programs

To date, NASA has received $2.9 million in funding from FAA for the

development of APMS. NASA contributed $300,000 to the project in fiscal

year 1997. The extent of future NASA and FAA funding for further

development and implementation of APMS has not yet been determined.

Structural Loads Program. As part of FAA’s Aging Aircraft Research and

Development Program, the Structural Loads Program is a cooperative FAA

and NASA effort to collect information about the external loads to which

airframe components are subjected during flight. The collected data will

be used to develop and maintain an extensive database of transport

aircraft usage to continuously validate and update flight and landing load

airworthiness certification standards on the basis of actual measured

usage. To date, the Structural Loads Program has equipped with QARs six

MD-80s at Alaska Airlines and six Boeing 737-400s at US Airways. Data

collected from these QARs are also being made available for FOQA analysis.

Advanced Qualification Program. FAA’s Advanced Qualification Program

(AQP) is an alternate method of qualifying, training, certifying, and ensuring

the competency of flight crew members and other operations personnel

subject to the training and evaluation requirements of Federal Aviation

Regulation (FAR) parts 121 and 135. AQP’s intent is to achieve the highest

possible standards of individual and crew performance without undue

increases in training costs. FOQA and APMS will be used to continuously

evaluate air crews’ technical skills and airlines’ procedures and training in

support of AQP. For example, FOQA data could be used to identify problems

occurring during recurrent flight simulator training and to highlight

training areas for increased emphasis.

Global Analysis and Information Network. The Global Analysis and

Information Network (GAIN) is a concept being actively explored by the

aviation community to facilitate the analysis, sharing, and dissemination of

aviation safety information with a goal of achieving zero accidents. GAIN

would have many information sources—FOQA information would be one of

the most important. Proposed by FAA in May 1996, GAIN will function as a

“significantly improved operational early warning capability that is

sensitive enough to detect and alert the aviation community to existing

and emerging problems.” Information will be shared among airlines and

manufacturers and at the different functional levels within organizations.

Although GAIN is still in the conceptual phase, the aviation community and

FAA are working to address the needs and concerns of prospective

members as well as explore potential designs for a prototype system.

Page 32 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix IV 

FAA’s Voluntary Safety Reporting: Selected

Programs

FAA has implemented a number of voluntary programs involving the

self-reporting of safety-related information to enhance aviation safety.

Although these programs involve the reporting of information by people

instead of by automated systems, they are similar to FOQA in that they

involve voluntary efforts to identify and correct potential safety problems.

We have highlighted three such programs.

Aviation Safety Reporting Program. Established by FAA in 1975 and

administered by NASA, the Aviation Safety Reporting Program (ASRP)

promotes the voluntary reporting of problems into the Aviation Safety

Reporting System (ASRS). Under 14 C.F.R. 91.25, FAA will not use reports

submitted under the program in any enforcement action (except accidents

or criminal offenses). Under FAA’s policy, although a finding of violation

may be made, no sanction will be imposed if (1) the violation was

inadvertent and not deliberate, (2) the violation did not involve a criminal

offense or an accident or an action that discloses a lack of qualification or

competency, (3) the person filing the report has not been found in any

prior FAA enforcement action to have committed a violation of federal

aviation regulations or law within a period of 5 years prior to the

occurrence, and (4) the report was filed within 10 days after the violation.

AC 00-46D (Feb. 26, 1997), “Aviation Safety Reporting Program.”27

Air Carrier Voluntary Disclosure Reporting Procedures. Initiated by FAA in

1990 for air carriers,28 the Voluntary Disclosure Procedures encourage

airlines to promptly disclose to FAA any instances of noncompliance with

the requirements for maintenance, flight operations, and security. FAA

initiated a policy of forgoing civil penalty actions if five conditions are met:

(1) the certificate holder immediately notifies FAA of the apparent violation

after detecting it and before the agency learns of it; (2) the apparent

violation is inadvertent; (3) the apparent violation does not indicate a lack,

or reasonable question, of the basic qualification of the certificate holder;

(4) immediate action must have been taken, or begun, upon discovery to

terminate the conduct that resulted in the apparent violation; and (5) the

certificate holder must develop and implement a comprehensive solution

satisfactory to the FAA. AC 120-56 (Jan. 23, 1992), “Air Carrier Voluntary

Disclosure Reporting Procedures.”

27Provisions concerning air traffic controllers involved in incidents reported under ASRS are addressed

in FAA Order 7210.3.G, Facility Operations and Administration.

28According to an assistant chief counsel at FAA, the procedures have since been expanded to include

production approval holders and repair stations.

Page 33 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix IV 

FAA’s Voluntary Safety Reporting: Selected

Programs

Aviation Safety Action Programs. FAA has established several

demonstration Aviation Safety Action Programs (ASAP), including the

USAir Altitude Awareness Program, the Alaska Airlines Altitude

Awareness Program, and the American Airlines Safety Action Program.29

These programs established incentives to encourage the employees of the

air carriers that are participating in the programs to disclose information

and identify possible violations of the Federal Aviation Regulations

without fear of punitive legal enforcement sanctions. FAA has recently

expanded the use of ASAP through the implementation of a 2-year

demonstration program. Under this program, apparent violations will

normally be addressed with administrative action if the apparent

violations do not involve (1) deliberate misconduct, (2) a substantial

disregard for safety and security, (3) criminal conduct, or (4) conduct that

demonstrates or raises a lack of qualifications. For apparent violations not

excluded under an ASAP, neither administrative action nor punitive legal

enforcement actions will be taken against an individual unless there is

sufficient evidence of the violation other than the individual’s

safety-related report. AC 120-66 (Jan. 8, 1997), “Aviation Safety Action

Programs (ASAP).”

29The Altitude Awareness programs at USAir and Alaska Airlines were joint programs with the Air Line

Pilots Association and FAA to eliminate altitude deviations. USAir’s program, in operation from

October 1990 through February 1992, and Alaska Airlines’ program, in operation from August 1994

through February 1995, encouraged flight crews to report altitude problems so that corrective action

could be taken. The American Airlines Safety Action Program is a joint program with the Allied Pilots

Association (American’s pilot union) and FAA. Begun in June 1994, the program encourages pilots to

report all types of potential safety problems.

Page 34 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix V 

Discovery-Related Court Actions

Airline officials and pilot unions’ representatives are concerned about the

use of discovery in civil litigation to reveal voluntarily collected safety

information. In two recent cases, the courts have sought to find a balance

between the airlines’ desire to protect such information and the plaintiffs’

right to a fair trial. In one case, the documents were required to be

produced, but under a protective order. In the other case, the court

recognized a new qualified privilege.

In 1995, the United States District Court, District of South Carolina,

Columbia Division, rejected USAir, Inc.’s30 argument that certain safety

data were protected under the self-critical evaluation privilege. Court

Order of Oct. 26, 1995, In re Air Crash at Charlotte, North Carolina, on

July 2, 1994, MDL Docket No. 1041 (D.S.C. 1995). This privilege, when

recognized, protects documents that reflect self-analysis.

The district court noted that the self-critical evaluation privilege is a

privilege of recent origin and one that is narrowly applied even in those

jurisdictions where it is recognized. The court described the privilege by

citing to Dowling v. American Hawaii Cruises, Inc., 971 F.2d 423, 425-426

(9th Cir. 1992), which explained:

“[O]ther courts have generally required that the party asserting the privilege demonstrate

that the material to be protected satisfies at least three criteria: first, the information must

result from a critical self-analysis undertaken by the party seeking protection; second, the

public must have a strong interest in preserving the free flow of the type of information

sought; finally, the information must be of the type whose flow would be curtailed if

discovery were allowed. . . .To these requirements should be added the general proviso that

no document will be accorded a privilege unless it was prepared with the expectation that

it would be kept confidential, and has in fact been kept confidential.”

The court found that the safety documents did not meet the criteria for the

privilege. According to the court, the most significant stumbling block for

the airline was meeting the third criterion—that the flow of the

information would be curtailed if discovery was allowed. Specifically, the

court found that the airline industry is highly competitive and tightly

regulated and that airlines have a keen interest in advancing and

promoting safety as well as services. Thus, the court reasoned that the

airlines were likely to conduct internal audits. The court reasoned that

while the disclosure of such audits to competitors would deter their use in

the future, disclosure for limited use in litigation is unlikely to have such

an impact.

30USAir changed its name to US Airways on Feb. 27, 1997.

Page 35 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix V 

Discovery-Related Court Actions

Subsequently, the court limited the possible uses of the documents it

ordered to be produced. Specifically, the court ordered:

“. . . plaintiff and their counsel shall be prohibited from disclosing, disseminating or

communicating in any manner to any person or entity not involved in this litigation any

portion of the information contained in those documents. . . . Plaintiff and their counsel

shall be further precluded from utilizing these documents or the information contained in

them for any purpose other than for this multidistrict litigation.

“In furtherance of this order, plaintiffs’ counsel shall insure that each person who is to be

given access to the referenced documents, including plaintiff and their attorneys, shall first

sign a document acknowledging that they are aware of and will comply with this order.

Plaintiffs’ counsel shall maintain a list of those persons which shall be provided to USAir’s

attorney upon request, subject to protection upon application to this court for good cause

shown.” Court Order of Nov. 14, 1995, In Re: Air Crash at Charlotte, North Carolina, on

July 2, 1994, MDL Docket No. 1041 (D.S.C. 1995).

In October 1996, the Supreme Court let stand the district court order

rejecting the airline’s assertion of a self-critical evaluation privilege. 65

U.S.L.W. 3221 (Oct. 8, 1996).

Recently, in another case involving documents prepared by American

Airlines’ employees collected under the American Airlines Safety Action

Program, the United States District Court, Southern District of Florida, on

a motion for reconsideration, also rejected the airline’s self-critical

analysis privilege claim. However, in this case the court recognized a new

qualified privilege to protect these documents. In re Air Crash Near Cali,

Colombia, on Dec. 20, 1995, 959 F. Supp. 1529 (S.D. Fla. 1997).

With respect to the self-critical analysis privilege, the court stated that “the

touchstone of a self-critical analysis is that it is an ’in house’ review

undertaken primarily, if not exclusively, for the purpose of internal quality

control.” In this case, the court rejected the application of the privilege,

finding the following:

“Even assuming that the materials prepared by American’s pilots in conjunction with the

ASAP program may be of a type whose creation might be curtailed if discovery is allowed,

these materials were prepared for dissemination to representatives of entities unaffiliated

with American (a federal regulatory agency and a union).”

The court, however, recognized a new, qualified common law privilege for

the ASAP materials. In recognizing a new privilege, the court considered the

Page 36 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix V 

Discovery-Related Court Actions

principles for evaluating claims of federal common law privileges recently

articulated in the Supreme Court case, Jaffee v. Redmond, __ U.S. __, 116

S. Ct. 1923 (1996): (1) the “private interest” involved—in other words

whether the dissemination of the information would chill the frank and

complete disclosure of fact; (2) the “public interests” furthered by the

proposed privilege; (3) the “likely evidentiary benefit that would result

from the denial of the privilege;” and (4) the extent to which the privilege

has been recognized by state courts and legislatures.31

The court found that American had met its burden of proving that a

qualified “ASAP privilege” is appropriate. Specifically, the court stated as

follows:

“The ASAP materials in dispute . . . were prepared voluntarily, in confidence and for use in a

discrete, limited context in cooperation with the FAA and the pilot’s union. There is a

genuine risk of meaningful and irreparable chill from the compelled disclosure of ASAP

materials in connection with the pending litigation.”

The court specified that the privilege should be qualified. Accordingly, the

plaintiffs could overcome the privilege with a persuasive showing of need

and hardship. The plaintiffs did not make such a showing in the case.

31The court recognized a psychotherapist-patient privilege in this case.

Page 37 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix VI 

Major Contributors to This Report

Resources,

Community, and

Economic

Development Division

David E. Bryant, Jr.

Thomas F. Noone

Robert E. White

Office of the General

Counsel

Mindi Weisenbloom

(341514) Page 38 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100