Report to Congressional Requesters

United States Government Accountability Office

GAO

September 2005 AVIATION SECURITY

Flight and Cabin Crew 

Member Security 

Training Strengthened, 

but Better Planning 

and Internal Controls 

Needed 

GAO-05-781 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-781.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Cathleen 

Berrick at (202) 512-8777 or 

berrickc@gao.gov. 

Highlights of GAO-05-781, a report to 

congressional requesters 

September 2005

AVIATION SECURITY

Flight and Cabin Crew Member Security 

Training Strengthened, but Better 

Planning and Internal Controls Needed 

Since the terrorist attacks of September 11, 2001, TSA enhanced guidance 

and standards for flight and cabin crew member security training with input 

from stakeholders. Specifically, TSA revised the guidance and standards to 

include additional training elements required by law and to improve the 

organization and clarity of the guidance and standards. Some stakeholders 

we interviewed and our own review generally found that the revised 

guidance and standards improved upon previous versions in terms of 

organization and clarity of the information provided. However, some 

stakeholders identified concerns about, for example, the reasonableness of 

applying parts of the guidance and standards to both flight and cabin crew 

members and the difficulty in implementing some of the standards without 

additional information or training tools from TSA. Additionally, TSA has not 

established strategic goals and performance measures for assessing the 

effectiveness of the training because it considers its role in the training 

program as regulatory. In this regard, TSA views the individual air carriers as 

responsible for establishing performance goals and measures for their 

training programs, but has not required them to do so. Without goals and 

measures, TSA and air carriers will be limited in their ability to fully assess 

accomplishments and target associated improvements. 

TSA recently took steps to strengthen its efforts to oversee air carriers’ flight 

and cabin crew security training to ensure they are complying with the 

required guidance and standards. For example, in January 2005, TSA added 

staff with expertise in designing training programs to review air carriers’ 

crew member security training curriculums and developed a standard form 

for staff to use to conduct their reviews. However, TSA lacks adequate 

controls for monitoring and reviewing air carriers’ crew member security 

training, including written procedures for conducting and documenting these 

reviews. TSA plans to develop written procedures, but has not established a 

timeframe for completing this effort. 

TSA has developed an advanced voluntary self-defense training program 

with input from stakeholders and implemented the program in December 

2004, as required by law. However, stakeholders and our own analysis 

identified concerns about the training design and delivery, such as the lack 

of recurrent training and the lack of a realistic training environment. Also, 

TSA has not yet established performance measures for the program or 

established a time frame for evaluating the program’s effectiveness. 

Training flight and cabin crew 

members to handle potential 

threats against domestic aircraft is 

an important element in securing 

our nation’s aviation system. The 

responsibility for ensuring that 

crew members are prepared to 

handle these threats is a shared 

responsibility between the private 

sector—air carriers—and the 

federal government, primarily the 

Transportation Security 

Administration (TSA). This report 

addresses (1) actions TSA has 

taken to develop guidance and 

standards for flight and cabin crew 

member security training and to 

measure the effectiveness of the 

training, (2) how TSA ensures 

domestic air carriers comply with 

the training guidance and 

standards, and (3) efforts TSA has 

taken to develop and assess the 

effectiveness of its voluntary selfdefense training program. 

What GAO Recommends

GAO is recommending that the 

Secretary of Homeland Security 

direct TSA to (1) establish strategic 

goals for crew security training, 

develop guidance and standards for 

air carriers to use to develop goals 

and measures for their training, and 

review air carriers’ goals and 

measures, (2) develop written 

procedures for monitoring air 

carriers’ crew security training, and 

(3) establish performance 

measures and a time frame for 

evaluating the effectiveness of the 

voluntary self-defense training. TSA

reviewed a draft of this report and 

generally agreed with GAO’s 

findings and recommendations. 
Page i GAO-05-781 Aviation Security 

Letter 1

Results in Brief 3 

Background 8 

TSA Has Enhanced Guidance and Standards for Flight and Cabin 

Crew Security Training with Input from Stakeholders, but Better 

Planning Is Needed 15 

Recent Progress Made to Strengthen Oversight of Flight and Cabin 

Crew Member Security Training Program, but Weaknesses 

Remain 20 

TSA Developed and Implemented Crew Member Self-Defense 

Training, but Has Not Established Performance Measures or 

Evaluated Training Effectiveness 26 

Conclusions 35 

Recommendations 36 

Agency Comments and Our Evaluation 37 

Appendix I Objectives, Scope, and Methodology 40 

Appendix II TSA’s Deployment of an Advanced Voluntary Crew 

Member Self-Defense Training from December 2004 

through September 2005, as of July 2005 42 

Appendix III Comments from the Department of Homeland 

Security 43 

Appendix IV GAO Contact and Staff Acknowledgments 45 

Tables 

Table 1: List of Basic Crew Member Security Training Elements 

Required By Law 13 

Table 2: List of Advanced Voluntary Crew Member Self-Defense 

Training Elements Required By Law 14 

Contents 
Page ii GAO-05-781 Aviation Security 

Table 3: Summary of Stakeholders’ Concerns on TSA’s Prototype 

Advanced Voluntary Self-Defense Training and Actions 

Taken by TSA in Response to the Concerns 29 

Table 4: Crew Member Participation in Advanced Voluntary Crew 

Member Self-Defense Training from December 2004 

through June 2005 30 

Figure 

Figure 1: Number of Domestic Air Carriers by Carrier Group and 

Percentage of Passengers Flown Domestically by Carrier 

Group 11 

Abbreviations 

ATSA Aviation and Transportation Security Act 

DHS Department of Homeland Security 

DOD Department of Defense 

DOT Department of Transportation 

FAA Federal Aviation Administration 

FAMS Federal Air Marshal Service 

FBI Federal Bureau of Investigation 

GPRA Government Performance and Results Act 

OMB Office of Management and Budget 

TSA Transportation Security Administration 

Vision 100 Vision100-Century of Aviation Reauthorization Act 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-05-781 Aviation Security 

September 6, 2005 

The Honorable Daniel K. Inouye 

Co-Chairman 

Committee on Commerce, Science, and Transportation 

United States Senate 

The Honorable John D. Rockefeller, IV 

Ranking Minority Member 

Subcommittee on Aviation 

Committee on Commerce, Science, and Transportation 

United States Senate 

Training flight and cabin crew members (pilots and flight attendants) to 

handle potential threats against domestic commercial aircraft is an 

important element in securing our nation’s aviation system. These flight 

and cabin crew members play a key role in ensuring the security of 

commercial aircraft and the safety of airline passengers and are the last 

line of defense in the event of an attempted terrorist attack onboard a 

commercial aircraft. The responsibility for ensuring that flight and cabin 

crew members are prepared to respond to these threats is a shared 

responsibility between the private sector—air carriers—and the federal 

government, primarily the Transportation Security Administration (TSA). 

In an effort to strengthen aviation security, title 49 of the United States 

Code, section 44918, requires that air carriers develop and deliver a basic 

security training program for flight and cabin crew members in 

accordance with training elements prescribed in the law. The law also 

requires that TSA monitor and periodically review air carriers’ training 

programs to ensure they adequately prepare crew members for potential 

threat conditions. Furthermore, the law requires TSA to develop and 

provide an advanced voluntary self-defense training program for flight and 

cabin crew members. Although there has not been another successful 

terrorist attack against commercial aircraft in the United States since 

September 11, 2001, concerns remain about whether flight and cabin crew 

members are prepared to handle a potential hijacking and other threats to 

commercial aircraft. 

To determine the progress TSA has made in developing and monitoring 

flight and cabin crew security training, we examined TSA’s efforts to 

develop guidance and standards for air carriers’ flight and cabin crew 

security training, monitor air carriers’ compliance with the guidance and 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-05-781 Aviation Security 

standards, and develop and deliver advanced voluntary self-defense 

training for crew members. Specifically, this report addresses the 

following questions: (1) What actions has TSA taken to develop guidance 

and standards for flight and cabin crew security training and to measure 

the effectiveness of the training? (2) How does TSA ensure domestic air 

carriers comply with required training guidance and standards? (3) What 

efforts has TSA taken to develop, implement, and measure the 

effectiveness of advanced voluntary self-defense training for flight and 

cabin crew members? 

In conducting our work, we reviewed TSA and Federal Aviation 

Administration (FAA) documentation related to flight and cabin crew 

member security training guidance and standards and advanced voluntary 

crew member self-defense training. The security training guidance 

provides detailed guidance from which air carriers must develop their 

flight and cabin crew security training programs. The security training 

standards set forth the specific requirements for crew member security 

training, such as the training elements that must be included in air carriers’ 

basic and recurrent (refresher) crew member security training programs. 

We also interviewed officials from 19 domestic air carriers, 2 air carrier 

associations, and 6 crew member labor organizations regarding security 

training guidance and standards, and advanced voluntary self-defense 

training.1

 We selected domestic air carriers based on whether they were 

currently offering initial and/or recurrent security training2

 and on the size 

of the air carrier in an effort to include a mixture of various domestic air 

carriers and air carriers of varying sizes. The size of an air carrier is based 

on the annual operating revenues and the number of revenue passenger 

boardings. We visited 8 of these air carriers to observe their training and to 

 1

Foreign air carriers operating to and from the United States are not regulated under the 

mandates of the Aircraft Operator Security Program (under which domestic air carriers 

operate) but are subject to the security requirements of 49 C.F.R. part 1546. Foreign air 

carriers operating under part 1546 must maintain a security program that TSA accepts as 

providing passengers a level of protection similar to the level of protection provided by 

U.S. air carriers serving the same airports. See 49 U.S.C. § 44906. Further, foreign air 

carriers must abide by the standards and recommended practices established by the 

International Civil Aviation Organization, which adopted expanded security training 

requirements for crew members in March 2002 that are similar to those found in the 

Aviation and Transportation Security Act (ATSA), Pub. L. No. 107-71, § 107, 115 Stat. 611, 

610-11 (2001). 

2

Initial security training is provided to newly hired flight and cabin crew members as well 

as crew members returning from a leave of absence of more than 3 years. Recurrent 

security training is annual refresher training provided to all flight and cabin crew members. 
Page 3 GAO-05-781 Aviation Security 

interview flight and cabin crew members about the training. Because we 

selected a nonprobability sample of domestic air carriers, the information 

we obtained from these interviews and visits cannot be generalized to all 

domestic air carriers. We also observed two sessions of TSA’s advanced 

voluntary crew member self-defense training. Information obtained from 

our interviews and visits cannot be generalized to all air carriers and 

stakeholders. Additionally, we interviewed officials responsible for 

developing security training guidance and standards and reviewing air 

carriers security training programs at TSA headquarters. We also 

interviewed officials at the FAA; Federal Bureau of Investigation (FBI); 

Bureau of Immigration and Customs Enforcement, Federal Air Marshal 

Service (FAMS); Department of Defense (DOD); crew member labor 

organizations; and associations representing air carriers to obtain 

information on their involvement in the development of crew member 

security training guidance and standards and advanced voluntary crew 

member self-defense training. A more detailed description of our scope 

and methodology is contained in appendix I. 

We conducted our work from June 2004 through August 2005 in 

accordance with generally accepted government auditing standards. 

Building on the legislatively mandated guidance developed by FAA in 

January 2002 and corresponding standards, TSA enhanced guidance and 

standards for flight and cabin crew member security training with input 

from various stakeholders. However, TSA has not established strategic 

goals and performance measures for assessing the effectiveness of crew 

member security training, nor required air carriers to do so. TSA officials 

stated that they revised the guidance and standards for two main reasons. 

First, the law, as amended by the Vision 100-—Century of Aviation 

Reauthorization Act (Vision 100), enacted in December 2003, required that 

air carriers include additional training elements in their basic crew 

member security training programs to prepare flight and cabin crew 

Results in Brief 
Page 4 GAO-05-781 Aviation Security 

members for potential threat conditions.3 Second, TSA determined that the 

guidance and standards needed to be better organized and to more clearly 

define security training elements, in part due to feedback from air carriers, 

flight and cabin crew member labor organizations, and associations 

representing air carriers. Some stakeholders stated and our own review 

found that the organization of the previous security training standards 

were difficult to follow and lacked clarity in some areas. For example, the 

previous guidance did not define what constitutes life-threatening 

behavior, whereas the revised guidance provides both a definition of this 

behavior and examples. Although the revised guidance and standards were 

an improvement over the previous versions in terms of organization and 

clarity, some stakeholders identified concerns about the reasonableness of 

applying parts of the guidance and standards to both flight and cabin crew 

members, difficulty in implementing some of the standards without 

additional information or training tools from TSA, and the vagueness of 

some of the guidance and standards. For example, some of the training 

standards remain generalized to both flight attendants and pilots, rather 

than targeted to their specific job functions in responding to a security 

threat. Additionally, TSA has not established strategic goals and 

performance measures for the flight and cabin crew member security 

training program. The Government Performance and Results Act (GPRA) 

of 1993 requires, among other things, that agencies use outcome-oriented 

goals and measures that assess results of a program or activity compared 

to its intended purpose. TSA training officials stated that they decided not 

to develop strategic goals or performance measures for flight and cabin 

crew security training because the officials view their role in the training 

program as regulatory—that is, monitoring air carriers’ compliance with 

the training guidance and standards established by TSA. In this regard, 

officials stated that it is the individual air carriers’ responsibility to 

establish performance goals and measures specific to their security 

 3

49 U.S.C. § 44918. The Aviation and Transportation Security Act (ATSA), enacted in 

November 2001, created the Transportation Security Administration (TSA) and mandated 

the Federal Aviation Administration (FAA) to develop a detailed guidance for a scheduled 

passenger air carrier flight and cabin crew member training program. FAA continued to be 

responsible for overseeing flight and cabin crew members’ security training until TSA 

assumed that responsibility pursuant to ATSA. Subsequently, the Vision 100—Century of 

Aviation Reauthorization Act (Vision 100), enacted in December 2003, revised the basic 

security training program requirements and mandated that TSA develop and implement an 

advanced voluntary self-defense training program for crew members. Pub. L. No. 108-176, § 

603, 117 Stat. 2490, 2563-65. Together, these acts mandated actions for TSA, FAA, and air 

carriers to strengthen flight and cabin crew security training under § 44918, including the 

development of security training guidance and standards. 
Page 5 GAO-05-781 Aviation Security 

training programs. However, without overall strategic goals developed by 

TSA, air carriers do not have a framework from which to develop their 

individual performance goals and measures. Further, TSA has not 

explicitly required air carriers to develop performance goals and measures 

or provided air carriers with guidance and standards for doing so. The 

absence of performance goals and measures for flight and cabin crew 

security training limits the ability of TSA and air carriers to fully assess the 

accomplishments of the flight and cabin crew member security training 

program in adequately training crew members, and to target appropriate 

improvements. 

Although TSA has recently taken steps to strengthen its efforts to oversee 

air carriers’ flight and cabin crew security training to ensure carriers are 

complying with required guidance and standards, TSA lacks adequate 

internal controls for monitoring and reviewing air carriers’ flight and cabin 

crew member security training. The law requires TSA to monitor and 

periodically review air carriers’ security training to ensure that the training 

is adequately preparing crew members for potential threat conditions. The 

law also requires TSA to consider complaints from crew members in 

determining when to review an air carriers’ crew member security training 

program. Further, the Comptroller General’s Standards for Internal 

Control in the Federal Government, which describes the minimum level of 

quality acceptable for internal controls in government and the basis 

against which internal controls should be evaluated, calls for, among other 

things, that controls generally be designed to assure that ongoing 

monitoring occurs during the course of normal operations, transactions 

and other significant events be documented clearly, and documentation be 

readily available for examination.4

 TSA has recently taken several steps to 

strengthen its review of air carriers’ crew member security training 

curriculum, including (1) adding staff with expertise in designing training 

programs to review the overall design of the air carriers’ crew member 

security training curriculums, rather than solely ensuring that each of the 

training elements is discussed in the training curriculums; (2) developing a 

standard form for TSA inspectors and training staff to use to conduct and 

document their reviews of air carrier security training curriculums to 

ensure training requirements stated in the revised security training 

standards are included in the air carriers’ security training and to enhance 

consistency in the review process; (3) requiring air carriers to obtain 

 4

GAO, Internal Control: Standards for Internal Control in the Federal Government, 

GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). 
Page 6 GAO-05-781 Aviation Security 

written participant feedback at the end of flight and cabin crew member 

security training to capture information on participants’ overall 

satisfaction with the training; and (4) reorganizing the inspection staff into 

a newly created Office of Compliance and issuing position announcements 

to fill vacant TSA inspector positions, which should allow for greater 

monitoring of air carriers’ flight and cabin crew security training 

programs. Although TSA made these recent improvements, the agency 

lacks adequate internal controls for monitoring and reviewing air carriers’ 

crew member security training. Specifically, TSA does not have written 

procedures for (1) completing the standard form used by TSA inspectors 

and training staff in reviewing air carriers’ flight and cabin crew security 

training curriculum; (2) conducting and documenting observations of air 

carriers’ classroom delivery of flight and cabin crew security training—a 

function performed by TSA inspectors—and selecting the air carriers to 

visit; (3) ensuring that air carriers track flight and cabin crew members’ 

completion of required security training; and (4) considering flight and 

cabin crew members’ written feedback on security training and securityrelated complaints from flight and cabin crew members in determining 

when to review air carriers’ flight and cabin crew security training. TSA 

officials stated that they plan to develop a handbook that documents 

procedures for TSA inspectors to use in reviewing air carriers’ flight and 

cabin crew members security training as well as guidance for the training 

staff to use during their reviews. However, officials stated that they have 

not established a time frame for completing these efforts. Until TSA 

documents procedures for monitoring and reviewing air carriers’ flight and 

cabin crew member security training, TSA will continue to lack a structure 

that provides reasonable assurance that TSA inspectors and training staff 

are performing thorough assessments of air carriers’ security training. 

TSA has developed an advanced voluntary self-defense training program 

for flight and cabin crew members with input from key stakeholders and 

implemented the program in December 2004 as required by law. However, 

some stakeholders and our own analysis identified concerns about the 

training design and delivery, and TSA has not yet developed performance 

measures for the program or established a time frame for evaluating the 

program’s overall effectiveness. In developing the advanced voluntary selfdefense training program, TSA consulted with law enforcement experts, 
Page 7 GAO-05-781 Aviation Security 

the Federal Air Marshal Service,5 air carriers, air carrier associations, crew 

member labor organizations, and other subject matter experts and selfdefense experts. TSA also piloted the training course in five cities and 

made refinements to the course based on feedback from participants. For 

example, training participants stated that the course included too much 

lecture time and that the training course was too long, particularly given 

that crew members must attend the training on their own time and must 

pay for their travel, lodging, and meals. In response, TSA increased the 

amount of hands-on training and shortened the training program from 

4 days to 3 days. During the first 7 months of the training program, 

participation in the training was relatively low, with only 474 flight and 

cabin crew members (39 percent of total capacity) attending the training. 

TSA officials attributed the low participation to crew members having a 

difficult time obtaining 3 consecutive days of leave to attend the training. 

Although TSA incorporated some stakeholder input into the course design, 

some stakeholders, including individuals identified as experts by TSA, and 

our own analysis identified several concerns regarding the training design 

and delivery, such as the lack of recurrent training, the lack of a realistic 

setting in which to conduct the training, and instructors’ lack of 

knowledge of crew members’ actual work environment. Our prior human 

capital work has found that in implementing a training program, an agency 

should ensure that implementation involves effective and efficient delivery 

of training6

—that is, the training should be conducted in a setting that 

approximates the participants’ working conditions and be taught by 

individuals who are knowledgeable about the subject matter and work 

environment. Furthermore, TSA has not yet developed performance 

measures for the program or established a time frame for evaluating the 

program’s overall effectiveness. TSA training officials stated that although 

they recognize the importance of measuring the effectiveness of the selfdefense training program, they have not been able to undertake this effort 

due to resource constraints within which they are operating and numerous 

internal process improvements currently under way in TSA. Without 

performance measures and an evaluation of the program’s effectiveness, 

TSA will not have meaningful information from which to determine 

 5

The Federal Air Marshal Service provides air security and promotes public confidence in 

our nation’s civil aviation system through the deployment of Federal Air Marshals who are 

specially trained civil aviation security specialists that are deployed onboard aircraft to 

protect passengers, crew, and aircraft from terrorist activities on both domestic and 

international flights. 

6

GAO, Human Capital: A Guide for Assessing Strategic Training and Development 

Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). 
Page 8 GAO-05-781 Aviation Security 

whether the voluntary training program is achieving its intended results 

and to make any needed improvements. 

To assist TSA in further strengthening its flight and cabin crew security 

training program and to provide the tools necessary to monitor the 

delivery and accomplishments of the training, we are making a number of 

recommendations to the Secretary of the Department of Homeland 

Security (DHS). These recommendations include directing the Assistant 

Secretary, Transportation Security Administration, to develop a 

framework from which to assess the accomplishments of flight and cabin 

crew member security training, including establishing strategic goals for 

the training; developing guidance for air carriers to use in establishing 

performance goals and measures for their individual training programs; 

and reviewing air carriers’ goals and measures as part of its monitoring 

efforts. We are also recommending that the Assistant Secretary, 

Transportation Security Administration, strengthen internal controls for 

monitoring and reviewing air carriers’ flight and cabin crew member 

security training programs and for documenting the results of its 

monitoring efforts; and establish performance measures for the advanced 

voluntary crew member self-defense training program and a time frame for 

evaluating the effectiveness of the training. 

We provided a draft of this report to the Department of Homeland Security 

for its review and comment. DHS, in its written comments, generally 

concurred with the findings and recommendations in the report, and 

agreed that efforts to implement our recommendations are critical to a 

successful flight and cabin crew member security training program. DHS 

described some actions TSA has taken or planned to take to implement 

these recommendations. For example, DHS stated that TSA has begun to 

establish strategic goals for the flight and cabin crew member security 

training program. DHS also stated that TSA is in the process of developing 

a monitoring plan, to the extent that resources permit, and a handbook for 

reviewing air carriers’ flight and cabin crew member security training 

programs. Additionally, DHS stated that TSA is currently working with the 

Office of Management and Budget (OMB) to establish performance 

measures for use in OMB’s Performance Assessment Rating Tool for TSA’s 

flight security training. The full text of DHS’s comments is included in 

appendix III. 

After the terrorist attacks of September 11, 2001, the President signed the 

Aviation and Transportation Security Act (ATSA) into law on November 

19, 2001, with the primary goal of strengthening the security of the nation’s 

Background 
Page 9 GAO-05-781 Aviation Security 

aviation system. ATSA created TSA as the agency responsible for securing 

all modes of transportation, including aviation.7

 The President also issued 

the National Strategy for Homeland Security in July 2002. The National 

Strategy for Homeland Security sets forth a plan to strengthen homeland 

security through the cooperation of federal, state, local, and private-sector 

organizations in various areas. The National Strategy for Homeland 

Security aligns and focuses homeland security functions into six critical 

mission areas: (1) intelligence and warning, (2) border and transportation 

security, (3) domestic counterterrorism, (4) protecting critical 

infrastructures and key assets, (5) defending against catastrophic threats, 

and (6) emergency preparedness and response. A theme of the national 

strategy is that homeland security is a shared responsibility among these 

stakeholders, not solely the responsibility of the federal government. In 

the case of flight and cabin crew member security training, air carriers and 

TSA both play an important role. Air carriers are responsible for 

developing and delivering security training programs for their crew 

members. TSA (and previously FAA) is responsible for developing the 

guidance and standards that air carriers are to use to design and deliver 

their security training and for monitoring air carriers’ flight and cabin crew 

member security training programs for compliance with the guidance and 

standards.8

 If TSA finds an air carrier to be noncompliant with developing 

and conducting the required flight and cabin crew member security 

training, TSA has a range of actions it can take, including imposing fines, 

and in extreme circumstances, force the air carrier to shut down its 

operations. 

The Bureau of Transportation Statistics reported that 105 domestic 

passenger air carriers were operating in the United States in 2004. Of the 

 7

ATSA created TSA as an agency within the Department of Transportation (DOT). The 

Homeland Security Act of 2002, Pub. L. No. 107-296, 116 Stat. 2135, signed into law on 

November 25, 2002, transferred TSA from the DOT to the new Department of Homeland 

Security. FAA continued to be responsible for overseeing flight and cabin crew members’ 

security training until TSA assumed that responsibility pursuant to ATSA. On or about 

February 17, 2002, TSA assumed responsibility for flight and cabin crew members’ security 

training. Initially, however, TSA was neither staffed nor organized to meet that 

responsibility on that date. Accordingly, while TSA “ramped up” its operations FAA’s Office 

of Aviation Security, while being largely absorbed into TSA, continued to assist TSA in its 

new roles as regulator and overseer. 

8

The flight and cabin crew security training implemented by air carriers pursuant to their 

security programs, the corresponding guidance and standards, and the related oversight 

discussed in this report are the responsibility of TSA, and not of FAA, which is responsible 

for approved training programs required under 14 C.F.R. part 121. 
Page 10 GAO-05-781 Aviation Security 

105 air carriers, 12 (11 percent) are major air carriers that carried over 

76 percent of the passengers in 2004. With a few exceptions for small 

aircraft, every commercial flight in the United States has at least two flight 

crew members and one cabin crew member onboard. These crew 

members are viewed as the last line of defense in what TSA describes as 

its layered security system, which includes perimeter security (e.g., airport 

security fencing), 100 percent passenger and checked baggage screening, 

hardened flight deck doors, armed federal air marshals, and armed pilots.9

Figure 1 provides the number of domestic air carriers by carrier group 

(major, national, and regional) and the percentage of passengers flown 

domestically by carrier group during fiscal year 2004.10

 9

Pilots (commercial, charter, and all cargo aircraft) who volunteer to participate in the 

Federal Flight Deck Officers Program are trained and armed to protect the aircraft cockpit. 

The pilots are deputized as federal flight deck officers. 

10The Bureau of Transportation Statistics defines major carriers as those with over 

$1 billion in annual operating revenues, national carriers as those with $100 million to 

$1 billion in annual operating revenues, and regional (large and medium air carriers) as 

those with up to $100 million in annual operating revenues. 
Page 11 GAO-05-781 Aviation Security 

Figure 1: Number of Domestic Air Carriers by Carrier Group and Percentage of 

Passengers Flown Domestically by Carrier Group 

Federal guidance for air carriers to use to develop their flight and cabin 

crew security training programs has been in place for over 20 years. FAA 

developed the crew member security training guidance, referred to as 

Common Strategy I, in the early 1980’s in response to numerous hijacking 

incidents in the late 1970’s. Common Strategy I generally instructed air 

carriers to develop training programs that called for flight and cabin crew 

members to cooperate with threatening passengers or hijackers and slow 

compliance with their demands. Based on this guidance, FAA also 

developed corresponding security training standards that set forth the 

requirements for flight and cabin crew member security training. Air 

carriers were required to incorporate the guidance and standards into 

their security training programs. FAA principal security inspectors and 

principal operations inspectors were responsible for monitoring air 

carriers’ compliance with the security training standards. The nature of the 

terrorist attacks on September 11, 2001, however, demonstrated that the 

philosophy of Common Strategy I—to cooperate with hijackers—was 

flawed as it presumed that hijackers would not use aircraft as weapons of 

mass destruction. 

Following the events of September 11, 2001, section 107 of ATSA required 

FAA, in consultation with TSA and other stakeholders, to develop detailed 

guidance for flight and cabin crew security training programs within 

60 days after the enactment of the act. FAA developed and issued security 

20% 76%

4%

12

21

72

Total number of domestic 

carriers by carrier group: 105

Annual operating

revenues

Percentage of passengers flown 

domestically by carrier group

Source: GAO analysis of Bureau of Transportation Statistics data.

Major

National

Regional

Over $1 billion

$100 million to $1 billion

$20 million to $100 million
Page 12 GAO-05-781 Aviation Security 

training guidance, in accordance with the requirements of ATSA, on 

January 19, 2002. In February 2002, TSA assumed responsibility for 

monitoring air carriers’ security training for United States passenger air 

carriers and the air carrier security inspections function was transferred 

from FAA to TSA.11

Following the enactment of ATSA, the President signed into law two acts 

that amended the flight and cabin crew training requirements codified at 

title 49 of the U.S. Code, section 44918—the Homeland Security Act of 

2002 and Vision 100. The Homeland Security Act, enacted on November 

25, 2002, amended the law by, among other things, mandating that, if TSA 

updated training guidance, it must issue a rule to include elements of selfdefense in the training programs.12 Vision 100, subsequently enacted on 

December 12, 2003, amended the flight and cabin crew security training 

law in its entirety to require that 

• air carriers providing scheduled passenger air transportation carry out a 

training program that addresses the 10 elements listed in table 1; 

• TSA approve the air carrier’s training programs; 

• TSA, in consultation with FAA, monitor air carrier training programs and 

periodically review an air carrier’s training program to ensure the program 

is adequately preparing crew members for potential threat conditions; 

• TSA, in consultation with FAA, order air carriers to modify training 

programs to reflect new or different security threats; and 

• TSA develop and provide an advanced voluntary self-defense training 

program to provide both classroom and effective hands-on training in, at 

least, the six training elements listed in table 2. 

 11TSA currently oversees about 84 air carriers. Bureau of Transportation Statistics data 

show that there were 105 commercial passenger air carriers in fiscal year 2004. TSA 

officials stated that this difference of 21 air carriers may be due to air carriers merging or 

going out of business. 

12TSA, however, took no action pursuant to § 1403 of the Homeland Security Act. TSA 

maintained the standards prescribed by ATSA until the passage of Vision 100, which 

prompted TSA to revise crew member security training guidance and standards. 
Page 13 GAO-05-781 Aviation Security 

Table 1 lists the minimum training elements required by law, as enacted by 

ATSA and as amended by Vision 100, for basic crew member security 

training. 

Table 1: List of Basic Crew Member Security Training Elements Required By Law 

Legislative requirements for crew member security training ATSA Vision 100

Determination of the seriousness of any occurrence • • 

Crew communication and coordination • • 

Appropriate responses to defend oneself • • 

Use of protective devices assigned to crew members • • 

Psychology of terrorists to cope with hijacker behavior and 

passenger responses 

• • 

(Live) situational training exercises regarding various threat 

conditions 

• • 

Flight deck procedures or aircraft maneuvers to defend the aircraft • • 

Recognizing suspicious activities • 

The proper commands to give passengers and attackers • 

The proper conduct of a cabin search, including explosive device 

recognition 

 • 

Source: ATSA and Vision 100. 

Table 2 lists the training elements that TSA must include in an advanced 

voluntary self-defense training program for flight and cabin crew members 

under the law, as amended by Vision 100. 
Page 14 GAO-05-781 Aviation Security 

Table 2: List of Advanced Voluntary Crew Member Self-Defense Training Elements 

Required By Law 

Legislative requirements for crew member security training 

Deterring a passenger who might present a threat 

Advanced control, striking, and restraint techniques 

Training to defend oneself against edged or contact weapons 

Methods to subdue and restrain an attacker 

Use of available items aboard the aircraft for self-defense 

Appropriate and effective responses to defend oneself including the use of force against 

an attacker 

Source: Vision 100. 

Over the years, our work on best practices in training has found that 

generally high-performing organizations follow certain key steps in 

developing and measuring the effectiveness of training programs.13 These 

steps include 

• planning—developing a strategic approach that establishes priorities and 

leverages investments in training to achieve agency results and identify the 

competencies—commonly referred to as knowledge, skills, abilities, and 

behaviors—needed to achieve organizational missions and goals, and 

measure the extent to which their employees possess these competencies; 

• design and development—identifying specific initiatives that the agency 

will use, along with other strategies, to include individual and 

organizational performance; 

• implementation—ensuring effective and efficient delivery of training 

opportunities in an environment that supports learning and change; and 

• evaluation—assessing the extent to which training efforts contribute to 

improved performance and results. 

 13GAO-04-546G and GAO, Human Capital: Selected Agencies’ Experiences and Lessons 

Learned in Designing Training and Development Program, GAO-04-291 (Washington, 

D.C.: January 30, 2004). 
Page 15 GAO-05-781 Aviation Security 

Building on the legislatively mandated guidance developed by FAA and the 

corresponding standards, TSA enhanced crew member security training 

guidance and standards with input from stakeholders in accordance with 

the law, as amended by Vision 100. TSA policy and training officials stated 

that they revised the guidance and standards for two main reasons. First, 

the law required that air carriers include additional training elements in 

their basic crew member security training programs to prepare flight and 

cabin crew members for potential threat conditions.14 Second, TSA 

determined that the guidance and standards needed to be better organized 

and to more clearly define security training elements, in part due to 

feedback from air carriers, flight and cabin crew member labor 

organizations, and associations representing air carriers.15 For example, 

stakeholders we interviewed and our own review found that the 

organization of the previous security training standards was difficult to 

follow in that several requirements were addressed in multiple sections of 

the document rather than focused in a single section. During the summer 

of 2003 and May 2004, TSA established two internal working groups 

comprised of representatives of its policy, training, regulatory, and/or legal 

 14The additional elements that Vision 100 required air carriers to include in their basic crew 

member security training programs are (1) proper commands to give passenger and 

attackers, (2) recognizing suspicious activities, and (3) proper conduct of a cabin search, 

including explosive device recognition. 

15TSA convened a 2-day working session in the summer of 2003 with representatives from 

all the associations on a number of topics related to flight and cabin crew security training. 

Information collected during these sessions, and ultimately the legislative elements set 

forth in Vision 100, triggered TSA to revise the common strategy and the standards. 

TSA Has Enhanced 

Guidance and 

Standards for Flight 

and Cabin Crew 

Security Training with 

Input from 

Stakeholders, but 

Better Planning Is 

Needed 

TSA Enhanced Flight and 

Cabin Crew Member 

Security Training Guidance 

and Standards with Input 

from Stakeholders, but 

Stakeholders Continue to 

Have Concerns 
Page 16 GAO-05-781 Aviation Security 

offices. One working group was responsible for revising the security 

training guidance, and the other working group was responsible for 

revising the corresponding security training standards—the standards 

from which air carriers must train their flight and cabin crew members. 

TSA officials stated that these working groups determined the 

reasonableness and appropriateness of the security training elements 

contained in the existing guidance and standards in place at that time and 

what additional training elements were needed. 

During the development of the revised guidance and standards, TSA 

provided external stakeholders16 with two opportunities to provide 

comments. In July 2004, the first comment period, TSA convened a 

meeting of external stakeholders to present an overview of the draft 

revised guidance and standards and to provide copies of the documents 

for their review and comment. TSA initially requested that stakeholders 

provide comments on the draft revised guidance and standards within 

2 weeks. However, in response to stakeholder concerns about the short 

comment period, TSA extended the comment period for an additional 

2 weeks. After consolidating all stakeholder comments, TSA’s internal 

working group reviewed the comments to determine which to incorporate 

in the guidance and standards. In August 2004, the second comment 

period, TSA convened additional meetings with external stakeholders—

one meeting with air carrier associations and another with crew member 

labor organizations—to review each of the stakeholders’ comments and to 

discuss changes made to the revised guidance and standards in response 

to these comments.17 In September 2004, TSA provided the stakeholders 

with a 30-day comment period on the revised guidance and standards. 

After receiving comments and determining which of the suggested 

changes to include in the revised guidance and standards, TSA issued the 

finalized guidance and standards to air carriers on January 3, 2005. 

 16These external stakeholders included representatives of air carriers, associations 

representing air carriers (Air Transport Association and Regional Airline Association), 

crew member labor organizations (Association of Flight Attendants and Air Line Pilots 

Association), and federal agencies, such as the Federal Aviation Administration, Federal 

Air Marshal Service, the Federal Bureau of Investigation, and the Department of Defense 

North American Airspace Defense Command. According to a Federal Air Marshal Service 

official, Federal Air Marshal Service officials attended meetings at TSA related to the flight 

and cabin crew security training to ensure that the TSA training was not in conflict with the 

Federal Air Marshal Service operations. 

17TSA officials stated that in the past crew member labor organizations had not been 

provided the opportunity to review and provide comments on the security training 

standards. 
Page 17 GAO-05-781 Aviation Security 

Stakeholders we interviewed and our own analysis of revisions made to 

the guidance and standards generally found the revised guidance and 

standards to be better organized and to provide some additional clarity on 

security training requirements for crew members. For example, we found 

that the previous standards only implicitly addressed the requirement for 

training on the psychology of terrorists and addressed it in multiple 

sections in the document. In contrast, the revised standards organized 

information on this requirement in a single section and clearly identified 

the requirement as “psychology of terrorists.” Additionally, the previous 

guidance did not define what constitutes life-threatening behavior, 

whereas the revised guidance provides both a definition of this behavior 

and examples. 

Although TSA made these enhancements, stakeholders we interviewed 

and stakeholders identified by TSA provided concerns about the 

reasonableness of applying parts of the guidance and standards to both 

flight and cabin crew members, the difficulty in implementing some of the 

standards without additional information or training tools from TSA, and 

the vagueness of some of the guidance and standards. Our interviews with 

officials from 19 air carriers and 8 representatives from aviation 

associations and crew member labor organizations, after the revised 

guidance and standards were finalized in January 2005, also identified 

similar concerns. Regarding the applicability of the standards to flight and 

cabin crew members, officials from 9 of the 19 air carriers that we 

interviewed stated that some of the training standards remained 

generalized to both pilots and flight attendants, rather than targeted to 

their specific job functions in responding to a security threat. For example, 

TSA requires both pilots and flight attendants to have annual hands-on 

training on how to use restraining devices. However, 2 of the 19 air 

carriers we interviewed stated that training pilots annually on how to use 

restraining devices is not necessary because pilots are trained to stay 

inside the flight deck at all times, even when an incident occurs in the 

aircraft cabin. TSA officials stated that all crew members need annual 

hands-on training on how to use restraining devices because off-duty flight 

crew members frequently fly, and if an incident occurs in the aircraft 

cabin, they will know how to use the devices. One crew member labor 

organization agreed with TSA’s position, stating that incidents could occur 

in which pilots may need to apply the restraints. Additionally, the crew 

member labor organization official stated since pilots in command are the 

security coordinators on flights, they must be familiar with the strategies, 

tactics, and techniques that flight attendants may use in defense of 

themselves, the passengers, and the aircraft. 
Page 18 GAO-05-781 Aviation Security 

Additionally, some stakeholders expressed concerns about the difficulty in 

implementing some of the standards without additional information or 

training tools from TSA. For example, officials from 12 of the 19 air 

carriers we interviewed stated that TSA had not provided sufficient 

training materials or tools to enable them to deliver certain elements of the 

security training. These air carriers stated that although they requested the 

additional information or tools, TSA responded that air carriers were 

responsible for identifying and providing the required tools needed to 

deliver the security training. A labor union organization official stated that 

relying on training organizations and air carriers to develop the training 

materials “perpetuates the disparate quality and breadth of training 

available throughout the industry, which does little to assure a common 

strategy approach to securing United States skies.” Additionally, officials 

from 4 of the 19 air carriers we interviewed expressed concerns that TSA 

did not take into consideration that some air carriers do not have the 

expertise and personnel to conduct the annual basic self-defense training. 

TSA responded that basic self-defense training is legislatively required and 

the Federal Air Marshal Service, FBI, and other agencies are willing to 

work with the air carriers on their overall flight and cabin crew security 

training. TSA officials further stated that the air carriers should have an 

established line of communication with these agencies, but if the air 

carriers are seeking a point of contact, TSA would provide agency contact 

information. According to a Federal Air Marshal Service official, a Federal 

Air Marshal Service liaison meets periodically with the air carriers and 

aviation industry associations representing the air carriers and crew 

members to discuss overall communications including flight and cabin 

crew training issues. 

Furthermore, 9 of 27 stakeholders (air carriers, associations representing 

air carriers, and crew member labor organizations) we interviewed were 

concerned about the lack of definition, guidance, and clarity for parts of 

the revised security training guidance and standards. For example, the 

crew member security training standards require that crew members 

demonstrate proficiency in various security training elements, such as the 

use of protective and restraining devices and proper conduct of a cabin 

search. However, the standards do not define proficiency. Officials from a 

crew member labor organization stated that without clear, measurable 

training objectives for individual air carrier training departments to 

determine crew member proficiency and training objectives, the likelihood 

that training quality and content will vary from air carrier to air carrier 

increases. TSA training officials stated that air carriers, in conjunction 

with their training departments, are required to develop a method for 

determining crew member proficiency in the required training elements. 
Page 19 GAO-05-781 Aviation Security 

TSA officials stated that the air carriers developed the training program, 

not TSA, and are therefore in the best position to define proficiency. TSA 

officials stated that their training staff’s review of the training materials 

include verifying that there are opportunities built into the training for 

flight and cabin crew members to demonstrate proficiency in the required 

elements. TSA officials further stated that air carriers should have the 

latitude to tailor their desired level of proficiency for the various standards 

to their individual operations. We found, however, that without standards 

for proficiency, which commonly serve as criteria for success in training 

programs, TSA will only be able to document training activity, and not the 

results of the training, i.e., whether the intended knowledge was in fact 

transferred to the training participants at a level acceptable to TSA. 

TSA has not established strategic goals or performance measures for flight 

and cabin crew member security training, nor required air carriers to do 

so. GPRA requires that agencies use outcome-oriented goals and measures 

that assess results, effects, or impacts of a program or activity compared 

to its intended purpose. GPRA also requires federal agencies to consult 

with key stakeholders—those with a direct interest in the success of the 

program—in developing goals and measures. Strategic goals explain the 

results that are expected from a program and when to expect those 

results. These goals should be expressed in a manner that could be used to 

gauge success in the future. Performance measures (indicators used to 

gauge performance) are meant to cover key facets of performance and 

help decision makers assess program accomplishments and improve 

program performance. With respect to flight and cabin crew security 

training, strategic goals would represent the key outcomes that TSA 

expects air carriers to achieve in providing flight and cabin crew member 

security training, and performance measures would gauge to what extent 

air carriers are achieving these outcomes. 

TSA training officials stated that they decided not to develop strategic 

goals or performance measures because they view their role in the crew 

member security training program as purely regulatory—that is, 

monitoring air carriers’ compliance with the training guidance and 

standards established by TSA. In this regard, TSA is the regulatory agency 

responsible for determining whether the security training program is 

adequately preparing flight and cabin crew members for potential threat 

conditions. TSA training officials also stated that due to the varying nature 

of the air carriers’ training programs, TSA believes that it is the individual 

air carriers that are responsible for establishing goals and performance 

measures specific to their security training programs and for using the 

TSA Has Not Established 

Strategic Goals or 

Performance Measures for 

Crew Member Security 

Training 
Page 20 GAO-05-781 Aviation Security 

results to make program improvements.18 However, without overall 

strategic goals established by TSA in collaboration with air carriers, air 

carriers do not have a framework from which to develop their individual 

performance goals and measures. Furthermore, TSA has not explicitly 

required air carriers to develop performance goals and measures or 

provided them with guidance and standards for doing so. Without 

guidance and standards, the 84 individual air carriers may establish 

inconsistent performance goals and measures. Additionally, the absence of 

performance goals and measures for flight and cabin crew security 

training limits the ability of TSA and air carriers to fully assess the 

accomplishments of the flight and cabin crew member security training 

and to target program improvements. 

TSA has recently taken steps to improve its oversight of air carriers’ crew 

member security training. One step includes adding staff with training 

expertise to review air carriers’ crew member security training 

curriculums to determine whether there is evidence that each applicable 

training standard is being met. When we began our review, TSA’s review of 

air carriers’ crew member security training programs was solely the 

 18In 2004, the House Appropriations Committee, in its report accompanying H.R. 4567—the 

fiscal year 2005 DHS appropriations bill—directed that TSA expeditiously promulgate 

basic, performance-based training requirements for flight attendant security training. In 

May 2005, the House Appropriations Committee’s report accompanying H.R. 2360—the 

fiscal year 2006 DHS appropriations bill—expressed concern with TSA’s lack of progress in 

developing minimum regulatory standards for basic, industrywide flight attendant security 

training programs and stated that it expects TSA to report on the status of these 

performance-based training requirements no later than July 16, 2006. 

Recent Progress Made 

to Strengthen 

Oversight of Flight 

and Cabin Crew 

Member Security 

Training Program, but 

Weaknesses Remain 

TSA Has Recently Taken 

Steps to Strengthen 

Oversight Efforts of Air 

Carriers Security Training 
Page 21 GAO-05-781 Aviation Security 

responsibility of the principal security inspectors.19 These TSA inspectors 

were responsible for conducting a regulatory review to determine whether 

air carriers’ crew member security training curriculums met the 

requirements set forth in the standards. Beginning in January 2005, TSA 

began using training staff with expertise in designing training programs to 

review the overall design of the air carriers’ crew member security training 

curriculum, how the information is to be conveyed, the expected setting of 

the practice environment, and the way in which the information is to be 

presented—and to ensure that the security training curriculum satisfies 

the required security training standards. TSA inspectors are responsible 

for identifying which standards apply to each of the air carriers, based on 

their knowledge of the air carrier’s flight operations, size of aircraft, and 

presence or absence of international routes. TSA officials stated that 

between January 2005, when the revised guidance and standards were 

issued, and August 2, 2005, the training staff were involved in the review of 

the 71 security training curriculums that had been submitted to TSA.20

In January 2005, TSA took another step to strengthen its review of air 

carriers’ flight and cabin crew member security training by developing a 

standard form for TSA inspectors and training staff to use to conduct and 

document their reviews of air carriers’ security training curriculums. Also, 

TSA developed an internal memorandum, dated January 5, 2005, that 

generally describes the review process TSA inspectors and training staff 

are to use when reviewing air carriers’ crew member security training 

curriculums. The standard form, which lists the required training elements, 

is used by TSA inspectors to document the requirements stated in the 

revised security training standards that apply to a particular air carrier, 

and by the training staff to verify that air carrier’s initial and recurrent 

training plans include the applicable requirements and to document their 

comments.21 Prior to the development of this form, there were no 

documented procedures for how the inspections were to be conducted or 

a standard form for TSA inspectors to use to document their reviews of air 

 19For this report, we refer to TSA principal security inspectors as TSA inspectors. 

20As of August 2, 2005, TSA officials stated that TSA received 73 air carriers’ security 

training curriculums. Of the 73, 71 curriculums were reviewed by the training staff and 

63 were approved by the TSA inspectors. Ten security training curriculums remain with the 

TSA inspectors for modifications. 

21According to TSA officials, not all of the requirements apply to all air carriers. For 

example, if an air carrier is not hiring new crew members, the requirements for initial 

security training would not apply to that air carrier. 
Page 22 GAO-05-781 Aviation Security 

carriers’ crew member security training curriculums. Additionally, TSA 

lacked complete documentation of its reviews of air carriers’ security 

training. Specifically, although TSA officials stated that TSA inspectors 

reviewed all 84 air carriers’ revised security training curriculums in 

response to January 2002 guidance and the corresponding standards, TSA 

was only able to provide us documentation related to 11 reviews.22 The 

Comptroller General’s Standards for Internal Control in the Federal 

Government states that agencies should document all transactions and 

other significant events and should be able to make this documentation 

readily available for examination. With the development of a standard 

form for reviewing air carriers’ security training curriculums in January 

2005, TSA was able to provide us with documentation for all 18 of the 

reviews of air carriers’ security training curriculums that TSA inspectors 

and training staff had conducted between January 2005 and April 20, 

2005.23

Additionally, in January 2005, TSA began requiring air carriers to obtain 

participant feedback at the end of crew member security training.24

According to our human capital work,25 participant feedback can be useful 

in providing the agency with varied perspectives on the effect of the 

training. However, TSA training officials stated that they are not certain 

how, if at all, they will use the participant feedback in conducting 

oversight of air carriers’ crew member security training programs. TSA 

officials stated that it is the responsibility of the individual air carriers to 

assess the results of participant feedback and to make changes to improve 

the security training as necessary. In May 2005, TSA training officials 

acknowledged that it would be useful for its inspectors to review 

participant feedback on an annual basis to assess flight and cabin crew 

members’ views of their air carriers’ security training programs and to 

 22Officials from 15 of the 19 air carriers we interviewed reported that their training 

curriculums developed in response to the January 2002 standards were reviewed and 

approved by TSA. 

23TSA officials stated that they had received 18 of the 84 air carriers’ security training 

curriculums as of April 20, 2005. 

24TSA provided air carriers with a sample form they could use to develop their end of the 

course participant evaluation. The sample form solicited feedback in a number of areas, 

such as the appropriateness of the security training course material and design, the 

instructor’s knowledge of the content of the course material, whether training exercises 

simulate tasks needed to perform the job, and the relevancy of the training to improve 

knowledge or skills to accomplish crew members’ job functions. 

25GAO-04-546G. 
Page 23 GAO-05-781 Aviation Security 

identify trends within and across air carriers. The official acknowledged 

that these results could provide TSA inspectors information they could use 

to prioritize their reviews of air carriers’ crew member security training. 

However, the official stated that reviewing the participant feedback is a 

resource intensive process that also requires a certain level of expertise 

and is not feasible for TSA to undertake at this time. Without plans for 

reviewing participant feedback, TSA is not making use of available 

information on possible deficiencies in the quality of air carriers’ security 

training programs or identifying best practices that could be shared. 

Furthermore, TSA is taking steps to address a staffing shortage among its 

TSA inspector workforce to enable greater monitoring of air carriers’ flight 

and cabin crew member security training. Specifically, on April 1, 2005, 

TSA reorganized its inspection staff into a newly created Office of 

Compliance.26 TSA officials stated that this reorganization should help 

address the staffing shortfalls that previously existed. TSA also issued 

position announcements in an effort to fill vacant inspector positions. TSA 

officials stated that they had about 23 TSA inspectors onboard when the 

inspection function transferred from FAA to TSA in February 2002. As of 

February 2005, TSA had 15 inspectors onboard, 5 of whom were in the 

position for less than 5 months. Between January 2004 and September 

2004, the TSA inspector workforce ranged from about 7 to 14 inspectors. 

TSA officials stated that a number of these staff subsequently left TSA 

because of advancement opportunities within the Department of 

Homeland Security and personal reasons. As part of TSA’s monitoring 

efforts, TSA inspectors periodically visit air carriers to observe classroom 

delivery of flight and cabin crew member security training and to review 

air carrier records documenting flight and cabin crew member completion 

of required security training. TSA officials stated that with the existing 

inspector workforce, they were only able to conduct observations of about 

25 of air carriers’ classroom delivery of flight and cabin crew member 

security training during fiscal year 2004.27 Although TSA is not required to 

observe the classroom delivery of all air carriers’ flight and cabin crew 

member security training on an annual basis, TSA officials stated that 

 26TSA inspectors were previously located in the Aviation Regulatory Inspections Division, 

which has been merged into the newly established Office of Compliance. 

27TSA officials also stated that overseeing air carriers’ crew member security training is a 

small function relative to other TSA inspector responsibilities. For example, TSA 

inspectors spend much of their time responding to air carriers’ security incidents such as 

air carriers’ failure to conduct required cabin searches or security situations involving 

hostile passengers. 
Page 24 GAO-05-781 Aviation Security 

these observations allow them to determine whether security training is 

being delivered consistent with air carriers’ approved security training 

curriculums and to identify potential problems with the training delivery. 

While TSA has taken steps to strengthen its oversight of air carriers’ crew 

member security training, TSA has not fully developed procedures for 

monitoring this training. TSA is required by law to monitor and 

periodically review air carriers’ security training to ensure that the training 

is adequately preparing crew members for potential threat conditions. The 

Comptroller General’s Standards for Internal Control in the Federal 

Government calls for controls generally to be designed to assure that 

ongoing monitoring occurs during the course of normal operations and 

that transactions and other significant events be documented clearly and 

the documentation be readily available for examination.28 We identified 

weaknesses in TSA’s controls in these areas with regard to monitoring and 

reviewing air carriers’ flight and cabin crew security training. 

First, although TSA recently developed a standard form for its inspectors 

and training staff to use in reviewing air carriers’ flight and cabin crew 

member security training, TSA has not developed procedures for 

completing this form. TSA officials acknowledged that there are no 

documented procedures or criteria for staff to use to complete the 

standard form or for determining which standards apply to individual air 

carriers and whether or not to approve an air carrier’s security training 

curriculum.29 The lack of written procedures may result in inconsistent 

assessments of the air carriers’ security training curriculums and 

inconsistent application of the standards to air carriers. Formal 

procedures for reviewing air carriers’ flight and cabin crew security 

training could provide standardization when TSA inspectors and training 

staff assess the air carriers’ security training curriculum. 

 28GAO/AIMD-00-21.3.1. 

29TSA’s inspectors generally determine which standards apply to a particular air carrier. If 

air carriers are not hiring new crew members, then they will not submit an initial security 

training curriculum but only submit the recurrent security training curriculum to TSA for 

review and approval. The training staff review the air carriers’ security training curriculum 

to ensure that the curriculum satisfies the security training standards. According to air 

carrier officials, initial security training is provided to newly hired flight and cabin crew 

members as well as crew members returning from leave of absence of more than 3 years. 

Recurrent security training is annual training provided to all flight and cabin crew 

members. 

TSA Has Not Fully 

Developed Procedures for 

Monitoring Air Carriers’ 

Crew Member Security 

Training 
Page 25 GAO-05-781 Aviation Security 

Second, TSA does not have documented procedures for conducting and 

documenting observations of air carriers’ classroom delivery of flight and 

cabin crew member security training. During fiscal year 2004, according to 

TSA officials, TSA inspectors visited about 25 air carriers to observe crew 

member security training and review files, such as records documenting 

crew members’ completion of required security training.30 TSA officials 

stated that they did not have sufficient resources to visit all 84 air carriers 

to observe their security training. We requested records documenting TSA 

inspectors’ visits to air carriers to assess the completeness and 

consistency of these reviews. However, TSA officials stated that they were 

unable to provide us with the requested documentation. Without written 

procedures to guide TSA inspectors in observing security training and 

assessing the results of their observations, its inspectors may not conduct 

comprehensive and consistent reviews. Additionally, without a mechanism 

for documenting and maintaining TSA inspectors’ reviews of air carriers’ 

security training delivery in a standard format, TSA lacks the ability to 

track the results of these reviews and identify patterns, including strengths 

and weaknesses, in training delivery within and across air carriers. In June 

2005, a TSA official stated that TSA inspectors will monitor at least one 

flight and cabin crew member training class per year to ensure the 

curriculum is being followed. TSA inspectors are to provide the results of 

the monitoring to the principal operations inspector via memo or email. 

TSA officials stated that inspections of monitoring crew member security 

training will be maintained in a database, but TSA has not established a 

time frame for completing this database or documented procedures for 

this process. 

Additionally, although the law requires TSA to consider complaints from 

crew members in determining when to review air carriers’ flight and cabin 

crew member security training programs, TSA does not have procedures 

for considering such complaints. TSA inspection officials stated that they 

were not aware of any instances in which crew members had complained 

to TSA about security. However, in the event that TSA does receive 

complaints from crew members in the future, it is important that TSA have 

 30TSA relies on air carriers to track flight and cabin crew members’ completion of required 

security training. TSA officials stated that when conducting site visits to observe air 

carriers’ crew member security training, they may, but are not required to, review air 

carriers’ records of crew member training completion, although not required to do so by 

law. However, officials stated that air carriers are accountable for their crew members’ 

completion of required security training. All 19 air carriers we interviewed stated that they 

maintain information on their flight and cabin crew members’ security training completion 

using either training records and/or a database to track the information electronically. 
Page 26 GAO-05-781 Aviation Security 

established and documented procedures to inform its inspectors of how to 

consider the complaints in reviewing air carriers’ security training 

programs. TSA officials stated that complaints from flight and cabin crew 

members will be directed to their Office of Transportation Security Policy 

for review and all decisions regarding flight and cabin crew member 

security training program modifications or policy changes will be 

evaluated and disseminated by this office. The officials also stated that if 

the complaints involve the training delivery process, the TSA inspectors 

may be required to increase the frequency of on-site inspections based 

upon an evaluation of the seriousness of the complaints that are received. 

TSA officials stated that they plan to develop a handbook for its inspectors 

and guidance for its training staff to use in monitoring and reviewing air 

carriers’ flight and cabin crew member security training to help provide 

assurance that standardized monitoring occurs. However, TSA has not 

established a time frame for completing these efforts. 

In December 2004, as required by law, TSA implemented an advanced 

voluntary crew member self-defense training program for flight and cabin 

crew members after obtaining stakeholder input.31 Participation in the 

voluntary training course has been relatively low, with only 474 flight and 

cabin crew members (39 percent of total capacity) attending the training 

during the first 7 months of the program. TSA training officials attributed 

the low participation to crew members having a difficult time in obtaining 

3 consecutive days of leave to enable them to participate in the training. 

Additionally, although TSA incorporated some stakeholder concerns into 

the course design, some stakeholders, including individuals identified as 

experts by TSA and our own analysis identified concerns regarding the 

training design and delivery, including the training’s voluntary nature, the 

setting’s lack of realism, the training’s lack of recurrence, and the 

instructor’s lack of knowledge of crew members’ actual work 

environment. TSA has not developed performance measures for the 

program or established a time frame for evaluating the program’s overall 

effectiveness, including the effectiveness of the training design and 

delivery. 

 31TSA refers to this program as Crew Member Self-Defense Training. As with the TSA’s 

basic flight and cabin crew security training program, this advanced voluntary self-defense 

training program and its related oversight is not the responsibility of FAA. 

TSA Developed and 

Implemented Crew 

Member Self-Defense 

Training, but Has Not 

Established 

Performance 

Measures or 

Evaluated Training 

Effectiveness 
Page 27 GAO-05-781 Aviation Security 

TSA developed and implemented an advanced voluntary self-defense 

training program for flight and cabin crew members in consultation with 

key stakeholders by December 12, 2004, as required by law. TSA consulted 

with law enforcement personnel, security experts with expertise in selfdefense training, representatives of air carriers, flight attendants, labor 

organizations representing flight attendants, terrorism experts, Federal Air 

Marshal Service officials, and educational institutions offering law 

enforcement training programs, in developing the self-defense training 

program and determining how to apply the training elements specified by 

law. 32 According to TSA officials, in 2002, in anticipation of having to 

develop a mandatory self-defense training program as required by the 

Homeland Security Act,33 TSA established a working group comprised of 

law enforcement experts, Federal Air Marshals,34 and other subject matter 

experts, such as aviation security experts and self-defense/martial arts 

training experts, to assess what elements should be included in the 

training. This working group collaborated on the overall program design 

and delivery, including the program goals and objectives and the course 

content and delivery method. The working group’s efforts were placed on 

hold in 2003 when TSA was advised that legislation would be enacted to 

make the training a voluntary program to be provided by TSA, rather than 

a mandatory training program to be delivered by individual air carriers. 

After the enactment of Vision 100 in December 2003, TSA continued its 

efforts to develop an advanced voluntary self-defense training program 

until the program’s official implementation in December 2004, building on 

the input of the initial working group. The overall goal of the advanced 

voluntary crew member self-defense training, as defined by TSA, is to 

enable crew members to develop a higher level of competency in selfdefense tactics to prevent or reduce the possibility of injury or death to 

 32Section 44918, as amended by Vision 100, required TSA to address several training 

elements in the advanced voluntary crew member self-defense training, including 

(1) deterring a passenger who might present a threat; (2) advanced control, striking, and 

restraint techniques; (3) training to defend oneself against edged or contact weapons; 

(4) methods to subdue and restrain an attacker; (5) use of available items aboard the 

aircraft for self-defense; and (6) appropriate and effective responses to defend oneself 

including the use of force against an attacker. 

33The amendments to § 44918 made by Vision 100 superseded those of the Homeland 

Security Act, which required TSA, in updating its training guidance, to issue a rule 

mandating the inclusion of self-defense elements into the training program delivered by air 

carriers. 

34According to Federal Air Marshal Service officials, Federal Air Marshal Service officials 

attended meetings at TSA related to the crew member self-defense training to ensure that 

the TSA training was not in conflict with the Federal Air Marshal Service procedures. 

TSA Developed and 

Implemented Advanced 

Voluntary Self-Defense 

Training in Consultation 

with Stakeholders, but 

Participation in Training 

Remains Low 
Page 28 GAO-05-781 Aviation Security 

one’s person or the takeover of an aircraft. TSA also established several 

objectives for the training, including 

• recognize potential threats before an act of violence occurs; 

• interpret behaviors that lead to potential hostile acts; 

• conclude appropriate courses of action crew members must take to avert 

hostile actions intended to injure crew members or passengers or to take 

over an aircraft; and 

• apply appropriate individual self-protection measures and self-defense 

tactics to prevent or reduce the possibility of injury or death to one’s 

person or the takeover of an aircraft. 

Prior to implementing the voluntary training in December 2004, TSA 

piloted the prototype advanced voluntary self-defense training in August 

and September 2004 in five cities with major airline hubs and refined the 

training based on comments from participants.35 The participants provided 

positive feedback in four areas, including (1) the repetitive moves taught 

throughout the course made the self-defense tactics easy to learn; (2) the 

training prepared them mentally and physically to defend themselves and 

provided a good foundation in self-defense; (3) the small class size and 

instructor to student ratio of 1 to 8 was conducive to a productive learning 

environment;36 and (4) the location of the training facility and lodging was 

well received. TSA also received feedback on changes that could be made 

to enhance the training. Table 3 provides a summary of the stakeholders’ 

concerns on TSA’s prototype advanced voluntary self-defense training and 

actions taken by TSA in response to the concerns. 

 35The prototype training was held in Alexandria, Virginia; Wilmington, California; Fort 

Worth, Texas; Miami, Florida; and Chicago, Illinois. Crew member self-defense training in 

Chicago was cancelled due to low participation. TSA used a training contractor to deliver 

the training and provide the training facilities. 

36The classes are limited to 24 volunteer crew members per site to ensure an instructor to 

student ratio of 1 to 8. 
Page 29 GAO-05-781 Aviation Security 

Table 3: Summary of Stakeholders’ Concerns on TSA’s Prototype Advanced Voluntary Self-Defense Training and Actions 

Taken by TSA in Response to the Concerns 

Comments Action taken by TSA in response to stakeholder concerns 

Training course was too long. Decreased the number of training days from 4 to 3. 

Too much lecture time. Increased the amount of hands-on training from about 70 percent to 85 percent 

of the total course time. 

Lack of aircraft cabin or cockpit to promote realistic 

training scenarios. 

Encouraged instructors to attempt to simulate the aircraft cabin through creative 

means, such as creating an aisle using chairs and having participants perform 

the tactics within the confines of the chairs. 

Some instructors were not knowledgeable in 

aviation terminology, and aircraft environment, or 

the airline industry. 

Advised the contractor to consider hiring instructors with flight attendant and pilot 

experience. 

Lack of communication in getting the notice out 

about the class scheduling. 

Committed to providing more advance notice in announcing the official training. 

Lack of information on recurrent training. Although TSA has not ruled out recurrent training in the future, TSA is currently 

not planning to conduct such training because the law only requires TSA to 

provide the advanced voluntary crew member self-defense training. Under the 

basic flight and cabin crew member security training program, air carriers are to 

provide instructions on basic defensive techniques. 

Crew members’ exposure to potential liability if a 

crew member hurts someone when using 

techniques taught in the self-defense training to 

address an incident onboard an aircraft. 

TSA advises crew members participating in the advanced voluntary crew 

member self-defense training program of the liability protection offered by 49 

U.S.C. § 44903(k), which shields individuals, including crew members, from 

liability in the event they take action to defend themselves or others based upon 

a reasonable belief that an act of criminal violence or aircraft piracy was 

occurring or about to occur on board the aircraft. 

Source: GAO analysis of interviews with stakeholders. 

As of June 2005, a total of 474 crew members had participated in the 

training in 51 classes. During the initial deployment of the advanced 

voluntary crew member self-defense training in December 2004, 

participation was only about 14 percent of the total capacity being utilized, 

and about 38 percent of total enrolled participants actually attended the 

training course. Participation increased in January through March 2005, 

but declined in April through June 2005, with only 23 percent of total 

capacity utilized in June 2005. TSA plans to offer 46 additional advanced 

voluntary self-defense training courses during the remainder of fiscal year 

2005 in 10 cities.37 According to TSA officials, TSA estimated that 

 37The ten cities include Alexandria, Virginia; Miami, Florida; Chicago, Illinois; Fort Worth, 

Texas; Wilmington, California; Broomall, Pennsylvania; Decatur/Clarkston, Georgia; San 

Francisco, California; Aurora, Colorado; and Tempe, Arizona. TSA officials stated that they 

selected the cities in which to deliver the training based on whether there was a major 

airline hub within or nearby the city. Appendix II provides information on TSA actual and 

planned deployment of advanced voluntary crew member self-defense training from 

December 2004 through September 2005. 
Page 30 GAO-05-781 Aviation Security 

approximately 21,700 crew members will participate in the training, based 

on information obtained from air carrier associations and crew member 

labor organizations. Table 4 provides information on flight and cabin crew 

member participation in the advanced voluntary self-defense training from 

December 2004 through June 2005.38

Table 4: Crew Member Participation in Advanced Voluntary Crew Member Self-Defense Training from December 2004 through 

June 2005 

Month 

Number of classes 

in which training 

took place 

Class 

capacity

Number 

enrolled

Percentage 

of capacity 

enrolled

Number 

attended 

Percentage 

of enrolled 

attended

Percentage 

of total 

capacity 

attended

December 2004 4 96 34 35% 13 38% 14%

January 2005 8 192 115 60 75 65 39

February 2005 10 240 159 66 144 91 60

March 2005 10 240 135 56 123 91 51

April 2005 7 168 63 38 49 78 29

May 2005 8 192 61 32 48 79 25

June 2005 4 96 41 43 22 54 23

Total 51 1224 608 50 474 78 39

Source: GAO analysis of TSA data. 

TSA officials stated that the low participation rate in December 2004 was 

largely due to the short advance notice they provided stakeholders 

regarding the training. TSA announced the availability of the December 

2004 training courses 6 days before the training was to take place. This 

short notice significantly restricted the ability of flight and cabin crew 

members to participate in the training because they generally submit their 

scheduling bids 30 days prior to their work schedule.39 TSA attributed the 

short notice provided to the length of time it took to reallocate funds from 

other TSA programs to enable implementation of the advanced voluntary 

 38Section 44918, as amended by Vision 100, requires that TSA develop and deliver an 

advanced voluntary crew member self-defense program to all flight and cabin crew 

members. TSA officials defined voluntary as not only allowing crew member to choose 

whether they want to attend the training, but also allowing them to observe and not 

participate in the hands-on portion of the training. 

39Scheduling bids are tentative crew member flight schedules, leave, and vacation times. 
Page 31 GAO-05-781 Aviation Security 

self-defense training program.40 TSA provided crew members with more 

than 30 days notice for the remaining sessions. However, TSA training 

officials stated that participation continued to remain low in January 2005 

due to crew members calling in sick, crew members being called for flight 

duty at the last moment, and inclement weather. TSA training officials 

stated that although they projected that crew member participation would 

increase over time, crew member participation decreased in June 2005 due 

to crew members having a difficult time obtaining 3 consecutive days of 

leave to attend training. TSA training officials also stated that based on 

their experience with the Federal Flight Deck Officer training program, 

crew members’ ability to obtain leave for the purpose of attending training 

may be associated with seasonal variances, with low participation usually 

occurring during the spring and summer months. Stakeholders, including 

air carriers and crew member labor organizations, attributed the low 

participation to crew members having to attend the training on their own 

time and pay the cost of travel, lodging, and meals. TSA training officials 

stated that they were continuing to gather information from flight and 

cabin crew members through the training contractor in an effort to 

identify the causes for the low participation and, ultimately, to try to 

address these causes. 

Stakeholders, including individuals that TSA identified as subject matter 

experts, and our own analysis identified concerns with the design and 

delivery of the advanced voluntary crew member self-defense training.41

These concerns include the training’s voluntary nature, the setting’s lack 

of realism, the training’s lack of recurrence, and the instructor’s lack of 

knowledge of crew members’ actual work environment. These same 

concerns were identified by stakeholders in response to the prototype selfdefense training. As previously stated, our prior human capital work has 

found that in implementing a training program, an agency should ensure 

 40Congress has not appropriated funds for the advanced voluntary crew member selfdefense training program. In fiscal year 2004, TSA reallocated $500,000 from other training 

programs to support the development of advanced voluntary crew member self-defense 

training. In fiscal year 2005, TSA reallocated $2 million from other programs to support the 

delivery of this training. The Senate Appropriations Committee Report, which accompanied 

H.R. 2360—the fiscal year 2006 DHS appropriations bill, as passed by the Senate—would 

provide $3 million for voluntary flight crew training. 

41We interviewed 33 stakeholders regarding TSA’s advanced voluntary self-defense training 

program, including individuals TSA identified as subject matter experts and representatives 

of air carriers, air carrier associations, crew member labor organizations, and four federal 

agencies. 

Stakeholders Identified 

Concerns about the SelfDefense Training Design 

and Delivery 
Page 32 GAO-05-781 Aviation Security 

that implementation involves effective and efficient delivery of training—

that is, the training should be conducted in a setting that approximates the 

participants’ working conditions and taught by individuals who are 

knowledgeable about the subject matter and work environment. 

In the case of the advanced voluntary self-defense training program, 13 of 

the 33 stakeholders expressed concerns about the voluntary nature of the 

training and stated that the training should be mandatory. Six of these 

13 stakeholders stated that the program’s voluntary nature is inconsistent 

with TSA’s revised security training guidance that seeks to establish a 

common strategy that would enable individuals involved in an incident 

onboard an aircraft to know what others involved will be thinking and 

doing. These same stakeholders stated that because the training is not 

mandatory, if some crew members have had the self-defense training while 

others have not, a breakdown in communication could occur. TSA training 

officials stated that because the security training standards require all 

crew members to receive training on how to communicate and coordinate 

during a disturbance, they are not concerned about the voluntary nature of 

the self-defense training program. 

Additionally, 14 of the 33 stakeholders expressed concerns about the lack 

of a realistic training setting during the delivery of advanced voluntary 

crew member self-defense training. The self-defense techniques are taught 

in an open-space setting, unlike the narrow aisles crew members have to 

work within on an actual aircraft. In the two training sessions we observed 

in two different cities, participants had to be constantly reminded by the 

instructors of the restricted training space because participants repeatedly 

made defensive moves, such as spins and wide kicks, which could not be 

performed inside an aircraft cabin. Our prior human capital work has 

found that for learning to be successful, the training environment—

training facility and equipment—should be favorable to successful 

learning.42 TSA officials stated they examined the possibility of purchasing 

aircraft simulators for the self-defense training and found that it would 

cost TSA about $100,000 per simulator. Officials stated that they have 

advised the instructors to try to create a setting, using chairs, tape, or 

other means, to simulate the narrow aisles on an aircraft. We informed 

TSA that in the two training sessions we attended, instructors did not use 

these techniques. TSA officials stated that they would follow up with the 

instructors to ensure they use these techniques. 

 42GAO-04-546G. 
Page 33 GAO-05-781 Aviation Security 

Fifteen of the 33 stakeholders also expressed concerns about the lack of 

recurrent self-defense training given that self-defense skills are difficult to 

sustain if not consistently practiced over time. Stakeholders stated that a 

3-day, one-time self-defense training course would not enable crew 

members to develop proficiency in self-defense. TSA officials responsible 

for developing the advanced voluntary training program stated that the 

self-defense training is not intended to make participants proficient in selfdefense. Rather, the training is intended to enable crew members to 

develop a higher level of competency in self-defense tactics by extending 

their knowledge and skills in the use of self-defense techniques and 

improvised weapons.43 TSA officials also stated that the key benefit of the 

training is a change in the mindset of participants that enables a greater 

awareness of threat conditions onboard an aircraft and in their daily lives. 

Additionally, they stated that it is the responsibility of the individual 

participants to practice the various self-defense techniques they were 

taught. Furthermore, they stated that although TSA is not currently 

offering a recurrent training program, it has not ruled out the possibility of 

recurrent training in the future. Eleven of the 33 stakeholders also 

expressed concerns that the self-defense training could give participants a 

false sense of security. For example, two stakeholders stated that the false 

sense of security lies in the participants who take the course once and 

expect to be skilled and proficient using the self-defense techniques 

without realizing that they may not be capable of following through when 

an incident occurs. 

Finally, 6 of the 33 stakeholders, including subject matter experts and 

crew member labor organizations, identified concerns about the lack of 

knowledge instructors had about the crew members’ actual working 

environment. While some stakeholders commended the instructors for 

their technical knowledge of crew members’ actual work environment, 

others expressed concerns that some instructors lacked technical 

knowledge and expertise of the aviation industry. For example, a training 

participant we interviewed stated that the instructor did not understand 

how safety devices onboard an aircraft operate. The instructor suggested 

inflating an emergency raft while inflight to protect the flight deck. 

However, according to the training participant, inflating an emergency raft 

in flight could injure or kill passengers and crew members. Our prior 

human capital work found that the use of instructors who are 

 43TSA defines improvised weapons as items commonly found aboard an aircraft, such as 

coffee pots, beverage carts, wine bottles, and fire extinguishers. 
Page 34 GAO-05-781 Aviation Security 

knowledgeable of the subject matter and experienced in aviation industry 

issues can help provide assurance that instructors can effectively transfer 

these skills and knowledge to others. TSA officials stated that they advised 

the training contractor to hire instructors with law enforcement, martial 

arts/self-defense, and aviation backgrounds. Additionally, TSA provided 

the instructors with training on aviation terminology so instructors could 

better communicate with the students throughout the course. 

TSA officials stated that they were aware of the stakeholder concerns 

regarding the self-defense training course. The officials stated that their 

ability to address these concerns is limited by funding constraints and 

competing priorities. TSA officials further stated that they will continue to 

work with the contractor that is delivering the training to obtain any 

information that would be beneficial to the design and implementation of 

the training program. 

TSA has not yet developed performance measures for the advanced 

voluntary crew member self-defense training program or established a 

time frame for evaluating the program’s overall effectiveness. Our prior 

human capital work on best practices in training has found that generally, 

high-performing organizations evaluate the effectiveness of their training 

programs and use the results to target performance improvements.44 In 

February 2005, TSA began conducting end-of-course evaluations—

participant feedback—of the training and is planning to assess these 

evaluations to ensure the training is consistently achieving results over 

time. Additionally, TSA will use the results to modify the training, if 

appropriate. Although these evaluations should enable TSA to assess 

participants’ views on the training facilities, materials, and instructors, 

they will not enable TSA to determine whether the training increased the 

participants’ knowledge and skills. TSA officials stated that they recognize 

the importance of developing performance measures and evaluating the 

effectiveness of the program to ensure that it is consistently achieving the 

goals and target performance improvements. Although TSA plans to 

undertake these efforts, it has not established time frames for doing so. 

TSA officials stated that the numerous internal process improvements 

currently under way in TSA that compete for time and resources will affect 

how soon the agency can establish performance measures and conduct an 

evaluation of the training program. Without performance measures and an 

 44GAO-04-546G and GAO-04-291. 

TSA Has Not Established 

Performance Measures or 

a Time Frame for 

Evaluating Training 

Effectiveness 
Page 35 GAO-05-781 Aviation Security 

evaluation of the program’s overall effectiveness, TSA will not have 

meaningful information with which to determine whether the training 

program is actually enabling crew members to develop a higher level of 

competency in self-defense tactics—the intended goal of the training 

program. 

It has been less than 4 years since TSA assumed responsibility for aviation 

security. During this period, TSA implemented numerous initiatives to 

strengthen the various layers of security in commercial aviation. These 

efforts have largely focused on passenger and checked-baggage 

screening—among the first lines of defense in preventing terrorist attacks 

on commercial aircraft. TSA has recently taken steps to ensure that flight 

and cabin crew members—the last line of defense—are prepared to 

handle potential threat conditions onboard commercial aircraft. The 

revised guidance and standards TSA developed for air carriers to follow in 

developing and delivering their flight and cabin crew member security 

training is a positive step forward in strengthening security onboard 

commercial aircraft. However, guidance and standards alone do not 

provide assurances that the training delivered by air carriers is achieving 

TSA’s intended results. TSA views its role in flight and cabin crew member 

security training as regulatory, and that air carriers are responsible for 

measuring the success of their individual training programs. We agree that 

air carriers have responsibility for assessing the effectiveness of their 

training programs. However, we believe that overall responsibility for 

ensuring that flight and cabin crew members are prepared to respond to 

terrorist threats must be shared between the air carriers and TSA. In 

supporting this partnership, TSA should establish strategic goals for the 

flight and cabin crew security training program so that air carriers can 

develop their security programs, and measure the effectiveness of these 

programs, based on desired results, or goals, clearly defined by TSA. 

Without strategic goals to inform air carriers of what is expected from 

their training programs, and in the absence of guidance and standards to 

help ensure that air carriers establish consistent performance goals and 

measures, it will be difficult for TSA and the air carriers to gauge the 

success of training programs over time and to determine how to direct 

improvement efforts most effectively. 

Additionally, while we are encouraged by the recent steps TSA has taken 

to improve its monitoring and review of air carriers’ security training 

programs, without enhanced controls, such as written procedures for TSA 

staff to follow in conducting and documenting their reviews, TSA lacks 

reasonable assurance that its monitoring and review efforts will be 

Conclusions 
Page 36 GAO-05-781 Aviation Security 

conducted in a consistent and complete manner. Furthermore, a key 

source of information on the effectiveness of air carriers’ security training 

is participant feedback on the training. TSA’s recent requirement that air 

carriers obtain written feedback from flight and cabin crew members at 

the end of security training is a step in the right direction. However, 

without a process in place for considering this information during its 

oversight efforts, TSA is not effectively utilizing available information that 

could assist it in prioritizing and focusing its monitoring and review 

activities. 

Through developing and implementing the advanced voluntary selfdefense training program, TSA took another step forward in its efforts to 

prepare flight and cabin crew members to handle potential threat 

conditions onboard commercial aircraft. However, TSA has not yet 

established performance measures or a timeframe for evaluating the 

effectiveness of the training program, including the training design and 

delivery. Congress enacted the Government Performance and Results Act 

of 1993 to focus the federal government on achieving results and providing 

objective, results-oriented information to improve congressional decision 

making. Without performance measures or a method for evaluating the 

effectiveness of the training, TSA may not have information with which to 

systematically assess the program’s strengths, weaknesses, and 

performance. Performance measures and an evaluation of the program’s 

effectiveness can assist TSA in focusing its improvement efforts and 

provide Congress with information to assess the impact of an advanced 

voluntary self-defense training program. 

To help provide TSA management with reasonable assurance that its 

security training guidance and standards for flight and cabin crew 

members are preparing crew members for potential threat conditions, and 

to enable TSA and air carriers to assess the accomplishments of the 

security training and target program improvements, we recommend that 

the Secretary of the Department of Homeland Security direct the Assistant 

Secretary, Transportation Security Administration, to take the following 

three actions: 

• establish strategic goals for the flight and cabin crew member security 

training program, in collaboration with air carriers, and communicate 

these goals to air carriers to explain the results that are expected from the 

training; 

Recommendations 
Page 37 GAO-05-781 Aviation Security 

• develop guidance and standards for air carriers to use in establishing 

performance goals and measures for their individual flight and cabin crew 

member security training programs to help ensure consistency in the 

development of goals and measures; and 

• review air carriers’ goals and measures as part of its monitoring efforts to 

help ensure that they are linked to strategic goals established by TSA and 

to assess whether the training programs are achieving their intended 

results. 

To strengthen TSA’s internal controls and help ensure that air carriers are 

complying with TSA’s guidance and standards, we also recommend that 

the Assistant Secretary, Transportation Security Administration, establish 

a time frame for finalizing written procedures for monitoring and 

reviewing air carriers’ flight and cabin crew security training. These 

procedures should address the process for completing flight and cabin 

crew member curriculum review forms, determining which standards 

apply to individual air carriers and whether or not to approve an air 

carrier’s training curriculum, conducting and documenting observations of 

air carriers’ classroom delivery of security training, reviewing air carriers’ 

security training goals and measures, and considering security related 

complaints from flight and cabin crew members. As part of its efforts to 

develop written procedures, TSA should examine ways to incorporate 

participant feedback into its monitoring and review efforts. 

In addition, to help ensure that the advanced voluntary crew member selfdefense training is achieving its intended results, we recommend that the 

Assistant Secretary, Transportation Security Administration, establish 

performance measures for the advanced voluntary crew member selfdefense training program and a time frame for evaluating the effectiveness 

of the training, including the effectiveness of the training design and 

delivery. 

We provided a draft of this report to DHS for review and comment. On 

August 29, 2005, we received written comments on the draft report, which 

are reproduced in full in appendix III. DHS generally concurred with the 

findings and recommendations in the report, and agreed that efforts to 

implement our recommendations are critical to a successful flight and 

cabin crew member security training program. With regard to our 

recommendations that TSA establish strategic goals for the flight and 

cabin crew member security training program and develop guidance and 

standards for air carriers to use in establishing performance goals and 

Agency Comments 

and Our Evaluation 
Page 38 GAO-05-781 Aviation Security 

measures for their individual flight and cabin crew member security 

training programs, DHS stated that TSA has begun efforts to establish 

strategic goals for the program and, air carriers would benefit from 

additional guidance—that is, guidance in addition to the flight and cabin 

crew security training standards—to use in establishing performance goals 

and measures for their individual flight and cabin crew security training 

programs. While TSA has established standards for air carriers’ to use in 

developing their flight and cabin crew security training, these standards do 

not include strategic goals for the training nor provide any guidance for 

establishing performance goals and measures. In addition, at the time of 

our review, TSA had not begun developing strategic goals for flight and 

cabin crew security training. Therefore, we cannot assess the extent to 

which the goals TSA is currently developing satisfy our recommendation. 

With respect to our recommendation that TSA establish a time frame for 

finalizing written procedures for monitoring and reviewing air carriers’ 

flight and cabin crew security training, DHS stated that TSA is in the 

process of developing a monitoring plan, to the extent that resources 

permit, and a handbook for reviewing air carriers’ flight and cabin crew 

member security training programs. DHS further stated that the handbook 

is currently under development and will be completed and ready for 

implementation in fiscal year 2006. Finally, regarding our recommendation 

that TSA establish performance measures for the advanced voluntary crew 

member self-defense training program and a time frame for evaluating the 

effectiveness of the training, DHS stated that TSA is working with OMB to 

establish performance measures for use in OMB’s Performance 

Assessment Rating Tool for flight security training and will finalize these 

measures using fiscal year 2005 data as the baseline. According to DHS, 

these measures will provide TSA with information that can be used in 

evaluating the effectiveness of the advanced voluntary crew member selfdefense training. DHS also stated that TSA has begun to reach out to 

stakeholders to obtain feedback on this training. TSA’s successful 

completion of these ongoing and planned activities should address the 

concerns we raised in this report. We also provided relevant sections of 

this report to FAA, FBI, and DOD for their review, and incorporated their 

technical comments into the report as appropriate.

As agreed with your office, unless you publicly announce its contents 

earlier, we plan no further distribution of this report until 2 days from the 

date of this report. At that time, we will send copies of this report to the 

Secretary of the Department of Homeland Security and the Administrator 

of the Transportation Security Administration and interested 

congressional committees. We will also make copies available to others 
Page 39 GAO-05-781 Aviation Security 

upon request. In addition, the report will be made available at no charge 

on GAO’s Web site at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact me 

at (202) 512-8777 or berrickc@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made major contributions to this report are 

listed in appendix IV. 

Cathleen A. Berrick 

Director, Homeland Security and 

 Justice Issues 
Appendix I: Objectives, Scope, and 

Methodology 

Page 40 GAO-05-781 Aviation Security 

To determine the progress TSA has made in developing and monitoring 

flight and cabin crew security training, we examined TSA’s efforts to 

develop guidance and standards for air carriers’ flight and cabin crew 

security training, monitor air carriers’ compliance with the guidance and 

standards, and develop and deliver advanced voluntary self-defense 

training for crew members. Specifically, this report addresses the 

following questions: (1) What actions has TSA taken to develop guidance 

and standards for flight and cabin crew security training and to measure 

the effectiveness of the training? (2) How does TSA ensure domestic air 

carriers comply with required training guidance and standards? (3) What 

efforts has TSA taken to develop, implement, and measure the 

effectiveness of advanced voluntary self-defense training for flight and 

cabin crew members? 

To determine the actions TSA has taken to develop guidance and 

standards for flight and cabin crew security training and to measure the 

effectiveness of the training as well as how TSA ensures domestic air 

carriers comply with required training guidance and standards, we 

obtained and analyzed relevant legislation, guidance, and standards 

developed by TSA and FAA, and TSA records documenting its reviews of 

air carriers’ security training programs. We reviewed the security training 

guidance and standards to determine whether they contained the statutory 

requirements for flight and cabin crew security training of 49 U.S.C. §

44918, as established by the Aviation and Transportation Security Act, and 

as amended by the Homeland Security Act of 2002 and Vision 100—

Century of Aviation Reauthorization Act. We also interviewed TSA 

training, policy, and inspections officials to identify their reasons for 

revising the guidance and standards and the process they used to revise 

these documents and ensure air carriers’ compliance with the guidance 

and standards. Additionally, we compared TSA’s process for monitoring 

and reviewing air carrier compliance with flight and cabin crew security 

training guidance and standards to standards for internal control in the 

federal government. To assess stakeholder involvement in the 

development of the guidance and standards and identify any stakeholder 

concerns, we interviewed officials from the FAA, FBI, FAMS, DOD, crew 

member labor organizations, and associations representing air carriers. At 

eight domestic air carriers we visited, we interviewed air carrier officials 

to obtain their views on the security training guidance and standards and 

TSA’s efforts to ensure air carriers’ compliance with the guidance and 

standards and to observe the flight and cabin crew initial or recurrent 

(refresher) security training. We selected these domestic air carriers based 

on whether they were currently offering initial and/or recurrent security 

training and on the size of the air carrier in an effort to include a mixture 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology 

Page 41 GAO-05-781 Aviation Security 

of various domestic air carriers and air carriers of varying sizes. The size 

of an air carrier is based on the annual operating revenues and the number 

of revenue passenger boardings. Finally, we conducted phone interviews 

with representatives of 11 additional domestic air carriers—which we 

selected using the same criteria we used to select the 8 air carriers to 

visit—to obtain their views on the flight and cabin crew member guidance 

and standards and TSA’s monitoring of air carriers’ compliance with these 

standards. Because we selected a nonprobability sample of domestic air 

carriers, the information we obtained from these interviews and visits 

cannot be generalized to all domestic air carriers. 

To determine the efforts TSA has taken to develop, implement, and 

measure the effectiveness of advanced voluntary self-defense training for 

flight and cabin crew members, we obtained and analyzed relevant 

legislation, TSA’s course training manual for the self-defense training, and 

feedback provided by flight and cabin crew members who participated in 

the prototype training. We also interviewed TSA training officials 

responsible for designing and implementing the voluntary advanced crew 

member self-defense training program. Additionally, we observed the final 

training in two cities. Furthermore, we interviewed relevant stakeholders, 

including representatives of air carriers; labor organizations representing 

flight attendants and pilots as well as individual flight attendants and 

pilots; aviation industry associations representing air carriers; individuals 

identified as subject matter experts or self-defense training experts; and 

federal officials at the FBI, FAMS, and FAA to determine whether TSA 

consulted them when developing the crew member self-defense training. 

We identified subject matter experts or self-defense training experts based 

on recommendations from TSA and crew member labor organizations. We 

also interviewed representatives of the 19 domestic air carriers mentioned 

above to obtain their views on the design and delivery of the advanced 

voluntary crew member self-defense training. We assessed the extent to 

which TSA incorporated stakeholder input into the training program, and 

the basis for TSA’s decisions on which stakeholder input to incorporate 

into the training. Finally, we assessed TSA’s efforts to develop the training 

programs relative to our guidance for assessing training and development 

efforts in the federal government. 

We conducted our work from June 2004 through August 2005 in 

accordance with generally accepted government auditing standards. 
Appendix II: TSA’s Deployment of an 

Advanced Voluntary Crew Member SelfDefense Training from December 2004 

through September 2005, as of July 2005 

Page 42 GAO-05-781 Aviation Security 

 2005 

Training location December 2004 January February March April May June July August September 

Alexandria, Virginia • • • • • • c c 

• • 

Miami, Florida • • • • • • c • • • 

Chicago, Illinois c • • • • • • c • • 

Fort Worth, Texas • • • • • • • • • • 

Wilmington, 

California 

• • • • c c c c • • 

Aurora, Colorado • • • • • • • • • 

Broomall, 

Pennsylvania 

 c • • c c c c • No class 

currently 

scheduled 

Decatur/Clarkston, 

Georgia 

 • • • c • c • • • 

San Francisco, 

California 

 • • • • • • • • • 

Tempe, Arizona • • • • • • • 

Source: GAO analysis of TSA data. 

Note: c = cancelled class. Crew member self-defense training was cancelled due to low participation. 

Appendix II: TSA’s Deployment of an 

Advanced Voluntary Crew Member SelfDefense Training from December 2004 

through September 2005, as of July 2005 
Appendix III: Comments from the Department 

of Homeland Security 

Page 43 GAO-05-781 Security Aviation 

Appendix III: Comments from the 

Department of Homeland Security 
Appendix III: Comments from the Department 

of Homeland Security 

Page 44 GAO-05-781 Security Aviation 
Appendix IV: GAO Contact and Staff 

Acknowledgments 

Page 45 GAO-05-781 Security Aviation 

Cathleen A. Berrick (202) 512-8777 

In addition to the contact named above, Katherine Davis, Kimberly 

Gianopoulos, Sally Gilley, Stan Kostyla, Tom Lombardi, Gary Malavenda, 

Maria Strudwick, Carol Willett, and Su Jin Yon made key contributions to 

this report. 

Appendix IV: GAO Contact and Staff 

Acknowledgments 

GAO Contact 

Acknowledgements 

(440324) 
The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER