Report to the Chairman, Subcommittee 

on Aviation, Committee on 

Transportation and Infrastructure, 

House of Representatives 

United States Government Accountability Office

GAO 

May 2005 AVIATION SECURITY

Screener Training and 

Performance 

Measurement 

Strengthened, but 

More Work Remains 

GAO-05-457 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-457.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Cathleen 

Berrick, 202-512-8777, berrickc@gao.gov. 

Highlights of GAO-05-457, a report to the 

Chairman, Subcommittee on Aviation, 

Committee on Transportation and 

Infrastructure, House of Representatives 

May 2005

AVIATION SECURITY

Screener Training and Performance 

Measurement Strengthened, but More 

Work Remains 

TSA has initiated a number of actions designed to enhance screener training, 

such as updating the basic screener training course. TSA also established a 

recurrent training requirement and introduced the Online Learning Center, 

which makes self-guided training courses available over TSA’s intranet and 

the Internet. Even with these efforts, Federal Security Directors reported 

that insufficient screener staffing and a lack of high-speed Internet/intranet 

connectivity at some training facilities have made it difficult to fully utilize 

training programs and to meet the recurrent training requirement of 3 hours 

per week, averaged over a quarter year, within regular duty hours. TSA 

acknowledged that challenges exist in recurrent training delivery and is 

taking steps to address these challenges, including factoring training into 

workforce planning efforts and distributing training through written 

materials and CD-ROMs. However, TSA has not established a plan 

prioritizing the deployment of high-speed Internet/intranet connectivity to all 

airport training facilities to facilitate screener access to training materials. 

TSA lacks adequate internal controls to provide reasonable assurance that 

screeners receive legislatively mandated basic and remedial training, and to 

monitor its recurrent training program. Specifically, TSA policy does not 

clearly specify the responsibility for ensuring that screeners have completed 

all required training. In addition, TSA officials have no formal policies or 

methods for monitoring the completion of required training and were unable 

to provide documentation identifying the completion of remedial training. 

TSA has implemented and strengthened efforts to measure and enhance 

screener performance. For example, TSA has increased the number of 

covert tests it conducts at airports, which test screeners’ ability to detect 

threat objects on passengers, in their carry-on baggage, and in checked 

baggage. These tests identified that overall, weaknesses and vulnerabilities 

continue to exist in passenger and checked baggage screening systems at 

airports of all sizes, at airports with federal screeners, and at airports with 

private-sector screeners. While these test results are an indicator of 

performance, they cannot solely be used as a comprehensive measure of any 

airport’s screening performance or any individual screener’s performance. 

We also found that TSA’s efforts to measure and enhance screener 

performance have primarily focused on passenger screening, not checked 

baggage screening. For example, TSA only uses threat image software on 

passenger screening X-ray machines, and the recertification testing program 

does not include an image recognition module for checked baggage 

screeners. TSA is taking steps to address the overall imbalance in passenger 

and checked baggage screening performance data. TSA also established 

performance indexes for the passenger and checked baggage screening 

systems, to identify an overall desired level of performance. However, TSA 

has not established performance targets for each of the component 

indicators that make up the performance indexes, including performance 

targets for covert testing. TSA plans to finalize these targets by the end of 

fiscal year 2005. 

The screening of airport passengers 

and their checked baggage is a 

critical component in securing our 

nation’s commercial aviation 

system. Since May 2003, GAO has 

issued six products related to 

screener training and performance. 

This report updates the information

presented in the prior products and 

incorporates results from GAO’s 

survey of 155 Federal Security 

Directors—the ranking 

Transportation Security 

Administration (TSA) authority 

responsible for the leadership and 

coordination of TSA security 

activities at the nation’s 

commercial airports. Specifically, 

this report addresses (1) actions 

TSA has taken to enhance training 

for passenger and checked baggage 

screeners and screening 

supervisors, (2) how TSA ensures 

that screeners complete required 

training, and (3) actions TSA has 

taken to measure and enhance 

screener performance in detecting 

threat objects. 

What GAO Recommends

GAO is recommending that the 

Secretary of Homeland Security 

direct TSA to develop a plan for 

completing the deployment of highspeed connectivity at airport 

training facilities, and establish and 

communicate appropriate internal 

controls for monitoring the 

completion of training. 

TSA reviewed a draft of this report 

and generally agreed with GAO’s 

findings and recommendations. 
Page i GAO-05-457 Screener Training and Performance Measurement Strengthened 

Letter 1

Results in Brief 3 

Background 7 

TSA Has Enhanced and Expanded Training, but Some Screeners 

Have Encountered Difficulty Accessing and Completing 

Recurrent Training 14 

TSA Lacks Adequate Controls to Provide Reasonable Assurance 

That Screeners Receive Required Training 26 

Progress Has Been Made in Implementing Tools to Measure and 

Enhance Screener Performance, but Key Performance Targets 

Have Not Been Finalized 30 

Conclusions 45 

Recommendations for Executive Action 46 

Agency Comments and Our Evaluation 47 

Appendix I Summary of Previous Findings Related to Screener 

Training and Performance 49 

Appendix II Objectives, Scope, and Methodology 51 

Appendix III TSA Screener Training Tools Designed to Help 

Improve Screener Performance 55 

Appendix IV Summary of TSA’s Short-Term Action Items for 

Strengthening Passenger Screener Performance 57 

Appendix V Comments from the Department of Homeland 

Security 58 

Appendix VI GAO Contacts and Acknowledgments 60 

Contents 
Page ii GAO-05-457 Screener Training and Performance Measurement Strengthened 

Tables 

Table 1: Categories of Required Training Provided to TSA 

Screeners 13 

Table 2: Training Needed to Enhance Screener Supervision, as 

Reported by FSDs 20 

Table 3: Checkpoint and Checked Baggage Tests Conducted by 

OIAPR, September 10, 2002–September 30, 2004 33 

Table 4: Modules Included in Recertification Knowledge and Skills 

Assessment 38 

Table 5: Screener Recertification Module Testing Percentage Pass 

Rates, October 2003—March 2004 39 

Table 6: Recertification Testing Modules by Screening Function 40 

Table 7: Components of TSA’s Performance Indexes 44 

Figures 

Figure 1: Commercial Airports by Airport Security Category as of 

December 2003 8 

Figure 2: Passenger Checkpoint and Checked Baggage Screening 

Operations 11 

Figure 3: Description of Screening-Related Positions 12 

Figure 4: Newly Hired Screeners Trained as Dual-Function 

Screeners between April 1, 2004, and September 1, 2004 16 

Figure 5: Percentage of Airports Reported to Have High-Speed 

Connectivity for Training Purposes as of October 2004 25 

Figure 6: Screener Recertification Results, October 2003–

March 2004 39 
Page iii GAO-05-457 Screener Training and Performance Measurement Strengthened 

Abbreviations 

ATSA Aviation and Transportation Security Act 

DHS Department of Homeland Security 

DOT Department of Transportation 

EDS explosive detection system 

ETD electronic trace detection 

FAA Federal Aviation Administration 

FSD Federal Security Director 

IED improvised explosive device 

MBS modular bomb set 

MTAT Mobile Training Assist Team 

OIAPR Office of Internal Affairs and Program Review 

OIG Office of the Inspector General 

SIDA security identification display area 

TIP Threat Image Projection 

TSA Transportation Security Administration 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-05-457 Screener Training and Performance Measurement Strengthened 

May 2, 2005 

The Honorable John Mica 

Chairman, Subcommittee on Aviation 

Committee on Transportation and Infrastructure 

House of Representatives 

Dear Mr. Chairman:

The screening of airport passengers and their checked baggage is a critical 

component in securing our nation’s commercial aviation system. In an 

effort to strengthen the security of commercial aviation, the President 

signed the Aviation and Transportation Security Act on 

November 19, 2001. The act created the Transportation Security 

Administration (TSA) and mandated actions designed to strengthen 

aviation security, including requiring that TSA assume responsibility for 

conducting passenger and checked baggage screening at over 450 

commercial airports in the United States by November 19, 2002. It has 

been over 2 years since TSA assumed this responsibility, and the agency 

has spent billions of dollars and implemented a wide range of initiatives to 

enhance its passenger and checked baggage screening operations. Despite 

the attention to passenger and checked baggage screening operations, 

however, concerns about the effectiveness of the screening system 

remain. For example, covert testing conducted by TSA’s Office of Internal 

Affairs and Program Review and the Department of Homeland Security’s 

(DHS) Office of Inspector General identified weaknesses in the ability of 

screeners to detect threat objects. (The results of our analysis of TSA’s 

covert testing data and test program are included in a separate classified 

GAO report.) 

To determine the progress TSA has made in strengthening its passenger 

and checked baggage screening operations, the Subcommittee on 

Aviation, House Committee on Transportation and Infrastructure, 

requested that we examine TSA efforts to train screeners and to measure 

and enhance screener performance. Since we began our work in May 2003, 

we have issued six products that address issues related to screener 

training and performance, including four to this Subcommittee (see app. 

I). This report updates some of the information presented in our prior 

 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-05-457 Screener Training and Performance Measurement Strengthened 

products. In addition, it incorporates results from our surveys of 155 

Federal Security Directors (FSD).1

 The surveys were designed to obtain 

information related to, among other issues, TSA’s efforts to train screeners 

and supervisors and assess screener performance in detecting threat 

objects. Specifically, this report addresses the following questions: 

(1) What actions has TSA taken to enhance training for passenger and 

checked baggage screeners and screening supervisors? (2) How does TSA 

ensure that screeners complete required training? (3) What actions has 

TSA taken to measure and enhance screener performance in detecting 

threat objects? 

In conducting our work, we reviewed TSA documentation related to 

screener training requirements and performance testing. We also analyzed 

data from our survey responses from all 155 FSDs about screener training, 

supervision and performance. We also visited 29 airports of various sizes 

and geographic locations to obtain a cross-section of all airports, including 

the 5 airports with private-sector screeners.2

 To gain a better 

understanding of training and performance issues, during these visits, we 

interviewed FSDs, members of their management teams, passenger and 

checked baggage screeners, and airport officials. However, information 

obtained during these visits cannot be generalized to all airports across the 

nation. Additionally, we interviewed officials at TSA headquarters and 

TSA’s transportation security laboratory about their experiences with 

training and screener performance. We compared TSA practices and 

 1

The Federal Security Director is the ranking TSA authority responsible for the leadership 

and coordination of TSA security activities at the nation’s commercial airports. We sent 

two surveys—a general survey and an airport-specific survey—to all 155 Federal Security 

Directors on March 23, 2004. In the general survey, we asked each Federal Security 

Director to answer security-related questions that pertain to all of the airports for which 

he/she is responsible. In the airport-specific survey, we asked Federal Security Directors a 

number of airport-specific questions about screening and other security concerns for one 

or two airports, depending on the number of airports they were responsible for. By early 

May 2004, we had received responses from 100 percent of the Federal Security Directors 

for both surveys.

2

Pursuant to section 108 of the Aviation and Transportation Security Act (ATSA), TSA 

conducted a 2-year private screening pilot program at five airports—one in each airport 

security category (Pub. L. No. 107-71). The mission of the pilot, as defined by TSA, was to 

test the effectiveness of using private screening contractors in a post-September 11 

environment. The pilot concluded on November 18, 2004. On November 19, 2004, 

consistent with ATSA, TSA began allowing airports to apply to opt out of using federal 

screeners in favor of private contractors. For additional information on TSA’s progress in 

developing the opt-out program, see GAO, Aviation Security: Preliminary Observations 

on TSA’s Progress to Allow Airports to Use Private Passenger and Baggage Screening 

Services, GAO-05-126 (Washington, D.C.: Nov. 19, 2004). 
Page 3 GAO-05-457 Screener Training and Performance Measurement Strengthened 

procedures for monitoring completion of training with the Comptroller 

General’s Standards for Internal Control in the Federal Government. We 

assessed the reliability of the data we acquired from TSA regarding 

screener testing and training completion, and found the data to be 

sufficiently reliable for our purposes. A more detailed description of our 

scope and methodology is contained in appendix II. 

We conducted our work from May 2003 through April 2005 in accordance 

with generally accepted government auditing standards. 

TSA has initiated a number of actions designed to enhance passenger 

screener, checked baggage screener, and screener supervisor training. 

However, at some airports screeners encountered difficulty accessing and 

completing recurrent (refresher) training because of technological and 

staffing constraints. Among the actions TSA has taken to enhance training 

are changes and updates to the basic training program. For example, TSA 

added a modular dual-function course during basic training that covers 

passenger and checked baggage screening functions and allows newly 

hired screeners to perform either function upon completion of the 

training. TSA also established a requirement for recurrent screener 

training and developed and introduced the Online Learning Center, which 

makes self-guided training courses available to employees over TSA’s 

intranet and the Internet. In addition, TSA has provided Federal Security 

Directors with hands-on training tools to use for local recurrent training 

and testing. Finally, TSA has taken steps to provide leadership and 

technical training to Screening Supervisors. Despite these improvements, 

some Federal Security Directors, in response to open-ended survey 

questions, identified a desire for more training in specific areas, including 

leadership, communication, and supervision. Further in survey responses, 

Federal Security Directors reported that largely because of insufficient 

screener staffing, screeners were not always able to meet the recurrent 

training requirement within regular duty hours. Federal Security Directors 

at some airports also reported ongoing problems with a lack of high-speed 

Internet/intranet connectivity, which severely limited screener access to 

the Online Learning Center and diminished its value as a learning tool. As 

of October 2004, nearly half of the screener workforce did not have highspeed access to the Online Learning Center at their training facility. TSA 

has acknowledged that challenges exist in recurrent screener training 

delivery and is taking steps to address these challenges, including 

factoring training requirements into workforce planning efforts and 

distributing training through written materials and CD-ROMs until full 

Internet/intranet connectivity is achieved. However, TSA does not have a 

plan for prioritizing and scheduling the deployment of high-speed 

Results in Brief 
Page 4 GAO-05-457 Screener Training and Performance Measurement Strengthened 

connectivity to all airport training facilities once funding is available. The 

absence of such a plan limits TSA’s ability to make prudent decisions 

about how to move forward with deploying connectivity to all airports to 

provide screeners access to online training. 

TSA lacks adequate internal controls to provide reasonable assurance that 

screeners are receiving legislatively mandated basic and remedial training, 

and to monitor the status of its recurrent training program. The 

Comptroller General’s Standards for Internal Control call for (1) areas of 

authority and responsibility to be clearly defined and appropriate lines of 

reporting established, (2) transactions and other significant events to be 

documented clearly and documentation to be readily available for 

examination, and (3) controls generally to be designed to ensure that 

ongoing monitoring occurs in the course of normal operations.3

 In 

addition, the standards also require that information be communicated 

within an organization to enable individuals to carry out their internal 

control responsibilities. However, our review of TSA’s training program 

noted weaknesses in each of these control areas. First, TSA policy does 

not clearly define the responsibility for ensuring that screeners have 

completed all required training. Second, TSA officials were unable to 

produce documentation of remedial training completion for our 

examination. Third, TSA has no formal policies for monitoring the 

completion of required training. Moreover, because of the lack of highspeed Internet/intranet connectivity at some airports, staff had to manually 

input training data that would otherwise be recorded automatically by the 

Online Learning Center, making it challenging for some airports to keep 

accurate and up-to-date training records. TSA headquarters officials 

acknowledged that it can be difficult for airports to keep the Online 

Learning Center up-to-date with the most recent training records without 

high-speed connectivity. 

TSA has improved its efforts to measure and enhance screener 

performance. In September 2003, we first reported on the need for TSA to 

strengthen its efforts to measure and enhance screener performance. At 

that time, TSA had collected limited data on screener performance. 

Specifically, limited covert testing—unannounced undercover tests in 

which TSA agents attempt to pass threat objects through screening 

checkpoints and in checked baggage—had been performed, the Threat 

 3

GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 

(Washington, D.C.: November 1999). 
Page 5 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Image Projection system was not fully operational,4 and TSA had not fully 

implemented the annual screener recertification program.5

 Since then, TSA 

has implemented and strengthened efforts to collect screener performance 

data as part of its overall effort to enhance screener performance. For 

example, TSA has increased the number of covert tests it conducts at 

airports. These tests have identified that overall, weaknesses and 

vulnerabilities continue to exist in the passenger and checked baggage 

screening systems at airports of all sizes, at airports with federal 

screeners, and at airports with private-sector screeners. While these test 

results are an indicator of screener performance, they cannot solely be 

used as a comprehensive measure of any airport’s screening performance 

or any individual screener’s performance. TSA has also implemented other 

efforts to measure and enhance screener performance. However, these 

efforts have primarily focused on passenger screening, not checked 

baggage screening. Specifically, the Threat Image Projection system is 

only available for passenger screening; the recertification testing program 

does not include an image recognition module for checked baggage 

screeners; and the screener performance improvement study focused 

solely on passenger screeners. TSA is taking steps to address the overall 

imbalance in passenger and baggage screening performance data, 

including working toward implementing the Threat Image Projection 

system for checked baggage screening and developing an image 

recognition module for checked baggage screener recertification. TSA has 

also implemented a number of other improvements, which it identified in a 

passenger screener performance improvement study and incorporated 

into a screener performance improvement plan. Furthermore, TSA has 

established two performance indexes for the screening systems—one for 

passenger and one for checked baggage screening.6

 These indexes 

measure overall performance through a composite of component 

indicators. However, TSA has not established performance targets for 

each of the component indicators—such as covert testing—which would 

 4

The Threat Image Projection system is designed to test screeners’ detection capabilities by 

projecting threat images, including images of guns and explosives, into bags as they are 

screened. Screeners are responsible for positively identifying the threat image and calling 

for the bag to be searched. 

5

ATSA requires that TSA collect performance information on all passengers and baggage 

screeners by conducting an annual proficiency evaluation to ensure each screener 

continues to meet all qualifications and standards related to the functions he or she 

performs. 

6

TSA’s performance indexes show how well screening systems are functioning on a scale of 

1 to 5, using a weighted average of the values of four distinct performance indicators. 
Page 6 GAO-05-457 Screener Training and Performance Measurement Strengthened 

allow it to draw more meaningful conclusions about its performance and 

most effectively direct its improvement efforts. Although TSA has not yet 

established performance targets for each of the component indicators, 

TSA plans to finalize performance targets for the indicators by the end of 

fiscal year 2005. 

Certain information we obtained and analyzed regarding screener training 

and performance is classified or is considered by TSA to be sensitive 

security information. Accordingly, the results of our review of this 

information are not included in this report.7

To help ensure access to and completion of required training, we are 

making recommendations to the Secretary of the Department of Homeland 

Security to direct the Assistant Secretary, Transportation Security 

Administration, to (1) develop a plan for completing the deployment of 

high-speed Internet/intranet connectivity to all TSA’s airport training 

facilities and (2) establish appropriate responsibilities and other internal 

controls for monitoring and documenting screener compliance with 

training requirements. 

We provided a draft copy of this report to DHS for review. DHS, in its 

written comments, generally concurred with the findings and 

recommendations in the report and agreed that efforts to implement our 

recommendations are critical to successful passenger and checked 

baggage screening training and performance. DHS described some actions 

TSA had taken or planned to take to implement these recommendations. 

DHS also stated that TSA had already developed a plan for prioritizing and 

scheduling the deployment of high-speed Internet/intranet connectivity to 

all TSA airport training facilities. However, although we requested a copy 

of the plan several times during our review and after receiving written 

comments from DHS, TSA did not provide us with a copy of this plan. 

Therefore, we cannot assess the extent to which the plan DHS referenced 

in its written comments fulfills our recommendation. Additionally, DHS 

stated that it is taking steps to define responsibilities for monitoring the 

 7

We issued two additional reports detailing the results of our review, which discuss 

information deemed to be classified or sensitive security information. The report 

containing sensitive security information is GAO, Aviation Security: Screener Training 

and Performance Measurement Strengthened, but More Work Remains, GAO-05-143SU

(Washington D.C.: Feb. 28, 2005). The report containing classified and sensitive security 

information is GAO, Aviation Security: Results of Transportation Security 

Administration’s Covert Testing for Passenger and Checked Baggage Screening for 

September 2002 through September 2004, GAO-05-437C (Washington D.C.: Apr. 7, 2005). 
Page 7 GAO-05-457 Screener Training and Performance Measurement Strengthened 

completion of required training, and to insert this accountability into 

performance plans of all TSA supervisors. The full text of DHS’s comments 

is included in appendix V. 

The performance of passenger and checked baggage screeners in 

detecting threat objects at the nation’s airports has been a long-standing 

concern. In 1978, screeners failed to detect 13 percent of the potentially 

dangerous objects that Federal Aviation Administration (FAA) agents 

carried through airport screening checkpoints during tests. In 1987, 

screeners did not detect 20 percent of the objects in similar tests. In tests 

conducted during the late 1990s, as the testing objects became more 

realistic, screeners’ abilities to detect dangerous objects declined further. 

In April 2004, we, along with the DHS Office of the Inspector General 

(OIG), testified that the performance of screeners continued to be a 

concern. More recent tests conducted by TSA’s Office of Internal Affairs 

and Program Review (OIAPR) also identified weaknesses in the ability of 

screeners to detect threat objects, and separate DHS OIG tests identified 

comparable screener performance weaknesses. In its July 2004 report, The 

National Commission on Terrorist Attacks Upon the United States, known 

widely as the 9/11 Commission, also identified the need to improve 

screener performance and to better understand the reasons for 

performance problems.8

 

After the terrorist attacks of September 11, 2001, the President signed the 

Aviation and Transportation Security Act (ATSA) into law on November 

19, 2001, with the primary goal of strengthening the security of the nation’s 

aviation system. ATSA created TSA as an agency with responsibility for 

securing all modes of transportation, including aviation.9

 As part of this 

responsibility, TSA oversees security operations at the nation’s more than 

450 commercial airports, including passenger and checked baggage 

screening operations. Prior to the passage of ATSA, air carriers were 

responsible for screening passengers and checked baggage, and most used 

private security firms to perform this function. FAA was responsible for 

ensuring compliance with screening regulations. 

 8

The National Commission on Terrorist Attacks Upon the United States: The 9/11 

Commission Report (Washington, D.C., July 2004). 

9

Consistent with ATSA, TSA was created as an agency within the Department of 

Transportation (DOT) with responsibility for securing all modes of transportation, 

including aviation. The Homeland Security Act of 2002, signed into law on 

November 25, 2002, transferred TSA from the DOT to the new Department of Homeland 

Security (Pub. L. No. 107-296). 

Background 
Page 8 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Today, TSA security activities at airports are overseen by FSDs. Each FSD 

is responsible for overseeing security activities, including passenger and 

checked baggage screening, at one or more commercial airports. TSA 

classifies the over 450 commercial airports in the United States into one of 

five security risk categories (X, I, II, III, and IV) based on various factors, 

such as the total number of takeoffs and landings annually, the extent to 

which passengers are screened at the airport, and other special security 

considerations. In general, category X airports have the largest number of 

passenger boardings and category IV airports have the smallest. TSA 

periodically reviews airports in each category and, if appropriate, updates 

airport categorizations to reflect current operations. Figure 1 shows the 

number of commercial airports by airport security category as of 

December 2003. 

Figure 1: Commercial Airports by Airport Security Category as of December 2003 

Note: TSA periodically reviews and updates airport categories to reflect current operations. We used 

the categories in place in December 2003 to conduct our analyses during this review. 

Total: 454

X

Category

Source: GAO analysis of TSA data.

21 Airports

I

II

III

IV

61 Airports

51 Airports

197 Airports 124 Airports

4.6%

13.4%

11.2%

27.3%

43.3%
Page 9 GAO-05-457 Screener Training and Performance Measurement Strengthened 

In addition to establishing TSA and giving it responsibility for passenger 

and checked baggage screening operations, ATSA set forth specific 

enhancements to screening operations for TSA to implement, with 

deadlines for completing many of them. These requirements included 

• assuming responsibility for screeners and screening operations at more 

than 450 commercial airports by November 19, 2002; 

• establishing a basic screener training program composed of a minimum of 

40 hours of classroom instruction and 60 hours of on-the-job training; 

• conducting an annual proficiency review of all screeners; 

• conducting operational testing of screeners;10 

• requiring remedial training for any screener who fails an operational test; 

and 

• screening all checked baggage for explosives using explosives detection 

systems by December 31, 2002.11

Passenger screening is a process by which authorized TSA personnel 

inspect individuals and property to deter and prevent the carriage of any 

unauthorized explosive, incendiary, weapon, or other dangerous item 

aboard an aircraft or into a sterile area.12 Passenger screeners must inspect 

individuals for prohibited items at designated screening locations.13 The 

four passenger screening functions are: 

 10TSA defines an operational screening test as any covert test of a screener conducted by 

TSA, on any screening function, to assess the screener’s threat item detection ability and/or 

adherence to TSA-approved procedures. 

11Pursuant to the Homeland Security Act, the deadline for screening all checked baggage 

using explosive detection systems was, in effect, extended until December 31, 2003. 

12Sterile areas are located within the terminal where passengers wait after screening to 

board departing aircraft. Access to these areas is controlled by TSA screeners at 

checkpoints where they conduct physical screening of passengers and their carry-on 

baggage for weapons and explosives. 

13Screeners must deny passage beyond the screening location to any individual or property 

that has not been screened or inspected in accordance with passenger screening standard 

operating procedures. If an individual refuses to permit inspection of any item, that item 

must not be allowed into the sterile area or aboard an aircraft. 
Page 10 GAO-05-457 Screener Training and Performance Measurement Strengthened 

• X-ray screening of property, 

• walk-through metal detector screening of individuals, 

• hand-wand or pat-down screening of individuals, and 

• physical search of property and trace detection for explosives. 

Checked baggage screening is a process by which authorized security 

screening personnel inspect checked baggage to deter, detect, and prevent 

the carriage of any unauthorized explosive, incendiary, or weapon 

onboard an aircraft. Checked baggage screening is accomplished through 

the use of explosive detection systems14 (EDS) or explosive trace detection 

(ETD) systems,15 and through the use of alternative means, such as manual 

searches, K-9 teams, and positive passenger bag match,16 when EDS and 

ETD systems are unavailable on a temporary basis. Figure 2 provides an 

illustration of passenger and checked baggage screening operations. 

 14Explosive detection systems use probing radiation to examine objects inside baggage and 

identify the characteristic signatures of threat explosives. EDS equipment operates in an 

automated mode. 

15Explosive trace detection works by detecting vapors and residues of explosives. Human 

operators collect samples by rubbing bags with swabs, which are chemically analyzed to 

identify any traces of explosive materials. 

16Positive passenger bag match is an alternative method of screening checked baggage, 

which requires that the passenger be on the same aircraft as the checked baggage.
Page 11 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Figure 2: Passenger Checkpoint and Checked Baggage Screening Operations 

Manual searches

Manual or ETD searches

2. Checked Baggage Screening

2a. Primary screening

2b. Alternative screening

1. Passenger Checkpoint Screening

TSA certified K-9 teams

Video surveillance

Source: GAO and Nova Development Corporation.

Passenger and baggage screening functions

X-ray

X-ray

X-ray screening

Physical barriers

(walls/partitions)

Physical barriers

(walls/partitions)

Types of Screening:

1. Passenger

2. Checked baggage

Explosive detection

system (EDS)

Explosive trace

detection (ETD)

Explosive detection

or

Only if passenger is identified or 

randomly selected for additional 

screening; meets other particular 

criteria or to resolve hand-wand alerts.

Only if the passenger is identified 

or randomly selected for additional 

screening or if screener identifies a 

potential prohibited item on X-ray.

Walk-through metal detector

and X-ray screening

Hand-wand or pat-down

Positive passenger

bag match
Page 12 GAO-05-457 Screener Training and Performance Measurement Strengthened 

There are several positions within TSA for employees that perform and 

directly supervise passenger and checked baggage screening functions. 

Figure 3 provides a description of these positions. 

Figure 3: Description of Screening-Related Positions 

(1) identifies, distributes, and balances workload and tasks among screeners; (2) makes necessary 

adjustments to accomplish the screening workload; (3) trains or arranges for the technical training of 

screeners; (4) monitors and reports on the status and progress of screening work; (5) maintains records of 

work accomplishments and administrative information; (6) represents a team of screeners for the 

purpose of obtaining resources, and secures needed information or decisions from the supervisor on 

major work problems and issues that arise; (7) serves as a coach and facilitator to a team of 

screeners; (8) resolves simple, informal complaints of screeners and refers formal grievances to 

the appropriate management official; and (9) approves leave as delegated by management. 

Screening

Manager

Screening 

Supervisor

Screener

(1) manages screening operations; (2) ensures quality and consistency of screening procedures; (3) schedules screening 

personnel to screening operations; (4) manages overall screening workforce issues; (5) manages external relationships; 

and (5) interprets technical aspects of TSA policies, regulations, and directives. 

(1) provides frontline security protection to air travelers, airports, and airplanes; 

(2) wands and performs pat-down searches, operates an X-ray machine, 

screens baggage, and reviews tickets; (3) identifies dangerous objects on 

passengers or in baggage or cargo, and prevents these objects from 

being transported onto aircraft; and (4) uses diverse electronic 

detection and imaging equipment. 

(1) performs preboard screening of persons and their carry-on 

or checked baggage; (2) implements security-screening 

procedures that are central to TSA objectives and will 

serve to protect the traveling public by preventing any 

deadly or dangerous objects from being transported 

onto the aircraft; and (3) assists in monitoring the 

flow of passengers through the screening 

checkpoint. 

Positions Major functions

Source: GAO analysis of TSA data.

Lead 

Screener

(Number of positions TSA

was budgeted for in FY04)

(1,210 positions)

(3,419 positions)

(3,891 positions)

(37,023 positions)
Page 13 GAO-05-457 Screener Training and Performance Measurement Strengthened 

To prepare screeners to perform screening functions, to keep their skills 

current, and to address performance deficiencies, TSA provides three 

categories of required screener training. Table 1 provides a description of 

the required training. 

Table 1: Categories of Required Training Provided to TSA Screeners 

Category of 

training Description of training requirement 

Basic training ATSA requires newly hired screeners to complete basic training 

composed of a minimum of 40 hours of classroom instruction and 

60 hours of on-the-job training before they independently perform 

screening functions. 

Recurrent training TSA policy requires screeners to complete 3 hours of skills 

refresher training per week averaged over each quarter. 

Remedial training ATSA requires screeners who fail any operational test to complete 

remedial training on the function they failed before they resume 

performing that function. 

Source: ATSA and TSA. 

In September 2003, we reported on our preliminary observations of TSA’s 

efforts to ensure that screeners were effectively trained and supervised 

and to measure screener performance.17 We found that TSA had 

established and deployed a basic screener training program and required 

remedial training but had not fully developed or deployed a recurrent 

training program for screeners or supervisors. We also reported that TSA 

had collected limited data to measure screener performance. Specifically, 

TSA had conducted limited covert testing, the Threat Image Projection 

System was not fully operational, and TSA had not implemented the 

annual screener proficiency testing required by ATSA. In subsequent 

products, we reported progress TSA had made in these areas and 

challenges TSA continued to face in making training available to screeners 

and in measuring and enhancing screener performance. A summary of our 

specific findings is included in appendix I. 

 17GAO, Airport Passenger Screening: Preliminary Observations on Progress Made and 

Challenges Remaining, GAO-03-1173 (Washington, D.C.: Sept. 24, 2003). 
Page 14 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA has taken a number of actions to enhance the training of screeners 

and Screening Supervisors but has encountered difficulties in providing 

access to recurrent training. TSA has enhanced basic training by, among 

other things, adding a dual-function (passenger and checked baggage) 

screening course for new employees. Furthermore, in response to the 

need for frequent and ongoing training, TSA has implemented an Online 

Learning Center with self-guided training courses available to employees 

over TSA’s intranet and the Internet and developed and deployed a 

number of hands-on training tools.18 Moreover, TSA now requires 

screeners to participate in 3 hours of recurrent training per week, 

averaged over each quarter year. TSA has also implemented leadership 

and technical training programs for Screening Supervisors. However, 

some FSDs, in response to open-ended survey questions, identified a 

desire for more training in specific areas, including leadership, 

communication, and supervision. Further, despite the progress TSA has 

made in enhancing and expanding screener and supervisory training, TSA 

has faced challenges in providing access to recurrent training. FSDs 

reported that insufficient staffing and a lack of high-speed 

Internet/intranet connectivity at some training facilities have made it 

difficult to fully utilize these programs and to meet training requirements. 

TSA has acknowledged that challenges exist in recurrent screener training 

delivery and is taking steps to address these challenges, including 

factoring training requirements into workforce planning efforts and 

distributing training through written materials and CD-ROMs until full 

Internet/intranet connectivity is achieved. However, TSA does not have a 

plan for prioritizing and scheduling the deployment of high-speed 

connectivity to all airport training facilities once funding is available. The 

absence of such a plan limits TSA’s ability to make prudent decisions 

about how to move forward with deploying connectivity to all airports to 

provide screeners access to online training. 

TSA has enhanced its basic screener training program by updating the 

training to reflect changes to standard operating procedures, deploying a 

new dual-function (passenger and checked baggage screening) basic 

training curriculum, and allowing the option of training delivery by local 

staff. As required by ATSA, TSA established a basic training program for 

 18The Online Learning Center is TSA’s central, official electronic source of all training and 

related performance/development accomplishments for TSA employees. It serves as the 

delivery platform for online training and is the official repository for TSA training records. 

TSA launched the Online Learning Center on October 31, 2003. 

TSA Has Enhanced 

and Expanded 

Training, but Some 

Screeners Have 

Encountered 

Difficulty Accessing 

and Completing 

Recurrent Training 

TSA Has Enhanced Basic 

Screener Training 
Page 15 GAO-05-457 Screener Training and Performance Measurement Strengthened 

screeners composed of a minimum of 40 hours of classroom instruction 

and 60 hours of on-the-job training. TSA also updated the initial basic 

screener training courses at the end of 2003 to incorporate changes to 

standard operating procedures and directives, which contain detailed 

information on how to perform TSA-approved screening methods. 

However, a recent study by the DHS OIG found that while incorporating 

the standard operating procedures into the curricula was a positive step, a 

number of screener job tasks were incompletely addressed in or were 

absent from the basic training courses.19

In addition to updates to the training curriculum, in April 2004, TSA 

developed and implemented a new basic screener training program, dualfunction screener training that covers the technical aspects of both 

passenger and checked baggage screening. Initially, new hire basic 

training was performed by a contractor and provided a screener with 

training in either passenger or checked baggage screening functions. A 

screener could then receive basic training in the other function later, at the 

discretion of the FSD, but could not be trained in both functions 

immediately upon hire. The new dual-training program is modular in 

design. Thus, FSDs can chose whether newly hired screeners will receive 

instruction in one or both of the screening functions during the initial 

training. In addition, the individual modules can also be used to provide 

recurrent training, such as refreshing checked baggage screening skills for 

a screener who has worked predominately as a passenger screener. TSA 

officials stated that this new approach provides the optimum training 

solution based on the specific needs of each airport and reflects the fact 

that at some airports the FSD does not require all screeners to be fully 

trained in both passenger and checked baggage screening functions. 

Some FSDs, particularly those at smaller airports, have made use of the 

flexibility offered by the modular design of the new course to train 

screeners immediately upon hire in both passenger and checked baggage 

screening functions. Such training up front allows FSDs to use screeners 

for either the passenger or the checked baggage screening function, 

immediately upon completion of basic training. Figure 4 shows that 

58 percent (3,324) of newly hired screeners trained between April 1, 2004, 

and September 1, 2004, had completed the dual-function training. 

 19Department of Homeland Security, Office of Inspector General: An Evaluation of the 

Transportation Security Administration’s Screener Training and Methods of Testing, 

OIG-04-045 (Washington, D.C., Sept. 2004). 
Page 16 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Figure 4: Newly Hired Screeners Trained as Dual-Function Screeners between April 1, 2004, and September 1, 2004 

In April 2004, TSA also provided FSDs with the flexibility to deliver basic 

screener training using local instructors. TSA’s Workforce Performance 

and Training Office developed basic screener training internally, and 

initially, contractors delivered all of the basic training. Since then, TSA has 

provided FSDs with the discretion to provide the training using local TSA 

employees or to use contractors. The flexibility to use local employees 

allows FSDs and members of the screener workforce to leverage their 

first-hand screening knowledge and experience and address situations 

unique to individual airports.20 As of December 10, 2004, TSA had trained 

1,021 local FSD staff (representing 218 airports) in how to instruct the 

dual-function screener training course. TSA officials stated that they 

expect the use of TSA-approved instructors to increase over time. 

 20For local employees to be approved as instructors by TSA, they must be nominated by an 

FSD and have a current or prior instructor certification by a recognized training and 

development organization or have had at least 2 years of experience as an instructor. In 

addition, local TSA instructors must have successfully completed the course of instruction 

they will be teaching and demonstrate instructional skills by assisting a TSA-approved 

instructor in classroom instruction and monitoring actual classroom instruction. 

Airport category:

Percentage dual-trained

Percentage trained as passenger screeners only

Percentage trained as checked baggage screeners only

X

43%

2%

55%

18% 14%

1% 2%

79% 86% 97% 96%

I II III IV

Source: GAO analysis of TSA data.

1% 3%

58%

All airports

40%

2%

3%

0.3%

Total number

of airports: 21 61 57 124 197 454
Page 17 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA has taken several steps to deploy a recurrent screener training 

program, including implementing the Online Learning Center—which 

includes several recurrent training modules developed by TSA—

implementing a recurrent training requirement, and providing hands-on 

training tools. According to TSA, comprehensive and frequent training is 

key to a screener’s ability to detect threat objects. In September 2003, we 

reported that TSA had not fully developed or deployed a recurrent training 

program for passenger screeners. At that time, some FSDs expressed 

concern with the lack of training available to the screener workforce, and 

in the absence of headquarters-provided training, they were developing 

and implementing locally based recurrent training programs. Similar to 

our findings, TSA’s April 2004 consultant study reported that most FSDs 

and their staffs were generally unsatisfied with the training support 

provided by headquarters.21 Specifically, the study found that: 

“Numerous interviews revealed concerns with training curriculum, communication, and 

coordination issues that directly affect security screening. Unsatisfied with the quantity 

and breadth of topics, many Training Coordinators have developed supplementary lectures 

on both security and non-security related topics. These additional lectures...have been very 

highly received by screeners.” 

In October 2003, TSA introduced the Online Learning Center to provide 

screeners with remote access to self-guided training courses. As of 

September 14, 2004, TSA had provided access to over 550 training courses 

via the Online Learning Center and made the system available via the 

Internet and its intranet. TSA also developed and deployed a number of 

hands-on training modules and associated training tools for screeners at 

airports nationwide. These training modules cover topics including handwanding and pat-down techniques, physical bag searches, X-ray images, 

prohibited items, and customer service. Additionally, TSA instituted 

another module for the Online Learning Center called Threat in the 

Spotlight, that, based on intelligence TSA receives, provides screeners 

with the latest in threat information regarding terrorist attempts to get 

threat objects past screening checkpoints. Appendix III provides a 

summary of the recurrent training tools TSA has deployed to airports and 

the modules currently under development. 

 21In April 2004, a TSA-sponsored independent evaluation of screening operations was 

completed by a consultant (Private Screening Operations Performance Evaluation 

Summary Report, BearingPoint, Apr. 16, 2004). The study was designed to evaluate the 

performance of federal and private screening operations. 

TSA Has Deployed 

Recurrent Screener 

Training and Provided 

Additional Training Tools, 

but Some FSDs Identified 

the Need for More Courses 
Page 18 GAO-05-457 Screener Training and Performance Measurement Strengthened 

In December 2003, TSA issued a directive requiring screeners to receive 

3 hours of recurrent training per week averaged over a quarter year. One 

hour is required to be devoted to X-ray image interpretation and the other 

2 hours to screening techniques, review of standard operating procedures, 

or other mandatory administrative training, such as ethics and privacy act 

training. 

In January 2004, TSA provided FSDs with additional tools to facilitate and 

enhance screener training. Specifically, TSA provided airports with at least 

one modular bomb set (MBS II) kit—containing components of an 

improvised explosive device—and one weapons training kit, in part 

because screeners had consistently told TSA’s OIAPR inspectors that they 

would like more training with objects similar to ones used in covert 

testing.22 

Although TSA has made progress with the implementation of recurrent 

training, some FSDs identified the need for several additional courses, 

including courses that address more realistic threats. TSA acknowledged 

that additional screener training is needed, and officials stated that the 

agency is in the process of developing new and improved screener 

training, including additional recurrent training modules (see app. III). 

TSA has arranged for leadership training for screening supervisors through 

the Department of Agriculture Graduate School and has developed 

leadership and technical training courses for screening supervisors. 

However, some FSDs reported the need for more training for Screening 

Supervisors and Lead Screeners. The quality of Screening Supervisors has 

been a long-standing concern. In testifying before the 9/11 Commission in 

May 2003, a former FAA Assistant Administrator for Civil Aviation Security 

stated that following a series of covert tests at screening checkpoints to 

determine which were strongest, which were weakest, and why, invariably 

the checkpoint seemed to be as strong or as weak as the supervisor who 

was running it. Similarly, TSA’s OIAPR identified a lack of supervisory 

training as a cause for screener covert testing failures. Further, in a July 

2003 internal study of screener performance, TSA identified poor 

 22The MBS II and weapons training kits were provided to airports to address the identified 

training gap by allowing screeners to see and feel the threat objects that they are looking 

for. These kits contain some of the test objects used by TSA’s OAIPR to conduct the covert 

testing. In February 2004, TSA issued guidance to FSDs on use of these kits to conduct 

local screener testing. These guidelines were updated in June 2004. 

TSA Provides Leadership 

and Technical Training for 

Supervisors, but Some 

FSDs Would Like More 

Training for Screening 

Supervisors and Lead 

Screeners 
Page 19 GAO-05-457 Screener Training and Performance Measurement Strengthened 

supervision at the screening checkpoints as a cause for screener 

performance problems. In particular, TSA acknowledged that many Lead 

Screeners, Screening Supervisors, and Screening Managers did not 

demonstrate supervisory and management skills (i.e., mentoring, coaching, 

and positive reinforcement) and provided little or no timely feedback to 

guide and improve screener performance. In addition, the internal study 

found that because of poor supervision at the checkpoint, supervisors or 

peers were not correcting incorrect procedures, optimal performance 

received little reinforcement, and not enough breaks were provided to 

screeners. A September 2004 report by the DHS OIG supported these 

findings, noting that Screening Supervisors and Screening Managers 

needed to be more attentive in identifying and correcting improper or 

inadequate screener performance.23

TSA recognizes the importance of Screener Supervisors and has 

established training programs to enhance their performance and 

effectiveness. In September 2003, we reported that TSA had begun 

working with the Department of Agriculture Graduate School to tailor the 

school’s off-the-shelf supervisory course to meet the specific needs of 

Screening Supervisors, and had started training the existing supervisors at 

that time through this course until the customized course was fielded. 

According to TSA’s training records, as of September 2004, about 3,800 

Screening Supervisors had completed the course—approximately 

92 percent of current Screening Supervisors. In response to our survey, 

one FSD noted that the supervisory training was long overdue because 

most of the supervisors had no prior federal service or, in some cases, no 

leadership experience. This FSD also noted that “leadership and 

supervisory skills should be continuously honed; thus, the development of 

our supervisors should be an extended and sequential program with 

numerous opportunities to develop skills—not just a one-time class.” 

In addition to the Department of Agriculture Graduate School course, 

TSA’s Online Learning Center includes over 60 supervisory courses 

designed to develop leadership and coaching skills. In April 2004, TSA 

included in the Online Learning Center a Web-based technical training 

course—required for all Lead Screeners and Screening Supervisors. This 

course covers technical issues, such as resolving alarms at screening 

 23Department of Homeland Security Office of Inspector General, Audit of Passenger and 

Baggage Screening Procedures at Domestic Airports, OIG-04-37 (Washington, D.C., 

September 2004). 
Page 20 GAO-05-457 Screener Training and Performance Measurement Strengthened 

checkpoints. TSA introduced this course to the field in March 2004, and 

although the course is a requirement, TSA officials stated that they have 

not set goals for when all Lead Screeners and Screening Supervisors 

should have completed the course. In June 2004, TSA training officials 

stated that a second supervisor technical course was planned for 

development and introduction later in 2004. However, in December 2004, 

the training officials stated that planned funding for supervisory training 

may be used to support other TSA initiatives. The officials acknowledged 

that this would reduce TSA’s ability to provide the desired type and level 

of supervisory training to its Lead Screener, Screening Supervisor, and 

Screening Manager staff. TSA plans to revise its plans to provide Lead 

Screener, Screening Supervisor, and Screening Manager training based on 

funding availability. 

Although TSA has developed leadership and technical courses for 

Screening Supervisors, many FSDs, in response to our general survey, 

identified additional types of training needed to enhance screener 

supervision. Table 2 provides a summary of the additional training needs 

that FSDs reported. 

Table 2: Training Needed to Enhance Screener Supervision, as Reported by FSDs 

Training topic 

Percentage of FSDs 

indicating this need

Leadership and management 41

Public speaking/communications 26

Conflict management 24

Human resource-related issues (hiring, termination, discipline) 19

Counseling, mentoring, and coaching 18

Writing skills 16

Additional supervisory training 13

Interpersonal skills 13

Crisis management and incident response 6

Source: GAO general survey of FSDs. 

TSA training officials stated that the Online Learning Center provides 

several courses that cover these topics. Such courses include 

• Situation Leadership II; 

• Communicating with Difficult People: Handling Difficult Co-Workers; 

• Team Participation: Resolving Conflict in Teams; 
Page 21 GAO-05-457 Screener Training and Performance Measurement Strengthened 

• Employee Performance: Resolving Conflict; 

• High Impact Hiring; 

• Team Conflict: Overcoming Conflict with Communication; 

• Correcting Performance Problems: Disciplining Employees; 

• Team Conflict: Working in Diversified Teams; 

• Correcting Performance Problems: Identifying Performance Problems; 

• Resolving Interpersonal Skills; 

• Grammar, Skills, Punctuation, Mechanics and Word Usage; and 

• Crisis in Organizations: Managing Crisis Situations. 

TSA training officials acknowledged that for various reasons FSDs might 

not be aware that the supervisory and leadership training is available. For 

example, FSDs at airports without high-speed Internet/intranet access to 

the Online Learning Center might not have access to all of these courses. It 

is also possible that certain FSDs have not fully browsed the contents of 

the Online Learning Center and therefore are not aware that the training is 

available.24 Furthermore, officials stated that online learning is relatively 

new to government and senior field managers, and some of the FSDs may 

expect traditional instructor-led classes rather than online software. 

Some FSDs responded to our general survey that they faced challenges 

with screeners receiving recurrent training, including insufficient staffing 

to allow all screeners to complete training within normal duty hours and a 

lack of high-speed Internet/intranet connectivity at some training facilities. 

According to our guide for assessing training, to foster an environment 

conducive to effective training and development, agencies must take 

actions to provide sufficient time, space, and equipment to employees to 

complete required training.25 TSA has set a requirement for 3 hours of 

recurrent training per week averaged over a quarter year, for both full-time 

and part-time screeners. However, FSDs for about 18 percent (48 of 263) 

of the airports in our airport-specific survey reported that screeners 

received less than 9 to 12 hours of recurrent training per month.26

Additionally, FSDs for 48 percent (125 of 263) of the airports in the survey 

 24TSA headquarters training officials stated that the Training Coordinators at airports 

should be aware of the availability of the supervisory and leadership training courses. 

25GAO, Human Capital: A Guide for Assessing Strategic Training and Development 

Efforts in the Federal Government, GAO-04-546G (Washington, D.C., March 2004). 

26Methods used by FSDs for determining and reporting recurrent training time have not 

been verified by GAO. 

Some FSDs Reported 

Impediments to Screeners 

Receiving Recurrent 

Training 
Page 22 GAO-05-457 Screener Training and Performance Measurement Strengthened 

reported that there was not sufficient time for screeners to receive 

recurrent training within regular work hours. 

At 66 percent of those airports where the FSD reported that there was not 

sufficient time for screeners to receive recurrent training within regular 

work hours, the FSDs cited screener staffing shortages as the primary 

reason. We reported in February 2004 that FSDs at 11 of the 15 category X 

airports we visited reported that they were below their authorized staffing 

levels because of attrition and difficulties in hiring new staff. In addition, 

three of these FSDs noted that they had never been successful in hiring up 

to the authorized staffing levels. We also reported in February 2004 that 

FSDs stated that because of staffing shortages, they were unable to let 

screeners participate in training because it affected the FSD’s ability to 

provide adequate coverage at the checkpoints. In response to our survey, 

FSDs across all categories of airports reported that screeners must work 

overtime in order to participate in training. A September 2004 DHS OIG 

report recommended that TSA examine the workforce implications of the 

3-hour training requirement and take steps to correct identified 

imbalances in future workforce planning to ensure that all screeners are 

able to meet the recurrent training standard. The 3-hours-per-week 

training standard represents a staff time commitment of 7.5 percent of fulltime and between 9 and 15 percent of part-time screeners’ nonovertime 

working hours. TSA headquarters officials have stated that because the 3-

hours-per-week requirement is averaged over a quarter, it provides 

flexibility to account for the operational constraints that exist at airports. 

However, TSA headquarters officials acknowledged that many airports are 

facing challenges in meeting the 3-hour recurrent training requirement. 

TSA data for the fourth quarter of fiscal year 2004 reported that 75 percent 

of airports were averaging less than 3 hours of recurrent training per week 

per screener. The current screener staffing model, which is used to 

determine the screener staffing allocations for each airport, does not take 

the 3-hours-per-week recurrent training requirement into account.27

However, TSA headquarters officials said that they are factoring this 

 27The staffing model took into account factors such as the number of screening 

checkpoints and lanes at an airport; originating passengers; projected air carrier service 

increases and decreases during calendar year 2003; and hours needed to accommodate 

some screener training, leave, and breaks. 
Page 23 GAO-05-457 Screener Training and Performance Measurement Strengthened 

training requirement into their workforce planning efforts, including the 

staffing model currently under development.28 

Another barrier to providing recurrent training is the lack of high-speed 

Internet/intranet access at some of TSA’s training locations.29 TSA officials 

acknowledged that many of the features of the Online Learning Center, 

including some portions of the training modules and some Online Learning 

Center course offerings, are difficult or impossible to use in the absence of 

high-speed Internet/intranet connectivity. As one FSD put it, “the delayed 

deployment of the high-speed Internet package limits the connectivity to 

TSA HQ for various online programs that are mandated for passenger 

screening operations including screener training.” One FSD for a category 

IV airport noted the lack of a high-speed connection for the one computer 

at an airport he oversees made the Online Learning Center “nearly 

useless.” 

TSA began deploying high-speed access to its training sites and 

checkpoints in May 2003 and has identified high-speed connectivity as 

necessary in order to deliver continuous training to screeners. TSA’s July 

2003 Performance Improvement Study recommended accelerating highspeed Internet/intranet access in order to provide quick and systematic 

distribution of information and, thus, reduce uncertainty caused by the 

day-to-day changes in local and national procedures and policy. In 

October 2003, TSA reported plans to have an estimated 350 airports online 

with high-speed connectivity within 6 months. However, in June 2004, TSA 

reported that it did not have the resources to reach this goal. 

 28In May 2003, TSA hired a contractor to develop a staffing model for its screening 

workforce. TSA officials told us that the contractor completed the staffing model in June 

2004, and all airports now have the capability to use the contractor’s stand-alone software. 

TSA completed installation of the software on its intranet in March 2005. This installation 

provides TSA headquarters with access to the staffing models used by airports. The 

Intelligence Reform and Terrorism Prevention Act of 2004 requires TSA to develop and 

submit to the appropriate congressional committees, standards for determining aviation 

security staffing at commercial airports no later than 90 days after December 17, 2004, the 

date of the act’s enactment, and GAO to conduct an analysis of these standards (Pub. L. No. 

108-458). As of April 15, 2005, these standards had not been submitted. 

29High-speed Internet/intranet access is provided by a series of technologies that give users 

the ability to send and receive data at volumes and speeds far greater than Internet access 

over traditional telephone lines. In addition to offering speed, the technology provides a 

continuous, “always on” connection (no need to dial up) and a “two-way” capability, that is, 

the ability to both receive and transmit data at high speeds. 
Page 24 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA records show that as of October 2004, TSA had provided high-speed 

access for training purposes to just 109 airports, where 1,726 training 

computers were fully connected.30 These 109 airports had an authorized 

staffing level of over 24,900 screeners, meaning that nearly 20,100 

screeners (45 percent of TSA’s authorized screening workforce) still did 

not have high-speed Internet/intranet access to the Online Learning Center 

at their training facility. In October 2004, TSA officials stated that TSA’s 

Office of Information Technology had selected an additional 16 airport 

training facilities with a total of 205 training computers to receive highspeed connectivity by the end of December 2004. As of January 19, 2005, 

TSA was unable to confirm that these facilities had received high-speed 

connectivity. Additionally, they could not provide a time frame for when 

they expected to provide high-speed connectivity to all airport training 

facilities because of funding uncertainties. Furthermore, TSA does not 

have a plan for prioritizing and scheduling the deployment of high-speed 

connectivity to all airport training facilities once funding is available. 

Without a plan, TSA’s strategy and timeline for implementing connectivity 

to airport training facilities is unclear. The absence of such a plan limits 

TSA’s ability to make prudent decisions about how to move forward with 

deploying connectivity once funding is available. Figure 5 shows the 

percentage of airports reported to have high-speed connectivity for their 

training computers by category of airport as of October 2004. 

 30TSA defines fully connected as a training computer with the new network image installed 

and connected to the TSA broadband network. 
Page 25 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Figure 5: Percentage of Airports Reported to Have High-Speed Connectivity for Training Purposes as of October 2004 

Note: These data show airports with high-speed connectivity on training computers. Some airports 

have high-speed connectivity, but for purposes other than training. 

To mitigate airport connectivity issues in the interim, on April 1, 2004, TSA 

made the Online Learning Center courses accessible through public 

Internet connections, which enable screeners to log on to the Online 

Learning Center from home, a public library, or other locations. However, 

TSA officials stated that the vast majority of screeners who have used the 

Online Learning Center have logged in from airports with connectivity at 

their training facilities. TSA also distributes new required training 

products using multiple delivery channels, including written materials and 

CD-ROMs for those locations where access to the Online Learning Center 

is limited. Specifically, TSA officials stated that they provided airports 

without high-speed connectivity with CD-ROMs for the 50 most commonly 

used optional commercial courseware titles covering topics such as 

information technology skills, customer service, and teamwork. 

Additionally, officials stated that as technical courses are added to the 

Online Learning Center, they are also distributed via CD-ROM and that 

until full connectivity is achieved, TSA will continue to distribute new 

training products using multiple delivery channels. 

Airport category:

Total number

of airports: 21 61 51 124 197 454

Airports with high-speed connectivity

Airports without high-speed connectivity

X

57% 43%

25%

43%

83%

2%

75%

57%

17%

98%

24%

76%

I II III IV All airports

Source: GAO analysis of TSA data.
Page 26 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Because of a lack of internal controls, TSA cannot provide reasonable 

assurance that screeners are completing required training. First, TSA 

policy does not clearly define responsibility for ensuring that screeners 

have completed all required training. Additionally, TSA has no formally 

defined policies or procedures for documenting completion of remedial 

training, or a system designed to facilitate review of this documentation 

for purposes of monitoring. Further, TSA headquarters does not have 

formal policies and procedures for monitoring completion of basic training 

and lacks procedures for monitoring recurrent training. Finally, at airports 

without high-speed connectivity, training records must be entered 

manually, making it challenging for some airports to keep accurate and upto-date training records. 

TSA’s current guidance for FSDs regarding the training of the screener 

workforce does not clearly identify responsibility for tracking and 

ensuring compliance with training requirements. In a good control 

environment, areas of authority and responsibility are clearly defined and 

appropriate lines of reporting are established.31 In addition, internal 

control standards also require that responsibilities be communicated 

within an organization. The Online Learning Center provides TSA with a 

standardized, centralized tool capable of maintaining all training records 

in one system. It replaces an ad hoc system previously used during initial 

rollout of federalized screeners in which contractors maintained training 

records. A February 2004 management directive states that FSDs are 

responsible for ensuring the completeness, accuracy, and timeliness of 

training records maintained in the Online Learning Center for their 

employees. For basic and recurrent training, information is to be entered 

into the Online Learning Center within 30 days of completion of the 

training activity. However, the directive does not clearly identify who is 

responsible for ensuring that employees comply with training 

requirements. Likewise, a December 2003 directive requiring that 

screeners complete 3 hours of training per week averaged over a quarter 

states that FSDs are responsible for ensuring that training records for each 

screener are maintained in the Online Learning Center. Although both 

directives include language that requires FSDs to ensure training records 

are maintained in the Online Learning Center, neither specifies whether 

FSDs or headquarters officials are responsible for ensuring compliance 

with the basic, recurrent, and remedial training requirements. Even so, 

 31GAO/AIMD-00-21.3.1. 

TSA Lacks Adequate 

Controls to Provide 

Reasonable 

Assurance That 

Screeners Receive 

Required Training 

TSA Policy Does Not 

Clearly Define 

Responsibility for 

Ensuring That Screeners 

Are in Compliance with 

Training Requirements 
Page 27 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA headquarters officials told us that FSDs are ultimately responsible for 

ensuring screeners receive required training. However, officials provided 

no documentation clearly defining this responsibility. Without a clear 

designation of responsibility for monitoring training completion, this 

function may not receive adequate attention, leaving TSA unable to 

provide reasonable assurance that its screening workforce receives 

required training. In April 2005, TSA officials responsible for training 

stated that they were updating the February 2004 management directive on 

training records to include a specific requirement for FSDs to ensure that 

screeners complete required training. They expect to release the revised 

directive in May 2005. 

TSA has not established and documented policies and procedures for 

monitoring completion of basic and recurrent training. Internal control 

standards advise that internal controls should be designed so that 

monitoring is ongoing and ingrained in agency operations.32 However, TSA 

headquarters officials stated that they have no formal policy for 

monitoring screeners’ completion of basic training. They also stated that 

they have neither informal nor formal procedures for monitoring the 

completion of screeners’ recurrent training requirements, and 

acknowledged that TSA policy does not address what is to occur if a 

screener does not meet the recurrent training requirement. Officials 

further stated that individual FSDs have the discretion to determine what 

action, if any, to take when screeners do not meet this requirement. 

In July 2004, TSA training officials stated that headquarters staff recently 

began running a report in the Online Learning Center to review training 

records to ensure that newly hired screeners had completed required basic 

training. In addition, they stated that in June 2004, they began generating 

summary-level quarterly reports from the Online Learning Center to 

quantify and analyze hours expended for recurrent screener training. 

Specifically, TSA training officials stated that reports showing airport-level 

compliance with the 3-hour recurrent requirement were generated for the 

third and fourth quarters of fiscal year 2004 and delivered to the Office of 

Aviation Operations for further analysis and sharing with the field. 

However, Aviation Operations officials stated that they did not use these 

reports to monitor the status of screener compliance with the 3-hour 

recurrent training requirement and do not provide them to the field unless 

 32GAO/AIMD-00-21.3.1. 

TSA Lacks Formal Policies 

and Procedures for 

Monitoring Completion of 

Required Training 
Page 28 GAO-05-457 Screener Training and Performance Measurement Strengthened 

requested by an FSD. TSA training officials said that while headquarters 

intends to review recurrent training activity on an ongoing basis at a 

national and airport level, they view FSDs and FSD training staff as 

responsible for ensuring that individuals receive all required training. 

Further, they acknowledged that weaknesses existed in the reporting 

capability of the Online Learning Center and stated that they plan to 

upgrade the Online Learning Center with improved reporting tools by the 

end of April 2005. Without clearly defined policies and procedures for 

monitoring the completion of training, TSA lacks a structure to support 

continuous assurance that screeners are meeting training requirements. 

TSA has not established clear policies and procedures for documenting 

completion of required remedial training. The Standards for Internal 

Control state that agencies should document all transactions and other 

significant events and should be able to make this documentation readily 

available for examination.33 A TSA training bulletin dated October 15, 2002, 

specifies that when remedial training is required, FSDs must ensure the 

training is provided and a remedial training reporting form is completed 

and maintained with the screener’s local records.34 However, when we 

asked to review these records, we found confusion as to how and where 

they were to be maintained. TSA officials stated that they are waiting for a 

decision regarding how to maintain these records because of their 

sensitive nature. In the meantime, where and by whom the records should 

be maintained remains unclear. 

In September 2004, officials from TSA’s OIAPR—responsible for 

conducting covert testing—stated that they maintain oversight to ensure 

screeners requiring remedial training receive required training by 

providing a list of screeners that failed covert testing and therefore need 

remedial training to TSA’s Office of Aviation Operations. Aviation 

Operations is then to confirm via memo that each of the screeners has 

received the necessary remedial training and report back to OIAPR. 

Accordingly, we asked TSA for all Aviation Operations memos confirming 

 33GAO-AIMD-00-21.3.1. 

34Consistent with ATSA, TSA requires remedial training for any passenger or baggage 

screener who fails an operational (covert) test and prohibits screeners from performing the 

screening function related to the test they failed until they successfully complete the 

training. 

TSA Lacks Clearly Defined 

Policies and Procedures 

for Documenting 

Completion of Remedial 

Training 
Page 29 GAO-05-457 Screener Training and Performance Measurement Strengthened 

completion of remedial training, but we were only able to obtain 1 of the 

12 memos.35

In addition, during our review, we asked to review the remedial training 

reporting forms at five airports to determine whether screeners received 

required training, but we encountered confusion about requirements for 

maintaining training records and inconsistency in record keeping on the 

part of local TSA officials. Because of the unclear policies and procedures 

for recording completion of remedial training, TSA does not have adequate 

assurance that screeners are receiving legislatively mandated remedial 

training. 

Although training computers with high-speed Internet/intranet 

connectivity automatically record completion of training in the Online 

Learning Center, airports without high-speed access at their training 

facility must have these records entered manually. The February 2004 

management directive that describes responsibility for entering training 

records into the Online Learning Center also established that all TSA 

employees are required to have an official TSA training record in the 

Online Learning Center that includes information on all official training 

that is funded wholly or in part with government funds. Without highspeed access, TSA officials stated that it can be a challenge for airports to 

keep the Online Learning Center up to date with the most recent training 

records. TSA headquarters officials further stated that when they want to 

track compliance with mandatory training such as ethics or civil rights 

training, they provide the Training Coordinators with a spreadsheet on 

which to enter the data rather than relying on the Online Learning Center. 

As one FSD told us, without high-speed connectivity at several of the 

airports he oversees, “this is very time consuming and labor intensive and 

strains my limited resources.” The difficulty that airports encounter in 

maintaining accurate records when high-speed access is absent could 

compromise TSA’s ability to provide reasonable assurance that screeners 

are receiving mandated basic and remedial training. 

 35According to TSA officials, between September 2002 and December 8, 2004, the Office of 

Internal Affairs and Program Review issued 12 memorandums to Aviation Operations that 

identified screeners requiring remedial training based on covert testing conducted from 

September 2002 through September 30, 2004. 

Lack of High-Speed 

Connectivity Limits TSA’s 

Ability to Document and 

Track the Completion of 

Screener Training 
Page 30 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA has improved its efforts to measure and enhance screener 

performance. However, these efforts have primarily focused on passenger 

screening rather than checked baggage screening, and TSA has not yet 

finalized performance targets for several key performance measures. For 

example, TSA has increased the amount of covert testing it performs at 

airports. These tests have identified that, overall, weaknesses and 

vulnerabilities continue to exist in the passenger and checked baggage 

screening systems. TSA also enabled FSDs to conduct local covert testing, 

fully deployed the Threat Image Projection (TIP) system to passenger 

screening checkpoints at commercial airports nationwide, and completed 

the 2003/2004 annual screener recertification program for all eligible 

screeners. However, not all of these performance measurement and 

enhancement tools are available for checked baggage screening. 

Specifically, TIP is not currently operational at checked baggage screening 

checkpoints, and the recertification program does not include an image 

recognition component for checked baggage screeners. However, TSA is 

taking steps to address the overall imbalance in passenger and checked 

baggage screening performance data, including working toward 

implementing TIP for checked baggage screening and developing an image 

recognition module for checked baggage screener recertification. To 

enhance screener and screening system performance, TSA has also 

conducted a passenger screener performance improvement study and 

subsequently developed an improvement plan consisting of multiple action 

items, many of which TSA has completed. However, TSA has not 

conducted a similar study for checked baggage screeners. In addition, TSA 

has established over 20 performance measures for the passenger and 

checked baggage screening systems as well as two performance indexes 

(one for passenger and one for checked baggage screening). However, 

TSA has not established performance targets for each of the component 

indicators within the indexes, such as covert testing. According to The 

Office of Management and Budget, performance goals are target levels of 

performance expressed as a measurable objective, against which actual 

achievement can be compared. Performance goals should incorporate 

measures (indicators used to gauge performance); targets (characteristics 

that tell how well a program must accomplish the measure), and time 

frames. Without these targets, TSA’s performance management system, 

and these performance indexes, specifically, may not provide the agency 

with the complete information necessary to assess achievements and 

make decisions about where to direct performance improvement efforts. 

Although TSA has not yet established performance targets for each of the 

component indicators, TSA plans to finalize performance targets for the 

indicators by the end of fiscal year 2005. 

Progress Has Been 

Made in Implementing 

Tools to Measure and 

Enhance Screener 

Performance, but Key 

Performance Targets 

Have Not Been 

Finalized 
Page 31 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA headquarters has increased the amount of covert testing it performs 

and enabled FSDs to conduct additional local covert testing at passenger 

screening checkpoints. TSA’s OIAPR conducts unannounced covert tests 

of screeners to assess their ability to detect threat objects and to adhere to 

TSA-approved procedures. These tests, in which undercover OIAPR 

inspectors attempt to pass threat objects through passenger screening 

checkpoints and in checked baggage, are designed to measure 

vulnerabilities in passenger and checked baggage screening systems and 

to identify systematic problems affecting performance of screeners in the 

areas of training, policy, and technology.36 TSA considers its covert testing 

as a “snapshot” of a screener’s ability to detect threat objects at a 

particular point in time and as one of several indicators of system wide 

screener performance. 

OIAPR conducts tests at passenger screening checkpoints and checked 

baggage screening checkpoints. According to OIAPR, these tests are 

designed to approximate techniques terrorists might use. These covert test 

results are one source of data on screener performance in detecting threat 

objects as well as an important mechanism for identifying areas in 

passenger and checked baggage screening needing improvement. In 

testimony before the 9/11 Commission, the Department of Transportation 

Inspector General stated that emphasis must be placed on implementing 

an aggressive covert testing program to evaluate operational effectiveness 

of security systems and equipment.37

 

Between September 10, 2002, and September 30, 2004, OIAPR conducted a 

total of 3,238 covert tests at 279 different airports. In September 2003, we 

reported that OIAPR had conducted limited covert testing but planned to 

double the amount of tests it conducted during fiscal year 2004, based on 

an anticipated increase in its staff from about 100 full-time equivalents to 

about 200 full-time equivalents. TSA officials stated that based on budget 

constraints, OIAPR’s fiscal year 2004 staffing authorization was limited to 

 36OIAPR designs its covert testing methods based, in part, on intelligence regarding the 

most recent threats. 

37Statement before the National Commission on Terrorist Attacks upon the United States, 

Statement of the Honorable Kenneth M. Mead, Inspector General, U.S. Department of 

Transportation, May 22, 2003. 

TSA Has Increased Its 

Covert Testing and 

Allowed Local Covert 

Testing at Passenger 

Screening Checkpoints 
Page 32 GAO-05-457 Screener Training and Performance Measurement Strengthened 

183 full-time-equivalents,38 of which about 60 are located in the field.39

Despite a smaller than expected staff increase, by the end of the second 

quarter of fiscal year 2004, OIAPR had already surpassed the number of 

tests it performed during fiscal year 2003, as shown in table 3. 

 38Covert testing is an ancillary duty and not a full-time assignment for the majority of 

OIAPR staff. According to OIAPR, of the approximately 123 full-time-equivalent positions in 

headquarters, 14 are dedicated fully to the covert testing program, which includes covert 

testing of all modes of transportation, not just airports. These 14 full-time-equivalents are in 

OIAPR’s Special Operations group and form the core of team leaders for the covert testing 

trips. In addition, two full-time-equivalents from OIAPR’s Office of Program Analysis 

support the covert testing program full-time in terms of data analysis, report writing, and 

quality assurance. The remaining OIAPR staff in headquarters and the field are responsible 

for conducting criminal and noncriminal investigations of employee misconduct; 

conducting program reviews, inspections, and special inquiries into security incidents; and 

managing OIAPR. 

39TSA established five mission support centers staffed with OIAPR, training, and other 

personnel. These centers are located in Atlanta, Dallas, Detroit, Philadelphia, and San 

Francisco. In fiscal year 2004, OIAPR began to use field staff to support its covert testing 

activities. In addition, OIAPR has 12 investigators located at seven airports. 
Page 33 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Table 3: Checkpoint and Checked Baggage Tests Conducted by OIAPR, 

September 10, 2002–September 30, 2004 

Testing period 

fiscal 

year/quarter Airportsa

Passenger 

screening 

checkpoint tests

Checked 

baggage testsb

Total tests

2002 fourth 2 30 3 33

2003 first 14 120 1 121

2003 second 31 231 27 258

2003 third 28 198 19 217

2003 fourth 25 217 23 240

2004 first 41 171 110 281

2004 second 111 770 182 952

2004 third 56 379 102 481

2004 fourth 64 527 128 655

Total 2,643 595 3,238

Source: GAO analysis of OIAPR data. 

Note: Some airports have been tested more than once. 

a

OIAPR conducted covert testing at three additional airports in September and October 2002. 

However, at the time of the testing, federal screeners had not yet been deployed to these airports. 

We excluded these tests from our analysis. 

b

TSA did not begin reporting the results of checked baggage tests until January 2003. However, four 

of these tests were conducted in September and October 2002. 

In October 2003, OIAPR committed to testing between 90 and 150 airports 

by April 2004 as part of TSA’s short-term screening performance 

improvement plan. OAIPR officials stated that this was a onetime goal to 

increase testing. This initiative accounts for the spike in testing for the 

second quarter of fiscal year 2004. 

OIAPR has created a testing schedule designed to test all airports at least 

once during a 3-year time frame. Specifically, the schedule calls for OIAPR 

to test all category X airports once a year, category I and II airports once 

every 2 years, and category III and IV airports at least once every 3 years. 

In September 2003 and April 2004, we reported that TSA covert testing 

results had identified weaknesses in screeners’ ability to detect threat 
Page 34 GAO-05-457 Screener Training and Performance Measurement Strengthened 

objects.40 More recently, in April 2005, we, along with the DHS OIG, 

identified that screener performance continued to be a concern. 

Specifically, our analysis of TSA’s covert testing results for tests 

conducted between September 2002 and September 2004 identified that 

overall, weaknesses still existed in the ability of screeners to detect threat 

objects on passengers, in their carry-on bags, and in checked baggage. 41

Covert testing results in this analysis cannot be generalized either to the 

airports where the tests were conducted or to airports nationwide.42 These 

weaknesses and vulnerabilities were identified at airports of all sizes, at 

airports with federal screeners, and airports with private-sector screeners. 

For the two-year period reviewed, overall failure rates for covert tests 

(passenger and checked baggage) conducted at airports using privatesector screeners were somewhat lower than failure rates for the same 

tests conducted at airports using federal screeners for the airports tested 

during this period.43 Since these test results cannot be generalized as 

discussed above, each airport’s test results should not be considered a 

comprehensive measurement of the airport’s performance or any 

individual screener’s performance in detecting threat objects, or in 

determining whether airports with private sector screeners performed 

better than airports with federal screeners. 

On the basis of testing data through September 30, 2004, we determined 

that OIAPR had performed covert testing at 61 percent of the nation’s 

commercial airports. TSA has until September 30, 2005, to test the 

additional 39 percent of airports and meet its goal of testing all airports 

 40GAO, Airport Passenger Screening: Preliminary Observations on Progress Made and 

Challenges Remaining, GAO-03-1173 (Washington, D.C.: Sept. 24, 2003). GAO, Aviation 

Security: Private Screening Contractors Have Little Flexibility to Implement Innovative 

Approaches, GAO-04-544T (Washington, D.C.: Apr 22, 2004). 

41Results of TSA’s covert testing of passenger and checked baggage screening are classified 

and are discussed in a separate classified report, GAO, Results of Transportation Security 

Administration’s Covert Testing for Passenger and Checked Baggage Screening for 

September 2002 through September 2004, GAO-05-437C (Washington, D.C.: Apr. 5, 2005). 

42Test results cannot be generalized because sample tests were not identified using the 

principles of probability sampling. In a probability sample to assess screener detection of 

threat objects, each screening of a passenger or baggage would have to have a chance of 

being selected. A well-designed probability sample would enable failure rates to be 

generalized to all airports. However, for cost and operational reasons, probability sampling 

may not be feasible for passenger and checked baggage screening because it would require 

a very large sample size and an exhaustive examination of each sampled passenger or 

baggage to determine if there was a threat object to detect. 

43Test failure rates are classified and have been excluded from this report. 
Page 35 GAO-05-457 Screener Training and Performance Measurement Strengthened 

within 3 years. Although officials stated that they have had to divert 

resources from airport testing to conduct testing of other modes and that 

testing for other modes of transportation may affect their ability to 

conduct airport testing, they still expect to meet the goal. 

In February 2004, TSA provided protocols to help FSDs conduct their own 

covert testing of local airport passenger screening activities—a practice 

that TSA had previously prohibited.44 Results of local testing using these 

protocols are to be entered into the Online Learning Center. This 

information, in conjunction with OAIPR covert test results and TIP threat 

detection results, is intended to assist TSA in identifying specific training 

and performance improvement efforts. In February 2005, TSA released a 

general procedures document for local covert testing at checked baggage 

screening locations. 

TSA officials said that they had not yet begun to use data from local covert 

testing to identify training and performance needs because of difficulties 

in ensuring that local covert testing is implemented consistently 

nationwide. These officials said that after a few months of collecting and 

assessing the data, they will have a better idea of how the data can be 

used. 

TSA has nearly completed the reactivation of the TIP system at airports 

nationwide and plans to use data it is collecting to improve the 

effectiveness of the passenger screening system.45 TIP is designed to test 

passenger screeners’ detection capabilities by projecting threat images, 

including guns, knives, and explosives, onto bags as they are screened 

during actual operations. Screeners are responsible for identifying the 

threat image and calling for the bag to be searched. Once prompted, TIP 

identifies to the screener whether the threat is real and then records the 

screener’s performance in a database that could be analyzed for 

 44The local covert testing protocols were updated in June 2004 and August 2004 to provide 

information on alternative testing methods. 

45TIP is not yet operational at one airport because of construction at the screening 

checkpoint to prepare for its installation. However, the TIP-ready X-ray machines have 

already been procured for the airport and are expected to be installed once the 

construction issues have been resolved. 

TSA Has Reinstated the 

Threat Image Projection 

System and Plans to Use 

Its Data to Improve the 

Screening System 
Page 36 GAO-05-457 Screener Training and Performance Measurement Strengthened 

performance trends.46 TSA is evaluating the possibility of developing an 

adaptive functionality to TIP. Specifically, as individual screeners become 

proficient in identifying certain threat images, such as guns or knives, they 

will receive fewer of those images and more images that they are less 

proficient at detecting, such as improvised explosive devices. 

TIP was activated by FAA in 1999 with about 200 threat images, but it was 

shut down immediately following the September 11 terrorist attacks 

because of concerns that it would result in screening delays and panic, as 

screeners might think that they were actually viewing threat objects. In 

October 2003, TSA began reactivating and expanding TIP. In April 2004, 

we reported that TSA was reactivating TIP with an expanded library of 

2,400 images at all but one of the more than 1,800 checkpoint lanes 

nationwide. To further enhance screener training and performance, TSA 

also plans to develop at least an additional 50 images each month. 

Despite these improvements, TIP is not yet available for checked baggage 

screening. In April 2004, we reported that TSA officials stated that they 

were working to resolve technical challenges associated with using TIP for 

checked baggage screening on EDS machines and have started EDS TIP 

image development. The DHS OIG reported in September 2004 that TSA 

plans to implement TIP on all EDS machines at checked baggage stations 

nationwide in fiscal year 2005. However, in December 2004, TSA officials 

stated that because of severe budget reductions, TSA will be unable to 

begin implementing a TIP program for checked baggage in fiscal year 2005. 

They did not specify when such a program might begin. 

TSA plans to use TIP data to improve the passenger screening system in 

two ways. First, TIP data can be used to measure screener threat detection 

effectiveness by different threats. Second, TSA plans to use TIP results to 

help identify specific recurrent training needs within and across airports 

and to tailor screeners’ recurrent training to focus on threat category areas 

that indicate a need for improvement. TSA considers February 2004 as the 

first full month of TIP reporting with the new library of 2,400 images. TSA 

began collecting these data in early March 2004 and is using the data to 

determine more precisely how they can be used to measure screener 

 46The TIP database records both the TIP hit rate and TIP false alarm rate. These two results 

are used to determine the probability of detection and probability of false alarm, which 

determine overall TIP performance. 
Page 37 GAO-05-457 Screener Training and Performance Measurement Strengthened 

performance in detecting threat objects and to determine what the data 

identify about screener performance. 

TSA does not currently plan to use TIP data as an indicator of individual 

screener performance because TSA does not believe that TIP by itself 

adequately reflects a screener’s performance. Nevertheless, in April 2004, 

TSA gave FSDs the capability to query and analyze TIP data in a number of 

ways, including by screener, checkpoint, and airport. FSDs for over 

60 percent of the airports included in our airport-specific survey stated 

that they use or plan to use TIP data as a source of information in their 

evaluations of individual screener performance. Additionally, FSDs for 

50 percent of the airports covered in our survey reported using data 

generated by TIP to identify specific training needs for individual 

screeners. 

In September 2004, the DHS OIG reported that TSA is assessing the cost 

and feasibility of modifying TIP so that it recognizes and responds to 

specific threat objects with which individual screeners are most and least 

competent in detecting, over time. This feature would increase the utility 

of TIP as a training tool. The DHS OIG also reported that TSA is 

considering linking TIP over a network, which would facilitate TSA’s 

collection, analysis, and information-sharing efforts around TIP user 

results. The report recommended that TSA continue to pursue each of 

these initiatives, and TSA agreed. However, in December 2004, TSA 

officials stated that the availability of funding will determine whether or 

not they pursue these efforts further. 

TSA has completed its first round of the screener recertification program, 

and the second round is now under way. However, TSA does not currently 

include an image recognition component in the test for checked baggage 

screener recertification. ATSA requires that each screener receive an 

annual proficiency review to ensure he or she continues to meet all 

qualifications and standards required to perform the screening function. In 

September 2003, we reported that TSA had not yet implemented this 

requirement.47 To meet this requirement, TSA established a recertification 

program, and it began recertification testing in October 2003 and 

 47GAO-03-1173. 

TSA Has Completed Its 

First Round of Screener 

Recertification Testing, but 

Testing for Checked 

Baggage Screeners Is Not 

as Comprehensive as for 

Passenger Screeners 
Page 38 GAO-05-457 Screener Training and Performance Measurement Strengthened 

completed the testing in March 2004. 48 The first recertification program 

was composed of two assessment components, one of screeners’ 

performance and the other of screeners’ knowledge and skills. During the 

performance assessment component of the recertification program, 

screeners are rated on both organizational and individual goals, such as 

maintaining the nation’s air security, vigilantly carrying out duties with 

utmost attention to tasks that will prevent security threats, and 

demonstrating the highest levels of courtesy to travelers to maximize their 

levels of satisfaction with screening services. The knowledge and skills 

assessment component consists of three modules: (1) knowledge of 

standard operating procedures, (2) image recognition, and (3) practical 

demonstration of skills. Table 4 provides a summary of these three 

modules. 

Table 4: Modules Included in Recertification Knowledge and Skills Assessment 

Testing modules Description 

Knowledge of standard operating 

procedures 

Computerized 50-question multiple-choice test. It 

is either passenger- or baggage-specific. 

Image recognition Computerized test that consists of 100 images 

and is used to evaluate a screener’s skill and 

ability in detecting threat or prohibited objects 

within X-ray images. 

Practical demonstration of skills Hands-on simulated work sample to evaluate a 

screener’s knowledge, skills, and ability when 

performing specific screener tasks along with 

ability to provide customer service. 

Source: TSA. 

To be recertified, screeners must have a rating of “met” or “exceeded” 

standards on their annual performance assessments and have passed each 

of the applicable knowledge and skills modules. Screeners that failed any 

of the three modules were to receive study time or remedial training as 

well as a second opportunity to take and pass the modules. Screeners who 

failed on their second attempt were to be removed from screening duties 

and subject to termination. Screeners could also be terminated for 

receiving a rating of below “met” standards. 

 48Some screeners, such as those on extended leave, leave without pay, military leave, or 

leave because of an on-the-job injury were not tested. These screeners are retested as they 

return to work. 
Page 39 GAO-05-457 Screener Training and Performance Measurement Strengthened 

TSA completed its analysis of the recertification testing and performance 

evaluations in May 2004. TSA’s analysis shows that less than 1 percent of 

screeners subject to recertification failed to complete this requirement. 

Figure 6 shows the recertification results. 

Figure 6: Screener Recertification Results, October 2003–March 2004 

Across all airports screeners performed well on the recertification testing. 

Over 97 percent of screeners passed the standard operating procedures 

test on their first attempt. Screeners faced the most difficulty on the 

practical demonstration of skills component. However, following 

remediation, 98.6 percent of the screeners who initially failed this 

component passed on their second attempt. Table 5 shows the results of 

the recertification testing by module. 

Table 5: Screener Recertification Module Testing Percentage Pass Rates, 

October 2003—March 2004 

Standard operating 

procedures Image recognition 

Practical skills 

demonstration

First attempt 97.4 96.0 75.2

Retest 96.8 84.3 98.6

Overall 99.9 99.5 99.7

Source: TSA.

Source: GAO analysis of TSA data.

42,690 Screeners recertified

352 Screeners not recertified

99.2%

0.8%

Screeners recertified

Screeners not recertified
Page 40 GAO-05-457 Screener Training and Performance Measurement Strengthened 

As shown in table 6, screeners hired as checked baggage screeners were 

not required to complete the image recognition module in the first round 

of the recertification testing.49 

Table 6: Recertification Testing Modules by Screening Function 

Testing modules Passenger 

Checked 

baggage 

Knowledge of standard operating procedures √ √

Image recognition √ 

Practical demonstration of skills √ √

Source: TSA. 

In addition, during the first year of recertification testing, which took place 

from October 2003 through May 2004, dual-function screeners who were 

actively working as both passenger and checked baggage screeners were 

required to take only the recertification test for passenger screeners. They 

were therefore not required to take the recertification testing modules 

required for checked baggage, even though they worked in that capacity.50

TSA began implementing the second annual recertification testing in 

October 2004 and plans to complete it no later than June 2005. This 

recertification program includes components for dual-function screeners. 

However, TSA still has not included an image recognition module for 

checked baggage screeners—which would include dual-function screeners 

performing checked baggage screening. TSA officials stated that a decision 

was made to not include an image recognition module for checked 

baggage screeners during this cycle because not all checked baggage 

screeners would have completed training on the onscreen resolution 

protocol by the time recertification testing was conducted at their 

airports.51 In December 2004, TSA officials stated that they plan on 

 49According to TSA records, between October 2003 and May 2004, 13,516 screeners 

completed the recertification testing as baggage screeners. 

50As of January 7, 2005, TSA reported that its workforce included approximately 25,947 

dual-trained screeners who were certified to serve as passenger or baggage screeners. 

51TSA’s onscreen resolution protocol requires that when an EDS machine alarm goes off, 

indicating the possibility of explosives, TSA screeners, by reviewing computer-generated 

images of the inside of the bag, attempt to determine whether or not a suspect item or 

items are in fact explosive materials. If the screener is unable to make this determination, 

the bag is diverted from the main conveyor belt into an area where it receives a secondary 

screening by a screener with an ETD machine. 
Page 41 GAO-05-457 Screener Training and Performance Measurement Strengthened 

developing an image recognition module for checked baggage and dualfunction screeners, and that this test should be available for next year’s 

recertification program. The development and implementation of the 

image recognition test will be contingent, they stated, upon the availability 

of funds. 

TSA has implemented a number of improvements designed to enhance 

screener performance, based on concerns it identified in a July 2003 

Passenger Screener Performance Improvement Study and 

recommendations from OIAPR. To date, however, these efforts have 

primarily focused on the performance of passenger screeners, and TSA 

has not yet undertaken a comparable performance study for checked 

baggage screeners. The Passenger Screener Performance Improvement 

Study relied in part on the findings of OIAPR’s covert testing. At the time 

the study was issued, OIAPR had conducted fewer than 50 tests of 

checked baggage screeners. The July 2003 study focused on and included 

numerous recommendations for improving the performance of passenger 

screeners, but recommended waiting to analyze the performance of 

checked baggage screeners until some time after implementation of the 

recommendations, some of which TSA indicated, also applied to checked 

baggage screeners. TSA officials told us that this analysis has been 

postponed until they have reviewed the impact of implementing the 

recommendations on passenger screening performance. 

In October 2003, to address passenger screener performance deficiencies 

identified in the Screener Performance Improvement Study, TSA 

developed a Short-Term Screening Performance Improvement Plan. This 

plan included specific action items in nine broad categories—such as 

enhance training, increase covert testing, finish installing TIP, and 

expedite high-speed connectivity to checkpoints and training computers—

that TSA planned to pursue to provide tangible improvements in passenger 

screener performance and security (see app. IV for additional information 

on the action items). In June 2004, TSA reported that it had completed 57 

of the 62 specific actions. As of December 2004, two of these actions still 

had not been implemented—full deployment of high-speed connectivity 

TSA Has Identified and 

Implemented Efforts to 

Enhance Screener 

Performance, but These 

Efforts Primarily Focused 

on Passenger Screeners 
Page 42 GAO-05-457 Screener Training and Performance Measurement Strengthened 

and a time and attendance package—both of which continue to be 

deferred pending the identification of appropriate resources.52

In addition to the Performance Improvement Study and corresponding 

action plans, TSA’s OIAPR makes recommendations in its reports on 

covert testing results. These recommendations address deficiencies 

identified during testing and are intended to improve screening 

effectiveness. As of December 2004, OIAPR had issued 18 reports to TSA 

management on the results of its checkpoint and checked baggage covert 

testing.53 These reports include 14 distinct recommendations,54 some of 

which were included in TSA’s screener improvement action plan. All but 

two of these reports included recommendations on corrective actions 

needed to enhance the effectiveness of passenger and checked baggage 

screening. 

TSA has established performance measures, indexes, and targets for the 

passenger and checked baggage screening systems, but has not 

established targets for the various components of the screening indexes. 

The Government Performance and Results Act of 1993 provides, among 

other things, that federal agencies establish program performance 

measures, including the assessment of relevant outputs and outcomes of 

each program activity.55 Performance measures are meant to cover key 

aspects of performance and help decision makers to assess program 

accomplishments and improve program performance. A performance 

target is a desired level of performance expressed as a tangible, 

 52TSA officials stated that in early fiscal year 2005, TSA’s Office of Information Technology 

committed to extending high-speed connectivity to an additional 16 locations, but is still 

awaiting funding to further expand network coverage and to extend the network to 

checkpoints. 

53OIAPR has issued its reports to the TSA Administrator; the TSA Administrator’s Chief of 

Staff; Associate Undersecretary for Aviation Operations; Associate Undersecretary for 

Workforce, Performance and Training; Assistant Secretary and Chief Technology Officer; 

Assistant Administrator and Chief Support Systems Officer; Chief Operating Officer, the 

Office of Transportation Security and Intelligence, and the Office of Security Policy. The 

report recommendations are directed to the office(s) responsible for taking the corrective 

action. 

54Some recommendations appear repeatedly in multiple reports issued by OIAPR. 

55According to the Government Performance and Results Act, the Office of Management 

and Budget, and GAO, outcomes assess actual results as compared with the intended 

results or consequences that occur from carrying out a program or activity. Outputs count 

the goods and services produced by a program or organization. 

TSA Has Established 

Screening Performance 

Measures and Indexes but 

Has Not Established Key 

Performance Targets 
Page 43 GAO-05-457 Screener Training and Performance Measurement Strengthened 

measurable objective, against which actual achievement will be compared. 

By analyzing the gap between target and actual levels of performance, 

management can target those processes that are most in need of 

improvement, set improvement goals, and identify appropriate process 

improvements or other actions. 

An April 2004 consultant study commissioned by TSA found that FSDs and 

FSD staffs generally believed the lack of key performance indicators 

available to monitor passenger and checked baggage screening 

performance represented a significant organizational weakness. Since 

then, TSA has established over 20 performance measures for the 

passenger and checked baggage screening systems. For example, TSA 

measures the percentage of screeners meeting a threshold score on the 

annual recertification testing on their first attempt, the percentage of 

screeners scoring above the national standard level on TIP performance, 

and the number of passengers screened, by airport category. 

TSA also has developed two performance indexes to measure the 

effectiveness of the passenger and checked baggage screening systems.56

These indexes measure overall performance through a composite of 

indicators and are derived by combining specific performance measures 

relating to passenger and checked baggage screening, respectively. 

Specifically, these indexes measure the effectiveness of the screening 

systems through machine probability of detection and covert testing 

results;57 efficiency through a calculation of dollars spent per passenger or 

bag screened; and customer satisfaction through a national poll, customer 

surveys, and customer complaints at both airports and TSA’s national call 

center. According to TSA officials, the agency has finalized targets for the 

two overall indexes, but these targets have not yet been communicated 

throughout the agency. Further, TSA plans to provide the FSDs with only 

the performance index score, not the value of each of the components, 

because the probabilities of detection are classified as secret and TSA is 

concerned that by releasing components, those probabilities could be 

 56TSA analyzed data from fiscal years 2003 and 2004 to establish baselines and establish 

performance targets. 

57According to TSA, the machine probabilities of detection are established by the 

certification standards for each particular model of machines, and machines are not 

deployed unless they have met those standards. 
Page 44 GAO-05-457 Screener Training and Performance Measurement Strengthened 

deduced.58 Table 7 summarizes the components of the performance 

indexes developed by TSA. 

Table 7: Components of TSA’s Performance Indexes 

Performance index Components 

Weight 

(percent) 

FY 2005–2010 

performance 

target 

(1-5 scale)

• Machine probability of 

detection x person 

probability of detection 

(covert testing) 50

• Cost per person screened 25 3.3

Passenger screening 

program 

• Consumer satisfaction 25

• Machine probability of 

detection x person 

probability of detection 

(covert testing) 50

• Cost per bag screened 25 3.2

Checked baggage 

screening program 

• Consumer satisfaction 25

Source: TSA. 

TSA has not yet established performance targets for the various 

components of the screening indexes, including performance targets for 

covert testing (person probability of detection). TSA’s strategic plan states 

that the agency will use the performance data it collects to make tactical 

decisions based on performance. The screening performance indexes 

developed by TSA can be a useful analysis tool, but without targets for 

each component of the index, TSA will have difficulty performing 

meaningful analyses of the parts that add up to the index. For example, 

without performance targets for covert testing, TSA will not have 

identified a desired level of performance related to screener detection of 

threat objects. Performance targets for covert testing would enable TSA to 

focus its improvement efforts on areas determined to be most critical, as 

100 percent detection capability may not be attainable. In January 2005, 

TSA officials stated that the agency plans to track the performance of 

individual index components and establish performance targets against 

 58TSA headquarters officials stated that their intent is to provide FSDs with various 

mechanisms to assess their screening effectiveness at the airports for which they are 

responsible. Specifically, they stated that FSDs are provided with TIP data and the results 

of OIAPR’s covert testing at their airports. Additionally, they have access to the results of 

local covert testing at their airports. 
Page 45 GAO-05-457 Screener Training and Performance Measurement Strengthened 

which to measure these components. They further stated that they are 

currently collecting and analyzing baseline data to establish these targets 

and plan to finalize them by the end of fiscal year 2005. 

It has been over 2 years since TSA assumed responsibility for passenger 

and checked baggage screening operations at the nation’s commercial 

airports. TSA has made significant accomplishments over this period in 

meeting congressional mandates related to establishing these screening 

operations. With the congressional mandates now largely met, TSA has 

turned its attention to assessing and enhancing the effectiveness of its 

passenger and checked baggage screening systems. An important tool in 

enhancing screener performance is ongoing training. As threats and 

technology change, the training and development of screeners to ensure 

they have the competencies—knowledge, skills, abilities, and behaviors—

needed to successfully perform their screening functions become vital to 

strengthening aviation security. Without addressing the challenges to 

delivering ongoing training, including installing high-speed connectivity at 

airport training facilities, TSA may have difficulty maintaining a screening 

workforce that possesses the critical skills needed to perform at a desired 

level. In addition, without adequate internal controls designed to help 

ensure screeners receive required training that are also communicated 

throughout the agency, TSA cannot effectively provide reasonable 

assurances that screeners receive all required training. Given the 

importance of the Online Learning Center in both delivering training and 

serving as the means by which the completion of screener training is 

documented, TSA would benefit from having a clearly defined plan for 

prioritizing the deployment of high-speed Internet/intranet connectivity to 

all airport training facilities. Such a plan would help enable TSA to move 

forward quickly and effectively in deploying high-speed connectivity once 

funding is available. 

Additionally, history demonstrates that U.S. commercial aircraft have long 

been a target for terrorist attacks through the use of explosives carried in 

checked baggage, and covert testing conducted by TSA and DHS OIG have 

identified that weaknesses and vulnerabilities continue to exist in the 

passenger and checked baggage screening systems, including the ability of 

screeners to detect threat objects. While covert test results provide an 

indicator of screening performance, they cannot solely be used as a 

comprehensive measure of any airport’s screening performance or any 

individual screener’s performance, or in determining the overall 

performance of federal versus private-sector screening. Rather, these data 

should be considered in the larger context of additional performance data, 

Conclusions 
Page 46 GAO-05-457 Screener Training and Performance Measurement Strengthened 

such as TIP and recertification test results, when assessing screener 

performance. While TSA has undertaken efforts to measures and 

strengthen performance, these efforts have primarily focused on 

passenger screening and not on checked baggage screening. TSA’s plans 

for implementing TIP for checked baggage screening, and establishing an 

image recognition component for the checked baggage screeners 

recertification testing—plans made during the course of our review—

represent significant steps forward in its efforts to strengthen checked 

baggage screening functions. Additionally, although TSA has developed 

passenger and checked baggage screening effectiveness measures, the 

agency has not yet established performance targets for the individual 

components of these measures. Until such targets are established, it will 

be difficult for TSA to draw more meaningful conclusions about its 

performance and how to most effectively direct its improvement efforts. 

For example, performance targets for covert testing would enable TSA to 

focus its improvement efforts on areas determined to be most critical, as 

100 percent detection capability may not be attainable. We are encouraged 

by TSA’s recent plan to establish targets for the individual components of 

the performance indexes. This effort, along with the additional 

performance data TSA plans to collect on checked baggage screening 

operations, should assist TSA in measuring and enhancing screening 

performance and provide TSA with more complete information with which 

to prioritize and focus its screening improvement efforts. 

To help ensure that all screeners have timely and complete access to 

screener training available in the Online Learning Center and to help 

provide TSA management with reasonable assurance that all screeners are 

receiving required passenger and checked baggage screener training, we 

recommend that the Secretary of the Department of Homeland Security 

direct the Assistant Secretary, Transportation Security Administration, to 

take the following two actions: 

• develop a plan that prioritizes and schedules the deployment of high-speed 

Internet/intranet connectivity to all TSA’s airport training facilities to help 

facilitate the delivery of screener training and the documentation of 

training completion, and 

• develop internal controls, such as specific directives, clearly defining 

responsibilities for monitoring and documenting the completion of 

required training, and clearly communicate these responsibilities 

throughout the agency. 

Recommendations for 

Executive Action 
Page 47 GAO-05-457 Screener Training and Performance Measurement Strengthened 

We provided a draft of this report to DHS for review and comment. On 

February 4, 2005, we received written comments on the draft report, which 

are reproduced in full in appendix V. DHS generally concurred with the 

findings and recommendations in the report, and agreed that efforts to 

implement our recommendations are critical to successful passenger and 

checked baggage screening training and performance. With regard to our 

recommendation that TSA develop a plan that prioritizes and schedules 

the deployment of high-speed Internet/intranet connectivity to all TSA’s 

airport training facilities, DHS stated that TSA has developed such a plan. 

However, although we requested a copy of the plan several times during 

our review and after receiving written comments from DHS, TSA did not 

provide us with a copy of the plan. Therefore, we cannot assess the extent 

to which the plan DHS referenced in its written comments fulfills our 

recommendation. In addition, regarding our recommendation that TSA 

develop internal controls clearly defining responsibilities for monitoring 

and documenting the completion of required training, and clearly 

communicate those responsibilities throughout TSA, DHS stated that it is 

taking steps to define responsibility for monitoring the completion of 

required training and to insert this accountability into the performance 

plans of all TSA supervisors. TSA’s successful completion of these ongoing 

and planned activities should address the concerns we raised in this 

report. DHS has also provided technical comments on our draft report, 

which we incorporated where appropriate. 

As agreed with your office, we will send copies of this report to relevant 

congressional committees and subcommittees and to the Secretary of the 

Department of Homeland Security. We will also make copies available to 

others upon request. In addition, the report will be made available at no 

charge on GAO’s Web site at http://www.gao.gov. 

Agency Comments 

and Our Evaluation 
Page 48 GAO-05-457 Screener Training and Performance Measurement Strengthened 

If you have any questions about this report or wish to discuss it further, 

please contact me at (202) 512-8777. Key contributors to this report are 

listed in appendix VI. 

Sincerely yours, 

Cathleen A. Berrick 

Director, Homeland Security 

 and Justice Issues 
Appendix I: Summary of Previous Findings 

Related to Screener Training and 

Performance 

Page 49 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Product date Summary of previous findings related to screener training and performance 

Title and GAO product 

number

September 2003 The Transportation Security Administration (TSA) had deployed a basic screener 

training program and required remedial training but had not fully developed or 

deployed a recurrent training program for screeners or supervisors. 

TSA had collected little information to measure screener performance in detecting 

threat objects. 

• TSA’s Office of Internal Affairs and Program Review’s (OIAPR) covert testing 

was the primary source of information collected on screeners’ ability to detect 

threat objects. However, TSA did not consider the covert testing a measure of 

screener performance. 

• TSA was not using the Threat Image Projection system (TIP) but planned to 

fully activate the system with significantly more threat images than previously 

used in October 2003. 

• TSA had not yet implemented an annual proficiency review to ensure that 

screeners met all qualifications and standards required to perform their 

assigned screening functions. 

Airport Passenger 

Screening: Preliminary 

Observations Made and 

Challenges Remaining 

GAO-03-1173

November 2003 Although little data existed on the effectiveness of passenger screening, TSA was 

implementing several efforts to collect performance data. 

• TSA’s OIAPR had conducted little covert testing of the screener workforce but 

planned to double the number of tests it conducted during fiscal year 2004. 

• TSA only recently began activating TIP on a wide-scale basis and expected it 

to be fully operational at every checkpoint at all airports by April 2004. 

• TSA only recently began implementing the annual recertification program and 

did not expect to complete testing at all airports until March 2004. 

• TSA was developing performance indexes for individual screeners and the 

screening system as a whole but had not fully established these indexes. TSA 

expected to have them in place by the end of fiscal year 2004. 

Aviation Security: Efforts to 

Measure Effectiveness and 

Address Challenges 

GAO-04-232T

November 2003 Although TSA had not fully developed or deployed recurrent or supervisory 

training programs, it was in the process of 

• deploying six recurrent training modules and was pilot-testing an online 

learning management system, and 

• working with the U.S. Department of Agriculture’s Graduate School to tailor its 

off-the-shelf supervisory course to meet the specific training needs of screening 

supervisors. 

Aviation Security: Efforts to 

Measure Effectiveness and 

Strengthen Security 

Programs 

GAO-04-285T

February 2004 While TSA had taken steps to enhance its screener training programs, staffing 

imbalances, and lack of high-speed connectivity at airport training facilities had 

made it difficult for screeners at some airports to fully utilize these programs. 

Although TSA was making progress in measuring the performance of passenger 

screeners, it had collected limited performance data related to its checked 

baggage screening operations. However, TSA had begun collecting additional 

performance data related to its checked baggage screening operations and 

planned to increase these efforts in the future. 

As part of its efforts to develop performance indexes, TSA was developing 

baseline data for fiscal year 2004 and planned to report the indexes to DHS in 

fiscal year 2005. 

Aviation Security: 

Challenges Exist in 

Stabilizing and Enhancing 

Passenger and Baggage 

Screening Operations 

GAO-04-440T

Appendix I: Summary of Previous Findings 

Related to Screener Training and 

Performance 
Appendix I: Summary of Previous Findings 

Related to Screener Training and 

Performance 

Page 50 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Product date Summary of previous findings related to screener training and performance 

Title and GAO product 

number

April 2004 With the exception of covert testing and recent TIP data, data were not yet 

available to assess how well screeners were performing and what steps if any 

TSA needed to take to improve performance. Also, TSA was not using TIP as a 

formal indicator of screening performance, but instead was using it to identify 

individual screener training needs. 

Aviation Security: Private 

Screening Contractors 

Have Little Flexibility to 

Implement Innovative 

Approaches 

GAO-04-505T

Source: GAO. 
Appendix II: Objectives, Scope, and 

Methodology 

Page 51 GAO-05-457 Screener Training and Performance Measurement Strengthened 

To examine efforts by the Transportation Security Administration to 

enhance their passenger and checked baggage screening programs, we 

addressed the following questions: (1) What actions has TSA taken to 

enhance training for screeners and supervisors? (2) How does TSA 

monitor compliance with screener training requirements? (3) What is the 

status of TSA’s efforts to assess and enhance screener performance in 

detecting threat objects? 

To determine how TSA has enhanced training for screeners and 

supervisors and how TSA has monitored compliance with screener 

training requirements, we obtained and analyzed relevant legislation, as 

well as TSA’s training plans, guidance, and curriculum. We reviewed data 

from TSA’s Online Learning Center and assessed the reliability of the 

Online Learning Center database. We compared TSA’s procedures for 

ensuring that screeners receive required training according to Standards 

for Internal Controls in the Federal Government. We interviewed TSA 

officials from the Office of Workforce Performance and Training and the 

Office of Aviation Operations in Arlington, Virginia. At the airports we 

visited, we interviewed Federal Security Directors and their staffs, such as 

Training Coordinators. We also met with officials from four aviation 

associations—the American Association of Airport Executives, Airports 

Council International, the Air Transport Association, and the Regional 

Airline Association. We did not assess the methods used to develop TSA’s 

screener training program, nor did we analyze the contents of TSA’s 

curriculum. Although we could not independently verify the reliability of 

all of this information, we compared the information with other supporting 

documents, when available, to determine data consistency and 

reasonableness. We found the data to be sufficiently reliable for our 

purposes. 

To determine what efforts TSA has taken to assess and to enhance 

screener performance in detecting threat objects, we reviewed related 

reports from the Department of Transportation and the Department of 

Homeland Security (DHS) Inspector General, Congressional Research 

Service, and TSA, as well as prior GAO reports. We obtained and reviewed 

TSA’s covert test data and results of the annual recertification testing. 

(Results of the covert testing are classified and will be the subject of a 

separate classified GAO report.) We discussed methods for inputting, 

compiling, and maintaining the data with TSA officials. We also assessed 

the methodology of TSA’s covert tests and questioned OIAPR officials 

about the procedures used to ensure the reliability of the covert test data. 

When we found discrepancies between the data OIAPR maintained in 

spreadsheets and the data included in the hard copy reports we obtained 

Appendix II: Objectives, Scope, and 

Methodology 
Appendix II: Objectives, Scope, and 

Methodology 

Page 52 GAO-05-457 Screener Training and Performance Measurement Strengthened 

from TSA, we worked with OIAPR to resolve the discrepancies. Further, 

we visited TSA headquarters to review TSA’s annual recertification testing 

modules and discuss TSA’s process for validating the recertification 

exams. As a result, we determined that the data provided by TSA were 

sufficiently reliable for the purposes of our review. We also reviewed 

TSA’s performance measures, targets, and indexes. Finally, we interviewed 

TSA headquarters officials from several offices in Arlington, Virginia, 

including Aviation Operations, Workforce Performance and Training, 

Strategic Management and Analysis, and Internal Affairs and Program 

Review. 

In addition, in accomplishing our objectives, we also conducted site visits 

at select airports nationwide to interview Federal Security Directors and 

their staffs and conducted two Web-based surveys of Federal Security 

Directors. Specifically, we conducted site visits at 29 airports (13 category 

X airports, 9 category I airports, 3 category II airports, 3 category III 

airports, and 1 category IV airport) to observe airport security screening 

procedures and discuss issues related to the screening process with TSA, 

airport, and airline officials. We chose these airports to obtain a crosssection of all airports by size and geographic distribution. In addition, we 

selected each of the five contract screening pilot airports. The results from 

our airport visits provide examples of screening operations and issues but 

cannot be generalized beyond the airports visited because we did not use 

statistical sampling techniques in selecting the airports. The category X 

airports we visited were Baltimore Washington International Airport, 

Boston Logan International Airport, Chicago O’Hare International Airport, 

Dallas/Fort Worth International Airport, Denver International Airport, 

Washington Dulles International Airport, John F. Kennedy International 

Airport, Los Angeles International Airport, Newark Liberty International 

Airport, Orlando International Airport, Ronald Reagan Washington 

National Airport, San Francisco International Airport, Seattle-Tacoma 

International Airport. The category I airports we visited were BurbankGlendale-Pasadena Airport, John Wayne Airport, Chicago Midway 

International Airport, Dallas Love Field, Kansas City International Airport, 

Little Rock National Airport, Metropolitan Oakland International Airport, 

Portland International Airport, and Tampa International Airport. The 

category II airports we visited were Jackson International Airport, Dane 

County Regional Airport, and Greater Rochester International Airport. The 

category III airports we visited were Idaho Falls Regional Airport, Jackson 

Hole Airport, and Orlando Sanford International Airport. The category IV 

airport we visited was Tupelo Regional Airport. 
Appendix II: Objectives, Scope, and 

Methodology 

Page 53 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Further, we administered two Web-based surveys to all 155 Federal 

Security Directors who oversee security at each of the airports falling 

under TSA’s jurisdiction. One survey, the general survey, contained 

questions covering local and national efforts to train screeners and 

supervisors and the status of TSA’s efforts to evaluate screener 

performance, including the annual recertification program and TIP. The 

second survey attempted to gather more specific airport security 

information on an airport(s) under the Federal Security Director’s 

supervision. For the airport-specific survey, each Federal Security 

Director received one or two surveys to complete, depending on the 

number of airports they were responsible for. Where a Federal Security 

Director was responsible for more than two airports, we selected the first 

airport based on the Federal Security Director’s location and the second 

airport to obtain a cross-section of all airports by size and geographic 

distribution. In all, we requested information on 265 airports. However, 

two airports were dropped from our initial selection because the airlines 

serving these airports suspended operations and TSA employees were 

redeployed to other airports. As a result our sample size was reduced to 

263 airports, which included all 21 category X, and 60, 49, 73, and 

60 category I through IV airports respectively. In that we did not use 

probability sampling methods to select the sample of airports that were 

included in our airport-specific survey, we cannot generalize our findings 

beyond the selected airports. 

A GAO survey specialist designed the surveys in combination with other 

GAO staff knowledgeable about airport security issues. We conducted 

pretest interviews with six Federal Security Directors to ensure that the 

questions were clear, concise, and comprehensive. In addition, TSA 

managers and an independent GAO survey specialist reviewed the survey. 

We conducted these Web-based surveys from late March to mid-May 2004. 

We received completed general surveys from all 155 Federal Security 

Directors and completed airport-specific surveys for all 263 separate 

airports for which we sought information, for 100 percent response rates. 

The surveys’ results are not subject to sampling errors because all Federal 

Security Directors were asked to participate in the surveys and we did not 

use probability-sampling techniques to select specific airports. However, 

the practical difficulties of conducting any survey may introduce other 

errors, commonly referred to as nonsampling errors. For example, 

difficulties in how a particular question is interpreted, in the sources of 

information that are available to respondents, or in how the data are 

entered into a database or were analyzed can introduce unwanted 

variability into the survey results. We took steps in the development of the 
Appendix II: Objectives, Scope, and 

Methodology 

Page 54 GAO-05-457 Screener Training and Performance Measurement Strengthened 

surveys, the data collection, and the data editing and analysis to minimize 

these nonsampling errors. Also, in that these were Web-based surveys 

whereby respondents entered their responses directly into our database, 

there was little possibility of data entry or transcription error. In addition, 

all computer programs used to analyze the data were peer reviewed and 

verified to ensure that the syntax was written and executed correctly. 

We performed our work from May 2003 through April 2005 in accordance 

with generally accepted government auditing standards. Certain 

information we obtained and analyzed regarding screener training and 

performance are classified or are considered by TSA to be sensitive 

security information. Accordingly, the results of our review of this 

information are not included in this report.1

 1

We issued two additional reports detailing the results of our review, which discuss 

information deemed to be classified or sensitive security information. The report 

containing sensitive security information is GAO, Aviation Security: Screener Training 

and Performance Measurement Strengthened, but More Work Remains, GAO-05-143SU

(Washington D.C.: Feb. 28, 2005). The report containing classified and sensitive security 

information is GAO, Results of Transportation Security Administration’s Covert Testing 

for Passenger and Checked Baggage Screening for September 2002 through 

September 2004, GAO-05-437C (Washington D.C.: Apr. 7, 2005). 
Appendix III: TSA Screener Training Tools 

Designed to Help Improve Screener 

Performance 

Page 55 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Training tool Purpose Status reported by TSA 

Hand Held Metal Detector 

and Pat Down Video 

Provide an informative and effective learning tool to enhance 

screeners’ skills in the areas of hand-wanding and pat-down 

searches of passengers. 

Deployed November 21, 2003 

MBS II Weapons Kits This tool allows screeners to touch actual improvised explosive 

device (IED) components and build their own devices. This 

experiential learning will enable screeners to more readily detect real 

IEDs during screening. These weapons are also used to assist in 

training by using them for live testing conducted by FSD staff. 

Deployed January 26, 2004 

Firearms Weapons Kits This tool allows screeners to touch actual firearms and begin to 

understand how they can be broken down into various parts. By 

understanding this and experiencing it, screeners are better able to 

see the components of a firearm during actual screening. These 

weapons are also used to assist in training by using them for live 

testing conducted by FSD staff. 

Deployed January 26, 2004 

X-Ray Operator Video Maintain and enhance the screeners’ X-ray image operational skills. Deployed February 5, 2004 

X-Ray Tutor Version 1 Provide a tool that includes about 14,000 image combinations to 

practice threat identification. 

Deployed February 15, 2004 

Basic Supervisory Technical 

Web-Based Training 1 

Assist screener supervisors to understand additional roles they must 

perform, including enhanced technical skills needed to adequately 

supervise screening functions and resolve alarms using interactive, 

performance-based training tool 

Deployed April 16, 2004 

Mobile Training Assist 

Teams (MTAT) 

These teams go into airports where data shows performance needs 

attention. The team offers a variety of services to assist in improving 

the performance, such as on-the-spot training and consulting 

services. Team visits can be initiated by FSDs, Internal Affairs 

reports, Quality Assurance trips, or MTAT Supervisors proactively 

visiting the airport and FSD. 

Site visits completed from 

October 2003 through 

December 3, 2004: 

• North Central (37 visits) 

• South Central (51 visits) 

• Northeast (25 visits) 

• Southeast (60 visits) 

• Western (53 visits) 

• 54 FSD Stakeholder 

Relations Meetings 

Basic Screener Supervisory 

Classroom Training 

Improve screener supervisors’ knowledge of federal government and 

TSA personnel rules and how to effectively coach and communicate 

with employees. 

Approximately 3,800 supervisors 

have been trained. 

ETD Maintenance Course Certification of screeners to perform supervisory maintenance tasks 

above and beyond operator training. 

Delivered April 1, 2004 

Credential Verification 

Training 

Provide students with basic skills needed to verify the identity of flying 

armed law enforcement officers. 

Deployed April 15, 2004 

Threat in the Spotlight This weekly product brings to light actual cases of weapons being 

found by law enforcement, with an explanation of how those weapons 

could be used to attack aviation. 

Deployed weekly in 2004 

EDS Operations WebBased Training Provide interactive, performance based recurrent Web-based training 

modules for checked baggage explosive detection systems (EDS). 

Deployed April 30, 2004 

Appendix III: TSA Screener Training Tools 

Designed to Help Improve Screener 

Performance 
Appendix III: TSA Screener Training Tools 

Designed to Help Improve Screener 

Performance 

Page 56 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Training tool Purpose Status reported by TSA 

Handwanding and Pat 

Down Web-Based Training 

Improve screener performance by providing an interactive tool 

complementary to Hand Held Metal Detector and Pat Down Video 

that allows the screener to practice proper techniques and receive 

immediate feedback. 

Deployed April 30, 2004 

Customer Service WebBased Training Reinforces TSA’s customer service principles and places the 

screener in various situations requiring effective customer service 

responses. 

Deployed April 30, 2004 

Checkpoint and Checked 

Baggage Operations WebBased Training 

Provide interactive, performance-based recurrent training modules for 

checkpoint and checked baggage operations. 

Deployed April 30, 2004 

Physical Bag Search Video Maintain and enhance screeners’ explosive trace detection (ETD) 

and physical bag search skills for carry-on and checked baggage. 

Deployed April 30, 2004 

ETD and Physical Bag 

Search Web-Based Training 

Provide interactive recurrent Web-based training modules for ETD 

and physical bag search. 

Deployed April 30, 2004 

Prohibited Items WebBased Training Provide an interactive, performance-based training tool to enhance 

screener’s ability to identify prohibited items. 

Deployed June 25, 2004 

Effectively Screening 

Prosthetics Video 

Provide an informative and effective learning tool to maintain and 

enhance the skills of screeners in the areas of persons with 

prosthetics. 

Deployed December 16, 2004 

X-Ray Tutor Version 2 Provide a tool to practice threat identification with about 10,000,000 

image combinations. 

Scheduled to be deployed 

during the second quarter of 

fiscal year 2005 

X-Ray Simulator Training Sharing the X-Ray Tutor Version 2 library, this tool will allow 

screeners to practice finding threat items using the full capabilities of 

the TIP-ready X-ray machines. 

Scheduled to be deployed 

during the second quarter of 

fiscal year 2005 

Basic Supervisory Technical 

Web-Based Training 2 

Provide an interactive, performance-based tool to convey how the 

supervisor is to handle screening situations, handed off by the 

screening, following standard operator procedures. 

To be developed 

Safety, Lifting, and Twisting 

Web-Based Training 

Provide a Web-based training that will engage the student with 3-

dimensional representations of the muscular frame, showing proper 

lifting techniques and the results of improper techniques. 

Scheduled to be deployed 

during the second quarter of 

fiscal year 2005 

Source: TSA. 
Appendix IV: Summary of TSA’s Short-Term 

Action Items for Strengthening Passenger 

Screener Performance 

Page 57 GAO-05-457 Screener Training and Performance Measurement Strengthened 

People Action item Description Benefit 

1 Increase FSD support 

and accountability 

Hold FSDs accountable for screening 

performance and delivery of security 

Management accountability is driven down to the local 

airport 

FSD performance is linked to screener performance, 

creating incentives for maintaining and improving 

security 

2 Enhance training Provide ongoing training for screeners 

and supervisors to maintain their skills 

and provide new skills and techniques 

based on evolving threats and lessons 

learned 

Maintains and improves knowledge base of screeners 

Ensures proper oversight by supervisors 

Ensures that screeners are capable of addressing 

evolving threats 

3 Increase Internal 

Affairs covert testing 

Increase the frequency of TSA covert 

testing 

Improved identification of systemic vulnerabilities in 

airport security systems 

Immediate implementation of limited remedial actions 

4 Continue to pursue 

human performance 

improvements 

Better understand reasons and causes for 

human errors and interactions with 

technology in order to identify 

opportunities for performance 

improvements, with a goal of identifying 

optimum work conditions 

Reduces human-based errors 

Increases workforce morale and working conditions, 

leading to improved performance 

Technology 

5 Continue to identify 

screening technology 

improvements 

Continue to research alternative 

technologies and seek short-term 

technological solutions, especially for 

potential vectors 

Identifies threats more accurately and quickly 

Decreases number of false positives from equipment 

6 Finish installing TIP The TIP system is a series of 2,400 

images of threat objects that can be 

automatically fed into X-ray machines 

during actual screening 

Maintains alertness of screeners 

Identifies individual screener performance issues 

7 Expedite high-speed 

connectivity to 

checkpoints and 

training computers 

Connect all TSA offices, checkpoints and 

screening equipment (X-rays, EDS 

machines) to the Internet in order to 

automate and improve processes that are 

currently done manually or not at all 

Provides immediate feedback on and response to 

screener performance issues 

Improves communication with managers in the field 

Process 

8 Refresh aviation 

operations policy, 

procedures, and 

practice 

Conduct a thorough and expedited review 

of all policies and procedures developed 

during the rollout of TSA with a focus on 

increasing screening performance and 

capabilities 

Maintains “freshness” of standard operating 

procedures based on most recent intelligence about 

security threats 

Removes or updates outdated or unnecessary 

screening techniques based on lessons learned 

9 Improve workforce 

management 

Determine the optimal workforce staffing 

levels based on latest passenger flows 

and other factors 

Maximizes utilization of existing resources 

Source: TSA. 

Appendix IV: Summary of TSA’s Short-Term 

Action Items for Strengthening Passenger 

Screener Performance
Appendix V: Comments from the Department of Homeland 

Security 

Page 58 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Appendix V: Comments from the Department 

of Homeland Security 
Appendix V: Comments from the Department of Homeland 

Security 

Page 59 GAO-05-457 Screener Training and Performance Measurement Strengthened 
Page 60 GAO-05-457 Screener Training and Performance Measurement Strengthened 

Cathleen A. Berrick (202) 512-8777 

Maria D. Strudwick (202) 512-5419 

In addition to those named above, David Alexander, Leo Barbour, 

Lisa Brown, Elizabeth Curda, Kevin Dooley, Kathryn Godfrey, 

David Hooper, Christopher Jones, Stuart Kaufman, Kim Gianopoulos, 

Thomas Lombardi, Cady S. Panetta, Minette Richardson, Sidney Schwartz, 

Su Jin Yon, and Susan Zimmerman were key contributors to this report. 

Appendix VI: GAO Contacts and 

Acknowledgments 

GAO Contacts 

Staff 

Acknowledgments 

(440391) 
The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER