DEPARTMENT OF HOMELAND SECURITY 

Office of Inspector General 

An Evaluation of the Transportation
	

Security Administration’s Screener
	

Training and Methods of Testing
	

Notice: The Department of Homeland Security, Office of Inspector General, has redacted this 

report for public release based on input from the relevant government entity pursuant to 49 

C.F.R. Part 1520. Those redactions are identified as Part 1520 in the report. A review under the 

Freedom of Information Act, 5 U.S.C. § 552, will be conducted upon request. 

Office of Inspections, Evaluations, & Special Reviews 

OIG-04-045 September 2004 

appreciation to all of those who contributed to the preparation of this report. 

Clark Kent Ervin 

Inspector General 

Offi ce of Inspector General 

U.S. Department of Homeland Security 

Washington, DC 20528 

Preface 

The Department of Homeland Security (DHS) Office of Inspector General (OIG) was established 

by the Homeland Security Act of 2002 (Public Law 107-296) by amendment to the Inspector 

General Act of 1978. This is one of a series of audit, inspection, investigative, and special reports 

prepared by the OIG as part of its DHS oversight responsibility to identify and prevent fraud, 

waste, abuse, and mismanagement. 

This report assesses the strengths and weaknesses of the program or operation under review. It 

is based on interviews with employees and officials of relevant agencies and institutions, direct 

observations, and a review of applicable documents. 

The recommendations herein have been developed to the best knowledge available to the OIG, 

and have been discussed in draft with those responsible for implementation. It is my hope that 

this report will result in more effective, efficient, and economical operations. I express my 

Contents
	

Introduction......................................................................................................................................5
	

Results in Brief ................................................................................................................................6
	

Background......................................................................................................................................9
	

Purpose, Scope, and Methodology.................................................................................................13
	

Findings.......................................................................................... .................14
	

Basic Screener Classroom Training.........................................................................................14
	

Analysis..............................................................................................................................14
	

Design ................................................................................................................................16
	

Implementation ..................................................................................................................28
	

Evaluation ..........................................................................................................................37
	

Basic Screener On-The-Job Training.......................................................................................41
	

Analysis..............................................................................................................................41
	

Design ................................................................................................................................43
	

Implementation ..................................................................................................................47
	

Evaluation ..........................................................................................................................48
	

Recurrent Training and Recertification Testing .......................................................................49
	

Recurrent Program Has Been Established .........................................................................50
	

Annual Screener Recertification Testing for FY 2003-04 Has Been Completed...............58
	

Other Issues Related to Screener Training, Screener Performance, and an Increase in the 

Detection of Prohibited Items ..................................................................................................62
	

Use of the Online Learning Center Is Limited by Network Access...................................62
	

Specific Screening Equipment Training and Certification Is Necessary............................66
	

Efforts to Advance the Development and Deployment of New Screening Technologies 

Should Be Accelerated....................................................................................................... 70
	

Future Planning Should Account for Continually Changing Training Requirements .......73
	

An Evaluation of TSA’s Screener Training and Methods of Testing Page 1
	
Contents
	

Appendices 

Appendix A: Management Comments.......................................................................................75
	

Appendix B: OIG Evaluation of Management Comments ........................................................94
	

Appendix C: Recommendations ..............................................................................................107
	

Appendix D: TSA’s Interim Screener Recurrent Training Program Guidance ........................110
	

Appendix E: Major Contributors to This Report .....................................................................115
	

Appendix F: Report Distribution.............................................................................................116
	

Figures 

Figure 1: Summary of Basic Screener Training Observations ..............................................8
	

Figure 2: Checked Baggage Screener Basic Curriculum Activities Conducted..................33
	

Figure 3: Required Cross-Training Hours for EDS Co-Located with ETD ........................45
	

Figure 4: Screener Recertification Performance..................................................................60

Figure 5: OLC Use ..............................................................................................................63

Figure 6: Security Screening Equipment Models Currently in Use ....................................66
	

Figure 7: Model-specific Screener ETD Training ...............................................................68
	

Figure 8: On-Duty Screener EDS Certifications .................................................................69
	

Page 2 An Evaluation of TSA’s Screener Training and Methods of Testing 
Contents
	

Abbreviations 

AMTAP Automated Multi-Level Training Assessment Program 

ATSA Aviation and Transportation Security Act 

CBT Computer-Based Training 

DFS Dual Function Screener 

DHS Department of Homeland Security 

DOT Department of Transportation 

EDS Explosives Detection System 

ETD Explosives Trace Detection 

EQA Office of Evaluation and Quality Assurance 

FAA Federal Aviation Administration 

FSD Federal Security Director 

FY Fiscal Year 

GAO Government Accountability Office 

IED Improvised Explosive Device 

IIT Image Interpretation Test 

IMT Image Mastery Test 

IPR Image Proficiency Review 

LEO Law Enforcement Officer 

MBS-2 Modular Bomb Set, Version II 

OIAPR Office of Internal Affairs and Program Review 

OIG Office of Inspector General 

OJT On-The-Job Training 

OLC Online Learning Center 

SOP Standard operating procedure 

SPR Standard Proficiency Review 

TAI TSA approved instructor 

TIP Threat Image Projection 

TRX TIP-Ready X-ray Machine 

TSA Transportation Security Administration 

TSL Transportation Security Lab 

WPT Office of Workforce Performance and Training 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 3 
Contents
	

Page 4 An Evaluation of TSA’s Screener Training and Methods of Testing 
 

OIG
	 Department of Homeland Security 

Office of Inspector General 

Introduction 

In response to the September 11, 2001, terrorist attacks in the United States, 

Congress enacted the Aviation and Transportation Security Act, Public Law No. 

107-71 (ATSA), which established the Transportation Security Administration 

(TSA). On February 17, 2002, TSA officially assumed responsibility for civil 

aviation security functions and began hiring and training federal employees to 

perform passenger checkpoint and checked baggage screening functions. ATSA 

requires a basic screener training program that consists of 40 hours of classroom 

instruction, 60 hours of on-the-job training (OJT), and an OJT examination 

before screeners may perform duties independently. After meeting these initial 

requirements, screeners must also receive recurrent training, operational testing, 

and an annual proficiency review, which includes TSA’s annual screener 

recertification testing. 

TSA developed two basic training courses to address the needs of passenger and 

checked baggage screeners. Although both types of screeners are taught some 

shared knowledge and skills, passenger checkpoint screeners develop additional 

skills in X-ray operation, screening of persons, and searching accessible property, 

while checked baggage screeners develop additional skills in searching checked 

baggage and operating certain machines that detect explosives. TSA’s early 

versions of the basic training courses received criticism from the Department 

of Transportation and Department of Homeland Security Offices of Inspector 

General for the quality of OJT and checked baggage screener testing. In June 

2003, TSA’s Office of Workforce Performance and Training announced plans for 

course revisions. We initiated this review to determine whether TSA updated the 

courses and is meeting the training requirements set by ATSA. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 5 
Results in Brief
	

Since the initial deployment of federal screeners in 2002, TSA’s basic screener 

training program has been improved through multiple revisions and amendments. 

TSA’s December 2003 revisions to the passenger and checked baggage screener 

basic training aligned the course materials with TSA’s latest standard operating 

procedures, presented detailed and technically accurate information, and 

addressed most topics in sufficient depth without devoting excess class time to 

extraneous matter. These revisions, however, were not supported by a systematic 

or comprehensive instructional systems design process and, as a result, were 

incomplete. 

For classroom courses, passenger checkpoint screeners received adequate 

instruction, practice, and testing to demonstrate that they learned how to perform 

screening functions described in the standard operating procedures, such as 

screening passengers with the hand-held metal detector. The same was not 

true, however, of checked baggage screeners, who received far less hands-on 

practice, partly because of insufficient access to practice equipment. In addition, 

some checked baggage screeners received training on a model of Explosives 

Trace Detection (ETD) machine different from the one on which they would be 

working. Furthermore, neither passenger nor checked baggage screeners received 

instruction, practice, or testing for some skills necessary to their functions, such 

as safety skills to handle deadly or dangerous weapons and objects. Although 

TSA planned for screeners to acquire additional skills during OJT, guidelines for 

both types of screeners did not include materials to support training on deferred 

content. Classroom and OJT could have benefited from more thorough advance 

planning and analysis to select course content and frame the curriculum. 

Test design and administration for the classroom and OJT require further revision. 

Since we faulted the checked baggage classroom training quizzes and written 

examinations for a lack of rigor in June 2003, TSA revised and eliminated 

repetitive and simplistic test questions. However, TSA deployed new checked 

baggage examinations in December 2003 without pilot-testing and validating the 

passing score, resulting in a spike in examination failures, which TSA redressed 

with further test revision and by providing the option of re-training to terminated 

screeners. For both the passenger checkpoint and checked baggage practical 

demonstration examinations, TSA did not standardize the instructor delivery, test 

Page 6 An Evaluation of TSA’s Screener Training and Methods of Testing 
scenarios, scoring of performance steps, and test equipment or props available. 

Variation in administration of the practical demonstration examinations, which 

occurred in both classroom and OJT, resulted in the deployment of screeners 

with different levels of course material mastery. We understand TSA’s interest in 

expediting course and test design, but view the short-cuts as significant flaws that 

have weakened the training program. The following table summarizes our major 

observations of the December 2003 passenger checkpoint and checked baggage 

basic training courses that are discussed in this report: 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 7 
Figure 1. Summary of Basic Screener Training Observations
	

Observations 

As of December 1, 2003 

Passenger Checked 

Baggage 

Page 

References 

C 

L 

A 

S 

S 

R 

O 

O 

M 

Analysis Framed Course Design no no 14 

Necessary Skills Trained and 

Tested no no 17, 25 

Students Trained on Equipment 

Models in Use at Their Airports N/A1 no 66 

Curriculum Materials 

Technically Accurate and 

Detailed 

yes yes 16 

Lesson Objectives Conveyed to 

Students yes no 20 

Ample Hands-on Practice yes no 32 

Sufficient Training Materials 

and Equipment Available no no 25, 32 

Written Exams Pilot-Tested and 

Adequately Validated no no 21, 37 

Standardized Administration of 

Practical Demonstration Exams no no 24, 34, 37 

Sufficient Steps Taken to 

Prevent Exam Compromise no no 23 

O 

N 

T 

H 

E 

J 

O 

B 

Analysis Framed Course Design no no 41 

Curriculum Materials 

Adequately Detailed no no 43 

Necessary Skills Trained and 

Tested no no 41, 46 

Monitors Certified to Train 

Screeners no no 47 

Standardized Administration of 

Practical Demonstration Exams no no 48 

1 The December 1, 2003, passenger basic curriculum did not provide model-specific training for the equipment that passenger screeners 

use, including the Explosives Trace Detection machines. Checked Baggage screeners receive model-specific training for the Explosives 

Trace Detection machines and Explosives Detection Systems. 

Page 8 An Evaluation of TSA’s Screener Training and Methods of Testing 
In addition to the basic screener training programs, we also reviewed TSA’s 

recurrent training, annual screener recertification testing, and issues related to 

increasing screeners’ performance in detecting prohibited items. TSA made 

significant improvements to the recurrent training program, setting a screener 

training requirement of three hours per week and distributing an array of 

training materials and tools to airports. Also, TSA recently completed its first 

annual screener recertification testing. However, both programs require further 

development. For example, the Online Learning Center, TSA’s intranet-based 

tool for tracking and disseminating recurrent training, is only partially available 

to the majority of airports because TSA has not yet established sufficient network 

connectivity for computer training labs. TSA is planning to address other 

identified needs, such as fully testing cross-trained screeners2 in both passenger 

checkpoint and checked baggage screening skills during the next annual screener 

recertification. 

Because screener performance in detecting prohibited items is not a reflection of 

training alone, TSA should also improve supervisory reinforcement of screener 

training and account for continually changing training requirements. Most 

importantly, TSA should accelerate efforts to develop and deploy advanced 

screener technologies because their application holds the greatest long-term 

potential for reducing airport security system vulnerabilities and increasing the 

detection of prohibited items. 

An April 2004 revision to the basic screener training program, the Dual Function 

Screener Course, begins to address some issues raised in our review. In addition, 

TSA’s Office of Workforce Performance and Training has developed internal 

plans and analyses to help guide improvements to the screener training program. 

This report contains 22 recommendations to the TSA Administrator to improve 

security screener training and methods of testing (see Appendix C). 

Background 

In November 2001, the President signed the Aviation and Transportation Security 

Act (ATSA), which created TSA within the Department of Transportation (DOT). 

2

 Cross-training certifies baggage screeners on passenger screener operations or passenger screeners on baggage screener operations. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 9 
ATSA mandated that TSA deploy federal personnel to oversee the screening 

of all commercial airline passengers by November 19, 2002, and checked 

baggage by December 31, 2002. TSA hired, trained, and deployed over 55,000 

federal passenger and checked baggage screeners by the mandated deadlines 

to commercial airports across the United States.3 On March 1, 2003, TSA was 

integrated into the Department of Homeland Security (DHS) under the Border and 

Transportation Security Directorate. 

Before September 11, 2001, screeners operating under the Federal Aviation 

Administration’s (FAA) regulatory authority were required to undergo 12 hours 

of classroom instruction and 40 hours of OJT. In contrast, ATSA required that 

within 60 days of its enactment TSA develop a basic screener training program 

with a minimum of 40 hours classroom instruction, 60 hours of OJT, and an 

OJT examination. In response, TSA developed separate basic training programs 

for passenger checkpoint and checked baggage screeners that met the ATSA 

requirements. During the passenger checkpoint basic training program, TSA 

trained passenger checkpoint screeners to screen persons and their accessible 

property by operating walk-through metal detectors, X-ray machines, hand-held 

metal detectors, and ETD machines and by performing pat-downs of persons and 

physical searches of bags. During the checked baggage basic training program, 

TSA trained checked baggage screeners to screen checked baggage by operating 

Explosives Detection System (EDS) and ETD machines and by performing 

physical searches of bags. 

TSA conducts classroom training at airports where a large number of newly hired 

screeners await training. Newly hired screeners located at airports where training 

is not taking place are scheduled and deployed to airports where classroom 

training is conducted. The number of courses offered depends upon the number of 

newly hired screeners in need of training.4 

3

 Five commercial airports (San Francisco; Rochester, N.Y.; Tupelo, Miss.; Jackson, Wyo.; and Kansas City, Mo.) participate in a pilot 

program that allows them to employ screeners privately. These non-federal screeners must meet TSA training standards, and they attend the 

same basic training courses and participate in recurrent training and annual recertification. 4

 In December 2003, the typical class size for the Checked Baggage Screener Basic Course and the Passenger Screener Combo Basic 

Training Course was 15 and 24 students, respectively. 

Page 10 An Evaluation of TSA’s Screener Training and Methods of Testing 
 

To prepare federal screeners for their initial deployment, TSA contracted 

with Lockheed Martin to provide basic training and track OJT for passenger 

checkpoint screeners. Boeing Service Company received the contract to provide 

a similar training program for checked baggage screeners. Both companies 

employed several subcontractors to perform contracted tasks. In May 2003, 

Lockheed Martin was awarded the contract to provide both types of training. TSA 

was appropriated approximately $206 million for screener training and trained 

25,387 newly hired screeners in FY 2003. In FY 2004, approximately $112 

million was appropriated, and 9,579 newly hired screeners were trained as of June 

18, 2004. TSA requested a total of $145 million for training in FY 2005.5 

Since the initial deployment of TSA’s screener workforce, the screener training 

program and the performance of federal screeners have been the subject of 

numerous reports and articles. On January 26 and January 28, 2003, Newsday 

printed two articles alleging that TSA instructors of checked baggage courses 

held at LaGuardia Airport in New York provided students answers to questions 

prior to their taking the final examination. TSA’s Office of Internal Affairs and 

Program Review (OIAPR) conducted a review of these allegations. Following 

OIAPR’s report, we initiated an inspection of OIAPR’s review on June 12, 2003. 

Although OIAPR’s findings were supported by its interviews and analysis, we 

noted serious concerns regarding TSA’s testing plan. Specifically, TSA instituted a 

training and testing method that prepared each student to answer a specific set of 

questions upon examination. By doing this, TSA lost the opportunity to evaluate 

the student’s mastery and comprehension of the course material. In addition, we 

expressed concerns about the content and phrasing of examination questions that 

provided obvious clues to correct answers.6 

Also on June 12, 2003, TSA’s Workforce Performance and Training Division 

(WPT) issued an internal memorandum that provided information on actions 

planned by TSA to update, modify, and improve the training of checked baggage 

screeners. These actions included (1) a complete review of the program and 

the refinement of areas needing improvement; (2) a revamping of all quizzes 

5

 Data provided by the Transportation Security Administration, June 18, 2004.
	 6 Department of Homeland Security, Office of Inspector General, Office of Inspections, Evaluations, and Special Reviews, “Transportation 

Security Administration’s Checked Baggage Screener Training and Certification: A Letter Report,” Report #ISP-02-03, August 2003.
	

An Evaluation of TSA’s Screener Training and Methods of Testing Page 11 
and formal test material to ensure there is no duplication of questions; and (3) a 

review of the updated testing materials to ensure test validity. 

In our initial inspection, we committed to a follow-up review of TSA’s training 

and testing methods to evaluate the effectiveness of the changes TSA announced it 

would make. In response to growing public attention to airport security measures 

and the performance of both passenger and checked baggage screeners, we 

included the quality of both the passenger and checked baggage screener training 

programs in our review. 

When we began fieldwork for this report in December 2003, TSA continued to 

require newly hired checked baggage screeners to complete 40 hours of classroom 

instruction by attending the Checked Baggage Screener Basic Training course 

and 60 hours of OJT. However, in partial fulfillment of TSA’s June 2003 pledge, 

the December 2003 checked baggage course included revised training and 

testing materials. For passenger checkpoint screeners, TSA required successful 

completion of 47 hours of classroom instruction by attending the Passenger 

Screener Combo Basic Training course and 60 hours of OJT. Similar to the 

checked baggage course, the December 2003 passenger checkpoint course offered 

revised training and testing materials. 

On April 5, 2004, TSA introduced its new 91.5-hour Dual Function Screener 

(DFS) training course to replace the previous basic screener curriculum for 

newly hired screeners. This course, taught over nine days, builds on and 

updates the materials from the December 2003 courses. The DFS course is a 

longer, modularized course designed to equip newly hired screeners with the 

knowledge, skills, and ability to perform successfully all the duties associated 

with both passenger and checked baggage screening. Newly hired screeners 

who are expected to operate EDS machines must pass all job-knowledge testing 

associated with the DFS course and receive an additional two days of training on 

EDS machine operation. Because it was introduced near the completion of our 

fieldwork, we did not review the DFS course curriculum. TSA staff commented 

that the DFS course revisions resolve some of the issues raised in this report, 

particularly regarding test development. Where we were able to verify the 

revisions, they are incorporated in the text of the report; further TSA comments 

are in Appendices A and B. 

Page 12 An Evaluation of TSA’s Screener Training and Methods of Testing 
Purpose, Scope, and Methodology 

We reviewed TSA’s basic screener training and methods of testing to determine 

whether (1) screener training established the qualifications, knowledge, skills, 

and aptitudes necessary to detect items not authorized by federal law on aircraft, 

and (2) testing methods enabled TSA to identify students who will successfully 

perform as passenger or checked baggage screeners. In addition, we followed 

up to determine whether the TSA revamped the quizzes and final examinations 

given during classroom training, which we criticized in our first report. Finally, 

we determined whether recurrent training, weekly X-ray image interpretation 

training, screener performance improvement videos, and recertification testing 

were planned and implemented effectively by TSA. 

We analyzed ATSA and reports related to screener training written by DHS 

OIG, DOT OIG, Government Accountability Office (GAO), and TSA’s OIAPR. 

Additionally, we analyzed TSA policies, procedures, documentation, standard 

operating procedures (SOPs), and contracts relevant to passenger and checked 

baggage screener basic training. We also examined TSA and contractor passenger 

and checked baggage screener basic training curriculum materials and manuals, 

methods of testing and tools used, and related records. 

We interviewed TSA personnel at headquarters and six TSA field offices, 

including Federal Security Directors (FSDs), Deputy Federal Security Directors, 

training coordinators, scheduling officers, supervisory transportation security 

screeners, and screening managers. We also interviewed TSA contractor training 

personnel from Lockheed Martin Corporation and its subcontractors, including 

basic screener classroom training instructors. Last, we interviewed officials 

from the manufacturers of security screening equipment, including InVision, GE 

IonTrack, Smiths Detection, and Thermo Electron Corporation, and officials from 

TSA’s Transportation Security Laboratory. 

We observed three 40-hour Checked Baggage Screener Basic Courses — two 

on the operation of CTX 2500/5500 EDS and Ionscan 400B ETD machines and 

one on the operation of 3DX 6000 EDS and Ionscan 400B ETD machines. We 

also observed three 47-hour Passenger Screener Combo Basic Courses and one 

Passenger to Checked Baggage Cross-Training Course. In addition, we visited 

three field locations to observe OJT. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 13 
We patterned our review of these courses after a common instructional systems 

design model, which focuses on identifying employee performance needs, 

designing training to meet the needs, and evaluating training to ensure the needs 

have been met. The model organizes instructional design into systematic phases: 

analysis, design, development, implementation, and evaluation. In general, 

analysis involves identifying training needs and selecting training content; design 

involves outlining the course, including lesson objectives and tests; development 

involves producing course materials; implementation includes training delivery; 

and evaluation involves assessing student learning and course quality in order 

to identify improvements. Our evaluation emphasized the analysis, design, 

implementation, and evaluation phases.7 

Our fieldwork was conducted from December 2003 to May 2004. The inspection 

was conducted under the authority of the Inspector General Act of 1978, as 

amended, and according to the Quality Standards for Inspections issued by the 

President’s Council on Integrity and Efficiency. 

Findings 

Basic Screener Classroom Training 

Analysis 

Training development should begin with a formal analysis to identify employee 

training needs, a process that federal regulations require agencies to perform 

at least annually.8 While ATSA establishes minimum training needs for TSA, 

such as 40 hours of classroom instruction for new screeners, systematic and 

comprehensive analysis is fundamental to selecting the specific training content 

7 Assessing the efficiency and effectiveness of TSA’s actions during the development phase, such as contracting out the production of 

training materials, was outside the scope of this review. 

8

 5 C.F.R. 410.203; Executive Order 11348, Providing for the Further Training of Government Employees, April 20, 1967; and OPM’s 

Human Capital Assessment and Accountability Framework.
	

Page 14 An Evaluation of TSA’s Screener Training and Methods of Testing 
and aims.9 TSA has not yet completed or applied training analysis to the design 

of the basic screener training program. Several of TSA’s work products contain 

information pertinent to the analysis, including the 2002 Skills Standards for 

Transportation Security Screeners, the SOPs and Aviation Operations Directives 

that delineate screening tasks, and the 2003 Passenger Screener Performance 

Improvement Study. To date, WPT applied this guiding information to the basic 

training courses only partially. However, WPT indicated to us that it plans to 

conduct a full analysis in support of its next course revisions. 

Because TSA began hiring and training screeners within two months of its 

formation, the initial basic screener training courses were not shaped by a 

formal analysis, which would have consumed the time TSA spent designing and 

developing the courses prior to rollout. However, as TSA revised the courses 

several times over the eighteen months following their first use, TSA did not 

revisit the analysis phase and complete a full assessment of specific screener 

training needs and course aims. WPT staff said that the comprehensive March 

2002 Skills Standards for Transportation Security Screeners10 could serve as 

the foundation of analysis for the basic screener training program. However, 

WPT has not used the study to guide the revision of the basic screener training 

program. Instead, WPT primarily applied TSA’s SOPs and Aviation Operations 

Directives to determine the content of the December 2003 revisions. The SOPs 

and directives contain detailed information on how to perform TSA-approved 

screening methods. While their incorporation in the curricula was a positive 

step, they contain a less comprehensive list of screener job tasks than the Skills 

Standards. A number of job tasks listed in the Skills Standards are incompletely 

addressed in or absent from the basic training courses. Further, neither the Skills 

Standards nor the SOPs and Aviation Operation Directives set levels for screener 

9

 One interpretation of analysis defines its primary components as needs assessment, performance analysis, and task analysis. During needs 

assessment, the agency identifies what employee performance changes are needed to meet organizational goals. If changes are needed, 

performance analysis helps the agency identify and select tools and strategies for change, which may include not only training but also 

environmental and motivational options. When the agency selects a training strategy, task analysis helps the agency develop job duties into 

discrete tasks, training content, and measurable objectives. 

10 The study inventories screener responsibilities and includes a weighted, supporting list of knowledge and skills that passenger checkpoint 

screeners must show in order to succeed on the job, ranging from customer service skills to screening animal cages. The study, which has 

undergone several revisions and accounts for some checked baggage screener responsibilities, would help WPT to select and define the job 

tasks in the basic courses. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 15 
performance to help guide TSA’s expectations of screener performance during 

training and testing. 

Another study TSA did not incorporate in its analysis is the July 2003 Passenger 

Screener Performance Improvement Study, which identified causes for and 

solutions to the screener failure rate on operational tests. The study identified 

over 100 recommendations that address environmental and motivational factors, 

as well as inadequate screener knowledge and skills. Because it examined the 

performance gaps of passenger checkpoint screeners who already completed 

basic training, the training-related solutions involve recurrent training. Some 

recommended solutions, however, also have implications for basic training. For 

example, one solution involves providing screeners with guidance on optimal 

on-screen bag search techniques, something that would be useful when screeners 

are first learning X-ray image interpretation. This solution was not included in the 

December 2003 screener curriculum revision. 

WPT recognizes that completing and applying an analysis to frame the screener 

basic training courses is overdue. A December 2003 study by WPT’s Performance 

Consulting team recommended that WPT complete an analysis for the basic 

screener training program. This had not been done at the time of our evaluation. 

Since the existing work products that support the analysis focus mainly on 

passenger checkpoint screeners, it is important for WPT to extend the analysis 

to checked baggage screeners. A complete analysis will help TSA ensure that 

the basic screener training program includes the appropriate content and, by 

extension, leads to organizational improvement. 

We recommend that the TSA Administrator: 

Recommendation 1: Complete the analysis for basic screener classroom training, 

both for passenger checkpoint and checked baggage screeners, and institute 

mechanisms to ensure that staff update the analysis at least annually and apply the 

results to curriculum revisions. 

Design 

The design phase involves transforming the screener training needs identified 

in the analysis phase into course objectives, strategies, and curricula. Partly 

Page 16 An Evaluation of TSA’s Screener Training and Methods of Testing 
because TSA did not use a systematic training analysis to guide the revised 

design, the December 2003 design for the passenger checkpoint and checked 

baggage classroom courses did not fully meet basic training needs. The revisions 

improved previous basic course versions by updating the training content to 

reflect the latest version of TSA’s SOPs and Aviation Operations Directives, 

and the revised content provided technically accurate instruction for all the 

TSA-approved screening methods. However, important knowledge and skills 

outlined in the Skills Standards for Transportation Security Screeners were not 

included in the course objectives. Instructional strategies, such as the explicit 

presentation of course objectives and the effective use of training materials and 

tools, were not fully developed. Further, the test design for both the written and 

practical examinations did not comprehensively address course objectives; written 

examinations require further validation; and practical examinations require greater 

standardization. Therefore, although the December 2003 revisions improved the 

course design, TSA should continue the revision process. 

Some Standard Knowledge and Skills Identified as Important for Transportation 

Security Screeners Were Missing from the Course Curriculum 

Neither the passenger checkpoint nor checked baggage curriculum covered 

the full scope of the occupational knowledge and technical skills necessary to 

effectively perform screening functions. In addition, certain lessons in each did 

not engage subject matter in appropriate depth and were, therefore, incomplete. 

TSA identified occupational knowledge and technical skills that screeners should 

develop in the Skills Standards for Transportation Security Screeners and the 

screening SOPs. There is no requirement to teach all of these skills, e.g., heart 

attack recognition, in basic training. However, the basic courses should prepare 

screeners to perform routine duties. For the most part, the classroom courses 

address TSA-approved screening functions, background knowledge about TSA, 

and customer service skills. The courses do not address all of the skills that the 

Skills Standards and SOPs suggest new screeners need, such as: 

• Reading airline tickets and baggage tags when screening selectees who 

require special attention; 

• Handling deadly or dangerous weapons and objects; 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 17 
• Recognizing acceptable forms of identification for travelers claiming a 

right to bring weapons on aircraft, such as Law Enforcement Officers 

(LEOs); and 

• Repacking baggage after searches. 

While some of these skills could be taught during OJT, there is no requirement 

that they be covered then. 

Both the passenger checkpoint and checked baggage SOPs identify the ability to 

recognize artfully concealed threats as a requirement for the successful execution 

of all screening functions. However, while the lecture presentations for all of the 

passenger checkpoint and checked baggage screening functions contained some 

discussion of artful concealment, the curricula required very little practice and 

testing. Only the course training on X-ray operation required all screeners to 

receive practice and skills testing in the identification of artfully concealed threats, 

such as a Lockerbie-type radio bomb. For other passenger checkpoint screening 

functions, such as physical search of property and hand-wanding, most practice 

scenarios did not involve artfully concealed threats, and thus only a few screeners 

received hands-on practice identifying them. The checked baggage curriculum 

required no practice or skills testing of screeners in identifying artfully concealed 

items. 

The checked baggage curriculum omitted other knowledge and skills critical 

to screeners. Although checked baggage screeners learn during classroom 

training to interpret EDS images to recognize common objects, such as food 

items and clothing, the lesson does not train screeners to recognize threat items, 

such as Improvised Explosive Devices (IEDs). Unlike x-ray machines, EDS 

machines identify threat items for additional screening, without screener image 

interpretation. However, screeners who can interpret EDS images to identify IEDs 

can take the safety precaution of holding threats in the explosive-resistant EDS 

tunnels,11 thereby avoiding triggering explosive or incendiary bag contents when 

the luggage is ejected from the EDS machine or receives additional screening. 

TSA provides practice in EDS IED recognition during on-the-job training. TSA 

11 Unlike the Passenger Checkpoint SOPs, the Checked Baggage SOPs do not explicitly require threat items to be held in the EDS tunnel 

for further examination by a supervisor and law enforcement officer. 

Page 18 An Evaluation of TSA’s Screener Training and Methods of Testing 
should take the opportunity to begin this training in the classroom because 

classroom training is structured and standardized. 

Another skill identified by TSA as critical to screeners is the handling and 

screening of animals and cages. Many airlines permit house pets to be checked as 

baggage. This subject, however, is omitted from the checked baggage curriculum. 

Because live animals cannot be screened using EDS machines, checked baggage 

screeners must know how to handle and physically search animals and their cages 

for prohibited items. 

Also absent from the passenger checkpoint curriculum, and reviewed in a cursory 

manner in the checked baggage curriculum, was discussion, demonstration, and 

practice of proper lifting techniques. During FY 2003, TSA employees’ injury 

and illness rate (19.4%) and lost time rate (8.9%) were the highest in the federal 

government.12 Although TSA may provide some instruction on proper lifting 

techniques during recurrent and on-the-job training, not all screeners received the 

training. A review of training records at one airport showed that just over half of 

active screeners had documented training in proper lifting techniques. TSA should 

provide thorough training, including practice, before screeners are required to lift 

baggage. Proper lifting skills are critical to both checked baggage and passenger 

screeners and are necessary to avoid injury and limit time away from the job. 

Some occupational knowledge and technical skills identified by TSA as important 

to passenger checkpoint and checked baggage screeners were addressed 

perfunctorily or incompletely in the curricula. While classroom training materials 

included learning objectives on these topics, they devoted little pertinent 

content. For example, while both courses had “Screener Awareness” modules 

that addressed safety-related issues, little information was provided on how to 

identify hazardous substances in bags or respond to contact with toxic items in 

bag searches. The safety information put forward in the course was also devoid of 

any discussion of pathogens like HIV and hepatitis that may be communicated to 

screeners through contact with needles or bodily fluids in or on baggage. 

In addition, ETD machine operation training in the passenger checkpoint course 

lacked sufficient depth. Unlike the basic checked baggage course, the passenger 

12 Federal Illness and Injury Statistics for FY 2003, OSHA. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 19 
checkpoint course did not include training on ETD component familiarization, 

maintenance and alarm documentation requirements, start-up and shut-down 

procedures, calibration/verification procedures, and shift maintenance. Also, 

the course abbreviated the training on ETD decontamination procedures and 

machine errors and corrective actions. While the responsibility for performing 

these important ETD operations has yet to be plainly designated by TSA, some 

of the associated tasks are related to basic machine operation and should be 

taught to rank and file screeners. Curricular shortcomings in this area could 

create confusion and lead to the improper use of ETD machines, thus increasing 

aviation security system vulnerability. At a minimum, the limited ETD training for 

passenger checkpoint screeners needlessly burdens screener supervisors with the 

tasks and creates the opportunity for screening delays. 

Checked Baggage Course Objectives Were Not Clearly Presented 

Course objectives specify what students should be able to do in order to be 

considered competent. Because they provide students the means to organize their 

efforts toward acquiring certain knowledge and skills and accomplishing desired 

tasks, course objectives should be stated clearly at the beginning of each module 

and lesson. Moreover, by referring to training curricula that clearly identify the 

results that students should attain, instructors can more easily determine whether 

students have gained the appropriate knowledge and skills. 

The coverage of identified instructional objectives was exceptionally thorough 

in the passenger checkpoint curriculum. In line with model training standards, 

the passenger checkpoint curriculum explicitly identified specific objectives for 

all 17 modules of the training program. The curriculum materials addressed each 

objective in the content for that section and presented a review of the objectives at 

the end of each module. 

The checked baggage curriculum, however, rarely made course and lesson 

expectations evident to students. Care was not taken in the design of the 

curriculum to ensure that learning objectives were presented to students at the 

beginning of the lessons. For the most part, course materials only made provision 

for the verbal presentation of lesson objectives to students, rather than, for 

example, by presentation on a slide at the beginning of a module. Of 11 modules 

and lessons presented during each checked baggage course, only one clearly 

Page 20 An Evaluation of TSA’s Screener Training and Methods of Testing 
specified the objectives to students at the beginning of the lessons. Objective 

review sections were typically included at the end of the modules and lessons. 

Testing Issues Remained
	

a. Written Examinations 

As planned in June 2003, TSA revamped all checked baggage quizzes and 

examinations to prevent duplication of questions and reviewed the updated 

materials to ensure test validity.13 By cutting and revising quizzes, TSA eliminated 

rehearsal of examination questions. Also, for the new, December 2003 written 

examinations, TSA reviewed the content for validity and eliminated simplistic 

First, each of the checked baggage course’s three 25-question, multiple-choice 

examinations was administered immediately following relevant classroom 

modules. As a result, they had limited ability to assess students’ long-term 

retention. Second, some of the test questions did not have a precise match to the 

lessons. Some “incorrect” answers were factually correct,14 and some “correct” 

answers did not reflect what instructors emphasized to screeners. Ambiguous 

questions caused screener confusion and blurred the distinction between screeners 

who retained the lesson material and those that did not. 

questions. Nevertheless, testing issues remained. 

13 For a test to be validated, analysis of related evidence and theory should support the proposed interpretation of the test scores; that is, the 

test measures what it is intended to measure.
	 14 TSA elected to use “best answer” format, in which more than one answer may be correct for some questions. Only one answer, however, 

is the best correct answer in terms of degree of correctness. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 21 
Also, several questions, which should not be used during future testing, did not 

correspond to information presented in course materials. 

Page 22 An Evaluation of TSA’s Screener Training and Methods of Testing 
Instead of a series of written examinations, the basic passenger checkpoint course 

had a single, 78-question multiple-choice test. Administered at the course’s 

conclusion, after screeners had significant time to practice and study the course 

material, this test provided a better assessment of screeners’ retention. In addition, 

passenger checkpoint test questions were generally challenging without being 

ambiguous or misleading. 

Unfortunately, the written exams for both the passenger checkpoint and checked 

baggage courses were unvaried, rendering them vulnerable to compromise. The 

questions on the exams were always presented to test takers in the same sequence 

and TSA had no other versions of the exams to issue in the event that answers 

were leaked. Staff from TSA’s Standards, Testing, and Certification division said 

that alternate exams have been prepared for current basic screener courses should 

test answers from the primary version of an exam be inappropriately disclosed. 

Finally, neither the passenger checkpoint course nor checked baggage course 

material had post-test review sessions. Instructors could have used post-test 

reviews to address misunderstandings identified by the testing. Instead, screeners 

who failed to answer questions correctly on the written examination may have 

carried their associated misperceptions to the workplace. While subsequent 

training in an OJT setting may address those misconceptions, OJT cannot be 

relied on to discover or remedy them before they have become entrenched. 

15 TSA did not pilot test the checked baggage course written examinations before their release. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 23 
b. Practical Demonstration of Proficiency Evaluations 

Following initial classroom training, students perform a practical demonstration 

of their skills and ability to perform basic screening functions. The administration 

of practical skills tests for checked baggage was inconsistent. Test procedures for 

instructor evaluations did not provide sufficient detail on assessing demonstrated 

tasks. The brevity of the test administrator guidelines fostered broad instructor 

discretion. Practical demonstration test materials included a list of tasks students 

were to demonstrate, but did not provide a sequenced list of steps that must be 

executed for a student to be credited with successfully performing a given task. 

For example, the 100% Checked Baggage practical test required instructors to 

give screeners a passing or failing mark for “General ETD Sampling Procedures/ 

Sampling Environment (tables, lifting)” and “General ETD Sampling Procedures/ 

Cross-Contamination.” With the components of these tasks undefined, instructors 

gave some screeners passing marks even though they failed to sample and 

decontaminate the screening table, a step fundamental to both tasks. Further 

undermining uniformity, the test guidelines for practical demonstrations of ETDswiping and physical inspections of luggage did not provide for standardized test 

bags and contents. 

The passenger checkpoint practical skills testing showed similar weaknesses. 

The absence of test administration instructions and detailed guidance on how 

to assess performance of screening tasks permitted a subjective evaluation. 

Important information about appropriate test scenarios and evaluative criteria was 

missing from the practical assessment framework. For the walk-through metal 

detector practical test, for example, students screened widely varying numbers of 

passengers, and whether the passengers exhibited threat behavior was arbitrary 

and inconsistent for each test. Throughout the practical exams, students were 

required to discuss rather than demonstrate alarm resolution, even when sufficient 

props were available to enable realistic demonstration. Furthermore, instructors 

were not cautioned against coaching students during the evaluation. 

Page 24 An Evaluation of TSA’s Screener Training and Methods of Testing 
Methods of Testing Failed to Measure Adequately Whether Some Objectives 

Were Met 

The curriculum for passenger checkpoint and checked baggage screeners did not 

properly assess student comprehension on some areas of knowledge and skills 

relevant to screening. 

TSA checked baggage course written examinations and practical demonstrations 

of proficiency were not structured to test trainees on all key learning objectives. 

Checked baggage screeners were not tested on their familiarity with IED 

components and configurations, knowledge of safety and maintenance reporting 

and documentation requirements, customer service skills, or general familiarity 

with airports and security systems. While much of this knowledge and many 

related skills may be addressed during OJT, guidelines do not call for testing in 

any of these areas after completion of OJT. 

Similarly, when considered as a whole, the series of examinations administered 

to screeners following the passenger checkpoint classroom training did not 

test students on several important objectives and necessary job knowledge and 

skills presented in the curriculum. For example, students’ knowledge of the 

different types of explosives, situations when full-body and limited pat-downs are 

required, screening procedures for animals, screening procedures for persons with 

prosthetics, communication with and information about people with disabilities, 

passenger rights during screening, and the legal basis for the same, were not 

formally tested. Guidelines for the OJT portion of basic screener training do not 

call for testing in any of these areas, either. 

Materials and Tools Needed to Support Training Were Inadequate 

Interesting and effective training materials and tools are necessary to supplement 

good training techniques. They reinforce learning, bring variety to the session, 

allow learners to use more than one sense to increase learning, provide take-home 

references, and encourage active participation during the instructional process. 

The resources and support materials devoted to the checked baggage course were 

inadequate. Basic learning aids such as training manuals and slide presentation 

summaries were not provided to students. Students could have benefited from 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 25 
the use of these materials. They permit students to access information at a later 

time, absorb information at their own pace, and eliminate the need to take notes 

frantically. Resource and support shortcomings also were evident in multimedia 

presentations included in the course. In one particularly notable instance, a video 

presented during the customer service portion of the course was produced for 

another audience and did not address the customer service issues associated with 

checked baggage screening. The materials for the passenger checkpoint course 

were somewhat better. Although the course did not offer a training manual for 

students, students received procedure summary sheets for each screening function 

during training. 

Students enrolled in basic screener courses were not permitted to remove 

course materials from the classroom. This practice limited student initiative to 

pursue learning independently and reduced opportunities to reinforce classroom 

instruction. Furthermore, because of scheduling constraints on studying in class 

before the examinations, students’ inability to refer to course materials outside 

of class may have adversely affected test performance. In April 2004, TSA 

introduced new training manuals and materials for the new DFS course. TSA also 

adopted a new policy permitting students enrolled in the DFS course to use these 

materials in class and for individual study after hours. 

TSA also recognized the need to improve another important training tool—image 

recognition software used during the passenger checkpoint screener basic 

training. X-ray object recognition is arguably the most challenging aspect of 

passenger screening; training in this area is essential to screener job preparedness. 

To cultivate skill in this arena, the basic passenger screener course employed 

a computer-based training (CBT) tool. The original Smart Systems CBT tool 

permitted students to choose from simulated representations of all three models 

of X-ray machines in use at commercial airport screening stations in the U.S. 

Students using Smart System would make judgments about whether X-ray images 

of bags presented on their screens contained obvious threats, possible threats, 

or no threats. Feedback on student judgments was provided at the end of an 

image review session. Unfortunately, this system had several flaws. First, threats 

and bags were often repeated, allowing students to memorize them. While this 

repetitiveness reinforced students’ image recall, it reduced the emphasis that could 

have been placed on critical review and evaluation of new images with novel 

threat presentations. Second, the software regarded certain items as threats that are 

Page 26 An Evaluation of TSA’s Screener Training and Methods of Testing 
not currently regarded as threats in the U.S. And finally, the Smart System CBT
	

simulator offered images that were of inferior quality when compared to those 

presented on actual X-ray machines. 

In response to these concerns, TSA is now using a more advanced image training 

tool, X-Ray Tutor. This application has substantially more training images 

(14.4 million) and adapts to user object recognition capabilities based on past 

performance. In addition to customized training, X-Ray Tutor provides users 

with instant feedback on decisions to clear or further examine bags. Unlike the 

former CBT feedback mechanism, which presented information on the accuracy 

of screener judgments at the end of training sessions, X-Ray Tutor software 

educates students on decision-making failures while their rationale for clearing or 

further examining a bag is still fresh in their minds. The use of X-Ray Tutor also 

represents a major improvement in CBT because it is accompanied by a formal Xray image interpretation training module, which includes a lesson on threat object 

pattern recognition, module objectives, a summary, and review test. 

For all current and future basic screener classroom curricula, we recommend that 

the TSA Administrator: 

Recommendation 2: Ensure that passenger checkpoint and checked baggage 

basic course objectives (1) address the knowledge and skills necessary for routine 

screener performance, as identified by the most current task analysis, and (2) are 

presented to students at the beginning of course lessons. 

Recommendation 3: Further revise written examinations and administration 

procedures, including the following steps: 

• Ensure all course objectives receive an appropriate assessment of student 

learning; 

• Thoroughly validate tests, including passing scores, if they are to be used 

for selection decisions; 

• Pilot-test examination questions to ensure their clarity and objectivity; 

• Schedule tests later during the course to enhance their ability to measure 

students’ long-term retention of course material; and 

• Incorporate post-test reviews to prevent screener misconceptions from 

being carried into the workplace. 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 27 
Recommendation 4: Develop and distribute detailed test administration 

guidelines for Practical Demonstration of Proficiency Evaluations with the aim of 

increasing standardization. 

Recommendation 5: Distribute effective training materials to reinforce learning 

and allow for individual study outside the classroom. 

Implementation 

The implementation phase of training development focuses on the delivery of a 

training program. Improvements that TSA should make to the implementation of 

the basic screener classroom training include: reinstating the OJT requirement 

for newly hired instructors, requiring instructors to undergo annual recertification 

testing, analyzing the cost and feasibility of establishing designated training sites, 

increasing the use of instructional methods other than lecture, ceasing coaching of 

students during practical skills testing, and ensuring leadership within the screener 

workforce diligently monitor screeners. 

Instructor Requirements Do Not Mandate Additional Practical Experience or 

Annual Recertification 

No person is authorized to instruct basic screener training, excluding OJT, 

unless that person is an approved instructor who has been certified by TSA. 

Currently, Lockheed Martin provides TSA with contracted instructors for both 

passenger checkpoint and checked baggage classroom training. In order to teach 

basic screener courses, contracted instructors are required to meet the following 

qualifications: 

• Be a graduate of a formalized instructor-training program, such as 

American Society for Training and Development or military/federal 

instructor training; 

• Prior to instructing, attend a basic passenger or checked baggage 

classroom training course and pass any courses they will be teaching; 

• Observe and assist a qualified instructor teaching the same course they 

will be teaching; and 

Page 28		 An Evaluation of TSA’s Screener Training and Methods of Testing 
• Successfully demonstrate the required qualifications with a one-hour 

“practice teach” for a panel of TSA or contractor master instructors and a 

quality assurance representative. 

In May 2003, TSA added another requirement. After attending classroom training, 

newly hired contracted instructors were required to observe live screening 

operations for the course they instruct—three days of passenger OJT or two 

days of checked baggage OJT. In December 2003, however, TSA cancelled 

this instructor requirement due to the salary and travel costs. In addition, some 

instructors complained that the instructor OJT requirement had limited value 

because, as contractors, they were restricted to passive observance of the 

passenger checkpoints and checked baggage stations, rather than performance of 

screening duties. 

Despite the cost and practical limitations of this OJT requirement, it is an 

important step in ensuring that instructors have the experience necessary to train 

newly hired screeners on screening equipment and procedures. The observations 

included in this modified OJT allow instructors to learn how the security 

screening process operates under real airport conditions, with local variations, 

and to learn from screeners, lead screeners, supervisory screeners, and screening 

managers about their experiences on the job. TSA said that it has plans to 

develop a new OJT program for new instructors that is “more meaningful,” but 

still includes observation of screening operations and dedicated time with lead 

and supervisory screeners. The additional experience gained during OJT would 

better equip instructors with the knowledge and skills to lecture confidently and 

successfully on screening procedures, to demonstrate screening skills, and to 

answer questions posed by students during classroom training. 

Instructors are not currently required to undergo annual recertification testing. By 

May 1, 2004, 170 contract instructors completed training on the new DFS course. 

Each of these instructors was recertified after passing three written tests with a 

score of 70% or better and demonstrating multiple practical skills. Instructors 

who did not attend the DFS training, however, have not been recertified. While 

the instructors teaching the courses appeared competent, all instructors, like 

security screeners, should undergo an annual recertification to determine their 

mastery of the changing screening requirements and ability to continue to 

perform successfully as a security screening instructor. Although TSA’s Office 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 29 
of Evaluation and Quality Assurance (EQA) conducts quality assurance checks 

by evaluating instructors, debriefing them, and removing any instructor deemed 

incompetent, the staff dedicated to conducting these checks is small; therefore, the 

number of quality assurance site visits and instructor evaluations is very limited. 

Staffing within TSA’s EQA division permits visits to only five percent of training 

sites on a monthly basis. Annual recertification of instructors could serve as an 

additional monitoring and quality control element in verifying current competence 

of instructors and ensuring that each has maintained the necessary knowledge and 

skills. It also allows instructors to keep up with the changes and developments 

made to security screening. 

In June 2003, an Aviation Operations directive was issued to FSDs giving 

guidance on how to nominate staff to attend training to become voluntary TSA 

approved instructors (TAI). In addition to the contract instructors who are TAIs, 

TSA employees, including screeners, may become TAIs as an additional duty. 

TSA intends to have these TAIs train fellow staff on-site in order to reduce the 

need for contract instructors to travel to specific field locations when additional 

instruction is needed or new training requirements arise.16 Thus far, TSA has 

certified approximately 700 employees as TAIs. While TSA-employee TAIs were 

initially used for recurrent and some cross-training, TSA began to permit them to 

train new hires on April 12, 2004. As of May 30, 2004, FSDs had the option of 

requesting full support, partial support, or no support from contracted instructors, 

and could do so on a class by class basis. Although TSA still permits the use of 

contracted instructors, TSA expects that the use of TSA-employee TAIs to train 

newly hired screeners will increase over time. 

Using TSA employees as TAIs not only will minimize dependency on contracted 

instructors and increase the flexibility and control that each FSD has over 

training of their local screeners, but also it will leverage local experience. TAIs 

who are drawn from the screener workforce have direct experience working 

within aviation security, are familiar with screening procedures and equipment, 

are required to undergo an annual recertification test, and are required to take 

16 In order to become a TAI, an individual must have a current or prior instructor certification by a recognized training and development 

organization, or have had at least two years of experience as an instructor. In addition, TAIs must have successfully completed the course 

of instruction they will be teaching and demonstrate instructional skills by assisting a TAI in classroom instruction and monitoring actual 

classroom instruction. 

Page 30 An Evaluation of TSA’s Screener Training and Methods of Testing 
training on a continual basis. As a result, TAIs will be able to bring their own 

experiences into the classroom and address situations unique to their local 

airports. While TSA aims eventually to use TSA-employee TAIs to conduct new 

hire and cross-training on an exclusive basis, this most likely will not occur in 

the near future. All instructors, contracted or employed by TSA, should receive 

additional experience through OJT and be required to demonstrate their continued 

competence as a security screener instructor through annual recertification testing. 

Limited Access to Screening Equipment and Training Aids Adversely Impacted 

Training 

TSA currently conducts classroom training at airports where a large number of 

newly hired screeners await training. Newly hired screeners located at airports 

where training is not taking place travel to locations where classroom training is 

being conducted. Classroom training is delivered at a variety of different locales, 

including local airports, TSA office buildings, and hotels within the vicinity of 

the airports. Many of these training settings, however, are not dedicated training 

spaces. As a result, access to screening equipment for demonstration and practice 

is extremely limited at best, and sometimes not available at all. 

Screening equipment is furnished for classroom instruction by TSA at local 

airports or is shipped to the training sites by Lockheed Martin. Both of these 

options pose difficulties, however, as screening equipment at airports is often 

in use and unavailable, and shipping equipment to training sites is difficult 

logistically. Due to size, it is not possible to ship or move some screening 

equipment, such as EDS machines, to a training facility. While TSA has some 

EDS machine simulators located at some training locations, it does not have 

enough simulators to equip all training locations. 

In addition to screening equipment, training aids were not always available at the 

training locations. These objects include props such as luggage, prohibited items 

such as knives or scissors, permitted items that may conceal a prohibited item, 

such as a camera or book, and clothing. Training aids, critical to hands-on practice 

and role-playing, were often drawn from the instructors’ personal items and were 

too few to provide all students with realistic practice opportunities. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 31 
Easier access to actual screening equipment, simulators, and training aids 

would ensure that instructors have the necessary tools to demonstrate screening 

procedures and students have the opportunity to practice new skills. Through 

demonstration and hands-on practice, students better comprehend, master, and 

retain the necessary knowledge and skills required to perform successfully as a 

security screener. 

Because it is logistically difficult to make screening equipment, simulators, and 

training aids available at each training location, TSA should analyze the cost and 

feasibility of establishing fewer designated training locations or regional training 

centers at which these items are permanently located and readily available. 

Ideally, these designated training locations would have multiple mock passenger 

checkpoints and checked baggage screening stations to facilitate role-playing and 

mock realistic situations. Designated training locations would contribute to the 

standardization of screener training. 

Delivery of Checked Baggage Screener Basic Training Focused Too Heavily on 

Lecture 

In the delivery of courses, instructors primarily lectured to convey the training 

content to newly hired screeners. This was especially true for the checked 

baggage courses. During these courses, lecture was used for an estimated 95% 

of the 40 hours of training, and instructors did not use the majority of practice 

activities suggested in the curriculum. 

While lecture is an instructional method by which training is commonly delivered, 

it can be among the least effective of methods. Research on students exposed to 

lectures indicates that they were attentive and readily assimilated lecture material 

during the first 10-20 minutes, but that attention dropped dramatically thereafter.17 

In order to enhance student attention and retention, lectures should be punctuated 

with periodic activities. 

In addition to maintaining attention, the delivery methods of demonstration, 

hands-on practice, and role-playing can be much more effective in screeners’ 

17 Johnston, Susan and Cooper, Jim, “Quick-thinks: Active-thinking Tasks in Lecture Classes and Televised Instruction,” Cooperative 

Learning and Colleges Teaching newsletter, Vol. 8, no. 1, Fall 1997. 

Page 32 An Evaluation of TSA’s Screener Training and Methods of Testing 
comprehension and retention of newly learned knowledge and skills. It is common 

knowledge that some adults are visual learners, some are auditory learners, and 

some are tactile learners, while others learn best through a combination of these. 

Demonstration, hands-on practice, and role-playing engage several senses: sight, 

hearing, and touch. By using alternative delivery methods in addition to lecture, 

each student’s favored learning medium can be used during the course. Finally, 

students are often receptive to ideas from lecture when they are reintroduced 

during demonstration and practical exercises. 

During the checked baggage courses, different activities that could be performed 

with students were presented in the instructor’s manuals. Very few of these 

activities were conducted. Although some of these activities did not provide 

screeners with an opportunity to practice real on-the-job skills, others were 

of high quality and provided an opportunity for skills practice, such as ETD 

decontamination procedures. Of 46 different activities provided to instructors in 

two checked baggage courses, seven and four were fully conducted, respectively. 

Figure 2. Checked Baggage Screener Basic Curriculum Activities Conducted 

Lesson Title Course 1 Activities 

Provided/Conducted 

Course 2 Activities 

Provided/Conducted 

TSA Overview 5/3 5/0 

Customer Service 5/0 5/0 

Persons With Disabilities 5/2 5/1 

Screener Awareness 2/0 2/0 

Improvised Explosive Devices 0/0 0/0 

EDS Overview 0/0 0/0 

EDS Common Object Recognition 0/0 1/0 

EDS SP Operations 1/1 0/0 

EDS Operational Procedures 1/0 1/0 

Barringer IONSCAN 400B 15/0 15/1 

100% Checked Baggage 

Total 

12/1 

46/7 (15.2%) 

12/2 

46/4 (8.6%) 

The small amount of demonstration and hands-on practice in the checked baggage 

courses was in large part due to limited access to checked baggage screening 

equipment. Other activities that did not require screening equipment, however, 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 33 
were also not conducted. For example, activities that involved demonstrating 

correct methods of sampling for traces of explosives or completing maintenance 

forms and logbooks were not conducted. Instructors also did not use activities 

identified in the curriculum to practice proper customer service behavior. 

While lecture was still the primary method of delivery during the passenger 

checkpoint course, alternate methods of delivery were used more frequently than 

in the checked baggage screener basic classroom courses. Passenger checkpoint 

course instructors performed many demonstrations and conducted multiple 

activities in which all students participated.18 Passenger checkpoint students 

even had the opportunity to visit a closed passenger screening checkpoint at an 

airport prior to written and practical skill examinations. Instructors also chose to 

complete additional activities that provided opportunities to model or practice 

proper customer service behavior in situations pertinent to passenger checkpoint 

screening. These demonstrations and activities in the passenger checkpoint 

course not only helped hold students’ attention and aided in their comprehension 

of the new knowledge and skills presented to them, but also gave the students 

confidence that they could perform successfully as screeners. 

Although lecture is the most common means of delivering large quantities of 

information, a variety of delivery methods should be used to ensure that all 

students have full opportunity to comprehend, master, and retain the knowledge 

and skills necessary to perform successfully as a screener. While OJT provides 

a significant amount of demonstration and hands-on practice, demonstration and 

hands-on practice within the classroom are needed to prepare student for the 

practical examinations given during classroom training. Moreover, practice in 

the risk-free, controlled classroom environment should help screeners make the 

transition from classroom training to OJT, where screeners experience real-world 

job conditions for the first time. 

Administration of Practical Skills Testing Was Not Standardized 

For both the passenger checkpoint and checked baggage courses, instructors 

clearly communicated written test procedures by reading from a TSA-approved 

18 The greater number of demonstration and activities performed during the passenger checkpoint training was in part due to easier access 

to real or mock passenger checkpoint screening equipment. 

Page 34 An Evaluation of TSA’s Screener Training and Methods of Testing 
test administration script. There was, however, no standardization in the 

administration of the practical skills tests, otherwise known as the Practical 

Demonstration of Proficiency Evaluations. 

Proper assessment of individual skills in performing screening tasks requires 

the independent exercise of these tasks without assistance. However, instructors 

coached students through the practical skills tests. The level to which students 

were coached varied depending on the instructor and the student undergoing the 

test. Practical Demonstration of Proficiency Evaluation Test Procedures given to 

instructors, which provide a brief overview and general guidance in conducting 

practical skills tests, did not address whether instructors were permitted to 

provide coaching. However, according to Practical Demonstration of Proficiency 

Evaluation Instructions that are given to students, if a student asks the instructor a 

question during the evaluation, the instructor may only respond by informing the 

student to “act in accordance with your training.” 

While the associated scoring was pass/fail, coaching during practical tests may 

have offered an unfair advantage to some students. Also, coaching prevented 

some students from fully demonstrating on their own that they have the skills 

to perform successfully as a screener. It is imperative that students demonstrate 

their abilities within a controlled environment before proceeding to OJT, where 

they will be conducting actual screening in a live environment with multiple 

distractions. As a result, all coaching during the Practical Demonstration of 

Proficiency Evaluations should cease. 

In addition, suitable testing environments were not established for the 

administration of practical skills evaluations and contributed to the lack of 

standardization. Testing took place in one room where students could watch other 

students take the skills test before them. This enabled those students waiting to 

take the test to memorize how others demonstrated their skills using precisely 

the same scenarios they would encounter. By placing students in an environment 

where they can memorize and repeat what others have done before them, TSA lost 

the opportunity to challenge students and assess whether they can successfully 

perform screening functions independently. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 35 
Security Supervision Does Not Reinforce Basic Screener Training 

No matter how successful training is in establishing the necessary basic 

knowledge and skills to perform screening functions, it is crucial that this 

knowledge and these skills are continually reinforced. A screener’s work is 

often monotonous and requires repeated application of careful methods and 

strict procedures. The repetitiveness of screener tasks can have a physical toll 

on screeners that may result in short-cutting procedures or applying improper 

techniques. To avoid this, lead screeners, supervisory screeners, and screening 

managers must reinforce basic knowledge and skills learned in basic screener 

training by diligently observing screeners and immediately correcting negligent 

screening techniques and failure to adhere to SOPs. 

During some of our site visits, we observed poor supervision at passenger 

checkpoint and checked baggage screening stations. Lead screeners, supervisory 

screeners, and screening managers were not diligent in correcting screeners’ 

lack of adherence to the SOPs and sloppy screening techniques. For example, 

when using ETD machines, screeners are trained on the use of specific sampling 

techniques for detecting trace explosives that include sampling in one direction. 

However, several screeners on the job used back-and-forth, circular, or zigzag 

sampling techniques. Also, several screeners positioned hand-held metal detectors 

at distances further from the body of a person than what has been prescribed 

and taught during basic training. These screening techniques can be corrected 

when pointed out to a screener. In addition, if screeners know that they are under 

constant supervisory observation, the screeners will be motivated to be more 

diligent and use precise and proper screening techniques on their own. 

Because screeners may not retain all of the procedures and proper techniques 

learned during basic screener training and their attentiveness may decrease over 

time, it is imperative that supervisors and managers within the screener workforce 

diligently observe screeners, correct identified deficiencies, and promote a 

positive team environment where every screener is comfortable speaking out 

when they observe peers performing incorrectly. 

For all current and future basic screener classroom curricula, we recommend that 

the TSA Administrator: 

Page 36 An Evaluation of TSA’s Screener Training and Methods of Testing 
Recommendation 6: Require all TSA approved instructors to complete an 

appropriate form of OJT prior to providing instruction and to undergo annual 

recertification testing. 

Recommendation 7: Analyze the cost and feasibility of establishing designated 

training sites where screening equipment, simulators, and training aids can be 

permanently located and readily available to students. 

Recommendation 8: Use alternatives to lectures more frequently during 

classroom training to help maintain student attentiveness, contribute to the 

comprehension and mastery of new knowledge and skills, and foster retention of 

the material. 

Recommendation 9: Cease all coaching of students during practical 

demonstration of proficiency evaluations. 

Recommendation 10: Ensure that leadership within the screener workforce 

diligently monitors screeners and immediately corrects identified failures to 

adhere to screening procedures and negligent screening techniques. 

Evaluation 

The evaluation phase helps ensure that training is relevant, effective, and achieves 

training goals. While there are many types of evaluation to assess a training 

program’s effectiveness, most models reframe Donald Kirkpatrick’s four levels 

of evaluation: (1) student reaction to the course; (2) student learning during the 

course; (3) student application of learning on the job; and (4) organizational 

performance improvements from the training.19 WPT formally collected data on 

student reactions and learning during the course, but WPT was still in the process 

of identifying valid measurements to evaluate student application of learning 

on the job and organizational improvements driven by training. WPT needs to 

improve its efforts to ensure that its measurements of student learning during the 

course are valid and comprehensive, too. 

19 Donald L. Kirkpatrick, Evaluating Training Programs, 1975. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 37 
TSA has a system in place to collect and evaluate data on student reactions for 

the basic screener classroom training. The primary benefit of collecting this data 

is that negative student reactions reveal opportunities for improving the training. 

Instructors collected from all screeners a course evaluation form that asked 36 

questions, some open-ended, covering the appropriateness of course content, 

degree of instructor knowledge, realism of practical exercises, ease of use for 

computer-based training, self-assessment of screener learning, and other factors. 

The Evaluation Branch of WPT’s EQA scans the forms into the Automated MultiLevel Training Assessment Program (AMTAP) database to collect, analyze, and 

report results for course managers, EQA staff, and training coordinators. EQA 

staff monitor student responses in order to make course adjustments, such as 

sending Quality Assurance staff into the field to review an instructor or course 

following screener complaints. 

For evaluations of student learning during the course, TSA conducts written 

and practical examinations to measure whether screeners fully meet course 

objectives. When we observed the December 2003 versions of passenger and 

checked baggage classroom training, flaws in the examinations’ design and 

implementation, particularly for the checked baggage course, limited their 

precision in showing which screeners met the course objectives and which did 

not. First, both passenger and checked baggage courses contained objectives that 

were not tested in either of the written and practical examinations. Second, for the 

checked baggage written examinations: 

• TSA did not pilot test the new examinations to identify and revise 

ambiguous test questions. 

• 

Page 38 An Evaluation of TSA’s Screener Training and Methods of Testing 

20 
• TSA administered the checked baggage examinations prior to practice 

sessions. Several screeners complained that practice prior to the 

examinations would have better developed their understanding of the 

test material, much of which addressed performance of procedures that 

screeners had not yet performed. 

Third, TSA did not standardize the instructor delivery, test scenarios, scoring 

of performance steps, and test equipment and props available for the practical 

examinations. Without a detailed, standardized process for the practical 

examinations, instructors did not ensure that screeners demonstrated similar levels 

of mastery of the course material. For some screeners but not others, instructors 

simulated threat items and scenarios, provided coaching, or talked through 

portions of the practical exam without requiring screeners to exhibit the steps. 

Given these weaknesses in the written and practical examinations, we question 

why TSA chose to use high-stakes tests without remediation in the absence 

of a requirement to do so. The Standards for Educational and Psychological 

Testing point out, “[t]he need for precision [in scoring tests] increases as the 

consequences of decisions and interpretations grow in importance,” and “[t]he 

probability of misclassification will generally be relatively high for persons 

with scores close to the cut points.” 21 TSA should consider resetting the passing 

score to retain marginal screeners for remediation. This will reduce TSA’s risk 

of terminating screeners, whose selection and training TSA has already financed, 

who may demonstrate acceptable performance with minor additional training and 

more realistic, practical examinations. After all, at the end of classroom training, 

screeners have 60 hours of OJT to continue to develop their knowledge and skills. 

TSA should take the opportunity to identify screeners’ remediation needs for 

supplemental training and testing during OJT. A new, April 2004 administration 

guide for practical examinations shows that TSA is moving in this direction. 

Since the purpose of the basic screener training program is to provide the 

knowledge and skills necessary to screener on-the-job performance, evaluation of 

how screeners apply their training on the job is another measurement WPT should 

undertake. WPT’s EQA is planning such measurements that may involve direct 

21 American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, 

Standards for Educational and Psychological Testing, 1999, p. 50. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 39 
observation of screener performance. While time-consuming and labor-intensive, 

direct observation should be informative. 

Additionally, WPT is in the process of developing valid performance 

measurements to apply in evaluations that assess whether screener training makes 

a difference in TSA’s organizational performance. In November 2003, GAO 

testified, “Our recent work on TSA’s passenger screening program showed that 

although TSA has made numerous enhancements in passenger screening, it has 

collected limited information on how effective these enhancements have been 

in improving screeners’ ability to detect threat objects.”22 The same can be said 

of checked baggage screening; the development of its training programs and 

screener performance measurements trails the passenger screening program’s. To 

date, generalizations about TSA’s organizational performance in threat detection 

have been based on OIAPR and DHS OIG operational test results, which 

involve only a small fraction of airports and screeners.23 The Passenger Screener 

Performance Index, which TSA expects to compile by the end of FY 2004, will 

use recertification examination scores and other data to track screener ability 

to detect prohibited items. In addition, EQA has discussed developing its own 

measurements. Regardless of the measurements taken, TSA will have difficulty 

isolating basic screener training as the cause of improved organizational results. 

Ongoing changes to TSA’s recurrent training program, supervisory development, 

equipment configurations, and other factors may also alter the frequency of 

detection of items that threaten flight safety. Nevertheless, it is important that TSA 

measure organizational results. An absence of improvement in organizational 

results may indicate a critical need to adjust the basic screener training program. 

We recommend that the TSA Administrator: 

Recommendation 11: Continue the development of evaluations that will relate 

training to screener application of learning on the job and to organizational 

results. 

22 Aviation Security: Efforts to Measure Effectives and Address Challenges, GAO-04-232T, November 5, 2003. 23 For further discussion of operational testing, see page 55. 

Page 40 An Evaluation of TSA’s Screener Training and Methods of Testing 
Basic Screener On-The-Job Training 

ATSA requires both newly hired passenger checkpoint and checked baggage 

screeners to complete 60 hours of OJT and an OJT examination upon completion 

of classroom training. Until screeners meet these requirements, they may not 

make independent judgments regarding whether passengers or property may 

proceed without further screening. In February 2003, DOT OIG criticized TSA’s 

OJT program for not requiring screeners “to demonstrate proficiency in all job 

related tasks.”24 In response, TSA issued the Passenger Checkpoint OJT Interim 

Guidance for Federal Transportation Security Screeners, Revision 2.0, and the 

OJT Procedures (Interim Guidance) for Federal Checked Baggage Security 

Screeners in April 2003. Both interim guidelines set requirements for screeners 

to practice and demonstrate proficiency in screening techniques. TSA also has 

assumed responsibility for administration of all OJT, which was contracted out 

during the initial federalization of screeners in 2002. 

Analysis 

TSA has not yet completed analysis to identify the training needs and objectives 

that screener OJT must address. In its current form, OJT primarily assists 

screeners to increase their proficiency in screening techniques learned in 

the classroom. However, OJT also should teach objectives that screeners do 

not master during classroom instruction, such as following airport-specific 

procedures. The interim OJT guidelines do not require screeners to meet some 

objectives missed in the classroom, and they provide little instructional support 

for OJT monitors to teach the new material. Extending analysis for basic screener 

training to OJT should help TSA identify and address these gaps in a systematic 

manner. 

In both the passenger and checked baggage classroom training, instructors and 

training materials identified skills not taught in the classroom that screeners need 

to acquire at their airports during OJT. Examples of training deferred to OJT 

include: how to work within airport-specific screening configurations, such as 

inline systems; how to use the features and ensure proper operation of specific 

24 DOT OIG SC-2003-0026, Audit of Screener Security Program, February 28, 2003, p. 2. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 41 
models of X-ray and walk-through metal detector equipment; how to gain access 

to checked baggage that is locked; how to verify firearms declaration forms in 

checked baggage; how to apply airport-specific breach procedures; how to write 

incident reports; and how to identify selectees for mandatory additional screening. 

In some cases, however, the OJT guidelines left out the requirement for screeners 

to learn the new skill. For example, after the classroom “Screener Awareness” 

training introduced passenger screeners to the concept of reportable security 

incidents, instructors were to note: “The quality of the report you write is critical 

in resolving situations. In some instances, your report may be admitted as 

evidence in a court of law. Many forms will exist at your screening locations and 

you will learn more how to fill them out during OJT.” However, report writing 

is not addressed in the passenger OJT guidelines. Likewise, checked baggage 

classroom training tells screeners to segregate the bags of selectees for the most 

thorough type of search, but neither classroom training nor OJT instructs checked 

baggage screeners on how to identify selectees. The checked baggage OJT 

guidelines do not mention selectee screening at all. 

In other cases, the OJT guidelines require practice of new skills but contain 

no instructional materials to assist OJT monitors in teaching the skills to the 

screener. For example, unlike checked baggage screeners, passenger screeners 

do not receive model-specific training on screening equipment during classroom 

training. The passenger classroom curriculum explicitly notes that screeners will 

learn how to operate model-specific features of X-ray and walk-through metal 

detector equipment during OJT, and the same is implied for the ETD. However, 

the passenger OJT guidelines do not explain which model-specific features and 

operations should be taught and how. Without such instructional support, it is 

unlikely that OJT monitors, who have limited qualifications as instructors,25 can 

provide standardized training on the new material to all screeners and ensure that 

training objectives are met. 

TSA needs to complete an analysis for basic screener training that takes into 

account the hand-off of objectives from classroom training to OJT, and the scope 

and level of proficiency required of screeners at the end of OJT. TSA should use 

the analysis to guide the design of OJT instructional materials as well. Without 

25 For further discussion of OJT monitor qualifications, see page 47. 

Page 42 An Evaluation of TSA’s Screener Training and Methods of Testing 
systematic analysis to drive curriculum revisions, gaps in basic screener training 

will persist. 

Design 

TSA should: (1) design OJT training administration guidance to provide detailed 

lists of specific job tasks and establish function-specific time requirements for 

both newly hired and cross-trained screeners; (2) enforce policy regarding the 

OJT training image recognition test; and (3) require students to be tested on all 

key screening functions. 

On-The-Job Training Guidance Needs Improvement 

Each FSD was given the April 2003 OJT guidance to define the conduct of OJT 

for newly hired passenger checkpoint and checked baggage screeners. These 

documents, however, provide significantly different guidance on the structure 

and content of OJT. The OJT guidance for passenger screening contains seven 

OJT checklists for each applicable passenger screening function and establishes 

a specific minimum amount of time to be spent on each function in order to 

complete OJT. All seven checklists cite specific tasks that must be executed in 

order for an OJT student to be credited with successfully completing screening 

functions. OJT monitors are to verify that each screener has sufficient knowledge 

of these tasks, can perform the tasks with little intervention from the monitor, and 

ultimately can perform the tasks without any intervention from the monitor. In 

contrast, the OJT guidance for checked baggage screening contains a checklist 

of more general tasks that OJT monitors are to verify a student has “completed” 

during each OJT session. The brevity and lack of detailed tasks found in the OJT 

guidelines undermines the effectiveness of checked baggage OJT. 

On the other hand, while both OJT guidance documents specifically set the 

number of hours that each screener completing OJT should spend on each 

screening function, only the checked baggage OJT guidance addresses time 

requirements for cross-trained screeners, as well as those who need “differences 

training,” i.e., training on different models of ETD machines. The passenger 

checkpoint OJT guidance document also should set time standards for the 

functional training of cross-trained screeners. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 43 
Although time requirements are established in the checked baggage OJT 

guidance, passenger screeners who have been cross-trained to screen checked 

baggage are not required to have the full 60 hours of OJT. These cross-trained 

screeners are not required to spend time training on knowledge and skills they 

have already acquired such as security and local procedures, TSA values, and 

customer service. Instead, cross-trained screeners dedicate their OJT training time 

to new knowledge and skills. A significantly lesser amount of time, however, is 

required of them than of newly hired screeners. For example, EDS operation is 

a new task for both newly hired checked baggage screeners and cross-trained 

passenger checkpoint screeners. The former are required to spend 23 hours of OJT 

working with EDS machines, while the latter spend 5.5 hours. In addition, some 

tasks that are required of newly hired checked baggage screeners are not required 

of cross-trained screeners, and vice-versa. For example, a cross-trained passenger 

checkpoint screener does not have to dedicate training time to learning how to 

recognize improvised explosive devices in EDS images. 

Page 44 An Evaluation of TSA’s Screener Training and Methods of Testing 
Figure 3. Required Cross-Training Hours for EDS Co-Located with ETD
	

OJT Task 

Newly Hired 

Checked Baggage 

Screener Time 

Requirements 

Cross-Trained 

Checked Baggage 

Screener Time 

Requirements 

Security and Local Procedures 3 hours 0 hours 

TSA Values 1 hour 0 hours 

Customer Service 1 hour 0 hours 

Persons with Disabilities 1 hour 0 hours 

Screener Awareness 1 hour 0 hours 

Checked Baggage Overview and 

Operations 

Overview of EDS Operations 

1 hour 

1 hour 

0 hours 

30 minutes 

EDS Operations 13 hours 3 hours 

EDS Improvised Explosive 

Devices 3 hours 0 hours 

Common Object Recognition/ 

Location/Bag Handling 6 hours 2 hours 

Bag Transportation and Proper 

Handling Procedures for ETD/ 

Physical Search 

Not Applicable 30 minutes 

Overview of ETD Operations 1 hour 1 hour 

ETD Operations 10 hours 0 hours 

100% Checked Baggage 

Procedures and Alarm Resolution 9 hours 1 hour 

Daily Shift 4 hours 0 hours 

Questions and Answers 

Total Time 

5 hours 

60 hours 

1 hour 

9 hours 

Because certain tasks are new to both, the time requirements that newly hired and 

cross-trained checked baggage screeners are to dedicate to each of these should be 

of equal duration. It is possible that nine hours of OJT for a cross-trained screener 

is too little. On the other hand, it is possible that 60 hours of OJT for a newly 

hired checked baggage screener is too much. TSA should conduct an evaluation 

of the OJT program to determine the most appropriate amount of time that should 

be spent on each screening function during OJT to establish skills to perform all 

job related tasks and equally apply these to both newly hired and cross-trained 

screeners. In addition, while the checked baggage OJT includes “differences 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 45 
training” to familiarize screeners with models of ETD machines not taught during 

their classroom training, the OJT does not provide differences training for EDS 

machines. Because EDS models also have operational differences and some 

airports have more than one model of EDS, TSA should address this shortcoming. 

To ensure that OJT is a prescriptive, standardized process, TSA should craft 

uniform guidance that provides detailed, sequenced lists of specific tasks that each 

OJT student must perform. Where needed, TSA should provide more detailed 

instructional materials to guide OJT monitors in providing the training. TSA 

should also establish appropriate, function-specific time requirements for both 

newly hired and cross-trained screeners. 

OJT Testing Issues 

OJT test mechanisms for both passenger and checked baggage screener skills are 

limited, and to a certain extent, they are less rigorous than the testing at the end 

of classroom training. Yet ATSA requires testing after the completion of OJT, not 

classroom training. 

Upon completion of passenger checkpoint OJT, passenger screeners are required 

to take only an image recognition test, called the Image Mastery Test (IMT), to 

become certified. TSA’s policy gives students three opportunities to pass the IMT. 

Although there is no provision in the OJT program to authorize IMT remediation 

and testing more than three times, many TSA officials in the field said that FSDs 

permitted additional opportunities to retake the test. Through February 2004, TSA 

data indicate that of those tested, 1,154 screeners, or two percent, took the IMT 

more than the permitted maximum three times. In some cases, screeners took the 

IMT as many as eight times. Because this is the last image test threshold screeners 

must pass before they are permitted to make independent judgments, TSA should 

strictly define policies on IMT retesting. 

In addition, problems remain with the IMT. The IMT software installed on 

machines around the nation regards several items as non-threats even though they 

are currently considered threats by TSA. For example, the IMT treats screwdrivers 

and scissors as non-threats even though they are not permitted onboard aircraft. 

TSA’s EQA division identified this problem during the summer of 2002, and 

consequently, the software manufacturer made corrective adjustments to the IMT. 

Page 46 An Evaluation of TSA’s Screener Training and Methods of Testing 
During a site visit in December 2003, however, we noted that the new edition of 

the IMT software had not been installed. 

Finally, while passenger checkpoint OJT testing assesses students on their ability 

to recognize threat images, it does not test skills in the performance of other jobrelated tasks such as operation of the hand-held or walk-through metal detectors. 

As discussed, the passenger checkpoint OJT interim guidance calls for monitors to 

verify that each student can perform all screening functions without intervention. 

However, the guidance does not provide for realistic, practical testing of students 

on any screening function, such as the operation of a hand-held metal detector, 

other than the monitoring of X-ray machines. Likewise, the checked baggage 

OJT guidance calls for a practical “three-bag test” during which OJT students are 

required to perform a closed bag, limited bag, and open bag search with the use of 

an ETD machine. For those students trained on the operation of an EDS machine, 

however, there is no EDS practical testing component. Both guidance documents 

should be expanded to require post-OJT testing on all TSA-approved screening 

methods that the screener will be required to use. 

Implementation 

TSA should take steps to ensure that all OJT monitors have the necessary skills 

and experience to deliver OJT and that OJT test administration is standardized. 

OJT Monitors Have Varying Qualifications to Teach Screening Skills 

TSA does not require OJT monitors for basic screener OJT to be approved or 

certified. According to passenger checkpoint OJT guidance, any screener who has 

successfully completed the appropriate basic screener training course and the 60-

hour OJT program may act as an OJT monitor. Also, according to the guidance, 

FSDs or their designees are encouraged to select OJT monitors with the maximum 

amount of experience and a demonstrated ability to mentor and counsel other 

screeners. 

Each OJT monitor should be an expert in the necessary subject knowledge and 

skills as well as have the desire to teach and the ability to relate well with the 

person they are mentoring. But most importantly, TSA should ensure that FSDs, 

or their designees, choose monitors who have the ability to transfer their subject 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 47 
knowledge and skills to students in an organized, systematic, easy to retain 

manner. To ensure that TSA has the most effective and credible OJT program, 

TSA should certify individual screeners who are specially designated to deliver 

and monitor OJT. Through certification, TSA will ensure that trainers are qualified 

and can competently transfer knowledge and skills. In addition, certification will 

ensure that trainers practice a structured training delivery method, including test 

administration, which promotes consistency. TSA should consider expanding the 

TAI program to certify employees to conduct OJT as well as classroom training. 

Administration of Checked Baggage Practical “Three Bag Test” Was Not 

Standardized 

According to the checked baggage OJT guidelines, upon completion of the 

60 hours of OJT, persons administering the practical “three bag test” must use a 

strip with a very small amount of explosives to contaminate a test item that will 

cause an ETD machine to produce an alarm. Guidelines further specify that the 

explosive material “should be placed in the same locations for each screener. 

Each bag should produce an ETD alarm. Screeners must perform each procedure 

without error and receive an alarm on all three bags to pass this test.” 

Many local TSA staff and OJT administrators, however, are not testing with 

the explosive material strip. According to these officials, its use contaminates 

an ETD machine, and it is difficult to clean from an ETD machine. Officials at 

TSA’s Transportation Security Lab (TSL), however, disagree that use of the test 

explosive material results in persistent machine contamination or alarm. Although 

more advanced testing products are currently being developed, TSL officials 

believe that, for now, the test strip with explosive material should be used in the 

field during testing. The test strip should be used, so that when an ETD machine 

produces an alarm, the OJT test administrator will know that proper sampling 

techniques are being used. 

Evaluation 

Because TSA’s EQA is short-staffed, TSA does not conduct an analysis of the 

overall effectiveness of the OJT portion of basic screener training. While OJT 

monitors are required to measure the extent to which students improve knowledge 

and increase skills as a result of the 60 hours of OJT, TSA does not measure the 

Page 48 An Evaluation of TSA’s Screener Training and Methods of Testing 
reaction and satisfaction of OJT students to the training program, the extent to 

which change in behavior has occurred as a result of OJT, nor the impact of OJT 

on TSA’s organizational goals, such as an improved detection rate of prohibited 

items. As the level of staffing increases, however, EQA plans to work with TSA’s 

Office of Aviation Operations and begin conducting a complete evaluation of the 

OJT program. 

We recommend that the TSA Administrator: 

Recommendation 12: Ensure that OJT guidance provides detailed, sequenced 

lists of specific tasks that each OJT student must perform, including instructional 

materials where needed, and establishes appropriate, function-specific time 

requirements for both newly hired and cross-trained screeners. 

Recommendation 13: Revise OJT examinations in order to: 

• Test screeners on all TSA-approved screening methods that they will be 

required to use upon completion of OJT; 

• Enforce the use of tests with explosive material during ETD practical 

examinations; and 

• Standardize and enforce limits on OJT retesting opportunities, including 

the Image Mastery Test. 

Recommendation 14: Ensure OJT monitors are certified as having the skills 

and experience necessary to deliver and monitor OJT training and administer 

subsequent testing. 

Recurrent Training and Recertification Testing 

In an October 16, 2003, statement before the Committee on Transportation and 

Infrastructure, Subcommittee on Aviation, TSA’s Administrator said TSA was 

in the process of developing a recurrent training program and that recertification 

testing for FY 2003-04 was underway. 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 49 
Recurrent Training Program Has Been Established 

ATSA specifically requires screeners to receive “training to ensure that [they] are 

proficient in using the most up-to-date technology and to ensure their proficiency 

in recognizing new threats and new weapons.”26 While TSA made important 

strides in developing recurrent training tools, TSA has opportunities to enhance 

their effectiveness. In addition, TSA should examine the workforce implications 

of the current three-hour-per-week training requirement of each screener. 

Important Strides in Developing Recurrent Training Tools Have Been Made 

On December 22, 2003, TSA’s WPT issued interim guidance to FSDs that 

mandated the institution of a standard recurrent training program for all screeners 

to ensure that they maintain and enhance their knowledge, skills, and abilities 

to effectively screen persons and their property. To comply with the interim 

guidance, all FSDs are required to develop an annual screener training plan for 

each screener under their authority. These plans are to guarantee that all screeners 

(1) receive recurrent training to effectively screen persons and their property, 

(2) meet other mandatory administrative training requirements, such as ethics 

training, and (3) further their professional development (See the interim guidance 

in Appendix D). 

According to the interim guidance, FSDs are required to develop a screener 

training plan that meets the legal and regulatory requirements as well as the 

individual performance and developmental needs of each screener. As a result, the 

substance of the recurrent training provided to screeners is to be developed both 

nationally and locally. This coupling of national and local training is designed 

to offer standardized training options and allow flexibility for FSDs to create 

their own training to develop individual competencies and to address unique 

operational needs at the airport level. For example, on a national level, TSA 

developed several recurrent training tools, including performance videos and webbased training courses on common screening procedures. On a local level, FSDs 

may elect to supplement these training tools with safety presentations developed 

in coordination with a local fire department, teamwork techniques, and workplace 

communication seminars. 

26 Codified in 49 U.S.C. §44935(h). 

Page 50 An Evaluation of TSA’s Screener Training and Methods of Testing 
 

 

 

 

 

 

 

 

 

To support the recurrent training program, TSA recently developed or made 

available a number of training tools and practices, including the following: 

• The “Excellence in Screener Performance” video series provides 

instruction on physical bag searches, hand-held metal detector and pat 

down searches, X-ray operation, and screening persons with disabilities. 

• Eight computer-based training modules reinforce topics introduced in 

the “Excellence in Screener Performance” videos and introduce new 

material, including prohibited and dual-use items,27 operating EDS 

machines, and technical training for supervisors. 

•		 X-Ray Tutor, one of of the eight computer-based training modules, 

enables screeners to practice X-ray image interpretation with sample 

images adapted to individual performance ability and training needs. 

TSA recommends an average of one hour per week of practice on image 

interpretation. 

•		 “Threat in the Spotlight” presentations describe and show pictures, in 

some cases X-ray images, of threats recently found by screeners or from 

other operational and intelligence resources. 

• The Threat Image Projection System trains and tests screeners on threat 

image recognition using X-ray machines and EDS equipment. This 

learning opportunity is enhanced when screeners are provided timely 

coaching and feedback on their individual performance. 

• Increased operational testing, including testing conducted at the local 

level, assesses screeners’ abilities to detect a threat item under covert and 

realistic conditions or in a training environment. 

• 366 computer-based, off-the-shelf courses from NetG provide general 

business and professional development training.28 

While some of these tools are still in development, TSA has made important 

strides in fielding training tools and practices to exercise and sharpen screener 

skills necessary to detect prohibited items. 

27 49 U.S.C. 44935 (h) specifically requires that screeners receive training in recognizing dual-use items.
	 28 While some courses already relate to screener tasks, such as “Excellence in Service: Working with Upset Customers,” TSA is augmenting 

these course offerings with TSA-specific training developed in house, such as “Sensitive Security Information Awareness.” 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 51 
 

Recurrent Training Requirements May Have Implications for the Screener 

Workforce 

According to the interim guidance on recurrent training, all screeners, both 

full- and part-time, are to receive three hours of recurrent, administrative, and 

professional development training each week during scheduled duty time. To 

accommodate airports’ operational constraints and scheduling limitations, this 

three-hour-per-week standard may be met if screeners receive this level of training 

on average over an entire quarter. Accordingly, an airport providing a screener six 

hours of recurrent training one week and none the next would be in compliance.29 

While TSA has developed necessary recurrent training standards, it is not clear 

what impact associated requirements will have on localities experiencing screener 

staffing shortages. The three-hour-per-week training standard represents a sizeable 

staff time commitment: 7.5 percent of full-time and between nine and 15 percent 

of part-time screeners’ non-overtime working hours. At certain airports, this major 

commitment of staff time is concurrent with a substantial screener shortage. 

Several training coordinators and FSDs remarked that some would find it difficult 

to meet this new training requirement, because doing so would leave them with 

insufficient numbers to staff checkpoints and checked baggage stations. When 

considered system-wide, screener shortages are significant. Numerous articles 

published in spring 2004 highlighted problems associated with multiple airports 

that are understaffed. In addition, between May 2003 and January 2004, TSA 

airports used the equivalent of more than five percent of its authorized screener 

workforce in overtime hours.30 For the majority of that time, the three-hour 

recurrent training requirement was not yet in effect. 

While it is vital that screeners receive high-quality recurrent training to enhance 

skills and learn about new developments, technological advances, and the latest 

threats, many airports may not be able to meet the current requirement. TSA 

should examine the workforce implications of the three-hour training requirement 

and consider these implications in future workforce planning to ensure that all 

screeners meet the recurrent training standard by performing high-quality training 

29 The guidance further recommends that this recurrent training include at least one hour of screening procedures review and one hour of 

X-ray image interpretation practice per week over a calendar quarter.
	 30 Challenges Exist in Stabilizing and Enhancing Passenger and Checked Baggage Operations, GAO-04-440T, February 12, 2004.
	

Page 52 An Evaluation of TSA’s Screener Training and Methods of Testing 
activities and that FSDs are, without exception, able to staff checkpoints and 

checked baggage stations with the required number of screeners. 

Threat Image Projection Program Features Should Be Fine-Tuned to Maximize 

Training Benefits 

TSA uses Threat Image Projection (TIP) for training and testing threat image 

recognition on operating X-ray machines at passenger checkpoints. The EDS 

machines at checked baggage stations may also be configured to support TIP. 

Regarded in part as a component of TSA’s recurrent training program, TIP 

transmits simulated threats onto X-ray or EDS operator screens on active 

machines at intervals governed by an image frequency algorithm. The program 

records screener responses to the simulated threats and provides immediate 

feedback to screeners on their performance. TIP notifies screeners when they have 

correctly identified a TIP threat image and cautions them when they have not 

recognized a TIP threat image or taken too long to evaluate a TIP image. 

The image library that TIP draws upon for these projected images includes IEDs, 

firearms, knives, and other prohibited items and threats like opaque objects and 

dual-use items, e.g., screwdrivers. To challenge screener detection skills, many 

of the threats in these images are presented in “non-standard” orientations. An 

image of a gun from behind, such that only the butt of the weapon is visible, is an 

example of one such non-standard orientation. 

In 1997, FAA initiated TIP deployment to active screening locations as a means of 

increasing screener vigilance, providing continual on-the-job image interpretation 

training, and assessing screener performance. FAA’s efforts to institute TIP 

widely were halted by the events of September 11, 2001, out of concern that 

TIP would result in screening delays and increased passenger anxiety. FAA’s 

original logic and plans formed the basis for TSA’s later initiatives to use and 

expedite deployment of TIP at all U.S. airports. Because TSA perceived greater 

advantages to installing TIP on X-ray machines at passenger checkpoints rather 

than EDS machines at checked baggage stations, it prioritized TIP implementation 

at passenger checkpoints. In October 2002, TSA’s Administrator directed the 

formation of a TIP Integrated Product Team to develop program performance 

criteria, identify system data collection requirements, and ensure the deployment 

of TIP-Ready X-ray (TRX) machines at all airports. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 53 
TSA adopted a December 31, 2003, deadline for TIP installation on X-ray 

machines at every passenger checkpoint in the nation. To meet this goal, TSA 

purchased and installed hundreds of TRX machines while its Image Development 

Center generated 2,400 TIP images for each of the three makes of TRX. TSA 

ultimately succeeded in installing TIP in 98% of X-ray machines nationwide by 

the deadline, but due to vendor software difficulties and TRX procurement delays, 

not all 1,800 X-ray machines had TIP until March 2, 2004. 

TSA has espoused a time line that calls for TIP implementation on all EDS 

machines at checked baggage stations in FY 2005. TIP is presently installed on 

many EDS machines, but the image galleries that it uses are not standardized for 

all models of EDS. Although it has not been installed on EDS machines at airports 

to date, TSA has developed a common TIP image library for all EDS machines 

used in the U.S. TSA also is evaluating technical solutions to the challenges of 

TIP image projection in a three-dimensional environment. 

To optimize the program’s training potential, TSA is assessing the cost and 

feasibility of adding a user-adaptability feature to TIP. If developed, this feature 

will tailor TIP sessions to address individual screener weaknesses revealed in 

user performance data. If a screener has particular difficulty identifying IEDs, for 

instance, this software feature would trigger the projection of a higher proportion 

of simulated IEDs than under standard circumstances. 

TSA plans also include the networking of all TRX machines. Once completed, 

TRX connectivity will provide substantial benefits. First, it will facilitate 

information sharing on user performance among TRX machines, which is 

important to the currency of user-adaptability settings on different machines. 

Networked TRX machines also will be able to share user performance information 

with off-site, computer-based image training tools, so that these training tools can 

be similarly customized to individual screeners’ image recognition skill levels. 

Finally, TRX connectivity may ease airport-based TIP administrator obligations. 

Connectivity holds out the potential to streamline TIP data reporting and ease 

the process of adding newly prohibited items and future threats to TIP image 

galleries. 

TSA intends to use TIP as both a training tool and a performance measurement 

mechanism. To prepare for this latter function, TSA is developing TIP 

Page 54 An Evaluation of TSA’s Screener Training and Methods of Testing 
performance criteria. Setting national TIP performance standards poses some 

significant challenges. To confront these challenges, TSA is using TIP data from 

TRX machines around the nation to establish a TIP performance baseline. 

TIP data analysis will provide the basis for work in other areas, as well. TSA plans 

to make adjustments in particular TIP image difficulty settings, for example, based 

on observed screener detection performance presented in TIP data. In addition, 

TSA is developing a secure website that will generate TIP summary reports with 

summaries of the performance results of individual screeners, particular airports, 

TSA regions, and U.S. airports as a whole. Headquarters, FSDs, and staff will be 

able to use these TIP data reports to customize training efforts to address observed 

screener threat detection shortcomings. 

TIP is an important recurrent training tool and offers great potential for measuring 

of screener performance. Critical improvements to the program will advance 

its usefulness as a training tool and promote its reliability as a performance 

measurement mechanism. 

Aspects of Operational Testing Can Be Enhanced to Maximize Training and 

Testing Benefits 

Guidance on how to use the Modular Bomb Sets, Version II (MBS-2), and 

Weapons Training kits as effective training tools was included with the kit 

packages. Screeners are allowed to handle and assemble these items as well as 

experiment with how they appear on an X-ray monitor at various angles and in 

disassembled form, when placed in a simulated carry-on bag. As a result, both kits 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 55 

31 
can improve screener performance by providing realistic hands-on practice tools 

and sharpening screeners’ ability to detect IEDs and firearms. 

ATSA mandates the operational testing of screeners.32 Operational tests are 

performed by authorized staff using simulated or inoperable threat items, 

such as those found in MBS-2 and Weapons Training kits, in a live screening 

environment. A screener’s failure to identify these simulated or inoperable threats 

and follow appropriate procedures results in the disqualification of the screener 

from performing the associated screening function until they have completed 

remedial training. Remedial training consists of a review of the SOPs and 

appropriate videos associated with the screener’s failure. 

Thus far, operational testing has been conducted on a very limited scale and only 

a small fraction of the screener workforce has been subjected to it. TSA’s OIAPR 

conducted 1,095 checkpoint operational tests from November 2003 to January 

2004 and 192 checked baggage operational tests from January 2003 to January 

2004. DHS OIG conducted an additional 687 checkpoint operational tests and 146 

checked baggage operational tests from July to November 2003. In addition, the 

GAO conducted a very limited number of operational tests in 2003. 

While operational tests provide key information about screener performance, they 

also offer opportunities to identify and address critical recurrent training needs. 

Recognizing the value of such testing, TSA took steps to expand the allowed 

usage of the MBS-2 and Weapons kits. On February 23, 2004, TSA issued an 

Aviation Operations Directive, which was later revised in June 2004, to FSDs 

that provided guidance on using these kits to conduct passenger screener tests 

by placing the simulated threat items in accessible property submitted for X-ray 

examination or hidden on a person. Tests can be “practice” 33 or “operational.” 34 

32 Codified in 49 U.S.C. §44935(f).
	 33 A practice test assesses a screener’s ability to detect a threat item under covert and realistic conditions or in a training environment. 

Failure to detect a threat item during these “practice” tests does not result in a screener being disqualified from performing the associated 

screening function until remedial training is completed.
	 34 An operational test assesses a screener’s ability to detect a threat item under covert and realistic conditions only. Failure to detect a threat 

item during actual operational tests results in a screener being disqualified from performing the associated screening function until remedial 

training is completed. 

Page 56 An Evaluation of TSA’s Screener Training and Methods of Testing 
The revised Aviation Operations Directive and associated guidance were enhanced 

to allow firearms and IEDs to be assembled or disassembled when used during 

local testing. In addition, local test administrators are allowed to place assembled 

firearms in baggage at deceptive angles and in deceptive locations. Other possible 

enhancements to the local testing, however, would be to allow test objects to be 

artfully concealed and carried on one of the sensitive areas of the body.

35 Current 

procedures do not permit either. Expanding the procedures to include artful 

concealment and placement of test objects on sensitive areas is being considered 

at this time. Both of these enhancements are necessary to increase test realism and 

heighten its level of difficulty. 

The guidelines for local testing do not authorize practice and operational testing 

to be conducted at checked baggage screening locations since the items in the kits 

have not been validated for EDS or ETD equipment. TSA said that it is currently 

developing operational testing tools for use with ETD and EDS machines. TSA 

reports, however, that completion dates for these items have not been set. We 

believe that the distribution of test aids of this type to FSDs around the nation 

would significantly increase the operational testing of checked baggage screening 

locations and aid in the detection of prohibited items in checked baggage. 

With TSA’s recent approval to use on-screen resolution36 and future wide-scale 

adoption of resolution procedures by EDS operators, EDS operational testing is an 

increasingly vital airport security system assessment mechanism. 

We recommend that the TSA Administrator: 

Recommendation 15: Examine the workforce implications of the three-hour 

training requirement and take steps to correct identified imbalances in future 

workforce planning to ensure that all screeners are able to meet the recurrent 

training standard. 

Recommendation 16: Continue to pursue the development and application of TIP 

user adaptability features to maximize TIP training benefits. 

35 Guidance allows items to be placed on the upper-inner thigh, which TSA does not consider a sensitive area. TSA considers the breasts 

(females only), genitals, and buttocks to be sensitive body areas. 

36 On February 25, 2004, TSA approved the use of on-screen resolution, otherwise known as Alarm Resolution Protocol, in which the EDS 

operator interprets the bag image and may resolve benign alarms without explosives trace detection sampling or physical inspection. With 

the exception of a few pilot programs, however, on-screen resolution is not yet being used. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 57 
Recommendation 17: Expedite TRX connectivity to realize administrative and 

information sharing gains related to TIP. 

Recommendation 18: Further enhance local operational testing efforts by: (1) 

revising procedures and protocols to increase opportunities for realistic and 

difficult testing, and (2) expediting the development, certification, and distribution 

of ETD and EDS operational testing tools to enable assessment of screeners 

performing these duties. 

Annual Screener Recertification Testing for FY 2003-04 Has Been 

Completed 

To determine whether the TSA security screener workforce has the necessary 

knowledge and skills to continue to perform successfully screening functions, 

ATSA requires TSA to conduct and document an annual proficiency review of 

both passenger checkpoint and checked baggage screeners. ATSA provides: 

“An individual employed as a security screener may not continue to be 

employed in that capacity unless the evaluation demonstrates that the 

individual -

(A) continues to meet all qualifications and standards required to perform a 

screening function; 

(B) has a satisfactory record of performance and attention to duty based on 

the standards and requirements in the security program; and 

(C) demonstrates the current knowledge and skills necessary to 

courteously, vigilantly, and effectively perform screening functions.”37 

To comply with this requirement, TSA undertook recertification testing of its 

screeners, including lead and supervisory screeners, on October 1, 2003, with 

an estimated completion date of March 31, 2004. This recertification process 

consisted of two parts: (1) a knowledge and skills assessment program; and (2) a 

final rating on screeners’ annual performance agreement, an annual assessment of 

screeners signed by their supervisors. To be recertified, screeners who completed 

OJT prior to June 30, 2003, were required to pass specified components of 

37 Codified in 49 U.S.C. § 44935(f)(5). 

Page 58 An Evaluation of TSA’s Screener Training and Methods of Testing 
the knowledge and skills assessment program and achieve a rating of “met or 

exceeded” standards on their performance assessments. 

The knowledge and skills assessment portion of the recertification tests screeners 

on their knowledge of passenger checkpoint or checked baggage SOPs, Aviation 

Operations Directives, and other screening operations guidance, as well as their 

skill in performing security screening functions. Three components make up the 

knowledge and skills assessment program. 

Passenger checkpoint screeners are required to take all three components, while 

checked baggage screeners are required to take the job knowledge and practical 

skills demonstration components. Screeners receive a rating of either “pass” or 

“fail” for each component. If a screener fails any component, he or she is provided 

remedial training and an opportunity to retake the component. If he or she fails 

the retest for that component, the screener is separated from TSA employment. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 59 
As of May 2004, all screeners completed the FY 2003-04 recertification testing.38 

Overall, 99.9% of screeners passed the SPR, while 99.5% of passenger screeners 

passed the IPR. The practical skills demonstration component, delivered by 

Lockheed Martin, was the last component to be completed. Almost one quarter 

of screeners failed their first practical skills demonstration, while less than two 

percent of those who initially failed did so on their second attempt after receiving 

remedial training. Overall, 99.6% of screeners evaluated on their practical skills 

passed. As of May 19, 2004, recertified screeners numbered 42,682.39 Dismissed 

screeners numbered 350, as a result of their failing one of the recertification 

components or not having a satisfactory rating on their performance assessments. 

Figure 4. Screener Recertification Performance 

Recertification 

Components 

Screener Recertification Performance 

Tested 

Passed 

1st 

Attempt 

Retested 

Passed 

2nd 

Attempt 

In 

Progress† 

Overall 

Passed Dismissed 

 Component 1 

SPR 

48,518 47,264 

97.4% 938 908 96.8% 

296 48,204 

99.9%

30 

 Component 2 

IPR 

33,724 32,357 

96.0% 1,078 909 84.3% 

290 33,267 

99.5%

169 

 Component 3 

Practicals 

42,970 32,312 

75.2% 10,683 10,534 98.6% 

148 42,770 

99.6%

148 

*Statistics provided by Transportation Security Administration, May 19, 2004. 

†Screeners who are on extended leave, workers’ compensation, or Performance Improvement 

Plan. 

A media report on the practical skills testing conducted during the third 

recertification component cited screener reports that evaluations were conducted 

38 This does not include 1,152 screeners who have not completed the re-certification testing due to reasons such as being on extended 

leave, workers’ compensation, or a Performance Improvement Plan, i.e., a screener who did not receive a satisfactory rating on his/her 

performance assessment but has been given the opportunity to improve. 

39 The number of re-certified screeners is significantly lower than the 48,518 screeners who took the first re-certification component and 

reflects screeners who failed a re-certification component, have yet to take one or more re-certification components, have left TSA after 

completing one or more re-certification components, or have been reassigned. 

Page 60 An Evaluation of TSA’s Screener Training and Methods of Testing 
using subjective criteria and that retests were “watered down.”40 Over the full 

course of its delivery, the third component of the recertification process has been 

the subject of WPT quality assurance monitoring. Quality assurance monitors 

visited approximately 40 airports, or 11%, with staff undergoing practical skills 

evaluations between October 2003 and February 2004. Substantive comments 

from quality assurance visits do not support claims that retests were less rigorous 

than initial evaluations, but they do raise concerns about the administration of 

the practical skills testing. Quality assurance reports from a quarter of airports 

that monitors visited pointed to significant test administration problems.41 Quality 

assurance officials responded to significant issues of this nature by ensuring that 

corrective actions were taken immediately and that the process was improved for 

future administration of the third recertification component. For example, after 

detecting early deficiencies, quality assurance staff developed and distributed 

guidelines that assisted Lockheed Martin’s evaluators in assessing the practical 

skills of screeners. 

Under the current TSA recertification guidelines, screeners who have been 

cross-trained and are “actively” working as both a passenger checkpoint and 

checked baggage screener are required to take only the recertification test for 

passenger screeners. Cross-trained screeners are, therefore, not required to take 

the SPR specific to checked baggage or demonstrate the practical skills necessary 

to perform checked baggage screening functions. As of May 17, 2004, TSA’s 

workforce included approximately 18,588 cross-trained screeners who were 

certified to serve as passenger or checked baggage screeners. 

TSA’s future training plans call for an increase in the number of “dual function” 

screeners, i.e., those that perform passenger checkpoint as well as checked 

baggage screening functions. TSA’s WPT currently is developing a process and 

timelines for meeting the annual screener recertification mandate for FY 2004-05. 

As a result of the current number of cross-trained screeners and the future increase 

of dual function screeners, TSA’s WPT reportedly plans to establish a dual 

function screener recertification test for the next recertification cycle. We agree 

40 Strohm, Chris, “Airport Screeners Slam TSA Re-certification Program,” Government Executive, February 18, 2004.
	 41 The following were considered significant test administration problems: coaching of screeners during the evaluation, evaluation scenarios 

compromised, evaluation props in plain sight, evaluation on items not prohibited under TSA SOPs, evaluation on incorrect scenarios, 

evaluation scenarios too simplistic, inadequate evaluation orientation, and evaluators lacking adequate knowledge of the screening process.
	

An Evaluation of TSA’s Screener Training and Methods of Testing Page 61 
 

 

that screeners should be required to be recertified on every screening function 

that they are expected to continue to perform and agree that a dual function 

recertification track should be established for all future recertification testing. 

Other Issues Related to Screener Training, Screener Performance, and an 

Increase in the Detection of Prohibited Items 

Other issues must be addressed in order to reduce further airport security system 

vulnerabilities. 

Use of the Online Learning Center Is Limited by Network Access 

In March 2004, the TSA Administrator testified, “From the standpoint of training 

delivery, our most significant accomplishment is the launching of our learning 

management system, the TSA Online Learning Center (OLC).”42 Among its many 

advantages, an online learning management system can: 

• Accommodate remote locations and flexible schedules; 

• Reach audiences large or small; 

• Make available self-paced courses, references, and job aids, including 

the “Excellence in Screener Performance” video series and web-based 

training tools, such as technical training for supervisors; 

• Organize professional development plans; 

• Enable rapid updates to materials; 

• Ensure training delivery is highly consistent; 

• Simplify changing course schedules and controlling sign-ups; 

• Support standardized testing and enhanced test security measures; and 

• Provide automated record-keeping of an employee’s progress. 

TSA has begun to realize some of these benefits since launching the OLC on 

October 31, 2003. However, TSA needs to improve screeners’ access to computers 

and the TSA intranet in order to enable widespread use of the system. 

42 House Appropriations Subcommittee on Homeland Security, Hearing on the Transportation Security Administration FY05 Budget 

Request, March 11, 2004. 

Page 62		 An Evaluation of TSA’s Screener Training and Methods of Testing 
Use of the OLC continues to grow since the system was launched, increasing 

from 1,400 average weekday student logins in February 2004 to 3,000 weekday 

student logins in April 2004.43 Much of the OLC’s recent growth has been in 

available content, such as the screener recurrent training courses that TSA is 

making available online. Use of OLC training courses and record-keeping is 

growing. 

Figure 5. OLC Use 

Learning events recorded 

or completed online 

March 

2004 

April 

2004 

Screener recurrent technical training, Threat in the 

Spotlight, and local testing events 66,000 93,000 

Web-based online courses 25,000 35,000 

Briefings and reading assignments 77,000 35,000 

Instructor-led classes and miscellaneous training 

records 7,000 15,000 

Total 175,000 265,000 

TSA is using or investigating the many advantages of an online learning 

management system, from managing employee development plans to posting 

reference materials. Handbooks for using the OLC have been posted and 

updated online; and other materials pertinent to screening, such the SOPs and 

Aviation Operations Directives, are to be posted in the near future. Advantages 

of maintaining these references on the OLC include better version control and 

replacing time-consuming, manual updates to materials with quick, centralized 

electronic uploads over the intranet. In addition, TSA is beginning to tap the 

OLC’s standardized testing features. The OLC has the capability to deliver online 

written tests with randomized questions and answers and proctor codes to control 

test administration and security. When this capability is exercised, it could help 

TSA address the test security concerns discussed on page 20. TSA plans to pilot 

OLC-based tests for the DFS course in summer 2004. 

43 For comparison purposes, figures include only student logins from the TSA intranet. Student logins from the TSA extranet, which TSA 

launched in March 2004, increase the April figures; see page 65. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 63 
In February 2004, TSA began to require the centralization of training records in 

the OLC. Currently, TSA does not have full, centralized records to verify that 

screeners meet the training requirements of ATSA and complete three hours 

of recurrent training for every 40 hours worked. WPT recently added analysis 

software to the OLC that will enable staff to compile and analyze reports of 

employees’ certifications and completed classes, once the training records are 

added to the OLC. Before the launching of the OLC, TSA allowed training 

coordinators and contractor trainers to develop independent and non-standard 

training record systems. As a result, TSA’s training records are fragmented 

between WPT and contractor databases, plus spreadsheets, databases, and paper 

files unique to local airports. We reviewed a judgmental sample of paper and 

electronic training records at four airports, where FSDs are responsible for 

record retention. More often than not, records of ATSA-required training were 

incomplete. Shortfalls included missing or partial records of classroom training, 

missing records of equipment-specific training and examinations, documentation 

of fewer than 60 OJT hours, and missing OJT examination records. Establishing 

complete training records in the OLC will do much to improve the quality of 

screener learning histories. TSA has begun to use the OLC to schedule and 

record new training, but the compilation of historical records from contractors, 

WPT, and training coordinators has been delayed. TSA has several challenges to 

address in order to build the OLC learning histories, such as the loss of historical 

training documentation. Because some contractor training records are no longer 

accessible, TSA will need to formulate a strategy for verifying that screeners have 

fulfilled the training requirements. 

Other challenges include limited network connections to the OLC in the field and 

the manual data entry workload imposed on training coordinators when training is 

conducted offline. Many of the OLC’s features, such as automated record keeping 

and secure testing, are difficult or impossible to use in the absence of computers 

with high-speed connectivity to TSA’s intranet. However, of the 425 airports with 

TSA screeners, only 112, or 26.4%, have computer training labs with high-speed 

connectivity. At least 50 additional airports have computer training labs that could 

be networked, but TSA ceased these high-speed network installations in April 

2004 after exhausting funds. TSA’s information technology office does not expect 

Page 64 An Evaluation of TSA’s Screener Training and Methods of Testing 
supplemental funding adequate to resume installations in FY 2005. This leaves 

screeners at 313 airports with insufficient access to the OLC. 44 

Until connectivity is improved, WPT improvised several methods to enable the 

use of some OLC features. In March 2004, TSA launched an extranet that enables 

employees to log in to the OLC from non-TSA computers, such as from home or 

a public library. By the end of April 2004, TSA recorded a daily average of 950 

weekday and 550 weekend student logins from the extranet. In addition, WPT 

distributes online course materials to training coordinators in alternate formats, 

including CD-Rom, videos, and paper. Training coordinators, most of whom have 

high-speed intranet access, may download and print OLC materials or use them 

for overhead projections. All of the new screener recurrent web-based training 

and performance videos, and 50 of the 366 general NetG courses, are available for 

offline use. Nevertheless, the improvised solutions have limitations. For example, 

offline delivery of OLC content prevents training coordinators from using the 

OLC’s automated record keeping feature, and security concerns prevent TSA from 

offering all but the NetG courses and account administration features over the 

extranet. 

Providing computers with high-speed connectivity to all the airports will enable 

TSA to make better use of the many training features the OLC offers. In the 

July 2003 Passenger Screener Performance Improvement Study, TSA staff 

recommended that TSA “[e]stablish adequate airport learning centers at all 

airports that have not yet done so” and “[a]ccelerate broadband access and LMS 

[Learning Management System, or the OLC] at all airports” in order to remedy 

screeners’ lack of skills, knowledge, or information. TSA should continue to fund 

the implementation of these recommendations. 

We recommend that the TSA Administrator: 

Recommendation 19: Fund and resume installation of computer training labs and 

high-speed network connectivity to provide all screeners with full access to the 

Online Learning Center. 

44 In FY 2003, TSA prioritized delivery of network connectivity to the administrative locations of the FSDs and their staff, installing the 

computers and networking for 148 out of 159 FSDs. However, many of these networked computers were dedicated to staff administrative 

work and distant from screening locations. TSA’s original vision for installation included extending the network beyond the offices 

to airport operations areas, including training rooms, break rooms, operations centers, and screening locations, which would improve 

screeners’ access to the OLC. One airport has received this full installation. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 65 
Specific Screening Equipment Training and Certification Is Necessary 

TSA is required by ATSA to provide equipment-specific training to all of its 

security screeners. According to ATSA, “An individual employed as a security 

screener may not use any security screening device or equipment in the scope 

of that individual’s employment unless the individual has been trained on that 

device or equipment and has successfully completed a test on the use of the 

device or equipment.”45 There are different models of ETD machines in use at 

both passenger checkpoints and checked baggage screening stations at airports 

around the country. In addition, more than one model of EDS machine is in use at 

TSA checked baggage screening stations. The following figure lists the different 

models of ETD and EDS machines currently in use at commercial airports: 

Figure 6. Security Screening Equipment Models Currently in Use 

TSA Screening 

Equipment Makes & Models Manufacturers 

Explosives Trace Ionscan 400A & B Smiths Detection 

Detection Machines Itemiser - Windows GE IonTrack 

(ETD) EGIS II & III Thermo Electron Corp.

Explosives Detection 3DX 6000 L-3 Communications 

Systems Machines CTX 2500, 5500 InVision 

(EDS) CTX 9000 InVision 

Each model of ETD and EDS machines has significant operational and preventive 

maintenance requirements specific to that model. For ETD machines, sampling 

media, placement of sampling swab, calibration/ verification procedures, 

machine start-up procedures, shift maintenance, decontamination procedures, 

and operations monitor display differ among each of the three makes currently 

in use. For EDS machines, the imaging orientation, operator image manipulation 

capabilities, and operator panel and console presentation and display are critically 

different. 

45 Codified in 49 U.S.C. § 44935(g). 

Page 66 An Evaluation of TSA’s Screener Training and Methods of Testing 
To achieve ATSA compliance and foster screener understanding of the operational 

and preventative maintenance differences of the machines, it is essential that 

screeners be trained on the particular make and model of EDS and ETD machine 

with which they will work. Staff reported that TSA attempted to schedule newly 

hired checked baggage screeners on the models of EDS and ETD machines 

that are located at the airport at which they will be working. If a course was not 

available with both the required EDS and ETD models, however, the EDS model 

took precedence. For newly hired passenger checkpoint screeners who would be 

using ETD machines, TSA did not make an effort to schedule them in courses that 

would certify them to operate a specific model of ETD machine because TSA did 

not consider ETD certification a job requirement for these screeners. 

We compared the model of screening devices offered during instruction in the 

basic passenger checkpoint and checked baggage courses throughout December 

2003 against the models of ETD and EDS screening systems available at 

screeners’ home airports. In general, screeners were scheduled to train on the 

appropriate EDS model for their airports. However, of 481 students enrolled 

in the checked baggage courses, 96, or 20%, returned to home airports listed 

as not having the ETD model that they had been trained to use. For passenger 

checkpoint screeners, 201 of 808, or 25%, returned to home airports listed as 

having ETD models different from the one on which they trained. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 67 
Screener ETD Training Basic Screener 

Training Course Student Trained on Not Trained on 

Enrollment for Home Airport Home Airport 

Dec. 2003 ETD Model* ETD Model* 

 Checked Baggage 481 385 96 

80% 20% 

 Passenger Combo 808 607 201 

75% 25% 

Total 1,289 992 297 

77% 23%

*According to TSA EDS and ETD Machine Inventory, December 2003. 

Figure 7. Model-specific Screener ETD Training
	

While it may be difficult logistically to schedule newly hired screeners in basic 

courses that certify them on the appropriate EDS and ETD models for their 

airports, it is imperative all checked baggage screeners have specific training 

that addresses the specific operation and nuances of the makes and models of 

ETD and EDS machines. The checked baggage OJT guidelines provide for 

checked baggage screeners to receive “differences training” that orients them to 

ETD models not taught during their scheduled classroom training, but the OJT 

environment is less structured and standardized than classroom training. 

In addition, because passenger checkpoint screeners should be able to perform 

tasks related to basic ETD machine operation and maintenance, these screeners 

should be trained on the specific operation and nuances of the makes and models 

of ETD machines. While the alarm resolution procedures using ETD are different 

for passenger checkpoint screeners than checked baggage screeners, the operation 

and preventative maintenance requirements of the actual ETD machines are the 

same for both passenger checkpoint and checked baggage screening. 

The deficiency in training on specific makes and models of ETD machines has 

been addressed in part by TSA through its design of the DFS course. During this 

course, TSA incorporated time for all newly hired screeners to learn both the 

Page 68 An Evaluation of TSA’s Screener Training and Methods of Testing 
Ionscan and Itemiser ETD machines.46 However, selected screeners train on only 

one make and model of EDS machine. As a result, TSA should continue to ensure 

that these newly hired screeners are scheduled for basic courses that certify them 

on the correct make and model of EDS machine for their airports. 

As a corollary to training screeners on the appropriate ETD and EDS models 

for their airports, TSA should schedule trained screeners to operate the machine 

models for which they are qualified. Although the training coordinators said they 

communicate with scheduling officers about screener qualifications, TSA does 

not have a system in place to prevent scheduling officers from assigning screeners 

to operate equipment they are not qualified to use. During one of our site visits, 

we reviewed training records available at the checked baggage training stations 

for a random sample of checked baggage screeners to determine whether these 

screeners were certified on the model of EDS machine located at the checked 

baggage station to which they had been assigned.47 Of 40 checked baggage 

screeners, 21, or 53%, were scheduled to work at stations with models of EDS 

machines different from the model on which they were certified. 

Figure 8. On-Duty Screener EDS Certifications 

Checked 

Baggage 

Screening Station 

EDS Model 

at Station 

 Station #1 3DX 6000 

 Station #2 CTX 5500 

 Station #3 3DX 6000 

 Station #4 CTS 5500 

Total Not Certified on 

Model at Screening Station 

On-Duty Screener EDS Certifications 

CTX 5500 CTX 9000 3DX 6000 

5 3 12 

3 8 0 

3 0 3 

1 2 0 

8 13 0 

Screeners 

Not Certified on 

Model at 

Screening Station 

8 (40%) 

8 (73%) 

3 (50%) 

2 (67%) 

21 (53%) 

46 Because it is being phased out of the security screening system, TSA is not training screeners on the operation of the EGIS ETD machine. 

47 In some instances, screeners, who were not certified on the equipment located at the checked baggage to which they were assigned, were 

not operating the equipment but performed other duties such as loading baggage onto the machine conveyor belts. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 69 
Some scheduling officers responsible for scheduling new screeners for OJT
	

and for shifts are informed of screener equipment certifications by training 

coordinators on an ad hoc basis, often via email or verbally. In other cases, 

screening managers located at passenger checkpoints and checked baggage 

stations were responsible for determining the equipment certifications of screeners 

assigned to their station by either talking to the screener, calling the local training 

office, or referring to sometimes outdated or incomplete training records that 

are located at the checkpoints and stations. One training coordinator whom we 

interviewed planned to improve local records of screener certification and hoped 

to use them as a management control to ensure appropriate work assignments. 

TSA is headed in this direction as well, and recently added analysis software to 

the OLC that will facilitate the generation of certification reports. The OLC itself 

offers more than 80 reports for training coordinators. To ensure that screeners are 

scheduled to work on machines on which they have been certified and to assist 

scheduling officers determine the certifications that each screener has earned, TSA 

should develop a scheduling system that will interface with the training records 

found in the OLC. 

We recommend that the TSA Administrator: 

Recommendation 20: Ensure that screeners are scheduled for basic classroom 

training that provides initial certification on the specific make and model of ETD 

and EDS machines that they operate. 

Recommendation 21: Improve management controls for the screener scheduling 

system, such as linking scheduling to employee qualifications in the Online 

Learning Center, to ensure that TSA schedules screeners to operate only 

equipment on which they are certified. 

Efforts to Advance the Development and Deployment of New Screening 

Technologies Should Be Accelerated 

Current TSA screener training is largely shaped by the screening technologies in 

use today. Passenger checkpoint screeners, for example, dedicate considerable 

training time to learning how to recognize threats in X-ray images, because 

the effectiveness of X-ray screening depends on whether the X-ray operators 

Page 70 An Evaluation of TSA’s Screener Training and Methods of Testing 
recognize threat images. Checked baggage screeners, in contrast, learn very 

little image interpretation because the EDS machines they use have a different 

technology that identifies and locates potential threats without operator 

intervention. The effectiveness of TSA’s screening technologies depends in 

varying degree on operator performance. In other words, screeners’ overall 

performance in guarding against transportation security threats relies both on 

human skill, which training can enhance to a certain extent, and on the capabilities 

of screening technology. Since there are limitations on human performance, 

investing in improved screening technologies is one way that TSA can improve 

overall performance in detecting threat items. 

On their own, screeners cannot detect all threat items all of the time. Several 

“human factors” combine to limit screeners’ ability to do so, including fatigue, 

task complexity, and social conventions. Screeners performing repetitive tasks, 

such as hand-wanding or ETD sampling, must combat a degree of monotony 

while strictly applying SOPs without short-cutting or adopting improper 

techniques. The repetitiveness of screening passengers and baggage creates 

fatigue, which can lead to decreased attentiveness and accuracy. X-ray operation 

also requires unflagging vigilance for an uncertain threat, plus extremely wellhoned image recognition capabilities. Property that passes through X-ray 

machines for screening contains a universe of items assembled into a virtually 

limitless number of configurations. Given the complexity of object recognition 

efforts and the impact of fatigue, there are limits on human performance in this 

area, even if the screener has an aptitude for the work and substantial training and 

experience. In another example, social conventions discourage screeners from 

encroaching on travelers’ privacy to pat down passengers close to sensitive body 

areas while searching for threat items. While the SOPs prohibit screeners from 

touching certain sensitive areas during a full-body pat down, some screeners 

give these areas particularly wide berth when conducting their searches. Aspects 

of screeners’ work are extremely challenging and present substantial hurdles in 

vigilance, skill development, and technical and procedural adherence. 

Screener selection, training, testing, and monitoring can help screeners overcome 

these challenges in many cases, but will not guarantee complete success. There 

are fundamental limitations on human performance in any line work; airport 

security screening is no exception. Such screener performance deficiencies create 

openings in the security system that render aviation more vulnerable. In fact, 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 71 
recent operational test results from DHS OIG48 indicate that screener performance 

was a factor in 82% of passenger checkpoint and 61% of checked baggage 

screening failures.49 

To a significant extent, technological limitations result in vulnerabilities 

independent of human performance factors. DHS OIG staff conducting 

operational penetration tests in 2003 reported that screening technology 

limitations were a factor in 30% of security failures at checkpoints and 30% of 

failures at checked baggage screening stations.50 Some of the currently deployed 

screening technologies lack detection capabilities for certain threats, and some 

depend too heavily on the detection capabilities of screeners. 

In its own Passenger Screener Performance Improvement Study, TSA 

acknowledged that “technological limitations combine with human factors to 

prevent 100% detection” of threats. The report continues by recommending that 

TSA “accelerate efforts to identify, test, and deploy new technologies.” In line 

with this analysis, TSA is actively involved in the development of new systems 

to improve screener performance and materially aid in threat detection efforts. 

One goal of development efforts is to furnish screeners with more performance 

monitoring aids. The implementation of TIP, discussed earlier, is an associated 

effort. TSA is also pursuing testing and approval of a pressure gauge for ETD 

sampling wands to provide feedback to screeners on the amount of pressure they 

are using in sampling bags. Other current TSA technological development efforts 

are geared toward reducing the human factor in checkpoint and checked baggage 

screening by developing superior automated detection aids. 

A substantial measure of current airport security system vulnerability is linked to 

screener performance. Accordingly, some security system vulnerabilities can be 

addressed with improved selection, training, and monitoring of screeners. These 

48 Department of Homeland Security, Office of Inspector General, Office of Audits, “Audit of Passenger and Baggage Screening 

Procedures at Domestic Airports.” OIG-04-37, September 2003.
	 49 For the purposes of this section, operational test failures due to “inadequate training” or “failure to adhere to standard operating 

procedures” were considered failures due to screener performance. In several instances, non-human factors, e.g., technological limitations, 

contributed to operational test failures in which screener performance was also a factor. 50 Operational test failures were sometimes attributable to multiple causes. In several instances, human factors contributed to operational 

test failures in which technological limitations were also a factor. 

Page 72 An Evaluation of TSA’s Screener Training and Methods of Testing 
efforts will ultimately experience diminishing returns, however, as the quality of 

human performance of current screening functions is finite. At checkpoints and 

checked baggage screening locations, the best way to address these vulnerabilities 

is through the development of new technologies that provide more substantial 

screening assistance to operators and reduce the impact of human limitations. 

While organizational and individual actions will continue to play a critical role in 

airport screening, the application of new screening technologies holds the greatest 

long-term potential for reducing airport security system vulnerabilities and 

increasing the detection of prohibited items. 

We recommend that the TSA Administrator: 

Recommendation 22: Continue efforts toward the development and advancement 

of technologies to support screening efforts. Resource investment should place 

particular emphasis on technologies for passenger checkpoint screening, as 

passenger screening procedures are more operator dependent and, thus, more 

vulnerable to human factors than checked baggage screening. 

Future Planning Should Account for Continually Changing Training 

Requirements 

Transportation security screening is a dynamic field fueled by evolving threats 

and changing means of confronting these threats. Terrorists and others posing a 

threat to aviation security will continue to use different and evolving techniques to 

avoid detection. In response to information on these changing threats and ongoing 

appraisals of the screening process, TSA adjusts screening procedures. While 

many screening technologies under development are geared toward reducing 

the human factor in screening, screeners will nonetheless have to absorb new 

technical information on an ongoing basis to accommodate the deployment of 

these new technologies. Moreover, given sufficient resources, TSA’s research 

and development efforts could result in a radical transformation of passenger and 

checked baggage screening operations in a matter of years. 

Anticipated changes in threats, procedures, and technologies present a major 

future training challenge for TSA. Unlike more static fields, it will be necessary 

for the security screening workforce to have evolving training requirements. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 73 
Consequently, it is imperative that TSA’s administrative apparatus be capable of 

continuously developing and implementing new training tools. 

Page 74 An Evaluation of TSA’s Screener Training and Methods of Testing 



















Appendix B 

OIG Evaluation of Management Comments 

We evaluated TSA’s written comments to the draft report and made changes, as 

appropriate, to the final version. Below is a summary of our analysis of TSA’s 

response to the recommendations contained in the draft report. 

Recommendation 1: Complete the analysis for basic screener classroom training, 

both for passenger checkpoint and checked baggage screeners, and institute 

mechanisms to ensure that staff update the analysis at least annually and apply the 

results to curriculum revisions. 

TSA Response: WPT’s Screener Training Division will re-examine the basic 

screener training program with assistance from WPT’s Instructional Design, 

Performance Consulting, Standards and Testing, and Quality and Assurance and 

Evaluation Divisions. TSA also noted that its Performance Consulting Division 

completed a performance assessment of internal training development procedures 

in February 2004, which examined overall improvements for the basic screener 

training program. 

OIG Evaluation: The Performance Consulting Division’s study that TSA cites 

provided comprehensive recommendations on program management for basic 

screener training. The study does not contain analysis of the training needs of new 

passenger checkpoint and checked baggage screeners, nor does it develop job 

competencies into discrete tasks, training content and strategies, and measurable 

objectives. However, the study recommends that WPT adopt an instructional 

design model that includes the analysis phase and “immediately” launch a second 

study to examine and link job competencies, performance expectations, learning 

objectives, learning activities, and test items. The Performance Consulting 

Division also recommended that WPT develop and annually revise a long-range 

program plan for screener training. Based on TSA’s response, it appears that WPT 

has not yet completed these recommendations from the February 2004 study. 

Their completion should satisfy our recommendation. In its action plan, TSA 

should provide an update on its progress toward completing the second study and 

long-range plan. Recommendation 1 is resolved – open. 

Recommendation 2: Ensure that passenger checkpoint and checked baggage 

basic course objectives (1) address the knowledge and skills necessary for routine 

screener performance, as identified by the most current task analysis, and (2) are 

presented to students at the beginning of course lessons. 

Page 94 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix B 

OIG Evaluation of Management Comments 

TSA Response: Internal and contractor subject matter experts evaluated the 

course objectives for the April 2004 version of the Dual Function Screening 

course and found that they support the knowledge and skills necessary for 

routine screener performance. The new course materials present objectives at 

the beginning and end of each lesson. TSA added that when WPT completes the 

analysis for the basic screener training program, per our first recommendation, 

staff will update the course objectives and content, as needed. 

OIG Evaluation: We accept TSA’s response and look forward to reviewing 

TSA’s course materials once they are revised based on the completed analysis. 

Recommendation 2 is resolved – open. 

Recommendation 3: Further revise written examinations and administration 

procedures, including the following steps: 

• Ensure all course objectives receive an appropriate assessment of student 

learning; 

• Thoroughly validate tests, including passing scores, if they are to be used 

for selection decisions; 

• Pilot-test examination questions to ensure their clarity and objectivity; 

• Schedule tests later during the course to enhance their ability to measure 

students’ long-term retention of course material; and 

• Incorporate post-test reviews to prevent screener misconceptions from 

being carried into the workplace. 

TSA Response: TSA’s goal is to ensure that all course objectives receive an 

appropriate assessment of student learning and are covered by test material from 

one of the DFS written, image interpretation, or practical skills demonstration 

tests. To validate tests, TSA directly mapped DFS written test items back to 

training content and revised cut scores using Angoff panels, a technique where 

subject matter experts rate the probability that a minimally competent screener 

would answer each item correctly. Furthermore, TSA piloted all DFS tests before 

the course was released. Finally, TSA now administers tests at the conclusion of 

the course and conducts standardized post-test reviews. During these reviews, 

missed concepts, and not individual questions, are discussed. 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 95 
Appendix B 

OIG Evaluation of Management Comments 

OIG Evaluation: We are satisfied that TSA has thoroughly validated and piloted 

written tests. In addition, we are pleased that TSA currently is administering 

written tests near the end of its DFS course and conducting post-test reviews. 

We maintain, however, that instructors need to review all concepts missed by 

screeners who will advance to OJT, whether associated test items were missed by 

one or many students. Finally, we agree that TSA’s goal should be to ensure that 

all course objectives receive an assessment of student learning. We request that 

TSA provide, as part of its action plan, the document that maps course objectives 

to test items. Recommendation 3 is resolved – open. 

Recommendation 4: Develop and distribute detailed test administration 

guidelines for Practical Demonstration of Proficiency Evaluations with the aim of 

increasing standardization. 

TSA Response: TSA revised the practical skills checklists and administration 

guidelines for implementation with the new DFS course. The more detailed 

practical skills checklists specify exactly what behaviors need to be demonstrated 

for each task. The Administration Guide for Practical Skills Demonstrations 

conducted during Checkpoint, Checked Baggage, or Dual Functioning Screener 

Training, May 2004, standardizes the process by clearly stipulating that a student 

is required to perform the practical skills demonstrations without coaching from 

the test administrator. 

OIG Evaluation: TSA’s revisions may address this recommendation by providing 

instructors with sufficient detail on assessing demonstrated tasks and a list of 

steps that must be executed by students for them to be credited with successfully 

performing each task. As part of its action plan to resolve this recommendation, 

TSA should provide us with a copy of the practical skills checklists and the 

Administration Guide for Practical Skills Demonstration conducted during 

Checkpoint, Checked Baggage, or Dual Functioning Screener Training, May 

2004, from the new DFS course. This recommendation will be closed when we 

have reviewed revised DFS practical skills checklists and administration guide. 

Recommendation 4 is resolved – open. 

Recommendation 5: Distribute effective training materials to reinforce learning 

and allow for individual study outside the classroom. 

Page 96 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix B 

OIG Evaluation of Management Comments 

TSA Response: With the deployment of the DFS course, TSA developed detailed 

student guides that include slide contents and content notes for each associated 

slide. Students are permitted to take these guides outside of the classroom 

allowing for individual study and must return the guides before taking the job 

knowledge test for each module. 

OIG Evaluation: TSA should provide us with a copy of the DFS student 

guide. This recommendation will be closed upon our review of that document. 

Recommendation 5 is resolved – open. 

Recommendation 6: Require all TSA approved instructors to complete an 

appropriate form of OJT prior to providing instruction and to undergo annual 

recertification testing. 

TSA Response: TAIs have completed security screener training, including OJT, 

and are required to meet all recertification requirements. TSA also lists a number 

of contract instructor requirements, including a complete annual certification of 

proficiency in a Practical Skills Demonstration. 

OIG Evaluation: Because TAIs complete OJT and meet all recertification 

requirements, using TAIs to conduct new hire and cross-training on an exclusive 

basis is the direction that TSA should take. Until this is implemented fully, 

however, the additional experience gained from completing OJT would better 

equip contract instructors with the knowledge and skills to conduct screener 

training effectively. TSA’s response does not address OJT requirements for 

contract instructors. In addition, because changes and developments in security 

screening procedures are frequent, TSA should have a method to verify, on an 

annual basis, that instructors maintain the knowledge and skills necessary to teach 

current screening procedures and machine operation. Some contract instructors 

who recently completed a train-the-trainer course on how to instruct the DFS 

course passed written and practical tests on updated procedures. However, it is 

unclear whether contract instructors would receive annual evaluations of their 

familiarity with current procedures, in the absence of new train-the-trainer 

courses. Recommendation 6 is unresolved. To resolve this recommendation, TSA 

should adopt a means to verify that contractor instructors know current screening 

procedures, by annual recertifications or some alternative, and possess familiarity 

with screening in airports, through OJT or otherwise. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 97 
Appendix B 

OIG Evaluation of Management Comments 

Recommendation 7: Analyze the cost and feasibility of establishing designated 

training sites where screening equipment, simulators, and training aids can be 

permanently located and readily available to students. 

TSA Response: TSA is currently examining the cost and effectiveness of locating 

training centers near proposed hiring centers or in currently established training 

centers located at or near major training locations. 

OIG Evaluation: The establishment of designated training sites would improve 

the effectiveness of initial screener training. We are pleased that TSA is assessing 

the cost of establishing such training sites. Upon completion of TSA’s analysis, 

we will close this recommendation. Recommendation 7 is resolved – open. 

Recommendation 8: Use alternatives to lectures more frequently during 

classroom training to help maintain student attentiveness, contribute to the 

comprehension and mastery of new knowledge and skills, and foster retention of 

the material. 

TSA Response: With the release of the DFS course, a number of alternative 

delivery modes are employed in basic training, including practical labs in a 

simulated checkpoint environment, additional demonstration and practice during 

the full-body pat down and screener safety and awareness lessons, probing 

questioning by instructors, a visit to an airport checkpoint, and new software 

applications for image interpretation. 

OIG Evaluation: Alternate methods of delivery were used more frequently in 

passenger checkpoint courses than in checked baggage courses. The alternate 

delivery methods listed by TSA in its response focus largely on passenger 

checkpoint screening. However, TSA must ensure that alternate methods of 

delivery are used by instructors when training screeners how to operate EDS 

machines and follow checked baggage screening procedures. This should include 

a visit to an airport checked baggage station and demonstration and practice on an 

EDS machine or simulator. In addition to reviewing TSA’s action plan to resolve 

this recommendation, we will review the DFS course curriculum, including 

the course schedule and course administration manual given to instructors. 

Recommendation 8 is resolved – open. 

Page 98 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix B 

OIG Evaluation of Management Comments 

Recommendation 9: Cease all coaching of students during practical 

demonstration of proficiency evaluations. 

TSA Response: TSA’s Administration Guide for Practical Skills Demonstration 

conducted during Checkpoint, Checked Baggage, or Dual Functioning Screener 

Training, May 2004, standardizes the process a test administrator must follow 

when administering practical skills demonstrations. This guide clearly stipulates 

that a student is required to perform the practical skills demonstrations initially 

without coaching or feedback from the test administrator. 

OIG Evaluation: The new administration guide specifically prohibits coaching 

during practical demonstration of proficiency evaluations and allows quality 

assurance staff to observe the skills demonstration. Recommendation 9 is closed. 

Recommendation 10: Ensure that leadership within the screener workforce 

diligently monitors screeners and immediately corrects identified failures to 

adhere to screening procedures and negligent screening techniques. 

TSA Response: TSA is working on revising the Checked Baggage SOP, including 

a chapter on “Screening Oversight and Audits,” that will cover how a supervisor 

must verify that screeners are following procedures as outlined in the SOP. In 

addition, TSA allows flexibility at the local level to decide how supervisors 

appropriately monitor and evaluate screener performance. For example, leadership 

at one airport created a form for screener managers to observe and rate each 

screener’s performance on a weekly basis and correct mistakes or recommend 

recurrent training. 

OIG Evaluation: TSA may address part of this recommendation through the 

incorporation of a “Screening Oversight and Audits” chapter in the Checked 

Baggage SOP. However, TSA also should revise the Passenger Checkpoint SOP 

to include a chapter on oversight. In addition, while we are encouraged by the 

example of extra measures taken by a local airport to observe, rate, and correct 

screener performance, until such measures are adopted and enforced at the 

national level, room for improvement remains. As part of its action plan to further 

resolve this recommendation, TSA should provide us with a copy of the “Screener 

Oversight and Audits” chapter of the Checked Baggage SOP. Recommendation 10 

is resolved – open. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 99 
Appendix B 

OIG Evaluation of Management Comments 

Recommendation 11: Continue the development of evaluations that will relate 

training to screener application of learning on the job and to organizational 

results. 

TSA Response: Implementing a 4-level evaluation model, including evaluations 

that focus on the application of learning on the job and the organizational impact 

of training, is an industry standard and one of WPT’s goals. WPT is waiting 

for resources to become available to conduct these two additional levels of 

evaluation. 

OIG Evaluation: We accept TSA’s response. In its action plan, TSA should 

specify the resources required for conducting the evaluations. In the absence of 

a comprehensive 4-level evaluation regimen, TSA may be able to apply existing 

performance measurements, such as operational testing results, as indicators of 

training outcomes. Recommendation 11 is resolved – open. 

Recommendation 12: Ensure that OJT guidance provides detailed, sequenced 

lists of specific tasks that each OJT student must perform, including instructional 

materials where needed, and establishes appropriate, function-specific time 

requirements for both newly hired and cross-trained screeners. 

TSA Response: TSA redesigned its OJT checklists to include a detailed, 

sequenced list of tasks to be completed and minimum OJT time requirements 

for particular job functions. TSA is now in the process of establishing a team to 

consider OJT program improvements. 

OIG Evaluation: TSA’s updates to OJT checklists are responsive to this 

recommendation. However, TSA’s response did not specifically address function 

specific OJT time requirements for cross-trained screeners. If these requirements 

have not already been set out in the revised OJT checklists, the project team 

charged with devising ways to improve the OJT program should address this. As 

part of its action plan to resolve this recommendation, TSA should provide us 

with its revised OJT checklists. Recommendation 12 is resolved – open. 

Page 100 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix B 

OIG Evaluation of Management Comments 

Recommendation 13: Revise OJT examinations in order to: 

• Test screeners on all TSA-approved screening methods that they will be 

required to use upon completion of OJT; 

• Enforce the use of tests with explosive material during ETD practical 

examinations; and 

• Standardize and enforce limits on OJT retesting opportunities, including 

the Image Mastery Test. 

TSA Response: The current OJT checklists serve as examinations on the 

procedures outlined in the screening SOPs. Screener Training Division’s Project 

Team for OJT program improvements will address improvements to the OJT 

examinations. Improvements include working with the Transportation Security 

Lab to develop a reliable method for testing with explosive material during ETD 

practical examinations. TSA also noted that its Guidance for Administering 

Security Screener On-the-Job Training, version 3.5, limits retesting opportunities 

to two. 

OIG Evaluation: TSA’s comments are responsive to the first two parts of the 

recommendation. We modified the second part of the draft recommendation 

to account for TSA’s plan to develop and use an alternative explosive material 

during ETD examinations, as the current material, i.e., dry transfer strips, is not 

sufficiently reliable. Regarding the third part of the recommendation, TSA’s 

response establishes a standardized limit on OJT retesting opportunities. Our 

concern is that the same limit existed previously and was not enforced, thereby 

allowing some FSDs to grant screeners as many as eight opportunities to take the 

IMT. In its action plan, TSA should clarify how it plans to monitor and enforce 

the retesting limits in the Guidance for Administering Security Screener Onthe-Job Training. TSA also should provide copies of the revised OJT checklists 

and an update on the Transportation Security Lab’s progress in developing an 

alternative explosive test material for ETD examinations. Recommendation 13 is 

resolved – open. 

Recommendation 14: Ensure OJT monitors are certified as having the skills 

and experience necessary to deliver and monitor OJT training and administer 

subsequent testing. 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 101 
Appendix B 

OIG Evaluation of Management Comments 

TSA Response: TSA is analyzing the feasibility of implementing a formalized 

OJT program for OJT monitors. In addition, this recommendation is one of the 

improvements to the OJT program that TSA’s Project Team will address. 

OIG Evaluation: We are encouraged that TSA is analyzing the feasibility of 

implementing a formalized OJT program for OJT monitors. The implementation 

of an effective OJT program for OJT monitors would be fully responsive to our 

recommendation. Recommendation 14 is resolved – open. 

Recommendation 15: Examine the workforce implications of the three-hour 

training requirement and take steps to correct identified imbalances in future 

workforce planning to ensure that all screeners are able to meet the recurrent 

training standard. 

TSA Response: TSA summarized its “Interim Screener Recurrent Training 

Program Guidance.” TSA recounted the methodology for measuring and logging 

training hours, the types of available recurrent training, and its expectations 

for FSDs to develop and manage training plans that meet both the three-hour 

requirement for recurrent training and screener developmental needs. 

OIG Evaluation: TSA’s comments are not responsive to our recommendation 

because they do not indicate whether TSA’s workforce plans ensure that an 

additional three-hour demand on screeners’ workweek is feasible. TSA’s history 

of logging extensive screener overtime hours and comments from TSA field staff 

suggest that the demand may not be feasible at current staffing levels. We are 

particularly concerned for part-time screeners, who also must meet the threehour requirement but who typically are not scheduled to work during periods 

of low passenger volume when airports might conduct screening and training 

concurrently. During the course of our review, we made four requests for a copy 

of TSA’s 2004 workforce plan in order to evaluate the extent to which the plan 

accounted for the recurrent training requirement. In its action plan, TSA should 

provide a copy of the workforce plan and an explanation of how the recurrent 

training requirement affected or will affect planned staffing levels. Other steps 

that TSA could take toward resolving this recommendation include providing 

statistics from the Online Learning Center regarding the extent to which airports 

have been able to log the required three hours per screener. Recommendation 15 

is unresolved. 

Page 102 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix B 

OIG Evaluation of Management Comments 

Recommendation 16: Continue to pursue the development and application of TIP 

user adaptability features to maximize TIP training benefits. 

TSA Response: TSA is finalizing the Functional Requirements for the Second 

Generation TIP System for x-ray machines that is envisioned to include an 

adaptive learning capability. Upon finalization of requirements, TSA will work 

with TRX vendors to analyze the feasibility of development and implementation. 

OIG Evaluation: TSA’s efforts are responsive to this recommendation. We are 

hopeful that TSA’s analysis will result in the actual development and application 

of an adaptive learning capability to TIP. Recommendation 16 is resolved – open. 

Recommendation 17: Expedite TRX connectivity to realize administrative and 

information sharing gains related to TIP. 

TSA Response: TSA is currently conducting a TRX/TIP Network Pilot Program 

at five airports. This pilot is designed to connect existing TRX machines and the 

National TIP Server to the TSA Network, test and evaluate the TIP Network, 

standardize a networking and connectivity solution that can be deployed to 

additional airports, and lay the groundwork for continued screening performance 

improvement. In addition, the pilot provides a foundation for TSA to develop the 

Civil Aviation Security Screening Network (CASSNET) which will provide the 

next generation of connectivity for airport security equipment including ETD, 

TIP-ready x-rays, EDS, and WTMD. 

OIG Evaluation: We are encouraged by the TRX/TIP Network Pilot Program 

currently being conducted and are hopeful that connectivity and networking 

at all other airports is achieved in the near future. Recommendation 17 is 

resolved — open. 

Recommendation 18: Further enhance local operational testing efforts by: 

(1) revising procedures and protocols to increase opportunities for realistic and 

difficult testing, and (2) expediting the development, certification, and distribution 

of ETD and EDS operational testing tools to enable assessment of screeners 

performing these duties. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 103 
Appendix B 

OIG Evaluation of Management Comments 

TSA Response: TSA will study the addition of more difficult types of operational 

tests, including the use of threat objects that are artfully concealed and the 

placement of threat objects on sensitive areas. Other efforts already taken by TSA 

to increase the realism and difficulty of local operational testing include the use of 

“bogus” boarding passes from aircraft operators so that the individual carrying a 

test object into the checkpoint appears to be a regular passenger and the proposal 

of adding new test items to the MBS-2 and Weapons kits. In addition, TSA is 

testing a new ETD training and testing aid that will be used to assess screener 

trace sampling skills and is beginning to develop simulated items for EDS that 

can be inserted into the testing and training kits. 

OIG Evaluation: We are hopeful that TSA’s study will result in the permission 

to use threat objects that are artfully concealed in baggage and placed on sensitive 

areas of the body during local operation testing. In addition, we are encouraged 

by the current testing and development of ETD and EDS operational testing tools. 

Recommendation 18 is resolved – open. 

Recommendation 19: Fund and resume installation of computer training labs and 

high-speed network connectivity to provide all screeners with full access to the 

Online Learning Center. 

TSA Response: The TSA Chief Information Officer has a comprehensive plan 

to implement high-speed connectivity at more than 500 screener and support 

staff sites. To date, TSA has established high-speed connectivity primarily for 

FSD office locations. Further progress is dependent on funding availability. TSA 

asserted that funding the network installations is a high-priority issue for TSA and 

that TSA has the support of the DHS CIO Council. 

OIG Evaluation: We recognize the efforts of TSA’s Office of Information 

Technology in planning extensively for the implementation of high-speed 

connectivity at field sites. We also acknowledge that TSA has many competing 

demands for its resources. Choosing to fund the installation of high-speed 

connectivity at all screener sites will enable TSA to use the advanced training 

features in the Online Learning Center, plus it will decrease the clerical burden 

on training coordinators who must log training by hand. We encourage TSA 

to continue to improve connectivity at field sites. Recommendation 19 is 

resolved – open. 

Page 104 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix B 

OIG Evaluation of Management Comments 

Recommendation 20: Ensure that screeners are scheduled for basic classroom 

training that provides initial certification on the specific make and model of ETD 

and EDS machines that they operate. 

TSA Response: TSA does not concur with the recommendation. TSA stated that 

all screeners do receive initial training and certification on any specific make and 

model of equipment they operate. Due to logistical and resource issues, TSA does 

not believe it is practical or feasible to require annual re-certification on particular 

makes and models of machines. 

OIG Evaluation: While TSA policy may require certification for each make 

and model of equipment a screener operates, our field observations were that the 

policy was not consistently followed. In cases where screeners received formal 

classroom training on equipment models inappropriate for their duty assignment, 

the screeners may later have received “differences” training for the appropriate 

models at their airports. However, machine certification should be completed 

in a controlled classroom setting, not an active checkpoint. It is important that 

screeners have a comprehensive understanding and familiarity with the specific 

types of machines they will be operating before they do so in a live screening 

environment during OJT. The risks of improperly screening carry-on items and 

checked baggage, damaging expensive machinery, and creating screening delays 

are too great to permit machine certification to occur after screeners are working 

in a live setting. 

Given the current structure of TSA’s annual screener re-certification process, we 

accept TSA’s explanation that machine make- and model-specific re-certification 

would not be feasible at all airports. Therefore, we modified our recommendation 

to exclude an annual re-certification requirement on specific makes and models of 

ETD and EDS equipment. Nevertheless, because TSA has made no commitment 

to certify screeners on specific machine makes and models during initial 

classroom training, Recommendation 20 is unresolved. 

Recommendation 21: Improve management controls for the screener scheduling 

system, such as linking scheduling to employee qualifications in the Online 

Learning Center, to ensure that TSA schedules screeners to operate only 

equipment on which they are certified. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 105 
Appendix B 

OIG Evaluation of Management Comments 

TSA Response: TSA agreed that the Online Learning Center is the official 

repository of employee certifications data and that it could be integrated with the 

Sabre scheduling tool through the proposed Office of Information Technology 

Integration Services program. 

OIG Evaluation: TSA staff envisioned enabling the Online Learning Center 

and Sabre scheduling tool to share data as a long-range goal. In its action 

plan, TSA should include an update on progress toward the data integration. 

Recommendation 21 is resolved – open. 

Recommendation 22: Continue efforts toward the development and advancement 

of technologies to support screening efforts. Resource investment should place 

particular emphasis on technologies for passenger checkpoint screening, as 

passenger screening procedures are more operator dependent and, thus, more 

vulnerable to human factors than checked baggage screening. 

TSA Response: TSA agrees that efforts toward the development and 

advancement of security screening technologies, with an emphasis on passenger 

checkpoint technologies, should be continued. TSA is conducting pilot programs 

to test technologies that will identify explosives carried on a person as well as 

on documents, such as boarding passes. In addition, an explosives detection 

technology prototype that will act as an automated certified EDS for carryon baggage is being tested and will be an adjunct system to the current x-ray 

technology at screening checkpoints. 

OIG Evaluation: TSA’s response demonstrates a continued commitment to 

pursue needed technological development and deployment efforts. Recent 

pilot testing of new tools for passenger screening are directly responsive to this 

recommendation. Recommendation 22 is closed. 

Page 106 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix C 

Recommendations 

Recommendations 

Recommendation 1: Complete the analysis for basic screener classroom training, 

both for passenger checkpoint and checked baggage screeners, and institute 

mechanisms to ensure that staff update the analysis at least annually and apply the 

results to curriculum revisions. 

Recommendation 2: Ensure that passenger checkpoint and checked baggage 

basic course objectives (1) address the knowledge and skills necessary for routine 

screener performance, as identified by the most current task analysis, and (2) are 

presented to students at the beginning of course lessons. 

Recommendation 3: Further revise written examinations and administration 

procedures, including the following steps: 

• Ensure all course objectives receive an appropriate assessment of student 

learning; 

• Thoroughly validate tests, including passing scores, if they are to be used 

for selection decisions; 

• Pilot-test examination questions to ensure their clarity and objectivity; 

• Schedule tests later during the course to enhance their ability to measure 

students’ long-term retention of course material; and 

• Incorporate post-test reviews to prevent screener misconceptions from 

being carried into the workplace. 

Recommendation 4: Develop and distribute detailed test administration 

guidelines for Practical Demonstration of Proficiency Evaluations with the aim of 

increasing standardization. 

Recommendation 5: Distribute effective training materials to reinforce learning 

and allow for individual study outside the classroom. 

Recommendation 6: Require all TSA approved instructors to complete an 

appropriate form of OJT prior to providing instruction and to undergo annual 

recertification testing. 

An Evaluation of TSA’s Screener Training and Methods of Testing		 Page 107 
Appendix C 

Recommendations 

Recommendation 7: Analyze the cost and feasibility of establishing designated 

training sites where screening equipment, simulators, and training aids can be 

permanently located and readily available to students. 

Recommendation 8: Use alternatives to lectures more frequently during 

classroom training to help maintain student attentiveness, contribute to the 

comprehension and mastery of new knowledge and skills, and foster retention of 

the material. 

Recommendation 9: Cease all coaching of students during practical 

demonstration of proficiency evaluations. 

Recommendation 10: Ensure that leadership within the screener workforce 

diligently monitors screeners and immediately corrects identified failures to 

adhere to screening procedures and negligent screening techniques. 

Recommendation 11: Continue the development of evaluations that will relate 

training to screener application of learning on the job and to organizational 

results. 

Recommendation 12: Ensure that OJT guidance provides detailed, sequenced 

lists of specific tasks that each OJT student must perform, including instructional 

materials where needed, and establishes appropriate, function-specific time 

requirements for both newly hired and cross-trained screeners. 

Recommendation 13: Revise OJT examinations in order to: 

• Test screeners on all TSA-approved screening methods that they will be 

required to use upon completion of OJT; 

• Enforce the use of tests with explosive material during ETD practical 

examinations; and 

• Standardize and enforce limits on OJT retesting opportunities, including 

the Image Mastery Test. 

Recommendation 14: Ensure OJT monitors are certified as having the skills 

and experience necessary to deliver and monitor OJT training and administer 

subsequent testing. 

Page 108		 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix C 

Recommendations 

Recommendation 15: Examine the workforce implications of the three-hour 

training requirement and take steps to correct identified imbalances in future 

workforce planning to ensure that all screeners are able to meet the recurrent 

training standard. 

Recommendation 16: Continue to pursue the development and application of TIP 

user adaptability features to maximize TIP training benefits. 

Recommendation 17: Expedite TRX connectivity to realize administrative and 

information sharing gains related to TIP. 

Recommendation 18: Further enhance local operational testing efforts by: (1) 

revising procedures and protocols to increase opportunities for realistic and 

difficult testing, and (2) expediting the development, certification, and distribution 

of ETD and EDS operational testing tools to enable assessment of screeners 

performing these duties. 

Recommendation 19: Fund and resume installation of computer training labs and 

high-speed network connectivity to provide all screeners with full access to the 

Online Learning Center. 

Recommendation 20: Ensure that screeners are scheduled for basic classroom 

training that provides initial certification on the specific make and model of ETD 

and EDS machines that they operate. 

Recommendation 21: Improve management controls for the screener scheduling 

system, such as linking scheduling to employee qualifications in the Online 

Learning Center, to ensure that TSA schedules screeners to operate only 

equipment on which they are certified. 

Recommendation 22: Continue efforts toward the development and advancement 

of technologies to support screening efforts. Resource investment should place 

particular emphasis on technologies for passenger checkpoint screening, as 

passenger screening procedures are more operator dependent and, thus, more 

vulnerable to human factors than checked baggage screening. 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 109 
 

 

 

Appendix D
	

TSA’s Interim Screener Recurrent Training Program Guidance—December 22, 2003
	

Page 110 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix D
	

TSA’s Interim Screener Recurrent Training Program Guidance—December 22, 2003
	

An Evaluation of TSA’s Screener Training and Methods of Testing Page 111
	
Appendix D 

TSA’s Interim Screener Recurrent Training Program Guidance—December 22, 2003 

Page 112 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix D
	

TSA’s Interim Screener Recurrent Training Program Guidance—December 22, 2003
	

An Evaluation of TSA’s Screener Training and Methods of Testing Page 113
	
Appendix D
	

TSA’s Interim Screener Recurrent Training Program Guidance—December 22, 2003
	

Page 114 An Evaluation of TSA’s Screener Training and Methods of Testing 
Appendix E 

Major Contributors to this Report 

Melissa M. Howard, Chief Inspector, Department of Homeland Security, Office of 

Inspections, Evaluations, and Special Reviews 

Meredith L. Megles, Senior Inspector, Department of Homeland Security, Office 

of Inspections, Evaluations, and Special Reviews 

E. Wynne Krause, Inspector, Department of Homeland Security, Office of 

Inspections, Evaluations, and Special Reviews 

Justin H. Brown, Inspector, Department of Homeland Security, Office of 

Inspections, Evaluations, and Special Reviews 

Jennifer A. Lindsey, Inspector, Department of Homeland Security, Office of 

Inspections, Evaluations, and Special Reviews 

An Evaluation of TSA’s Screener Training and Methods of Testing Page 115 
Appendix F 

Report Distribution 

Department of Homeland Security 

Secretary 

Deputy Secretary 

Chief of Staff 

General Counsel 

Under Secretary, Border and Transportation Security 

DHS OIG Liaison 

DHS Public Affairs 

DHS Deputy Chief Security Officer 

Transportation Security Administration 

Assistant Secretary of Homeland Security for the Transportation Security 

Administration 

Director, Workforce Performance and Training 

Director, Aviation Operations 

Director, Human Resources 

Director, Information Technology 

Director, Internal Affairs and Program Review 

Director, Security Technology 

Director, Operations Policy 

DHS OIG Audit Liaison, Office of Transportation Security Policy 

Office of Management and Budget 

Chief, Homeland Security Branch 

DHS OIG Program Examiner 

Congress 

Congressional Oversight and Appropriations Committees 

Page 116 An Evaluation of TSA’s Screener Training and Methods of Testing 

Additional Information and Copies 

To obtain additional copies of this report, call the Office of Inspector General (OIG) at 

(202) 254-4100, fax your request to (202) 254-4285, or visit the OIG web site at www. 

dhs.gov/oig. 

OIG Hotline 

To report alleged fraud, waste, abuse or mismanagement, or any other kind of criminal 

or noncriminal misconduct relative to department programs or operations, call the OIG 

Hotline at 1-800-323-8603; write to Department of Homeland Security, Washington, DC 

20528, Attn: Office of Inspector General, Investigations Division – Hotline. The OIG 

seeks to protect the identity of each writer and caller.