Department of Homeland Security

Office of Inspector General 

Audit of the Effectiveness of the 


Checked Baggage Screening System and Procedures 


Used to Identify and Resolve Threats 


(Unclassified Summary)
 

OIG-09-42 March 2009 
 

Office of Inspector General 

U.S. Department of Homeland Security

Washington, DC 20528 

March 17, 2009 

Preface 

The Department of Homeland Security (DHS) Office of Inspector General (OIG) was 

established by the Homeland Security Act of 2002 (Public Law 107-296) by amendment 

to the Inspector General Act of 1978. This is one of a series of audit, inspection, and 

special reports prepared as part of our oversight responsibilities to promote economy, 

efficiency, and effectiveness within the department. 

This report addresses the strengths and weaknesses of the Transportation Security 

Administration’s procedures, equipment, and supervision to ensure that Transportation 

Security Officers are able to prevent threat items from being introduced into the checked 

baggage systems of the nation’s airports. It is based on interviews with Transportation 

Security Officers and Transportation Security Administration officials, direct 

observations, field testing, and a review of applicable documents. 

The recommendations herein have been developed to the best knowledge available to our 

office, and have been discussed in draft with those responsible for implementation. We 

trust this report will result in more effective, efficient, and economical operations. We 

express our appreciation to all of those who contributed to the preparation of this report. 

Richard L. Skinner 


Inspector General 

OIG
 

Department of Homeland Security 

Office of Inspector General 

Background 

The Transportation Security Administration (TSA) is responsible 

for overseeing aviation security and ensuring the safety of the air 

traveling public. This includes the screening of checked baggage 

and safeguarding it from prohibited items. We conducted 

unannounced, covert testing to evaluate how effective the OnScreen Alarm Resolution Protocol (OSARP) certified operators are 

at determining whether an object causing an explosives detection 

system to alarm poses a serious threat and how those threats are 

being resolved. 

We conducted unannounced, covert testing at eight domestic 

airports, from April 2008 through June 2008. Because TSA has 

deployed explosives trace detection machines at most of the major 

domestic airports for operational use, we included locations that 

were using these technologies in conjunction with trained 

Transportation Security Officers (TSOs) utilizing the OSARP 

when we made our airport selections. 

Our purpose in testing was to determine whether: (1) TSA’s 

implementation of OSARP is effective; (2) TSA’s checked 

baggage screening procedures and standards are adequate; 

(3) TSOs are following those screening procedures and standards; 

and (4) aviation security screening equipment and technologies are 

functioning properly and as intended to prevent threat items from 

entering the checked baggage system. 

Audit of the Effectiveness of the Checked Baggage Screening System and Procedures Used to Identify and Resolve Threats 

 Page 1 
Results of Audit 

The number of tests conducted, the names of the airports tested, 

and the quantitative and qualitative results of our testing are 

classified. We have shared that information with the department, 

the Transportation Security Administration, and appropriate 

congressional committees. 

We identified vulnerabilities in the security of checked baggage at 

the eight domestic airports where we conducted testing. As a 

result of our testing, we made four recommendations. TSA 

concurred with three of the four recommendations and concurred 

in part with the other recommendation. When fully implemented, 

these recommendations should strengthen the checked baggage 

screening process. 

We gave a copy of our draft report to TSA for review and 

comment. The unclassified portion of TSA’s response to our draft 

report is summarized below. 

TSA’s Response 

TSA appreciates the work done by OIG on this engagement, and 

can generally validate OIG’s data against TSA’s own covert 

testing programs when TSA analysis rules are applied. While we 

disagree over OIG’s characterizations of Explosives Detection 

System (EDS) performance and have excluded them from our 

analysis, the remainder of OIG’s data is in line with TSA test 

results in both agency-run covert testing programs. 

OIG’s data continues to demonstrate the effectiveness of TSA’s 

checked baggage security system in detecting and removing threat 

objects. 

TSA Covert Assessment Programs Have Similar Results 

Aviation Screening Assessment Program 

TSA’s Aviation Screening Assessment Program (ASAP) is 

designed to measure screening performance using a realistic 

standardized test scenario framework. Through ASAP, TSA 

conducts covert testing of the TSA screening process. The primary 

purpose is to provide statistical data to identify points of and 

reasons for failures within the checkpoint and checked baggage 

screening process. The data generated from ASAP testing is used 

Audit of the Effectiveness of the Checked Baggage Screening System and Procedures Used to Identify and Resolve Threats 

Page 2 
to make national recommendations to improve screening 

performance. 

TSA Office of Inspection Testing 

TSA’s Office of Inspection has a very robust testing program 

designed to identify systemic vulnerabilities in transportation 

security systems. Through the Office of Inspection testing 

program, subject matter experts develop and test specific 

hypotheses regarding potential system vulnerabilities. These tests 

are not designed to be performance measures. Rather, they are 

evaluations of system vulnerabilities that can be used to design 

countermeasures. When viewed in this light, the qualitative results 

from these experiments are highly valuable in analyzing 

vulnerabilities, with conclusions from these experiments informing 

decisions at the strategic level. 

TSA appreciates the work done by OIG on this engagement, which 

further validates TSA’s own testing of the checked baggage 

screening system effectiveness. 

Audit of the Effectiveness of the Checked Baggage Screening System and Procedures Used to Identify and Resolve Threats 

Page 3 
Appendix A 

Major Contributors to this Report 

Patrick O’Malley, Director, Transportation Security Division 

James Yeager, Program Manager 

Cecilia Barela, Analyst-in-Charge 

Sharon Trodden, Senior Auditor 

Danielle de Sanctis, Program Analyst 

Barry Cleveland, Program Analyst 

Vanessa Santos, Independent Referencer 

Audit of the Effectiveness of the Checked Baggage Screening System and Procedures Used to Identify and Resolve Threats 

 Page 4 
Appendix B 

Report Distribution 

Department of Homeland Security 

Secretary 

Acting Deputy Secretary 

Chief of Staff for Operations 

Chief of Staff for Policy 

Acting General Counsel 

Executive Secretary 

Director, GAO/OIG Liaison Office 

Assistant Secretary for Office of Policy 

Assistant Secretary for Office of Public Affairs 

Assistant Secretary for Office of Legislative Affairs 

Acting Assistant Secretary, Transportation Security Administration 

DHS OIG Liaison, TSA 

Office of Management and Budget 

Chief, Homeland Security Branch 

DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriations Committees, as appropriate 

Audit of the Effectiveness of the Checked Baggage Screening System and Procedures Used to Identify and Resolve Threats 


Page 5
 
ADDITIONAL INFORMATION AND COPIES 

To obtain additional copies of this report, please call the Office of Inspector General (OIG) at (202) 254-4199, 

fax your request to (202) 254-4305, or visit the OIG web site at www.dhs.gov/oig. 

OIG HOTLINE 

To report alleged fraud, waste, abuse or mismanagement, or any other kind of criminal or noncriminal 

misconduct relative to department programs or operations: 

• Call our Hotline at 1-800-323-8603; 

• Fax the complaint directly to us at (202) 254-4292; 

• Email us at DHSOIGHOTLINE@dhs.gov; or 

• Write to us at: 

DHS Office of Inspector General/MAIL STOP 2600, 

Attention: Office of Investigations - Hotline, 

245 Murray Drive, SW, Building 410, 

Washington, DC 20528. 

The OIG seeks to protect the identity of each writer and caller.