TSA Needs to Improve 

Monitoring of the 

Deployed Advanced 

Imaging Technology 

System 

May 8, 2020 

OIG-20-33 
 

 

 

 

 

 

 

 

DHS OIG HIGHLIGHTS
 

TSA Needs to Improve Monitoring of the

Deployed Advanced Imaging Technology System
 

May 8, 2020 

Why We Did 

This Audit 

Prior audit reports from both 

the U.S. Government 

Accountability Office and 

DHS OIG highlighted 

diminished detection 

performance and inefficient 

screening capabilities of 

TSA’s AIT system. While TSA 

committed to addressing 

these gaps, new threats 

continue to emerge requiring 

advanced security measures. 

We conducted this audit to 

determine to what extent 

TSA’s monitoring of the AIT 

addresses needed 

capabilities. 

What We 

Recommend 

We made two 

recommendations designed 

to improve TSA’s monitoring 

of the AIT system. 

For Further Information: 

Contact our Office of Public Affairs at 

(202) 981-6000, or email us at 

DHS-OIG.OfficePublicAffairs@oig.dhs.gov 

What We Found 

The Transportation Security Administration (TSA) does not 

monitor the Advanced Imaging Technology system (AIT) to 

ensure it continues to fulfill needed capabilities for 

detecting non-metallic threat items concealed on air 

passengers. Specifically, although AIT met the 

requirement for system availability, TSA did not monitor 

the AIT system’s probability of detection rate and 

throughput rate requirements set forth in TSA’s 

operational requirements document. These issues 

occurred because TSA has not established comprehensive 

guidance to monitor performance of the AIT system. 

Without continuous monitoring and oversight, TSA cannot 

ensure AIT is meeting critical system performance 

requirements — a consistent weakness found in prior DHS 

Office of Inspector General (OIG) reports. As a result, TSA 

may fund and acquire future systems without knowing 

whether the technology addresses new and emerging 

threats, potentially placing the traveling public at risk. 

TSA Response 

TSA concurred with our recommendations. We have 

included a copy of TSA’s response to our draft report in 

appendix A. We consider both recommendations resolved 

and open. 

www.oig.dhs.gov OIG-20-33 
0D\
 
 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Background
 

The Transportation Security Administration (TSA) is responsible for conducting 

passenger checkpoint and checked baggage screening operations at roughly 

450 airports nationwide. To achieve its mission, TSA mitigates threats to 

aviation security through the deployment of advanced technologies. 

In 2007, TSA piloted the Advanced Imaging 

Technology system (AIT) to address a 

critical weakness in security at passenger 

screening checkpoints. TSA began using 

the AIT to detect non-metallic threats such 

as weapons, explosives, and other items 

concealed on passengers that may not be 

identified by a walk-through metal 

detector. Currently, AIT is the primary onperson screening device at passenger 

screening checkpoints in 340 airports 

nationwide. TSA has deployed 962 AIT 

systems totaling an estimated $126 

million.1 Figure 1 shows an AIT deployed 

at an airport checkpoint. 

Prior to deploying AIT, TSA tested the system in both a controlled environment 

and at selected airports. The purpose of the test and evaluation process was to 

certify the AIT system met all operational needs as intended. TSA defined these 

detection and performance needs in its Operational Requirements Document for 

Second-Generation Advanced Imaging Technology System for Checkpoint 

Operations. This document outlined four performance metrics the AIT system 

must achieve to be fully operational: 

x Probability of Detection - AIT must identify the location of a concealed 

threat in any orientation on a passenger, at or greater than the defined 

detection rate value. TSA considers the detection rate classified. 

x Throughput - AIT must screen an average of 150 persons per hour 

during continuous use. 

x Availability - AIT shall maintain an availability rate of 98.5 percent based 

on a 16-hour day of operation. 

x Safety - AIT shall not present electrical, radiation, tripping, bodily injury, 

or other hazards to passengers and operators due to sharp corners or 

edges. It also must not have adverse impact on electronic medical 

devices such as pacemakers. We did not include safety in the scope of 

our review. 

1 Calculation is based on the average procurement unit cost for AIT in TSA’s 2017 Life Cycle 

Cost Estimate for Passenger Screening Program. 

Figure 1: TSA AIT System 

Source: TSA.gov 

www.oig.dhs.gov 2 OIG-20-33 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

On September 19, 2017, DHS’ Acquisition Review Board declared the AIT 

system at full operational capability2 and approved it to move forward as a 

standalone program in January 2018. As a standalone program, TSA is 

required to oversee all operational activities of the program as described in the 

DHS Systems Engineering Life Cycle Guidebook.3 These activities include 

operating and maintaining the system, performing operational analyses, 

conducting post-implementation reviews, and periodically reviewing the 

system’s performance. 

The Performance Management Branch, under TSA’s Security Operations 

Division, oversees AIT deployed to passenger screening lanes. PMB’s main 

oversight responsibility is to maximize use of the AIT system. In March 2018, 

PMB developed a field guide, Improving Security Effectiveness by Optimizing 

Utilization of Advanced Imaging Technology, aimed at improving security 

effectiveness by optimizing use of AIT. The field guide provides throughput 

calculations and target metrics for determining the operational capacity of the 

AIT system. PMB collects passenger throughput data and develops reports for 

TSA managers responsible for monitoring AIT performance and implementing 

corrective action. 

Over the years, TSA has received a high degree of scrutiny regarding 

effectiveness of the AIT system. Prior audit reports from both the U.S. 

Government Accountability Office and DHS Office of Inspector General (OIG) 

highlighted diminished detection performance and inefficient screening 

capabilities of the AIT system. While TSA has committed to addressing these 

gaps, new threats continue to emerge requiring advanced security measures. 

We conducted this audit to determine to what extent TSA’s monitoring of the 

AIT addresses needed capabilities. 

TSA Does Not Monitor AIT Operational Requirements 

TSA does not monitor the AIT system to ensure it continues to fulfill needed 

capabilities for detecting non-metallic threat items concealed on air 

passengers. Specifically, although AIT met the requirement for system 

availability, TSA did not monitor the AIT system’s probability of detection rate 

and throughput rate requirements set forth in TSA’s operational requirements 

document. These issues occurred because TSA has not established 

comprehensive guidance to monitor performance of the AIT system. 

Without continuous monitoring and oversight, TSA cannot ensure AIT is 

meeting critical system performance requirements — a consistent weakness 

2 An acquisition reaches full operational capability when there is a full deployment of capability
 

and when the requirements established in the operational requirements document are 

demonstrated through test and evaluation.
 3 DHS Guidebook, 102-01-103-01, Systems Engineering Life Cycle Guidebook, April 18, 2016.
 

www.oig.dhs.gov 3 OIG-20-33 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

found in prior DHS OIG reports. As a result, TSA may fund and acquire future 

systems without knowing whether the technology addresses new and emerging 

threats, potentially placing the traveling public at risk. 

TSA Does Not Monitor the Probability of Detection 

Standards for Internal Control in the Federal Government calls for agencies to 

continually monitor programs during the course of normal operations to help 

evaluate performance over time. TSA’s operational requirements document 

requires the AIT system to identify the location of a concealed threat at, or 

greater than, the (classified) defined detection rate. Although TSA certified the 

AIT system met the probability of detection rate during the testing and 

evaluation process, TSA has not conducted any further assessments of the 

system’s performance since deploying AIT to passenger screening lanes. The 

test and evaluation phase certifies the system, at a minimum, meets threshold 

detection requirements at the time of the test. However, this certification does 

not ensure the system will continue to perform at the same rate once deployed. 

As the AIT system ages, it is possible for performance to degrade over time. 

Currently, TSA does not require periodic assessments of AIT to ensure it 

operates as intended and identify whether the system continues to detect 

concealed threats at the rate it did when the system was first tested prior to 

deployment. 

While TSA does conduct other assessments, these assessments do not ensure 

the AIT system continues to perform as intended. For instance, Transportation 

Security Officers calibrate the AIT system before each shift by placing a test 

object in the machine to confirm it correctly scans and displays the object on 

the image screen. Although daily calibration demonstrates the AIT system is 

operational, it does not determine the probability of detection rate. 

TSA’s Office of Inspection also conducts covert tests to identify vulnerabilities 

in the passenger screening program. Covert testing includes on-body 

concealment of threat items designed to test and identify vulnerabilities in the 

security systems and identify corrective actions. Covert tests are conducted 

periodically for all types of technologies at the passenger screening checkpoint, 

and in only some cases include the AIT system as part of the test. However, 

these tests are not specifically designed with AIT detection standards in mind 

and are not a substitute for continual monitoring AIT detection requirements. 

Further, airports conduct localized covert tests to assess the Transportation 

Security Officer’s ability to resolve an AIT’s alarm and not the performance of 

the system itself. TSA’s Office of Inspection has not reviewed the AIT system 

detection standards as part of its covert testing in over 5 years. 

www.oig.dhs.gov 4 OIG-20-33 
 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

TSA Does Not Monitor Throughput Requirements 

According to the operational requirements document for the system, the AIT 

system must be able to screen an average of 150 passengers per hour. 

Transportation Security Officers manually enter daily AIT throughput data into 

the Performance Management Information System. TSA uses the information 

to generate AIT utilization reports to measure throughput performance. 

According to TSA’s field guide, Improving Security Effectiveness by Optimizing 

Utilization of Advanced Imaging Technology, the national target for AIT 

utilization is greater than 95 percent. However, we found TSA’s Federal 

Security Directors (FSD), responsible for monitoring throughput performance, 

do not always review utilization reports or implement corrective action when 

airports do not comply with the set target.4 Specifically, we contacted 10 FSDs 

with airports below target during an 18-month timeframe. We determined 3 of 

10 (30 percent) FSDs were unaware their airports fell below the AIT utilization 

target and 10 of 10 (100 percent) could not provide any examples of corrective 

actions taken. Further, 5 of 10 FSDs responded that managing passenger wait 

times was a larger priority than AIT utilization. 

Further, we determined TSA provided inconsistent oversight of AIT utilization. 

We contacted six Regional Directors responsible for reviewing utilization 

reports and communicating with the FSDs in their regions to determine how 

and when they provided oversight. We received varying responses from six 

Regional Directors who applied AIT utilization targets differently.5 For 

instance, one Regional Director said he reaches out to the FSD if AIT utilization 

falls below 75 percent, while another has weekly meetings with FSDs but 

applies a goal of 87 percent. Three Regional Directors responded they review 

utilization reports and reach out on an “as-needed” basis. The remaining 

Regional Director we interviewed said he reviews utilization reports daily and 

reaches out to FSDs if AIT utilization falls below 95 percent. These 

inconsistent targets do not hold FSDs accountable for meeting AIT utilization 

targets and prevent corrective action to ensure the AIT is meeting critical 

system performance requirements. 

TSA Has Not Developed Policies and Procedures to Monitor Requirements 

TSA has not established comprehensive guidance to ensure adequate 

monitoring of AIT system performance. According to Standards for Internal 

Control in the Federal Government, agencies should periodically review policies 

for continued relevance and update them as needed. Although TSA established 

the field guide for optimizing the use of the AIT, the field guide only focuses on 

utilization targets and reporting requirements. The field guide provides 

4 The Federal Security Director is responsible for security operations at each federalized airport. 

5 TSA’s national operations are divided into seven geographic regions. TSA assigns a Regional 

Director to oversee the FSDs in each region. 

www.oig.dhs.gov 5 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

throughput calculations and target metrics for determining the operational 

capacity of the AIT system. The Performance Management Branch collects 

passenger throughput data and develops reports for TSA managers responsible 

for monitoring AIT performance and implementing corrective action. However, 

the field guide does not require that TSA leadership regularly review these 

reports to make informed decisions on the use of the AIT machine. In addition, 

the field guide does not require TSA to conduct any follow-up or take corrective 

action once TSA leadership determines an airport has not met its target 

utilization rate. 

TSA also lacks requirements for ensuring airports continuously monitor AIT 

detection rates. As of August 2019, TSA was in the process of developing draft 

procedures for how to monitor detection requirements. However, TSA officials 

we spoke with identified challenges in designing such procedures since it is not 

feasible to use live explosives to test AIT at airports. While using live explosives 

may not be possible, TSA could consider other measures such as test articles 

manufactured in a TSA laboratory. Further, TSA should ensure the new policy 

not only includes procedures for monitoring detection rates, but also 

requirements to monitor AIT throughput to ensure TSA is carrying out ongoing 

monitoring and implementing corrective actions. 

Since 2007, TSA has invested about $126 million to acquire the AIT system. 

Given the security mission and this significant investment in technology, TSA 

must ensure the AIT system is fulfilling critical system performance 

requirements. Without adequate monitoring and oversight, TSA may continue 

to fund and acquire future systems without knowing whether the technology 

addresses new and emerging threats, potentially placing the traveling public at 

risk. 

Recommendations 

We recommend the TSA Administrator direct: 

Recommendation 1: The Acquisition Program Management Assistant 

Administrator to develop an effective monitoring strategy and implementation 

plan, including necessary policies and procedures for ensuring probability of 

detection through defined continued monitoring and assessment of the 

deployed AIT system. 

Recommendation 2: The Security Operations Executive Assistant 

Administrator to develop an effective monitoring strategy and implementation 

plan, including necessary policies and procedures for managing AIT 

throughput, to ensure TSA is carrying out ongoing monitoring and 

implementing corrective actions. 

www.oig.dhs.gov 6 OIG-20-33 
 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Management Comments and OIG Analysis 

TSA concurred with the recommendations. Appendix A contains a copy of 

TSA’s management comments in their entirety. We also received technical 

comments from TSA and made revisions to the report where appropriate. We 

consider the two recommendations resolved and open. A summary of TSA’s 

responses and our analysis follows. 

TSA Response to Recommendation 1: TSA concurred with the 

recommendation. TSA’s Assistant Administrator for Acquisition Program 

Management is drafting guidance for Post Implementation Reviews and 

recurring reviews. When completed, the guidance will provide a strategy for 

conducting performance reviews on transportation security equipment, 

including the AIT system. The guidance will provide instructions for the 

program to use to develop an AIT-specific plan, which will detail the activities 

to monitor and assess the AIT system’s performance. According to TSA, 

completion of the plan is highly dependent on methodologies currently under 

development. The estimated completion date is December 31, 2021. 

OIG Analysis: We consider these actions responsive to the recommendation. 

We will close the recommendation when we receive documentation showing 

TSA implemented guidance for Post Implementation Reviews, and the recurring 

reviews include an AIT-specific plan detailing continuous and consistent 

monitoring and assessment of the AIT system’s performance. The 

recommendation is resolved and open. 

TSA Response to Recommendation 2: TSA concurred with the 

recommendation. TSA’s Assistant Administrator for Aviation Operations will 

set national throughput targets and thresholds for the initiation of corrective 

actions. TSA’s Executive Directors will regularly review performance of FSDs. 

Also, FSDs’ annual performance evaluations will continue to include 

assessments of the FSDs’ management of corrective actions associated with 

unsatisfactory Key Performance Indicators, such as AIT throughput. The 

estimated completion date is December 31, 2020. 

OIG Analysis: We consider these actions responsive to the recommendation. 

We will close the recommendation when we receive documentation showing 

that TSA implemented a monitoring process to ensure ongoing monitoring of 

AIT throughput and implementation of corrective actions. The 

recommendation is resolved and open. 

www.oig.dhs.gov 7 OIG-20-33 
 

 

 

 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Objective, Scope, and Methodology
 

The Department of Homeland Security Office of Inspector General was 

established by the Homeland Security Act of 2002 (Public Law 107−296) by 

amendment to the Inspector General Act of 1978. 

We conducted this audit to determine to what extent TSA’s monitoring of AIT 

addresses needed capabilities. To accomplish our objective, we researched and 

analyzed DHS Instruction 102-01-001; DHS Guidebook, 102-01-103-01; Systems 

Engineering Life Cycle Guidebook; and the U.S. Government Accountability 

Office Standards for Internal Control in the Federal Government. We interviewed 

key personnel and obtained operational requirements documentation related to 

the AIT system’s availability, probability of detection, and passenger 

throughput. We did not include safety in the scope of our review. 

To understand how TSA established operational requirements for the AIT 

program, we conducted interviews with staff at TSA’s program offices 

responsible for developing the AIT program requirements. We obtained 

relevant acquisition documents, including – 

x Capability Analysis Report for Screening Traveler’s/Non-Traveling 

Individual’s Person; 

x Passenger Screening Program Mission Need Statement; 

x Passenger Screening Program Acquisition Plan; 

x Concept of Operations for Second-Generation Advanced Imaging 

Technology-2 System for Checkpoint Operations; 

x Operational Requirements Document for Second-Generation Advanced 

Imaging Technology System for Checkpoint Operations; 

x Functional Requirements Document for Second-Generation Advanced 

Imaging Technology System for Checkpoint Operations; 

x Passenger Screening Program Life Cycle Cost Estimate; 

x Advanced Imaging Technology System for Passenger Screening 

Alternatives Analysis; and 

x Passenger Screening Program-Advanced Imaging Technology Updated 

Project Test and Evaluation Master Plan Addendum. 

To assess throughput requirements, we spoke to Federal Security Directors at 

the Boston, Providence, and Manchester-Boston regional airports. We 

judgmentally selected these airports based on proximity to audit team 

locations. We obtained throughput data from the Performance Information 

Management System for fiscal year 2018 through the first two quarters of FY 

2019. 

To assess the reliability of TSA’s Performance Information Management System 

data, we reviewed documentation on how throughput information is submitted, 

www.oig.dhs.gov 8 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

interviewed FSDs responsible for entering throughput data into the 

Performance Measurement Information System, and reviewed the Executive 

Summary of the Verification and Validation of performance measure data as it 

related to the reliability of throughput data. Although we noted some data 

reliability inaccuracies, we determined throughput data was sufficiently reliable 

for the purpose of our audit. 

To evaluate whether TSA is maximizing use of the AIT system, we reviewed and 

analyzed AIT utilization reports to determine the number of airports not 

meeting the national utilization target. We also obtained and reviewed relevant 

guidance on AIT utilization and interviewed all Regional Directors, by phone or 

via email, as well as 10 FSDs based in airports that were not meeting the 

utilization target. 

To assess probability of detection requirements, we interviewed personnel from 

TSA’s Acquisition Program Management Branch, Requirements and 

Capabilities Analysis, Office of Inspection, Office of Intelligence, and Test and 

Evaluation Division. We assessed pertinent documentation to review the 

requirements set forth for probability of detection. 

To determine whether TSA provides oversight to ensure deployed AIT are 

meeting the availability requirement, we interviewed AIT’s program manager 

and obtained and reviewed maintenance reports from the Federal Data 

Reporting System; Performance Management Review documents; and FY 2018 

Transportation Security Equipment Useful Life Analysis. 

To understand how the AIT program recently became a standalone Level 2 

program, we obtained Acquisition Decision Memoranda indicating the AIT 

program had moved into the sustainment, or full operational capability phase. 

We met with TSA’s Portfolio Director and the Acquisition Program Management 

Team to understand the Passenger Screening Program’s realignment. We also 

reviewed the Post Implementation Review Strategy. Because the document was 

in draft, we did not report on its contents; instead, we verified whether 

implementation actions were taken. 

We conducted this performance audit between April 2018 and October 2019 

pursuant to the Inspector General Act of 1978, as amended, and according to 

generally accepted government auditing standards. Those standards require 

we plan and perform the audit to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based upon our 

audit objectives. We believe the evidence obtained provides a reasonable basis 

for our findings and conclusions based upon our audit objectives. The Office of 

Audits major contributors to this report are Carolyn Hicks, Audit Director; Paul 

Exarchos, Audit Manager; Jeff Mun, Auditor-in-Charge; Michael Levy, Auditor; 

Michaela Stuart, Program Analyst; Lindsey Koch, Communications Analyst; 

and Kendra Loper, Independent Referencer. 

www.oig.dhs.gov 9 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix A 

TSA Comments to the Draft Report 

www.oig.dhs.gov 10 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 11 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 12 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

www.oig.dhs.gov 13 OIG-20-33 
 

OFFICE OF INSPECTOR GENERAL 

Department of Homeland Security 

Appendix B 

Report Distribution 

Department of Homeland Security 

Secretary 

Deputy Secretary 

Chief of Staff 

Deputy Chiefs of Staff 

General Counsel 

Executive Secretary 

Director, GAO/OIG Liaison Office 

Assistant Secretary for Office of Policy 

Assistant Secretary for Office of Public Affairs 

Assistant Secretary for Office of Legislative Affairs 

TSA Audit Liaison 

Office of Management and Budget 

Chief, Homeland Security Branch 

DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriations Committees 

www.oig.dhs.gov 14 OIG-20-33 
Additional Information and Copies 

To view this and any of our other reports, please visit our website at: 

www.oig.dhs.gov. 

For further information or questions, please contact Office of Inspector General 

Public Affairs at: DHS-OIG.OfficePublicAffairs@oig.dhs.gov. 

Follow us on Twitter at: @dhsoig. 

OIG Hotline 

To report fraud, waste, or abuse, visit our website at www.oig.dhs.gov and click 

on the red "Hotline" tab. If you cannot access our website, call our hotline at 

(800) 323-8603, fax our hotline at (202) 254-4297, or write to us at: 

Department of Homeland Security 

Office of Inspector General, Mail Stop 0305 

Attention: Hotline 

245 Murray Drive, SW 

Washington, DC 20528-0305