AVIATION SECURITY

9/11 Anniversary 

Observations on TSA’s 

Progress and Challenges in 

Strengthening Aviation 

Security 

Statement of Stephen M. Lord, Director 

Homeland Security and Justice Issues

Testimony 

Before the Subcommittee on 

Transportation Security, Committee on 

Homeland Security, House of 

Representatives

For Release on Delivery 

Expected at 2:00 p.m. EDT

Tuesday, September 11, 2012

GAO-12-1024T

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-12-1024T, a testimony 

before the Subcommittee on Transportation 

Security, Committee on Homeland Security, 

U.S. House of Representatives

September 11, 2012

AVIATION SECURITY

9/11 Anniversary Observations on TSA’s Progress 

and Challenges in Strengthening Aviation Security

Why GAO Did This Study

Securing commercial aviation 

operations remains a daunting task, 

with hundreds of airports, thousands of 

aircraft, and thousands of flights daily 

carrying millions of passengers and 

pieces of carry-on and checked 

baggage. The attempted terrorist 

bombing of Northwest flight 253 on 

December 25, 2009, and the October 

2010 discovery of explosive devices in 

air cargo packages on an all-cargo 

aircraft bound for the United States 

from Yemen highlight the continuing 

need for effective passenger, cargo, 

and baggage screening. This 

statement discusses actions TSA has 

taken to (1) validate the scientific basis 

of its behavior-based passenger 

screening program (the Screening of 

Passengers by Observation 

Techniques, or SPOT); (2) strengthen 

the security of inbound air cargo (3) 

acquire checked baggage screening 

technology in accordance with 

established guidance; and (4) vet 

foreign nationals training at U.S. flight 

schools. This statement is based on 

GAO’s work issued from September 

2009 through July 2012, and includes 

selected updates on air cargo 

screening conducted from July through 

September 2012. For the selected 

updates, GAO interviewed TSA 

officials.

What GAO Recommends

GAO is not making any new 

recommendations. GAO has 

previously recommended that TSA 

take actions to improve aviation 

security. In general, TSA concurred

with the recommendations, and is 

taking actions to address them. 

What GAO Found

The Transportation Security Administration (TSA) has taken actions to validate 

the science underlying its behavior-based passenger screening program, the 

Screening of Passengers by Observation Techniques, or SPOT, program, but 

more work remains. GAO reported in May 2010 that (1) TSA deployed SPOT 

before first determining whether there was a scientifically valid basis for using 

behavior and appearance indicators to reliably identify passengers who may 

pose a risk; and (2) it is unknown if the SPOT program has ever resulted in the 

arrest of anyone who is a terrorist, or who was planning to engage in terrorist 

related activity, although there is other evidence that terrorists have transited 

through SPOT airports. GAO recommended in May 2010 that the Department of 

Homeland Security (DHS) convene an independent panel of experts to review 

the methodology of the ongoing validation study on the SPOT program to 

determine whether it is sufficiently comprehensive to validate the program. DHS 

concurred and subsequently revised its validation study to include an

independent expert review. DHS’s study, completed in April 2011, found that 

SPOT was more effective than random screening to varying degrees; however, 

DHS noted limitations to the study, such as that it was not designed to 

comprehensively validate whether SPOT can be used to reliably identify 

individuals who pose a security risk. GAO is currently reviewing the program and 

will issue our report next year.

TSA has taken actions to enhance the security of cargo on inbound aircraft, but 

challenges remain. For example, TSA issued new screening requirements aimed 

at enhancing the security of cargo on aircraft, such as prohibiting the transport of 

air cargo on passenger aircraft from Yemen. In June 2010, GAO recommended 

that TSA develop a mechanism to verify the accuracy of all screening data. TSA 

concurred in part and required air carriers to report inbound cargo screening 

data, but has not yet fully addressed the recommendation. In June 2012, TSA 

required air carriers to screen 100 percent of inbound air cargo transported on 

passenger aircraft by December 3, 2012. However, air carriers and TSA face 

challenges in implementing this requirement and in providing reasonable 

assurance that screening is being conducted at reported levels. 

DHS and TSA have experienced difficulties establishing acquisition program 

baselines, schedules, and cost estimates for the Electronic Baggage Screening 

Program (EBSP). For example, GAO reported in July 2011 that TSA had 

established a schedule for the acquisition of the explosives detection systems 

(EDS) TSA deploys to screen checked baggage, but it did not fully comply with 

leading practices. GAO recommended that DHS develop and maintain a 

schedule for the EBSP in accordance with leading practices. DHS concurred. 

GAO reported in July 2012 that TSA has worked to enhance general aviation 

security, such as though issuing regulations, but there are weaknesses in its

process for vetting foreign flight school student applicants, and in DHS’s process 

for identifying flight school students who may be in the country illegally. For 

example, TSA’s program to help determine whether flight school students pose a 

security threat does not determine whether they entered the country legally. GAO

recommended actions that DHS and TSA could take to address these concerns, 

with which DHS and TSA have concurred, and are starting to take actions. 

View GAO-12-1024T. For more information, 

contact Stephen M. Lord at (202) 512-4379 or 

lords@gao.gov.
 

Page 1 GAO-12-1024T 

Chairman Rogers, Ranking Member Jackson-Lee, and Members of the 

Subcommittee:

I appreciate the opportunity to participate in today’s hearing on the 

anniversary of the September 11, 2001, terrorist attacks to discuss our 

work examining key layers of aviation security: (1) the Transportation 

Security Administration’s (TSA) behavior-based passenger screening 

program; (2) the security of air cargo on flights bound for the United 

States from foreign countries (known as inbound air cargo); (3) the 

deployment of checked baggage screening technology; and (4) the 

federal government’s vetting process for individuals training at U.S. flight 

schools. This work may help inform future deliberations about any 

potential challenges and corrective actions regarding U.S. aviation 

security. 

In the years that have passed since TSA assumed responsibility for 

aviation security, TSA has spent billions of dollars and implemented a 

wide range of initiatives to strengthen aviation security. Our work has 

shown that TSA has enhanced aviation security with respect to 

passenger, checked baggage, and air cargo screening, among other 

areas. Securing commercial aviation operations, however, remains a 

daunting task—with hundreds of airports, thousands of aircraft, and 

thousands of flights daily carrying millions of passengers and their 

property, as well as cargo. The attempted terrorist bombing of Northwest 

flight 253 on December 25, 2009, and the October 2010 discovery of 

explosive devices in air cargo packages on an all-cargo aircraft bound for 

the United States from Yemen provides a vivid reminder that civil aviation 

remains an attractive terrorist target and highlights the continuing need for 

effective passenger, cargo, and baggage screening. According to the 

President’s National Counterterrorism Strategy, released in June 2011, 

aviation security and screening is an essential tool in our ability to detect, 

disrupt, and defeat plots to attack the homeland.1

My statement today discusses actions TSA has taken to (1) validate the 

scientific basis of its behavior-based passenger screening program 

(known as the Screening of Passengers by Observation Techniques, or 

SPOT program), (2) strengthen the security and screening of inbound air 

cargo, (3) acquire checked baggage screening technology in accordance 

with established guidance, and (4) vet foreign nationals seeking to 

 

 

1 National Strategy for Counterterrorism, (Washington, D.C.: June 28, 2011).

 
 

Page 2 GAO-12-1024T 

undertake flight training at U.S. flight schools,

2 as well as the challenges 

associated with implementing these actions.

This statement is based on our prior work issued from May 2010 through 

July 2012, and includes selected updates conducted from July 2012 

through September 2012 on TSA’s efforts to improve security of inbound 

air cargo.

3Our previously published products contain additional details on 

the scope and methodology, including data reliability, for these reviews. 

For the updated information on air cargo screening, we obtained TSA 

views on our findings and incorporated technical comments where 

appropriate. We conducted our work in accordance with generally 

accepted government auditing standards. 

The Aviation and Transportation Security Act established TSA as the 

federal agency with primary responsibility for securing the nation’s civil 

aviation system, which includes the screening of all passenger and 

property transported by commercial passenger aircraft.4

 

2 Flight schools fall within the general aviation community, which also includes 

nonscheduled aircraft operations such as air medical-ambulance, corporate aviation, and 

privately owned aircraft—generally, aircraft not available to the general public for 

transport. 

 At the more than 

450 TSA-regulated airports in the United States, prior to boarding an 

aircraft, all passengers, their accessible property, and their checked 

baggage are screened pursuant to TSA-established procedures. TSA 

relies upon multiple layers of security to deter, detect, and disrupt persons 

posing a potential risk to aviation security. These layers include behavior 

detection officers (BDOs), who examine passenger behaviors and 

appearances to identify passengers who might pose a potential security 

3 See GAO, Aviation Security: Efforts to Validate TSA’s Passenger Screening Behavior 

Detection Program Underway, but Opportunities Exist to Strengthen Validation and 

Address Operational Challenges, GAO-10-763 (Washington, D.C.: May 20, 2010); Aviation 

Security: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated Airports, 

but Could Strengthen Cost Estimates, GAO-12-266 (Washington, D.C.: Apr. 27, 2012); 

Aviation Security: Actions Needed to Address Challenges and Potential Vulnerabilities 

Related to Securing Inbound Air Cargo, GAO-12-632 (Washington, D.C.: May 10, 2012); 

and General Aviation Security: Weaknesses Exist in TSA's Process for Ensuring Foreign 

Flight Students Do Not Pose a Security Threat, GAO-12-875 (Washington, D.C.: July 18, 

2012).

4 See Pub. L. No. 107-71, 115 Stat. 597 (2001). For purposes of this testimony,

“commercial passenger aircraft” refers to U.S. or foreign-flagged air carriers operating 

under TSA-approved security programs with regularly scheduled passenger operations to 

or from a U.S. airport. 

Background 
 

Page 3 GAO-12-1024T 

risk at TSA-regulated airports;5 travel document checkers, who examine 

tickets, passports, and other forms of identification; transportation security 

officers (TSO), who are responsible for screening passengers and their 

carry-on baggage at passenger checkpoints using x-ray equipment, 

magnetometers, Advanced Imaging Technology, and other devices, as 

well as for screening checked baggage; random employee screening; and 

checked baggage screening systems.6 The Implementing 

Recommendations of 9/11 Commission Act of 2007 further mandates that 

the Secretary of Homeland Security establish a system to screen 100 

percent of cargo transported on passenger aircraft, and defines screening 

for purposes of meeting this mandate, in general, as a physical 

examination or the use of nonintrusive methods to assess whether cargo 

poses a threat to transportation security.7 Such cargo ranges in size from 

1 pound to several tons and ranges in type from perishable commodities 

to machinery. In 2011, all-cargo carriers transported approximately 66 

percent (6.9 billion pounds) of the total cargo (10.4 billion pounds) 

transported to the United States.8 Additionally, TSA has responsibilities 

for general aviation security, and developed the Alien Flight Student 

Program (AFSP) to help determine whether foreign students enrolling at 

flight schools pose a security threat.9

 

5 TSA designed SPOT to provide BDOs with a means of identifying persons who may 

pose a potential security risk at TSA-regulated airports by focusing on behaviors and 

appearances that deviate from an established baseline and that may be indicative of 

stress, fear, or deception.

U.S. government threat 

assessments have discussed plans by terrorists to use general aviation 

aircraft to conduct attacks. Further, analysis conducted on behalf of TSA 

has indicated that larger general aviation aircraft may be able to cause 

significant damage to buildings and other structures. 

6 Advanced Imaging Technology screens passengers for metallic and non-metallic threats 

including weapons, explosives, and other objects concealed under layers of clothing. At 

airports participating in TSA’s Screening Partnership Program, screeners employed by 

private sector entities under contract to and overseen by TSA, and not TSOs, perform the 

passenger and checked baggage screening function in accordance with TSA 

requirements. See 49 U.S.C. § 44920. 

7 See 49 U.S.C. § 44901(g). 

8 Based on 2011 TSA data. 

9 See 49 C.F.R. pt. 1552, subpt. A. 
 

Page 4 GAO-12-1024T 

We reported in May 2010 that TSA deployed SPOT nationwide before 

first determining whether there was a scientifically valid basis for using 

behavior and appearance indicators as a means for reliably identifying 

passengers who may pose a risk to the U.S. aviation system.10

According to TSA, SPOT was deployed before a scientific validation of 

the program was completed to help address potential threats to the 

aviation system, such as those posed by suicide bombers. TSA also 

stated that the program was based upon scientific research available at 

the time regarding human behaviors. We reported in May 2010 that 

approximately 14,000 passengers were referred to law enforcement 

officers under SPOT from May 2004 through August 2008.11 Of these 

passengers, 1,083 were arrested for various reasons, including being 

illegal aliens (39 percent), having outstanding warrants (19 percent), and 

possessing fraudulent documents (15 percent). The remaining 27 percent 

were arrested for other reasons such as intoxication, unruly behavior, 

theft, domestic violence, and possession of prohibited items. As noted in 

our May 2010 report, SPOT officials told us that it is not known if the 

SPOT program has ever resulted in the arrest of anyone who is a 

terrorist, or who was planning to engage in terrorist-related activity. More 

recent TSA data covering the period from November 1, 2010, to April 18, 

2012, indicates that SPOT referred 60,717 passengers for additional 

screening, which resulted in 3,803 referrals to law enforcement officers 

and 353 arrests. Of these 353 arrests, 23 percent were related to 

immigration status, 23 percent were drug-related, 9 percent were related 

to fraudulent documents, 22 percent were related to outstanding 

warrants, and 28 percent were for other offenses.12

A 2008 report issued by the National Research Council of the National 

Academy of Sciences stated that the scientific evidence for behavioral 

monitoring is preliminary in nature.13

 

10 See GAO-10-763. 

The report also noted that an 

11 See GAO-10-763. 

12 These percents add to more than 100 percent (specifically, 105 percent) because some 

of the passengers were arrested for multiple offenses.

13 National Research Council, Protecting Individual Privacy in the Struggle Against 

Terrorists: A Framework for Assessment (Washington, D.C.: National Academies Press, 

2008). We reviewed the approach used and the information provided in this study and 

found the study and its results to be reliable for the purposes for which we used it in this 

report. 

TSA Has Taken 

Actions to Validate 

the Science 

Underlying Its 

Behavior Detection 

Program, but More 

Work Remains
 

Page 5 GAO-12-1024T 

information-based program, such as a behavior detection program, 

should first determine if a scientific foundation exists and use scientifically 

valid criteria to evaluate its effectiveness before deployment. The report 

added that such programs should have a sound experimental basis and 

that the documentation on the program’s effectiveness should be 

reviewed by an independent entity capable of evaluating the supporting 

scientific evidence.14

As we reported in May 2010, an independent panel of experts could help 

DHS determine if the SPOT program is based on valid scientific principles 

that can be effectively applied in an airport environment for 

counterterrorism purposes. Thus, we recommended that the Secretary of 

Homeland Security convene an independent panel of experts to review 

the methodology of DHS’s Science and Technology Directorate’s ongoing 

validation study on the SPOT program being conducted to determine 

whether the study’s methodology is sufficiently comprehensive to validate 

the SPOT program. We also recommended that this assessment include 

appropriate input from other federal agencies with expertise in behavior 

detection and relevant subject matter experts.15 DHS concurred and 

stated that its validation study, completed in April 2011, included input 

from a broad range of federal agencies and relevant experts, including 

those from academia.16 DHS’s validation study found that SPOT was 

more effective than random screening to varying degrees. For example, 

the study found that SPOT was more effective than random screening at 

identifying individuals who possessed fraudulent documents and 

identifying individuals who law enforcement officers ultimately arrested.17

 

14 A study performed by the JASON Program Office raised similar concerns. The JASON 

Program Office is an independent scientific advisory group that provides consulting 

services to the U.S. government on matters of defense science and technology.

15 See GAO-10-763. 

16 See DHS, SPOT Referral Report Validation Study Final Report Volume I: Technical 

Report, (Washington, D.C.: Apr. 5, 2011). DHS’s study was conducted to determine the 

extent to which SPOT was more effective than random screening at identifying security 

threats and how the program’s behaviors correlate to identifying high-risk travelers. The 

study defines high-risk passengers as travelers that knowingly and intentionally try to 

defeat the security process including those carrying serious prohibited or illegal items, 

such as weapons, drugs, or fraudulent documents; or those that were ultimately arrested 

by law enforcement.

17 The extent to which SPOT is more effective than random screening at identifying 

fraudulent documents and individuals ultimately arrested by law enforcement officers is 

deemed sensitive security information by TSA.
 

Page 6 GAO-12-1024T 

According to DHS’s study, no other counterterrorism or screening 

program incorporating behavior and appearance-based indicators is 

known to have been subjected to such a rigorous, systematic evaluation 

of its screening accuracy. However, DHS noted that the identification of 

high-risk passengers was rare in both the SPOT and random tests. DHS’s 

study also noted that the assessment was an initial validation step, and 

was not designed to fully validate whether behavior detection can be used 

to reliably identify individuals in an airport environment who pose a 

security risk. According to DHS, further research will be needed to 

comprehensively validate the program.

In addition, DHS determined that the base rate, or frequency, of SPOT 

behavioral indicators observed by TSA to detect suspicious passengers 

was very low and that these observed indicators were highly varied 

across the traveling public. Although details about DHS’s findings related 

to these indicators are sensitive security information, the low base rate 

and high variability of traveler behaviors highlights the challenge that TSA 

faces in effectively implementing a standardized list of SPOT behavioral 

indicators. In addition, DHS outlined several limitations to the study. For 

example, the study noted that BDOs were aware of whether individuals 

they were screening were selected as the result of identified SPOT 

indicators or random selection. DHS stated that this had the potential to 

introduce bias into the assessment. DHS also noted that SPOT data from 

January 2006 through October 2010 were used in its analysis of 

behavioral indicators even though questions about the reliability of the 

data exist.18

The study also noted that it was not designed to comprehensively validate 

whether SPOT can be used to reliably identify individuals in an airport 

environment who pose a security risk. The DHS study also made several 

additional recommendations related to strengthening the program and 

conducting a more comprehensive validation of whether the science can 

be used for counterterrorism purposes in the aviation environment.19

 

18 DHS officials stated that this historical SPOT data was not used in their analysis to 

determine whether SPOT was more effective than random screening.

Some of these recommendations, such as the need for a comprehensive 

19 The study made recommendations related to SPOT in three areas: (1) future validation 

efforts, (2) comparing SPOT with other screening programs, and (3) broader program 

evaluation issues. TSA designated the specific details of these recommendations 

sensitive security information.
 

Page 7 GAO-12-1024T 

program evaluation including a cost-benefit analysis, reiterate 

recommendations made in our prior work. In March 2011, we reported 

that Congress may wish to consider the study’s results in making future 

funding decisions regarding the program.20 TSA is reviewing the study’s 

findings and assessing the steps needed to address DHS’s 

recommendations. If TSA decides to implement the recommendations in 

the April 2011 DHS validation study, it may be years away from knowing 

whether there is a scientifically valid basis for using behavior detection 

techniques to help secure the aviation system against terrorist threats 

given that the initial study took about 4 years to complete. We are 

conducting a follow-on review of TSA’s behavior detection program, and 

its related variant, the so-called “Assessor Program”, which incorporates 

more extensive verbal interactions (“chat downs”) with the traveling 

public. The Assessor program is currently being test piloted in Boston and 

Detroit. Our follow-on report on this program will be issued early next 

year.

DHS and TSA have taken four primary actions to enhance the security of 

inbound cargo on passenger and all-cargo aircraft following the October 

2010 bomb attempt originating in Yemen.

TSA issued new screening requirements aimed at enhancing 

the security of cargo on passenger and all-cargo aircraft. 

Beginning in October 2010, TSA imposed new risk-based security 

procedures on passenger and all-cargo aircraft aimed at focusing more 

detailed screening measures on high risk shipments and, among other 

things, prohibited the transport of cargo on passenger aircraft from 

Yemen and Somalia due to threats stemming from those areas.21 

DHS instituted working groups with air cargo industry stakeholders 

to identify ways to enhance air cargo security. In January 2011, the 

Secretary of Homeland Security established an Air Cargo Security 

Working Group to obtain advice and consultations from air cargo security 

 

20 See GAO, Opportunities to Reduce Potential Duplication in Government Programs, 

Save Tax Dollars, and Enhance Revenue, GAO-11-318SP (Washington, D.C.: Mar. 1, 

2011).

21 TSA imposed requirements on both U.S. and foreign-flagged passenger and all-cargo 

carriers. All-cargo carriers are generally aircraft configured solely for the transport of 

cargo (e.g., FedEx and United Parcel Service). 

DHS and TSA Have 

Taken Actions to 

Enhance the Security 

of Cargo on Inbound 

Aircraft, but 

Challenges Remain
 

Page 8 GAO-12-1024T 

stakeholders on ways to enhance the security of the air cargo system.22

DHS initiated an Air Cargo Advance Screening (ACAS) pilot to 

identify high-risk cargo for screening prior to transport to the United 

States. The aim of the pilot, which is ongoing, is to determine whether it 

is feasible for air carriers to submit air cargo manifest data to CBP prior to 

departure from all foreign last point of departure airports to allow CBP to 

analyze, target, and, if needed, issue instructions to air carriers to provide 

additional cargo information or take additional security measures before 

such cargo is loaded onto aircraft. DHS initially focused on all-cargo 

express carriers and companies due to the elevated risk highlighted by 

the October 2010 incident.

The Air Cargo Security Working Group briefed the Secretary of Homeland 

Security, the Commissioner of U.S. Customs and Border Protection 

(CBP), and the TSA Administrator in April 2011 on proposed solutions, 

and recommended that TSA reevaluate the agency’s implementation 

plan, timeline, and resources related to TSA’s program to recognize the 

security programs of foreign countries, known as the National Cargo 

Security Program (NCSP). According to TSA officials, participants of this 

working group have reconvened as part of the Aviation Security Advisory 

Committee, which held its first meeting in May 2012, and the committee 

will meet again in mid-September 2012 to discuss the implementation of 

the recommendations.

23

 

22 DHS’s Air Cargo Security Working Group consists of four subgroups: (1) Information 

subgroup, whose objective is to, among other things, enhance intelligence and information 

sharing among federal stakeholders and between the U.S. government and private sector 

entities; (2) technology and capacity building subgroup, whose objective is to review 

technology standards and develop suggestions for addressing technology limitations; (3) 

global cargo programs subgroup, whose objective is to review and explore opportunities 

for enhanced public-private coordination as DHS works to address statutory requirements 

for screening 100 percent of inbound air cargo; and (4) global mail subgroup, whose 

objective is to, among other things, identify potential vulnerabilities for global mail and 

propose alternative processes and procedures to ensure the safety of mail transported by 

air. 

As of August 2012, the ACAS pilot included 

3 passenger air carriers and 4 all-cargo carriers that service the United 

States and is focused on about 189 geographic locations. Under existing 

CBP requirements, CBP must receive manifest data for air cargo 

shipments from air carriers no later than 4 hours prior to the flight’s arrival 

in the United States or no later than the time of departure (that is, “wheels 

23 All-cargo express carriers and companies focus on transporting cargo under quick time 

frames. 
 

Page 9 GAO-12-1024T 

up” and en route directly to the United States) from locations in North 

America.24

TSA developed a program to recognize foreign air cargo security 

programs. TSA has developed the NCSP recognition program to review 

and recognize the air cargo security programs of foreign countries if TSA 

deems those programs as providing a level of security commensurate 

with TSA’s air cargo security standards. In May 2012, TSA recognized 

Canada as providing a level of security commensurate with U.S. air cargo 

security standards, and in June 2012, the agency recognized the 

European Union and Switzerland as also providing this same level of 

security based on the principle of “mutual recognition.”

Under the pilot program, however, participants provide 

manifest data prior to loading cargo aboard aircraft.

25

Despite these actions, air carriers and TSA face three key challenges 

that, among other things, could limit TSA’s ability to meet the 9/11 

Commission Act mandate to screen 100 percent of cargo transported on 

passenger aircraft as it applies to inbound air cargo and to provide 

reasonable assurance that screening is being conducted at reported 

levels.

TSA officials 

stated that the NCSP recognition program is a key effort in meeting the 

100 percent screening mandate because it will eliminate the need for air 

carriers to comply with two countries’ security programs.

26 All-cargo carriers subject to TSA regulation also reported facing 

challenges in implementing new TSA screening requirements established 

after the October 2010 Yemen incident. 

Passenger air carriers reported logistical challenges implementing 

proposed screening requirements. In January 2011, TSA proposed 

changes to passenger aircraft security requirements outlined in the 

Aircraft Operator Standard Security Program and the Model Security 

Program to further enhance the security of air cargo departing foreign 

locations by requiring 100 percent screening of inbound cargo previously 

exempt from screening. TSA requirements currently call for air carriers to 

 

24 See 19 C.F.R. § 122.48a(b). 

25 TSA had previously recognized France and the United Kingdom as providing a level of 

security commensurate with U.S. air cargo security standards. 

26 See 49 U.S.C. § 44901(g).
 

Page 10 GAO-12-1024T 

screen a certain percentage of all cargo.27

TSA faces challenges verifying screening data on inbound 

passenger cargo. TSA relies on data submitted to the agency by air 

carriers to determine the amount of inbound air cargo screened in 

accordance with TSA screening requirements. As of September 2011, 

TSA officials stated that air carrier-reported screening percentages—

which they estimate to be about 80 percent—are based on actual data 

reported by air carriers, but agreed that it is difficult to verify the accuracy 

of the screening data reported by air carriers with reasonable assurance. 

According to TSA, as of August 2012, the air carrier data have not been 

independently verified for accuracy since TSA has not developed a 

mechanism to cross-reference local screening logs with screening reports 

submitted by air carriers to TSA that do not contain such information. To 

more accurately identify the level of screening being conducted on 

inbound air cargo, we recommended in June 2010 that TSA develop a 

mechanism to verify the accuracy of all screening data through random 

checks or other practical means.

TSA proposed changes that 

would require passenger air carriers to screen 100 percent of cargo as 

part of its efforts to meet the 9/11 Commission Act mandate. Passenger 

air carriers expressed concerns about being able to meet the 100 percent

screening mandate as it applies to inbound cargo stating that it would 

cause significant disruptions in the air cargo supply chain, among other 

issues. In response to these concerns, TSA officials stated that they 

revised the proposed requirements and issued new passenger security 

requirements in June 2012. Agency officials said they plan to require air 

carriers to screen 100 percent of inbound air cargo transported on 

passenger aircraft by December 3, 2012.

28

 

27 Details on TSA’s screening requirements are deemed sensitive security information and 

not included in this statement. 

TSA concurred in part and stated that 

as of May 1, 2010, they had issued changes to air carriers’ standard 

security programs that require air carriers to report inbound cargo 

screening data to TSA. Specifically, TSA officials told us that in May 2010 

the agency created a reporting requirement for air carriers to provide 

screening data on a monthly basis. TSA also stated that inspectors 

review screening data, among other things, when inspecting air carriers 

as part of the agency’s air carrier compliance inspections. However, since 

TSA still has not developed a mechanism to verify the accuracy of the 

28 GAO-10-446. 
 

Page 11 GAO-12-1024T 

data reported by air carriers, the agency has not yet fully met the intent of 

the recommendation. It will be important for TSA to continue to work 

towards ensuring verification of inbound air cargo screening data 

submitted by air carriers and that inbound air cargo is screened in 

accordance with the mandate.

Reporting screening data could facilitate oversight of all-cargo 

carrier compliance requirements. TSA relies on data submitted by 

passenger carriers to determine the amount of air cargo screened on 

inbound passenger aircraft but there is no requirement for all-cargo 

carriers to report comparable screening data to TSA, even though most of 

the cargo shipped from abroad into the United States is shipped on allcargo carriers. Thus, TSA does not know the extent to which all-cargo 

carriers are screening cargo or meeting the enhanced screening 

requirements introduced after the October 2010 incident in Yemen. 

Officials from two global all-cargo carriers said that submitting such 

information to TSA would be feasible because they are already collecting 

this data internally, but officials from two other all-cargo carriers stated 

that reporting screening data to TSA would be challenging because of 

staffing limitations or because such data may not be available. TSA 

officials said that TSA does not require that all-cargo carriers submit 

screening data because it has focused its efforts on collecting data from 

passenger air carriers in support of meeting the 100 percent mandate. 

TSA officials stated that TSA may consider opportunities to capture 

additional inbound air cargo information, but has not yet weighed the 

costs and benefits of doing so because it has focused its efforts on 

establishing the ACAS pilot program, which DHS established to more 

readily indentify high-risk cargo. The pilot program is a key effort to 

identify high-risk cargo prior to aircraft departing from foreign airports, but 

is not intended to provide TSA with screening data, which if collected and 

verified, could provide additional assurance that all-cargo carriers are 

complying with TSA’s enhanced screening requirements. To help TSA 

better determine what actions are needed, if any, to ensure that all-cargo 

carriers are complying with the agency’s enhanced screening 

requirements, we recommended in May 2012 that DHS assess the costs 

and benefits of requiring all-cargo carriers to report data on screening 

conducted.29

 

29 See 

DHS concurred with the recommendation and is taking 

actions to address it.

GAO-12-632. 
 

Page 12 GAO-12-1024T 

TSA’s Electronic Baggage Screening Program (EBSP) reports that 76 

percent of the airports (337 of 446) the agency regulates for security have 

a mix of in-line and stand-alone baggage screening configurations that 

best meet airport needs (i.e., optimal systems). Our prior work on TSA’s 

checked baggage screening program—EBSP—identified a number of 

shortcomings in DHS and TSA’s process for establishing program 

baselines, program schedules, and cost estimates.

Acquisition program baselines. We found that realistic acquisition 

program baselines with stable requirements for cost, schedule, and 

performance are among the factors that are important to successful 

acquisitions delivering capabilities within cost and schedule.30 Further, we 

reported in April 2009 that program performance metrics for cost and 

schedule can provide useful indicators of the health of acquisition 

programs and, when assessed regularly for changes and the reasons that 

cause changes, such indicators can be valuable tools for improving 

insight and oversight of individual programs as well as the total portfolio of 

major acquisitions.31

We reported in April 2012 that TSA has not had a DHS-approved 

acquisition program baseline for EBSP since the program’s inception 

more than 8 years ago.

According to DHS’s acquisition guidance, the 

program baseline is the contract between the program and departmental 

oversight officials and must be established at program start to document 

the program’s expected cost, deployment schedule, and technical 

performance. By tracking and measuring actual program performance 

against this baseline, management can be alerted to potential problems, 

such as cost growth or changing requirements, and has the ability to take 

corrective action.

32

 

30

Further, DHS did not require TSA to complete 

an acquisition program baseline until November 2008. TSA officials said 

they have twice submitted an acquisition program baseline to DHS for 

approval—first in November 2009 and again February 2011. However, 

according to DHS officials TSA did not have a fully developed life cycle 

cost estimate. In November 2011, DHS told TSA that it needed to revise 

the life cycle cost estimates as well as its procurement and deployment 

GAO-10-588SP. 

31 Defense Acquisitions: Measuring the Value of DOD’s Weapon Programs Requires 

Starting with Realistic Baselines, GAO-09-543T (Washington, D.C.: April 1, 2009). 

32 GAO-12-266. 

DHS and TSA Have 

Experienced 

Difficulties 

Establishing 

Acquisition Program 

Baselines, Schedules, 

and Cost Estimates 

for Checked Baggage 

Screening Systems
 

Page 13 GAO-12-1024T 

schedules to reflect budget constraints. DHS officials told us that they 

could not approve the acquisition program baseline as written because 

TSA’s estimates were significantly over budget. An approved baseline will 

provide DHS with additional assurances that TSA’s approach is 

appropriate and that the capabilities being pursued are worth the 

expected costs. TSA officials stated that TSA is working with DHS to 

amend the draft program baseline and plans to resubmit a revised life 

cycle cost estimates with a revised acquisition program baseline by 

December 31, 2012. As we reported, establishing and approving a 

program baseline, as DHS and TSA plan to do for the EBSP, could help 

DHS assess the program’s progress in meeting its goals and achieve 

better program outcomes.

Schedules. In July 2011, we reported that TSA had established a 

schedule for the acquisition of the explosives detection systems (EDS) 

TSA deploys to screen checked baggage, but it did not fully comply with 

leading practices, and TSA had not developed a plan to upgrade its EDS 

fleet to meet the 2010 explosives detection requirements.33 We noted that 

some of TSA’s approximately 2,200 deployed systems met 2005 

explosive requirements while the remainder met 1998 explosive detection 

requirements.34

 

33 GAO, Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for 

Checked Baggage, but Additional Screening Actions Are Need,

Leading practices state that the success of a large-scale 

system acquisition, such as TSA’s EDS acquisition, depends in part on 

having a reliable schedule that identifies when the program’s set of work 

activities and milestone events will occur, amongst other things. We 

reported that the schedule for the EDS acquisition is not reliable because 

it does not include a timeline to deploy EDS or plans to procure EDS to 

meet subsequent phases of explosive detection requirements. We stated 

that developing a reliable schedule would help TSA better monitor and 

oversee the progress of the EDS acquisition. DHS concurred with the 

recommendation to develop and maintain a schedule for the entire EBSP 

in accordance with the leading practices we identified for preparing a 

schedule. DHS commented that TSA had already begun working with key 

stakeholders to develop and define requirements for a schedule and to 

ensure that the schedule aligns with the leading practices. In April 2012, 

GAO-11-740, 

(Washington, D.C.: July 11, 2011). 

34 The specific number of EDS operating at particular detection levels is considered 

sensitive security information.
 

Page 14 GAO-12-1024T 

TSA stated that it had secured contractor resources to support 

development of an integrated master schedule in accordance with our 

and industry best practices, and that it anticipated completion of this 

schedule by September 2013.

Cost estimates. In April 2012, we reported that TSA’s methods for 

developing life cycle cost estimates for the EBSP did not fully adhere to 

best practices for developing these estimates.35 We reported in March 

2009 that a high-quality, reliable cost estimation process provides a 

sound basis for making accurate and well-informed decisions about 

resource investments, budgets, assessments of progress, and 

accountability for results and thus is critical to the success of a program.36

We reported that TSA’s estimates partially met three characteristics and 

minimally met one characteristic of a reliable cost estimate. 37 DHS 

concurred with the recommendation that TSA ensure that its life cycle 

cost estimates conform to cost estimating best practices, and identified 

efforts underway to address it.

 

35 See GAO-12-266. 

36 GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs. (Supersedes GAO-07-1134SP). GAO-09-3SP, 

(Washington, D.C.: Mar. 2, 2009). 

37 Specifically, we found their life cycle cost estimate to be partially comprehensive, 

partially documented, partially accurate, and minimally credible: (1) Partially 

comprehensive because the cost estimate does not incorporate costs associated with all 

security threats, lacks a detailed scope of work, and lacks a single technical baseline; (2) 

partially documented because TSA did not adequately document many assumptions or 

methodologies underlying its cost model, and provided little or no evidence that the 

assumptions and methodologies underlying the cost estimate were approved by 

management; (3) partially accurate because differences between planned and actual 

costs are not fully documented, explained, or reviewed; and (4) minimally credible 

because TSA did not complete relevant activities, such as an independent cost estimate, -

to ensure that the estimate accounts for bias and uncertainty. See GAO-12-875. 
 

Page 15 GAO-12-1024T 

As we reported in July 2012, TSA has worked with industry and other 

stakeholders to enhance general aviation security, such as issuing 

regulations and enhancing outreach and awareness, but there are 

weaknesses in the agency’s process for vetting foreign flight student 

applicants and in DHS’s process for identifying flight students who may 

be in the country illegally. We recommended two actions that DHS and 

TSA could take to address these concerns, with which DHS concurred.

Vetting foreign flight student applicants. Under AFSP, foreign 

nationals seeking flight training in the United States undergo a TSA 

security threat assessment before receiving flight training to determine 

whether each applicant is a security threat to the United States. 

According to TSA officials, when a foreign national applies to AFSP to 

obtain flight training, TSA uses information submitted by the foreign 

national—such as name, date of birth, and passport information—to 

conduct a criminal history records check, a review of the Terrorist 

Screening Database, and a review of the Department of Homeland 

Security’s TECS system.38 According to TSA officials, most foreign 

nationals taking training from a U.S. flight training provider will apply for a 

Federal Aviation Administration (FAA) airman certificate (pilot’s license) 

once their flight training is completed. Information obtained by FAA as 

part of this application for certification is placed in the airmen registry. 

From January 2006 through September 2011, 25,599 foreign nationals 

had applied for FAA airman certificates, indicating they had completed 

flight training. However, TSA computerized matching of FAA data 

determined that some known number of foreign nationals did not match 

with those in TSA’s database, raising questions as to whether they had 

been vetted.39

Since 2009, TSA has vetted all new and existing FAA airman certificate 

holders against the Terrorist Screening Database on an ongoing basis, 

which would include the foreign nationals identified through TSA’s 

 

38Information in the Terrorist Screening Center’s consolidated database of known or 

suspected terrorists—the Terrorist Screening Database—is used for security-related 

screening of foreign nationals applying to AFSP, among other purposes. TECS, an 

updated and modified version of the former Treasury Enforcement Communications 

System, is an information-sharing platform that allows users to access different databases 

relevant to the antiterrorism and law enforcement mission of numerous other federal 

agencies. 

39 The exact number is considered to be sensitive security information.

TSA Established a 

Process for Vetting 

Foreign Flight 

Students but 

Weaknesses Remain
 

Page 16 GAO-12-1024T 

analysis. However, this vetting does not occur until after the foreign 

national has obtained flight training. Thus, foreign nationals obtaining 

flight training with the intent to do harm—such as three of the pilots and 

leaders of the September 11, 2001, terrorist attacks—could have already 

obtained the training needed to operate an aircraft before they received 

any type of vetting.40

Identifying flight students entering the country illegally. We also 

reported that AFSP is not designed to determine whether a foreign flight 

student entered the country legally; thus, a foreign national can be 

approved for training through AFSP after entering the country illegally. A 

March 2010 U.S. Immigration and Customs Enforcement (ICE) 

investigation of a flight school led to the arrest of six such foreign 

nationals, including one who had a commercial pilot’s license. As a result, 

TSA and ICE jointly worked on vetting names of foreign students against 

immigration databases, but had not specified desired outcomes and time 

frames, or assigned individuals with responsibility for fully instituting the 

program as of July 2012. Thus, this weakness still exists today. Having a 

We recommended that TSA take steps to identify 

any instances where foreign nationals receive FAA airman certificates 

without first undergoing a TSA security threat assessment and examine 

those instances so that TSA can identify the reasons for these 

occurrences and strengthen controls to prevent future occurrences. DHS 

concurred with this recommendation and stated that TSA signed a 

memorandum of understanding with FAA in February 2012 to help 

address this issue. The memorandum outlines a process for FAA to 

provide certain data from its airmen registry on a monthly basis and 

authorizes TSA to use the data to ensure flight training providers are 

providing TSA with information to conduct background checks prior to 

flight instruction. This is an important step toward addressing the first part 

of our recommendation, provided that TSA uses the data to identify 

instances where foreign nationals receive FAA airman certificates without 

first undergoing a TSA security threat assessment, identifies reasons for 

these occurrences, and strengthens controls to prevent future 

occurrences, as we recommended.

 

40 TSA likewise does not vet flight student applicants claiming U.S. citizenship. H.R. 

6159—the Flight School Security Act of 2012, introduced in July 2012—would require, 

among other things, a determination by TSA that the individual seeking training is a nonthreat to aviation prior to beginning flight training. See H.R. 6159, 112th Cong. (2d Sess. 

2012). The bill, sponsored by Representative Bennie G. Thompson, was referred to the 

Committee on Homeland Security, Subcommittee on Transportation Security. 
 

Page 17 GAO-12-1024T 

road map, with steps and time frames, and assigning individuals the 

responsibility for fully instituting a pilot program could help TSA and ICE 

better identify and prevent potential risk. We recommended that TSA and 

ICE develop a plan, with time frames, and assign individuals with 

responsibility and accountability for assessing the results of their pilot 

program to check TSA AFSP data against information DHS has on 

applicants’ admissibility status to help detect and identify violations by 

foreign flight students, and institute that pilot program if it is found to be 

effective. DHS concurred and stated that TSA will prepare a plan by 

December 2012 to assess the results of the pilot program with ICE to 

determine the lawful status of the active AFSP population. We believe 

that these are positive actions that could help TSA address the 

weaknesses identified in our report. We will continue to monitor TSA’s 

progress on the proposed solutions as the agency proceeds.

Chairman Rogers, Ranking Member Jackson-Lee, and Members of the 

Subcommittee, this concludes my prepared statement. I look forward to 

responding to any questions that you may have.

For questions about this statement, please contact Stephen M. Lord at 

(202) 512-4379 or lords@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this statement. Individuals making key contributions to this testimony 

are Steve Morris, Assistant Director, and Michelle Woods, Analyst in 

Charge. Additional contributors include David M. Bruno, Glenn Davis, 

Jessica Lucas Judy; Assistant Directors; Joel Aldape; Adam Hoffman; 

Susanna Kuebler; Thomas Lombardi; Lara Miklozek; Linda Miller; Daniel 

Rodriguez; and Douglas Sloane.

GAO Contact and 

Staff 

Acknowledgements 

(441107)
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.