United States Government Accountability Office

GAO Report to Congressional Committees

AVIATION SECURITY

TSA Is Enhancing Its 

Oversight of Air 

Carrier Efforts to 

Identify Passengers on 

the No Fly and 

Selectee Lists, but 

Expects Ultimate 

Solution to Be 

Implementation of 

Secure Flight 

September 2008 

GAO-08-992 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

September 2008

AVIATION SECURITY

TSA Is Enhancing Its Oversight of Air Carrier Efforts

to Identify Passengers on the No Fly and Selectee 

Lists, but Expects Ultimate Solution to Be 

Implementation of Secure Flight 

Highlights of GAO-08-992, a report to 

congressional committees 

Air carriers remain a front-line

defense against acts of terrorism 

that target the nation’s civil

aviation system. A key 

responsibility of air carriers is to 

check passengers’ names against 

terrorist watch-list records to 

identify persons who should be

prevented from boarding (the No

Fly List) or who should undergo

additional security scrutiny (the

Selectee List). Eventually, the

Transportation Security

Administration (TSA) is to assume

this responsibility through its 

Secure Flight program. However, 

due to program delays, air carriers

retain this role. You asked GAO to 

review domestic air carriers’ 

watch-list-matching processes. 

GAO examined (1) the watch-listmatching requirements air carriers

must follow that have been 

established by TSA, and (2) the

extent to which TSA has assessed 

air carriers’ compliance with these

requirements. GAO reviewed TSA’s 

security directives, internal

guidance used by TSA’s inspectors 

to assess air carriers’ compliance 

with requirements, and inspection

results, as well as interviewed staff 

from 14 of 95 domestic air carriers

(selected to reflect a range in

operational sizes). This report is 

the public version of a restricted

report (GAO-08-453SU) issued in

July 2008. 

 

What GAO Recommends

GAO is not making any 

recommendations because TSA 

initiated actions in April 2008 to 

strengthen watch-list-matching

requirements and its oversight of

air carriers’ implementation of

these requirements. 

TSA’s requirements for domestic air carriers to conduct watch-list matching 

include a requirement to identify passengers whose names are either identical 

or similar to those on the No Fly and Selectee lists. Similar-name matching is 

important because individuals on the watch list may try to avoid detection by 

making travel reservations using name variations. According to TSA’s Office 

of Intelligence, there have been incidents of air carriers failing to identify 

potential matches by not successfully conducting similar-name matching. 

However, until revisions were initiated in April 2008, TSA’s security directives

did not specify what types of similar-name variations were to be considered 

by air carriers. Thus, in interviews with 14 air carriers GAO found inconsistent 

approaches to conducting similar-name matching. Due to such inconsistency, 

a passenger could be identified as a match by one air carrier and not by 

another. In addition, not every air carrier reported conducting similar name 

comparisons. Further, in January 2008, TSA conducted an evaluation of air 

carriers and found deficiencies in their capability to conduct similar-name 

matching. Shortly thereafter, in April 2008, TSA revised the No Fly List 

security directive to specify a baseline capability for conducting watch-list 

matching, and TSA reported that it planned to similarly revise the Selectee 

List security directive. Because the baseline capability requires that air 

carriers compare only the types of name variations specified in the directive,

TSA recognizes that the new baseline capability will not address all 

vulnerabilities. However, TSA emphasized that establishing the baseline 

capability should improve air carriers’ performance of watch-list matching 

and, in TSA’s view, is the best interim solution pending the implementation of

Secure Flight. 

TSA has undertaken various efforts to assess domestic air carriers’ 

compliance with watch-list matching requirements; however, until 2008, TSA

had conducted limited testing of air carriers’ similar-name-matching 

capability. In 2005, for instance, TSA conducted an evaluation to determine 

whether air carriers had the capability to identify names that were identical—

but not similar—to those on the No Fly List. Also, regarding regularly 

conducted inspections, TSA’s guidance did not specifically direct inspectors 

to test air carriers’ similar-name-matching capability, nor did the guidance 

specify the number or types of name variations to be assessed. Records in 

TSA’s database for regular inspections conducted during 2007 made reference 

to name-match testing in 61 of the 1,145 watch-list-related inspections that 

GAO reviewed. Without criteria or standards for air carriers to follow in 

comparing name variations, TSA did not have a uniform basis for assessing 

compliance and addressing deficiencies. However, during the course of GAO’s 

review and prompted by findings of the evaluation conducted in January 2008, 

TSA reported that its guidance for inspectors would be revised to help ensure 

air carriers’ compliance with security directives. Although TSA has plans to 

strengthen its oversight of air carriers’ compliance with the revised security

directives, it is too early to assess the extent of such oversight since TSA’s

efforts are ongoing and not completed.

To view the full product, including the scope 

and methodology, click on GAO-08-992. 

For more information, contact Cathleen A. 

Berrick at (202) 512-3404 or

berrickc@gao.gov. 
Contents 

Letter 1

Results in Brief 7

Background 10

TSA Took Action in 2008 to Enhance Watch-List Matching 

Conducted by Air Carriers but Believes the Ultimate Solution 

Will Be Implementation of Secure Flight 15

Until a 2008 Special Emphasis Inspection, TSA Had Conducted 

Limited Testing of Air Carriers’ Capability to Perform SimilarName Matching 25

Concluding Observations 32

Agency Comments 33

Appendix I Objectives, Scope, and Methodology 37 

Appendix II Overview of Selected Domestic Air Carriers’ 

Watch-List-Matching Processes 46 

Appendix III GAO Contact and Staff Acknowledgments 53 

Tables 

Table 1: TSA Watch-List-Matching Requirements Prior to the April 

2008 Revision to the No Fly List Security Directive 18

Table 2: Requirements for Matching Passenger Data to No Fly and 

Selectee Lists and Inspection Guidelines Used to Assess 

Compliance with the Requirements 28

Table 3: Watch-List-Matching Requirements and the Related

Inspection Guidelines (Fiscal Year 2007) 43

Figure 

Figure 1: Overview of the Current Passenger Watch-List-Matching 

Process 49

Page i GAO-08-992 Aviation Security and Watch List Matching
Abbreviations 

CAPPS Computer Assisted Passenger Prescreening System

DHS Department of Homeland Security 

FBI Federal Bureau of Investigation 

PARIS Performance and Results Information System 

PNR passenger name record 

TRIP Traveler Redress Inquiry Program 

TSA Transportation Security Administration 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page ii GAO-08-992 Aviation Security and Watch List Matching
United States Government Accountability Office

Washington, DC 20548 

 

September 9, 2008 

Congressional Committees 

Currently, more than 6 years after the terrorist attacks on September 11, 

2001, air carriers remain a front-line defense against acts of terrorism that 

target the nation’s civil aviation system. A key aspect of air carriers’ 

security responsibilities is to conduct preboarding checks of all 

passengers’ personal information against terrorist watch-list records that 

contain information on thousands of individuals with known or potential 

links to terrorism. This process, referred to hereafter as watch-list 

matching, involves comparing passenger data—most prominently name 

and date of birth—against the No Fly List to identify individuals who 

should be prevented from boarding an aircraft, and against the Selectee 

List to identify individuals who must undergo enhanced screening at the 

checkpoint prior to boarding.1

The Transportation Security Administration (TSA) requires that domestic 

air carriers operating to, from, and within the United States conduct 

watch-list matching.2

 Data compiled by TSA’s Office of Intelligence

indicate that, at times, these air carriers have failed to identify individuals 

who are on the No Fly List. For instance, for the 3-year period from 

January 2005 through December 2007, TSA documented several known 

incidents involving individuals on the No Fly List who, because of failures 

of domestic air carriers’ watch-list-matching processes, were allowed to 

 1

 Watch-list matching is one of two TSA-mandated prescreening processes conducted by air 

carriers. The other mandated prescreening activity is the Computer Assisted Passenger 

Prescreening System, discussed later this report, which does not involve matching 

passenger information against the No Fly and Selectee lists. These lists contain applicable 

records from the Terrorist Screening Center’s consolidated database of known or

appropriately suspected terrorists. See GAO, Terrorist Watch List Screening: 

Recommendations to Promote a Comprehensive and Coordinated Approach to TerroristRelated Screening, GAO-08-253T (Washington, D.C.: Nov. 8, 2007). 

2

 The number of domestic air carriers has varied over time, for example, from 95 in 2005 to 

about 70 in 2007. For the purposes of this report, domestic air carriers are those with 

operations based in the United States that maintain full security programs in accordance

with 49 C.F.R. part 1544. Foreign air carriers—air carriers with operations based outside 

the United States—must also comply with U.S. security regulations, including applicable 

requirements for watch-list matching, when operating flights to or from the United States in

accordance with 49 C.F.R. part 1546. Both domestic and foreign air carriers may conduct 

international flights to and from the United States; however, these operations are outside 

the scope of this report. 

Page 1 GAO-08-992 Aviation Security and Watch List Matching
board international flights traveling to or from the United States.3

 Data for 

these types of incidents, referred to as false negative watch-list-matching 

results, generally are not available for domestic flights—that is, domestic 

air carrier operations between two points within the United States or its 

territories.4

 Nevertheless, because the requirements for air carriers to 

conduct watch-list matching are generally the same irrespective of the 

departure or arrival location, false negative incidents may be occurring on 

domestic flights if watch-listed individuals attempt to fly domestically. 

At present, domestic air carriers generally conduct watch-list matching in 

accordance with requirements that TSA sets forth in security directives—a

regulatory tool through which TSA may impose security measures on a 

regulated entity, in this case air carriers, generally in response to an 

immediate or imminent threat.5

 For example, security directives require 

that air carriers execute comparisons of passenger information with No 

Fly and Selectee list information within 24 hours of a flight’s scheduled 

departure. TSA also has responsibility for overseeing how air carriers 

implement the requirements set forth in security directives. Critical to this 

effort are the agency’s aviation security inspectors, who oversee air carrier 

efforts at air carriers’ corporate security offices (principal security 

inspectors) and at airport locations (transportation security inspectors). 

As required by law, TSA is to take over from air carriers the function of 

matching passenger information to the No Fly and Selectee lists for 

domestic flights.6

 Since 2003, we have been assessing TSA’s efforts to 

 3

 See GAO, Terrorist Watch List Screening: Opportunities Exist to Enhance Management

Oversight, Reduce Vulnerabilities in Agency Screening Processes, and Expand Use of the 

List, GAO-08-110 (Washington, D.C.: Oct. 11, 2007). We reported that TSA’s Office of 

Intelligence documented various incidents (for the period January 1, 2005, through June 3,

2007) in which air carriers—both domestic and foreign—allowed individuals on the No Fly 

List to board international flights traveling to or from the United States. Several of these 

incidents involved flights of domestic air carriers. We asked TSA’s Office of Intelligence to 

identify any additional incidents in which a No Fly listed individual flew on a domestic air 

carrier for the period June 4, 2007, through December 31, 2007, and TSA identified no

additional incidents occurring within this time period. 

4

 This issue of false negatives is addressed later in this report. 

5

 See, e.g., 49 C.F.R. § 1544.305. Although generally issued in response to an immediate or

imminent threat, security directives may be effective for an indefinite duration if TSA 

determines that a continuing need for such measures exists. In some cases, aviation-related 

measures implemented through a security directive have been discontinued, amended, or 

incorporated into air carrier security programs. 

6

 See 49 U.S.C. 44903(j)(2)(C).

Page 2 GAO-08-992 Aviation Security and Watch List Matching
develop such a watch-list-matching program, currently known as Secure 

Flight, and have reported that significant challenges, including the need to 

follow a more structured systems development approach and to fully 

address how the program would protect passengers’ privacy rights, have 

delayed its implementation.7

 In April 2008, we reported that TSA has made 

significant progress in developing Secure Flight, but that challenges 

remained in a number of areas, including the need to develop more robust 

cost and schedule estimates.8 We are continuing to review TSA’s 

development and implementation of Secure Flight in response to requests

from the U.S. Senate (Committee on Commerce, Science, and

Transportation, and its Subcommittee on Aviation Operations, Safety, and 

Security; Committee on Appropriations, Subcommittee on Homeland 

Security; Committee on Homeland Security and Governmental Affairs; and 

Committee on the Judiciary) and the U.S. House of Representatives 

(Committee on Transportation and Infrastructure, Committee on 

Homeland Security, and the Committee on Oversight and Government 

Reform). In addition, the Consolidated Appropriations Act, 2008, requires 

that we report to the Committees on Appropriations of the Senate and 

House of Representatives on the Department of Homeland Security’s 

(DHS) certification of 10 conditions outlined in section 522(a) of the 

Department of Homeland Security Appropriations Act, 2005, related to the

development and implementation of the Secure Flight program.9

 The 

report is to be submitted 90 days after the DHS’s Secretary certifies that all 

10 conditions have been successfully met. 

Pending Secure Flight’s implementation, air carriers will continue to have 

primary responsibility for the watch-list-matching function. In conjunction 

with our ongoing evaluation of Secure Flight, we testified in June 2006 that 

due to delays and uncertainty surrounding Secure Flight’s implementation, 

some air carriers were enhancing their watch-list-matching processes. We 

 7

 GAO, Aviation Security: Computer-Assisted Passenger Prescreening System Faces 

Significant Implementation Challenges, GAO-04-385 (Washington, D.C.: Feb. 13, 2004); 

Aviation Security: Management Challenges Remain for the Transportation Security

Administration’s Secure Flight Program, GAO-06-864T (Washington, D.C.: June 14, 2006); 

and Aviation Security: Transportation Security Administration Has Strengthened

Planning to Guide Investments in Key Aviation Security Programs, but More Work 

Remains, GAO-08-456T (Washington, D.C.: Feb. 28, 2008). 

8

 GAO, Transportation Security: Efforts to Strengthen Aviation and Surface 

Transportation Security Continue to Progress, but More Work Remains, GAO-08-651T 

(Washington, D.C.: Apr. 15, 2008). 

9

 See Pub. L. No. 110-161, Div. E, § 513, 121 Stat. 1844, 2072-73 (2007). 

Page 3 GAO-08-992 Aviation Security and Watch List Matching
further identified that these improvements, though beneficial to the 

respective air carrier’s operations, could further exacerbate differences 

that currently exist among the various air carriers, and could result in 

varying levels of effectiveness across air carriers in matching passenger 

information to the No Fly and Selectee lists.10

Due to the importance of identifying passengers who may pose a threat to

commercial aviation, we were asked to review the current processes that 

domestic air carriers use to conduct watch-list matching for domestic 

flights.11 Accordingly, this report addresses the following questions: 

• What are TSA’s requirements for domestic air carriers to conduct 

watch-list matching for domestic flights? 

• To what extent has TSA assessed domestic air carriers’ compliance 

with watch-list-matching requirements? 

This report is a public version of the restricted report (GAO-08-453SU) that 

we provided to you on July 10, 2008. DHS and TSA deemed some of the 

information in the restricted report as Sensitive Security Information, 

which must be protected from public disclosure. Therefore, this report 

omits this information, such as the specific details associated with the 

current processes that domestic air carriers use to conduct watch-list 

matching. Although the information provided in this report is more limited

in scope, it addresses the same principal questions as the restricted report.

Also, the overall methodology used for both reports is generally the same.

To determine TSA’s requirements for matching passenger information 

against the No Fly and Selectee lists for domestic flights, we reviewed 

TSA’s security directives, policies, and other guidance applicable to watchlist matching. We also interviewed officials at TSA’s Office of 

Transportation Sector Network Management, Office of Security 

 10 GAO, Aviation Security: Management Challenges Remain for the Transportation 

Security Administration’s Secure Flight Program, GAO-06-864T (Washington, D.C.: June 

14, 2006). 

11 We are conducting this review in response to requests from the House of Representatives

(Committee on Transportation and Infrastructure, Committee on Homeland Security, and 

Committee on Oversight and Government Reform). These requesters asked that we review

the current passenger prescreening system in conjunction with our ongoing work related to

TSA’s progress with Secure Flight. In addition, we are reporting on this issue to the U.S.

Senate requesters and the mandate committees associated with our Secure Flight work. 

Page 4 GAO-08-992 Aviation Security and Watch List Matching
Operations, Office of Intelligence, and Office of Chief Counsel. We also 

reviewed key policy documents for Secure Flight, as well as our most 

recent reports and testimonies on the program to determine the planned 

matching process. In addition, to identify the composition and use of the 

No Fly and Selectee lists, we interviewed officials with the Department of 

Justice, Federal Bureau of Investigation’s (FBI) Terrorist Screening 

Center, which has responsibility for managing the use of terrorist 

information in screening processes.12 We also contacted officials from a 

federally sponsored working group on identity matching to discuss the 

challenges associated with name-based matching. Moreover, to 

understand how air carriers have responded to watch-list-matching 

requirements, we conducted telephone interviews with officials from 14 

domestic air carriers.13 Our selection of air carriers was based, in part, on 

operational size with the goal of obtaining a range of sizes based on 

operating revenue. For example, the Department of Transportation 

classifies eight of the air carriers in our review as major air carriers that 

provide service to locations across the nation and, with the exception of 

one air carrier, around the world.14 The remaining six air carriers had 

comparatively smaller business operations that generally provided service 

covering a geographical area, such as the Pacific Northwest, or commuter 

service.15 Although the 14 air carriers we spoke with represent a range in

the types of air carriers that conduct watch-list matching, and, according 

to our calculations, accounted for approximately 70 percent of all 

passengers that boarded domestic flights in 2005, the results of our 

telephone interviews are not generalizable to the domestic operations of 

 12 Pursuant to Homeland Security Presidential Directive 6, dated September 16, 2003, the

Terrorist Screening Center—an entity that has been operational since December 2003

under the administration of the FBI—was established to develop and maintain the U.S.

government’s consolidated terrorist screening database (the watch list) and to provide for 

the use of watch-list records during security-related screening processes. 

13 All 14 air carriers we interviewed operate under full security programs in accordance 

with 49 C.F.R. part 1544 and conduct watch-list matching in accordance with the No Fly 

and Selectee list security directives issued by TSA. 

14 The Department of Transportation groups U.S.-based air carriers according to their 

operating revenue. In the 2005 groupings, each of the “major” air carriers had over 

$1 billion in operating revenue. 

15 Of these six, the Department of Transportation’s 2005 revenue groupings identified three

as “national” air carriers, with each having over $100 million to $1 billion in operating

revenue, and one as a “regional” air carrier, with $100 million or less in operating revenue.

The other two air carriers were not included in the department’s revenue groupings, given 

the small scale of operations, but were identified by the department as air carriers that

provide commuter service. Major air carriers have over $1 billion in operating revenue. 

Page 5 GAO-08-992 Aviation Security and Watch List Matching
all domestic air carriers. However, our selection allowed us to understand 

how watch-list matching was performed for the majority of passengers 

flying domestically in 2005. In addition, although our work summarizes the 

14 air carriers’ watch-list-matching capabilities as described to us in 

interviews, we did not independently verify each air carrier’s reported 

method of implementation to determine the reliability of the data. 

To determine the extent to which TSA has assessed domestic air carriers’ 

compliance with watch-list-matching requirements in the No Fly and 

Selectee list security directives,16 we first assessed TSA’s inspection 

process, including the focus of inspections and inspection methods. We 

also examined TSA’s national inspection plans and related guidance and 

policy documents. Further, at TSA headquarters, we interviewed officials 

responsible for developing and implementing inspection guidance and 

compiling and analyzing inspection results. Specifically, we interviewed 

representatives from the Office of Security Operations and the Office of 

Transportation Sector Network Management. We analyzed the results of 

both regular inspections (i.e., inspections conducted in conjunction with 

annual inspection plans) and nonroutine watch-list-related inspections 

that TSA conducted. For instance, we analyzed regular watch-list-related 

inspections that TSA conducted during fiscal year 2007 to ensure that air 

carriers were in compliance with applicable requirements. Although we 

concluded that these regular inspection data were sufficiently reliable for 

the purposes of this report, we have concerns about the potential for error 

based on TSA’s process for querying its inspection database (we discuss 

these concerns in more detail in app. I). To assess data reliability, we 

performed electronic testing, discussed the data system and any data 

inconsistencies we found with knowledgeable TSA officials, and reviewed

existing information about the data system. We also reviewed results from

a special emphasis assessment that TSA conducted in 2005, and a special 

emphasis inspection it conducted in January 2008, both of which 

addressed air carriers’ capability to conduct watch-list matching.17 We 

determined that the sampling and related procedures used for the special

emphasis assessment were insufficient for providing a reliable estimate of

 16 The No Fly and Selectee list security directives also address the screening of air carrier

employees against the No Fly and Selectee lists, but our scope was confined to the

passenger-specific prescreening requirements in the security directives. 

17 Special emphasis assessments and special emphasis inspections are nonroutine activities 

undertaken at the direction of TSA headquarters. According to TSA, a special emphasis

assessment addresses a vulnerability that generally is not tied to a regulation, while a 

special emphasis inspection is tied to a regulatory requirement. 

Page 6 GAO-08-992 Aviation Security and Watch List Matching
the success rate of all attempted matches by air carriers. We did not assess 

the initial data TSA provided in February 2008 for the special emphasis 

inspection it conducted the previous month.18

We conducted this performance audit from July 2006 to September 2008 in

accordance with generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on the audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions 

based on the audit objectives. More details about the scope and 

methodology of our work are presented in appendix I. 

TSA has issued two security directives (one for the No Fly List and 

another for the Selectee List) that delineate requirements related to air 

carrier watch-list matching, including the identification of passengers with

names similar to those on the lists. Identifying passengers with names 

similar to those on the No Fly and Selectee lists—a process TSA refers to 

as similar-name matching—is a critical component of watch-list matching 

because individuals may travel using abbreviated name forms or other 

variations of their names. Therefore, searching for only an exact match of

the passenger’s name may not result in identifying all watch-listed 

individuals. There have been incidents, according to TSA’s Office of 

Intelligence, of air carriers failing to identify potential matches by not 

successfully conducting similar-name matching. Before revisions to the 

security directives were initiated in 2008, TSA expected air carriers to find 

similar names but provided no specificity on the extent to which air 

carriers should make these comparisons. The 14 air carriers we 

interviewed reported implementing varied approaches to similar-name 

matching. Because air carriers used different approaches, a passenger 

could be identified as a match to a watch-list record by one carrier and not 

by another carrier, which results in uneven effectiveness of watch-list 

matching. Generally, TSA had been aware that air carriers were not using 

equivalent processes to compare passenger names with names on the No

Fly and Selectee lists. However, in early 2008 the significance of such 

Results in Brief 

 18 In September 2008, TSA provided us the results of a special emphasis assessment 

(conducted during May 2008) of seven air carriers’ compliance with new requirements in 

the No Fly List security directive, which was revised in April 2008 to specify a baseline 

capability for conducting watch-list matching. This special emphasis assessment is 

discussed later in this report. 

Page 7 GAO-08-992 Aviation Security and Watch List Matching
differences was crystallized during the course of our review and following 

TSA’s special emphasis inspection of air carriers’ name-matching 

capability. On the basis of these inspection results, TSA issued a revised 

security directive governing the use of the No Fly List in April 2008 to 

establish a baseline capability for similar-name matching to which all air 

carriers must conform. Also, TSA announced that it plans to revise the 

Selectee List security directive to similarly require the new baseline 

capability.19 According to TSA officials, the new baseline capability is 

intended to improve the effectiveness of watch-list matching, particularly 

for those air carriers that did not compare the types of name variations 

specified by the new baseline capability or that compared none at all.

However, TSA officials noted that the new baseline is not intended to 

address all possible types of name variations and the related security 

vulnerabilities. Agency officials explained that based on their analysis of 

the No Fly and Selectee lists and interviews with intelligence community 

officials, the newly established baseline covers the most critical types of 

name variations. TSA officials further stated that this is an interim solution 

that will strengthen security while not requiring air carriers to invest in 

significant modifications to their watch-list-matching processes, given 

TSA’s expected implementation of Secure Flight beginning in 2009. These 

officials added that when implemented, Secure Flight will be better able to

use passenger names and other identifying information to more accurately 

match passengers to the subjects of watch-list records. 

TSA has undertaken various efforts to assess domestic air carriers’ 

compliance with watch-list-matching requirements in the No Fly and 

Selectee list security directives; however, until 2008, TSA had conducted 

limited testing of air carriers’ similar-name-matching capability. In 2005,

for instance, TSA conducted a special emphasis assessment that focused 

on air carriers’ capability to prescreen passengers for exact-name matches 

with the No Fly List, but did not address the air carriers’ capability to 

conduct similar-name comparisons. Regarding inspections conducted as 

part of regular inspection cycles, TSA’s guidance establishes that 

regulatory requirements encompassing critical layers of security need 

intensive oversight, and that testing is the preferred method for validating 

compliance. However, before being revised in 2008, TSA’s inspection 

 19 In September 2008, TSA informed us that the revised Selectee List security directive was 

still in the agency’s internal clearance process but did not provide us a targeted issuance 

date. 

Page 8 GAO-08-992 Aviation Security and Watch List Matching
guidelines (called PARIS prompts)20 for watch-list-related inspections were 

broadly stated and did not specifically direct inspectors to test air carriers’ 

similar-name-matching capability. Moreover, TSA’s guidance provided no 

baseline criteria or standards regarding the number or types of such 

variations that must be assessed. In response to our inquiry, 6 of TSA’s 9 

principal security inspectors told us that their assessments during annual

inspection cycles have not included examining air carriers’ capability to 

conduct certain basic types of similar-name comparisons. Also, in 

reviewing documentation of the results of the most recent inspection cycle 

(fiscal year 2007), we found that available records in TSA’s database made

references to name-matching tests in 6 of the 36 watch-list-related 

inspections that principal security inspectors conducted, and in 55 of the 

1,109 inspections that transportation security inspectors conducted.21

Without baseline criteria or standards for air carriers to follow in 

conducting similar-name comparisons, TSA has not had a uniform basis 

for assessing compliance. Further, without routinely and uniformly testing 

how effectively air carriers are conducting similar-name matching, TSA 

may not have had an accurate understanding of the quality of air carriers’ 

watch-list-matching processes. However, TSA began taking corrective 

actions during the course of our review and after it found deficiencies in 

the capability of air carriers to conduct similar-name matching during a 

January 2008 special emphasis inspection.22 More specifically, following 

the January 2008 inspection, TSA officials reported that TSA immediately 

began working with individual air carriers to address deficiencies. Also, 

officials reported that, following the issuance of TSA’s revised No Fly List 

security directive in April 2008, the agency had plans to assess air carriers’ 

progress in meeting the baseline capability specified in the new security 

directive after 30 days, and that the annual inspection plan for

transportation security inspectors would be revised to help ensure 

compliance by air carriers with requirements in the new security directive. 

In September 2008, TSA provided us with results from a May 2008 special 

emphasis assessment of seven air carriers’ compliance with the revised No

 20 PARIS is the acronym for the Performance and Results Information System, which is

TSA’s inspections database. This database assists TSA management by providing factual 

and analytical information on the compliance of TSA-regulated entities. There are

approximately 1,700 PARIS prompts, which serve as guidelines for TSA inspectors. 

21 According to TSA data, these 1,145 watch-list-related inspections (36 plus 1,109) covered 

60 domestic air carriers, and most of the air carriers were inspected multiple times. 

22 TSA reported that the January 2008 special emphasis inspection covered 52 domestic air

carriers and 31 foreign air carriers. 

Page 9 GAO-08-992 Aviation Security and Watch List Matching
Fly List security directive. Although the details of this special emphasis 

assessment are classified, TSA generally characterized the results as 

positive. Further, the TSA officials noted that the agency’s internal 

handbook—which provides guidance to transportation security inspectors 

on how to inspect air carriers’ performance of various requirements, 

including watch-list-matching requirements—was being revised and was 

expected to be released later this year. Thus, the TSA officials stated that 

the new inspection guidance would be used in conjunction with the 

nationwide regulatory activities plan for fiscal year 2009. While these 

actions and plans are positive developments, it is too early to determine 

the extent to which TSA will assess air carriers’ compliance with watchlist-matching requirements moving forward since these efforts are still 

underway. 

We provided a draft of our restricted report to DHS and the Department of 

Justice for review and comment. DHS had no comments. The Department 

of Justice provided technical comments to the restricted version of this 

report, which we incorporated where appropriate. 

TSA uses a layered system of defense to secure civil aviation whereby 

additional layers provide security when any one security measure may fail.

Watch-list matching is one such layer of defense. Air carriers began 

checking passenger names against government-supplied terrorist watch 

lists (compiled by the FBI and distributed by the Federal Aviation 

Administration) in the early 1990s. After the attacks of September 11, 2001, 

and the subsequent establishment of TSA during the same year, primary 

responsibility for civil aviation security, including overseeing the watchlist-matching process, fell to TSA.23 The Aviation and Transportation 

Security Act, enacted in November 2001, requires that a system be used to 

evaluate all passengers before they board an aircraft and ensure that 

selected individuals and their carry-on and checked baggage are 

adequately screened.24 TSA fulfilled this mandate by continuing to require 

Background 

 23 In accordance with 49 U.S.C. § 114(h), TSA adopted policies and procedures for ensuring

that air carriers use information from government agencies to identify individuals on

passenger lists who may be a threat to civil aviation or national security and, if such an 

individual is identified, notify appropriate law enforcement agencies, prevent the individual 

from boarding an aircraft, or take other appropriate action with respect to that individual.

24 Pub. L. No. 107-71, § 136, 115 Stat. 597, 637 (2001) (codified at 49 U.S.C. § 44903(j)(2)(A)) 

(requiring use of the Computer Assisted Passenger Prescreening System or any successor 

system).

Page 10 GAO-08-992 Aviation Security and Watch List Matching
and oversee air carrier operation of the Computer Assisted Passenger 

Prescreening System (CAPPS)—an electronic application that selects 

individuals for enhanced screening at the passenger checkpoint based on 

certain travel characteristics identified by TSA as indicating potential 

risk—and by issuing security directives in April 2002 that continued and 

amended the requirements that domestic air carriers match passenger 

information against the No Fly and Selectee lists. These security directives 

are the No Fly List Procedures security directive, requiring domestic air 

carriers to conduct checks of passenger information against the No Fly 

List to identify individuals who should be precluded from boarding flights,

and the Selectee List Procedures security directive, directing domestic air 

carriers to conduct checks of passenger information against the Selectee 

List to identify individuals who should receive enhanced screening (e.g., 

additional physical screening or a hand-search of carry-on baggage) before 

proceeding through the security checkpoint.25 Since 2002, TSA has issued 

numerous revisions to the No Fly and Selectee list security directives to 

strengthen and clarify requirements, and has issued guidance to assist air 

carriers in implementing their watch-list-matching processes.26

So that they may carry out watch-list-matching requirements, TSA 

provides air carriers with access to the No Fly and Selectee lists—subsets 

of the terrorist screening database managed by the FBI’s Terrorist 

Screening Center. The terrorist screening database is composed of records 

that contain identifying information (e.g., name and date of birth) on both 

foreign and U.S. citizens with known or appropriately suspected links to 

terrorism. Only those nominations in the terrorist screening database 

submitted by elements within the intelligence community, including the

 25 For the purposes of this report, we address policies and procedures applicable to air

carriers regulated under 49 C.F.R. part 1544 (U.S.-flagged air carriers), which we refer to as

domestic air carriers. For these air carriers, we limit our discussion to the watch-list 

matching TSA requires to secure the aviation sector for domestic flights—air carrier 

operations between two points within the United States or its territories. TSA requirements 

also address the international operations of domestic air carriers, and the operations of 

foreign-flagged air carriers flying to and from destinations within the United States and its 

territories in accordance with 49 U.S.C. part 1546; however, these requirements are outside 

the scope of our review. 

26 The most recent version of the No Fly List Procedures security directive is SD 1544-01-

20F, dated April 9, 2008, and the most recent version of the Selectee List Procedures 

security directive is SD 1544-01-21F, dated March 8, 2007. 

Page 11 GAO-08-992 Aviation Security and Watch List Matching
FBI, that meet criteria specified by the Homeland Security Council27

relating to the threat that an individual poses to civil aviation are exported 

as records to be included on the No Fly or Selectee lists.28 At present, the 

Terrorist Screening Center forwards the No Fly and Selectee lists to TSA’s 

Office of Intelligence, which generally posts new lists daily to a secure 

Web board that air carriers may access to retrieve the lists.29 The Terrorist 

Screening Center provides TSA’s Office of Intelligence with new No Fly 

and Selectee lists on a daily basis as well as any time a nominating entity

submits additions and deletions that require immediate notification to the

aviation community. 

TSA’s Regulatory 

Inspection Framework 

TSA is responsible for ensuring air carriers’ compliance with regulatory 

requirements, including requirements reflected in TSA security directives 

and TSA-approved security programs. According to TSA inspection 

guidance, compliance with regulatory requirements may be validated in

various ways, depending on the risk associated with the requirements. For 

example, when regulatory requirements are largely administrative and 

encompass the least critical layers of security, compliance may be 

validated largely through inspections based on documentation reviews. 

However, when regulatory requirements encompass more critical layers of 

security, more intensive oversight is needed, and compliance typically is to 

be validated through testing, inspections, surveillance, special emphasis 

assessments, and special emphasis inspections. 

TSA conducts inspections of air carriers throughout the year as part of 

regular inspection cycles based on annual inspection plans. These 

inspections are based on inspection guidelines known as PARIS prompts, 

 27 On June 10, 2008, the Department of Justice provided us comments on a draft of the 

restricted version of this report (GAO-08-453SU) and noted that the Principals Committee, 

which is a senior interagency forum under the Homeland Security Council, had approved

additional criteria that the Terrorist Screening Center would begin implementing on June 

23, 2008. The Homeland Security Council was established to ensure coordination of all 

homeland-security-related activities among executive departments and agencies and 

promote the effective development and implementation of all homeland security policies. 

See The White House, Homeland Security Presidential Directive/HSPD-1, Organization 

and Operation of the Homeland Security Council (Washington, D.C.: Oct. 29, 2001). 

28 Each watch-list record, however, does not necessarily indicate a separate individual on 

the list. Some listed individuals have multiple records attributed to them due to the 

inclusion of known aliases and name variations. 

29 The lists may also be provided via password-protected e-mail. 

Page 12 GAO-08-992 Aviation Security and Watch List Matching
which address a broad range of regulatory requirements (including airport 

perimeter security and cargo security, as well as screening of employees, 

baggage, and passengers). With respect to watch-list matching, an 

inspection guideline (PARIS prompt) instructs inspectors to determine, for 

example, whether the air carrier is comparing the names of all passengers 

against names on the most current No Fly and Selectee lists in accordance 

with the procedures outlined in TSA’s security directives. 

TSA conducts watch-list-related inspections at air carriers’ corporate 

security offices (where policies and procedures are established on how 

watch-list matching is to be performed) and at airports (where policies 

and procedures for responding to a potential match are implemented). 

TSA’s principal security inspectors are responsible for conducting 

inspections at domestic air carriers’ corporate headquarters. These 

inspectors assess air carriers’ compliance with security requirements and 

provide direct oversight of air carriers’ implementation of and compliance 

with TSA-approved security programs. TSA considers principal security 

inspectors to be subject-matter experts for the air carrier community 

concerning implementation of and compliance with security programs and 

other requirements. As of January 2008, nine principal security inspectors 

were responsible for assessing the compliance of domestic air carriers 

with requirements in the No Fly and Selectee list security directives (as 

well as with other regulatory requirements pertaining to commercial 

aviation). Each of these inspectors has responsibility for one or more 

domestic air carriers. For fiscal year 2007, there were 72 domestic air 

carriers to which the No Fly and Selectee list security directives applied.

Field inspectors—known as transportation security inspectors—conduct 

watch-list-related inspections at airports. They are responsible for a 

multitude of TSA-related activities, including conducting inspections and 

investigations of airports and air carriers, monitoring compliance with 

applicable civil aviation security policies and regulations, resolving routine 

situations that may be encountered in the assessment of airport security, 

participating in testing of security systems in connection with compliance 

inspections, identifying when enforcement actions should be initiated, 

and providing input on the type of action and level of penalty 

commensurate with the nature and severity of a violation that is ultimately 

recommended to TSA’s Office of Chief Counsel. As of June 2008, there 

were 681 transportation security inspectors responsible for 459 

commercial airports across the United States. 

Page 13 GAO-08-992 Aviation Security and Watch List Matching
TSA began developing a program to take over watch-list-matching 

capability from air carriers in March 2003.30 TSA cancelled this earlier 

effort, known as CAPPS II, due to development challenges and privacy 

concerns. In July 2004, the National Commission on Terrorist Attacks 

Upon the United States (the 9/11 Commission) recommended that the 

federal government take over the watch-list-matching function from air 

carriers.31 Subsequently, the Intelligence Reform and Terrorism Prevention 

Act of 2004 required that TSA develop such a watch-list-matching 

capability.32 Shortly after suspending work on the CAPPS II program in 

August 2004, TSA initiated development of Secure Flight, a program that 

the agency expects will allow the federal government to perform watch-list 

matching for passengers on all flights within the United States and 

ultimately for international flights with departures from or arrivals in the 

United States. 

Secure Flight: 

Development of a 

Government-Run WatchList-Matching Process 

In February 2006, we testified that although some progress had been made 

in developing Secure Flight, long-standing issues related to systems 

development and testing, program management, privacy protections, and 

redress remained.33 We reported in testimony that as a result of these 

deficiencies the program was at risk of failure. Following our February 

2006 testimony, TSA announced a temporary suspension of Secure Flight’s

development to reassess program goals and capabilities. TSA completed 

this reassessment in January 2007, moved forward to complete its 

 30 TSA initiated this effort in response to the Aviation and Transportation Security Act, 

which requires that TSA ensure that a system is used to evaluate all passengers before they

board an aircraft and ensure that selected individuals and their carry-on and checked

baggage are adequately screened. See Pub. L. No. 107-71, § 136, 115 Stat. at 637 (codified at 

49 U.S.C. § 44903(j)(2)(A)). 

31 The National Commission on Terrorist Attacks Upon the United States, The 9/11 

Commission Report – Final Report of the National Commission on Terrorist Attacks 

Upon the United States (Washington, D.C.: 2004), p. 393. 

32 Pub. L. No. 108-458, § 4012(a)(1), 118 Stat. 3638, 3714-17 (2004) (codified at 49 U.S.C. 

§ 44903(j)(2)(C) (2004)). A separate provision enacted at section 4012(a)(2) addressed the 

predeparture screening of international passengers, with the Secretary of Homeland

Security giving this responsibility to U.S. Customs and Border Protection. See 49 U.S.C. 

§ 44909(c)(6). 

33 With regard to redress protections, DHS must have a process whereby aviation 

passengers determined to pose a threat to aviation security by Secure Flight may appeal 

that determination and correct erroneous information contained within the prescreening

system. See GAO, Aviation Security: Significant Management Challenges May Adversely

Affect Implementation of the Transportation Security Administration’s Secure Flight

Program, GAO-06-374T (Washington, D.C.: Feb. 9, 2006). 

Page 14 GAO-08-992 Aviation Security and Watch List Matching
concept-of-operations plan for the Secure Flight program and strengthen 

systems development efforts, and, in August 2007, issued a notice of 

proposed rulemaking describing the requirements TSA will expect air 

carriers to implement to facilitate the government-run prescreening 

process.34 TSA expects that, beginning in early calendar year 2009, the 

Secure Flight program will begin assuming from air carriers the watch-listmatching responsibility for domestic flights. At some point following this 

assumption for domestic flights, TSA plans to assume from U.S. Customs 

and Border Protection this watch-list-matching function for international

flights that depart from or arrive in the United States. However, we 

testified in February 2008 that despite significant progress in the 

development of Secure Flight, TSA did not fully follow best practices for

developing Secure Flight’s life-cycle cost and schedule estimates, and that

failure to do so put the program at risk of cost overruns, missed deadlines, 

and performance shortfalls, among other issues.35

Through its security directives, TSA has issued requirements for watch-list 

matching, which include identifying passengers with names similar to 

those on the No Fly and Selectee lists—a process TSA refers to as similarname matching. Before undertaking revisions of the relevant security 

directives in 2008, TSA expected air carriers to conduct similar-name 

matching but TSA’s security directives did not specify how many and what 

types of such name variations air carriers should compare. Consequently, 

some of the 14 air carriers we interviewed reported that they compared 

more name variations than others. Air carriers that do not conduct similarname comparisons and carriers that conduct relatively limited 

comparisons are less effective in identifying watch-listed individuals who 

travel under name variations. Also, due to inconsistent air carrier 

processes, a passenger could be identified as a match by one carrier and 

not by another. In April 2008, during the course of our review, TSA revised 

and issued the No Fly List security directive to specify a baseline 

capability for similar-name matching to which all air carriers must 

conform. Also, in April 2008, TSA officials reported that the agency had 

TSA Took Action in 

2008 to Enhance 

Watch-List Matching 

Conducted by Air 

Carriers but Believes 

the Ultimate Solution 

Will Be 

Implementation of 

Secure Flight 

 34 See 72 Fed. Reg. 48,356 (Aug. 23, 2007). Requirements described in the notice of 

proposed rulemaking are subject to revisions based on various considerations, including

input that TSA received during the public comment period. As of the date of this report’s 

issuance, DHS had not issued a final Secure Flight rule. 

35 GAO, Aviation Security: Transportation Security Administration Has Strengthened

Planning to Guide Investments in Key Aviations Security Programs, but More Work

Remains, GAO-08-465T (Washington, D.C.: Feb. 28, 2008). 

Page 15 GAO-08-992 Aviation Security and Watch List Matching
plans to similarly revise the Selectee List security directive to require the 

same baseline capability.36 TSA officials acknowledged that the new 

baseline capability will not address all vulnerabilities identified by TSA. 

However, the officials stated that the new baseline capability was their 

best interim approach for improving air carriers’ matching efforts because, 

among other reasons, it will strengthen watch-list matching without 

requiring considerable investment in a solution that will be replaced when 

Secure Flight is implemented. TSA officials further stated that the longer 

term solution for watch-list matching is Secure Flight, which will have the 

capability to undertake more advanced searches for individuals on the No 

Fly and Selectee lists. 

Prior to April 2008, TSA 

Watch-List-Matching 

Requirements Were Broad 

and Allowed Air Carriers 

to Implement Less 

Effective Processes 

Prior to a revision of the No Fly List security directive in April 2008—and a 

similar revision planned for the Selectee List security directive—TSA’s 

watch-list-matching requirements for domestic flights (summarized in 

table 1) addressed five key processes: (1) retrieval of the No Fly and 

Selectee lists, (2) the matching of passenger and list information, (3) the 

use of TSA’s Cleared List,

37 (4) notification procedures, and (5) recordkeeping activities.38 In April 2008, TSA revised the No Fly List security 

directive for watch-list matching and also reported plans for similarly 

revising the Selectee List security directive. The security directive 

revisions—discussed later in this section—still address the five key 

process areas, but provide greater specificity on TSA’s requirements for 

matching passenger and watch-list information (the second key process 

 36 In September 2008, TSA informed us that the revised Selectee List security directive was 

still in the agency’s internal clearance process but did not provide us a targeted issuance 

date. 

37 When making determinations on matches, air carriers must use the TSA Cleared List, 

which is composed of names and other personal-identifying information on individuals 

whom the Department of Homeland Security has reviewed and determined are not 

individuals on the No Fly or Selectee lists. 

38 Specifically, we reviewed and discussed the No Fly and Selectee list security directives 

and identified within each the key requirements pertaining to domestic flights. Although

the same requirements generally apply to the international flights of both domestic and 

foreign air carriers, such operations fall outside the scope of our review. For more 

information on how we identified requirements for watch-list matching, see appendix I. 

Page 16 GAO-08-992 Aviation Security and Watch List Matching
shown in table 1).39 Prior to the April 2008 revision of the No Fly List 

security directive, TSA’s requirements in this area lacked specificity for 

purposes of implementation, although the then-current security directives 

addressed the need for air carriers to identify passengers with names that 

are either identical or similar to those on the No Fly List or the Selectee 

List. To identify passengers with similar names—an activity known as 

similar-name matching—air carriers’ automated programs or manual 

reviews were expected to capture No Fly and Selectee list names that are 

variations of the name on the passenger’s reservation. 

 39 TSA’s revised No Fly List Procedures security directive (SD 1544-01-20F) is dated April 9,

2008. Also, in April 2008, TSA reported that the current Selectee List Procedures security 

directive (SD 1544-01-21F) would be similarly revised. In September 2008, TSA informed us

that the revised Selectee List security directive was still in the agency’s internal clearance

process but did not provide us a targeted issuance date. 

Page 17 GAO-08-992 Aviation Security and Watch List Matching
Table 1: TSA Watch-List-Matching Requirements Prior to the April 2008 Revision to the No Fly List Security Directive 

Requirements (key processes) Discussion

(1) Retrieving the No Fly and Selectee lists • Air carriers must monitor the TSA Web board throughout the day for the most recent 

postings of the No Fly and Selectee lists. 

(2) Matching passenger data to No Fly and 

Selectee lists 

• Within 24 hours of scheduled flight departure time, but no later than passenger checkin, air carriers are to compare records from the most recently issued No Fly and 

Selectee lists with identifying information on passengers found in the respective air 

carrier’s reservation system and offered by passengers at the time of check-in. 

• When comparing data, air carriers must identify name matches to the No Fly and 

Selectee lists. To identify similar-name matches, automated and manual processes 

are expected to have the capability to compare name variations. 

• To determine which passengers are matches, a passenger’s name and one piece of 

identifying information (found either within the air carrier’s reservation system or 

supplied by the passenger at check-in) must match with corresponding information 

provided on the No Fly or Selectee lists. 

(3) Using the TSA Cleared List

a • When making determinations on matches, air carriers must use the TSA Cleared List, 

which is composed of names and other personal-identifying information on individuals 

whom the Department of Homeland Security has reviewed and determined are not 

individuals on the No Fly or Selectee lists. Individuals determined to be on the TSA 

Cleared List should be accepted for travel and not be subject to further procedures for 

handling matches to No Fly or Selectee lists identified in the security directives. 

(4) Notifying authorities • Upon identifying a passenger whose information matches with the No Fly or Selectee 

lists and who is not on the TSA Cleared List, air carriers must follow certain 

notification procedures, such as to contact the federal security director and the 

appropriate local law enforcement officer (for matches to the No Fly List) or to 

designate the passenger as a selectee for enhanced checkpoint screening

procedures (for matches to the Selectee List). 

(5) Keeping records • Air carriers must keep records on the results of watch-list matching for specified time 

periods—for example, air carriers must keep a record of all flights operated with 

passengers designated as selectees for 7 calendar days from the date of the flight’s 

departure. 

Sources: GAO analysis of TSA’s No Fly List Procedures security directive (SD 1544-01-20 series) and Selectee List Procedures

security directive (SD 1544-01-21 series), versions dated July 8, 2004, and March 8, 2007. 

a

Security directives in effect prior to the April 2008 revision of the No Fly List Procedures security

directive referenced a “cleared column,” a format for clearing passengers. TSA eventually replaced

this format with the Cleared List, and revised language for the April 2008 No Fly List security 

directive. 

Air carriers must conduct similar-name matching because watch-listed 

individuals may travel using variations of the names attributed to them on

the No Fly or Selectee lists and, thus, would not be identified if air carriers 

searched only for an exact-name match. At present, TSA does not require 

that air carriers collect the full name from passengers making travel 

reservations, thus, passengers may travel using variations of their legally

documented names; for example, abbreviated name forms or portions of 

their names. Such name variations may arise due to unintentional 

errors—for example, a travel agent mistakenly books travel for “Jon” 

Page 18 GAO-08-992 Aviation Security and Watch List Matching
when the name spelling is actually “John,” or the agent accidentally 

transposes a passenger’s first and middle names for a flight reservation. 

Traveling under a name variation could also represent a watch-listed 

individual’s intentional effort to evade detection. For example, an 

individual identified as John Robert Smith on his driver’s license may 

make a travel reservation using a common name variation—such as using 

his middle and last names (Robert Smith) or his initials and last name (J.R. 

Smith). If the John Robert Smith in this example were a name on the No 

Fly List, an exact, letter-for-letter comparison of the passenger’s 

reservation name (either Robert Smith or J.R. Smith) with the No Fly List 

would fail to identify the watch-listed individual. However, a comparison 

of possible variations of the watch-list name (John Robert Smith) could 

identify either Robert Smith or J.R. Smith as a potential match—that is, an 

individual who is a possible match to the No Fly List or Selectee List and 

whose personal identifying information requires further review before a 

match can be determined. 

Regarding similar-name matching, before 2008, TSA’s security directives 

had broad requirements that allowed air carriers discretion in determining 

the extent to which they compared name variations. For instance, to 

identify watch-listed individuals who travel using variations of their name,

TSA’s security directives did not specify how many possible combinations 

of name elements should be compared. TSA officials explained that the 

agency initially issued broad security directives to allow air carriers 

flexibility in implementing requirements and—until the April 2008 revision

of the No Fly List security directive—left the directives relatively 

unchanged because the agency was developing a government-run 

capability to take over this function. The operations of those air carriers 

that are subject to the watch-list-matching requirements of TSA’s security 

directives range from commuter providers to international-service 

providers. According to TSA officials, broad security directive 

requirements permit air carriers with such diverse operations to 

implement processes that best meet their operational needs and 

technological capabilities. 

Before 2008, TSA’s Security 

Directives Allowed Air Carriers 

More Discretion in Comparing 

Name Variations 

Officials further explained that TSA’s focus has been on developing its 

own watch-list-matching capability (now Secure Flight) since 2003. TSA 

officials noted that, though not an impetus for making requirements broad 

when first articulated in 2002, this focus on developing a government-run 

watch-list-matching program is one reason why these requirements 

remained relatively unchanged until April 2008. 

Page 19 GAO-08-992 Aviation Security and Watch List Matching
The 14 air carriers we interviewed reported adopting different approaches 

to name matching. Although each of the 14 air carriers we spoke with 

during our review reported conducting comparisons to identify exactname matches of passengers and names on the No Fly List or the Selectee 

List, not every air carrier reported conducting similar-name comparisons.40

Those air carriers that conducted similar-name comparisons reported 

using various approaches, some of which compared more name variations 

than others. 

Failure to Conduct SimilarName Matching or Comparing 

Name Variations to a Lesser 

Extent Reduces the 

Effectiveness of Watch-List 

Matching 

According to air carriers, a critical factor affecting their implementation of 

similar-name-matching requirements was their observation that 

conducting more comparisons for variations results in longer lines at 

ticket counters and passenger inconvenience. Specifically, 10 air carriers 

commented that conducting similar-name comparisons resulted in more 

passengers being identified as potential matches. At the time of check-in, 

air carriers must perform additional checks at the ticket counter of each

potentially matched passenger’s government-issued identification against 

data on the No Fly and Selectee lists. Therefore, according to 12 of the 14

air carriers we spoke with, a large number of potential matches can lead to 

congestion at the ticket counter and longer wait times for all passengers. 

Inconsistent approaches to conducting similar-name matching could lead 

a passenger to be identified as a match by one air carrier and not by 

another. Further, not conducting similar-name matching—or conducting 

such matching to only a very limited extent—compromises the usefulness

of the No Fly List and Selectee List. There have been incidents, according

to TSA’s Office of Intelligence, of air carriers failing to identify potential

matches by not effectively conducting similar-name matching. In these 

incidents, the air carriers’ processes led to false negative watch-listmatching results—that is, individuals who were on the No Fly List and 

were not identified by the respective air carrier’s watch-list-matching 

process. In some of these incidents, the individual’s flight reservation 

contained a name that varied somewhat from the name on the No Fly List, 

and the air carrier’s watch-list-matching process did not identify the name 

as a possible match. 

 40 We did not independently verify the air carriers’ approaches to watch-list matching. 

Unless noted otherwise, our summary of the air carriers’ approaches is based on system 

capabilities reported to us in 14 separate interviews with the respective air carriers. 

Appendix II provides more detail on the 14 air carriers’ reported approaches to watch-list 

matching. 

Page 20 GAO-08-992 Aviation Security and Watch List Matching
In most of these cases, the failures of the air carriers to identify the 

potential matches were discovered as a result of the U.S. Customs and 

Border Protection’s comparison of passenger and watch-list data for 

international flights. Specifically, TSA learned of the failures through U.S. 

Customs and Border Protection, which identified the No Fly listed 

individual when conducting its own comparison of passenger information 

against the No Fly and Selectee lists for international flights.

41 These 

comparisons, performed as part of U.S. Customs and Border Protection’s 

border security mission, took place after the air carriers completed their 

comparisons, in effect constituting a second check of passenger and 

watch-list information. U.S. Customs and Border Protection does not 

screen passengers on domestic flights; thus, there is no opportunity for a 

second comparison of passenger information against the No Fly and 

Selectee lists for domestic flights. Therefore, it is difficult to determine the 

extent to which domestic air carriers may be failing to identify watchlisted individuals who are able to board domestic flights. 

In October 2007, we reported that of the known cases in which individuals 

on the No Fly List flew on international flights bound to or from the United 

States, some were allowed to fly because the respective air carrier’s 

process failed to identify the passenger’s name as a match.42 Although 

these individuals were subsequently identified in-flight by other means, the 

onboard security threats required an immediate counterterrorism 

response, which in some instances resulted in diverting the aircraft to a 

location other than its original destination.43 According to TSA’s Office of 

Intelligence, some of these incidents may be attributed to air carriers’ 

inability to identify similar-name matches when passengers travel using 

variations of their name.

TSA had been aware that air carriers were not using equivalent processes 

to compare passenger names with names on the No Fly and Selectee lists. 

For instance, in June 2006, we reported that the improvements air carriers 

 41 Some of these flights involved passengers who flew from one domestic location to

another domestic location, where they boarded an international flight. TSA learned that the 

individual on the No Fly List flew domestically after U.S. Customs and Border Protection 

identified the individual on the international leg. 

42 GAO, Terrorist Watch List Screening: Opportunities Exist to Enhance Management 

Oversight, Reduce Vulnerabilities in Agency Screening Processes, and Expand the Use of

the List, GAO-08-110 (Washington, D.C.: Oct. 11, 2007). 

43 GAO-08-110. 

Page 21 GAO-08-992 Aviation Security and Watch List Matching
were making to their individual watch-list-matching processes, though 

beneficial to the respective air carrier’s operations, could further 

exacerbate differences that currently exist among the various air carriers 

and could result in varying levels of effectiveness across air carriers in 

matching passenger information to the No Fly and Selectee lists.44

Furthermore, TSA’s March 2007 Secure Flight Program Baseline explained 

“because each aircraft operator conducts its own matching process, the 

ability to conduct watch-list matching and coordinate law enforcement 

responses is not consistent across the aviation industry.”45 Moreover, in 

several interviews over the course of our work, TSA officials 

acknowledged that in general, some air carriers were performing more 

similar-name comparisons than other air carriers. TSA’s understanding of 

the significance of these differences was crystallized in January 2008, 

when results of a special emphasis inspection identified deficiencies in air 

carriers’ similar-name-matching capability. 

To Address Deficiencies in 

Air Carriers’ Similar-NameMatching Capability, TSA 

Issued a Revised No Fly 

List Security Directive in 

April 2008 to Provide More 

Specific Requirements 

During the course of our work and in response to findings of the January 

2008 special emphasis inspection that identified deficiencies in air carriers’ 

similar-name-matching capability, TSA officials reported that the agency 

immediately began to assess options for corrective actions to implement 

across the aviation industry. In doing so, officials noted that they 

consulted with representatives from the intelligence community, the 

Secure Flight program, and the aviation industry. On the basis of its 

assessment, TSA revised the No Fly List security directive in April 2008 to

establish a specific baseline capability for air carriers in conducting 

similar-name matching. Also, in April 2008, TSA officials reported that the 

agency had plans to similarly revise the Selectee List security directive to

require the same baseline capability.46

 44 GAO, Aviation Security: Management Challenges Remain for the Transportation 

Security Administration’s Secure Flight Program, GAO-06-864T (Washington, D.C.: June 

14, 2006). 

45 Upon completing a reassessment of the Secure Flight program in February 2007, TSA 

produced this document to identify decisions made about Secure Flight’s capabilities 

during the reassessment. See TSA, Secure Flight Program Baseline (Washington, D.C.:

March 2007), p. 5. 

46 As mentioned previously, in September 2008, TSA informed us that the revised Selectee 

List security directive was still in the agency’s internal clearance process but did not 

provide us a targeted issuance date. 

Page 22 GAO-08-992 Aviation Security and Watch List Matching
TSA officials acknowledged that the new baseline capability will not 

address all vulnerabilities identified by TSA. However, TSA officials 

explained that they expect the new similar-name matching baseline 

capability to strengthen the watch-list matching currently performed by air 

carriers. In particular, the officials expect the newly established baseline

capability to improve the matching processes of those air carriers that do 

not compare the kinds of variations required by the new baseline or that 

compare none at all. Furthermore, according to agency officials, the 

variations specified by the new baseline address the types of situations air 

carriers will encounter due to passengers making their own reservations. 

Accordingly, TSA concluded that requiring air carriers to conduct similarname comparisons beyond the baseline capability specified in the revised 

No Fly List security directive was not warranted for the interim period 

pending the implementation of Secure Flight. TSA was not able to provide 

us with data or analysis to support this assertion, and we did not 

undertake an independent analysis to determine the sufficiency of the 

newly established baseline. 

TSA officials also explained they determined that revising the security 

directives to be the most feasible approach for strengthening the current 

watch-list-matching process over other options because it was expedient 

and would have the least negative impact on air carriers’ operations. 

Specifically, TSA officials determined that upon issuing the revised No Fly 

List security directive, air carriers would need only 2 to 4 weeks to 

implement new requirements. When considering how this option would 

affect air carrier operations, TSA officials explained that they considered 

the number of potential matches that likely would be generated by the 

new baseline capability. As previously discussed, air carriers reported that

comparing more name variations results in more passengers being 

identified as potential matches, who then must go to the ticket counter to

obtain their boarding passes. Thus, large numbers of potential matches 

could overwhelm air carriers’ check-in operations. TSA officials explained 

that the industry officials with whom they consulted in developing the new 

baseline capability believed it would produce a manageable number of 

potential matches. 

In exploring actions to strengthen the watch-list-matching process, TSA 

considered two other options—one that would have required each air 

carrier to contract with third-party providers to develop customized 

watch-list-matching software, and another that involved the creation of an 

expanded version of the No Fly and Selectee lists to include name 

variations so that air carriers need only conduct comparisons to identify 

an identical match. TSA identified significant obstacles to implementing 

Page 23 GAO-08-992 Aviation Security and Watch List Matching
these options. Specifically, TSA determined that contracting with thirdparty vendors was impracticable due to availability and timing concerns. 

For instance, identifying appropriate vendors and implementing vendorprovided solutions could take almost 2 years—an unrealistic time frame 

given that Secure Flight’s implementation is scheduled to begin in 2009. In

this regard, TSA officials also expressed reluctance to requiring air 

carriers to undertake the expense of contracting with third-party vendors

for an interim approach, while at the same time requiring that air carriers 

invest in system changes for Secure Flight. With regard to the option of 

adding name variations to the No Fly and Selectee lists, according to TSA 

officials, creating these variations would have greatly expanded the total

size of the No Fly List, which could overwhelm the name-matching 

capability of some air carriers and could potentially send an 

unmanageable number of potential matches to the ticket counters of air 

carriers. As previously discussed, in our air carrier interviews, 10 of the 14

air carriers reported that searching for more name variations leads to the 

identification of more potential matches. In this regard, there is some 

support for TSA’s determination that expansion of the No Fly and Selectee 

lists could produce an unmanageable number of potential matches. 

However, we did not independently assess this issue. 

Although TSA officials characterized the new baseline capability as a good

interim solution for strengthening watch-list matching—one that balances 

TSA’s need to strengthen watch-list matching with the air carriers’ need 

for efficient operations—they stressed that the Secure Flight program is 

ultimately the solution. For example, in its development of Secure Flight, 

TSA plans to develop a name-matching process that will have the 

capability to identify name variations beyond those specified by the new 

baseline. Further, according to TSA, Secure Flight will be better able to 

use passenger names and other identifying information (such as date of 

birth and gender) to more accurately match passengers to the subjects of 

watch-list records and, thereby, further reduce the risks of false negatives 

without unacceptably increasing the number of false positives (mistakenly 

identifying a passenger’s name as a potential match with watch-list 

records). 

Page 24 GAO-08-992 Aviation Security and Watch List Matching
Although TSA assessed air carriers’ compliance with watch-list-matching 

requirements through a special emphasis assessment conducted in 2005 

and through planned inspections conducted in conjunction with annual 

inspection cycles, the agency had tested similar-name matching to a 

limited extent until 2008. For instance, the 2005 special emphasis 

assessment focused on air carriers’ capability to identify passenger names 

that were exact matches with names on the No Fly List, but did not 

address the capability to conduct similar-name matching. Also, during the 

most recent annual inspection cycle (fiscal year 2007), although some TSA 

inspectors tested air carriers’ effectiveness in conducting similar-name 

matching, the inspectors did so at their own discretion and without 

specific evaluation criteria. However, during a special emphasis inspection 

conducted in January 2008, TSA found deficiencies in the capability of air 

carriers to conduct similar-name matching.47 Thereafter, following TSA’s 

revision of the No Fly List security directive in April 2008, officials planned 

to issue new guidance for inspectors to better ensure compliance by air 

carriers with requirements in the new security directive (e.g., by providing 

uniform evaluation criteria consistent with the new requirements). In 

response to our request for updated information on its oversight efforts, 

TSA provided us the results of a special emphasis assessment (conducted 

in May 2008) of seven air carriers’ compliance with the revised No Fly List 

security directive. Although the details of this special emphasis 

assessment are classified, TSA officials generally characterized the results 

as positive. Further, TSA’s noted that the agency’s internal handbook—

which provides guidance to transportation security inspectors on how to

inspect air carriers’ performance of various requirements, including watchlist-matching requirements—was being revised and was expected to be

released later this year. Thus, TSA indicated that the new inspection 

guidance would be used in conjunction with the nationwide regulatory 

activities plan for fiscal year 2009. While these actions and plans are 

positive developments, it is too soon to determine the extent to which air 

carriers’ compliance with watch-list-matching requirements will be 

assessed based on the new security directives since these efforts are still 

underway. 

Until a 2008 Special 

Emphasis Inspection, 

TSA Had Conducted 

Limited Testing of Air 

Carriers’ Capability to 

Perform Similar-Name

Matching 

 47 TSA reported that the January 2008 special emphasis inspection covered 52 domestic air

carriers and 31 foreign air carriers. 

Page 25 GAO-08-992 Aviation Security and Watch List Matching
TSA conducted a special emphasis assessment in 2005 that tested the 

capability of domestic air carriers to find passenger names that were exact 

matches to names on the No Fly List. The 2005 special emphasis 

assessment was undertaken at the request of the TSA Administrator due to 

serious failures in air carriers’ watch-list-matching processes, according to

a senior TSA official. To conduct the assessment, TSA inspectors made 

flight reservations using the exact name of an individual who was on the 

No Fly List and not on the TSA Cleared List. If the air carrier identified the 

name on the reservation as a potential match to the individual on the No 

Fly List—and the check-in agent identified through the reservation system

that further assistance was needed to finish the check-in process (e.g., to 

call security)—the test was considered to be successfully completed. 

According to TSA data: 

TSA’s Special Emphasis 

Assessment in 2005 

Focused on Air Carriers’ 

Exact-Name-Matching 

Capability 

• air carriers passed a large majority of the initial tests conducted in 

June and July 2005, although several air carriers failed one or more 

tests and 

• those air carriers that failed a test were retested in September 2005, 

and a large majority of these air carriers passed the tests.48

Although TSA conducted a large number of tests, TSA officials stated—

and our own analyses confirmed—that results from this special emphasis 

assessment would not produce a reliable estimate of the success rate of all 

attempted matches by air carriers because TSA did not randomly select 

the air carriers, airports, or individual flights for review. As a result, the 

findings from this assessment cannot be used to infer overall or individual 

rates of success in identifying exact name matches in accordance with the

No Fly and Selectee list security directives. That is, although the 2005 

special emphasis assessment provided insight into air carriers’ 

effectiveness in conducting a basic form of name matching, the picture 

provided was incomplete. Moreover, the air carriers’ failure rates may 

have been considerably higher had the special emphasis assessment tested 

similar-name-matching capability, given that this capability involves more 

 48 According to TSA officials, the agency had planned to conduct tests of all 81 domestic air 

carriers that were subject to the No Fly List Procedures security directive at that time. 

However, the officials explained that due to limited resources, initial testing covered 63 air 

carriers (encompassing operations at 354 airports), and the retesting covered 36 air carriers 

(encompassing operations at 290 airports). 

Page 26 GAO-08-992 Aviation Security and Watch List Matching
than finding a name that is a letter-for-letter match to another name. 

However, TSA officials told us that at the time of the special emphasis 

assessment in 2005, exact-name matching was the agency’s focus. 

TSA Conducted Planned 

Watch-List-Related 

Inspections throughout the 

Year, but Inspectors Tested 

Air Carriers’ Effectiveness 

at Similar-Name Matching 

at Their Own Discretion 

and without Baseline 

Evaluation Criteria 

Since issuing the No Fly and Selectee list security directives in 2002, TSA 

has incorporated watch-list-related inspections into its regular inspection 

cycle, but inspectors tested air carriers’ effectiveness in similar-name 

matching during these planned inspections to a limited extent and without 

specific evaluation criteria. In the most recent annual inspection cycle 

(fiscal year 2007), TSA conducted 1,145 inspections of air carriers’ 

compliance with watch-list-related requirements in the No Fly and 

Selectee security directives; 1,109 of these inspections were conducted at 

air carriers’ airport locations by transportation security inspectors and 36 

at air carriers’ corporate security offices by principal security inspectors.49

The 1,145 inspections covered 60 of the 72 domestic air carriers to which 

the security directives applied during fiscal year 2007, and most of the 

carriers were inspected multiple times that year.50 TSA found air carriers in 

compliance with required procedures in 1,133 (99 percent) of the 1,145 

inspections.51

These inspections were based on one or more inspection guidelines 

(called PARIS prompts) and were sometimes conducted in combination 

with inspections related to other regulatory requirements, such as 

performing criminal history record checks on employees or implementing 

CAPPS procedures. Table 2 presents the inspection guidelines TSA used to 

assess a key security directive requirement that we reviewed—matching 

 49 As noted earlier, we concluded that these inspection data were sufficiently reliable for 

the purposes of this report, but we have concerns about the potential for error based on 

TSA’s process for querying its inspection database (we discuss these concerns in more 

detail in app. I). 

50 Regarding the air carriers that did not receive a watch-list-related inspection during fiscal 

year 2007, TSA does not require inspectors to inspect each air carrier every year in terms of 

watch-list-related requirements. However, a senior TSA official in the compliance area who 

supervises inspectors stated that annually inspecting every air carrier is a goal, at least for 

principal security inspectors. 

51 We did not evaluate the basis for the inspectors’ assessment decisions regarding 

compliance with requirements. Although TSA’s security directives require comparisons of

passenger and employee names to the No Fly and Selectee lists, our review was confined to

requirements related to passengers only. 

Page 27 GAO-08-992 Aviation Security and Watch List Matching
passenger names to the No Fly and Selectee lists.

52 Additional guidelines 

used to assess other requirements in our review are presented in appendix

I.53

Table 2: Requirements for Matching Passenger Data to No Fly and Selectee Lists and Inspection Guidelines Used to Assess 

Compliance with the Requirements 

Inspection guidelines Requirements for matching passenger 

data to No Fly and Selectee lists Transportation security inspectors Principal security inspectors 

• Within 24 hours of scheduled flight 

departure time, air carriers are to 

compare records from the most recently 

issued No Fly and Selectee lists with 

identifying information on passengers 

found in the respective air carrier’s 

reservation system and offered by 

passengers at the time of check-in. 

• When comparing data, air carriers must 

identify name matches (including similarname matches) to the No Fly and 

Selectee lists. 

• To determine which passengers are 

matches, a passenger’s name and one 

piece of identifying information (found 

either within the air carriers’ reservation 

system or supplied by the passenger at 

check-in) must match with 

corresponding information provided on 

the No Fly or Selectee lists. 

• All passenger names are compared to 

the most current No Fly and Selectee 

lists. 

• The aircraft operator is comparing all 

passenger names to the most current No 

Fly and Selectee lists in accordance with 

the procedures outlined in Security 

Directive 1544-01-20 series (No Fly) and 

Security Directive 1544-01-21 series 

(Selectee). 

• Procedures are in place to ensure the 

most recently issued No Fly List is 

utilized within 24 hours of receipt. 

• Procedures are in place to ensure the 

most recently issued Selectee List is 

utilized within 24 hours of receipt. 

• Procedures are in place to contact the 

Federal Security Director, local law 

enforcement, the FBI, and TSA Office of 

Intelligence for matches to the No Fly 

List. 

• Records are maintained of all flights 

operated with passengers who were 

determined by local law enforcement, 

U.S. legal attaché, or TSA Office of 

Intelligence not to be a match. 

Sources: GAO analysis of TSA’s No Fly List Procedures security directive (SD 1544-01-20 series) and Selectee List Procedures

security directive (SD 1544-01-21 series), versions dated July 8, 2004, and March 8, 2007, and inspection guidelines applicable during 

fiscal year 2007. 

The inspections conducted by transportation security inspectors at 

airports used the guidelines in table 2 to assess air carriers’ compliance in

matching passenger data to the No Fly and Selectee lists in fiscal year 

2007. However, these inspectors tested exact-name and similar-name 

matching during these inspections at their own discretion; moreover, an 

official in TSA’s Office of Security Operations, Compliance Division, stated

that, generally, transportation security inspectors test exact-name- 

 52 To report their findings in TSA’s automated database, inspectors select one of four 

options from a computer-generated list: not inspected, not applicable, not in compliance, 

and in compliance. If the inspectors wish to add narrative to describe their findings, they

can do so in a data field reserved for comments. 

53 In appendix I, see table 3. 

Page 28 GAO-08-992 Aviation Security and Watch List Matching
matching capability only. This inspection guideline is broadly written and 

does not specify the methods for validating compliance with the 

requirement to perform name comparisons. According to a TSA official in 

the Office of Security Operations, field inspectors may validate compliance 

by asking check-in agents to demonstrate that they have access to the 

current No Fly and Selectee lists and that any hard copies of the lists are 

properly protected; they may also interview check-in agents to ensure that

they understand the security directive requirements, observe them as they 

process passengers who have been identified as Selectee or No Fly 

individuals, and/or test the air carriers’ system by requesting a gate pass in 

the name of an individual on the watch list. We found evidence of field

inspectors testing air carriers’ name matching systems in 55 of the 1,109 

inspections they conducted in fiscal year 2007 (such tests may have been 

administered during the other inspections conducted in fiscal year 2007 

but were not documented). 

For the 36 inspections conducted by principal security inspectors at air 

carriers’ corporate security offices, we found 6 inspection records that 

referred to tests of exact-name and similar-name matching capability (they 

may have administered such tests during the other inspections they 

conducted that year but did not document the tests). Principal security 

inspectors did not have an inspection guideline directing them to assess 

exact-name and similar-name matching capability specifically—thus they 

tested this capability at their own initiative, and then reported their 

methods and results in conjunction with one of the four guidelines 

presented in table 2. Further, in response to our inquiry, 6 of TSA’s 9 

principal security inspectors told us that their assessments have not 

included examining air carriers’ capability to conduct certain basic types 

of similar-name comparisons. 

TSA establishes in guidance for inspections (including watch-list-related 

inspections) that testing is the preferred method for assessing air carriers’ 

compliance with regulations whenever possible and that it is only through 

testing that security can be assured.54 TSA further establishes in inspection 

guidance that when regulatory requirements encompass critical layers of 

security, more intensive oversight is needed, and compliance typically is to 

be validated through testing, inspections, surveillance, special emphasis 

 54 TSA, National Inspection Manual, 2007. Inspections for all regulated areas (not just 

watch-list-related inspections) generally incorporate all of four methods—testing, 

document review, interviews, and surveillance.

Page 29 GAO-08-992 Aviation Security and Watch List Matching
assessments, and special emphasis inspections.55 Without routinely testing 

air carriers’ compliance with the similar-name-matching requirement, TSA 

may not have reliable data on the effectiveness of air carriers’ watch-listmatching processes and could be hindered in taking timely action to 

address any deficiencies. 

Inspectors who have tested air carriers’ effectiveness in performing 

similar-name matching have done so without specific evaluation criteria.

As discussed earlier, for any given name there are a number of possible 

name variations that could be used for travel, but TSA inspectors did not 

have baseline criteria on the number or types of such variations that must 

be evaluated. In the absence of specific standards for similar-name 

matching that all air carriers must follow, TSA has had no assurance that 

its inspections are based on uniform evaluation criteria. The inspections 

may not have been conducted uniformly and may have produced 

inconsistent results, given the absence of specific standards. In fall 2007, 

TSA began to review the adequacy of inspection guidance used by 

principal security inspectors, including guidance for watch-list-related 

inspections. As discussed in the following section, TSA expects to provide 

baseline criteria on the number and types of such variations inspectors 

must evaluate, but had not completed these efforts as of early September 

2008. 

A Special Emphasis 

Inspection Conducted in 

2008 Found Deficiencies in 

Air Carriers’ Similar-NameMatching Capabilities, and 

TSA Has Plans for 

Corrective Actions 

During the course of our review and following TSA’s discovery of a major 

air carrier’s inability to effectively conduct both exact-name and similarname-matching against the No Fly List, TSA initiated a 3-day, special 

emphasis inspection in January 2008 that tested the capability of 83 air 

carriers to conduct watch-list matching.56 According to TSA officials, this 

inspection covered 52 domestic air carriers and 31 foreign air carriers. To 

implement the special emphasis inspection, TSA used 100 names on the 

No Fly List to test the 83 air carriers’ capability to identify both exactname and similar-name matches based on various types of possible name 

variations. On the basis of test results, a senior TSA official stated that the

agency has confidence in air carriers’ capability to identify exact-name 

matches. Regarding the capability to identify similar-name matches, TSA 

 55 TSA, Regulatory Activities Plan for Transportation Security Inspectors Fiscal Year

2008. 

56 We briefed the TSA Administrator and other senior officials on the results of our work in

November 2007. 

Page 30 GAO-08-992 Aviation Security and Watch List Matching
found that no air carrier was successful in identifying matches involving 

all types of name variations, although some carriers were more effective 

than others. 

On the basis of this inspection, TSA officials stated that they began to

strengthen oversight of air carriers’ similar-name-matching capability. For

example, the TSA officials explained that—after a 30-day period following 

issuance of the revised No Fly List security directive in April 2008—the 

agency’s inspectors would begin to evaluate air carriers’ performance in 

complying with the new requirements. TSA officials explained that these 

initial inspections would be conducted at air carriers’ corporate security 

offices and at airports. Officials further stated that after these initial 

inspections, others would be conducted periodically and, if applicable, 

TSA would impose progressively stronger enforcement actions against air 

carriers that are not successful in meeting the new standards. 

In September 2008, in response to our request for updated information on 

the status of its oversight efforts, TSA provided us the results of a special

emphasis assessment (conducted during May 20–29, 2008) of seven air 

carriers’ compliance with new requirements in the No Fly List security 

directive. Although the details of this special emphasis assessment are 

classified, TSA generally characterized the results as positive. Also, TSA 

plans to work with individual air carriers, as applicable, to analyze specific

failures, improve system performance, and conduct follow-up testing as 

needed. 

In further reference to revision of the No Fly List security directive in April 

2008, TSA officials stated that the agency’s internal guidance is being 

updated to align inspection guidance with the revised directive. The 

officials elaborated that the new inspection guidance will place more 

emphasis on testing the effectiveness of security measures rather than 

using a checklist approach to determine whether an air carrier has a 

particular procedure in place. Regarding the emphasis on testing, our 

review noted that the draft guidance being developed for principal security 

inspectors included testing scenarios based on the types of name 

variations that air carriers must be capable of conducting in accordance

with the revised watch-list-matching requirements. Also, according to TSA, 

guidance for transportation security inspectors is being developed (as part 

of the 2009 Regulatory Activities Plan) to provide more specific direction 

to inspectors for assessing name-matching capability. In September 2008, 

in response to our inquiry, TSA noted that the agency’s internal 

handbook—which provides guidance to transportation security inspectors 

on how to inspect air carriers’ performance of various requirements, 

Page 31 GAO-08-992 Aviation Security and Watch List Matching
including watch-list-matching requirements—was being revised and was 

expected to be released later this year. Thus, TSA indicated that the new 

inspection guidance would be used in conjunction with the nationwide 

regulatory activities plan for fiscal year 2009. Overall, the actions taken 

(and planned to be taken) by TSA are positive developments, although it is

too soon to determine the extent to which TSA will assess air carriers’ 

compliance with the revised watch-list-matching requirements. 

According to TSA officials, there were other benefits stemming from the 

January 2008 special emphasis inspection. For example, officials stated 

that in considering options for corrective actions, TSA consulted with 

representatives from the intelligence community, which is responsible for 

identifying names (and variations of names)57 for inclusion on the No Fly 

and Selectee lists. According to TSA, these discussions enhanced the

intelligence community’s understanding of how air carriers use the No Fly 

and Selectee lists, and as a result, the intelligence community is better 

positioned to carefully consider which name variations are appropriate for 

being added to the lists and whether these variations would be helpful for

the purposes of watch-list matching. Further, TSA officials noted that such 

considerations, in turn, could benefit air carriers and the public by limiting 

the number of passengers who are misidentified as being potential 

matches with watch-list records. TSA officials added that insights 

regarding the extent to which name variations exist on the No Fly and 

Selectee lists also have benefited ongoing efforts to design and implement 

the Secure Flight program. Specifically, officials explained that TSA now 

has a fuller understanding of the types of name variations presently 

contained in watch-list records and, in turn, a fuller understanding of what 

types of comparisons Secure Flight should be capable of performing. 

Shortcomings that have national security implications exist in the watchlist-matching capability of domestic air carriers, as confirmed by the 

results of TSA’s recent special emphasis inspection. Specifically, TSA 

found differences among air carriers in the thoroughness and effectiveness 

of their processes for comparing passengers’ names with those on the No

Fly List. A particular concern involves similar-name comparisons. 

However, TSA’s April 2008 revision of the No Fly List security directive 

Concluding 

Observations 

 57 As noted previously, each watch-list record does not necessarily indicate a separate 

individual on the list. Some listed individuals have multiple records attributed to them due 

to the inclusion of known aliases and name variations. 

Page 32 GAO-08-992 Aviation Security and Watch List Matching
establishes a baseline name-matching capability by specifying the types of 

name variations that air carriers’ processes must be capable of identifying. 

Effective implementation of the baseline capability should strengthen 

watch-list-matching processes, especially for those air carriers that had 

been using less thorough approaches for identifying similar-name matches. 

Concurrently, revised internal guidance for TSA’s inspectors can help

ensure that compliance decisions are based upon testing and that these 

tests are carried out regularly, using the standards specified within the 

security directives as evaluation criteria. Also, if properly documented in 

inspection reports, the results of these tests could give TSA management 

better information on the quality of watch-list matching being conducted 

by air carriers, thereby improving TSA’s monitoring of the overall security 

posture of the aviation sector. At the time of our review, TSA’s process for 

revising its guidance was in the initial stages; thus it is too early to 

determine the extent to which updated guidance for principal security 

inspectors and transportation security inspectors would strengthen 

oversight of air carriers’ compliance with the security directive 

requirements. Given continued delays in the implementation of the Secure 

Flight program, TSA’s oversight of air carriers’ compliance with watch-listmatching requirements remains an important responsibility. TSA officials 

acknowledge that the baseline capability specified in the revised No Fly 

List security directive and the similar revision planned for the Selectee List

security directive—while an improvement—does not address all 

vulnerabilities identified by TSA and does not provide the level of risk 

mitigation that is expected to be achieved from Secure Flight. Thus, TSA 

intends to deploy the Secure Flight program beginning in January 2009 so

that it may implement this more robust matching capability. 

We provided a draft of our restricted report (GAO-08-453SU) to the 

Department of Homeland Security and the Department of Justice for 

review and comment. The Department of Homeland Security had no 

comments. The Department of Justice provided technical comments on 

the restricted version of this report, which we incorporated where 

appropriate.

Agency Comments 

We will send copies of this report to the appropriate congressional 

committees; the Secretary of Homeland Security; and the U.S. Attorney 

General. We will make copies available to others upon request. The 

report will also be available at no charge on our Web site at 

http://www.gao.gov. 

Page 33 GAO-08-992 Aviation Security and Watch List Matching
If you or your staff have any questions about this report or wish to discuss 

the matter further, please contact me at (202) 512-3404 or 

berrickc@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this report. GAO staff who made 

major contributions to this report are listed in appendix III. 

Cathleen A. Berrick 

Director, Homeland Security and Justice Issues 

Page 34 GAO-08-992 Aviation Security and Watch List Matching
List of Congressional Committees

The Honorable Robert C. Byrd 

Chairman 

The Honorable Thad Cochran 

Ranking Member 

Committee on Appropriations 

United States Senate 

The Honorable Daniel K. Inouye 

Chairman 

The Honorable Kay Bailey Hutchison 

Ranking Member 

Committee on Commerce, Science, and Transportation 

United States Senate 

The Honorable Joseph I. Lieberman

Chairman 

The Honorable Susan M. Collins

Ranking Member 

Committee on Homeland Security and Governmental Affairs 

United States Senate 

The Honorable Patrick J. Leahy 

Chairman 

The Honorable Arlen Specter 

Ranking Member 

Committee on the Judiciary 

United States Senate 

The Honorable John D. Rockefeller, IV 

Chairman 

The Honorable Kay Bailey Hutchison 

Ranking Member 

Subcommittee on Aviation Operations, Safety, and Security 

Committee on Commerce, Science, and Transportation 

United States Senate 

Page 35 GAO-08-992 Aviation Security and Watch List Matching
The Honorable Dave Obey 

Chairman 

The Honorable Jerry Lewis 

Ranking Member 

Committee on Appropriations 

House of Representatives 

The Honorable Bennie G. Thompson 

Chairman 

The Honorable Peter T. King 

Ranking Member 

Committee on Homeland Security 

House of Representatives 

The Honorable Henry A. Waxman 

Chairman 

The Honorable Tom Davis 

Ranking Member 

Committee on Oversight and Government Reform 

House of Representatives 

The Honorable James L. Oberstar

Chairman 

The Honorable John L. Mica 

Ranking Republican Member 

Committee on Transportation and Infrastructure 

House of Representatives 

The Honorable David E. Price 

Chairman 

The Honorable Harold Rogers 

Ranking Member 

Subcommittee on Homeland Security 

Committee on Appropriations 

House of Representatives 

The Honorable Judd Gregg 

United States Senate 

The Honorable Don Young 

House of Representatives 

Page 36 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: 

Methodology 

Objectives, Scope, and 

Page 37 GAO-08-992

Appendix I: Objectives, Scope, and 

Methodology 

To examine the current processes that domestic air carriers use to 

conduct watch-list matching for domestic flights, we addressed the 

following questions: (1) What are TSA’s requirements for domestic air 

carriers to conduct watch-list matching for domestic flights? (2) To what 

extent has TSA assessed domestic air carriers’ compliance with watch-listmatching requirements? 

In addressing the principal questions, we drew upon our previous work 

and reports on aviation security—specifically, reports covering TSA’s 

inspection process, Secure Flight, and other passenger prescreening 

programs. We also consulted our most recent reports and testimonies on 

terrorist watch lists. In addition, we reviewed relevant studies conducted 

by other governmental agencies, including the Congressional Research 

Service and the Department of Justice’s Office of Inspector General. This 

report is a public version of the restricted report that we provided to 

congressional committees in July 2008.1

More details about the scope and methodology of our work to address 

each of the principal questions are presented in the following sections, 

respectively.

Objectives 

Scope and 

Methodology 

TSA’s Requirements for Air 

Carriers to Conduct WatchList Matching for Domestic 

Flights 

To determine TSA’s requirements for air carriers to match passenger 

information against the No Fly List and the Selectee List for domestic 

flights, we assessed two key TSA documents—the No Fly List Procedures

security directive and the Selectee List Procedures security directive.2

 We 

reviewed versions of these security directives—including the revisions 

 1

 GAO, Aviation Security: Pending Implementation of Secure Flight, TSA Is Enhancing 

Its Oversight of Air Carrier Efforts to Identify Passengers on the No Fly and Selectee

Lists, GAO-08-453SU (Washington, D.C.: July 10, 2008). 

2

 These directives apply to domestic air carriers—that is, U.S. air carriers that maintain 

security programs in accordance with 49 C.F.R. part 1544. The directives govern watch-list 

matching for flights operating between two points within the United States or its 

territories. Although outside the scope of our review, the directives also apply to domestic 

air carriers’ international operations. At the start of our review, we based our analysis on 

the No Fly List Procedures (1544-01-20D) security directive and the Selectee List 

Procedures (1544-01-21E) security directive, both dated July 8, 2004. Over the course of our 

review, TSA first issued revised security directives in 2007 and has undertaken to revise 

them again in April 2008. The 2007 revisions of the No Fly and Selectee list security 

directives (SD 1544-01-20E and SD1544-01-21F, respectively) clarified certain elements of

the directives but resulted in no substantive changes in the requirements. Generally, in this 

report, we focus on the changes in requirements resulting from revisions undertaken in

April 2008 (SD 1544-01-20F and anticipated SD 1544-01-21G (Selectee List), respectively). 

Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

made in April 2008—to identify applicable requirements for watch-list 

matching. For the purposes of this report, we considered applicable 

requirements to be those that, according to TSA, would be assumed by the 

Secure Flight program, once operational, and those that TSA had itself 

identified for its oversight activities.3 Thus, we identified the following 

requirements (or key processes) as being within this scope (see table 1, 

which is presented earlier in this report): (1) the retrieval of the No Fly 

and Selectee lists, (2) the matching of passenger and watch-list 

information, (3) the use of the TSA Cleared List, (4) procedures for 

notifying authorities, and (5) keeping appropriate records.4

To further our understanding of these requirements, we reviewed TSA 

policies and other guidance applicable to watch-list matching. We also 

interviewed officials from TSA’s Office of Security Operations, which had 

primary responsibility for writing the security directives, and officials from 

two TSA offices that collaborated with the Office of Security Operations in 

crafting critical sections of the directives—the Office of Transportation 

Sector Network Management and the Office of Intelligence. To better 

understand TSA’s rationale for similar-name-matching requirements as 

well as the challenges associated with name-based matching, we attended

meetings of the interagency Federal Identity Match Search Engine 

Performance Standards Working Group, which was organized by the 

Terrorist Screening Center to help ensure awareness of best practices with 

regard to identity matching among federal agencies, and spoke with one of

 3 We based our understanding of TSA’s planned capabilities for Secure Flight on our 

February 2006 testimony before the Senate Committee on Commerce, Science, and

Transportation, our most recent, comprehensive testimony on the program when we 

initiated our work in July 2006. See GAO, Aviation Security: Significant Management 

Challenges May Adversely Affect Implementation of the Transportation Security

Administration’s Secure Flight Program, GAO-06-374T (Washington, D.C.: Feb. 9, 2006). 

4

 Although addressed in the security directives, other requirements that we excluded from

our scope involved, for example, procedures involving the screening of employees and 

procedures related to the international operations of domestic air carriers. We did not 

consider requirements for domestic air carriers’ international flights as part of our review

because at the time we were planning our review, TSA intended for Secure Flight to take

over the watch-list-matching function for only domestic flights. U.S. Customs and Border 

Protection was expected to conduct the watch-list-matching function for flights arriving 

from or departing to locations outside the United States, not Secure Flight. However, in 

February 2008 we reported in testimony that, as agreed to by the respective agencies, TSA 

will also take over the matching of international passengers against the No Fly and Selectee 

lists from U.S. Customs and Border Protection. GAO, Aviation Security: Transportation 

Security Administration Has Strengthened Planning to Guide Investments in Key

Aviation Security Programs, but More Work Remains, GAO-08-465T (Washington, D.C.: 

Feb. 28, 2008). 

Page 38 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

the group’s experts working in the field of name matching.5 To obtain 

information on the composition and use of the No Fly and Selectee lists, 

we spoke with officials from the Department of Justice’s Terrorist 

Screening Center and TSA’s Office of Intelligence. Further, to understand 

how TSA compiles and disseminates its Cleared List to air carriers, we 

spoke with officials from the Department of Homeland Security’s Traveler 

Redress Inquiry Program (TRIP) and TSA’s Office of Transportation 

Security Redress, which share responsibility for managing the TSA 

Cleared List for the current watch-list-matching process. Finally, to 

compare the current watch-list-matching process with that proposed once 

the federal government performs watch-list matching, we reviewed recent 

Secure Flight program documents.6

To generally understand how domestic air carriers have responded to 

TSA’s requirements, we selected for interviews a nonprobability sample of 

14 air carriers from a TSA-provided list of 95 air carriers that were subject 

to the watch-list-matching security directives for fiscal year 2005. To 

ensure that our sample of air carriers reflected a range of operational 

sizes, we based our selections partly on data from the U.S. Department of 

Transportation, which places air carriers in size categories based on 

operating revenue. Specifically, we selected 8 that were considered 

“major” air carriers, each having more than $1 billion in operating revenue

in 2005; all but one of these 8 major air carriers flew internationally. In 

addition, we selected 3 air carriers the Department of Transportation 

identified as “national” air carriers, each having more $100 million to 

$1 billion in operating revenue in 2005, and 1 air carrier the department 

identified as a “regional” air carrier, with $100 million or less in operating 

revenue. We also selected two air carriers from the list that were not 

included in the Department of Transportation’s revenue groupings, given 

the small scale of their operations, but were identified by the department 

as air carriers that provide commuter service. National, regional, and 

commuter air carriers—which generally provided service covering a 

 5

 One objective of the Federal Identity Match Search Engine Performance Standards 

Working Group is to provide guidance to improve the effectiveness of the automated

search engines that federal agencies use for conducting identity matching. The group began 

meeting in December 2005. It included representatives from the departments of Homeland 

Security, State, and Defense; FBI; the intelligence community; and the National Institute of

Standards and Technology. 

6

 Specifically, we reviewed the Secure Flight notice of proposed rulemaking (72 Fed. Reg.

48,356 (Aug. 23, 2007)) and final concept of operations for Secure Flight (dated Mar. 9,

2007). We also reviewed our most recent reports and testimonies on the program.

Page 39 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

geographical area, such as the Pacific Northwest—had comparatively

smaller business operations. 

In selecting the 14 air carriers, we also considered the number of 

passengers transported. To determine this number, we used the 

Department of Transportation’s data for number of revenue passengers 

who enplaned (boarded) domestic air carriers during calendar year 2005—

the most recent year for which data were available when making our 

selections in 2006.7

 To the extent possible, we identified the number of 

domestic enplanements for those air carriers required to perform watchlist matching in 2005, identified within the previously cited TSA list. 

According to our calculations, the 14 air carriers in our study accounted 

for approximately 70 percent of all passengers who boarded domestic air 

carriers’ flights during calendar year 2005, and thus, our selection allowed

us to understand how watch-list matching was performed for the majority 

of passengers flying domestically in 2005. Although the 14 domestic air 

carriers we selected represent a range in size of air carrier operations and 

transported a majority of passengers that boarded domestic flights in 

calendar year 2005, the results of our interviews are not generalizable to 

all domestic air carriers. 

To help ensure consistency in conducting our interviews with air carriers, 

we developed a data collection instrument with questions focusing on air 

carriers’ implementation of certain requirements of the No Fly and 

Selectee list security directives. We conducted four of these interviews in 

person at the air carriers’ headquarters and the rest via telephone. In 

addition, to clarify our understanding of air carriers’ processes, we 

conducted follow-up phone interviews with four selected air carriers and 

received written answers to our follow-up questions from an additional 

four selected air carriers. The air carrier officials who answered our 

questions generally held positions in corporate security and regulatory 

affairs; however, half of the air carriers also had information technology 

systems specialists participate to answer technical questions related to 

automated name-matching systems. We did not audit or independently 

verify each air carrier’s implementation of TSA’s security directive 

requirements; rather, our work summarizes the capabilities as reported by 

officials at the 14 air carriers. 

 7

 Specifically, the data reflect the number of domestic passengers who boarded (enplaned) 

at a flight’s point of origin in calendar year 2005. The data include only revenue passengers,

or passengers from whom the air carrier received payment. As such, the data exclude

passengers using frequent flier vouchers, infants, air carrier employees, etc. 

Page 40 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

Finally, to understand challenges air carriers have experienced in 

implementing watch-list-matching requirements, we examined TSA’s case 

files on all regulatory violations of the No Fly List Procedures and the 

Selectee List Procedures security directives reported since the directives 

were first issued by TSA in 2002 to the time TSA provided us with the data

in November 2007—a total of 32 cases.8

 We reviewed these case files, 

which contained documentation and other legal analyses pertaining to 

TSA’s inspection findings following the discovery of the violation, to 

determine the nature and causes (i.e., human or electronic) of the 

violations and to identify any patterns among the cases. Finally, to clarify

the agency’s process for investigating and adjudicating security directive 

violations, we spoke with officials from TSA’s Office of Chief Counsel. 

Extent to Which TSA Has 

Assessed Domestic Air 

Carriers’ Compliance with 

Watch-List-Matching 

Requirements for 

Prescreening Passengers 

To address this objective, we first obtained an overview of TSA’s plans and 

guidance for assessing air carriers’ compliance with regulatory

requirements. For instance, to understand the inspection process, the 

focus of inspections, and inspection methods, we reviewed TSA’s National 

Inspection Manual, the Principal Security Inspector Handbook, and 

related implementing guidance and policy documents. Further, we 

interviewed or received written responses to our submitted questions from 

the general manager of TSA’s Office of Transportation Sector Network 

Management, the two branch chiefs in the office’s Commercial Aviation 

Sector, and all nine of the office’s principal security inspectors. We 

particularly focused on contacting the principal security inspectors 

because they are responsible for conducting inspections at air carriers’ 

corporate security offices (where watch-list-matching policies and 

procedures are formulated) that apply across an air carrier’s operations. In

addition, to obtain information on the creation of inspection plans and 

guidance and the compilation and analysis of inspection data, we spoke 

with individuals in the Office of Security Operations and the Office of 

 8

 The earliest case was dated December 3, 2003; the most recent was dated August 24, 2007. 

Because some domestic air carriers that are subject to security directives fly 

internationally, 7 of the 32 cases involved flights arriving from or departing to international

locations. Although we excluded such flights from our review of watch-list-matching 

requirements, as mentioned previously, we retained these 7 cases in our analysis of 

regulatory violations. We did so because (1) the requirements for air carriers to perform 

watch-list matching for flights involving an international location are, for the most part, the 

same as those for air carrier operations between two points within the United States or its 

territories, and (2) in August 2007, TSA announced that Secure Flight would eventually 

assume watch-list matching for passengers on flights arriving from or departing to 

locations outside the United States. 

Page 41 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

Transportation Sector Network Management. Also, to obtain

management’s perspectives on inspections, we spoke with the assistant 

general managers of the Office of Security Operations’ Compliance 

Division and its Procedures Division. We also interviewed two federal 

security directors9

 and two transportation security inspectors, also within 

TSA’s Office of Security Operations and who were located in the 

Washington, D.C., metropolitan area, on planning and conducting 

inspections. 

After obtaining an understanding of TSA’s plans and guidance for 

assessing air carriers’ compliance with regulatory requirements, we 

reviewed the results of TSA inspections that are scheduled on a regular 

basis in conjunction with annual inspection plans. In conducting 

inspections each year, TSA’s inspectors use an extensive list of inspection 

guidelines (known as PARIS prompts)10 that cover a broad range of 

applicable topics—including topics outside the scope of our review, such

as airport perimeter security and cargo security, as well as screening of

employees and baggage.

11 As presented in table 3, we determined that TSA 

used 11 inspection guidelines during fiscal year 2007 that were relevant to

the objectives of our review.12 Of these, guidelines 1, 2, and 6 through 11 

were applicable to inspections conducted by principal security inspectors, 

while guidelines 3 through 5 were applicable to inspections conducted by

transportation security inspectors. 

 9

 Federal security directors are responsible for leading and coordinating TSA security 

activities at airports across the nation. 

10 The Performance and Results Information System (PARIS) is an inspections database 

that assists TSA management by providing factual and analytical information on the 

compliance of TSA-regulated entities. 

11 As mentioned previously, the watch-list-matching requirements relevant to the objectives 

of our review are shown in table 1, which is presented earlier in this report. 

12 TSA provided us with data for 12 inspection guidelines. These 12 are the 11 guidelines 

shown in table 3—plus the following guideline, which was replaced in March 2007 with

guideline 4 in table 3: “All passenger names are compared to the most current No Fly and 

Selectee Lists in accordance with the procedures outlined in Security Directive 1544-01-20 

series (No Fly) and Security Directive 1544-01-21 series (Selectee).” Because these two 

guidelines were used for the same purpose but at different times during fiscal year 2007, we 

combined the data associated with each one and treated them as one inspection guideline

for the purposes of this report.

Page 42 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

Table 3: Watch-List-Matching Requirements and the Related Inspection Guidelines (Fiscal Year 2007) 

Requirements (key processes) Inspection guidelines (prompts) 

Retrieving the No Fly and Selectee lists 1. Procedures are in place to ensure the most recently issued No Fly List is utilized 

within 24 hours of receipt. 

2. Procedures are in place to ensure the most recently issued Selectee List is utilized 

within 24 hours of receipt. 

Matching passenger data to No Fly and 

Selectee lists 

3. All passenger names are compared to the most current No Fly and Selectee lists in 

accordance with the Private Charter Standard Security Program. 

4. The aircraft operator is comparing all passenger names to the most current No Fly 

and Selectee lists in accordance with the procedures outlined in Security Directive 

1544-01-20 series (No Fly) and Security Directive 1544-01-21 series (Selectee). 

Using the TSA Cleared List 5. A passenger identified as a match on the Selectee List is cleared, along with his or 

her accessible property. 

Notifying authorities 6. Procedures are in place to contact the federal security director, local law 

enforcement, FBI, and TSA Office of Intelligence for matches to the No Fly List. 

7. Procedures are in place to contact the TSA Office of Intelligence for matches to the 

Selectee List. 

Keeping recordsa8. Records are maintained of all flights operated with passengers who were determined 

by a local law enforcement, U.S. legal attaché, or TSA Office of Intelligence not to be 

a match. 

9. Records are maintained of every flight operated with passengers who are designated

as selectees. 

10. Records are maintained of every flight with an individual who is cleared to fly utilizing 

data in the TSA Cleared List including the name of the cleared individual and the 

accepting aircraft operator representative.b

 (No Fly List) 

11. Records are maintained of every flight with an individual who is cleared to fly utilizing 

data in the TSA Cleared List including the name of the cleared individual and the 

accepting aircraft operator representative.b

 (Selectee List) 

Sources: GAO analysis of TSA’s security directives and related guidance. 

a

Maintaining accurate records, according to TSA officials, provides a starting point for an investigation

in the event of a terrorist incident. 

b

This inspection guideline reflects the current process, which is to use the TSA Cleared List. Security 

directives in effect prior to April 2008 referenced a “cleared column,” a format for clearing passengers. 

TSA eventually replaced this format with the Cleared List and updated language in the April 2008 

revision of the No Fly List Procedures security directive to reflect the new process. 

In reference to the 12 inspection guidelines—the 11 guidelines listed in 

table 3 and the 1 guideline discussed in footnote 12 of this appendix—TSA 

queried its PARIS database to identify all inspections of domestic air 

carriers conducted during fiscal year 2007 that used at least one of these 

guidelines. In addition to determining the number of inspections, we 

reviewed the fiscal year 2007 inspection data to calculate compliance 

Page 43 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

rates.13 We did not evaluate the substantive basis for the inspectors’ 

assessment decisions regarding compliance with requirements. 

To determine whether and to what extent TSA’s inspectors tested the air 

carriers’ capability to conduct exact-name and similar-name matching, we 

also reviewed documentation of testing in a data field (in the PARIS 

database) that allowed inspectors to enter narrative comments regarding 

similar-name matching, among other inspection activities. In doing so, we 

conducted a formal content analysis by having two analysts independently

review comments in the data field and then resolve any inconsistencies 

between the two sets of analytical observations. Moreover, we submitted 

written questions to each of TSA’s nine principal security inspectors 

asking them to describe their practices for testing air carriers’ capability to 

identify similar-name variations. 

In contrast to these regular inspections, TSA also conducted a special 

emphasis assessment and a special emphasis inspection, nonroutine 

activities conducted at the direction of TSA headquarters. A special 

emphasis assessment addresses a vulnerability that generally is not tied to 

a regulation, while a special emphasis inspection is tied to a regulatory 

requirement. TSA provided us information on the scope, methodology, and 

results of a special emphasis assessment that TSA conducted during June,

July, and September 2005. We reviewed the scope, methodology, and 

results of this assessment with our methodologists and with TSA officials. 

We determined that the sampling and related procedures used for the 

special emphasis assessment were insufficient for providing a reliable 

estimate of the success rate of all attempted matches by air carriers; thus, 

the results cannot be used to infer overall or individual rates of

compliance with the name-matching requirements in TSA’s security 

directives. 

In February 2008, TSA provided us a briefing on the scope and 

methodology of a special emphasis inspection conducted the month 

before in which the similar-name-matching capability of 52 domestic air 

carriers and 31 foreign air carriers was tested. The briefing also covered

analyses of the results to date of the special emphasis inspection and a 

discussion of the corrective actions that TSA was planning to implement 

to address deficiencies. In April 2008, TSA provided us with an updated 

briefing on its plans for corrective actions. In September 2008, we 

 13 Our calculations were based only on the 12 inspection guidelines relevant to our review. 

Page 44 GAO-08-992 Aviation Security and Watch List Matching
Appendix I: Objectives, Scope, and 

Methodology 

requested information on TSA’s progress with these corrective actions. In

response, TSA provided us the results of a special emphasis assessment 

(conducted during May 20–29, 2008) of seven air carriers’ compliance with 

requirements in the April 2008 No Fly List security directive. We did not 

assess the reliability of the data TSA collected during the January 2008 

special emphasis inspection nor the May 2008 special emphasis 

assessment. 

In assessing the reliability of the fiscal year 2007 data that TSA provided us

for watch-list-related inspections based on annual inspection cycles, we 

performed electronic testing, discussed the data system and any data 

inconsistencies we found with knowledgeable TSA officials, and reviewed

existing information about the system. Although we determined that the 

data were reliable for the purposes of this report, we have concerns about 

TSA’s process for querying its inspection database, and the potential for 

faulty output. The process is cumbersome and prone to user error due, in

part, to differences that occur in the verbiage of inspection guidelines and 

types of inspections as they are revised over time.

Reliability of Fiscal 

Year 2007 Inspections 

Data 

We conducted this performance audit from July 2006 to September 2008 in

accordance with generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on the audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions 

based on the audit objectives. 

Page 45 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic Appendix II: Overview of Selected Domestic Air Carriers’ Watch-List-Matching Processes 

Air Carriers’ Watch-List-Matching Processes 

TSA’s watch-list-matching requirements for domestic flights address five 

key process areas: (1) retrieval of the No Fly and Selectee lists, (2) the 

matching of passenger and list information, (3) the use of TSA’s Cleared 

List, (4) notification procedures, and (5) record-keeping activities (see 

table 1).1

To generally understand how TSA’s requirements for watch-list matching 

were being implemented, we reviewed documents in which TSA provided

general information on air carriers’ processes. We also interviewed 14 

domestic air carriers with operations ranging in size from international to 

commuter service about their watch-list-matching processes. All 14 air 

carriers were subject to TSA’s requirements for comparing passenger 

information with records on the No Fly and Selectee lists and the TSA 

Cleared List.2

 We asked each of the 14 to describe their processes for 

meeting TSA’s requirements.3

 The air carriers’ implementation of these 

requirements can be discussed in reference to three time periods—before 

passenger check-in, at passenger check-in, and after passenger check-in—

as reflected in the following sections, respectively, and as illustrated in 

figure 1. 

 1

 To identify these requirements, we reviewed the No Fly List Procedures and Selectee List 

Procedures security directives (series SD 1544-01-20 and SD 1544-01-21, respectively). This 

report discusses only the requirements within the two security directives pertaining to 

domestic flights (defined as flights occurring between points within the United States and 

its territories), though these same requirements generally apply to the international flights

of both domestic and foreign air carriers. For more information on how we identified 

requirements for watch-list matching, see appendix I. 

2

 For information on our methodology for selecting the 14 air carriers and conducting the

interviews, see appendix I.

3

 The implementation methods described in this appendix are based on descriptions 

obtained from the 14 air carriers. We did not undertake audits of the air carriers’ processes

to confirm that the processes functioned as described in the interviews. Specifically, we

asked air carriers questions on methods for securing the most recent No Fly and Selectee 

lists, executing comparisons within required time frames, determining valid matches, and 

implementing required notification and reporting procedures. 

Page 46 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic 

Air Carriers’ Watch-List-Matching Processes 

The 14 air carriers told us that they obtain new versions of the No Fly and 

Selectee lists through one or both of the following methods (1) assigning 

an employee to monitor TSA’s Web board for new postings at certain 

intervals throughout the day, and (2) receiving an e-mail message from 

TSA to the respective air carrier’s security staff informing them of new No 

Fly and Selectee lists. Also, all 14 air carriers reported using passenger 

name record (PNR) data—data collected from the passenger at the time a

reservation is made—to make comparisons against the No Fly and 

Selectee lists. Specifically, the air carriers said that they have implemented 

procedures to execute comparisons of PNR and watch-list data prior to 

scheduled flight departure. Most of the air carriers told us they do this by 

using computerized matching programs that automatically execute 

comparisons. 

Before Passenger 

Check-in: Retrieving 

the No Fly and 

Selectee Lists and 

Executing Name 

Comparisons 

Because the 14 air carriers we interviewed did not collect date of birth (an 

identifying data element that air carriers receive on the No Fly and 

Selectee lists) within PNR data, this information generally was not 

available for matching purposes prior to check-in. However, as discussed 

later in this appendix, several air carriers reported developing systems 

capable of accessing passenger date-of-birth information collected and 

stored outside of PNR data for use in comparisons conducted prior to 

check-in, but this information was not available for all of their passengers. 

Thus, the 14 air carriers we spoke with were limited to performing nameonly comparisons—that is, comparisons of passenger names with names 

on the No Fly and Selectee lists—prior to check-in for at least some, if not 

all, passengers. All 14 air carriers we spoke with reported conducting 

comparisons to identify exact-name matches of passengers and watch-list 

names. However, not every air carrier reported conducting comparisons to 

identify similar-name matches. 

Page 47 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic 

Air Carriers’ Watch-List-Matching Processes 

In accordance with TSA requirements, air carriers are to collect additional 

identifying information at check-in to assist in identifying passengers who 

are matches with information on the No Fly or Selectee lists. Air carriers 

collect additional identifying information at check-in only for those 

passengers identified as potential matches to the No Fly or Selectee lists 

through the name-only comparisons they conduct prior to check-in. To 

prevent individuals who are potential matches from checking in by other 

means, such as using Internet or airport kiosk check-in, air carriers with 

automated systems place an automatic “lock” on boarding passes (see 

fig. 1).4

 By doing so, the air carriers force all potentially matched 

passengers to check in at the ticket counter, where an agent is to collect a 

valid form of identification with date of birth (typically, a governmentissued identification document such as a driver’s license or passport) to 

complete the comparison of passenger and watch-list information. 

At Passenger Checkin: Completing 

Comparisons of 

Passenger and WatchList Information and 

Using TSA’s Cleared 

List 

To check the potentially matched passenger’s date of birth information 

against the No Fly and Selectee lists, most of the 14 air carriers we 

interviewed reported comparing the two dates manually, and the other air

carriers reported keying the passenger’s date of birth into a computer 

system that would automatically execute the comparison.5

 The 14 air 

carriers reported that if they determine that the dates of birth do not 

match, they unlock the boarding pass without consulting TSA, in 

accordance with TSA requirements, thereby allowing the passenger to 

continue the boarding process (see fig. 1, post-check-in number 1).6

However, if a passenger’s date of birth matches with that of an individual

on the No Fly or Selectee lists, the 14 air carriers said that they consider 

 4

 The one air carrier in our review without an automated system reported requiring all 

passengers, regardless of whether they were a potential match, to check in at the ticket

counter. To identify those passengers who should submit additional information for further 

comparison against the No Fly and Selectee lists at check-in, this air carrier reported

having its employee in charge of watch-list matching make a written notation next to the

name of all identified potential matches on a printed list of passengers with reservations. 

5

 In addition, to check potentially matched passenger information against the No Fly and 

Selectee lists, three air carriers reported that they had developed kiosks with capabilities to

read electronic date of birth information from certain forms of identification that are

machine readable. 

6

 After this point, the passenger generally experiences no further inconvenience due to

watch-list matching. However, the passenger may be selected for enhanced checkpoint 

screening as a result of the Computer Assisted Passenger Prescreening System (CAPPS)—

an electronic application that selects individuals for enhanced screening at the passenger 

checkpoint based on certain travel characteristics identified by TSA as indicating potential 

risk. 

Page 48 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic 

Air Carriers’ Watch-List-Matching Processes 

the passenger to be a match and followed the procedures outlined in TSA’s 

security directives for handling matches to the No Fly or Selectee lists (see 

fig. 1, post-check-in numbers 2 and 3). 

Figure 1: Overview of the Current Passenger Watch-List-Matching Process 

PNR is created

LIST

Air carrier system compares

PNR

data

List

data

 Check-in for 

Nonlocked PNRs

Source: GAO analysis.

Watch-list data

Pre

check-in

Check-in

To

Post

check-in

Checkpoint

screening

Additional

screening

Passenger

proceeds to flight

Air carrier

contacts

appropriate 

officials

Boarding 

pass

denied

BOARDING

PASS

Cleared

Potential match

YES

Clearance process for locked PNRs

● Passengers present government ID at ticket counter

● Agent compares passenger’s ID to No Fly, Selectee, and

 TSA Cleared list data. Internet, kiosk or

ticket counter

Potential

match

NO

Matching results

1. Cleared

 Passenger identified as not

being on No Fly or Selectee lists

 Passenger identified as a match

 to Cleared List

2. Selectee match

 Passenger identified as a match

 to Selectee List

3. No Fly

 Passenger identified as a match

 to No Fly List

Passenger name Record (PNR) locked PNR unlocked

Passenger makes

reservation

TSA posts No Fly, Selectee,

and TSA Cleared lists to 

secure Web board

Air carrier retrieves

list data

Downgraded to selectee

Not cleared

Page 49 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic 

Air Carriers’ Watch-List-Matching Processes 

Also, 10 air carriers reported using the TSA Cleared List to identify and

clear passengers misidentified as a match to the No Fly List or the Selectee 

List, generally at the time of check in. The other 4 air carriers reported not 

using the list—despite TSA’s requirement that all air carriers do so. In 

addition, of the 10 air carriers that reported using the cleared list, 2 

reported using the list in conjunction with their independently developed 

processes to “pre-clear” individuals (discussed below). Development of 

such processes was undertaken to allow air carriers to identify and clear 

misidentified passengers without requiring them to check in at the ticket 

counter. Specifically, 11 of the 14 air carriers we interviewed reported that

individuals on the TSA Cleared List still must approach the ticket counter 

at check in.7

 Consequently, 6 of the 14 air carriers that we interviewed 

reported developing alternative clearance processes to decrease the 

number of potentially matched individuals who are required to check in at

the ticket counter. These 6 carriers explained that their internally 

developed clearance processes operate by using additional data sources, 

such as passenger information collected in frequent flier databases, to 

resolve potential matches prior to check in. For example, if an air carrier 

collected date of birth within its frequent flier database, its internal 

clearance system would compare the date of birth of a potentially matched 

passenger who had entered a frequent flier number when making a 

reservation with the date of birth of the respective individual on the No Fly 

List or the Selectee List.8

 

 7

 These individuals are required to check in at the ticket counter because the air carrier 

must confirm that the passenger is the cleared individual by comparing the passenger’s

legal identifying documentation with the TSA Cleared List. 

8

 Air carriers with frequent flier programs generally have the capability to collect a frequent

flier number within the PNR; therefore, unlike date of birth information, frequent flier 

numbers are available to air carriers prior to a passenger’s arrival at check-in and can be

used to assist in the confirmation of a passenger’s identity because of the presence of date

of birth information in the passenger’s frequent flier account. 

Page 50 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic 

Air Carriers’ Watch-List-Matching Processes 

For match determinations made at the time of passenger check in, TSA’s 

No Fly and Selectee list security directives require that air carriers follow 

certain notification and record-keeping procedures. With regard to 

notification procedures: 

• If the air carrier identifies a passenger as a potential match to the No 

Fly List, the air carrier must contact both the applicable federal 

security director and the appropriate law enforcement officer. Then, if 

the law enforcement officer confirms that the passenger is a match, the 

air carrier is to contact the local Federal Bureau of Investigation (FBI) 

field office and TSA’s Office of Intelligence. 

• If the air carrier identifies a passenger as a potential match to the 

Selectee List, the air carrier must mark the passenger’s boarding pass 

to indicate to checkpoint screeners that the passenger should be 

subject to enhanced checkpoint screening. Also, the air carrier must 

notify TSA’s Office of Intelligence that the passenger has been matched 

with the Selectee List. 

After Passenger 

Check in: 

Implementing the 

Notification and 

Record-Keeping 

Procedures Specified 

in TSA’s No Fly and 

Selectee Security 

Directives 

With regard to record-keeping procedures, TSA’s security directives 

require that air carriers maintain a record of (1) all passengers cleared 

using the TSA Cleared List, (2) all flights that had potentially matched 

passengers who were determined by local law enforcement not to be a

match to the No Fly List, and (3) all passengers identified as matches with 

the Selectee List. 

Generally, the 14 air carriers told us that they followed the notification and 

record-keeping requirements specified in TSA’s security directives, but 

reported having different procedures in place to implement these 

requirements. For example, upon identifying a potential match to the No

Fly List, 5 air carriers reported requiring their ticket agents to notify their 

respective air carrier’s ground security coordinator, who would then make 

the necessary calls to the applicable TSA federal security director and to 

local law enforcement. Three other air carriers reported requiring that 

ticket agents contact security staff at a centralized call center, and these 

staff would then make the necessary notifications.

9

 In addition, some of 

the carriers reported using some slight deviations from the stated 

requirements. For example, rather than notifying the local FBI field office

 9

 Another air carrier reported requiring the ticket agent to make these notifications; the

other five air carriers we interviewed did not discuss this aspect of the watch-list-matching 

process. 

Page 51 GAO-08-992 Aviation Security and Watch List Matching
Appendix II: Overview of Selected Domestic 

Air Carriers’ Watch-List-Matching Processes 

and TSA’s Office of Intelligence of a match only after a local law 

enforcement officer has confirmed the match, 8 air carriers reported 

contacting TSA’s Office of Intelligence for every passenger whose 

information matched the No Fly List, regardless of the local law 

enforcement officer’s input.10

 10 Two air carriers reported that (per the security directive requirement) they waited for 

local law enforcement officer confirmation before calling the FBI field office or TSA’s 

Office of Intelligence. One air carrier reported that it could not answer the question; that is, 

having never identified an individual as a name and date of birth match to the No Fly List, 

the air carrier could not say what its actions would be. During our interviews, three air 

carriers did not discuss this aspect of the watch-list-matching process. 

Page 52 GAO-08-992 Aviation Security and Watch List Matching
Appendix III: GAO

A

 Contact and Staff 

cknowledgments 

Page 53 GAO-08-992

Appendix III: GAO Contact and Staff 

Acknowledgments 

Cathleen A. Berrick, (202) 512-3404 or berrickc@gao.gov

In addition to the contact named above, Danny Burton and Christine 

Fossett (Assistant Directors) and Mona Blake and Mike Bollinger 

(Analysts-in-Charge) managed this assignment. 

GAO Contact 

Staff 

Acknowledgments 

Suzanne Heimbach, Matt Mohning, Justin Monroe, Alison Sands, and 

Susan Woodward made significant contributions to the work. 

David Alexander, Michele Fejfar, and Rich Hung assisted with design, 

methodology, and data analysis. 

Tom Lombardi and David Plocher provided legal support. 

Richard Ascarate, Ryan Consaul, Kevin Copping, Kristen Jensen, Lara 

Kaskie, Maria Soriano, William D. Updegraff, and Margaret Vo provided 

assistance in report preparation. 

(440744) Aviation Security and Watch List Matching
GAO’s Mission The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “E-mail Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, DC 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Ralph Dawn, Managing Director, dawnr@gao.gov, (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, DC 20548 

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER