Aviation Security: TSA Coordinates with Stakeholders on Changes to Screening Rules but Could Clarify Its Review Processes and Better Measure Effectiveness
Released: 2019-11-20T00:00:00

Letter

Background

TSA’s Secure Flight Program

TSA began implementing its Secure Flight program in 2009 to identify passengers who may pose security risks before boarding an aircraft. The program requires U.S. and foreign commercial aircraft operators traveling to, from, within or overflying the United States, as well as U.S. commercial aircraft operators with international point-to-point flights, to collect information from passengers and transmit it electronically to TSA. This information includes personally identifiable information, such as full name, gender, date of birth, passport information (if available), and certain non-personally identifiable information, such as itinerary information and the unique number associated with a travel record (record number locator).

The Secure Flight program matches the passenger-provided personally identifiable information against federal government watchlists and other information to determine if passengers may pose a security risk and to assign them a risk category. Since January 2009, Secure Flight has matched passengers to two subsets of the Terrorist Screening Database—the No Fly List, composed of individuals who should be precluded from boarding an aircraft or entering the sterile area of a U.S. airport, and the Selectee List, composed of individuals who should receive enhanced screening prior to boarding an aircraft or entering an airport sterile area. The risk categories are not specifically communicated to the air carriers, but for each passenger Secure Flight provides responses to air carriers commensurate with the risk levels identified (e.g., an air carrier will receive a response of “inhibited” if the passenger was identified as being in the highest-risk category, or the boarding pass printed for a high-risk passenger will identify that passenger as a selectee for enhanced screening at the security checkpoint).

In April 2011, in response to the December 25, 2009 attempted attack, TSA also began matching passengers to a third subset of the Terrorist Screening Database—the Expanded Selectee List—to designate known or suspected terrorists not otherwise included on the No Fly or Selectee Lists as selectees for enhanced screening. The Expanded Selectee List, in general, includes all records in the Terrorist Screening Database with a full name (first name and surname) and full date of birth not otherwise included on the No Fly or Selectee Lists. The Secure Flight system, which also screens passengers against the Silent Partner and Quiet Skies Lists, among others, results in passengers receiving one of four prescreening outcomes:

Low risk (expedited screening). Passengers who are eligible for expedited screening, such as those with TSA Pre®,

Unknown Risk (standard screening). Passengers who warrant standard screening,

High Risk (enhanced screening). Passengers who receive enhanced screening such as a pat down and explosives trace detection, because they have been identified as matches to government watchlists, including the Selectee, Expanded Selectee, Silent Partner and Quiet Skies Lists, or

Highest Risk (denied boarding). Passengers who are not permitted to board a commercial aircraft, such as passengers who are on the No Fly List or the Centers for Disease Control and Prevention Do Not Board List (see fig. 1).

Secure Flight also randomly identifies passengers for enhanced screening. Although subject to the same screening measures as high risk passengers, they have not been determined to be high risk. Similarly, individuals included on the Silent Partner and Quiet Skies Lists have not been determined to be of high risk, but rather have been identified using rules based on current intelligence and other factors that may indicate an elevated risk.

The Silent Partner List

TSA leverages CBP information and targeting capabilities to create the Silent Partner List. Specifically, TSA leverages (1) data CBP collects regarding passengers traveling internationally (such as citizenship, passport country of issuance, and address information), and (2) CBP’s Automated Targeting System. CBP uses the Automated Targeting System to identify potentially high risk passengers arriving or departing the United States by comparing passenger information with law enforcement, intelligence, and other enforcement data using risk-based targeting scenarios and assessments.

Analysts within TSA I&A’s Threat Analysis Division review intelligence to identify factors that may indicate elevated passenger risk. TSA works with CBP to create Silent Partner and Quiet Skies rules in the Automated Targeting System based on these factors. The system returns information on passengers who match with the rules and are scheduled to fly on U.S.- bound flights. TSA then omits any individuals on the Silent Partner cleared list (i.e. travelers exempted from further enhanced screening based on a specific rule) before placing the remaining passengers on the Silent Partner List. The Secure Flight program designates passengers who are on the Silent Partner List as selectees for enhanced screening for a particular international flight.

The Quiet Skies List

In April 2012, TSA’s Quiet Skies List became fully operational. The Quiet Skies List is a subset of passengers on the Silent Partner List. Specifically, TSA identifies certain Silent Partner rules that warrant continued enhanced screening for passengers’ subsequent domestic or outbound travel after arriving in the United States. Passengers identified via these rules—the Quiet Skies rules—comprise the Quiet Skies List. Passengers matched to the Quiet Skies List are designated as selectees and receive enhanced screening on any subsequent domestic flights for a designated period of time, or for a designated number of flights, whichever comes first. After the designated time period has elapsed (or number of flights is flown), passengers’ names and identifying information are moved to a cleared list.

TSA Modernization Act Requirements

Pursuant to the TSA Modernization Act, TSA I&A is to identify and review its Silent Partner and Quiet Skies screening rules, in coordination with DHS and TSA stakeholders, every 120 days and provide notification to these stakeholders no later than two days after making a change to a rule. Table 1 lists the DHS and TSA stakeholders TSA I&A must coordinate with under the Act.

TSA Coordinates with Stakeholders as Required, but TSA Guidance Is Not Clear About Criteria for Review of Rule Changes TSA I&A Coordinates Quarterly Rule Reviews and Notifies Oversight Offices of Rule Changes

According to DHS and TSA officials, TSA has coordinated quarterly rule review meetings with DHS and TSA stakeholders since the inception of the Silent Partner and Quiet Skies programs. We reviewed documentation of the reviews that occurred from December 2018 through March 2019. The quarterly review meetings are called for in DHS’s Automated Rule Review SOP and its Quiet Skies Implementation Plan.

Pursuant to the TSA Modernization Act, TSA I&A is to identify and review its screening rules in coordination with DHS and TSA stakeholders every 120 days—or at least three times a year. TSA I&A officials stated that they plan to continue convening four times a year because, given the difficulty of scheduling these large meetings, it will help them ensure they meet the 120 day requirement. Since October 2018, TSA I&A has also included representatives of DHS’s Traveler Redress Inquiry Program and the Federal Air Marshal Service in these quarterly review meetings, as required by the Act. Officials from these offices told us in August 2019 that they are still determining their role in the rule review process, but expect the coordination to be beneficial.

DHS and TSA SOPs set forth the process for the quarterly review meetings. TSA I&A and stakeholder officials stated that the process generally happens as described in the SOP. Two weeks prior to the meeting, TSA I&A sends out materials including a list of new rules, rule changes, archived (discontinued) rules, and the rationale and links to the underlying intelligence supporting each rule change. According to TSA officials, TSA and DHS stakeholders review the rules from their particular areas of expertise. For example, TSA Chief Counsel officials reported that they review rules and the supporting intelligence to ensure that the rules meet legal sufficiency standards. A TSA Privacy official stated that they review rules and the supporting intelligence to ensure rules do not violate passengers’ rights. All stakeholders review the rules to ensure they are based on current intelligence that identify specific threats. If a stakeholder finds that there is insufficient current intelligence to support the rule, TSA I&A officials stated that they would modify it to ensure it is tailored to current intelligence or archive a rule when the intelligence- based threat is no longer relevant. For example, during the March 2019 quarterly review meeting TSA I&A officials discussed archiving a Silent Partner rule due to insufficient current intelligence to support it. According to TSA I&A officials, the rule was archived in April 2019.

TSA I&A officials and stakeholders generally agreed that the quarterly reviews provide a good mechanism for oversight of both programs. Stakeholders told us these meetings provide a forum to discuss the scope of the rules and whether or not they were supported by current intelligence or if they are sufficiently specific. For example, a TSA stakeholder questioned the basis for a rule that identified a particular travel pattern as a high risk factor. As a result, TSA I&A officials reviewed the intelligence and revised the rule.

TSA I&A officials stated that since enactment of the TSA Modernization Act in October 2018, they have also notified DHS and TSA stakeholders within two days of making changes to a rule. We reviewed the eight notifications that TSA I&A sent to stakeholders regarding rule changes during the period from October 2018 through May 2019. These notifications detailed changes to rules, new rules, and rules that were archived. DHS and TSA stakeholders we spoke with said that the two day notifications are helpful in keeping them informed in between quarterly meetings. In addition, stakeholders said it allowed them to proactively reach out to TSA I&A to ask questions and share more timely feedback about rule changes. TSA I&A has implemented the two day notifications and other steps required in the TSA Modernization Act, but TSA I&A’s Standard Operating Procedures have not yet been updated to reflect these changes. TSA I&A officials stated that they have plans to do so in fall 2019.

TSA Has a Standard and Expedited Rule Review Process, but TSA Guidance Is Unclear about Criteria for Each Process

TSA I&A’s standard operating procedures establish two situation- dependent processes for reviewing and approving rule changes, as shown in figure 2. First, under standard circumstances, TSA I&A’s standard operating procedures detail a four-part vetting process by which TSA I&A drafts support for the rule change and it is subsequently approved by TSA Chief Counsel, the TSA I&A Assistant Administrator, and ultimately TSA senior leadership. TSA procedures specify that in standard circumstances, all rule changes are to be supported and approved in writing prior to implementation. Specifically, TSA I&A is to draft a memo with the nature of the threat and how all components of the rule address the concerns from intelligence reporting. The memo, along with all pertinent intelligence sources, is then required to be routed through TSA Chief Counsel and TSA leadership for intelligence, legal, and policy review. TSA’s April 2012 Quiet Skies Implementation Plan specified that the Chief Counsel’s review is to ensure that the proposed rule targets the threat presented in the assessment, the assessment properly documents the reasons for the recommendation, and the recommendation is in compliance with relevant legal authorities, regulations, and DHS policies. Upon approval, the memo is referred to TSA senior leadership—the TSA Administrator or TSA Deputy Administrator—for final written approval. Following this, the rule change can be implemented.

A second process, called exigent, is also briefly described in the SOPs. In exigent circumstances—circumstances requiring immediate action—the TSA I&A Assistant Administrator or his or her designee may direct that the rule be implemented immediately without a signed decision memo. The signed memo is still required, but can be drafted, reviewed, and approved after the change is implemented. TSA I&A officials stated that the exigent process entails verbal direction to implement a rule.

It is unclear if TSA I&A has followed the exigent rule review process in standard circumstances because the SOP is unclear on the criteria for each process. TSA’s SOP states that the exigent review process may be used “if TSA determines that exigent circumstances require immediate implementation of a Silent Partner rule.” However, the SOP does not clarify who or which office within TSA makes this determination or what types of circumstances would be appropriately characterized as exigent.

TSA I&A officials told us that exigent circumstances were very rare. They estimated that in the last 3 years exigent circumstances had occurred once. Yet, the same officials also estimated that they implemented approximately 90 percent of the rule changes following verbal approval from either TSA or I&A leadership and drafted the required memos after the fact. This indicates that TSA I&A officials have not followed the standard review process when implementing rule changes in circumstances they regard as standard, and the process followed appears to be closer to what would occur in exigent circumstances. These TSA I&A officials explained that drafting and processing the approval memo after they implement a rule change allows them to more quickly respond to changing intelligence. TSA’s SOP provides flexibility for this in exigent circumstances. However, given the absence of clarity in the SOP about when the exigent process is to be used and who is to make that decision, it is unclear whether or not TSA I&A used the exigent review process—a process which is not, initially, contingent upon TSA’s legal review or I&A’s written support—in circumstances that DHS and TSA leadership who oversee the program would regard as standard.

According to Standards for Internal Control in the Federal Government, management should implement control activities through policies by, for example, documenting responsibilities in policies and periodically reviewing policies and procedures for continued relevance and effectiveness. As TSA I&A updates its Silent Partner and Quiet Skies SOPs in fall 2019, clarifying the criteria for standard and exigent rule review procedures would provide greater assurance that screening rule changes are reviewed as intended.

TSA I&A officials further told us that they do not document or otherwise have a way of determining what proportion of rule changes have been reviewed in accordance with the standard process versus the exigent process because they had not identified a need to do so. According to the 2012 TSA memo establishing Quiet Skies as a permanent program, at the program’s outset a working group of DHS and TSA stakeholders identified the need for transparency as the first of seven key areas of consensus. Further, DHS’s Integrated Risk Management Framework establishes transparency and documentation as important characteristics of homeland security risk management. Documenting which review process TSA I&A uses for each rule change could improve transparency.

TSA Tracks Some Data on Rule Implementation, but Has Not Comprehensively Assessed Effectiveness TSA Has Monitored List Size and Number of Rule Matches, but Has Not Identified a Means to Comprehensively Measure Rule Effectiveness

TSA I&A monitors some operational data on its passenger screening rules. For example, TSA I&A officials track the number of individuals on the Silent Partner and Quiet Skies Lists, and the number of Silent Partner and Quiet Skies rules triggered by the passengers’ travel. TSA I&A officials stated that rule matches and list size are helpful for oversight purposes because they allow TSA I&A to monitor for Secure Flight system errors. Officials identified one example in which a Secure Flight software update created a system error that prevented 808 passengers from being moved to the Quiet Skies cleared list after a designated number of flights. According to the officials, monitoring list size and the number of rules triggered by passengers’ travel allowed them to identify and correct this error within 10 days of identifying the system error.

TSA I&A has not identified a means to comprehensively measure rule effectiveness. TSA I&A officials explained that they would find it helpful to demonstrate the effectiveness of the program, but had not yet done so because it was difficult to measure. TSA I&A officials reported that the approach they have used was to count the number of Quiet Skies passengers who were later identified as a known or suspected terrorist and added to the Terrorist Screening Database. TSA I&A officials reported that in January 2019 they reviewed all Quiet Skies passengers from January 2014 through July 2018 to determine how many were subsequently added to the Terrorist Screening Database. However, because it included Quiet Skies only, this analysis excluded about 93 percent of the rules. TSA officials reported that it is not feasible to do a similar analysis for Silent Partner rules because of the higher numbers of rules and matches and the difficulty matching Silent Partner rules to data in the Terrorist Screening Database. Further, TSA officials noted that without comparable information on the rate that non-Quiet Skies passengers were added to the Terrorist Screening Database during that time period, it is difficult to interpret what the results indicate about rule effectiveness.

TSA’s April 2012 Quiet Skies Implementation Plan established that TSA would continually evaluate the performance of the rules in the Silent Partner and Quiet Skies programs. Further, GAO and the Office of Management and Budget have previously identified useful practices to enhance performance management and measurement processes. GAO has previously reported that measuring performance allows organizations to track the progress they are making toward their goals and gives managers critical information on which to base decisions for improving their progress. Office of Management and Budget guidance has also focused specifically on common challenges associated with measuring effectiveness, including data availability and identifying measurable outcomes for a program. This guidance suggests using a variety of approaches such as outlining short-term milestones, identifying target outcomes, and using proxy measures to assess these programs.

Assessing the effectiveness of Silent Partner and Quiet Skies rules may be difficult, but I&A could explore using other data sources to assess program effectiveness in addition to further developing their consideration of Terrorist Screening Database additions. For example, TSA I&A could consider analyzing TSA data on the outcomes of the enhanced screening of Silent Partner and Quiet Skies passengers at passenger security checkpoints. CBP officials said that they review secondary inspection results to help them assess CBP’s rules-based program. TSA I&A officials noted that they were considering this measure and would need to determine what comparison group would make sense, and if they want to focus on specific screening outcomes versus all outcomes.

TSA I&A could also consider using the results of air marshals’ monitoring of Quiet Skies passengers. According to senior Federal Air Marshal Service officials, the service—with a budget of approximately $780 million for fiscal year 2019—began deploying air marshals on as many flights as possible with Quiet Skies passengers in March 2018. According to TSA’s Privacy Impact Assessment for Silent Partner and Quiet Skies and a Federal Air Marshal Service official, after air marshals complete a flight with a Quiet Skies List match, they file a report saying either “nothing to report” or, if they observe that the individual was involved in a security incident or suspicious activity, they will describe this in an after-action report. TSA I&A officials told us that while they have seen individual after-action reports, they do not review them regularly. These after-action reports are another source of information TSA I&A could consider using to gauge program effectiveness.

Given the TSA resources being devoted to the enhanced screening and in-flight monitoring of many passengers matching the Silent Partner and Quiet Skies Lists, and the burden on the traveling public, it is important that TSA understand the value of its screening rules programs. Exploring additional data sources—such as checkpoint screening results and Federal Air Marshal Service after-action reports—could help TSA refine and supplement their existing efforts to measure program effectiveness.

Conclusions

The attempted attack of December 25, 2009, highlighted the unknown threats to U.S. civil aviation. TSA has created the Silent Partner and Quiet Skies Lists to help address these unknown threats by ensuring that certain potentially higher risk passengers receive enhanced screening when traveling to, from, or within the United States. TSA created an oversight process that was further bolstered by the TSA Modernization Act, and DHS and TSA officials we met with generally regard the process as effective. However, TSA SOPs are not clear about when it is appropriate for TSA to use an expedited review process and they do not document which review process they used. The lack of clear SOPs inhibits program oversight. By establishing clear criteria for and documentation of each review process, TSA could increase transparency and ensure rule changes are reviewed as intended. Moreover, TSA has not identified a means to comprehensively measure the effectiveness of its Silent Partner and Quiet Skies rules. Exploring additional data sources—such as checkpoint screening results and Federal Air Marshal Service after-action reports—could help TSA refine and supplement their existing efforts to measure program effectiveness.

Recommendations for Executive Action

We are making the following three recommendations to TSA: The Administrator of TSA should clarify the criteria for exigent circumstances and standard rule review procedures; (Recommendation 1)

The Administrator of TSA should document which rule review process TSA I&A uses (exigent or standard) for each new rule or rule change; (Recommendation 2)

The Administrator of TSA should explore additional data sources measuring the effectiveness of Silent Partner and Quiet Skies rules. (Recommendation 3)

Agency Comments

We provided a draft of our report to DHS for comment. In written comments, which are included in appendix I, DHS concurred with our three recommendations and described steps they plan to take to address them. DHS also provided technical comments, which we have incorporated, as appropriate.

We are sending copies of this report to the appropriate congressional committees and to the Acting Secretary of Homeland Security. In addition, this report is available at no charge on the GAO website at http://gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-8777 or russellw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made significant contributions to this report are listed in Appendix II.

Appendix I: Comments from the Department of Homeland Security

Appendix II: GAO Contacts and Staff Acknowledgments

Appendix II: GAO Contacts and Staff Acknowledgments Error! No text of specified style in document.

GAO Contact

William Russell, (202) 512-8777 or russellw@gao.gov.

Staff Acknowledgments

In addition to the contact named above Claudia Becker, Assistant Director; Imoni Hampton, Analyst-in-Charge; Melissa Greenaway, John De Ferrari, Michele Fejfar, Eric Hauswirth, Tom Lombardi, and Kevin Reeves made key contributions to this work.