Aviation Security: TSA's Managed Inclusion Process Expands Passenger Expedited Screening, But TSA Has Not Tested Its Security Effectiveness
Released: 2015-03-25T00:00:00

Letter

Background

When TSA began offering expedited screening at airports in the summer of 2011, transportation security officers (TSO) initially provided such screenings in standard lanes to passengers aged 12 and younger, and subsequently extended expedited screening to certain flight crew members and then to passengers aged 75 and older. In October 2011, TSA began to expand the concept of expedited airport screening to more of the flying public by piloting the TSA PreTM program. This pilot program allowed certain frequent fliers of two air carriers to experience expedited screening at four airports. These frequent fliers became eligible for screening in dedicated expedited screening lanes, called TSA PreTM lanes, because they had opted into the TSA PreTM program through the air carrier with which they had attained frequent flier status. TSA also allowed certain members of the U.S. Customs and Border Protection’s (CBP) Trusted Traveler programs to experience expedited screening as part of the TSA PreTM pilot. TSA provided expedited screening in dedicated screening lanes to these frequent fliers and eligible CBP Trusted Travelers during the TSA PreTM pilot program because TSA used information available to it to determine that eligible passengers in these groups were lower risk. When traveling on one of the air carriers and departing from one of the airports participating in the pilot, these passengers were eligible to be screened in dedicated TSA PreTM screening lanes where the passengers were not required to remove their shoes; divest light outerwear, jackets, and belts; or remove liquids, gels, and laptops from carry-on baggage.

Since October 2011, TSA has further expanded the known traveler populations eligible for expedited screening. After TSA piloted TSA PreTM with certain passengers who are frequent fliers and members of CBP’s Trusted Traveler programs, TSA established separate TSA PreTM lists for additional low-risk passenger populations, including members of the U.S. armed forces, Congressional Medal of Honor Society Members, members of the Homeland Security Advisory Council, and Members of Congress, among others.other agencies or entities, TSA created its own TSA PreTM list composed of individuals who apply to be preapproved as low-risk travelers through the TSA PreTM Application Program, an initiative launched in December 2013. To apply, individuals must visit an enrollment center where they provide biographic information (i.e., name, date of birth, and address), valid identity and citizenship documentation, In addition to TSA PreTM lists sponsored by and fingerprints to undergo a TSA Security Threat Assessment. TSA leveraged existing federal capabilities to both enroll and conduct threat assessments for program applicants using enrollment centers previously established for the Transportation Worker Identification Credential Program, and existing transportation vetting systems to conduct applicant Applicants must be U.S. citizens, U.S. nationals or threat assessments.lawful permanent residents, and cannot have been convicted of certain crimes. As of March 2015, about 7.2 million individuals were eligible, through TSA PreTM lists, for expedited screening. Figure 1 shows the populations for each TSA PreTM list.

In addition to passengers who are included on one of the TSA PreTM lists, in October 2013, TSA continued to expand the opportunities for expedited screening to broader groups of passengers through the TSA PreTM Risk Assessment program and the Managed Inclusion process, both of which are described in greater detail below. Figure 2 shows a snapshot from February 25, 2015, through March 3, 2015, of the percentage of weekly passengers receiving non-expedited screening and expedited screening, and further shows whether known crew members experienced expedited screening, and whether expedited screening occurred in TSA PreTM lanes (for passengers designated as known travelers or through the TSA PreTM Risk Assessment program, or passengers chosen for expedited screening using Managed Inclusion), or in standard lanes.

As noted in figure 2, during the week ending March 3, 2015, 28 percent of passengers nationwide received expedited screening were issued TSA PreTM boarding passes, but were provided expedited screening in a standard screening lane, meaning that they did not have to remove their shoes, belts, and light outerwear, but they had to divest their liquids, gels, and laptops. TSA provides expedited screening to TSA PreTM -eligible passengers in standard lanes when airports do not have dedicated TSA PreTM screening lanes because of airport space constraints and limited TSA PreTM throughput.

TSA Uses Three Methods to Assess Risk for Participation in Expedited Screening

As we found in 2014, TSA determines a passenger’s eligibility for or opportunity to experience expedited screening at the airport using one of three risk assessment methods. These include (1) inclusion on a TSA PreTM list of known travelers, (2) identification of passengers as low risk by TSA’s Risk Assessment algorithm, or (3) a real-time threat assessment at the airport using the Managed Inclusion process.

Passenger Eligibility based on TSA PreTM Lists of Known Travelers

TSA has determined that the individuals included on the TSA PreTM lists of known travelers are low risk by virtue of their membership in a specific group or based on group vetting requirements. For example, TSA determined that members of the Congressional Medal of Honor Society, a group whose members have been awarded the highest U.S. award for valor in action against enemy forces, present a low risk to transportation security and are appropriate candidates to receive expedited screening. In other cases, TSA determined that members of groups whose members have undergone a security threat assessment by the federal government, such as individuals working for agencies in the intelligence community and who hold active Top Secret/Sensitive Compartmentalized Information clearances, are low risk and can be provided expedited screening. Similarly, TSA designated all active and reserve service members of the United States armed forces, whose combined members total about 2 million people, as a low risk group whose members were eligible for expedited screening. TSA determined that active duty military members were low risk and appropriate candidates to receive expedited screening because the Department of Defense administers common background checks of its members.

Members of the list-based, low-risk populations who requested, or were otherwise deemed eligible, to participate in TSA PreTM were provided a unique known traveler number. Their personal identifying information (name and date of birth) along with the known traveler number are included on lists used by Secure Flight for screening. To be recognized as low risk by the Secure Flight system, individuals on TSA PreTM lists with known traveler numbers must submit these numbers when making a flight reservation. maintain them by ensuring that individuals continue to meet the criteria for inclusion and to update the lists as needed.

TSA also continues to provide expedited screening on a per-flight basis to the almost 1.5 million frequent fliers who opted to participate in the TSA PreTM program pilot. According to TSA, this group of eligible frequent fliers met the standards set for the pilot based on their frequent flier status as of October 1, 2011. According to TSA officials, TSA determined that these frequent fliers were an appropriate population to include in the program for several reasons, including the fact that frequent fliers are vetted against various watchlists, such as the No-Fly list, each time they travel to ensure that they are not listed as known or suspected terrorists, and are screened appropriately at the checkpoint.

Passenger Eligibility Based On TSA PreTM Risk Assessments

As we found in December 2014, the TSA PreTM Risk Assessment program evaluates passenger risk based on certain information available for the passenger’s specific flight and determines the likelihood that passengers will be designated as eligible to receive expedited screening through TSA PreTM. Beginning in 2011, TSA piloted the process of using the Secure Flight system to obtain Secure Flight Passenger Data from air carriers and other data to assess whether the passenger is low risk on a per-flight basis and thus eligible to receive a TSA PreTM designation on his or her boarding pass to undergo expedited screening. In September 2013 after completing this pilot, TSA decided to explore expanding this risk assessment approach to every traveler. In order to develop the set of low-risk rules used to determine the passengers’ relative risk, TSA formed an Integrated Project Team consisting of officials from the Offices of Security Operations, Intelligence and Analysis, Security Capabilities, and Risk-Based Security. The team used data from multiple sources, including passenger data from the Secure Flight system from calendar year 2012, to derive a baseline level of relative risk for the entire passenger population. Our review of TSA’s documentation in our 2014 report showed that TSA considered the three elements of risk assessment—Threat, Vulnerability, and Consequence—in its development of the risk assessment. These three elements constitute the framework for assessing risk as called for in the Department of Homeland Security’s National Infrastructure Protection Plan. We found that TSA worked with a contractor to evaluate the data elements taken from information available for passengers’ specific flights and the proposed risk model rules used to determine the baseline level of relative risk. In its assessment of the algorithm used for the analysis, the contractor agreed with TSA’s analysis of the relationship between the data elements and relative risk assigned to the data elements. TSA officials stated that as of March 2015, the agency is continuing to refine the algorithm to include additional variables to help determine passenger risk.

As we found in December 2014, although TSA determined that certain combinations of data elements in its risk-based algorithm are less likely to include unknown potential terrorists, it also noted that designating passengers as low risk based solely on the algorithm carries some risk. To mitigate these risks, TSA uses a random exclusion factor that places passengers, even those who are otherwise eligible for expedited screening, into standard screening a certain percentage of the time. TSA adjusts the level of random exclusion based on the relative risk posed by the combinations of various data elements used in the algorithm. The result is that passengers associated with some data combinations that carry more risk are randomly excluded from expedited screening more often than passengers associated with other data combinations. For example, TSA’s assessment indicated that combinations of certain data elements are considered relatively more risky than other data groups and passengers who fit this profile for a given flight should seldom be eligible for expedited screening, while combinations of other data on a given flight pose relatively less risk and therefore passengers who fit these combinations could be made eligible for expedited screening a majority of the time. TSA developed a risk algorithm that scores each passenger on each flight, and passengers with a high enough score receive a TSA PreTM boarding pass designation making them eligible for expedited screening for that trip.

Passenger Eligibility Based on Real Time Threat Assessments Using Managed Inclusion Process

As we found in December 2014, Managed Inclusion is designed to provide expedited screening to passengers not deemed low risk prior to arriving at the airport. TSA uses Managed Inclusion as a tool to direct passengers who are not on a TSA PreTM list, or designated as eligible for expedited screening via the TSA PreTM Risk Assessments, into the expedited screening lanes to increase passenger throughput in these lanes when the volume of TSA PreTM -eligible passengers is low. In addition, TSA developed Managed Inclusion to improve the efficiency of dedicated TSA PreTM screening lanes as well as to help TSA reach its internal goal of providing expedited screening to at least 25 percent of passengers by the end of calendar year 2013.

To operate Managed Inclusion, TSA randomly directs a certain percentage of passengers not previously designated that day as eligible for expedited screening to the TSA PreTM expedited screening lane. To screen passengers who have been randomly directed into the expedited screening lane, TSA uses real time threat assessments including combinations of Behavior Detection Officers (BDOs), canine teams and Explosives Trace Detection (ETD) devices to ensure that passengers do not exhibit high-risk behaviors or otherwise present a risk at the airport.

According to TSA, it designed the Managed Inclusion process using a layered approach to provide security when providing expedited screening to passengers via Managed Inclusion. Specifically, these layers include (1) the Secure Flight vetting TSA performs to identify high-risk passengers required to undergo enhanced screening at the checkpoint and to ensure these passengers are not directed to TSA PreTM expedited screening lanes, (2) a randomization process that TSA uses to include passengers into TSA PreTM screening lanes who otherwise were not eligible for expedited screening, (3) BDOs who observe passengers and look for certain high-risk behaviors, (4) canine teams and ETD devices that help ensure that passengers have not handled explosive materials prior to travel, and (5) an unpredictable screening process involving walk-through metal detectors in expedited screening lanes that randomly select a percentage of passengers for additional screening.

Managed Inclusion Process

When passengers approach a security checkpoint that is operating Managed Inclusion, they approach a TSO who is holding a randomizer device, typically an iPad that directs the passenger to the expedited or standard screening lane. TSA officials stated that the randomization layer of security is intended to ensure that passengers cannot count on being screened in the expedited screening lane even if they use a security checkpoint that is operating Managed Inclusion. FSDs can adjust the percentage of passengers randomly sent into the Managed Inclusion lane depending on specific risk factors. Figure 3 illustrates how these layers of security operate when FSDs use Managed Inclusion lanes.

According to TSA, it designed the Managed Inclusion process to use BDOs stationed in the expedited screening lane as one of its layers of security when Managed Inclusion is operational to observe passengers’ behavior as they move through the security checkpoint queue. When BDOs observe certain behaviors that indicate a passenger may be higher risk, the BDOs are to refer the passenger to a standard screening lane so that the passenger can be screened using standard or enhanced screening procedures. In our November 2013 report on TSA’s behavior detection and analysis program, we concluded that although TSA had taken several positive steps to validate the scientific basis and strengthen program management of behavior detection and analysis program, TSA had not demonstrated that BDOs can reliably and effectively identify high- risk passengers who may pose a threat to the U.S. aviation system. Further, we recommended that the Secretary of Homeland Security direct the TSA Administrator to limit future funding support for the agency’s behavior detection activities until TSA can provide scientifically validated evidence that demonstrates that behavioral indicators can be used to identify passengers who may pose a threat to aviation security. The Department of Homeland Security did not concur with this recommendation, in part, because it disagreed with GAO’s analysis of TSA’s behavioral indicators. In February 2015, TSA officials told us that they had revised the behavioral indicators, were conducting pilot tests on the use of new BDO protocols, and anticipated concluding the testing at 5 airports in late 2015. At that time, TSA plans to make a determination about whether the new protocols are ready for further testing, including an operational test in 10 airports to determine the protocols’ effectiveness, which has an estimated completion date in the latter half of 2016.

According to a TSA decision memorandum and its accompanying analysis, TSA uses canine teams and ETD devices at airports as an additional layer of security when Managed Inclusion is operational to determine whether passengers may have interacted with explosives prior to arriving at the airport. In airports with canine teams, passengers must walk past a canine and its handler in an environment where the canine is trained to detect explosive odors and to alert the handler when a passenger has any trace of explosives on his or her person. For example, passengers in the Managed Inclusion lane may be directed to walk from the travel document checker through the passageway and past the canine teams to reach the X-ray belt and the walk-through metal detector. According to TSA documents, the canines, when combined with the other layers of security in the Managed Inclusion process provide effective security. According to TSA, it made this determination by considering the probability of canines detecting explosives on passengers, and then designed the Managed Inclusion process to ensure that passengers would encounter a canine a certain percentage of the time.

Our prior work examined data TSA had on its canine program, what these data showed, and to what extent TSA analyzed these data to identify program trends. Further we analyzed the extent to which TSA deployed canine teams using a risk-based approach and determined their effectiveness prior to deployment. As a result of this work, we recommended in January 2013, among other things, that TSA take actions to comprehensively assess the effectiveness of canine teams. The Department of Homeland Security concurred with this recommendation and has taken steps to address it. Specifically, according to TSA canine test results, TSA has conducted work to assess canine teams and to ensure they meet the security effectiveness thresholds TSA established for working in the Managed Inclusion lane, and the canines met these thresholds as a requirement to screen passengers in Managed Inclusion lanes.

In those airports where canines are unavailable or not working, TSA uses ETD devices as a layer of security when operating Managed Inclusion. TSOs stationed at the ETD device are to select passengers to have their hands swabbed as they move through the expedited screening lane. TSOs are to wait for a passenger to proceed through the Managed Inclusion queue and approach the device, where the TSO is to swab the passenger’s hands with an ETD pad and place the pad in the ETD device to determine whether any explosive residue is detected on the pad. Once the passenger who was swabbed is cleared, the passenger then proceeds through the lane to the X-ray belt and walk-through metal detector for screening. TSA procedures require FSDs to meet certain performance requirements when ETD devices are operating, such as swabbing passengers at a designated rate, and TSA data from January 1, 2014, through April 1, 2014, show that these requirements were not always met. Beginning in May 2014, TSA’s Office of Security Operations began tracking compliance with the ETD swab requirements and developed and implemented a process to ensure that the requirements are met. In March 2015 TSA officials confirmed this process was still in place.

According to TSA, it uses unpredictable screening procedures as an additional layer of security after passengers who are using expedited screening pass through the walk-through metal detector. This random selection of passengers for enhanced screening occurs after they have passed all security layers TSA uses for Managed Inclusion, and provides one more chance for TSA to detect explosives on a passenger.

TSA Has Not Tested the Overall Effectiveness of the Managed Inclusion Process

As we reported in December 2014, according to TSA, it designed the Managed Inclusion process using a layered approach to security when providing expedited screening to passengers via Managed Inclusion. Specifically, the Office of Security Capabilities’ proof of concept design noted that the Managed Inclusion process was designed to provide a more rigorous real-time threat assessment layer of security when compared to standard screening or TSA PreTM screening. According to the design concept, this real-time threat assessment, utilizing both BDOs and explosives detection, allows TSA to provide expedited screening to passengers who have not been designated as low risk without decreasing overall security effectiveness. While TSA has tested the security effectiveness of each of these layers of security, TSA has not yet tested the security effectiveness of the overall Managed Inclusion process as it functions as a whole.

Conducting Effectiveness Testing of the Managed Inclusion Process in Accordance with Evaluation Best Practices Would Enhance Reliability of the Process

GAO-14-159, GAO-10-763, GAO-13-239, GAO-14-695T, and GAO-11-740. addressing those recommendations but they have not yet been fully implemented. TSA officials stated that they have not yet tested the security effectiveness of the overall Managed Inclusion process as it functions as a whole, as TSA has been planning for such testing over the course of the last year. TSA documentation shows that the Office of Security Capabilities recommended in January 2013 that TSA test the security effectiveness of Managed Inclusion as a system. We reported in 2014 that according to officials, TSA anticipated that testing would begin in October 2014 and estimated that testing could take 12 to 18 months to complete. In March 2015, TSA officials provided us a schedule for the development and completion of BDO and Canine testing supporting the Managed Inclusion process. TSA scheduled a pilot for testing BDOs which was set to begin October 2014 and run through May 2015. Further, the schedule TSA provided indicates that a proof of concept for Canine Covert Testing was scheduled for November 2014 and that operational testing of canines was scheduled to begin in June 2015 and be completed in March 2016. Testing the security effectiveness of the Managed Inclusion process is consistent with federal policy, which states that agencies should assess program effectiveness and make improvements as needed.

We have previously reported on challenges TSA has faced in designing studies and protocols to test the effectiveness of security systems and programs in accordance with established methodological practices. For example, in our March 2014 assessment of TSA’s acquisition of Advanced Imaging Technology, we found that TSA conducted operational and laboratory tests, but did not evaluate the performance of the entire system, which is necessary to ensure that mission needs are met. element of evaluation design is to define purpose and scope, to establish what questions the evaluation will and will not address.

See GAO-14-357. We recommend that TSA better measure the effectiveness of its entire AIT system. TSA concurred with the recommendation. and analysis program. For example we found that TSA did not randomly select airports to participate in the study, so the results were not generalizeable across airports. In addition, we found that TSA collected the validation study data unevenly and experienced challenges in collecting an adequate sample size for the randomly selected passengers, facts that might have further affected the representativeness of the findings. According to established evaluation design practices, data collection should be sufficiently free of bias or other significant errors that could lead to inaccurate conclusions.

In our December 2014 report we concluded that ensuring the planned effectiveness testing of the Managed Inclusion process adheres to established evaluation design practices would help TSA provide reasonable assurance that the effectiveness testing will yield reliable results. The specific design limitations we identified in TSA’s previous studies of Advanced Imaging Technology and behavior detection and analysis program may or may not be relevant design issues for an assessment of the effectiveness of the Managed Inclusion process, as evaluation design necessarily differs based on the scope and nature of the question being addressed. In general, evaluations are most likely to be successful when key steps are addressed during design, including defining research questions appropriate to the scope of the evaluation, and selecting appropriate measures and study approaches that will permit valid conclusions. As a result, we recommended that to ensure that TSA’s planned testing yields reliable results, the TSA Administrator take steps to ensure that TSA’s planned effectiveness testing of the Managed Inclusion process adheres to established evaluation design practices. DHS concurred with our recommendations and began taking steps to ensure that its planned effectiveness testing of the Managed Inclusion process adheres to established evaluation practices. Specifically, DHS stated that TSA plans to use a test and evaluation process—which calls for the preparation of test and evaluation framework documents including plans, analyses, and a final report describing the test results—for its planned effectiveness testing of Managed Inclusion.

Chairman Katko, Ranking Member Rice, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contact and Staff Acknowledgements

For questions about this statement, please contact Jennifer Grover at (202) 512-7141 or groverj@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement include Glenn Davis (Assistant Director), Brendan Kretzschmar, Ellen Wolfe, David Alexander, Thomas Lombardi, Susan Hsu, Caroline Neidhold, and Serena Epstein. Key contributors for the previous work that this testimony is based on are listed in each product.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.