Aviation Security: Rapid Growth in Expedited Passenger Screening Highlights Need to Plan Effective Security Assessments
Released: 2014-12-12T00:00:00

Letter

Background

According to TSA, the majority of air carrier passengers present a low risk to aviation security, but until recently, with the exception of passengers matched to watchlists, TSA has used the same security screening procedures for passengers without regard for the risk passengers posed. TSA, through the Secure Flight system, utilizes watchlists derived from the Terrorist Screening Database—the U.S. government’s consolidated watchlist of known or suspected terrorists—and other sources to identify potentially high-risk passengers. For example, the No Fly List contains information on individuals who are prohibited from boarding an aircraft, and the Selectee List contains information on individuals who must undergo additional security screening before being permitted to board an aircraft. As part of TSA PreTM, TSA conducts terrorism related checks using lists derived from the database as a part of the security threat assessment process.

In 2011, TSA began to explore the benefits of using risk-based, intelligence-driven security screening procedures that would allow TSA to learn more about passengers prior to travel because, according to TSA, it can better assess passenger risk when it knows more about passengers. In addition, TSA worked to develop procedures to ensure screening resources are focused on passengers determined to be high risk and passengers about whom TSA has less information while expediting the screening of passengers TSA has assessed as being lower risk based on information it has on such passengers. Using this risk-based approach, TSA began to identify and define lower risk passenger populations or “known travelers”—that is, those who have volunteered personal information to TSA so that TSA can confirm these known travelers are lower risk.

Expedited Screening

When TSA began offering expedited airport screening in the summer of 2011, TSA initially provided such screenings in standard lanes to passengers aged 12 and younger, and subsequently extended expedited screening to certain flight crew members and then to passengers aged 75 and older. In October 2011, TSA began to expand the concept of expedited airport screening to more of the flying public by piloting the TSA PreTM program. This pilot program allowed certain frequent fliers of two air carriers to experience expedited screening at four airports. These frequent fliers became eligible for screening in dedicated expedited screening lanes, which became known as TSA PreTM lanes, because they had opted into the TSA PreTM program through the air carrier with which they had attained frequent flier status. TSA also allowed certain members of the U.S. Customs and Border Protection’s (CBP) Trusted Traveler programs to experience expedited screening as part of the TSA PreTM pilot. TSA provided expedited screening in dedicated screening lanes to these frequent fliers and CBP’s Trusted Travelers during the TSA PreTM pilot program because TSA had information about these passengers, and TSA used this information to determine that passengers in these groups were lower risk. When traveling on one of the air carriers and departing from one of the airports participating in the pilot, these passengers were eligible to be screened in dedicated TSA PreTM screening lanes where the passengers were not required to remove their shoes; divest light outerwear, jackets, and belts; or remove liquids, gels, and laptops from carry-on baggage.

After the pilot concluded, in February 2012, and transitioned into a formal program, TSA began to add additional air carriers and passenger groups to the TSA PreTM program. For an air carrier to participate in TSA PreTM, the air carrier must have the technological capability to send the necessary passenger information to Secure Flight for vetting against federal government watchlists, and print the low-risk designation in the encrypted boarding pass bar code and the TSA PreTM designation on the boarding pass. As each air carrier joined TSA PreTM, the carrier’s frequent fliers became eligible to opt in for expedited screening only when traveling on that air carrier.

Since October 2011, TSA further expanded the known traveler populations eligible for expedited screening. After TSA piloted TSA PreTM with certain passengers who are frequent fliers and members of CBP’s Trusted Traveler programs, TSA established separate TSA PreTM lists for additional low-risk passenger populations, including members of the U.S. armed forces, Congressional Medal of Honor Society Members, members of the Homeland Security Advisory Council, and Members of Congress, among others. In addition to TSA PreTM lists sponsored by other agencies or entities, TSA created its own TSA PreTM list composed of individuals who apply to be preapproved as low-risk travelers through the TSA PreTM Application Program, an initiative launched in December 2013. To apply, individuals must visit an enrollment center where they provide biographic information (i.e., name, date of birth, and address), valid identity and citizenship documentation, and fingerprints to undergo a TSA Security Threat Assessment. TSA leveraged existing federal capabilities to both enroll and conduct threat assessments for program applicants using enrollment centers previously established for the Transportation Worker Identification Credential Program, and existing transportation vetting systems to conduct applicant threat assessments. Applicants must be U.S. citizens, U.S. nationals or lawful permanent residents, and cannot have been convicted of certain crimes. As of April 2014, there were about 5.6 million individuals who, through TSA PreTM lists, were eligible for expedited screening. Figure 1 shows the populations for each TSA PreTM list.

In addition to passengers who are included on one of the TSA PreTM lists, in October 2013, TSA began implementing the TSA PreTM Risk Assessment program, which evaluates passenger risk using data available to TSA to determine a certain likelihood that passengers will be designated as eligible to receive expedited screening through TSA PreTM.

The TSA PreTM Process at the Airport

According to TSA officials, in February 2013, TSA established a policy to notify passengers of their eligibility for expedited screening using the air carrier reservation systems in order to improve passenger movement through airports for TSA PreTM eligible passengers. For every passenger, TSA uses the Secure Flight system to automatically match passenger information collected by the air carriers against the various watchlists (e.g., the No Fly and Selectee Lists) up to 72 hours before passengers’ scheduled air travel. After checking for matches to the watchlists, Secure Flight directs the air carrier to mark a passenger’s boarding pass for enhanced screening or expedited screening, or to identify a passenger as being prohibited from boarding an aircraft, or to identify a passenger for standard screening.TSA uses a similar process to identify passengers who are eligible for expedited screening at the airport by using the same information provided by the air carriers to the Secure Flight system to match against lists of individuals who have been designated as low risk. TSA informs passengers of this eligibility by directing the air carriers to mark the boarding pass with the TSA PreTM designation. Figure 2 shows examples of boarding passes with the TSA PreTM designation.

At airports, the mechanism TSA uses to screen passengers who have the TSA PreTM designation on their boarding pass depends on the configuration of the airport. At some airports, TSA has dedicated TSA PreTM expedited screening lanes where passengers with the TSA PreTM designation on their boarding passes are not required to divest shoes, light outerwear, laptops, liquids, and gels. Because TSA PreTM expedited screening is voluntary, a passenger designated as eligible for TSA PreTM expedited screening may choose not to use a TSA PreTM dedicated screening lane. Also, some airports do not have dedicated TSA PreTM expedited screening lanes either because of space restrictions that preclude the airport from installing a dedicated screening lane in a security checkpoint or because the number of passengers with the TSA PreTM boarding pass designations is low and therefore does not warrant a separate dedicated lane. According to TSA officials, at these airports, passengers with a TSA PreTM boarding pass can still experience expedited screening of “their persons” (i.e., passengers are not required to divest shoes, light jackets, and belts) and use a walk through metal detector in the standard screening lane; however, they must divest their liquids, gels, and laptops from baggage because the screening process used in the standard screening lanes should result in the transportation security officer (TSO) identifying these items and searching the baggage—slowing throughput in the standard screening lane—if these items are not removed.

For airports at which there are dedicated screening lanes, security checkpoints may not consistently have a volume of TSA PreTM passengers that warrants operating such lanes because the TSO working in the lanes would be underutilized. Also, an underutilized expedited screening lane can result in longer wait times in standard screening lanes because airports that offer expedited screening in dedicated screening lanes generally do so at the expense of standard screening lane availability, according to airport and TSA officials. To ensure the utility of expedited screening lanes for non-TSA PreTM passengers, TSA implemented the Managed Inclusion process in November 2012. The Managed Inclusion process involves using real-time threat assessment methods, including randomization procedures and behavior detection officers (BDO), as well as either canine teams or explosives trace detection (ETD) devices to screen non-TSA PreTM passengers in lanes that are otherwise dedicated to TSA PreTM passengers. TSA operates Managed Inclusion at the discretion of the airport’s FSD and is available at airports that have dedicated TSA PreTM expedited screening lanes, and either canine teams, ETD devices, or both. The Managed Inclusion process will be more fully described later in this report.

As of April 2014, TSA officials stated they provided expedited screening at essentially all of the approximately 450 airports at which TSA performs, or oversees the performance of, security screening, including 118 airports where TSA offers expedited screening in dedicated TSA PreTM screening lanes. The 118 airports where expedited screening is offered in dedicated TSA PreTM screening lanes represent about 95 percent of total air carrier enplanements based on Federal Aviation Administration calendar year 2012 data. Figure 3 shows the locations of the airports with dedicated TSA PreTM screening lanes. Also, appendix I provides a list of the airports included on the map.

TSA PreTM Risk Assessments and Other Programs Have Significantly Increased Availability and Use of Expedited Screening

TSA’s implementation of the TSA PreTM Risk Assessments, expansion of Managed Inclusion, and increasing the number of TSA PreTM airports accounted for a significant increase in the overall number of passengers designated as eligible for expedited screening, as well as the number of passengers who generally underwent such screening. Specifically, according to TSA data and as shown in figure 4, the number of TSA PreTM boarding passes issued each month grew slowly from October 2011, when TSA PreTM was launched, through September 2013, increasing from about 673,000 to about 3 million. In October 2013, when TSA began the TSA PreTM Risk Assessment process, TSA issued almost 9 million TSA PreTM boarding passes. Furthermore since October 2011, air carrier participation has expanded from two air carriers to nine air carriers as of April 2014. Figure 4 also shows the number of passengers receiving TSA PreTM expedited screening, the dates when air carriers began participating in the TSA PreTM program, and the dates when various programs intended to expand the use of expedited screening were implemented.

Figure 4 also shows the difference between the number of TSA PreTM boarding passes issued and the number of passengers who receive expedited screening at the airport. According to TSA officials, this difference occurs because TSA PreTM is a voluntary program and not all passengers who are eligible necessarily use expedited screening. For example, a passenger may be traveling with a group in which not all passengers in the group are eligible for expedited screening so the passenger may choose to forgo expedited screening. Prior to October 2013, TSA designated passengers as eligible for expedited screening because they were members of one of the populations TSA had designated as low risk. These populations included individuals who opted to participate in the TSA PreTM lists shown in figure 1, as well as almost 1.5 million frequent fliers who opted in to participate in the TSA PreTM program, and passengers aged 12 and under and 75 and older. In October 2013, TSA began to provide expedited screening to a much larger population of travelers using the TSA PreTM Risk Assessment program and Managed Inclusion. After TSA implemented the TSA PreTM Risk Assessment program any passenger flying on a participating air carrier could be designated as low risk and provided a TSA PreTM boarding pass designation.

At the same time the number of TSA PreTM boarding pass designations increased, the expedited screening throughput in TSA PreTM lanes also increased. For example, according to TSA data and as shown in figure 4, the number of passengers receiving expedited screening in September 2013 was about 2 million. In October 2013, about 8 million passengers received expedited screening, about a 300 percent increase. According to TSA officials, the increased throughput in TSA PreTM lanes was due to the implementation of the TSA PreTM Risk Assessments and the expansion from 40 to 100 airports at that time with dedicated expedited screening lanes. In addition, TSA increased the number of passengers with the opportunity to experience expedited screening in October and November 2013 by expanding Managed Inclusion, in which TSA uses real-time threat assessment methods to screen standard passengers in lanes that are otherwise dedicated to TSA PreTM passengers. Whereas TSA first implemented Managed Inclusion in November 2012 using canine teams to screen passengers for explosives as part of a real-time threat assessment, TSA determined it could operate Managed Inclusion at more airports by screening for explosives using ETD devices. TSA piloted Managed Inclusion with ETD devices in Boston and Seattle in July 2013, and implemented the program nationwide in October and November 2013 when it increased the number of airports operating Managed Inclusion with ETD devices. Figure 5 shows a snapshot from May 11, 2014, through May 18, 2014, of the percentage of weekly passengers receiving non-expedited screening and expedited screening, and further shows whether known crew members experienced expedited screening, and whether expedited screening occurred in TSA Pre™ lanes (for passengers designated as known travelers or through the TSA Pre™ Risk Assessment program, or passengers chosen for expedited screening using Managed Inclusion), or in standard lanes.

As noted in figure 5, out of the 41 percent of passengers nationwide receiving expedited screening during the week ending May 18, 2014, nearly 40 percent of them were issued TSA Pre™ boarding passes, but were provided expedited screening in a standard screening lane, meaning that they were provided expedited screening of their persons and did not have to remove their shoes, belts, and light outerwear, but they had to divest their liquids, gels, and laptops. TSA provides expedited screening to TSA PreTM-eligible passengers in standard lanes when airports do not have dedicated TSA PreTM screening lanes because of airport space constraints and limited TSA PreTM throughput.

TSA Used Various Methods to Collaborate with Stakeholders about Expedited Screening; TSA has begun Taking Action to Improve Communication with Passengers

TSA collaborated with stakeholders about how TSA PreTM works in various ways, including presenting information about TSA PreTM at industry conferences and events, holding monthly meetings with air carriers and airport authorities, and conducting stakeholder briefings with air carriers and airport authorities when implementing TSA PreTM at airports. In addition, TSA officials stated that they have provided Power Point presentations, reference materials, and promotional materials such as advertisements to air carrier and industry stakeholders. TSA officials said that they have also worked with a number of airports to set up tables and information booths where passengers can obtain information about TSA PreTM and expedited screening. Further, officials stated TSA has updated their website and provided TSA PreTM information via the My TSA App to inform passengers about how to enroll in the TSA PreTM Application Program, which air carriers are participating in TSA PreTM, and which airports have dedicated TSA PreTM screening lanes. TSOs informally communicate information about TSA PreTM at airports by explaining to passengers waiting in the screening queue how they could enroll in the TSA PreTM Application Program to be eligible for expedited screening.

Stakeholders told us that TSA coordinated with them at the local level to implement dedicated TSA PreTM lanes. For example, one airport authority we spoke with stated that TSA held regular meetings with airport management and with the air carriers and included them in the implementation process for opening expedited screening lanes. In addition, representatives from two of the six industry associations that represent passenger groups and that we interviewed noted that TSA’s coordination with them was effective and that TSA gave the associations the opportunity to provide input to TSA on how the program was first implemented. According to these industry associations, TSA met with them to encourage the associations to advertise expedited screening and TSA PreTM to members. Further, three of five air carriers said that they worked closely with TSA officials at the local airport level and provided input to TSA on issues like where to place TSA PreTM lanes within the airport.

Eight of the 16 stakeholders that we interviewed stated that TSA could do a better job of communicating to passengers details about expedited screening in dedicated TSA PreTM lanes. For example, representatives from air carriers and industry associations stated that TSA could better inform passengers about (1) the TSA PreTM expedited screening eligibility requirements, including the fact that expedited screening is not always guaranteed even when a passenger is on one of the TSA PreTM lists; (2) how to know if they are eligible for expedited screening on a flight on a given day; and (3) the divestiture requirements and procedures in TSA PreTM screening lanes, among other things. In addition, officials from one airport authority noted that they observe passengers who are confused and improperly using expedited screening and stated that passengers who travel infrequently could be better educated about TSA PreTM. Furthermore, at two of the six airports we visited we observed customers divesting liquids, laptops, shoes, and outerwear in TSA PreTM screening lanes, which we noted caused the throughput to decrease and wait times to increase for other passengers in the TSA PreTM expedited screening lanes. TSA officials stated that it takes time to train passengers about the expedited screening process and that some confusion on the screening procedures is to be expected as passengers are retrained on this process after becoming accustomed to the security measures instituted since the September 11, 2001, terrorist attacks.

TSA tracks customers’ experiences using the TSA Contact Center, where passengers can call or e-mail TSA officials about passenger screening experiences or ask TSA staff for information about security screening. Contact Center staff record details of each call and label the calls regarding TSA PreTM as compliments, complaints, or information requests about the program. TSA has collected the number of calls received regarding TSA PreTM since October of 2011, reviews these data weekly, and produces a weekly report on the call data. Our analysis of Contact Center data shows that each month since October 2011, passengers have submitted between 303 and 4,211 information requests about TSA PreTM screening lanes. As shown in table 1, as of April 2014 TSA had received over 97,000 calls regarding TSA PreTM since October of 2011.

As of July 2014, TSA officials stated they are beginning to analyze trends and patterns from the Contact Center data in order to determine the effectiveness of their advertising and messaging to passengers, and to identify potential ways to make improvements. The Office of the Chief Risk Officer has developed a trend analysis to record the number of TSA PreTM-related calls the Contact Center received per month since October of 2012. The trend analysis also tracks the types of calls received over time to identify changes in the number of compliments, complaints, or information requests and highlights trends relative to events such as the start up of TSA PreTM Risk Assessments in October 2013 and TSA’s roll-out of the TSA PreTM Application Program in December 2013. While these efforts could address concerns raised by stakeholders, they are still in the early stages, so it is too soon to determine whether TSA’s planned actions will effectively address these concerns.

TSA Uses Three Methods to Assess Risk for Participation in Expedited Screening, But Has Not Tested the Overall Effectiveness of the Managed Inclusion Process

TSA determines a passenger’s eligibility for or opportunity to experience expedited screening at the airport using one of three risk assessment methods. These include (1) inclusion on a TSA PreTM list of known travelers, (2) identification of passengers as low risk by TSA’s Risk Assessment algorithm, or (3) a real-time threat assessment at the airport using the Managed Inclusion process.

Passenger Eligibility Based on Inclusion on a TSA PreTM List of Known Travelers

TSA has determined that the individuals included on the TSA PreTM lists of known travelers are low risk by virtue of their membership in a specific group or based on group vetting requirements. For example, TSA determined that members of the Congressional Medal of Honor Society, a group whose members have been awarded the highest U.S. award for valor in action against enemy forces, present a low risk to transportation security and are good candidates to receive expedited screening. In other cases, TSA determined that members of groups whose individual members have undergone a security threat assessment by the federal government, such as individuals working for agencies in the intelligence community and who hold active Top Secret/Sensitive Compartmentalized Information clearances, are low risk and can be provided expedited screening. Similarly, TSA designated all active and reserve service members of the United States armed forces, whose combined members total over 2 million people, as a low risk group of trusted travelers. TSA determined that active duty military members were low risk and good candidates to receive expedited screening because the Department of Defense administers common background checks of its members.

Except for those who joined through the TSA Pre™ Application program, the TSA Pre™ lists include populations for which TSA coordinated with a lead agency or outside entity willing to compile and maintain the lists. TSA has entered into separate agreements with the various agencies and entities to administer these lists. Generally, according to these agreements, Secure Flight has responsibility for receiving and processing the lists, but the originating agencies or entities are to maintain them by ensuring that individuals continue to meet the criteria for inclusion and to update the lists as needed.

TSA also continues to provide expedited screening on a per-flight basis to the almost 1.5 million frequent fliers who opted to participate in the TSA Pre™ program pilot. According to TSA, this group of eligible frequent fliers met the standards set for the pilot based on their frequent flyer status as of October 1, 2011. TSA determined that these frequent fliers were an appropriate population to include in the program for several reasons, including the fact that frequent fliers are vetted against various watchlists each time they travel to ensure that they are not listed as known or suspected terrorists and screened at the checkpoint.

Passenger Eligibility Based on TSA PreTM Risk Assessments

The TSA PreTM Risk Assessment program evaluates passenger risk based on certain information available for a specific flight and determines the likelihood that passengers will be designated as eligible to receive expedited screening through TSA PreTM. Beginning in 2011, TSA piloted the process of using the Secure Flight system to obtain Secure Flight Passenger Data from air carriers and other data to assess whether the passenger is low risk on a per-flight basis and thus eligible to receive a TSA PreTM designation on his or her boarding pass to give the flier access to expedited screening. In September 2013 after completing this pilot, TSA decided to explore expanding this risk assessment approach to every traveler. In order to develop the set of low-risk rules to determine the passengers’ relative risk, TSA formed an Integrated Project Team consisting of officials from the Offices of Security Operations, Intelligence and Analysis, Security Capabilities, and Risk-Based Security. The team used data from multiple sources, including passenger data from the Secure Flight system from calendar year 2012, to derive a baseline level of relative risk for the entire passenger population. Our review of TSA’s documentation showed that TSA considered the three elements of risk assessment—Threat, Vulnerability, and Consequence—in its development of the risk assessment. These three elements constitute the framework for assessing risk as called for in the Department of Homeland Security’s National Infrastructure Protection Plan. TSA worked with a contractor to evaluate the data elements and the proposed risk model rules used for the baseline level of relative risk. In its assessment of the algorithm used for the analysis, the contractor agreed with TSA’s analysis of the relationship between the data elements and relative risk assigned to the data elements.

Although TSA determined that certain combinations of data elements in its risk-based algorithm are less likely to include unknown potential terrorists, it also noted that designating passengers as low risk based solely on the algorithm carries some risk. To mitigate these risks, TSA uses a random exclusion factor that places passengers, even those who are otherwise eligible for expedited screening, into standard screening a certain percentage of the time. TSA adjusts the level of random exclusion based on the relative risk of the combinations of various data elements used in the algorithm, such that data combinations carrying more risk are randomly excluded from expedited screening more often than other data combinations. For example, TSA’s assessment indicated that combinations of certain data elements are considered relatively more risky than other data groups and passengers who fit this profile for a given flight should seldom be eligible for expedited screening, while combinations of other data on a given flight pose relatively less risk and could therefore be made eligible for expedited screening most of the of the time. TSA developed a risk algorithm that scores each passenger on each flight, and passengers with a high enough score receive a TSA PreTM boarding pass designation making them eligible for expedited screening for that trip.

For both the TSA PreTM known traveler lists and the TSA PreTM Risk Assessments, TSA uses the Secure Flight system to determine passengers’ risk levels and to assign low-risk passengers the TSA PreTM designation to their boarding passes. Air carriers collect each passenger’s name and date of birth when the passenger books travel. When a passenger has a known traveler number, he or she may enter the number into the air carrier reservation system when booking travel. The air carrier sends this passenger information, along with the travel itinerary, to the Secure Flight system 72 hours before the passenger’s scheduled air travel. The Secure Flight system checks passenger data against watchlists and TSA PreTM lists, and runs the data through the low-risk rules in the TSA PreTM Risk Assessment algorithm, including applying the random exclusion rate to some passengers. Secure Flight then directs the air carrier to mark a passenger’s boarding pass for enhanced screening or expedited screening, or to identify a passenger as prohibited from boarding an aircraft or for standard screening. Figure 6 illustrates the process by which TSA uses Secure Flight to determine the boarding pass result.

Passenger Eligibility Based on Real Time Threat Assessments Using Managed Inclusion Process

Managed Inclusion is designed to provide expedited screening to passengers not deemed low risk prior to arriving at the airport. TSA uses Managed Inclusion as a tool to direct passengers who are not on a TSA PreTM list or designated as eligible for expedited screening via the TSA PreTM Risk Assessments into the expedited screening lanes to increase passenger throughput in these lanes when the volume of TSA PreTM- eligible passengers is low. In addition, Managed Inclusion was developed to improve the efficiency of dedicated TSA PreTM screening lanes as well as to help TSA reach its internal goal of providing expedited screening to at least 25 percent of passengers by the end of calendar year 2013. TSA randomly selects passengers to enter the Managed Inclusion queue using a randomizer device that directs a certain percentage of passengers not previously designated that day as eligible for expedited screening to the TSA PreTM expedited screening lane. To screen passengers who have been randomly directed into the expedited screening lane, TSA uses real time threat assessments including combinations of BDOs, canine teams and ETD devices to ensure that passengers do not exhibit high-risk behaviors or otherwise present a risk at the airport.

According to TSA, it designed the Managed Inclusion process using a layered approach to provide security when providing expedited screening to passengers via Managed Inclusion. Specifically, the Office of Security Capabilities’ proof of concept design noted that the Managed Inclusion process was designed to provide a more rigorous real-time threat assessment layer of security when compared to standard screening or TSA PreTM screening. According to the design concept, this real-time threat assessment, utilizing both BDOs and canine teams, allows TSA to provide expedited screening to passengers who have not been designated as low risk without decreasing overall security effectiveness. These layers include (1) the Secure Flight vetting TSA performs to identify high-risk passengers required to undergo enhanced screening at the checkpoint and to ensure these passengers are not directed to TSA PreTM expedited screening lanes, (2) a randomization process that TSA uses to include passengers into TSA Pre™screening lanes who otherwise were not eligible for expedited screening, (3) BDOs who observe passengers and look for certain high-risk behaviors, (4) canine teams and ETD devices that help ensure that passengers have not handled explosive materials prior to travel, and (5) an unpredictable screening process involving walk-through metal detectors in expedited screening lanes that randomly select a percentage of passengers for additional screening.

Managed Inclusion Process

When passengers approach a security checkpoint that is operating Managed Inclusion, they approach a TSO who is holding a randomizer device, typically an iPad that directs the passenger to the expedited or standard screening lane. TSA officials stated that the randomization layer of security is intended to ensure that passengers cannot count on being screened in the expedited screening lane even if they use a security checkpoint that is operating Managed Inclusion. FSDs can adjust the percentage of passengers randomly sent into the Managed Inclusion lane depending on specific risk factors. Figure 7 illustrates how these layers of security operate when FSDs use Managed Inclusion lanes.

According to TSA, it designed the Managed Inclusion process to use BDOs stationed in the expedited screening lane as one of its layers of security when Managed Inclusion is operational to observe passengers’ behavior as they move through the security checkpoint queue. When BDOs observe certain behaviors that indicate a passenger may be higher risk, the BDOs are to refer the passenger to a standard screening lane so that the passenger can be screened using standard or enhanced screening procedures. We have conducted past work on TSA’s behavior detection analysis program, including the Screening of Passengers by Observation Techniques (SPOT) program, which BDOs use to identify potential high-risk passengers. In our November 2013 report, we reported that although TSA has taken several positive steps to validate the scientific basis and strengthen program management of behavior detection analysis and the SPOT program, TSA has not demonstrated that BDOs can reliably and effectively identify high-risk passengers who may pose a threat to the U.S. aviation system. In our 2013 report, we recommended that the Secretary of Homeland Security direct the TSA Administrator to limit future funding support for the agency’s behavior detection activities until TSA can provide scientifically validated evidence that demonstrates that behavioral indicators can be used to identify passengers who may pose a threat to aviation security. The Department of Homeland Security did not concur with this recommendation; however, in August 2014, TSA noted that it is taking actions to optimize the effectiveness of its behavior detection program and plans to begin testing this effort in October 2014.

According to a TSA decision memorandum and its accompanying analysis, TSA uses canine teams and ETD devices at airports as an additional layer of security when Managed Inclusion is operational to determine whether passengers may have interacted with explosives prior to arriving at the airport. In airports with canine teams, passengers must walk past a canine and its handler in an environment where the canine is trained to detect explosive odors and to alert the handler when a passenger has any trace of explosives on his or her person. For example, passengers in the Managed Inclusion lane may be directed to walk from the travel document checker through the passageway and past the canine teams to reach the X-ray belt and the walk-through metal detector. According to TSA documents, the canines, when combined with the other layers of security in the Managed Inclusion process provide effective security. According to TSA, it made this determination by considering the probability of canines detecting explosives on passengers, and then designed the Managed Inclusion process to ensure that passengers would encounter a canine a certain percentage of the time.

Our prior work examined data TSA had on its canine program, what these data show, and to what extent TSA analyzed these data to identify program trends. Further we analyzed the extent to which TSA deployed canine teams using a risk-based approach and determined their effectiveness prior to deployment. As a result of this work, we recommended in January 2013, among other things, that TSA take actions to comprehensively assess the effectiveness of canine teams. The Department of Homeland Security concurred with this recommendation and has taken steps to address it. Specifically, according to TSA canine test results, TSA has conducted work to assess canine teams and to ensure they meet the security effectiveness thresholds TSA established for working in the Managed Inclusion lane; and the canines met these thresholds as a requirement to screen passengers in managed inclusion lanes.

In those airports where canines are unavailable, TSA uses ETD devices as a layer of security when operating Managed Inclusion. TSOs stationed at the ETD device are to select passengers to have their hands swabbed as they move through the expedited screening lane. TSOs are to wait for a passenger to proceed through the Managed Inclusion queue and approach the device, where the TSO is to swab the passenger’s hands with an ETD pad and place the pad in the ETD device to determine whether any explosive residue is detected on the pad. Once the passenger who was swabbed is cleared, the passenger then proceeds through the lane to the X-ray belt and walk-through metal detector for screening. TSA procedures require FSDs to meet certain performance requirements when ETD devices are operating, and TSA data from January 1, 2014, through April 1, 2014, show that these requirements were not always met. Beginning in May 2014, TSA’s Office of Security Operations began tracking compliance with the ETD swab requirements and developed and implemented a process to ensure that the requirements are met.

According to TSA, it uses unpredictable screening procedures as an additional layer of security after passengers who are using expedited screening pass through the walk-through metal detector. This random selection of passengers for enhanced screening after they have passed all security layers TSA uses for Managed Inclusion provides one more chance for TSA to detect explosives on a passenger.

TSA Should Ensure that Its Effectiveness Testing of the Managed Inclusion Process is Conducted in Accordance with Established Methodological Practices

TSA officials stated that they tested the security effectiveness of the individual components of the Managed Inclusion process before implementing Managed Inclusion, and determined that each layer alone provides an effective level of security. For example, TSA tested the threat detection ability of its canines using a variety of variables such as concealment location and the length of time the item was concealed prior to the encounter with the canine team. We did not evaluate the security effectiveness testing TSA conducted on the individual layers of the Managed Inclusion process. However, we have previously conducted work on several of the layers used in the Managed Inclusion process, including BDOs, ETD, and canine teams and raised concerns regarding their effectiveness and recommended actions to address those concerns. As discussed earlier in this report, TSA has made progress in addressing those recommendations. TSA determined through the initial testing of the Managed Inclusion layers that Managed Inclusion provides a higher level of security than TSA baseline security levels. In addition, according to TSA standard operating procedures, Managed Inclusion passengers are more likely than other passengers to be screened for explosives. TSA officials stated that they have not yet tested the security effectiveness of the overall Managed Inclusion process as it functions as a whole, as TSA has been planning for such testing over the course of the last year. TSA documentation shows that the Office of Security Capabilities recommended in January 2013 that TSA test the security effectiveness of Managed Inclusion as a system. According to officials, TSA anticipates that testing will begin in October 2014 and estimates that testing could take 12 to 18 months to complete. However, TSA could not provide us with specifics or a plan or documentation showing how the testing is to be conducted, the locations where it is to occur, how those locations are to be selected, or the timeframes for conducting testing at each location. Testing the security effectiveness of the Managed Inclusion process is consistent with federal policy, as laid out in Executive Order 13450—Improving Government Program Performance.

We have previously reported on challenges TSA has faced in designing studies and protocols to test the effectiveness of security systems and programs in accordance with established methodological practices. For example, in our March 2014 assessment of TSA’s acquisition of Advanced Imagining Technology, we found that TSA conducted operational and laboratory tests, but did not evaluate the performance of the entire system, which is necessary to ensure that mission needs are met. A key element of evaluation design is to define purpose and scope, to establish what questions the evaluation will and will not address.

Further, in November 2013 we identified methodological weaknesses in the overall design and data collection of TSA’s April 2011 validation comparison study to determine the effectiveness of the SPOT program.

For example we found that TSA did not randomly select airports to participate in the study, so the results were not generalizeable across airports. In addition, we found that TSA collected the validation study data unevenly and experienced challenges in collecting an adequate sample size for the randomly selected passengers, facts that might have further affected the representativeness of the findings. According to established evaluation design practices, data collection should be sufficiently free of bias or other significant errors that could lead to inaccurate conclusions.

Ensuring its planned effectiveness testing of the Managed Inclusion process adheres to established evaluation design practices will help TSA provide reasonable assurance that the effectiveness testing will yield reliable results. The specific design limitations we identified in TSA’s previous studies of Advanced Imaging Technology and SPOT may or may not be relevant design issues for an assessment of the effectiveness of the Managed Inclusion process, as evaluation design necessarily differs based on the scope and nature of the question being addressed. In general, evaluations are most likely to be successful when key steps are addressed during design, including defining research questions appropriate to the scope of the evaluation, and selecting appropriate measures and study approaches that will permit valid conclusions.

TSA Could Improve Program Assessment by Aligning Goals with Measures and Has Estimated Savings from Expedited Screening for Fiscal Year 2015

TSA has two goals and one measure intended to assess the performance of its expedited screening programs, but our analysis shows that the goals and measure are not aligned. Also, TSA has estimated savings from expedited screening and has reduced its fiscal year 2015 budget request by the amount of the estimated savings.

TSA’s Expedited Screening Performance Measure Does Not Accurately Link to Goals on Passenger Eligibility

TSA has two goals intended to assess the performance of its expedited screening programs and uses one measure to track progress toward these goals. However, our analysis shows that the measure used does not align with the goals. Specifically, TSA’s program goals are to ensure: (1) that 25 percent of air passengers were eligible for expedited screening by the end of calendar year 2013; and (2) that 50 percent of passengers are eligible for expedited screening by the end of calendar year 2014. According to TSA documents, TSA uses one measure—the total number of air passengers screened daily using expedited screening as a percentage of the total number of passengers screened daily—to assess progress towards these goals. TSA collects data for this measure by reporting, not the number of passengers designated as eligible for expedited screening, but the number of passengers who actually receive such screening. As noted earlier in this report, because expedited screening is voluntary, not all passengers who are eligible necessarily use expedited screening. For example, a passenger may be traveling with a group in which not all passengers in the group are eligible for expedited screening, so the passenger may choose to forgo expedited screening. Also, TSA may not use boarding pass scanners at the airport or at a specific checkpoint, which would preclude TSA from offering expedited screening at the airport to passengers with TSA PreTM boarding passes who were not otherwise eligible for expedited screening due to the age of the passenger. As a result, the information that TSA is reporting to show that it is meeting its goal may be understated and inaccurate.

TSA’s Chief Risk Officer agreed that the goals and the measure are not linked, but said that tracking actual screening data rather than eligibility data presents a more accurate picture of the expedited screening program performance. Specifically, the Chief Risk Officer noted that TSA’s definition of “eligible” is broader than passengers who receive the TSA PreTM designation on the boarding pass and that eligible passengers also include those individuals who have opted into the TSA PreTM program as members of one of the TSA PreTM lists, and passengers who fly on one of the participating TSA PreTM air carriers who could become eligible through the TSA PreTM Risk Assessment process. He stated that broadly measuring the number of passengers eligible rather than those who receive a TSA PreTM boarding pass result (making them eligible for expedited screening) would overstate progress toward the eligibility goal. According to TSA, the Administrator set an internal TSA target in September 2012 that 25 percent of daily travelers screened by TSA were to receive some form of expedited screening by the end of the calendar year 2013 and 50 percent by the end of the calendar year 2014 and did so to encourage the development and expansion of TSA’s risk-based security initiatives and expedited screening of passengers. TSA also noted that it began achieving the 25 percent of passengers receiving expedited screening goal in November 2013 and attributed reaching this goal to an increase in the number of participating air carriers from seven in July 2013 to nine in November 2013, an increase in the number of airports where TSA PreTM dedicated screening lanes are available to over 100 airports in October 2013, the implementation of TSA PreTM Risk Assessments on a flight-by-flight basis in October 2013, and the increased use of Managed Inclusion in October 2013.

While we agree that tracking actual screening data may provide insights about expedited screening program performance, ensuring that the goals and measure are aligned is important to provide more accurate performance measurement data to guide program performance. Best practices regarding the key attributes of successful performance measurement state that performance measures should link and align with agency-wide goals and the mission should be clearly communicated throughout the organization. Furthermore, in response to a requirement of the Department of Homeland Security Appropriations Act, 2014, TSA submitted a report to the Committees on Appropriations of the Senate and House of Representatives certifying that one in four (or 25 percent of) passengers are eligible for expedited screening without lowering security standards and outlined a strategy to increase the number of passengers eligible for expedited screening to 50 percent by the end of calendar year 2014. Aligning TSA’s measures and goals could help ensure that TSA, as well as lawmakers, has a clear and accurate picture of the program’s performance and can make improvements as needed.

TSA’s Fiscal Year 2015 Budget Estimates Savings Resulting from Expedited Screening Programs

TSA’s fiscal year 2015 budget request includes savings noted as a budget decrease resulting from efficiencies anticipated from risk-based security initiatives including expedited screening. In its fiscal year 2015 budget request, TSA estimated savings of $100 million as a result of risk- based security initiatives and primarily based these savings on an expected reduction in screening staff at airports. Specifically, TSA estimated that it could reduce staff costs by about $92 million (or 1,441 full-time-equivalent staff) with an additional savings of almost $8 million realized from an associated reduction in indirect costs such as training, information technology support, and recruitment, among other expenses. According to TSA officials, implementing expedited screening allows for staff reductions because TSA is able to operate fewer screening lanes while maintaining throughput rates and short wait times. Also, TSA officials noted that these staff reductions are to be realized through staff attrition. Table 2 shows TSA’s estimated cost savings as a result of risk- based security initiatives.

Identifying potential cost savings and including these savings amounts in proposed budgets is consistent with Office of Management and Budget federal budgeting guidance. Specifically, for the fiscal year 2014 budget cycle, the Office of Management and Budget instructed agencies to identify programs where legislative, budget, or administrative changes could improve program effectiveness and efficiency and result in cost savings so that the deficit reductions included in the Budget Control Act of 2011 could be realized. Further, the Office of Management and Budget’s fiscal year 2015 budget guidance instructed agencies to continue to identify areas where cost savings might be realized, consistent with the fiscal year 2014 budget guidance. In addition, identifying potential savings from program efficiencies in spending estimates is consistent with Office of Management and Budget guidance for developing budget submissions and included in the instructions provided to federal agencies for preparing annual budgets. We compared TSA’s savings estimates with the Office of Management and Budget guidance for preparing agency budgets and found that TSA’s estimates were consistent with this guidance. Specifically, the guidance states that agencies’ budgets must reflect all requirements anticipated by the agency at the time of the budget submission, including budget decreases for activities proposed for reduction.

Conclusions

TSA’s new methods to assess passenger risk, such as TSA PreTM Risk Assessments and Managed Inclusion, have significantly increased the use of expedited screening. As a result, it will be important for TSA to evaluate the security effectiveness of the Managed Inclusion process as a whole, to ensure that it is functioning as intended and that passengers are being screened at a level commensurate with their risk. According to TSA, this testing is scheduled to begin in October 2014 and could take 12 to 18 months to complete. Ensuring that its planned effectiveness testing of the Managed Inclusion process adheres to established evaluation design practices will help TSA provide reasonable assurance that the effectiveness testing yields reliable results. Regarding overall program performance, although TSA collects information to assess the performance of its expedited screening programs, its assessment could be improved because the performance goals and the measures used to evaluate those goals are currently not aligned. Aligning TSA’s measures and goals could help ensure that TSA, as well as lawmakers, has a clear and accurate picture of the program’s performance and can make improvements as needed.

Recommendations for Executive Action

GAO is recommending that TSA take the following two actions.

To ensure that TSA’s planned testing yields reliable results, we recommend that the TSA Administrator take steps to ensure that TSA’s planned effectiveness testing of the Managed Inclusion process adheres to established evaluation design practices.

To ensure that TSA has accurate information by which to measure the performance of its expedited screening programs, we recommend that the TSA Administrator ensure that the expedited screening performance goals and measures align.

Agency Comments and Our Evaluation

We provided DHS with a copy of this report for review and comment. On November 19, 2014, DHS provided written comments, which are summarized below and reproduced in full in appendix II. DHS concurred with our two recommendations and described actions under way or planned to address them. In addition, DHS provided written technical comments, which we incorporated into the report as appropriate.

DHS concurred with our first recommendation that TSA take steps to ensure that its planned effectiveness testing of the Managed Inclusion process adheres to established evaluation practices. DHS stated that TSA plans to use a test and evaluation process—which calls for the preparation of test and evaluation framework documents including plans, analyses, and a final report describing the test results—for its planned effectiveness testing of Managed Inclusion. In addition, TSA has begun collaborating with National Institute of Standards and Technology statistical engineering staff who are to share their knowledge and experience with TSA as well as assist with the design, analysis, and reporting of the test and evaluation process. These actions, if implemented effectively, should address the intent of our recommendation.

DHS concurred with our second recommendation that TSA ensure that the expedited screening performance goals and measure align. According to DHS, all measures are approved by DHS and the Office of Management and Budget, and changes must be approved by both agencies. In June 2014, TSA began working with both agencies to modify the measure to ensure that it aligned with the expedited screening goals. This action, once completed, should address the intention of our recommendation.

We are sending copies of this report to the appropriate Congressional committees, the Secretary of the Department of Homeland Security, and other interested parties. In addition, this report will be made publicly available at no extra charge on the GAO website at http://www.gao.gov.

Should you or your staff have any questions about this report, please contact Jennifer A. Grover at 202-512-7141 or GroverJ@gao.gov. Key contributors to this report are acknowledged in appendix III.

Appendix I: Listing of 118 Airports at Which Transportation Security Administration (TSA) Offers Expedited Screening with Dedicated Screening Lanes as of April 2014

Appendix II: Comments from the Department of Homeland Security

Appendix III: GAO Contact and Staff Acknowledgments

Contact

Staff Acknowledgments

In addition to the contact named above, Glenn Davis (Assistant Director), Ellen Wolfe (Analyst-in-Charge), Chuck Bausell, Eric Hauswirth, Suling Homsy, Susan Hsu, Brendan Kretzschmar, Stanley Kostyla, Thomas Lombardi, Linda Miller, and Jean Orland made key contributions to this report.