Airport Passenger Screening: Preliminary Observations on Progress Made and Challenges Remaining
Released: 2003-09-25T00:00:00

Letter

Background

ATSA created TSA as an agency within the Department of Transportation (DOT) to ensure security for all modes of transportation, to include aviation. ATSA set forth specific enhancements to aviation security for TSA to implement and established deadlines for completing many of them. These enhancements included federalizing passenger screeners at more than 440 commercial airports by November 19, 2002; screening checked baggage for explosives by December 31, 2002; enhancing screener training standards; and establishing and managing a 2-year pilot program at five airports—one in each airport category—where screening of passengers and property would be conducted by a private screening company and overseen by TSA. Additionally, ATSA included a provision that allows airport operators to apply to opt-out of using federal screeners in favor of private screeners beginning November 19, 2004.

Prior to the passage of ATSA, air carriers were responsible for screening passengers and most used private security firms to perform this function. Longstanding concerns existed regarding screener performance in detecting threat objects. Inadequate training and poor supervision, along with rapid turnover and inadequate attention to human factors, were historically identified as key contributors to poor screener performance.As early as 1987, we reported that too little attention had been paid to (1) individual aptitudes for effectively performing screening duties; (2) the sufficiency of screener training and screeners’ ability to comprehend training; and (3) the monotony of the job and distractions that reduced screeners’ vigilance. Additional studies have shown that effective training can lead to more effective performance and lower turnover rates for passenger screeners.

Concerns have long existed over screeners’ inability to detect threat objects during covert tests at passenger screening checkpoints. In 1978, screeners failed to detect 13 percent of the potentially dangerous objects FAA agents carried through checkpoints during tests—a level that was considered “significant and alarming.” In 1987, screeners did not detect 20 percent of the objects during the same types of tests. In addition, we reported that FAA tests conducted between 1991 and 1999 showed that screeners’ ability to detect objects was not improving, and in some cases was worsening. In tests conducted in the late 1990s, as the testing objects became more realistic and more closely approximated how a terrorist might attempt to penetrate a checkpoint, screeners’ ability to detect dangerous objects declined even further.

Scope and Methodology

Our preliminary observations are based on our review of TSA documentation related to screener training, testing, and supervision; the contract screening pilot program; screener staffing levels; and airport security concerns. We interviewed TSA headquarters’ officials in Arlington, Virginia; and interviewed FSDs, their staffs, and screeners at 12 commercial airports throughout the nation; 10 airport operators; officials at 5 air carriers; and officials from 4 aviation associations— American Association of Airport Executives (AAAE), Airports Council International (ACI), Air Transport Association, and Regional Airline Association. We also reviewed our prior reports that addressed issues related to the performance of airport passenger screeners. We conducted our work from May through September 2003 in accordance with generally accepted government auditing standards. Because our review is still ongoing, the results presented in this report are preliminary.

To complete our work, we will continue to collect and review TSA documentation related to each of our four objectives, including obtaining and analyzing the results of TSA’s operational tests. We will also administer a survey to all 158 FSDs to obtain their perspectives on general and airport specific information related to each of our four objectives. Additionally, we will visit at least 8 additional airports to conduct interviews with FSDs, their staffs, members of the screener workforce, and airport operators. We will also interview representatives of all 5 pilot program airports, as well as airport operators at all category X airports, to obtain information on their coordination with TSA and their plans, if any, to apply to opt-out of the federal screening program beginning November 19, 2004. Finally, we will continue to meet with TSA headquarters officials to obtain current information related to the issues addressed in this report. We anticipate issuing a final report in April 2004.

Recurrent and Supervisory Training Programs Not Fully Developed

TSA developed basic and remedial screener training programs, but has not fully developed or deployed a recurrent or supervisory training program to ensure that screeners are effectively trained and supervised. Comprehensive and frequent training is key to passenger screeners’ ability to detect threat objects. Studies have shown that on-going training can lead to more effective performance and lower turnover rates for passenger screeners. According to TSA, there are three key elements of passenger screener training: (1) basic training, (2) remedial training, and (3) recurrent training. As required by ATSA, TSA established a basic screener-training program comprised of 40 hours of classroom instruction and 60 hours of on-the-job training (OJT). TSA reported that all of its screeners who work independently have completed basic screener training and that those who failed an operational test received required remedial training.

Basic Training

TSA requires screeners to complete a minimum of 40 hours of classroom instruction and 60 hours of OJT prior to making independent screening decisions. This requirement is an increase over FAA’s basic training requirements when it oversaw passenger screening, which called for 12 hours of classroom instruction and 40 hours of OJT. According to TSA officials, all screeners who work independently have met the basic screener training requirements. TSA contractors are responsible for delivering and tracking basic screener classroom training, while OJT is tracked locally at each airport. TSA encourages, but does not require, screening managers, who are responsible for overseeing screening functions to participate in classroom training, even if they do not have prior screening experience. Nevertheless, 2 of the 12 FSDs we interviewed said that they require their screening managers to observe basic screener training.

Remedial Training

Consistent with ATSA, TSA requires remedial training for any screener who fails an operational test and prohibits screeners from performing the screening function related to the test they failed until they successfully complete the training. FSDs must certify that screeners identified as requiring remedial training complete the training before they can perform the screening function identified as a performance weakness. TSA’s Aviation Operations Division is responsible for tracking the completion of remedial training following the failure of covert tests. The tracking of remedial training initiated for reasons other than failing a covert test is the responsibility of the FSDs or their designees. TSA reported that all screeners requiring remedial training have received the training.

Recurrent Training

TSA has not fully developed or deployed a recurrent training program, but has recognized that ongoing training of screeners on a frequent basis is critical to maintaining and enhancing screener skills. According to agency officials, TSA established a training task force comprised of airport Training Coordinators, screeners, and headquarters officials to conduct an assessment of training needs. As a result of the task force’s suggestions, TSA is developing six recurrent training modules—the first of which TSA plans to deploy to all airports beginning in October 2003. TSA plans to release each of the remaining five modules as they are finalized, which they anticipate will occur throughout 2004. TSA officials also said that they designed and are currently pilot testing an On-Line Learning Management System (LMS) comprised of 366 various training courses, which they expect to field in October 2003. Officials said that they were not further along in implementing their recurrent training modules or LMS due to budget considerations.

Fourteen of the 22 passenger screeners and supervisors we interviewed expressed the need for recurrent training. They were particularly interested in receiving additional training related to recognizing x-ray images of threat objects. In addition, 10 of the 12 FSDs we interviewed reported implementing their own locally developed recurrent training courses rather than waiting for the training modules to be deployed by headquarters. TSA’s OIAPR found that screeners at airports that conducted frequent, on-going training performed better during covert tests—TSA’s form of operational testing—than screeners who did not receive recurrent training.

Supervisory Training

TSA describes its screening supervisors as the key to a strong defense in detecting threat objects. In September 2001, we reported on the results of our survey of aviation stakeholders and aviation and terrorism experts concerning options for conducting screening. The respondents identified better supervision as one of the factors necessary for improving screener performance. Additionally, DOT’s Office Inspector General (OIG) recently reported that screener supervisors are the key to effective screening, and TSA’s OIAPR identified a lack of supervisory training as a cause for screener testing failures. FSDs and TSA headquarters officials recognize the need to enhance the skills of screening supervisors through supervisory training.TSA is currently working with USDA to tailor its off- the-shelf supervisory course to the specific needs of TSA’s screening supervisors. TSA recently reported that it is sending supervisors to the basic USDA supervisor’s course until the customized course is fielded, which it expects to occur in April 2004. To supplement the classroom training, TSA also plans to establish a supervisory training module for recurrent training. We plan to review TSA’s training initiatives further during the remainder of our evaluation.

Little Information Exists to Measure Screeners’ Performance in Detecting Threat Objects

Currently, the results of TSA’s OIAPR’s operational, or covert, testing is the only indication of screener performance in detecting threat objects. However, TSA does not view the results of OIAPR’s covert testing as a measure of screener performance, but rather as a “snapshot” of a screener’s ability to detect threat objects at a particular point in time. Although OAIPR conducts fewer covert tests of passenger screeners than previously conducted by the FAA, TSA considers its tests more rigorous than FAA’s tests because they more closely approximate techniques terrorists might use. In addition to conducting operational testing, TSA plans to fully activate the Threat Image Projection system and implement a screener certification program in October 2003 to collect additional information on screener performance. TSA also developed a Performance Management Information System to collect and maintain information on the performance of its passenger and baggage screening operations. However, PMIS contains little data on screener performance in detecting threat objects. TSA officials said that they plan to expand PMIS to collect some performance information, but did not identify a timeframe for when the data will be collected.

Operational Testing

TSA defines an operational screening test as any covert test of a screener, conducted by TSA, on any screener function to assess the screener’s threat item detection ability and/or adherence to TSA-approved procedures. When a screener fails a test, he or she is required to receive immediate remedial training, and is prohibited from performing the function related to the failed test until he or she satisfactory completes the training. Currently, OIAPR’s covert testing is the only source of operational testing conducted of passenger screeners. These tests are designed to identify systematic problems affecting the performance of screeners in the areas of training, policy, and equipment. TSA does not view the results of OIAPR’s covert testing as a measure of screener performance, but rather a “snapshot” of a screener’s ability to detect threat objects at a particular point in time and as an indicator of systemwide screener performance. OIAPR testing to date has shown weaknesses in screeners’ ability to detect threat objects. Testing conducted by the DOT’s OIG, the Department of Homeland Security’s OIG, and GAO have also identified screener performance weaknesses.

Prior to the creation of TSA, FAA conducted thousands of covert tests annually of passenger screeners. Most of these tests were compliance tests in which FAA agents attempted to get nine test objects, such as guns and grenades, past screeners conducting x-ray, metal detector, and physical searches at airport checkpoints. The DOT OIG described these tests as unlike the techniques that terrorists would employ. In 1997, FAA incorporated simulated improvised explosive devices into its compliance testing and performed, on average, more than 2,000 of these test each year. In addition to compliance tests, FAA’s special headquarters based testing unit, often called the Red Team, conducted more realistic tests using harder to detect threat objects by agents not known to screeners.

TSA’s OIAPR has conducted fewer covert tests than conducted by FAA, but considers its testing methods more rigorous than either of FAA’s compliance or Red Team tests because they more closely approximate techniques terrorists might use. OIAPR officials further said that their tests are intentionally designed to have a high probability of failure in an effort to identify vulnerabilities and areas needing improvement. Additionally, unlike testing conducted under FAA, OIAPR staff that perform the tests reported that they provide immediate feedback to screeners, their managers, and the FSDs to explain how they beat the system and provide instant remedial training. We plan to review OIAPR’s operational testing in more detail during the remainder of our evaluation.

Based on an anticipated increase in staff from about 100 in fiscal year 2003 to 200 in fiscal year 2004, OIAPR plans to conduct twice as many covert tests next year. In addition, TSA recently established 5 mission support centers located throughout the country, which according to TSA, will be staffed with OIAPR personnel available to conduct additional covert tests. These centers will also be staffed with mobile testing teams that will work with FSDs in their region to conduct screener training using some of the test objects OIAPR uses in its covert tests.

Threat Image Projection (TIP) System

In late 1999, to help screeners remain alert, train them to become more adept at detecting harder to spot threat objects, and continuously measure screener performance, FAA began deploying TIP. TIP places images of threat objects on x-ray screens during actual operations and records whether screeners identify the threat object. By frequently exposing screeners to a variety of images of dangerous objects on the x-ray screens, the system provides continuous OJT and allows for immediate supervisory feedback, on-the-spot training, and remedial training.

According to TSA officials, TIP was shut down immediately following the September 11th terrorist attacks due to concerns that it would result in screening delays and panic, as screeners might think that they were actually viewing a threat object. TSA officials recognize that TIP is a key tool in maintaining and enhancing screener performance, and said that they had begun reactivating TIP with significantly more images than FAA had in place. TSA officials said that TIP had not been reactivated sooner due to a lack of automated data collection via cellular modems; competing priorities; a lack of training; and a lack of resources needed to deploy TIP activation teams.

Annual Screener Certification

ATSA requires that each passenger screener receive an annual proficiency review to ensure he or she continues to meet all qualifications and standards required to perform the screening function. Although TSA has not yet implemented this requirement, it plans to develop an annual screener certification program comprised of three components, including (1) image recognition test; (2) knowledge of standard operating procedures (SOPs); and (3) practical demonstration of skills, to be administered by a contractor. TSA has not yet determined the level of performance that screeners must achieve to be certified, but officials said that they plan to require performance at a high, but reasonable level. Officials also said that they plan to remediate and retest screeners who fail any portion of the test, but have not yet determined the number of times a screener may retake the test before termination. Certification is scheduled to begin in October 2003 and to be completed at all 442 airports by January 2004, in the order in which the airports began federal screening operations. TSA officials recently reported that they awarded a contract to conduct the practical demonstration component of the test; however, TSA has not developed a schedule for when the program will be fielded to the airports. We plan to review TSA’s annual screener certification program during the remainder of our evaluation.

Performance Management Information System

TSA’s Performance Management Information System—PMIS—for passenger and baggage screening operations contains little data on screener performance in detecting threat objects. PMIS collects information on workload, staffing, and equipment and is used to identify some performance and policy issues, such as the level of absenteeism, average time for equipment repairs, and status of TSA’s efforts to meets goals for 100 percent baggage screening. (See app. I for examples of information collected and contained in PMIS.) Additionally, TSA uses PMIS data to identify needed changes to SOPs. Officials further reported that PMIS has the ability to generate reports that enable TSA to track its progress toward meeting its performance goals as well as to generate reports by region, FSD, airport, and/or individual screening checkpoint. PMIS has been deployed to all airports with federal screeners. FSDs are responsible for designating a staff person to enter performance data into PMIS on a daily basis.

TSA officials reported that they are planning to integrate performance information from various systems into PMIS to assist the agency in making strategic decisions. TSA also recently reported that it is developing a screener performance index, which is supposed to include information such as the results of TIP tests, training tests, and certification tests. We plan to review these plans in more detail during the remainder of our evaluation.

Screener Performance Improvement Study

TSA is taking steps to improve screener performance. In July 2003, TSA completed a Screener Performance Improvement Study, which was designed to identify root causes for gaps between current screener performance and TSA’s desired performance—defined as 100 percent interception of prohibited items coming through the passenger screening checkpoints. As part of its study, TSA identified four significant screener performance deficiencies. TSA concluded that four key factors contributed to the identified deficiencies: (1) lack of skills, knowledge, or information; (2) low motivation; (3) ineffective work environment; and (4) incorrect or missing incentives. To address the screener performance deficiencies identified in the study, TSA developed several key solutions, including the need to establish adequate training facilities at airports; staff airports adequately to allow time for training; reconfigure checkpoints to eliminate distractions; implement TIP at all airports; and enhance supervisory skills. According to TSA officials, the appropriate TSA components are currently developing action plans for each of the deficiencies identified in the Performance Improvement Study. The plans are to include action steps, timelines, required resources, and anticipated outcomes. We plan to review these plans during the remainder of our evaluation.

An Assessment of the Contract Screening Pilot Program Has Not Yet Begun

TSA has implemented a pilot program using contract screeners at 5 airports, but has not determined how to evaluate and measure the performance of the pilot program airports. The purpose of the 2-year pilot program is to determine the feasibility of using private screening companies rather than federal screeners. Initially, TSA required private screening companies to adhere to all of the procedures and protocols used for federal screeners. However, TSA recently provided the pilot contractors with some flexibility, such as allowing them to determine and maintain their own staffing levels and make independent hiring decisions. While TSA has not yet determined how to evaluate and measure the performance of the pilot program airports, it plans to award a contract by October 1, 2003, to compare the performance of pilot screeners with federal screeners and determine the reasons for any differences. TSA officials said that the Office of Management and Budget requested that they include in their evaluation ways to allow more innovation by contract screening companies.

Although ATSA allows airports to apply to opt-out of using federal screeners beginning in November 2004, TSA has not begun to plan for the possible transition of airports from a federal system to a private screening company. Airports Council International officials said that numerous airports have contacted them expressing an interest in obtaining more information to assist in their decision regarding opting-out. Six of the 10 airport operators we interviewed said that they had not made any decisions regarding opting-out, and all 10 said they had not received any information from TSA regarding the option. However, the airport operators said that they would like information to assist them in deciding whether to opt-out, such as determining who bears responsibility for funding the screening contract; airport liability in the event of an incident linked to a screener failure; how well the current pilot program airports are performing; performance standards to which contract screeners will be held; and TSA’s role in overseeing contracted screening.

TSA Continuing to Work to Identify Appropriate Staffing Levels at the Nation’s Airports

Initially, TSA headquarters determined screener-staffing levels for all airports without actively seeking input from FSDs. Eight of the 12 FSDs we interviewed said that they had limited authority to respond to airport specific staffing needs, such as reacting to fluctuations in daily and/or seasonal passenger flow. However, TSA headquarters officials said that during the second stage of their workforce reduction process, they solicited input from FSDs, airport officials, and air carriers. TSA headquarters officials acknowledged that their initial staffing efforts created imbalances in the screener workforce and have taken steps to correct identified imbalances, such as such as authorizing the hiring of part-time screeners at over 200 airports—the first of which began working on September 15, 2003.

TSA determined the current screener staffing levels using a computer- based modeling process that took into account the number of screening checkpoints and lanes at an airport; originating passengers; the number of airport workers requiring screening; projected air carrier service increases and decreases during calendar year 2003; and hours needed to accommodate screener training, leave, and breaks. TSA recently hired an outside consultant to conduct a study of screener staffing levels at various airports. TSA officials stated that they will continue to review the staffing allocation provided through the modeling efforts to assess air carrier and airport growth patterns, and adjustments will be made as appropriate. We plan to review TSA’s efforts to determine appropriate staffing levels for passenger screeners during the remainder of our evaluation.

As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 5 days after its issue date. At that time, we will send copies of this report to the Secretary of the Department of Homeland Security and interested congressional committees. We will also make copies available to others upon request. In addition, the report will be available at no charge on GAO’s Web site at http://www.gao.gov.

If you have any questions about this report, or wish to discuss it further, please contact me at (202) 512-8777 or Jack Schulze, Assistant Director, at (202) 512-4390. Key contributors to this report are listed in appendix II.

Appendix I: Examples of Information Maintained in TSA’s Performance Management Information System

Number of weapons surrendered at sweep screening Number of cleared Explosive Trace Detection (ETD) alarms Lost, stolen, or damaged items Number of Screeners on light duty Number of screeners trained on baggage only/passenger only/cross- trained Percent of airports using the CAPPS II system Average wait time at passenger screening checkpoints for federalized airports Number of gates in use Number of ETS, x-ray machines, explosive detection systems (EDS) machines Number of enplanements Number of screeners on duty Individual airport measures to achieve change in threat level by date and time TSA officials described sweep screening as a method of screening in which screeners randomly stop passengers in the airport concourse for additional screening.

Appendix II: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to those named above, David Alexander, Lisa Brown, Christopher Jones, Stuart Kaufmann, Thomas Lombardi, Jan Montgomery, Edward Stephenson, Maria Strudwick, and Susan Zimmerman were key contributors to this report.