Aviation Security: 9/11 Anniversary Observations on TSA's Progress and Challenges in Strengthening Aviation Security
Released: 2012-09-11T00:00:00

Letter

Background

The Aviation and Transportation Security Act established TSA as the federal agency with primary responsibility for securing the nation’s civil aviation system, which includes the screening of all passenger and property transported by commercial passenger aircraft. At the more than 450 TSA-regulated airports in the United States, prior to boarding an aircraft, all passengers, their accessible property, and their checked baggage are screened pursuant to TSA-established procedures. TSA relies upon multiple layers of security to deter, detect, and disrupt persons posing a potential risk to aviation security. These layers include behavior detection officers (BDOs), who examine passenger behaviors and appearances to identify passengers who might pose a potential security risk at TSA-regulated airports; travel document checkers, who examine tickets, passports, and other forms of identification; transportation security officers (TSO), who are responsible for screening passengers and their carry-on baggage at passenger checkpoints using x-ray equipment, magnetometers, Advanced Imaging Technology, and other devices, as well as for screening checked baggage; random employee screening; and checked baggage screening systems. The Implementing Recommendations of 9/11 Commission Act of 2007 further mandates that the Secretary of Homeland Security establish a system to screen 100 percent of cargo transported on passenger aircraft, and defines screening for purposes of meeting this mandate, in general, as a physical examination or the use of nonintrusive methods to assess whether cargo poses a threat to transportation security. Such cargo ranges in size from 1 pound to several tons and ranges in type from perishable commodities to machinery. In 2011, all-cargo carriers transported approximately 66 percent (6.9 billion pounds) of the total cargo (10.4 billion pounds) transported to the United States. Additionally, TSA has responsibilities for general aviation security, and developed the Alien Flight Student Program (AFSP) to help determine whether foreign students enrolling at flight schools pose a security threat. U.S. government threat assessments have discussed plans by terrorists to use general aviation aircraft to conduct attacks. Further, analysis conducted on behalf of TSA has indicated that larger general aviation aircraft may be able to cause significant damage to buildings and other structures.

TSA Has Taken Actions to Validate the Science Underlying Its Behavior Detection Program, but More Work Remains

We reported in May 2010 that TSA deployed SPOT nationwide before first determining whether there was a scientifically valid basis for using behavior and appearance indicators as a means for reliably identifying passengers who may pose a risk to the U.S. aviation system.

According to TSA, SPOT was deployed before a scientific validation of the program was completed to help address potential threats to the aviation system, such as those posed by suicide bombers. TSA also stated that the program was based upon scientific research available at the time regarding human behaviors. We reported in May 2010 that approximately 14,000 passengers were referred to law enforcement officers under SPOT from May 2004 through August 2008. Of these passengers, 1,083 were arrested for various reasons, including being illegal aliens (39 percent), having outstanding warrants (19 percent), and possessing fraudulent documents (15 percent). The remaining 27 percent were arrested for other reasons such as intoxication, unruly behavior, theft, domestic violence, and possession of prohibited items. As noted in our May 2010 report, SPOT officials told us that it is not known if the SPOT program has ever resulted in the arrest of anyone who is a terrorist, or who was planning to engage in terrorist-related activity. More recent TSA data covering the period from November 1, 2010, to April 18, 2012, indicates that SPOT referred 60,717 passengers for additional screening, which resulted in 3,803 referrals to law enforcement officers and 353 arrests. Of these 353 arrests, 23 percent were related to immigration status, 23 percent were drug-related, 9 percent were related to fraudulent documents, 22 percent were related to outstanding warrants, and 28 percent were for other offenses.

A 2008 report issued by the National Research Council of the National Academy of Sciences stated that the scientific evidence for behavioral monitoring is preliminary in nature. The report also noted that an information-based program, such as a behavior detection program, should first determine if a scientific foundation exists and use scientifically valid criteria to evaluate its effectiveness before deployment. The report added that such programs should have a sound experimental basis and that the documentation on the program’s effectiveness should be reviewed by an independent entity capable of evaluating the supporting scientific evidence.

As we reported in May 2010, an independent panel of experts could help DHS determine if the SPOT program is based on valid scientific principles that can be effectively applied in an airport environment for counterterrorism purposes. Thus, we recommended that the Secretary of Homeland Security convene an independent panel of experts to review the methodology of DHS’s Science and Technology Directorate’s ongoing validation study on the SPOT program being conducted to determine whether the study’s methodology is sufficiently comprehensive to validate the SPOT program. We also recommended that this assessment include appropriate input from other federal agencies with expertise in behavior detection and relevant subject matter experts. stated that its validation study, completed in April 2011, included input from a broad range of federal agencies and relevant experts, including those from academia. DHS’s validation study found that SPOT was more effective than random screening to varying degrees. For example, the study found that SPOT was more effective than random screening at identifying individuals who possessed fraudulent documents and identifying individuals who law enforcement officers ultimately arrested.

See GAO-10-763.

According to DHS’s study, no other counterterrorism or screening program incorporating behavior and appearance-based indicators is known to have been subjected to such a rigorous, systematic evaluation of its screening accuracy. However, DHS noted that the identification of high-risk passengers was rare in both the SPOT and random tests. DHS’s study also noted that the assessment was an initial validation step, and was not designed to fully validate whether behavior detection can be used to reliably identify individuals in an airport environment who pose a security risk. According to DHS, further research will be needed to comprehensively validate the program.

In addition, DHS determined that the base rate, or frequency, of SPOT behavioral indicators observed by TSA to detect suspicious passengers was very low and that these observed indicators were highly varied across the traveling public. Although details about DHS’s findings related to these indicators are sensitive security information, the low base rate and high variability of traveler behaviors highlights the challenge that TSA faces in effectively implementing a standardized list of SPOT behavioral indicators. In addition, DHS outlined several limitations to the study. For example, the study noted that BDOs were aware of whether individuals they were screening were selected as the result of identified SPOT indicators or random selection. DHS stated that this had the potential to introduce bias into the assessment. DHS also noted that SPOT data from January 2006 through October 2010 were used in its analysis of behavioral indicators even though questions about the reliability of the data exist.

The study made recommendations related to SPOT in three areas: (1) future validation efforts, (2) comparing SPOT with other screening programs, and (3) broader program evaluation issues. TSA designated the specific details of these recommendations sensitive security information. program evaluation including a cost-benefit analysis, reiterate recommendations made in our prior work. In March 2011, we reported that Congress may wish to consider the study’s results in making future funding decisions regarding the program. TSA is reviewing the study’s findings and assessing the steps needed to address DHS’s recommendations. If TSA decides to implement the recommendations in the April 2011 DHS validation study, it may be years away from knowing whether there is a scientifically valid basis for using behavior detection techniques to help secure the aviation system against terrorist threats given that the initial study took about 4 years to complete. We are conducting a follow-on review of TSA’s behavior detection program, and its related variant, the so-called “Assessor Program”, which incorporates more extensive verbal interactions (“chat downs”) with the traveling public. The Assessor program is currently being test piloted in Boston and Detroit. Our follow-on report on this program will be issued early next year.

DHS and TSA Have Taken Actions to Enhance the Security of Cargo on Inbound Aircraft, but Challenges Remain

DHS and TSA have taken four primary actions to enhance the security of inbound cargo on passenger and all-cargo aircraft following the October 2010 bomb attempt originating in Yemen.

TSA issued new screening requirements aimed at enhancing the security of cargo on passenger and all-cargo aircraft. Beginning in October 2010, TSA imposed new risk-based security procedures on passenger and all-cargo aircraft aimed at focusing more detailed screening measures on high risk shipments and, among other things, prohibited the transport of cargo on passenger aircraft from Yemen and Somalia due to threats stemming from those areas.

DHS instituted working groups with air cargo industry stakeholders to identify ways to enhance air cargo security. In January 2011, the Secretary of Homeland Security established an Air Cargo Security Working Group to obtain advice and consultations from air cargo security stakeholders on ways to enhance the security of the air cargo system.The Air Cargo Security Working Group briefed the Secretary of Homeland Security, the Commissioner of U.S. Customs and Border Protection (CBP), and the TSA Administrator in April 2011 on proposed solutions, and recommended that TSA reevaluate the agency’s implementation plan, timeline, and resources related to TSA’s program to recognize the security programs of foreign countries, known as the National Cargo Security Program (NCSP). According to TSA officials, participants of this working group have reconvened as part of the Aviation Security Advisory Committee, which held its first meeting in May 2012, and the committee will meet again in mid-September 2012 to discuss the implementation of the recommendations.

All-cargo express carriers and companies focus on transporting cargo under quick time frames. up” and en route directly to the United States) from locations in North America. Under the pilot program, however, participants provide manifest data prior to loading cargo aboard aircraft.

TSA developed a program to recognize foreign air cargo security programs. TSA has developed the NCSP recognition program to review and recognize the air cargo security programs of foreign countries if TSA deems those programs as providing a level of security commensurate with TSA’s air cargo security standards. In May 2012, TSA recognized Canada as providing a level of security commensurate with U.S. air cargo security standards, and in June 2012, the agency recognized the European Union and Switzerland as also providing this same level of TSA officials security based on the principle of “mutual recognition.” stated that the NCSP recognition program is a key effort in meeting the 100 percent screening mandate because it will eliminate the need for air carriers to comply with two countries’ security programs.

Despite these actions, air carriers and TSA face three key challenges that, among other things, could limit TSA’s ability to meet the 9/11 Commission Act mandate to screen 100 percent of cargo transported on passenger aircraft as it applies to inbound air cargo and to provide reasonable assurance that screening is being conducted at reported levels. All-cargo carriers subject to TSA regulation also reported facing challenges in implementing new TSA screening requirements established after the October 2010 Yemen incident.

TSA had previously recognized France and the United Kingdom as providing a level of security commensurate with U.S. air cargo security standards. screen a certain percentage of all cargo. TSA proposed changes that would require passenger air carriers to screen 100 percent of cargo as part of its efforts to meet the 9/11 Commission Act mandate. Passenger air carriers expressed concerns about being able to meet the 100 percent screening mandate as it applies to inbound cargo stating that it would cause significant disruptions in the air cargo supply chain, among other issues. In response to these concerns, TSA officials stated that they revised the proposed requirements and issued new passenger security requirements in June 2012. Agency officials said they plan to require air carriers to screen 100 percent of inbound air cargo transported on passenger aircraft by December 3, 2012.

GAO-10-446. data reported by air carriers, the agency has not yet fully met the intent of the recommendation. It will be important for TSA to continue to work towards ensuring verification of inbound air cargo screening data submitted by air carriers and that inbound air cargo is screened in accordance with the mandate.

Reporting screening data could facilitate oversight of all-cargo carrier compliance requirements. TSA relies on data submitted by passenger carriers to determine the amount of air cargo screened on inbound passenger aircraft but there is no requirement for all-cargo carriers to report comparable screening data to TSA, even though most of the cargo shipped from abroad into the United States is shipped on all- cargo carriers. Thus, TSA does not know the extent to which all-cargo carriers are screening cargo or meeting the enhanced screening requirements introduced after the October 2010 incident in Yemen. Officials from two global all-cargo carriers said that submitting such information to TSA would be feasible because they are already collecting this data internally, but officials from two other all-cargo carriers stated that reporting screening data to TSA would be challenging because of staffing limitations or because such data may not be available. TSA officials said that TSA does not require that all-cargo carriers submit screening data because it has focused its efforts on collecting data from passenger air carriers in support of meeting the 100 percent mandate. TSA officials stated that TSA may consider opportunities to capture additional inbound air cargo information, but has not yet weighed the costs and benefits of doing so because it has focused its efforts on establishing the ACAS pilot program, which DHS established to more readily indentify high-risk cargo. The pilot program is a key effort to identify high-risk cargo prior to aircraft departing from foreign airports, but is not intended to provide TSA with screening data, which if collected and verified, could provide additional assurance that all-cargo carriers are complying with TSA’s enhanced screening requirements. To help TSA better determine what actions are needed, if any, to ensure that all-cargo carriers are complying with the agency’s enhanced screening requirements, we recommended in May 2012 that DHS assess the costs and benefits of requiring all-cargo carriers to report data on screening conducted.actions to address it.

DHS and TSA Have Experienced Difficulties Establishing Acquisition Program Baselines, Schedules, and Cost Estimates for Checked Baggage Screening Systems

TSA’s Electronic Baggage Screening Program (EBSP) reports that 76 percent of the airports (337 of 446) the agency regulates for security have a mix of in-line and stand-alone baggage screening configurations that best meet airport needs (i.e., optimal systems). Our prior work on TSA’s checked baggage screening program—EBSP—identified a number of shortcomings in DHS and TSA’s process for establishing program baselines, program schedules, and cost estimates.

Acquisition program baselines. We found that realistic acquisition program baselines with stable requirements for cost, schedule, and performance are among the factors that are important to successful acquisitions delivering capabilities within cost and schedule. Further, we reported in April 2009 that program performance metrics for cost and schedule can provide useful indicators of the health of acquisition programs and, when assessed regularly for changes and the reasons that cause changes, such indicators can be valuable tools for improving insight and oversight of individual programs as well as the total portfolio of major acquisitions.program baseline is the contract between the program and departmental oversight officials and must be established at program start to document the program’s expected cost, deployment schedule, and technical performance. By tracking and measuring actual program performance against this baseline, management can be alerted to potential problems, such as cost growth or changing requirements, and has the ability to take corrective action.

According to DHS’s acquisition guidance, the We reported in April 2012 that TSA has not had a DHS-approved acquisition program baseline for EBSP since the program’s inception more than 8 years ago. Further, DHS did not require TSA to complete an acquisition program baseline until November 2008. TSA officials said they have twice submitted an acquisition program baseline to DHS for approval—first in November 2009 and again February 2011. However, according to DHS officials TSA did not have a fully developed life cycle cost estimate. In November 2011, DHS told TSA that it needed to revise the life cycle cost estimates as well as its procurement and deployment schedules to reflect budget constraints. DHS officials told us that they could not approve the acquisition program baseline as written because TSA’s estimates were significantly over budget. An approved baseline will provide DHS with additional assurances that TSA’s approach is appropriate and that the capabilities being pursued are worth the expected costs. TSA officials stated that TSA is working with DHS to amend the draft program baseline and plans to resubmit a revised life cycle cost estimates with a revised acquisition program baseline by December 31, 2012. As we reported, establishing and approving a program baseline, as DHS and TSA plan to do for the EBSP, could help DHS assess the program’s progress in meeting its goals and achieve better program outcomes.

Schedules. In July 2011, we reported that TSA had established a schedule for the acquisition of the explosives detection systems (EDS) TSA deploys to screen checked baggage, but it did not fully comply with leading practices, and TSA had not developed a plan to upgrade its EDS fleet to meet the 2010 explosives detection requirements. We noted that some of TSA’s approximately 2,200 deployed systems met 2005 explosive requirements while the remainder met 1998 explosive detection requirements.system acquisition, such as TSA’s EDS acquisition, depends in part on having a reliable schedule that identifies when the program’s set of work activities and milestone events will occur, amongst other things. We reported that the schedule for the EDS acquisition is not reliable because it does not include a timeline to deploy EDS or plans to procure EDS to meet subsequent phases of explosive detection requirements. We stated that developing a reliable schedule would help TSA better monitor and oversee the progress of the EDS acquisition. DHS concurred with the recommendation to develop and maintain a schedule for the entire EBSP in accordance with the leading practices we identified for preparing a schedule. DHS commented that TSA had already begun working with key stakeholders to develop and define requirements for a schedule and to ensure that the schedule aligns with the leading practices. In April 2012, Leading practices state that the success of a large-scale TSA stated that it had secured contractor resources to support development of an integrated master schedule in accordance with our and industry best practices, and that it anticipated completion of this schedule by September 2013.

Cost estimates. In April 2012, we reported that TSA’s methods for developing life cycle cost estimates for the EBSP did not fully adhere to best practices for developing these estimates. We reported in March 2009 that a high-quality, reliable cost estimation process provides a sound basis for making accurate and well-informed decisions about resource investments, budgets, assessments of progress, and accountability for results and thus is critical to the success of a program. We reported that TSA’s estimates partially met three characteristics and minimally met one characteristic of a reliable cost estimate. DHS concurred with the recommendation that TSA ensure that its life cycle cost estimates conform to cost estimating best practices, and identified efforts underway to address it.

TSA Established a Process for Vetting Foreign Flight Students but Weaknesses Remain

As we reported in July 2012, TSA has worked with industry and other stakeholders to enhance general aviation security, such as issuing regulations and enhancing outreach and awareness, but there are weaknesses in the agency’s process for vetting foreign flight student applicants and in DHS’s process for identifying flight students who may be in the country illegally. We recommended two actions that DHS and TSA could take to address these concerns, with which DHS concurred.

Vetting foreign flight student applicants. Under AFSP, foreign nationals seeking flight training in the United States undergo a TSA security threat assessment before receiving flight training to determine whether each applicant is a security threat to the United States. According to TSA officials, when a foreign national applies to AFSP to obtain flight training, TSA uses information submitted by the foreign national—such as name, date of birth, and passport information—to conduct a criminal history records check, a review of the Terrorist Screening Database, and a review of the Department of Homeland Security’s TECS system. According to TSA officials, most foreign nationals taking training from a U.S. flight training provider will apply for a Federal Aviation Administration (FAA) airman certificate (pilot’s license) once their flight training is completed. Information obtained by FAA as part of this application for certification is placed in the airmen registry. From January 2006 through September 2011, 25,599 foreign nationals had applied for FAA airman certificates, indicating they had completed flight training. However, TSA computerized matching of FAA data determined that some known number of foreign nationals did not match with those in TSA’s database, raising questions as to whether they had been vetted.

Since 2009, TSA has vetted all new and existing FAA airman certificate holders against the Terrorist Screening Database on an ongoing basis, which would include the foreign nationals identified through TSA’s analysis. However, this vetting does not occur until after the foreign national has obtained flight training. Thus, foreign nationals obtaining flight training with the intent to do harm—such as three of the pilots and leaders of the September 11, 2001, terrorist attacks—could have already obtained the training needed to operate an aircraft before they received any type of vetting. We recommended that TSA take steps to identify any instances where foreign nationals receive FAA airman certificates without first undergoing a TSA security threat assessment and examine those instances so that TSA can identify the reasons for these occurrences and strengthen controls to prevent future occurrences. DHS concurred with this recommendation and stated that TSA signed a memorandum of understanding with FAA in February 2012 to help address this issue. The memorandum outlines a process for FAA to provide certain data from its airmen registry on a monthly basis and authorizes TSA to use the data to ensure flight training providers are providing TSA with information to conduct background checks prior to flight instruction. This is an important step toward addressing the first part of our recommendation, provided that TSA uses the data to identify instances where foreign nationals receive FAA airman certificates without first undergoing a TSA security threat assessment, identifies reasons for these occurrences, and strengthens controls to prevent future occurrences, as we recommended.

Identifying flight students entering the country illegally. We also reported that AFSP is not designed to determine whether a foreign flight student entered the country legally; thus, a foreign national can be approved for training through AFSP after entering the country illegally. A March 2010 U.S. Immigration and Customs Enforcement (ICE) investigation of a flight school led to the arrest of six such foreign nationals, including one who had a commercial pilot’s license. As a result, TSA and ICE jointly worked on vetting names of foreign students against immigration databases, but had not specified desired outcomes and time frames, or assigned individuals with responsibility for fully instituting the program as of July 2012. Thus, this weakness still exists today. Having a road map, with steps and time frames, and assigning individuals the responsibility for fully instituting a pilot program could help TSA and ICE better identify and prevent potential risk. We recommended that TSA and ICE develop a plan, with time frames, and assign individuals with responsibility and accountability for assessing the results of their pilot program to check TSA AFSP data against information DHS has on applicants’ admissibility status to help detect and identify violations by foreign flight students, and institute that pilot program if it is found to be effective. DHS concurred and stated that TSA will prepare a plan by December 2012 to assess the results of the pilot program with ICE to determine the lawful status of the active AFSP population. We believe that these are positive actions that could help TSA address the weaknesses identified in our report. We will continue to monitor TSA’s progress on the proposed solutions as the agency proceeds.

Chairman Rogers, Ranking Member Jackson-Lee, and Members of the Subcommittee, this concludes my prepared statement. I look forward to responding to any questions that you may have.

GAO Contact and Staff Acknowledgements

For questions about this statement, please contact Stephen M. Lord at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this testimony are Steve Morris, Assistant Director, and Michelle Woods, Analyst in Charge. Additional contributors include David M. Bruno, Glenn Davis, Jessica Lucas Judy; Assistant Directors; Joel Aldape; Adam Hoffman; Susanna Kuebler; Thomas Lombardi; Lara Miklozek; Linda Miller; Daniel Rodriguez; and Douglas Sloane.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.