Ensuring the security of our nation’s commercial aviation system has been a long-standing concern. As demonstrated by the 1988 bombing of a U.S. airliner over Lockerbie, Scotland, and the 1995 plot to blow up as many as 12 U.S. aircraft in the Pacific region discovered by Philippine authorities, U.S. aircraft have long been a target for terrorist attacks. Many efforts have been made to improve aviation security, but as we and others have documented in numerous reports and studies, weaknesses in the system continue to exist. It was these weaknesses that terrorist exploited to hijack four commercial aircraft in September 2001, with tragic results. On November 19, 2001, the President signed into law the Aviation and Transportation Security Act, with the primary goal of strengthening the security of the nation’s aviation system. ATSA created TSA as an agency within the Department of Transportation with responsibility for securing all modes of transportation, including aviation. ATSA mandated specific improvements to aviation security and established deadlines for completing many of them. TSA’s main focus during its first year of operation was on meeting these ambitious deadlines, particularly federalizing the screener workforce at commercial airports nationwide by November 19, 2002, while at the same time establishing a new federal organization from the ground up. The Homeland Security Act, signed into law on November 25, 2002, transferred TSA from the Department of Transportation to the new Department of Homeland Security. Virtually all aviation security responsibilities now reside with TSA, including the screening of air passengers and baggage, a function that had previously been the responsibility of air carriers. TSA is also responsible for ensuring the security of air cargo and overseeing security measures at airports to limit access to restricted areas, secure airport perimeters, and conduct background checks for airport personnel with access to secure areas, among other responsibilities. TSA has implemented numerous initiatives designed to enhance aviation security but has collected little information on the effectiveness of these initiatives. ATSA requires that TSA establish acceptable levels of performance and develop annual performance plans and reports to measure and document the effectiveness of its security initiatives.Although TSA has developed these performance tools, as required by ATSA, it currently focuses on progress toward meeting ATSA deadlines, rather than on the effectiveness of its programs and initiatives. However, TSA is taking steps to collect objective data to assess its performance. TSA currently has limited information on the effectiveness of its aviation security initiatives. As we reported in September 2003, the primary source of information collected on screeners’ ability to detect threat objects is the covert testing conducted by TSA’s Office of Internal Affairs and Program Review. However, TSA does not consider the results of these covert tests to be a measure of performance but rather a “snapshot” of a screener’s ability to detect threat objects at a particular point in time, and as a system-wide performance indicator. At the time we issued our report, the Office of Internal Affairs and Program Review had conducted 733 covert tests of passenger screeners at 92 airports. Therefore, only about 1 percent of TSA’s nearly 50,000 screeners had been subject to a covert test. In addition to conducting covert tests at screening checkpoints, TSA conducts tests to determine whether the current Computer-Assisted Passenger Screening System is working as designed, threat objects are detected during the screening of checked baggage, and access to restricted areas of the airport is limited only to authorized personnel. While the Office of Internal Affairs has conducted about 2,000 access tests, it has conducted only 168 Computer-Assisted Passenger Screening System and checked baggage tests. Based on an anticipated increase in staff from about 100 in fiscal year 2003 to 200 in fiscal year 2004, the Office of Internal Affairs and Program Review plans to conduct twice as many covert tests next year. Another key source of data on screener performance in detecting threat objects is the Threat Image Projection (TIP) system, which places images of threat objects on the X-ray screen during actual operations and records whether screeners identify the threat object. The Federal Aviation Administration began deploying TIP in late 1999 to continuously measure screener performance and to train screeners in becoming more adept at detecting hard-to-spot threat objects. However, TIP was shut down immediately following the September 11 terrorist attacks because of concerns that it would result in screening delays and panic, as screeners might think that they were actually viewing a threat object. Although TSA officials recognized that TIP is a key tool in measuring, maintaining, and enhancing screener performance, they only recently began reactivating TIP on wide-scale basis because of competing priorities, a lack of training, and a lack of resources needed to deploy TIP activation teams. Once TIP is fully deployed and operational at every checkpoint at all airports, as it is expected to be in April 2004, TSA headquarters and federal security directors will have the capability to analyze this performance data in a number of ways, including by individual screeners, checkpoints, terminals, and airports. When fully deployed, the annual screener recertification test results will provide another source of data on screener performance. ATSA requires that TSA collect performance information on each screener through conducting an annual proficiency review to ensure he or she continues to meet all qualifications and standards required to perform the screening function. Although TSA began deploying federal screeners to airports in April 2002, TSA only recently began implementing the annual recertification program and does not expect to complete testing at all airports until March 2004. The recertification testing is comprised of three components: (1) image recognition; (2) knowledge of standard operating procedures; and (3) practical demonstration of skills, to be administered by a contractor. TSA officials consider about 28,000 screeners as having already completed the first two components because they successfully passed competency tests TSA administered at many airports as part of a screener workforce reduction effort. However, these competency tests did not include the third component of TSA’s planned annual screener recertification program—the practical demonstration of skills. TSA officials awarded a contract for this component of the annual proficiency reviews in September 2003. TSA’s Performance Management Information System for passenger and baggage screening operations is designed to collect performance data, but it currently contains little information on screener performance in detecting threat objects. The Performance Management Information System collects a wide variety of metrics on workload, staffing, and equipment and is used to identify some performance indicators, such as the level of absenteeism, the average time for equipment repairs, and the status of TSA’s efforts to meet goals for 100 percent electronic baggage screening. However, the system does not contain any performance metrics related to the effectiveness of passenger screeners. TSA is planning to integrate performance information from various systems into the Performance Management Information System to assist the agency in making strategic decisions. TSA further plans to continually enhance the system as it learns what data are needed to best manage the agency. In addition to making improvements to the Performance Management Information System, TSA is currently developing performance indexes for both individual screeners and the screening system as a whole. The screener performance index will be based on data such as the results of performance evaluations and recertification tests, and the index for the screening system will be based on information such as covert test results and screener effectiveness measures. TSA has not yet fully established its methodology for developing the indexes, but it expects to have the indexes developed by the end of fiscal year 2004. In conjunction with measuring the performance of its passenger screening operations, TSA must also assess the performance of the five pilot airports that are currently using contract screeners to determine the feasibility of using private screening companies instead of federal screeners. Although ATSA allows airports to apply to opt out of using federal screeners beginning in November 2004, TSA has not yet determined how to evaluate and measure the performance of the pilot program. In early October 2003, TSA awarded a contract to BearingPoint, Inc., to compare the performance of pilot screening with federal screening, including the overall strengths and weaknesses of both systems, and determine the reasons for any differences. The evaluation is scheduled to be completed by March 31, 2004. TSA has acknowledged that designing an effective evaluation of the screeners at the pilot airports will be challenging because key operational areas, including training, assessment, compensation, and equipment, have to a large extent been held constant across all airports, and therefore are not within the control of the private screening companies. In its request for proposal for the pilot airport evaluation, TSA identified several data sources for the evaluation, including the Performance Management Information System and the Office of Internal Affairs and Program Review’s covert testing of passenger screeners. However, as we recently reported, data from both of these systems in measuring the effectiveness of screening operations is limited. As a result, it will be a challenge for TSA to effectively compare the performance of the contract pilot airports with the performance of airports using federal screeners. TSA has recognized the need to strengthen the assessment of its performance, and has initiated efforts to develop and implement strategic and performance plans to clarify goals, establish performance measures, and measure the performance of its security initiatives. Strategic plans are the starting point for an agency’s planning and performance measurement efforts. Strategic plans include a comprehensive mission statement based on the agency’s statutory requirements, a set of outcome-related strategic goals, and a description of how the agency intends to achieve these goals. The Government Performance and Results Act (GPRA) establishes a framework for strategic plans that requires agencies to clearly establish results-oriented performance goals in strategic and annual performance plans for which they will be held accountable, measure progress toward achieving those goals, determine the strategies and resources to effectively accomplish the use performance information to make programmatic decisions necessary to improve performance, and formally communicate results in performance reports.