Aviation Security: A National Strategy and Other Actions Would Strengthen TSA's Efforts to Secure Commercial Airport Perimeters and Access Controls
Released: 2009-09-30T00:00:00

Letter

Background

Airport Security Roles and Responsibilities

On February 17, 2002, pursuant to ATSA, TSA assumed responsibility for the security of the nation’s civil aviation system from the Federal Aviation Administration (FAA), including FAA’s existing aviation security programs, plans, regulations, orders, and directives covering airports, air carriers, and other related entities. Among other things, ATSA directs TSA to improve the security of airport perimeters and the access controls leading to secured areas, and take measures to reduce the security risks posed by airport workers. (See app. II for more specific details on ATSA requirements and TSA’s actions to address these requirements.) TSA has 158 FSDs who oversee the implementation of, and adherence to, TSA requirements at the approximately 450 commercial airports nationwide. As part of TSA’s oversight role, it also conducts compliance inspections, covert testing, and vulnerability assessments to analyze and improve security. (See app. III for information on how TSA uses compliance inspections and covert testing to identify possible airport security vulnerabilities.)

In general, TSA funds its perimeter and access control security–related activities out of its annual appropriation and in accordance with direction set forth in congressional committee reports. For example, the Explanatory Statement accompanying the DHS Appropriations Act, 2008, directed that TSA allocate $15 million of its appropriation to a worker screening pilot program. TSA does not track the amount of funds spent in total for perimeter and access controls because related efforts and activities can be part of broader security programs that also serve other aspects of aviation security. In addition, airports may receive federal funding for perimeter and access control security, such as through federal grant programs or TSA pilot programs. (For more information on such airport security costs and funding, see app. IV.)

Airport operators have direct responsibility for day-to-day aviation operations, including, in general, the security of airport perimeters, access controls, and workers, as well as for implementing TSA security requirements. Airport operators implement security requirements in accordance with their TSA-approved security programs. Elements of a security program may include, among other things, procedures for performing background checks on airport workers, applicable training programs for these workers, and procedures and measures for controlling access to secured airport areas. Security programs may also be required to describe the secured areas of the airport, including a description and map detailing boundaries and pertinent features of the secured areas, and the measures used to control access to such areas.

Commercial airports are generally divided into designated areas that have varying levels of security, known as secured areas, security identification display areas (SIDA), air operations areas (AOA), and sterile areas. Sterile areas, located within the terminal, are where passengers wait after screening to board departing aircraft. Access to sterile areas is controlled by TSA screeners at security checkpoints, where they conduct physical screening of passengers and their property. Airport workers may access the sterile area through the security checkpoint or through other access points secured by the airport operator in accordance with its security program. The SIDA and the AOA are not to be accessed by passengers, and typically encompass baggage loading areas, areas near terminal buildings, and other areas close to parked aircraft and airport facilities, as illustrated in figure 1.

Securing access to the sterile area from other secured areas—such as the SIDA—and security within the area, is the responsibility of the airport operator, in accordance with its security program. Airport perimeter and access control security is intended to prevent unauthorized access into secured areas—either from outside the airport complex or from within the airport’s sterile area. Individual airport operators determine the boundaries for each of these areas on a case-by-case basis, depending on the physical layout of the airport and in accordance with TSA requirements. As a result, some of these areas may overlap. Within these areas, airport operators are responsible for safeguarding their airfield barriers, preventing and detecting unauthorized entry into secured areas, and conducting background checks of workers with unescorted access to secured areas.

Methods used by airports to control access through perimeters or into secured areas vary because of differences in the design and layout of individual airports, but all access controls must meet minimum performance standards in accordance with TSA requirements. These methods typically involve the use of one or more of the following: pedestrian and vehicle gates, keypad access codes using personal identification numbers, magnetic stripe cards and readers, turnstiles, locks and keys, and security personnel.

According to TSA officials, airport security breaches occur within and around secured areas at domestic airports (see fig. 2 for the number of security breaches reported by TSA from fiscal year 2004 through fiscal year 2008). While some breaches may represent dry runs by terrorists or others to test security or criminal incidents involving airport workers, most are accidental. TSA requires FSDs to report security breaches that occur both at the airports for which they are responsible and on board aircraft destined for their airports. TSA officials said that they review security breach data and report them to senior management as requested, and provide data on serious breaches to senior management on a daily basis, as applicable.

According to a TSA official, the increase in known breaches from fiscal years 2004 through 2005 reflects a change in the requirements for reporting security breaches that TSA issued in December 2005. This change provided more specific instructions to FSDs on how to categorize different types of security incidents. Regarding increases in security breaches from fiscal years 2005 through 2008, TSA officials said that while they could not fully explain these increases, there could be several reasons to account for this growth. For example, according to TSA officials, changes in TSA management often trigger increases in specific types of breaches reported, such as since 2004, when the priorities of the new Administrator resulted in an increase in the reporting of restricted items. TSA officials also stated that a report of a security breach at a major U.S. airport is likely to cause security and law enforcement officials elsewhere to subsequently raise the overall awareness of security requirements for a period of time. In addition, TSA noted that certain inspections conducted by TSA officials tend to produce heightened awareness by federal and airport employees whose perimeter security and access control procedures are being inspected for compliance with regulations.

Risk Management Approach Can Help Guide Homeland Security Efforts

Risk management is a tool for informing policymakers’ decisions about assessing risks, allocating resources, and taking actions under conditions of uncertainty. We have previously reported that a risk management approach can help to prioritize and focus the programs designed to combat terrorism. Risk management, as applied in the transportation security context, can help federal decision makers determine where and how to invest limited resources within and among the various modes of transportation. In accordance with Homeland Security Presidential Directive (HSPD) 7, the Secretary of Homeland Security designated TSA as the sector-specific agency for the transportation security sector, requiring TSA to identify, prioritize, and coordinate the protection of critical infrastructure and key resources within this sector and integrate risk management strategies into its protective activities. In June 2006, in accordance with HSPD-7 and the Homeland Security Act of 2002, DHS released the NIPP, which it later updated in 2009. The NIPP developed a risk management framework for homeland security. In accordance with the NIPP, TSA developed the TS-SSP to govern its strategy for securing the transportation sector, as well as annexes for each mode of transportation, including aviation. The NIPP and TS-SSP set forth risk management principles, including a comprehensive risk assessment process for considering threat, vulnerability, and consequence assessments to determine the likelihood of terrorist attacks and the severity of the impacts. Figure 3 illustrates the interrelated activities of the NIPP’s risk management framework.

Set security goals: Define specific outcomes, conditions, end points, or performance targets that collectively constitute an effective protective posture.

Identify assets, systems, networks, and functions: Develop an inventory of the assets, systems, and networks that constitute the nation’s critical infrastructure, key resources, and critical functions. Collect information pertinent to risk management that takes into account the fundamental characteristics of each sector.

Assess risks: Determine risk by combining potential direct and indirect consequences of a terrorist attack or other hazards (including seasonal changes in consequences and dependencies and interdependencies associated with each identified asset, system, or network), known vulnerabilities to various potential attack vectors, and general or specific threat information.

Prioritize: Aggregate and analyze risk assessment results to develop a comprehensive picture of asset, system, and network risk; establish priorities based on risk; assess the mitigation of risk for each proposed activity based on a specific investment; and determine protection and business continuity initiatives that provide the greatest mitigation of risk.

Implement protective programs: To reduce or manage identified risk, select sector-appropriate protective actions or programs that offer the greatest mitigation of risk for any given resource/expenditure/investment. Secure the resources needed to address priorities.

Measure effectiveness: Use metrics and other evaluation procedures at the national and sector levels to measure progress and assess the effectiveness of the national Critical Infrastructure and Key Resources Protection Program in improving protection, managing risk, and increasing resiliency.

Within the risk management framework, the NIPP also establishes core criteria for risk assessments. According to the NIPP, risk assessments are a qualitative determination, a quantitative determination, or both of the likelihood of an adverse event occurring and are a critical element of the NIPP risk management framework. Risk assessments also help decision makers identify and evaluate potential risks so that countermeasures can be designed and implemented to prevent or mitigate the potential effects of the risks. The NIPP characterizes risk assessment as a function of three elements: Threat: The likelihood that a particular asset, system, or network will suffer an attack or an incident. In the context of risk associated with a terrorist attack, the estimate of this is based on the analysis of the intent and the capability of an adversary; in the context of a natural disaster or accident, the likelihood is based on the probability of occurrence.

Vulnerability: The likelihood that a characteristic of, or flaw in, an asset’s, system’s, or network’s design, location, security posture, process, or operation renders it susceptible to destruction, incapacitation, or exploitation by terrorist or other to intentional acts, mechanical failures, and natural hazards.

Consequence: The negative effects on public health and safety, the economy, public confidence in institutions, and the functioning of government, both direct and indirect, that can be expected if an asset, system, or network is damaged, destroyed, or disrupted by a terrorist attack, natural disaster, or other incident.

Information from the three elements used in assessing risk—threat, vulnerability, and consequence—can lead to a risk characterization and provide input for prioritizing security goals.

TSA Has Taken Steps to Assess Threats and Vulnerabilities for Airport Security, but Has Not Conducted a Comprehensive Risk Assessment to Help Identify Priorities and Allocate Resources

While TSA has taken steps to assess risk, it has not conducted a comprehensive risk assessment based on assessments of threats, vulnerabilities, and consequences. TSA officials reported that they have identified threats to airport security as part of an overall assessment of threats to the civil aviation system. While TSA has conducted vulnerability assessment activities at select airports, it has not analyzed whether the select assessments reflect the overall vulnerability of airport security nationwide. Further, TSA has not yet assessed the consequences of an attack against airport perimeter and access control security.

TSA Has Taken Steps to Assess Risk, but a Comprehensive Risk Assessment Would Identify Priorities and Inform Resource Allocation

According to the NIPP, risk assessments are to be documented, reproducible (so that others can verify the results), defensible (technically sound and free of significant errors), and complete. The NIPP maintains that these qualities are necessary to risk assessments so they can be used to support national-level, comparative risk assessment, planning, and resource prioritization. For a risk assessment to be considered complete, the NIPP states that it must specifically assess threat, vulnerability, and consequence; after these three components have been assessed, they are to be combined to produce a risk estimate. According to the NIPP, comprehensive risk assessments are necessary for determining which assets or systems face the highest risk for prioritizing risk mitigation efforts and the allocation of resources and for effectively measuring how security programs reduce risks.

In March 2009 we reported that a lack of information that fully depicts threats, vulnerabilities, and consequences limits an organization’s ability to establish priorities and make cost-effective security measure decisions. TSA officials told us that they have not completed a comprehensive risk assessment for airport security, although they said that they have prepared and are currently reviewing a draft of a comprehensive, scenario-based air domain risk assessment (ADRA), which officials said is to serve as a comprehensive risk assessment for airport security. According to officials, the ADRA is to address all three elements of risk for domestic commercial aviation, general aviation, and air cargo. However, TSA has not released it as originally planned for in February 2008. As of May 2009 TSA officials had not provided revised dates for when the agency expects to finalize the ADRA, and they could not provide documentation to demonstrate to what extent the ADRA will address all three components of risk for airport perimeter and access control security. As a result, it is not clear whether the ADRA will provide the risk analysis needed to inform TSA’s decisions and planning for airport perimeter and access control security. Standard practices in program management call for documenting the scope of the program and milestones (i.e., time frames) to ensure results are achieved. Conducting a comprehensive risk assessment for airport security and documenting milestones for its implementation would help ensure that TSA’s intended actions will be implemented, and would allow TSA to more confidently ensure that its investments in airport security are risk informed and allocated toward the highest-priority risks.

TSA Uses a Variety of Products to Assess Threat to Airport Security

A threat assessment is the identification and evaluation of adverse events that can harm or damage an asset. TSA uses several products to identify and assess potential threats to airport security, such as daily intelligence briefings, weekly suspicious incident reports, and situational awareness reports, all of which are available to internal and external stakeholders. TSA also issues an annual threat assessment of the U.S. civil aviation system, which includes an assessment of threats to airport perimeter and access control security. According to TSA officials, these products collectively form TSA’s assessment of threats to airport perimeter and access control security. TSA’s 2008 Civil Aviation Threat Assessment cites four potential threats related to perimeter and access control security, one of which is the threat from insiders—airport workers with authorized access to secured areas. The 2008 assessment characterized the insider threat as “one of the greatest threats to aviation,” which TSA officials explained is meant to reflect the opportunity insiders have to do damage, as well as the vulnerability of commercial airports to an insider attack, which these officials stated as being very high. As of May 2009, TSA had no knowledge of a specific plot by terrorists or others to breach the security of any domestic commercial airport. However, TSA has also noted that airports are seen as more accessible targets than aircraft, and that airport perimeters may become more desirable targets as terrorists look for new ways to circumvent aviation security.

Intelligence is necessary to inform threat assessments. As we reported in March 2009, TSA has not clarified the levels of uncertainty—or varying levels of confidence—associated with the intelligence information it has used to identify threats to the transportation sector and guide its planning and investment decisions. Both Congress and the administration have recognized uncertainty inherent in intelligence analysis, and have required analytic products within the intelligence community to properly caveat and express uncertainties or confidence in resulting conclusions or judgments. As a result, the intelligence community and the Department of Defense have adopted this practice in reporting threat intelligence. Since TSA does not assign confidence levels to its analytic judgments, it is difficult for TSA to correctly prioritize its tactics and investments based on uncertain intelligence. In March 2009 we recommended that TSA work with the Director of National Intelligence to determine the best approach for assigning uncertainty or confidence levels to analytic intelligence products and apply this approach. TSA agreed with this recommendation and said that it has begun taking action to address it.

Additional Analysis Could Help Inform TSA’s Assessment Activities for Airport Security Vulnerabilities

Analyzing the Extent to Which Joint Vulnerability Assessments Provide an Assessment of Nationwide Vulnerabilities Could Strengthen TSA’s Ability to Mitigate Risk

The NIPP requires that a risk assessment include a comprehensive assessment of vulnerabilities in assets or systems, such as a physical design feature or type of location, that make them susceptible to a terrorist attack. As we reported in June 2004, these assessments are intended to facilitate airport operators’ efforts to comprehensively identify and effectively address perimeter and access control security weaknesses. TSA officials told us that their primary measures for assessing the vulnerability of commercial airports to attack are the collective results of joint vulnerability assessments (JVA) and professional judgment. TSA officials said that the agency plans to expand the number of JVAs conducted in the future but, as of May 2009, did not have a plan for doing so.

According to TSA officials, JVAs are assessments that teams of TSA special agents and other officials conduct jointly with the Federal Bureau of Investigation (FBI) and, as required by law, are generally conducted every 3 years for airports identified as high risk. In response to our 2004 recommendation that TSA establish a schedule and analytical approach for completing vulnerability assessments for evaluating airport security, TSA developed criteria to select and prioritize airports as high-risk for assessment. TSA officials stated that in addition to assessing airports identified as high risk, the agency has also assessed the vulnerability of other airports at the request of FSDs. According to TSA’s TS-SSP, after focusing initially on airports deemed high risk, JVAs are to be conducted at all commercial airports. TSA officials stated that JVA teams assess all aspects of airport security and operations, including fuel, cargo, catering, general aviation, terminal area and law enforcement operations, and the controls that limit access to secured areas and the integrity of the airport perimeter. However, officials emphasized that a JVA is not intended to be a review of an airport’s compliance with security requirements and teams do not impose penalties for noncompliance. From fiscal years 2004 through 2008, TSA conducted 67 JVAs at a total of 57 airports—about 13 percent of the approximately 450 commercial airports nationwide. In 2007 TSA officials conducted a preliminary analysis of the results of JVAs conducted at 23 domestic airports during fiscal years 2004 and 2005, and found 6 areas in which 20 percent or more of the airports assessed were identified as vulnerable. Specific vulnerabilities included the absence of blast resistant glass in terminal windows, lack of bollards/barriers in front of terminals, lack of blast resistant trash receptacles, and insufficient electronic surveillance of perimeter lines and access points. As of May 2009 TSA officials said that the agency had not finalized this analysis and, as of that date, did not have plans to do so. TSA officials also told us that they have shared the results of JVA reports with TSA’s Office of Security Technology to prioritize the distribution of relevant technology to those airports with vulnerabilities that these technologies could strengthen.

TSA characterizes U.S. airports as a system of interdependent hubs and links (spokes) in which the security of all is affected or disrupted by the security of the weakest one. The interdependent nature of the system necessitates that TSA protect the overall system as well as individual assets. TSA maintains that such a “systems-based approach” allows it to focus resources on reducing risks across the entire system while maintaining cost-effectiveness and efficiency. TSA officials could not explain to what extent the collective JVAs of specific airports constitute a reasonable systems-based assessment of vulnerability across airports nationwide or whether the agency has considered assessing vulnerabilities across all airports. Although TSA has conducted JVAs at each category of airport, 58 of the 67 were at the largest airports. According to TSA data, 87 percent of commercial airports—most of the smaller Category II, III, and IV airports—have not received a JVA. TSA officials said that because they have not conducted JVAs for these airports, they do not know how vulnerable they are to an intentional security breach. In 2004 we reported that TSA intended to compile baseline data on airport security vulnerabilities to enable it to conduct a systematic analysis of airport security vulnerabilities nationwide. At that time TSA officials told us that such analysis was essential since it would allow the agency to determine the adequacy of security policies and help TSA and airport operators better direct limited resources. According to TSA officials, conducting JVAs at all airports would allow them to compile national baseline data on perimeter and access control security vulnerabilities. As of May 2009, however, TSA officials had not yet completed a nationwide vulnerability assessment, evaluated whether the current approach to JVAs would provide the desired systems-based approach to assessing airport security vulnerabilities, or explained why a nationwide assessment or evaluation has not been conducted. In subsequent discussions, TSA officials told us that based on our review they intend to increase the number of JVAs conducted at airports that are not categorized as high risk—primarily Category II, III, and IV airports. According to officials, the resulting data are to assist TSA in prioritizing the allocation of limited resources. However, TSA officials could not tell us how many additional airports they plan to assess in total or within each category, the analytical approach and time frames for conducting these assessments, and to what extent these additional assessments, in combination with past JVAs, will constitute a reasonable systems-based assessment of vulnerability across airports nationwide. Standard practices for program management call for establishing a management plan and milestones to meet stated objectives and achieve results. It is also unclear to what extent the ADRA, when it is completed, will represent a systems-based vulnerability assessment, an assessment of airports nationwide, or both. Given that TSA officials believe that the vulnerability of airports to an insider attack is very high and the security of airports is interconnected, this vulnerability would extend throughout the nationwide system of airports. Evaluating the extent to which the agency’s current approach assesses systems-based vulnerabilities, including the vulnerabilities of smaller airports, would better position TSA to provide reasonable assurance that it is identifying and addressing the areas of greatest vulnerability and the spectrum of vulnerability across the entire airport system. Further, should TSA decide to conduct a nationwide assessment of airport vulnerability, developing a plan that includes milestones for completing the assessment would help TSA ensure that it takes the necessary actions to accomplish desired objectives within reasonable time frames.

TSA Could Strengthen Its Understanding of Risks by Considering Vulnerability Assessment Activities Conducted by Airport Operators

According to the NIPP, DHS and lead security agencies, such as TSA, are to seek to use information from the risk assessments of security partners, whenever possible, to contribute to an understanding of sector and national risks. Moreover, the NIPP states that DHS and lead agencies are to work together to assist security partners in providing vulnerability assessment tools that may be used as part of self-assessment processes, and provide recommendations regarding the frequency of assessments, particularly in light of emergent threats. According to the NIPP, stakeholder vulnerability assessments may serve as a basis for developing common vulnerability reports that can help identify strategic needs and more fully investigate interdependencies.

However, TSA officials could not explain to what extent they make use of relevant vulnerability assessments conducted independently by airport operators to contribute to the agency’s understanding of airport security risks, or have worked with security partners to help ensure that tools are available for airports to conduct self-assessment processes of vulnerability. Officials from two prominent airport industry associations estimated that the majority of airports, particularly larger airports, have conducted vulnerability assessments, although they could not give us a specific number. In addition, officials from 8 of the 10 airports whom we interviewed on this issue told us that their airports had conducted vulnerability assessment activities. Some of these analyses could be useful to TSA in conducting a systematic analysis of airport security vulnerabilities nationwide. By taking advantage, to the extent possible, of existing vulnerability assessment activities conducted by airport operators, TSA could enrich its understanding of airport security vulnerabilities and therefore better inform federal actions for reducing airport vulnerabilities.

TSA Has Not Conducted a Consequence Assessment for Airport Security

According to TSA officials, the agency has not assessed the consequences of a successful attack against airport perimeters or a breach to secured areas within airports, even though the NIPP asserts that the potential consequence of an incident is the first factor to be considered in developing a risk assessment. According to the NIPP, risk assessments should include consequence assessments that evaluate negative effects to public health and safety, the economy, public confidence in national economic and political institutions, and the functioning of government that can be expected if an asset, system, or network is damaged, destroyed, or disrupted by a terrorist attack.

Although TSA officials agree that a consequence assessment for airport security is needed, and have stated that the ADRA is intended to provide a comprehensive consequence assessment based on risk scenarios, the agency has not provided additional details as to what the assessment will include, the extent to which it will assess consequence for airport security, or when it will be completed. Standard management practices call for documenting milestones (i.e., time frames) to ensure that results are achieved. TSA officials have agreed that a consequence assessment for airport perimeter and access controls security is an important element in assessing risk to airport security. In addition, TSA officials commented that although the immediate consequences of a breach of airport security would likely be limited, such an event could be the first step in a more significant attack against an airport terminal or aircraft, or an attempt to use an aircraft as a weapon. Conducting a consequence assessment could help TSA in developing a comprehensive risk assessment and increase its assurance that the resulting steps it takes to strengthen airport security will more effectively reduce risk and mitigate the consequences of an attack on individual airports and the aviation system as a whole.

TSA Has Taken a Variety of Protective Actions to Strengthen Airport Security, but Did Not Follow Accepted Practices in Developing Its Worker Screening Pilot Program; Additionally, Issues Remain regarding Worker Security, Technology, and Other Initiatives

TSA has implemented a variety of programs and protective actions to strengthen airport security, from additional worker screening to assessing different technologies. For example, consistent with the Explanatory Statement, TSA piloted several methods to screen workers accessing secured areas, but clear conclusions could not be drawn because of significant design limitations, and TSA did not develop or document an evaluation plan to guide design and implementation of the pilot. Further, while TSA has strengthened other worker security programs, assessed various technologies, and added to programs aimed at improving general airport security, certain issues, such as whether security technologies meet airport needs, have not been fully resolved.

TSA Has Taken a Variety of Protective Actions to Improve and Strengthen the Security of Commercial Airports since 2004

TSA has taken a variety of protective actions to improve and strengthen the security of commercial airports through the development of new programs or by enhancing existing efforts. Since we last reported on airport perimeter and access control security in June 2004, TSA has implemented efforts to strengthen worker screening and security programs, improve access control technology, and enhance general airport security by providing an additional security presence at airports. According to TSA, each of its security actions—or layers—is capable of stopping a terrorist attack, but when used in combination (what TSA calls a layered approach), a much stronger system results. To better address the risks posed by airport workers, TSA, in accordance with the Explanatory Statement accompanying the DHS Appropriations Act, 2008, initiated a worker screening pilot program to assess various types of screening methods for airport workers. TSA also implemented a random worker screening program and is currently working to apply its screening procedures consistently across airports. In addition, TSA has expanded its requirements for conducting worker background checks. TSA has also taken steps, such as implementing two pilot programs, to identify and assess technologies to strengthen the security of airport perimeters and access controls to secured areas. Further, TSA has taken steps to strengthen general airport security processes. For example, TSA has developed a program in which teams of TSA officials, law enforcement officers, and airport officials temporarily augment airport security through various actions such as randomly inspecting workers, property, and vehicles and patrolling secured areas. Table 1 lists the actions TSA has taken since 2004 to strengthen airport security.

TSA Has Pilot Tested Various Worker Screening Methods, but Significant Program Limitations and Lack of a Sound Evaluation Plan May Limit the Usefulness of the Results

From May through July 2008 TSA piloted a program to screen 100 percent of workers at three airports and to test a variety of enhanced screening methods at four other airports. (See app. V for more detailed information on the pilot program, including locations and types of screening methods used.) According to TSA, the objective of the pilot was to compare 100 percent worker screening and enhanced random worker screening based on (1) screening effectiveness, (2) impact on airport operations, and (3) cost considerations. TSA officials hired a contractor—HSI, a federally funded research and development center—to assist with the design, implementation, and evaluation of the data collected. In July 2009 TSA released a report on the results of the pilot program, which included HSI’s findings. HSI concluded that random screening is a more cost-effective approach because it appears “roughly” as effective in identifying contraband items—or items of interest—at less cost than 100 percent worker screening. However, HSI also emphasized that the pilot program “was not a robust experiment” because of limitations in the design and evaluation, such as the limited number of participating airports, which led HSI to identify uncertainties in the results. Given the significance of these limitations, we believe that it is unclear whether random worker screening is more or less cost-effective than 100 percent worker screening.

Specifically, HSI identified what we believe to be significant limitations related to the design of the pilot program and the estimation of costs and operational effects. Limitations related to program design include (1) a limited number of participating airports, (2) the short duration of screening operations (generally 90 days), (3) the variety of screening techniques applied, (4) the lack of a baseline, and (5) limited evaluation of enhanced methods. For example, HSI noted that while two of the seven pilot airports performed complete 100 percent worker screening, neither was a Category X airport; a third airport—a Category X—performed 100 percent screening at certain locations for limited durations. HSI also reported that the other four pilot airports used a range of tools and screening techniques—magnetometers, handheld metal detectors, pat- downs—which reduced its ability to assess in great detail any one screening process common to all the pilot airports. In addition, HSI cited issues regarding the use of baseline data for comparison of screening methods. HSI attempted to use previous Aviation Direct Access Screening Program (ADASP) screening data for comparison, but these data were not always comparable in terms of how the screening was conducted. In addition, HSI identified a significant limitation in generalizing pilot program results across airports nationwide, given the limited number and diversity of the pilot airports. HSI noted that because these airports were chosen based on geographic diversity and size, other unique airport factors that might affect worker screening operations—such as workforce size and the number and location of access points—may not have been considered.

HSI also recognized what we believe to be significant limitations in the development of estimates of the costs and operational effects of implementing 100 percent worker screening and random worker screening nationwide. HSI’s characterization of its cost estimates as “rough order of magnitude”—or imprecise—underscores the challenge of estimating costs for the entire airport system in the absence of detailed data on individual airports nationwide and in light of the limited amount of information gleaned from the pilot on operational effects and other costs. HSI noted that the cost estimates do not include costs associated with operational effects, such as longer wait times for workers, and potentially costly infrastructure modifications, such as construction of roads and shelters to accommodate vehicle screening. HSI developed high- and low-cost estimates based on current and optimal numbers of airport access points and the amount of resources (personnel, space, and equipment) needed to conduct 100 percent and random worker screening. According to these estimates, the direct cost—including personnel, equipment, and other operation needs—of implementing 100 percent worker screening would range from $5.7 billion to $14.9 billion for the first year, while the direct costs of implementing enhanced random worker screening would range from $1.8 billion to $6.6 billion.

HSI noted that the random worker screening methods applied in the worker screening pilot program were a “significant step” beyond TSA’s ongoing worker screening program—ADASP—which the agency characterizes as a “random” worker screening program. For the four pilot airports that applied random screening methods, TSA and airport associations agreed to screen a targeted 20 percent of workers who entered secured areas each day. TSA officials also told us that this 20 percent threshold was significantly higher than that applied through ADASP, although officials said that they do not track the percentage of screening events processed through ADASP. TSA officials told us that they do not have sufficient resources to track this information.

In addition to the limitations recognized by HSI, TSA and HSI did not document key aspects of the design and implementation of the pilot program. For example, while they did develop and document a data collection plan that outlined the data requirements, sources, and collection methods to be followed by the seven pilot airports in order to evaluate the program’s costs, benefits, and impacts, they did not document a plan for how such data would be analyzed to formulate results. Standards for Internal Control for the Federal Government states that significant events are to be clearly documented and the documentation should be readily available for examination to inform management decisions. In addition, in November 2008, based in part on our guide for designing evaluations, we reported that pilot programs can more effectively inform future program rollout when an evaluation plan is developed to guide consistent implementation of the pilot and analysis of the results. At minimum, a well-developed, sound evaluation plan contains several key elements, including measurable objectives, standards for pilot performance, a clearly articulated methodology, detailed data collection methods, and a detailed data analysis plan. Incorporating these elements can help ensure that the implementation of a pilot generates performance information needed to make effective management decisions. While TSA and HSI completed a data collection plan, and generally defined specific measurable objectives for the pilot program, they did not address other key elements that collectively could have strengthened the effectiveness of the pilot program and the usefulness of the results: Performance standards. TSA and HSI did not develop and document criteria or standards for determining pilot program performance, which are necessary for determining to what extent the pilot program is effective.

Clearly articulated evaluation methodology. TSA and HSI did not fully articulate and document the methodology for evaluating the pilot program. Such a methodology is to include plans for sound sampling methods, appropriate sample sizes, and comparing the pilot results with ongoing efforts. TSA and HSI documented relevant elements, such as certain sampling methods and sample sizes, in both its overall data collection plan for the program and in individual pilot operations plans for each airport implementing the pilot. However, while officials stated that the seven airports were selected to obtain a range of physical size, worker volume, and geographical dispersion information, they did not document the criteria they used in this process, and could not explain the rationale used to decide which screening methods would be piloted by the individual airports. Because the seven airports tested different screening methods, there were differences in the design of the individual pilots as well as in the type and frequency of the data collected. While design differences are to be expected given that the pilot program was testing disparate screening methods, there were discrepancies in the plans that limited HSI’s ability to compare methods across sites. For example, those airports that tested enhanced screening methods—as opposed to 100 percent worker screening—used different rationales to determine how many inspections would be conducted each day. TSA officials said that this issue and other discrepancies and points of confusion were addressed through oral briefings with the pilot airports, but said that they did not provide additional written instructions to the airports responsible for conducting the pilots. TSA and HSI officials also did not document how they would address deviations from the piloted methods, such as workers who avoided the piloted screening by accessing alternative entry points, or suspension of the pilot because of excessive wait times for workers or passengers (some workers were screened through passenger screening checkpoints). Further, TSA and HSI officials did not develop and document a plan for comparing the results of the piloted worker screening methods with TSA’s ongoing random worker screening program to determine whether the piloted methods had a greater impact on reducing insider risk than ongoing screening efforts.

Detailed data analysis. Although the agreement between TSA and HSI also called for the development of a data analysis plan, neither HSI nor TSA developed an analysis plan to describe how the collected data would be used to track the program’s performance and evaluate the effectiveness of the piloted screening methods, including 100 percent worker screening. For example, HSI used the number of confiscated items as a means of comparing the relative effectiveness of each screening method. However, HSI reported that the number of items confiscated during pilot operations was “very low” at most pilot airports, and some did not detect any. Based on these data, HSI concluded that random worker screening appeared to be “roughly” as effective in identifying confiscated items as 100 percent worker screening. However, it is possible that there were few or no contraband items to detect, as workers at the pilot airports were warned in advance when the piloted screening methods would be in effect and disclosure signs were posted at access points. As a result, comparing the very low rate—and in some cases, nonexistence—of confiscated items across pilots, coupled with the short assessment period, may not fully indicate the effectiveness of different screening methods at different airports. If a data analysis plan had been developed during pilot design, it could have been used to explain how such data would be analyzed, including how HSI’s analysis of the pilots’ effectiveness accounted for the low confiscation rates.

Because of the significance of the pilot program limitations reported by HSI, as well as the lack of documentation and detailed information regarding the evaluation of the program, the reliability of the resulting data and any subsequent conclusions about the potential impacts, costs, benefits, and effectiveness of 100 percent worker screening and other screening methods cannot be verified. For these reasons, it would not be prudent to base major policy decisions regarding worker screening solely on the results of the pilot program. HSI reported that the wide variation— such as size, traffic flow, and design—of U.S. commercial airports makes it difficult to generalize the seven pilot results to all commercial airports. While we agree it is difficult to generalize the results of such a small sample to an entire population, a well-documented and sound evaluation plan could have helped ensure that the pilot program generated the data and performance information needed to draw reasonable conclusions about the effectiveness of 100 percent worker screening and other methods to inform nationwide implementation. Incorporating these elements into an evaluation plan when designing future pilots could help ensure that TSA’s pilots generate the necessary data for making management decisions and that TSA can demonstrate that the results are reliable.

TSA Has Taken Steps to Strengthen Worker Security Programs, but Issues Remain

Aviation Direct Access Screening Program

According to TSA officials, FSDs and others in the aviation community have long recognized the potential for insiders to do harm from within an airport. TSA officials said that they developed ADASP—a random worker screening program—to counteract the potential vulnerability of airports to an insider attack. According to TSA officials, ADASP serves as an additional layer of security and as a deterrent to workers who seek to smuggle drugs or weapons or to do harm. According to senior TSA officials, FSDs decide when and how to implement ADASP, including the random screening of passengers at the boarding gate or workers at SIDA access points to the sterile area.

TSA officials said that ADASP was initially developed as a pilot project at one airport in March 2005 to deter workers from breaching access controls and procedures for secured areas at that particular airport. According to officials, after concluding that the pilot was successful in deterring airport workers from bringing restricted items into secured areas, TSA began implementing ADASP on a nationwide voluntary basis in August 2006 using existing resources. In March 2007, in response to several incidents of insider criminal activity, TSA directed that ADASP be conducted at all commercial airports nationwide. For example, on March 5, 2007, two airline employees smuggled 14 firearms and 8 pounds of marijuana on board a commercial airplane at Orlando International Airport (based on information received through an anonymous tip, the contraband was confiscated when the plane landed in San Juan, Puerto Rico).

In its October 2008 report, the DHS Office of the Inspector General (OIG) found that ADASP was being implemented in a manner that allowed workers to avoid being screened, and that the program had been applied inconsistently across airports. For example, at most of the seven airports the DHS OIG visited, ADASP screening stations were set up in front of worker access points, which allowed workers to identify that ADASP was being implemented and potentially choose another entry and avoid being screened. However, at another airport, the screening location was set up behind the access point, which prevented workers from avoiding being screened. ADASP standard operating procedures allow ADASP screening locations to be set up in front of or behind direct access points as long as there is signage alerting workers that ADASP screening is taking place. However, the DHS OIG found that the location of the screening stations— either in front of or behind direct access points—affected whether posted signs were visible to workers. The DHS OIG recommended that TSA apply consistent ADASP policies and procedures at all airports, and establish an ADASP working group to consider policy and procedure changes based on an accumulation of best practices across the country. TSA agreed with the DHS OIG’s recommendations, and officials stated that they have begun to take action to address them.

Expanded Worker Background Checks

Since April 2004, and in response to our prior recommendation, TSA has taken steps to enhance airport worker background checks. TSA background checks are composed of security threat assessments (STA), which are name-based records checks against various terrorist watch lists, and criminal history record checks (CHRC), which are fingerprint-based criminal records checks. TSA requires airport workers to undergo both STAs and CHRCs before being granted unescorted access to secured areas in which they perform their duties.

In July 2004 TSA expanded STA requirements by requiring workers in certain secured areas to submit current biographical information, such as date of birth. TSA further augmented STAs in 2005 to include a citizenship check to identify individuals who may be subject to coercion because of their immigration status or who may otherwise pose a threat to transportation security. In 2007 TSA expanded STA requirements beyond workers with sterile area or SIDA access to apply to all individuals seeking or holding airport-issued identification badges or credentials. Finally, in June 2009 TSA began requiring airport operators to renew all airport identification media every 2 years, deactivate expired media and require workers to resubmit biographical information in the event of certain changes, and expand the STA requirement to include individuals with unescorted access to the AOA, among other things.

TSA has taken steps to strengthen its background check requirements and is considering additional actions to address certain statutory requirements and issues that we identified in 2004. For example, TSA is considering revising its regulation listing the offenses that if a conviction occurred within 10 years of applying for this access, would disqualify a person from receiving unescorted access to secured areas. TSA officials told us that TSA and industry stakeholders are considering whether some disqualifying offenses may warrant a lifelong ban. In addition, while TSA has not yet specifically addressed a statutory provision requiring TSA to require, by regulation, that individuals with regularly escorted access to secured airport areas undergo background checks, TSA officials told us that they believe the agency’s existing measures address the potential risk presented by such workers. They also said that it would be challenging to identify the population of workers who require regularly escorted access because such individuals—for example, construction workers—enter airports on an infrequent and unpredictable basis.

TSA Has Taken Steps to Improve Security Technology, but the Extent to Which TSA Has Addressed Airport Technology Needs Is Unclear

Biometric Access Control Initiatives

Since 2004, TSA has taken some steps to develop biometric worker credentialing; however, it is unclear to what extent TSA plans to address statutory requirements regarding biometric technology, such as developing or requiring biometric access controls at commercial airports in consultation with industry stakeholders. For instance, in October 2008 the DHS OIG reported that TSA planned to mandate phased-in biometric upgrades for all airport access control systems to meet certain specifications. However, as of May 2009, according to TSA officials, the agency had not made a final decision on whether to require airports to implement biometric access controls, but it intends to pursue a combination of rule making and other measures to encourage airports to voluntarily implement biometric credentials and control systems. While TSA officials said that the agency issued a security directive in December 2008 that encourages airports to implement biometric access control systems that are aligned with existing federal identification standards, TSA officials also reported the need to ensure that airports incorporate up- to-date standards. These officials also said that TSA is considering establishing minimum requirements to ensure consistency in data collection, card information configuration, and biometric information. Airport operators and industry association officials have called for a consensus-based approach to developing biometric technology standards for airports, and have stressed the need for standards that allow for flexibility and consider the significant investment some airports have already made in biometric technology. Airport operators have also expressed a reluctance to move forward with individual biometric projects because of concerns that their enhancements will not conform to future federal standards.

Although TSA has not decided whether it will mandate biometric credentials and access controls at airports, it has taken steps to assess and develop such technology in response to stakeholder concerns and statutory requirements. For example, TSA officials said the agency has assisted the aviation industry and RTCA, Inc., a federal aviation advisory committee, in developing recommended security standards for biometric access controls, which officials said provide guidelines for acquiring, designing, and implementing access control systems. TSA officials also noted that the agency has cooperated with the Biometric Airport Security Identification Consortium, or BASIC—a working group of airport operators and aviation association representatives—which has developed guidance on key principles that it believes should be part of any future biometric credential and access control system. In addition, TSA is in the early stages of developing the Aviation Credential Interoperability Solution (ACIS) program. ACIS is conceived as a credentialing system in which airports use biometrics to verify the identities and privileges of workers who have airport- or air carrier–issued identification badges before granting them entry to secured areas. According to TSA, ACIS would provide a trusted biometric credential based on smart card technology (about the size of a credit card, using circuit chips to store and process data) and specific industry standards, and establish standard airport processes for enrollment, card issuance, vetting, and the management of credentials. Although these processes would be standardized nationwide, airports would still be individually responsible for determining access authority. According to TSA officials, the agency is seeking to build ACIS on much of the airports’ existing infrastructure and systems and has asked industry stakeholders for input on key considerations, including the population of workers who would receive the credential, program policies, process, technology considerations, operational impacts, and concerns regarding ACIS.

However, as of May 2009, TSA officials could not explain the status of ACIS or provide additional information on the possible implementation of the program since the agency released the specifications for industry comment in April 2008. As a result, it is unclear when and how the agency plans to address the requirements of the Intelligence Reform and Terrorism Prevention Act, including establishing minimum standards for biometric systems and determining the best way to incorporate these decisions into airports’ existing practices and systems. As of May 2009 TSA officials had not provided any further information, such as scheduled milestones, on TSA’s plans to implement biometric technology at airports. Standard practices in program management suggest that developing scheduled milestones can help define the scope of the project, achieve key deliverables, and communicate with key stakeholders. In addition, until TSA communicates its decision on whether it plans to mandate—such as through a rule making—or collaboratively implement biometric access controls at airports, and what approach is best—be it ACIS or another system—operators may be hesitant to upgrade airport security in this area. As we reported in 2004, airport operators do not want to run the risk of installing costly technology that may not comply with future TSA requirements and standards. Developing milestones for implementing a biometric system could help ensure that TSA addresses statutory requirements. In addition, such milestones will provide airports and the aviation industry with the scheduling information needed to plan future security improvements and expenditures.

Technology Pilot Programs

In addition to biometric technology efforts, TSA has also initiated efforts to assess other airport perimeter and access control technology. Pursuant to ATSA, TSA established two pilot programs to assess perimeter and access control security technology, the Airport Access Control Pilot Program (AACPP) in 2004 and the Airport Perimeter Security (APS) pilot program in 2006. AACPP piloted various new and emerging airport security technologies, including biometrics. TSA issued the final report on AACPP in December 2006, but did not recommend any of the piloted technologies for full-scale implementation. TSA officials said that a second round of pilot projects would be necessary to allow time for project evaluation and limited deployments, but as of May 2009 TSA officials said that details for this second round were still being finalized. The purpose of the APS pilot, according to TSA officials, is to identify and mitigate existing airport perimeter security vulnerabilities using commercially available technology. APS was originally scheduled to be completed in December 2007, but according to TSA officials, though five of the six pilot projects have been completed, the remaining pilot has been delayed because of problems with the acquisition process. According to TSA officials, the final pilot project is to be completed by October 2009.

TSA officials told us that the agency has also taken steps to provide some technical and financial support to small- and medium-sized airports through AACPP and the APS pilot program, as both tested technologies that could be suitable for airports of these sizes. TSA officials also stated that smaller airports could potentially benefit from the agency’s efforts to test the Virtual Perimeter Monitoring System, which was developed by the U.S. Navy and is being installed and evaluated at four small airports. Further, officials noted that TSA has also provided significant funding to support cooperative agreements for the deployment of law enforcement officers at airports—including Category II, III, and IV airports—to help defray security costs. However, according to TSA officials, as of May 2009 TSA had not yet developed a plan, or a time frame for developing a plan, to provide technical information and funding to small- and medium-sized airports, as required by ATSA. According to TSA officials, funds had not been appropriated or specifically directed to develop such a plan, and TSA’s resources and management attention have been focused on other statutory requirements for which it has more direct responsibility and deadlines, including passenger and baggage screening requirements. (For a summary of TSA actions to address certain statutory requirements for airport security technology, see app. II.)

TSA Has Taken Action to Improve General Airport Security, but Concerns Exist regarding Implementation of Security Requirements Established by Security Directives

TSA has taken actions to improve general airport security by establishing programs and requirements. For example, TSA has augmented access control screening and general airport security by increasing the presence of transportation security officers and law enforcement officials through the Screening of Passengers by Observation Techniques (SPOT) program and the Law Enforcement Officer Reimbursement Program. In addition, it uses the Visible Intermodal Prevention and Response (VIPR) program, which is used across the transportation sector, to augment airport security efforts. (For more information on these TSA programs, see app. VI.)

TSA uses a variety of regulatory mechanisms for imposing requirements within the transportation sector. In the aviation environment, TSA uses the security directive as one of its regulatory tools for imposing requirements to strengthen the security of civil aviation, including security at the nation’s commercial airports. Pursuant to TSA regulation, the agency may decide to use security directives to impose requirements on airport operators if, for example, it determines that additional security measures are needed to respond to general or specific threats against the civil aviation system. As of March 2009 TSA identified 25 security directives or emergency amendments in effect that related to various aspects of airport perimeter and access control security. As shown in table 2, TSA imposed requirements through security directives that address areas such as worker and vehicle screening, criminal history record checks, and law enforcement officer deployments.

According to TSA officials, security directives enable the agency to respond rapidly to immediate or imminent threats and provide the agency with flexibility in how it imposes requirements on airport operators. This function is especially relevant given the adaptive, dynamic nature of the terrorist threat. Moreover, according to TSA, imposing requirements through security directives is less time consuming than other processes, such as the lengthier notice-and-comment rule making process, which generally provides opportunity for more stakeholder input, requires cost- benefit analysis, and provides the regulated entities with more notice before implementation and enforcement.

Officials from two prominent aviation associations and eight of nine airports we visited identified concerns regarding requirements established through security directive: Officials from the two aviation associations noted inconsistencies between requirements established through separate security directives. For example, they noted that the requirements for airport-issued identification badges are different from those for badges issued by an air carrier. Workers employed by the airport, air carrier, or other entities who apply for an airport identification badge granting unescorted access to a secured area are required to undergo an immigration and citizenship status check, whereas workers who apply through an air carrier, which can grant similar unescorted access rights, are not. Both airport and air carrier workers can apply to an airport operator for airport-issued identification badges, but only air carrier workers can apply to their aircraft operator (employer) for an air carrier–issued identification badge. TSA officials told us that the agency plans to address this inconsistency—which has been in effect since December 2002—and is working on a time frame for doing so.

Airport operator officials from eight of the nine airports we visited and officials from two industry associations expressed concern that requirements established through security directives related to airport security are often issued for an indefinite time period. Our review of 25 airport security directives and emergency amendments showed that all except one were issued with no expiration date. The two aviation industry associations have expressed concerns directly to TSA that security directive requirements should be temporary and include expiration dates so that they can be periodically reviewed for relevancy.

According to senior officials, TSA does not have internal control procedures for monitoring and coordinating requirements established through security directives related to airport perimeter and access control security. In November 2008 TSA officials told us that the agency had drafted an operations directive that documents procedures for developing, coordinating, issuing, and monitoring civil aviation security directives. According to officials, this operations directive also is to identify procedures for conducting periodic reviews of requirements imposed through security directives. However, while TSA officials told us that they initially planned to issue the operations directive in April 2009, in May 2009 they said that they were in the process of adopting the recommendations of an internal team commissioned to review and identify improvements to TSA’s policy review process, including the proposed operations directive. In addition, as of May 2009, officials did not have an expected date for finalizing the directive. TSA officials explained that because the review team’s recommendations will require organizational changes and upgrades to TSA’s information technology infrastructure, it will take a significant amount of time before an approved directive can be issued. As a result, it is unclear to what extent the operations directive will address concerns expressed by aviation operators and industry stakeholders. Standard practices in program management call for documented milestones to ensure that results are achieved. Establishing milestones for implementing guidance to periodically review airport security requirements imposed through security directives would help TSA formalize review of these directives within a time frame authorized by management.

In addition to the stakeholder issues previously discussed, representatives from two prominent aviation industry associations have expressed concern that TSA has not issued security directives in accordance with the law. Specifically, these representatives noted that the Transportation Security Oversight Board (TSOB) has not reviewed TSA’s airport perimeter and access control security directives in accordance with a provision set forth in ATSA. This provision, as amended, establishes emergency procedures by which TSA may immediately issue a regulation or security directive to protect transportation security, and provides that any such regulation or security directive is subject to review by the TSOB. The provision further states that any regulation or security directive issued pursuant to this authority may remain in effect for a period not to exceed 90 days unless ratified or disapproved by the TSOB. According to TSA officials, the agency has not issued security directives related to airport perimeter and access control security under this emergency authority. Rather, officials explained, the agency has issued such security directives (and all aviation-related security directives) in accordance with its aviation security regulations governing airport and aircraft operators, which predate ATSA and the establishment of TSA. FAA implemented regulations—promulgated through the notice-and- comment rule making process—establishing FAA’s authority to issue security directives to impose requirements on U.S. airport and aircraft operators. With the establishment of TSA, FAA’s authority to regulate civil aviation security, including its authority to issue security directives, transferred to the new agency. TSA does not consider ATSA to have altered this existing authority.

A National Strategy for Airport Security Could Help Ensure Program Effectiveness, Inform Cost and Resource Decisions, Ensure Collaboration, and Increase Accountability

Although TSA has developed a variety of individual protective actions to mitigate identified airport security risks, it has not developed a unified national strategy aimed at enhancing airport perimeter and access control security. Through our prior work on national security planning, we have identified characteristics of effective security strategies, several of which are relevant to TSA’s numerous efforts to enhance perimeter and access control security. For example, TSA has not developed goals and objectives for related programs and activities, prioritized protective security actions, or developed performance measures to assess the results of its perimeter and access control security efforts beyond tracking outputs (the level of activity provided over a period of time). Further, although TSA has identified some cost information that is used to inform programmatic decision making, it has not fully assessed the costs and resources necessary to implement its airport security efforts. Finally, TSA has not fully outlined how activities are to be coordinated among stakeholders, integrated with other aviation security priorities, or implemented within the agency.

Leading Practices Show That Strategies Help Guide Decision Making and Increase Accountability

Developing a strategy to accomplish goals and desired outcomes helps organizations manage their programs more effectively and is an essential mechanism to guide progress in achieving desired results. Strategies are the starting point and foundation for defining what an agency seeks to accomplish, and we have reported that effective strategies provide an overarching framework for setting and communicating goals and priorities and allocating resources to inform decision making and help ensure accountability. Moreover, a strategy that outlines security goals, as well as mechanisms and measures to achieve such goals, and that is understood and available to all relevant stakeholders strengthens implementation of and accountability to common principles.

A national strategy to guide and integrate the nation’s airport security activities could strengthen decision making and accountability for several reasons. First, TSA has identified airport perimeter and access control security—particularly the mitigation of risks posed by workers who have unescorted access to secured areas—as a top priority. Historically, TSA has recognized the importance of developing strategies for high-priority security programs involving high levels of perceived risk and resources, such as air cargo security and the SPOT program. Second, in security networks that rely on the cooperation of all security partners—in this case TSA, airport operators, and air carriers—strategies can provide a basis for communication and mutual understanding between security partners that is fundamental for such integrated protective programs and activities. In addition, because of the mutually dependent roles that TSA and its security partners have in airport security operations, TSA’s ability to achieve results depends on the ability of all security partners to operate under common procedures and achieve shared security goals. Finally, officials from two prominent industry organizations that represent the majority of the nation’s airport operators said that the industry would significantly benefit from a TSA-led strategy that identified long-term goals for airport perimeter and access control security. In addition to providing a unifying framework, a strategy that clearly identifies milestones, developed in cooperation with industry security partners, could make it easier for airport operators to plan, fund, and implement security enhancements that according to industry officials can require intensive capital improvements.

While TSA has taken steps to assess threat and vulnerability related to airport security and developed a variety of protective actions to mitigate risk, TSA has not developed a unifying strategy to guide the development, implementation, and assessment of these varied actions and those of its security partners. TSA officials cited three reasons why the agency has not developed a strategy to guide national efforts to enhance airport security. First, TSA officials cited a lack of congressional emphasis on airport perimeter and access control security relative to other high-risk areas, such as passenger and baggage screening. Second, these officials noted that airport operators, not TSA, have operational responsibility for airport security. Third, they cited a lack of resources and funding.

While these issues may present challenges, they should be considered in light of other factors. First, Congress has long recognized the importance of airport security, and has contributed to the establishment of a variety of requirements pertaining to this issue. For example, the appropriations committees, through reports accompanying DHS’s annual appropriations acts, have directed TSA to focus its efforts on enhancing several aspects of airport perimeter and access control security. Moreover, developing a strategy that clearly articulates the risk to airport security and demonstrates how those risks can be addressed through protective actions could help inform decision making. Second, though we recognize that airport operators, not TSA, generally have operational responsibility for airport perimeter and access control security, TSA—as the regulatory authority for airport security and the designated lead agency for transportation security—is responsible for identifying, prioritizing, and coordinating protection efforts within aviation, including those related to airport security. TSA currently exercises this authority by ensuring compliance with TSA-approved airport operator security programs and, pursuant to them, by issuing and ensuring compliance with requirements imposed through security directives or other means. Finally, regarding resource and funding constraints, federal guidelines for strategies and planning include linking program activities and anticipated outcomes with expected program costs. In this regard, a strategy could strengthen decision making to help allocate limited resources to mitigate risk, which is a cornerstone of homeland security policy. Additionally, DHS’s risk management approach recognizes that resources are to be focused on the greatest risks, and on protective activities designed to achieve the biggest reduction in those risks given the limited resources at hand. The NIPP risk management framework provides guidance for agencies to develop strategies and prioritize activities to those ends.

A strategy helps to link individual programs to specific performance goals and describe how the programs will contribute to the achievement of those goals. A national strategy could help TSA, airport operators, and industry stakeholders in aligning their activities, processes, and resources to support mission-related outcomes for airport perimeter and access control security, and, as a result, in determining whether their efforts are effective in meeting their goals for airport security.

TSA Has Not Identified Security Goals or Priorities or Fully Assessed the Effectiveness of Its Actions to Strengthen Airport Security

Our previous work has identified that an essential characteristic of effective strategies is the setting of goals, priorities, and performance measures. This characteristic addresses what a strategy is trying to achieve and the steps needed to achieve and measure those results. A strategy can provide a description of an ideal overall outcome, or “end- state,” and link individual programs and activities to specific performance goals, describing how they will contribute to the achievement of the end- state. The prioritization of programs and activities, and the identification of milestones and performance measures, can aid implementing parties in achieving results according to specific time frames, as well as enable effective oversight and accountability. The NIPP also calls for the development of goals, priorities, and performance measures to guide DHS components, including TSA, in achieving a desired end-state.

Goals

Security goals allow stakeholders to identify the desired outcomes that a security program intends to achieve and that all security partners are to work to attain. Defining goals and desired outcomes, in turn, enables stakeholders to better guide their decision making to develop protective security programs and activities that mitigate risks. The NIPP also states that security goals should be used in the development of specific protective programs and considered for distinct assets and systems. However, according to TSA officials, the agency has not developed goals and objectives for airport security, including specific targets or measures related to the effectiveness of security programs and activities. TSA officials told us that the agency sets goals for aviation security as a whole but has not set goals and objectives for the airport perimeter and access control security area. Developing a baseline set of security goals and objectives that consider, if not reflect, the airport perimeter and access control security environment would help provide TSA and its security partners with the fundamental tools needed to define outcomes for airport perimeter and access control security. Furthermore, a defined outcome that all security partners can work toward will better position TSA to provide reasonable assurance that it is taking the most appropriate steps for ensuring airport security.

Priorities

Our past work has also shown that the identification of program priorities in a strategy aids implementing parties in achieving results, which enables more effective oversight and accountability. Although TSA has implemented protective programs and activities that address risks to airport security, according to TSA officials it has not prioritized these activities nor has it yet aligned them with specific goals and objectives. TSA officials told us that in keeping with legislative mandates, they have focused agency resources on aviation security programs and activities that were of higher priority, such as passenger and baggage screening and air cargo security. Identifying priorities related to airport perimeter and access control security could assist TSA in achieving results within specified time frames and limited resources because it would allow the agency to concentrate on areas of greatest importance.

Performance Measures

In addition to our past work on national strategies, the NIPP and other federal guidance require agencies to assess whether their efforts are effective in achieving key security goals and objectives so as to help drive future investment and resource decisions and adapt and adjust protective efforts as risks change. Decision makers use performance measurement information, including activity outputs and descriptive information regarding program operations, to identify problems or weaknesses in individual programs, identify factors causing the problems, and modify services or processes to try to address problems. Decision makers can also use performance information collectively, and, according to the NIPP, examine a variety of data to provide a holistic picture of the health and effectiveness of a security approach from which to make security improvements. If significant limitations on performance measures exist, the strategy might address plans to obtain better data or measurements, such as national standards or indicators of preparedness.

TSA officials told us that TSA has not fully assessed the effectiveness of its protective activities for airport perimeters and secured areas, but they said that the agency has taken some steps to collect certain performance data for some airport security programs and activities to help inform programmatic decision making. For example, TSA officials told us that they require protective programs, such as ADASP and VIPR, to report certain output data and descriptive program information, which officials use to inform administrative or programmatic decisions. For ADASP, TSA requires FSDs to collect information on, among other things, the number of workers screened, vehicles inspected, and prohibited items surrendered. TSA officials said that they use these descriptive and output data to inform programmatic decisions, such as determining the number of staff days needed to support ADASP operations nationwide. However, TSA was not able to provide documentation on how such analysis has been conducted. For VIPR, officials said that they require team members to complete after-action reports that include data on the number of participants, locations, and types of activities conducted. TSA officials said that they are analyzing and categorizing this descriptive and output information to determine trends and identify areas of success and failure, which they will use to improve future operations, though they did not provide us with examples of how they have done this. TSA officials also told us that they require SPOT to report descriptive operations data and situational report information, which are to be used to assign necessary duties and correct problems with program implementation. However, TSA officials could not tell us how they use these descriptive and output data to inform program development and administrative decisions. While the use of descriptive and output data to inform program development and administration is both appropriate and valuable, leading management practices emphasize that successful performance measurement focuses on assessing the results of individual programs and activities.

TSA officials also told us that while they recognize the importance of assessing the effectiveness of airport security programs and activities in reducing known threats, it is difficult to do so because the primary purpose of these activities is deterrence. Assessing the deterrent benefits of a program is inherently challenging because it involves determining what would have happened in the absence of an intervention, or protective action, and it is often difficult to isolate the impact of the individual program on behavior that may be affected by multiple other factors. Because of this difficulty, officials told us that they have instead focused their efforts on assessing the extent to which each airport security activity supports TSA’s overall layered approach to security. We recognize that assessing the effectiveness of deterrence-related activities is challenging and that it continues to be the focus of ongoing analytic effort and policy review. For example, a January 2007 report by the Department of Transportation addressed issues related to measuring deterrence in the maritime sector, and a February 2007 report by the RAND Corporation acknowledged the challenges associated with measuring the benefits of security programs aimed at reducing terrorist risk. However, as a feature of TSA’s layered security approach, many of its airport activities address other aspects of security in addition to deterrence. Like other homeland security efforts, TSA’s airport security activities also seek to limit the potential for attack, safeguard critical infrastructure and property, identify wrongdoing, and ensure an effective and efficient response in the event of an attack; the desired outcome of its efforts is to reduce the risk of an attack. Deterrence is an inherent benefit of any protective action, and methods designed to detect wrongdoing and measures taken to safeguard critical infrastructure and property, for example, also help deter terrorist attacks. There are a number of activities that TSA has implemented that seek to reduce this risk, such as requiring security threat assessments for all airport workers. Some of these activities serve principally to deter, such as ADASP, while others are more focused on safeguarding critical infrastructure and property, such as conducting compliance inspections of aviation security regulations or installing perimeter fencing. Some activities serve multiple purposes, such as VIPR, which seeks to provide a visual deterrent to terrorist or other criminal activity, but also seeks to safeguard critical infrastructure in various modes of transportation. Examining the extent to which its activities have effectively addressed these various purposes would enable TSA to more efficiently implement and manage its programs.

There are several methods available that TSA could explore to gain insight on the extent to which its security activities have met their desired purpose and to ultimately improve program performance. For example, TSA could work with stakeholders, such as airport operators and other security partners, to identify and share lessons learned and best practices across airports to better tailor its efforts and resources and continuously improve security. TSA could also use information gathered through covert testing or compliance inspections—such as noncompliance or security breaches—to make adjustments to specific security activities and to identify which aspects require additional investigation. In addition, TSA could develop proxy measures—indirect measures or signs that approximate or represent the direct measure—to show how security efforts correlate to an improved security outcome. Appendix VII provides a complete discussion on these methods, as well as information on other alternatives TSA could explore.

TSA Has Identified Costs for Some Airport Security Activities, but Has Not Fully Identified Costs and Resource Needs, and Has Generally Not Conducted Cost-Benefit Analysis to Prioritize and Allocate Resources for Airport Security Activities

Our prior work shows that effective strategies address costs, resources, and resource allocation issues. Specifically, effective strategies address the costs of implementing the individual components of the strategy, the sources and types of resources needed (such as human capital or research and development), and where those resources should be targeted to better balance risk reductions with costs. Effective strategies may also address in greater detail how risk management will aid implementing parties in prioritizing and allocating resources based on expected benefits and costs. Our prior work found that strategies that provide guidance on costs and needed resources help implementing parties better allocate resources according to priorities, track costs and performance, and shift resources as appropriate.

Costs and Resources

Statutory requirements and federal cost accounting standards also stress the benefits of developing and reporting on the cost of federal programs and activities, as well as using that information to more effectively allocate resources and inform program management decisions. TSA has identified the costs and resources it needs for some specific activities and programs that exclusively support airport security, such as JVAs of selected commercial airports. However, for programs that serve airport security as well as other aspects of aviation security, TSA has not identified the costs and resources devoted to airport security. For example, TSA has identified its expenditures for compliance inspections and other airport security–related programs and activities, which collectively totaled nearly $850 million from fiscal years 2004 through 2008. However, TSA has not identified what portion of these funds was directly allocated for airport security activities versus other aviation security activities, such as passenger screening. (For a more detailed discussion of airport security costs, see app. IV.) Further, TSA has not fully identified the resources it needs to mitigate risks to airport perimeter and access control security. According to TSA officials, identifying collective agency costs and resource needs for airport security activities is challenging because airport security is not a separately funded TSA program, and many airport security activities are part of broader security programs. However, without attempting to identify total agency costs, it will be difficult for TSA to identify costs associated with individual security activities, and therefore it will be hindered in determining the resources it needs to sustain desired activity levels and realize targeted results. While TSA officials told us that they are starting to identify costs for airport security activities and plan to complete this effort by the end of 2009, they could provide no additional information to illustrate their approach for doing so. As a result, it is unclear what costs the agency will identify, and to what extent TSA will be able to identify costs for specific security activities in order to identify the resources it needs to sustain desired activity levels and realize targeted results.

TSA officials also told us that they have not yet identified or estimated costs to the aviation industry for implementing airport security requirements, such as background checks for their workers, or capital costs—such as construction and equipment—that airport operators incur to enhance the security of their facilities. According to these officials, the agency does not have the resources and funds to collect cost information from airport operators. However, TSA officials could not tell us how and to what extent they had assessed the resources and funds needed to collect this information or whether they had explored other options for collecting cost data, such as working with industry associations to survey airport operators. Estimating general cost information on the types and levels of resources needed for desired outcomes would provide TSA and other stakeholders with valuable information with which to make informed resource and investment decisions, including decisions about future allocation needs, to mitigate risks to airport security.

Prioritizing and Allocating Resources

According to our previous work on effective national strategies, as well as NIPP guidance, risk management focuses security efforts on those activities that bring about the greatest reduction in risk given the resources used. According to federal guidance, employing systematic cost-benefit analysis helps ensure that agencies choose the security priorities that most efficiently and effectively mitigate risk for the resources available. The Office of Management and Budget (OMB) cites cost-benefit analysis as one of the key principles to be considered when an agency allocates resources for capital expenditures because it provides decision makers with a clear indication of the most efficient alternative. DHS’s Cost-Benefit Analysis Guidebook also states that cost-benefit analysis identifies the superior financial solution among competing alternatives, and that it is a proven management tool to support planning and managing costs and risks. While TSA has made efforts to consider costs for some airport security programs, it has not used cost-benefit analysis to allocate or prioritize resources toward the most cost-effective alternative actions for mitigating risk.

According to TSA officials, certain factors have limited TSA’s ability to conduct cost-benefit analysis, such as resource constraints and the need to take immediate action to address new and emerging security threats. However, officials could not demonstrate that they had attempted to conduct cost-benefit analysis for programs and activities related to airport security within the constraints of current resources, or explain how, or to what extent, they had assessed the resources that would be needed to conduct cost-benefit analysis. Further, TSA officials could not cite a situation in which the need to take immediate action—outside of issuing security directives—in response to a threat prevented them from conducting cost-benefit analysis. TSA officials agreed that conducting cost-benefit analysis is beneficial, but also said that it is not always practical because of the difficulty in quantifying the benefits of deterrence- based activities. Because of this challenge, officials said that they have used professional judgment, past experience, law enforcement principles, and intelligence information to evaluate alternative airport security activities to mitigate risks. While TSA’s approach to identifying security actions includes accepted risk reduction decision-making tools, such as professional judgment, it does not provide a means to fully weigh the benefits versus the costs of implementing alternative actions. However, despite the challenges TSA cited to developing cost-benefit analysis, TSA officials told us that as of January 2009, the agency was in the early stages of investigating costs and benefits related to airport perimeter access control. According to these officials, TSA plans to initially focus on developing cost estimates associated with improving access control, a process the agency expects to complete by the end of 2009. However, because TSA officials did not explain how they expect to identify and estimate these costs and how, in the future, they plan to identify and estimate benefits for alternative actions, especially those actions that focus on deterrence, it is not yet clear to what extent TSA’s efforts will constitute cost-benefit analysis.

The use of systematic cost-benefit analysis when considering future airport security measures would help TSA to choose the most cost- effective security options for mitigating risk. We recognize the difficulties in quantifying the benefits of deterrence-based activities, but there are alternatives that TSA could pursue to assess benefits, such as examining the extent to which its activities address other purposes besides deterrence. Moreover, OMB recognizes that in some circumstances—such as when data are insufficient—costs and benefits cannot be quantified, in which case costs and benefits are to be assessed in qualitative terms. By exploring ways to identify expected costs associated with alternatives, and balancing these with estimated security benefits, TSA can more fully ensure that it is efficiently allocating and prioritizing its limited resources, as well as those of individual airports, in a way that maximizes the effectiveness of its airport security efforts.

TSA Has Collaborated with Stakeholders regarding Airport Security Activities, but Has Not Always Fully Coordinated or Integrated Airport Security with Other Aspects of Aviation Security

Our prior work shows that effective national strategies address how to coordinate efforts and resolve conflicts among stakeholders, address ways in which each strategy relates to the goals of other strategies, and devise plans for implementing the strategies. Because the responsibility for airport perimeter and access control security involves multiple stakeholders, including federal entities, individual airport operators, air carriers, and industry organizations, coordination among stakeholders is critical. In such an environment, the implementation of security activities is strengthened when a strategy addresses how federal efforts will coordinate and integrate with other federal and private sector initiatives, relate to the goals and objectives of other strategies and plans, and be implemented and coordinated by relevant parties.

Coordination

Representatives from industry associations told us that while TSA has collaborated with industry stakeholders on the development of multiple airport security activities and initiatives, the agency has not always fully coordinated the development and implementation of specific security activities and initiatives. For example, although TSA has worked with the industry in the development of some aspects of airport security technology, such as biometrics, industry association officials told us that the agency has not yet recommended specific technology based on the results of technology-based pilot programs it completed over 2 years ago in 2007. These officials also noted that TSA did not fully coordinate with the industry in its decision to impose stronger requirements on worker credentialing practices in the wake of security incidents at individual airports. TSA officials said that they have worked closely with industry stakeholders in addressing airport security issues, and have established working groups to continue to coordinate on issues such as biometric access control security. Our prior work found that a strategy should provide both direction and guidance to government and private entities so that missions and contributions can be more appropriately coordinated.

Integration and Implementation

TSA has not demonstrated how it relates the activities of airport security to the goals, objectives, and activities of TSA’s other aviation security strategies, such as passenger screening, air cargo screening, and baggage screening. In addition, TSA has not identified how these various security areas are coordinated at the national level. For example, TSA officials told us that some security efforts, such as the random worker screening program and roving security response teams, are used to address multiple security needs, such as both passenger and worker screening, but could not identify the extent to which program resources are planned for and applied between competing security needs. TSA officials said that decisions to allocate random worker screening resources between passenger and worker screening are made at the local airport level by FSDs. However, a clear understanding of how TSA’s needs and goals for airport security align with those of its other security responsibilities would enable the agency to better coordinate its programs, gauge the effectiveness of its actions, and allocate resources to its highest-priority needs. Finally, it is not clear to what extent TSA has coordinated airport security activities within the agency, the responsibilities for which are spread among multiple offices. TSA officials explained that agency efforts to enhance and oversee airport perimeter and access control security are spread across multiple programs within five TSA component offices. No one office or program has responsibility for coordinating and integrating actions that affect the numerous aspects of perimeter and access control security, including operations, technology, intelligence, program policy, credentialing, and threat assessments. TSA officials agreed that the diffusion of responsibilities across offices can present coordination challenges. Developing an overarching, integrated framework for coordinating actions between implementing parties could better position TSA to avoid unnecessary duplication, overlap, and conflict in the implementation of these actions. According to our past work, strategies that provide guidance to clarify and link the roles, responsibilities, and capabilities of the implementing parties can foster more effective implementation and accountability.

Conclusions

Commercial airports facilitate the movement of millions of passengers and tons of goods each week and are an essential link in the nation’s transportation network. Given TSA’s position that the interconnected commercial airport network is only as strong as its weakest asset, determining vulnerability across this network is fundamental to determining the actions and resources that are necessary to reasonably protect it. Evaluating whether existing, select vulnerability assessments reflect the network of airports will help TSA ensure that its actions strengthen the whole airport system. If TSA finds that additional assessments are needed to identify the extent of vulnerabilities nationwide, then developing a plan with milestones for conducting those assessments, and leveraging existing available assessment information from stakeholders, would help ensure the completion of these assessments and that intended results are achieved. In addition, although the consequences of a successful terrorist breach in airport security have not been assessed, based on the past events, the potential impact on U.S. assets, safety, and public morale could be profound. For this reason, assessing the likely consequences of an attack is an essential step in assessing risks to the nation’s airports. Further, a comprehensive risk assessment that combines threat, vulnerability, and consequence would help TSA determine which risks should be addressed—and to what degree—and would help guide the agency in identifying the necessary resources for addressing these risks. Moreover, documenting milestones for completing the risk assessment would help ensure its timely completion.

Implementing and evaluating a pilot program can be challenging, especially given the individual characteristics of the sites involved in the worker screening pilot, such as the variation in airport size, traffic flows, and layouts. However, a well-developed and documented evaluation plan, with well-defined and measurable objectives and standards as well as a clearly articulated methodology and data analysis plan, can help ensure that a pilot program is implemented and evaluated in ways that generate reliable information to inform future program development decisions. By making such a plan a cornerstone of future pilot programs, TSA will be better able to ensure that the results of those pilot programs will produce the reliable data necessary for making the best program and policy decisions.

Integrating biometric technology into existing airport access control systems will not be easy given the range of technologies available, the number of stakeholders involved, and potential differences in the biometric controls already in use at airports. Yet Congress, the administration, and the aviation industry have emphasized the need to move forward in implementing such technology to better control access to sensitive airport areas. But until TSA decides whether, when, and how it will mandate biometric access controls at airports, individual airport operators will likely continue to delay investing in potentially costly technology in case it does not comply with future federal standards. Establishing milestones for addressing requirements would not only provide airports with the necessary information to appropriately plan future security upgrades, but give all stakeholders a road map by which they can anticipate future developments.

TSA uses security directives as a means for establishing additional security measures in response to general or specific threats against the civil aviation system, including the security of airport perimeters and the controls that limit access to secured airport areas. Just as it is important that federal agencies have flexible mechanisms for responding to the adaptive, dynamic nature of the terrorist threat, it is also important that requirements remain consistent with current threat information. Establishing milestones for periodically reviewing airport perimeter and access control requirements imposed through security directives would help provide TSA and stakeholders with reasonable assurance that TSA’s personnel will review these directives within a time frame authorized by management.

TSA, along with industry partners, has taken a variety of steps to implement protective measures to strengthen airport security, and many of these efforts have required numerous stakeholders to implement a range of activities to achieve desired results. These various actions, however, have not been fully integrated and unified toward achieving common outcomes and effectively using resources. A national risk-informed strategy—that establishes measurable goals, priorities, and performance measures; identifies needed resources; and is aligned and integrated with related security efforts—would help guide decision making and hold all public and private security partners accountable for achieving key shared outcomes within available resources. Moreover, a strategy that identifies these key elements would allow TSA to better articulate its needs—and the challenge of meeting those needs—to industry stakeholders and to Congress. Furthermore, balancing estimated costs against expected security benefits, and developing measures to assess the effectiveness of security activities, would help TSA provide reasonable assurance that it is properly allocating and prioritizing its limited resources, or those of airports, in a way that maximizes the effectiveness of its airport security efforts.

Recommendations for Executive Action

To help ensure that TSA’s actions in enhancing airport security are guided by a systematic risk management approach that appropriately assesses risk and evaluates alternatives, and that it takes a more strategic role in ensuring that government and stakeholder actions and resources are effectively and efficiently applied across the nationwide network of airports, we recommend that the Assistant Secretary of TSA work with aviation stakeholders to implement the following five actions: Develop a comprehensive risk assessment for airport perimeter and access control security, along with milestones (i.e., time frames) for completing the assessment, that (1) uses existing threat and vulnerability assessment activities, (2) includes consequence analysis, and (3) integrates all three elements of risk—threat, vulnerability, and consequence.

As part of this effort, evaluate whether the current approach to conducting JVAs appropriately and reasonably assesses systems vulnerabilities, and whether an assessment of security vulnerabilities at airports nationwide should be conducted.

If the evaluation demonstrates that a nationwide assessment should be conducted, develop a plan that includes milestones for completing the nationwide assessment. As part of this effort, leverage existing assessment information from industry stakeholders, to the extent feasible and appropriate, to inform its assessment.

Ensure that future airport security pilot program evaluation and implementation efforts include a well-developed and well-documented evaluation plan that includes criteria or standards for determining program performance, a clearly articulated methodology, a detailed data collection plan, and a detailed data analysis plan.

Develop milestones for meeting statutory requirements, in consultation with appropriate aviation industry stakeholders, for establishing system requirements and performance standards for the use of biometric airport access control systems.

Develop milestones for establishing agency procedures for reviewing airport perimeter and access control requirements imposed through security directives.

To better ensure a unified approach among airport security stakeholders for developing, implementing, and assessing actions for securing airport perimeters and access to controlled areas, develop a national strategy for airport security that incorporates key characteristics of effective security strategies, including the following: Measurable goals, priorities, and performance measures. TSA should also consider using information from other methods, such as covert testing and proxy measures, to gauge progress toward achieving goals.

Program cost information and the sources and types of resources needed. TSA should also identify where those resources would be most effectively applied by exploring ways to develop and implement cost-benefit analysis to identify the most cost-effective alternatives for reducing risk.

Plans for coordinating activities among stakeholders, integrating airport security goals and activities with those of other aviation security priorities, and implementing security activities within the agency.

Agency Comments and Our Evaluation

We provided a draft of our report to DHS and TSA on August 3, 2009, for review and comment. On September 24, 2009, DHS provided written comments, which are reprinted in appendix VIII. In commenting on our report, DHS stated that it concurred with all five recommendations and identified actions planned or under way to implement them.

In its comments to our draft report, DHS stated that the Highlights page of our report includes a statement that is inaccurate. We disagree. Specifically, DHS contends that it is not accurate to state that TSA “has not conducted vulnerability assessments for 87 percent of the nation’s 450 commercial airports” because this statement does not recognize that TSA uses other activities to assess airport vulnerabilities, and that these activities are conducted for every commercial airport. For example, DHS stated that (1) every commercial airport must have a TSA-approved ASP, which is to cover personnel, physical, and operational security measures; (2) each ASP is reviewed on a regular basis by a FSD; and (3) such FSD reviews “include a review of security measures applied at the perimeter.” As we noted in our report, TSA identified JVAs, along with professional judgment, as the agency’s primary mechanism for assessing airport security vulnerabilities in accordance with NIPP requirements. Moreover, it is not clear to what extent the FSD reviews and other activities TSA cites in its comments address airport perimeter and access control vulnerabilities or to what extent such reviews have been applied consistently on a nationwide basis, since TSA has not provided us with any documentary evidence regarding these or other reviews. Finally, in meeting with TSA, its officials acknowledged that because they have not conducted a joint vulnerability assessment for 87 percent of commercial airports, they do not know how vulnerable these airports are to an intentional breach in security or an attack. Thus, we consider the statement on our Highlights page to be accurate. TSA also stated that “as provided in our draft report” the foundation of TSA’s national strategy is its individual layers—or actions—of security, which, when combined, generate an exponential increase in deterrence and detection capability. However, we did not evaluate TSA’s layered approach to security or the extent to which this approach provides increased deterrence and detection capabilities.

Regarding our first recommendation that TSA develop a comprehensive risk assessment for airport perimeter and access control security, DHS stated that TSA will develop such an assessment through its ongoing efforts to conduct a comprehensive risk assessment for the transportation sector. TSA intends to provide the results of the assessment to Congress by January 2010. According to DHS, the aviation domain portion of the sector risk assessment is to address, at the national level, nine airport perimeter and access control security scenarios. It also stated that the assessment is to integrate all three elements of risk—threat, vulnerability and consequence—and will rely on existing assessment activities, including JVAs. In developing this assessment, it will be important that TSA evaluate whether its current approach to conducting JVAs, which it identifies as one element of its risk assessment efforts, appropriately assesses vulnerabilities across the commercial airport system, and whether additional steps are needed. Since TSA has repeatedly stated the need to develop baseline data on airport security vulnerabilities to enable it to conduct systematic analysis of vulnerabilities on a nationwide basis, TSA could also benefit from exploring the feasibility of leveraging existing assessment information from industry stakeholders to inform this assessment.

DHS also agreed with our second recommendation that a well-developed and well-documented evaluation plan should be part of TSA’s efforts to evaluate and implement future airport security pilot programs. In addition, DHS concurred with our third recommendation that TSA develop milestones for meeting statutory requirements for establishing system requirements and performance standards for the use of biometric airport access control systems. DHS noted that while mandatory use of such systems is not required by statute, TSA is still considering whether it will mandate the use of biometric access control systems at airports, and in the meantime it will continue to encourage airport operators to voluntarily utilize biometrics in their access control systems. We agree that mandatory use of biometric access control systems is not required by statute, but establishing milestones would help guide TSA’s continued work with the airport industry to develop and refine existing biometric access control standards. In regard to our fourth recommendation that TSA develop milestones for establishing agency procedures for reviewing airport security requirements imposed through security directives, DHS concurred that milestones are necessary.

Finally, in regard to our fifth recommendation that TSA develop a national strategy for airport security that incorporates key characteristics of effective security strategies, DHS concurred and stated that TSA will develop a national strategy by updating the TS-SSP. DHS stated that TSA intends to solicit input on the plan from its Sector Coordinating Council, which represents key private sector stakeholders from the transportation sector, before releasing the updated TS-SSP in the summer of 2010. However, given that the TS-SSP is to focus on detailing how the NIPP framework will apply to the entire transportation sector, it may not be the most appropriate vehicle for developing a national strategy that addresses the various management issues specific to airport security that we identified in our report. A more effective approach might be to issue the strategy as a stand-alone plan, in keeping with the format TSA has used for its air cargo, passenger checkpoint screening, and SPOT strategies. A stand-alone strategy might better facilitate key stakeholder involvement, focus attention on airport security needs, and allow TSA to more thoroughly address relevant challenges and goals. But irrespective of the format, it will be important that TSA fully address the key characteristics of an effective strategy, as identified in our report. The intent of a national strategy is to provide a unifying framework that guides and integrates stakeholder activities toward desired results, which may be best achieved when planned efforts are clear and sustainable, and transparent enough to ensure accountability. Thus, it is important that the strategy fully incorporate the following characteristics: (1) measurable goals, priorities, and performance measures; (2) program cost information, including the sources and types of resources needed; and (3) plans for coordinating activities among stakeholders, integrating airport security goals and activities with those of other aviation security priorities, and implementing security activities within the agency.

TSA also provided us with technical comments, which we considered and incorporated in the report where appropriate.

We are sending copies of this report to the Secretary of Homeland Security, the Secretary of Transportation, the Assistant Secretary of the Transportation Security Administration, appropriate congressional committees, and other interested parties. The report also is available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any further questions about this report or wish to discuss these matters further, please contact me at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix IX.

Appendix I: Objectives, Scope, and Methodology

This report evaluates to what extent the Transportation Security Administration (TSA) has assessed the risk to airport security consistent with the National Infrastructure Protection Plan’s (NIPP) risk management framework; implemented protective programs to strengthen airport security, and evaluated its worker screening pilot program; and established a national strategy to guide airport security decision making.

To evaluate the extent to which TSA has assessed risks for airport perimeter and access control security efforts, we relied on TSA to identify risk assessment activities for these areas, and we then examined documentation for these activities, such as TSA’s 2008 Civil Aviation Threat Assessment, and interviewed TSA officials responsible for conducting assessment efforts. We examined the extent to which TSA generally conducted activities intended to assess threats, vulnerabilities, and consequences to the nation’s approximately 450 airports. We also reviewed the extent to which TSA’s use of these three types of assessments met the NIPP criteria for completing a comprehensive risk assessment. However, while we assessed the extent to which the individual threat and vulnerability assessment activities that TSA identified addressed the area of airport perimeter and access controls, the scope of our work did not include individual evaluations of these activities to determine whether they were consistent with the NIPP criteria for conducting threat and vulnerability assessments. In addition, we reviewed and summarized critical infrastructure and aviation security requirements set out by Homeland Security Presidential Directives 7 and 16, the Aviation and Transportation Security Act (ATSA), and other statutes and related materials. We also examined the individual threat and vulnerability assessment activities and discussed them with senior TSA and program officials, to evaluate how TSA uses this information to set goals and inform its decision making. We compared this information with the NIPP, TSA’s Transportation Security Sector-Specific Plan, and our past guidance and reports on recommended risk management practices. In addition, we obtained and analyzed data from TSA regarding joint vulnerability assessments, which are conducted with the Federal Bureau of Investigation (FBI), to determine the extent to which TSA has used this information to assess risk to airport perimeter and access control security. We also obtained information on the processes used to schedule and track these activities to determine the reliability with which these data were collected and managed, and we determined that the data were sufficiently reliable for the purposes of this report. We interviewed TSA and FBI officials responsible for conducting joint vulnerability assessments to discuss the number conducted by TSA since 2004, the scope of these assessments, and how they are conducted.

In addition, we interviewed selected TSA officials responsible for risk management and security programs related to airport perimeter and access control to clarify the extent to which TSA has assessed risk in these areas. We selected these officials based upon their relevant expertise with TSA’s risk management efforts and its airport perimeter and access control efforts. We also analyzed TSA data on security breaches by calculating the total number of security breaches from fiscal years 2004 through 2008. To determine that the data were sufficiently reliable to present contextual information regarding all breaches to secured areas (including airport perimeters) in this report, we obtained information on the processes used to collect, tabulate, and assess these data, and discussed data quality control procedures with appropriate officials and found that the data were sufficiently reliable for this purpose. Because the data include security breaches that occurred within any type of secured areas, including passenger-related breaches, they are not specific to perimeter and access control security. In addition, the data have not been adjusted to reflect potential issues that could also influence or skew the number of overall breaches, such as annual increases in the number of passengers or specific incidences occurring within individual airports that account for more breaches than others. Furthermore, because TSA does not require its inspectors to enter a description of the breach when documenting an incident, and general reports on breach data do not show much variation between incidences unless a report includes a description of the breach, we did not ask TSA for descriptive information on breaches that occurred.

To evaluate the extent to which TSA has implemented protective programs to strengthen airport security consistent with the NIPP risk management framework, we asked TSA to identify agency-led activities and programs for strengthening airport security. For the purposes of this report, we categorized TSA’s responses into four main areas of effort: (1) worker screening pilot program, (2) worker security programs, (3) technology, and (4) general airport security. To determine the extent to which TSA evaluated its worker screening pilot program, we analyzed TSA’s final report on it worker screening pilot program, including conclusions and limitations cited by the contractor—the Homeland Security Institute (HSI)—TSA hired to assist with the pilot’s design, implementation, and evaluation. We also reviewed standards for internal control in the federal government and our previous work on pilot program development and evaluation to identify accepted practices for ensuring reliable results, including key features of a sound evaluation plan. Further, we analyzed TSA and HSI’s documentation of the worker screening pilot program methodology to determine whether TSA and HSI had documented their plans for conducting the program, whether each pilot was carried out in a consistent manner, and if participating airports were provided with written requirements or guidance for conducting the pilots. To evaluate TSA’s efforts for its worker security programs, we assessed and summarized relevant program information, operations directives, and standard operating procedures for the Aviation Direct Access Screening Program (ADASP) and enhanced background checks. We also informed this assessment with recent work by the Department of Homeland Security’s (DHS) Office of the Inspector General (OIG) regarding worker screening. We reviewed the DHS OIG’s methodology and analysis to determine whether its findings were reliable for use in our report. We analyzed TSA’s documentation of its background checks to determine if TSA sufficiently addressed relevant ATSA requirements and recommendations from our 2004 report on airport security. We also interviewed TSA officials responsible for worker background checks to determine the agency’s efforts to develop a plan to meet outstanding ATSA requirements.

With respect to perimeter and access control technology, we reviewed and summarized TSA documentation and evaluations of the Airport Access Control Pilot Program (AACPP), documentation related to the Airport Perimeter Security (APS) pilot program, and the dissemination of information regarding technology to airports. We interviewed officials with the DHS Directorate for Science and Technology, the National Safe Skies Alliance, and RTCA, Inc., regarding research, development, and testing efforts, and challenges and potential limitations of applicable technologies to airport perimeter and access control security. We selected these entities because of their role in the development of such technology. We also interviewed TSA Headquarters officials to obtain views on the nature and scope of technology-related efforts and other relevant considerations, such as how they addressed relevant ATSA requirements and recommendations from our 2004 report, or how they plan to do so. With regard to TSA’s efforts for general airport security, we examined TSA’s procedures for developing and issuing airport perimeter and access control requirements through security directives and other methods, and analyzed the extent to which TSA disseminated security requirements to airports through security directives. At our request, TSA identified 25 security directives and emergency amendments that imposed requirements related to airport perimeter and access control security, which we examined to identify specific areas of regulation. In addition, we assessed and summarized relevant program information and documentation, such as operations directives, for other programs identified by TSA, such as the Visible Intermodal Prevention and Response (VIPR) program, Screening of Passengers by Observation Techniques (SPOT) program, and the Law Enforcement Officer Reimbursement Program.

To evaluate the extent to which TSA established a national strategy to guide airport security decision making, we considered guidance on effective characteristics for security strategies and planning that we previously reported, Government Performance and Results Act (GPRA) requirements, and generally accepted strategic planning practices for government agencies. In order to evaluate TSA’s approach to airport security, we reviewed TSA documents to identify major security goals and subordinate objectives for airport perimeter and access control security, and relevant priorities, goals, objectives, and performance measures. We also analyzed relevant program documentation, including budget, cost, and performance information, including relevant information TSA developed and maintains for the Office of Management and Budget’s Performance Assessment Rating Tool. We compared TSA’s approach with criteria identified in NIPP, other DHS guidance, GPRA, and other leading practices in strategies and planning. We also interviewed relevant TSA program and budget officials, Federal Aviation Administration (FAA) officials, and selected aviation industry officials regarding the cost of airport perimeter and access control security for fiscal years 2004 through 2008.

To determine the extent to which TSA collaborated with stakeholders on airport security activities, and to obtain their insights on airport security operations, costs, and regulation, we interviewed industry officials from the Airports Council International-North America—whose commercial airport members represent 95 percent of domestic airline passenger and air cargo traffic in North America—and from the American Association of Airport Executives—whose members represent 850 domestic airports. We selected these industry associations based on input from TSA and from industry stakeholders, who identified the two associations representing commercial airport operators. We also attended aviation association conferences at which industry officials presented information on national aviation security policy and operations, and we conducted a group discussion with 17 officials representing various airport and aircraft operators and aviation associations to obtain their views regarding key issues affecting airport security. While the views expressed by these industry, airport, and aircraft operator officials cannot be generalized to all airport industry associations and operators, these interviews provided us with additional perspectives on airport security and an understanding of the extent to which TSA has worked and collaborated with airport stakeholders.

We also conducted site visits at nine U.S. commercial airports—Orange County John Wayne Airport, Washington-Dulles International Airport, Miami International Airport, Orlando International Airport, John F. Kennedy International Airport, Westchester County Airport, Logan International Airport, Barnstable Municipal Airport, and Salisbury/Wicomico County Regional Airport. During these visits we observed airport security operations and discussed issues related to perimeter and access control security with airport officials and on-site TSA officials, including federal security directors (FSD). We selected these airports based on several factors, including airport category, size, and geographical dispersion; whether they faced problems with perimeter and access control security; and the types of technological initiatives tested or implemented. Because we selected a nonprobability sample of airports to visit, those results cannot be generalized to other U.S. commercial airports; however, the information gathered provides insight into TSA and airport programs and procedures. In addition, at Miami International Airport and John F. Kennedy International Airport we conducted separate interviews with airport officials to discuss their ongoing, or anticipated, efforts to implement additional worker screening methods at their respective airports. We also conducted telephone interviews with airport officials and FSDs from four airports that had implemented, or planned to implement, various forms of 100 percent screening of airport workers to discuss their efforts. These were Cincinnati/Northern Kentucky International Airport, Dallas/Fort Worth International Airport, Denver International Airport, and Phoenix Sky Harbor International Airport. While the views of the officials we spoke with regarding additional worker screening methods cannot be generalized to all airport security officials, they provided insight into how airport security programs were chosen and developed. We also conducted an additional site visit at Logan International Airport to observe TSA’s implementation of various worker screening methods as part of the agency’s worker screening pilot program. While the experiences of this pilot location cannot be generalized to all airports participating in the pilot, we chose this airport based on airport category and the variety of worker screening methods piloted at this location.

We conducted this performance audit from May 2007 through September 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: TSA Actions to Address Selected Statutory Requirements for Airport Security

TSA has taken steps since 2004 to address some of the requirements related to airport perimeter and access control security prescribed by ATSA. The related ATSA requirements, and TSA’s actions as of May 2009 to address these requirements, are summarized in table 3.

Appendix III: TSA Also Uses Compliance Inspections and Covert Testing to Detect Possible Airport Security Vulnerabilities

TSA officials told us that they use the results of compliance inspections and covert testing to augment their assessment of potential vulnerabilities in airport security. Compliance inspections examine a regulated entity’s— such as an airport operator or air carrier—adherence to federal regulations, which TSA officials say they use to determine if airports adequately address known threats and vulnerabilities. According to TSA, while regulatory compliance is just one dimension of airport security, compliance with federal requirements allows TSA to determine the general level of security within an airport. As a result, according to TSA, compliance with regulations suggests less vulnerability within an airport and, conversely, failure to meet critical compliance rates suggests the likelihood of a larger problem within an airport and helps the agency identify and assess vulnerabilities. TSA allows its inspectors to conduct compliance inspections based on observations of various activities, such as ADASP, VIPR, and local covert testing, and to conduct additional inspections based on vulnerabilities identified through assessments or the results of regular inspections.

Covert tests are any test of security systems, personnel, equipment, and procedures to obtain a snapshot of the effectiveness of that security measure, and they are used to improve airport performance, safety, and security. TSA officials stated that covert testing assists the agency in identifying airport vulnerabilities because such tests are designed based on threat assessments and intelligence to approximate techniques that terrorists may use to exploit gaps in airport security. TSA conducts four types of covert tests for airport access controls: Access to security identification display areas (SIDA): TSA inspectors not wearing appropriate identification attempt to penetrate SIDA access points, such as boarding gates, employee doors, and other entrances.

Access to air operations areas (AOA): TSA inspectors not wearing appropriate identification attempt to penetrate AOA via access points from public areas, such as perimeter gates and cargo areas.

Access to aircraft: TSA inspectors not wearing appropriate identification (or not carrying valid boarding passes) attempt to penetrate passenger access points that lead to aircraft from sterile areas, such as boarding gates, employee doors, and jet ways.

SIDA challenges: Once inside a SIDA, TSA inspectors attempt to walk around these areas, such as the tarmac and baggage loading areas, without displaying appropriate identification.

TSA also requires FSDs to conduct similar, locally controlled tests of access controls to ensure compliance and identify possible vulnerabilities with airport security. These tests are selected by the FSDs and based on locally identified risks and can include challenging procedures in the secure area, piggybacking (following authorized airport workers into secured areas), and attempting to access an aircraft from sterile area.

According to TSA officials, the agency uses the results of its covert tests to inform decision making for airport security, but officials could not provide examples of how this information has specifically informed past decisions.

Appendix IV: Costs for Airport Security

Various TSA offices and programs contribute to the overall operations and costs of airport perimeter and access control security. According to TSA officials, the agency does not develop a cost estimate specific to perimeter and access control security because such efforts are often part of broader security activities or related programs—for example, VIPR and SPOT are also used for passenger screening. As a result, it is difficult to identify what percentage of program costs has been expended on airport perimeter and access control security activities. At our request, TSA officials identified the estimated spending related to perimeter and access control security programs from fiscal years 2004 through 2008 (see table 4).

Airports can receive funding for purposes related to perimeter and access control security via grants awarded through FAA’s Airport Improvement Program. TSA officials also told us that the agency generally does not collect or track cost information for airport security efforts funded through the Airport Improvement Program. This program is one of the principal sources of funding for airport capital improvements in the United States, providing approximately $3 billion in grants annually to enhance airport capacity, safety, and environmental protection, as well as perimeter security. According to FAA officials, many factors are considered when awarding grants to airports for perimeter security enhancements, although security projects required by statute or regulation receive the highest priority. Projects that receive funding have included computerized access controls for ramps, infrastructure improvements to house central computers, surveillance systems, and perimeter fencing. According to FAA, more than $365 million in airport perimeter and access control–related grants were provided through the Airport Improvement Program for fiscal years 2004 through 2008.

TSA officials also told us that the agency does not track funds spent by individual airport operators to enhance or maintain perimeter and access control security. In 2009 the Airports Council International-North America—an aviation industry association—surveyed commercial airports regarding the funding needed for airport capital projects from 2009 to 2013. As part of this effort, the association surveyed airport operators on the amount of funds they planned to expend on airport security as a percentage of their overall budgets. The association reported that planned airport operator spending on airport security, as a percentage of total spending, ranged from 3.8 percent (about $2 billion) for large hub airports to 3.9 percent (about $230 million) for small hub airports. The association surveys did not include information on the types of security projects undertaken by airports. However, during our site visits we obtained data from selected airport operators on the costs of perimeter and access control security projects they had recently concluded or estimated costs for projects in progress. Examples of airport spending on perimeter and access control security include $30 million to install a full biometric access system; $6.5 million to install an over 8,000-foot-long blast/crash resistant wall along the airport perimeter; $8 million to install over 680 bollards in front of passenger terminals and vehicle access points; and $3 million to develop and install an infrared intrusion detection system.

Appendix V: TSA Worker Screening Pilot Program

From May through July 2008 TSA implemented worker screening pilots at seven airports in accordance with the Explanatory Statement accompanying the DHS Appropriations Act, 2008 (see table 5 for a summary of text directing the worker screening pilot program). At three airports, TSA conducted 100 percent worker screening—inspections of all airport workers and vehicles entering secure areas; at four others TSA randomly screened 20 percent of workers and tested other enhanced security measures. Screening of airport workers was to be done at either the airport perimeter or the passenger screening checkpoints. TSA was directed to collect data on the methods it utilized, and evaluate the benefits, costs, and impacts of 100 percent worker screening to determine the most effective and cost-efficient method of addressing and deterring potential security risks posed by airport workers.

The enhanced measures that TSA tested at the four airports not implementing 100 percent screening are summarized below: Employee training: TSA provided a security awareness training video, which all SIDA badgeholders were required to complete. According to TSA, the training intended reduce security breaches by increasing workers’ understanding of their security responsibilities and awareness of threats and abnormal behaviors.

Behavioral recognition training: TSA provided funding to participating airports to teach select law enforcement officers and airport personnel to identify potentially high-risk individuals based on their behavior. A condensed version of the SPOT course, this training was intended to equip personnel with skills to enhance existing duties, according to TSA officials.

Targeted physical inspections: TSA conducted random inspections of vehicles and individuals entering the secured areas of airports to increase the coverage of ADASP. Inspections consisted of bag, vehicle, and identification checks; scanning bottled liquids; and random security sweeps of specific airport areas.

Deployment of technology: TSA employed additional technology at selected airports to assist with the screening of employees, such as walk- through and handheld metal detectors, bottled liquid scanners, and explosive detection systems. TSA also tested biometric access control systems at selected airports.

Appendix VI: Additional TSA Efforts to Improve General Airport Security

According to TSA, VIPR operations augment existing airport security activities, such as ADASP, and provide a visual deterrent to terrorist or other criminal activity. VIPR was first implemented in 2005, and according to TSA officials, VIPR operations are deployed through a risk-based approach and in response to specific intelligence information or known threats. In a VIPR operation, TSA officials, including transportation security officers and inspectors, behavioral detection officers, bomb appraisal officers, and federal air marshals work with local law enforcement and airport officials to temporarily enhance aviation security. According to TSA officials, VIPR operations for perimeter and access control security can include random inspections of individuals, property, and vehicles, as well as patrols of secured areas and random checks to ensure that employees have the proper credentials. TSA officials told us that although they do not know how many VIPR deployments have specifically addressed airport perimeter and access control security, from March 2008 through April 2009 TSA performed 1,042 commercial and general aviation airport or cargo VIPR operations. According to TSA officials, the majority of these operations involved the observation and patrolling of secured airport areas and airport perimeters. As of May 2009 TSA officials also said that the agency is in the process of enhancing its VIPR database to more accurately capture and track specific operational objectives, such as enhancing the security of airport perimeters and access controls, and developing an estimated time frame for completing this effort.

Since 2004 TSA has used SPOT—a passenger screening program in which behavior detection officers observe and analyze passenger behavior to identify potentially high-risk individuals—to determine if an individual or individuals may pose a risk to aircraft or airports. Although SPOT was originally designed for passenger screening, TSA officials stated that FSDs can also use behavior detection officers to assess worker behavior as they pass through the passenger checkpoint, as part of random worker screening operations or as part of VIPR teams deployed at an airport. However, TSA officials could not determine how often behavior detection officers have participated in random worker screening or VIPR operations, or identify which airports have used behavior detection officers for random worker screening. According to TSA officials, the agency is in the process of redesigning its data collection efforts and anticipates that it will be able to more accurately track this information in the future, though officials did not provide a time frame for doing so. TSA officials also told us that when participating in random worker screening, behavior detection officers observe workers for suspicious behavior as they are being screened and may engage workers in casual conversation to assess potential threats. According to TSA officials, the agency has provided behavior detection training to law enforcement personnel as part of its worker screening pilot program, as well as to selected airport security and operations personnel at more than 20 airports. We currently have ongoing work assessing SPOT, and will issue a report on this program at a later date.

Law Enforcement Officer Reimbursement Program

TSA undertakes efforts to facilitate the deployment of law enforcement personnel authorized to carry firearms at airport security checkpoints, and in April 2002, the Law Enforcement Officer Reimbursement Program was established to provide partial reimbursement for enhanced, on-site law enforcement presence in support of the passenger screening checkpoints. Since 2004, the program has expanded to include law enforcement support along the perimeter and to assist with worker screening. According to TSA, the program is implemented through a cooperative agreement process that emphasizes the ability of both parties to identify and agree as to how law enforcement officers will support the specific security requirements at an airport. For example, the FSD, in consultation with the airport operator and local law enforcement, may determine that rather than implementing fixed-post stationing of law enforcement officers, it may be more appropriate to implement flexible stationing of law enforcement officers. TSA may also provide training or briefings on an as- needed basis on relevant security topics, including improvised explosive device recognition, federal criminal statutes pertinent to aviation security, and procedures and processes for armed law enforcement officers. Awards made under the reimbursement program are subject to the availability of appropriated funds, among other things, and are to supplement not supplant state and local funding. According to TSA officials, however, no applicant has been denied funds based on lack of appropriated funds.

Appendix VII: Alternative Methods Available to Assist TSA in Assessing the Effectiveness of Its Actions to Strengthen Airport Security

Program evaluation methods exist whereby TSA could attempt to assess whether its activities are meeting intended objectives. These methods center on reducing the risk of both external and internal threats to the security of airport perimeters and access controls, and seek to use information and resources available to help capture pertinent information.

First, recognizing that there are challenges associated with measuring the effectiveness of deterrence-related activities, the NIPP’s Risk Management Framework provides mechanisms for qualitative feedback that although not considered a metric, could be applied to augment and improve the effectiveness and efficiency of protective programs and activities. For example, working with stakeholders—such as airport operators and other security partners—to identify and share lessons learned and best practices across airports could assist TSA in better tailoring its efforts and resources and continuously improving security. Identifying a range of qualitative program information—such as information gathered through vulnerability assessment activities or compliance inspections—could also allow TSA to determine whether activities are effective. As discussed in appendix III, compliance inspections and covert tests could be used to identify noncompliance with regulations or security breaches within designated secured areas. For example, TSA could use covert tests to determine if transportation security officers are following TSA procedures when screening airport workers or whether certain worker screening procedures detect prohibited items. However, in order to improve the usefulness of this technique, we previously recommended to TSA that the agency develop a systematic process for gathering and analyzing specific causes of all covert testing failures, record information on processes that may not be working properly during covert tests, and identify effective practices used at airports that perform well on covert tests.

Second, as TSA has already begun to do with some activities, it could use data it already collects to identify trends and establish baseline data for a future comparison of effectiveness. For example, a cross-sectional analysis of the number of workers caught possessing prohibited items at specific worker screening locations over time, while controlling for variables such as increased law enforcement presence or airport size, could provide insights into what type of security activities help to reduce the possession of prohibited items. Similarly, an examination of airport workers apprehended, fired, or referred to law enforcement while on the job could provide insights into the quality of worker background checks and security threat assessments. Essentially, the these types of analyses provide a useful context for drawing conclusions about whether certain security practices are reasonable and appropriate given certain conditions and, gradually, with the accumulation of relevant data, should allow TSA to start identifying cause-and-effect relationships.

Third, according to the Office of Management and Budget (OMB), the use of proxy measures may also allow TSA to determine how well its activities are functioning. Proxy measures are indirect measures or indicators that approximate or represent the direct measure. TSA could use proxy measures to address deterrence, other security goals as identified above, or a combination of both. According to OMB, proxy measures are to be correlated to an improved security outcome, and the program should be able to demonstrate—for example, through the use of modeling—how the proxies tie to the eventual outcome. The Department of Transportation has also highlighted the need for proxy measures when assessing maritime security efforts pertaining to deterrence. For example, according to the Department of Transportation, while a direct measure of access to seaports might be the number of unauthorized intruders detected, proxy measures for seaport access may include related information on gates and guards—combined with crime statistics relating to unauthorized entry in the area of the port—to support a broader view of port security. In terms of aviation security, because failure to prevent a worker from placing a bomb on a plane could be catastrophic, proxy measures may include information on access controls, worker background checks, and confiscated items. Proxy measures could also include information on aircraft operators’ efforts to secure the aircraft. In using a variety of proxy measures, failure in any one of the identified measures could provide an indication on the overall risk to security.

Lastly, the use of likelihood, or “what-if scenarios,” which are used to describe a series of steps leading to an outcome, could allow TSA to assess whether potential activities and efforts effectively work together to hypothetically achieve a positive outcome. For example, the development of such scenarios could help TSA to consider whether an activity’s procedures could be modified in response to identified or projected changes in terrorist behaviors, or if an activity’s ability to reduce or combat a threat is greater if used in combination with other activities.

Appendix VIII: Comments from the Department of Homeland Security

Appendix IX: GAO Contact and Staff Acknowledgments

Acknowledgments

In addition to the contact named above, Steve Morris, Assistant Director, and Barbara Guffy, Analyst-in-Charge, managed this assignment. Scott Behen, Valerie Colaiaco, Dorian Dunbar, Christopher Keisling, Matthew Lee, Sara Margraf, Spencer Tacktill, Fatema Wachob, and Sally Williamson made significant contributions to the work. Chuck Bausell, Jr. provided expertise on risk management and cost-benefit analysis. Virginia Chanley and Michele Fejfar assisted with design, methodology, and data analysis. Thomas Lombardi provided legal support; Elizabeth Curda and Anne Inserra provided expertise on performance measurement; and Pille Anvelt developed the report’s graphics.