Aviation Security: TSA Has Taken Steps to Enhance Its Foreign Airport Assessments, but Opportunities Exist to Strengthen the Program
Released: 2011-10-31T00:00:00

Letter

Background

DHS Responsibilities for Enhancing the Security of Airports with U.S.-Bound Flights from Foreign Countries

Shortly after the September 11, 2001, terrorist attacks, Congress passed, and the President signed into law, the Aviation and Transportation Security Act, which established TSA and gave the agency responsibility for securing all modes of transportation, including the nation’s civil aviation system, which includes domestic and international commercial aviation operations. In furtherance of its civil aviation security responsibilities, TSA is statutorily required to assess the effectiveness of security measures at foreign airports served by a U.S. air carrier, from which a foreign air carrier serves the United States, that pose a high risk of introducing danger to international air travel, and at other foreign airports deemed appropriate by the Secretary of Homeland Security. This provision of law also identifies measures that the Secretary must take in the event that he or she determines that an airport is not maintaining and carrying out effective security measures based on TSA assessments. See appendix II for a detailed description of the process for taking secretarial actions against a foreign airport. In addition, TSA conducts inspections of U.S. air carriers and foreign air carriers that service the United States from foreign airports pursuant to its authority to ensure that air carriers certified or permitted to operate to, from, or within the United States meet applicable security requirements, including those set forth in an air carrier’s TSA-approved security program.

The Secretary of Homeland Security delegated to the Assistant Secretary of TSA the responsibility for conducting foreign airport assessments, but retained responsibility for making the determination that a foreign airport does not maintain and carry out effective security measures. Currently, the Global Compliance Division and Office of International Operations, within TSA’s Office of Global Strategies, are responsible for conducting foreign airport assessments. Table 1 highlights the roles and responsibilities of the TSA positions within these divisions that are responsible for implementing the foreign airport assessment program.

TSA assesses the effectiveness of security measures at foreign airports using select aviation security standards and recommended practices adopted by ICAO, a United Nations organization representing 190 countries. ICAO standards and recommended practices address operational issues at an airport, such as ensuring that passengers and baggage are properly screened and that unauthorized individuals do not have access to restricted areas of an airport. ICAO standards and recommended practices also address non-operational issues, such as whether a foreign government has implemented a national civil aviation security program for regulating security procedures at its airports and whether airport officials implementing security controls go through background investigations, are appropriately trained, and are certified according to a foreign government’s national civil aviation security program. ICAO member states have agreed to comply with these standards, and are strongly encouraged to comply with ICAO- recommended practices. The ICAO standards and recommended practices TSA assesses foreign airports against are referred to collectively in this report as ICAO standards or standards. See appendix III for a description of the ICAO standards TSA uses to assess security measures at foreign airports.

TSA uses a risk-informed approach to schedule foreign airport assessments by categorizing airports into three tiers. Specifically, Tier 1 airports—airports that are determined to be low risk—are assessed once every 3 years; Tier 2 airports—airports determined to be medium risk— are assessed every 2 years; and Tier 3 airports—those determined to be high risk—are assessed annually. TSA’s assessments of foreign airports are conducted by a team of inspectors, which generally includes one team leader and one team member. According to TSA, it generally takes 3 to 7 days to complete a foreign airport assessment. However, the amount of time required to conduct an assessment varies based on several factors, including the size of the airport, the number of air carrier station inspections to be conducted at the airport, the threat level to civil aviation in the host country, and the amount of time it takes inspectors to travel to and from the airport where the assessment will take place.

TSA uses a multistep process to conduct assessments of foreign airports. Specifically, the TSA Representative (TSAR) must obtain approval from the host government to allow TSA to conduct an airport assessment, and schedule the date for the on-site assessment. After conducting an entry briefing with Department of State, host country officials, and airport officials, the team conducts an on-site visit to the airport. During the assessment, the team of inspectors uses several methods to determine a foreign airport’s level of compliance with ICAO standards, including conducting interviews with airport officials, examining documents pertaining to the airport’s security measures, and conducting a physical inspection of the airport. For example, inspectors are to examine the integrity of fences, lighting, and locks by walking the grounds of the airport. Inspectors also make observations on access control procedures, such as looking at employee and vehicle identification methods in secure areas, as well as monitoring passenger and baggage screening procedures in the airport. At the close of an airport assessment, inspectors brief foreign airport and government officials on the results of the assessment. TSA inspectors also prepare a report summarizing their findings on the airport’s overall security posture and security measures, which may contain recommendations for corrective action and must be reviewed by the TSAR, the ROC manager, and TSA headquarters officials. See appendix IV for more information on the multistep process TSA uses to conduct its assessments of foreign airports.

Along with conducting airport assessments, the same TSA inspection team also conducts air carrier inspections when visiting a foreign airport to ensure that air carriers are in compliance with TSA security requirements. Both U.S. air carriers and foreign air carriers with service to the United States are subject to inspection. When conducting air carrier inspections, TSA inspectors examine compliance with applicable security requirements, including TSA-approved security programs, emergency amendments to the security programs, and security directives. As in the case of airport assessments, air carrier inspections are conducted by a team of inspectors, which generally includes one team leader and one team member. An inspection of an air carrier typically takes 1 or 2 days, but can take longer depending on the extent of service by the air carrier. Inspection teams may spend several days at a foreign airport inspecting air carriers if there are multiple airlines serving the United States from that location. During an inspection, inspectors are to review applicable security manuals, procedures, and records; interview air carrier station personnel; and observe air carrier employees processing passengers from at least one flight from passenger check-in until the flight departs the gate to ensure that the air carrier is in compliance with applicable requirements. Inspectors evaluate a variety of security measures, such as passenger processing, checked baggage acceptance and control, aircraft security, and passenger screening. If an inspector finds that an air carrier is not complying with applicable security requirements, additional steps are to be taken to record such instances and, in some cases, pursue them with further investigation.

If the inspectors report that an airport’s security measures do not meet minimum ICAO standards, particularly critical standards, such as those related to passenger and checked baggage screening and access controls, TSA headquarters officials are to inform the Secretary of Homeland Security. If the Secretary, based on TSA’s airport assessment results, determines that a foreign airport does not maintain and carry out effective security measures, he or she must, after advising the Secretary of State, take secretarial action. See appendix II for a detailed description of the process for taking secretarial actions against a foreign airport.

In 2007, we issued a report on TSA ‘s foreign airport assessment program, including the results of TSA’s foreign airport assessments, actions taken and assistance provided by TSA when security deficiencies were identified at foreign airports, TSA oversight of its program, and TSA’s efforts to address challenges in conducting foreign airport assessments. Specifically, we reported that TSA’s oversight of the foreign airport assessment program could be strengthened. For example, TSA did not have adequate controls in place to track whether scheduled assessments and inspections were actually conducted, deferred, or canceled. TSA also did not always document foreign officials’ progress in addressing security deficiencies identified by TSA. Further, TSA did not have outcome-based performance measures to assess the impact of its assessments on the security of U.S.-bound flights. As a result, we recommended that TSA develop controls for tracking and documenting information and establish outcome-based performance measures to strengthen oversight of its foreign airport and air carrier evaluation programs. DHS concurred with the recommendations and has since taken several actions to address them, which we discuss later in our report.

TSA Has Taken Steps to Enhance Foreign Airport Assessments, but Challenges Remain

TSA Steps to Update and Streamline Its Assessments

Since 2007, TSA has taken a number of steps to update and streamline its foreign airport assessment program, as discussed below.

TSA revised and updated its Standard Operating Procedures (SOP) for the program. In 2010, TSA revised the SOP, which prescribes program and operational guidance for assessing security measures at foreign airports. TSA also streamlined the assessment process by reducing the number of ICAO standards it assesses foreign airports against from 86 to 40. Of the 40, TSA officials we interviewed told us the agency has identified 22 standards as key for determining an airport’s level of security. In addition, TSA reduced the assessment report writing cycle time for inspectors from 38 calendar days to 20 calendar days, which was intended to expedite the delivery of assessment reports to host governments. This new requirement has helped TSA reduce the time needed to deliver its assessment results to foreign countries, but all 23 inspectors we interviewed told us this requirement was often difficult to meet due to a variety of factors. For example, upon returning from a visit, TSA inspectors reported that they need to document both the airport assessment and air carrier inspections, and plan their next trip, which makes the reduced reporting time requirement difficult to meet. However, the Director of Global Compliance told us that for larger airports with many air carriers, TSA recently began separating the airport assessment and air carrier inspection visits into two separate visits, thus reducing the documentation workload. Moreover, the deadline to submit documentation has been delayed for some back-to-back assessment trips in order to provide sufficient time for inspectors to complete the documentation. The Director of Global Compliance also stated that, in fiscal year 2012, all employees will have training opportunities in order to improve writing skills and reduce the amount of time dedicated to editing and rewriting assessments. In addition, to address resource needs we identified in 2007, TSA hired 6 additional international inspectors in 2007 and 10 international cargo inspectors in 2008 and created 25 new international inspector positions, of which 15 were filled as of July 2011. TSA plans to fill the remaining 10 positions by the end of 2011. The Director of Global Compliance stated that the burden of writing and processing assessment reports should be lessened as the agency hires additional inspectors because this will create a greater pool of available inspectors to conduct and document the assessments.

TSA implemented a new risk-informed methodology for prioritizing and scheduling its assessments at foreign airports in 2010. Specifically, TSA now categorizes foreign airports as high, medium or low risk. Of the roughly 300 foreign airports TSA assesses, TSA identified some airports as high risk and others as medium risk as of August, 2011. The remaining airports were deemed low risk. TSA’s methodology for determining an airport’s risk category is based on the likelihood of a location being targeted (threat), the protective measures in place at that location (vulnerability), and the potential impact of an attack on the international transportation system (consequence). TSA uses current threat information, airport passenger and flight data, and prior airport assessment results to assign each airport a numerical risk score, which is then used to determine its overall risk ranking. As part of this calculation, TSA assigns each airport an overall vulnerability score of 1–5. These scores, or categories, are numerical representations of compliance or noncompliance with the ICAO standards the agency assesses each foreign airport against. Specifically, using an airport’s most recent assessment report, the ROC Manager and TSA’s Director of Global Compliance assign an overall vulnerability category for each airport based on the following descriptions provided in the 2010 Foreign Airport Assessment Program SOP:  Category 1: Fully Compliant;  Category 2: Capability Exists with Minor Episodes of Noncompliance;  Category 3: Capability Exists, Compliance is Generally Noted,  Category 4: Capability Exists, Serious Lack of Implementation  Category 5: Egregious Noncompliance.

Once the vulnerability score is determined, it is then combined with each airport’s related threat and consequence information to determine its risk category. TSA attempts to assess high-risk airports every year, medium- risk airports once every 2 years, and low-risk airports once every 3 years. TSA’s Director of Global Compliance told us this new approach allows the agency to better allocate resources to identify and mitigate security concerns at foreign airports it assesses. In addition, all the TSA ROC managers and 19 of the 23 inspectors we interviewed during our site visits told us that this new foreign airport risk prioritization methodology was an improvement over the previous process. These officials also stated that this new approach has helped them reduce the number of assessments conducted annually, enabling inspectors to better adhere to the annual schedule. On the basis of our analysis, TSA’s approach for scheduling foreign airport assessments is consistent with generally accepted risk management principles, which define risk as a function of threat, vulnerability, and consequence.

TSA developed a 2011 strategic implementation plan. This plan establishes annual program objectives and milestones, and links program activities to broader agency aviation security goals providing a road map for their completion.

TSA began declassifying its foreign airport assessment reports. Since 2007, TSA has been declassifying the reports from Confidential and designating them SSI to facilitate better access to and the dissemination of program results, while still providing protection for foreign government information deemed sensitive. TSA officials noted that the declassification of assessment results is essential for TSA because staff could not easily access the specifics of prior results and deficiencies from reports that have not yet been declassified.

TSA formed the Capacity Development Branch (CDB). TSA created the CDB in 2007 to manage all TSA international aviation security capacity building assistance efforts, including requests for assistance in response to a host government’s airport assessment results. Through CDB, TSA provides six aviation security training courses that address, among other things, preventive security measures, incident management and response, and cargo security. In 2008, TSA also developed the Aviation Security Sustainable International Standards Team (ASSIST) Program to provide more long-term, sustainable, technical aviation security assistance to select foreign countries. Thus far, TSA has partnered with five countries under the ASSIST program: St. Lucia, Liberia, Georgia, Haiti, and Palau. See appendix V for more specific information on TSA assistance provided these countries under ASSIST.

TSA developed assessment tracking tools to provide better oversight of program information. In 2007 we reported that TSA did not have controls in place to track the status of scheduled foreign airport assessments, including whether assessments were actually conducted or whether they were deferred or canceled, which could make it difficult for the agency to ensure that scheduled assessments are actually completed. We also reported that TSA did not always document the results of follow-up conducted by TSA staff to determine progress made by foreign governments in addressing security deficiencies identified by TSA inspectors during assessments, and that such follow-up would enable the agency to have access to updated information on the security status of foreign airports that provide service to the United States. In response to the findings and recommendations we made in our 2007 report, TSA implemented a tool to track its annual foreign airport assessment schedule, including reasons why assessments were deferred or canceled, and a tracking sheet to compile the results of its prior airport assessments. Specifically, this sheet documents the frequency with which foreign airports complied with particular categories of ICAO standards, such as passenger screening, baggage screening, and access controls, among others. TSA also developed a tool whereby deficiencies previously identified during an assessment can be tracked to monitor the progress made by host governments in rectifying security deficiencies. TSA’s Director of Global Compliance told us these tracking sheets have helped TSA provide better oversight and monitoring of key program information.

TSA signed several working arrangements to facilitate its assessments. Since 2007, TSA has signed a multilateral working arrangement with the European Union (EU), and several bilateral working arrangements with individual foreign nations, to facilitate, among other things, TSA assessments at foreign airports. Specifically, in 2007, we reported that TSA had taken steps toward harmonizing airport assessment processes with the European Commission (EC). As part of these efforts, TSA and the EC established six working groups to facilitate, among other things, sharing of SSI between TSA and the EC, TSA observation of EU airport assessments, as well as EC observation of TSA assessments of airports in the United States. In 2008, TSA signed a multilateral working arrangement with the EU to facilitate joint assessments and information sharing between TSA and the EU. Specifically, under the arrangement, TSA and the EC coordinate assessment schedules annually to identify airport locations at which to conduct joint assessments. EC officials we interviewed told us their main goal under the arrangement was to better leverage resources and reduce the number of TSA visits per year to European airports because of concerns from EU member states on the frequency of visits from EC and U.S. audit teams. TSA officials we interviewed said they also wanted to better leverage existing resources while ensuring continued TSA access to European airports for the purposes of conducting security assessments. While TSA agreed to conduct assessments at EU airports no more than once every 5 years, EU and TSA officials we interviewed said the EC permits TSA to approach a country bilaterally if scheduling conflicts do not allow for an assessment to be conducted jointly. TSA also occasionally conducts table-top reviews in place of on-site airport visits. Specifically, if the EC inspected an airport within the last 2 years, TSA will sometimes meet with EC officials to review the EC inspection report–- referred to as a table top–-which typically contains enough information for TSA to make its evaluations. However, TSA officials said table-top reviews should not serve as a permanent substitute for TSA onsite assessments.

TSA has also entered into several bilateral working arrangements with foreign countries to facilitate its airport assessments. Specifically, TSA has signed arrangements with Brazil, Germany, India, the United Kingdom, Russia, and is in the process of establishing arrangements with Nicaragua and Portugal. These arrangements specify certain conditions, practices, and protocols for sharing key information with TSA, but also impose some constraints, such as limiting the number of TSA visits per year, including the length of the visit.

Challenges Affecting TSA Assessment Efforts

Even with TSA’s efforts to enhance the program, challenges remain in several areas: gaining access to some foreign airports, developing an automated database to manage program information, prioritizing and providing training and technical assistance, and expanding the scope of TSA’s airport assessments to include all-cargo operations, as discussed below.

TSA access to some foreign airports has been limited by sovereignty concerns. In 2007, we reported that some host governments expressed concerns that TSA assessments infringe upon their authority to regulate airports and air carriers within their borders, and that some foreign governments had denied TSA access to their airports. TSA’s multilateral and bilateral arrangements have helped to facilitate assessments in some foreign countries, but TSA has had difficulty gaining access to some foreign airports due to sovereignty concerns raised by host governments. For example, TSA has not been able to assess any of the four airports in Venezuela or conduct TSA compliance inspections for air carriers flying out of Venezuela into the United States, including U.S. air carriers, since 2006. Thus, TSA has been unable to determine the security posture of flights from Venezuela bound for the United States. On September 8, 2008, the Secretary of Homeland Security issued a Public Notice that informs the public that the U.S. Government is unable to determine whether airports in Venezuela that serve as the last point of departure for nonstop flights to the United States maintain and carry out effective aviation security measures.

A TSA official told us that a TSA representative traveled to Venezuela recently to start discussions with the Venezuelan government about TSA regaining access to Venezuelan airports to conduct assessments and air carrier inspections. Since it is unclear what the outcome of these discussions will be, and when TSA will regain access to airports and air carriers in Venezuela, the Public Notice remains in effect. Until TSA is able to regain access to airports and air carriers in Venezuela to conduct assessments and air carrier inspections, the agency will be unable to determine to what extent, if at all, airports in Venezuela are maintaining and carrying out effective security measures, or the extent to which air carriers are complying with TSA security requirements for U.S.-bound flights. The Director of Global Compliance indicated TSA is concerned about sovereignty issues with other foreign countries and their willingness to allow TSA inspectors to assess their airports and air carriers. TSA has been working on establishing a Memorandum of Understanding with one country to ensure continued TSA access to its airports. Moreover, TSA indicated that working arrangements it developed with two other countries were undertaken to address government sovereignty concerns over TSA’s assessments.

TSA has experienced difficulties developing an automated database. Since 2007, TSA has been in the process of trying to develop an integrated, automated database management system to allow for more timely submission of foreign airport assessment results, as well as perform more substantive analysis and comparisons of foreign airport trends and issues. Specifically, in response to our 2007 recommendations, TSA stated that they were exploring an automated means of capturing foreign airport assessment data to track airport deficiencies identified, corrective actions recommended by TSA, and any resulting actions taken by the host nation. In 2010, TSA field tested a system, called the Foreign Airport Assessment Reporting System (FAARS), which was intended to store results of airport assessments for easier data extraction and manipulation. For example, while airport assessments are currently prepared as word documents (typically around 60 pages in length), FAARS was intended to put information into database fields, which would have allowed the Office of Global Strategies (OGS) to run reports on specific indicators, such as which foreign airport checkpoints are using Advanced Imaging Technology (AIT) units. However, the Director of Global Compliance told us FAARS ultimately did not meet TSA’s needs and was discontinued because, among other things, data entry was cumbersome and certain data fields could not be edited. Further, the database was not web-based, and instead had to be installed on users’ hard drives, not allowing for easy integration of multiple users and data.

In April 2011, TSA developed a comprehensive functional requirements document, which outlines the capabilities and functions required for a new proposed software solution. TSA officials told us they provided it to officials in TSA’s Offices of Acquisition and Information Technology who developed a contract for developing, testing, fielding, and distributing a software solution that meets programs needs. TSA officials told us that the contractor who will develop the product has received the Statement of Work, and initial implementation of the product is planned for fiscal year 2012, with full capability planned to follow in fiscal year 2013. Given these time frames, it will be important for TSA to monitor the status of this effort to ensure a solution is implemented within reasonable time frames, particularly since we raised this issue in our 2007 report and it is still not clear when a solution will be fully vetted and implemented. TSA’s Director of Global Compliance also told us that identifying a database management system that meets of the needs of the program has been a long-standing challenge for the program.

TSA’s training and technical assistance efforts face several challenges, and TSA’s new equipment loan program has raised concerns. TSA has initiated several capacity building efforts since our 2007 report, but these efforts have been affected by conflicting Department of State priorities, and TSA’s new equipment loan program has raised concerns about ensuring that loaned equipment is properly operated and maintained. Specifically, in addition to its own training courses and technical assistance, CDB provides training and technical assistance sponsored by the Department of State’s Anti-Terrorism Assistance (ATA) program and from the Organization of American States Inter-American Committee Against Terrorism, which is funded through the State Department. A CDB official stated they currently have eight employees and limited funds to provide aviation security technical and training assistance to partner nations overseas. As a result, a CDB official told us their training schedule often has a 3-month lag from when training is requested to when it is provided. In addition, four TSARs we spoke with stated they sometimes have difficulty getting their requests for TSA training from host nations fulfilled because of a lack of resources. According to a TSA official we spoke with, during the past 2 years, the U.S. government’s aviation security training and assistance priorities have been largely driven by State Department priorities. For example, of the 64 course offerings CDB had planned to provide in 47 foreign countries at the beginning of fiscal year 2011, 33 were sponsored by State ATA or the Organization of American States, and some number of those countries have high-risk airports as identified by TSA. In addition, TSA’s 2010 training schedule showed that of the 53 course offerings CDB provided in 33 countries, 29 were sponsored by State ATA or the Organization of American States, and some number of those countries have high-risk airports as identified by TSA. CDB and State Department officials told us they plan to work more closely in the future to better align their respective priorities.

In addition to providing various types of training and technical assistance, TSA has also provided aviation security equipment to foreign countries to help these countries enhance their existing capabilities and practices. Specifically, one of TSA’s goals in its CDB fiscal year 2011–2015 Strategic Plan is to develop the necessary procedures for a system of long-term lending of decommissioned TSA screening equipment to partner countries. In accordance with authority granted under the Aviation and Transportation Security Act, TSA has undertaken to provide or loan security technologies and other equipment to foreign governments. According to TSA officials, the agency exercises this authority in coordination with the Department of State, and has obtained authority from the Department of State to negotiate and conclude agreements with foreign governments to provide technical cooperation and assistance, referred to as “Circular 175” agreements. For example, following the October 2010 discovery of explosive devices in air cargo packages bound for the United States from Yemen, TSA loaned six hand- held explosives trace detection devices to Yemen in an expedited fashion as a response to an emergent threat to help enhance the government’s passenger and cargo screening processes. TSA officials also told us that the agency has provided security technology and equipment to Aruba, Bahamas, Bermuda, Haiti, Ireland, and Malta under this same authority.

While TSA has provided some equipment to foreign countries, TSA and EC officials we spoke with identified potential challenges associated with doing so. For example, TSA officials cited some foreign governments’ inability to properly maintain and operate TSA-provided screening equipment once provided. TSA officials told us it will be important for the agency to ensure that a foreign government has the appropriate staff, and that they are properly trained and ready to operate the equipment as well as conduct any necessary maintenance, to ensure that the U.S.-provided equipment is being used as intended and remains operational. TSA officials also explained that while under its existing authority it can donate or otherwise transfer equipment, such authority does not authorize TSA to provide maintenance and service contracts for this equipment. TSA officials we spoke with told us they would support congressional efforts to provide the agency with this additional authority. In addition, EC officials we interviewed identified similar challenges to their current and potential future efforts to provide various types of capacity building assistance to foreign countries. TSA officials said it will be important for TSA to establish user agreements with recipient countries that ensure U.S. government resources are not wasted or inappropriately used.

Several factors may complicate TSA assessments of foreign all- cargo operations. Following the attempted bombing of an all-cargo flight bound for the United States from Yemen in October 2010, TSA decided to devote additional resources to assessing all-cargo airports. While TSA is still in the early planning stages of its efforts to assess all-cargo operations at foreign airports, several factors may complicate these efforts. Specifically, TSA’s Director of Global Compliance stated that the agency has identified 17 foreign airports that serve as all-cargo last points of departure to the United States. As of July 2011, TSA has conducted two all-cargo assessments of two airports in China. Moreover, TSA plans to assess two additional all-cargo airports by the end of fiscal year 2011. According to TSA, from these first visits, TSA is making some adjustments to the assessment process. For fiscal years 2012 through 2013, TSA plans to schedule visits to the remaining 15 airports that serve as all-cargo last points of departure to the United States, pending host government permission. However, TSA stated that it is too early to tell how many additional inspectors may be needed to complete these assessments.

TSA officials we interviewed identified several factors that may complicate TSA’s assessments of all-cargo operations at foreign airports. For example, all of the 23 TSA inspectors we interviewed expressed concerns about incorporating additional assessment visits into their annual schedules given their current workloads. In addition, these officials stated that it is uncertain whether foreign governments will allow TSA inspectors to assess their all-cargo operations and all-cargo airports. For example, while TSA has several bilateral arrangements with foreign countries to facilitate its assessments, TSA officials told us these arrangements do not specify access to cargo operations or all-cargo airports. Moreover, all four cargo inspectors we met with said it is logistically difficult to assess “upstream” cargo originating from other non-last point of departure airports. These inspectors said these logistical challenges will be an important factor for the agency to consider when selecting foreign airports to assess as well as in making determinations on the security posture of cargo on flights departing foreign airports for the United States. In addition, these inspectors also said that travel to some foreign all-cargo operation airports may be logistically difficult because of the lack of direct passenger flights and may require long travel by car or train. The Director of Global Compliance acknowledged that this new effort is challenging and stated that the agency will address these issues on a case-by-case basis. However, the Director also stated that with the increase to the inspector workforce, the cross-training of generalist international aviation inspectors to perform cargo inspections, and the limited additional locations to visit, TSA will be able to perform these additional visits over the next 2 years.

Some Number of Foreign Airports Complied with ICAO Standards, but TSA Could Better Use Its Assessment Results

Some Foreign Airports Complied with ICAO Standards, but TSA Identified Serious Noncompliance Issues at Other Airports

Based on our analysis of the results of TSA’s foreign airport assessments conducted during fiscal year 2006 through May 9, 2011, some number of the foreign airports TSA assessed complied with all of TSA’s aviation security assessment standards. However, TSA has identified serious or egregious noncompliance issues at a number of other foreign airports. Common areas of noncompliance included weaknesses in airport access controls and passenger and baggage screening. Moreover, our analysis of TSA’s assessments showed variation in compliance across regions, among various individual standards, and by airports’ risk level. For example, our analysis showed that some number of regions of the world had no airports with egregious noncompliance while some regions had several such airports. Specific information related to our analysis of TSA’s airport assessment results is deemed SSI.

TSA Has Not Yet Analyzed Its Assessment Results to Identify Trends and Better Inform Future Activities

TSA has not taken steps to analyze or evaluate its foreign airport assessment results in the aggregate to identify regional and other trends over time, which could assist the agency in informing and prioritizing its future activities. TSA officials have access to results of foreign airport assessments dating back to fiscal year 1997, but they have not analyzed the information to gain insight into how foreign airports’ security posture may have changed over time or identified regional and other patterns and trends over time. Specifically, TSA’s airport assessment reports are collected in an online repository that can be accessed by employees, and TSA’s Director of Global Compliance compiles high-level information from each airport assessment in a tracking tool, which allows her to view the overall results of assessments without having to go back to individual narrative reports. However, according to TSA, the agency has not analyzed the data contained in this tracking tool, which could assist TSA in informing and prioritizing its future activities and assessing the results of its past assessment efforts. In addition, while the spreadsheet provides a snapshot of airports and their results compared to the ICAO standards, it does not indicate why a standard was not met by an airport. If TSA employees would like to know why a certain airport did not meet a standard in a previous year, they must locate and read the report for that assessment. TSA’s Director of Global Compliance told us that this is labor intensive, and makes it difficult to identify anomalies or trends over time.

Standards for Internal Control in the Federal Government require agencies to ensure that ongoing monitoring occurs during the course of normal operations to help evaluate program effectiveness. TSA’s Director of Global Compliance as well as all TSA ROC managers and inspectors we interviewed agreed that information pertaining to identified vulnerabilities in foreign airports should be compiled in regional-, country-, and airport-specific aggregates to help conduct planning and assess the results of program activities. TSA’s Director of Global Compliance stated that TSA has prepared a vacancy announcement for a program analyst position which may, when filled, be tasked with compiling overall results and analyzing assessment results.

TSA’s Director of Global Compliance as well as all ROC managers and inspectors we interviewed also agreed that analysis of foreign airport assessment results would be helpful in identifying the aviation security training needs of foreign aviation security officials. TSA has one internally funded program in place that is specifically intended to provide aviation security training and technical assistance to foreign aviation security officials. However, TSA also coordinates with other federal agencies, such as the Department of State, to identify global and regional training needs and provide instructors for the aviation security training courses State offers to foreign officials. While TSA does not always determine which foreign countries receive aviation security training and technical assistance offered by other federal agencies, TSA could use the cumulative results of TSA’s foreign airport assessments to better support TSA’s priorities for aviation security training and technical assistance. Moreover, with analysis of airport assessment results, TSA could better inform its risk management decision making by identifying trends and security gaps, and target capacity building efforts accordingly. Specifically, this evaluation could include an analysis of the frequency of noncompliance issues TSA inspectors identified, including regional variations and perspectives on the security posture of individual airports over time. Further, a mechanism to evaluate cumulative foreign airport assessment results could help the agency better allocate and target its future resources and better understand its results, including the impact the program is having on enhancing foreign nations’ ability to comply with ICAO standards.

TSA Has Not Yet Developed Outcome-Based Performance Measures

In 2007, we reported that TSA was taking steps to assess whether the goals of the foreign airport assessment program were being met, but that it had not yet developed outcome-based performance measures to evaluate the impact TSA assistance has on improving foreign airport compliance with ICAO standards. As a result, we recommended that TSA establish outcome-based performance measures to strengthen oversight of the program. While DHS officials agreed with the recommendation in 2007, according to TSA, the agency has not yet developed such measures. The goal of the foreign airport assessment program is to ensure the security of U.S.-bound flights by evaluating the extent to which foreign governments are complying with applicable security requirements. The Government Performance and Results Act (GPRA) of 1993, as amended by the GPRA Modernization Act of 2010, requires executive branch departments to use performance measures to assess progress toward meeting program goals and to help decision makers assess program accomplishments and improve program performance. Performance measures can be categorized either as outcome measures, which describe the results of carrying out a program or activity, or as output measures, which describe the direct products or services delivered by a program or activity, or as process measures, which address the type or level of program activities conducted, such as timeliness or quality.

TSA has taken some steps to develop a variety of measures and is reporting this information to the Office of Management and Budget. These measures include:  average number of international inspections conducted annually per  percentage of foreign airports serving as Last Point of Departure operating in compliance with leading security indicators,  percentage of countries with direct flights to the U.S. that are provided aviation security assistance, and  percentage of countries/territories with no direct flights to the U.S. that are provided aviation security assistance.

While these measures are useful in determining, for example, the percentage of airports operating in compliance with security indicators, they do not address the ultimate results of the program, as outcome measures could. Outcome-based measures could help determine the extent to which TSA programs that assess and provide training and technical assistance to foreign airports have helped to improve security at airports that service the United States. However, TSA’s Director of Global Compliance noted several possible challenges with applying such outcome measures to the assessment program. Specifically, the Director stated that the foreign airport assessment program is designed to identify—not correct—security deficiencies at foreign airports, and that whether or not foreign officials improve security at their airports is not within TSA’s control. The Director added that such measures may create a disincentive for inspectors to objectively assess an airport’s level of compliance. Despite these challenges, the Director acknowledged the importance of developing outcome measures and stated that their development should be the responsibility of TSA’s Office of Global Strategies, not individual programs within this office, such as the foreign airport assessment program that she leads. Even without full control over the outcomes associated with such measures, we continue to believe our prior recommendation is still valid and that it would be useful for TSA to develop reasonable outcome-based measures, such as the percentage of security deficiencies that were addressed as a result of TSA onsite assistance or related technical assistance and training offered by the CDB, and TSA recommendations for corrective action. As we previously recommended, such measures would help TSA establish greater accountability over the way in which TSA uses its resources and, in conjunction with its existing measures, enable the agency to evaluate and improve the impact of its assistance on improving security at foreign airports.

Opportunities Exist to Clarify Airport Assessment Criteria, Further Target Airport Assessments, and Systematically Identify Security Best Practices

While TSA has taken a number of steps to improve and streamline its foreign airport assessment program since our 2007 report, opportunities exist for TSA to make additional improvements in several key areas. For example, TSA has taken steps to make its foreign airport assessments more risk informed, but the agency lacks clearly defined criteria to determine a foreign airport’s level of noncompliance with ICAO standards. For example, as stated earlier, TSA provides each airport an overall vulnerability category, or score, of 1 through 5, which is a numerical representation of compliance or level of noncompliance with the ICAO standards the agency assesses each foreign airport against. However, TSA has not developed any specific criteria, definitions, or implementing guidelines to ensure ROC managers and other program management officials apply these categories consistently across airports. For example, the SOP does not define how to assess whether an airport should receive a vulnerability rating of 3—“capability exists, compliance is generally noted, shortfalls remain,” versus a vulnerability rating of 2—“capability exists with minor episodes of noncompliance.” In the absence of more specific and transparent criteria and guidance, it is not clear how TSA applied these related categories—which describe the level of noncompliance—to the results of the assessments, or whether they were applied consistently over time. The lack of documented guidance prevented us from making an analysis or comparison of how TSA made its determinations. This is particularly important given that these scores represent an overall assessment of an airport’s level of compliance or noncompliance with ICAO standards that TSA has deemed critical to airport security, and also are a key component of TSA foreign airport risk- ranking determinations.

TSA’s Director of Global Compliance agreed these category determinations are largely subjective judgments based on many facts and circumstances. TSA’s Director of Global Compliance stated that it is challenging to establish specific guidance for how to assign these categories because of the numerous factors that can influence the decision for assigning vulnerability scores. The Director also noted that because she reviews each assessment report and weighs in on each assigned category, she in effect serves to institutionalize the scores and ensure they are consistent from airport to airport. Standards for Internal Control in the Federal Government call for controls and other significant events to be clearly documented in directives, policies, or manuals to help ensure operations are carried out as intended. This is especially important should key staff leave the agency. Although we recognize the inherently subjective nature of the standards, providing TSA decision makers with more specific criteria and definitions for determining a foreign airport’s level of compliance with ICAO standards would provide greater assurance that such determinations are consistent across airports over time. The Director acknowledged that additional guidance, such as examples to illustrate what these categories mean, could help ensure greater transparency and consistency over how airport vulnerability scores are determined. Such consistency is important since airport vulnerability determinations are used to calculate an airport’s overall security risk level, which in turn affects the program’s activities and resource needs.

In addition, TSA officials we spoke with identified opportunities for TSA to increase program efficiency by conducting more targeted airport assessments. Specifically, ROC managers and inspectors at all the locations we visited stated there are opportunities for TSA to conduct more targeted, smaller scale assessments at foreign airports that could focus more exclusively on the key security issues at a particular airport rather than having inspectors conduct full-scale assessments every visit. For example, the ROC Manager of one location we visited stated that the Federal Aviation Administration previously conducted supplemental-type visits of foreign airports that were reduced in scope and only focused on specific issues or deficiencies that needed to be addressed. He said that TSA should consider ways to incorporate this type of assessment philosophy into its current operations as it may help further streamline the assessment process and associated time frames. ROC managers at all the locations we visited also said inspectors often know, from their prior visits and assessment reports, what specific issues are present at specific airports, and that focusing more time on key issues could provide a more effective way of addressing and correcting security deficiencies. Twenty of 23 inspectors we spoke with said this type of assessment would also reduce repetitive and duplicative data gathering. In addition, these inspectors stated they sometimes do not have the opportunity to conduct all necessary onsite operational observations, document reviews, and interviews because they spend a significant amount of time addressing other descriptive, less critical aspects of the assessment. They said more targeted risk-informed assessments would allow them to focus more time and attention on key security issues, resulting in higher quality and more useful assessment results. Exploring opportunities to conduct more targeted assessments could help TSA enhance the efficiency and value of TSA’s foreign airport assessment program.

TSA’s Director of Global Compliance told us they have begun to conduct abbreviated and targeted airport assessments in some cases due to the security risks associated with traveling and working in certain countries. For example, in 2011 TSA conducted abbreviated assessments at airports in Mexico and Iraq, due to the current security situation, which focused on a select number of critical areas rather than on all topics typically covered during an assessment. While targeted or abbreviated assessments are viewed as beneficial in some circumstances, TSA’s Director of Global Compliance also stated that conducting a comprehensive assessment is important because inspectors may visit an airport only once every 3 years, to document any security changes, deficiencies, or improvements since the previous visit. The Director also raised a concern about conducting additional targeted assessments if they limited opportunities to conduct regularly scheduled comprehensive assessment visits. However, we believe TSA’s use of abbreviated or targeted assessments could be expanded in cases where it would not have a negative impact on the program. For example, as TSA works to systematically analyze the results of its assessments, it may determine that specific regions of the world need additional assistance in meeting certain critical standards. TSA could use this information to focus or target its assessments to address these higher risk scenarios, thus leveraging program resources. Such efforts are consistent with TSA’s ongoing risk- informed activities, as discussed earlier in this report. Moreover, we have previously reported that risk-informed, priority driven decisions can help inform decision makers when allocating finite resources to the areas of greatest need.

In addition, TSA has not taken steps to systematically compile or analyze security best practice information that could contribute to enhancing the security of both foreign and U.S. airports. TSA officials acknowledged possible opportunities to better identify, compile, and analyze aviation security best practices through their assessments at foreign airports. We have previously reported that in order to identify innovative security practices that could help further mitigate terrorism-related risk to transportation sector assets, it is important to assess the feasibility as well as the costs and benefits of implementing security practices currently used by foreign countries. While TSA compiles information in its foreign airport assessment reports to evaluate the degree to which airports are in compliance with select ICAO standards, it does not have a process in place to identify and analyze aviation security best practices that are being used by foreign airports to secure their operations and facilities. TSA officials agreed that identifying relevant best practices could help TSA better leverage their assessment activities by assisting foreign airports in increasing their level of compliance with ICAO standards, as well as in identifying security practices and technologies that may be applicable to enhancing the security of U.S. airports.

In December 2, 2010, testimony before the Senate Committee on Commerce, Science and Transportation, TSA’s Director of Global Compliance confirmed that there are a variety of ways in which foreign airports can effectively meet ICAO standards. For example, one airport might address access control security by using coded door locks and swipe cards, while another may lock its doors and limit the number of available keys to certain personnel. Airports may also establish perimeter security in different ways, such as through fencing or natural barriers. In addition, TSA inspectors, as part of the assessment, often obtain detailed information and understanding of the various types of security technologies and methods being used by foreign governments, which may also be applicable and cost-effective for U.S. airports. For example, while accompanying TSA inspectors during an airport assessment, we observed TSA inspectors being briefed on various passenger screening processes, technologies, and equipment that were comparable to, and in some cases may have exceeded, those used in the U.S. We believe establishing a mechanism to systematically compile and analyze this type of information could help ensure TSA is more effectively able to assist foreign airports in meeting ICAO standards and improve security practices, as well as identify security practices and technologies that may be applicable to enhancing the security of U.S. airports.

Conclusions

Securing commercial aviation operations remains a daunting task—with hundreds of airports and thousands of flights carrying millions of passengers and pieces of checked baggage to the United States every year. TSA’s foreign airport assessment program is aimed at enhancing this system by identifying critical security weaknesses and gaps in airports serving the United States, which in turn can help inform and guide needed efforts to mitigate these deficiencies. TSA has taken a number of actions to enhance its foreign airport assessment program since 2007, but additional steps can help further strengthen the program. For example, developing a mechanism to evaluate assessment results to determine security trends and patterns could enable TSA to target and prioritize future assessment activities, including training and other capacity building resources. Moreover, establishing criteria and guidance for determining the vulnerability of individual foreign airports would provide for greater consistency of these vulnerability ratings across airports over time. Such consistency is important since airport vulnerability determinations are used to calculate an airport’s overall security risk level. Further, exploring the feasibility of conducting more targeted assessments could help enhance the efficiency and value of TSA’s foreign airport assessment program. Moreover, systematically compiling information on aviation security best practices could help ensure TSA is more effectively able to assist foreign airports in meeting ICAO standards and improve security practices, as well as identifying security practices and technologies that may be applicable to enhancing the security of U.S. airports.

Recommendations for Executive Action

To help further enhance TSA’s foreign airport assessment program, we recommend that the Secretary of Homeland Security direct the Assistant Secretary for the Transportation Security Administration to take the following three actions:  Develop a mechanism to evaluate the results of completed assessment activities to determine any trends and target future activities and resources. This evaluation could include frequency of noncompliance issues, regional variations, and perspectives on the security posture of individual airports over time.  Establish criteria and guidance to assist TSA decision makers when determining the vulnerability rating of individual foreign airports.  Consider the feasibility of conducting more targeted assessments and systematically compiling information on aviation security best practices.

Agency Comments and Our Evaluation

We provided a draft of the sensitive version of this report to DHS and TSA on September 1, 2011, for review and comment. DHS provided written comments which are reprinted in appendix VI. In commenting on our report, DHS stated that it concurred with all three of the recommendations and identified actions taken or planned to implement them. DHS also highlighted new initiatives under way by the Office of Global Strategies.

Regarding the first recommendation that TSA develop a mechanism to evaluate the results of completed assessment activities to determine any trends and target activities and resources, and that this evaluation could include frequency of noncompliance issues, regional variations, and perspectives on the security posture of individual airports over time, DHS concurred. DHS stated that TSA has taken several steps to address this recommendation including utilizing a program analyst to create analyses reflecting temporal and site-specific trends and anomalies. DHS also stated that TSA established a project team to evaluate regional, country, and airport vulnerabilities and determine those problem areas that could be effectively addressed by training. DHS also noted that TSA is developing workshops that can be presented by inspectors at the conclusion of an airport assessment which will be tailored to address specific shortfalls observed during the assessment, which could be effectively mitigated through training. These actions, when fully implemented, should address the intent of the recommendation.

DHS concurred with the second recommendation that TSA establish criteria and guidance to assist TSA decision makers when determining the vulnerability rating of individual foreign airports. DHS stated that the most recent version of the Foreign Airport Assessment Program Standard Operating Procedures now contains several scenarios for managers to use as a set of guidelines in determining the vulnerability rating for each open standard and for the airport overall. DHS also stated that the Director of Global Compliance and ROC managers will collaborate on the development of a scenario archive to promote more long-term consistency in the event that key staff leave the agency. We support TSA’s efforts to ensure greater transparency and consistency over how airport vulnerability scores are determined and believe it will be important for TSA to provide sufficient detail in the criteria and guidance that the agency develops. Such actions, when fully implemented, should address the intent of the recommendation.

DHS concurred with the third recommendation that TSA consider the feasibility of conducting more targeted assessments and systematically compiling information on aviation security best practices. In its response, DHS stated that TSA is developing a pre-audit questionnaire that will be sent to each host government in advance of a planned airport assessment which will assist assessment teams in obtaining administrative information and key documents, such as the Airport Security Program, prior to the visit. DHS added that when the questionnaire is returned to TSA, the agency will obtain an official translation of all submitted items so that the assessment team has a better understanding of the current policies, procedures, and practices in place at the site. According to DHS, this practice may enable the team to tailor its efforts at the airports to focus on those areas of concern as indicated in the responses to the questionnaire, as well as the critical standards. DHS stated that TSA plans to complete development of the questionnaire by mid-fiscal year 2012, with wide-scale deployment beginning in October 2012. We support TSA’s planned actions but also believe that there may be additional opportunities for TSA to expand its use of targeted assessments as it works to implement the first recommendation related to developing a mechanism to evaluate the results of completed assessment activities to determine any trends and target activities and resources. For example, as TSA works to systematically analyze the results of its assessments, it may determine that specific regions of the world need assistance in meeting certain critical standards. Such action, in conjunction with TSA’s planned efforts, would meet the intent of the recommendation.

With regard to aviation security best practices, DHS stated that the five volumes of the International Civil Aviation Organization (ICAO) Security Manual for Safeguarding Civil Aviation Against Acts of Unlawful Interference (Document 8973) contains the globally-recognized best practices and alternative methods for meeting the ICAO standards and recommended practices. DHS stated that TSA participates in the development and review of this document and draws from it when recommending improvements to foreign airport authorities. However, it noted that an infrequently-populated portion of the foreign airport assessment reports is available for inspectors to capture particularly noteworthy practices. DHS stated that during fiscal year 2012, inspectors will be encouraged to more conscientiously identify and document new approaches encountered at airports that are not reflected in the security manual but effectively address the ICAO standards and recommended practices. We support these efforts but also believe that it will be important for TSA to capture information identifying security best practices and technology that may be applicable to enhancing the security of U.S. airports. Such action, in conjunction with TSA’s planned efforts, would meet the intent of the recommendation.

DHS also provided us with technical comments, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 10 days from the report date. At that time, we will send copies to the Secretary of Homeland Security, appropriate congressional committees, and other interested parties. This report also will be available at no charge on the GAO web site at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix VII.

Appendix I: Scope and Methodology

To examine the efforts made by the Transportation Security Administration (TSA) to determine whether foreign airports that provide service to the United States are maintaining and carrying out effective security measures, we addressed the following questions: (1) to what extent has TSA taken steps to enhance its foreign airport assessment program since 2007, and what challenges remain; (2) what are the results of TSA’s foreign airport assessments, and to what extent does TSA use the results of these assessments to guide its future assessment activities; and (3) what opportunities, if any, exist to enhance the value of TSA’s foreign airport assessment program?

To collectively address all three questions, we reviewed relevant laws and regulations, including statutory provisions that identify specific actions to be taken by the Secretary of Homeland Security when the Secretary determines that a foreign airport does not maintain and carry out effective security measures. We reviewed various TSA program management and strategic planning documents and interviewed TSA officials located at TSA headquarters and in the field. We interviewed other federal and nonfederal stakeholders, such as the Department of State, International Civil Aviation Organization (ICAO), and the European Commission (EC). We outline the specific steps taken to answer each objective below.

To determine the steps TSA has taken to enhance its foreign airport assessment program since 2007, we reviewed various TSA program management and strategic planning documents to identify revisions to its current and planned future strategy. Specifically, we reviewed TSA’s 2010 Foreign Airport Assessment Program Standard Operating Procedures (SOP) document, which prescribes program and operational guidance for assessing security measures at foreign airports, and informs TSA personnel at all levels of what is expected of them in the implementation of the program. We also reviewed the job aids TSA inspectors use during each assessment, which ensure that the TSA-specified ICAO aviation security standards and recommended practices are fully evaluated during each assessment.

To determine TSA’s current and planned future strategy, we reviewed available strategic planning documents that TSA uses to guide its program. Specifically, we reviewed TSA’s Office of Global Strategies International Strategy to Enhance Aviation Security for 2010–2012, TSA’s Office of Global Strategies Global Compliance Strategic Implementation Plan Fiscal Year 2011, and the TSA Capacity Development Strategic Plan for fiscal years 2011–2015. In addition, we also obtained and reviewed multilateral and bilateral arrangements TSA has established with the European Union (EU) and several foreign nations to facilitate coordination in the area of aviation security, including facilitation of TSA’s foreign airport assessments.

To understand how TSA assesses and manages its foreign airport risk information, we obtained and reviewed various program documents. Specifically, we obtained and reviewed documents on TSA’s methodology for assigning individual risk rankings (called Tier rankings) to each foreign airport it assesses. TSA’s rankings are based on the likelihood of a location being targeted, the protective measures in place at that location, and the potential impact of an attack on the international transportation system. Airports are then categorized as high, medium, or low risk. While we did not evaluate the quality of TSA’s risk rankings, as this analysis was outside the scope of our work, we generally determined that the rankings addressed all three components of risk (threat, vulnerability, and consequence).

To obtain a greater understanding of the foreign airport assessment process, including how TSA works with host nation officials, we accompanied a team of TSA inspectors during an assessment of the Toronto Pearson International Airport. We based our selection on several factors, including the airport locations TSA had plans to assess during the course of our audit work, host government willingness to allow us to accompany TSA, and travel costs.

To obtain information on the extent to which TSA provided oversight of its assessment efforts, we obtained and reviewed various TSA program management documents and tools TSA uses to track and manage information for the program. Specifically, we reviewed the TSA Airport and Air Carrier Comprehensive Tool (known as the A.C.T.), which TSA uses to track its foreign airport assessment schedule, including when various airports are due to be assessed. We also reviewed the Open Standards and Recommended Practices Tracking Tool, which the TSA Representatives (TSAR) use to monitor and track a foreign airport’s progress in resolving security deficiencies identified by TSA inspectors during previous assessments. In addition, we reviewed the tracking sheet TSA’s Director of Global Compliance uses to compile and track current- and prior-year assessment results, including individual airport vulnerability scores and information on which specific ICAO standards were in noncompliance.

To obtain stakeholder views and perspectives on steps TSA has taken to enhance its foreign airport assessment program since 2007, we interviewed and obtained information from various federal and nonfederal stakeholders. Specifically, we interviewed TSA officials located in the Office of Global Strategies (OGS), Global Compliance (GC), Office of International Operations (OIO), and Capacity Development Branch (CDB). In addition, we also conducted site visits to four of the five TSA Regional Operations Centers (ROC) located in Los Angeles, Dallas, Miami, and Frankfurt where we met with the ROC managers and 23 international aviation security inspectors who conduct TSA’s foreign airport assessments. We based our site visit selections on the number of available inspectors at each location and geographic dispersion.

We conducted telephone and in-person interviews with 9 of the 27 TSARs, located in various embassies and consulates throughout the world, who schedule TSA airport assessment visits and follow up on host governments’ progress in addressing identified security deficiencies. When possible, we conducted in-person interviews with TSARs who were at TSA ROCs during our site visits. We based our TSAR selections on geographic dispersion and varying years of experience. During each of these interviews, we discussed these officials’ responsibilities related to the program, including their role in assisting foreign officials in correcting security deficiencies identified during assessments.

We met with Department of State officials to better understand how they coordinate with TSA through their Anti-Terrorism Assistance (ATA) Program and other related efforts aimed at assisting foreign partners’ capacity to secure their airports. Additionally, we met with officials from the EC, International Air Transport Association, and ICAO to discuss efforts and programs these organizations have in place to enhance international aviation security. We interviewed or received responses to questions from five foreign embassies to obtain perspectives of foreign transportation security officials on TSA’s airport assessment program. We based our selection on geographic dispersion and countries with the highest risk airports, as designated by TSA. However, information from our interviews with government officials, members of the aviation industry, and TSA officials and inspectors cannot be generalized beyond those that we spoke with because we did not use statistical sampling techniques in selecting individuals to interview.

To identify challenges affecting TSA’s foreign airport assessment program, we interviewed TSA program management officials and field officials located at the TSA ROCs on the challenges they experience obtaining access to foreign airports to conduct assessments, the development of an automated database management system, and the provision of aviation security training to foreign governments. In addition, we met with TSA’s Director of Global Compliance, and ROC managers and inspectors located in the field, to discuss potential future challenges TSA may experience when attempting to conduct assessments at foreign airports with all-cargo flights to the United States. Specifically, we obtained their perspectives on foreign governments that have been reluctant to allow TSA inspectors to visit their airports. We interviewed TSA’s Director of Global Compliance on the agency’s progress in developing an automated database to manage program information, including the challenges the agency has experienced finding a solution that meets program needs. We conducted telephone and in-person interviews with nine TSARs to obtain their perspectives on challenges to scheduling airport assessment visits. In addition, we interviewed officials within TSA’s CDB to better understand the scope and types of requests for assistance they receive from foreign countries, including challenges they experience in attempting to provide assistance, such as resource constraints and aligning security priorities with the Department of State.

To determine the results of TSA’s foreign airport assessments and the extent to which the agency evaluates its results to inform future activities, we interviewed TSA officials on the results of its assessments, obtained and reviewed assessment reports and relevant program documents, and conducted our own independent analysis of TSA’s assessment results. To better understand the scope and type of information contained in TSA’s foreign airport assessment reports, we obtained and reviewed the most recently available assessments for all high-risk airports. We also selected a randomized sample of assessment reports from current and prior years. We reviewed sections of these reports for completeness and general consistency with TSA guidance for preparing assessment reports. We obtained and reviewed TSA’s foreign airport risk-ranking sheet to better understand which airports TSA identified as high, medium, and low risk, including how the results of TSA’s assessments influence an airport’s risk ranking. In addition, we obtained and reviewed TSA’s foreign airport assessment program vulnerability results tracking sheet used by the Director of Global Compliance to compile and track current and prior- year assessment results. This tracking sheet included records of TSA’s compliance assessments for each airport that TSA assessed from fiscal year 1997 through May 9, 2011. Specifically, the tracking sheet recorded assessment results for each of the ICAO standards used in the airport assessments, as well as an overall vulnerability score of 1 through 5 assigned after each assessment. This overall vulnerability score is a representation of compliance or noncompliance with all the ICAO standards against which TSA assesses foreign airports. We interviewed the Director of Global Compliance on the steps taken to develop the tracking sheet, including how TSA manages and updates data, and how TSA assigns vulnerability scores. In addition, we conducted our own independent analysis of TSA’s assessment results from fiscal year 2006 through May 9, 2011. Specifically, we analyzed data from TSA’s foreign airport assessment program vulnerability results tracking sheet to identify the number of airports in each vulnerability category by region. We also analyzed TSA assessment results data to determine the frequency with which foreign airports complied with particular ICAO standards, such as access control, quality control, passenger screening, and baggage screening, among others. For those airports that TSA has identified as high risk, we analyzed TSA assessment results data to determine the number of resolved and remaining compliance issues at high-risk airports by region, as well as the level of noncompliance found at high-risk airports.

To assess the reliability of TSA’s data, we selected a random sample of records from TSA’s foreign airport assessment program vulnerability results tracking sheet. Next, we examined the corresponding reports to locate those ICAO standards that had been identified as less than fully compliant in the tracking sheet (a score of 2 through 5 on a 5-point scale). The actual scores assigned to the compliance ratings and found in the tracking sheet were determined by the Director of Global Compliance using guidance in the 2010 SOP in consultation with individuals involved in the assessment process (ROC managers, Supervisory Transportation Security Specialists, and Transportation Security Specialists). Our comparison of the results in the tracking sheet with the compliance information provided in the corresponding reports did not match in several cases. However, in discussions with TSA we determined that the differences were the result of changes to the ICAO standards used in the assessments or a change in the definition of the standards. Specifically, TSA told us that Amendments 10 and 11 to ICAO Annex 17 changed the definitions of some standards, and the numbers assigned to identify them. For example, a standard concerning Hold Baggage Security is now identified as 4.5.1. However, in years prior to Amendments 10 and 11 to Annex 17, that same standard was identified as 4.1.1. TSA’s Director of Global Compliance told us that she updated the foreign airport assessment program vulnerability results tracking sheet with the new definitions and numbers, and the associated results, each time an ICAO amendment came out. As a result, we determined that any analysis of the assessment results for specific ICAO standards would need to take into account the changes TSA identified. Based on our overall analysis of the data and reports, we determined that the data were sufficiently reliable to provide a general indication, by type or category, of the standards TSA assesses against and the level of compliance, and frequency of compliance, for TSA’s airport assessments over the period of our analysis.

In addition, we interviewed TSA’s Director of Global Compliance on the steps TSA takes to analyze its assessment results to inform the agency’s future efforts and compared these efforts to Standards for Internal Control in the Federal Government. We discussed the status of implementation of our 2007 recommendation to develop outcome-oriented performance measures to evaluate the impact that TSA assessments have on improving foreign airport compliance with ICAO standards. We interviewed TSA managers and inspectors located in the field on their roles and responsibilities in determining and documenting assessment results. We assessed TSA’s efforts to analyze its assessment results against Standards for Internal Control in the Federal Government, which require agencies to ensure that ongoing monitoring occurs during the course of normal operations to help evaluate program effectiveness.

To identify opportunities for TSA to enhance the value of TSA’s foreign airport assessment program, we reviewed all relevant program management and strategic documentation, and interviewed TSA officials as well as various other federal and nonfederal stakeholders. Specifically, we reviewed the 2011 Foreign Airport Assessment Program SOP and job aids; OGS, GC, and CDB strategic planning documents; foreign airport risk assessment and ranking information; program management tools TSA uses to track and manage its schedule and the status of foreign airport security deficiencies; and reviewed TSA foreign airport assessment results and reports. We also reviewed our prior work concerning how risk-informed and priority driven decisions can help inform agency decision makers in allocating finite resources to the areas of greatest need.

Moreover, we reviewed the process TSA uses to assign vulnerability ratings of 1-5 to each foreign airport it assesses and then evaluated this process against Standards for Internal Control in the Federal Government, which call for controls and other significant events to be clearly documented in directives, policies, or manuals to help ensure operations are carried out as intended. In addition, we visited the Toronto Pearson International Airport to observe TSA inspectors during the assessment thereby obtaining a greater understanding of the foreign airport assessment process, including opportunities for TSA to improve its program. We reviewed prior GAO work discussing the importance of identifying potential best practices, as part of conducting U.S. federal government security assessments in other countries.

To obtain stakeholder views and perspectives on opportunities to enhance the program, we interviewed and obtained information from various TSA and nonfederal stakeholders. Specifically, we interviewed TSA headquarters officials in GC, OIO, and CDB. During our site visits, we interviewed ROC managers and international inspectors on possible opportunities that exist for TSA to improve its foreign airport assessment program. We discussed opportunities to improve the program during our telephone and in-person interviews with nine TSARs. In addition, we discussed ways in which TSA could improve its program during our interviews with officials from the EC, ICAO, and select foreign embassies.

We conducted this performance audit from August 2010 through October 2011 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Process for Taking Secretarial Action against a Foreign Airport

If the Secretary, based on the TSA airport assessment results, determines that a foreign airport does not maintain and carry out effective security measures, he or she must, after advising the Secretary of State, take secretarial action. Below is a list of these actions. Figure 4 describes the process for taking secretarial action against an airport.  90-day action—The Secretary notifies foreign government officials that they have 90 days to address security deficiencies that were identified during the airport assessment and recommends steps necessary to bring the security measures at the airport up to ICAO standards.  Public notification—If, after 90 days, the Secretary finds that the government has not brought security measures at the airport up to ICAO standards, the Secretary notifies the general public that the airport does not maintain and carry out effective security measures.  Modification to air carrier operations—If, after 90 days, the Secretary finds that the government has not brought security measures at the airport up to ICAO standards:  The Secretary may withhold, revoke, or prescribe conditions on the operating authority of U.S.-based and foreign air carriers using that airport to provide transportation to the U.S., following consultation with appropriate host government officials and air carrier representatives, and with the approval of the Secretary of State.  The President may prohibit a U.S.-based or foreign air carrier from providing transportation between the United States and any foreign airport that is the subject of a secretarial determination.  Suspension of service— The Secretary, with approval of the Secretary of State, shall suspend the right of any U.S.-based or foreign air carrier to provide service to or from an airport if the Secretary determines that a condition exists that threatens the safety or security of passengers, aircraft, or crew traveling to or from the airport, and the public interest requires an immediate suspension of transportation between the United States and that airport.

Appendix III: ICAO Standards TSA Uses to Assess Security Measures at Foreign Airports

TSA inspectors use 40 ICAO standards and 1 recommended practice when conducting foreign airport assessments. Of the 40, TSA identified 22 as critical. These 22 critical standards are in bold.

Airport Operations:

3.2.1 Each Contracting State shall require each airport serving civil aviation to establish, implement and maintain a written Airport Security Program appropriate to meet the requirements of the National Civil Aviation Security Programme. 3.2.2 Each Contracting State shall ensure that an authority at each airport serving civil aviation is responsible for coordinating the implementation of security controls. 3.2.3 Each Contracting State shall ensure that an airport security committee at each airport serving civil aviation is established to assist the authority mentioned under 3.2.2 in its role of coordinating the implementation of security controls and procedures as specified in the airport security programme.

Quality Control:

3.4.1 Each Contracting State shall ensure that the persons implementing security controls are subject to background checks and selection procedures. 3.4.2 Each Contracting State shall ensure that the persons implementing security controls possess all competencies required to perform their duties and are appropriately trained according to the requirements of the national civil aviation security programme and that appropriate records are maintained up to date. Relevant standards of performance shall be established and initial and periodic assessments shall be introduced to maintain those standards. 3.4.3 Each Contracting State shall ensure that the persons carrying out screening operations are certified according to the requirements of the National Civil Aviation Security Program to ensure that performance standards are consistently and reliably achieved. 3.4.5 Each Contracting State shall ensure that the implementation of security measures is regularly subjected to verification of compliance with the national civil aviation security programme. The priorities and frequency of monitoring shall be determined on the basis of risk assessment carried out by the relevant authorities. 3.4.6 Each Contracting State shall arrange for audits, tests, surveys and inspections to be conducted on a regular basis, to verify compliance with the National Civil Aviation Security Program and to provide for the rapid and effective rectification of any deficiencies.

Measures Relating to Access Control:

4.2.1 Each Contracting State shall ensure that the access to airside areas at airports serving civil aviation is controlled in order to prevent unauthorized entry. 4.2.2 Each Contracting State shall ensure that security restricted areas are established at each airport serving civil aviation designated by the State based upon a security risk assessment carried out by the relevant national authorities. 4.2.3 Each Contracting State shall ensure that identification systems are established in respect of persons and vehicles in order to prevent unauthorized access to airside areas and security restricted areas. Identity shall be verified at designated checkpoints before access is allowed to airside areas and security restricted areas. 4.2.4 Each Contracting State shall ensure that background checks are conducted on persons other than passengers granted unescorted access to security restricted areas of the airport prior to granting access to security restricted areas. 4.2.5 Each Contracting State shall ensure that the movement of persons and vehicles to and from the aircraft is supervised in security restricted areas in order to prevent unauthorized access to aircraft. 4.2.6 Each Contracting State shall ensure that persons other than passengers, together with items carried, being granted access to security restricted areas are screened; however, if the principle of 100 per cent screening cannot be accomplished, other security controls, including but not limited to proportional screening, randomness and unpredictability, shall be applied in accordance with a risk assessment carried out by the relevant national authorities. 4.2.7 Each Contracting State shall ensure that vehicles being granted access to security restricted areas, together with items contained within them, are subject to screening or other appropriate security controls in accordance with a risk assessment carried out by the relevant national authorities.

Measures Relating to Aircraft: 4.3.1 Each Contracting State shall ensure that aircraft security checks of originating aircraft engaged in commercial air transport movements are performed or an aircraft security search is carried out. The determination of whether it is an aircraft security check or a search that is appropriate shall be based upon a security risk assessment carried out by the relevant national authorities. 4.3.2 Each Contracting State shall ensure that measures are taken to ensure that any items left behind by passengers disembarking from transit flights are removed from the aircraft or otherwise dealt with appropriately before departure of an aircraft engaged in commercial flights. 4.3.3 Each Contracting State shall require its commercial air transport operators to take measures as appropriate to ensure that during flight unauthorized persons are prevented from entering the flight crew compartment. 4.3.4 Each Contracting State shall ensure that an aircraft subject to 4.3.1 is protected from unauthorized interference from the time the aircraft search or check has commenced until the aircraft departs.

Measures Relating to Passengers and Their Cabin Baggage: 4.4.1 Each Contracting State shall establish measures to ensure that originating passengers of commercial air transport operations and their cabin baggage are screened prior to boarding an aircraft departing from a security restricted area. 4.4.2 Each Contracting State shall ensure that transfer passengers of commercial flights and their cabin baggage are screened prior to boarding an aircraft, unless it has established a validation process and continuously implements procedures, in collaboration with the other Contracting State where appropriate, to ensure that such passengers and their cabin baggage have been screened to an appropriate level at the point of origin and, subsequently, protected from unauthorized interference from the point of screening at the originating airport to the departing aircraft at the transfer airport. 4.4.3 Each Contracting State shall ensure that passengers and their cabin baggage which have been screened are protected from unauthorized interference from the point of screening until they board their aircraft. If mixing or contact does take place, the passengers concerned and their cabin baggage shall be re-screened before boarding an aircraft. 4.4.4 Each Contracting State shall ensure that passengers and their cabin baggage which have been screened are protected from unauthorized interference from the point of screening until they board their aircraft. If mixing or contact does take place, the passengers concerned and their cabin baggage shall be re-screened before boarding an aircraft.

Measures Relating to Hold Baggage:

4.5.1 Each Contracting State shall establish measures to ensure that originating hold baggage is screened prior to being loaded onto an aircraft engaged in commercial air transport operations departing from a security restricted area. 4.5.2 Each Contracting State shall ensure that all hold baggage to be carried on a commercial aircraft is protected from unauthorized interference from the point it is screened or accepted into the care of the carrier, whichever is earlier, until departure of the aircraft on which it is to be carried. If the integrity of the hold baggage is jeopardized, the hold baggage shall be re-screened before being placed on board an aircraft. 4.5.3 Each Contracting State shall ensure that commercial air transport operators do not transport the baggage of passengers who are not on board the aircraft unless that baggage is identified as unaccompanied and subjected to additional screening. 4.5.4 Each Contracting State shall ensure that transfer hold baggage is screened prior to being loaded onto an aircraft engaged in commercial air transport operations, unless it has established a validation process and continuously implements procedures, in collaboration with the other Contracting State where appropriate, to ensure that such hold baggage has been screened at the point of origin and subsequently protected from unauthorized interference from the originating airport to the departing aircraft at the transfer airport. 4.5.5 Each Contracting State shall ensure that aircraft commercial air transport operators transport only items of hold baggage that have been individually identified as accompanied or unaccompanied, screened to the appropriate standard, and accepted for carriage on that flight by the air carrier. All such baggage should be recorded as meeting these criteria and authorized for carriage on that flight.

Measures Relating to Cargo, Mail and Other Goods:

4.6.1 Each Contracting State shall ensure that appropriate security controls, including screening where practicable, are applied to cargo and mail, prior to their being loaded onto an aircraft engaged in passenger commercial air transport operations. 4.6.2 Each Contracting State shall establish a supply chain security process, which includes the approval of regulated agents and/or known consignors, if such entities are involved in implementing screening or other security controls of cargo and mail. 4.6.3 Each Contracting State shall ensure that cargo and mail to be carried on a passenger commercial aircraft are protected from unauthorized interference from the point screening or other security controls are applied until departure of the aircraft. 4.6.4 Each Contracting State shall ensure that operators do not accept cargo or mail for carriage on an aircraft engaged in passenger commercial air transport operations unless the application of screening or other security controls is confirmed and accounted for by a regulated agent, or such consignments are subjected to screening. Consignments which cannot be confirmed and accounted for by a regulated agent are to be subjected to screening. 4.6.5 Each Contracting State shall ensure that catering, stores and supplies intended for carriage on passenger commercial flights are subjected to appropriate security controls and thereafter protected until loaded onto the aircraft. 4.6.6 Each Contracting State shall ensure that merchandise and supplies introduced into security restricted areas are subject to appropriate security controls, which may include screening. 4.6.7 Each Contracting State shall ensure that security controls to be applied to cargo and mail for transportation on all-cargo aircraft are determined on the basis of a security risk assessment carried out by the relevant national authorities.

Measures Relating to Special Categories of Passengers: 4.7.1 Each Contracting State shall develop requirements for air carriers for the carriage of potentially disruptive passengers who are obliged to travel because they have been the subject of judicial or administrative proceedings.

Measures Relating to the Landside:

4.8.1 Recommendation.— Each Contracting State should ensure that security measures in landside areas are established to mitigate possible threats of acts of unlawful interference in accordance with a risk assessment carried out by the relevant authorities.

Prevention:

5.1.4 Each Contracting State shall ensure that contingency plans are developed and resources made available to safeguard civil aviation, against acts of unlawful interference. The contingency plans shall be tested on a regular basis. 5.1.5 Each Contracting State shall ensure that authorized and suitably trained personnel are readily available for deployment at its airports serving international civil aviation to assist in dealing with suspected, or actual, cases of unlawful interference with civil aviation.

Aerodrome Emergency Planning:

9.1.1 An aerodrome emergency plan shall be established at an aerodrome, commensurate with the airport operations and other activities conducted at the aerodrome.

Fencing:

9.10.3 Suitable means of protection shall be provided to deter the inadvertent or premeditated access of unauthorized persons into ground installations and facilities essential for the safety of civil aviation located off the aerodrome.

Appendix IV: TSA Process for Conducting Foreign Airport Assessments

TSA uses a multistep process to conduct its assessments of foreign airports. Figure 5 describes the process TSA uses.

Appendix V: TSA Aviation Security Sustainable International Standards Team (ASSIST) Program

The mission of the ASSIST program is to raise and strengthen international aviation security standards in foreign countries and airports, and to ensure that improvements in standards are long-term and sustainable. Specifically, TSA deploys teams consisting of six to seven individuals for 1 week in partnership with the host nation in order to evaluate and develop recommendations for building the aviation security capacity. Following the initial visit, TSA conducts follow-up focused visits to deliver training and technical assistance when agreed upon by the host nation. To date, TSA has partnered with five foreign countries under the ASSIST program. These countries are St. Lucia, Liberia, Georgia, Haiti, and Palau. TSA selects countries to partner with based on a variety of factors, which include focusing on countries with last point of departure service to the United States, foreign airport risk rankings, a foreign government’s demonstrated willingness to engage TSA, and a foreign government’s demonstrated ability to sustain ASSIST initiatives after the conclusion of ASSIST. See below for specific information on the countries TSA partnered with during 2009-2011.

St. Lucia:

St. Lucia was the first nation to partner with TSA under the ASSIST program. It was selected as the pilot country for ASSIST because it is a last point of departure location to the U.S., a popular destination for U.S. passengers, and the TSA Representative in the region requested the assistance. The inaugural survey visit to St. Lucia was conducted in January 2009. Subsequent follow-up visits were held in March and June of 2009, and focused on training in Emergency Communications, Improvised Explosive Device Familiarization, Essential Instructor Skills, and Basic Screener Training. The ASSIST program closed out in St. Lucia in 2010. TSA officials told us that TSA partnered with St. Lucia because it was the pilot country for the ASSIST program. The Capacity Development Branch did not want to pilot the ASSIST program in a country that was “ultra challenging” in terms of security deficiencies.

Liberia:

Liberia was the second nation to partner with TSA under ASSIST. Liberia was chosen for ASSIST after President George W. Bush visited the nation in February 2008, pledging U.S. support in the area of aviation security. In addition, Delta Airlines wanted to reestablish service between the U.S. and Liberia and, in order to do so, Liberia’s national civil aviation program needed improvement. Liberia received a survey visit in April 2009. TSA conducted Essential Instructor Skills and Basic Screening Skills Training in May 2009. This training was followed by monthly visits to assess the impact of training and other technical assistance. In January 2010, TSA coordinated Fraudulent Document Detection training in conjunction with the U.S. Customs and Border Protection and Immigration and Customs Enforcement. In August 2010, TSA conducted its National Inspectors Training. The ASSIST program was closed out in Liberia in November 2010.

Georgia:

Georgia, the third nation to partner with TSA under ASSIST, received a survey visit in September 2009. TSA coordinated its ASSIST program activities in Georgia with the European Civil Aviation Conference (ECAC). Georgia is a member state of ECAC and ECAC initiated a program of technical assistance in Georgia following its March 2009 audit of the Tbilisi Airport. In addition, TSA officials also told us that the State Department also requested that TSA work with Georgia. In April 2010, ECAC and TSA conducted ECAC’s Best Practices for National-level Auditors course. In August 2010, TSA conducted a review of passenger and baggage screening. The ASSIST program was closed out in Georgia in December 2010.

Palau:

TSA deployed an ASSIST program representative to Palau in August 2010. TSA officials told us that Palau was selected for the ASSIST program as a result of the results from the TSA foreign airport assessment program. In addition, Palau was a last point of departure to the United States and the host government was willing to engage TSA and make a commitment to sustain its aviation security enhancements.

Haiti

Currently, the ASSIST program is working with Haiti. Haiti was selected for ASSIST as a result of past program assessment recommendations. Specifically, in October 2010, the ASSIST team was in the process of conducting a training “needs assessment” in Haiti to determine what is needed to rectify aviation security deficiencies found by the program.

Appendix VI: Comments from the Department of Homeland Security

Appendix VII: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Steve D. Morris, Assistant Director, and Christopher E. Ferencik, Analyst-in-Charge, managed this review. Wendy C. Johnson, Lisa A. Reijula, and Rebecca Kuhlmann Taylor made significant contributions to the work. Thomas F. Lombardi provided legal support. Stanley J. Kostyla and Minette D. Richardson assisted with design, methodology, and data analysis. Linda S. Miller provided assistance in report preparation. Tina Cheng helped develop the report’s graphics.