TSA Explosives Detection Canine Program: Actions Needed to Analyze Data and Ensure Canine Teams Are Effectively Utilized
Released: 2013-01-31T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: TSA has implemented a multilayered system composed of people, processes, and technology to protect the transportation system. One of TSA's security layers is NCP, composed of over 760 deployed explosives detection canine teams, including PSC teams trained to detect explosives on passengers. As requested, GAO examined (1) data TSA has on its canine program, what these data show, and to what extent TSA analyzed these data to identify program trends, and (2) the extent to which TSA deployed PSC teams using a risk-based approach and determined their effectiveness prior to deployment. To address these questions, GAO conducted visits to four geographic locations selected based on the number and type of canine teams deployed. The results of these site visits are not generalizable, but provided insights into NCP. GAO also analyzed TSA data from 2011 through 2012, such as utilization data; reviewed documents, including response protocols; and interviewed DHS officials.

section_title: What GAO Found
paragraphs: The Transportation Security Administration (TSA), the federal agency that administers the National Canine Program (NCP), is collecting and using key data on its canine program, but could better analyze these data to identify program trends. TSA collects canine team data using the Canine Website System (CWS), a central management database. TSA uses CWS to capture the amount of time canine teams conduct training as well as searching for explosives odor, among other functions. However, TSA has not fully analyzed the data it collects in CWS to identify program trends and areas that are working well or in need of corrective action. Such analyses could help TSA to determine canine teams' proficiency, inform future deployment efforts, and help ensure that taxpayer funds are used effectively. For example:

GAO analysis of canine team training data from May 2011 through April 2012 showed that some canine teams were repeatedly not in compliance with TSA's monthly training requirement, which is in place to ensure canine teams remain proficient in explosives detection.

GAO analysis of TSA's cargo-screening data from September 2011 through July 2012 showed that canine teams primarily responsible for screening air cargo placed on passenger aircraft exceeded their monthly screening requirement. This suggests that TSA could increase the percentage of air cargo it requires air cargo canine teams to screen or redeploy teams.

TSA has not deployed passenger screening canines (PSC)--trained to identify and track explosives odor on a person--consistent with its risk-based approach, and did not determine PSC teams' effectiveness prior to deployment. TSA's 2012 Strategic Framework calls for the deployment of PSC teams based on risk; however, GAO found that PSC teams have not been deployed to the highest-risk airport locations. TSA officials stated that the agency generally defers to airport officials on whether PSC teams will be deployed, and some airport operators have decided against the use of PSC teams at their airports because of concerns related to the composition and capabilities of PSC teams. As a result of these concerns, the PSC teams deployed to higher-risk airport locations are not being used for passenger screening as intended, but for other purposes, such as screening air cargo or training. TSA is coordinating with aviation stakeholders to resolve concerns related to PSC team deployment, but has been unable to resolve these concerns, as of September 2012. Furthermore, TSA began deploying PSC teams in April 2011 prior to determining the teams' operational effectiveness and before identifying where within the airport these teams would be most effectively utilized. TSA is in the process of assessing the effectiveness of PSC teams in the operational environment, but testing is not comprehensive since it does not include all areas at the airport or compare PSCs with already deployed conventional canines (trained to detect explosives in stationary objects). As a result, more comprehensive testing could provide TSA with greater assurance that PSC teams are effective in identifying explosives odor on passengers and provide an enhanced security benefit.

This is a public version of a sensitive report that GAO issued in December 2012. Information TSA deemed Sensitive Security Information has been redacted.

section_title: What GAO Recommends
paragraphs: GAO is recommending that TSA (1) regularly analyze data to identify program trends and areas working well or in need of corrective action, and (2) take actions to comprehensively assess the effectiveness of PSCs. If PSCs are determined to be effective, GAO is recommending that TSA coordinate with stakeholders to deploy PSC teams to the highest-risk airport locations and utilize them as intended. DHS concurred with GAO's recommendations.

section_title: Letter
paragraphs: Recent foiled terrorist plots in the fall of 2010—including a planned attack  on the Washington, D.C., Metro system, and the discovery of explosive  devices in air cargo packages transported on all-cargo aircraft bound for  the United States from Yemen—underscore the existing threat to the U.S.  transportation system. Within the Department of Homeland Security  (DHS), the Transportation Security Administration (TSA) is the primary  federal agency responsible for security of the nation’s transportation  system. Since the terrorist attacks of September 11, 2001, TSA has  implemented a multilayered system of security composed of people,  processes, and technology to protect the transportation system. One of  TSA’s security layers is the Office of Security Operations’ (OSO) National  Canine Program (NCP), composed of over 760 explosives detection  canine teams—a canine paired with a handler—aimed at deterring and  detecting the use of explosive devices in the U.S. transportation system.

Through NCP, TSA trains, deploys, and certifies explosives detection  canine teams. The program began under the Federal Aviation  Administration in 1972 as a partnership with state and local law  enforcement agencies with jurisdiction over airports by pairing law  enforcement officer (LEO) handlers with conventional canines trained to  detect explosives in objects (e.g., baggage and vehicles). In accordance  with the Aviation and Transportation Security Act, which established TSA,  the transfer of NCP to TSA was accomplished in March 2003. TSA  subsequently expanded the program beyond airports to other  transportation modes, including mass transit, and in January 2008, further  expanded the program to include civilian transportation security inspector  (TSI) canine teams responsible for screening air cargo. In 2011, TSA  again expanded the program by deploying TSI handlers to airports with  passenger screening canines (PSC)—conventional canines also trained  to detect explosives being carried or worn on a person.

In 2008, pursuant to the Implementing Recommendations of the 9/11  Commission Act of 2007, we reviewed TSA’s plans and capacity to  increase its explosives detection canine teams. In that review, we  reported that TSA had begun to increase its capacity to implement its  plan to train, deploy, and certify additional explosives detection canine  teams, as required. You requested additional information on TSA’s canine  deployment plans, and canines’ effectiveness in detecting explosives.  This report addresses the following questions:  (1) What data does TSA have on its canine program, what do  these data show, and to what extent has TSA analyzed these data  to identify program trends?  (2) To what extent has TSA deployed passenger screening canine  teams using a risk-based approach and determined their  effectiveness prior to deployment?

In addition, this report includes information on the actions TSA has taken  to enhance NCP since we issued our last report, in 2008 (see app. I).

This report is a public version of a prior sensitive report that we provided  to you. TSA deemed some of the information in the prior report Sensitive  Security Information (SSI), which must be protected from public  disclosure. The Department of Defense (DOD) also determined some of  the information in our report to be operationally sensitive. Therefore, this  report omits sensitive information regarding the number of deployed  canine teams by type, results of our analysis of canine team data, and  TSA’s assessment of PSC teams, among other things. The information  provided in this report is more limited in scope as it excludes such  sensitive information, but it addresses the same questions as the  sensitive report and the overall methodology used for both reports is the  same.

To determine what data TSA has on its canine program, what these data  show, and the extent to which TSA has analyzed these data to identify  program trends, we interviewed TSA OSO officials about the type of  canine team data they collect and analyze to monitor canine team  performance consistent with Standards for Internal Control in the Federal  Government. Using data from the agency’s Canine Website System  (CWS), we analyzed the number of training and utilization minutes  recorded by canine handlers from May 2011 through April 2012—the  most recent data available at the time of our analysis. We compared the  number of minutes canine teams spent training each month (known as  training minutes) with TSA’s requirement. We analyzed the number of  minutes canine teams were utilized each month (known as utilization  minutes) by type of handler (LEO and TSI) to identify any differences. In  addition, we collected and reviewed monthly reports on canine team  training and utilization from canine team supervisors—an area canine  coordinator and field canine coordinators. Further, we analyzed data from  CWS for May 2012—the most recent data available at the time of our  analysis—to determine the number of certified and decertified canine  teams by type of handler. Moreover, we reviewed all available certification  data from January 2011 through August 2012 provided by TSA. To  assess the reliability of the CWS data on canine team training, utilization,  and certification, we interviewed knowledgeable officials, reviewed the  data for obvious errors and anomalies, and reviewed documentation. We  determined the data were sufficiently reliable for our purposes. We also  reviewed and analyzed the number of final canine responses—instances  when the canine sits, indicating to its handler that it detects explosives  odor—documented in CWS, and corresponding swab samples sent to  TSA’s Canine Explosives Branch (CEB) to validate the canines’  responses and determine their effectiveness in detecting explosives odor.  We found limitations in the final canine response data, as discussed later  in this report, but we believe the data to be sufficiently reliable for  generally comparing the number of canine responses with the number of  swab samples.

To assess the extent to which TSA deployed PSC teams in a risk-based  manner, consistent with TSA’s 2012 Strategic Framework, we compared  TSA’s risk ranking of airports with the NCP’s canine asset spreadsheet  that tracks the deployment location of canine teams. We reviewed TSA’s  Recommended Operational Procedures for Canine Screening of  Individuals to understand TSA’s procedures for deploying PSC teams and  protocols for responding to final responses from PSCs. In addition, we  interviewed PSC handlers at the two locations we visited where PSC  teams were initially deployed. We also interviewed officials from Amtrak  police and three major aviation industry associations—Airport Law  Enforcement Agencies Network (ALEAN), Airports Council International- North America (ACI-NA), and American Association of Airport Executives  (AAAE) because they represent stakeholders who expressed views to  TSA on its deployment of PSC teams. While the results from our  interviews are not generalizable to all PSC handlers and aviation  stakeholders, they provided a range of perspectives on TSA’s PSC team  utilization and response protocols. To determine the extent to which TSA  determined the effectiveness of PSC teams, we compared the PSC  assessment approach used by TSA and the DHS Science and  Technology Directorate (S&T) with DHS guidance and best practices.  Moreover, we reviewed the available test results from TSA and DHS  S&T’s operational assessment of PSC teams’ effectiveness in different  areas at the airport. In addition, we interviewed DHS S&T officials  regarding the assessment methodology, and determined that the results  were reliable. Finally, we reviewed TSA’s canine team cost data, as of  January 2012, to identify the difference in costs among the various types  of canine teams. To assess the reliability of the canine team cost data, we  interviewed knowledgeable officials, and reviewed the data for obvious  errors and anomalies. We determined the data were sufficiently reliable  for the purposes of our report.

To summarize the actions TSA has taken to enhance NCP, we observed  canine training at TSA’s Canine Training and Evaluation Section (CTES)  at Lackland Air Force Base in Texas, as well as Auburn University’s  Canine Detection Training Center in Alabama, because this is where  some PSC teams were trained. Furthermore, we conducted site visits to  the three locations with the greatest number of deployed canine teams in  the mass transit, airport, and air cargo environments to observe canine  teams training in their operational environments and interview LEO and  TSI canine handlers. We also observed canine teams and interviewed  handlers during a site visit we conducted at a fourth location so we could  observe additional PSC teams. While the results from our interviews are  not generalizable to all canine handlers, they provided a range of  perspectives on canine handlers’ views on TSA’s training and evaluation  process. Appendix II provides more details about our objectives, scope,  and methodology.

We conducted this performance audit from January 2012 through January  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: NCP’s mission is to deter and detect the introduction of explosive devices  into the transportation system. As of September 2012, NCP has deployed  762 of 921 canine teams for which it is able to fund across the  transportation system. Table 1 shows the number of canine teams by  type for which funding is available, as well as describes their roles,  responsibilities, and costs to TSA. There are four types of LEO teams:  aviation, mass transit, maritime, and multimodal, and three types of TSI  teams: air cargo, multimodal, and PSC.

TSA’s start-up costs for LEO teams include the costs of training the  canine and handler, and providing the handler’s agency a stipend. The  annual costs to TSA for LEO teams reflect the amount of the stipend.  TSA’s start-up and annual costs for TSI canine teams are greater than  those for LEO teams, because TSI handlers are TSA employees, so the  costs include the handlers’ pay and benefits, service vehicles, and cell  phones, among other things. PSC teams come at an increased cost to  TSA compared with other TSI teams because of the additional training  costs associated with providing decoys (i.e., persons pretending to be  passengers who walk around the airport with explosive training aids).

From fiscal year 2010 to 2012, TSA funding for NCP increased from  approximately $52 million to $101 million. During this time, TSA  requested and was provided funding to increase the stipend it provides to  law enforcement agencies for participating in NCP, and of amounts  appropriated in fiscal year 2012, $5 million was directed for 20 new  canine teams. For fiscal year 2013, TSA is requesting approximately  $96 million for its canine program, which is about $5 million less in  funding than was appropriated in fiscal year 2012. TSA plans to continue  to fund 921 canine teams, and cited savings on supplies, and efficiencies  in operations, among others, as reasons for the reduced funding request.

Figure 1 shows LEO, TSI, and PSC teams performing searches in  different environments.

TSA obtains canines at no cost to TSA through an interagency agreement  with the Department of Defense. DOD reported that the most recent  agreement, dated January 2012, requires TSA to request the number of  canines it needs based on its annual requirement. TSA reported that on  the basis of how many canine teams it plans to train, it requests up to 190  canines each year. TSA also breeds canines at its Canine Breeding and  Development Center at Lackland Air Force Base. Canines undergo 10  weeks of explosives detection training before being paired with a handler  at TSA’s CTES, also located at Lackland Air Force Base. Conventional  canine handlers attend a 10-week training course, and PSC handlers  attend a 12-week training course. Canines are paired with a LEO or TSI  handler during their training course. After canine teams complete this  initial training, and are acclimated to their home operating environment,  they undergo a 10- to 14-day “training mission” with a CTES evaluator to  obtain initial certification to work in their home operating environment.  During the training mission, the evaluator uses a checklist as a guide to  determine if a canine team should be certified. Some items on the  checklist are considered critical, and the canine team must demonstrate  those necessary skills in order to be certified. After initial certification,  canine teams are evaluated on an annual basis to maintain certification.

During the conventional explosives detection evaluation, canine teams  must demonstrate their ability to detect all the explosive training aids the  canines were trained to detect in five search areas. The five search  areas are randomly selected among all the possible areas, but according  to CTES, include the area that is most relevant to the type of canine team  (e.g., teams assigned to airports will be evaluated in areas such as  aircraft and cargo). Canine teams must find a certain percentage of the  explosive training aids to pass their annual evaluation. In addition, a  specified number of nonproductive responses (NPR)—when a canine  responds to a location where no explosives odor is present—are allowed  to pass an evaluation and maintain certification. After passing the  conventional evaluation, PSC teams are required to undergo an  additional annual evaluation that includes detecting explosives on a  person, or being carried by a person. PSC teams are tested in different  locations within the sterile area of an airport. A certain number of  persons must be detected, and a specified number of NPRs are allowed  for PSC certification.

section_title: TSA Collects and Uses Data to Track Canine Program Performance, but Could Better Analyze These Data to Identify Program Trends
paragraphs: TSA collects and uses key canine program data in its Canine Website  System, a central management database, but it could better analyze  these data to identify program trends. Through CWS, TSA captures the  amount of time canine handlers spend on proficiency training as well as  screening for explosives, among other functions. Table 2 highlights  some of the key data elements included in CWS.

NCP uses CWS data to track and monitor canine teams’ performance.  Specifically, FCCs review CWS data to determine how many training and  utilization minutes canine teams are conducting on a monthly basis. NCP  management also analyzes CWS data to determine, for example, how  many canine teams are certified in detecting explosive odors, as well as  the number of teams that passed short notice assessments. However,  TSA has not fully analyzed the performance data it collects in CWS to  identify program trends and areas that are working well or in need of  corrective action. Such analyses could better position TSA to determine  canine teams’ proficiency, guide future deployment efforts, and help  ensure that taxpayer funds are being used effectively.
subsections: section_title: Training Minutes
paragraphs: TSA tracks the number of training minutes canine teams conduct on a  monthly basis, as well as the types of explosives and search areas used  when training, to ensure teams maintain their proficiency in detecting  explosive training aids. However, TSA does not analyze training minute  data over time (from month to month) and therefore is unable to  determine trends related to canine teams’ compliance with the  requirement. On the basis of our analysis of TSA’s data, we determined  that some canine teams were repeatedly not in compliance with TSA’s  240-minute training requirement, and identified differences in the rate of  compliance between TSI and LEO teams.

According to senior NCP officials, reasons canine teams may not meet  the requirement include the canine or handler taking leave, sometimes as  a result of sickness or injury, and participating in an annual evaluation,  among other reasons. While these circumstances may lead to a team not  completing the required minutes certain months during a year, our  analysis further revealed that a certain number of canine teams did not  meet the training requirement for a more extended period, at least 6  months during the period analyzed. In light of this broader trend, NCP  officials agreed that it would be useful to monitor compliance with training  requirements over time to ensure canine teams maintain their proficiency  in detecting explosives odor and stated that this information was  previously unknown to TSA until we brought it to its attention. Analyzing  the existing training data could help alert TSA to trends in noncompliance  that could assist the agency in identifying canine team deficiencies and  areas in need of corrective action. Specifically, such analyses could be  used by program managers to place the canine handlers who do not meet  the training requirement on a performance improvement plan, or  ultimately remove the handlers from the canine program, among other  actions.

section_title: Utilization Minutes
paragraphs: TSA tracks and monitors on a monthly basis the number of utilization  minutes canine teams conduct searching for explosives, screening air  cargo, and serving as a deterrent. TSA also collects monthly data on the  amount of cargo TSI air cargo teams screen in accordance with the  agency’s requirement, but hasn’t analyzed these data over time to  determine if, for example, changes are in needed in the screening  requirement or the number of teams deployed. Our analysis of CWS  utilization data for the period from May 2011 through April 2012 showed  that LEO teams consistently reported greater levels of monthly utilization  minutes than TSI teams, and that on average, from September 2011  through July 2012, TSI air cargo teams exceeded their monthly screening  requirement. However, according to senior NCP officials, it is difficult to  compare utilization rates directly because LEO and TSI teams record  utilization minutes differently. LEO teams report their utilization minutes  based on the amount of time they spend patrolling transportation  terminals, searching baggage and vehicles, and screening air cargo,  among other things. TSI teams, on the other hand, report their utilization  minutes based on the time spent primarily screening air cargo.

According to senior NCP officials, the differences in how utilization  minutes are recorded explains why utilization minutes vary so widely  between LEO and TSI teams. While variances exist in utilization minutes  for canine teams, opportunities exist to determine whether LEO and TSI  teams are being utilized effectively. Unlike TSA’s monthly 240-minute  training requirement, there is no minimum monthly canine utilization  requirement to help ensure that canine teams are effectively utilized.  According to senior NCP officials, the agency is considering whether a  minimum requirement for, and a consistent definition of, utilization  minutes should be applied to LEO and TSI teams in response to our  review. In the absence of a consistent definition of utilization minutes or  goals for utilization, it will be difficult for TSA to determine whether canine  teams are being utilized effectively. Analysis of utilization minutes in light  of stated goals could help TSA determine where LEO and TSI canine  teams are most effectively utilized and could leverage such analysis to  drive future deployment efforts.

In addition to capturing utilization minutes, TSA also collects and  analyzes data monthly on the amount of cargo TSI air cargo canine  teams screen in accordance with the agency’s requirement. However, it is  unclear how the agency uses this information to identify trends to guide  longer-term future program efforts and activities. Our analysis of TSA’s  cargo screening data from September 2011 through July 2012 showed  that TSI air cargo teams nationwide generally exceeded their monthly  requirement to screen air cargo placed on passenger aircraft. This  suggests that TSA could increase the percentage of air cargo it requires  canine teams to screen, or redeploy additional TSI air cargo teams. For  example, while TSI air cargo teams have generally met or exceeded their  monthly screening requirement, as of September 2012, the agency has  yet to deploy all of its 120 TSI air cargo teams for which funding is  available. However, TSA has not utilized the trends in cargo screening  data to guide agency decision making as to whether the current and  planned deployment number of TSI teams by location is appropriate.  Three of the four supervisory TSI officials we interviewed stated that  staffing levels of TSI teams were sufficient. However, one supervisory TSI  official stated that his TSI staffing level was more than double what was  needed to meet TSA’s screening requirement and provide adequate  coverage for the airport. However, TSA plans to deploy additional TSI  teams to this airport in the near future. As TSA continues to roll out TSI  air cargo teams to reach its goal of 120 teams nationwide for which it has  funding, an analysis of utilization data could better inform agency decision  makers regarding future TSI air cargo screening requirements, and  number of canine team deployments by location.

section_title: Certification Rates
paragraphs: TSA tracks the number of certified and decertified canine teams, but is  unable to analyze these data to identify trends in certification rates  because these data were not consistently tracked and recorded prior to  2011. Our analysis of CWS certification data revealed that approximately  the same percentage of LEO and TSI teams were TSA certified in 2011  and 2012. While the certification data show a high percentage of certified  canine teams for the time periods analyzed, it is unclear whether  certifications rates have changed over time for LEO and TSI canine  teams because reliable data for older periods of time were not available.

According to TSA officials, while LEO certification rates have remained  constant for a 2-year time period, TSI certification rates have increased  substantially since 2008, when TSI teams were initially deployed. TSA  officials attributed the initially low TSI certification rates to lack of  management support and limited availability of explosive training aids.  However, we could not determine what, if any, variances existed in the  certification rates among LEO and TSI teams over time because CTES  reported it was unable to provide certification rates by type of canine team  for calendar years 2008 through 2010. According to CTES, the agency  recognizes this deficiency and is in the process of implementing  procedures to address data collection, tracking, and record-keeping  issues. It will be important for TSA to continue to capture and analyze  these data to understand the differences that may exist between LEO and  TSI canine teams; how these differences may affect their effectiveness;  and what steps, if any, need to be taken to address any differences.

section_title: Short Notice Assessments
paragraphs: To help provide reasonable assurance that canine teams are proficient in  detecting explosives odor, NCP conducts and collects information on  covert tests conducted by FCCs that assess canine teams’ operational  effectiveness in detecting and responding to possible explosives, which  TSA refers to as short notice assessments. However, TSA has not  analyzed the results of these assessments beyond the pass and fail rate,  which limits NCP’s ability to determine the effectiveness of its canine  teams. Furthermore, these assessments were suspended in May 2012  because of FCC staffing shortages. NCP began conducting short notice  assessments in April 2011, and FCCs conducted 159 short notice  assessments in 2011. While these tests can be helpful in identifying  canine deficiencies, without analyzing the results of these tests to  determine if there are any search areas or type of explosives in which  canine teams are more effective compared with others, and what, if any,  training may be needed to mitigate deficiencies, TSA is missing an  opportunity to fully utilize them. Resuming and analyzing the results of  short notice assessments will provide TSA with additional information on  the effectiveness of its canine teams in detecting explosives odor,  including determining the conditions under which canine teams are most  or least effective. According to NCP officials, TSA is in the process of  hiring 10 additional FCCs and expects all 10 will be employed in early  2013, at which time NCP plans to resume the assessments, and analyze  the results.

section_title: Final Canine Responses
paragraphs: Our analysis of final canine responses and data on corresponding swab  samples used to verify the presence of explosives odor revealed that  canine teams are not submitting swab samples to NCP’s Canine  Explosives Branch, which is an important component of TSA’s quality  assurance process. In March 2011, TSA issued guidance to FCCs  instructing canine handlers on how to collect and submit swab samples of  final responses to CEB to verify the presence of explosives odor. Such  samples can confirm canines are effectively detecting explosives odor,  although there are limitations to taking swab samples, and a sample that  does not confirm the presence of odor does not necessarily mean the  canine falsely detected explosives odor (e.g., the sample may have been  taken from the wrong part of the item). The purpose of the guidance was  to develop a standardized process of collecting swab samples to  determine canine teams’ effectiveness in detecting explosives odor.

However, we found that canine handlers did not always submit swab  samples. Specifically, we determined that the number of swab samples  sent by canine handlers to TSA’s CEB for scientific review was far lower  than the number of final canine responses recorded in CWS. For  instance, in 2011, of the total number of final canine responses recorded  in CWS, LEO and TSI canine teams submitted swab samples for 8  percent of the final responses to CEB for scientific study after the  guidance was issued. NCP stated that 49 percent of the final canine  responses entered in CWS were improperly or incorrectly recorded by  canine handlers, and the remaining 51 percent of cases did not have to  be submitted to CEB for scientific study since submission under the  guidance was not a requirement. However, NCP officials agreed that  tracking and scientifically measuring final canine responses could help  NCP measure canine teams’ effectiveness in detecting explosives in their  operational environments, such as an airport or air cargo facility.  According to these officials, the agency will take steps to require canine  handlers to submit swab samples to CEB. For example, the requirement  will be added to NCP’s Standard Operating Practices and Procedures,  which is expected to be issued by March 2013. In addition, senior NCP  officials told us the agency will clarify this policy with program supervisors  and canine handlers, and modify CWS to more easily allow handlers to  document when a swab sample is collected and sent to CEB for study. If  these steps are implemented effectively, the issues we raised will be  addressed.

Of the final canine responses that could have been submitted to CEB,  NCP reported that almost half were resolved locally by identifying sources  of odors consistent with those canines are trained to detect. However,  NCP could not determine the resolution of the remaining cases that were  not submitted to CEB, and for which no final resolution was documented  at the local level. As our analysis has shown, capturing and analyzing  additional information on final canine responses over time is an important  quality assurance step and may help TSA and other interested  stakeholders to more accurately determine the extent to which canine  teams are effectively detecting explosive materials in real world  scenarios.

Standards for Internal Control in the Federal Government requires  agencies to ensure that ongoing monitoring occurs during the course of  normal operations to help evaluate program effectiveness. TSA collects  and uses key canine team performance data, but it has not fully analyzed  these data, consisting of training and utilization minutes, certification  rates, and results of short notice assessments and final canine  responses. Such analyses could help TSA management identify program  trends to better target resources and activities based on what is working  well and what may be in need of corrective action. According to NCP  officials, while FCCs review training and utilization data on a monthly  basis, NCP does not have the time or staff resources to analyze such  data over time. However, canine handlers collectively spend a significant  amount of time providing the data to NCP, and such analyses could better  position TSA to determine the proficiency of its canine teams, guide future  deployment efforts, and help ensure taxpayer funds are being used  effectively.

section_title: TSA Has Not Deployed Passenger Screening Canine Teams to the Highest- Risk Airports and Did Not Determine Their Effectiveness Prior to Deployment
paragraphs: TSA’s 2012 Strategic Framework calls for the deployment of PSC teams  based on risk; however, airport stakeholder concerns about the  appropriateness of TSA’s response resolution protocols for these teams  have resulted in PSC teams not being deployed to the highest-risk airport  terminals and concourses. Moreover, TSA began deploying PSC teams  prior to determining the teams’ operational effectiveness and before  identifying where within the airport these teams could be most effectively  utilized to screen passengers.
subsections: section_title: TSA Has Deployed PSC Teams to Airports; However, PSC Teams Have Not Been Deployed to the Highest-Risk Terminals and Concourses
paragraphs: In April 2011, TSA began deploying PSC teams to airports terminals and  concourses, and plans to deploy all 120 PSC teams for which it has  funding by the end of calendar year 2013. TSA’s Strategic Framework  calls for the deployment of PSC teams based on risk; however, we found  that PSC teams have not been deployed to the highest-risk airport  terminals and concourses based on TSA’s high-risk list. TSA officials  stated that PSC teams were not deployed to the highest-risk terminals  and concourses for various reasons, including concerns from an airport  law enforcement association about TSA’s decision to deploy PSC teams  with civilian TSI handlers and the appropriateness of TSA’s response  resolution protocols. These protocols require the canine handler to be  accompanied by two additional personnel that may, but not always,  include a law enforcement officer. According to representatives from an  airport law enforcement association, these protocols are not appropriate  for a suicide bombing attempt requiring an immediate law enforcement  response.

According to TSA officials, the lack of agreed-upon response resolution  protocols with local law enforcement officials has complicated efforts to  introduce PSCs at some airports. For example, seven airport operators  have declined the deployment of PSC teams because of concerns  expressed by airport stakeholders regarding TSA’s response resolution  protocols and whether they are adequate for dealing with potential  threats, such as suicide bombers. According to TSA’s Assistant  Administrator for the Office of Security Operations, TSA’s goal is to  deploy PSC teams to airports where there is a supportive working  environment, and accordingly, TSA does not generally require that PSC  teams be accepted for use at an airport. However, TSA reported that if it  identified a specific threat to an airport, it would use its authority to deploy  the PSC teams regardless of any airport stakeholder opposition to the  decision, but it has not had to do so.

Representatives from three major aviation industry associations we  interviewed support using PSC teams in airports, but also raised  concerns surrounding TSA’s PSC response resolution protocols. These  concerns were primarily related to TSA deploying PSCs with unarmed  TSI handlers rather than LEO handlers because of the differences in their  abilities to respond to a potential threat, such as a suicide bomber.  According to TSA officials, TSA deployed PSCs with TSI handlers in a  manner consistent with the report of the Committee on Appropriations of  the House of Representatives, which accompanies the committee’s DHS  fiscal year 2013 appropriations bill. Officials from an association  representing airport law enforcement officers recommended that PSC  teams be accompanied by three LEOs, among other reasons, to help  reduce the potential liability and response times associated with  responding to a possible suicide bomber. Representatives from an  association representing airport law enforcement officers and TSA  officials stated that they have been working together to try to address  concerns regarding TSA’s response resolution protocols; however, they  have not been able to reach an agreement on this issue, as of September  2012. TSA maintains that PSCs are similar to the agency’s other  screening tools used to detect explosives, and thus a final response from  a PSC should trigger the same type of law enforcement response to  potential threats identified through other screening measures.  Additionally, TSA officials stated that airport operators are required to  adopt and maintain TSA-approved security programs that, among other  things, provide for a law enforcement presence and capability at the  airport that is adequate to ensure the safety of passengers, and that their  current approach of using LEOs in support of TSI-led PSC operations is  consistent with this requirement.

However, TSA’s decision to deploy PSC teams only to airports where  they would be willingly accepted by stakeholders has resulted in PSC  teams not being deployed to the highest-risk airport terminals and  concourses on its high-risk list. Given that PSC teams cost $164,000  annually per team, TSA is not using the teams in the most cost-effective  manner to enhance security if it is limited to deploying them at lower-risk  airports and concourses. Moreover, PSC teams at the two high-risk  airports we visited are not being used for passenger screening because  TSA and the local law enforcement agencies have not reached  agreement on the PSC response resolution protocols. Thus, rather than  being utilized for their intended primary purpose—passenger screening— PSC teams are being used to screen air cargo or conduct training. TSA  officials agreed that the decision to defer to airport stakeholders’  willingness to have PSC teams deployed to their airports has resulted in  PSCs not being optimally utilized. While we recognize the value of  obtaining airport stakeholder buy-in, deploying PSC teams to the highest- risk airport terminals and concourses in accordance with its Strategic  Framework, could help TSA better deploy future PSC teams in a cost- effective manner, and utilize the PSC teams that have already been  deployed for their intended purposes to enhance security.

section_title: TSA Plans to Complete PSC Effectiveness Assessments in 2013; Initial Results Indicate Performance Challenges
paragraphs: TSA began deploying PSC teams in April 2011 prior to determining the  teams’ operational effectiveness and before identifying where within the  airport environment these teams would be most effectively utilized.  According to TSA officials, operational assessments did not need to be  conducted prior to their deployment because canines were being used to  screen passengers by other entities, such as airports in the United  Kingdom, and TSA leadership focused on initially deploying PSC teams  to a single location within the airport—the sterile area—because it thought  it would be the best way to foster stakeholders’ acceptance of the teams.  However, in June 2012, DHS S&T and TSA began conducting operational  assessments to help demonstrate the effectiveness of PSC teams, and  plans to complete additional PSC effectiveness testing in 2013 at  additional airports. To assess the effectiveness of PSC teams in detecting  explosives odor, TSA and DHS S&T are assessing their capabilities  based on several factors. On the basis of these results, DHS S&T and  TSA’s NCP recommended that the assessment team conduct additional  testing and that additional training and guidance be provided to canine  teams.

As part of our review, we visited two airports at which PSC teams have  been deployed and observed training exercises in which PSC teams  accurately detected explosives odor (i.e., positive response), failed to  detect explosives odor (i.e., miss), and falsely detected explosives odor  (i.e., non productive response). See the hyperlink in the note for figure 2  for videos of training exercises at one airport showing instances when  PSC teams detected, and failed to detect explosives odor.

On the basis of the results of DHS’s assessments, TSA could have  benefited from completing operational assessments of PSCs before  deploying them on a nationwide basis to determine whether they are an  effective method of screening passengers in the U.S. airport environment.  TSA officials stated that operational assessments did not need to occur  prior to deployment because the agency does not consider PSCs a new  screening method. Moreover, according to DHS S&T’s Canine Explosives  Detection Project Manager, some PSC teams needed to be deployed  operationally at airports before their effectiveness could be assessed.  While we agree that some PSC teams needed to be deployed for  operational assessments to occur, this testing could have been  completed before TSA started deploying additional PSC teams on a  nationwide basis beyond the three pilot site airports. As discussed earlier,  TSA plans to deploy all 120 teams for which it has funding by the end of  calendar year 2013. Best practices for program performance call for  agencies to use performance information to assess efficiency, identify  performance gaps, and ensure intended goals are met. We previously  reported on the need for TSA to validate detection methods before  deploying new screening methods. For example, TSA proceeded with  deploying its Screening of Passengers by Observation Techniques  behavior detection program on a nationwide basis before determining  whether the list of passenger behaviors and appearances underpinning  the program were scientifically validated, and whether these techniques  could be applied on a large scale for counterterrorism purposes in an  airport environment. Assessing the effectiveness of PSCs in the  operational environment could help provide TSA with reasonable  assurance that PSCs are effective in identifying explosives odor on  passengers and provide an enhanced security benefit.

Additionally, TSA has not completed an assessment to determine where  within the airport PSC teams would be most effectively utilized. According  to TSA’s Assistant Administrator for the Office of Security Operations, to  alleviate airport stakeholders’ concerns regarding TSA’s response  resolution protocols, the agency initially deployed PSC teams to the  sterile areas, thereby enabling TSA to gather data on the value of PSC  teams in the airport environment while reducing the likelihood of a final  response from a PSC resulting in the closure of a checkpoint since an  individual has already passed through several layers of screening when  entering the sterile area. However, aviation stakeholders we interviewed,  raised concerns about this deployment strategy, stating that PSC teams  would be more effectively utilized in non sterile areas of the airport, such  as curbside or in the lobby areas. Specifically, two of the three aviation  industry representatives we interviewed stated that deploying PSC teams  to the sterile area, rather than the public area, could limit their ability to  detect and deter potential suicide bombers entering the airport. According  to TSA’s Assistant Administrator for the Office of Security Operations,  TSA initially deployed PSC teams to the sterile area because it thought it  would foster stakeholders’ acceptance of PSC teams and does not  generally consider the deployment of PSC teams to the sterile area as a  permanent or most optimal use of the PSC teams. Instead, the agency  maintains that the best use of PSC teams is in the public area or at the  screening checkpoint.

TSA has since deployed PSC teams to the passenger screening  checkpoints. According to NCP officials, the agency is also considering  providing some PSCs to LEOs to work on the public side of the airport.  However, DHS S&T does not plan to assess the effectiveness of PSCs  on the public side, beyond the checkpoint, since TSA was not planning to  deploy PSCs to the public side of the airport when DHS S&T designed its  test plan. Comprehensive operational assessments that include a  comparison of PSC teams in both the sterile and public areas of the  airport could help TSA determine if it is beneficial to deploy PSCs to the  public side of airports, in addition to or in lieu of the sterile area and  checkpoint. Moreover, Standards for Internal Control in the Federal  Government stresses the need for agencies to provide reasonable  assurance of the effectiveness and efficiency of operations, including the  use of the entity’s resources.

During the June 2012 assessment of PSC teams’ effectiveness, TSA  conducted one of the search exercises with three conventional canine  teams. Although this assessment was not intended to be included as part  of DHS S&T and TSA’s formal assessment of PSC effectiveness, the  results of this assessment suggest, and TSA officials and DHS S&T’s  Canine Explosives Detection Project Manager agreed, that a systematic  assessment of PSCs with conventional canines could provide TSA with  information to determine whether PSCs provide an enhanced security  benefit compared with conventional LEO aviation canine teams that have  already been deployed to airport terminals. An assessment would help  clarify whether additional investments for PSC training are warranted.  Moreover, since PSC teams are trained in both conventional and  passenger screening methods, TSA could decide to convert existing PSC  teams to conventional canine teams, thereby limiting the additional  resource investments associated with training and maintaining the new  PSC teams. Additionally, as previously discussed, should TSA decide to  deploy some of the PSCs scheduled to be deployed in 2013 with LEO  handlers, rather than TSI handlers, TSA could further reduce program  costs and increase costs savings, as a PSC team led by a LEO handler  could cost TSA $53,000 annually per team, compared with $164,000, the  current annual cost per TSI-led PSC team. Moreover, TSA officials and  aviation industry stakeholders stated that deploying PSCs with  experienced LEO canine handlers could reduce the learning curve  associated with becoming a PSC handler.

section_title: Conclusions
paragraphs: Securing the nation’s vast and diverse transportation system is a  challenging task that is complicated by the ever-changing and dynamic  threat environment. TSA’s National Canine Program is an important and  growing component of its efforts to achieve this goal. However, a  systematic analysis of key program data that TSA currently collects,  consisting of training and utilization minutes, certification rates, results of  short notice assessments (covert tests), and canine responses to the  possible presence of explosives, could better position TSA to identify  program trends to better target resources and activities based on what is  working well and what may be in need of corrective action. In addition, a  comprehensive assessment of passenger screening canines and  conventional canine teams in all areas of the airport could provide TSA  with greater assurance that passenger screening canine teams are a  cost-effective screening tool and are being optimally utilized. Such  analyses could help TSA determine the appropriate number, type, and  placement of canine teams needed to best secure the U.S. transportation  system. Finally, if effective, deploying passenger screening canine teams  to the highest-risk airport terminals and concourses will provide TSA with  greater assurance that the PSC teams are deployed in a manner that  targets the most pressing security needs in a cost-effective manner.

section_title: Recommendations for Executive Action
paragraphs: To help ensure TSA analyzes canine team data to identify program  trends, and determines if PSC teams provide an added security benefit to  the civil aviation system, and if so, deploys PSC teams to the highest-risk  airports, we recommend that the Administrator of the Transportation  Security Administration direct the Manager of the NCP to take the three  following actions:

Regularly analyze available data to identify program trends and areas  that are working well and those in need of corrective action to guide  program resources and activities. These analyses could include, but  not be limited to, analyzing and documenting trends in proficiency  training, canine utilization, results of short notice assessments (covert  tests) and final canine responses, performance differences between  LEO and TSI canine teams, as well as an assessment of the optimum  location and number of canine teams that should be deployed to  secure the U.S. transportation system.

Expand and complete testing, in conjunction with DHS S&T, to assess  the effectiveness of PSCs and conventional canines in all airport  areas deemed appropriate (i.e., in the sterile area, at the passenger  checkpoint, and on the public side of the airport) prior to making  additional PSC deployments to help (1) determine whether PSCs are  effective at screening passengers, and resource expenditures for PSC  training are warranted, and (2) inform decisions regarding the type of  canine team to deploy and where to optimally deploy such teams  within airports.

If PSCs are determined to provide an enhanced security benefit, TSA  should coordinate with airport stakeholders to deploy future PSC  teams to the highest-risk airports, and ensure that deployed PSC  teams are utilized as intended, consistent with its statutory authority to  provide for the screening of passengers and their property.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of the sensitive version of this report to the  Departments of Defense and Homeland Security on November 13, 2012,  for their review and comment. DOD did not provide comments. DHS  provided written comments, which are reprinted in appendix III. DHS  concurred with all three recommendations, and identified actions taken or  planned to implement the recommendations.

DHS concurred with the first recommendation, that TSA regularly analyze  canine team data to identify program trends, and areas working well and  those in need of correction action. DHS stated that NCP is planning to  reestablish yearly comprehensive assessments of canine teams’  compliance with program requirements, and implement corrective action,  when necessary. Furthermore, TSA is upgrading CWS, and in March  2013 will be able to capture data on PSC teams, as it currently does for  conventional canine teams. We support TSA’s actions to implement these  yearly assessments, as well as the upgrade to CWS to collect information  on PSC teams. We believe it is important for TSA to also review its  analysis of canine teams’ performance screening air cargo in order to  inform future decisions on the percentage of air cargo that it requires  canine teams to screen, as well as the number of TSI air cargo canine  teams needed to meet TSA’s air cargo screening requirement. TSA  should also analyze its data, including results of short notice assessments  and final canine responses, to assess canine team performance in order  to mitigate any explosives detection challenges and determine canine  teams’ strengths. Such actions, in conjunction with TSA’s planned efforts,  would meet the intent of the recommendation.

DHS concurred with the second recommendation, to expand and  complete testing to assess the effectiveness of PSC teams and  conventional canines in areas of the airport deemed appropriate. DHS  stated that TSA and DHS S&T will conclude testing of PSC teams in the  sterile areas of airports in February 2013, and have begun testing PSC  teams in the public areas of the airports. In addition, TSA will discuss with  DHS S&T developing testing protocols for conventional canines. We  support TSA’s actions to assess the effectiveness of PSC teams, and  encourage TSA to incorporate conventional canines into future testing to  demonstrate if PSCs provide an enhanced explosives detection benefit  relative to conventional canines, given their increased cost.

DHS also stated that TSA anticipates deploying all 120 of the PSC teams  for which it received authorization by the end of calendar year 2013. If  PSC teams are not proven to be effective based on TSA and DHS S&T’s  assessments, we strongly encourage TSA to reconsider deploying PSC  teams for which it has not already allocated or obligated resources. Such  action, in conjunction with TSA’s planned efforts, would meet the intent of  the recommendation.

DHS concurred with the third recommendation, to deploy PSC teams to  the highest-risk airports. DHS stated that TSA will deploy future teams to  the highest-priority airports as identified by both its operational and risk- based analyses, and it continues to evaluate and modify its risk-based  analysis as needed. We support TSA’s efforts to deploy PSC teams to the  highest-risk airports in order to provide an additional layer of security  where it is most needed, and continue to believe that TSA first needs to  determine if PSCs provide an enhanced security benefit and, if so,  coordinate such deployments with airport stakeholders. Such action, in  conjunction with TSA’s planned efforts, should address the intent of the  recommendation.

DHS also provided technical comments which we incorporated, as  appropriate.

If you or your staff have any questions about this report, please contact  me at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made key contributions to this report are  listed in appendix IV.

section_title: Appendix I: TSA Has Taken Actions to Enhance Program Training and Evaluation, and Research, Development, and Testing
paragraphs: Since our review of the Transportation Security Administration’s (TSA)  National Canine Program (NCP) in 2008, the agency has taken actions to  enhance the program in the areas of (1) training and evaluation, and (2)  research, development, and testing with the DHS Science and  Technology Directorate (S&T). Table 3 provides an overview of these  efforts.

Training and evaluation. Canine handlers we interviewed in four  locations (74 of 75 handlers), generally spoke positively of the instruction  they had received during their training courses and missions. For  example, some canine handlers cited the hands-on instruction and  feedback they received from TSA Canine Training and Evaluation Section  (CTES) evaluators during their training missions as particularly beneficial.  However, some canine handlers we interviewed (26 of 67 evaluated)  suggested that TSA seek ways to improve the consistency and objectivity  of the evaluation process. For example, some canine handlers stated that  while one evaluator may consider a canine sitting a few feet away from  the explosive training aid a “fringe” response and count it positively,  another evaluator would consider the same scenario a nonproductive  response (NPR). As noted earlier, a specified number of NPRs are  allowed to pass an evaluation and maintain certification. In July 2011,  CTES formed a committee comprising various TSA officials and canine  handlers to conduct a review of its evaluation process. The committee  suggested creating a “Train the Evaluator” course to reduce  inconsistencies in the evaluation process. CTES subsequently created an  evaluation unit, which will be implementing the course on consistently  applying evaluation guidelines later this calendar year. Furthermore, in  January 2012, CTES began sending its most experienced evaluators to  all annual evaluations to help improve the consistency and objectivity of  the evaluation process. Specifically, according to CTES officials, one of  the two evaluators conducting the evaluation is to be from the evaluation  unit, which is composed of CTES’s 12 most experienced evaluators. The  committee supported actions to provide greater consistency during the  evaluation process but stated that the evaluations are fair, and ultimately  not the reason for canine teams failing evaluations. Rather, the committee  cited a lack of local training support for canine teams once they are  deployed to their home units. As a result, CTES is in the process of  deploying regional trainers to the field to assist canine teams with  proficiency training. Thus far, 23 of 27 regional trainers have been  deployed to the field. CTES planned to deploy all 27 regional trainers by  December 2012.

Research, development, and testing. Through its Canine Explosives  Detection Project, DHS S&T is working to enhance TSA canine  effectiveness through two efforts: (1) conducting operational assessments  of canine detection capabilities in the air cargo environment, and (2)  determining the behavioral, physiological, and genetic identifiers of a  successful explosives detection canine.

Operational assessments: DHS S&T has conducted a series of  tests assessing the ability of canine teams to detect explosives in  air cargo. During the most recent series of tests, conducted in  2009 and 2010, DHS S&T determined canines can be an effective  tool in detecting explosives in the air cargo environment, but  recommended more frequent and appropriate training.

DHS S&T and TSA also conducted a pilot test to determine if  private, or third-party, canine teams could be used by the air cargo  industry to screen air cargo for the purposes of meeting the  mandate to screen 100 percent of air cargo. According to TSA,  the pilot concluded in September 2011, and TSA determined that  utilizing third-party canine teams to screen air cargo could be a  viable option, but there would also be challenges, including  providing the resources needed to maintain oversight of the  program. TSA leadership is evaluating whether to move forward  with the implementation of a third-party canine program. Allowing  the air cargo industry to use canines could increase its ability to  meet the screening mandate by overcoming challenges we have  previously reported.

Behavior and genetics: DHS S&T, in collaboration with a private  company, Dog Genetics, and the University of Texas, is  researching behavioral, physiological, and genetic identifiers of a  successful explosives detection canine. DHS S&T is reviewing the  history of proven explosives detection canines within NCP and is  attempting to identify behaviors and DNA markers that are  indicative a canine will be successful at explosives detection. The  goal of this research is to allow TSA to better select which canines  to breed, and which to select for explosives detection training.  This is a long-term effort and DHS S&T estimates this research  will be completed in fiscal year 2015.

section_title: Appendix II: Objectives, Scope, and Methodology
paragraphs: This report addresses the following objectives:  (1) What data does TSA have on its canine program, what do  these data show, and to what extent has TSA analyzed these data  to identify program trends?  (2) To what extent has TSA deployed passenger screening canine  teams using a risk-based approach and determined their  effectiveness prior to deployment?

In addition, this report summarizes the actions TSA has taken to enhance  its National Canine Program since we issued our last report, in 2008.

To determine what data TSA has on its canine program, what these data  show, and the extent to which TSA has analyzed these data to identify  program trends, we interviewed TSA’s Office of Security Operations  headquarters and field officials about the type of data they collect and  how they analyze the data to monitor canine team effectiveness  consistent with Standards for Internal Control in the Federal  Government. Specifically, we interviewed canine branch chiefs at  headquarters who oversee the operations of NCP. We also interviewed  an area canine coordinator and field canine coordinators, and supervisory  transportation security inspectors who oversee canine teams in the field.  Using data from the agency’s Canine Website System (CWS), we  analyzed training and utilization minutes from May 2011 through April  2012—the most recent data available at the time of our analysis. We  compared the number of training minutes canine teams conducted each  month with TSA’s training requirement documented in its Canine Manual  and cooperative agreements with law enforcement agencies. We  analyzed the number of minutes canine teams were utilized by type of  canine team to identify any differences between law enforcement officer  (LEO) and transportation security inspector (TSI) teams. In addition, we  collected and reviewed monthly reports on canine team training and  utilization from an area canine coordinator and field canine coordinators.  Further, we analyzed certification data from CWS for May 2012—the  most recent data available at the time of our analysis—to determine the  number of certified and decertified canine teams by type of team (LEO  and TSI). Moreover, we reviewed TSA certification data by type of team  from January 2011 through August 2012. To assess the reliability of the  CWS data on canine team training, utilization, and certification, we  interviewed knowledgeable officials, reviewed the data for obvious errors  and anomalies, and reviewed documentation. We determined the data  were sufficiently reliable for our purposes. We also reviewed and  analyzed the number of final canine responses documented in CWS, and  corresponding swab samples sent to TSA’s Canine Explosives Branch  (CEB) to validate canines’ ability to detect explosives odor. Specifically,  we compared the number of final canine responses recorded in CWS with  the number of swab samples sent to CEB from the time the guidance was  issued on collecting swab samples in March 2011 to July 2012—the most  current data available at the time of our request. We found limitations in  the final canine response data, as discussed in the report, but we believe  the data to be sufficiently reliable for generally comparing the number of  canine responses with the number of swab samples.

To assess the extent TSA deployed passenger screening canines (PSC)  in a risk-based manner, consistent with TSA’s 2012 Strategic Framework,  we compared the risk rankings TSA assigned to airport terminals with  NCP’s September 2012 canine asset spreadsheet—which tracks the  deployment location of TSA’s canine assets. We also interviewed senior  TSA and NCP headquarters and field officials to obtain information on the  extent to which risk and other factors, such as stakeholder concerns,  were considered when deploying PSC teams. We also reviewed TSA’s  Recommended Operational Procedures for Canine Screening of  Individuals to understand the agency’s operational procedures and  protocols for deploying and resolving final responses by PSC teams. In  addition, we interviewed PSC handlers at two locations we visited where  PSCs were initially deployed. We also interviewed officials from Amtrak  police, and three major aviation industry associations—Airport Law  Enforcement Agencies Network (ALEAN), Airports Council International- North America (ACI-NA), and American Association of Airport Executives  (AAAE)—to discuss their views on PSC utilization and response  protocols. While the results from our interviews are not generalizable to  all PSC handlers and aviation stakeholders, they provided views on  TSA’s utilization of PSC teams and protocols for resolving final  responses.

To determine the extent to which TSA determined the effectiveness of  PSC teams, we compared the PSC assessment approach used by TSA  and DHS S&T with DHS guidance and best practices. We also  interviewed senior TSA and DHS S&T officials to obtain information on  the extent to which a comprehensive assessment of the effectiveness of  PSC teams had been completed in the operational environment, including  determining (1) where within the airport environment (i.e. sterile,  checkpoint, and public areas) PSC teams would most effectively utilized,  and (2) whether PSCs provide an added benefit in terms of cost and  security to conventional canines. Moreover, we reviewed the test results  from TSA and DHS S&T’s assessment to determine the extent to which  PSC teams have been proven effective in different areas at the airport,  consistent with the environments described in TSA’s Recommended  Operational Procedures for Canine Screening of Individuals and best  practices in Standards for Internal Control in the Federal Government. In  addition, we interviewed DHS S&T officials regarding the testing  methodology, and determined that the results are reliable. We also  observed PSC teams in their operational environments during our site  visits, and served as decoys—passengers with concealed explosive  training aids—during PSC training exercises in airport terminals, and  videotaped the PSC teams. We reviewed canine team costs data, as of  January 2012, to identify the difference in costs among the various types  of canine teams. To assess the reliability of the canine team cost data, we  interviewed knowledgeable officials, reviewed the data for obvious errors  and anomalies, and reviewed documentation. We determined the data  were sufficiently reliable for our purposes.

To summarize the actions TSA has taken to enhance NCP since our  2008 review, we reviewed TSA’s canine team training and evaluation  procedures. We discussed these procedures with trainers, instructors,  and evaluators at TSA’s CTES. In addition, we observed canine training  at CTES at Lackland Air Force Base in Texas, as well as Auburn  University’s Canine Detection Training Center in Alabama because this is  where some PSC teams were trained. We discussed canine training and  evaluation practices with the Department of Defense’s Military Working  Dog Program, and the Scientific Working Group on Dog and Orthogonal  Detector Guidelines. Furthermore, we conducted site visits to the three  locations with the greatest number of deployed canine teams in the mass  transit, airport, and air cargo environments to observe canine teams  training in their operational environments and interview LEO and TSI  canine handlers. We also observed canine teams and interviewed  handlers during a site visit we conducted to a fourth location so we could  observe additional PSC teams. While the results from our interviews are  not generalizable to all canine handlers, they provided a range of  perspectives on canine handlers’ views on the training and evaluation  process. We reviewed DHS S&T’s efforts to conduct research,  development, and testing to enhance canine effectiveness through its  Canine Explosives Detection Project. Specifically, we reviewed  agreements and contracts with project participants to develop homemade  explosive training aids to broaden threat detection, evaluate the  effectiveness of canine teams in detecting explosives in air cargo, and  conduct research on breeding and genetics to select canines best suited  for explosives detection. We also discussed the status of these efforts  with DHS S&T officials.

We conducted this performance audit from January 2012 through January  2013 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix III: Comments from the Department of Homeland Security

section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Steve Morris, Assistant Director,  and Lisa Canini, Analyst-in-Charge, managed this review. Josh Diosomito  and Michelle Woods made significant contributions to the work. Lydia  Araya, Chuck Bausell, Michele Fejfar, Richard Hung, Thomas F.  Lombardi, Douglas Manor, Linda Miller, Erin O’Brien, Jessica Orr and  Jessica Smith also contributed to this report.


Question-summary annotations: questions: question: What is TSA doing with regards to NCP?
answer: The Transportation Security Administration (TSA), the federal agency that administers the National Canine Program (NCP), is collecting and using key data on its canine program, but could better analyze these data to identify program trends.
child_questions: question: How does TSA collect canine team data?
answer: TSA collects canine team data using the Canine Website System (CWS), a central management database. TSA uses CWS to capture the amount of time canine teams conduct training as well as searching for explosives odor, among other functions.
child_questions: question: What is the problem with TSA's collection?
answer: However, TSA has not fully analyzed the data it collects in CWS to identify program trends and areas that are working well or in need of corrective action.

question: How could analysis of TSA's data be helpful?
answer: Such analyses could help TSA to determine canine teams' proficiency, inform future deployment efforts, and help ensure that taxpayer funds are used effectively.

questions: question: What has TSA not yet done?
answer: TSA has not deployed passenger screening canines (PSC)--trained to identify and track explosives odor on a person--consistent with its risk-based approach, and did not determine PSC teams' effectiveness prior to deployment.
child_questions: question: Why have some airport operators not used PSC teams?
answer: TSA officials stated that the agency generally defers to airport officials on whether PSC teams will be deployed, and some airport operators have decided against the use of PSC teams at their airports because of concerns related to the composition and capabilities of PSC teams.
child_questions: question: What is the result of not using PSC teams?
answer: As a result of these concerns, the PSC teams deployed to higher-risk airport locations are not being used for passenger screening as intended, but for other purposes, such as screening air cargo or training.

question: What is TSA in the process of doing?
answer: TSA is in the process of assessing the effectiveness of PSC teams in the operational environment, but testing is not comprehensive since it does not include all areas at the airport or compare PSCs with already deployed conventional canines (trained to detect explosives in stationary objects).
child_questions: question: How would comprehensive testing be more beneficial to TSA?
answer: As a result, more comprehensive testing could provide TSA with greater assurance that PSC teams are effective in identifying explosives odor on passengers and provide an enhanced security benefit.

questions: question: What has TSA implemented?
answer: TSA has implemented a multilayered system composed of people, processes, and technology to protect the transportation system.
child_questions: question: What is the importance of NCP?
answer: One of TSA's security layers is NCP, composed of over 760 deployed explosives detection canine teams, including PSC teams trained to detect explosives on passengers.

question: What has GAO examined regarding TSA?
answer: As requested, GAO examined (1) data TSA has on its canine program, what these data show, and to what extent TSA analyzed these data to identify program trends, and (2) the extent to which TSA deployed PSC teams using a risk-based approach and determined their effectiveness prior to deployment.
child_questions: question: How did GAO examine TSA?
answer: To address these questions, GAO conducted visits to four geographic locations selected based on the number and type of canine teams deployed. GAO also analyzed TSA data from 2011 through 2012, such as utilization data; reviewed documents, including response protocols; and interviewed DHS officials.

questions: question: What does GAO recommend?
answer: GAO is recommending that TSA (1) regularly analyze data to identify program trends and areas working well or in need of corrective action, and (2) take actions to comprehensively assess the effectiveness of PSCs.
child_questions: question: What does GAO recommend if PSCs are effective?
answer: If PSCs are determined to be effective, GAO is recommending that TSA coordinate with stakeholders to deploy PSC teams to the highest-risk airport locations and utilize them as intended.

question: How did DHS respond to GAO's recommendations?
answer: DHS concurred with GAO's recommendations.