Explosives Detection Canines: TSA Has Taken Steps to Analyze Canine Team Data and Assess the Effectiveness of Passenger Screening Canines
Published: 2014-06-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: TSA has implemented a multilayered system composed of people, processes, and technology to protect the nation's transportation system. One of TSA's security layers is NEDCTP, composed of over 800 deployed explosives detection canine teams, including PSC teams trained to detect explosives on passengers.
This testimony addresses the extent to which TSA has (1) regularly analyzed data to identify program trends and areas working well or in need of corrective action, and (2) comprehensively assessed the effectiveness of PSCs, and coordinated with stakeholders to deploy PSC teams to the highest-risk airports and utilize them as intended. This statement is based on a report GAO issued in January 2013 and selected updates obtained from October 2013 through June 2014. For the selected updates, GAO reviewed TSA documentation, including the results of PSC effectiveness assessments, and interviewed agency officials on the status of implementing GAO's recommendations.
section_title: What GAO Found
paragraphs: In January 2013, GAO reported that the Transportation Security Administration (TSA) collected and used key canine program data in support of its National Explosives Detection Canine Team Program (NEDCTP), but could better analyze these data to identify program trends. For example, GAO found that in reviewing short notice assessments (covert tests), TSA did not analyze the results beyond the pass and fail rates. Therefore, TSA was missing an opportunity to determine if there were any search areas or types of explosives in which canine teams were more effective compared with others, and what, if any, training may be needed to mitigate deficiencies. GAO recommended that TSA regularly analyze available data to identify program trends and areas that are working well and those in need of corrective action to guide program resources and activities. TSA concurred and has taken actions that address the intent of our recommendation. For example, in the event a team fails a short notice assessment, TSA now requires that canine team supervisors complete an analysis of the team's training records to identify an explanation for the failure.
In January 2013, GAO found that TSA began deploying passenger screening canine (PSC) teams—teams of canines trained to detect explosives being carried or worn on a person—in April 2011 prior to determining the teams' operational effectiveness and where within an airport PSC teams would be most effectively utilized. GAO recommended that TSA expand and complete testing to assess the effectiveness of PSCs and conventional canines (trained to detect explosives in stationary objects) in all airport areas deemed appropriate prior to making additional PSC deployments. This would help (1) determine whether PSCs are effective at screening passengers, and resource expenditures for PSC training are warranted, and (2) inform decisions regarding the type of canine team to deploy and where to optimally deploy such teams. TSA concurred and has taken steps to address the recommendation, but additional action is needed. Specifically, TSA launched a PSC training and assessment initiative and determined PSCs to be most effective when working at the airport checkpoint, but TSA does not plan to conduct a comparison of PSC teams with conventional canine teams as GAO recommended. In January 2013, GAO also found that TSA's 2012 Strategic Framework calls for the deployment of PSC teams based on risk; however, airport stakeholder concerns related to the composition and capabilities of PSC teams resulted in the teams not being deployed to the highest-risk airports. GAO recommended that if PSCs are determined to provide an enhanced security benefit compared with conventional canine teams, TSA should coordinate with airport stakeholders to deploy future PSC teams to the highest-risk airports. TSA concurred and has taken steps to address the recommendation. Specifically, the PSC teams for which TSA had funding and not already deployed to a specific airport at the time GAO's report was issued have been deployed to, or allocated to, the highest-risk airports.
section_title: What GAO Recommends
paragraphs: GAO is making no new recommendations in this statement.

section_title: Letter
paragraphs: I appreciate the opportunity to discuss our work on the Transportation  Security Administration’s (TSA) National Explosives Detection Canine  Team Program (NEDCTP). Within the Department of Homeland Security  (DHS), TSA is the primary federal agency responsible for security of the  nation’s transportation system. Since the terrorist attacks of September  11, 2001, TSA has implemented a multilayered system of security  composed of people, processes, and technology to protect the  transportation system. One of TSA’s security layers is NEDCTP,  composed of over 800 explosives detection canine teams—a canine  paired with a handler—aimed at deterring and detecting the use of  explosive devices in the U.S. transportation system.
Through NEDCTP, TSA trains, deploys, and certifies explosives detection  canine teams. The program began under the Federal Aviation  Administration in 1972 as a partnership with state and local law  enforcement agencies with jurisdiction over airports by pairing law  enforcement officer (LEO) handlers with conventional canines trained to  detect explosives in objects (e.g., baggage and vehicles). In accordance  with the Aviation and Transportation Security Act, which established TSA,  the transfer of the canine program from the Federal Aviation  Administration to TSA was accomplished in March 2003. subsequently expanded the program beyond airports to other  transportation modes, including mass transit, and in January 2008, further  expanded the program to include civilian transportation security inspector  (TSI) canine teams responsible for screening air cargo. In 2011, TSA  again expanded the program by deploying TSI handlers to airports with  passenger screening canines (PSC)—conventional canines also trained  to detect explosives being carried by or worn on a person.
Enacted in November 2001, the Aviation and Transportation Security Act established,  within the Department of Transportation, TSA as the agency responsible for securing the  nation’s transportation systems. See Pub. L. No. 107-71, § 101(a), 115 Stat. 597 (2001).  The Homeland Security Act of 2002 subsequently transferred TSA to the newly  established Department of Homeland Security. See Pub. L. No. 107-296, § 403, 116 Stat.  2135, 2178 (2002).
My testimony today addresses the extent to which TSA has (1) regularly  analyzed data to identify program trends and areas working well or in  need of corrective action, and (2) comprehensively assessed the  effectiveness of PSCs, and coordinated with stakeholders to deploy PSC  teams to the highest-risk airports and utilize them as intended. This  statement is based on our January 2013 report and includes selected  updates on the status of TSA’s efforts to implement the recommendations  in that report. The report cited in this statement provides detailed  information on our scope and methodology. To update our work, we  obtained related documentation from TSA from October 2013 through  June 2014, including reports used by NEDCTP to monitor canine team  training minute requirements, results of PSC effectiveness assessments,  and PSC deployment schedules. We also interviewed agency officials in  June 2014 on the progress made by TSA to implement the  recommendations in our January 2013 report. The work upon which this  statement is based was conducted in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: NEDCTP’s mission is to deter and detect the introduction of explosive  devices into the transportation system. As of June 2014, NEDCTP has  deployed 802 of 985 canine teams for which it is able to fund across the  transportation system.type for which funding is available, as well as describes their roles,  responsibilities, and costs to TSA. There are four types of LEO teams:  aviation, mass transit, maritime, and multimodal, and three types of TSI  teams: air cargo, multimodal, and PSC.
section_title: TSA Has Taken Steps to Analyze Canine Team Data to Identify Program Trends
paragraphs: Since our January 2013 report, TSA has taken steps to analyze key data  on the performance of its canine teams to better identify program trends,  as we recommended. In January 2013, we reported that TSA collected  and used key canine program data in its Canine Website System (CWS),  a central management database, but it could better analyze these data to  identify program trends. Table 2 highlights some of the key data elements  included in the CWS.
In January 2013, we found that NEDCTP was using CWS data to track  and monitor canine teams’ performance. Specifically, field canine  coordinators reviewed CWS data to determine how many training and  utilization minutes canine teams conducted on a monthly basis. NEDCTP  management used CWS data to determine, for example, how many  canine teams were certified in detecting explosive odors, as well as the  number of teams that passed short notice assessments. However, in our  January 2013 report, we also found that TSA had not fully analyzed the  performance data it collected in CWS to identify program trends and  areas that were working well or in need of corrective action. For example:
Training minutes: TSA tracked the number of training minutes canine  teams conducted on a monthly basis, as well as the types of  explosives and search areas used when training, to ensure teams  maintained their proficiency in detecting explosive training aids.  However, we found that TSA did not analyze training minute data over  time (from month to month) and therefore was unable to determine  trends related to canine teams’ compliance with the requirement. On  the basis of our analysis of TSA’s data, we determined that some  canine teams were repeatedly not in compliance with TSA’s 240- minute training requirement, in some cases for 6 months or more in a  1-year time period.
Utilization minutes: We found that TSA collected and analyzed data  monthly on the amount of cargo TSI air cargo canine teams screened  in accordance with the agency’s requirement. However, it was unclear  how the agency used this information to identify trends to guide  longer-term future program efforts and activities, since our analysis of  TSA’s cargo screening data from September 2011 through July 2012  showed that TSI air cargo teams nationwide generally exceeded their  monthly requirement. We concluded that TSA could increase the  percentage of cargo it required TSI canine teams to screen.
Certification rates: We found that TSA tracked the number of certified  and decertified canine teams, but was unable to analyze these data to  identify trends in certification rates because these data were not  consistently tracked and recorded prior to 2011. Specifically, we could  not determine what, if any, variances existed in the certification rates  among LEO and TSI teams over time because CTES reported it was  unable to provide certification rates by type of canine team for  calendar years 2008 through 2010. According to CTES, the agency  recognized the deficiency and was in the process of implementing  procedures to address data collection, tracking, and record-keeping  issues.
Short notice assessments (covert tests): We found that when TSA  was performing short notice assessments (prior to their suspension in  May 2012), it was not analyzing the results beyond the pass and fail  rates. We concluded that without conducting the assessments and  analyzing the results of these tests to determine if there were any  search areas or type of explosives in which canine teams were more  effective compared with others, and what, if any, training may have  been needed to mitigate deficiencies, TSA was missing an opportunity  to fully utilize the results.
Final canine responses: Our analysis of final canine responses and  data on corresponding swab samples used to verify the presence of  explosives odor revealed that canine teams were not submitting swab  samples to NEDCTP’s Canine Explosives Unit (CEU). Specifically, we  determined that the number of swab samples sent by canine handlers  to CEU for scientific review was far lower than the number of final  canine responses recorded in CWS. We concluded that without the  swab samples, TSA was not able to more accurately determine the  extent to which canine teams were effectively detecting explosive  materials in real-world scenarios.
In January 2013, we recommended that TSA regularly analyze available  data to identify program trends and areas that are working well and those  in need of corrective action to guide program resources and activities.  These analyses could include, but not be limited to, analyzing and  documenting trends in proficiency training minutes, canine utilization,  results of short notice assessments and final canine responses,  performance differences between LEO and TSI canine teams, as well as  an assessment of the optimum location and number of canine teams that  should be deployed to secure the U.S. transportation system. TSA  concurred with our recommendation and has taken actions to address it.  Specifically, TSA is monitoring canine teams’ training minutes over time  by producing annual reports. TSA also reinstated short notice  assessments in July 2013, and in the event a team fails, the FCC  completes a report that includes an analysis of the team’s training records  to identify an explanation for the failure. In April 2013, TSA reminded  canine handlers of the requirement to submit swab samples of their  canines’ final responses, and reported that the number of samples  submitted that same month, increased by 450 percent, when compared  with sample submissions in April 2012. CEU is producing reports on the  results of its analysis of the swab samples for the presence of explosives  odor. In June 2014, TSA officials told us that in March 2014, NEDCTP  stood up a new office, known as the Performance Measurement Section,  to perform analyses of canine team data. We believe that these actions  address the intent of our recommendation and could better position TSA  to identify program trends to better target resources and activities based  on what is working well and what may be in need of corrective action.
section_title: TSA Has Conducted Additional PSC Team Effectiveness Assessments and Deployed Some Teams to Highest- Risk Airports, but Additional Actions Are Needed
subsections: section_title: TSA Has Conducted Additional PSC Team Effectiveness Assessments, but Has Not Compared PSC Teams with Conventional Canine Teams
paragraphs: In our January 2013 report, we found that TSA began deploying PSC  teams in April 2011 prior to determining the teams’ operational  effectiveness. However, in June 2012, the DHS Science and Technology  Directorate (S&T) and TSA began conducting effectiveness assessments   On the basis of  to help demonstrate the effectiveness of PSC teams.these assessments, DHS S&T and TSA’s NEDCTP recommended that  the assessment team conduct additional testing and that additional  training and guidance be provided to canine teams. See the hyperlink in  the note for figure 2 for videos of training exercises at one airport showing  instances when PSC teams detected, and failed to detect, explosives  odor. In January 2013, we concluded that TSA could have benefited from  completing effectiveness assessments of PSCs before deploying them on  a nationwide basis to determine whether they are an effective method of  screening passengers in the U.S. airport environment.
We also reported in January 2013 that TSA had not completed an  assessment to determine where within the airport PSC teams would be  most effectively utilized, but rather TSA leadership focused on initially  deploying PSC teams to a single location within the airport—the sterile  area—because it thought it would be the best way to foster stakeholders’,  specifically airport operators’ and law enforcement agencies’, acceptance  of the teams. Stakeholders were resistant to the deployment of PSC  teams because they have civilian handlers, and TSA’s response  resolution protocols do not require the teams to be accompanied by a law  enforcement officer. According to TSA’s Assistant Administrator for the  Office of Security Operations, to alleviate airport stakeholders’ concerns  regarding TSA’s response resolution protocols, the agency initially  deployed PSC teams to the sterile areas, thereby enabling TSA to gather  data on the value of PSC teams in the airport environment while reducing  the likelihood of a final response from a PSC, since an individual has  already passed through several layers of screening when entering the  sterile area. However, aviation stakeholders we interviewed raised  concerns about this deployment strategy, stating that PSC teams would  be more effectively utilized in non-sterile areas of the airport, such as  curbside or in the lobby areas. TSA subsequently deployed PSC teams to  the passenger screening checkpoints. However, DHS S&T did not plan to  assess the effectiveness of PSCs on the public side, beyond the  checkpoint, since TSA was not planning to deploy PSCs to the public side  of the airport when DHS S&T designed its test plan. In January 2013, we  concluded that comprehensive effectiveness assessments that include a  comparison of PSC teams in both the sterile and public areas of the  airport could help TSA determine if it is beneficial to deploy PSCs to the  public side of airports, in addition to or in lieu of the sterile area and  checkpoint.
During the June 2012 assessment of PSC teams’ effectiveness, TSA  conducted one of the search exercises with three conventional canine  teams. Although this assessment was not intended to be included as part  of DHS S&T’s and TSA’s formal assessment of PSC effectiveness, the  results of the assessment suggested, and TSA officials and DHS S&T’s  Canine Explosives Detection Project Manager agreed, that a systematic  assessment of PSCs with conventional canines could provide TSA with  information to determine whether PSCs provide an enhanced security  benefit compared with conventional LEO aviation canine teams that have  already been deployed to airport terminals. In January 2013, we  concluded that an assessment would help clarify whether additional  investments for PSC training are warranted. We also concluded that since  PSC teams are trained in both conventional and passenger screening  methods, TSA could decide to convert existing PSC teams to  conventional canine teams, thereby limiting the additional resource  investments associated with training and maintaining the new PSC  teams.
We recommended that TSA expand and complete testing, in conjunction  with DHS S&T, to assess the effectiveness of PSCs and conventional  canines in all airport areas deemed appropriate prior to making additional  PSC deployments to help (1) determine whether PSCs are effective at  screening passengers, and resource expenditures for PSC training are  warranted, and (2) inform decisions regarding the type of canine team to  deploy and where to optimally deploy such teams within airports. TSA  concurred and has taken some actions to address our recommendation,  but further action is needed to fully address it. Specifically, in June 2014,  TSA reported that through its PSC Focused Training and Assessment  Initiative, a two-cycle assessment to establish airport-specific optimal  working areas, assess team performance, and train teams on best  practices, it had assessed PSC teams deployed to 27 airports, cumulating  in a total of 1,048 tests. On the basis of these tests, TSA determined that  PSC teams are effective and should be deployed at the checkpoint  queue. In February 2014, TSA launched a third PSC assessment cycle to  determine how PSCs’ effectiveness changes over time in order to  determine their optimal duration time when working the checkpoint queue  (i.e., how many minutes they can work and continue to be effective).
Although TSA has taken steps to determine whether PSC teams are  effective and where in the airport environment to optimally deploy such  teams, as of June 2014, TSA has not compared the effectiveness of  PSCs and conventional canines in order to determine if the greater cost  of training canines in the passenger screening method is warranted.  According to TSA, the agency does not plan to include conventional  canine teams in PSC assessments because conventional canines have  not been through the process used with PSC canines to assess their  temperament and behavior when working in proximity to people. While we  recognize TSA’s position that half of deployed conventional canines are  of a breed not accepted for use in the PSC program, other conventional  canines are suitable breeds, and have been paired with LEO aviation  handlers working in proximity with people since they patrol airport  terminals, including ticket counters and curbside areas. We continue to  believe that TSA should conduct an assessment to determine whether  conventional canines are as effective detecting explosives odor on  passengers when compared with PSC teams working in the checkpoint  queue. As we reported, since PSC teams are trained in both conventional  and passenger screening methods, TSA could decide to convert existing  PSC teams to conventional canine teams, thereby limiting the additional  resource investments associated with training and maintaining PSC  teams.
section_title: TSA Deployed Some PSC Teams to Highest-Risk Airports
paragraphs: In our January 2013 report, we found that TSA’s 2012 Strategic  Framework calls for the deployment of PSC teams based on risk;  however, airport stakeholder concerns about the appropriateness of  TSA’s response resolution protocols for these teams resulted in PSC  teams not being deployed to the highest-risk airports. TSA officials stated  that PSC teams were not deployed to the highest-risk airports for various  reasons, including concerns from an airport law enforcement association  about TSA’s decision to deploy PSC teams with civilian TSI handlers and  the appropriateness of TSA’s response resolution protocols. These  protocols require the canine handler to be accompanied by two additional  personnel that may, but not always, include a law enforcement officer.  According to representatives from an airport law enforcement association,  these protocols are not appropriate for a suicide bombing attempt  requiring an immediate law enforcement response. TSA’s decision to  deploy PSC teams only to airports where they would be willingly accepted  by stakeholders resulted in PSC teams not being deployed to the highest- risk airports on its high-risk list. Moreover, PSC teams that were deployed  to high-risk airports, specifically two airports we visited, were not being  used for passenger screening because TSA and the local law  enforcement agencies had not reached agreement on the PSC response  resolution protocols.
We recommended that if PSCs are determined to provide an enhanced  security benefit, TSA should coordinate with airport stakeholders to  deploy future PSC teams to the highest-risk airports, and ensure that  deployed PSC teams are utilized as intended, consistent with its statutory  authority to provide for the screening of passengers and their property.  TSA concurred with our recommendation, and has taken action to  address it. Specifically, as of June 2014, the PSC teams for which TSA  had funding and not already deployed to a specific airport at the time our  report was issued have been deployed to, or allocated to, the highest-risk  airports. According to TSA, it was successful in deploying PSC teams to  airports where they were previously declined by aviation stakeholders for  various reasons. For example, TSA officials explained that stakeholders  have realized that PSCs are an effective means for detecting explosives  odor, and no checkpoints have closed because of a nonproductive  response. PSCs also help reduce wait times at airport checkpoints  because PSC teams are one method by which TSA can operate  Managed Inclusion—a tool that allows passengers who have not, for  example, enrolled in TSA PreTM to access to PreTM screening lanes.
According to TSA, PSC teams provide an added layer of security, making  it possible for TSA to provide expedited screening to passengers who  have not enrolled in TSA PreTM and therefore have not had a  background check. In November 2013, TSA also reported it was making  progress in working with stakeholders to allow PSC teams to work at  checkpoints at airports where PSC teams were not previously performing  passenger screening, but rather were training and screening air cargo. In  June 2014, TSA officials reported that of all the airports where PSC  teams had been deployed, all but one airport agreed to allow TSA to  conduct screening of individuals at passenger screening checkpoint  queues. We believe that these actions address the intent of our  recommendation, contingent upon TSA comparing PSC teams with  conventional canine teams.
Chairman Hudson, Ranking Member Richmond, and members of the  subcommittee, this completes my prepared statement. I would be happy  to respond to any questions you may have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions about this statement, please contact Jennifer Grover at  (202) 512-7141 or GroverJ@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. Individuals making key contributions to this statement  include Chris Ferencik (Assistant Director), Chuck Bausell, Lisa Canini,  Josh Diosomito, Michele Fejfar, Eric Hauswirth, Richard Hung, Thomas  Lombardi, Jessica Orr, and Michelle Woods. Key contributors to the  previous work that this testimony is based on are listed in the report.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.