Aviation Security: TSA Has Policies that Prohibit Unlawful Profiling But Should Improve Its Oversight of Behavior Detection Activities
Released: 2019-06-04T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: In 2016, TSA began using behavior detection in a more limited way to identify potentially high-risk passengers who exhibit certain behaviors it asserts are indicative of stress, fear, or deception, and refer them for additional screening or, when warranted, to law enforcement. TSA's policies and procedures prohibit unlawful profiling, i.e., screeners are prohibited from selecting passengers for additional screening based on race, ethnicity, or other factors. Allegations of racial profiling have raised questions about TSA's use of behavior detection.

GAO was asked to review TSA's measures to prevent behavior detection activities from resulting in unlawful profiling. This report examines, among other things, (1) TSA's oversight of behavior detection activities and (2) the number of complaints alleging violations of civil rights and civil liberties related to passenger screening and actions taken by TSA to address them.

GAO reviewed TSA policies and procedures; analyzed passenger complaint data received by TSA from October 2015 through February 2018 and actions taken to address them; and interviewed TSA officials. Complaint data we analyzed alleged conduct that occurred at the screening checkpoint and was not specific to behavior detection activities.

section_title: What GAO Found
paragraphs: Transportation Security Administration (TSA) policy requires managers to ensure behavior detection is conducted without regard to race or ethnicity, among other factors. TSA uses seven oversight checklists to assess whether behavior detection activities are conducted in accordance with TSA policy, such as monitoring whether screeners trained in behavior detection observe and engage passengers correctly. However, these checklists do not instruct supervisors to monitor for indications of profiling. TSA officials stated that the training screeners receive, adherence to operating procedures, and general supervisory oversight are sufficient to alert supervisors to situations when unlawful profiling may occur. However, developing a specific mechanism to monitor behavior detection activities for compliance with policies prohibiting unlawful profiling would provide TSA with greater assurance that screeners are adhering to such policies.

From October 2015 through February 2018, TSA received about 3,700 complaints alleging civil rights and civil liberties violations related to passenger screening. These complaints are not specific to behavior detection activities. The TSA Contact Center (TCC), the office that initially receives these complaints, reported that about half of the complaints did not have complete information from passengers necessary for further review, such as the airport and date of the incident. According to TCC officials, they attempt to obtain the additional information from passengers, but often the complaint does not include the correct contact information or the passenger does not respond to the TCC's request for additional information. The TCC complaint data show that the remaining 51 percent (about 1,900) of complaints were referred to the TSA Multicultural Branch, the office responsible for reviewing complaints alleging civil rights and civil liberties violations. The Multicultural Branch reported reviewing 2,059 complaints, including approximately 1,900 complaints from TCC, as well as complaints referred from other TSA offices. For about half of the complaints (1,066) the Multicultural Branch reviewed, it found indications of potential discrimination and unprofessional conduct that involved race or other factors and recommended a range of refresher training across airports or for screeners at individual airports identified in the complaints.

TSA's Multicultural Branch Reviewed 2,059 Complaints Alleging Violations of Civil Rights and Civil Liberties from October 2015 through February 2018

section_title: What GAO Recommends
paragraphs: TSA should develop a specific oversight mechanism to monitor behavior detection activities for compliance with policies that prohibit unlawful profiling. DHS concurred with GAO's recommendation.

section_title: Letter
paragraphs: In 2006, the Transportation Security Administration (TSA) began using  behavior detection to identify potentially high-risk passengers who exhibit  certain behaviors it asserts are indicative of stress, fear, or deception, and  refer them for additional screening or, when warranted, to law  enforcement. TSA’s policies and procedures require screeners to apply  behavior detection without regard to race, ethnicity, nationality, and  religion, among other factors. Over the years, allegations of racial profiling  have raised questions about whether behavior detection officers  inappropriately consider these factors when referring passengers for  additional screening or to law enforcement. As we previously reported,  TSA and the Department of Homeland Security’s (DHS) Office of  Inspector General investigated allegations of behavior detection officers  unlawfully profiling passengers because of their race, ethnicity, or  nationality at Newark Liberty International Airport in 2010, at Honolulu  International Airport in 2011, and at Boston Logan International Airport in  2012.

In 2013, we found that TSA was unable to demonstrate that the agency’s  behavior detection activities could reliably and effectively identify high-risk  passengers who may pose a threat to the U.S. aviation system. We  recommended that TSA limit future funding support for the agency’s  behavior detection activities until TSA could provide scientifically  validated evidence that demonstrates that behavioral indicators can be  used to identify passengers who may pose a threat to aviation security.  In 2016, TSA began using behavior detection in a more limited way and in  2017, we reported that TSA had reduced funding for behavior detection  and revised its behavioral indicators. We stated that TSA should continue  to limit funding for such activities until it can provide valid evidence  demonstrating that behavioral indicators can be used to identify  passengers who may pose a threat to aviation security. In early fiscal  year 2018, TSA ended the stand-alone behavior detection program and  eliminated the behavior detection officer position. TSA also began  integrating the former behavior detection officers into the screener  workforce to support increased passenger volume at TSA’s checkpoints,  according to TSA officials. Screeners trained in behavior detection  continue to use it, largely in support of canine units.

In light of these efforts, you asked us to review the mechanisms TSA  uses to prevent unlawful profiling while screening passengers using  behavior detection. This report (1) describes how TSA trains screeners  who engage in behavior detection on policies and procedures that prohibit  unlawful profiling and evaluates the training; (2) examines TSA’s  oversight of behavior detection activities including monitoring compliance  with policies that prohibit unlawful profiling; (3) examines the number of  complaints related to passenger screening TSA received from October  2015 through February 2018 alleging violations of civil rights and civil  liberties, and actions TSA took to address them; and (4) describes how  TSA used these complaint data to inform screener training.

To address all four objectives, we reviewed relevant statutes, regulations,  and strategic documents, such as TSA’s Behavior Detection Strategic  Plan and Standard Operating Procedures. Additionally, we conducted  interviews with TSA headquarters officials from Security Operations,  Training and Development, the TSA Contact Center, and the Multicultural  Branch. In addition, we selected a nongeneralizable sample of three  airports out of 15 that offered behavior detection training classes at the  time of our review and attended the training classes: Hartsfield-Jackson  Atlanta International Airport, Dallas-Fort Worth International Airport, and  Dulles International Airport. We also interviewed TSA screeners,  managers, and the TSA designated point of contact at these airports. We  selected the airports based on their size and the number of participants  signed up to take the Optimized Behavior Detection Basic Training  course.

To determine how TSA trains screeners on policies that prohibit unlawful  profiling and evaluates the training, we reviewed TSA’s training directives,  course materials, and policies and procedures related to prohibiting  profiling during the operation of behavior detection activities from fiscal  year 2016 through November 2018. We selected this time frame because  TSA’s most current behavior detection procedures—Optimized Behavior  Detection—were approved as the new standard for behavior detection  training and operations starting in fiscal year 2017. To determine TSA’s  policies for prohibiting unlawful profiling, we reviewed various documents  pertaining to behavior detection including standard operating procedures,  the behavior detection handbook, and the behavior detection strategic  plan for fiscal years 2016 through 2018. To determine what training  screeners using behavior detection receive on the policies and  procedures that prohibit profiling in carrying out behavior detection  activities, we reviewed TSA’s 2018 National Training Plan, which lays out  annual training requirements for screeners, and TSA’s management  directive on training standards to determine how TSA developed and  selected training requirements. We also reviewed behavior detection– related National Shift Briefings, a presentation on racial profiling  developed by the Multicultural Branch in collaboration with a member of  TSA’s Multicultural Coalition, handouts, and other relevant materials  provided to screeners. Lastly, to determine how TSA evaluates training  related to behavior detection, we reviewed relevant TSA documentation  on behavior detection training, including training evaluation documents,  and we compared the documentation to the Kirkpatrick Evaluation Model  for training evaluation.

To examine TSA’s oversight of behavior detection activities including  monitoring compliance with policies prohibiting unlawful profiling, we  analyzed the Optimized Behavior Detection Standard Operating  Procedures and oversight guidance from Security Operations, which is  the office responsible for managing and overseeing the use of behavior  detection. We also reviewed the Department of Justice’s (DOJ) oversight  guidance on the use of race and other factors—guidance that DHS has  adopted. We interviewed Security Operations officials who oversee the  use of behavior detection to discuss the oversight process for monitoring  compliance with standard operating procedures. To learn about how  these practices are implemented at airports, we met with TSA officials at  each of the three airports we visited and observed how screeners trained  in behavior detection use behavioral indicators while conducting canine  and airport and airline employee screening operations. Additionally, we  reviewed TSA’s efforts in response to the Secretary of Homeland  Security’s 2012 directive to enhance data collection systems to facilitate  appropriate supervision and monitor the use of behavior detection in  response to allegations of profiling by behavior detection officers.  Further, we compared TSA’s efforts to relevant federal internal control  standards related to monitoring internal control systems.

To examine the number of complaints related to passenger screening that  TSA has received alleging violations of civil rights and civil liberties, and  steps taken by TSA to address them, we analyzed complaint data for  October 2015 through February 2018 from the TSA Contact Center (TCC)  and reviewed actions taken to address the complaints. The complaints  we reviewed were not specific to behavior detection and generally  reflected alleged conduct occurring at the screening checkpoint through  the application of screening measures. Further, complaints we reviewed  alleged discrimination or profiling based on personal attributes and  characteristics related to, among other things, an individual’s race,  ethnicity, national origin, language, gender, age, and hair. The TCC uses  the Airport Information Management (AIM) system to record complaints  and actions taken to resolve them, including referrals to the TSA  Multicultural Branch—the office responsible for reviewing complaints  alleging civil rights and civil liberties violations. We selected this time  frame because it covered the period during which TSA began  transitioning to Optimized Behavior Detection through the most recent  date for which TSA had received complaints at the time we began  reviewing the complaint data.

On the basis of documentation from and discussions with TSA officials  responsible for maintaining the data, we determined that data from AIM  were sufficiently reliable for the purpose of describing information on the  content and number of the complaints the TCC received. We also found  that the data from AIM was sufficiently reliable for reporting the status of  TCC’s review and referral of complaints related to allegations of civil  rights and civil liberties violations to the Multicultural Branch from October  2015 through February 2018.

We conducted a content analysis of the TCC complaints alleging civil  rights and civil liberties violations. In the first step, three analysts  independently coded a sample of complaints and then worked together to  resolve any discrepancies. The analysts reviewed and discussed the  complaints until they reached an agreement on the coding. During this  first step, we reviewed TSA’s codebook to understand the TCC’s process  for categorizing complaints. Based on our initial review of a subset of  complaints we developed a list of codes to categorize the complaints. The  codes we developed include certain protected classes and other personal  attributes and characteristics, as well as the alleged adverse actions  identified in the complaints.

We interviewed TSA officials from the Multicultural Branch and the  designated point of contact at the three airports we visited to obtain their  perspectives on using complaint data to monitor or enhance screening  operations. Additionally, we reviewed relevant TSA policies and  procedures related to the complaint process for complaints alleging civil  rights and civil liberties violations, such as management directives,  standard operating procedures that guide the TCC’s review and referral of  these complaints to the Multicultural Branch, and the Multicultural  Branch’s checklist for determining whether to conduct an inquiry (e.g.,  complaint was reported within a specified time, incident occurred at a  federalized airport). We analyzed data on the resolutions of the  complaints reviewed by the Multicultural Branch, which uses the  Entellitrak database to document complaint resolutions. On the basis of  information from and discussions with TSA officials related to the controls  in place to maintain the integrity of TSA’s complaint data, we determined  that data from Entellitrak were sufficiently reliable for the purpose of  describing the Multicultural Branch’s reported resolution of the complaints  it reviewed.

In addition, we analyzed a nongeneralizable sample of six complaint files  selected by the Multicultural Branch to gather more information about the  documentation, resolution, and subsequent actions for passenger  complaints.

To describe how TSA used the passenger complaint data to inform  screener training, we reviewed TSA documentation related to trainings  and briefings that were developed in response to passenger complaints  alleging violations of civil rights and civil liberties. We also interviewed  TSA officials to discuss actions taken to analyze the complaint data for  trends and to use the results of the complaint reviews to identify training  for screeners. We also reviewed how TSA shares this information with  TSA executive leadership, TSA airport customer service managers, and  screeners in the field, among others.

We conducted this performance audit from August 2017 through April  2019 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: DHS’s 2012 Memorandum on Addressing Allegations of Unlawful Profiling
paragraphs: In 2012, the Secretary of Homeland Security issued a memorandum  directing TSA to take a number of actions in response to allegations of  profiling by behavior detection officers. These actions included, among  others, working with the DHS Office of Civil Rights and Civil Liberties to  (1) review, and revise as necessary, behavior detection officer training  policies, training curriculum, and supervisory guidance to ensure they  adequately address and train against profiling; (2) enhance data  collection to facilitate appropriate supervision and monitoring of behavior  detection activities; and (3) ensure passengers are aware of complaint  mechanisms and ensure complaints are appropriately handled.

TSA has taken some actions to address these directives. For example,  TSA has  revised its standard operating procedures and training materials to  more clearly instruct personnel trained in behavior detection and other  TSA personnel on how to avoid unlawful profiling;  initiated a study to collect data on the race and national origin of  passengers referred for behavior detection screening and examine  whether disparities exist in the referral trends, and if so, whether these  differences suggest discrimination or bias in the referral process; and  issued a Management Directive establishing TSA policy and  procedures for receiving, documenting, and referring passenger  screening complaints resulting from the application of TSA security  screening policies and procedures, including processes for all  involved offices in headquarters and the field that handle passenger  complaints.

section_title: TSA’s Use of Behavior Detection
paragraphs: The Aviation and Transportation Security Act established TSA as the  federal agency with primary responsibility for securing the nation’s civil  aviation system, which includes the screening of all passengers and  property transported by commercial passenger aircraft. At the  approximately 440 TSA-regulated airports in the United States, all  passengers, their accessible property, and their checked baggage are  screened prior to boarding an aircraft or entering the sterile area of an  airport pursuant to statutory and regulatory requirements and TSA-established standard operating procedures. TSA began using behavior  detection in 2006 as an added layer of security to identify potentially high- risk passengers.

Through the end of fiscal year 2016, TSA’s behavior detection screening  process was a stand-alone program that used specially trained behavior  detection officers to observe passengers at the screening checkpoint and  engage them in brief verbal exchanges. During this period, behavior  detection officers had brief interactions with passengers in the queue  leading up to the screening checkpoint. If the behavior detection officers  determined during this interaction that a passenger exhibited a certain  number of behavioral indicators, the behavior detection officer was to  refer the passenger for additional screening or, if circumstances  warranted, contact a law enforcement officer. According to TSA  procedures, if a passenger was referred for additional screening, one  behavior detection officer conducted a pat-down of the passenger and  search of his or her personal property while another checked documents  and conversed with the passenger, attempting to understand why the  behavioral indicators were being displayed and continuing to look for  additional behavioral indicators. If a passenger did not exhibit a certain  number of additional indicators, he or she was allowed to proceed to the  boarding gate. If the passenger did exhibit a certain number of additional  indicators, or other events occurred, such as the discovery of a fraudulent  document, the behavior detection officer was to call a law enforcement  officer. The law enforcement officer then would determine next steps,  which could include questioning the passenger or conducting a criminal  background check. The law enforcement officer then determined whether  to release the passenger, refer the passenger to another law enforcement  agency, or arrest him or her.

In fiscal year 2017, consistent with the Aviation Security Act of 2016, TSA  eliminated the stand-alone behavior detection officer position. TSA transferred the former behavior detection officers to serve as part of the  screener workforce and began assigning them to the checkpoint to  screen passengers. According to TSA officials, when screeners trained in  behavior detection are assigned to a position, TSA policies and  procedures permit them to use behavior detection when applicable.  Furthermore, TSA’s checkpoint standard operating procedures do not  currently include the use of behavior detection, as behavior detection’s  use continues to be guided by its own policies established in 2016.

However, some screeners trained in behavior detection continue to use  behavior detection to support passenger screening canine teams as part  of expedited screening. As part of this process, screeners trained in  behavior detection work in conjunction with canine teams to observe  passenger behavior and identify passenger behaviors that may indicate  that a passenger poses a higher risk to the aviation system.

section_title: Overview of Optimized Behavior Detection Training
paragraphs: The Training and Development Division (Training Division), within TSA  headquarters, oversees the development, delivery, and evaluation of  training programs for TSA employees. The National Training Plan,  developed annually by the Training Division and Security Operations,  contains the core curriculum for screeners to meet their yearly training  requirements. In addition, Security Operations works with the Traveler  Engagement Division to develop and deliver specific training on topics  such as disability profiling, racial profiling, and screening transgender  persons.

In August 2017, TSA began training screeners on its new behavioral  indicators. TSA revised the behavioral indicators by eliminating and  combining some of the indicators used to observe passenger behavior,  which TSA refers to as Optimized Behavior Detection. According to TSA  officials, Optimized Behavior Detection includes 36 revised behavioral  indicators—which TSA pared down from a list of 96 indicators. As of  January 2019, TSA officials told us out of the approximately 43,000  screeners nationwide, a total of 2,541 screeners had been trained at 117  airports in Optimized Behavior Detection.

Screeners must be trained in passenger and accessible property  screening before they are eligible to attend Optimized Behavior Detection  training. Upon successful completion of Optimized Behavior Detection  training, screeners are permitted to utilize behavior detection in  accordance with the standard operating procedures, such as when  operating in conjunction with canine teams or screening airport and airline  workers. In addition, screeners must complete all requirements in the  National Training Plan which includes elements of training on TSA’s  mechanisms for preventing unlawful profiling.

section_title: TSA’s Oversight of Behavior Detection
paragraphs: TSA’s Security Operations is responsible for overseeing the use of  behavior detection. TSA’s behavior detection policies and procedures  prohibit screeners from selecting passengers for additional screening  based on race, ethnicity, religion, and other factors, whether through  behavior detection or other security measures. This responsibility  includes overseeing officers trained in behavior detection to ensure they  conduct behavior detection without regard to race/ethnicity, color,  gender/sex, gender identity, religion, national origin, sexual orientation, or  disability, in accordance with constitutional, statutory, regulatory, and  other legal and DHS policy requirements to protect the civil rights and civil  liberties of individuals. Although the stand-alone behavior detection officer  position was eliminated and the program ended in 2017, the requirement  to conduct oversight and verify compliance with TSA policies still applies  when behavior detection is used, such as when behavior detection is  used in conjunction with passenger screening canine teams. According to  TSA’s policies and procedures, supervisors must conduct oversight  observations of behavior detection activities a minimum of 8 hours every  14 days to verify and document compliance with behavior detection  policies, standard operating procedures, the handbook, and training,  among other things, and submit a compliance checklist documenting the  review to TSA Security Operations.

section_title: Passenger Complaint Review and Referral Process
paragraphs: The TSA Contact Center (TCC) is the primary point of contact for  collecting, documenting, and responding to public questions, concerns, or  complaints regarding passengers’ screening experience; reports and  claims of lost, stolen, or damaged items; and complaints submitted by  TSA employees. The TCC may refer screening complaints for resolution  to other TSA headquarters offices, depending on the specific allegation.  For example, complete complaints alleging violations of civil rights and  civil liberties, which include allegations implicating color, race, ethnicity,  gender, genetic information, national origin, religion, sexual orientation,  and parental status, must be referred to the Multicultural Branch. Figure 1  describes the TCC’s complaint review process.

TSA’s Multicultural Branch is responsible for collecting, monitoring, and  adjudicating passenger complaints alleging civil rights and civil liberties  violations at the passenger screening checkpoint, including complaints  alleging unlawful profiling and discrimination, among other things. The  Multicultural Branch receives complaints alleging civil rights and civil  liberties violations from several sources within TSA including the TCC.  When TCC officials determine a complete complaint involves a potential  civil rights or civil liberties violation, they are to forward the complaint to  the Multicultural Branch where staff are to input the complaint into a  database and track the resolution of each complaint they receive. The  Multicultural Branch, in consultation with Security Operations, determines  whether a screener followed standard operating procedures while  screening the complainant by reviewing available video of an incident or  interviewing witnesses who saw the incident.

Depending on the nature and severity of the allegation, TSA airport staff  may also elevate the complaint and evidence to the airport’s Assistant  Federal Security Director (FSD) for Screening. If the investigation finds  fault with the screener, the screener’s supervisor or manager is to  determine the corrective action to be taken. Corrective actions specified  in TSA’s guidelines for disciplinary actions to address misconduct range  from mandating that the screener take additional training to correct the  behavior to terminating the screener’s employment for multiple repeat  offenses or a single egregious action. Following the outcome of the  complaint review and any resulting corrective actions, the TSA  headquarters unit or the TSA customer support manager at the airport is  to communicate the status of the resolution, if any, to the complainant— such as by using a template letter that explains TSA’s policies and  procedures or issuing an apology. According to Multicultural Branch  protocols for reviewing passenger complaints, complaints may be  resolved in three ways:

Closed-Administratively: If the complainant does not respond within  10 days to the Multicultural Branch’s first contact for additional  information, such as a request for additional information on the  alleged civil rights and civil liberties violation, the complaint is to be  closed.

Closed-No Jurisdiction: Complaints that are not within the Multicultural  Branch’s jurisdiction, such as complaints involving rude and  unprofessional conduct that are not related to allegations of civil rights  and civil liberties violations, are to be closed and referred to other TSA  offices or the TSA designated point of contact at the airport for further  handling.

Closed-Resolved: Following the outcome of the investigation, the  Multicultural Branch is to send a letter to the complainant summarizing  the allegations reviewed, explaining whether TSA procedures were  followed, and in some cases, issuing an apology or informing the  complainant of the type of training offered to the screener(s). The  Multicultural Branch may recommend training and provide refresher  training materials for distribution at the airport to the screener(s)  involved, if identified, or for all screeners at the airport’s checkpoint at  which the complaint originated. According to TSA officials, the  Multicultural Branch recommends training when standard operating  procedures for screening were not followed or when it determines that  the proactive measure of refresher training would be useful. According  to TSA, the designated TSA point of contact at the airport is required  to verify when the training is completed.

section_title: Screeners Using Behavior Detection Receive Basic and Recurrent Training Related to Profiling, and TSA Evaluates Training Effectiveness Using the Kirkpatrick Model
subsections: section_title: Screeners Conducting Behavior Detection Receive Training on TSA’s Policies and Procedures That Prohibit Unlawful Profiling
paragraphs: Before screeners are eligible to conduct any behavior detection activities,  they must first complete a 5-day Optimized Behavior Detection Basic  Training course, and undergo on-the-job training at their local airport. This  course includes an overview of DHS and TSA policies that prohibit  unlawful profiling, and trains screeners to apply behavioral indicators to  passengers without regard to race/ethnicity, color, gender/sex, gender  identity, religion, national origin, sexual orientation, or disability.  Participants must complete the Optimized Behavior Detection Basic  Training course and pass a 40 question job knowledge test at the end of  the class, in addition to completing 32 hours of on-the-job training under  the supervision of an officer already trained in behavior detection. If a  participant fails the job knowledge test, he or she is to receive 1 hour of  remedial training before retaking the test. Screeners must pass the test in  two attempts to be eligible to conduct behavior detection activities.

In the four Optimized Behavior Detection Basic Training courses we  attended, the training instructors covered TSA’s policies on prohibiting  unlawful profiling on day one of the course, and explained that profiling  passengers based on discernible traits was not only illegal, but that such  practices are ineffective at identifying potentially high-risk passengers. In  addition, the course manual included a copy of DHS’s 2013 memorandum  defining racial profiling, which all participants were required to review.  To test their understanding of TSA policy and the Optimized Behavior  Detection Standard Operating Procedures, the instructors presented  various scenarios to engage participants in practicing how they would  apply behavior detection at the checkpoint.

The 2018 National Training Plan required behavior detection–trained  screeners to complete four recurrent technical training courses related to  behavior detection, including two that contain material reinforcing DHS’s  and TSA’s policies prohibiting unlawful profiling. Screeners participate in  each of the four interactive training courses using a computer and the  courses contain knowledge checks that the participant must answer  correctly before completing the training. Table 1 describes the training  courses screeners trained in behavior detection are required to complete  and appendix I includes a list of additional training related to unlawful  profiling.

section_title: TSA Evaluates Training Courses Using the Kirkpatrick Evaluation Model
paragraphs: TSA determines the effectiveness of particular training programs using  the Kirkpatrick Evaluation Model, a commonly accepted training  evaluation model endorsed by the Office of Personnel Management and  used throughout the federal government. In May 2018, TSA updated its  training standards based on the ADDIE model, a methodology comprising  five phases: Analysis, Design, Development, Implementation, and  Evaluation (ADDIE). TSA uses the Kirkpatrick model as part of the  evaluation stage of ADDIE. The Kirkpatrick model consists of a four-level  approach for soliciting feedback from training course participants and  evaluating the impact the training had on individual development, among  other things.

TSA conducts Levels 1 and 2 evaluations on selected training courses.  Table 2 provides an overview of the Kirkpatrick model and the evaluation  levels for courses related to behavior detection and unlawful profiling.

TSA officials told us they will continue to evaluate the Optimized Behavior  Detection Basic Training course and Level 3 evaluations are under  development, as they roll out their training evaluation process. According  to TSA’s Training Standards, a review team determines the frequency of  curriculum review, which should occur at least once every 5 years. As  part of this review, TSA plans to leverage data reported in evaluations at  Kirkpatrick Levels 1 through 3.

section_title: TSA Has Oversight Policies for Behavior Detection and Prohibits Unlawful Profiling but Does Not Specifically Assess Whether Profiling Occurs
paragraphs: TSA’s 2016 Optimized Behavior Detection Program Handbook and  Operational Oversight Compliance Guidance require supervisors to  conduct routine checks of behavior detection operations to monitor  compliance with standard operating procedures. TSA’s behavior detection  Operational Oversight Compliance Guidance outlines seven specific  assessments of behavior detection operations and includes a checklist for  each assessment for managers to document completion of these routine  oversight tasks. According to TSA officials, these assessments should  occur when screeners use behavior detection in conjunction with canine  operations and while screening airline and airport workers, among other  activities. When conducting these assessments, supervisors are to  conduct 1-hour observations and use detailed checklists to document  how screeners trained in behavior detection perform the behavior  detection in practice. For example, one checklist requires supervisors to  observe how screeners trained in behavior detection monitor passenger  flow and communicate with passengers while observing for behavioral  indicators, such as ensuring screeners using behavior detection do not  ask passengers intrusive or offensive questions, among other activities  related to the use of behavior detection.

However, our review of the oversight checklists found that they do not  specifically instruct supervisors to monitor for compliance with procedures  intended to prohibit unlawful profiling. According to TSA officials, TSA’s  guidance and checklists do not include this type of monitoring for unlawful  profiling because officials believe that the training screeners receive,  adherence to the standard operating procedures, and the general  supervisory oversight in place are sufficient to prevent unlawful profiling  and could alert supervisors to situations where unlawful profiling happens.  However, the 2013 DHS memorandum on DHS’s policy on unlawful  profiling states that each component, including TSA, should both  implement specific policy and procedures on racial profiling, and ensure  all personnel are trained and held accountable for meeting the standards  set forth in DHS policy. In addition, Standards for Internal Control in the  Federal Government states that management should establish and  implement activities to monitor the internal control system and evaluate  the results, as well as remediate identified internal control deficiencies.  Such a mechanism could be an item added to a checklist for supervisors  to document, based on their observations, whether screeners selected  individuals for additional scrutiny in a manner consistent with policies and  procedures.

Another oversight mechanism, as noted in DOJ’s guidance on the use of  race and other factors, could be studying the implementation of policies  and procedures that prohibit unlawful profiling through targeted, data- driven research projects. As previously discussed, in 2013, TSA initiated  a study and collected data through October 2017 on passengers referred  for secondary screening to monitor compliance with policies that prohibit  unlawful profiling. TSA discontinued the study and did not analyze the  data collected because the stand-alone behavior detection program  ended in November 2017. As a result of not conducting the analysis, TSA  does not know what the data would have shown regarding compliance  with policies that prohibit unlawful profiling.

TSA officials said they plan to update the behavior detection and  checkpoint screening policies, procedures, and guidance during fiscal  year 2019. As a part of this update, TSA officials told us they plan to  include language in the standard operating procedures reinforcing the use  of behavior detection simultaneously with other checkpoint duties, such  as the document checker position. However, TSA officials told us they are  not planning to add an oversight mechanism specific to profiling as part of  the updates because, as previously noted, they believe screener training,  adherence to the standard operating procedures, and general supervisory  oversight are sufficient. Developing a specific oversight mechanism,  such as a checklist or a data-driven study, to monitor screeners’  compliance with policies that prohibit unlawful profiling would provide TSA  with greater assurance that its personnel are adhering to these policies  when using behavior detection, and better position TSA to identify  potential incidents of unlawful profiling.

section_title: TSA Received About 3,700 Complaints Alleging Violations of Civil Rights and Civil Liberties from October 2015 to February 2018 and Recommended Screener Training to Address Complaints
subsections: section_title: The TCC Received 3,663 Complaints Related to Passenger Screening and a Majority of the Complaints Alleged Discrimination or Profiling Based on Personal Attributes and Characteristics
paragraphs: The TCC received 3,663 complaints related to passenger screening  alleging violations of civil rights and civil liberties from October 2015  through February 2018. These complaints are not specific to behavior  detection activities and generally reflect alleged conduct occurring at the  screening checkpoint through the application of screening measures. We  analyzed the 3,663 complaints and found that the majority (2,251 of  3,663) of the complaints alleged discrimination or profiling based on  personal attributes and characteristics. For example, the TCC received  complaints alleging discrimination that involved assertions by passengers  that they had been selected for pat-downs based on race and ethnicity,  among other reasons, when the passengers believed they did not trigger  an alarm prompting the pat-downs. The TCC also received complaints  related to passengers’ transgender identity alleging selection for  additional screening because of their transgender status. Additionally, the  TCC received passenger complaints alleging that screening procedures  were aggressive or inappropriate for senior citizens. Table 3 provides a  list of complaint types based on our analysis. In addition, appendix II  provides additional detail about our content analysis of complaints  alleging civil rights and civil liberties violations, and appendix III provides  a list of 10 airports most often identified in the complaints.

As TSA’s primary point of contact for passenger complaints, the TCC is  responsible for the initial review and referral of all complaints that involve  allegations of civil rights and civil liberties violations to the Multicultural  Branch. According to the TCC standard operating procedures, TCC  analysts review the complaints to ensure that they contain the necessary  information to be considered complete, including the airport, passenger’s  name, date of the incident, and description of the alleged civil rights and  civil liberties violation. In addition, complaints reported over the phone or  made on behalf of another person without the person’s consent are  initially considered incomplete. For complaints that are not complete,  the TCC sends the passenger a document request for information when  the passenger has provided correct contact information. According to  TCC officials, passengers often do not provide the correct contact  information or do not respond with the necessary information to complete  the complaint. TCC officials said that incomplete complaints are typically  sent to the Multicultural Branch for informational purposes. Multicultural  Branch officials told us that they consider information from incomplete  complaints to inform its policy and training initiatives, and to improve how  TSA engages with the public.

From October 2015 through February 2018, the TCC referred 51 percent  (1,865) of the 3,663 complaints it received to the Multicultural Branch for  review. The TCC reported that 48 percent (1,764) of the 3,663 complaints  did not have complete information necessary for further review, such as  the airport and date of the incident. According to TSA officials, these  complaints were sent to the Multicultural Branch for informational  purposes. TCC’s passenger complaint data show that the remaining 1  percent (34) of the complaints were from TSA employees and were  referred to other TSA offices for review.

section_title: TSA’s Multicultural Branch Reviewed More Than 2,000 Complaints and Recommended a Range of Screener Training
paragraphs: TSA’s Multicultural Branch receives and reviews complete complaints  related to allegations of violations of civil rights and civil liberties that are  referred to it from the TCC, DHS’s Office of Civil Rights and Civil  Liberties, TSA’s Disability Branch, and TSA personnel at airports. From  October 2015 through February 2018, the Multicultural Branch received  2,059 complaints alleging violations of civil rights and civil liberties, as  shown in figure 2. Multicultural Branch officials stated that the majority of  these complaints were referred from the TCC.

As shown in figure 2, for 1,066 (52 percent) of the complaints,  Multicultural Branch staff found indications of potential discrimination,  such as instances of rude or unprofessional conduct that included the use  of race or other protected characteristics. According to Multicultural  Branch staff, to resolve the 1,066 complaints, they recommended a range  of refresher training. Multicultural Branch staff explained that when issues  are identified, their policy is to address the issues through screener  training. Multicultural Branch officials reported that these trainings were  provided through National Shift Briefings, which were circulated across  TSA, or through training provided at a particular airport. For example:  In one of the complaint cases we reviewed, a passenger alleged  profiling based on headwear. Multicultural Branch officials used  camera recordings and statements from officers involved in the  encounter to substantiate that screening procedure violations had  occurred. As a result, Multicultural Branch officials recommended  refresher training to the airport on headwear screening protocols for  all screeners at the airport to review.

In another complaint case we reviewed, a passenger alleged profiling  based on the use of a tribal-issued photo identification card. In  response, Multicultural Branch officials sent refresher training on  verifying tribal identification and the screening of Native American  passengers to the TSA designated point of contact at the airport  involved for distribution to TSA personnel identified in the complaint.

In a third complaint reviewed, a passenger alleged being profiled at  the screening checkpoint, without including any additional details.  According to TSA officials, based on the particular allegations of the  complaint and the lack of details, TSA was unable to substantiate the  allegations made in the complaint. As a result, Multicultural Branch  sent National Shift Briefings on TSA’s policies and procedures that  prohibit unlawful profiling and inappropriate comments to the TSA  designated point of contact at the airport involved for distribution to  TSA personnel identified in the complaint.

As shown in figure 2, there were 993 complaints that the Multicultural  Branch reviewed but did not address through training. The Multicultural  Branch closed 121 of these complaints because it determined that the  complainant did not provide sufficient information about the alleged civil  rights and civil liberties violation for Multicultural Branch review and the  complainant did not respond with additional information requested by the  Multicultural Branch within 10 days. The Multicultural Branch determined  that the remaining 872 complaints were not substantiated based on its  review of the camera recording of the alleged incident, or were not within  its jurisdiction. For the complaints not within its jurisdiction, the  Multicultural Branch referred them to other TSA offices, to TSA officials at  the airport or airports identified in the complaints for review, or to other  federal agencies (e.g., U.S. Customs and Border Protection, Department  of Transportation, or the Federal Aviation Administration) as appropriate.  These complaints involved allegations of unprofessional conduct and  other issues that did not involve allegations of civil rights and civil liberties  violations.

According to Multicultural Branch guidance, the designated TSA point of  contact at the airport along with the Multicultural Branch analyst are to  determine appropriate next steps for resolving complaints, such as  preparing a briefing for screeners that is tailored to address the concerns  raised by the complainant. TSA officials stated that resolutions to the  complainant are tailored to reflect the allegation, type of inquiry  conducted, and investigation of the facts and evidence underlying the  complaint. TSA’s responses to the complainant include, but are not  limited to, apologizing for the screening experience or informing the  complainant about the next steps such as the agency’s plans to address  the complaint or underlying conduct that gave rise to the complaint. For  example, in a letter we reviewed, TSA apologized for the “unprofessional  and inappropriate personal questions” the passenger experienced during  screening, and stated that refresher training would be distributed to  screeners at the airport involved. According to documentation we  reviewed related to this complaint, the Multicultural Branch sent refresher  training materials on avoiding inappropriate comments to the designated  TSA point of contact at the airport involved. In addition, TSA’s office of  Human Capital Employee Relations reported that it took a range of  disciplinary actions—from letters of reprimand to termination—for 100  screeners from October 2015 through February 2018, in part in response  to passenger complaints alleging civil rights and civil liberties violations.

section_title: TSA’s Multicultural Branch Analyzes and Shares Passenger Complaint Data to Inform Screener Training
paragraphs: TSA’s Multicultural Branch regularly collects and analyzes data on  passenger civil rights and civil liberties and discrimination complaints and  their resolution status, and shares this information with TSA executive  leadership, TSA airport customer service managers, and screeners in the  field, among others. Multicultural Branch officials told us their staff are  assigned to specific airports based on geographic region, and they  continually analyze passenger complaints referred to their office from the  TCC to identify trends. Staff members meet weekly to discuss trends in  complaints for their geographic regions, and they review weekly,  quarterly, and annual reports on the number and category of complaints  referred to their office by the TCC. In addition, Multicultural Branch  officials track the resolution of the cases for which they have jurisdiction  and submit this information to their senior leadership each week.  Specifically, the Multicultural Branch uses a database to track complaints  by type, airport, submission date, and resolution status, such as how  many cases are open, closed, or whether they have been resolved.

Multicultural Branch officials share trends in complaints throughout TSA in  several ways, including conference calls, monthly briefings, reporting  metrics to TSA executive leadership, and on-site training events at  airports each year. For example, Multicultural Branch officials hold  monthly conference calls with customer service managers at airports to  review complaint trends, upcoming on-site airport trainings, and job aids  they have developed to help screeners understand issues, such as  screening passengers wearing religious headwear. Multicultural Branch  officials stated they also share information with screeners and supervisors  through National Shift Briefings that are distributed at all airports, and  focus on bringing awareness to screeners on events they need to be  aware of when screening passengers, such as religious observances  occurring that month.

According to TSA officials, the Multicultural Branch uses its analysis of  passenger complaints and the results of complaint investigations to  develop training aids and materials on areas where they determine  screeners need more training, such as multicultural awareness or  screening of transgender passengers. For example, the Multicultural  Branch has developed briefings focusing on unlawful profiling and  unconscious bias which reiterated that unlawful profiling is against TSA  policy, defined unconscious bias, and provided scenario-based examples.  Additionally, members from the Multicultural Branch hold on-site training  for screeners at selected airports each year based on complaint data  analysis and other factors. These training sessions last three days,  include topics stemming from complaint data TSA has analyzed, and can  include webinars, role-playing, and other forms of instruction.

section_title: Conclusions
paragraphs: DHS and TSA have policies prohibiting unlawful profiling—using race,  ethnicity, gender, or other protected characteristics to identify passengers  for additional screening—when using behavior detection, as well as other  screening measures. While TSA has oversight guidance and checklists to  monitor screeners’ use of behavior detection, these policies and  procedures do not include a specific mechanism to monitor whether  screeners may be using behavior detection to unlawfully profile  passengers. Although TSA officials report that they are working to update  the standard operating procedures in 2019, they currently have no plan to  add a specific mechanism to monitor compliance with policies that  prohibit unlawful profiling. Developing a specific oversight mechanism  would provide TSA with greater assurance that screeners are adhering to  such policies and help TSA identify any potential incidents of unlawful  profiling.

section_title: Recommendation for Executive Action
paragraphs: We are making the following recommendation to TSA.

The TSA Administrator should direct Security Operations to develop a  specific oversight mechanism to monitor the use of behavior detection  activities for compliance with DHS and TSA policies that prohibit unlawful  profiling. (Recommendation 1)

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DHS for review and comment. DHS  provided written comments which are reproduced in appendix IV. In its  comments, DHS concurred with our recommendation and described  actions planned to address it. Security Operations, TCC, and the  Multicultural Branch also provided technical comments, which we  incorporated as appropriate.

DHS correctly noted in its letter that GAO’s analysis of civil rights and civil  liberties complaints related to every aspect of TSA’s passenger and  baggage screening and is not specific to behavior detection. We agree  with DHS’s observation, as this analysis provides information on what  passengers alleged in their complaints and how TSA addressed them. It  is important to note that the complaint data provided by TSA did not  preclude behavior detection activities as a potential contributing factor to  any number of the complaints submitted.

With regard to our recommendation, that the TSA Administrator should  direct Security Operations to develop a specific oversight mechanism to  monitor the use of behavior detection activities for compliance with DHS  and TSA policies that prohibit unlawful profiling, DHS stated that TSA  plans to take additional steps to continue to ensure behavior detection  activities adhere to polices that prohibit unlawful profiling. In fiscal year  2019, TSA plans to modify existing oversight checklists used by  managers and supervisors to include specific terminology for monitoring  unlawful profiling. DHS estimated that this effort would be completed by  September 30, 2019. This action, if fully implemented, should address the  intent of the recommendation.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees and the Secretary of Homeland Security. In  addition, the report will be available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  William Russell at (202) 512-8777 or RussellW@gao.gov. Contact points  for our Offices of Congressional Relations and Public Affairs may be  found on the last page of this report. GAO staff who made key  contributions to this report are listed in appendix V.

section_title: Appendix I: Additional Training Related to Unlawful Profiling
paragraphs: The Transportation Security Administration (TSA) provided examples of  refresher training materials that are provided to screeners on TSA’s  prohibition on the use of unlawful profiling at the passenger screening  checkpoint. Table 4 provides information on these materials, including the  methods used to distribute the materials to screeners.

section_title: Appendix II: GAO Analysis of Complaints Submitted to the Transportation Security Administration Contact Center
paragraphs: From October 2015 through February 2018, the Transportation Security  Administration (TSA) Contact Center (TCC) received 3,663 complaints  that it classified as alleging violations of civil rights and civil liberties. Of  the 3,663 complaints, the TCC received 707 complaints, or about 19  percent, by phone. Table 5 summarizes our analysis of the complaints  the TCC received.

section_title: Appendix III: GAO Analysis of Complaints Submitted to the Transportation Security Administration Contact Center by Airport
paragraphs: From October 2015 through February 2018, the Transportation Security  Administration (TSA) Contact Center (TCC) received 3,663 complaints  that it classified as alleging violations of civil rights and civil liberties. The  TCC received 707 of these complaints, or about 19 percent, by phone.  Of the 3,663 complaints, Los Angeles International Airport was identified  most often in the complaint data. Table 6 lists the 10 airports most often  identified in these complaints.

section_title: Appendix IV: Agency Comments

section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: William Russell, (202) 512-8777 or RussellW@gao.gov.

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Ellen Wolfe (Assistant Director),  Natalie Maddox (Analyst in Charge), Saida Hussain, and Brendan  Kretzschmar made key contributions to this report. Also contributing to  the report were Alyssa Bertoni, David Dornisch, Ben Emmel, Eric  Hauswirth, Susan Hsu, Tom Lombardi, Amanda Miller, Sam Portnow,  Rachel Stoiko, and Adam Vogt.

Fast facts: TSA says that its screeners may refer passengers for additional screening if they exhibit behaviors that indicate stress, fear, or deception. Screeners are prohibited from picking passengers based on factors such as race or ethnicity.

TSA has guidance and checklists to monitor screeners’ use of behavior detection, but should better target unlawful profiling.

We recommended TSA develop additional oversight specifically focused on monitoring behavior detection activities for compliance with policies that prohibit unlawful profiling.

Question-summary annotations: questions: question: What is the TSA policy regarding bias or profiling in behavior detection?
answer: Transportation Security Administration (TSA) policy requires managers to ensure behavior detection is conducted without regard to race or ethnicity, among other factors.
child_questions: question: What resources does the TSA have for assessing behavior detection?
answer: TSA uses seven oversight checklists to assess whether behavior detection activities are conducted in accordance with TSA policy, such as monitoring whether screeners trained in behavior detection observe and engage passengers correctly.
child_questions: question: Where does this checklist fall short?
answer: However, these checklists do not instruct supervisors to monitor for indications of profiling.

question: What comprised TSA's statement about bias training?
answer: TSA officials stated that the training screeners receive, adherence to operating procedures, and general supervisory oversight are sufficient to alert supervisors to situations when unlawful profiling may occur.
child_questions: question: What would be a more appropriate action for TSA to take?
answer: However, developing a specific mechanism to monitor behavior detection activities for compliance with policies prohibiting unlawful profiling would provide TSA with greater assurance that screeners are adhering to such policies.

questions: question: How did the TSA begin behavior detection?
answer: In 2016, TSA began using behavior detection in a more limited way to identify potentially high-risk passengers who exhibit certain behaviors it asserts are indicative of stress, fear, or deception, and refer them for additional screening or, when warranted, to law enforcement.

question: How does TSA prohibit profiling?
answer: TSA's policies and procedures prohibit unlawful profiling, i.e., screeners are prohibited from selecting passengers for additional screening based on race, ethnicity, or other factors.
child_questions: question: What raised suspicion about bias in TSA behavior detection?
answer: Allegations of racial profiling have raised questions about TSA's use of behavior detection.