Screening Partnership Program: TSA Issued Application Guidance and Developed a Mechanism to Monitor Private versus Federal Screener Performance
Published: 2014-01-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: TSA maintains a federal workforce to screen passengers and baggage at the majority of the nation's commercial airports, but it also oversees a workforce of private screeners at airports who participate in the SPP. The SPP allows commercial airports to apply to have screening performed by private screeners, who are to provide a level of screening services and protection that equals or exceeds that of federal screeners. In recent years, TSA's SPP has evolved to incorporate changes in policy and federal law, prompting enhanced interest in measuring screener performance.
This testimony addresses the extent to which TSA (1) has provided guidance to airport operators for the SPP application process and (2) assesses and monitors the performance of private and federal screeners. This statement is based on a report GAO issued in December 2012 and selected updates conducted in January 2014. To conduct the selected updates, GAO reviewed documentation, such as the SPP Annual Report issued in January 2013, and interviewed agency officials on the status of implementing GAO's recommendations.
section_title: What GAO Found
paragraphs: Since GAO reported on this issue in December 2012, the Transportation Security Administration (TSA) has developed application guidance for airport operators applying to the Screening Partnership Program (SPP). In December 2012, GAO reported that TSA had not provided guidance to airport operators on its application and approval process, which had been revised to reflect requirements in the Federal Aviation Administration Modernization and Reform Act of 2012. Further, airport operators GAO interviewed at the time generally stated that they faced difficulties completing the revised application, such as how to obtain cost information. Therefore, GAO recommended that TSA develop application guidance, and TSA concurred. To address GAO's recommendation, TSA updated its SPP website in December 2012 by providing general application guidance and a description of the criteria and process the agency uses to assess airports' SPP applications. The guidance provides examples of information that airports could consider providing to TSA to help assess their suitability for the program and also outlines how the agency will analyze cost information. The new guidance addresses the intent of GAO's recommendation and should help improve transparency of the SPP application process as well as help airport operators determine whether their airports are good candidates for the SPP.
TSA has also developed a mechanism to regularly monitor private versus federal screener performance. In December 2012, GAO found differences in performance between SPP and non-SPP airports based on its analysis of screener performance data. However, while TSA had conducted or commissioned prior reports comparing the performance of SPP and non-SPP airports, TSA officials stated at the time that they did not plan to conduct similar analyses in the future, and instead stated that they were using across-the-board mechanisms to assess screener performance across all commercial airports. In December 2012, GAO found that these across-the-board mechanisms did not summarize information for the SPP as a whole or across years, which made it difficult to identify changes in private screener performance. GAO concluded that monitoring private screener performance in comparison with federal screener performance was consistent with the statutory provision authorizing TSA to enter into contracts with private screening companies and recommended that TSA develop a mechanism to regularly monitor private versus federal screener performance. TSA concurred with the recommendation. To address GAO's recommendation, in January 2013, TSA issued its first SPP Annual Report, which provides an analysis of private versus federal screener performance. Further, in September 2013, a TSA Assistant Administrator signed an operations directive that provides internal guidance for preparing the SPP Annual Report, including the requirement that the report annually verify that the level of screening services and protection provided at SPP airports is equal to or greater than the level that would be provided by federal screeners. These actions address the intent of GAO's recommendation and could assist TSA in identifying performance changes that could lead to improvements in the program.
section_title: What GAO Recommends
paragraphs: GAO is making no new recommendations in this statement.

section_title: Letter
paragraphs: I appreciate the opportunity to discuss our work on the Transportation  Security Administration’s (TSA) Screening Partnership Program (SPP).  TSA, a component of the Department of Homeland Security (DHS), is  responsible for ensuring the security of the traveling public through,  among other things, screening passengers traveling by aircraft for  explosives and other prohibited items. To fulfill this responsibility, TSA  maintains a federal workforce of screeners at a majority of the nation’s  commercial airports, but also oversees a smaller workforce of private  screeners employed by companies under contract to TSA at airports that  participate in TSA’s SPP. The SPP, established in 2004 in accordance  with provisions of the Aviation Transportation Security Act (ATSA), allows  commercial airports an opportunity to “opt out” of federal screening by  applying to TSA to have private screeners perform the screening  function. At airports with private screeners, TSA continues to be  responsible for overseeing airport screening operations and ensuring that  the private contractors provide effective and efficient security operations  in a manner consistent with law and other TSA requirements; however,  the screening of passengers and baggage at these airports is performed  by private screening contractors selected and approved by TSA.January 2014, there are 14 airports participating in the SPP, as well as 6   As of  airports approved for participation and awaiting the selection of a  screening contractor.
The standard by which TSA evaluates airport applications for participation  in the SPP has changed since the program’s inception in 2004. First, in  January 2011, the TSA Administrator announced his decision not to  expand the SPP beyond the 16 airports that were participating in the  program at that time “unless a clear and substantial advantage to do so  emerges in the future.” In so doing, the Administrator cited his interest in  helping the agency evolve into a “more agile, high-performing  organization that can meet the security threats of today and the future” as  the reason for his decision. Of the 6 airports that submitted applications  from March 2009 through January 2012 and that were evaluated under  the “clear and substantial advantage” standard, TSA approved the  application of 1 airport and denied the applications of the other 5. Second,  according to TSA officials, the Federal Aviation Administration  Modernization and Reform Act of 2012 (FAA Modernization Act), enacted  in February 2012, prompted TSA to change the standard by which it  evaluates SPP applications and requires, among other things, that the  TSA Administrator approve an SPP application submitted by an airport  operator if the Administrator determines that the approval would not  compromise security or detrimentally affect the cost-efficiency or the  effectiveness of the screening of passengers or property at the airport.
My testimony today addresses the extent to which TSA (1) has provided  guidance to airport operators for the SPP application process and (2)  assesses and monitors the performance of private and federal screeners.  This statement is based on our December 2012 report and selected  updates on the status of recommendations made in our December 2012  report. For our December 2012 report, we analyzed past and current  SPP application forms and instructions and interviewed airport operators,  screeners, SPP contractors, SPP applicants, TSA headquarters officials,  and Federal Security Directors (FSD). We also compared TSA’s process  for reviewing applications and monitoring performance of private and  federal screeners against Standards for Internal Control in the Federal  Government to determine if any improvements were needed. To  determine how screener performance compares at SPP and non-SPP  airports, we compared screener performance for the 16 SPP airports that  were participating in the SPP at that time to the average performance of  other airports in their category, as well as nationally, from fiscal year 2009  through 2011. More detailed information on the scope and methodology  appears in our December 2012 report. To update our work, we obtained  related documentation, such as the SPP Annual Report issued in January  2013, and interviewed agency officials in January 2014 on progress made  to implement the recommendations from our December 2012 report  related to application guidance and monitoring of private versus federal  screener performance. We conducted the work on which this statement is  based in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: On November 19, 2002, pursuant to ATSA, TSA began a 2-year pilot  program at 5 airports using private screening companies to screen  passengers and checked baggage. In 2004, at the completion of the  pilot program, and in accordance with ATSA, TSA established the SPP,  whereby any airport authority, whether involved in the pilot or not, could  request a transition from federal screeners to private, contracted  screeners. All of the 5 pilot airports that applied were approved to  continue as part of the SPP, and since its establishment, 21 additional  airport applications have been accepted by the SPP.
In March 2012, TSA revised the SPP application to reflect requirements  of the FAA Modernization Act, enacted in February 2012. Among other provisions, the act provides that
Not later than 120 days after the date of receipt of an SPP application  submitted by an airport operator, the TSA Administrator must approve  or deny the application.
The TSA Administrator shall approve an application if approval would  not (1) compromise security, (2) detrimentally affect the cost-efficiency  of the screening of passengers or property at the airport, or (3)  detrimentally affect the effectiveness of the screening of passengers  or property at the airport.
Within 60 days of a denial, TSA must provide the airport operator, as  well as the Committee on Commerce, Science, and Transportation of  the Senate and the Committee on Homeland Security of the House of  Representatives, a written report that sets forth the findings that  served as the basis of the denial, the results of any cost or security  analysis conducted in considering the application, and  recommendations on how the airport operator can address the  reasons for denial.
All commercial airports are eligible to apply to the SPP. To apply, an  airport operator must complete the SPP application and submit it to the  SPP Program Management Office (PMO), as well as to the FSD for its  airport, by mail, fax, or e-mail. Figure 1 illustrates the SPP application  process.
Although TSA provides all airports with the opportunity to apply for  participation in the SPP, authority to approve or deny the application  resides in the discretion of the TSA Administrator. According to TSA  officials, in addition to the cost-efficiency and effectiveness considerations  mandated by FAA Modernization Act, there are many other factors that  are weighed in considering an airport’s application for SPP participation.  For example, the potential impacts of any upcoming projects at the airport  are considered. Once an airport is approved for SPP participation and a  private screening contractor has been selected by TSA, the contract  screening workforce assumes responsibility for screening passengers  and their property and is required to adhere to the same security  regulations, standard operating procedures, and other TSA security  requirements followed by federal screeners at non-SPP airports.
section_title: TSA Has Developed Program Application Guidance to Help Improve Transparency of Its Process and Assist Airports in Completing their Applications
paragraphs: Since our December 2012 report, TSA has developed guidance to assist  airport operators in completing their SPP applications, as we  recommended. In December 2012, we reported that TSA had developed  some resources to assist SPP applicants, but it had not provided  guidance on its application and approval process to assist airports. As the  application process was originally implemented in 2004, the SPP  application process required only that an interested airport operator  submit an application stating its intention to opt out of federal screening  as well as its reasons for wanting to do so. In 2011, TSA revised its SPP  application to reflect the “clear and substantial advantage” standard  announced by the Administrator in January 2011. Specifically, TSA  requested that the applicant explain how private screening at the airport  would provide a clear and substantial advantage to TSA’s security  operations. At that time, TSA did not provide written guidance to airports  to assist them in understanding what would constitute a “clear and  substantial advantage to TSA security operations” or TSA’s basis for  determining whether an airport had met that standard.
As previously noted, in March 2012 TSA again revised the SPP  application in accordance with provisions of the FAA Modernization Act,  which became law in February 2012. Among other things, the revised  application no longer included the “clear and substantial advantage”  question, but instead included questions that requested applicants to  discuss how participating in the SPP would not compromise security at  the airport and to identify potential areas where cost savings or  efficiencies may be realized. In December 2012, we reported that while  TSA provided general instructions for filling out the SPP application as  well as responses to frequently asked questions (FAQ), the agency had  not issued guidance to assist airports with completing the revised  application nor explained to airports how it would evaluate applications  given the changes brought about by the FAA Modernization Act. For  example, neither the application instructions or the FAQs addressed  TSA’s SPP application evaluation process or its basis for determining  whether an airport’s entry into the SPP would compromise security or  affect cost-efficiency and effectiveness.
Further, we found that airport operators who completed the applications  generally stated that they faced difficulties in doing so and that additional  guidance would have been helpful. For example, one operator stated  that he needed cost information to help demonstrate that his airport’s  participation in the SPP would not detrimentally affect the cost-efficiency  of the screening of passengers or property at the airport and that he  believed not presenting this information would be detrimental to his  airport’s application. However, TSA officials at the time said that airports  do not need to provide this information to TSA because, as part of the  application evaluation process, TSA conducts a detailed cost analysis  using historical cost data from SPP and non-SPP airports. The absence  of cost and other information in an individual airport’s application, TSA  officials noted, would not materially affect the TSA Administrator’s  decision on an SPP application.
Therefore, we reported in December 2012 that while TSA had approved  all applications submitted since enactment of the FAA Modernization Act,  it was hard to determine how many more airports, if any, would have  applied to the program had TSA provided application guidance and  information to improve transparency of the SPP application process.  Specifically, we reported that in the absence of such application guidance  and information, it may be difficult for airport officials to evaluate whether  their airports are good candidates for the SPP or determine what criteria  TSA uses to accept and approve airports’ SPP applications. Further, we  concluded that clear guidance for applying to the SPP could improve the  transparency of the application process and help ensure that the existing  application process is implemented in a consistent and uniform manner.  Thus, we recommended that TSA develop guidance that clearly (1) states  the criteria and process that TSA is using to assess whether participation  in the SPP would compromise security or detrimentally affect the cost- efficiency or the effectiveness of the screening of passengers or property  at the airport, (2) states how TSA will obtain and analyze cost information  regarding screening cost-efficiency and effectiveness and the implications  of not responding to the related application questions, and (3) provides  specific examples of additional information airports should consider  providing to TSA to help assess an airport’s suitability for the SPP.
TSA concurred with our recommendation and has taken actions to  address it. Specifically, TSA updated its SPP website in December 2012  by providing (1) general guidance to assist airports with completing the  SPP application and (2) a description of the criteria and process the  agency will use to assess airports’ applications to participate in the SPP.  While the guidance states that TSA has no specific expectations of the  information an airport could provide that may be pertinent to its  application, it provides some examples of information TSA has found  useful and that airports could consider providing to TSA to help assess  their suitability for the program. Further, the guidance, in combination with  the description of the SPP application evaluation process, outlines how  TSA plans to analyze and use cost information regarding screening cost- efficiency and effectiveness. The guidance also states that providing cost  information is optional and that not providing such information will not  affect the application decision. We believe that these actions address the  intent of our recommendation and should help improve transparency of  the SPP application process as well as help airport officials determine  whether their airports are good candidates for the SPP.
section_title: Performance between SPP and Non-SPP Airports Varied; TSA Recently Developed a Mechanism to Monitor Private versus Federal Screener Performance
subsections: section_title: Performance Varied between SPP and Non- SPP Airports for Some Measures, but Differences Cannot Be Entirely Attributed to the Use of Private or Federal Screeners
paragraphs: In our December 2012 report, we analyzed screener performance data for  four measures and found that there were differences in performance  between SPP and non-SPP airports, and those differences could not be  exclusively attributed to the use of either federal or private screeners. The  four measures we selected to compare screener performance at SPP and  non-SPP airports were Threat Image Projection (TIP) detection rates,  recertification pass rates, Aviation Security Assessment Program (ASAP)  test results, and Presence, Advisement, Communication, and Execution  (PACE) evaluation results (see table 1). For each of these four measures,  we compared the performance of each of the 16 airports then  participating in the SPP with the average performance for each airport’s  category (X, I, II, III, or IV), as well as the national performance averages  for all airports for fiscal years 2009 through 2011.
As we reported in December 2012, on the basis of our analyses, we  found that, generally, certain SPP airports performed slightly above the  airport category and national averages for some measures, while others  performed slightly below. For example, SPP airports performed above  their respective airport category averages for recertification pass rates in  the majority of instances, while the majority of SPP airports that took  PACE evaluations in 2011 performed below their airport category  averages. For TIP detection rates, SPP airports performed above their  respective airport category averages in about half of the instances.  However, we also reported in December 2012 that the differences we  observed in private and federal screener performance cannot be entirely  attributed to the type of screeners at an airport, because, according to  TSA officials and other subject matter experts, many factors, some of  which cannot be controlled for, affect screener performance. These  factors include, but are not limited to, checkpoint layout, airline schedules,  seasonal changes in travel volume, and type of traveler.
We also reported in December 2012 that TSA collects data on several  other performance measures but, for various reasons, the data cannot be  used to compare private and federal screener performance for the  purposes of our review. For example, passenger wait time data could not  be used because we found that TSA’s policy for collecting wait times  changed during the time period of our analyses and that these data were  not collected in a consistent manner across all airports. We also considered reviewing human capital measures such as attrition,  absenteeism, and injury rates, but did not analyze these data because  TSA’s Office of Human Capital does not collect these data for SPP  airports. We reported that while the contractors collect and report this  information to the SPP PMO, TSA does not validate the accuracy of the  self-reported data nor does it require contractors to use the same human  capital measures as TSA, and accordingly, differences may exist in how  the metrics are defined and how the data are collected. Therefore, we  found that TSA could not guarantee that a comparison of SPP and non- SPP airports on these human capital metrics would be an equal  comparison.
section_title: TSA Has Developed a Mechanism to Monitor Private Screener Performance Separately from the Performance of Federal Screeners
paragraphs: Since our December 2012 report, TSA has developed a mechanism to  regularly monitor private versus federal screener performance, as we  recommended. In December 2012, we reported that while TSA monitored  screener performance at all airports, the agency did not monitor private  screener performance separately from federal screener performance or  conduct regular reviews comparing the performance of SPP and non-SPP  airports. Beginning in April 2012, TSA introduced a new set of  performance measures to assess screener performance at all airports  (both SPP and non-SPP) in its Office of Security Operations Executive  Scorecard (the Scorecard). Officials told us at the time of our December  2012 review that they provided the Scorecard to FSDs every 2 weeks to  assist the FSDs with tracking performance against stated goals and with  determining how performance of the airports under their jurisdiction  compared with national averages. According to TSA, the 10 measures  used in the Scorecard were selected based on input from FSDs and  regional directors on the performance measures that most adequately  reflected screener and airport performance.
Performance measures in  the Scorecard included the TIP detection rate, and the number of  negative and positive customer contacts made to the TSA Contact Center  through e-mails or phone calls per 100,000 passengers screened, among  others.
We also reported in December 2012 that TSA had conducted or  commissioned prior reports comparing the cost and performance of SPP  and non-SPP airports. For example, in 2004 and 2007, TSA  commissioned reports prepared by private consultants, while in 2008 the  agency issued its own report comparing the performance of SPP and  non-SPP airports.performed at a level equal to or better than non-SPP airports. However,  TSA officials stated at the time that they did not plan to conduct similar  analyses in the future, and instead, they were using across-the-board  mechanisms of both private and federal screeners, such as the  Scorecard, to assess screener performance across all commercial  airports.
Generally, these reports found that SPP airports  In addition to using the Scorecard, we found that TSA conducted monthly  contractor performance management reviews (PMR) at each SPP airport  to assess the contractor’s performance against the standards set in each  SPP contract. The PMRs included 10 performance measures, including  some of the same measures included in the Scorecard, such as TIP  detection rates and recertification pass rates, for which TSA establishes  acceptable quality levels of performance. Failure to meet the acceptable  quality levels of performance could result in corrective actions or  termination of the contract.
However, as we reported in December 2012, the Scorecard and PMR did  not provide a complete picture of screener performance at SPP airports  because, while both mechanisms provided a snapshot of private screener  performance at each SPP airport, this information was not summarized  for the SPP as a whole or across years, which made it difficult to identify  changes in performance. Further, neither the Scorecard nor the PMR  provided information on performance in prior years or controlled for  variables that TSA officials explained to us were important when  comparing private and federal screener performance, such as the type of  X-ray machine used for TIP detection rates. We concluded that  monitoring private screener performance in comparison with federal  screener performance was consistent with the statutory requirement that  TSA enter into a contract with a private screening company only if the  Administrator determines and certifies to Congress that the level of  screening services and protection provided at an airport under a contract  will be equal to or greater than the level that would be provided at the  airport by federal government personnel. Therefore, we recommended  that TSA develop a mechanism to regularly monitor private versus federal  screener performance, which would better position the agency to know  whether the level of screening services and protection provided at SPP  airports continues to be equal to or greater than the level provided at non- SPP airports.
TSA concurred with our recommendation, and has taken actions to  address it. Specifically, in January 2013, TSA issued its first SPP Annual  Report. The report highlights the accomplishments of the SPP during  fiscal year 2012 and provides an overview and discussion of private  versus federal screener cost and performance. The report also describes  the criteria TSA used to select certain performance measures and  reasons why other measures were not selected for its comparison of  private and federal screener performance. The report compares the  performance of SPP airports with the average performance of airports in  their respective category, as well as the average performance for all  airports, for three performance measures: TIP detection rates,  recertification pass rates, and PACE evaluation results. Further, in  September 2013, the TSA Assistant Administrator for Security Operations  signed an operations directive that provides internal guidance for  preparing the SPP Annual Report, including the requirement that the SPP  PMO must annually verify that the level of screening services and  protection provided at SPP airports is equal to or greater than the level  that would be provided by federal screeners. We believe that these  actions address the intent of our recommendation and should better  position TSA to determine whether the level of screening services and  protection provided at SPP airports continues to be equal to or greater  than the level provided at non-SPP airports. Further, these actions could  also assist TSA in identifying performance changes that could lead to  improvements in the program and inform decision making regarding  potential expansion of the SPP.
Chairman Mica, Ranking Member Connolly, and members of the  subcommittee, this completes my prepared statement. I would be happy  to respond to any questions you may have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions about this statement, please contact Jennifer Grover at  (202) 512-7141 or GroverJ@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. Individuals making key contributions to this statement  include Glenn Davis (Assistant Director), Stanley Kostyla, Brendan  Kretzschmar, Thomas Lombardi, Erin O’Brien, and Jessica Orr. Key  contributors for the previous work that this testimony is based on are  listed in the product.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.