Screening Partnership Program: TSA Has Improved Application Guidance and Monitoring of Screener Performance, and Continues to Improve Cost Comparison Methods
Published: 2014-07-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: TSA maintains a federal workforce to screen passengers and baggage at the majority of the nation's commercial airports, but it also oversees a workforce of private screeners at airports who participate in the SPP. The SPP allows commercial airports to apply to have screening performed by private screeners, who are to provide a level of screening services and protection that equals or exceeds that of federal screeners.
This testimony addresses the extent to which TSA (1) provides guidance to airport operators for the SPP application process, (2) assesses and monitors the performance of private versus federal screeners, and (3) compares the costs of federal and private screeners. This statement is based on reports and a testimony GAO issued from January 2009 through January 2014.
section_title: What GAO Found
paragraphs: Since GAO's December 2012 report on the Screening Partnership Program (SPP), the Transportation Security Administration (TSA) has developed guidance for airport operators applying to the SPP. In December 2012, GAO found that TSA had not provided guidance to airport operators on its SPP application and approval process, which had been revised to reflect statutory requirements. Further, airport operators GAO interviewed at the time identified difficulties in completing the revised application, such as obtaining cost information requested in the application. GAO recommended that TSA develop application guidance and TSA concurred. In December 2012, TSA updated its SPP website with general application guidance and a description of TSA's assessment criteria and process. The new guidance addresses the intent of GAO's recommendation.
TSA has also developed a mechanism to regularly monitor private versus federal screener performance. In December 2012, TSA officials stated that they planned to assess overall screener performance across all commercial airports instead of comparing the performance of SPP and non-SPP airports as they had done previously. Also in December 2012, GAO reported differences between the performance at SPP and non-SPP airports based on screener performance data. In addition, GAO reported that TSA's across-the-board mechanisms did not summarize information for the SPP as a whole or across years, making it difficult to identify changes in private screener performance. GAO concluded that monitoring and comparing private and federal screener performance were consistent with the statutory provision authorizing TSA to contract with private screening companies. As a result, GAO recommended that TSA develop a mechanism to regularly do so. TSA concurred with the recommendation and in January 2013, issued its SPP Annual Report , which provided an analysis of private versus federal screener performance. In September 2013, TSA provided internal guidance requiring that the report annually verify that the level of screening services and protection provided at SPP airports is equal to or greater than the level that would be provided by federal screeners. These actions address the intent of GAO's recommendation.
TSA has faced challenges in accurately comparing the costs of screening services at SPP and non-SPP airports. In 2007, TSA estimated that SPP airports cost about 17 percent more to operate than airports using federal screeners. In January 2009, GAO noted strengths in TSA's methodology, but also identified seven limitations that could affect the accuracy and reliability of cost comparisons. GAO recommended that TSA update its analysis to address the limitations. TSA generally concurred with the recommendation. In March 2011, TSA described efforts to address the limitations and a revised cost comparison estimating that SPP airports would cost 3 percent more to operate in 2011 than airports using federal screeners. In March 2011, GAO found that TSA had taken steps to address some of the limitations, but needed to take additional actions. In July 2014, TSA officials stated that they are continuing to make additional changes to the cost estimation methodology and GAO is continuing to monitor TSA's progress in this area through ongoing work.
section_title: What GAO Recommends
paragraphs: GAO has made several recommendations since 2009 to improve SPP operations and oversight, which GAO has since closed as implemented based on TSA actions to address them.

section_title: Letter
paragraphs: I appreciate the opportunity to discuss our work on the Transportation  Security Administration’s (TSA) Screening Partnership Program (SPP).  TSA, a component of the Department of Homeland Security (DHS), is  responsible for ensuring the security of the traveling public through,  among other things, screening passengers traveling by aircraft for  explosives and other prohibited items. To fulfill this responsibility, TSA  maintains a federal workforce of screeners at a majority of the nation’s  commercial airports, but also oversees a smaller workforce of private  screeners employed by companies under contract to TSA at airports that  participate in TSA’s SPP. The SPP, established in 2004 in accordance  with provisions of the Aviation Transportation Security Act (ATSA), allows  commercial airports an opportunity to “opt out” of federal screening by  applying to TSA to have private screeners perform the screening  function. At airports with private screeners, the screening of passengers  and baggage is performed by private screening contractors selected and  approved by TSA; however, TSA continues to be responsible for  overseeing airport security operations and ensuring that the private  contractors provide effective and efficient screening operations in a  manner consistent with law and other TSA requirements at these  airports. As of July 2014, there are currently 18 airports participating in  the program, 14 of which are currently operating with contracted  screeners and four of which that have not yet transitioned to private  screeners. Two additional approved airports are awaiting the selection of  a screening contractor and one application is pending.
The standard by which TSA evaluates airport applications for participation  in the SPP has changed since the program’s inception in 2004. First, in  January 2011, the TSA Administrator announced his decision not to  expand the SPP beyond the 16 airports that were participating in the  program at that time “unless a clear and substantial advantage to do so  emerges in the future.” In so doing, the Administrator cited his interest in  helping the agency evolve into a “more agile, high-performing  organization that can meet the security threats of today and the future” as  the reason for his decision. Of the 6 airports that submitted applications  from March 2009 through January 2012 and that were evaluated under  the “clear and substantial advantage” standard, TSA approved the  application of 1 airport and denied the applications of the other 5. Second,  according to TSA officials, the Federal Aviation Administration  Modernization and Reform Act of 2012 (FAA Modernization Act), enacted  in February 2012, prompted TSA to change the standard by which it  evaluates SPP applications and requires, among other things, that the  TSA Administrator approve an SPP application submitted by an airport  operator if the Administrator determines that the approval would not  compromise security or detrimentally affect the cost-efficiency or the  effectiveness of the screening of passengers or property at the airport.
My testimony today addresses the extent to which TSA (1) provides  guidance to airport operators for the SPP application process, (2)  assesses and monitors the performance of private versus federal  screeners, and (3) compares the costs of private and federal operations.  This statement is based on reports and a testimony we issued from  January 2009 to January 2014. Our March 2011 report was based on our  review of TSA’s updated cost analysis, which was provided in response to  recommendations in our January 2009 report, as well as discussions with  agency officials. For our December 2012 report, among other things we  analyzed past and current SPP application forms and instructions and  interviewed airport operators, screeners, SPP contractors, SPP  applicants, TSA headquarters officials, and federal security directors  (FSD). More detailed information on the scope and methodology appears  in our January 2009, March 2011, and December 2012 reports, and our  January testimony. For the January 2014 testimony, we obtained related  documentation, such as the SPP Annual Report issued in January 2013,  and interviewed agency officials on progress made to implement the  recommendations from our December 2012 report related to application  guidance and monitoring of private versus federal screener performance.
We conducted the work on which this statement and the underlying  products are based in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: On November 19, 2002, pursuant to ATSA, TSA began a 2-year pilot  program at 5 airports using private screening companies to screen  passengers and checked baggage. In 2004, at the completion of the pilot  program, and in accordance with ATSA, TSA established the SPP,  whereby any airport authority, whether involved in the pilot or not, could  request a transition from federal screeners to private, contracted  screeners. All of the 5 pilot airports that applied were approved to  continue as part of the SPP, and since its establishment, 21 additional  airport applications have been accepted by the SPP.
In March 2012, TSA revised the SPP application to reflect requirements  of the FAA Modernization Act, enacted in February 2012. Among other  provisions, the act provides the following:
Not later than 120 days after the date of receipt of an SPP application  submitted by an airport operator, the TSA Administrator must approve  or deny the application.
The TSA Administrator shall approve an application if approval would  not (1) compromise security, (2) detrimentally affect the cost-efficiency  of the screening of passengers or property at the airport, or (3)  detrimentally affect the effectiveness of the screening of passengers  or property at the airport.
Within 60 days of a denial, TSA must provide the airport operator, as  well as the Committee on Commerce, Science, and Transportation of  the Senate and the Committee on Homeland Security of the House of  Representatives, a written report that sets forth the findings that  served as the basis of the denial, the results of any cost or security  analysis conducted in considering the application, and  recommendations on how the airport operator can address the  reasons for denial.
All commercial airports are eligible to apply to the SPP. To apply, an  airport operator must complete the SPP application and submit it to the  SPP Program Management Office (PMO), as well as to the FSD for its  airport. Figure 1 illustrates the SPP application process.
Although TSA provides all airports with the opportunity to apply for  participation in the SPP, authority to approve or deny the application  resides in the discretion of the TSA Administrator. According to TSA  officials, in addition to the cost-efficiency and effectiveness considerations  mandated by FAA Modernization Act, there are many other factors that  are weighed in considering an airport’s application for SPP participation.  For example, the potential impacts of any upcoming projects at the airport  are considered. Once an airport is approved for SPP participation and a  private screening contractor has been selected by TSA, the contract  screening workforce assumes responsibility for screening passengers  and their property and is required to adhere to the same security  regulations, standard operating procedures, and other TSA security  requirements followed by federal screeners at non-SPP airports.
section_title: TSA Has Developed Program Application Guidance to Help Improve Transparency of Its Process and Assist Airports in Completing Their Applications
paragraphs: TSA has developed guidance to assist airport operators in completing  their SPP applications, as we recommended in December 2012.  Specifically, in December 2012, we reported that TSA had developed  some resources to assist SPP applicants, but it had not provided  guidance on its application and approval process to assist airports. As it  was originally implemented in 2004, the SPP application process required  only that an interested airport operator submit an application stating its  intention to opt out of federal screening as well as its reasons for wanting  to do so. In 2011, TSA revised its SPP application to reflect the “clear and  substantial advantage” standard announced by the Administrator in  January 2011. Specifically, TSA requested that the applicant explain how  private screening at the airport would provide a clear and substantial  advantage to TSA’s security operations. At that time, TSA did not  provide written guidance to airports to assist them in understanding what  would constitute a “clear and substantial advantage to TSA security  operations” or TSA’s basis for determining whether an airport had met  that standard.
As previously noted, in March 2012 TSA again revised the SPP  application in accordance with provisions of the FAA Modernization Act,  which became law in February 2012. Among other things, the revised  application no longer included the “clear and substantial advantage”  question, but instead included questions that requested applicants to  discuss how participating in the SPP would not compromise security at  the airport and to identify potential areas where cost savings or  efficiencies may be realized. In December 2012, we reported that while  TSA provided general instructions for filling out the SPP application as  well as responses to frequently asked questions (FAQ), the agency had  not issued guidance to assist airports with completing the revised  application or explained to airports how it would evaluate applications  given the changes brought about by the FAA Modernization Act. For  example, neither the application instructions nor the FAQs addressed  TSA’s SPP application evaluation process or its basis for determining  whether an airport’s entry into the SPP would compromise security or  affect cost-efficiency and effectiveness.
Further, in December 2012, we found that airport operators who  completed the applications generally stated that they faced difficulties in  doing so and that additional guidance would have been helpful. For  example, one operator stated that he needed cost information to help  demonstrate that his airport’s participation in the SPP would not  detrimentally affect the cost-efficiency of the screening of passengers or  property at the airport and that he believed not presenting this information  would be detrimental to his airport’s application. However, TSA officials at  the time said that airports do not need to provide this information to TSA  because, as part of the application evaluation process, TSA conducts a  detailed cost analysis using historical cost data from SPP and non-SPP  airports. The absence of cost and other information in an individual  airport’s application, TSA officials noted, would not materially affect the  TSA Administrator’s decision on an SPP application.
Therefore, we reported in December 2012 that while TSA had approved  all applications submitted since enactment of the FAA Modernization Act,  it was hard to determine how many more airports, if any, would have  applied to the program had TSA provided application guidance and  information to improve transparency of the SPP application process.  Specifically, we reported that in the absence of such application guidance  and information, it may be difficult for airport officials to evaluate whether  their airports are good candidates for the SPP or determine what criteria  TSA uses to accept and approve airports’ SPP applications. We  concluded that clear guidance for applying to the SPP could improve the  transparency of the application process and help ensure that the existing  application process is implemented in a consistent and uniform manner.  Thus, we recommended that TSA develop guidance that clearly (1) states  the criteria and process that TSA is using to assess whether participation  in the SPP would compromise security or detrimentally affect the cost- efficiency or the effectiveness of the screening of passengers or property  at the airport, (2) states how TSA will obtain and analyze cost information  regarding screening cost-efficiency and effectiveness and the implications  of not responding to the related application questions, and (3) provides  specific examples of additional information airports should consider  providing to TSA to help assess an airport’s suitability for the SPP.
TSA concurred with our recommendation and, in January 2014, we  reported that TSA had taken actions to address it. Specifically, TSA  updated its SPP website in December 2012 by providing (1) general  guidance to assist airports with completing the SPP application and (2) a  description of the criteria and process the agency will use to assess  airports’ applications to participate in the SPP. While the guidance states  that TSA has no specific expectations of the information an airport could  provide that may be pertinent to its application, it provides some  examples of information TSA has found useful and that airports could  consider providing to TSA to help assess their suitability for the program.  Further, the guidance, in combination with the description of the SPP  application evaluation process, outlines how TSA plans to analyze and  use cost information regarding screening cost-efficiency and  effectiveness. The guidance also states that providing cost information is  optional and that not providing such information will not affect the  application decision. As we reported in January 2014, these actions  address the intent of our recommendation.
section_title: Performance between SPP and Non-SPP Airports Varied; TSA Recently Developed a Mechanism to Monitor Private versus Federal Screener Performance
paragraphs: In our December 2012 report, we analyzed screener performance data for  four measures and found that there were differences in performance  between SPP and non-SPP airports, and those differences could not be  exclusively attributed to the use of either federal or private screeners. The  four measures we selected to compare screener performance at SPP and  non-SPP airports were Threat Image Projection (TIP) detection rates;  recertification pass rates; Aviation Security Assessment Program (ASAP)  test results; and Presence, Advisement, Communication, and Execution  (PACE) evaluation results (see table 1). For each of these four measures,  we compared the performance of each of the 16 airports then  participating in the SPP with the average performance for each airport’s  category (X, I, II, III, or IV), as well as the national performance averages  for all airports for fiscal years 2009 through 2011.
As we reported in December 2012, on the basis of our analyses, we  found that, generally, screeners at certain SPP airports performed slightly  above the airport category and national averages for some measures,  while others performed slightly below. For example, at SPP airports,  screeners performed above their respective airport category averages for  recertification pass rates in the majority of instances, while at the majority  of SPP airports that took PACE evaluations in 2011, screeners performed  below their airport category averages. For TIP detection rates,  screeners at SPP airports performed above their respective airport  category averages in about half of the instances. However, we also  reported in December 2012 that the differences we observed in private  and federal screener performance cannot be entirely attributed to the type  of screeners at an airport, because, according to TSA officials and other  subject matter experts, many factors, some of which cannot be controlled  for, affect screener performance. These factors include, but are not  limited to, checkpoint layout, airline schedules, seasonal changes in travel  volume, and type of traveler.
We also reported in December 2012 that TSA collects data on several  other performance measures but, for various reasons, the data cannot be  used to compare private and federal screener performance for the  purposes of our review. For example, passenger wait time data could not  be used because we found that TSA’s policy for collecting wait times  changed during the time period of our analyses and that these data were  not collected in a consistent manner across all airports. We also  considered reviewing human capital measures such as attrition,  absenteeism, and injury rates, but did not analyze these data because  TSA’s Office of Human Capital does not collect these data for SPP  airports. We reported that while the contractors collect and report this  information to the SPP PMO, TSA does not validate the accuracy of the  self-reported data nor does it require contractors to use the same human  capital measures as TSA, and accordingly, differences may exist in how  the metrics are defined and how the data are collected. Therefore, we  found that TSA could not guarantee that a comparison of SPP and non- SPP airports on these human capital metrics would be an equal  comparison.
Moreover, in December 2012, we found that while TSA monitored  screener performance at all airports, the agency did not monitor private  screener performance separately from federal screener performance or  conduct regular reviews comparing the performance of SPP and non-SPP  airports. Beginning in April 2012, TSA introduced a new set of  performance measures to assess screener performance at all airports  (both SPP and non-SPP) in its Office of Security Operations Executive  Scorecard (the Scorecard). Officials told us at the time of our December  2012 review that they provided the Scorecard to FSDs every 2 weeks to  assist the FSDs with tracking performance against stated goals and with  determining how performance of the airports under their jurisdiction  compared with national averages. According to TSA, the 10 measures  used in the Scorecard were selected based on input from FSDs and  regional directors on the performance measures that most adequately  reflected screener and airport performance. Performance measures in  the Scorecard included the TIP detection rate and the number of negative  and positive customer contacts made to the TSA Contact Center through  e-mails or phone calls per 100,000 passengers screened, among  others.
We also reported in December 2012 that TSA had conducted or  commissioned prior reports comparing the cost and performance of SPP  and non-SPP airports. For example, in 2004 and 2007, TSA  commissioned reports prepared by private consultants, while in 2008 the  agency issued its own report comparing the performance of SPP and  non-SPP airports. Generally, these reports found that SPP airports  performed at a level equal to or better than non-SPP airports. However,  TSA officials stated at the time that they did not plan to conduct similar  analyses in the future, and instead, they were using across-the-board  mechanisms of both private and federal screeners, such as the  Scorecard, to assess screener performance across all commercial  airports.
We found that In addition to using the Scorecard, TSA conducted monthly  contractor performance management reviews (PMR) at each SPP airport  to assess the contractor’s performance against the standards set in each  SPP contract. The PMRs included 10 performance measures, including  some of the same measures included in the Scorecard, such as TIP  detection rates and recertification pass rates, for which TSA establishes  acceptable quality levels of performance. Failure to meet the acceptable  quality levels of performance could result in corrective actions or  termination of the contract.
However, in December 2012, we found that the Scorecard and PMR did  not provide a complete picture of screener performance at SPP airports  because, while both mechanisms provided a snapshot of private screener  performance at each SPP airport, this information was not summarized  for the SPP as a whole or across years, which made it difficult to identify  changes in performance. Further, neither the Scorecard nor the PMR  provided information on performance in prior years or controlled for  variables that TSA officials explained to us were important when  comparing private and federal screener performance, such as the type of  X-ray machine used for TIP detection rates. We concluded that  monitoring private screener performance in comparison with federal  screener performance was consistent with the statutory requirement that  TSA enter into a contract with a private screening company only if the  Administrator determines and certifies to Congress that the level of  screening services and protection provided at an airport under a contract  will be equal to or greater than the level that would be provided at the  airport by federal government personnel. Therefore, we recommended  that TSA develop a mechanism to regularly monitor private versus federal  screener performance, which would better position the agency to know  whether the level of screening services and protection provided at SPP  airports continues to be equal to or greater than the level provided at non- SPP airports.
TSA concurred with the recommendation, and has taken actions to  address it. Specifically, in January 2013, TSA issued its first SPP Annual  Report. The report highlights the accomplishments of the SPP during  fiscal year 2012 and provides an overview and discussion of private  versus federal screener cost and performance. The report also describes  the criteria TSA used to select certain performance measures and  reasons why other measures were not selected for its comparison of  private and federal screener performance. The report compares the  performance of SPP airports with the average performance of airports in  their respective category, as well as the average performance for all  airports, for three performance measures: TIP detection rates,  recertification pass rates, and PACE evaluation results. Further, in  September 2013, the TSA Assistant Administrator for Security Operations  signed an operations directive that provides internal guidance for  preparing the SPP Annual Report, including the requirement that the SPP  PMO must annually verify that the level of screening services and  protection provided at SPP airports is equal to or greater than the level  that would be provided by federal screeners. We believe that these  actions address the intent of our recommendation and should better  position TSA to determine whether the level of screening services and  protection provided at SPP airports continues to be equal to or greater  than the level provided at non-SPP airports. Further, these actions could  also assist TSA in identifying performance changes that could lead to  improvements in the program and inform decision making regarding  potential expansion of the SPP.
section_title: TSA Continues to Address Limitations in the Methodology for Comparing the Costs of SPP and Non-SPP Screening Services
paragraphs: TSA has faced challenges in accurately comparing the costs of screening  services at SPP and non-SPP airports. In 2007, TSA estimated that SPP  airports would cost about 17 percent more to operate than airports using  federal screeners. In our January 2009 report we noted strengths in the  methodology’s design, but also identified seven limitations in TSA’s  methodology that could affect the accuracy and reliability of cost  comparisons, and its usefulness in informing future management  decisions. We recommended that if TSA planned to rely on its  comparison of cost and performance of SPP and non-SPP airports for  future decision making, the agency should update its analysis to address  the limitations we identified. TSA generally concurred with our findings  and recommendation. In March 2011, TSA provided us with an update on  the status of its efforts to address the limitations we cited in our report, as  well as a revised comparison of costs for screening operations at SPP  and non-SPP airports. This revised cost comparison generally addressed  three of the seven limitations and provided TSA with a more reasonable  basis for comparing the screening cost at SPP and non-SPP airports. In  the update, TSA estimated that SPP airports would cost 3 percent more  to operate in 2011 than airports using federal screeners. In March 2011,  we found that TSA had also taken actions that partially addressed the  four remaining limitations related to cost, but needed to take additional  actions or provide additional documentation. In July 2014, TSA officials  stated they are continuing to make additional changes to the cost  estimation methodology and we are continuing to monitor TSA’s progress  in this area through ongoing work.
Chairman Hudson, Ranking Member Richmond, and members of the  subcommittee, this completes my prepared statement. I would be happy  to respond to any questions you may have at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions about this statement, please contact Jennifer Grover at  (202) 512-7141 or GroverJ@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. Individuals making key contributions to this statement  include Glenn Davis (Assistant Director), Charles Bausell, Kevin Heinz,  Susan Hsu, Tyler Kent, Stanley Kostyla, and Thomas Lombardi. Key  contributors for the previous work that this testimony is based on are  listed in the products.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.