Screening Partnership Program: TSA Should Issue More Guidance to Airports and Monitor Private versus Federal Screener Performance
Published: 2012-12-06T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: TSA maintains a federal workforce to screen passengers and baggage at the majority of the nation's commercial airports, but also oversees a workforce of private screeners at airports who participate in the SPP. The SPP allows commercial airports to use private screeners, provided that the level of screening matches or exceeds that of federal screeners. In recent years, TSA's SPP has evolved to incorporate changes in policy and federal law, prompting enhanced interest in measuring screener performance. GAO was asked to examine the (1) status of SPP applications and airport operators', aviation stakeholders', and TSA's reported advantages and disadvantages of participating in the SPP; (2) extent to which TSA has provided airports guidance to govern the SPP application process; and (3) extent to which TSA assesses and monitors the performance of private and federal screeners. GAO surveyed 28 airport operators that had applied to the SPP as of April 2012, and interviewed 5 airport operators who have not applied and 1 airport operator who applied to the SPP after GAO's survey. Although not generalizable, these interviews provided insights. GAO also analyzed screener performance data from fiscal years 2009-2011. This is a public version of a sensitive report that GAO issued in November 2012. Information that TSA deemed sensitive has been redacted.
section_title: What GAO Found
paragraphs: Since implementation of the Screening Partnership Program (SPP) in 2004, 29 airports have applied to the program, citing various advantages and relatively few disadvantages. Of the 25 approved, 16 are participating in the program, 6 are currently in the contractor procurement process, and the remainder withdrew from participation because their commercial airline services were discontinued. In 2011, the Transportation Security Administration (TSA) denied applications for 6 airports because, according to TSA officials, the airports did not demonstrate that participation in the program would "provide a clear and substantial advantage to TSA security operations." After enactment of the Federal Aviation Administration Modernization and Reform Act of 2012 (FAA Modernization Act) in February 2012, TSA revised its SPP application, removing the "clear and substantial advantage" question. Four of the 6 airports that had been denied in 2011 later reapplied and were approved. In GAO's survey and in interviews with airport operators (of SPP and non-SPP airports) and aviation stakeholders, improved customer service and increased staffing flexibilities were most commonly cited as advantages or potential advantages of the SPP. Individual Federal Security Directors we interviewed cited reduced involvement in human resource management as an advantage; however, TSA generally remains neutral regarding the SPP. Few disadvantages were cited; however, some airport operators cited satisfaction with federal screeners and concerns with potential disruption from the transition to private screening services.
TSA has developed some resources to assist SPP applicants; however, it has not provided guidance to assist airports applying to the program. Consistent with the FAA Modernization Act, TSA's revised SPP application requested that applicants provide information to assist TSA in determining if their participation in the SPP would compromise security or detrimentally affect the cost-efficiency or screening effectiveness of passengers and property at their airport. TSA also developed responses to frequently asked questions and has expressed a willingness to assist airports that need it. However, TSA has not issued guidance to assist airports with completing applications and information on how the agency will assess them. Three of five airport operators who applied using the current application stated that additional guidance is needed to better understand how to respond to the new application questions. Developing guidance could better position airports to evaluate whether they are good candidates for the SPP.
TSA recently improved its screener performance measures, but could benefit from monitoring private versus federal screener performance. In April 2012, TSA added measures to ensure that the set of measures it uses to assess screener performance at private and federal airports better addresses its airport screening strategic goals and mission. However, TSA does not monitor private screener performance separately from federal screener performance. Instead, TSA conducts efforts to monitor screener performance at individual SPP airports, but these efforts do not provide information on SPP performance as a whole or across years, which makes it difficult to identify program trends. A mechanism to consistently monitor SPP versus non-SPP performance would better position TSA to ensure that the level of screening services and protection provided at SPP airports continues to match or exceed the level provided at non-SPP airports, thereby ensuring that SPP airports are operating as intended.
section_title: What GAO Recommends
paragraphs: GAO recommends that the TSA Administrator develop guidance for SPP applicants and a mechanism to monitor private versus federal screener performance. TSA concurred with the recommendations.

section_title: Letter
paragraphs: The Transportation Security Administration (TSA) is responsible for  ensuring the security of the traveling public through, among other things,  screening passengers traveling by aircraft for explosives and other  prohibited items. To fulfill this responsibility, TSA maintains a federal  workforce of screeners at a majority of the nation’s commercial airports,  but also oversees a smaller workforce of private screeners employed by  companies under contract to TSA at airports that participate in TSA’s  Screening Partnership Program (hereafter referred to as the SPP, or the  Program). The SPP, established in 2004 in accordance with provisions  of the Aviation Transportation Security Act (ATSA), allows commercial  airports an opportunity to “opt out” of federal screening by applying to  TSA to have private-sector screeners perform the screening function.private screening airports, TSA continues to be responsible for  overseeing airport screening operations and ensuring that the contractors  provide effective and efficient security operations in a manner consistent  with law and other TSA requirements; however, the screening of   At  passengers and baggage at these airports is performed by private  screening contractors selected and approved by the TSA.
Questions about the cost-effectiveness and screening efficiency of private  screeners compared with that of federal screeners have increased  scrutiny of the SPP. Some representatives of the aviation industry and  certain airport operators contend that the SPP capitalizes on the private  sector’s innovation and flexibility to provide screening services more  efficiently and with enhanced customer service. However, the SPP model  allows for little variance in screening operations at SPP airports. For  example, in accordance with ATSA, private screeners must meet the  same standards and requirements for hiring and training that apply to  federal screeners, abide by the same standard operating procedures, and  be provided compensation and benefits at a level not less than their  federal counterparts.
In addition to the 16 airports, TSA approved the applications of 3 other airports (including  one heliport) prior to January 2011 that are not participating in the program because  commercial airline services at these airports were terminated. application submitted by an airport operator if the Administrator  determines that the approval would not compromise security or  detrimentally affect the cost-efficiency or the effectiveness of the  screening of passengers or property at the airport.
Citing the recent SPP airport approvals, the Committee on  Appropriations, House of Representatives, recommended approximately  $160 million for privatized screening for fiscal year 2013, which  represents $15 million above the amount requested in the Department of  Homeland Security’s (DHS) budget request and about $14 million above  the enacted fiscal year 2012 level. increased to ensure adequate resources to support potential new SPP  participants and to encourage TSA to make greater use of the program.  The report of the Committee on Appropriations of the Senate, however,  recommended approximately $143 million for privatized screening for  fiscal year 2013, the same amount requested in the budget and  approximately $1 million below the enacted fiscal year 2012 level.report, the Senate Committee explained that it expects TSA to not  approve any new contract application for privatized screening if the  annual cost of the contract exceeds the annual cost to TSA of providing  federal screening services at that airport.
See H.R. Rpt. No. 112-492, at 64-65 (May 23, 2012) (accompanying H.R. 5855, 112th  Cong. (2d Sess. 2012)).
GAO, Aviation Security: TSA’s Cost and Performance Study of Private-Sector Airport  Screening, GAO-09-27R (Washington, D.C.: Jan. 9, 2009). the limitations we identified. TSA generally concurred with our  recommendation, and in March 2011, we reported that TSA has made  progress in addressing the limitations related to costs and estimated that  SPP airports would cost 3 percent more to operate in 2011 than airports  using federal screeners.
This report addresses the (1) status of airport applications made to the  SPP, and airport operator, other stakeholders, and TSA views on the  advantages and disadvantages of participating in the SPP; (2) extent to  which TSA has provided guidance to govern the SPP application process;  and (3) extent to which TSA assesses and monitors the performance of  private and federal screeners.
This report is a public version of the prior sensitive report that we  provided to you. DHS deemed some of the information in the prior report  as Sensitive Security Information (SSI), which must be protected from   Therefore, this report omits information about the  public disclosure.specific results of our comparison of SPP screener performance with  performance of federal screeners across four performance measures we  analyzed. Although the information provided in this report is more limited  in scope, it addresses the same questions as the sensitive report. Also,  the overall methodology used for both reports is the same.
To address all three of these objectives, we interviewed Federal Security  Directors (FSD); airport operators; screeners; and where applicable, SPP  contractors at 10 airports. We selected the 10 airports by matching an  SPP to a non-SPP airport, in each of the five airport categories (category  X, I, II, III, and IV), based primarily on (1) annual passenger and baggage  volumes, (2) screener staffing model full-time equivalent allocation, and  (3) number of check-points and screening lanes. Additionally, on the basis of available travel resources, we visited 7 of the 10 airports to  observe airport screening operations, including any unique challenges  faced by these airports. Our observations from these airport visits and  interviews are illustrative and provide insights about private and federal  screening operations but are not generalizable to all airports across the  country.
To determine the status of SPP applications, and airport operators’, other  stakeholders’, and TSA’s views on the advantages and disadvantages of  participating in the SPP, we interviewed officials of TSA’s SPP Program  Management Office (PMO) and reviewed the 15 SPP applications that  had been submitted since fiscal year 2009, as well as TSA’s available  decision memos on the applications. Further, we surveyed the 28 airport  operators who have applied to the SPP since its inception through April  2012 (when the survey was implemented) to solicit their views on  advantages and disadvantages for airports for participating in the SPP.  A 29th airport, Bozeman Yellowstone International Airport, applied to the  SPP for the first time in June 2012 and, therefore, was not included in our  survey. However, we interviewed this airport, as well as the 5 non-SPP  airports we visited or interviewed, to obtain their perspectives on the  potential advantages and disadvantages of participating in the SPP. We  also interviewed representatives of three aviation industry associations to  identify the advantages and disadvantages of using federal and  nonfederal screeners. We selected the three associations because they  represent the majority of aviation industry stakeholders, including airport  operators.
To determine the extent to which TSA has provided guidance to govern  the SPP application process, we analyzed past and current SPP  application forms and instructions, as well as interviewed TSA  headquarters officials to identify the requirements and process for  applying to the SPP. We surveyed operators of all approved SPP airports  as well as operators of airports that have applied but are currently not  participating in the SPP because TSA denied their application, or their  participation is pending the procurement of a contractor, to determine  their perspectives on the SPP application process. To determine if any  improvements are needed to the SPP application process, we compared  TSA’s application process and requirements with Standards for Internal  Control in the Federal Government.
GAO/AIMD-00-21.3.1 and GAO, Executive Guide: Effectively Implementing the  Government Performance and Results Act, GAO/GGD-96-118 (Washington, D.C.: June  1996); and Tax Administration: IRS Needs to Further Refine Its Tax Filing Season  Performance Measures, GAO-03-143 (Washington, D.C.: November 2002). performance for the four performance measures we analyzed. However,  as we note later, because there are many factors that may account for  differences in screener performance, some of which cannot be controlled  for, any difference we found in screener performance at SPP and non- SPP airports cannot be entirely attributed to the use of either federal or  private screeners.
We conducted this performance audit from November 2011 to November  2012, in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives. More details about the  scope and methodology of our work are presented in appendix I.
subsections: section_title: Background
paragraphs: ATSA established TSA and charged it with responsibility for securing all  modes of transportation, including civil aviation. Prior to ATSA and the  establishment of TSA, passenger and baggage screening had generally  been performed by private screening companies under contract to airlines  and in accordance with FAA regulations. In accordance with ATSA, TSA  currently employs personnel who screen passengers at the vast majority  of TSA-regulated (also referred to as commercial) airports nationwide. On  November 19, 2002, pursuant to ATSA, TSA began a 2-year pilot  program at 5 airports using private screening companies to screen  passengers and checked baggage. In 2004, at the completion of the  pilot program, and in accordance with ATSA, TSA established a  permanent program known as the Screening Partnership Program  whereby any airport authority, whether involved in the pilot or not, could  request a transition from federal screeners to private, contracted  screeners. Each of the 5 pilot airports applied and was approved to  continue as part of the SPP, and since its establishment, 20 additional  airport applications have been accepted by the SPP. Once an airport is  approved for SPP participation and a private screening contractor has  been selected, the contract screening workforce assumes responsibility  for screening passengers and their property and must adhere to the same  security regulations, standard operating procedures, and other TSA  security requirements followed by federal screeners at commercial  airports.
subsections: section_title: Federal and Private Screening Roles and Responsibilities
paragraphs: TSA’s SPP PMO, located within TSA’s Office of Security Operations  (OSO), coordinates with local TSA officials to support an airport’s  transition from federal to private screening operations and supports the  day-to-day management of the SPP. The PMO facilitates the SPP  application process by reviewing SPP applications, organizing SPP  application review meetings with other relevant TSA offices, and  preparing and routing relevant application documentation to these offices  and the TSA Administrator. Along with the TSA Office of Acquisition, the  office plays a significant role in contract oversight and administration, as  well as actively participates in contract source selection processes.
TSA’s FSDs provide day-to-day operational direction for security  operations at the airports within their jurisdiction, including those  participating in the SPP. However, FSD management responsibilities  differ at airports using federal versus private screeners. For example, at  airports with a federal workforce, the FSD directly supervises and controls  the screening workforce. However, at SPP airports, the FSD has  responsibility for overall security but does not have direct control over  workforce management; rather the SPP contractor is contractually  obligated to effectively and efficiently manage its screening workforce.
The SPP contractor’s responsibilities include recruiting, assessing, and  training screening personnel to provide security screening functions in  accordance with TSA regulations, policies, and procedures. SPP  contractors are also expected to take operational direction from TSA,  through the FSDs, to help ensure they meet the terms and conditions of  the contract. In addition, SPP contractors are rewarded for identifying and  proposing ideas that TSA accepts for possible innovations in recruiting,  training, and security procedures, such as the practice of conducting pre- hire orientations to inform prospective screener candidates of the position  requirements, which is 1 of over 200 ideas submitted to TSA by SPP  contractors to date.
section_title: Overview of the SPP Application Process
paragraphs: In March 2012, TSA revised the SPP application to reflect requirements   Among other  of the FAA Modernization Act enacted in February 2012.provisions, the act provides that
Not later than 120 days after the date of receipt of an SPP  application submitted by an airport operator, the TSA  Administrator must approve or deny the application.
The TSA Administrator shall approve an application if approval  would not (1) compromise security, (2) detrimentally affect the  cost-efficiency of the screening of passengers or property at the  airport, or (3) detrimentally affect the effectiveness of the  screening of passengers or property at the airport.
The airport operator shall include as part of its application  submission a recommendation as to which private screening  company would best serve the security screening and passenger  needs of the airport.
Within 60 days of a denial TSA must provide the airport operator,  as well as the Committee on Commerce, Science, and  Transportation of the Senate and the Committee on Homeland  Security of the U.S. House of Representatives, a written report  that sets forth the findings that served as the basis of the denial,  the results of any cost or security analysis conducted in  considering the application, and recommendations on how the  airport operator can address the reasons for denial.
All commercial airports are eligible to apply to the SPP. To apply, an  airport operator must complete the SPP application and submit it to the  SPP PMO, as well as to the airport FSD, by mail, fax, or e-mail. As  required by the FAA Modernization Act, not later than 120 days after the  application is received by TSA, the Administrator must make a final  decision on the application. Figure 1 illustrates the SPP application  process.
Although TSA provides all airports with the opportunity to apply for  participation in the SPP, authority to approve or deny the application  resides in the discretion of the TSA Administrator. According to TSA  officials, in addition to the cost-efficiency and effectiveness considerations  mandated by FAA Modernization Act, there are many other factors that  are weighed in considering an airport’s application for SPP participation.  For example, the potential impact on the workload of the Office of  Information Technology and the potential impact of any upcoming  projects at the airport are considered. SPP PMO officials said that by  considering all relevant factors, they do not expect to identify a specific  piece of information that would definitively deny an application’s approval  based on the standards in the FAA Modernization Act. However, in doing  so, they hope to ensure that the Administrator has the complete picture  and could therefore make a decision using all factors in combination,  consistent with the FAA Modernization Act. Nonetheless, factors found to  be cost-prohibitive are likely to result in the airport being denied  participation in the program.
section_title: TSA Implemented Prior Efforts Comparing Private and Federal Screening
paragraphs: In May 2007, TSA awarded a contract to Catapult Consultants to conduct  a cost and performance analysis of airports with private screeners versus   This analysis would be used to assist  airports with federal screeners.senior TSA leadership with strategic decisions regarding the degree to  which TSA should leverage public/private partnerships in the area of  screening services. According to the December 2007 report the  contractor issued on its analysis, SPP airports performed at a level equal  to or better than non-SPP airports for the four performance measures  included in the analysis. Following this study, in February 2008, TSA  issued a report on a study TSA conducted comparing the cost and  performance of screening at SPP and non-SPP airports. The study  compared performance measures at each of six SPP airports to the non- SPP airports in the same airport category and found that SPP airports  generally performed consistently with non-SPP airports in their category  for the performance measures included in its analysis.
section_title: TSA Has Approved 25 SPP Applications; Enhanced Customer Service Was the Most Commonly Cited Advantage of the SPP
subsections: section_title: SPP Applicants and Participating Airports
paragraphs: Since the inception of the SPP in 2004, 29 airports have applied for  participation in the program; 25 airports have been approved, and as we  noted earlier in this report, 16 airports are participating in the SPP as of  October 2012. A detailed timeline and status of each airport application  are provided in figure 2 and appendix II.
Nine airports were approved but are not currently participating in the  program because they are either (1) in the process of having an SPP  contractor procured, (2) were once part of the SPP but ceased screening  services when commercial airline service placing the airport under TSA  regulation was discontinued, or (3) never transitioned to the SPP because  commercial airline service bringing the airport under TSA regulation to  these airports was discontinued before private screening services began.  Specifically, 6 airports—West Yellowstone Airport, Montana; Orlando  Sanford International Airport, Florida; Glacier Park International Airport,  Montana; Sacramento International Airport, California; Bert Mooney  Airport, Montana; and Bozeman Yellowstone International Airport,  Montana—have been approved but are not yet currently participating in  the SPP pending TSA’s selection of the screening contractor to provide  services at each airport. Two airports—the East 34th Street Heliport, New  York, and Gallup Municipal Airport, New Mexico were participating in the  SPP, but according to TSA officials, the air carriers servicing these  airports discontinued service after the contract was awarded, and thus  these airports no longer required TSA screening services. Additionally,  Florida Keys Marathon Airport, Florida, was approved for participation in  the SPP, but the air carrier servicing the airport discontinued services  prior to the start of the screening contract, and accordingly screening  services were no longer required.
TSA denied applications from 6 airports—submitted from March 2009  through December 2011. Five of these applications were submitted to  TSA before the Administrator announced in January 2011 that the agency  would not expand the SPP beyond the then current 16 airports “unless a  clear and substantial advantage to do so emerges in the future.” The sixth  application was submitted for consideration approximately 1 week after  the Administrator’s announcement. Prior to the enactment of the FAA  Modernization Act in February 2012, 1 of the 6 airports whose application  TSA denied re-applied under TSA’s “clear and substantial advantage”  standard and was approved. Following enactment of the FAA  Modernization Act, which provided that TSA shall approve an application  if approval would not compromise security or detrimentally affect the cost- efficiency or the effectiveness of the screening of passengers or property  at the airport, TSA approved the applications of 3 other airports who  reapplied. Two of the 6 airports that had been denied never reapplied for  participation in the SPP (see fig. 2 for additional details). Figure 3 and  appendix III show the locations of the 16 airports currently participating in  the SPP as well as the 6 airports that TSA recently approved for  participation.
Directions: Place mouse over each symbol for airport category and SPP status N.Dak.
Minn.
S.Dak.
Wisc.
N.Y.
N.H.
Wyo.
Mich.
Mass.
Pa.
R.I.
Nebr.
Ind.
Conn.
Colo.
Ill.
N.J.
Kans.
W. Va.
Mo.
Va.
Ky.
Del.
N.C.
Md.
N.Mex.
Tenn.
Okla.
D.C.
Ark.
S.C.
Miss.
Ala.
Ga.
Tex.
La.
Fla.
As figure 3 shows, 10 of 16 of the airports currently participating in the  SPP are smaller, category III and IV airports, with 9 of those located in  the western region of the United States.
In recent years, the number of airports applying for participation in the  SPP has generally declined. Specifically, from 2004 through 2008, 21  airports applied to the SPP, including the 5 airports that participated in  TSA’s SPP pilot program. Since 2009, TSA has received SPP  applications from 8 airports.
section_title: Reported Advantages and Disadvantages of Joining the SPP
paragraphs: Airport operators we surveyed and interviewed, as well as aviation  industry stakeholders (i.e., aviation associations) and TSA officials we  interviewed, most commonly cited customer service and staffing flexibility  as advantages of participating in the SPP, but also expressed concerns  about the SPP transition process and satisfaction with existing TSA  screening services as potential disadvantages of participating in the  program. We surveyed 28 airport operators who had applied to the SPP  from its inception in 2004 through April 2012. Twenty-six operators  responded. Because all 26 survey respondents were airport operators  who have applied to the SPP, these airport operators may be more likely  to present positive views of, or what they perceived of, the SPP. In  addition, perspectives may also be influenced by whether or not the  operators were approved for participation in the SPP at the time the  survey was conducted. We also interviewed 6 airport operators that were  not included in our survey. Five of these airport operators have not  applied for participation in the SPP, and 1 airport operator had applied for  participation after our survey was conducted, and therefore was not  included as part of our survey.
Our 2012 survey and interviews of airport operators include the following  highlights:  The advantages most frequently identified by the airport operators that  had applied to the SPP and responded to our survey and those we  interviewed (including those that had not applied to the SPP) were related  to providing better customer service and obtaining flexibility assigning  staff. The airport associations most commonly cited obtaining flexibility in  assigning staff as an advantage. Because TSA generally remains neutral  regarding the SPP, the views of TSA officials expressed are attributed to  the individual FSDs we interviewed and do not reflect the views of the  agency.
Customer service. Sixteen airport operators we surveyed and  interviewed reported customer service as an advantage—15 had  applied to the SPP and 1 had not. Specifically, 14 of 26 airport  operators responding to the survey indicated this was a realized or  potential advantage to a great or very great extent. In addition, 2  of the 6 airport operators we interviewed, 1 of which applied to the  SPP, stated that the level of customer service provided by security  screeners is particularly important for smaller community-based  airports. These airports constitute the majority of the airports  participating in the SPP, because passengers who have negative  encounters with the screening process generally associate their  experiences with the specific airport. Thus, airport officials stated  that this might increase the likelihood that the passengers involved  will seek alternative modes of transportation or different airports  for future travel. Representatives from the three airport  associations we interviewed did not identify customer service as  an advantage of the SPP. TSA officials stated that federal  screeners can and do provide similar levels of customer service  and that most commercial airports are content to have a TSA  workforce at their airports. TSA also stated that customer service  is an important aspect of their work, and that the agency is taking  steps to improve customer service in a way that does not  jeopardize the agency’s core mission, which is to ensure the  security of the traveling public. Specifically, TSA officials said that  they have enhanced their performance management processes to  better gauge customer service, such as tracking negative contacts  received at airports.
Staffing flexibility. Fifteen airport operators we surveyed and  interviewed—14 had applied to the SPP and 1 had not—and  representatives from two aviation industry associations reported  that private screening contractors are generally more responsive  and flexible than TSA to increasing staffing needs in response to  fluctuations in passenger volume at the airport. Specifically, 13 of  26 airport operators responding to our survey cited flexibility in  assigning staff as a realized or potential advantage to a great or  very great extent of participating in the SPP. Two of the 6 airport  operators we interviewed, 1 of which had applied to the SPP, also  cited staffing flexibility as an advantage. For example, an airport  operator highlighted challenges the airport has faced in adjusting  the number of screening staff to accommodate the seasonal  changes in passenger volume at his airport. Specifically, the  airport operator, a current SPP participant, commented that unlike  TSA screeners, private screening contractors are able to staff  screeners in split shifts—a work period divided into two or more  periods of time, such as morning and evening, with a break of  several hours between—thereby enabling them to adjust to the  airport’s flight schedule and changes in passenger volume. TSA  officials disagreed with this view and stated that TSA provides  FSDs with discretion to utilize federal screeners in split shifts  during the course of the workday, provided that such discretion is  exercised as the direct result of operational need. Furthermore,  TSA officials stated that all category IV and many category III  airports use split shifts. Four of six FSDs we interviewed cited a  reduced involvement in human resource management as an  advantage to the federal government for participating in the SPP.  For example, one FSD said that because TSA oversees the  screening operations of SPP airports and FSDs are not involved  with deploying and managing screening staff, they are better able  to focus on their security oversight functions, including ensuring  that proper standard operating procedures are being followed.
Cost savings. During our follow-up interviews with survey  respondents, 4 airport operators said that participating in the SPP  could help alleviate TSA resource constraints and result in cost  savings to the federal government because some airports that are  currently participating in or applied for participation in the SPP are  located in certain rural or high-cost communities where the federal  government has difficulty hiring screeners and must utilize federal  personnel deployed for temporary assignments, which results in  increased costs. An FSD of an SPP airport located in a small,  high-cost community we interviewed agreed that the salary offered  by TSA made it difficult to fill screening positions at the airport,  stating that prior to the airport’s transition to the SPP, TSA had  difficulty hiring screeners from the local area, and as a result had  to use screeners from its National Deployment Force (NDF), a  deployable federal screening workforce, because of the high cost  of living in the area. To maintain the requisite level of screening  services at airports in environments where it is hard to recruit,  TSA often uses screeners from its NDF, which TSA stated can be  more expensive than SPP screeners because the NDF screeners  are compensated on a per diem basis when deployed and incur  other costs such as temporary housing expenses.
Airport operators generally cited few realized or potential disadvantages  of participating in the SPP. Six airport operators we surveyed and  interviewed cited the discontinuation of federal screening services as a  potential disadvantage of participating in the SPP. Specifically, the 4 of 25  survey respondents who had applied to the SPP program cited the  discontinuation of federal screening services as a potential disadvantage  of participating in the SPP. In addition, 2 airport operators who have not  applied to the SPP expressed concerns about the potential disruption  associated with the transition from TSA screeners to private screeners at  their airports, and the associated risk of doing so if the process does not  proceed as smoothly as intended. One of these airport operators stated  that concerns about the transition process—going from federal screeners  to private screeners—is the primary reason the airport has not submitted  an application. Further, this airport operator also cited concerns about  maintaining screener morale, and hence security, as a major reason for   Officials from the aviation  the airport’s decision to not apply to the SPP.industry associations we interviewed did not cite any realized or potential  disadvantages. As noted earlier, TSA generally remains neutral regarding  the SPP, and accordingly did not cite disadvantages of participating in the  SPP.
Additionally, airport operators from 3 airports that have not applied to the  SPP expressed no interest in the SPP, and stated that they are generally  satisfied with the level of screening service provided by TSA. Similarly, an  Airport Council International-North America (ACI-NA) March 2007 study  found that 71 percent of 31 survey respondents were not interested in the  SPP, and cited satisfaction with TSA screening services, among other  things, for not having any interest in the SPP. When asked, representatives from all three aviation industry associations we  interviewed either expressed no opinion on the SPP or cited no  disadvantages to participating in the SPP. Two of these industry  representatives added that the majority of the airports they represent are  generally satisfied with the screening services provided by TSA.
section_title: TSA Has Developed Application Resources, but Could Provide Guidance for SPP Applicants
paragraphs: TSA has developed some resources to assist applicants; however, it has  not provided guidance on its application and approval process to assist  airports with applying to the program. As the application process was  originally implemented, TSA required that an airport operator interested in  applying to the program submit an application stating its intention to opt  out of federal screening as well as its reason(s) for wanting to do so.  However, in 2011, TSA revised its SPP application to reflect the “clear  and substantial advantage” standard announced by the Administrator in  January 2011. Specifically, TSA requested that the applicant explain how  private screening at the airport would provide a clear and substantial  advantage to TSA’s security operations. At the time, TSA did not  provide written guidance to airports to assist them in understanding what  would constitute a “clear and substantial advantage to TSA security  operations” or TSA’s basis for determining whether an airport had  established that opting out would present a clear and substantial  advantage to TSA security operations. TSA officials told us that they did  not issue guidance at the time in conjunction with the new standard  because the agency desired to maintain a neutral position on the SPP  and did not want to influence an airport’s decision to participate in the  program. In the absence of such guidance, SPP officials told us that they  were available to provide assistance, if requested, to airports that sought  assistance or information on completing their application.
In March 2012, TSA again revised the SPP application in accordance with  provisions of the FAA Modernization Act enacted in February 2012.  Among other things, the revised application no longer includes the “clear  and substantial advantage” question, but instead includes questions that  request applicants to discuss how participating in the SPP would not  compromise security at the airport and to identify potential areas where  cost savings or efficiencies may be realized. Additionally, in accordance  with the FAA Modernization Act, applicants must recommend a contractor  that would best serve the security screening and passenger needs of the  airport. TSA officials told us that the agency offers potential applicants  numerous points of contact and methods with which the applicants can  discuss the program before applying to participate. Specifically,  applicants can discuss the program with their FSD, the SPP program  manager, or their recommended screening contractor. Further, according  to TSA officials, once an airport operator submits an application, TSA  assigns a program official as a point of contact for the application, and  works with the applicant to ensure the application is complete and to keep  the applicant informed. TSA also provides general instructions for filling  out the SPP application as well as responses to frequently asked  questions (FAQ). However, TSA has not issued guidance to assist  airports with completing the new application and has not explained to  airports how it will evaluate applications given the changes brought about  by the new law. Neither the current application instructions nor the FAQs  address TSA’s SPP application evaluation process or its basis for  determining whether an airport’s entry into SPP would compromise  security or affect cost-efficiency and effectiveness.
We interviewed 4 of the 5 airport operators that applied to the SPP since  TSA revised its application in the wake of the FAA Modernization Act.  Three of the 5 told us that they struggled to answer the application  questions related to the cost-efficiency of converting to the SPP because  they did not have data on federal screening costs, while the fourth airport  operator said that she did not need additional information or guidance to  respond to the question. One of the 4 airport operators stated that he  needed the cost information to help demonstrate that his airport’s  participation in the SPP would not detrimentally affect the cost-efficiency  of the screening of passengers or property at the airport and that he  believes not presenting this information would be detrimental to his  airport’s application. However, TSA officials said that the cost information  required to answer the questions is basic cost information that airports  should already maintain and that airports do not need to provide this  information to TSA because, as part of the application evaluation process,  TSA conducts a more detailed cost analysis using historical cost data  from SPP and non-SPP airports. TSA officials added that the SPP  application and the cost information requested only serve to alert TSA of  things it may not be already aware of about the airport. The absence of  cost and other information in an individual airport’s application, TSA  officials noted, would not materially affect the TSA Administrator’s  decision on an SPP application.
Three of the 4 airport operators we interviewed, and whose applications  TSA subsequently approved after enactment of the FAA Modernization  Act, said that additional guidance would have been helpful in completing  the application and determining how TSA evaluates the applications. A  representative from 1 of the 3 airports stated that while TSA officials have  been more responsive and accessible since enactment of the FAA  Modernization Act, the agency has not necessarily been helpful with the  application process. Moreover, all 4 airport operators we interviewed told  us that TSA did not specifically assign a point of contact when they  applied to the program. Rather, all 4 airport operators reported consulting  the SPP PMO, their FSD, or their recommended contractor because they  needed information on such issues as screening cost, the list of current  SPP contractors, TSA screener staffing levels, and examples of additional  information they should provide TSA because they could not answer  some of the application questions without this information. Specifically, 1  of the 4 airport operators reported contacting the FSD to request  assistance with completing the application, while 2 of the four said they  did not because FSDs generally are not knowledgeable about the  program or are able to provide only general as opposed to detailed  information about the application process. Instead of contacting their  FSDs, these 2 airport operators told us that they contacted the SPP PMO  and stated that the office were helpful in providing general information,  such as a list of current SPP contractors, but not screening cost or other  specific application information that would help the airports demonstrate  whether the use of private screeners would compromise security or  detrimentally affect the cost-efficiency or effectiveness of the screening of  passengers or property at the airport. Another airport operator who  reported contacting the SPP PMO stated that she learned about TSA’s  SPP selection criteria and processes in the course of her discussions with  one of the SPP managers with whom she had developed a working  relationship over the years, and added that had she not contacted this  particular manager, she would not have obtained this information  because TSA does not publish the information for other airports that may  be interested in obtaining the information. Three of the 4 airport operators  who told us they sought information to complete their application from  their recommended contractor as advised by TSA stated that the  contractors told them they did not have the necessary cost information to  assist the airports with responding to the application questions related to  the cost-efficiency of converting to the SPP.
Following enactment of the FAA Modernization Act, TSA officials initially  stated that application guidance is not needed because the “clear and  substantial” basis for joining the SPP has been eliminated and responses  to the two new application questions related to cost-efficiency and  effectiveness are optional responses. However, the Assistant  Administrator for the Office of Security Operations now agrees that  providing additional high-level guidance on the kind of information TSA  considers during the application review phase would be helpful to SPP  applicants. TSA SPP officials also stated that they routinely talk about the  SPP at industry briefings and that they have done a good job of  explaining the new application to industry. However, as of September  2012, representatives of all three aviation industry associations we  interviewed told us that TSA has not provided any information on the SPP  to their association since enactment of the FAA Modernization Act in  February 2012. Additionally, representatives of two of the three aviation  industry associations said that providing guidance or information on the  criteria TSA uses to evaluate applications would be helpful to their  members, while a representative from the third aviation association that  represents domestic and international airline carriers said that its  members would appreciate any basic information on the SPP. In  interviews we conducted prior to the enactment of the FAA Modernization  Act, these same aviation industry representatives told us that the absence  of guidance provided by TSA is a barrier to applying to the program. They  added that most airports do not want to invest in preparing an application  when they are unsure as to how it would be evaluated by TSA.
TSA has approved all applications submitted since enactment of the FAA  Modernization Act; however, it is hard to determine how many more  airports, if any, would have applied to the program had TSA provided  application guidance and information to improve transparency of the SPP  application process. In the absence of such application guidance and  information, it will be difficult for more airport officials to evaluate whether  their airports are good candidates for the SPP or determine what criteria  TSA uses to accept and approve airports’ SPP applications. Further,  airports may be missing opportunities to provide TSA with cost and other  information that TSA would find useful in reviewing airport applications.  According to Standards for Internal Control in the Federal Government,  internal control and all transactions and other significant events need to  be clearly documented, and the documentation should be readily  available for examination. The documentation should appear in  management directives, administrative policies, or operating manuals and  may be in paper or electronic form. Clear guidance for applying to the  SPP could improve the transparency of the SPP application process and  help ensure that the existing application process is implemented in a  consistent and uniform manner.
section_title: TSA Has Measures to Assess Screener Performance, but Enhanced Processes for Evaluating Screener Performance Could Be Beneficial
paragraphs: TSA improved its set of screener performance measures in 2012 by  adding measures that address passenger satisfaction, thereby ensuring  that the measures address all aspects of the agency’s airport screening  strategic goals and mission. However, a mechanism to monitor private  versus federal screener performance could help TSA to routinely ensure  that the level of screening services and protection provided at SPP  airports continues to be conducted at acceptable levels provided at non- SPP airports, and could help inform TSA managers when making  decisions regarding the future of the SPP, such as whether to expand the  program to more non-SPP airports. While we found differences in  screener performance between SPP and non-SPP airports, those  differences cannot be entirely attributed to the use of either private or  federal screeners.
subsections: section_title: Performance between SPP and Non-SPP Airports Varies for Some Measures, but Differences Cannot Be Entirely Attributed to the Use of Private or Federal Screeners
paragraphs: We analyzed screener performance data for four measures and found  that while there are differences in performance between SPP and non- SPP airports, those differences cannot be exclusively attributed to the use  of either federal or private screeners. We selected these measures  primarily based on our review of previous studies that compared screener  performance of SPP and non-SPP airports as well as on our interviews  with aviation security subject matter experts, including TSA’s FSDs, SPP  contractors, and airport and aviation industry stakeholders. We also  selected performance measures for which TSA has, for the most part,  consistently and systematically collected data from fiscal year 2009  through 2011.performance at SPP and non-SPP airports are TIP detection rates,  recertification pass rates, Aviation Security Assessment Program (ASAP)  test results, and Presence, Advisement, Communication, and Execution  (PACE) evaluation results (see table 1). For each of these four measures,  we compared the performance of each of the 16 SPP airports with the  average performance for each airport’s category (X, I, II, III, or IV), as well  as the national performance averages for all airports for fiscal years 2009  through 2011.
While it is useful for TSA managers to compare an SPP airport’s performance against its  airport category for TIP detection rate and recertification pass rate in the PMRs, it is also  important that the set of measures used to compare screener performance at SPP and  non-SPP airports address a variety of agency priorities, such as passenger satisfaction.  For more on the key attributes of successful performance measures, see appendix V. information on the format of this tool and how it will be used. Further,  neither the Scorecard nor the PMR provides information on performance  in prior years nor controls for variables that TSA officials explained to us  are important when comparing private and federal screener performance,  such as the type of X-ray machine used for TIP detection rates.
Monitoring private screener performance in comparison with federal  screener performance is consistent with the statutory requirement that  TSA enter into a contract with a private screening company only if the  Administrator determines and certifies to Congress that the level of  screening services and protection provided at an airport under a contract  will be equal to or greater than the level that would be provided at the   Further, according to TSA  airport by federal government personnel.guidance on the SPP, one of TSA’s major goals for the SPP is that  private screeners must perform at the same or better level as federal  screeners. A mechanism to monitor private versus federal screener  performance would better position TSA to know whether the level of  screening services and protection provided at SPP airports continues to  be equal to or greater than the level provided at non-SPP airports. TSA  officials stated that it is not TSA’s goal to ensure that SPP airports  continue to perform at levels equal to or greater than non-SPP airports,  but to ensure that all airports operate at their optimal level, which they  monitor using across-the-board mechanisms, such as the Scorecard.  However, monitoring private versus federal screener performance could  also help TSA to identify positive or negative trends in SPP performance  that could lead to improvements in the program and TSA’s monitoring of  SPP airports in general, and inform decision-making regarding potential  future expansion of the SPP.
section_title: Conclusions
paragraphs: TSA faces a daunting task in ensuring that a screening workforce is in  place to consistently implement security protocols across the nation’s  commercial airports while facilitating passenger travel. Questions about  the performance of private screeners compared with federal screeners,  recently enacted statutory provisions, and changes to the program’s  application and approval process underscore the need for TSA to ensure  that the program’s application requirements are clearly defined and  consistently applied so that aviation stakeholders have a full and fair  opportunity to participate in the program. Thus, a well-defined and clearly  documented application guideline that states (1) the criteria and process  that TSA is using to assess airport’s participation in the SPP, (2) how TSA  will obtain and analyze cost information regarding screening cost- efficiency and effectiveness and the implications of not responding to  related application questions, and (3) specific examples of additional  information airports should consider providing to TSA to help assess  airports’ suitability for SPP could benefit TSA. Specifically, guidelines  could help alleviate airports’ uncertainty about the application process  and better inform TSA to determine whether to approve an airport’s SPP  application.
It is also incumbent on TSA to be capable of determining if airports  participating in the program are performing at a level that is equal to or  greater than the level of security that would be provided by federal  screeners at the airports through regular monitoring and reporting.  Although not a prerequisite for approving an application for participation in  the SPP, TSA must certify to Congress that the level of screening  services and protection provided by a private screening contractor will be  equal to or greater than the level that would be provided at the airport by  federal government personnel before entering into a contract with a  private screening company. While TSA regularly tracks screener  performance at all airports and reevaluates the measures it uses to  assess this performance, TSA has not conducted regular reviews  comparing private and federal screener performance and does not have  plans to do so. Regular comparison reviews would enable TSA to know  whether the level of screening services provided by private screening  contractors is equal to or greater than the level provided at non-SPP  airports. These reviews could also assist TSA in identifying performance  changes that could lead to improvements in the program and inform  decision making regarding potential expansion of the SPP.
section_title: Recommendations for Executive Action
paragraphs: To improve TSA’s SPP application process and to inform decisions  regarding the future of the SPP, we recommend that the Secretary of the  Department of Homeland Security direct the Administrator of TSA to take  the following two actions: develop guidance that clearly (1) states the criteria and process  that TSA is using to assess whether participation in the SPP  would compromise security or detrimentally affect the cost- efficiency or the effectiveness of the screening of passengers or  property at the airport; (2) states how TSA will obtain and analyze  cost information regarding screening cost-efficiency and  effectiveness and the implications of not responding to the related  application questions; and (3) provides specific examples of  additional information airports should consider providing to TSA to  help assess an airport’s suitability for SPP, and develop a mechanism to regularly monitor private versus federal  screener performance.
section_title: Agency Comments and Our Evaluation
paragraphs: We requested comments on a draft of the sensitive version of this report  from TSA. On November 7, 2012, DHS provided written comments, which  are reprinted in appendix VI and provided technical comments, which we  incorporated as appropriate. DHS generally concurred with our two  recommendations and described actions planned to address them.  Specifically,
DHS stated that TSA will provide as much information as is  prudent on how the agency would evaluate if an airport’s  participation in the SPP would compromise security or  detrimentally affect the cost-efficiency or the effectiveness of the  screening of passengers or property at the airport. Further, DHS  stated that TSA will provide general categories of information in  the SPP application guidance it plans to issue and will continually  review the guidance to ensure that airports are comfortable with  the SPP application process and understand how all the  information provided will be used to evaluate their application.  TSA expects to post an overview of the SPP application process  to the agency’s website by November 30, 2012, that would specify  details on the data it will use to assess applications and discuss its  cost-estimating methodology and definition of cost efficiency. We  believe that these are beneficial steps that would address our  recommendation once adopted, and help address stakeholder  concerns about the transparency of the SPP application process.
DHS stated that starting in the first quarter of fiscal year 2013,  TSA will produce semi-annual reports that will include an  evaluation of SPP airport performance against the performance of  TSA airports as a whole, as well as performance against each  SPP airport category. Additionally, DHS noted that TSA is in the  initial planning phase of deploying an electronic data collection  system to facilitate systematic collection and reporting of SPP  data, as well as TSA oversight of SPP contractor activities.
Deployment of the electronic data collection system is targeted for  the latter part of fiscal year 2013.  Once implemented, these new  reporting mechanisms will address our recommendation by  facilitating TSA’s efforts to assess private versus federal screener  performance.
We are sending copies of this report to the Secretary of Homeland  Security, the TSA Administrator, the House Infrastructure and  Transportation Committee, and other interested parties. In addition, the  report is available at no charge on the GAO web-site at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4379 or at lords@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  that last page of this report. GAO staff who made major contributions to  this report are listed in appendix VII.
section_title: Appendix I: Objectives, Scope, and Methodology
subsections: section_title: Objectives
paragraphs: This appendix describes how we did our work to address (1) the status of  Screening Partnership Program (SPP) applications, and airport operator,  other stakeholder, and the Transportation Security Administration’s (TSA)  views on the advantages and disadvantages of participating in the SPP;  (2) the extent to which TSA has provided guidance to govern the SPP  application process; and (3) the extent to which TSA assesses and  monitors the performance of private and federal screeners.
section_title: Scope and Methodology
paragraphs: To address all three of these objectives, we interviewed Federal Security  Directors (FSD); airport operators; screeners; and where applicable, SPP  contractors at 10 airports. We selected the 10 airports by matching an  SPP to a non-SPP airport, in each of the five airport categories (category  X, I, II, III, and IV), based primarily on (1) annual passenger and baggage  volumes, (2) screener staffing model full-time equivalent allocation, and  (3) number of check-points and screening lanes. Additionally, on the  basis of available travel resources, we visited 7 of the 10 airports to  observe airport screening operations, including any unique challenges  faced by these airports. We surveyed the 28 airport operators who have  applied to the SPP since its inception up until April 2012 to obtain their  perspectives on the SPP application process, the advantages and  disadvantages of participating in private or federal screening, and   performance measures TSA uses to assess screeners. The 28 airports  whose airport operators we surveyed include 16 from airports that were  participating in the SPP at the time of the survey, 2 airports that withdrew  their applications before TSA made a decision, 3 airports that were  approved but never transitioned to the SPP because commercial airline  service was discontinued at the airport, and 7 airports that initially applied  from March 2009 through April 2012 (when we implemented our survey).A 29th airport, Bozeman Yellowstone International Airport, applied to the  SPP for the first time in June 2012 and therefore was not included in our  survey. Two airport operators did not respond to our survey. One of the 2  was an airport that had withdrawn its application to the SPP before a  decision was made and a second an airport whose application was  denied in January 2011 while the “clear and substantial advantage”  application standard was in effect. We conducted two expert reviews of  the survey with major aviation associations, and three survey pretests  with airport operators. In addition to the 28 airport operators in our survey,  we also interviewed the airport operators of Bozeman Yellowstone  International Airport and the 5 non-SPP airports we visited to obtain their  perspectives on the potential advantages and disadvantages of  participating in the SPP. For this study, our focus is on assessing airport  screening performance as opposed to individual screener performance.  We assessed the aggregate of individual screener performance  measures only to the extent that they reflect overall screening  performance at airports.
To determine the status of SPP applications, and airport operator, other  stakeholders’, and TSA’s views on the advantages and disadvantages of  participating in the SPP, we interviewed officials of TSA’s SPP Program  Management Office (PMO) and reviewed the 15 SPP applications that  had been submitted since fiscal year 2009, as well as TSA’s available  decision memos on the applications. We also analyzed the results of our  survey of SPP airport operators and operators of airports that have  applied to the SPP. We also conducted semistructured interviews with  TSA, contractor, and airport officials during our airport site visit interviews  as well as interviewed aviation industry stakeholders to identify the  advantages and disadvantages of using federal and nonfederal  screeners.
To determine the extent to which TSA has provided guidance to govern  the SPP application process, we reviewed key statutes and policies to  identify requirements related to the SPP. We also analyzed past and  current SPP application forms and instructions, as well as interviewed  TSA headquarters officials, to identify the requirements and process for  applying to the SPP. As previously noted, we surveyed airport operators,  which included operators of all 16 SPP airports and the 6 airports whose  applications TSA denied for not establishing that transitioning to the SPP  would provide a “clear and substantial advantage to TSA security  operations,” to determine their perspectives on the SPP application  process. Further, we interviewed airport officials at the 8 airports that  have applied to the SPP since 2009, which includes the 6 airports that  applied under TSA’s “clear and substantial advantage” standard, to obtain  their perspectives on the clarity of the SPP application process. We also  compared TSA’s application process and requirements against standards  in Standards for Internal Control in the Federal Government which calls  for an agency’s transactions and other significant events to be clearly  documented and well defined.
To determine the extent to which TSA assesses and monitors the  performance of private and federal screeners, we reviewed TSA’s  screener performance measurement documents, reports, and data  systems. We also interviewed TSA headquarters officials knowledgeable  about TSA’s performance management process to identify current  screener performance measures. At the airports we visited, we observed  screening operations to identify areas where screener performance could  be assessed, and interviewed contractor, airport, and TSA officials to  obtain their perspectives on the current set of performance measures. We  reviewed TSA’s most recent set of performance measures in the Office of  Security Operations Executive Scorecard as well as its previous set in the  Management Objective Report to determine what, if any, improvements  had been made. To do so, we evaluated the sets of measures against the  nine key attributes of successful performance measures, which we  developed in prior reports based on GAO’s prior efforts to examine  agencies that were successful in implementing the performance  measurement aspects of the Government Performance and Results Act  (GPRA). the performance of federal and private screeners against standards in  Standards for Internal Control in the Federal Government and best  practices for performance management.
GAO-03-143. nationally, from fiscal year 2009 through 2011. For our comparison, we  focused on four performance measures: threat image projection (TIP)  detection rates; recertification pass rates; aviation screening  assessment program (ASAP) covert test results; and presence,  advisement, communication, and execution (PACE) evaluation results.  We selected these measures primarily based on our review of previous  studies that compared screener performance of SPP and non-SPP  airports as well as on our interviews with aviation security subject matter  experts, including TSA’s FSD, SPP contractors, and airport and aviation  industry stakeholders. We also selected performance measures for which  TSA has, for the most part, consistently and systematically collected data  for our study years. For some of the measures we selected, such as  PACE evaluations, data were not available for all 3 years or all airports;  nonetheless, we selected these measures because they represent  integral aspects of screener performance. We explain these  circumstances further when we present the data. To ensure the reliability  of the performance measures data, we (1) interviewed TSA officials who  use and maintain the data; (2) checked the data for missing information,  outliers, and obvious errors; and (3) reviewed documentation for the  relevant data systems to ensure the data’s integrity. On the basis of the  steps we took, we found the data reliable for the purpose of providing  summary statistics of screener performance for the four performance  measures we analyzed. However, as noted earlier in this report, there are  many factors, some of which cannot be controlled for, that may account  for differences in screener performance; therefore, the differences we  found in screener performance at SPP and non-SPP airports may not be  attributed entirely to the use of either federal or private screeners.
section_title: Appendix II: TSA’s Screening Partnership Program Application History
paragraphs: As of October 2012, 29 airports have applied for participation in the SPP  since the inception of the program in 2004 (see table 3).
section_title: Appendix III: Map of Screening Partnership Program Airports
paragraphs: As of October 2012, 16 airports are participating in the SPP and 6 airports  were recently approved for participation (see figure 4 and table 4).
section_title: Appendix IV: Other Measures to Assess Screener Performance
paragraphs: TSA collects data on several other performance measures, but, for  various reasons, they cannot be used to compare private and federal  screener performance for the purposes of our review. Below, we discuss  four variables occasionally cited by the airport officials and aviation  stakeholders we interviewed as possible measures for comparing federal  and private screening and the reasons we did not use them to compare  private and federal screener performance.
Wait times: A wait time is the total cycle time for a passenger to  reach the advanced imaging technology (AIT) machine or walkthrough  metal detector (whichever is available) from entering the queue. TSA  officials at some airports collect these data by passing out a card to a  passenger at the end of the line. We do not present passenger wait  time data because we found that TSA’s policy for collecting wait times  changed during the time period of our analyses and that these data  were not collected in a consistent manner across all airports. Further,  TSA officials noted that wait times are affected by a number of  variables that TSA cannot control, such as airline flight schedules.
Passenger throughput: Passenger throughput is the number of  passengers screened in each of the screening lanes per hour. These  data are collected automatically by the screening machines. TSA  officials stated that they review this measure to ensure that  passengers are not being screened too quickly, which may mean that  screeners are not being thorough, or are screened too slowly, which  may mean that screeners could be more efficient. According to TSA  officials, passenger throughput is affected by a number of factors that  are unique to individual airports, including technology, capacity and  configuration of the checkpoint, type of traveler, and various factors  related to the flight schedules. While officials noted that there is a goal  for how many passengers should be screened per hour, a rate below  this goal is not necessarily indicative of a problem, but could be due to  a reduced passenger volume, as is likely during nonpeak travel hours.  For example, at one of the airports we visited, there are few flights  scheduled for the morning and evening, at which point passenger  throughput is very low, and several flights scheduled around lunch- time, at which point the passenger throughput is relatively high.
Human capital measures: We also considered reviewing human  capital measures such as attrition, absenteeism, and injury rates.  However, TSA’s Office of Human Capital does not collect these data  for SPP airports because, according to these officials, maintaining  information on human capital measures is the sole responsibility of  the contractor. While the contractors collect and report this information  to TSA, TSA does not validate the accuracy of the self-reported data.  Further, TSA does not require that the contractors use the same  human capital measures as TSA, and accordingly, differences may  exist in how the metrics are defined and how the data are collected.  Therefore, TSA cannot guarantee that a comparison of SPP and non- SPP airports on these human capital metrics would be an equal  comparison. TSA officials also stated that they do not use human  capital measures to compare SPP and non-SPP airports because  these measures are affected by variables that are not within the  control of TSA or the contractor. For example, some airports are  located in areas that have a high cost of living, and as a result, it can  be difficult to hire screeners because the screener salary may not be  competitive there. “Red team” covert tests: In addition to ASAP tests, TSA’s Office of  Inspections also conducts covert tests, the results of which are also  classified. These covert tests are commonly referred to as red team  tests, and are designed to identify potential vulnerabilities in TSA’s  screening operations, as opposed to test screeners’ compliance with  standard operating procedures. We have previously reported that an  airport’s red team test results represent a snapshot in time and should  not be considered a comprehensive measurement of any one airport’s  performance or any individual airport’s performance. Further, while  GAO analyzed red team tests in these reports, we determined, for  reasons we cannot report here due to the sensitive security nature of  the information, that it would not be appropriate to analyze the tests  for the purpose of comparing screener performance at SPP and non- SPP airports.
section_title: Appendix V: GAO’s Assessment of Screener Performance Measures
paragraphs: By adding measures to the Scorecard that addressed other non-security- related TSA priorities, TSA improved the set of performance measures it  uses to asses screener performance. In the past, we have examined  agencies that were successful in implementing the performance  measurement aspects of the Government Performance and Results Act  and concluded that these agencies exhibit certain key characteristics that  it characterized as the nine key attributes of successful performance  measures. While the Management Objective Report (MOR) addressed  eight of the key attributes, it did not address balance because the set of  performance measures did not address a variety of agency priorities.  Balance among a set of performance measures is important because it  helps to ensure that performance measurement efforts are not  overemphasizing one or two priorities at the expense of others, which  may keep managers from understanding the effectiveness of their  program in supporting the agency’s overall missions and goals.  Specifically, the MOR did not contain measures related to passenger  satisfaction which, according to TSA’s Strategic Plan, is part of the  agency’s mission. However, the Office of Security Operations (OSO)  Executive Scorecard (Scorecard) includes passenger satisfaction  measures, such as the number of negative and positive customer  contacts made to the TSA Contact Center through e-mails or phone calls  per 100,000 passengers screened through the airport, which were not  previously included in the MOR. By adding measures related to  passenger satisfaction to the Scorecard, TSA ensured balance in the set  of performance measures the agency uses to assess screener  performance and thereby ensured that its assessment of screening  operation performance would be representative of a variety of program  and agency goals (see table 5).
section_title: Appendix VI: Comments from the Department of Homeland Security
section_title: Appendix VII: GAO Contact and Staff Acknowledgments
paragraphs: Appendix VII: GAO Contact and Staff  Acknowledgments Error! No text of specified style in document.
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Glenn Davis, Assistant Director,  and Edith Sohna, Analyst-in-Charge, managed this assignment. Erin  O’Brien and Michelle Woods made significant contributions to the work.  Carl Barden, Stuart Kaufman, Stanley Kostyla, and Minette Richardson  assisted with design and methodology. Tom Lombardi provided legal  support. Linda Miller provided assistance in report preparation, and Lydia  Araya made contributions to the graphics presented in the report.