Screening Partnership Program: TSA Should Issue More Guidance to Airports and Monitor Private versus Federal Screener Performance
Released: 2012-12-06T00:00:00

Letter

Background

ATSA established TSA and charged it with responsibility for securing all modes of transportation, including civil aviation. Prior to ATSA and the establishment of TSA, passenger and baggage screening had generally been performed by private screening companies under contract to airlines and in accordance with FAA regulations. In accordance with ATSA, TSA currently employs personnel who screen passengers at the vast majority of TSA-regulated (also referred to as commercial) airports nationwide. On November 19, 2002, pursuant to ATSA, TSA began a 2-year pilot program at 5 airports using private screening companies to screen passengers and checked baggage. In 2004, at the completion of the pilot program, and in accordance with ATSA, TSA established a permanent program known as the Screening Partnership Program whereby any airport authority, whether involved in the pilot or not, could request a transition from federal screeners to private, contracted screeners. Each of the 5 pilot airports applied and was approved to continue as part of the SPP, and since its establishment, 20 additional airport applications have been accepted by the SPP. Once an airport is approved for SPP participation and a private screening contractor has been selected, the contract screening workforce assumes responsibility for screening passengers and their property and must adhere to the same security regulations, standard operating procedures, and other TSA security requirements followed by federal screeners at commercial airports.

Federal and Private Screening Roles and Responsibilities

TSA’s SPP PMO, located within TSA’s Office of Security Operations (OSO), coordinates with local TSA officials to support an airport’s transition from federal to private screening operations and supports the day-to-day management of the SPP. The PMO facilitates the SPP application process by reviewing SPP applications, organizing SPP application review meetings with other relevant TSA offices, and preparing and routing relevant application documentation to these offices and the TSA Administrator. Along with the TSA Office of Acquisition, the office plays a significant role in contract oversight and administration, as well as actively participates in contract source selection processes.

TSA’s FSDs provide day-to-day operational direction for security operations at the airports within their jurisdiction, including those participating in the SPP. However, FSD management responsibilities differ at airports using federal versus private screeners. For example, at airports with a federal workforce, the FSD directly supervises and controls the screening workforce. However, at SPP airports, the FSD has responsibility for overall security but does not have direct control over workforce management; rather the SPP contractor is contractually obligated to effectively and efficiently manage its screening workforce.

The SPP contractor’s responsibilities include recruiting, assessing, and training screening personnel to provide security screening functions in accordance with TSA regulations, policies, and procedures. SPP contractors are also expected to take operational direction from TSA, through the FSDs, to help ensure they meet the terms and conditions of the contract. In addition, SPP contractors are rewarded for identifying and proposing ideas that TSA accepts for possible innovations in recruiting, training, and security procedures, such as the practice of conducting pre- hire orientations to inform prospective screener candidates of the position requirements, which is 1 of over 200 ideas submitted to TSA by SPP contractors to date.

Overview of the SPP Application Process

In March 2012, TSA revised the SPP application to reflect requirements Among other of the FAA Modernization Act enacted in February 2012.provisions, the act provides that

Not later than 120 days after the date of receipt of an SPP application submitted by an airport operator, the TSA Administrator must approve or deny the application.

The TSA Administrator shall approve an application if approval would not (1) compromise security, (2) detrimentally affect the cost-efficiency of the screening of passengers or property at the airport, or (3) detrimentally affect the effectiveness of the screening of passengers or property at the airport.

The airport operator shall include as part of its application submission a recommendation as to which private screening company would best serve the security screening and passenger needs of the airport.

Within 60 days of a denial TSA must provide the airport operator, as well as the Committee on Commerce, Science, and Transportation of the Senate and the Committee on Homeland Security of the U.S. House of Representatives, a written report that sets forth the findings that served as the basis of the denial, the results of any cost or security analysis conducted in considering the application, and recommendations on how the airport operator can address the reasons for denial.

All commercial airports are eligible to apply to the SPP. To apply, an airport operator must complete the SPP application and submit it to the SPP PMO, as well as to the airport FSD, by mail, fax, or e-mail. As required by the FAA Modernization Act, not later than 120 days after the application is received by TSA, the Administrator must make a final decision on the application. Figure 1 illustrates the SPP application process.

Although TSA provides all airports with the opportunity to apply for participation in the SPP, authority to approve or deny the application resides in the discretion of the TSA Administrator. According to TSA officials, in addition to the cost-efficiency and effectiveness considerations mandated by FAA Modernization Act, there are many other factors that are weighed in considering an airport’s application for SPP participation. For example, the potential impact on the workload of the Office of Information Technology and the potential impact of any upcoming projects at the airport are considered. SPP PMO officials said that by considering all relevant factors, they do not expect to identify a specific piece of information that would definitively deny an application’s approval based on the standards in the FAA Modernization Act. However, in doing so, they hope to ensure that the Administrator has the complete picture and could therefore make a decision using all factors in combination, consistent with the FAA Modernization Act. Nonetheless, factors found to be cost-prohibitive are likely to result in the airport being denied participation in the program.

TSA Implemented Prior Efforts Comparing Private and Federal Screening

In May 2007, TSA awarded a contract to Catapult Consultants to conduct a cost and performance analysis of airports with private screeners versus This analysis would be used to assist airports with federal screeners.senior TSA leadership with strategic decisions regarding the degree to which TSA should leverage public/private partnerships in the area of screening services. According to the December 2007 report the contractor issued on its analysis, SPP airports performed at a level equal to or better than non-SPP airports for the four performance measures included in the analysis. Following this study, in February 2008, TSA issued a report on a study TSA conducted comparing the cost and performance of screening at SPP and non-SPP airports. The study compared performance measures at each of six SPP airports to the non- SPP airports in the same airport category and found that SPP airports generally performed consistently with non-SPP airports in their category for the performance measures included in its analysis.

TSA Has Approved 25 SPP Applications; Enhanced Customer Service Was the Most Commonly Cited Advantage of the SPP

SPP Applicants and Participating Airports

Since the inception of the SPP in 2004, 29 airports have applied for participation in the program; 25 airports have been approved, and as we noted earlier in this report, 16 airports are participating in the SPP as of October 2012. A detailed timeline and status of each airport application are provided in figure 2 and appendix II.

Nine airports were approved but are not currently participating in the program because they are either (1) in the process of having an SPP contractor procured, (2) were once part of the SPP but ceased screening services when commercial airline service placing the airport under TSA regulation was discontinued, or (3) never transitioned to the SPP because commercial airline service bringing the airport under TSA regulation to these airports was discontinued before private screening services began. Specifically, 6 airports—West Yellowstone Airport, Montana; Orlando Sanford International Airport, Florida; Glacier Park International Airport, Montana; Sacramento International Airport, California; Bert Mooney Airport, Montana; and Bozeman Yellowstone International Airport, Montana—have been approved but are not yet currently participating in the SPP pending TSA’s selection of the screening contractor to provide services at each airport. Two airports—the East 34th Street Heliport, New York, and Gallup Municipal Airport, New Mexico were participating in the SPP, but according to TSA officials, the air carriers servicing these airports discontinued service after the contract was awarded, and thus these airports no longer required TSA screening services. Additionally, Florida Keys Marathon Airport, Florida, was approved for participation in the SPP, but the air carrier servicing the airport discontinued services prior to the start of the screening contract, and accordingly screening services were no longer required.

TSA denied applications from 6 airports—submitted from March 2009 through December 2011. Five of these applications were submitted to TSA before the Administrator announced in January 2011 that the agency would not expand the SPP beyond the then current 16 airports “unless a clear and substantial advantage to do so emerges in the future.” The sixth application was submitted for consideration approximately 1 week after the Administrator’s announcement. Prior to the enactment of the FAA Modernization Act in February 2012, 1 of the 6 airports whose application TSA denied re-applied under TSA’s “clear and substantial advantage” standard and was approved. Following enactment of the FAA Modernization Act, which provided that TSA shall approve an application if approval would not compromise security or detrimentally affect the cost- efficiency or the effectiveness of the screening of passengers or property at the airport, TSA approved the applications of 3 other airports who reapplied. Two of the 6 airports that had been denied never reapplied for participation in the SPP (see fig. 2 for additional details). Figure 3 and appendix III show the locations of the 16 airports currently participating in the SPP as well as the 6 airports that TSA recently approved for participation.

Directions: Place mouse over each symbol for airport category and SPP status N.Dak.

Minn.

S.Dak.

Wisc.

N.Y.

N.H.

Wyo.

Mich.

Mass.

Pa.

R.I.

Nebr.

Ind.

Conn.

Colo.

Ill.

N.J.

Kans.

W. Va.

Mo.

Va.

Ky.

Del.

N.C.

Md.

N.Mex.

Tenn.

Okla.

D.C.

Ark.

S.C.

Miss.

Ala.

Ga.

Tex.

La.

Fla.

As figure 3 shows, 10 of 16 of the airports currently participating in the SPP are smaller, category III and IV airports, with 9 of those located in the western region of the United States.

In recent years, the number of airports applying for participation in the SPP has generally declined. Specifically, from 2004 through 2008, 21 airports applied to the SPP, including the 5 airports that participated in TSA’s SPP pilot program. Since 2009, TSA has received SPP applications from 8 airports.

Reported Advantages and Disadvantages of Joining the SPP

Airport operators we surveyed and interviewed, as well as aviation industry stakeholders (i.e., aviation associations) and TSA officials we interviewed, most commonly cited customer service and staffing flexibility as advantages of participating in the SPP, but also expressed concerns about the SPP transition process and satisfaction with existing TSA screening services as potential disadvantages of participating in the program. We surveyed 28 airport operators who had applied to the SPP from its inception in 2004 through April 2012. Twenty-six operators responded. Because all 26 survey respondents were airport operators who have applied to the SPP, these airport operators may be more likely to present positive views of, or what they perceived of, the SPP. In addition, perspectives may also be influenced by whether or not the operators were approved for participation in the SPP at the time the survey was conducted. We also interviewed 6 airport operators that were not included in our survey. Five of these airport operators have not applied for participation in the SPP, and 1 airport operator had applied for participation after our survey was conducted, and therefore was not included as part of our survey.

Our 2012 survey and interviews of airport operators include the following highlights: The advantages most frequently identified by the airport operators that had applied to the SPP and responded to our survey and those we interviewed (including those that had not applied to the SPP) were related to providing better customer service and obtaining flexibility assigning staff. The airport associations most commonly cited obtaining flexibility in assigning staff as an advantage. Because TSA generally remains neutral regarding the SPP, the views of TSA officials expressed are attributed to the individual FSDs we interviewed and do not reflect the views of the agency.

Customer service. Sixteen airport operators we surveyed and interviewed reported customer service as an advantage—15 had applied to the SPP and 1 had not. Specifically, 14 of 26 airport operators responding to the survey indicated this was a realized or potential advantage to a great or very great extent. In addition, 2 of the 6 airport operators we interviewed, 1 of which applied to the SPP, stated that the level of customer service provided by security screeners is particularly important for smaller community-based airports. These airports constitute the majority of the airports participating in the SPP, because passengers who have negative encounters with the screening process generally associate their experiences with the specific airport. Thus, airport officials stated that this might increase the likelihood that the passengers involved will seek alternative modes of transportation or different airports for future travel. Representatives from the three airport associations we interviewed did not identify customer service as an advantage of the SPP. TSA officials stated that federal screeners can and do provide similar levels of customer service and that most commercial airports are content to have a TSA workforce at their airports. TSA also stated that customer service is an important aspect of their work, and that the agency is taking steps to improve customer service in a way that does not jeopardize the agency’s core mission, which is to ensure the security of the traveling public. Specifically, TSA officials said that they have enhanced their performance management processes to better gauge customer service, such as tracking negative contacts received at airports.

Staffing flexibility. Fifteen airport operators we surveyed and interviewed—14 had applied to the SPP and 1 had not—and representatives from two aviation industry associations reported that private screening contractors are generally more responsive and flexible than TSA to increasing staffing needs in response to fluctuations in passenger volume at the airport. Specifically, 13 of 26 airport operators responding to our survey cited flexibility in assigning staff as a realized or potential advantage to a great or very great extent of participating in the SPP. Two of the 6 airport operators we interviewed, 1 of which had applied to the SPP, also cited staffing flexibility as an advantage. For example, an airport operator highlighted challenges the airport has faced in adjusting the number of screening staff to accommodate the seasonal changes in passenger volume at his airport. Specifically, the airport operator, a current SPP participant, commented that unlike TSA screeners, private screening contractors are able to staff screeners in split shifts—a work period divided into two or more periods of time, such as morning and evening, with a break of several hours between—thereby enabling them to adjust to the airport’s flight schedule and changes in passenger volume. TSA officials disagreed with this view and stated that TSA provides FSDs with discretion to utilize federal screeners in split shifts during the course of the workday, provided that such discretion is exercised as the direct result of operational need. Furthermore, TSA officials stated that all category IV and many category III airports use split shifts. Four of six FSDs we interviewed cited a reduced involvement in human resource management as an advantage to the federal government for participating in the SPP. For example, one FSD said that because TSA oversees the screening operations of SPP airports and FSDs are not involved with deploying and managing screening staff, they are better able to focus on their security oversight functions, including ensuring that proper standard operating procedures are being followed.

Cost savings. During our follow-up interviews with survey respondents, 4 airport operators said that participating in the SPP could help alleviate TSA resource constraints and result in cost savings to the federal government because some airports that are currently participating in or applied for participation in the SPP are located in certain rural or high-cost communities where the federal government has difficulty hiring screeners and must utilize federal personnel deployed for temporary assignments, which results in increased costs. An FSD of an SPP airport located in a small, high-cost community we interviewed agreed that the salary offered by TSA made it difficult to fill screening positions at the airport, stating that prior to the airport’s transition to the SPP, TSA had difficulty hiring screeners from the local area, and as a result had to use screeners from its National Deployment Force (NDF), a deployable federal screening workforce, because of the high cost of living in the area. To maintain the requisite level of screening services at airports in environments where it is hard to recruit, TSA often uses screeners from its NDF, which TSA stated can be more expensive than SPP screeners because the NDF screeners are compensated on a per diem basis when deployed and incur other costs such as temporary housing expenses.

Airport operators generally cited few realized or potential disadvantages of participating in the SPP. Six airport operators we surveyed and interviewed cited the discontinuation of federal screening services as a potential disadvantage of participating in the SPP. Specifically, the 4 of 25 survey respondents who had applied to the SPP program cited the discontinuation of federal screening services as a potential disadvantage of participating in the SPP. In addition, 2 airport operators who have not applied to the SPP expressed concerns about the potential disruption associated with the transition from TSA screeners to private screeners at their airports, and the associated risk of doing so if the process does not proceed as smoothly as intended. One of these airport operators stated that concerns about the transition process—going from federal screeners to private screeners—is the primary reason the airport has not submitted an application. Further, this airport operator also cited concerns about maintaining screener morale, and hence security, as a major reason for Officials from the aviation the airport’s decision to not apply to the SPP.industry associations we interviewed did not cite any realized or potential disadvantages. As noted earlier, TSA generally remains neutral regarding the SPP, and accordingly did not cite disadvantages of participating in the SPP.

Additionally, airport operators from 3 airports that have not applied to the SPP expressed no interest in the SPP, and stated that they are generally satisfied with the level of screening service provided by TSA. Similarly, an Airport Council International-North America (ACI-NA) March 2007 study found that 71 percent of 31 survey respondents were not interested in the SPP, and cited satisfaction with TSA screening services, among other things, for not having any interest in the SPP. When asked, representatives from all three aviation industry associations we interviewed either expressed no opinion on the SPP or cited no disadvantages to participating in the SPP. Two of these industry representatives added that the majority of the airports they represent are generally satisfied with the screening services provided by TSA.

TSA Has Developed Application Resources, but Could Provide Guidance for SPP Applicants

TSA has developed some resources to assist applicants; however, it has not provided guidance on its application and approval process to assist airports with applying to the program. As the application process was originally implemented, TSA required that an airport operator interested in applying to the program submit an application stating its intention to opt out of federal screening as well as its reason(s) for wanting to do so. However, in 2011, TSA revised its SPP application to reflect the “clear and substantial advantage” standard announced by the Administrator in January 2011. Specifically, TSA requested that the applicant explain how private screening at the airport would provide a clear and substantial advantage to TSA’s security operations. At the time, TSA did not provide written guidance to airports to assist them in understanding what would constitute a “clear and substantial advantage to TSA security operations” or TSA’s basis for determining whether an airport had established that opting out would present a clear and substantial advantage to TSA security operations. TSA officials told us that they did not issue guidance at the time in conjunction with the new standard because the agency desired to maintain a neutral position on the SPP and did not want to influence an airport’s decision to participate in the program. In the absence of such guidance, SPP officials told us that they were available to provide assistance, if requested, to airports that sought assistance or information on completing their application.

In March 2012, TSA again revised the SPP application in accordance with provisions of the FAA Modernization Act enacted in February 2012. Among other things, the revised application no longer includes the “clear and substantial advantage” question, but instead includes questions that request applicants to discuss how participating in the SPP would not compromise security at the airport and to identify potential areas where cost savings or efficiencies may be realized. Additionally, in accordance with the FAA Modernization Act, applicants must recommend a contractor that would best serve the security screening and passenger needs of the airport. TSA officials told us that the agency offers potential applicants numerous points of contact and methods with which the applicants can discuss the program before applying to participate. Specifically, applicants can discuss the program with their FSD, the SPP program manager, or their recommended screening contractor. Further, according to TSA officials, once an airport operator submits an application, TSA assigns a program official as a point of contact for the application, and works with the applicant to ensure the application is complete and to keep the applicant informed. TSA also provides general instructions for filling out the SPP application as well as responses to frequently asked questions (FAQ). However, TSA has not issued guidance to assist airports with completing the new application and has not explained to airports how it will evaluate applications given the changes brought about by the new law. Neither the current application instructions nor the FAQs address TSA’s SPP application evaluation process or its basis for determining whether an airport’s entry into SPP would compromise security or affect cost-efficiency and effectiveness.

We interviewed 4 of the 5 airport operators that applied to the SPP since TSA revised its application in the wake of the FAA Modernization Act. Three of the 5 told us that they struggled to answer the application questions related to the cost-efficiency of converting to the SPP because they did not have data on federal screening costs, while the fourth airport operator said that she did not need additional information or guidance to respond to the question. One of the 4 airport operators stated that he needed the cost information to help demonstrate that his airport’s participation in the SPP would not detrimentally affect the cost-efficiency of the screening of passengers or property at the airport and that he believes not presenting this information would be detrimental to his airport’s application. However, TSA officials said that the cost information required to answer the questions is basic cost information that airports should already maintain and that airports do not need to provide this information to TSA because, as part of the application evaluation process, TSA conducts a more detailed cost analysis using historical cost data from SPP and non-SPP airports. TSA officials added that the SPP application and the cost information requested only serve to alert TSA of things it may not be already aware of about the airport. The absence of cost and other information in an individual airport’s application, TSA officials noted, would not materially affect the TSA Administrator’s decision on an SPP application.

Three of the 4 airport operators we interviewed, and whose applications TSA subsequently approved after enactment of the FAA Modernization Act, said that additional guidance would have been helpful in completing the application and determining how TSA evaluates the applications. A representative from 1 of the 3 airports stated that while TSA officials have been more responsive and accessible since enactment of the FAA Modernization Act, the agency has not necessarily been helpful with the application process. Moreover, all 4 airport operators we interviewed told us that TSA did not specifically assign a point of contact when they applied to the program. Rather, all 4 airport operators reported consulting the SPP PMO, their FSD, or their recommended contractor because they needed information on such issues as screening cost, the list of current SPP contractors, TSA screener staffing levels, and examples of additional information they should provide TSA because they could not answer some of the application questions without this information. Specifically, 1 of the 4 airport operators reported contacting the FSD to request assistance with completing the application, while 2 of the four said they did not because FSDs generally are not knowledgeable about the program or are able to provide only general as opposed to detailed information about the application process. Instead of contacting their FSDs, these 2 airport operators told us that they contacted the SPP PMO and stated that the office were helpful in providing general information, such as a list of current SPP contractors, but not screening cost or other specific application information that would help the airports demonstrate whether the use of private screeners would compromise security or detrimentally affect the cost-efficiency or effectiveness of the screening of passengers or property at the airport. Another airport operator who reported contacting the SPP PMO stated that she learned about TSA’s SPP selection criteria and processes in the course of her discussions with one of the SPP managers with whom she had developed a working relationship over the years, and added that had she not contacted this particular manager, she would not have obtained this information because TSA does not publish the information for other airports that may be interested in obtaining the information. Three of the 4 airport operators who told us they sought information to complete their application from their recommended contractor as advised by TSA stated that the contractors told them they did not have the necessary cost information to assist the airports with responding to the application questions related to the cost-efficiency of converting to the SPP.

Following enactment of the FAA Modernization Act, TSA officials initially stated that application guidance is not needed because the “clear and substantial” basis for joining the SPP has been eliminated and responses to the two new application questions related to cost-efficiency and effectiveness are optional responses. However, the Assistant Administrator for the Office of Security Operations now agrees that providing additional high-level guidance on the kind of information TSA considers during the application review phase would be helpful to SPP applicants. TSA SPP officials also stated that they routinely talk about the SPP at industry briefings and that they have done a good job of explaining the new application to industry. However, as of September 2012, representatives of all three aviation industry associations we interviewed told us that TSA has not provided any information on the SPP to their association since enactment of the FAA Modernization Act in February 2012. Additionally, representatives of two of the three aviation industry associations said that providing guidance or information on the criteria TSA uses to evaluate applications would be helpful to their members, while a representative from the third aviation association that represents domestic and international airline carriers said that its members would appreciate any basic information on the SPP. In interviews we conducted prior to the enactment of the FAA Modernization Act, these same aviation industry representatives told us that the absence of guidance provided by TSA is a barrier to applying to the program. They added that most airports do not want to invest in preparing an application when they are unsure as to how it would be evaluated by TSA.

TSA has approved all applications submitted since enactment of the FAA Modernization Act; however, it is hard to determine how many more airports, if any, would have applied to the program had TSA provided application guidance and information to improve transparency of the SPP application process. In the absence of such application guidance and information, it will be difficult for more airport officials to evaluate whether their airports are good candidates for the SPP or determine what criteria TSA uses to accept and approve airports’ SPP applications. Further, airports may be missing opportunities to provide TSA with cost and other information that TSA would find useful in reviewing airport applications. According to Standards for Internal Control in the Federal Government, internal control and all transactions and other significant events need to be clearly documented, and the documentation should be readily available for examination. The documentation should appear in management directives, administrative policies, or operating manuals and may be in paper or electronic form. Clear guidance for applying to the SPP could improve the transparency of the SPP application process and help ensure that the existing application process is implemented in a consistent and uniform manner.

TSA Has Measures to Assess Screener Performance, but Enhanced Processes for Evaluating Screener Performance Could Be Beneficial

TSA improved its set of screener performance measures in 2012 by adding measures that address passenger satisfaction, thereby ensuring that the measures address all aspects of the agency’s airport screening strategic goals and mission. However, a mechanism to monitor private versus federal screener performance could help TSA to routinely ensure that the level of screening services and protection provided at SPP airports continues to be conducted at acceptable levels provided at non- SPP airports, and could help inform TSA managers when making decisions regarding the future of the SPP, such as whether to expand the program to more non-SPP airports. While we found differences in screener performance between SPP and non-SPP airports, those differences cannot be entirely attributed to the use of either private or federal screeners.

Performance between SPP and Non-SPP Airports Varies for Some Measures, but Differences Cannot Be Entirely Attributed to the Use of Private or Federal Screeners

We analyzed screener performance data for four measures and found that while there are differences in performance between SPP and non- SPP airports, those differences cannot be exclusively attributed to the use of either federal or private screeners. We selected these measures primarily based on our review of previous studies that compared screener performance of SPP and non-SPP airports as well as on our interviews with aviation security subject matter experts, including TSA’s FSDs, SPP contractors, and airport and aviation industry stakeholders. We also selected performance measures for which TSA has, for the most part, consistently and systematically collected data from fiscal year 2009 through 2011.performance at SPP and non-SPP airports are TIP detection rates, recertification pass rates, Aviation Security Assessment Program (ASAP) test results, and Presence, Advisement, Communication, and Execution (PACE) evaluation results (see table 1). For each of these four measures, we compared the performance of each of the 16 SPP airports with the average performance for each airport’s category (X, I, II, III, or IV), as well as the national performance averages for all airports for fiscal years 2009 through 2011.

While it is useful for TSA managers to compare an SPP airport’s performance against its airport category for TIP detection rate and recertification pass rate in the PMRs, it is also important that the set of measures used to compare screener performance at SPP and non-SPP airports address a variety of agency priorities, such as passenger satisfaction. For more on the key attributes of successful performance measures, see appendix V. information on the format of this tool and how it will be used. Further, neither the Scorecard nor the PMR provides information on performance in prior years nor controls for variables that TSA officials explained to us are important when comparing private and federal screener performance, such as the type of X-ray machine used for TIP detection rates.

Monitoring private screener performance in comparison with federal screener performance is consistent with the statutory requirement that TSA enter into a contract with a private screening company only if the Administrator determines and certifies to Congress that the level of screening services and protection provided at an airport under a contract will be equal to or greater than the level that would be provided at the Further, according to TSA airport by federal government personnel.guidance on the SPP, one of TSA’s major goals for the SPP is that private screeners must perform at the same or better level as federal screeners. A mechanism to monitor private versus federal screener performance would better position TSA to know whether the level of screening services and protection provided at SPP airports continues to be equal to or greater than the level provided at non-SPP airports. TSA officials stated that it is not TSA’s goal to ensure that SPP airports continue to perform at levels equal to or greater than non-SPP airports, but to ensure that all airports operate at their optimal level, which they monitor using across-the-board mechanisms, such as the Scorecard. However, monitoring private versus federal screener performance could also help TSA to identify positive or negative trends in SPP performance that could lead to improvements in the program and TSA’s monitoring of SPP airports in general, and inform decision-making regarding potential future expansion of the SPP.

Conclusions

TSA faces a daunting task in ensuring that a screening workforce is in place to consistently implement security protocols across the nation’s commercial airports while facilitating passenger travel. Questions about the performance of private screeners compared with federal screeners, recently enacted statutory provisions, and changes to the program’s application and approval process underscore the need for TSA to ensure that the program’s application requirements are clearly defined and consistently applied so that aviation stakeholders have a full and fair opportunity to participate in the program. Thus, a well-defined and clearly documented application guideline that states (1) the criteria and process that TSA is using to assess airport’s participation in the SPP, (2) how TSA will obtain and analyze cost information regarding screening cost- efficiency and effectiveness and the implications of not responding to related application questions, and (3) specific examples of additional information airports should consider providing to TSA to help assess airports’ suitability for SPP could benefit TSA. Specifically, guidelines could help alleviate airports’ uncertainty about the application process and better inform TSA to determine whether to approve an airport’s SPP application.

It is also incumbent on TSA to be capable of determining if airports participating in the program are performing at a level that is equal to or greater than the level of security that would be provided by federal screeners at the airports through regular monitoring and reporting. Although not a prerequisite for approving an application for participation in the SPP, TSA must certify to Congress that the level of screening services and protection provided by a private screening contractor will be equal to or greater than the level that would be provided at the airport by federal government personnel before entering into a contract with a private screening company. While TSA regularly tracks screener performance at all airports and reevaluates the measures it uses to assess this performance, TSA has not conducted regular reviews comparing private and federal screener performance and does not have plans to do so. Regular comparison reviews would enable TSA to know whether the level of screening services provided by private screening contractors is equal to or greater than the level provided at non-SPP airports. These reviews could also assist TSA in identifying performance changes that could lead to improvements in the program and inform decision making regarding potential expansion of the SPP.

Recommendations for Executive Action

To improve TSA’s SPP application process and to inform decisions regarding the future of the SPP, we recommend that the Secretary of the Department of Homeland Security direct the Administrator of TSA to take the following two actions: develop guidance that clearly (1) states the criteria and process that TSA is using to assess whether participation in the SPP would compromise security or detrimentally affect the cost- efficiency or the effectiveness of the screening of passengers or property at the airport; (2) states how TSA will obtain and analyze cost information regarding screening cost-efficiency and effectiveness and the implications of not responding to the related application questions; and (3) provides specific examples of additional information airports should consider providing to TSA to help assess an airport’s suitability for SPP, and develop a mechanism to regularly monitor private versus federal screener performance.

Agency Comments and Our Evaluation

We requested comments on a draft of the sensitive version of this report from TSA. On November 7, 2012, DHS provided written comments, which are reprinted in appendix VI and provided technical comments, which we incorporated as appropriate. DHS generally concurred with our two recommendations and described actions planned to address them. Specifically,

DHS stated that TSA will provide as much information as is prudent on how the agency would evaluate if an airport’s participation in the SPP would compromise security or detrimentally affect the cost-efficiency or the effectiveness of the screening of passengers or property at the airport. Further, DHS stated that TSA will provide general categories of information in the SPP application guidance it plans to issue and will continually review the guidance to ensure that airports are comfortable with the SPP application process and understand how all the information provided will be used to evaluate their application. TSA expects to post an overview of the SPP application process to the agency’s website by November 30, 2012, that would specify details on the data it will use to assess applications and discuss its cost-estimating methodology and definition of cost efficiency. We believe that these are beneficial steps that would address our recommendation once adopted, and help address stakeholder concerns about the transparency of the SPP application process.

DHS stated that starting in the first quarter of fiscal year 2013, TSA will produce semi-annual reports that will include an evaluation of SPP airport performance against the performance of TSA airports as a whole, as well as performance against each SPP airport category. Additionally, DHS noted that TSA is in the initial planning phase of deploying an electronic data collection system to facilitate systematic collection and reporting of SPP data, as well as TSA oversight of SPP contractor activities.

Deployment of the electronic data collection system is targeted for the latter part of fiscal year 2013. Once implemented, these new reporting mechanisms will address our recommendation by facilitating TSA’s efforts to assess private versus federal screener performance.

We are sending copies of this report to the Secretary of Homeland Security, the TSA Administrator, the House Infrastructure and Transportation Committee, and other interested parties. In addition, the report is available at no charge on the GAO web-site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-4379 or at lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on that last page of this report. GAO staff who made major contributions to this report are listed in appendix VII.

Appendix I: Objectives, Scope, and Methodology

Objectives

This appendix describes how we did our work to address (1) the status of Screening Partnership Program (SPP) applications, and airport operator, other stakeholder, and the Transportation Security Administration’s (TSA) views on the advantages and disadvantages of participating in the SPP; (2) the extent to which TSA has provided guidance to govern the SPP application process; and (3) the extent to which TSA assesses and monitors the performance of private and federal screeners.

Scope and Methodology

To address all three of these objectives, we interviewed Federal Security Directors (FSD); airport operators; screeners; and where applicable, SPP contractors at 10 airports. We selected the 10 airports by matching an SPP to a non-SPP airport, in each of the five airport categories (category X, I, II, III, and IV), based primarily on (1) annual passenger and baggage volumes, (2) screener staffing model full-time equivalent allocation, and (3) number of check-points and screening lanes. Additionally, on the basis of available travel resources, we visited 7 of the 10 airports to observe airport screening operations, including any unique challenges faced by these airports. We surveyed the 28 airport operators who have applied to the SPP since its inception up until April 2012 to obtain their perspectives on the SPP application process, the advantages and disadvantages of participating in private or federal screening, and performance measures TSA uses to assess screeners. The 28 airports whose airport operators we surveyed include 16 from airports that were participating in the SPP at the time of the survey, 2 airports that withdrew their applications before TSA made a decision, 3 airports that were approved but never transitioned to the SPP because commercial airline service was discontinued at the airport, and 7 airports that initially applied from March 2009 through April 2012 (when we implemented our survey).A 29th airport, Bozeman Yellowstone International Airport, applied to the SPP for the first time in June 2012 and therefore was not included in our survey. Two airport operators did not respond to our survey. One of the 2 was an airport that had withdrawn its application to the SPP before a decision was made and a second an airport whose application was denied in January 2011 while the “clear and substantial advantage” application standard was in effect. We conducted two expert reviews of the survey with major aviation associations, and three survey pretests with airport operators. In addition to the 28 airport operators in our survey, we also interviewed the airport operators of Bozeman Yellowstone International Airport and the 5 non-SPP airports we visited to obtain their perspectives on the potential advantages and disadvantages of participating in the SPP. For this study, our focus is on assessing airport screening performance as opposed to individual screener performance. We assessed the aggregate of individual screener performance measures only to the extent that they reflect overall screening performance at airports.

To determine the status of SPP applications, and airport operator, other stakeholders’, and TSA’s views on the advantages and disadvantages of participating in the SPP, we interviewed officials of TSA’s SPP Program Management Office (PMO) and reviewed the 15 SPP applications that had been submitted since fiscal year 2009, as well as TSA’s available decision memos on the applications. We also analyzed the results of our survey of SPP airport operators and operators of airports that have applied to the SPP. We also conducted semistructured interviews with TSA, contractor, and airport officials during our airport site visit interviews as well as interviewed aviation industry stakeholders to identify the advantages and disadvantages of using federal and nonfederal screeners.

To determine the extent to which TSA has provided guidance to govern the SPP application process, we reviewed key statutes and policies to identify requirements related to the SPP. We also analyzed past and current SPP application forms and instructions, as well as interviewed TSA headquarters officials, to identify the requirements and process for applying to the SPP. As previously noted, we surveyed airport operators, which included operators of all 16 SPP airports and the 6 airports whose applications TSA denied for not establishing that transitioning to the SPP would provide a “clear and substantial advantage to TSA security operations,” to determine their perspectives on the SPP application process. Further, we interviewed airport officials at the 8 airports that have applied to the SPP since 2009, which includes the 6 airports that applied under TSA’s “clear and substantial advantage” standard, to obtain their perspectives on the clarity of the SPP application process. We also compared TSA’s application process and requirements against standards in Standards for Internal Control in the Federal Government which calls for an agency’s transactions and other significant events to be clearly documented and well defined.

To determine the extent to which TSA assesses and monitors the performance of private and federal screeners, we reviewed TSA’s screener performance measurement documents, reports, and data systems. We also interviewed TSA headquarters officials knowledgeable about TSA’s performance management process to identify current screener performance measures. At the airports we visited, we observed screening operations to identify areas where screener performance could be assessed, and interviewed contractor, airport, and TSA officials to obtain their perspectives on the current set of performance measures. We reviewed TSA’s most recent set of performance measures in the Office of Security Operations Executive Scorecard as well as its previous set in the Management Objective Report to determine what, if any, improvements had been made. To do so, we evaluated the sets of measures against the nine key attributes of successful performance measures, which we developed in prior reports based on GAO’s prior efforts to examine agencies that were successful in implementing the performance measurement aspects of the Government Performance and Results Act (GPRA). the performance of federal and private screeners against standards in Standards for Internal Control in the Federal Government and best practices for performance management.

GAO-03-143. nationally, from fiscal year 2009 through 2011. For our comparison, we focused on four performance measures: threat image projection (TIP) detection rates; recertification pass rates; aviation screening assessment program (ASAP) covert test results; and presence, advisement, communication, and execution (PACE) evaluation results. We selected these measures primarily based on our review of previous studies that compared screener performance of SPP and non-SPP airports as well as on our interviews with aviation security subject matter experts, including TSA’s FSD, SPP contractors, and airport and aviation industry stakeholders. We also selected performance measures for which TSA has, for the most part, consistently and systematically collected data for our study years. For some of the measures we selected, such as PACE evaluations, data were not available for all 3 years or all airports; nonetheless, we selected these measures because they represent integral aspects of screener performance. We explain these circumstances further when we present the data. To ensure the reliability of the performance measures data, we (1) interviewed TSA officials who use and maintain the data; (2) checked the data for missing information, outliers, and obvious errors; and (3) reviewed documentation for the relevant data systems to ensure the data’s integrity. On the basis of the steps we took, we found the data reliable for the purpose of providing summary statistics of screener performance for the four performance measures we analyzed. However, as noted earlier in this report, there are many factors, some of which cannot be controlled for, that may account for differences in screener performance; therefore, the differences we found in screener performance at SPP and non-SPP airports may not be attributed entirely to the use of either federal or private screeners.

Appendix II: TSA’s Screening Partnership Program Application History

As of October 2012, 29 airports have applied for participation in the SPP since the inception of the program in 2004 (see table 3).

Appendix III: Map of Screening Partnership Program Airports

As of October 2012, 16 airports are participating in the SPP and 6 airports were recently approved for participation (see figure 4 and table 4).

Appendix IV: Other Measures to Assess Screener Performance

TSA collects data on several other performance measures, but, for various reasons, they cannot be used to compare private and federal screener performance for the purposes of our review. Below, we discuss four variables occasionally cited by the airport officials and aviation stakeholders we interviewed as possible measures for comparing federal and private screening and the reasons we did not use them to compare private and federal screener performance.

Wait times: A wait time is the total cycle time for a passenger to reach the advanced imaging technology (AIT) machine or walkthrough metal detector (whichever is available) from entering the queue. TSA officials at some airports collect these data by passing out a card to a passenger at the end of the line. We do not present passenger wait time data because we found that TSA’s policy for collecting wait times changed during the time period of our analyses and that these data were not collected in a consistent manner across all airports. Further, TSA officials noted that wait times are affected by a number of variables that TSA cannot control, such as airline flight schedules.

Passenger throughput: Passenger throughput is the number of passengers screened in each of the screening lanes per hour. These data are collected automatically by the screening machines. TSA officials stated that they review this measure to ensure that passengers are not being screened too quickly, which may mean that screeners are not being thorough, or are screened too slowly, which may mean that screeners could be more efficient. According to TSA officials, passenger throughput is affected by a number of factors that are unique to individual airports, including technology, capacity and configuration of the checkpoint, type of traveler, and various factors related to the flight schedules. While officials noted that there is a goal for how many passengers should be screened per hour, a rate below this goal is not necessarily indicative of a problem, but could be due to a reduced passenger volume, as is likely during nonpeak travel hours. For example, at one of the airports we visited, there are few flights scheduled for the morning and evening, at which point passenger throughput is very low, and several flights scheduled around lunch- time, at which point the passenger throughput is relatively high.

Human capital measures: We also considered reviewing human capital measures such as attrition, absenteeism, and injury rates. However, TSA’s Office of Human Capital does not collect these data for SPP airports because, according to these officials, maintaining information on human capital measures is the sole responsibility of the contractor. While the contractors collect and report this information to TSA, TSA does not validate the accuracy of the self-reported data. Further, TSA does not require that the contractors use the same human capital measures as TSA, and accordingly, differences may exist in how the metrics are defined and how the data are collected. Therefore, TSA cannot guarantee that a comparison of SPP and non- SPP airports on these human capital metrics would be an equal comparison. TSA officials also stated that they do not use human capital measures to compare SPP and non-SPP airports because these measures are affected by variables that are not within the control of TSA or the contractor. For example, some airports are located in areas that have a high cost of living, and as a result, it can be difficult to hire screeners because the screener salary may not be competitive there. “Red team” covert tests: In addition to ASAP tests, TSA’s Office of Inspections also conducts covert tests, the results of which are also classified. These covert tests are commonly referred to as red team tests, and are designed to identify potential vulnerabilities in TSA’s screening operations, as opposed to test screeners’ compliance with standard operating procedures. We have previously reported that an airport’s red team test results represent a snapshot in time and should not be considered a comprehensive measurement of any one airport’s performance or any individual airport’s performance. Further, while GAO analyzed red team tests in these reports, we determined, for reasons we cannot report here due to the sensitive security nature of the information, that it would not be appropriate to analyze the tests for the purpose of comparing screener performance at SPP and non- SPP airports.

Appendix V: GAO’s Assessment of Screener Performance Measures

By adding measures to the Scorecard that addressed other non-security- related TSA priorities, TSA improved the set of performance measures it uses to asses screener performance. In the past, we have examined agencies that were successful in implementing the performance measurement aspects of the Government Performance and Results Act and concluded that these agencies exhibit certain key characteristics that it characterized as the nine key attributes of successful performance measures. While the Management Objective Report (MOR) addressed eight of the key attributes, it did not address balance because the set of performance measures did not address a variety of agency priorities. Balance among a set of performance measures is important because it helps to ensure that performance measurement efforts are not overemphasizing one or two priorities at the expense of others, which may keep managers from understanding the effectiveness of their program in supporting the agency’s overall missions and goals. Specifically, the MOR did not contain measures related to passenger satisfaction which, according to TSA’s Strategic Plan, is part of the agency’s mission. However, the Office of Security Operations (OSO) Executive Scorecard (Scorecard) includes passenger satisfaction measures, such as the number of negative and positive customer contacts made to the TSA Contact Center through e-mails or phone calls per 100,000 passengers screened through the airport, which were not previously included in the MOR. By adding measures related to passenger satisfaction to the Scorecard, TSA ensured balance in the set of performance measures the agency uses to assess screener performance and thereby ensured that its assessment of screening operation performance would be representative of a variety of program and agency goals (see table 5).

Appendix VI: Comments from the Department of Homeland Security

Appendix VII: GAO Contact and Staff Acknowledgments

Appendix VII: GAO Contact and Staff Acknowledgments Error! No text of specified style in document.

GAO Contact

Acknowledgments

In addition to the contact named above, Glenn Davis, Assistant Director, and Edith Sohna, Analyst-in-Charge, managed this assignment. Erin O’Brien and Michelle Woods made significant contributions to the work. Carl Barden, Stuart Kaufman, Stanley Kostyla, and Minette Richardson assisted with design and methodology. Tom Lombardi provided legal support. Linda Miller provided assistance in report preparation, and Lydia Araya made contributions to the graphics presented in the report.