Checked Baggage Screening: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated Airports, but Could Strengthen Cost Estimates
Published: 2012-04-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: TSAs EBSP, one of DHSs largest acquisition programs, aims to improve security and lower program life cycle costs by optimizing checked baggage screening systems that best meet the needs of the nations airports. This includes, among other things, the integration of baggage screening equipment into baggage handling systems, referred to as in-line systems. Installing in-line systems typically requires airports to undertake costly facility modification projects, for which TSA will generally reimburse up to the applicable federal cost share. As requested, GAO examined (1) the status of TSAs efforts to install optimal checked baggage screening systems in collaboration with airports, (2) how reducing the federal cost share for eligible airport modification projects from 90 percent to its previous level of 75 percent would affect the amount that TSA pays for modifications, and (3) whether TSAs methods for estimating and validating costs for the EBSP are consistent with best practices. GAO reviewed EBSP planning and status documents, compared TSAs cost estimation approach against GAO best practices, and visited 10 airports selected in part based on the status of the EBSP optimization at these airports. Although the results from these visits are not generalizable, they provided insights into the program.
section_title: What GAO Found
paragraphs: The Transportation Security Administrations (TSA) Electronic Baggage Screening Program (EBSP) reports that 76 percent of the airports (337 of 446) the agency regulates for security have a mix of in-line and stand-alone baggage screening configurations that best meet airport needs (i.e., optimal systems). However, only 36 percent (10 of 28) of the nations larger airportsbased on factors such as the total number of takeoffs and landings annuallyhave complete optimal systems. This is because the larger airports generally need more complex in-line systems and often require a significant amount of airport infrastructure modification and construction. In August 2011, TSA shifted its focus from installing optimal baggage screening systems to replacing aging machines (recapitalization). However, TSA plans to continue to optimize systems during many of its recapitalization projects.
Using TSA cost estimates, GAO estimates that reducing the portion of costs that TSA pays for facility modifications associated with the installation of optimal baggage screening systems, from 90 percent to 75 percent, would lower the federal governments cost for airport modification projects it supports by roughly $300 million from fiscal year 2012 through fiscal year 2030. Officials from all 10 airports with whom GAO spoke stated that airports benefit from the installation of integrated, in-line baggage screening systems. The primary benefitcited by representatives from 9 of the airports GAO visitedis that passenger congestion is reduced by removing stand-alone machines from lobbies or ticketing areas. Other benefits cited by airports included a reduction in lost baggage and increased screening and passenger throughput. However, for a variety of reasons, representatives from 8 of 10 airports GAO visited opposed a reduction in the federal cost share for related airport modifications.
TSA established cost estimates for the EBSP to help identify total program cost, recapitalization cost, and potential savings resulting from installing optimal systems, but its processes for developing these estimates do not fully comply with best practices. These include, among other things, ensuring that the estimates comprise all costs and are well documented. For example, TSAs estimates were properly adjusted for inflation and were developed using relevant data, such as existing contracts for equipment purchases and maintenance costs. However, the estimates did not include all costs, for example, the costs associated with detecting all security threats, and many assumptions and methodologies underlying the cost model were not clearly documented. As highlighted in GAOs past work, a high-quality, reliable cost estimation process provides a sound basis for making accurate and well-informed decisions about resource investments and budgets and thus is critical to the success of a program. Developing accurate cost estimates would help ensure that the program does not experience unanticipated cost growth and program funding needs resulting from future recapitalization and facility modification activities. In addition, TSA is working with the Department of Homeland Security (DHS) to develop an approved acquisition program baseline, which according to DHS guidance is the contract between program and departmental oversight officials for what will be delivered, how it will perform, and what it will cost. TSA expects the baseline to be approved in May 2012.
section_title: What GAO Recommends
paragraphs: GAO recommends that TSA ensure that its life cycle cost estimates conform to cost estimating best practices. DHS concurred with GAOs recommendation.

section_title: Letter
paragraphs: As demonstrated by the attempted attack of Northwest Airlines Flight 253  from Amsterdam to Detroit on Christmas Day in 2009, commercial  aviation continues to be a target of terrorist activity. The 1988 bombing of  a U.S. airliner over Lockerbie, Scotland, further demonstrates that  explosive devices placed in checked baggage have long been an area of  concern. The Aviation and Transportation Security Act (ATSA), enacted  on November 19, 2001, created the Transportation Security  Administration (TSA), now a component of the Department of Homeland  Security (DHS), and mandated, among other things, that it provide for the  screening of all checked baggage using explosive detection systems. To  satisfy this mandate, TSA has deployed two types of screening  equipment to airports in the United States where screening is required:  (1) explosives detection systems (EDS), which use X-rays with computer- aided imaging to automatically recognize the characteristic signatures of  threat explosives, and (2) explosives trace detection (ETD) machines, in  which a human operator (baggage screener) uses chemical analysis to  manually detect traces of explosive materials’ vapors and residue.
As amended, ATSA mandated that TSA provide for the screening of all  checked baggage using EDS machines no later than December 31,  2003. To meet the 100 percent checked baggage screening  requirement, TSA deployed screening equipment in a variety of  temporary screening solutions, including EDS machines placed in stand- alone configurations in airport terminal lobbies, and solutions that relied  on ETD machines for primary screening. According to TSA, these initial  stand-alone EDS machines have higher operating costs and can screen  fewer bags per hour than EDS machines that are integrated into a single,  coordinated “in-line” baggage handling system. To improve the efficiency  and effectiveness of baggage screening at airports, TSA’s Electronic  Baggage Screening Program (EBSP) aims to, among other goals, (1)  replace, reconfigure, and deploy equipment to increase throughput,  system capacity, and effectiveness while reducing staffing requirements  and airport lobby installations; (2) increase equipment reliability, reduce  equipment downtime, and extend service life; and (3) achieve new  capabilities to better detect changing terrorist threats.
Pursuant to ATSA, providing for checked baggage screening is a federal  responsibility, but airports must be capable of accommodating the  screening systems to secure safe travel for the nation’s passengers. TSA  collaborates with airports and airlines to determine the most appropriate  screening configuration and install the systems that best fit the airports’ or  airlines’ needs. Such optimal baggage screening systems may have a  mix of detection systems (EDS or ETD) and configurations (in-line or  stand-alone systems), depending on the airport’s needs, and may help  streamline airport and TSA operations, reduce screening costs, and  enhance security. However, in-line solutions may not be appropriate or  cost effective in airports or terminals with fewer bags per week to screen.  From fiscal years 2008 through 2011, TSA reported that it obligated  approximately $1.9 billion to support airport facility modification projects  required to install more efficient optimal baggage screening systems, and  estimates that it will obligate an additional $1.15 billion for this purpose  through fiscal year 2030. TSA has determined that manual ETD  screening is an optimal configuration for the nation’s smaller airports at  which TSA performs or oversees screening.
According to TSA’s Planning Guidelines and Design Standards for Checked Baggage  Inspections Systems, version 4.1, September 15, 2011, the project sponsor is assumed to  be an airport owner/operator or an airline (if the system is for an airline-owned terminal).  Some of the key responsibilities of the project sponsor include initiation and execution of  planning and design, application for TSA funding, initiation and execution of construction,  as well as testing and commissioning of the checked baggage inspection system and  operation and maintenance of the baggage handling system portion of the overall checked  baggage inspection system. TSA is responsible for, among other things, developing the  planning guidelines and design standards, working with airports and airlines to develop  and review designs, providing the EDS and ETD screening equipment, and final approval  of the checked baggage system. percent.which it generally funds 90 percent of the costs of modifying airports for  the installation of an optimal system, with airports or airlines funding the  remaining 10 percent of the projects’ cost.
Currently, TSA enters into reimbursable agreements through  A foundation for making informed budget decisions for acquisition  programs at DHS is the acquisition program baseline, which according to  DHS’s guidance is its contract with a component, such as TSA, on a  program’s critical costs, schedule, and program performance goals.  Establishing such a baseline at the program start is important for defining  its scope, assessing whether all life cycle costs are properly calculated,  and measuring how well the program is meeting its goals. By tracking and  measuring actual program performance against this baseline,  management can be alerted to potential problems, such as cost growth or  changing requirements, and has the ability to take early corrective action.
You asked us to assess the EBSP’s current status, costs, and alternative  cost sharing options. Accordingly, this report addresses the following  questions:
What is the status of TSA’s efforts to install optimal checked baggage  screening systems in collaboration with airports?
How would reducing the current federal cost share for eligible airport  modification projects from 90 percent to its previous level of 75  percent affect the amount that TSA pays for these modifications, and  what benefits, if any, do airports report receiving from in-line baggage  screening systems?
To what extent are TSA’s cost estimation procedures consistent with  best practices and is TSA’s acquisition baseline consistent with DHS  guidance?
To determine the status of TSA’s efforts to install optimal checked  baggage screening systems, we obtained the most current data available  from TSA as of December 2011 and January 2012 for the following: 1) on  the current number of airports with at least one in-line system, 2) the  number of airports that have optimal checked baggage screening  systems, 3) and the number of operational in-line systems and EDS and  ETD machines deployed to TSA-regulated airports. We reviewed  documentation from TSA’s EBSP, including the EBSP strategic plans for  fiscal years 2006, 2008, and 2009, and the 2011 EDS and ETD  Recapitalization and Optimization Plan.
To determine how reducing the current federal cost share for eligible  airport facility modification projects may affect the amount that TSA pays  for these modifications, we calculated estimates based on TSA’s August  2011 projections of how much airport modifications will cost in the future.  To estimate what cost shifts could occur in the future, we obtained TSA’s  projections of its airport modification costs as stated in its final August  2011 Life Cycle Cost Estimate (LCCE) report. We used TSA’s LCCE  projections of airport modification costs for each fiscal year, 2012 through  2030, and estimated the amount of expenditures for airport modifications  that could be shifted from TSA to airports and airlines if the federal  government’s cost share were reduced from the current 90 percent to  previous federal cost share of 75 percent.
We assessed the reliability of the various data TSA provided about  airports, including the number of TSA-regulated airports (by category) and  the numbers of airports with the different configurations of baggage  screening (in-line or stand-alone) systems, and the investment and  budget expenditure dollar values in the LOIs and OTAs by questioning  cognizant TSA officials and obtaining documentation about these various  data.our report.
We found these data to be sufficiently reliable for the purposes of  To assess the extent to which TSA’s cost estimation methods are  consistent with best practices and how its acquisition program baseline  efforts are consistent with DHS guidance, we analyzed the TSA August  2011 LCCE for EBSP. We used best practices in GAO’s Cost Estimating  and Assessment Guide to evaluate TSA’s estimating methodologies,  assumptions, and results to assess whether the official cost estimates  were comprehensive (i.e., include all costs), accurate, well documented,   We defined a characteristic of the LCCE as not met if the  and credible. agency provided no evidence that satisfied any portion of the criterion,  minimally met if the agency provided evidence that satisfied less than  one-half of the criterion, partially met if the agency provided evidence that  satisfied about one-half of the criterion, substantially met if the agency  provided evidence that satisfied more than one-half of the criterion, and  met if the agency provided complete evidence that satisfied the entire  criterion. Our analyses examined TSA’s practices in developing its LCCE.  Regarding the development of an acquisition program baseline, we  examined the EBSP Acquisition Review Board decisions and relevant  acquisition decision memos during the period 2005 through 2011, as well  as other documentation related to DHS and TSA efforts to develop an  acquisition program baseline.officials, including officials in the TSA Chief Financial Officer’s Office and  DHS’s Program Accountability and Risk Management Office, to identify   We also interviewed TSA and DHS  what procedures they have put in place to approve the acquisition  program baseline.
To gain a better understanding of issues across all of our objectives, we  conducted site visits to airports in California, New York, Massachusetts,  Washington, D.C., and Florida to interview local airport officials, regional  TSA officials, and airline representatives. To get a range of airports, we  chose these airports based on the size of airport, type of checked  baggage screening system installed, and whether the airport’s checked  baggage screening system had been optimized. Because we selected a  nonprobability sample of airports, the information we obtained from these  interviews and visits cannot be generalized to all airports. However, we  believe that observations obtained from these visits provided us with a  greater understanding of airport officials’ perspectives on TSA’s  collaboration with airports that seek to have optimal baggage screening  systems installed and allowable costs reimbursed, as well as the  perspective of local TSA officials. In addition, we interviewed officials from  the largest industry associations, which represent airport executives,  airports, and airlines (the American Association of Airport Executives, the  Airports Council International North America, and the Air Transport  Association). We also interviewed an official from the Association for  Airline Passenger Rights. These results cannot be generalized to the  entire industry but provided industry perspectives.
We conducted this performance audit from October 2010 through April  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit  to obtain sufficient, appropriate evidence to provide a reasonable basis  for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: With the passage of ATSA in November 2001, TSA assumed from the  Federal Aviation Administration (FAA) responsibility for securing the  nation’s civil aviation system. In accordance with ATSA, TSA is  responsible for the procurement, installation, and maintenance of  explosive detection systems, including EDS and ETD, used to screen  checked baggage for explosives (see figs. 1 and 2) at TSA-regulated  airports. EDS machines identify suspicious items or anomalies in  checked baggage that could indicate the presence of explosives or  detonation devices. At airports with EDS, EDS machines are generally  employed for primary screening of checked baggage while ETD machines  are used for secondary screening to help resolve questions raised by  EDS screening. At airports without EDS, ETD machines are used as the  primary method for screening checked baggage.
TSA deploys EDS machines in stand-alone and in-line configurations. In  a stand-alone configuration, checked baggage is manually loaded and  unloaded by screeners (see fig. 1). In contrast, an in-line configuration  integrates EDS machines with a baggage handling system—a conveyor  system that transports and sorts baggage from the ticket counter through  the baggage screening system. (See fig. 3, which shows an in-line  configuration with three EDS machines.)
Generally, an in-line checked baggage inspection system employs three  levels of screening (see fig. 4). EDS machines perform automated (Level  1) screening. If the EDS machine is unable to clear a bag, it sends an  alarm to a screener who performs a secondary (Level 2) inspection  known as On-Screen Resolution by reviewing an image of the contents of  the bag via computer monitor. If the screener cannot resolve the alarm  using on-screen resolution tools, the bag goes to the Checked Baggage  Resolution Area (Level 3) where another screener will perform manual  inspection of the bag assisted by an ETD machine.
TSA officials stated that deployment of an integrated, centralized in-line  system of EDS machines can enhance security, increase screening  efficiencies, and lower screening costs by, among other things, reducing  the number of screeners needed to conduct baggage screening and  reducing work-related injuries caused by lifting heavy bags. Installing an  in-line system can require modification of an airport terminal, including  removal of the existing system, installation of a new baggage handling  system and EDS machines, and the use of an interim solution to screen  checked baggage while the in-line system is built. TSA estimates that  depending on the size and complexity of an in-line project, installing an in- line system can take one to four years at larger (category X and I)  airports.
In 2005, we reported that although TSA made substantial progress in  installing EDS machines, the agency had not conducted a systematic,  prospective analysis to determine which airports could achieve long-term  savings and improve efficiencies and security by installing in-line systems  or, where in-line systems may not be economically justifiable, by making  greater use of stand-alone EDS rather than relying on the labor-intensive  and less efficient ETD screening processes. We recommended that  TSA systematically evaluate baggage screening needs at airports,  including identifying and prioritizing the airports where the benefits—such  as cost savings of screening operations and improved security—of  replacing stand-alone baggage screening machines with in-line systems  are likely to exceed the costs of the systems. TSA concurred and in  response released its Strategic Planning Framework in February 2006,  which identified and prioritized airports based on an analysis of several  factors, including security risk and the amount of estimated cost savings.
In March 2011, we reported that by continuing to replace or modify older  baggage screening systems with more efficient solutions, including in-line  screening systems, TSA could continue to eliminate baggage screener   TSA agreed that the deployment of more efficient systems  positions. offers potential personnel cost savings to the federal government.
GAO-11-318SP. airports, will be operated at the levels in established requirements and we  recommended that TSA develop a reliable schedule for the EBSP. DHS  concurred with these recommendations and has initiated actions to  implement them.
In February 2012, we reported that we continue to believe that TSA might  achieve savings in screening personnel costs by continuing to replace or  modify older checked baggage screening systems with more efficient  solutions, including in-line screening systems, to the extent possible.  TSA reported that since the issuance of GAO’s 2011 report, it had  replaced 60 stand-alone checked baggage screening machines with more  efficient in-line screening systems.
Since fiscal year 2006, about $6.8 billion has been made available to TSA  for activities related to checked baggage screening (see table 1), making  the EBSP one of DHS’s largest acquisition programs.
Following the terrorist attacks of September 11, 2001, and the enactment  of ATSA, airports relied upon various sources of funding to support  security-related capital improvement projects. For example, as enacted,  ATSA authorized the use of Airport Improvement Program (AIP) funds for  projects to support the installation of explosive detection systems.  Subsequently enacted statutes, such as the Vision 100—Century of  Aviation Reauthorization Act (Vision 100), however, either limited or  precluded the use of AIP to fund projects related to the installation of in- line systems. TSA, which is solely responsible for procuring and  deploying equipment to screen checked baggage for explosives, also  provides funding in support of related facility modifications. The  Consolidated Appropriations Resolution, 2003, first authorized the use of  LOIs by TSA for airport facility modification projects related to the  installation of in-line baggage screening systems. Although not a  binding commitment of federal funding, LOIs are agreements providing  that TSA will reimburse airports or airlines for a specified percentage of  an eligible project’s cost, subject to the availability of appropriations. This  in turn enables an airport to proceed with a project because the airport  and any investors are aware that the agreed-upon percentage of  allowable costs will likely be reimbursed. The airport or airline is  responsible for its share of the total funding needed to complete the  project and generally must be capable of funding the project in its entirety.
From fiscal years 2003 through 2007, TSA entered into 8 LOI agreements  covering 9 airports. Pursuant to the law then in effect, these LOI agreements provided for a 75 percent federal cost share of allowable  project costs. Beginning in fiscal year 2008 and in accordance with  Vision 100, any LOI entered into by TSA was to reflect a 90 percent  federal cost share. Since 2008, TSA has entered into 4 more LOIs at a  90 percent federal cost share. As of December 13, 2011, TSA reported  that its net cumulative obligations for all 12 LOIs were $1.46 billion. TSA  also uses OTAs to support airport facility modification projects related to  the installation of checked baggage screening equipment. TSA describes  OTAs, which have become the primary administrative vehicles through  which TSA financially supports such projects, generally as single-year  reimbursable agreements (in contrast to the multiyear LOI agreements).  According to TSA, OTAs take many forms and are generally not required  to comply with federal laws and regulations that apply to contracts or  cooperative agreements, such as the Federal Acquisitions Regulations,  thus enabling the parties to negotiate provisions that are mutually  agreeable. According to TSA, the federal cost share applied to OTAs may  be negotiated, but since fiscal year 2008 TSA has generally followed the  federal cost share applicable to LOIs. As of the end of fiscal year 2011,  TSA had used at least 150 OTAs to support airport facility modifications  related to the installation of in-line systems. According to TSA, these  OTAs have reflected federal cost shares ranging from 50 to100 percent.  As of December 13, 2011, TSA reported that its net cumulative  obligations for all OTAs were $1.74 billion.
section_title: TSA Has Made Progress in Deploying Optimal Checked Baggage Screening
subsections: section_title: TSA Has Collaborated with Airports to Install Optimal Baggage Screening Systems at 76 Percent of TSA-Regulated Airports
paragraphs: TSA reports that 337 of 446 airports (76 percent) the agency regulates for  security have optimal baggage screening systems. The remaining 109 of  446 airports (24 percent) do not yet have optimal baggage screening  systems in all screening areas. To be considered an airport with an  optimal baggage screening system, as TSA considers it and as we define  it for purposes of this report, an airport must have completed installation  and activation of the in-line or stand-alone systems that best fits the  airport’s screening needs without relying on temporary stand-alone  systems.may have a mix of explosive detection systems (EDS or ETD) and  configurations (in-line or stand-alone systems), depending on the airport’s  needs. TSA officials told us that they plan to deploy equipment for an  additional 201 in-line systems in the future, which will include the   Thus, an airport with an optimal baggage screening system  purchase of an estimated 685 new EDS machines for installation at  airports that are not screening with optimal configurations.
TSA aims to complete its efforts to deploy optimal screening systems by  using EDS machines as the primary means for screening checked  baggage at all category X, I, II, and III airports while continuing to use  ETD machines as the primary means at category IV airports.  Additionally, TSA plans to deploy EDS machines in in-line configurations  at all category X and I airports and in-line or stand-alone configurations at  category II and III airports. At each of the 10 airports we visited, we  observed distinct checked baggage screening needs, based on an  airport’s terminal configuration and the number of passenger boardings.  We discussed an airport’s willingness and financial ability to pay for  facility modifications required to install in-line systems. For each of these  airports, officials and engineers provided documentation (for example,  drawings and blueprints) on distinct facility modification projects to  accommodate baggage screening system upgrades. According to airport  officials, TSA and airport officials work together to determine the most  appropriate baggage screening configuration based on an airport’s  needs. Of the 337 airports where baggage screening systems are optimal  and no longer using temporary solutions, 55 airports use EDS in-line  systems exclusively for their primary checked baggage screening needs,  while 92 airports use EDS stand-alone machines only, and 167 use ETD  systems exclusively. The remaining 23 airports have a mix of systems.
See figure 5 for additional details on airport screening configurations.  Also, see appendix II for more details on the status of efforts to optimize  checked baggage screening systems.
Of the 337 airports with optimal baggage screening systems, larger  airports were less likely to have completed optimal solutions than smaller  airports. Specifically, 36 percent (10 of 28) of the category X airports  and 49 percent (28 of 57) of the category I airports were considered to  have optimal solutions, whereas 60 percent (46 of 77) of the category II  airports, 76 percent (96 of 127) of the category III airports, and all of the  157 of the category IV airports were considered to be optimally  configured, as shown in figure 6.
According to TSA and airport officials,  this is because the larger airports generally need to install more complex  in-line systems, which are more time and resource intensive to install and  often require a significant amount of airport infrastructure modification and  construction, while the smaller airports, particularly the category IV  airports that rely on the smaller ETD machines, require far less time and  resources to install these systems. Moreover, TSA officials stated that the  in-line systems that best meet the screening needs of larger airports take  longer to plan and build because of the added complexity and scale of the  upgrades and the coordination required among multiple stakeholders.
TSA anticipates that in the next 5 years about 60 percent (1,153 of 1,933)  of the EDS machines will reach the end of their useful life of about 10  years and will need to be replaced, as shown in figure 7. As a result, to  ensure that 100 percent of checked bags continue to be screened as  required by ATSA, TSA revised its focus from replacing stand-alone EDS  in airport lobbies with in-line systems to replacing its aging fleet of EDS  and ETD machines, a process it calls recapitalization. However, TSA  reported that it will continue to collaborate with airports or airlines to install  optimal in-line systems if it coincides with efforts to recapitalize aging  EDS machines or if the existing in-line systems do not meet current TSA  standards. In August 2011, TSA issued its EDS and ETD Recapitalization  and Optimization Plan, which establishes the method and criteria for  prioritizing when and how EBSP recapitalization and optimization will  occur. The plan notes that many in-line recapitalization projects will  include an optimization component. For example, a number of early in- line screening systems are likely to require optimization, among other  things, to improve performance, increase efficiency, and reduce operating  costs. At one airport we visited with an early in-line system, we observed  a baggage handling system that needed to be replaced because it had  sharp curves and steep grades that led to an excessive number of errors  and jams. The airport was involved in TSA’s recapitalization pilot  program, and airport officials anticipated TSA supporting the optimization  of this system as part of recapitalization.
section_title: Reducing the Federal Cost Share for Eligible Airport Facility Modification Projects Could Reduce TSA’s Costs
subsections: section_title: Reducing the Federal Cost Share
paragraphs: Consistent with current law, TSA enters into reimbursable agreements  through which it generally funds 90 percent of the cost of an eligible  airport facility modification project to support the installation of an optimal  system, with an airport or airline funding the remaining 10 percent of the  project’s cost. If the federal cost share for airport facility modification  projects is reduced, TSA may be able to use available funding to install a  greater number of optimal solutions than currently anticipated. From fiscal  year 2003 through fiscal year 2007, in accordance with law then in effect,  TSA entered into LOIs at a 75 percent federal cost share. Looking  forward, we used TSA’s projections of airport modification costs for each  fiscal year, 2012 through 2030, as presented in its latest LCCE, and  estimated the amount of expenditures for airport modifications that could  be shifted from TSA to airports if the federal government’s cost share  were reduced from the current 90 percent to 75 percent. As discussed  later in this report, we found that TSA’s LCCE data are of questionable  reliability for a precise estimate. However, the data can serve to provide a  rough indication of how much TSA could save if the cost share were  adjusted. Thus, we estimate that if the federal cost share for such projects  returned to the 75 percent TSA applied to many of the reimbursable  agreements it entered into prior to fiscal year 2008, rather than the  current federal share of 90 percent, TSA’s anticipated expenditures for  these modifications would be roughly $300 million less through fiscal year  2030.
TSA had previously determined that assigning costs among industry  stakeholders and the nation as a whole is difficult because operational  improvements to the baggage handling systems and national security  benefits are difficult to quantify. This, in turn, makes it difficult to develop a  cost share formula that would allow TSA to allocate costs in proportion to  benefits. Consistent with the Intelligence Reform and Terrorism  Prevention Act of 2004, TSA commissioned the 2006 Baggage Screening  Investment Study Working Group to prepare a report for the Aviation  Security Advisory Committee, which examined what an appropriate  federal government/airport cost share should be for the installation of  checked baggage screening equipment. The working group, which  consisted of over 60 members representing, among others, TSA, FAA,  airports, airlines, designers of baggage handling systems, and financial  institutions, were unable to develop a consensus on an appropriate cost  share formula, in large part because of the difficulties of measuring  benefits, differing views on the federal responsibility for funding capital  investments related to baggage screening, and the competing demands  on the federal budget. As a result, potential cost share options were not  submitted to the Congress as part of DHS’s fiscal year 2006 budget  submission.
section_title: Airport Officials Acknowledge Benefits from In-line Systems but Oppose Increasing Their Cost Share
paragraphs: Representatives of all 10 airports we visited told us that they benefit from  the installation of integrated, in-line baggage screening systems.  Specifically, officials from 9 of the 10 airports cited the reduction of passenger  congestion in airport terminals because stand-alone EDS machines  were removed from the lobby or ticketing areas, officials from half of the airports noted that in-line systems reduce the  number of lost or stolen bags by creating a streamlined process for  moving checked baggage directly from where baggage is checked by  the passenger and airline to the aircraft, and officials from 3 of 10 airports noted that in-line systems facilitate  airport growth.
However, for various reasons, officials representing 8 of the 10 airports  opposed a reduction in the federal cost share that would increase  airports’ share of modifications costs.the airports stated the following four concerns:   Specifically, officials from half of
Assuming a larger share of airport modification costs would pose  hardships because of current fiscal or funding constraints.
Airports incur additional (that is, nonallowable) costs that are  necessary to building an in-line system, but which TSA will not  reimburse. Examples of the nonallowable costs the airports cited  include the costs of designing an in-line system and constructing  rooms in which screeners manually screen bags that have not  previously been cleared. As a result, officials from 5 of the 10  airports we spoke with told us that after necessary, but nonallowable,  costs were included, the airports were already paying for more than  10 percent of the modification costs associated with in-line systems.
Airports have a backlog of capital projects or would rather fund  projects that will produce additional revenue, such as parking garages  or larger areas for concessions, than projects that are related to TSA’s  security responsibilities.
TSA will be the primary beneficiary of in-line baggage screening  systems because the integration and consolidation of these systems  will enable TSA to reduce the number of baggage screeners and  provide TSA with other operational efficiencies.
section_title: TSA’s Cost Estimation Processes Do Not Fully Comply with Best Practices, and TSA and DHS Are Working Together to Establish an Acquisition Program Baseline
subsections: section_title: TSA Estimates That the EBSP Will Cost Close to $50 Billion through 2030
paragraphs: TSA’s August 25, 2011, life cycle cost estimate identified a total program  cost for EBSP of $49.2 billion through fiscal year 2030. The $49.2 billion  includes $2.65 billion for EBSP program operation and management;  $11.03 billion for capital costs, including, among other things,  recapitalization and facility modifications for optimization; $14.89 billion  for operations and maintenance of equipment; $18.42 billion for screener  salaries; and $2.22 billion for research and development and other  miscellaneous related costs. Additionally, TSA officials reported that the  program is expected to provide life cycle cost savings of $537 million.  However, we found that the cost estimates are of questionable reliability  for a precise estimate. TSA continues to revise its life cycle cost  estimates. For example, its August 2011 EBSP life cycle cost estimate  report stated that new requirements, including recapitalization and  upgrading the efficiency of early in-line systems, will likely lead to a gap  between anticipated program needs and anticipated funding during fiscal  years 2012 to 2017, totaling up to $436 million. However, in December  2011 TSA officials told us the DHS Acquisition Review Board had  requested that TSA revise the EBSP funding plans and projections to  more accurately reflect current budget constraints and reduced funding  available for the program. According to TSA officials, they plan to  complete the revised EBSP planning estimates and funding projections to  help eliminate the potential funding gap before the next Acquisition  Review Board meeting in May 2012. EBSP senior program officials  explained that TSA will address the potential funding gap by   (1) controlling the costs associated with engineering initiatives and  improvements in technology performance, (2) delaying funding of some  new in-line systems and recapitalization projects, and (3) extending the  useful life of equipment beyond 10 years in cases where replacement  could be delayed.
section_title: TSA’s Life Cycle Cost Estimates Are Not Fully Consistent with Best Practices for Reliable and Credible Estimates
paragraphs: Although TSA’s methods for developing its LCCE reflect features of best  practices, its methods do not fully adhere to these practices. As  highlighted in our past work, a high-quality, reliable cost estimation  process provides a sound basis for making accurate and well-informed  decisions about resource investments, budgets, assessments of  progress, and accountability for results and thus is critical to the success  of a program. According to the Office of Management and Budget  (OMB), federal agencies must maintain current and well-documented  estimates of program costs, and these estimates must encompass the  program’s full life cycle.increased risk of making poorly informed investment decisions, securing   Without such an estimate, agencies are at  insufficient resources to effectively execute defined program plans and  schedules, and experiencing program cost and schedule overruns and  performance shortfalls. As highlighted in our Cost Estimating and  Assessment Guide, a reliable cost estimate has four characteristics—it is  comprehensive, well documented, accurate, and credible. We reviewed  TSA’s cost estimation procedures for the EBSP and assessed the extent  to which the agency met the four characteristics, as shown in table 2.
Our assessment showed that TSA’s EBSP estimates partially met three  characteristics and minimally met one characteristic of a reliable cost  estimate. Specifically, TSA’s cost estimate was as follows:
Partially comprehensive because the estimate defines the program,  reflects the current schedule, is technically reasonable, includes  assumptions identified by a team of personnel and engineers, and  provides risks related to detection standards. However, the cost  estimate is not considered fully comprehensive because it does not  incorporate costs associated with all security threats, lacks a detailed  product-oriented work breakdown structure that covers the entire  scope of work, and lacks a single technical baseline. Without fully  accounting for life cycle costs, management may have difficulty  successfully planning program resource requirements and making  wise decisions. Further, the program lacks a defined end date. A  reasonable criterion is that the estimate capture at least 10 years of  costs beyond the planned full operational capability date—the date at  which optimal systems are fully deployed and operating at all  locations. However, we cannot determine whether the time frame is  sufficient because we have not received documentation to support the  program’s official, planned full operational capability date. According  to TSA, the EBSP does not have a defined end date for procurement  because maintaining compliance with the 100 percent screening  mandate established by ATSA requires TSA to continuously procure  and replace equipment as it reaches the end of its useful life. TSA  also believes that it is following DHS acquisition guidance outlined  under the acquisition decision memorandum dated January 13, 2005,  for estimating threshold dates. Nevertheless, the EBSP still lacks a  defined, official full operational capability date, without which we can  neither determine whether the time frame used in the LCCE is  sufficient nor verify that the life cycle cost estimate is fully  comprehensive.
Partially documented because TSA used relevant data to help  develop the estimate. For example, TSA’s estimated price for the  equipment is based on existing contracts for EBSP equipment  purchases, maintenance costs, LOI agreements, and OTAs. TSA also  provided narratives, briefings, and documents to describe the program  requirements, purpose, technical characteristics, and acquisition  strategy, and explained how calculations were performed. However,  TSA did not adequately document many assumptions or  methodologies underlying its cost model to the extent that would allow  someone unfamiliar with the cost estimate, using only the available  documentation, to easily re-create the estimate. For example,  equipment purchase and hardware upgrade costs were based in part  on estimates from engineers and contract specialists rather than  historical or analogous data. Unless ground rules and assumptions  are clearly documented, the cost estimate will not have a basis for  areas of potential risk to be resolved. In addition, TSA also provided  little or no evidence that the assumptions and methodologies  underlying the cost estimate were approved by management.officials agreed that additional documentation could improve the  outside reviewers’ ability to evaluate the estimate. According to TSA,  the EBSP plans to follow DHS guidance to implement software  dedicated to the estimation, documentation, and reporting of costs  during all phases of the EBSP program life cycle to help address  documentation concerns.
Partially accurate because while the estimate is properly adjusted for  inflation, differences between planned and actual costs are not fully  documented, explained, or reviewed. In addition, we cannot determine  whether the cost estimate is unbiased—that is, neither overly  conservative nor overly optimistic—because the program did not  perform an uncertainty analysis that meets best practices. While TSA  agreed that costs should be documented, it could not explain why the  differences between planned and actual costs were not being fully  documented, explained, or reviewed.
Minimally credible because while TSA identified changes in cost for  each scenario against the baseline and developed a limited risk  analysis, TSA did not complete other relevant activities to ensure that  the estimate accounts for bias and uncertainty. For example, the  agency did not (1) document in detail the assumptions and  parameters associated with its sensitivity analysis, such as detailed  calculations on how each parameter was varied between its minimum  and maximum values; (2) conduct a fully objective uncertainty  analysis that derives the point estimate percentile rather than  assumes it; (3) cross-check major cost elements to see whether  results are similar; and (4) use an independent cost estimate to  validate the cost estimate. Given the important role that an  independent cost estimate provides in developing an objective and  unbiased assessment of whether the program estimate can be  achieved, developing and using an independent cost estimate would  provide decision makers with insight into a program’s potential costs  and reduce the risk of underfunding a program. TSA officials told us  they did not perform a complete uncertainty analysis because it was  too costly and time consuming. TSA officials concurred that an  independent cost estimate was not done for the EBSP and agreed  that completing an independent estimate would be helpful.
Regarding the credibility of TSA’s estimates, our past work has shown  that program cost estimates that are independently validated help  improve the confidence that the estimate is credible, are needed for  making timely and informed budget decisions, and help reduce the  likelihood of unanticipated program cost growth. Our prior reviews of  several DHS programs, including the EBSP, have also shown that if cost  estimates are not validated in accordance with DHS acquisition  management directive requirements at the start of an acquisition  program, it is difficult to assess whether a program is being deployed  within planned budgets. Further, DHS’s Acquisition Management  Directive requires major program cost estimates to be validated early in  the decision-making process, before programs can receive authorization  for acquisition contracts at the DHS Acquisition Review Board meeting.  However, we found that since 2008 the DHS Acquisition Review Board  allowed the EBSP to proceed with acquisition contracts before the LCCE  was independently validated by DHS, which is inconsistent with DHS  policy.
In May 2010, the DHS Cost Analysis Division reviewed the EBSP life  cycle cost estimate and found that it needed more comprehensive data  and that its accuracy could not be determined. As of December 2011, the  estimate had not been independently validated by DHS. According to  TSA, DHS guidance that required validation of cost estimates was not in  place until November 2008. The interim DHS directive for LCCEs and  validation was established in November 2008, and the Cost Analysis  Division was designated the authority responsible for independent cost  estimates on January 24, 2010. However, the DHS Acquisition Review  Board did not request a validated LCCE until February 25, 2011. TSA  officials also stated that since the program’s budget circumstances  changed over time, the cost estimate needed to be revised to reflect a  constrained budget for fiscal year 2013 and other program changes. For  example, TSA officials stated that new requirements for the EBSP,  particularly the shift in focus to recapitalization of the aging EDS and ETD  fleet and upgrades to in-line baggage screening systems and threat  detection levels, contributed to continued program cost growth and  delayed efforts to validate the latest LCCEs. As a result, the federal  government’s portion of the cost estimates for the EBSP has increased  from approximately $20.5 billion in fiscal year 2010 to $25.4 billion in  fiscal year 2011—a 24 percent increase.DHS on the validation of the cost estimate for the next Acquisition Review  Board meeting scheduled for May 2012.
TSA is currently working with  According to our 2009 Cost Estimating and Assessment Guide, endorsed  by the OMB and DHS, cost estimates are integral to determining and  communicating a realistic view of likely cost outcomes that can be used to  plan the work necessary to develop, produce, and support a program.  Taking steps to ensure that its cost estimates for the EBSP conform to  cost estimating best practices will help provide TSA with a sound basis for  understanding how the program can be sustained in future years.
section_title: TSA and DHS Are Working to Develop an EBSP Acquisition Program Baseline
paragraphs: Another foundation for making informed budget decisions is the  acquisition program baseline, which is to document a program’s critical  cost elements, including acquisition costs and life cycle costs. According  to DHS’s acquisition guidance, the program baseline is the contract  between the program and departmental oversight officials and must be  established at program start to document the program’s expected cost,  deployment schedule, and technical performance. Establishing such a  baseline at program start is important for defining the program’s scope,  assessing whether all life cycle costs are properly calculated, and  measuring how well the program is meeting its goals. As we have  previously reported, establishing realistic original baseline estimates is  important for minimizing the risks of poorly defined requirements and  achieving better program outcomes. By tracking and measuring actual  program performance against this formal baseline, management can be  alerted to potential problems, such as cost growth or changing  requirements, and has the ability to take early corrective action.
However, since the inception of the program more than eight years ago,  the EBSP has not had a DHS approved acquisition program baseline and  DHS did not require TSA to complete an acquisition program baseline  until November 2008. An approved baseline will provide DHS with  additional assurances that TSA’s approach is appropriate and that the  capabilities being pursued are worth the expected costs. DHS officials  told us that several reorganizations of DHS offices responsible for  approving the baseline and a lack of functional expertise within the  agency contributed to further delays in approving the EBSP acquisition  program baseline. According to TSA officials, they have twice submitted  an acquisition program baseline to DHS for approval. In November 2009  and February 2011, TSA requested approval of a program baseline, but  according to DHS officials TSA did not have a fully developed life cycle  cost estimate. In November 2011, DHS told TSA that it needed to revise  the life cycle cost estimates as well as its procurement and deployment  schedules to reflect budget constraints. DHS officials told us that they  could not approve the acquisition program baseline as written because  TSA’s estimates were significantly over budget. TSA officials stated that  TSA is currently working with DHS to amend the draft program baseline  for approval. TSA officials stated that they plan to resubmit the revised  acquisition program baseline before the next Acquisition Review Board  meeting in May 2012. Establishing and approving a program baseline, as  DHS and TSA currently plan to do for the EBSP, could help DHS assess  the program’s progress in meeting its goals and achieve better program  outcomes.
section_title: Conclusions
paragraphs: TSA’s EBSP is aimed at increasing airport screening efficiencies and  addressing the continuing threat of explosives concealed in checked  baggage, at a total estimated cost to the federal government and the  private sector of close to $50 billion through fiscal year 2030. Given the  size of the federal investment, it is vital that TSA ensures effective  stewardship over these resources and conveys useful information to the  Congress about the scope and cost of the program. However, the  limitations we identified in TSA’s EBSP cost estimates raise questions  about their reliability. Taking steps to ensure that its cost estimates meet  the four characteristics for high-quality and reliable cost estimates would  provide TSA with increased assurance about the reliability of the  estimated total cost of the program and better position it to account for all  resources and associated costs required to develop, implement, and  sustain the EBSP.
section_title: Recommendation for Executive Action
paragraphs: In order to strengthen the credibility, comprehensiveness, and reliability of  TSA’s cost estimates and related savings estimates for the EBSP, we  recommend that the Administrator of TSA ensure that its life cycle cost  estimates conform to cost estimating best practices.
section_title: Agency Comments and Our Evaluation
paragraphs: On March 30, 2012, we provided a draft of this report to DHS for its  review and comment. DHS provided written comments on April 19, 2012,  which are reprinted in appendix IV. In its written comments, DHS  concurred with our recommendation that TSA ensure that its life cycle  cost estimates conform to cost estimating best practices and discussed  efforts under way to address it. DHS further acknowledged the  importance of producing life cycle cost estimates that are comprehensive,  well documented, accurate, and credible so that they can be used to  support DHS funding and budget decisions. DHS also noted that after  conducting an internal review, TSA is implementing a management  directive that applies DHS guidance and the best practices from the GAO  Cost Estimating and Assessment Guide. As part of this effort, TSA is   (1) establishing a working group and executive board to review program  cost estimates to validate whether the estimates are credible and  affordable, (2) requiring all life cycle cost estimates to be approved by  DHS to ensure consistency and quality across TSA programs,   (3) purchasing and training its employees on specialized cost estimating  software, and (4) initiating hiring actions to hire additional cost estimating  personnel. TSA believes that this will institutionalize cost estimating best  practices within the organization and ultimately allow TSA and the  Department to make better-informed investment decisions. These are  positive steps; however, additional time will be needed to assess whether  they have been fully and consistently implemented in accordance with  GAO best practices.
We are sending copies of this report to the Secretary of Homeland  Security, the Assistant Secretary of the Transportation Security  Administration, and appropriate congressional committees. In addition,  this report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-8777 or lords@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page   of this report. GAO staff who made major contributions to this report are  listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: We examined the Department of Homeland Security’s (DHS)  Transportation Security Administration’s (TSA) operation of the Electronic  Baggage Screening Program (EBSP) to assess the program’s current  status, alternative cost sharing options, and cost estimates. Specifically,  we addressed the following questions:
What is the status of TSA’s efforts to install optimal checked baggage  screening systems in collaboration with airports?
How would reducing the current federal cost share for eligible airport  modification projects from 90 percent to its previous level of 75  percent affect the amount that TSA pays for these modifications, and  what benefits, if any, do airports report receiving from in-line baggage  screening systems?
To what extent are TSA’s cost estimation procedures consistent with  best practices and is TSA’s acquisition baseline consistent with DHS  guidance?
To determine the status of TSA’s efforts to install optimal checked  baggage screening systems, we obtained data as of December 2011 and  January 2012 from TSA, such as the current number of airports with at  least one in-line system, and the number of airports with optimal systems.  We also obtained data on the number of airports configured exclusively  with in-line screening, the number of airports configured with a mix in-line  and stand-alone explosives detection systems (EDS), and the number of  airports using only stand-alone EDS for the same time period. We also  collected data on the overall number of operational in-line systems and  EDS and explosives trace detection (ETD) machines as of December  2011. We reviewed documentation from TSA’s EBSP, including the EBSP  strategic plans for fiscal years 2006, 2008, and 2009, and the 2011 EDS  and ETD Recapitalization and Optimization Plan.
We assessed the reliability of the various data TSA provided about  airports, including the number of TSA-regulated airports (by category) and  the numbers of airports with the different configurations of baggage  screening systems (in-line or stand-alone), and the investment and  budget expenditure dollar values in the letters of intent and other  transaction agreements by questioning cognizant TSA officials and  obtaining extensive documentation about these various data. We found  these data to be sufficiently reliable for the purposes of our report.
To determine how reducing the federal cost share from the current 90  percent to the previous federal cost share of 75 percent for eligible airport  modification projects may affect the amount that TSA pays for these  modifications, we calculated estimates based on TSA’s August 2011  projections of how much airport modifications will cost in the future. These  projections represent TSA’s best estimate for how much it will spend on  airport modifications for in-line systems each year from fiscal years 2012  through 2030. We also reviewed the reliability of the cost estimate by  evaluating how well TSA followed best practices detailed in the GAO Cost  Estimating and Assessment Guide (see below). While TSA’s process for  estimating costs only partially meets the characteristics of a reliable cost  estimate, the data can serve to provide a rough indication of how much  could be saved by reducing the federal cost share for optimization. To  assess how the installation of in-line baggage handling systems may  benefit the airports that receive them, we visited a nonrandom sample of  10 airports. We chose these airports based on the size of airport, type of  checked baggage screening systems installed, and status of airport  facility modification completion. We discussed cost share with officials at  each airport representing the airport authority, tenant airlines, and TSA’s  Federal Security Director. In addition, we interviewed officials from the  largest industry associations that represent airport executives, airports  and airlines (the American Association of Airport Executives, the Airports  Council International North America, and the Air Transport Association).  We also interviewed an official from of the Association for Airline  Passenger Rights. In addition, we discussed potential benefits of  optimization with aviation security experts. These results cannot be  generalized to the entire industry, but did provide broader perspectives on  the issues and costs associated with the EBSP.
To assess the extent to which TSA’s methods for estimating costs for  EBSP are consistent with best practices and its acquisition program  baseline is consistent with DHS guidance, we analyzed TSA’s most  recent life cycle cost estimate and recapitalization report finalized in  August 2011. Specifically, we used best practices in the GAO Cost  Estimating and Assessment Guide to evaluate TSA’s estimating  methodologies, assumptions, and results to assess whether the official  cost estimates were comprehensive (i.e., includes all costs), accurate,  well documented, and credible. Our Cost Estimating and Assessment  Guide considers an estimate to be  comprehensive if its level of detail ensures that all pertinent costs are  included and no costs are double counted; accurate if it is not overly conservative, is based on an assessment of  the most likely costs, and is adjusted properly for inflation; well documented if the estimate can be easily repeated or updated  and can be traced to original sources through auditing; and  credible if the estimate has been cross-checked with an independent  cost estimate and a level of uncertainty associated with the estimate  has been identified.
We also interviewed the TSA EBSP office’s cost estimating team and its  consultants to obtain a detailed understanding of their methodology, the  cost model, and data. In doing so, we interviewed cognizant program  officials, including the Program Manager and cost analysis team,  regarding their respective roles, responsibilities, and actions in developing  the cost estimate, reviewing it, or both.
We examined data reliability of the cost estimate by doing the following:
Obtaining cost estimates and reviewing how each major element was  calculated with an emphasis on the basis for the estimate and  strength and quality of the supporting documentation.
Verifying that the parameters used to create each estimate were valid  and applicable by comparing to available cost estimating references,  posing questions to the cost estimators for clarification, and relying on  other technical sources for cross-checking.
Verifying that calculations were correct for each major element.
Verifying that escalation was properly applied and elements rolled up  accurately to the overall program cost estimate.
We reviewed TSA’s EBSP cost estimates to determine whether the  characteristic was (1) not met if the agency provided no evidence that  satisfied any portion of the criterion, (2) minimally met if the agency  provided evidence that satisfied less than one-half of the criterion,   (3) partially met if the agency provided evidence that satisfied about one- half of the criterion, (4) substantially met if the agency provided evidence  that satisfied more than one-half of the criterion, and (5) met if the agency  provided complete evidence that satisfied the entire criterion. One analyst  assigned a value ranging from 1 to 5 indicating the extent to which the  agencies met each best practice and averaged the values for the  practices that were associated with each characteristic. A second analyst  independently verified the results. We also interviewed program officials  from TSA and DHS responsible for each cost estimate about the  estimate’s derivation. In doing so, we independently assessed the cost  estimates for the current EBSP, as provided to us in August 2011, against  our best practices.
To understand how TSA is working to make better informed budget  decisions and complying with DHS guidance to develop an acquisition  program baseline, we reviewed DHS guidance on acquisitions and  documents related to TSA’s efforts to coordinate with DHS on developing  an acquisition program baseline for EBSP, which DHS considers the  contract between the program and departmental oversight officials to  document the program’s expected cost, deployment schedule, and  technical performance. We also reviewed EBSP Acquisition Review  Board decisions and relevant acquisition decision memos during the  period 2005 through 2011. Additionally, we interviewed TSA and DHS  officials, including officials in the TSA Chief Financial Officer’s office and  DHS’s Program Accountability and Risk Management Office to identify  what procedures have been put in place to approve the acquisition  program baseline.
To gain a better understanding of issues across all of our objectives,  including the development of optimal systems, status of implementation,  funding challenges, impact of a change in the cost share formula at the  airport level, and cost estimation, we conducted site visits to California,  New York, Massachusetts, Washington, D.C., and Florida to interview  local airport officials, regional TSA officials, and airline representatives.  To get a range of airports for our site visits, we made our selections  based on the size of airport, type of checked baggage screening systems  installed, and status of airport facility modification completion. We also  considered recommendations from TSA and industry association officials  about which airports to visit. Because we selected a nonprobability  sample of airports, the information we obtained from these interviews and  visits cannot be generalized to all airports. However, we believe that  observations obtained from these visits provided us with a greater  understanding of the airport officials’ perspectives. On these site visits,  we interviewed airport, airline, and TSA officials responsible for financing,  operating, and installing the checked baggage systems within their  respective airports.
We conducted this performance audit from October 2010 through April  2012 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit  to obtain sufficient, appropriate evidence to provide a reasonable basis  for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
section_title: Appendix II: Status of Implementation Efforts for TSA’s EBSP
paragraphs: TSA, through its EBSP, has deployed EDS and ETD machines in a  variety of in-line and stand-alone configurations at airports to streamline  airports and TSA operations, reduce screening costs, and enhance  security. The following three tables provide information on the status of  checked baggage screening systems. Table 3 highlights the different  system configurations for airports that have optimal checked baggage  systems by airport category. Table 4 shows the number of TSA-regulated  airports with at least one in-line system. Table 5 provides the numbers of  in-line systems and stand-alone EDS and ETD machines at TSA- regulated airports.
section_title: Appendix III: Our Evaluation of EBSP’s Cost Estimate Process
paragraphs: In determining that TSA’s processes for developing EBSP cost estimates  do not fully comply with best practices, we evaluated TSA’s cost  estimation methods against our 2009 Cost Estimating and Assessment  Guide. (See table 6.) We applied the following scale across the four  categories of best practices:
Not met: TSA provided no evidence that satisfies any portion of the  criterion.
Minimally met: TSA provided evidence that satisfies less than one-half  of the criterion.
Partially met: TSA provided evidence that satisfies about one-half of  the criterion.
Substantially met: TSA provided evidence that satisfies more than  one-half of the criterion.
Met: TSA provided complete evidence that satisfies the entire  criterion.
section_title: Appendix IV: Comments from the Department of Homeland Security
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Glenn Davis, Assistant Director,  and Daniel Rodriguez, Analyst-in-Charge, managed this assignment.  Wendy Dye, Daren Sweeney, and Yee Wong made major contributions to  the planning and all other aspects of the work. David Alexander and  Richard Hung assisted with design, methodology, and data reliability.  Chuck Bausell and Jack Wang assisted with economic analysis. Jason  Lee, Stacey Steele, and Karen Richey assisted with life cycle cost  estimate analysis. Nathan Tranquilli assisted with acquisition and  contracting issues. Linda Miller provided assistance in report preparation.  Thomas Lombardi provided legal support.