Aviation Security: TSA Should Ensure Screening Technologies Continue to Meet Detection Requirements after Deployment
Released: 2019-12-05T00:00:00

Letter

Background

TSA is responsible for implementing and overseeing security operations at roughly 440 commercial airports as part of its mission to protect the nation’s civil aviation system.

Screening Technologies

TSA is responsible for ensuring that all passengers, their carry-on bags, and their checked baggage are screened to detect and deter the smuggling of prohibited items, such as explosives, into the sterile areas of airports and onto aircraft. Agency procedures generally provide that passengers pass through security checkpoints where their person, identification documents, and carry-on bags are screened by transportation security officers (TSO). TSA uses a variety of screening technologies—screening systems, as well as software and hardware for those systems—to carry out its mission. Figure 1 depicts the various screening technologies a passenger may encounter in primary and secondary security screening.

Process for Acquiring and Deploying Screening Technologies

TSA develops detection standards that identify and describe the prohibited items—such as guns, knives, military explosives, and homemade explosives—that each technology is to detect during the screening process. The standards, which are classified, also identify how often the technology should detect prohibited items (referred to as the required probability of detection) and the maximum rate at which the technology incorrectly identifies prohibited items (the probability of false alarm). For explosive materials, the standards also identify what the screening technology is to be able to detect in terms of (1) the minimum amount or weight of the material (the minimum detection mass) and (2) the chemical and physical makeup of the material (density range of the explosive material).

S&T supports TSA in the development of standards by, among other things, analyzing the characteristics (threat mass, or the amount of material that constitutes a threat, and density) of explosive materials. The agency uses the resulting data to develop detection standards that are specific to each screening technology.

After a detection standard is approved, TSA decides whether to operationalize—put into effect—detection standards by acquiring and deploying technologies to update detection capabilities to meet the standard. That is, it decides whether to take steps to develop new technology capable of meeting the standard and put the new technology in place at commercial airports. Technology can mean new software to upgrade existing screening systems as well as entirely new screening systems. TSA does not always or immediately operationalize detection standards, for reasons which are explained later in this report.

To operationalize a detection standard, TSA must acquire technology capable of meeting the standard. TSA officials told us they follow DHS acquisition policies and procedures when acquiring new screening technologies. Officials said they adapt detection standards as detection requirements to guide the acquisition process, meaning the specifications described in the standards are incorporated into the requirements manufacturers must meet when developing new technology. Once manufacturers have developed new technologies that meet detection requirements, the technologies undergo a test and evaluation process, known as the qualification process. The following are key steps in that process: 1. Certification – Certification is a preliminary step in TSA’s qualification process. For TSA to certify that a screening technology meets its detection requirements, S&T’s Transportation Security Laboratory conducts certification testing on a manufacturer’s initial submission of its proposed screening technology to determine whether it meets TSA’s detection requirements (i.e., the rate at which it must accurately detect each category of explosive it is designed to detect, among other things). 2. Integration/Implementation Testing – TSA Systems Integration Facility administers qualification testing to test system performance against additional requirements, such as reliability, availability, and maintainability. TSA also conducts field testing to ensure readiness for operational test and evaluation. 3. Operational Test and Evaluation - TSA deploys units to selected airports to conduct operational testing. Operational testing allows TSA to evaluate the operational effectiveness, suitability, and cyber resiliency of the technology in a realistic environment.

After new technologies have been tested and approved, TSA can purchase and deploy them to commercial airports. When a deployed screening system can no longer be updated to meet new detection standards, TSA considers it obsolete and generally designates it for replacement with a newer version of the technology.

Figure 2 shows TSA’s process for acquiring and deploying new screening technologies to meet detection standards.

DHS Risk Management

DHS guidance provides that its components, including TSA, use risk information about security threats and analysis to inform decision-making. Risk management helps decision makers identify and evaluate potential risks so that actions can be taken to mitigate them. DHS defines a risk assessment as a function of threat, vulnerability, and consequence. DHS guidance also says that risk assessments and transparency are key elements of effective homeland security risk management.

TSA Has a Process for Developing Detection Standards, but Has Not Updated Its Guidance or Documented Key Decisions

TSA Has Consistently Followed Testing Protocols in Developing Detection Standards

TSA has a process to develop new explosives detection standard in response to emerging, credible threats involving a homemade explosive (see sidebar for more information on homemade explosives). According to TSA officials, the first step in the process is to determine whether a new detection standard is needed, which they do by working with S&T and other federal partners to ”characterize” the threat material—that is, identify the chemical and physical properties of the material, such as the threat mass and density. Below is the process (steps) TSA and S&T officials told us they use to characterize a threat material and determine whether a new detection standard is needed.

Homemade Explosives Homemade explosives are designed to cause destruction when used in improvised explosive devices. The picture below shows damage to an aircraft panel from a homemade explosive. Beginning in the early 2000s, homemade explosives replaced military and conventional explosives as the preferred tool of terrorists, and challenged the capabilities of existing screening technologies. Unlike conventional threats, homemade explosives are often made of common commercial items and it can be challenging to distinguish them from innocuous gels and liquids stored in personal baggage or cargo. They also have different detonation patterns from conventional explosives in that they often release energy much more slowly, which may lead to incomplete or delayed detonation. This pattern is not well understood, which makes it much more difficult to predict the resulting damage. of the explosive—the minimum amount of the material that constitutes a threat to civil aviation.

Material down selection (selection of possible mixtures for testing). Because the exact formulation of the explosive can vary, S&T must test and model various formulations in different proportions to gain an understanding of the homemade explosive. In this step, TSA determines the representative formulations and preparations that are to be prepared and tested, based on data provided by S&T.

Synthesis, formulation, and preparation of materials. S&T establishes how the threat material could be made, including its chemical synthesis (as applicable), possible formulations or mixtures of the material with other components, and the preparation of those mixtures. S&T uses this information to develop samples of the threat material for testing.

Data acquisition and analysis. S&T examines the samples using micro- computed tomography and explosives detection system, and the resulting data are sent to S&T’s Transportation Security Laboratory for verification. The verified data are then sent to the U.S. Department of Energy’s Lawrence Livermore National Laboratory for analysis.

The Transportation Security Administration and the Science and Technology Directorate have ranked 300 conventional and homemade explosives that pose the most likely threat to aviation security based on factors such as availability, stability, performance, and method of initiation.

Region of responsibility. Lawrence Livermore National Laboratory generates preliminary results in the form of the “region of responsibility,” which is a map or explosive detection “window” outlining the characteristics of the threat material in terms of density and effective atomic number. These preliminary results are discussed among TSA and S&T stakeholders, with TSA determining the final region of responsibility. The region of responsibility data are used to develop software algorithms that will allow screening technologies to recognize explosive materials whose characteristics fall within the region of responsibility.

Detection standard. TSA and S&T also use the region of responsibility data to determine whether the explosive material can already be detected by deployed screening technologies. If screening technologies can already detect the material, TSA will not contract with technology manufacturers to develop a new software algorithm or screening technology. But regardless of whether a new software algorithm or new technology is needed, TSA will draft a new detection standard for the material that, generally, will specify the minimum threat mass and density range to be detected, the acceptable probability of detection, and probability of false alarm. The draft standard is reviewed by TSA senior management before being approved.

We found that the work S&T and other stakeholders performed to characterize explosive threat materials was consistent across the threat materials. Specifically, we found that S&T consistently followed the process described to us (as outlined above) for characterizing a threat material in the seven material threat assessments we reviewed. We also reviewed documentation regarding additional testing and analysis S&T performed on select threat materials, and found the additional testing and analyses were performed consistently.

TSA Has Not Updated Its Guidance for Developing Detection Standards to Reflect Required Procedures, Key Stakeholder Roles, and New Organizational Structure

TSA has not updated its 2015 guidance for developing new detection standards to reflect key changes in their procedures. In December 2015, TSA issued the Detection Requirements Update Standard Operating Procedure, which a senior official told us served as the agency’s approved guidance for developing detection standards. Our review of the document found that, as of August 2019, it did not accurately reflect (1) designated procedures for developing detection standards, (2) the roles and responsibilities of key stakeholders such as S&T, and (3) TSA’s organizational structure. For example, one way in which the 2015 guidance has not been updated is in the designated procedures it describes for reviewing available intelligence information. Specifically, the guidance calls for an annual assessment of emerging threats, which a senior TSA official told us TSA no longer conducts because relevant emerging threats are now occurring more frequently and intelligence information is processed on an ongoing basis.

In another example, the guidance specifies that TSA will form working groups composed of agency officials and stakeholders to assess potential threat materials and develop an analysis plan, and that each working group will define the roles and responsibilities of its members. According to a senior TSA official, the agency does not convene working groups to assess intelligence or develop an analysis plan, although officials regularly meet with stakeholders to discuss the steps needed to characterize new threat materials and document the minutes from these meetings.

Finally, while the guidance discusses in detail which TSA offices and management positions are responsible for implementing and overseeing the process, the agency has since reorganized and these offices and positions no longer exist. Therefore, the 2015 guidance is no longer relevant in terms of which offices and positions are responsible for implementing and overseeing the approval of detection standards.

Officials told us that, as of August 2019, they had begun revising the guidance to reflect existing standard operating procedures for developing detection standards, but had yet to finalize a draft of the new guidance or document plans or timeframes for completing and approving it. Further, it is not clear to what extent the revised guidance will address designated procedures for developing detection standards, the key roles and responsibilities of stakeholders, and TSA’s new organizational structure. Officials said they had not updated the guidance earlier because both TSA and S&T had been undergoing agency reorganizations.

Standards for Internal Control in the Federal Government provides that agencies should identify, on a timely basis, significant changes to internal conditions that have already occurred, such as changes in programs or activities, oversight structure, and organizational structure. Additionally, agencies are to develop and maintain documentation of internal controls, such as policies and procedures necessary to achieve objectives and address related risks. By documenting the processes and procedures TSA uses to develop detection standards, clarifying the roles and responsibilities of stakeholders, and documenting organizational changes, TSA could have better assurance that detection standards are developed in accordance with established policies and practices.

TSA and S&T Did Not Document All Key Decisions Regarding the Development of Detection Standards

Our review of TSA’s steps to develop detection standards from fiscal years 2014 through 2018 found that TSA and S&T did not document all key decisions—those that could potentially affect outcomes—regarding the testing and analyses (characterization) of explosive threat materials and the development of explosives detection standards. We found that TSA and S&T produced a series of detailed material threat assessments to document the characterization of threat materials and consistently developed action memos to justify proposed detection standards. However, we also found that in five of the seven material threat assessments we reviewed TSA and S&T did not consistently document key steps in the testing and analyses of materials, such as how selected samples were prepared for testing. For example, one S&T material threat assessment we reviewed did not document the method used to synthesize (chemically produce) material samples used for testing. Not documenting the method could prevent officials from fully understanding the results of the analysis. Specifically, the assessment noted that there are multiple methods of synthesis, and that the chosen method could affect the makeup of the resulting material and therefore the ability of the screening technologies to detect it. Additionally, while two of the seven material threat assessments cited standard operating procedures for sample preparation for all participating laboratories, three did not cite standard operating procedures for at least one laboratory and two stated that sample preparation information had not been provided by one or more of the participating laboratories. Without documentation, TSA might not have all the necessary information to address future issues involving detection of these materials.

We also found four instances in which TSA did not clearly document why select materials were sent for additional testing or did not document key decisions regarding the development and consideration of detection standards. For example, S&T performed additional testing and analysis on select threat materials after the material threat assessment was finalized. However, the documentation of this additional testing left out key elements regarding how and why the additional testing was needed and conducted. The action memo documenting new standards based on the results of the additional testing did not include a justification for why specific threat materials were selected for additional data collection. While a test plan for equivalency testing of one material stated that the additional testing was conducted because data reported in the literature were not considered representative of current threat configurations, similar justification was not included in the action memo justifying changes to the new standard based on the additional testing. Finally, a senior TSA official told us he requested the additional equivalency testing because the values in the previous detection standards appeared to be more conservative than expected and there was no documentation explaining how TSA had arrived at those numbers. According to the official, the previous detection standard was approved before his tenure and the determining officials were no longer with TSA. He also stated that he did not know whether TSA required documentation of testing and analysis when the previous detection standard was being developed.

We found that TSA did not document key decisions regarding the development and consideration of detection standards. For example, officials could not provide documentation of conclusions reached on specific key decisions, such as the consideration and decision not to approve a proposed explosives trace detection standard. A senior TSA official said he did not know why the decision had not been documented because the officials involved were no longer with the agency.

According to Standards for Internal Control in the Federal Government, documentation is required for the effective design, implementation, and operating effectiveness of an agency. Documentation also provides a means to retain organizational knowledge and mitigate the risk of having that knowledge limited to a few personnel, as well as a means to communicate that knowledge as needed to external parties. By documenting key decisions regarding the development of detection standards, including instances in which draft standards are not approved, TSA could better ensure that effective decisions are made and that organizational knowledge is retained regardless of changes in personnel.

TSA Operationalizes Detection Standards by Updating its Screening Technologies, Which Can Take Years to Complete

TSA officials said one way to operationalize detection standards—acquire and deploy technologies to update detection capabilities and meet the detection standard—is to update existing screening systems with new technology, such as software or firmware. When possible, the agency installs software as part of routine maintenance. TSA can also deploy new hardware or replace screening systems to update detection capabilities. According to officials, the agency applies an incremental approach to updating existing screening technologies—it updates technologies when manufacturers are able to develop the requisite capabilities and as resources allow—which can take years to complete.

According to officials, all fully deployed TSA screening technologies had detection capabilities that met detection standards approved from 2006 through 2012. That is, as of August 2019, TSA’s fleet of screening technologies met detection standards that were approved in 2012 or earlier. For example:

Bottled liquid scanner units met a detection standard that was

Advanced technology x-ray units met two detection standards, depending on their manufacturer, that were both approved in 2010; and

Explosives trace detection units met a detection standard that was approved in 2012.

Further, for each screening technology, the agency has approved two to three new detection standards that have not been operationalized, as of August 2019. For example, in addition to the 2006 detection standard for bottled liquid scanner, TSA approved standards for bottled liquid scanner in 2012 and in 2017 that have not been operationalized.

TSA officials said they were working to operationalize some of the detection standards approved since 2012. Officials said they were working with manufacturers to develop new technologies to operationalize some of these standards. In other cases they were in the process of deploying new technologies that meet these standards. For example, as of August 2019, TSA was in the process of updating and replacing explosives detection systems to meet a detection standard that was approved in 2014. Officials said they expected to have the entire fleet updated by September 2023. TSA officials said they were also in the process of updating deployed advanced technology x-ray units for one of its two manufacturers to meet a standard that was approved in 2014. For more information about the detection standards TSA had approved for each technology as of August 2019, and the status of TSA’s progress in operationalizing them, see appendix I.

TSA shares information about the capabilities it needs with manufacturers through requests for proposal, requests for information, and broad agency announcements. The agency places approved technologies on a qualified products list—a list of technologies that have been tested and certified as meeting requirements by TSA and DHS—and the agency can then award a contract to one of the manufacturers to purchase and deploy the technology. Before deploying technologies to airports, TSA conducts testing to ensure consistency in the manufacturing process, system configuration, and functionality following production, and then again after the technology is installed at airports.

Our analysis of the acquisition information TSA provided found it took from 2 to 7 years to fully develop, certify, test, and deploy screening technologies to airports. For example, when operationalizing explosives trace detection standard 5.0, it took one manufacturer 4 years and a second manufacturer 7 years to develop, and for TSA to deploy, the software needed to update the capability of existing explosives trace detection units to meet the new standard.

Figure 3 provides our analysis of TSA’s timeline for operationalizing advanced imaging technology detection standards approved from 2010 through 2016. TSA officials said they approved detection standard 3.3 for advanced imaging technology in October 2010 and began deploying technology that met that standard to airports in August 2011. Officials said they approved a subsequent standard, 4.1, in January 2012, began deploying technology to meet it in October 2014, and completed the deployment in September 2017. Officials said it took 3 years to complete deployment because the demand for advanced imaging technology increased over time as airports experienced an increase in passenger volumes, among other reasons. Since 2012, TSA approved two additional detection standards for advanced imaging technology—4.3 in February 2016 and 4.3.1 in August 2016. TSA officials said they have not operationalized these two standards because the manufacturer has not been able to develop the requisite technology. As such, deployed advanced imaging technology units meet standards approved in 2010 and 2012.

TSA officials stated that they do not always, or immediately, operationalize detection standards after they are approved. They said they make these decisions on a case-by-case basis, depending on many factors. These include whether: (1) manufacturers have the technological ability, (2) a new technology is in development, and (3) screening technologies already have the capability.

Manufacturers do not have the technological ability. TSA officials said manufacturers do not always have the technical ability to meet detection standards. According to officials, it can be challenging for manufacturers to develop the technology necessary to detect new threats as presented in a detection standard, and in some cases impossible without further research and development. For example, TSA officials said that manufacturers have been unable to develop the requisite technology to meet the most recent detection standards (4.3 and 4.3.1) for advanced imaging technology. However, TSA officials said they have expanded their research and development efforts to try to develop the technology. TSA officials told us they plan to continue developing detection standards irrespective of the capabilities of currently deployed technologies so that they can focus on identifying emerging threats. The new detection standards then serve to set expectations for manufacturers about the capability to which they should aspire and justify research and development necessary to realize that capability. To better manage the difference between the capabilities of deployed technologies and the capabilities described in detection standards, TSA officials said they are in the process of developing a new position of Capability Manager, who would be responsible for managing the development of mission-essential capabilities—such as carry-on baggage screening—from start to finish. Officials said they expect this position will help bridge the gap between approved detection standards and the detection capabilities of deployed screening technologies over time, because the managers will have cross- cutting oversight of the process.

A new technology is in development. Officials said that they may not operationalize a detection standard if they expect a new type of screening technology will replace an existing one. For example, officials said that TSA is exploring new alarm resolution technologies—that is, screening technologies that are used to determine whether alarms are false positives. Officials said new alarm resolution technologies may replace the bottled liquid scanner in the future, and therefore they have not pursued operationalizing detection standard 2.3.

Screening technologies already have the capability. According to TSA officials, new detection standards do not always add significant detection capabilities. For example, officials decided not to operationalize bottled liquid scanner detection standard 3.0 when it was approved in 2017 because the deployed units already had most of the capabilities called for in the detection standard; TSA developed the new standard to better align with standards for other technologies.

TSA Deployment Decisions are Generally Based on Logistical Factors, and the Extent to Which TSA Considers Risk Is Unclear Because Decision- Making Lacks Documentation

TSA Assesses Risks and Capability Gaps When Determining Acquisition Needs

Our review of TSA acquisition documents found that TSA considers risk at the beginning of the screening technologies acquisition process.. Specifically, the agency considers risk in two phases—(1) a risk assessment developed from intelligence information and modeling tools, and (2) an annual capability analysis that analyzes and prioritizes capability gaps and determines mitigation options. Figure 4 provides an overview of TSA’s acquisition process for new screening technologies.

Risk assessment. TSA uses intelligence information and modeling tools, such as the Risk and Trade Space Portfolio Analysis, to assess risk to the aviation system. The Risk and Trade Space Portfolio Analysis was developed in 2014 to analyze the security effectiveness of alternate combinations of some aviation security countermeasures. Officials said a recent example of a risk-informed deployment decision influenced by the Risk and Trade Space Portfolio Analysis was TSA’s 2017 deployment of 141 advanced imaging technology units to category III and IV airports. Officials said that around 2014, TSA received intelligence about a potential terrorist threat to airports, as well as the results of covert testing at airports that identified screening vulnerabilities. Officials said a 2014 Risk and Trade Space Portfolio Analysis also identified disparities in screening capabilities at smaller airports. In part because of the vulnerability identified by these three factors, as well as ongoing conversations between TSA senior leadership, the DHS Inspector General, and members of Congress, officials said TSA procured and deployed additional advanced imaging technology units to some category III and IV airports that did not have them.

Capability analysis. TSA uses the Transportation Security Capability Analysis Process, a structured decision-making tool, to identify and prioritize capability gaps and help direct agency resources towards closing critical gaps to an acceptable level. When existing screening capabilities do not fully meet TSA’s mission needs, the associated capability gap presents a security risk. As part of the Transportation Security Capability Analysis Process, TSA produces Capability Analysis Reports that identify and recommend solutions to closing capability gaps. Recommendations have included procedural changes, such as new training for TSOs, and investments in new technology. TSA’s investment in computed tomography technology for checkpoint screening of carry-on baggage is an example of TSA’s implementation of the Transportation Security Capability Analysis Process to validate capability gaps and identify recommended courses of action. Officials said that in some cases the agency may identify a capability gap that cannot be resolved to an acceptable level with commercially available screening technology, in which case it will pursue additional research and development.

TSA’s Approach to How Risk Informs Deployment Decisions Lacks Documentation

TSA officials told us that they operate under the assumption that every airport is a possible entry point into the aviation system for a terrorist, and they do not consider there to be a significant difference in vulnerability among airports when deploying screening technologies. However, officials did not provide analysis or documentation that supported this conclusion. Officials noted the exception to this assumption is a handful of airports that are consistently considered to be the highest risk because of known threats and a high volume of international travelers.

Further, officials said that if they had information about a threat to a specific airport that would be mitigated by deploying a screening technology, they would modify their plans for deployment accordingly.

However, TSA’s process for how it would change its deployment plans to specific airports based on risk lacks transparency. For example, officials said that as part of the acquisition process they have ongoing discussions with stakeholders about their deployment strategies, including security and intelligence officials who would inform them of any relevant risk information. Officials said these discussions are generally informal and not documented—it was unclear how these discussions have incorporated information about risk in the past, and officials could not provide an example of when risk information at specific airports had directly influenced deployment of technologies to airports in the recent past.

In 2018, the agency released its Transportation Security Administration Systems Acquisition Manual, which called for deployment plans to be written documents, and officials said they began documenting their plans for deploying screening technologies in the last two years. TSA officials provided us with one deployment plan—for their 2018 deployment of explosives trace detection units—but we found that it was not transparent about how risk was a factor in officials’ methodology for determining the order of airports to receive the technology. The explosives trace detection plan documented TSA’s schedule of deployment and the roles and responsibilities of relevant stakeholders, among other things. However, while the plan indicated that officials would coordinate with relevant offices within the agency for information about risks that might impact their deployment strategy, we found that the plan did not document how risk had informed their decisions about where and how to deploy the technology, including the assumptions, methodology, and uncertainties considered.

Additionally, TSA officials did not document, and could not fully explain, how risk analyses contributed to and factored into the following specific deployment decisions.

Deployment of advanced imaging technology to smaller airports. Officials said many factors influenced their decision to deploy advanced imaging technology units to category III and IV airports, including information about threats and a related 2014 risk analysis. However, officials did not document their decisions and could not fully explain their risk analysis, including their process for analyzing and weighing relevant factors. According to officials, the decision was made during discussions with senior leadership, which were risk-informed and supported by whiteboard analyses and classified documents. Additionally, officials told us that, for practical reasons, they deployed units to those category III and IV airports that had the space to accommodate them, but did not further assess the priority of deployment among the smaller airports because they had determined that the risk was uniform and because they planned to deploy the units within a short timeframe. Officials did not document the risk assessment that led to this determination, and could not explain how the three elements of risk—threat, vulnerability, and consequence—were used or assessed.

Deployment of targeted threat algorithm. In 2016, TSA deployed a targeted threat algorithm—software to improve detection capabilities—to a limited number of advanced imaging technology units in response to a specific threat. After testing the operational impacts of the software algorithm, the agency decided to stop further deployment. The documentation TSA provided did not explain how officials had analyzed the risk-mitigation benefits of the algorithm, including the underlying assumptions and uncertainty, or how they had weighed those benefits against the operational impacts and costs when they made their decision not to fully deploy the algorithm.

TSA officials said they follow the DHS acquisition process to acquire and deploy technologies and their deployment decisions are based on, and informed by, their initial assessments of capability gaps, as well as their understanding that every airport offers equal entry into the aviation system. However, officials had not documented the rationale for these decisions and could not fully explain how risk had informed their decisions about where and in what order to deploy screening technologies.

DHS’s Risk Management Fundamentals states that components should consistently and comprehensively incorporate risk management into all aspects of the planning and execution of their organizational missions. Additionally, it says transparency is vitally important in homeland security risk management, and documentation should include transparent disclosure of the rationale behind decisions, including the assumptions, methodology, and the uncertainty considered. By fully disclosing what risk factors are weighed and how decisions are made, TSA officials can better ensure that their deployment of screening technologies matches potential risks (threats, vulnerabilities, and consequences). This is of particular importance given the agency’s limited resources and the fact that screening technologies are not easily relocated.

TSA Generally Deploys Screening Technologies Based on Logistical Factors

TSA officials said that absent a specific risk to an airport or category of airports that would be mitigated by deploying a screening technology, they consider a number of logistical factors that are aimed at maximizing the operational efficiency of the screening process. These factors influence the number of units of a technology the agency deploys to airports, the order in which they deploy them, and where they are deployed.

Officials said they use modeling software to determine the most efficient number of units to allocate to an airport for each type of screening system. This analysis takes into account variables such as the number of flights at an airport, airport passenger volumes, items per passenger, and secondary search rates. Additionally, agency officials said the layout of an airport is a significant determining factor for the number of units it receives. For example, an airport that has centralized checked baggage screening areas will need fewer explosives detection systems than an airport that has checked baggage screening areas dispersed in different locations.

Additionally, TSA officials said that logistical and funding factors can influence the order of deployment, including the manufacturer’s ability and resources to develop and deliver technologies. For example, as of June 2019, officials said the agency was in the process of updating the detection capabilities of 62 percent of its advanced technology x-ray fleet because one of its two manufacturers had completed testing and certification of the new technology, but the second manufacturer’s technology had yet to be certified. Officials said they also try to plan their deployment schedule around minimizing disruptions to airport operations, so if an airport could not absorb a full deployment of a technology because it would affect too many passengers, TSA would schedule the deployment in phases to minimize disruptions.

Further, TSA officials said that, as a result of these many logistical considerations, they generally fully deploy new screening technologies to category X airports first—generally, airports with the highest passenger volumes—and then proceed in order down to the airport with the lowest passenger volume. Officials said larger airports generally have the infrastructure in place to incorporate new technology without extensive disruption to operations, and they will screen the most passengers by deploying screening technologies to the largest airports first.

TSA Does Not Ensure That Screening Technologies Continue to Meet Detection Requirements after Deployment to Airports

TSA practices do not ensure that screening technologies continue to meet detection requirements after they have been deployed to airports. According to agency officials, the agency uses certification to confirm that technologies meet detection requirements before they are deployed to airports, and calibration to confirm that technologies are at least minimally operational while in use at airports. Officials stated these processes are sufficient to assure TSA that screening technologies are operating as intended. However, while certification and calibration serve important purposes in the acquisition and operation of screening technologies, they have not ensured that TSA screening technologies continue to meet detection requirements after they have been deployed.

Certification occurs prior to deployment. TSA’s certification process is designed to ensure screening technologies meet detection requirements during the acquisition process, prior to the procurement and deployment of the technologies, but it does not ensure screening technologies continue to meet detection requirements after deployment. As previously described, manufacturers provide an initial submission of the screening technology to TSA for certification testing as part of the acquisition process. During the certification process, S&T’s Transportation Security Laboratory tests the technology under controlled conditions to determine whether it meets TSA’s detection requirements. After TSA certifies that a screening technology meets detection requirements and it undergoes additional testing to determine whether it meets other TSA requirements in controlled testing facilities, TSA may deploy it to select airports for operational testing and evaluation to determine how it performs in an airport setting. Certification testing demonstrates that a manufacturer’s screening technology meets detection requirements during the acquisition process, which allows TSA to determine whether it should continue to consider the technology for acquisition.

Certification does not ensure that deployed technologies continue to meet detection requirements because it does not account for the possibility that performance of technologies can degrade over time throughout the technologies’ lifecycles after deployment. For example, in 2015 and 2016, DHS removed a sample of deployed explosives trace detection and bottled liquid scanner units from airports for testing in the Transportation Security Laboratory. The laboratory concluded that some deployed units for each technology it tested no longer met detection requirements— either the required probability of detection for certain explosives or the required rate for false alarm, or both. One explosives trace detection unit that was tested was found to have a probability of detection much lower than required. According to TSA officials, the units did not meet detection requirements because they were not adequately maintained, which affected their performance. In light of this, officials stated that they introduced better controls to ensure that routine preventative maintenance is performed as required. However, because TSA does not test the units after they are deployed to airports, it cannot determine the extent to which these controls ensure technologies continue to meet detection requirements. Officials noted that TSA uses a layered security approach at airports, so if one layer should fail—such as a deployed technology—the agency can still rely on other security measures among the various layers of security to detect threats. We have previously reported on the importance that TSA ensure each measure is effective to make the best use of its limited resources, in order to serve its aviation security mission.

Calibration does not test whether technologies meet detection requirements. TSA officials stated that daily calibration also helps ensure that screening technologies continue to meet detection requirements after deployment. However, while calibration demonstrates that the screening technology is at least minimally operational, it is not designed to test whether the screening technology meets detection requirements. For example, each explosives detection system is calibrated with an operational test kit that contains items of various densities. To calibrate explosives detection systems, a TSO must run the operational test kit through the unit and verify that the item is correctly displayed on the monitor (see figure 5 below). This process demonstrates whether the system can identify the known items’ densities, but it does not ensure that the system meets detection requirements. As a result, calibration could indicate that the unit is functioning even when its detection capabilities have degraded—that is, calibration determines that the technology is functional, but it does not ensure that the technology is meeting detection requirements.

TSA officials stated that they plan to develop a process to review screening technologies on an annual basis to analyze their performance, including detection over time. TSA officials stated that, as of August 2019, they were actively working on developing a review process for the explosives detection system but did not have a date for when they planned to complete it. TSA officials for the passenger and carry-on screening technologies stated that they had not yet started developing a review process for those technologies and the timeline for developing a review process will depend on funding.

TSA officials also noted that there are challenges in designing a process to ensure that screening technologies continue to meet detection requirements after deployment. For example, TSA and S&T officials stated that it is not feasible to conduct live explosives testing in airports. Further, according to TSA officials, while it is relatively easy to temporarily transfer smaller screening technologies, such as explosives trace detection and bottled liquid scanner units, to a controlled setting for live explosives testing, it would not be feasible to transfer larger installed units, such as advanced imaging technology. Although testing with live explosives in an airport poses undue risks and transferring larger machines for testing may be costly, TSA could develop other measures.

TSA officials stated that there is no requirement to ensure that its screening technologies continue to meet detection requirements after deployment to airports. However, Standards for Internal Control in the Federal Government calls for agencies to establish and operate a system to continuously monitor the quality of performance over time. Without taking additional steps to ensure screening technologies are meeting detection requirements, TSA may run the risk that its deployed screening technologies are not detecting explosives and other prohibited items. Developing and implementing a process to monitor screening technologies’ detection performance over time would help provide TSA assurance that screening technologies continue to meet detection requirements, as appropriate, after deployment. In doing so, TSA would also be better positioned to take any necessary corrective actions if or when screening technologies no longer operate as required.

TSA Spent an Estimated $3.1 Billion to Purchase, Deploy, Install, and Maintain its Fiscal Year 2018 Inventory of Screening Technologies

We estimate that TSA spent $3.1 billion to purchase, deploy, install, and maintain its inventory of screening technologies as of the end of fiscal year 2018, based on agency estimates of costs. Of this $3.1 billion, we estimate that TSA spent 71 percent to purchase screening technologies, 9 percent to deploy, about 12 percent to install, and, for fiscal year 2018, about 9 percent to maintain them for 1 year. The highest estimated total expenditures on a per-technology basis were for explosives detection systems ($2.1 billion, or 68 percent), advanced technology x-ray ($443 million, or 14 percent), explosives trace detection ($227 million, or 7 percent), and advanced imaging technology ($197 million, or 6 percent). Table 1 provides information on estimated expenditures for TSA’s September 2018 inventory of screening technologies, by screening technology and life-cycle phase (i.e., purchase, deploy, install, and maintain). Appendix III provides additional information on estimated TSA expenditures, such as prices per unit of technology and estimated expenditures by airport category.

TSA has also incurred costs, or has plans to incur costs, for additional actions related to screening technologies. Specifically, it has also incurred costs for modifications to commercial airport facilities to accommodate screening technologies. Further, TSA estimates additional life-cycle costs of $804 million to acquire, deploy, and maintain computed tomography systems through fiscal year 2026. The following provides more information on these estimated expenditures.

Airport modifications. TSA incurs costs related to modifying commercial airports to accommodate certain screening technologies, such as checked baggage screening systems (e.g., explosives detection systems). In December 2017, we reported that TSA had obligated at least $783 million from fiscal years 2012 through 2016 to reimburse airports for the allowable design and construction costs associated with installing, updating, or replacing screening technology. For example, TSA may enter into agreements to reimburse airport operators for a percentage of the allowable design and construction costs associated with facility modifications needed for installing, updating, or replacing in-line explosives detection systems. In-line screening systems use conveyor belts to route checked luggage through explosives detection systems, which capture images of the checked baggage to determine if a bag contains threat items not permitted for transport, including explosives. From fiscal years 2012 through 2016, agreements for TSA reimbursements to airports for checked baggage screening systems generally ranged in value from $50,000 to $150 million. As we reported in December 2017, in general, depending on the airport’s size, TSA will reimburse 90 or 95 percent of the allowable, allocable, and reasonable cost of certain projects. For other projects, TSA may provide 100 percent reimbursement—for example, where existing systems require the correction of security or safety deficiencies.

Computed tomography. In addition to its fiscal year 2018 inventory, TSA is currently in the process of deploying computed tomography to commercial airports to replace advanced technology x-ray systems. Computed tomography technology applies sophisticated algorithms to detect explosives and other prohibited items and creates a 3D image of carry-on baggage that a TSO can view and rotate 360 degrees. In fiscal year 2018, TSA determined that computed tomography is the best technology available to address rapidly evolving threats in the transportation sector, and plans to eventually deploy it to all checkpoints and replace advanced technology x-ray technology. As recorded in TSA’s Deployed Locations Report, TSA had deployed 11 computed tomography systems to category X and I airports as of September 24, 2018. According to TSA’s September 2018 life-cycle cost estimates, the agency plans to field 883 units by fiscal year 2026. As shown in table 2, TSA also planned to spend $805 million to purchase, deploy, and maintain this new technology through fiscal year 2026. However, in August 2019, TSA officials told us that they expect this estimated total procurement cost of $805 million to likely decrease as the per unit cost had decreased from $400,000 to $233,000 in the initial fiscal year 2019 contract for computed tomography.

Conclusions

TSA has invested billions of dollars in screening technologies as it responds to terrorists’ attempts to use homemade explosives to disrupt and damage civil aviation. Forecasted increases in passenger volumes and ongoing terrorist threats make it imperative that TSA employ recommended management and internal control practices. TSA could help ensure that critical detection standards are developed in accordance with approved practices, and that agency goals are effectively met by updating its guidance for developing standards. Additionally, by documenting key decisions in the development of detection standards, TSA could better assure the effectiveness of decision-making and the retention of organizational knowledge in the face of inevitable changes in personnel.

Similarly, when making technology deployment decisions, incorporating DHS-recommended practices for risk management would improve TSA’s ability to effectively fulfill its mission to secure the nation’s civil aviation system. While TSA assesses risk when deciding whether to invest in a new technology to address an identified capability gap, it is unclear the extent to which it considers risk when determining where and in what order to deploy approved screening technologies to airports. DHS guidance for homeland security risk management calls for risk to be considered consistently and comprehensively in all aspects of an agency’s work. Additionally, risk management includes transparent disclosure of the rationale behind decision-making so that stakeholders can understand how key factors were weighed. Incorporating these risk management principles into its decision-making for deploying screening technologies to airports would allow TSA to align its deployment strategies with potential threats, vulnerabilities, and consequences.

Lastly, TSA cannot ensure that its screening technologies continue to meet detection requirements after they have been deployed to airports. Developing and implementing a policy to ensure that TSA’s screening technologies continue to meet their respective detection requirements after deployment may assure the agency that its deployed screening technologies are effectively detecting explosives and other prohibited items that they are designed to identify, which is a critical part of TSA’s mission.

Recommendations for Executive Action

We are making the following five recommendations to TSA: The TSA Administrator should update TSA guidance for developing and approving screening technology explosives detection standards to reflect designated procedures, the roles and responsibilities of stakeholders, and changes in the agency’s organizational structure. (Recommendation 1)

The TSA Administrator should require and ensure that TSA officials document key decisions, including testing and analysis decisions, used to support the development and consideration of new screening technology explosives detection standards. (Recommendation 2)

The TSA Administrator should require and ensure that TSA officials document their assessments of risk and the rationale—including the assumptions, methodology, and uncertainty considered—behind decisions to deploy screening technologies. (Recommendation 3)

The TSA Administrator should develop a process to ensure that screening technologies continue to meet detection requirements after deployment to commercial airports. (Recommendation 4)

The TSA Administrator should implement the process it develops to ensure that screening technologies continue to meet detection requirements after deployment to commercial airports. (Recommendation 5)

Agency Comments and Our Evaluation

We provided a draft of this product to DHS for comment. We provided a draft of this report to DHS for review and comment. DHS provided written comments, which are reproduced in full in appendix IV. DHS concurred with our five recommendations and described actions undertaken or planned to address them. TSA also provided technical comments, which we incorporated as appropriate.

With regard to our first recommendation that TSA update guidance for developing and approving screening technology explosives detection standards, DHS concurred and stated that TSA has included updated guidance in its Requirements Engineering Integrated Process Manual, which was completed in September 2019. According to DHS, the update provides TSA’s process for developing and approving explosives detection standards, including designated procedures and roles and responsibilities of stakeholders, and reflects organizational changes to TSA. TSA provided us with the Requirements Engineering Integrated Process Manual in November 2019, concurrent with DHS comments. We will review the update and the extent to which it addresses the recommendation. This action, if fully implemented, should address the intent of the recommendation.

DHS concurred with our second recommendation that TSA ensure that officials document key decisions supporting the development of screening technology explosives detection standards. DHS stated that the updated Requirements Engineering Integrated Process Manual describes the process for documenting key decisions, including testing and analysis decisions, in the development of new detection standards. We will review the update and the extent to which it addresses the recommendation. This action, if fully implemented, should address the intent of the recommendation.

DHS also concurred with our third recommendation that TSA document its assessments of risk and the rationale behind its decisions to deploy screening technologies. According to DHS, TSA has instituted an improved process for documenting elements that contribute to deployment decisions—TSA’s August 2019 deployment plan for computed tomography is an example of the process. DHS stated that TSA will continue to include a comparable level of documentation in future deployment plans for screening technologies. We agree the computed tomography deployment site selection strategy is an example of how TSA can document the rationale governing the deployment of a screening technology. Future plans can further benefit by explaining the risk analysis itself along with the role that risk considerations played in the selection of airports for deployment. Formalizing guidance that directs TSA officials to document risk assessments and the rationale behind deployment decisions would help TSA ensure that its deployment of screening technologies matches potential risks.

DHS concurred with our fourth and fifth recommendations that TSA, respectively, develop and implement a process to ensure that screening technologies continue to meet all detection requirements after deployment to commercial airports. DHS stated that TSA will develop recurring individual post implementation reviews (PIR) for all screening technologies in accordance with DHS Directive 102-01, to assess multiple aspects of system performance, including detection over time. DHS also stated that TSA intends to examine the component performance of the detection chain rather than a direct measure of detection requirements, due to the limitations of using live explosives and simulants. DHS stated that because the detection chain for each technology is unique and will require individual reviews, TSA is developing a policy on the PIR development process, which it estimates will be completed by March 31, 2020. We appreciate the limitations live explosives and simulants present in testing and the need for reviews that are tailored to meet the unique characteristics of each screening technology. TSA plans to implement the review process on the first screening technology by December 31, 2020. These actions, if implemented across all applicable screening technologies, should address the intent of the recommendations.

We are sending copies of this report to the appropriate congressional committees and to the Acting Secretary of Homeland Security. In addition, this report is available at no charge on the GAO website at http://gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-8777 or russellw@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made significant contributions to this report are listed in Appendix V.

Appendix I: Transportation Security Administration (TSA) Screening Technologies

This appendix presents additional details on the TSA screening technologies we reviewed, including their function and the number of units deployed.

Appendix II: Objectives, Scope, and Methodology

This report addresses Transportation Security Administration’s (TSA) processes for developing and deploying screening technologies to airports regulated by TSA (i.e., “commercial” airports). Specifically, we examined 1. the extent to which TSA has a process for developing explosives detection standards for screening technologies in response to identified emerging threats; 2. how TSA operationalizes detection standards to update detection capabilities; 3. the extent to which TSA has considered risk when deploying screening technologies to commercial airports; 4. the extent to which TSA ensures screening technologies meet the requirements for detection standards after deployment; and 5. TSA estimated expenditures to purchase, deploy, install, and maintain its inventory of screening technologies as of the end of fiscal year 2018.

To address all of our objectives, we identified 11 screening technologies TSA used to screen passengers’ identification documents, person, carry- on bags, and checked baggage at commercial airports as of September 24, 2018, as recorded in TSA’s Government Property Management database. The seven screening technologies in use at commercial airport passenger checkpoints were advanced imaging technology, advanced technology x-ray machine, bottled liquid scanner, boarding pass scanner, chemical analysis device, threat image projection x-ray, and walk-through metal detector. The credential authentication technology and computed tomography, also used at checkpoint screening, were deployed and in use at select airports as TSA pilot projects. The two TSA screening technologies in use at commercial airports for checked baggage were explosives detection systems and explosives trace detection (TSA also uses explosives trace detection for checkpoint screening).

We assessed the reliability of TSA’s inventory data by interviewing agency officials and reviewing related documentation, such as the database user manual, among other things. We determined the data were sufficiently reliable to determine the type and number of TSA screening technologies deployed as of September 2018. To better understand how TSA screening technologies have been used, we reviewed reports from the U.S. Department of Homeland Security (DHS) Office of the Inspector General, the Congressional Research Service, past GAO reports, and relevant DHS and TSA documentation, such as DHS and TSA strategic documents and acquisition plans. To observe TSA screening procedures and the operation of screening technologies in the airport setting, we conducted site visits to seven commercial airports. During these visits we discussed screening technology issues with TSA federal security directors or their representatives. We selected these airports to reflect a range of airport categories, technologies, and geographic diversity. The results of these site visits and interviews cannot be generalized to all commercial airports, but they provided us with important context about the installation, use, and maintenance of TSA screening technologies across the different types of airports that TSA secures. We also conducted a site visit to the TSA Systems Integration Facility to better understand how screening technologies are tested and evaluated prior to deployment. Further, we interviewed officials from two industry associations and one screening technology manufacturers association based on input from TSA and DHS Science and Technology Directorate (S&T) officials.

To determine the extent to which TSA has a process for developing explosives detection standards, we examined TSA documents such as approved detection standards, action memos summarizing support for proposed detection standards, the Detection Requirements Update Standard Operating Procedure, and briefing slides describing TSA’s process, as of August 2019, for assessing threat materials and developing detection standards. We also evaluated Material Threat Assessment reports that summarized the testing and analyses performed by S&T’s Homemade Explosives Characterization Program, in coordination with S&T laboratories, to characterize (identify the physical density and mass of) explosive materials for detection standards developed from fiscal years 2014 through 2018. We evaluated S&T’s testing and analyses in accordance with TSA and S&T guidance to determine the extent to which these steps were consistent across materials; we did not analyze the sufficiency of the testing and analyses. We also assessed TSA and S&T processes and the extent to which they were documented in accordance with Standards for Internal Control in the Federal Government, and discussed the details of steps taken to develop standards with relevant TSA and S&T officials. In addition, we conducted a site visit to S&T’s Commercial Aircraft Vulnerability and Mitigation Program testing site at the U.S. Army Aberdeen Test Center, Maryland, to better understand how S&T tests the vulnerability of commercial aircraft to explosive materials.

To understand TSA’s process and timelines for operationalizing—putting into effect—detection standards, we requested information from TSA about screening technologies subject to explosives detection standards, deployed as of September 24, 2018: advanced imaging technology, advanced technology x-ray, bottled liquid scanner, explosives detection systems, and explosives trace detection. We requested information about the detection standards that deployed screening technologies met, as of August 2019, as well as subsequently approved detection standards, including the date the standards were approved, the dates when TSA achieved certain acquisition milestones when developing and deploying the associated technologies, and the status of ongoing and upcoming efforts to update detection capabilities to meet new standards. We identified the acquisition milestones by reviewing a past GAO report on TSA’s acquisition process and in consultation with GAO acquisition experts. We also reviewed a classified TSA report that evaluated the performance of a particular algorithm in order to understand TSA’s process for developing new screening technologies to meet detection standards. In addition, we reviewed relevant acquisition documents, such as DHS’s Acquisition Management Instruction 102, the 2018 Transportation Security Administration Systems Acquisition Manual, acquisition decision memos, acquisition plans, and Operations Requirements Documents. To understand TSA’s process for deciding whether to operationalize detection standards, we requested and reviewed available documentation for the standards that TSA had not operationalized, such as an operational status transition memo for bottled liquid scanner, and interviewed TSA officials about those decisions.

To understand how TSA had considered risk in its approach to deploying screening technologies at airports, we reviewed available documentation related to TSA’s deployment decisions. These included decision memos from acquisition review board meetings and action memos to TSA leadership; risk registers for checked baggage and checkpoint acquisition programs; available deployment plans, such as the agency’s Action Plan for deploying explosives trace detection units to airports in 2018; and acquisition guidance. To understand how TSA assesses capability needs and gaps, we interviewed agency officials about TSA’s Transportation Security Capability Analysis Process and reviewed capability analysis reports from 2018 and 2019, as well as TSA’s prioritized list of capability gaps and needs. We also interviewed acquisition officials, including TSA’s Component Acquisition Executive, about the role of risk in deployment decisions and requested written responses to specific questions. We assessed TSA’s decision-making process for deploying and updating screening technologies, generally, against DHS risk management criteria, such as DHS’s Risk Management Fundamentals. We also reviewed related areas of risk management and decision-making to understand the context in which TSA makes deployment decisions. Specifically, we reviewed the 2017 Transportation Sector Security Risk Assessment and the Cities and Airports Threat Assessment reports to understand the risks facing the nation’s aviation system. We also reviewed TSA’s enterprise risk management framework, such as the Enterprise Risk Management Policy Manual, to understand the role it played in TSA’s deployment decisions. We also interviewed an official from TSA’s Enterprise Performance and Risk office and the Executive Risk Steering Committee. To understand how TSA categorizes airports, we reviewed a 2017 Nationwide Airport Categorization Review memo from TSA’s Security Operations office and interviewed Security Operations officials.

To understand how TSA deploys screening technologies across airports and categories of airports, we analyzed TSA’s Deployed Locations Report, which reported on technologies that were in use or available for use at commercial airports from September 24 through September 30, 2018. We also reviewed TSA’s standardized methodology for determining the most efficient number of screening technologies at an airport. Additionally, we reviewed TSA’s Strategic Five-Year Technology Investment Plan from 2015 and the 2017 Biennial Refresh to understand TSA’s plans for ongoing investment in screening technologies. We reviewed various throughput data, such as annual passenger throughput for all commercial airports for fiscal year 2018 and enplanements data for calendar year 2017, to understand and compare TSA’s allocation of screening technologies with throughput data across airports and airport categories. We used this analysis to identify airports that had an unusually large or small number of screening technologies within a category, and interviewed TSA officials to understand the decisions that led to the allocation of screening technologies across airports and airport categories.

In addition, we reviewed the status of TSA’s limited deployment of computed tomography units to checkpoints. Specifically, we reviewed TSA’s 2018 Deployment Site Selection Strategy, which described the airports to which TSA would deploy computed tomography units and the methodology it used to select them, slides from recent conferences TSA held with industry representatives where it shared its plans for transitioning to computed tomography, and relevant Operational Requirements Documents. We also interviewed agency officials about their plans for the limited deployment and TSA’s transition from advanced technology x-ray to computed tomography for checkpoint screening.

To determine the extent to which TSA ensures its screening technologies continue meeting detection requirements after deploying them to airports, we reviewed TSA acquisition detection requirements for each screening technology as well as TSA guidance related to the testing and evaluation of screening technologies identified by TSA officials in interviews. We also interviewed TSA and S&T Transportation Security Laboratory officials about TSA requirements to test screening technologies, both prior to and after deployment, to determine the extent to which they meet detection requirements. We also observed transportation security officers and a transportation security specialist for explosives conduct verification and calibration procedures on screening technologies at the airports we visited. We reviewed TSA guidance to determine the extent to which its procedures ensure that screening technologies continued to meet detection requirements in airports. We then evaluated the procedures against Standards for Internal Control in the Federal Government for monitoring.

To identify TSA’s estimated expenditures to purchase, deploy, install, and maintain its inventory of screening technologies as of the end of fiscal year 2018, we reviewed TSA programs’ life-cycle cost estimates, which, for the purposes of acquisition planning, provide per unit estimates of the cost to purchase, deploy, install, and maintain passenger and checked baggage screening technologies. We chose this methodology in consultation with TSA officials and after determining that historical records of obligations and expenditures do not provided consistent and sufficient detail for the purposes of our analysis.

The life-cycle cost estimates include relevant phases for each screening technology (i.e., purchase, deploy, install, and maintain), although not all technologies have cost estimates for each phase of the life cycle. For example, some screening technologies may not specify deployment costs because such costs are included in the initial purchase price of the unit. In other cases, the technology does not have a deployment cost because the unit is small and portable, and placement of the unit is therefore handled by TSA airport staff at no charge. Estimated expenditures for installation also include costs associated with site acceptance testing, which is performed when a system is installed at its operational location. Unlike the purchase, deploy, and install unit prices, the maintenance unit price is the yearly cost of maintenance for one unit, and therefore recurs every year.

We assessed the reliability of the life-cycle cost estimates by reviewing documentation on the development of the estimates and interviewing TSA officials, among other things, and determined the estimates were sufficiently reliable for the purpose of estimating the amount of funds spent on acquiring, deploying, installing, and maintaining TSA’s inventory of screening technologies as of the end of fiscal year 2018. Because the life-cycle cost estimates were developed in different years, we used TSA guidelines to adjust costs for inflation and convert our estimates to 2018 dollars. We multiplied these estimates against the number of screening technologies deployed to commercial airports as of September 24, 2018, using data from TSA’s Government Property Management database. For computed tomography, we also obtained information on price and quantity from the technology’s life-cycle cost estimate and TSA officials. We also reviewed prior GAO work on TSA cost sharing programs for airport facility modification related to installation of some of the technologies in our review.

We conducted this performance audit from April 2018 to December 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Transportation Security Administration (TSA) Estimated Expenditures for Screening Technologies

We estimate that TSA spent $3.1 billion to purchase, deploy, install, and maintain its inventory of screening technologies, as of the end of fiscal year 2018, based on agency estimates of costs. Tables 3 through 5 provide information on estimated TSA expenditures by screening technology, life-cycle phase, and airport category. To analyze TSA’s estimated spending to purchase, deploy, install, and maintain its inventory of screening technologies as of the end of fiscal year 2018, we reviewed TSA life-cycle cost estimates, which, for the purposes of acquisition planning, provide per-unit estimates of the cost to purchase, deploy, install, and maintain passenger and checked baggage screening technologies at TSA-regulated airports (i.e., “commercial” airports). Because the life-cycle cost estimates were developed in different years, we used the same guidelines used by TSA to adjust costs for inflation to convert our estimates to 2018 dollars. We multiplied these estimates against the number of screening technologies deployed to commercial airports as of September 24, 2018.

Appendix IV: Comments from the Department of Homeland Security

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Kevin Heinz (Assistant Director), Barbara Guffy (Analyst in Charge), Kelsey Burdick, Jonathan Felbinger, Tyler Kent, Thomas Lombardi, Erin O’Brien, Kya Palomaki, Rebecca Parkhurst, and Dina Shorafa made key contributions to this report. In addition, key support was provided by Chuck Bausell, Richard Cederholm, Dominick Dale, Aryn Ehlow, Michele Fejfar, Eric Hauswirth, Richard Hung, and Alexis Olson.