Homeland Security: DHS and TSA Acquisition and Development of New Technologies
Released: 2011-09-22T00:00:00

Letter

Background

The Aviation and Transportation Security Act (ATSA) established TSA as the federal agency with primary responsibility for securing the nation’s civil aviation system, which includes the screening of all passenger and property transported from and within the United States by commercial passenger aircraft. In accordance with ATSA, all passengers, their accessible property, and their checked baggage are screened pursuant to TSA-established procedures at the 463 airports presently regulated for security by TSA. These procedures generally provide, among other things, that passengers pass through security checkpoints where they and their identification documents, and accessible property, are checked by transportation security officers (TSO), other TSA employees, or by private-sector screeners under TSA’s Screening Partnership Program. Airport operators, however, also have direct responsibility for implementing TSA security requirements such as those relating to perimeter security and access controls, in accordance with their approved security programs and other TSA direction.

TSA relies upon multiple layers of security to deter, detect, and disrupt persons posing a potential risk to aviation security. These layers include behavior detection officers (BDOs), who examine passenger behaviors and appearances to identify passengers who might pose a potential security risk at TSA-regulated airports; travel document checkers, who examine tickets, passports, and other forms of identification; TSOs responsible for screening passengers and their carry-on baggage at passenger checkpoints, using x-ray equipment, magnetometers, Advanced Imaging Technology, and other devices; random employee screening; and checked-baggage screening systems.

DHS’s Science and Technology Directorate (S&T) and TSA have taken actions to coordinate and collaborate in their efforts to develop and deploy technologies for aviation security. For example, they entered into a 2006 memorandum of understanding for using S&T’s Transportation Security Laboratory, and they established the Capstone Integrated Product Team for Explosives Prevention in 2006 to help DHS, TSA, and the U.S. Secret Service to, among other things, identify priorities for explosives prevention.

DHS and TSA Have Experienced Challenges in Developing and Meeting Key Performance Requirements for Various Technologies

Our past work has found that technology program performance cannot be accurately assessed without valid baseline requirements established at the program start. Without the development, review, and approval of key acquisition documents, such as the mission need statement, agencies are at risk of having poorly defined requirements that can negatively affect program performance and contribute to increased costs. For example, in June 2010, we reported that over half of the 15 DHS programs we reviewed awarded contracts to initiate acquisition activities without component or department approval of documents essential to planning acquisitions, setting operational requirements, or establishing acquisition program baselines. For example, TSA’s Electronic Baggage Screening Program did not have a department-approved program baseline or program requirements, but TSA is acquiring and deploying next- generation explosive detection technology to replace legacy systems. We made a number of recommendations to help address issues related to these procurements as discussed below. DHS has generally agreed with these recommendations and, to varying degrees, has taken actions to address them.

In addition, our past work has found that TSA faces challenges in identifying and meeting program requirements in a number of its programs. For example: In July 2011, we reported that TSA revised its explosive detection system (EDS) requirements to better address current threats and plans to implement these requirements in a phased approach. However, we reported that some number of the EDSs in TSA’s fleet are configured to detect explosives at the levels established in the 2005 requirements. The remaining EDSs are configured to detect explosives at 1998 levels. When TSA established the 2005 requirements, it did not have a plan with the appropriate time frames needed to deploy EDSs to meet the requirements. To help ensure that EDSs are operating most effectively, we recommended that TSA develop a plan to deploy and operate EDSs to meet the most recent requirements to ensure new and currently deployed EDSs are operated at the levels in established requirements. DHS concurred with our recommendation and has begun taking action to address them; for example, DHS reported that TSA has developed a plan to evaluate its current fleet of EDSs to determine the extent to which they comply with these requirements. However, our recommendation is intended to ensure that TSA operate all EDSs at airports at the most recent requirements. Until TSA develops a plan identifying how it will approach the upgrades for currently deployed EDSs—and the plan includes such items as estimated costs and the number of machines that can be upgraded—it will be difficult for TSA to provide reasonable assurance that its upgrade approach is feasible or cost- effective. Further, while TSA’s efforts are positive steps, it is too early to assess their effect or whether they address our recommendation.

In October 2009, we reported that TSA passenger screening checkpoint technologies were delayed because TSA had not consistently communicated clear requirements for testing the technologies. We recommended that TSA evaluate whether current passenger screening procedures should be revised to require the use of appropriate screening procedures until TSA determined that existing emerging technologies meet their functional requirements in an operational environment. TSA agreed with this recommendation. However, communications issues with the business community persist. In July 2011, we reported that vendors for checked-baggage screening technology expressed concerns about the extent to which TSA communicated with the business community about the current EDS procurement. TSA agreed with our July 2011 recommendation to establish a process to communicate information regarding TSA’s EDS acquisition to EDS vendors in a timely manner and reported taking actions to address it such as soliciting more feedback from vendors through kickoff meetings, industry days, and classified discussions of program requirements.

DHS and TSA Have Encountered Challenges in Overseeing and Testing New Technologies

Our prior work has also shown that not resolving problems discovered during testing can sometimes lead to costly redesign and rework at a later date. Addressing such problems before moving to the acquisition phase can help agencies better manage costs. Specifically: In June 2011 we reported that S&T’s Test & Evaluation and Standards Office, responsible for overseeing test and evaluation of DHS’s major acquisition programs, reviewed or approved test and evaluation documents and plans for programs undergoing testing, and conducted independent assessments for the programs that completed operational testing. DHS senior-level officials considered the office’s assessments and input in deciding whether programs were ready to proceed to the next acquisition phase. However, the office did not consistently document its review and approval of components’ test agents—a government entity or independent contractor carrying out independent operational testing for a major acquisition. In addition, the office did not document its review of other component acquisition documents, such as those establishing programs’ operational requirements. We recommended, among other things, that S&T develop mechanisms to document its review of component acquisition documentation. DHS concurred and reported actions underway to address them.

In July 2011, we reported that TSA experienced challenges in collecting explosives data on the physical and chemical properties of certain explosives needed by vendors to develop EDS detection software. These data are also needed by TSA for testing the machines to determine whether they meet established requirements prior to their procurement and deployment to airports. TSA and S&T have experienced these challenges because of problems associated with safely handling and consistently formulating some explosives.

The challenges related to data collection for certain explosives have resulted in problems carrying out the EDS procurement as planned. Specifically, attempting to collect data for certain explosives while simultaneously pursuing the EDS procurement delayed the EDS acquisition schedule. We recommended that TSA develop a plan to ensure that TSA has the explosives data needed for each of the planned phases of the 2010 EDS requirements before starting the procurement process for new EDSs or upgrades included in each applicable phase. DHS stated that TSA modified its strategy for the EDS’s competitive procurement in July 2010 in response to the challenges in working with the explosives for data collection by removing the data collection from the procurement process. While TSA’s plan to separate the data collection from the procurement process is a positive step, we feel, to fully address our recommendation, a plan is needed to establish a process for ensuring that data are available before starting the procurement process for new EDSs or upgrades for each applicable phase.

In July 2011, we also reported that TSA revised EDS explosives detection requirements in January 2010 to better address current threats and plans to implement these requirements in a phased approach. TSA had previously revised the EDS requirements in 2005 though it did not begin operating EDS to meet the 2005 requirements until 2009. Further, TSA deployed a number of EDSs that had the software necessary to meet the 2005 requirements, but because the software was not activated, these EDSs were still detecting explosives at levels established before TSA revised the requirements in 2005. TSA officials stated that prior to activating the software in these EDSs, they must conduct testing to compare the false-alarm rates for machines operating at one level of requirements to those operating at another level of requirements. According to TSA officials, the results of this testing would allow them to determine if additional staff are needed at airports to help resolve false alarms once the EDSs are configured to operate at a certain level of requirements. TSA officials told us that they plan to perform this testing as a part of the current EDS acquisition.

In October 2009, we reported that TSA deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, in January 2006 even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. TSA also lacked assurance that the portals would meet functional requirements in airports within estimated costs and the machines were more expensive to install and maintain than expected. In June 2006, TSA halted deployment of the explosives trace portals because of performance problems and high installation costs. We recommended that to the extent feasible, TSA ensure that tests are completed before deploying checkpoint screening technologies to airports. DHS concurred with the recommendation and has taken action to address it, such as requiring more-recent technologies to complete both laboratory and operational tests prior to deployment. For example, TSA officials stated that, unlike the explosive trace portal, operational testing for the Advanced Imaging Technology (AIT) was successfully completed late in 2009 before its deployment was fully initiated. We are currently evaluating the testing conducted on AIT as part of an ongoing review.

TSA Has Not Consistently Incorporated Information on Costs and Benefits in Making Acquisition Decisions

According to the National Infrastructure Protection Plan, security strategies should be informed by, among other things, a risk assessment that includes threat, vulnerability, and consequence assessments, information such as cost-benefit analyses to prioritize investments, and performance measures to assess the extent to which a strategy reduces or mitigates the risk of terrorist attacks. Our prior work has shown that cost-benefit analyses help congressional and agency decision makers assess and prioritize resource investments and consider potentially more cost-effective alternatives, and that without this ability, agencies are at risk of experiencing cost overruns, missed deadlines, and performance shortfalls. For example, we have reported that TSA has not consistently included these analyses in its acquisition decision making. Specifically: In October 2009, we reported that TSA had not yet completed a cost- benefit analysis to prioritize and fund its technology investments for screening passengers at airport checkpoints. One reason that TSA had difficulty developing a cost-benefit analysis was that it had not yet developed life-cycle cost estimates for its various screening technologies. We reported that this information was important because it would help decision makers determine, given the cost of various technologies, which technology provided the greatest mitigation of risk for the resources that were available. We recommended that TSA develop a cost-benefit analysis. TSA agreed with this recommendation and has completed a life-cycle cost estimate, but has not yet completed a cost-benefit analysis.

In March 2010, we reported that TSA had not conducted a cost- benefit analysis to guide the initial AIT deployment strategy. Such an analysis would help inform TSA’s judgment about the optim deployment strategy for the AITs, as well as provide information to inform the best path forward, considering all elements of the screening system, for addressing the vulnerability identified by the attempted December 25, 2009, terrorist attack. We recommended that TSA conduct a cost-benefit analysis. TSA completed a cost- effectiveness analysis in June 2011 and provided it to us in August 2011. We are currently evaluating this analysis as part of our ongoing AIT review.

DHS Has Efforts Underway to Strengthen Acquisition and Technology Development

Since DHS’s inception in 2003, we have designated implementing and transforming DHS as high risk because DHS had to transform 22 agencies—several with major management challenges—into one department. This high-risk area includes challenges in strengthening DHS’s management functions, including acquisitions; the effect of those challenges on DHS’s mission implementation; and challenges in integrating management functions within and across the department and its components. Failure to effectively address DHS’s management and mission risks could have serious consequences for U.S. national and economic security.

In part because of the problems we have highlighted in DHS’s acquisition process, implementing and transforming DHS has remained on our high- risk list. DHS currently has several plans and efforts underway to address the high-risk designation as well as the more specific challenges related to acquisition, technology development, and program implementation that we have previously identified.

In June 2011, DHS reported to us that it is taking steps to strengthen its investment and acquisition management processes across the department by implementing a decision-making process at critical phases throughout the investment life cycle. For example, DHS reported that it plans to establish a new model for managing departmentwide investments across their life cycles. Under this plan, S&T would be involved in each phase of the investment life cycle and participate in new councils and boards DHS is planning to create to help ensure that test and evaluation methods are appropriately considered as part of DHS’s overall research and development investment strategies. According to DHS, S&T will help ensure that new technologies are properly scoped, developed, and tested before being implemented. DHS also reports that it is working with components to improve the quality and accuracy of cost estimates and has increased its staff during fiscal year 2011 to develop independent cost estimates, a GAO best practice, to ensure the accuracy and credibility of program costs. DHS reports that four cost estimates for level 1 programs have been validated to date, but did not explicitly identify whether any of the Life Cycle Cost Estimates were for TSA programs.

The actions DHS reports taking or has underway to address the management of its acquisitions and the development of new technologies are positive steps and, if implemented effectively, could help the department address many of these challenges. However, showing demonstrable progress in executing these plans is key. In the past, DHS has not effectively implemented its acquisition policies, in part because it lacked the oversight capacity necessary to manage its growing portfolio of major acquisition programs. Since DHS has only recently initiated these actions, it is too early to fully assess their effect on the challenges that we will need have identified in our past work. Going forward, we believe DHS to demonstrate measurable, sustainable progress in effectively implementing these actions.

Chairman Rogers, Ranking Member Jackson Lee, and Members of the Subcommittee, this concludes my prepared statement. I would be pleased to respond to any questions that you or other members of the subcommittee may have.

GAO Contact and Staff Acknowledgments

For questions about this statement, please contact Steve Lord at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last p of this st atement. Individuals making key contributions to this testimony are David M. Bruno, Assistant Director; Robert Lowthian; Scott Behen; Ryan Consaul; Tom Lombardi; Bill Russell; Nate Tranquilli; and Julie Silvers.

Key contributors for the previous work that this testimony is based on are listed within each individual product.

Related GAO Products

Aviation Security: TSA Has Made Progress, but Additional Efforts Are Needed to Improve Security. GAO-11-938T. Washington, D.C.: September 16, 2011.

Department of Homeland Security: Progress Made and Work Remaining in Implementing Homeland Security Missions 10 Years after 9/11. GAO-11-881. Washington, D.C.: September 7, 2011.

Homeland Security: DHS Could Strengthen Acquisitions and Development of New Technologies. GAO-11-829T. Washington, D.C.: July 15, 2011.

Aviation Security: TSA Has Taken Actions to Improve Security, but Additional Efforts Remain. GAO-11-807T. Washington, D.C.: July 13, 2011.

Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for Checked Baggage, but Additional Screening Actions Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011.

Homeland Security: Improvements in Managing Research and Development Could Help Reduce Inefficiencies and Costs. GAO-11-464T. Washington D.C.: March 15, 2011.

High-Risk Series: An Update. GAO-11-278. Washington D.C.: February 16, 2011.

Department of Homeland Security: Assessments of Selected Complex Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010.

Aviation Security: Progress Made but Actions Needed to Address Challenges in Meeting the Air Cargo Screening Mandate. GAO-10-880T. Washington, D.C.: June 30, 2010.

Aviation Security: TSA Is Increasing Procurement and Deployment of Advanced Imaging Technology, but Challenges to This Effort and Other Areas of Aviation Security Remain. GAO-10-484T. Washington, D.C.: March 17, 2010.

Aviation Security: DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges. GAO-10-128. Washington, D.C.: October 7, 2009.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.