Homeland Security: DHS and TSA Acquisition and Development of New Technologies
Published: 2011-09-22T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Within the Department of Homeland Security (DHS), the Transportation Security Administration (TSA) is responsible for developing and acquiring new technologies to address homeland security needs. TSA's acquisition programs represent billions of dollars in life-cycle costs and support a wide range of aviation security missions and investments including technologies used to screen passengers, checked baggage, and air cargo, among others. GAO's testimony addresses three key challenges identified in past work: (1) developing technology program requirements, (2) overseeing and conducting testing of new technologies, and (3) incorporating information on costs and benefits in making technology acquisition decisions. This statement also addresses recent DHS efforts to strengthen its investment and acquisition processes. This statement is based on reports and testimonies GAO issued from October 2009 through September 2011 related to TSA's efforts to manage, test, and deploy various technology programs.
section_title: What GAO Found
paragraphs: GAO's past work has found that TSA has faced challenges in developing technology program requirements on a systemic and individual basis. Program performance cannot be accurately assessed without valid baseline requirements established at the program start. In June 2010, GAO reported that over half of the 15 DHS programs (including 3 TSA programs) GAO reviewed awarded contracts to initiate acquisition activities without component or department approval of documents essential to planning acquisitions, setting operational requirements, or establishing acquisition program baselines. At the program level, in July 2011, GAO reported that in 2010 TSA revised its explosive detection systems (EDS) requirements to better address current threats and plans to implement these requirements in a phased approach. However, GAO reported that some number of the EDSs in TSA's fleet are configured to detect explosives at the levels established in the 2005 requirements and TSA did not have a plan with time frames needed to deploy EDSs to meet the current requirements. GAO has also reported DHS and TSA challenges in overseeing and testing new technologies. For example, in July 2011, GAO reported that TSA experienced challenges in collecting data on the physical and chemical properties of certain explosives needed by vendors to develop EDS detection software and needed by TSA before procuring and deploying EDSs to airports. TSA and DHS Science and Technology Directorate have experienced these challenges because of problems associated with safely handling and consistently formulating some explosives. The challenges related to data collection for certain explosives have resulted in problems carrying out the EDS procurement as planned. In addition, in October 2009, GAO reported that TSA deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, in January 2006 even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. In June 2006, TSA halted deployment of the explosives trace portals because of performance problems and high installation costs. GAO's prior work has shown that cost-benefit analyses help congressional and agency decision makers assess and prioritize resource investments and consider potentially more cost-effective alternatives, and that without this ability, agencies are at risk of experiencing cost overruns, missed deadlines, and performance shortfalls. GAO has reported that TSA has not consistently included these analyses in its acquisition decision making. In June 2011, DHS reported that it is taking steps to strengthen its investment and acquisition management processes by implementing a decision-making process at critical phases throughout the investment life cycle. The actions DHS reports taking to address the management of its acquisitions and the development of new technologies are positive steps and, if implemented effectively, could help the department address many of these challenges.
section_title: What GAO Recommends
paragraphs: GAO is not making any new recommendations. In prior work, GAO made recommendations to address challenges related to deploying EDS to meet requirements, overseeing and conducting testing of new technologies, and incorporating information on costs and benefits in making technology acquisition decisions. DHS and TSA concurred and described actions underway to address the recommendations.

section_title: Letter
paragraphs: I am pleased to be here today to discuss our past work examining the  Transportation Security Administration’s (TSA) progress and challenges  in developing and acquiring new technologies to address homeland  security needs. TSA acquisition programs represent billions of dollars in  life-cycle costs and support a wide range of aviation security missions  and investments including technologies used to screen passengers,  checked baggage, and air cargo, among others. Within the Department of  Homeland Security (DHS), the Science and Technology Directorate  (S&T) has responsibility for coordinating and conducting basic and  applied research, development, demonstration, testing, and evaluation  activities relevant to DHS components, which also have responsibilities  for developing, testing, acquiring, and deploying such technologies. For  example, TSA is responsible for securing the nation’s transportation  systems and, with S&T, researching, developing, and deploying  technologies to, for example, screen airline passengers and their  property.
In recent years, we have reported that DHS has experienced challenges  in managing its multibillion-dollar acquisition efforts, including  implementing technologies that did not meet intended requirements and  were not appropriately tested and evaluated, and has not consistently  included completed analyses of costs and benefits before technologies  were implemented.
My testimony today focuses on the key findings of our prior work related  to TSA’s efforts to acquire and deploy new technologies to address  homeland security needs. Our past work has identified three key  challenges: (1) developing technology program requirements, (2)  overseeing and conducting testing of new technologies, and (3)  incorporating information on costs and benefits in making technology  acquisition decisions. This statement will also discuss recent DHS and  TSA efforts to strengthen its investment and acquisition processes.
This statement is based on reports and testimonies we issued from  October 2009 through September 2011 related to TSA’s efforts to  manage, test, and deploy various technology programs. For our past  work, we reviewed program schedules, planning documents, testing  reports, and other acquisition documentation. For some of the programs  we discuss in this testimony, we conducted site visits to a range of  facilities, such as national laboratories, airports, and other locations to  observe research, development, and testing efforts. We also conducted  interviews with DHS component program managers and S&T officials to  discuss issues related to individual programs. We conducted this work in  accordance with generally accepted government auditing standards.  More detailed information on the scope and methodology from our  previous work can be found within each specific report.
subsections: section_title: Background
paragraphs: The Aviation and Transportation Security Act (ATSA) established TSA as  the federal agency with primary responsibility for securing the nation’s  civil aviation system, which includes the screening of all passenger and  property transported from and within the United States by commercial  passenger aircraft. In accordance with ATSA, all passengers, their  accessible property, and their checked baggage are screened pursuant to  TSA-established procedures at the 463 airports presently regulated for  security by TSA. These procedures generally provide, among other  things, that passengers pass through security checkpoints where they  and their identification documents, and accessible property, are checked  by transportation security officers (TSO), other TSA employees, or by  private-sector screeners under TSA’s Screening Partnership Program.  Airport operators, however, also have direct responsibility for  implementing TSA security requirements such as those relating to  perimeter security and access controls, in accordance with their approved  security programs and other TSA direction.
TSA relies upon multiple layers of security to deter, detect, and disrupt  persons posing a potential risk to aviation security. These layers include  behavior detection officers (BDOs), who examine passenger behaviors  and appearances to identify passengers who might pose a potential  security risk at TSA-regulated airports; travel document checkers, who  examine tickets, passports, and other forms of identification; TSOs  responsible for screening passengers and their carry-on baggage at  passenger checkpoints, using x-ray equipment, magnetometers,  Advanced Imaging Technology, and other devices; random employee  screening; and checked-baggage screening systems.
DHS’s Science and Technology Directorate (S&T) and TSA have taken  actions to coordinate and collaborate in their efforts to develop and  deploy technologies for aviation security. For example, they entered into a  2006 memorandum of understanding for using S&T’s Transportation  Security Laboratory, and they established the Capstone Integrated  Product Team for Explosives Prevention in 2006 to help DHS, TSA, and  the U.S. Secret Service to, among other things, identify priorities for  explosives prevention.
section_title: DHS and TSA Have Experienced Challenges in Developing and Meeting Key Performance Requirements for Various Technologies
paragraphs: Our past work has found that technology program performance cannot be  accurately assessed without valid baseline requirements established at  the program start. Without the development, review, and approval of key  acquisition documents, such as the mission need statement, agencies are  at risk of having poorly defined requirements that can negatively affect  program performance and contribute to increased costs. For example, in  June 2010, we reported that over half of the 15 DHS programs we  reviewed awarded contracts to initiate acquisition activities without  component or department approval of documents essential to planning  acquisitions, setting operational requirements, or establishing acquisition  program baselines. For example, TSA’s Electronic Baggage Screening  Program did not have a department-approved program baseline or  program requirements, but TSA is acquiring and deploying next- generation explosive detection technology to replace legacy systems. We  made a number of recommendations to help address issues related to  these procurements as discussed below. DHS has generally agreed with  these recommendations and, to varying degrees, has taken actions to  address them.
In addition, our past work has found that TSA faces challenges in  identifying and meeting program requirements in a number of its  programs. For example:  In July 2011, we reported that TSA revised its explosive detection  system (EDS) requirements to better address current threats and  plans to implement these requirements in a phased approach.  However, we reported that some number of the EDSs in TSA’s fleet  are configured to detect explosives at the levels established in the  2005 requirements. The remaining EDSs are configured to detect  explosives at 1998 levels. When TSA established the 2005  requirements, it did not have a plan with the appropriate time frames  needed to deploy EDSs to meet the requirements. To help ensure that  EDSs are operating most effectively, we recommended that TSA  develop a plan to deploy and operate EDSs to meet the most recent  requirements to ensure new and currently deployed EDSs are  operated at the levels in established requirements. DHS concurred  with our recommendation and has begun taking action to address  them; for example, DHS reported that TSA has developed a plan to  evaluate its current fleet of EDSs to determine the extent to which  they comply with these requirements. However, our recommendation  is intended to ensure that TSA operate all EDSs at airports at the  most recent requirements. Until TSA develops a plan identifying how it  will approach the upgrades for currently deployed EDSs—and the  plan includes such items as estimated costs and the number of  machines that can be upgraded—it will be difficult for TSA to provide  reasonable assurance that its upgrade approach is feasible or cost- effective. Further, while TSA’s efforts are positive steps, it is too early  to assess their effect or whether they address our recommendation.
In October 2009, we reported that TSA passenger screening  checkpoint technologies were delayed because TSA had not  consistently communicated clear requirements for testing the  technologies. We recommended that TSA evaluate whether current  passenger screening procedures should be revised to require the use  of appropriate screening procedures until TSA determined that  existing emerging technologies meet their functional requirements in  an operational environment. TSA agreed with this recommendation.  However, communications issues with the business community  persist. In July 2011, we reported that vendors for checked-baggage  screening technology expressed concerns about the extent to which  TSA communicated with the business community about the current  EDS procurement. TSA agreed with our July 2011 recommendation  to establish a process to communicate information regarding TSA’s  EDS acquisition to EDS vendors in a timely manner and reported  taking actions to address it such as soliciting more feedback from  vendors through kickoff meetings, industry days, and classified  discussions of program requirements.
section_title: DHS and TSA Have Encountered Challenges in Overseeing and Testing New Technologies
paragraphs: Our prior work has also shown that not resolving problems discovered  during testing can sometimes lead to costly redesign and rework at a later  date. Addressing such problems before moving to the acquisition phase  can help agencies better manage costs. Specifically:  In June 2011 we reported that S&T’s Test & Evaluation and  Standards Office, responsible for overseeing test and evaluation of  DHS’s major acquisition programs, reviewed or approved test and  evaluation documents and plans for programs undergoing testing, and  conducted independent assessments for the programs that completed  operational testing. DHS senior-level officials considered the office’s  assessments and input in deciding whether programs were ready to  proceed to the next acquisition phase. However, the office did not  consistently document its review and approval of components’ test  agents—a government entity or independent contractor carrying out  independent operational testing for a major acquisition. In addition,  the office did not document its review of other component acquisition  documents, such as those establishing programs’ operational  requirements. We recommended, among other things, that S&T  develop mechanisms to document its review of component acquisition  documentation. DHS concurred and reported actions underway to  address them.
In July 2011, we reported that TSA experienced challenges in  collecting explosives data on the physical and chemical properties of  certain explosives needed by vendors to develop EDS detection  software. These data are also needed by TSA for testing the  machines to determine whether they meet established requirements  prior to their procurement and deployment to airports. TSA and S&T  have experienced these challenges because of problems associated  with safely handling and consistently formulating some explosives.
The challenges related to data collection for certain explosives have  resulted in problems carrying out the EDS procurement as planned.  Specifically, attempting to collect data for certain explosives while  simultaneously pursuing the EDS procurement delayed the EDS  acquisition schedule. We recommended that TSA develop a plan to  ensure that TSA has the explosives data needed for each of the  planned phases of the 2010 EDS requirements before starting the  procurement process for new EDSs or upgrades included in each  applicable phase. DHS stated that TSA modified its strategy for the  EDS’s competitive procurement in July 2010 in response to the  challenges in working with the explosives for data collection by  removing the data collection from the procurement process. While  TSA’s plan to separate the data collection from the procurement  process is a positive step, we feel, to fully address our  recommendation, a plan is needed to establish a process for ensuring  that data are available before starting the procurement process for  new EDSs or upgrades for each applicable phase.
In July 2011, we also reported that TSA revised EDS explosives  detection requirements in January 2010 to better address current  threats and plans to implement these requirements in a phased  approach. TSA had previously revised the EDS requirements in 2005  though it did not begin operating EDS to meet the 2005 requirements  until 2009. Further, TSA deployed a number of EDSs that had the  software necessary to meet the 2005 requirements, but because the  software was not activated, these EDSs were still detecting explosives  at levels established before TSA revised the requirements in 2005.  TSA officials stated that prior to activating the software in these EDSs,  they must conduct testing to compare the false-alarm rates for  machines operating at one level of requirements to those operating at  another level of requirements. According to TSA officials, the results  of this testing would allow them to determine if additional staff are  needed at airports to help resolve false alarms once the EDSs are  configured to operate at a certain level of requirements. TSA officials  told us that they plan to perform this testing as a part of the current  EDS acquisition.
In October 2009, we reported that TSA deployed explosives trace  portals, a technology for detecting traces of explosives on passengers  at airport checkpoints, in January 2006 even though TSA officials  were aware that tests conducted during 2004 and 2005 on earlier  models of the portals suggested the portals did not demonstrate  reliable performance in an airport environment. TSA also lacked  assurance that the portals would meet functional requirements in  airports within estimated costs and the machines were more  expensive to install and maintain than expected. In June 2006, TSA  halted deployment of the explosives trace portals because of  performance problems and high installation costs. We recommended  that to the extent feasible, TSA ensure that tests are completed before  deploying checkpoint screening technologies to airports. DHS  concurred with the recommendation and has taken action to address  it, such as requiring more-recent technologies to complete both  laboratory and operational tests prior to deployment. For example,  TSA officials stated that, unlike the explosive trace portal, operational  testing for the Advanced Imaging Technology (AIT) was successfully  completed late in 2009 before its deployment was fully initiated. We  are currently evaluating the testing conducted on AIT as part of an  ongoing review.
section_title: TSA Has Not Consistently Incorporated Information on Costs and Benefits in Making Acquisition Decisions
paragraphs: According to the National Infrastructure Protection Plan, security  strategies should be informed by, among other things, a risk assessment  that includes threat, vulnerability, and consequence assessments,  information such as cost-benefit analyses to prioritize investments, and  performance measures to assess the extent to which a strategy reduces  or mitigates the risk of terrorist attacks. Our prior work has shown that  cost-benefit analyses help congressional and agency decision makers  assess and prioritize resource investments and consider potentially more  cost-effective alternatives, and that without this ability, agencies are at  risk of experiencing cost overruns, missed deadlines, and performance  shortfalls. For example, we have reported that TSA has not consistently  included these analyses in its acquisition decision making. Specifically:  In October 2009, we reported that TSA had not yet completed a cost- benefit analysis to prioritize and fund its technology investments for  screening passengers at airport checkpoints. One reason that TSA  had difficulty developing a cost-benefit analysis was that it had not yet  developed life-cycle cost estimates for its various screening  technologies. We reported that this information was important  because it would help decision makers determine, given the cost of  various technologies, which technology provided the greatest  mitigation of risk for the resources that were available. We  recommended that TSA develop a cost-benefit analysis. TSA agreed  with this recommendation and has completed a life-cycle cost  estimate, but has not yet completed a cost-benefit analysis.
In March 2010, we reported that TSA had not conducted a cost- benefit analysis to guide the initial AIT deployment strategy. Such  an analysis would help inform TSA’s judgment about the optim deployment strategy for the AITs, as well as provide information to  inform the best path forward, considering all elements of the  screening system, for addressing the vulnerability identified by the  attempted December 25, 2009, terrorist attack. We recommended that  TSA conduct a cost-benefit analysis. TSA completed a cost- effectiveness analysis in June 2011 and provided it to us in August  2011. We are currently evaluating this analysis as part of our ongoing  AIT review.
section_title: DHS Has Efforts Underway to Strengthen Acquisition and Technology Development
paragraphs: Since DHS’s inception in 2003, we have designated implementing and  transforming DHS as high risk because DHS had to transform 22  agencies—several with major management challenges—into one  department. This high-risk area includes challenges in strengthening  DHS’s management functions, including acquisitions; the effect of those  challenges on DHS’s mission implementation; and challenges in  integrating management functions within and across the department and  its components. Failure to effectively address DHS’s management and  mission risks could have serious consequences for U.S. national and  economic security.
In part because of the problems we have highlighted in DHS’s acquisition  process, implementing and transforming DHS has remained on our high-  risk list. DHS currently has several plans and efforts underway to address  the high-risk designation as well as the more specific challenges related  to acquisition, technology development, and program implementation that  we have previously identified.
In June 2011, DHS reported to us that it is taking steps to strengthen its  investment and acquisition management processes across the  department by implementing a decision-making process at critical phases  throughout the investment life cycle. For example, DHS reported that it  plans to establish a new model for managing departmentwide  investments across their life cycles. Under this plan, S&T would be  involved in each phase of the investment life cycle and participate in new  councils and boards DHS is planning to create to help ensure that test  and evaluation methods are appropriately considered as part of DHS’s  overall research and development investment strategies. According to  DHS, S&T will help ensure that new technologies are properly scoped,  developed, and tested before being implemented. DHS also reports that it  is working with components to improve the quality and accuracy of cost  estimates and has increased its staff during fiscal year 2011 to develop  independent cost estimates, a GAO best practice, to ensure the accuracy  and credibility of program costs. DHS reports that four cost estimates  for level 1 programs have been validated to date, but did not explicitly  identify whether any of the Life Cycle Cost Estimates were for TSA  programs.
The actions DHS reports taking or has underway to address the  management of its acquisitions and the development of new technologies  are positive steps and, if implemented effectively, could help the  department address many of these challenges. However, showing  demonstrable progress in executing these plans is key. In the past, DHS  has not effectively implemented its acquisition policies, in part because it  lacked the oversight capacity necessary to manage its growing portfolio of  major acquisition programs. Since DHS has only recently initiated these  actions, it is too early to fully assess their effect on the challenges that we   will need  have identified in our past work. Going forward, we believe DHS to demonstrate measurable, sustainable progress in effectively  implementing these actions.
Chairman Rogers, Ranking Member Jackson Lee, and Members of the  Subcommittee, this concludes my prepared statement. I would be  pleased to respond to any questions that you or other members of the  subcommittee may have.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions about this statement, please contact Steve Lord at (202)  512-4379 or lords@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last p of this st atement. Individuals making key contributions to this testimony  are David M. Bruno, Assistant Director; Robert Lowthian; Scott Behen;  Ryan Consaul; Tom Lombardi; Bill Russell; Nate Tranquilli; and Julie  Silvers.
Key contributors for the previous work that this testimony is based on are  listed within each individual product.
section_title: Related GAO Products
paragraphs: Aviation Security: TSA Has Made Progress, but Additional Efforts Are  Needed to Improve Security. GAO-11-938T. Washington, D.C.:  September 16, 2011.
Department of Homeland Security: Progress Made and Work Remaining  in Implementing Homeland Security Missions 10 Years after 9/11.  GAO-11-881. Washington, D.C.: September 7, 2011.
Homeland Security: DHS Could Strengthen Acquisitions and  Development of New Technologies. GAO-11-829T. Washington, D.C.:  July 15, 2011.
Aviation Security: TSA Has Taken Actions to Improve Security, but  Additional Efforts Remain. GAO-11-807T. Washington, D.C.: July 13,  2011.
Aviation Security: TSA Has Enhanced Its Explosives Detection  Requirements for Checked Baggage, but Additional Screening Actions  Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011.
Homeland Security: Improvements in Managing Research and  Development Could Help Reduce Inefficiencies and Costs.  GAO-11-464T. Washington D.C.: March 15, 2011.
High-Risk Series: An Update. GAO-11-278. Washington D.C.: February  16, 2011.
Department of Homeland Security: Assessments of Selected Complex  Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010.
Aviation Security: Progress Made but Actions Needed to Address  Challenges in Meeting the Air Cargo Screening Mandate. GAO-10-880T.  Washington, D.C.: June 30, 2010.
Aviation Security: TSA Is Increasing Procurement and Deployment of  Advanced Imaging Technology, but Challenges to This Effort and Other  Areas of Aviation Security Remain. GAO-10-484T. Washington, D.C.:  March 17, 2010.
Aviation Security: DHS and TSA Have Researched, Developed, and  Begun Deploying Passenger Checkpoint Screening Technologies, but  Continue to Face Challenges. GAO-10-128. Washington, D.C.: October  7, 2009.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.