Homeland Security: DHS Could Strengthen Acquisitions and Development of New Technologies
Published: 2011-07-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This testimony discusses our past work examining the Department of Homeland Security's (DHS) progress and challenges in developing and acquiring new technologies to address homeland security needs. DHS acquisition programs represent hundreds of billions of dollars in life-cycle costs and support a wide range of missions and investments including border surveillance and screening equipment, nuclear detection equipment, and technologies used to screen airline passengers and baggage for explosives, among others. Since its creation in 2003, DHS has spent billions of dollars developing and procuring technologies and other countermeasures to address various threats and to conduct its missions. Within DHS, the Science and Technology Directorate (S&T) conducts general research and development and oversees the testing and evaluation efforts of DHS components, which are responsible for developing, testing, and acquiring their own technologies. This testimony focuses on the findings of our prior work related to DHS's efforts to acquire and deploy new technologies to address homeland security needs. Our past work has identified three key challenges: (1) developing technology program requirements, (2) conducting and completing testing and evaluation of technologies and (3) incorporating information on costs and benefits in making technology acquisition decisions. This statement will also discuss recent DHS efforts to strengthen its investment and acquisition processes.
section_title: What GAO Found
paragraphs: We have identified technologies that DHS has deployed that have not met key performance requirements. For example, in June 2010, we reported that over half of the 15 DHS programs we reviewed awarded contracts to initiate acquisition activities without component or department approval of documents essential to planning acquisitions, setting operational requirements, and establishing acquisition program baselines. Our prior work has also identified that failure to resolve problems discovered during testing can sometimes lead to costly redesign and rework at a later date and that addressing such problems during the testing and evaluation phase before moving to the acquisition phase can help agencies avoid future cost overruns. Specifically: (1) In March 2011, we reported that the independent testing and evaluation of SBInet's Block 1 capability to determine its operational effectiveness and suitability was not complete at the time DHS reached its decision regarding the future of SBInet or requested fiscal year 2012 funding to deploy the new Alternative (Southwest) Border Technology. (2) In September 2010, we reported that S&T's plans for conducting operational testing of container security technologies did not reflect all of the operational scenarios that CBP was considering for implementation. (3) In October 2009, we reported that TSA deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. TSA also lacked assurance that the portals would meet functional requirements in airports within estimated costs and the machines were more expensive to install and maintain than expected. In June 2006, TSA halted deployment of the explosives trace portals because of performance problems and high installation costs. Our prior work has shown that cost-benefit analyses help congressional and agency decision makers assess and prioritize resource investments and consider potentially more cost-effective alternatives and that without this ability, agencies are at risk of experiencing cost overruns, missed deadlines, and performance shortfalls. For example, we have reported that DHS has not consistently included these analyses in its acquisition decision making. Specifically: (1) In March 2011, we reported that the decision by the Secretary of Homeland Security to end the SBInet program was informed by, among other things, an independent analysis of cost-effectiveness. However, it was not clear how DHS used the results to determine the appropriate technology plans and budget decisions, especially since the results of SBInet's operational effectiveness were not complete at the time of the Secretary's decision to end the program. Furthermore, the cost analysis was limited in scope and did not consider all technology solutions because of the need to complete the first phase of the analysis in 6 weeks. (2) In October 2009, we reported that TSA had not yet completed a cost-benefit analysis to prioritize and fund its technology investments for screening passengers at airport checkpoints. One reason that TSA had difficulty developing a cost-benefit analysis was that it had not yet developed life-cycle cost estimates for its various screening technologies. (3) In June 2009, we reported that DHS's cost analysis of the Advanced Spectroscopic Portal (ASP) program did not provide a sound analytical basis for DHS's decision to deploy the portals.

section_title: Letter
paragraphs: I am pleased to be here today to discuss our past work examining the  Department of Homeland Security’s (DHS) progress and challenges in  developing and acquiring new technologies to address homeland security  needs. DHS acquisition programs represent hundreds of billions of dollars  in life-cycle costs and support a wide range of missions and investments  including border surveillance and screening equipment, nuclear detection  equipment, and technologies used to screen airline passengers and  baggage for explosives, among others. Since its creation in 2003, DHS  has spent billions of dollars developing and procuring technologies and  other countermeasures to address various threats and to conduct its  missions. Within DHS, the Science and Technology Directorate (S&T)  conducts general research and development and oversees the testing  and evaluation efforts of DHS components, which are responsible for  developing, testing, and acquiring their own technologies. For example,  the Transportation Security Administration (TSA) is responsible for  securing the nation’s transportation systems and, with S&T, researching,  developing, and deploying technologies to, for example, screen airline  passengers and their baggage. U.S. Customs and Border Protection  (CBP) is responsible for implementing measures and technologies to  secure the nation’s borders. In recent years, we have reported that DHS  has experienced challenges in managing its multibillion-dollar acquisition  efforts, including implementing technologies that did not meet intended  requirements and were not appropriately tested and evaluated, and has  not consistently included completed analyses of costs and benefits before  technologies were implemented.
My testimony today focuses on the key findings of our prior work related  to DHS’s efforts to acquire and deploy new technologies to address  homeland security needs. Our past work has identified three key  challenges: (1) developing technology program requirements, (2)  conducting and completing testing and evaluation of technologies and (3)  incorporating information on costs and benefits in making technology  acquisition decisions. This statement will also discuss recent DHS efforts  to strengthen its investment and acquisition processes.
This statement is based on reports and testimonies we issued from May  2009 through July 2011 related to DHS’s efforts to manage, test, and  deploy various technology programs and selected updates conducted in  July 2011 related to DHS’s efforts to strengthen its investment and  acquisition processes. For the updates, we reviewed recent DHS efforts  to strengthen its investment and acquisition processes, such as a June  2011 DHS report on the department’s progress and efforts in addressing  challenges identified in our biennial reports addressing high-risk  management issues. For our past work, we reviewed program  schedules, planning documents, testing reports, and other acquisition  documentation. For some of the programs we discuss in this testimony,  we conducted site visits to a range of facilities, such as national  laboratories, airports, and other locations to observe research,  development, and testing efforts. We also conducted interviews with DHS  component program managers and S&T officials to discuss issues related  to individual programs. We conducted this work in accordance with  generally accepted government auditing standards. More detailed  information on the scope and methodology from our previous work can be  found within each specific report.
subsections: section_title: DHS Has Experienced Challenges in Developing and Meeting Key Performance Requirements for Various Technologies
paragraphs: Our past work has found that program performance cannot be accurately  assessed without valid baseline requirements established at the program  start. Without the development, review, and approval of key acquisition  documents, such as the mission need statement, agencies are at risk of  having poorly defined requirements that can negatively affect program  performance and contribute to increased costs. We have also identified  technologies that DHS has deployed that have not met key performance  requirements. For example, in June 2010, we reported that over half of  the 15 DHS programs we reviewed awarded contracts to initiate  acquisition activities without component or department approval of  documents essential to planning acquisitions, setting operational  requirements, and establishing acquisition program baselines. We made  a number of recommendations to help address these issues as discussed  below. DHS has generally agreed with these recommendations and, to  varying degrees, has taken actions to address them.
In addition, our past work has found that DHS faces challenges in  identifying and meeting program requirements in a number of its  programs. For example:  In July 2011, we reported that TSA revised its explosive detection  system (EDS) requirements to better address current threats and  plans to implement these requirements in a phased approach.  However, we reported that only some of the EDSs in TSA’s fleet are  configured to detect explosives at the levels established in the 2005  requirements. The remaining EDSs are configured to detect  explosives at 1998 levels. When TSA established the 2005  requirements, it did not have a plan with the appropriate time frames  needed to deploy EDSs to meet the requirements. To help ensure that  EDSs are operating most effectively, we recommended that TSA  develop a plan to deploy and operate EDSs to meet the most recent  requirements to ensure new and currently deployed EDSs are  operated at the levels in established requirements. DHS concurred  with our recommendation.   In September 2010, we reported that the Domestic Nuclear Detection  Office (DNDO) was simultaneously engaged in the research and  development phase while planning for the acquisition phase of its  cargo advanced automated radiography system to detect certain  nuclear materials in vehicles and containers at ports. DNDO pursued  the deployment of the cargo advanced automated radiography system  without fully understanding the physical requirements of incorporating  the system in existing inspection lanes at ports of entry. We reported  that this occurred because, during the first year or more of the  program, DNDO and CBP had few discussions about operating  requirements for primary inspection lanes at ports of entry. DHS spent  $113 million on the program since 2005 and canceled the  development phase of the program in 2007.  In May 2010, we reported that not all of the Secure Border Initiative  Network (SBInet) operational requirements that pertain to Block 1  were achievable, verifiable, unambiguous, and complete. For  example, a November 2007 DHS assessment found problems with 19  operational requirements, which form the basis for the lower-level  requirements used to design and build the system. As a result, we  recommended that the Block 1 requirements, including key  performance parameters, be independently validated as complete,  verifiable, and affordable and any limitations found in the  requirements be addressed. DHS agreed with these  recommendations and CBP program officials told us that they  recognized the difficulties they experienced with requirements  development practices with the SBInet program. In January 2011, the  Secretary of Homeland Security announced her decision to end the  program as originally conceived because it did not meet cost- effectiveness and viability standards.  In October 2009, we reported that TSA passenger screening  checkpoint technologies were delayed because TSA had not  consistently communicated clear requirements for testing the  technologies. We recommended that TSA evaluate whether current  passenger screening procedures should be revised to require the use  of appropriate screening procedures until TSA determined that  existing emerging technologies meet its functional requirements in an  operational environment. TSA agreed with this recommendation and  reported taking actions to address it.
section_title: DHS Has Encountered Challenges in Conducting and Completing Testing and Evaluation
paragraphs: Our prior work has also identified that failure to resolve problems  discovered during testing can sometimes lead to costly redesign and  rework at a later date and that addressing such problems during the  testing and evaluation phase before moving to the acquisition phase can  help agencies avoid future cost overruns. Specifically:  In March 2011, we reported that the independent testing and  evaluation of SBInet’s Block 1 capability to determine its operational  effectiveness and suitability was not complete at the time DHS  reached its decision regarding the future of SBInet or requested fiscal  year 2012 funding to deploy the new Alternative (Southwest) Border  Technology. We reported that because the Alternative (Southwest)  Border Technology incorporates a mix of technology, including an  Integrated Fixed Tower surveillance system similar to that currently  used in SBInet, the testing and evaluation could have informed DHS’s  decision about moving forward with the new technology deployment.  In September 2010, we reported that S&T’s plans for conducting  operational testing of container security technologies did not reflect all  of the operational scenarios that CBP was considering for  implementation. We reported that until the container security  technologies are tested and evaluated consistent with all of the  operational scenarios, S&T cannot provide reasonable assurance that  the technologies will function as intended. For example, S&T did not  include certain scenarios necessary to test how a cargo container  would be transported throughout the maritime supply chain. We  recommended that DHS test and evaluate the container security  technologies consistent with all the operational scenarios DHS  identified for potential implementation. DHS concurred with our  recommendation.  In October 2009, we reported that TSA deployed explosives trace  portals, a technology for detecting traces of explosives on passengers  at airport checkpoints, even though TSA officials were aware that  tests conducted during 2004 and 2005 on earlier models of the portals  suggested the portals did not demonstrate reliable performance in an  airport environment. TSA also lacked assurance that the portals  would meet functional requirements in airports within estimated costs  and the machines were more expensive to install and maintain than  expected. In June 2006, TSA halted deployment of the explosives  trace portals because of performance problems and high installation  costs. We recommended that to the extent feasible, TSA ensure that  tests are completed before deploying checkpoint screening  technologies to airports. DHS concurred with the recommendation  and has taken action to address it, such as requiring more-recent  technologies to complete both laboratory and operational tests prior to  deployment.
section_title: DHS Has Not Consistently Incorporated Information on Costs and Benefits in Making Acquisition Decisions
paragraphs: Our prior work has shown that cost-benefit analyses help congressional  and agency decision makers assess and prioritize resource investments  and consider potentially more cost-effective alternatives and that without  this ability, agencies are at risk of experiencing cost overruns, missed  deadlines, and performance shortfalls. For example, we have reported  that DHS has not consistently included these analyses in its acquisition  decision making. Specifically:  In March 2011, we reported that the decision by the Secretary of  Homeland Security to end the SBInet program was informed by,  among other things, an independent analysis of cost-effectiveness.  However, it was not clear how DHS used the results to determine the  appropriate technology plans and budget decisions, especially since  the results of SBInet’s operational effectiveness were not complete at  the time of the Secretary’s decision to end the program. Furthermore,  the cost analysis was limited in scope and did not consider all  technology solutions because of the need to complete the first phase  of the analysis in 6 weeks. It also did not assess the technology  approaches based on the incremental effectiveness provided above  the baseline technology assets in the geographic areas evaluated. As  we reported, for a program of this importance and cost, the process  used to assess and select technology needs to be more robust.  In October 2009, we reported that TSA had not yet completed a cost- benefit analysis to prioritize and fund its technology investments for  screening passengers at airport checkpoints. One reason that TSA  had difficulty developing a cost-benefit analysis was that it had not yet  developed life-cycle cost estimates for its various screening  technologies. We reported that this information was important  because it would help decision makers determine, given the cost of  various technologies, which technology provided the greatest  mitigation of risk for the resources that were available. We  recommended that TSA develop a cost-benefit analysis. TSA agreed  with this recommendation and has completed a life-cycle cost  estimate and collected information for its checkpoint technologies, but  has not yet completed a cost-benefit analysis.  In June 2009, we reported that DHS’s cost analysis of the Advanced  Spectroscopic Portal (ASP) program did not provide a sound  analytical basis for DHS’s decision to deploy the portals. We also  reported that an updated cost-benefit analysis might show that  DNDO’s plan to replace existing equipment with advanced  spectroscopic portals was not justified, particularly given the marginal  improvement in detection of certain nuclear materials required of  advanced spectroscopic portals and the potential to improve the  current-generation portal monitors’ sensitivity to nuclear materials,  most likely at a lower cost. At that time, DNDO officials stated that  they planned to update the cost-benefit analysis. After spending more  than $200 million on the program, in February 2010 DHS announced  that it was scaling back its plans for development and use of the  portals technology.
section_title: DHS Has Efforts Under Way to Strengthen Acquisition and Technology Development
paragraphs: Since DHS’s inception in 2003, we have designated implementing and  transforming DHS as high risk because DHS had to transform 22  agencies—several with major management challenges—into one  department. This high-risk area includes challenges in strengthening  DHS’s management functions, including acquisitions; the impact of those  challenges on DHS’s mission implementation; and challenges in  integrating management functions within and across the department and  its components. Failure to effectively address DHS’s management and  mission risks could have serious consequences for U.S. national and  economic security.
In part because of the problems we have highlighted in DHS’s acquisition  process, implementing and transforming DHS has remained on our high-  risk list. DHS currently has several plans and efforts underway to address  the high-risk designation as well as the more specific challenges related  to acquisition and program implementation that we have previously  identified.
In June 2011, DHS reported to us that it is taking steps to strengthen its  investment and acquisition management processes across the  department by implementing a decision-making process at critical phases  throughout the investment life cycle. For example, DHS reported that it  plans to establish a new model for managing departmentwide  investments across their life cycles. Under this plan, S&T would be  involved in each phase of the investment life cycle and participate in new  councils and boards DHS is planning to create to help ensure that test  and evaluation methods are appropriately considered as part of DHS’s  overall research and development investment strategies. In addition, DHS  reported that the new councils and boards it is planning to establish to  strengthen management of the department’s acquisition and investment  review process would be responsible for, among other things, making  decisions on research and development initiatives based on factors such  as viability and affordability and overseeing key acquisition decisions for  major programs using baseline and actual data. According to DHS, S&T  will help ensure that new technologies are properly scoped, developed,  and tested before being implemented. DHS also reports that it is working  with components to improve the quality and accuracy of cost estimates  and has increased its staff during fiscal year 2011 to develop independent  cost estimates, a GAO best practice, to ensure the accuracy and  credibility of program costs. DHS reports that four cost estimates for level  1 programs have been validated to date.
The actions DHS reports taking or has under way to address the  management of its acquisitions and the development of new technologies  are positive steps and, if implemented effectively, could help the  department address many of these challenges. However, showing  demonstrable progress in implementing these plans is key. In the past,  DHS has not effectively implemented its acquisition policies, in part  because it lacked the oversight capacity necessary to manage its growing  portfolio of major acquisition programs. Since DHS has only recently  initiated these actions, it is too early to fully assess their impact on the  challenges that we have identified in our past work. Going forward, we  believe DHS will need to demonstrate measurable, sustainable progress  in effectively implementing these actions.
Chairman McCaul, Ranking Member Keating, and Members of the  Subcommittee, this concludes my prepared statement. I would be  pleased to respond to any questions that you or other members of the  subcommittee may have.
section_title: GAO Contact and Staff Acknowledgements
paragraphs: For questions about this statement, please contact David C. Maurer at  (202) 512-9627 or maurerd@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. Individuals making key contributions to this statement  include Chris Currie, Assistant Director; Bintou Njie; and Michael Kniss.  John Hutton; Katherine Trimble; Nate Tranquilli; and Richard Hung also  made contributions to this statement. Key contributors for the previous  work that this testimony is based on are listed within each individual  product.
section_title: Related GAO Products
paragraphs: Aviation Security: TSA Has Enhanced Its Explosives Detection  Requirements for Checked Baggage, but Additional Screening Actions  Are Needed. GAO-11-740. (Washington, D.C.: July 11, 2011).
Homeland Security: Improvements in Managing Research and  Development Could Help Reduce Inefficiencies and Costs.  GAO-11-464T. (Washington D.C.: March. 15, 2011).
Border Security: Preliminary Observations on the Status of Key  Southwest Border Technology Programs. GAO-11-448T. (Washington  D.C.: March 15, 2011).
High-Risk Series: An Update. GAO-11-278. (Washington D.C.: February  16, 2011).
Supply Chain Security: DHS Should Test and Evaluate Container  Security Technologies Consistent with All Identified Operational  Scenarios To Ensure the Technologies Will Function as Intended.  GAO-10-887. (Washington D.C.: September 29, 2010).
Combating Nuclear Smuggling: Inadequate Communication and  Oversight Hampered DHS Efforts to Develop an Advanced Radiography  System to Detect Nuclear Materials. GAO-10-1041T. (Washington D.C.:  September 15, 2010).
Department of Homeland Security: Assessments of Selected Complex  Acquisitions. GAO-10-588SP. (Washington, D.C.: June 30, 2010).
Secure Border Initiative, DHS Needs to Reconsider Its Proposed  Investment in Key Technology Program. GAO-10-340. (Washington,  D.C.: May 5, 2010).
Secure Border Initiative: DHS Needs to Address Testing and  Performance Limitations That Place Key Technology Program at Risk.  GAO-10-158. (Washington, D.C.: January 29, 2010).
Aviation Security: DHS and TSA Have Researched, Developed, and  Begun Deploying Passenger Checkpoint Screening Technologies, but  Continue to Face Challenges. GAO-10-128. (Washington, D.C.: October  7, 2009).
Combating Nuclear Smuggling: Lessons Learned from DHS Testing of  Advanced Radiation Detection Portal Monitors. GAO-09-804T.  (Washington, D.C.: June 25, 2009).
Combating Nuclear Smuggling: DHS Improved Testing of Advanced  Radiation Detection Portal Monitors, but Preliminary Results Show Limits  of the New Technology. GAO-09-655. (Washington, D.C.: May 21, 2009).
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.