Homeland Security: DHS and TSA Face Challenges Overseeing Acquisition of Screening Technologies
Released: 2012-05-09T00:00:00

Letter

Background

Since the department’s creation in 2003, we have designated the implementation and transformation of DHS as high risk because DHS had to combine 22 agencies—several with major management challenges— into one department, and failure to effectively address DHS’s management and mission risks could have serious consequences for U.S. national and economic security. This high-risk area includes (1) challenges in strengthening DHS’s management functions—financial management, human capital, information technology (IT), and acquisition management—(2) the effect of those challenges on DHS’s mission implementation, and (3) challenges in integrating management functions within and across the department and its components. On the basis of our prior work, in September 2010 we identified and provided to DHS 31 actions and outcomes that are critical to addressing the challenges within the department’s management areas and in integrating those functions across the department. These key actions and outcomes include, among others, validating required acquisition documents in accordance with a department-approved, knowledge-based acquisition process.

The Aviation and Transportation Security Act (ATSA) established TSA as the federal agency with primary responsibility for securing the nation’s civil aviation system, which includes the screening of all passengers and property transported from and within the United States by commercial passenger aircraft. In accordance with ATSA, all passengers, their accessible property, and their checked baggage are screened pursuant to TSA-established procedures at more than 450 airports presently regulated for security by TSA. These procedures generally provide, among other things, that passengers pass through security checkpoints where they and their identification documents, and accessible property, are checked by transportation security officers (TSO), other TSA employees, or by private-sector screeners under TSA’s Screening Partnership Program.

TSA relies upon multiple layers of security to deter, detect, and disrupt persons posing a potential risk to aviation security. These layers include TSOs responsible for screening passengers and their carry-on baggage at passenger checkpoints, using technologies that include x-ray equipment, magnetometers, and Advanced Imaging Technology (AIT), among others. In response to the December 2009 attempted terrorist attack, TSA revised its procurement and deployment strategy for AIT, commonly referred to as full-body scanners, increasing the number of AIT units it planned to procure and deploy. TSA stated that AIT provides enhanced security benefits compared with walk-through metal detectors, such as enhanced detection capabilities for identifying nonmetallic threat objects and liquids. AIT produces an image of a passenger’s body that a screener interprets. The image identifies objects, or anomalies, on the outside of the physical body but does not reveal items beneath the surface of the skin, such as implants. As of May 2012, TSA has deployed more than 670 AIT units to approximately 170 airports and reported that it plans to deploy a total of about 1,250 AIT units. In January 2012, we issued a classified report on TSA’s procurement and deployment of AIT that addressed the extent to which (1) TSA followed DHS acquisition guidance when procuring AIT and (2) deployed AIT units are effective at detecting threats. Another layer of security is checked-baggage screening, which uses technology referred to as explosive detection systems (EDS) and explosives trace detection (ETD).

DHS and TSA Have Experienced Challenges in Developing and Meeting Key Performance Requirements for Various Screening Technologies

Our past work has found that technology program performance cannot be accurately assessed without valid baseline requirements established at the program start. Without the development, review, and approval of key acquisition documents, such as the mission need statement and operational requirements document, agencies are at risk of having poorly defined requirements that can negatively affect program performance and contribute to increased costs. For example, in June 2010, we reported that more than half of 15 DHS programs we reviewed awarded contracts to initiate acquisition activities without component or department approval of documents essential to planning acquisitions, setting operational requirements, or establishing acquisition program baselines.currently have ongoing work related to this area and we plan to report the results later this year. We made a number of recommendations to help address issues related to these procurements as discussed below. DHS has generally agreed with these recommendations and, to varying degrees, has taken actions to address them.

In addition, our past work has found that TSA faces challenges in identifying and meeting program requirements in some of its aviation security programs. For example:

We reported in January 2012 that TSA did not fully follow DHS acquisition policies when acquiring AIT, which resulted in DHS approving full AIT deployment without full knowledge of TSA’s revised specifications. Specifically, DHS’s Acquisition Directive 102 required TSA to notify DHS’s Acquisition Review Board (ARB) if AIT could not meet any of TSA’s five key performance parameters (KPP) or if TSA changed a KPP during qualification testing. Senior TSA officials acknowledged that TSA did not comply with the directive’s requirements, but stated that TSA still reached a “good decision” in procuring AIT and that the ARB was fully informed of the program’s changes to its KPPs. Further, TSA officials stated that the program was not bound by the directive because it was a new acquisition process and they believed that the ARB was not fully functioning at the time. DHS officials stated that the ARB discussed the changed KPP but did not see the documents related to the change and determined that TSA must update the program’s key acquisition document, the Acquisition Program Baseline, before TSA could deploy AIT units. However, we reported that, according to a February 2010 acquisition decision memorandum from DHS, the ARB approved TSA for full-scale production without reviewing the changed KPP. DHS officials stated that the ARB should have formally reviewed changes made to the KPP to ensure that TSA did not change it arbitrarily. According to TSA, it should have submitted its revised requirements for approval, but it did not because there was confusion as to whether DHS should be informed of all changes. We had previously reported that programs procuring new technologies with fluctuating requirements will have a difficult time ensuring that the acquisition is meeting program needs. DHS acquisition oversight officials agreed that changing key requirements is not a best practice for system acquisitions already under way. As a result, we found that TSA procured and deployed a technology that met evolving requirements, but not the initial requirements included in its key acquisition requirements document that the agency initially determined were necessary to enhance the aviation system. We recommended that TSA should develop a roadmap that outlines vendors’ progress in meeting all KPPs. DHS agreed with our recommendation.

In July 2011, we reported that TSA revised its EDS requirements to better address current threats, and plans to implement these requirements in a phased approach. However, we reported that some number of EDS machines in TSA’s checked baggage screening fleet are configured to detect explosives at the levels established in the 2005 requirements. The remaining EDS machines are configured to detect explosives at 1998 levels. When TSA established the 2005 requirements, it did not have a plan with the appropriate time frames needed to deploy EDSs to meet the requirements. To help ensure that TSA’s checked baggage screening machines are operating most effectively, we recommended that TSA develop a plan to deploy EDSs to meet the most recent explosive-detection requirements and ensure that the new machines, as well as machines deployed in airports, are operated at the levels in established requirements. DHS concurred with our recommendation and has begun taking action to address it; for example, DHS reported that TSA has developed a plan to evaluate its current fleet of EDSs to determine the extent to which they comply with these requirements. However, our recommendation is intended to ensure that TSA operate all EDSs at airports at the most recent requirements. Until TSA develops a plan identifying how it will approach the upgrades for currently deployed EDSs—and the plan includes such items as estimated costs and the number of machines that can be upgraded—it will be difficult for TSA to provide reasonable assurance that its upgrade approach is feasible or cost effective.

DHS and TSA Have Encountered Challenges in Overseeing and Testing New Screening Technologies

Our prior work has also shown that not resolving problems discovered during testing can sometimes lead to costly redesign and rework at a later date. Addressing such problems before moving to the acquisition phase can help agencies better manage costs. Specifically: In January 2012, we reported that TSA began deploying AIT before it received approval for how it would test AIT. For example, DHS’s Acquisition Directive 102 required DHS to approve testing and evaluation master plans—the documents that ensure that programs are tested appropriately—prior to testing. However, we found that DHS did not approve TSA’s testing and evaluation master plan until January 2010, after TSA had completed qualification and operational tests and DHS had already approved TSA for full AIT deployment. According to DHS, the DHS Director of Operational Testing and Evaluation assessed the testing of AIT prior to the September 2009 ARB meeting and recommended approving the decision to procure AIT at that meeting, even though the ARB did not approve its testing plans. Additionally, we reported that DHS approved TSA’s AIT deployment in September 2009, on the basis of laboratory-based qualification testing results and initial field-based operational testing results that were not completed until later that year. According to DHS officials, the department initially had challenges providing effective oversight to projects already engaged in procurement when the directive was issued. For example, they noted that TSA had begun conducting qualification testing in 2009, but DHS’s first AIT oversight meeting under the new directive was not until later that year. As a result, we reported that TSA procured AIT without DHS’s full oversight and approval or knowledge of how TSA would test and evaluate AIT.

In July 2011, we reported that TSA revised the explosive detection requirements for EDS checked baggage screening machines in 2005 though it did not begin operating EDS systems to meet these 2005 requirements until 2009. We also reported that TSA made additional revisions to the EDS requirements in January 2010 but experienced challenges in collecting explosives data on the physical and chemical properties of certain explosives needed by vendors to develop EDS detection software to meet the 2010 requirements.also needed by TSA for testing the machines to determine whether they meet established requirements prior to their procurement and deployment to airports. TSA and S&T have experienced these challenges because of problems associated with safely handling and consistently formulating some explosives, which have also resulted in problems carrying out the EDS procurement as planned. Further, TSA deployed a number of EDSs that had the software necessary to meet the 2005 requirements, but because testing to compare false-alarm rates had not been completed, the software was not activated, subsequently; these EDSs were detecting explosives at levels established in 1998. According to TSA officials, once completed, the results of this testing to compare false alarm rates would allow them to determine if additional staff are needed at airports to help resolve false alarms once the EDSs are configured to operate at a certain level of requirements. TSA officials told us that they planned to perform this testing as a part of the ongoing EDS acquisition. We recommended that TSA develop a plan to ensure that TSA has the explosives data needed for each of the planned phases of the 2010 EDS requirements before starting the procurement process for new EDSs or upgrades included in each applicable phase. DHS stated that TSA modified its strategy for the EDS’s competitive procurement in July 2010 in response to the challenges in working with the explosives for data collection by removing the data collection from the procurement process. TSA’s plan to separate the data collection from the procurement process is a positive step, but to fully address our recommendation, a plan is needed to establish a process for ensuring that data are available before starting the procurement process for new EDSs or upgrades for each applicable phase.

In June 2011 we reported that S&T’s Test & Evaluation and Standards Office, responsible for overseeing test and evaluation of DHS’s major acquisition programs, reviewed or approved test and evaluation documents and plans for programs undergoing testing, and conducted independent assessments for the programs that completed operational testing. DHS senior-level officials considered the office’s assessments and input in deciding whether programs were ready to proceed to the next acquisition phase. However, the office did not consistently document its review and approval of components’ test agents—a government entity or independent contractor carrying out independent operational testing for a major acquisition. We recommended, among other things, that S&T develop mechanisms to document its review of component acquisition documentation. DHS concurred and reported actions underway to address them.

In October 2009, we reported that TSA deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, in January 2006 even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. In June 2006, TSA halted deployment of the explosives trace portals because of performance problems and high installation costs. In our 2009 report, we recommended that, to the extent feasible, TSA ensure that tests are completed before deploying new checkpoint screening technologies to airports. DHS concurred with the recommendation and has taken action to address it, such as requiring more-recent technologies to complete both laboratory and operational tests prior to deployment.

DHS and TSA Have Experienced Challenges Identifying Acquisition Program Baselines, Program Schedules, and Costs

We have found that realistic acquisition program baselines with stable requirements for cost, schedule, and performance are among the factors that are important to successful acquisitions delivering capabilities within cost and schedule. Our prior work has found that program performance metrics for cost and schedule can provide useful indicators of the health of acquisition programs and, when assessed regularly for changes and the reasons that cause changes, such indicators can be valuable tools for improving insight and oversight of individual programs as well as the total portfolio of major acquisitions.be accurately assessed without valid baseline requirements established Importantly, program performance cannot at the program start, particularly those that establish the minimum acceptable threshold required to satisfy user needs. According to DHS’s acquisition guidance, the program baseline is the contract between the program and departmental oversight officials and must be established at program start to document the program’s expected cost, deployment schedule, and technical performance. Establishing such a baseline at program start is important for defining the program’s scope, assessing whether all life-cycle costs are properly calculated, and measuring how well the program is meeting its goals. By tracking and measuring actual program performance against this baseline, management can be alerted to potential problems, such as cost growth or changing requirements, and has the ability to take early corrective action.

We reported in Aril 2012 that TSA has not had a DHS-approved acquisition program baseline since the inception of the EBSP program more than 8 years ago. Further, DHS did not require TSA to complete an acquisition program baseline until November 2008. According to TSA officials, they have twice submitted an acquisition program baseline to DHS for approval—first in November 2009 and again February 2011. An approved baseline will provide DHS with additional assurances that TSA’s approach is appropriate and that the capabilities being pursued are worth the expected costs. In November 2011, because TSA did not have a fully developed life-cycle cost estimate as part of its acquisition program baseline, DHS instructed TSA to revise the life cycle cost estimates as well as its procurement and deployment schedules to reflect budget constraints. DHS officials told us that they could not approve the acquisition program baseline as written because TSA’s estimates were significantly over budget. TSA officials stated that TSA is currently working with DHS to amend the draft program baseline and plans to resubmit the revised acquisition program baseline before the next Acquisition Review Board meeting, which is currently planned for July 2012. Establishing and approving a program baseline, as DHS and TSA currently plan to do for the EBSP, could help DHS assess the program’s progress in meeting its goals and achieve better program outcomes.

In our 2010 report of selected DHS acquisitions, 12 of 15 selected DHS programs we reviewed exhibited schedule delays and cost growth beyond initial estimates. We noted that DHS acquisition oversight officials have raised concerns about the accuracy of cost estimates for most major programs, making it difficult to assess the significance of the cost growth we identified. Leading practices state that the success of a large-scale system acquisition, such as the TSA’s EDS acquisition, depends in part on having a reliable schedule that identifies: (1) when the program’s set of work activities and milestone events will occur, (2) how long they will take, and (3) how they are related to one another. Leading practices also call for the schedule to expressly identify and define the relationships and dependencies among work elements and the constraints affecting the start and completion of work elements. Additionally, best practices indicate that a well-defined schedule also helps to identify the amount of human capital and fiscal resources that are needed to execute an acquisition.

We reported in January 2012 that TSA did not have plans to require vendors to meet milestones used during the AIT acquisition. We recommended that TSA should develop a roadmap that outlines vendors’ progress in meeting all KPPs because it is important that TSA convey vendors’ progress in meeting those requirements and full costs of the technology to decision makers when making deployment and funding decisions. TSA reported that it hoped vendors would be able to gradually improve meeting KPPs for AIT over time. We reported that TSA would have more assurance that limited taxpayer resources are used effectively by developing a roadmap that specifies development milestones for the technology and having DHS acquisition officials approve this roadmap. DHS agreed with our recommendation.

In July 2011, we reported that TSA had established a schedule for the acquisition of EDS machines but it did not fully comply with leading practices, and TSA had not developed a plan to upgrade its EDS fleet to meet the current explosives detection requirements. These leading practices state that the success of a large-scale system acquisition, such as TSA’s EDS acquisition, depends in part on having a reliable schedule that identifies when the program’s set of work activities and milestone events will occur, amongst other things. For example, the schedule for the EDS acquisition is not reliable because it does not reflect all planned program activities and does not include a timeline to deploy EDSs or plans to procure EDSs to meet subsequent phases of explosive detection requirements. We stated that developing a reliable schedule would help TSA better monitor and oversee the progress of the EDS acquisition. DHS concurred with our recommendation to develop and maintain a schedule for the entire EBSP in accordance with the leading practices identified by GAO for preparing a schedule. DHS commented that TSA had already begun working with key stakeholders to develop and define requirements for a schedule and to ensure that the schedule aligns with the best practices outlined by GAO.

In April 2012, we reported that TSA’s methods for developing life cycle cost estimates for the EBSP did not fully adhere to best practices for developing these estimates. As highlighted in our past work, a high-quality, reliable cost estimation process provides a sound basis for making accurate and well-informed decisions about resource investments, budgets, assessments of progress, and accountability for results and thus is critical to the success of a program. We reported that TSA’s estimates partially met three characteristics and minimally met one characteristic of a reliable cost estimate.concurred with our recommendation that TSA ensure that its life cycle cost estimates conform to cost estimating best practices, and identified efforts underway to address it. DHS also acknowledged the importance of producing life cycle cost estimates that are comprehensive, well documented, accurate, and credible so that they can be used to support DHS funding and budget decisions.

DHS Has Efforts Underway to Strengthen Oversight of Technology Acquisitions

In part due to the problems we have highlighted in DHS’s acquisition process, the implementation and transformation of DHS remains on our high-risk list. DHS currently has several plans and efforts underway to address the high-risk designation as well as the more specific challenges related to acquisition and program implementation that we have previously identified. For example, DHS initially described an initiative in the January 2011 version of its Integrated Strategy for High Risk Management to establish a framework, the Integrated Investment Life Cycle Model (IILCM), for managing investments across its components and management functions; strengthening integration within and across those functions; and ensuring mission needs drive investment decisions.

The department seeks to use the IILCM to enhance resource decision making and oversight by creating new department-level councils to identify priorities and capability gaps, revising how DHS components and lines of business manage acquisition programs, and developing a common framework for monitoring and assessing implementation of investment decisions. We reported in March 2012 that, from the time DHS first reported on the IILCM initiative in January 2011 to its December 2011 revision of its high-risk strategy, the initiative had made little progress though DHS plans to begin using the IILCM by the end of September 2012.

In October 2011, to enhance the department’s ability to oversee major acquisition programs, DHS realigned the acquisition management functions previously performed by two divisions within the Office of Chief Procurement Officer to establish the Office of Program Accountability and Risk Management (PARM). PARM, which is responsible for program governance and acquisition policy, serves as the Management Directorate’s executive office for program execution and works with DHS leadership to assess the health of major acquisitions and investments. To help with this effort, PARM is developing a database, known as the Decision Support Tool, intended to improve the flow of information from component program offices to the Management Directorate to support its governance efforts. DHS reported in its December 2011 Integrated Strategy for High Risk Management that senior executives are not confident enough in the data to use the Decision Support Tool developed by PARM to help make acquisition decisions. However, DHS’s plans to improve the quality of the data in this database are limited. At this time, PARM only plans to check the data quality in preparation for key milestone meetings in the acquisition process. This could significantly diminish the Decision Support Tool’s value because users cannot confidently identify and take action to address problems meeting cost or schedule goals prior to program review meetings.

We reported in March 2012 that DHS has made progress strengthening its management functions, but the department faces considerable challenges. Specifically, DHS has faced challenges overseeing the management, testing, acquisition, and deployment of various technology programs including AIT and EDS. Going forward, DHS needs to continue implementing its Integrated Strategy for High Risk Management and show measurable, sustainable progress in implementing its key management initiatives and corrective actions and achieving outcomes including those related to acquisition management. DHS reported that it plans to revise its Integrated Strategy for High Risk Management in June 2012, which includes management initiatives and corrective actions to address acquisition management challenges, among other management areas. We will continue to monitor and assess DHS’s implementation and transformation efforts through our ongoing and planned work, including the 2013 high-risk update that we expect to issue in early 2013.

Chairmen Issa and Mica, Ranking Members Cummings and Rahall, and members of the committees, this concludes my prepared statement. I would be pleased to respond to any questions that you may have.

GAO Contact and Staff Acknowledgments

For questions about this statement, please contact Steve Lord at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement include Dave Bruno, Assistant Director; Scott Behen, Analyst-in-Charge; Emily Gunn, and Katherine Trimble. Other contributors include: David Alexander, Tom Lombardi, Jason Lee, Linda Miller, and Jerry Seigler. Key contributors for the previous work that this testimony is based on are listed within each individual product.

Related GAO Products

Checked Baggage Screening: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated Airports, but Could Strengthen Cost Estimates. GAO-12-266. Washington D.C.: April 27, 2012.

Transportation Security Administration: Progress and Challenges Faced in Strengthening Three Key Security Programs. GAO-12-541T. Washington D.C.: March 26, 2012 Aviation Security: TSA Has Made Progress, but Additional Efforts Are Needed to Improve Security. GAO-11-938T. Washington, D.C.: September 16, 2011.

Department of Homeland Security: Progress Made and Work Remaining in Implementing Homeland Security Missions 10 Years after 9/11. GAO-11-881. Washington, D.C.: September 7, 2011.

Homeland Security: DHS Could Strengthen Acquisitions and Development of New Technologies. GAO-11-829T. Washington, D.C.: July 15, 2011.

Aviation Security: TSA Has Taken Actions to Improve Security, but Additional Efforts Remain. GAO-11-807T. Washington, D.C.: July 13, 2011.

Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for Checked Baggage, but Additional Screening Actions Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011.

Homeland Security: Improvements in Managing Research and Development Could Help Reduce Inefficiencies and Costs. GAO-11-464T. Washington D.C.: March 15, 2011.

High-Risk Series: An Update. GAO-11-278. Washington D.C.: February 16, 2011.

Department of Homeland Security: Assessments of Selected Complex Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010.

Aviation Security: Progress Made but Actions Needed to Address Challenges in Meeting the Air Cargo Screening Mandate. GAO-10-880T. Washington, D.C.: June 30, 2010.

Aviation Security: TSA Is Increasing Procurement and Deployment of Advanced Imaging Technology, but Challenges to This Effort and Other Areas of Aviation Security Remain. GAO-10-484T. Washington, D.C.: March 17, 2010.

Aviation Security: DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges. GAO-10-128. Washington, D.C.: October 7, 2009.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.