Aviation Security: Status of TSA's Acquisition of Technology for Screening Passenger Identification and Boarding Passes
Released: 2012-06-19T00:00:00

Letter

CAT/BPSS Is in the Operational Testing and Evaluation Phase, and the Life Cycle Cost Estimate Is Not Fully Consistent with Best Practices

CAT/BPSS, which is part of TSA’s Passenger Screening Program, has undergone initial testing and is in the operational testing and evaluation phase of acquisition, according to TSA. The goal of CAT/BPSS is to deploy a computerized system that will read and analyze data and embedded security features on every passenger’s identification and some boarding passes, and to identify fraudulent credentials and boarding passes. In 2011, TSA conducted qualification testing of this system at its System Integration Facility at Washington Reagan National Airport, including testing the systems against more than 530 genuine and fraudulent documents, such as state-issued driver’s licenses, passports, and military identification cards, according to TSA. The technology is designed to automatically compare a passenger’s identification with a set of embedded security features to seek to identify indicators of fraud and concurrently ensure that the information on the identification and boarding pass matches. This system is intended to help ensure that identity credentials and boarding passes presented at the checkpoint have not been tampered with or fraudulently produced, and that the information on the boarding pass matches that of the identity credential. According to TSA, CAT/BPSS is to compare identity credentials with an internal database of more than 2,400 templates for various types of credentials and to check for certain embedded security features, then alert the operator of any discrepancies.

In September 2011, TSA awarded contracts for approximately $3.2 million, which included the purchase of 30 units from three different vendors. In April 2012, TSA began deploying units to three airports— George Bush Intercontinental in Houston, Luis Muñoz Marín International in San Juan, and Washington Dulles International—in preparation for initial operational testing. TSA officials said that those airports were selected, in part, because of their high passenger volume and experience with detecting fraudulent documents. In preparation for initial testing, TSA tested the performance of its current process for comparison purposes. TSA is also training personnel on the CAT/BPSS systems, collecting preliminary data on system performance and availability, and assessing the adequacy of the concept of operations and standard operating procedures. According to TSA officials, these efforts will allow travel document checkers at the three airports to test the three systems in an operational environment and provide feedback on the systems’ performance. During operational testing, TSA plans to assess the systems’ performance against key performance parameters for detection, passenger throughput, and availability. Once operational testing is complete, TSA plans to produce a system evaluation report and recommend whether to move forward with the acquisition or make modifications. Vendors that successfully exit the operational testing phase will be eligible to compete for a contract to produce 1,400 units, according to TSA.

According to the life cycle cost estimate for the Passenger Screening Program, of which CAT/BPSS is a part, the estimated 20-year life cycle cost of CAT/BPSS is approximately $130 million based on a procurement of 4,000 units. As highlighted in our Cost Estimating and Assessment Guide, a reliable cost estimate has four characteristics—it is comprehensive, well documented, accurate, and credible. We reviewed TSA’s November 2011 life cycle cost estimate for the Passenger Screening Program and compared it with the four characteristics. Based on our assessment, the life cycle cost estimate is reasonably comprehensive and well documented. Regarding accuracy, the cost estimate assumes a 1 percent inflation rate from fiscal years 2015 through 2029, as compared with the historic inflation rates calculated for fiscal years 2009 through 2014, which ranged from 3.3 to 4.5 percent. If a larger inflation rate were used, costs would be much higher than what are currently estimated. In addition, we cannot make a determination as to the credibility of the life cycle cost estimate as it does not include a risk and uncertainty analysis or an independent cost estimate. The risk assessment would quantify risks and identify effects of changing key cost driver assumptions and factors. In the cost estimate, TSA indicates that it is pursuing the acquisition of risk analysis capability and plans on having such capabilities in time for the next life cycle cost estimate. Likewise, there is no evidence that an independent cost estimate was conducted by a group outside the acquiring organization to determine whether other estimating methods would produce similar results. TSA officials indicated that the agency is updating its life cycle cost estimate to include a risk and uncertainty analysis and independent cost estimate, but the document has not yet been approved.

The agency plans to expand the CAT/BPSS deployment schedule following successful implementation and testing in the selected airport environments. As of June 2012, TSA officials estimated that this could occur as soon as the end of this calendar year, depending on the results of the operational testing and evaluation phase.

Previously Identified Challenges TSA Faces in Overseeing Acquisition of Screening Technologies

Our past work has identified three key challenges related to TSA’s efforts to acquire and deploy technologies to address homeland security needs: (1) developing and meeting technology program requirements, (2) overseeing and conducting testing of new screening technologies, and (3) developing acquisition program baselines to establish initial cost, schedule, and performance parameters.

We have previously reported that DHS and TSA have faced challenges in developing and meeting program requirements when acquiring screening technologies, and that program performance cannot be accurately assessed without valid baseline requirements established at the program start. In June 2010, for example, we reported that more than half of the 15 DHS programs we reviewed awarded contracts to initiate acquisition activities without component or department approval of documents essential to planning acquisitions, setting operational requirements, or establishing acquisition program baselines. We made a number of recommendations to help address issues related to these procurements. DHS generally agreed with these recommendations and, to varying degrees, has begun taking actions to address them. We currently have ongoing work related to this area and we plan to report the results later this fall.fully follow DHS acquisition policies when acquiring advanced imaging At the program level, in May 2012, we reported that TSA did not technology (AIT), or body scanners, which resulted in DHS approving full AIT deployment without full knowledge of TSA’s revised specifications.

As a result, we found that TSA procured and deployed a technology that met evolving requirements, but not the initial requirements included in its key acquisition requirements document that the agency initially determined were necessary to enhance the aviation system. We recommended that TSA develop a road map that outlines vendors’ progress in meeting all key performance parameters. DHS agreed with our recommendation and has begun taking action to address it.

We have also reported on DHS and TSA challenges in overseeing and testing new screening technologies, which can lead to costly redesign and rework at a later date. Addressing such problems before moving to the acquisition phase can help agencies better manage costs. For example, in October 2009, we reported that TSA had deployed explosives trace portals, a technology for detecting traces of explosives on passengers at airport checkpoints, in January 2006 even though TSA officials were aware that tests conducted during 2004 and 2005 on earlier models of the portals suggested the portals did not demonstrate reliable performance in an airport environment. In June 2006, TSA halted deployment of the explosives trace portals because of performance problems and high installation costs. In our 2009 report, we recommended that, to the extent feasible, TSA ensure that tests are completed before deploying new checkpoint screening technologies to airports. DHS concurred with the recommendation and has taken action to address it, such as requiring more-recent technologies to complete both laboratory and operational tests prior to deployment.

GAO, Aviation Security: DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges, GAO-10-128 (Washington, D.C.: Oct. 7, 2009). cost and schedule can provide useful indicators of the health of acquisition programs. For example, we reported in April 2012 that TSA has not had a DHS-approved acquisition program baseline since the inception of the Electronic Baggage Screening Program (EBSP) more than 8 years ago. Further, DHS did not require TSA to complete an acquisition program baseline until November 2008. According to TSA officials, they have twice submitted an acquisition program baseline to DHS for approval—first in November 2009 and again in February 2011. An approved baseline would provide DHS with additional assurances that TSA’s approach is appropriate and that the capabilities being pursued are worth the expected costs. In November 2011, because TSA did not have a fully developed life cycle cost estimate as part of its acquisition program baseline for the EBSP, DHS instructed TSA to revise the life cycle cost estimates as well as its procurement and deployment schedules to reflect budget constraints. DHS officials told us that they could not approve the acquisition program baseline as written because TSA’s estimates were significantly over budget. TSA officials stated that TSA is currently working with DHS to amend the draft program baseline and plans to resubmit the revised acquisition program baseline before the next Acquisition Review Board meeting, which is planned for July or August 2012. Establishing and approving a program baseline, as DHS and TSA plan to do for the EBSP, could help DHS assess the program’s progress in meeting its goals and achieve better program outcomes.

Our prior work on TSA acquisition management identified oversight problems that have led to cost increases, delivery delays, and other operational challenges for certain assets, such as EBSP, but TSA has also taken several steps to improve its acquisition management. For example, while we continue to find that some TSA acquisition programs do not have key documents needed for properly managing acquisitions, CAT/BPSS has a DHS-approved mission needs statement, operational requirements document, and acquisition program baseline.

Congressional Oversight Issues

This hearing provides an opportunity for congressional stakeholders to focus a dialogue on how to continue a sufficient level of oversight of the CAT/BPSS acquisition and implementation and other key components of the Passenger Screening Program. For example, relevant questions that could be raised include the following:

To what extent, if any, have key performance parameters changed during the course of the acquisition, and how will these changes affect security and efficiency at the checkpoint? What would be TSA’s strategy if vendors have difficulty meeting the key performance parameters?

How will TSA ensure that implementation of the system addresses the security vulnerabilities previously identified?

What confidence does TSA have in its cost estimates and how is the agency mitigating the risk of cost escalation or schedule delays? In managing limited resources to mitigate a potentially unlimited range of security threats, how does CAT/BPSS fit into TSA’s broader aviation security strategy? What cost-benefit and related analyses, if any, are being used to guide TSA decision makers?

These types of questions and related issues warrant ongoing consideration by TSA management and continued oversight by congressional stakeholders.

Chairman Rogers, Ranking Member Jackson Lee, and Members of the Committee, this concludes my prepared statement. I look forward to responding to any questions that you may have.

GAO Contact and Staff Acknowledgment

For questions about this statement, please contact Steve Lord at (202) 512-4379 or lords@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this statement include Jessica Lucas-Judy, Assistant Director; Carissa Bryant; Jennifer Echard; Laurier Fish; Tom Lombardi; and Katherine Trimble. Key contributors for the previous work that this testimony is based on are listed within each individual product.

Related GAO Products

Homeland Security: DHS and TSA Face Challenges Overseeing Acquisition of Screening Technologies. GAO-12-644T. Washington, D.C.: May 9, 2012.

Checked Baggage Screening: TSA Has Deployed Optimal Systems at the Majority of TSA-Regulated Airports, but Could Strengthen Cost Estimates. GAO-12-266. Washington, D.C.: April 27, 2012.

Transportation Security Administration: Progress and Challenges Faced in Strengthening Three Key Security Programs. GAO-12-541T. Washington D.C.: March 26, 2012.

Homeland Security: DHS and TSA Acquisition and Development of New Technologies. GAO-11-957T. Washington, D.C.: September 22, 2011.

Aviation Security: TSA Has Made Progress, but Additional Efforts Are Needed to Improve Security. GAO-11-938T. Washington, D.C.: September 16, 2011.

Department of Homeland Security: Progress Made and Work Remaining in Implementing Homeland Security Missions 10 Years after 9/11. GAO-11-881. Washington, D.C.: September 7, 2011.

Homeland Security: DHS Could Strengthen Acquisitions and Development of New Technologies. GAO-11-829T. Washington, D.C.: July 15, 2011.

Aviation Security: TSA Has Taken Actions to Improve Security, but Additional Efforts Remain. GAO-11-807T. Washington, D.C.: July 13, 2011.

Aviation Security: TSA Has Enhanced Its Explosives Detection Requirements for Checked Baggage, but Additional Screening Actions Are Needed. GAO-11-740. Washington, D.C.: July 11, 2011.

Homeland Security: Improvements in Managing Research and Development Could Help Reduce Inefficiencies and Costs. GAO-11-464T. Washington D.C.: March 15, 2011.

High-Risk Series: An Update. GAO-11-278. Washington D.C.: February 16, 2011.

Department of Homeland Security: Assessments of Selected Complex Acquisitions. GAO-10-588SP. Washington, D.C.: June 30, 2010.

Aviation Security: Progress Made but Actions Needed to Address Challenges in Meeting the Air Cargo Screening Mandate. GAO-10-880T. Washington, D.C.: June 30, 2010.

Aviation Security: TSA Is Increasing Procurement and Deployment of Advanced Imaging Technology, but Challenges to This Effort and Other Areas of Aviation Security Remain. GAO-10-484T. Washington, D.C.: March 17, 2010.

Aviation Security: DHS and TSA Have Researched, Developed, and Begun Deploying Passenger Checkpoint Screening Technologies, but Continue to Face Challenges. GAO-10-128. Washington, D.C.: October 7, 2009.

GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. GAO-09-3SP. Washington, D.C.: March 2009.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.