Transportation Security: Status of GAO Recommendations on TSA's Security-Related Technology Acquisitions
Released: 2016-02-17T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: Within DHS, TSA is the federal agency responsible for securing domestic transportation systems. The Transportation Security Acquisition Reform Act (TSARA) contains a provision for GAO to submit a report to Congress containing an assessment of TSA's implementation of GAO recommendations regarding the acquisition of security-related technology. This report addresses (1) the status of TSA's implementation of relevant GAO recommendations since 2003 and the characteristics of those recommendations and (2) benefits realized by TSA in implementing those recommendations.

GAO determined the number and status of recommendations made to TSA from October 1, 2003, after TSA had become a part of the newly created DHS, through July 31, 2015, as well as the benefits derived from the recommendations TSA implemented, using an internal database that GAO maintains on the status of recommendations it makes. GAO specifically identified recommendations related to the acquisition of technology that helps TSA prevent or defend against threats to domestic transportation systems. TSA concurred with GAO's list of recommendations. TSA also provided technical comments on a draft of this report which GAO incorporated as appropriate. DHS did not provide formal comments.

section_title: What GAO Found
paragraphs: The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) has implemented 51 of the 58 recommendations GAO made from October 1, 2003, through July 31, 2015, to improve TSA's acquisition of security-related technology. GAO's recommendations generally directed TSA to develop a plan; conduct an analysis; or implement a program, policy, or procedure. For example, in March 2014, GAO recommended that TSA establish protocols to capture operational data on secondary screening of passengers at the checkpoint, to help ensure screening officers' performance.

TSA has not implemented 7 of the 58 recommendations. GAO closed 4 of the 7 recommendations because TSA stated that it would not take action or GAO determined that it was unlikely that TSA would take action. These recommendations were related to establishing the effectiveness of canine screening, conducting technical assessments to strengthen airport perimeter security and Advanced Imaging Technology. The remaining 3 open recommendations are focused on improving Advanced Imaging Technology operations. GAO continues to believe these recommendations are valid and should be fully addressed.

Since fiscal year 2003, GAO has identified approximately $1.7 billion in financial benefits, largely representing funds that TSA used to support other programs and activities, based on implementation of GAO's recommendations as well as findings from related GAO reports and testimonies on security-related technology acquisitions. GAO has also documented additional benefits, including programmatic and process improvements to TSA's programs, such as improvements to TSA's Electronic Baggage Screening Program's cost estimating processes.

section_title: Letter
paragraphs: The Transportation Security Acquisition Reform Act (TSARA), enacted in  December 2014, mandates reforms to acquisition policies and processes  that the Department of Homeland Security’s (DHS) Transportation  Security Administration (TSA) applies to its acquisition of security-related  technologies. Established by the Aviation and Transportation Security  Act (ATSA), TSA is the federal agency with primary responsibility for  securing the nation’s transportation systems, and to fulfill this mission,  acquires and uses comprehensive, effective, and efficient security-related  technologies. Initially established as an agency within the Department of  Transportation (DOT), TSA transferred to the newly created DHS in  March 2003 pursuant to the Homeland Security Act of 2002.. From  October 2011 through June 2015, TSA obligated approximately $4.6  billion to develop, purchase, and maintain security-related technologies,  with most of this amount supporting aviation-related acquisitions. For  example, two major TSA aviation security–related technology acquisitions  programs are the Passenger Screening Program (PSP) and the  Electronic Baggage Screening Program (EBSP). From fiscal year 2002  through August 2015, TSA reported obligating approximately $2.5 billion  and $10.9 billion respectively on planning, acquisitions, and maintenance  for these two programs. As TSA has obligated billions of dollars on  security-related technology acquisitions, it has also faced several  acquisitions challenges. For example, in 2009, we found that TSA lacked  assurances that its investment of $795 million in screening technologies  addressed the highest priority security needs at airport passenger  checkpoints. Also, in 2011, we found that TSA’s explosives detection  systems were not configured to meet the most current detection  requirements. GAO made numerous recommendations to discuss these  challenges. These recommendations are discussed in more detail later in  this report.

Section 4 of TSARA included a provision for GAO to submit a report to  Congress containing an assessment of TSA’s implementation of  recommendations GAO made on or before December 18, 2014— TSARA’s date of enactment—regarding TSA’s acquisition of security- related technology. Specifically, this report addresses the following  questions:  1.  What is the status of TSA’s implementation of GAO’s  recommendations relating to the acquisition of security-related  technology since 2003 and what are the characteristics of those  recommendations?  2.  What benefits have been realized by TSA implementing GAO’s  recommendations relating to the acquisition of security-related  technology?

To determine the status of TSA’s implementation of GAO’s  recommendations relating to security-related technology acquisitions, we  reviewed our body of work on TSA and, as appropriate, DHS, in order to  identify recommendations related to the acquisition of any technology that  assists TSA in the prevention of, or defense against, threats to the U.S.  transportation systems, including threats to people, property, and  information. For the purpose of this review, the acquisition of security- related technology encompasses acquisitions that support TSA activities  to screen passengers, checked baggage, and other property (including  cargo), including, but not limited to, Advanced Imaging Technology (AIT);  explosives detection systems (EDS); explosives trace detection (ETD)  machines and canines; the vetting and prescreening of passengers; and  services to maintain, modify, or improve such security-related  technologies. The acquisition of security-related technologies, for  purposes of this review, does not encompass acquisitions that relate  primarily to training on the use of security-related technologies; the  provision of screening services at an airport, such as the Screening  Partnership Program; or identification and access media, such as the  Transportation Worker Identification Credential.

To determine the number and status of the recommendations we made to  TSA from October 1, 2003, through July 31, 2015, we reviewed  information in our internal database that maintains information on the  status of recommendations we have made to all agencies. We selected  October 1, 2003, as the starting point for our review so we could include  all recommendations made to TSA after TSA transferred from the  Department of Transportation to the newly formed DHS in March 2003.  We relied on this database for the information contained in this report on  financial and non-financial benefits. Once TSA advised GAO that it had  taken action on a recommendation and we verified that action, we used a  multi-step process involving several non-GAO information sources, such  as agency program documentation, to estimate the related financial  benefits. The financial benefits that we report are net benefits—that is,  estimates of financial benefits that have been reduced by the estimated  costs associated with taking the action that we recommended. We  convert all estimates involving past and future years to their net present  value and use actual dollars to represent estimates involving the current  year. Financial benefit amounts vary depending on the nature of the  benefit.

To ensure the reliability of the recommendations data, we reviewed  documentation about the database used to produce the data and  interviewed GAO staff members responsible for maintaining and updating  the database. TSA officials also reviewed and concurred with our list of  recommendations. We determined the data to be sufficiently reliable for  the purposes of this report.

We conducted this performance audit from April 2015 through February  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Many of our reports and testimonies include recommendations that, if  acted upon, may result in tangible benefits for the U.S. taxpayer by  improving the federal government’s efficiency, effectiveness, and  accountability. Implemented recommendations can result in financial or  nonfinancial benefits for the federal government. An estimated financial  benefit is based on actions taken in response to our recommendations,  such as reducing government expenditures, increasing revenues, or  reallocating funds to other areas. For example, in fiscal year 2015, our  work resulted in $74.4 billion in financial benefits across the federal  government. Other benefits that result from our work cannot be measured  in dollar terms, and we refer to them as nonfinancial or other benefits.  These benefits are linked to specific recommendations or other work that  we completed over several years and could include improvements to  agency programs, processes, and policies. During fiscal year 2015, we  recorded a total of 1,286 other benefits government-wide that resulted  from our work, including improved services to the public and government  business operations.

As part of our responsibilities under generally accepted government  auditing standards, we follow up on recommendations we have made and  report their status to Congress. Agencies also have a responsibility to  monitor and maintain accurate records on the status of our  recommendations. After issuing a report, we follow up with reviewed  agencies at least once a year to determine the extent to which our  recommendations have been implemented and the benefits that have  been realized. During this follow-up, we identify what additional actions, if  any, would be needed to address our recommendations. A  recommendation is considered implemented when actions have been  taken that, consistent with our recommendation, address the issue or  deficiency we identified and upon which the recommendation is based.  Experience has shown that it takes time for some recommendations to be  implemented. For this reason, we actively track each unaddressed (i.e.,  open) recommendation for 4 years and review them to determine whether  implementation can be reasonably expected. The review includes  consideration of alternative strategies the agency may have for  implementing the recommendations. We will close the recommendation  as not implemented if TSA has indicated that it was not taking action or  we determined that it was unlikely that TSA would take action to close  these recommendations.

We maintain a publicly available database with information on the current  status of most open recommendations. The database allows searches  by agency, congressional committee, or key words and is available at  http://www.gao.gov/openrecs.html.

section_title: TSA Has Implemented 51 of 58 Recommendations Relating to the Acquisition of Security-Related Technology Aimed at Improving Agency Planning and Analysis
paragraphs: TSA has implemented 51 of the 58 recommendations we made from  October 1, 2003, through July 31, 2015, intended to improve TSA’s  acquisition of security-related technology, and has not implemented the  remaining 7 recommendations (see fig. 1).

The 58 recommendations relating to the acquisition of security-related  technology fall into three general categories: actions requiring the agency  to (1) develop a plan, process, protocol, or strategy; (2) implement a plan,  program, policy, procedure, or best practice; and (3) conduct a test,  study, or analysis. For example, in 2011 we recommended that TSA  develop a process to communicate information to EDS vendors in a  timely manner regarding EDS acquisition and to ensure that TSA takes a  comprehensive and cost-effective approach to procuring and deploying  EDS that meet the 2010 explosives detection requirements and any  subsequent revisions. Separately, in 2008 we recommended that TSA  fully incorporate best practices into developing Secure Flight life-cycle  cost and schedule estimates and develop a plan for mitigating cost and  schedule risks, among other things. In 2009, we recommended that  TSA conduct a cost-benefit analysis to assist in prioritizing investments in  new checkpoint screening technologies to help TSA take a  comprehensive, risk-informed approach to procuring and deploying  airport passenger screening technologies. As shown in figure 2, we  identified 31 recommendations for the first category, 7 recommendations  for the second category, and 20 recommendations for the third category.

TSA has not implemented 7 recommendations we made from fiscal year  2003 through July 31, 2015. Four of the 7 recommendations have been  closed, while 3 remain open. The 3 open recommendations are focused  on improving AIT operations, while the 4 closed recommendations were  related to establishing the effectiveness of canine screening, conducting  technical assessments to strengthen airport perimeter security, and AIT  operations. For example:  In 2014, we recommended that TSA establish protocols to facilitate  capturing operational data on secondary passenger screening at the  checkpoint to determine the extent to which rates of false alarms for  various AIT systems affect operational costs once Advanced Imaging  Technology-Automated Target Recognition (AIT-ATR) systems are  networked. TSA concurred with this recommendation. In its  comments on our report, TSA stated that it would monitor, update,  and report the results of its efforts to capture operational data and  evaluate its associated impacts on operational costs. When contacted  in November 2015 for an update on this recommendation, TSA  officials stated that they have taken steps toward implementing this  recommendation by evaluating the impact of false alarm rates on  operational costs (such as staffing) during testing for new AIT  systems. This recommendation remains open pending additional  actions by TSA to collect secondary screening data on an ongoing  basis, which could be used to obtain valuable insights on false alarm  rates and the resulting operational costs. By fully implementing the  recommendation, TSA could improve the overall performance of the  AIT system and make more informed decisions about checkpoint  screening.

In 2013, we recommended that TSA expand and complete testing, in  conjunction with the DHS Science and Technology Directorate, to  assess the effectiveness of passenger screening canines (PSC) and  conventional canines in all areas of an airport deemed appropriate by  TSA before deploying more passenger screening canine teams to  help (a) determine whether PSCs are effective at screening  passengers, and expenditures for PSC training are warranted and (b)  inform decisions about the type of canine teams to deploy and their  optimal locations in airports. TSA concurred with the  recommendation and took some action, but did not fully address the  recommendation. Specifically, in June 2014, TSA reported that it had  assessed PSC teams deployed to 27 airports, cumulating in a total of  1,048 tests. On the basis of these tests, TSA determined that PSC  teams are effective and should be deployed at the checkpoint queue.  However, when contacted in December 2014, officials reported that  they did not plan to expand or complete testing to compare the  effectiveness PSCs with the effectiveness of conventional canine  teams as we recommended, citing concerns about potential liability in  using conventional canines that have not been evaluated for their  suitability for screening passengers in an unfamiliar passenger  screening environment and the related risks to the program. We  disagreed and pointed out that conventional canines paired with  handlers already work in proximity with passengers since they patrol  airport terminals, including ticket counters and curbside areas. Given  that TSA does not plan on taking further action on this program, we  closed the recommendation as not implemented. However, we  continue to believe that the recommendation has merit and should be  fully implemented.

section_title: TSA’s Implementation of GAO Recommendations Has Resulted in Financial Benefits and Other Programmatic and Process Improvements
paragraphs: Since fiscal year 2003, we have identified approximately $1.7 billion in  financial benefits, largely representing funds that TSA used to support  other programs and activities, based on implementation of our  recommendations as well as findings from our related reports and  testimonies on security-related technology acquisitions. We have also  identified additional benefits, including programmatic and process  improvements to TSA’s programs stemming from implementation of our  recommendations and related work. The following are examples of the  financial, programmatic, and process benefits we identified.
subsections: section_title: Financial benefits:
paragraphs: In January 2012, we issued a report on TSA’s adherence to DHS  acquisition policy and efforts to test the effectiveness of AIT. Among  other things, we reported on the effectiveness of the systems and  recommended that TSA brief Congress. Congressional response to a  TSA briefing combined with our body of work on AIT resulted in TSA’s  decision to reduce the number of planned AIT purchases amounting  to approximately $1.4 billion.

In several reports and testimonies in fiscal year 2004, we reported on  delays and challenges in TSA’s development of the Computer- Assisted Passenger Prescreening System II (CAPPS II). We found  that TSA had not fulfilled statutory requirements concerning the  development and operations of CAPPS II. For example, TSA had not  determined the accuracy of the databases that would be used to  prescreen passengers, and had not conducted tests that would stress  the program and ensure system functionality. We made several  recommendations relating to developing project plans, including  schedules and estimated costs, conducting system testing; and  developing a process by which passengers can get erroneous  information corrected, among others. In part because of our initial  and subsequent evaluations, and congressional oversight hearings,  TSA reprogramed approximately $46 million in funding for CAPPS II  to other TSA activities in fiscal years 2003 through 2005. TSA  canceled CAPPS II development in August 2004 and, shortly after  that, announced plans to develop a successor passenger  prescreening program called Secure Flight. The projected program  funding for the canceled CAPPS II program resulted in total  programmatic savings of approximately $304 million.

section_title: Programmatic Improvements
paragraphs: In April 2012, we found that TSA had established cost estimates for  the EBSP to help identify total program cost, recapitalization cost, and  potential savings resulting from installing optimal systems, but its  processes for developing these estimates did not fully comply with  best practices. We recommended that in order to strengthen the  credibility, comprehensiveness, and reliability of TSA’s cost estimates  and related savings estimates for EBSP, TSA should follow cost- estimating best practices. In response, TSA implemented a  management directive that applies DHS guidance and best practices  from the GAO Cost Estimating and Assessment Guide and updated  its cost estimating best practices to include four characteristics of a  high quality and reliable cost estimate—comprehensive, well- documented, accurate, and credible. By implementing this  recommendation, TSA improved its ability to determine the cost of the  program and plan for the resources required to develop and manage  the EBSP.

In October 2009, we found that TSA had completed a strategic plan to  guide research, development, and deployment of passenger  checkpoint screening technologies; however, the plan was not risk- based. We recommended that TSA take a comprehensive, risk- informed approach to procuring and deploying technologies for airport  passenger checkpoint screening. Specifically, we recommended that,  to the extent feasible, TSA should complete operational tests and  evaluations before deploying screening technologies to airport  checkpoints. In response to our recommendation, in March 2010,  TSA updated its Aviation Modal Risk Assessment, which included a  comprehensive risk assessment of the potential for a terrorist attack  and implemented a test and evaluation process for all of its  technology procurements in accordance with DHS policy. TSA’s  actions increased its ability to successfully procure and deploy  passenger and checkpoint screening technologies.

section_title: Process Improvements
paragraphs: In 2011, we found that TSA did not effectively communicate in a  timely manner with vendors competing for EDS procurement. To help  ensure that TSA takes a comprehensive and cost-effective approach  to procuring and deploying EDS, in July 2011 we recommended that  TSA establish a process to communicate information to EDS vendors  in a timely manner about TSA’s EDS acquisitions, including changes  to the procurement schedule. In April 2012, TSA provided  information on a number of actions it had taken to improve  communication with EDS vendors, such as issuing 16 public  notifications that contained projected schedules and program updates.  In October 2012, TSA finalized its Explosives Detection System  Competitive Procurement Qualification Program Communications  Plan, which established a process for more timely communication with  vendors competing for EDS procurements. In addition, TSA used a  qualified products list in its EBSP acquisition plan that awarded  contracts only to precertified vendors. By establishing a process to  communicate with vendors in a timely manner, TSA was better  positioned to avoid delays and procurement cost overruns.

In May 2004, we found that TSA’s Office of Acquisition was at an  organizational level too low to effectively oversee the acquisition  process, coordinate acquisition activities, and enforce acquisition  policies effectively. The position of the office hindered its ability to help  ensure that TSA follows acquisition processes that enable the agency  to get the best value on goods and services. We recommended that  TSA elevate the Office of Acquisition to an appropriate level within  TSA to enable it to identify, analyze, prioritize, and coordinate agency- wide acquisition needs. In October 2004, TSA elevated its Office of  Acquisition to report directly to the Deputy Administrator of TSA. The  Office of Acquisition instituted an outreach program to provide  acquisition expertise to program offices on each area of the  acquisition life cycle. TSA also issued an Investment Review Process  guide in January 2005 that outlines acquisition personnel roles,  responsibilities, and procedures for conducting acquisition program  reviews at each key decision point. By taking these actions, DHS and  TSA were better positioned to make TSA’s acquisition process more  efficient and improved TSA’s ability to implement its acquisition  policies and procedures.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DHS for review and comment. DHS  did not provide formal comments but provided technical comments from  TSA which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional  committees and the Secretary of Homeland Security. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-7141 or GroverJ@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

section_title: Appendix I: Characteristics and Status of the Transportation Security Administration’s Implementation of GAO Recommendations
subsections: section_title: Product Number GAO-04-385
paragraphs: Recommendation  To address the challenges  associated with the  development, implementation,  and operation of the Computer  Assisted Passenger  Prescreening System (CAPPS  II), the Secretary of Homeland  Security should instruct the  Administrator of the  Transportation Security  Administration (TSA) to develop  plans identifying the specific  functionality that will be  delivered during each  increment of CAPPS II, the  specific milestones for  delivering this functionality, and  expected costs for each  increment.

To address the challenges  associated with the  development, implementation,  and operation of CAPPS II, the  Secretary of Homeland Security  should instruct the  Administrator of TSA to use  established plans to track  development progress to  ensure that promised  functionality is being delivered  on time and within established  cost estimates.

To address the challenges  associated with the  development, implementation,  and operation of CAPPS II, the  Secretary of Homeland Security  should instruct the  Administrator of TSA to develop  a schedule for critical security  activities, including finalizing  the security policy, the security  risk assessment, and system  certification and accreditation.

section_title: Product Number GAO-04-385
paragraphs: Recommendation  To address the challenges  associated with the  development, implementation,  and operation of CAPPS II, the  Secretary of Homeland Security  should instruct the  Administrator of TSA to develop  a strategy for mitigating the  high risk associated with  system and database testing  that ensures (1) accuracy  testing of commercial and  government databases is  conducted prior to the database  being used and (2) appropriate  stress testing is conducted to  demonstrate the system can  meet peak load requirements.

To address the challenges  associated with the  development, implementation,  and operation of CAPPS II, the  Secretary of Homeland Security  should instruct the  Administrator of TSA to develop  results-oriented performance  goals and measures to  evaluate the program’s  effectiveness, including  measures to assess  performance of the system in  generating reliable risk scores.

To help ensure that TSA  receives the goods and  services it needs at the best  value to the government, the  Secretary of Homeland Security  should direct the Administrator  of TSA to elevate the Office of  Acquisition to an appropriate  level within TSA to enable it to  identify, analyze, prioritize, and  coordinate agencywide  acquisition needs.

section_title: Product Number GAO-04-544
paragraphs: Recommendation  To help ensure that TSA  receives the goods and  services it needs at the best  value to the government, the  Secretary of Homeland Security  should direct the Administrator  of TSA to develop an adequate  system of internal controls,  performance measures, and  incentives to ensure that  policies and processes for  ensuring efficient and effective  acquisitions are implemented  appropriately.

To help ensure that TSA  receives the goods and  services it needs at the best  value to the government, the  Secretary of Homeland Security  should direct the Administrator  of TSA to direct the TSA  Human Capital Office to do the  following in coordination with  key offices in the Department of  Homeland Security: (1) assess  TSA’s current acquisition  workforce (as defined by the  Department of Homeland  Security) to determine the  number, skills, and  competencies of the workforce;  (2) identify any gaps in the  number, skills, and  competencies of the current  acquisition workforce; and (3)  develop strategies to address  any gaps identified, including  plans to attract, retain, and train  the workforce.

The Secretary of Homeland  Security should ensure that its  planned departmentwide  knowledge management  system provides TSA sufficient  data and analytic capability to  measure and analyze spending  activities and performance— and thereby highlight  opportunities to reduce costs  and improve service levels.

section_title: Product Number GAO-04-544
paragraphs: Recommendation  The Secretary of Homeland  Security should ensure that its  planned departmentwide  knowledge management  system provides TSA sufficient  data and analytic capability to  support effective oversight of  acquisitions.

To help ensure that TSA is able  to articulate and justify future  decisions on how best to  proceed with security  evaluations, fund and  implement security  improvements—including new  security technologies—and  implement additional measures  to reduce the potential security  risks posed by airport workers,  the Secretary of Homeland  Security should direct TSA’s  Administrator to develop and  provide Congress with a plan  for meeting the requirements of  the Aviation Transportation  Security Act (ATSA).

The Secretary of Homeland  Security should direct TSA’s  Administrator to develop and  provide Congress with a plan  for meeting the requirements of  ATSA by conducting  assessments of technology,  compile the results of these  assessments as well as  assessments conducted  independently by airport  operators, and communicate  the integrated results of these  assessments to airport  operators.

section_title: Product Number GAO-04-728
paragraphs: Recommendation  The Secretary of Homeland  Security should direct TSA’s  Administrator to develop and  provide Congress with a plan  for meeting the requirements of  ATSA by using the information  resulting from the security  evaluation and technology  assessment efforts cited above  as a basis for providing  guidance and prioritizing  funding to airports for  enhancing the security of the  commercial airport system as a  whole.

In developing the  comprehensive plan for  installing in-line explosives  detection systems (EDS)  baggage screening systems, as  directed by the fiscal year 2005  DHS Appropriation Act  Conference Report, and in  satisfying the requirements set  forth in the Intelligence Reform  and Terrorism Prevention Act of  2004, the Secretary of the  Department of Homeland  Security should direct the  Administrator of TSA to  systematically assess the costs  and benefits of deploying in-line  baggage screening systems at  airports that do not yet have in- line systems installed. As part  of this assessment, the  Administrator should identify  and prioritize the airports where  the benefits—in terms of cost  savings of baggage screening  operations and improved  security—of replacing stand- alone baggage screening  systems with in-line systems  are likely to exceed the costs of  the systems, or the systems are  needed to address security  risks or related factors.

section_title: Product Title
paragraphs: Recommendation  In developing the  comprehensive plan for  installing in-line EDS baggage  screening systems, as directed  by the fiscal year 2005 DHS  Appropriation Act Conference  Report, and in satisfying the  requirements set forth in the  Intelligence Reform and  Terrorism Prevention Act of  2004, the Secretary of the  Department of Homeland  Security should direct the  Administrator of TSA to  systematically assess the costs  and benefits of deploying in-line  baggage screening systems at  airports that do not yet have in- line systems installed. As part  of this assessment, the  Administrator should consider  the projected availability and  costs of baggage screening  equipment being developed  through research and  development efforts.

Recommendation  In developing the  comprehensive plan for  installing in-line EDS baggage  screening systems, as directed  by the fiscal year 2005 DHS  Appropriation Act Conference  Report, and in satisfying the  requirements set forth in the  Intelligence Reform and  Terrorism Prevention Act of  2004, the Secretary of the  Department of Homeland  Security should direct the  Administrator of TSA to  systematically assess the costs  and benefits of deploying in-line  baggage screening systems at  airports that do not yet have in- line systems installed. As part  of this assessment, the  Administrator should estimate  total funds needed to install in- line systems where appropriate,  including the federal funds  needed given different  assumptions regarding the  federal government and airport  cost-shares for funding the in- line systems.

Recommendation  In developing the  comprehensive plan for  installing in-line EDS baggage  screening systems, as directed  by the fiscal year 2005 DHS  Appropriation Act Conference  Report, and in satisfying the  requirements set forth in the  Intelligence Reform and  Terrorism Prevention Act of  2004, the Secretary of the  Department of Homeland  Security should direct the TSA  Administrator  to systematically  assess the costs and benefits  of deploying in-line baggage  screening systems at airports  that do not yet have in-line  systems installed. As part of  this assessment, the  Administrator should work  collaboratively with airport  operators, who are expected to  share the costs and benefits of  in-line systems, to collect data  and prepare the analyses  needed to develop plans for  installing in-line systems.

Systematic Planning  Needed to Optimize the  Deployment of Checked  Baggage Screening  Systems  assess the feasibility, expected  benefits, and costs of replacing  explosives trace detection  (ETD) machines with stand- alone EDS machines for  primary screening at those  airports where in-line systems  would not be either  economically justified or  justified for other reasons. In  conducting this assessment,  the Administrator should  consider the projected  availability and costs for  screening equipment being  developed through research  and development efforts.

section_title: Product Number GAO-05-356
paragraphs: Issue Date  March 28, 2005  To help manage risks  associated with Secure Flight’s  continued development and  implementation, and to assist  TSA in developing a framework  from which to support its efforts  in addressing congressional  areas of interest outlined in  Public Law 108-334, the  Secretary of the Department of  Homeland Security should  direct the Assistant Secretary of  TSA, to finalize the system  requirements document and the  concept of operations, and  develop detailed test plans to  help ensure that all Secure  Flight system functionality is  properly tested and evaluated.  These system documents  should address all system  functionality and include system  stress test requirements.

March 28, 2005  To help manage risks  associated with Secure Flight’s  continued development and  implementation, and to assist  TSA in developing a framework  from which to support its efforts  in addressing congressional  areas of interest outlined in  Public Law 108-334, the  Secretary of the Department of  Homeland Security should  direct the Assistant Secretary of  TSA to develop a plan for  establishing connectivity among  the air carriers, U.S. Customs  and Border Protection, and  TSA to help ensure the secure,  effective, and timely  transmission of data for use in  Secure Flight operations.

section_title: Product Number GAO-05-356
paragraphs: Issue Date  March 28, 2005  To help manage risks  associated with Secure Flight’s  continued development and  implementation, and to assist  TSA in developing a framework  from which to support its efforts  in addressing congressional  areas of interest outlined in  Public Law 108-334, the  Secretary of the Department of  Homeland Security should  direct the Assistant Secretary of  TSA, to develop reliable life- cycle cost estimates and  expenditure plans for Secure  Flight—in accordance with  guidance issued by the Office  of Management and Budget— to provide program managers  and oversight officials with  information needed to make  informed decisions regarding  program development and  resource allocations.

March 28, 2005  To help manage risks  associated with Secure Flight’s  continued development and  implementation, and to assist  TSA in developing a framework  from which to support its efforts  in addressing congressional  areas of interest outlined in  Public Law 108-334, the  Secretary of the Department of  Homeland Security should  direct the Assistant Secretary of  TSA, to develop results- oriented performance goals and  measures to evaluate the  effectiveness of Secure Flight  in achieving intended results in  an operational environment— as outlined in the Government  Performance and Results Act— including measures to assess  associated impacts on aviation  security.

section_title: Product Number GAO-05-356
paragraphs: Issue Date  March 28, 2005  To help manage risks  associated with Secure Flight’s  continued development and  implementation, and to assist  TSA in developing a framework  from which to support its efforts  in addressing congressional  areas of interest outlined in  Public Law 108-334, the  Secretary of the Department of  Homeland Security should  direct the Assistant Secretary of  TSA to, prior to achieving initial  operational capability, finalize  policies and issue associated  documentation specifying how  the Secure Flight program will  protect personal privacy,  including addressing how the  program will comply with the  requirements of the Privacy Act  of 1974 and related legislation.

March 28, 2005  To help manage risks  associated with Secure Flight’s  continued development and  implementation, and to assist  TSA in developing a framework  from which to support its efforts  in addressing congressional  areas of interest outlined in  Public Law 108-334, the  Secretary of the Department of  Homeland Security should  direct the Assistant Secretary of  TSA to, prior to achieving initial  operational capability, finalize  policies and procedures  detailing the Secure Flight  passenger redress process,  including defining the appeal  rights of passengers and their  ability to access and correct  personal data.

section_title: Product Title
paragraphs: Recommendation  To help improve TSA’s  management of EDS and ETD  maintenance costs and  strengthen oversight of contract  performance, the Secretary of  Homeland Security should  instruct the Assistant Secretary  of TSA to establish a timeline to  complete its evaluation and  close out the Boeing contract  and report to congressional  appropriations committees on  its actions, including any  necessary analysis, to address  the Department of Homeland  Security Office of Inspector  General’s recommendation to  recover any excessive fees  awarded to Boeing Service  Company.

To help improve TSA’s  management of EDS and ETD  maintenance costs and  strengthen oversight of contract  performance, the Secretary of  Homeland Security should  instruct the Assistant Secretary  of TSA to establish a timeline  for completing life-cycle cost  models for EDS, which TSA  recently began.

section_title: Product Title
paragraphs: Recommendation  To help improve TSA’s  management of EDS and ETD  maintenance costs and  strengthen oversight of contract  performance, the Secretary of  Homeland Security should  instruct the Assistant Secretary  of TSA to revise policies and  procedures to require  documentation of the  monitoring of EDS and ETD  maintenance contracts to  provide reasonable assurance  that contractor maintenance  cost data and performance data  are recorded and reported in  accordance with TSA  contractual requirements and  self-reported contractor mean  downtime data are valid,  reliable, and justify the full  payment of the contract  amount.

To assist TSA in further  strengthening the development  and implementation of the  Secure Flight program, the  Secretary of Homeland Security  should direct the Assistant  Secretary of TSA to fully  incorporate best practices into  the development of Secure  Flight life-cycle cost and  schedule estimates, to include:  (1) updating life-cycle cost and  schedule estimates; (2)  demonstrating that the Secure  Flight schedule has the logic in  place to identify the critical  path, integrates lower level  activities in a logical manner,  and identifies the level of  confidence in meeting the  desired end date; and (3)  developing and implementing a  plan for managing and  mitigating cost and schedule  risks, including performing a  schedule risk analysis and a  cost and schedule risk  assessment.

section_title: Product Title
paragraphs: Recommendation  To assist TSA in further  strengthening the development  and implementation of the  Secure Flight program, the  Secretary of Homeland Security  should direct the Assistant  Secretary of TSA to fully  implement the provisions in the  program’s risk management  plan to include developing an  inventory of risks with  prioritization and mitigation  strategies, report the status of  risks and progress to  management, and maintain  documentation of these efforts.

To assist TSA in further  strengthening the development  and implementation of the  Secure Flight program, the  Secretary of Homeland Security  should direct the Assistant  Secretary of TSA to finalize and  approve Secure Flight’s end-to- end testing strategy, and  incorporate end-to-end testing  requirements in other relevant  test plans, to include the test  and evaluation master plan.  The strategy and plans should  contain provisions for: (1)  testing that ensures that the  interrelated systems that  collectively support Secure  Flight will interoperate as  intended in an operational  environment; and (2) defining  and setting dates for key  milestone activities and  identifying who is responsible  for completing each of those  milestones and when.

section_title: Product Number GAO-09-292
paragraphs: Recommendation  To mitigate future risks of  performance shortfalls and  strengthen management of the  Secure Flight program moving  forward, the Secretary of  Homeland Security should  direct the Assistant Secretary of  TSA to periodically assess the  performance of the Secure  Flight system’s matching  capabilities and results to  determine whether the system  is accurately matching watch- listed individuals while  minimizing the number of false  positives—consistent with the  goals of the program; document  how this assessment will be  conducted and how its results  will be measured; and use  these results to determine  whether the system settings  should be modified.

October 7, 2009  To help ensure that DHS’s  Science and Technology  Directorate (S&T) and TSA take  a comprehensive, risk-informed  approach to the research,  development, test and  evaluation (RDT&E),  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should conduct a complete risk  assessment, including threat,  vulnerability, and consequence  assessments, which would  apply to the Passenger  Screening Program (PSP).

section_title: Product Number GAO-10-128
paragraphs: Category  Conduct a test, study,  or analysis  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should develop cost-benefit  analyses to assist in prioritizing  investments in new checkpoint  screening technologies.

October 7, 2009  To help ensure that DHS’s S&T  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should develop quantifiable  performance measures to  assess the extent to which  investments in research,  development, and deployment  of checkpoint screening  technologies achieve  performance goals for  enhancing security at airport  passenger checkpoints.

section_title: Product Number GAO-10-128
paragraphs: Category  Conduct a test, study,  or analysis  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should, after conducting a  complete risk assessment and  completing cost-benefit  analyses and quantifiable  performance measures for the  PSP, incorporate the results of  these efforts into the PSP  strategy as determined  appropriate.

October 7, 2009  To help ensure that DHS’s S&T  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should, to the extent feasible,  ensure that operational tests  and evaluations have been  successfully completed before  deploying checkpoint screening  technologies to airport  checkpoints.

section_title: Product Number GAO-10-128
paragraphs: Category  Conduct a test, study,  or analysis  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should evaluate whether TSA’s  current passenger screening  procedures should be revised  to require the use of  appropriate screening  procedures until it is  determined that existing  emerging technologies meet  their functional requirements in  an operational environment.

October 7, 2009  To help ensure that DHS’s S&T  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should, in the future, prior to  testing or using all checkpoint  screening technologies at  airports, determine whether  TSA’s passenger screening  procedures should be revised  to require the use of  appropriate screening  procedures until the  performance of the  technologies has been  validated through successful  testing and evaluation.

section_title: Product Number GAO-10-128
paragraphs: Category  Conduct a test, study,  or analysis  and TSA take a  comprehensive, risk-informed  approach to the RDT&E,  procurement, and deployment  of airport passenger checkpoint  screening technologies, and to  increase the likelihood of  successful procurements and  deployments of such  technologies, in the restricted  version of this report, we  recommended that the  Assistant Secretary for TSA  should evaluate the benefits of  the Explosives Trace Portals  that are being used in airports,  and compare the benefits to the  costs to operate and maintain  this technology to determine  whether it is cost-effective to  continue to use the machines in  airports.

To help ensure that TSA takes  a comprehensive and cost- effective approach to the  procurement and deployment of  EDSs that meet the 2010 EDS  requirements and any  subsequent revisions, the  Assistant Secretary for TSA  should develop a plan to  ensure that screening devices  or protocols are in place to  resolve EDS alarms if EDSs  are deployed that detect a  broader set of explosives than  existing ETD machines used to  resolve EDS screening alarms.

section_title: Product Number GAO-11-740
paragraphs: Recommendation  To help ensure that TSA takes  a comprehensive and cost- effective approach to the  procurement and deployment of  EDSs that meet the 2010 EDS  requirements and any  subsequent revisions, the  Assistant Secretary for TSA  should develop a plan to  ensure that TSA has the  explosives data needed for  each of the planned phases of  the 2010 EDS requirements  before starting the procurement  process for new EDSs or  upgrades included in each  applicable phase.

To help ensure that TSA takes  a comprehensive and cost- effective approach to the  procurement and deployment of  EDSs that meet the 2010 EDS  requirements and any  subsequent revisions, the  Assistant Secretary for TSA  should establish a process to  communicate information to  EDS vendors in a timely  manner regarding TSA’s EDS  acquisition, including  information such as changes to  the schedule.

To help ensure that TSA takes  a comprehensive and cost- effective approach to the  procurement and deployment of  EDSs that meet the 2010 EDS  requirements and any  subsequent revisions, the  Assistant Secretary for TSA  should develop and maintain an  integrated master schedule for  the entire Electronic Baggage  Screening Program (EBSP) in  accordance with the nine best  practices identified by GAO for  preparing a schedule.

section_title: Product Number GAO-11-740
paragraphs: Recommendation  To help ensure that TSA takes  a comprehensive and cost- effective approach to the  procurement and deployment of  EDSs that meet the 2010 EDS  requirements and any  subsequent revisions, the  Assistant Secretary for TSA  should ensure that key  elements of the program’s final  cost estimate reflect critical  issues, such as the potential  cost impacts resulting from  schedule slippage identified  once an integrated master  schedule for the Electronic  Baggage Screening Program  has been developed in  accordance with the nine best  practices identified by GAO for  preparing a schedule.

To help ensure that TSA takes  a comprehensive and cost- effective approach to the  procurement and deployment of  EDSs that meet the 2010 EDS  requirements and any  subsequent revisions, the  Assistant Secretary for TSA  should develop a plan to deploy  EDSs that meet the most  recent EDS explosives- detection requirements and  ensure that new machines, as  well as machines deployed in  airports, will be operated at the  levels established in those  requirements. This plan should  include the estimated costs for  new machines and upgrading  deployed machines, and the  time frames for procuring and  deploying new machines and  upgrading deployed machines.

In order to strengthen the  credibility, comprehensiveness,  and reliability of TSA’s cost  estimates and related savings  estimates for the EBSP, the  Administrator of TSA should  ensure that its life cycle cost  estimates conform to cost  estimating best practices.

section_title: Product Number GAO-13-239
paragraphs: Recommendation  To help ensure TSA analyzes  canine team data to identify  program trends, and  determines if PSC teams  provide an added security  benefit to the civil aviation  system, and if so, deploys PSC  teams to the highest-risk  airports, we recommend that  the Administrator of the  Transportation Security  Administration direct the  Manager of the National Canine  Program (NCP) to regularly  analyze available data to  identify program trends and  areas that are working well and  those in need of corrective  action to guide program  resources and activities. These  analyses could include, but not  be limited to, analyzing and  documenting trends in  proficiency training, canine  utilization, results of short  notice assessments (covert  tests) and final canine  responses, performance  differences between law  enforcement officer (LEO) and  TSI canine teams, as well as an  assessment of the optimum  location and number of canine  teams that should be deployed  to secure the U.S.  transportation system.

section_title: Product Number GAO-13-239
paragraphs: Recommendation  To help ensure TSA analyzes  canine team data to identify  program trends, and  determines if passenger  screening canine (PSC) teams  provide an added security  benefit to the civil aviation  system, and if so, deploys PSC  teams to the highest-risk  airports, we recommend that  the Administrator of TSA direct  the Manager of the NCP to  expand and complete testing, in  conjunction with DHS S&T, to  assess the effectiveness of  PSCs and conventional canines  in all airport areas deemed  appropriate (i.e., in the sterile  area, at the passenger  checkpoint, and on the public  side of the airport) prior to  making additional PSC  deployments to help (1)  determine whether PSCs are  effective at screening  passengers, and resource  expenditures for PSC training  are warranted, and (2) inform  decisions regarding the type of  canine team to deploy and  where to optimally deploy such  teams within airports.

section_title: Product Number GAO-14-357
paragraphs: Issue Date  March 31, 2014  To help ensure that TSA  improves Screening Officers’  (SO) performance on Advanced  Imaging Technology systems  equipped with Automated  Targeted Recognition (AIT- ATR) and uses resources  effectively, the Administrator of  the Transportation Security  Administration should establish  protocols that facilitate the  capturing of operational data on  secondary screening of  passengers at the checkpoint to  determine the extent to which  AIT-ATR system false alarm  rates affect operational costs  once AIT-ATR systems are  networked together.

March 31, 2014  To help ensure that TSA  invests in screening technology  that meets mission needs, the  Administrator of TSA should  measure system effectiveness  based on the performance of  the Advanced Imaging  Technology 2 (AIT-2)  technology and screening  officers who operate the  technology, while taking into  account current processes and  deployment strategies before  procuring AIT-2 systems.

March 31, 2014  To help ensure that TSA  invests in screening technology  that meets mission needs, the  Administrator of TSA should  use scientific evidence and  information from DHS’s  Science and Technology  Directorate, and the national  laboratories, as well as  information and data provided  by vendors to develop a  realistic schedule with  achievable milestones that  outlines the technological  advancements, estimated time,  and resources needed to  achieve TSA’s Tier IV end state  before procuring AIT-2  systems.

section_title: Product Title
paragraphs: Recommendation  To increase the likelihood of  timely and successful  acquisitions when enhancing  advanced imaging technology  (AIT) capabilities for airport  passenger checkpoint  screening, the Secretary of  Homeland Security, in  conjunction with the  Administrator of TSA, should  conduct a technical risk  assessment to determine the  extent to which AIT products  need additional development to  meet requirements. TSA should  complete this assessment prior  to award of production units  and should seek an  independent review from a  knowledgeable party, such as  the DHS Science and  Technology Directorate.

To increase the likelihood of  timely and successful  acquisitions when enhancing  AIT capabilities for airport  passenger checkpoint  screening, the Secretary of  Homeland Security, in  conjunction with the  Administrator of TSA, should  ensure that information from  technical risk assessments is  used to inform all future  iterations of Transportation  Security Administration’s  roadmap for enhancing AIT  capabilities.

section_title: Appendix II: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Glenn G. Davis (Assistant  Director), Nima Patel Edwards (Analyst-In-Charge), Rodney Bacigalupo,  Eric D. Hauswirth, Richard B. Hung, Thomas Lombardi, Luis E.  Rodriguez, Tovah Rom, Carley Shinault, and Edith Sohna made key  contributions to this report.


Question-summary annotations: questions: question: To what extent has TSA implemented GAO's recommendations?
answer: The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) has implemented 51 of the 58 recommendations GAO made from October 1, 2003, through July 31, 2015, to improve TSA's acquisition of security-related technology.
child_questions: question: What were GAO's recommendations to TSA?
answer: GAO's recommendations generally directed TSA to develop a plan; conduct an analysis; or implement a program, policy, or procedure.
child_questions: question: What is one example of such a recommendation?
answer: For example, in March 2014, GAO recommended that TSA establish protocols to capture operational data on secondary screening of passengers at the checkpoint, to help ensure screening officers' performance.

questions: question: What outstanding recommendations exist for TSA?
answer: TSA has not implemented 7 of the 58 recommendations.
child_questions: question: How did GAO treat these outstanding recommendations?
answer: GAO closed 4 of the 7 recommendations because TSA stated that it would not take action or GAO determined that it was unlikely that TSA would take action.
child_questions: question: What was the nature of these closed recommendations?
answer: These recommendations were related to establishing the effectiveness of canine screening, conducting technical assessments to strengthen airport perimeter security and Advanced Imaging Technology.

question: What is the purpose of the remaining open recommendations?
answer: The remaining 3 open recommendations are focused on improving Advanced Imaging Technology operations.
child_questions: question: How does GAO view these remaining open recommendations?
answer: GAO continues to believe these recommendations are valid and should be fully addressed.

questions: question: What is TSA's role within DHS?
answer: Within DHS, TSA is the federal agency responsible for securing domestic transportation systems.

question: What is GAO's relationship to TSA?
answer: The Transportation Security Acquisition Reform Act (TSARA) contains a provision for GAO to submit a report to Congress containing an assessment of TSA's implementation of GAO recommendations regarding the acquisition of security-related technology.
child_questions: question: What does this report address?
answer: This report addresses (1) the status of TSA's implementation of relevant GAO recommendations since 2003 and the characteristics of those recommendations and (2) benefits realized by TSA in implementing those recommendations.

questions: question: How did GAO monitor its recommendations to TSA?
answer: GAO determined the number and status of recommendations made to TSA from October 1, 2003, after TSA had become a part of the newly created DHS, through July 31, 2015, as well as the benefits derived from the recommendations TSA implemented, using an internal database that GAO maintains on the status of recommendations it makes.
child_questions: question: What type of recommendations did GAO identify?
answer: GAO specifically identified recommendations related to the acquisition of technology that helps TSA prevent or defend against threats to domestic transportation systems.
child_questions: question: How did TSA respond to these recommendations?
answer: TSA concurred with GAO's list of recommendations.

question: How else did TSA contribute to this report?
answer: TSA also provided technical comments on a draft of this report which GAO incorporated as appropriate. DHS did not provide formal comments.