Many of our reports and testimonies include recommendations that, if acted upon, may result in tangible benefits for the U.S. taxpayer by improving the federal government’s efficiency, effectiveness, and accountability. Implemented recommendations can result in financial or nonfinancial benefits for the federal government. An estimated financial benefit is based on actions taken in response to our recommendations, such as reducing government expenditures, increasing revenues, or reallocating funds to other areas. For example, in fiscal year 2015, our work resulted in $74.4 billion in financial benefits across the federal government. Other benefits that result from our work cannot be measured in dollar terms, and we refer to them as nonfinancial or other benefits. These benefits are linked to specific recommendations or other work that we completed over several years and could include improvements to agency programs, processes, and policies. During fiscal year 2015, we recorded a total of 1,286 other benefits government-wide that resulted from our work, including improved services to the public and government business operations. As part of our responsibilities under generally accepted government auditing standards, we follow up on recommendations we have made and report their status to Congress. Agencies also have a responsibility to monitor and maintain accurate records on the status of our recommendations. After issuing a report, we follow up with reviewed agencies at least once a year to determine the extent to which our recommendations have been implemented and the benefits that have been realized. During this follow-up, we identify what additional actions, if any, would be needed to address our recommendations. A recommendation is considered implemented when actions have been taken that, consistent with our recommendation, address the issue or deficiency we identified and upon which the recommendation is based. Experience has shown that it takes time for some recommendations to be implemented. For this reason, we actively track each unaddressed (i.e., open) recommendation for 4 years and review them to determine whether implementation can be reasonably expected. The review includes consideration of alternative strategies the agency may have for implementing the recommendations. We will close the recommendation as not implemented if TSA has indicated that it was not taking action or we determined that it was unlikely that TSA would take action to close these recommendations. We maintain a publicly available database with information on the current status of most open recommendations. The database allows searches by agency, congressional committee, or key words and is available at http://www.gao.gov/openrecs.html. TSA has implemented 51 of the 58 recommendations we made from October 1, 2003, through July 31, 2015, intended to improve TSA’s acquisition of security-related technology, and has not implemented the remaining 7 recommendations (see fig. 1). The 58 recommendations relating to the acquisition of security-related technology fall into three general categories: actions requiring the agency to (1) develop a plan, process, protocol, or strategy; (2) implement a plan, program, policy, procedure, or best practice; and (3) conduct a test, study, or analysis. For example, in 2011 we recommended that TSA develop a process to communicate information to EDS vendors in a timely manner regarding EDS acquisition and to ensure that TSA takes a comprehensive and cost-effective approach to procuring and deploying EDS that meet the 2010 explosives detection requirements and any subsequent revisions. Separately, in 2008 we recommended that TSA fully incorporate best practices into developing Secure Flight life-cycle cost and schedule estimates and develop a plan for mitigating cost and schedule risks, among other things. In 2009, we recommended that TSA conduct a cost-benefit analysis to assist in prioritizing investments in new checkpoint screening technologies to help TSA take a comprehensive, risk-informed approach to procuring and deploying airport passenger screening technologies. As shown in figure 2, we identified 31 recommendations for the first category, 7 recommendations for the second category, and 20 recommendations for the third category. TSA has not implemented 7 recommendations we made from fiscal year 2003 through July 31, 2015. Four of the 7 recommendations have been closed, while 3 remain open. The 3 open recommendations are focused on improving AIT operations, while the 4 closed recommendations were related to establishing the effectiveness of canine screening, conducting technical assessments to strengthen airport perimeter security, and AIT operations. For example: In 2014, we recommended that TSA establish protocols to facilitate capturing operational data on secondary passenger screening at the checkpoint to determine the extent to which rates of false alarms for various AIT systems affect operational costs once Advanced Imaging Technology-Automated Target Recognition (AIT-ATR) systems are networked. TSA concurred with this recommendation. In its comments on our report, TSA stated that it would monitor, update, and report the results of its efforts to capture operational data and evaluate its associated impacts on operational costs. When contacted in November 2015 for an update on this recommendation, TSA officials stated that they have taken steps toward implementing this recommendation by evaluating the impact of false alarm rates on operational costs (such as staffing) during testing for new AIT systems. This recommendation remains open pending additional actions by TSA to collect secondary screening data on an ongoing basis, which could be used to obtain valuable insights on false alarm rates and the resulting operational costs. By fully implementing the recommendation, TSA could improve the overall performance of the AIT system and make more informed decisions about checkpoint screening. In 2013, we recommended that TSA expand and complete testing, in conjunction with the DHS Science and Technology Directorate, to assess the effectiveness of passenger screening canines (PSC) and conventional canines in all areas of an airport deemed appropriate by TSA before deploying more passenger screening canine teams to help (a) determine whether PSCs are effective at screening passengers, and expenditures for PSC training are warranted and (b) inform decisions about the type of canine teams to deploy and their optimal locations in airports. TSA concurred with the recommendation and took some action, but did not fully address the recommendation. Specifically, in June 2014, TSA reported that it had assessed PSC teams deployed to 27 airports, cumulating in a total of 1,048 tests. On the basis of these tests, TSA determined that PSC teams are effective and should be deployed at the checkpoint queue. However, when contacted in December 2014, officials reported that they did not plan to expand or complete testing to compare the effectiveness PSCs with the effectiveness of conventional canine teams as we recommended, citing concerns about potential liability in using conventional canines that have not been evaluated for their suitability for screening passengers in an unfamiliar passenger screening environment and the related risks to the program. We disagreed and pointed out that conventional canines paired with handlers already work in proximity with passengers since they patrol airport terminals, including ticket counters and curbside areas. Given that TSA does not plan on taking further action on this program, we closed the recommendation as not implemented. However, we continue to believe that the recommendation has merit and should be fully implemented. Since fiscal year 2003, we have identified approximately $1.7 billion in financial benefits, largely representing funds that TSA used to support other programs and activities, based on implementation of our recommendations as well as findings from our related reports and testimonies on security-related technology acquisitions. We have also identified additional benefits, including programmatic and process improvements to TSA’s programs stemming from implementation of our recommendations and related work. The following are examples of the financial, programmatic, and process benefits we identified. In January 2012, we issued a report on TSA’s adherence to DHS acquisition policy and efforts to test the effectiveness of AIT. Among other things, we reported on the effectiveness of the systems and recommended that TSA brief Congress. Congressional response to a TSA briefing combined with our body of work on AIT resulted in TSA’s decision to reduce the number of planned AIT purchases amounting to approximately $1.4 billion. In several reports and testimonies in fiscal year 2004, we reported on delays and challenges in TSA’s development of the Computer- Assisted Passenger Prescreening System II (CAPPS II). We found that TSA had not fulfilled statutory requirements concerning the development and operations of CAPPS II. For example, TSA had not determined the accuracy of the databases that would be used to prescreen passengers, and had not conducted tests that would stress the program and ensure system functionality. We made several recommendations relating to developing project plans, including schedules and estimated costs, conducting system testing; and developing a process by which passengers can get erroneous information corrected, among others. In part because of our initial and subsequent evaluations, and congressional oversight hearings, TSA reprogramed approximately $46 million in funding for CAPPS II to other TSA activities in fiscal years 2003 through 2005. TSA canceled CAPPS II development in August 2004 and, shortly after that, announced plans to develop a successor passenger prescreening program called Secure Flight. The projected program funding for the canceled CAPPS II program resulted in total programmatic savings of approximately $304 million. In April 2012, we found that TSA had established cost estimates for the EBSP to help identify total program cost, recapitalization cost, and potential savings resulting from installing optimal systems, but its processes for developing these estimates did not fully comply with best practices. We recommended that in order to strengthen the credibility, comprehensiveness, and reliability of TSA’s cost estimates and related savings estimates for EBSP, TSA should follow cost- estimating best practices. In response, TSA implemented a management directive that applies DHS guidance and best practices from the GAO Cost Estimating and Assessment Guide and updated its cost estimating best practices to include four characteristics of a high quality and reliable cost estimate—comprehensive, well- documented, accurate, and credible. By implementing this recommendation, TSA improved its ability to determine the cost of the program and plan for the resources required to develop and manage the EBSP. In October 2009, we found that TSA had completed a strategic plan to guide research, development, and deployment of passenger checkpoint screening technologies; however, the plan was not risk- based. We recommended that TSA take a comprehensive, risk- informed approach to procuring and deploying technologies for airport passenger checkpoint screening. Specifically, we recommended that, to the extent feasible, TSA should complete operational tests and evaluations before deploying screening technologies to airport checkpoints. In response to our recommendation, in March 2010, TSA updated its Aviation Modal Risk Assessment, which included a comprehensive risk assessment of the potential for a terrorist attack and implemented a test and evaluation process for all of its technology procurements in accordance with DHS policy. TSA’s actions increased its ability to successfully procure and deploy passenger and checkpoint screening technologies. In 2011, we found that TSA did not effectively communicate in a timely manner with vendors competing for EDS procurement. To help ensure that TSA takes a comprehensive and cost-effective approach to procuring and deploying EDS, in July 2011 we recommended that TSA establish a process to communicate information to EDS vendors in a timely manner about TSA’s EDS acquisitions, including changes to the procurement schedule. In April 2012, TSA provided information on a number of actions it had taken to improve communication with EDS vendors, such as issuing 16 public notifications that contained projected schedules and program updates. In October 2012, TSA finalized its Explosives Detection System Competitive Procurement Qualification Program Communications Plan, which established a process for more timely communication with vendors competing for EDS procurements. In addition, TSA used a qualified products list in its EBSP acquisition plan that awarded contracts only to precertified vendors. By establishing a process to communicate with vendors in a timely manner, TSA was better positioned to avoid delays and procurement cost overruns. In May 2004, we found that TSA’s Office of Acquisition was at an organizational level too low to effectively oversee the acquisition process, coordinate acquisition activities, and enforce acquisition policies effectively. The position of the office hindered its ability to help ensure that TSA follows acquisition processes that enable the agency to get the best value on goods and services. We recommended that TSA elevate the Office of Acquisition to an appropriate level within TSA to enable it to identify, analyze, prioritize, and coordinate agency- wide acquisition needs. In October 2004, TSA elevated its Office of Acquisition to report directly to the Deputy Administrator of TSA. The Office of Acquisition instituted an outreach program to provide acquisition expertise to program offices on each area of the acquisition life cycle. TSA also issued an Investment Review Process guide in January 2005 that outlines acquisition personnel roles, responsibilities, and procedures for conducting acquisition program reviews at each key decision point. By taking these actions, DHS and TSA were better positioned to make TSA’s acquisition process more efficient and improved TSA’s ability to implement its acquisition policies and procedures. We provided a draft of this report to DHS for review and comment. DHS did not provide formal comments but provided technical comments from TSA which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees and the Secretary of Homeland Security. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7141 or GroverJ@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Recommendation To address the challenges associated with the development, implementation, and operation of the Computer Assisted Passenger Prescreening System (CAPPS II), the Secretary of Homeland Security should instruct the Administrator of the Transportation Security Administration (TSA) to develop plans identifying the specific functionality that will be delivered during each increment of CAPPS II, the specific milestones for delivering this functionality, and expected costs for each increment. To address the challenges associated with the development, implementation, and operation of CAPPS II, the Secretary of Homeland Security should instruct the Administrator of TSA to use established plans to track development progress to ensure that promised functionality is being delivered on time and within established cost estimates. To address the challenges associated with the development, implementation, and operation of CAPPS II, the Secretary of Homeland Security should instruct the Administrator of TSA to develop a schedule for critical security activities, including finalizing the security policy, the security risk assessment, and system certification and accreditation. Recommendation To address the challenges associated with the development, implementation, and operation of CAPPS II, the Secretary of Homeland Security should instruct the Administrator of TSA to develop a strategy for mitigating the high risk associated with system and database testing that ensures (1) accuracy testing of commercial and government databases is conducted prior to the database being used and (2) appropriate stress testing is conducted to demonstrate the system can meet peak load requirements. To address the challenges associated with the development, implementation, and operation of CAPPS II, the Secretary of Homeland Security should instruct the Administrator of TSA to develop results-oriented performance goals and measures to evaluate the program’s effectiveness, including measures to assess performance of the system in generating reliable risk scores. To help ensure that TSA receives the goods and services it needs at the best value to the government, the Secretary of Homeland Security should direct the Administrator of TSA to elevate the Office of Acquisition to an appropriate level within TSA to enable it to identify, analyze, prioritize, and coordinate agencywide acquisition needs. Recommendation To help ensure that TSA receives the goods and services it needs at the best value to the government, the Secretary of Homeland Security should direct the Administrator of TSA to develop an adequate system of internal controls, performance measures, and incentives to ensure that policies and processes for ensuring efficient and effective acquisitions are implemented appropriately. To help ensure that TSA receives the goods and services it needs at the best value to the government, the Secretary of Homeland Security should direct the Administrator of TSA to direct the TSA Human Capital Office to do the following in coordination with key offices in the Department of Homeland Security: (1) assess TSA’s current acquisition workforce (as defined by the Department of Homeland Security) to determine the number, skills, and competencies of the workforce; (2) identify any gaps in the number, skills, and competencies of the current acquisition workforce; and (3) develop strategies to address any gaps identified, including plans to attract, retain, and train the workforce. The Secretary of Homeland Security should ensure that its planned departmentwide knowledge management system provides TSA sufficient data and analytic capability to measure and analyze spending activities and performance— and thereby highlight opportunities to reduce costs and improve service levels. Recommendation The Secretary of Homeland Security should ensure that its planned departmentwide knowledge management system provides TSA sufficient data and analytic capability to support effective oversight of acquisitions. To help ensure that TSA is able to articulate and justify future decisions on how best to proceed with security evaluations, fund and implement security improvements—including new security technologies—and implement additional measures to reduce the potential security risks posed by airport workers, the Secretary of Homeland Security should direct TSA’s Administrator to develop and provide Congress with a plan for meeting the requirements of the Aviation Transportation Security Act (ATSA). The Secretary of Homeland Security should direct TSA’s Administrator to develop and provide Congress with a plan for meeting the requirements of ATSA by conducting assessments of technology, compile the results of these assessments as well as assessments conducted independently by airport operators, and communicate the integrated results of these assessments to airport operators. Recommendation The Secretary of Homeland Security should direct TSA’s Administrator to develop and provide Congress with a plan for meeting the requirements of ATSA by using the information resulting from the security evaluation and technology assessment efforts cited above as a basis for providing guidance and prioritizing funding to airports for enhancing the security of the commercial airport system as a whole. In developing the comprehensive plan for installing in-line explosives detection systems (EDS) baggage screening systems, as directed by the fiscal year 2005 DHS Appropriation Act Conference Report, and in satisfying the requirements set forth in the Intelligence Reform and Terrorism Prevention Act of 2004, the Secretary of the Department of Homeland Security should direct the Administrator of TSA to systematically assess the costs and benefits of deploying in-line baggage screening systems at airports that do not yet have in- line systems installed. As part of this assessment, the Administrator should identify and prioritize the airports where the benefits—in terms of cost savings of baggage screening operations and improved security—of replacing stand- alone baggage screening systems with in-line systems are likely to exceed the costs of the systems, or the systems are needed to address security risks or related factors. Recommendation In developing the comprehensive plan for installing in-line EDS baggage screening systems, as directed by the fiscal year 2005 DHS Appropriation Act Conference Report, and in satisfying the requirements set forth in the Intelligence Reform and Terrorism Prevention Act of 2004, the Secretary of the Department of Homeland Security should direct the Administrator of TSA to systematically assess the costs and benefits of deploying in-line baggage screening systems at airports that do not yet have in- line systems installed. As part of this assessment, the Administrator should consider the projected availability and costs of baggage screening equipment being developed through research and development efforts. Recommendation In developing the comprehensive plan for installing in-line EDS baggage screening systems, as directed by the fiscal year 2005 DHS Appropriation Act Conference Report, and in satisfying the requirements set forth in the Intelligence Reform and Terrorism Prevention Act of 2004, the Secretary of the Department of Homeland Security should direct the Administrator of TSA to systematically assess the costs and benefits of deploying in-line baggage screening systems at airports that do not yet have in- line systems installed. As part of this assessment, the Administrator should estimate total funds needed to install in- line systems where appropriate, including the federal funds needed given different assumptions regarding the federal government and airport cost-shares for funding the in- line systems. Recommendation In developing the comprehensive plan for installing in-line EDS baggage screening systems, as directed by the fiscal year 2005 DHS Appropriation Act Conference Report, and in satisfying the requirements set forth in the Intelligence Reform and Terrorism Prevention Act of 2004, the Secretary of the Department of Homeland Security should direct the TSA Administrator to systematically assess the costs and benefits of deploying in-line baggage screening systems at airports that do not yet have in-line systems installed. As part of this assessment, the Administrator should work collaboratively with airport operators, who are expected to share the costs and benefits of in-line systems, to collect data and prepare the analyses needed to develop plans for installing in-line systems. Systematic Planning Needed to Optimize the Deployment of Checked Baggage Screening Systems assess the feasibility, expected benefits, and costs of replacing explosives trace detection (ETD) machines with stand- alone EDS machines for primary screening at those airports where in-line systems would not be either economically justified or justified for other reasons. In conducting this assessment, the Administrator should consider the projected availability and costs for screening equipment being developed through research and development efforts. Issue Date March 28, 2005 To help manage risks associated with Secure Flight’s continued development and implementation, and to assist TSA in developing a framework from which to support its efforts in addressing congressional areas of interest outlined in Public Law 108-334, the Secretary of the Department of Homeland Security should direct the Assistant Secretary of TSA, to finalize the system requirements document and the concept of operations, and develop detailed test plans to help ensure that all Secure Flight system functionality is properly tested and evaluated. These system documents should address all system functionality and include system stress test requirements. March 28, 2005 To help manage risks associated with Secure Flight’s continued development and implementation, and to assist TSA in developing a framework from which to support its efforts in addressing congressional areas of interest outlined in Public Law 108-334, the Secretary of the Department of Homeland Security should direct the Assistant Secretary of TSA to develop a plan for establishing connectivity among the air carriers, U.S. Customs and Border Protection, and TSA to help ensure the secure, effective, and timely transmission of data for use in Secure Flight operations. Issue Date March 28, 2005 To help manage risks associated with Secure Flight’s continued development and implementation, and to assist TSA in developing a framework from which to support its efforts in addressing congressional areas of interest outlined in Public Law 108-334, the Secretary of the Department of Homeland Security should direct the Assistant Secretary of TSA, to develop reliable life- cycle cost estimates and expenditure plans for Secure Flight—in accordance with guidance issued by the Office of Management and Budget— to provide program managers and oversight officials with information needed to make informed decisions regarding program development and resource allocations. March 28, 2005 To help manage risks associated with Secure Flight’s continued development and implementation, and to assist TSA in developing a framework from which to support its efforts in addressing congressional areas of interest outlined in Public Law 108-334, the Secretary of the Department of Homeland Security should direct the Assistant Secretary of TSA, to develop results- oriented performance goals and measures to evaluate the effectiveness of Secure Flight in achieving intended results in an operational environment— as outlined in the Government Performance and Results Act— including measures to assess associated impacts on aviation security. Issue Date March 28, 2005 To help manage risks associated with Secure Flight’s continued development and implementation, and to assist TSA in developing a framework from which to support its efforts in addressing congressional areas of interest outlined in Public Law 108-334, the Secretary of the Department of Homeland Security should direct the Assistant Secretary of TSA to, prior to achieving initial operational capability, finalize policies and issue associated documentation specifying how the Secure Flight program will protect personal privacy, including addressing how the program will comply with the requirements of the Privacy Act of 1974 and related legislation. March 28, 2005 To help manage risks associated with Secure Flight’s continued development and implementation, and to assist TSA in developing a framework from which to support its efforts in addressing congressional areas of interest outlined in Public Law 108-334, the Secretary of the Department of Homeland Security should direct the Assistant Secretary of TSA to, prior to achieving initial operational capability, finalize policies and procedures detailing the Secure Flight passenger redress process, including defining the appeal rights of passengers and their ability to access and correct personal data. Recommendation To help improve TSA’s management of EDS and ETD maintenance costs and strengthen oversight of contract performance, the Secretary of Homeland Security should instruct the Assistant Secretary of TSA to establish a timeline to complete its evaluation and close out the Boeing contract and report to congressional appropriations committees on its actions, including any necessary analysis, to address the Department of Homeland Security Office of Inspector General’s recommendation to recover any excessive fees awarded to Boeing Service Company. To help improve TSA’s management of EDS and ETD maintenance costs and strengthen oversight of contract performance, the Secretary of Homeland Security should instruct the Assistant Secretary of TSA to establish a timeline for completing life-cycle cost models for EDS, which TSA recently began. Recommendation To help improve TSA’s management of EDS and ETD maintenance costs and strengthen oversight of contract performance, the Secretary of Homeland Security should instruct the Assistant Secretary of TSA to revise policies and procedures to require documentation of the monitoring of EDS and ETD maintenance contracts to provide reasonable assurance that contractor maintenance cost data and performance data are recorded and reported in accordance with TSA contractual requirements and self-reported contractor mean downtime data are valid, reliable, and justify the full payment of the contract amount. To assist TSA in further strengthening the development and implementation of the Secure Flight program, the Secretary of Homeland Security should direct the Assistant Secretary of TSA to fully incorporate best practices into the development of Secure Flight life-cycle cost and schedule estimates, to include: (1) updating life-cycle cost and schedule estimates; (2) demonstrating that the Secure Flight schedule has the logic in place to identify the critical path, integrates lower level activities in a logical manner, and identifies the level of confidence in meeting the desired end date; and (3) developing and implementing a plan for managing and mitigating cost and schedule risks, including performing a schedule risk analysis and a cost and schedule risk assessment. Recommendation To assist TSA in further strengthening the development and implementation of the Secure Flight program, the Secretary of Homeland Security should direct the Assistant Secretary of TSA to fully implement the provisions in the program’s risk management plan to include developing an inventory of risks with prioritization and mitigation strategies, report the status of risks and progress to management, and maintain documentation of these efforts. To assist TSA in further strengthening the development and implementation of the Secure Flight program, the Secretary of Homeland Security should direct the Assistant Secretary of TSA to finalize and approve Secure Flight’s end-to- end testing strategy, and incorporate end-to-end testing requirements in other relevant test plans, to include the test and evaluation master plan. The strategy and plans should contain provisions for: (1) testing that ensures that the interrelated systems that collectively support Secure Flight will interoperate as intended in an operational environment; and (2) defining and setting dates for key milestone activities and identifying who is responsible for completing each of those milestones and when. Recommendation To mitigate future risks of performance shortfalls and strengthen management of the Secure Flight program moving forward, the Secretary of Homeland Security should direct the Assistant Secretary of TSA to periodically assess the performance of the Secure Flight system’s matching capabilities and results to determine whether the system is accurately matching watch- listed individuals while minimizing the number of false positives—consistent with the goals of the program; document how this assessment will be conducted and how its results will be measured; and use these results to determine whether the system settings should be modified. October 7, 2009 To help ensure that DHS’s Science and Technology Directorate (S&T) and TSA take a comprehensive, risk-informed approach to the research, development, test and evaluation (RDT&E), procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should conduct a complete risk assessment, including threat, vulnerability, and consequence assessments, which would apply to the Passenger Screening Program (PSP). Category Conduct a test, study, or analysis and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should develop cost-benefit analyses to assist in prioritizing investments in new checkpoint screening technologies. October 7, 2009 To help ensure that DHS’s S&T and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should develop quantifiable performance measures to assess the extent to which investments in research, development, and deployment of checkpoint screening technologies achieve performance goals for enhancing security at airport passenger checkpoints. Category Conduct a test, study, or analysis and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should, after conducting a complete risk assessment and completing cost-benefit analyses and quantifiable performance measures for the PSP, incorporate the results of these efforts into the PSP strategy as determined appropriate. October 7, 2009 To help ensure that DHS’s S&T and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should, to the extent feasible, ensure that operational tests and evaluations have been successfully completed before deploying checkpoint screening technologies to airport checkpoints. Category Conduct a test, study, or analysis and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should evaluate whether TSA’s current passenger screening procedures should be revised to require the use of appropriate screening procedures until it is determined that existing emerging technologies meet their functional requirements in an operational environment. October 7, 2009 To help ensure that DHS’s S&T and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should, in the future, prior to testing or using all checkpoint screening technologies at airports, determine whether TSA’s passenger screening procedures should be revised to require the use of appropriate screening procedures until the performance of the technologies has been validated through successful testing and evaluation. Category Conduct a test, study, or analysis and TSA take a comprehensive, risk-informed approach to the RDT&E, procurement, and deployment of airport passenger checkpoint screening technologies, and to increase the likelihood of successful procurements and deployments of such technologies, in the restricted version of this report, we recommended that the Assistant Secretary for TSA should evaluate the benefits of the Explosives Trace Portals that are being used in airports, and compare the benefits to the costs to operate and maintain this technology to determine whether it is cost-effective to continue to use the machines in airports. To help ensure that TSA takes a comprehensive and cost- effective approach to the procurement and deployment of EDSs that meet the 2010 EDS requirements and any subsequent revisions, the Assistant Secretary for TSA should develop a plan to ensure that screening devices or protocols are in place to resolve EDS alarms if EDSs are deployed that detect a broader set of explosives than existing ETD machines used to resolve EDS screening alarms. Recommendation To help ensure that TSA takes a comprehensive and cost- effective approach to the procurement and deployment of EDSs that meet the 2010 EDS requirements and any subsequent revisions, the Assistant Secretary for TSA should develop a plan to ensure that TSA has the explosives data needed for each of the planned phases of the 2010 EDS requirements before starting the procurement process for new EDSs or upgrades included in each applicable phase. To help ensure that TSA takes a comprehensive and cost- effective approach to the procurement and deployment of EDSs that meet the 2010 EDS requirements and any subsequent revisions, the Assistant Secretary for TSA should establish a process to communicate information to EDS vendors in a timely manner regarding TSA’s EDS acquisition, including information such as changes to the schedule. To help ensure that TSA takes a comprehensive and cost- effective approach to the procurement and deployment of EDSs that meet the 2010 EDS requirements and any subsequent revisions, the Assistant Secretary for TSA should develop and maintain an integrated master schedule for the entire Electronic Baggage Screening Program (EBSP) in accordance with the nine best practices identified by GAO for preparing a schedule. Recommendation To help ensure that TSA takes a comprehensive and cost- effective approach to the procurement and deployment of EDSs that meet the 2010 EDS requirements and any subsequent revisions, the Assistant Secretary for TSA should ensure that key elements of the program’s final cost estimate reflect critical issues, such as the potential cost impacts resulting from schedule slippage identified once an integrated master schedule for the Electronic Baggage Screening Program has been developed in accordance with the nine best practices identified by GAO for preparing a schedule. To help ensure that TSA takes a comprehensive and cost- effective approach to the procurement and deployment of EDSs that meet the 2010 EDS requirements and any subsequent revisions, the Assistant Secretary for TSA should develop a plan to deploy EDSs that meet the most recent EDS explosives- detection requirements and ensure that new machines, as well as machines deployed in airports, will be operated at the levels established in those requirements. This plan should include the estimated costs for new machines and upgrading deployed machines, and the time frames for procuring and deploying new machines and upgrading deployed machines. In order to strengthen the credibility, comprehensiveness, and reliability of TSA’s cost estimates and related savings estimates for the EBSP, the Administrator of TSA should ensure that its life cycle cost estimates conform to cost estimating best practices. Recommendation To help ensure TSA analyzes canine team data to identify program trends, and determines if PSC teams provide an added security benefit to the civil aviation system, and if so, deploys PSC teams to the highest-risk airports, we recommend that the Administrator of the Transportation Security Administration direct the Manager of the National Canine Program (NCP) to regularly analyze available data to identify program trends and areas that are working well and those in need of corrective action to guide program resources and activities. These analyses could include, but not be limited to, analyzing and documenting trends in proficiency training, canine utilization, results of short notice assessments (covert tests) and final canine responses, performance differences between law enforcement officer (LEO) and TSI canine teams, as well as an assessment of the optimum location and number of canine teams that should be deployed to secure the U.S. transportation system. Recommendation To help ensure TSA analyzes canine team data to identify program trends, and determines if passenger screening canine (PSC) teams provide an added security benefit to the civil aviation system, and if so, deploys PSC teams to the highest-risk airports, we recommend that the Administrator of TSA direct the Manager of the NCP to expand and complete testing, in conjunction with DHS S&T, to assess the effectiveness of PSCs and conventional canines in all airport areas deemed appropriate (i.e., in the sterile area, at the passenger checkpoint, and on the public side of the airport) prior to making additional PSC deployments to help (1) determine whether PSCs are effective at screening passengers, and resource expenditures for PSC training are warranted, and (2) inform decisions regarding the type of canine team to deploy and where to optimally deploy such teams within airports. Issue Date March 31, 2014 To help ensure that TSA improves Screening Officers’ (SO) performance on Advanced Imaging Technology systems equipped with Automated Targeted Recognition (AIT- ATR) and uses resources effectively, the Administrator of the Transportation Security Administration should establish protocols that facilitate the capturing of operational data on secondary screening of passengers at the checkpoint to determine the extent to which AIT-ATR system false alarm rates affect operational costs once AIT-ATR systems are networked together. March 31, 2014 To help ensure that TSA invests in screening technology that meets mission needs, the Administrator of TSA should measure system effectiveness based on the performance of the Advanced Imaging Technology 2 (AIT-2) technology and screening officers who operate the technology, while taking into account current processes and deployment strategies before procuring AIT-2 systems. March 31, 2014 To help ensure that TSA invests in screening technology that meets mission needs, the Administrator of TSA should use scientific evidence and information from DHS’s Science and Technology Directorate, and the national laboratories, as well as information and data provided by vendors to develop a realistic schedule with achievable milestones that outlines the technological advancements, estimated time, and resources needed to achieve TSA’s Tier IV end state before procuring AIT-2 systems. Recommendation To increase the likelihood of timely and successful acquisitions when enhancing advanced imaging technology (AIT) capabilities for airport passenger checkpoint screening, the Secretary of Homeland Security, in conjunction with the Administrator of TSA, should conduct a technical risk assessment to determine the extent to which AIT products need additional development to meet requirements. TSA should complete this assessment prior to award of production units and should seek an independent review from a knowledgeable party, such as the DHS Science and Technology Directorate. To increase the likelihood of timely and successful acquisitions when enhancing AIT capabilities for airport passenger checkpoint screening, the Secretary of Homeland Security, in conjunction with the Administrator of TSA, should ensure that information from technical risk assessments is used to inform all future iterations of Transportation Security Administration’s roadmap for enhancing AIT capabilities. In addition to the contact named above, Glenn G. Davis (Assistant Director), Nima Patel Edwards (Analyst-In-Charge), Rodney Bacigalupo, Eric D. Hauswirth, Richard B. Hung, Thomas Lombardi, Luis E. Rodriguez, Tovah Rom, Carley Shinault, and Edith Sohna made key contributions to this report.

Summary: Within DHS, TSA is the federal agency responsible for securing domestic transportation systems. The Transportation Security Acquisition Reform Act (TSARA) contains a provision for GAO to submit a report to Congress containing an assessment of TSA's implementation of GAO recommendations regarding the acquisition of security-related technology. This report addresses (1) the status of TSA's implementation of relevant GAO recommendations since 2003 and the characteristics of those recommendations and (2) benefits realized by TSA in implementing those recommendations. GAO determined the number and status of recommendations made to TSA from October 1, 2003, after TSA had become a part of the newly created DHS, through July 31, 2015, as well as the benefits derived from the recommendations TSA implemented, using an internal database that GAO maintains on the status of recommendations it makes. GAO specifically identified recommendations related to the acquisition of technology that helps TSA prevent or defend against threats to domestic transportation systems. TSA concurred with GAO's list of recommendations. TSA also provided technical comments on a draft of this report which GAO incorporated as appropriate. DHS did not provide formal comments. The Department of Homeland Security's (DHS) Transportation Security Administration (TSA) has implemented 51 of the 58 recommendations GAO made from October 1, 2003, through July 31, 2015, to improve TSA's acquisition of security-related technology. GAO's recommendations generally directed TSA to develop a plan; conduct an analysis; or implement a program, policy, or procedure. For example, in March 2014, GAO recommended that TSA establish protocols to capture operational data on secondary screening of passengers at the checkpoint, to help ensure screening officers' performance. TSA has not implemented 7 of the 58 recommendations. GAO closed 4 of the 7 recommendations because TSA stated that it would not take action or GAO determined that it was unlikely that TSA would take action. These recommendations were related to establishing the effectiveness of canine screening, conducting technical assessments to strengthen airport perimeter security and Advanced Imaging Technology. The remaining 3 open recommendations are focused on improving Advanced Imaging Technology operations. GAO continues to believe these recommendations are valid and should be fully addressed. Since fiscal year 2003, GAO has identified approximately $1.7 billion in financial benefits, largely representing funds that TSA used to support other programs and activities, based on implementation of GAO's recommendations as well as findings from related GAO reports and testimonies on security-related technology acquisitions. GAO has also documented additional benefits, including programmatic and process improvements to TSA's programs, such as improvements to TSA's Electronic Baggage Screening Program's cost estimating processes.