Transportation Security Administration: Oversight of Explosive Detection Systems Maintenance Contracts Can Be Strengthened
Published: 2006-07-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Mandated to screen all checked baggage by using explosive detection systems at airports by December 31, 2003, the Transportation Security Administration (TSA) has deployed two types of screening equipment: explosive detection systems (EDS), which use computer-aided tomography X-rays to recognize explosives, and explosive trace detection (ETD) systems, which use chemical analysis to detect explosive residues. This report discusses (1) EDS and ETD maintenance costs, (2) factors that played a role in these costs, and (3) the extent to which TSA conducts oversight of maintenance contracts. GAO reviewed TSA's contract files and processes for reviewing contractor cost and performance data.
section_title: What GAO Found
paragraphs: TSA obligated almost $470 million from fiscal years 2002 through 2005 for EDS and ETD maintenance, according to TSA budget documents. In fiscal year 2006, TSA estimates it will spend $199 million and has projected it will spend $234 million in fiscal year 2007. TSA was not able to provide GAO with data on the maintenance cost per machine before fiscal year 2005 because, according to TSA officials, its previous contract with Boeing to install and maintain EDS and ETD machines was not structured to capture these data. Several factors have played a role in EDS and ETD maintenance costs. According to a September 2004 Department of Homeland Security's Office of Inspector General report, TSA did not follow sound contracting practices in administering the contract with Boeing, and TSA paid provisional award fees totaling $44 million through December 2003 without any evaluation of Boeing's performance. TSA agreed to recover any excessive award fees paid to Boeing if TSA determined that such fees were not warranted. In responding to our draft report, DHS told us that TSA and Boeing had reached an agreement in principle on this matter and that documentation was in the approval process with closure anticipated in July 2006. Moreover, TSA did not develop life-cycle cost models before any of the maintenance contracts were executed and, as a result, TSA does not have a sound estimate of maintenance costs for all the years the machines are expected to be in operation. DHS also stated in its comments on our draft report that a TSA contractor expected to complete a prototype life-cycle cost model by September 2006 and that TSA anticipated that the EDS model would be completed 12 months after the prototype was approved. Without such an analysis, TSA may not be identifying cost efficiencies and making informed procurement decisions on future purchases of EDS and ETD machines and maintenance contracts. TSA has taken actions to control costs, such as entering into firm-fixed-price contracts for maintenance starting in March 2005, which have advantages to the government because price certainty is guaranteed. Further, TSA incorporated standard performance requirements in the contracts including metrics related to machine reliability and monthly performance reviews. For EDS contractors, TSA has specified that the full agreed price would be paid only if mean downtime (i.e., the number of hours a machine is out of service in a month divided by the number of times that machine is out of service per month) requirements are met. Although TSA has policies for monitoring contracts, TSA officials provided no evidence that they are reviewing required contractor-submitted performance data, such as mean downtime data. TSA officials told GAO that they perform such reviews, but do not document their activities because there are no TSA policies and procedures requiring them to do so. As a result, without adequate documentation, TSA does not have reasonable assurance that contractors are performing as required and that full payment is justified based on meeting mean downtime requirements.

section_title: Letter
paragraphs: After the terrorist attacks of September 11, 2001, which highlighted the  vulnerability of U.S. aircraft to acts of terrorism, Congress passed and the  President signed into law the Aviation and Transportation Security Act  (ATSA), mandating, among other things, that all checked baggage at U.S.  airports be screened using explosive detection systems by December 31,  2002. To meet this requirement, the Transportation Security  Administration (TSA) deployed two types of equipment to screen checked  baggage for explosives: (1) explosive detection systems (EDS) that use  specialized X-rays to detect characteristics of explosives that may be  contained in baggage as it moves along a conveyor belt and (2) explosive  trace detection (ETD) machines, whereby an individual (i.e., a baggage  screener, or transportation security officer) swabs baggage and then  inserts the swab into the ETD machine, which in turn can detect chemical  residues that may indicate the presence of explosives within a bag.
Pursuant to ATSA, TSA assumed operational responsibility for conducting  the screening of checked baggage, which includes the procurement,  installation, and maintenance of EDS and ETD machines. By the end of  fiscal year 2006, TSA will have deployed over 1,400 EDS and 6,600 ETD  machines at baggage-screening locations in over 400 airports nationwide,  according to TSA budget documents. TSA has used contractors to perform  preventative and corrective maintenance on these EDS and ETD  machines.
House Conference Report 109-241, which accompanied the Department of  Homeland Security Appropriations Act, 2006 (Public Law 109-90) directed  that we report on the reasons for cost increases in maintaining TSA’s  explosive detection systems, including TSA’s contracting practices that  may have affected cost increases. With regard to this requirement, on  April 19, 2006, we provided you with information on the results of our  review (see app. I), which has been updated as appropriate in this report.  Subsequently, in May 2006, the House Appropriations Committee stated in  its report accompanying the Department of Homeland Security  Appropriations Bill for fiscal year 2007 its long-standing concerns about  the increasing costs for EDS and ETD maintenance and specifically, TSA’s  recovery of any excess award fees from a previous EDS and ETD  contractor. Further, the report stated that TSA should submit a report to  the House Appropriations Committee on any actions it has taken to collect  excessive award fees, how much has been received to date, and specific  plans to obligate these collections.
This report addresses the following questions:    What are the historical, current, and projected costs for the  maintenance of EDS and ETD machines?
What factors played a role in EDS and ETD maintenance costs, and  what factors could affect future costs?
What has TSA done to control EDS and ETD maintenance costs?
To what extent does TSA oversee the performance of EDS and ETD  maintenance contractors?
To address our objectives, we reviewed TSA contract files and related cost  data, TSA processes for reviewing contract performance data, budget  documents for fiscal years 2003 through 2007, acquisition and strategic  plans, a Department of Homeland Security’s Office of the Inspector  General (DHS OIG) report, GAO standards for internal controls, and our  previous work on TSA’s acquisition function. We interviewed TSA  headquarters officials, DHS OIG officials, and EDS and ETD contractor  representatives. For purposes of our review, we focused on amounts  obligated under contracts to maintain the machines. We did not review  TSA’s negotiations for maintenance services or the process for awarding  contracts. Nor did we assess other direct or indirect costs that may have  been related to TSA or DHS employees engaged in contract administration  or other related items.
We performed our work from January 2006 through July 2006 in  accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: With the passage of the Aviation and Transportation Security Act (ATSA)  in November 2001, TSA assumed from the Federal Aviation Administration  (FAA) the majority of the responsibility for civil aviation security,  including the commercial aviation system. ATSA required that TSA  screen 100 percent of checked baggage using explosive detection systems  by December 31, 2002. As it became apparent that certain airports would  not meet the December 2002 deadline, the Homeland Security Act of 2002  in effect extended the deadline to December 31, 2003, for noncompliant  airports. Under ATSA, TSA is responsible for the procurement,  installation, and maintenance of explosive detection systems used to  screen checked baggage for explosives. Airport operators and air carriers  continued to be responsible for processing and transporting passengers’  checked baggage from the check-in counter to the airplane.
Explosive detection systems include EDS and ETD machines (fig. 1). EDS  uses computer-aided tomography X-rays adapted from the medical field to  automatically recognize the characteristic signatures of threat explosives.  By taking the equivalent of hundreds of X-ray pictures of a bag from  different angles, EDS examines the objects inside of the baggage to  identify characteristic signatures of threat explosives. TSA has certified,  procured, and deployed EDS manufactured by three companies—L-3  Communications Security and Detection Systems (L-3); General Electric  InVision, Inc. (GE InVision); and Reveal Imaging Technologies, Inc.  (Reveal). In general, EDS is used for checked baggage screening. ETD  machines work by detecting vapors and residues of explosives. Human  operators collect samples by rubbing bags with swabs, which are then  chemically analyzed in the ETD machine to identify any traces of  explosive materials. ETD machines are used for both checked baggage and  passenger carry-on baggage screening. TSA has certified, procured, and  deployed ETD machines from three manufacturers, Thermo Electron  Corporation, Smiths Detection, and General Electric Company.
TSA’s EDS and ETD maintenance contracts provide for preventative and  corrective maintenance. Preventative maintenance includes scheduled  activities, such as changing filters or cleaning brushes, to increase  machine reliability and are performed monthly, quarterly, or yearly based  on the contractors’ maintenance schedules. Corrective maintenance  includes actions performed to restore machines to operating condition  after failure, such as repairing the conveyer belt mechanism after a bag  jams the machine. TSA is responsible for EDS and ETD maintenance costs  after warranties on the machines expire.
From June 2002 through March 2005, Boeing was the prime contractor  primarily for the installation and maintenance of EDS and ETD machines  at over 400 U.S. airports. TSA officials stated that the Boeing contract was  awarded at a time when TSA was a new agency with many demands and  extremely tight schedules for meeting numerous congressional mandates  related to passenger and checked baggage screening. The cost  reimbursement contract entered into with Boeing had been competitively  bid and contained renewable options through 2007. Boeing subcontracted  for EDS maintenance through firm-fixed-price contracts with the original  EDS manufacturers, GE InVision and L-3, which performed the  maintenance on their respective EDS. Boeing subcontracted for ETD  maintenance through a firm-fixed-price contract with Siemens. Consistent  with language in the fiscal year 2005 House Appropriations Committee  report and due to TSA’s acknowledgment of Boeing’s failure to control  costs, TSA received DHS authorization to negotiate new EDS and ETD  maintenance contracts in January 2005.
In March 2005, TSA signed firm-fixed-price contracts for EDS and ETD  maintenance. TSA awarded a competitively bid contract to Siemens to  provide maintenance for ETD machines. According to TSA, it negotiated  sole source contracts with L-3 and GE InVision for maintaining their  respective EDS because they are the original equipment manufacturers  and owners of the intellectual property rights of their respective EDS. In  September 2005, TSA awarded a competitively bid firm-fixed-price  contract to Reveal for both the procurement and maintenance of a  reduced size EDS.
section_title: Results
paragraphs: TSA obligated almost $470 million from fiscal year 2002 through fiscal year  2005 for EDS and ETD maintenance, according to TSA budget documents.  In fiscal year 2006, TSA estimates it will spend $199 million and has  projected it will spend $234 million in fiscal year 2007. According to TSA  officials, in fiscal year 2004, TSA requested and received approval to  reprogram about $32 million from another account to EDS/ETD  maintenance due to higher levels of maintenance costs than expected.  Similarly, in fiscal year 2005, TSA requested and received approval to  reprogram $25 million to fund the L-3 contract and to close out the Boeing  contract. TSA was not able to provide us with data on the maintenance  cost per machine before fiscal year 2005 because, according to TSA  officials, TSA’s previous contract with Boeing to maintain EDS and ETD  machines was not structured to capture these data. Table 1 identifies the  maintenance costs by type of EDS and ETD machine for fiscal years 2005  and 2006.
TSA did not provide us with projections of EDS and ETD maintenance  costs beyond 2007. TSA officials told us that future costs will be  influenced by the number, type, quantity, and locations of machines  necessary to support system configurations at airports, such as the extent  to which EDS are integrated with airport baggage conveyor systems or are  operated in stand-alone modes. Further, TSA officials told us that future  EDS and ETD maintenance costs are dependent on decisions related to  the deployment of new technologies and the refurbishment of existing  equipment, among other things. The current contracts would have  negotiated maintenance prices per machine through March 2009, if TSA  decides to exercise option years in the contracts.
We identified different factors that have played a role in costs to date and  that will influence future maintenance costs for EDS and ETD machines.  According to a September 2004 DHS OIG report, TSA did not follow sound  contracting practices in administering the Boeing contract, which was  primarily for the installation and maintenance of EDS and ETD machines.  According to DHS OIG officials, TSA’s failure to control costs under the  Boeing contract, including the lack of sound contracting practices,  contributed to increases in maintenance costs. Among other things, the  DHS OIG report stated that TSA had paid provisional award fees totaling  $44 million through December 2003 without any evaluation of Boeing’s  performance. In response to the DHS OIG, TSA agreed to recover any  excessive award fees paid to Boeing, if TSA determined that such fees  were not warranted. In commenting on our draft report in July 2006, DHS  stated that TSA has conducted a contract reconciliation process to ensure  that no fees would be paid on costs that exceeded the target due to poor  contractor performance. Further, DHS stated that TSA and Boeing had  reached an agreement in principle on this matter and that the  documentation was in the approval process with closure anticipated in  July 2006. In its report accompanying the DHS Appropriations Bill for  fiscal year 2007, the House Appropriations Committee stated its need for a  report from TSA on any actions it has taken to collect excessive award  fees, how much of the fees have been received to date, and specific plans  to obligate these collections and cited TSA’s plans to use any cost  recoveries to purchase and install additional EDS. These actions were  based on the committee’s long-standing concerns about the increasing  costs for EDS and ETD maintenance. In addition to matters related to the  Boeing contract, TSA officials stated that another factor contributing to  cost increases were the larger than expected number of machines that  came out of warranty and their related maintenance costs. According to  TSA officials, they were not able to determine the cost impact of these  additional machines because the Boeing contract was not structured to  provide maintenance costs for individual machines.
With regard to future EDS and ETD maintenance costs under firm-fixed- price contracts, maintenance costs per machine will increase primarily by  an annual escalation factor in the contracts that takes into account the  employment cost index and the consumer price index, if TSA decides to  exercise contract options. In addition, future maintenance costs may be  affected by a range of factors, including the number of machines deployed  and out of warranty, conditions under which machines operate, contractor  performance requirements, the emergence of new technologies or  improved equipment, and alternative screening strategies. Lastly, life-cycle  cost estimates were not developed for the Boeing, Siemens, L-3, and GE  InVision contracts before the maintenance contracts were executed, and,  as a result, TSA did not have a sound estimate of maintenance costs for all  the years the machines are expected to be in operation. In August 2005,  TSA hired a contractor to define parameters for a life-cycle cost model,  among other things. This contract states that TSA and the contractor will  work together to ensure that the full scope of work is planned,  coordinated, and executed according to approved schedules. In  commenting on our draft report in July 2006, DHS stated that the TSA  contractor estimated completing a prototype life-cycle cost model by  September 2006. Further, DHS stated that TSA’s evaluation of the  prototype would begin immediately upon delivery and that full  implementation of an EDS life-cycle cost model would be completed  within 12 months after the prototype had been approved. According to a  TSA official, the life-cycle cost model would be useful in determining  machine reliability and maintainability and to inform future contract  negotiations, such as when to replace a machine versus continuing to  repair it.
We identified several actions TSA has taken to control EDS and ETD  maintenance costs. First, TSA entered into firm-fixed-price contracts  starting in March 2005 with maintenance contractors, which offer TSA  certain advantages over cost reimbursement contracts because price  certainty is guaranteed for up to 5 years if TSA exercises options to 2009.  Also, TSA included several performance requirements in the Siemens, L-3,  GE InVision, and Reveal contracts, including the collection of metrics  related to machine reliability, maintainability, and availability and  required specific cost data related to maintenance and repair. TSA officials  told us that these data will assist them in monitoring the contractor  performance as well as informing future contract negotiations for  equipment and maintenance. These contracts also stipulate that  maintenance contractors meet monthly with TSA to review all pertinent  technical schedules and cost aspects of contracts. TSA also incorporated  provisions in the L-3 and GE InVision contracts to specify that the agreed  price for maintaining EDS would be paid only if the contractor performs  within specified mean downtime (MDT) requirements. Contractors  submit monthly invoices for 95 percent of the negotiated contract price for  the month and then submit a MDT report to justify the additional 5  percent. Consequently, if the contractor fails to fulfill the MDT  requirements, it is penalized 5 percent of the negotiated monthly  maintenance price. As of February 2006, neither GE InVision nor L-3 had  been penalized for missing their MDT requirements. The allowable MDT is  lowered from 2005 to subsequent renewable years in the contract, as  shown in table 2.
With regard to TSA’s oversight of EDS and ETD contractor performance,  TSA’s acquisition policies and GAO standards for internal controls call  for documenting transactions and other significant events, such as  monitoring contractor activities. The failure of TSA to develop internal  controls and performance measures has been recognized by other GAO  and DHS OIG reviews. TSA has policies and procedures for monitoring  its contracts and has included contractor performance requirements in the  current EDS and ETD maintenance contracts. However, TSA officials  provided no evidence that they are reviewing maintenance cost data  provided by the contractor because they are not required to document  such activities. For example, even though TSA officials told us that they  are reviewing required contractor data, including actual maintenance costs  related to labor hours and costs associated with replacing and shipping  machine parts, they did not have any documentation to support this. TSA  officials told us that they have begun to capture these data to assist them  in any future contract negotiations.
Further, TSA officials provided no evidence that performance data for  corrective and preventative maintenance required under contracts are  being reviewed. TSA officials told us that they perform such reviews, but  do not document their activities since there are no TSA policies or  procedures requiring them to do so. Therefore, TSA could not provide  assurance that contractors are complying with contract performance  requirements. For example, although TSA documents monthly meetings  with contractors to discuss performance data, TSA officials did not  provide evidence that they independently determine the reliability and  validity of data required by the contracts, such as mean time between  failures and mean time to repair, which are important to making informed  decisions about future purchases of EDS and ETD equipment and their  associated maintenance costs. Further, TSA officials provided no evidence  that they ensure that contractors are performing scheduled preventative  maintenance. TSA officials told us that they review the contractor- submitted data to determine whether contractors are fulfilling their  contractual obligations, but do not document their activities because there  are no TSA policies or procedures to require such documentation.
Additionally, for EDS contracts with possible financial penalties, TSA  officials told us that they review contractor-submitted mean downtime  data on a monthly basis to determine the reliability and validity of the data  and to determine whether contractors are meeting contract provisions or  should be penalized. However, TSA officials do not document these  activities because there are no TSA policies or procedures requiring them  to do so. As a result, without adequate documentation, there is no  assurance as to whether or not contractors are meeting contract  provisions or that TSA has ensured that it is making appropriate payments  for services provided.
section_title: Conclusions
paragraphs: The cost of maintaining checked baggage-screening equipment has  increased as more EDS and ETD machines have been deployed and  warranties expire. TSA’s move in March 2005 to firm-fixed-price  maintenance contracts for EDS and ETD maintenance was advantageous  to the government in that it helps control present and future maintenance  costs. Firm-fixed-price contracts also help ensure price certainty and  therefore are more predictable. However, unresolved issues remain with  the past contractor, specifically fees awarded to former contractor Boeing  that may have been excessive due to a lack of timely evaluation of the  contractor’s performance. The House Appropriations Committee has  expressed concern about these unresolved issues; specifically, what  actions TSA has taken to recover these excessive fees, and the extent to  which any collections might impact future TSA obligations. Closing out the  Boeing contract is essential to resolving these issues. In responding to our  draft report, DHS stated that the completion of an EDS life-cycle cost is  over a year away. Absent such a life-cycle cost model, TSA may not be  identifying cost efficiencies and making informed procurement decisions  regarding the future purchase of EDS and ETD machines and maintenance  contracts. Further, TSA must provide evidence of its reviews and analyses  of contractor-submitted data and perform analyses of contractor data to  determine the reliability and validity of the data and to provide assurance  of compliance with contract performance requirements and internal  control standards. Without stronger oversight, TSA will not have  reasonable assurance that contractors are performing as required and that  full payment is justified based on meeting mean downtime requirements.
section_title: Recommendations
paragraphs: To help improve TSA’s management of EDS and ETD maintenance costs  and strengthen oversight of contract performance, we recommend that the  Secretary of Homeland Security instruct the Assistant Secretary,  Transportation Security Administration, to take the following three  actions:    establish a timeline to complete its evaluation and close out the Boeing  contract and report to congressional appropriations committees on its  actions, including any necessary analysis, to address the Department of  Homeland Security Office of Inspector General’s recommendation to  recover any excessive fees awarded to Boeing Service Company;    establish a timeline for completing life-cycle cost models for EDS,  which TSA recently began; and  revise policies and procedures to require documentation of the  monitoring of EDS and ETD maintenance contracts to provide  reasonable assurance that    contractor maintenance cost data and performance data are  recorded and reported in accordance with TSA contractual  requirements and    self-reported contractor mean downtime data are valid, reliable, and  justify the full payment of the contract amount.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DHS for its review and comment. On  July 24, 2006, we received written comments on the draft report. DHS, in  its written comments, concurred with our findings and recommendations,  and agreed that efforts to implement these recommendations are essential  to a successful explosive detection systems program. DHS stated that it  has initiated efforts to improve TSA’s management of EDS and ETD  maintenance costs and strengthen oversight of contract performance.  Regarding our recommendation that TSA establish a timeline to close out  the Boeing contact and report to congressional committees on its actions  to recover any excessive fees, DHS stated that TSA has conducted a  contract reconciliation process to ensure that no fees would be paid on  costs that exceeded the target due to poor contractor performance and  that Boeing and TSA have reached an agreement in principle on this  matter and the documentation is in the approval process with closure  anticipated in July 2006. Regarding our recommendation to establish a  timeline for completing the EDS life-cycle cost model, DHS stated that  TSA expects to complete its prototype evaluation in September 2006 and  that the EDS life-cycle cost model will be completed 12 months after the  prototype has been approved. Regarding our recommendation to revise  TSA policies and procedures to require documentation of its monitoring of  EDS and ETD maintenance contracts, DHS stated that a TSA contractor is  developing automated tools to perform multiple analyses of contractor- submitted data that DHS said would allow TSA to accurately and  efficiently certify the contractors’ performance against their contractual  requirements and would allow TSA to independently validate and verify  maintenance and cost data. The department’s comments are reprinted in  appendix II.
We will send copies of this report to the Secretary of Homeland Security  and the Assistant Secretary, Transportation Security Administration, and  interested congressional committees. We will also make copies available  to others upon request. In addition, the report will be available at no  charge on GAO’s Web site at http://www.gao.gov. If you or your staffs have  any questions or need additional information, please contact me at (202)  512-8777 or berrickc@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report are acknowledged in  appendix III.
section_title: Appendix I: Information for Congressional Committees
subsections: section_title: Introduction
subsections: section_title: House Conference Report 109-241—which accompanied the Department of Homeland Security Appropriations Act, 2006 (Public Law 109-90)1—directed GAO to report on the reasons for cost increases in maintaining TSA’s explosive detection systems,2 including TSA’s related contracting practices. In February 2006, a House Appropriations staffer told us that this information would assist in the committee’s budget deliberations.
paragraphs: H.R. Conf. Rep. No. 109-241, at 52 (2005). TSA interprets the term explosive detection system to include both explosive detection systems (EDS) and explosive trace detection (ETD) machines.
section_title: Objectives
subsections: section_title: What are the historical, current, and projected costs for the
paragraphs: maintenance of explosive detection systems (EDS) and  explosive trace detection (ETD) machines?
What factors played a role in EDS and ETD maintenance  costs and what factors could affect future costs?
What has TSA done to control EDS and ETD maintenance  costs?
To what extent does TSA oversee the performance of EDS  and ETD maintenance contractors?
section_title: Scope and Methodology
paragraphs: To determine TSA costs to maintain EDS and ETD machines we reviewed TSA contract files and budget documents for fiscal years 2003 through 2007, and interviewed TSA headquarters officials, Department of Homeland Security Office of the Inspector General  (DHS OIG) officials, and EDS and ETD contractor representatives. For purposes of our review, we  focused on the amounts obligated under contracts to maintain the machines. We did not review TSA’s  negotiations for maintenance services or the process for awarding contracts, nor did we assess other  direct or indirect costs related to TSA or DHS employees engaged in contract administration or other  related items.
To determine what factors played a role in maintenance costs and what TSA has done to control costs, we: reviewed TSA contract files, acquisition and strategic plans, budget documents, TSA processes for  reviewing contract cost and performance data, and a DHS OIG report;and interviewed TSA headquarters officials, DHS OIG officials, and EDS and ETD contractor representatives.
To determine the extent of TSA contract oversight, we: reviewed TSA contract files and processes for reviewing contract performance data, interviewed TSA headquarters officials and EDS and ETD contractor representatives, and  reviewed GAO standards for internal controls We performed our work from January 2006 through June 2006 in accordance with generally accepted  government auditing standards.
GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).
According to TSA budget documents, TSA has obligated almost $470 million from fiscal year 2002 through fiscal year 2005 for EDS and ETD  maintenance. In fiscal year 2006, TSA estimates it will spend $199 million  and has projected it will spend $234 million in fiscal year 2007.
TSA was unable to provide us data on maintenance cost per machine  prior to fiscal year 2005 because, according to TSA officials, its  previous contract with Boeing Service Company (Boeing) to maintain  EDS and ETD machines was not structured to capture these data.
TSA did not provide us with projections of EDS and ETD maintenance  costs beyond fiscal year 2007, although TSA has negotiated  maintenance prices per machine through fiscal year 2009.
section_title: Background
paragraphs: TSA was mandated to screen all checked baggage using explosive  detection systems at airports by December 31, 2003.1  Explosive Detection Systems (EDS) use computer-aided  tomography X-rays to recognize the characteristics of explosives.   In general, EDS are used for checked baggage screening.
Explosive Trace Detection (ETD) machines use chemical analysis  to detect traces of explosive material vapors or residues.  ETD  machines are used for both passenger carry-on baggage and  checked baggage screening.
According to TSA budget documents, TSA will have deployed over  1,400 EDS and 6,600 ETD machines at baggage screening  locations in over 400 airports nationwide by the end of fiscal year  2006.
The Aviation and Transportation Security Act, Pub. L. No. 107-71 § 110(b), 115 Stat. 597, 615 (2001) mandated, among other things, that all checked baggage at  U.S. airports be screened using explosive detection systems by December 31, 2002. Section 425 of the subsequently enacted Homeland Security Act of 2002,  Pub. L. No. 107-296, 116 Stat. 2135, 2185-86, in effect, extended this mandate to December 31, 2003. See 49 U.S.C. § 44901(d).
section_title: Background
paragraphs: TSA is responsible for the EDS and ETD maintenance costs   EDS and ETD maintenance includes preventative  maintenance—scheduled activities to increase machine  reliability that are performed monthly, quarterly, and yearly  based on the contractors’ maintenance schedules and  corrective maintenance—actions performed to restore  machines to operating condition after failure.
A TSA official told us that typical EDS warranties are for one year and that ETD warranties are for 2 years.
subsections: section_title: Background—TSA Cost Reimbursement Contract with Boeing
paragraphs: From June 2002 through March 2005, Boeing was the prime contractor for the installation and  maintenance of EDS and ETD machines at over 400 U.S. airports. TSA officials stated that the  Boeing contract was awarded at a time when TSA was a new agency with many demands and  extremely tight schedules for meeting numerous congressional mandates related to passenger  and checked baggage screening. Boeing had a cost reimbursement contractwith TSA, which  was competitively bid and contained renewable options to 2007.
Firm fixed price contracts provide for a price that is not subject to any adjustment on the basis of the contractor’s cost experience in performing the contract. This contract type places upon the contractor maximum risk and full responsibility for all costs and resulting profit and loss. It provides maximum incentive for the contractor to control costs and perform effectively and imposes a minimum administrative burden upon the contracting parties.
In March 2005, TSA signed firm fixed price contracts for EDS and ETD  maintenance.
TSA awarded a competitively bid contract to Siemens to provide maintenance  for ETD machines.
TSA negotiated sole source contracts with L-3 and GE InVision because they  are the original equipment manufacturers and owners of the intellectual  property of their respective EDS.
TSA can exercise 4 1-year options on all three contracts through March 2009.
In September 2005, TSA awarded a competitively bid firm fixed price contract to  Reveal Imaging Technologies, Inc., (Reveal) for both the procurement and  maintenance of a reduced-size EDS.
section_title: Historical, Current, and Projected Costs for
section_title: EDS and ETD Maintenance Costs
paragraphs: According to TSA budget documents, TSA  has obligated almost $470 million for EDS  and ETD maintenance from fiscal years  2002 through 2005.
EDS and ETD Machine Maintenance Budget Amounts, Fiscal Years 2002 through 2007  (In millions) maintenance grew from $14 million in  fiscal year 2002 to an estimated $199  million in fiscal year 2006. In fiscal year  2007, TSA projects it will spend $234  million.
Appropriated (as revised)
75 100 205 200 234 the maintenance cost per machine prior to  fiscal year 2005 because, according to  TSA officials, its previous contract with  Boeing to maintain EDS and ETD  machines was not structured to capture  these data.
According to TSA officials, in fiscal year  2004, TSA requested and received  approval to reprogram about $32 million  due to higher levels of maintenance costs  than expected.
In fiscal year 2005, TSA requested and  received approval to reprogram $25 million  to fund the L-3 contract ($16.6 million) and  to closeout the Boeing contract ($8.4  million), which has yet to be closed.
section_title: EDS and ETD Maintenance Costs
paragraphs: TSA officials did not provide us with projections of costs  beyond 2007. However, current contracts have negotiated  maintenance prices per machine through March 2009, if TSA  decides to exercise option years in the contracts.
Future EDS and ETD maintenance costs depend on  decisions made as outlined in a February 2006 TSA strategic  planning framework for screening checked baggage using  EDS and ETD. Among other things, the plan discusses  options for the deployment of new technologies and  refurbishment of existing equipment.
subsections: section_title: Factors That Played a Role in EDS and ETD
section_title: Factors That Played a Role in Maintenance Costs and Could Impact Future Costs
paragraphs: Different factors have played a role in costs to date and will influence future maintenance costs for EDS  According to a September 2004 DHS OIG report, TSA did not follow sound contracting practices in  administering the Boeing contract, which was primarily for the installation and maintenance of EDS and  ETD machines.
Among other things, the DHS OIG found that TSA had paid provisional award fees totaling $44  million through December 2003 without any evaluation of Boeing’s performance.
GAO has identified similar instances of agencies’ failure to properly use incentives in making award fees. See GAO, Defense Acquisitions: DOD Has Paid Billions in  Award and Incentive Fees Regardless of Acquisition Outcomes, GAO-06-66 (Washington, D.C.: December 2005).
section_title: Factors That Played a Role in Maintenance Costs and Could Impact Future Costs
subsections: section_title: Under the current firm fixed price contracts, if TSA exercises
paragraphs: For EDS contracts, future labor and material costs could not be determined, so TSA negotiated an escalation factor to be used to determine pricing for the contract option years. For the ETD  contracts, TSA determined after a review of cost data, that it would apply a 4 percent escalation factor to prices in the contract option years. The employment cost index is a measure of the  change in the cost of labor, free from the influence of employment shifts among occupations and industries. The consumer price index is a measure of the average change in prices over time  of goods and services purchased by households.
section_title: Factors That Played a Role in Maintenance Costs and Could Impact Future Costs
subsections: section_title: Future maintenance costs will be impacted by a range of factors, including the number of
paragraphs: machines deployed and out of warranty, conditions under which machines operate, mean  downtime requirements, the emergence of new technologies or improved equipment, and  alternative screening strategies.
TSA’s February 2006 strategic plan framework for screening checked baggage over  the next 20 years discusses factors that may impact future maintenance costs. For  example, the framework discusses the refurbishment of existing machines and the  deployment of new technologies, but does not outline the number of machines or  specific time frames for implementation. Additionally, the impact of these strategies  on future maintenance costs is unknown.1 If no new equipment or maintenance providers emerge, TSA may pay a premium in  future sole source contracts where intellectual property rights are involved. For  example, because L-3 and GE InVision had intellectual property rights on their  machines, their maintenance contracts were not bid competitively and therefore  prices were not subject to the benefits of market forces.
TSA issued its strategic plan framework for screening checked baggage using EDS and ETD machines in response to various congressional mandates, congressional committee directives, and GAO recommendations.
section_title: Factors That Played a Role in Maintenance Costs and Could Impact Future Costs
subsections: section_title: Lifecycle cost estimates were not developed for the Boeing,
paragraphs: Siemens, L-3, and GE contracts before the maintenance  contracts were executed and, as a result, TSA did not have a  complete picture of all maintenance costs. In August 2005,  TSA hired a contractor to define parameters for a lifecycle  cost model. A TSA official told us that the contractor began  work on a lifecycle cost model for EDS in February 2006 and  did not know when the model would be completed.
section_title: TSA Actions to Control
section_title: TSA Actions to Control EDS and ETD Maintenance Cost Increases
subsections: section_title: Price certainty is guaranteed
paragraphs: for up to five years if TSA  exercises options to 2009.
section_title: contracts were competitively bid, although the GE InVision and L-3 contracts were sole source contracts.
paragraphs: TSA did not provide per-unit  maintenance costs prior to March  2005 because the Boeing contract  was not structured to capture these  data.
ETD Smiths Ionscan 400A Smiths Ionscan 400AE Smiths Ionscan 400B Thermo EGIS 3000 Thermo EGIS II GE Iontrack Itemiser-W 241 5 3,038 2 425 2,302 10,525 10,525 8,580 12,899 13,134 $  7,727 336 6 3,035 2 545 2,322 10,974 10,974 8,946 13,526 13,695 $  8,057 NOTE: Maintenance costs represent the negotiated prices in the maintenance contracts for EDS and  ETD machines.
section_title: TSA Actions to Control EDS and ETD Maintenance Cost Increases
paragraphs: TSA included several contractor performance requirements in the  Siemens, L-3, GE InVision, and Reveal contracts.
Metrics related to Reliability, Maintainability, and Availability  (RMA) of the machines must be reported to TSA.1  Specific cost data related to maintenance and repair must be  reported to TSA.
Contractors are required to meet monthly with TSA to review all  pertinent technical, schedule, and cost aspects of the contract, including an estimate of the work to be accomplished in the  next month; performance measurement information; and any  current and anticipated problems.
Includes metric s such as m ean tim e between failur es (gener ally the total time a m achi ne is av ailable to perform its required missi on divided by the num ber of  failur es over a  given per iod of time) and oper ational availability (generally the percentage of time, duri ng operational hours, that a machine is available to perform  its requir ed mission). Such reliability, maintainability, and  availability data ar e standard and appropri ate perform anc e requirements for maintenanc e contracts.
section_title: TSA Actions to Control EDS and ETD Maintenance Cost Increases
paragraphs: Provisions in the L-3 and GE InVision contracts specify that the agreed price for maintaining EDS will  be paid only if the contractor performs within specified mean downtime (MDT) requirements.
MDT is calculated by the number of hours a machine is out of service in a month divided by  the number of times that machine is out of service per month.
Contractors submit monthly invoices for 95 percent of the negotiated contract price for the  month and then submit an MDT report to justify the additional 5 percent. Consequently, if the  contractor fails to fulfill the MDT requirements, it is penalized 5 percent of the negotiated  monthly maintenance price.
As of February 2006, neither GE InVision nor L-3 have been penalized for missing their MDT.
The allowable MDT is lowered from 2005 to subsequent renewable years in the contract, as  shown in the table below.
subsections: section_title: EDS and ETD Contractor Performance
section_title: TSA Oversight of EDS and ETD Contractor Performance
paragraphs: TSA’s acquisition policiesand GAO’s standards for internal controlsTSA officials provided no evidence that they are reviewing maintenance cost data provided by the  contractor because they are not required to document such activities.  For example, even though TSA  officials told us they are reviewing required contractor data, including actual maintenance costs related  to labor hours, costs associated with replacement parts, and the costs of shipping machine parts, they  did not have any documentation to support this. TSA officials told us that they have begun to capture  these data to assist them in any future contract negotiations.
TSA officials provided no evidence that performance data for corrective and preventative maintenance  required under the contract is being reviewed. TSA officials told us that they perform such reviews, but  do not document their activities since there are no TSA policies or procedures requiring them to do so.  Therefore, TSA could not provide assurance that contractors are complying with contract performance  requirements. For example, although TSA documents monthly meetings with contractors to discuss  performance data, TSA did not provide evidence that it independently determines the reliability and  validity of data required by the contracts, such as mean time between failures and mean time to repair,  which are important to making informed decisions about future purchases of EDS and ETD equipment  and their associated maintenance costs.
GAO/AIMD-00-21.3.1.
For EDS contracts with possible financial penalties, TSA officials  told us that they review contractor-submitted mean downtime data  on a monthly basis to determine the reliability and validity of the  data and to determine whether contractors are meeting contract  provisions or should be penalized. However, TSA officials said they  do not document these activities because there are no TSA policies  or procedures to do so.
As a result, without adequate documentation, there is no assurance  on whether contractors are meeting contract provisions that TSA  has ensured that it is making appropriate payments for services  provided.
section_title: Conclusions
paragraphs: TSA’s move to firm fixed price maintenance contracts was advantageous to the  government in that it helps control present and future maintenance costs. Firm fixed price  contracts also help ensure price certainty and therefore are more predictable.
Unresolved issues remain with the past contractor, specifically fees awarded to former  contractor Boeing that may have been excessive due to a lack of timely evaluation of the  contractor’s performance.
Although TSA has begun to develop a lifecycle cost model in order to control costs and  negotiate future contracts, TSA has not set a timeframe to complete this model. Without  such a time frame, TSA may not be identifying cost efficiencies and making informed  procurement decisions.
Further, TSA must provide evidence of its reviews and analyses of contractor-submitted  data and perform analyses of contractor data to determine the reliability and validity of the  data and to provide assurance of contractor compliance with contract performance  requirements and internal control standards. Without stronger oversight, TSA will not have  reasonable assurance that contractors are performing as required and that full payment is  justified based on meeting mean downtime requirements.
section_title: Recommendations
subsections: section_title: To help improve TSA’s management of EDS and ETD maintenance costs and
paragraphs: strengthen oversight of contract performance, we recommend that the Secretary of  Homeland Security instruct the Assistant Secretary, Transportation Security  Administration to take the following three actions report to the congressional appropriations committees on its actions, including any  necessary analysis, to address the DHS OIG recommendation to recover any  excessive fees awarded to Boeing;  establish a time line for completing a lifecycle cost model for EDS, which TSA  revise its policies and procedures to require documentation of its monitoring of EDS  and ETD maintenance contracts to provide reasonable assurance that contractor maintenance cost data and performance data are recorded and  reported in accordance with TSA contractual requirements and self-reported contractor mean downtime data are valid, reliable, and justify the  full payment of the contract amount.
section_title: Agency Comments
paragraphs: TSA reviewed these slides in their entirety and provided several technical comments, which we incorporated as appropriate. TSA  officials told us that they are not making formal comments on our  recommendations.
section_title: Appendix II: Agency Comments
section_title: Appendix III: GAO Contact and Staff Acknowledgements
subsections: section_title: GAO Contact
section_title: Staff Acknowledgements
paragraphs: In addition to the individual names above, Charles Bausell, R. Rochelle  Burns, Glenn Davis, Katherine Davis, Michele Fejfar, Richard Hung, Nancy  Kawahara, Dawn Locke, Thomas Lombardi, Robert Martin, and William  Woods.