Defense Acquisitions: DOD Needs to Better Support Program Managers' Implementation of Anti-Tamper Protection
Published: 2004-03-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The U.S. government has invested hundreds of billions of dollars in developing the most sophisticated weapon systems and technologies in the world. Yet, U.S. weapons and technologies are vulnerable to exploitation, which can weaken U.S. military advantage, shorten the expected combat life of a system, and erode the U.S. industrial base's technological competitiveness. In an effort to protect U.S. technologies from exploitation, the Department of Defense (DOD) established in 1999 a policy directing each military service to implement anti-tamper techniques, which include software and hardware protective devices. This report reviews DOD's implementation of the anti-tamper policy as required by the Senate report accompanying the National Defense Authorization Act for Fiscal Year 2004.
section_title: What GAO Found
paragraphs: Program managers have encountered difficulties in implementing DOD's anti-tamper policy on individual weapon systems. First, defining a critical technology--a basis for determining the need for anti-tamper--is subjective, which can result in different conclusions regarding what needs anti-tamper protection. While different organizations can check on program managers' assessments, no organization has complete information or visibility across all programs. Some program managers said they needed assistance in determining which technologies were critical, but resources to help them were limited or unknown and therefore not requested. Second, anti-tamper protection is treated as an added requirement and can affect a program's cost and schedule objectives, particularly if the program is further along in the acquisition process. Programs GAO contacted experienced or estimated cost increases, and some encountered schedule delays when applying antitamper protection. Officials from one program stated that their existing budget was insufficient to cover the added cost of applying anti-tamper protection and that they were waiting for separate funding before attempting to apply such protection. Finally, anti-tamper techniques can be technically difficult to incorporate in some weapon systems--particularly when the techniques are not fully developed or when the systems are already in design or production. One program that had difficulty incorporating the techniques resorted to alternatives that provided less security. While DOD is overseeing the development of generic anti-tamper techniques and tools to help program managers, many of these efforts are still in progress, and program managers ultimately have to design and incorporate techniques needed for their unique systems.

section_title: Letter
paragraphs: The U.S. government has invested hundreds of billions of dollars in  developing the most sophisticated weapon systems and technologies in  the world. Yet, U.S. weapons and technologies can be exposed to the risk  of compromise when they are exported, stolen, lost during combat, or  damaged during routine missions. When U.S. technologies are  compromised, it can weaken U.S. military advantage, shorten the expected  combat life of a system, and erode the U.S. industrial base’s technological  competitiveness in the international marketplace.
In an effort to protect U.S. technologies from exploitation, the Under  Secretary of Defense for Acquisition, Technology, and Logistics in 1999  directed each military service to implement anti-tamper techniques.  Program managers are responsible for applying the techniques on  individual weapon systems. The Senate report accompanying the National  Defense Authorization Act for Fiscal Year 2004 required us to review the  Department of Defense’s (DOD) implementation of the anti-tamper policy.  We conducted our work between February 2003 and August 2003 in  accordance with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: DOD increasingly relies on advanced technology in its weapons for  effectiveness on the battlefield and actively seeks to include foreign  partners in weapon system development and acquisition. DOD’s policy  also encourages the sale of certain weapons to foreign governments  through the Foreign Military Sales Program and direct commercial sales  made by companies. While these efforts have the potential to enhance  coalition operations and reduce weapons’ unit costs, DOD has  acknowledged that the efforts also risk making U.S. technologies  potentially vulnerable to exploitation. DOD reported that an increasing  number of countries have reverse engineering capability and actively seek  to obtain U.S. technology through various means.
As a method to protect critical technologies, the Under Secretary of  Defense for Acquisition, Technology, and Logistics directed the military  services in 1999 to implement anti-tamper techniques. While the  techniques will not prevent exploitation, they are intended to delay or  discourage attempts to reverse engineer critical technologies in a weapon  system or develop countermeasures to a system or subsystem.
In 2001, the Under Secretary of Defense for Acquisition, Technology, and  Logistics designated the Air Force as the Executive Agent responsible for  implementing DOD’s anti-tamper policy. The Executive Agent oversees an  annual budget of about $8 million per year to implement policy and  manage anti-tamper technology projects through the Air Force Research  Laboratory. DOD, in conjunction with the Air Force Research Laboratory  and the Department of Energy’s Sandia National Laboratories, also holds  periodic information sessions to educate the acquisition community about  anti-tamper policy, guidance, and technology developments. In addition,  military services and defense agencies, such as the Missile Defense  Agency, have an anti-tamper focal point to coordinate activities.
Program managers are responsible for considering anti-tamper measures  on any weapon system with critical technologies. Since it is not feasible to  protect every technology, program managers are to conduct an assessment  to determine if anti-tamper protection is needed. The first step of the  decision process is to determine if the system has critical technologies. If  program managers determine the system has no critical technologies, they  are to document that decision according to draft guidance. Program  managers of systems that contain critical technologies complete the  remaining steps of the process. Based on draft guidance, program  managers are to conceptually address how they will implement anti- tamper measures at system development, otherwise known as   milestone B. DOD’s anti-tamper decision process is illustrated in figure 1.
Program managers can obtain assistance on their assessments from  government laboratories, contractors, and the intelligence community.  They are required to document the decision to use or not to use anti- tamper techniques in a classified annex of the program protection plan,  which is subject to approval from the program’s milestone decision  authority.
Anti-tamper techniques vary depending on the type of protection the  system requires. An example of an anti-tamper technique is software  encryption, which scrambles software instructions to make them  unintelligible without first being reprocessed through a deciphering  technique. Another example is a thin opaque coating placed on  microelectronic components, which makes it difficult to extract or dissect  the components without great damage. Programs can apply multiple anti- tamper techniques to a critical technology. For example, a program could  encrypt critical data on a microelectronic chip that is also covered with a  protective coating. Each layer of protection could act as an obstacle to  reverse engineering.
section_title: Anti-Tamper Implementation Has Been Hampered by Several Factors, and Support to Address Them Has Been Limited
paragraphs: Implementation of the anti-tamper policy has been hampered by several  factors. First, identification of critical technology is subject to  interpretation and program managers and DOD officials can and have  arrived at different conclusions about what needs to be protected. Second,  applying anti-tamper protection can take time and money, which may  compete with a program manager’s cost and schedule objectives. Finally,  some programs found it difficult to apply anti-tamper techniques when the  techniques were not fully developed, and others were unsure which  techniques were available to them. In general, the later anti-tamper  techniques are applied, the more difficult and costly it can be to  implement. Thus far, support to help program managers address some of  these factors has been limited.
subsections: section_title: Different Interpretations of Critical Technologies May Increase the Risk of Some Going Unprotected
paragraphs: DOD officials acknowledged that the identification of critical  technologies—a basis for determining if anti-tamper protection is  needed—is subjective, which can result in different conclusions regarding  what needs protection. DOD’s Program Managers Anti-Tamper Handbook  defines technology as critical if compromise results in degrading combat  effectiveness, shortening the expected combat life of the system, or  significantly altering program direction. While a broad definition allows  for flexibility to determine what is critical on individual systems, it may  increase the risk that the same technology is protected on some systems  but not on others or that different conclusions can be reached on whether  programs have critical technologies. For example:    An official from an intelligence agency described a case where two  services used the same critical technology, but only one identified the  technology as critical and provided protection. The intelligence agency  official speculated that if exploited, knowledge gained from the  unprotected system could have exposed the technology on both  systems to compromise. While both systems were ultimately protected,  the intelligence agency official stated that the situation could occur  again.
Officials from the Executive Committee told us that two program  managers stated that their systems had no critical technologies and  therefore were not subject to the anti-tamper policy. Both managers  were directed by the Executive Committee to reconsider their  determination and apply anti-tamper protection. As a result, one  program is in the process of determining which technologies are  critical, and the other program is applying anti-tamper protection as a  condition to export the system.
While different conclusions can be reached regarding what is critical,  various organizations can serve as a check on a program manager’s  assessment. However, no organization has complete information or  visibility of all programs across the services and agencies. For example,  the anti-tamper Executive Agent and the military service focal points do  not have full knowledge about which program offices have or have not  identified critical technologies or applied anti-tamper protection. In 2001,  DOD attempted to collect such information, but not all programs  provided data and DOD did not corroborate what was provided to ensure  that program officials were consistently assessing critical technologies.  The Executive Agent stated that there are no plans to update this data.  Conducting oversight over program managers’ assessments may be  difficult because of limited resources. Specifically, the Executive Agent  has two full-time staff and the military service focal points perform duties  other than anti-tamper management. Furthermore, according to a military  official, program offices that determine they have no critical technologies  are not required to obtain the focal points’ concurrence. While other  organizations can review a program manager’s critical technology  assessment as part of various acquisition and export processes, they may  not have a full perspective of the assessments made by all programs across  the services and the agencies. For example, different milestone decision  authorities only review an individual program manager’s critical  technology decisions for programs coming under their responsibility. Also,  the Executive Committee may weigh in on the determinations, but it only  reviews exports involving stealth technology.
While it was apparent that the systems had critical technologies, some  program managers needed assistance to determine which specific  technologies were critical. For example, a program office tasked the  contractor to identify critical technologies, and it has worked for months  with the contractor to agree upon and finalize a list of critical technologies  on the system. Also, an intelligence official, who is available to assist  program managers in assessing their systems’ criticality, found that some  program managers identified too many technologies as critical and that  others did not identify all of the systems’ critical elements. In one instance,  a program manager indicated that a system had 400 critical technologies,  but an intelligence agency narrowed down the list to about 50 that it  considered critical. In another case, a program manager concluded that an  entire system was one critical technology, but the intelligence agency  recommended that the system’s technologies be broken down and  identified approximately 15 as critical.
Although there are various resources to help program managers identify  critical technologies, they may have limited utility, or may not be known,  and therefore not requested. For example, the Militarily Critical  Technologies List—cited in guidance as a primary reference for program  managers—may not be up to date and may not include all technologies,  according to some DOD officials. Another resource—the Program  Managers Anti-Tamper Handbook—contains information regarding critical  technology determinations, but program managers are not always aware  that the handbook exists, in part because it is not widely distributed. In  addition, the Defense Intelligence Agency can conduct an independent  assessment of a system’s critical elements and technologies, if requested  by the program manager. However, many officials we interviewed were  unaware that the agency provides this assistance. According to a military  official, the focal points are available to review a program manager’s  assessment if requested.
In some instances, program managers may have differing perceptions of  what constitutes a critical technology. According to DOD’s guidance,  critical technologies can be either classified or unclassified. However, an  anti-tamper focal point stated that there is a perception that the anti- tamper policy only applies to classified programs. We found in one  instance that the manager for a weapon program stated that the program  did not require anti-tamper protection because it had no critical  technologies that were classified.
section_title: Applying Anti-Tamper Protection Can Affect Cost and Schedule Objectives
paragraphs: Applying anti-tamper protection takes money and time, which can affect a  program manager’s cost and schedule objectives. Generally, anti-tamper  implementation is treated as an added requirement that is not separately  funded for most programs. Program officials acknowledged that anti- tamper costs can be difficult to estimate and isolate because they are  intertwined with other costs, such as research and development or  production costs. As we have found in prior work, the later a requirement  is identified, the more costly it is to achieve.
Most programs we visited experienced or estimated cost increases, and  some encountered schedule delays as they attempted to apply anti-tamper  techniques. For example:    A program official told us the anti-tamper protection for a program  upgrade increased both design and production costs for the receiver  unit. The program official stated that the anti-tamper protection  increased total unit cost by an estimated $31 million, or 10 percent.  Program officials expressed concern that unit cost increases may affect  procurement decisions, particularly for one service, which is the  largest acquirer of units and may be unable to purchase the proposed  number.
A program office estimated that it needs a budget increase of   $56 million, or 10 percent, to fund the desired anti-tamper protections.  Officials from that program told us that the existing program budget  was inadequate to fund the added anti-tamper requirements. As a  result, the program manager requested, and is waiting for, separate  funding before attempting to apply anti-tamper protection to the  system.
One program office awarded a contract modification for the design,  implementation, and testing of anti-tamper techniques valued at   $12.5 million. Initially, the contractor had estimated the anti-tamper  costs to be $35 million, but the program office did not approve all  techniques suggested by the contractor. In addition, the contractor  estimated that the recurring unit price for anti-tamper protection on  future production lots may be $3,372 per unit. The U.S. government and  the contractor have not completed unit price negotiations. Program  officials told us that anti-tamper implementation contributed to a   6-month schedule delay.
Another program office estimated that $87 million is needed to protect  two critical technologies with multiple anti-tamper techniques. The  program office expects that half of the anti-tamper budget will be used  to test the techniques. The anti-tamper protection will only be applied if  the system is approved for export. At that time, program officials will  reexamine the anti-tamper cost estimates. In addition, it may take   5 years to adequately apply the techniques.
Officials from an international program stated that, thus far, they have  experienced a 60-day schedule delay while they wait for the contractor  to estimate the system’s anti-tamper cost. Program officials stated that  the potential for increased costs and additional schedule delays is high.
Program officials and representatives from the Executive Committee  stated that the cost of anti-tamper protection can be significantly  higher for an international program for various reasons, including that  the U.S. version and the international version of the system may require  different anti-tamper techniques.
Cost and schedule impacts may also be more significant if the programs  are further along in the acquisition process when program offices first  attempt to apply anti-tamper protection. Several programs that have  experienced significant cost increases or delays were in or beyond the  program development phase when they attempted to apply anti-tamper  techniques. For example, when the anti-tamper policy was issued, one  program had just obtained approval to begin system development and  program officials believed it was too late to implement anti-tamper  protection. As a result, the program received an interim waiver of the  anti-tamper policy, and it only plans to apply anti-tamper techniques if the  system is approved for export. While DOD has not systematically collected  cost data for anti-tamper application across programs, DOD officials have  stated that it is more cost-effective for programs to consider anti-tamper  requirements at program inception, rather than later in the acquisition  process. An official from a program that applied anti-tamper techniques in  the production phase stated that ideally a program should identify its anti- tamper needs, including cost and technology, as early as possible. Recent  Army anti-tamper guidance indicates that programs should receive  approval for their preliminary anti-tamper plans at the concept stage.
section_title: Needs Outpace Availability of Techniques and Tools
paragraphs: Anti-tamper techniques can be technically difficult to incorporate on a  weapon system, such as when the technology is immature. DOD is  working to oversee the development of generic anti-tamper techniques and  tools to help program managers identify potential techniques, but many of  these efforts are still in progress and it is uncertain how they will help  program managers. While program managers want knowledge about  generic techniques, they ultimately have to design and incorporate  techniques needed for their unique systems to ensure protection of critical  technologies and to meet performance objectives.
Problems in applying anti-tamper techniques typically arose when the  programs were already in design or production or when the techniques  were not fully developed or specifically designed for the system. For  example:    Officials from a program told us that they experienced problems when  applying an anti-tamper protective coating. Because the team applying  the coating did not coordinate with teams working on other aspects of  the system, the problems with the coating were not discovered until  just before production. Prior to an initial development test, the  program office received a temporary waiver to test the system without  the anti-tamper technique because the coating caused malfunctioning.  The program office and its contractor are working to resolve issues  with the anti-tamper technique.
A program office was not able to copy anti-tamper techniques used by a  similar program and, therefore, attempted to apply a generically  developed anti-tamper coating, which resulted in problems.  Specifically, the coating caused the system to malfunction, so the  program office requested assistance from a national laboratory, but the  laboratory’s solution melted key components of the system. Therefore,  the program office requested that the contractor develop a new coating  and other methods of protection for the system. The contractor’s anti- tamper techniques were successfully applied to the system.
One program required advanced anti-tamper techniques to protect  miniaturized internal components, but the technology was still in  development and not available for immediate application. According to  program officials, research and development of the anti-tamper  technique was originally expected to be completed in 2002 and is now  estimated to be available in 2006. Currently, officials are uncertain that  the technique will meet their needs because the technique is being  generically developed. In the absence of being able to apply the anti- tamper technique, the program received approval from DOD to use  procedural protections, whereby U.S. military personnel provide  physical security of the system when it is used in foreign countries,  which includes locking the unit in a protected room to restrict access  by foreign nationals. DOD officials stated that physical security can be  less reliable than actual anti-tamper protection.
Some program managers told us that they need more help in deciding  what anti-tamper techniques they should apply to their individual systems.  To provide information, DOD has a classified database that describes  current anti-tamper techniques. An Air Force Research Laboratory official  stated that they are in the process of updating this database, developing a  rating system on the value of various techniques to be included in the  database, and creating a classified technology road map that will prioritize  the needs for various anti-tamper techniques. These tools are currently  unavailable. DOD and Sandia National Laboratories also have provided  information on anti-tamper techniques and tools to program managers at  periodic workshops where attendance is voluntary.
To further assist program managers, DOD is in the process of overseeing  the development of generic anti-tamper techniques, but it is uncertain to  what extent such techniques address a program’s specific needs. In 2001,  DOD issued several contracts to encourage anti-tamper technology  development. To date, several defense contractors have provided anti- tamper technology concepts, but according to the Executive Agent,  programs need to further develop the technology before it can be applied  to and function on a particular system. According to Air Force Research  Laboratory and Sandia National Laboratories officials, generic anti-tamper  techniques can be considered, but program managers have to design and  incorporate the techniques needed for their unique systems. Program  managers ultimately have to ensure that the techniques protect critical  technologies and do not adversely affect performance objectives for the  system.
section_title: Conclusions
paragraphs: Anti-tamper protection is one of the key ways DOD can preserve U.S.  investment in critical technologies, while operating in an environment of  coalition warfare and a globalized defense industry. However,  implementation of the anti-tamper policy, thus far, has been difficult—in  part because DOD has not developed an implementation strategy to ensure  success. For program managers expected to implement anti-tamper  protection, the policy can compete with their goals of meeting cost and  schedule objectives, particularly when the anti-tamper requirement is  identified late in the system development process. Without providing more  oversight and guidance about what needs to be protected and how to do  so, DOD is at risk of program managers making decisions on individual  programs that can result in unprotected technologies and have negative  consequences for maintaining the military’s overall technological  advantage.
section_title: Recommendations for Executive Action
paragraphs: We are recommending that the Secretary of Defense direct the Under  Secretary of Acquisition, Technology, and Logistics and the anti-tamper  Executive Agent to take the following five actions to improve oversight  and assist program offices in implementing anti-tamper protection on  weapon systems.
To better oversee identification of critical technologies for all programs  subject to the anti-tamper policy, we recommend that the Secretary of  Defense direct the Under Secretary for Acquisition, Technology, and  Logistics, in coordination with the Executive Agent and the focal points, to  (1) collect from program managers information they are to develop on  critical technology identification and (2) appoint appropriate technical  experts to centrally review the technologies identified for consistency  across programs and services.
To better support program managers in the identification of critical  technologies, the Secretary of Defense should direct the Under Secretary  for Acquisition, Technology, and Logistics, in coordination with the  Executive Agent and the focal points, to (1) continue to identify available  anti-tamper technical resources, (2) issue updated policy identifying roles  and responsibilities of the technical support organizations, and (3) work  with training organizations to ensure training includes practical  information on how to identify critical technologies.
To help minimize the impact to program cost and schedule objectives, the  Secretary of Defense should direct the Under Secretary for Acquisition,  Technology, and Logistics to work with program managers to ensure that  the cost and techniques needed to implement anti-tamper protection are  identified early in a system’s life cycle and to reflect that practice in  guidance and decisions.
To maximize the return on investment of DOD’s anti-tamper technology  efforts, the Secretary of Defense should direct the Executive Agent to  monitor the value of developing generic anti-tamper techniques and  evaluate the effectiveness of the tools, once deployed, in assisting program  managers to identify and apply techniques on individual programs.
To ensure successful implementation of the anti-tamper policy, the  Secretary of Defense should direct the Under Secretary for Acquisition,  Technology, and Logistics to develop a business case that determines  whether the current organizational structure and resources are adequate  to implement anti-tamper protection and if not, what other actions are  needed to mitigate the risk of compromise of critical technologies.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD partially concurred  with one recommendation and offered an alternative solution, which we  did not incorporate. DOD concurred with our remaining four  recommendations and provided alternative language for two, which we  incorporated as appropriate. DOD’s letter is reprinted in the appendix.
DOD partially concurred with our recommendation to collect and centrally  review the program’s critical technology identifications and proposed,  instead, that it develop a standardized process to minimize subjectivity,  incorporate that process into anti-tamper policy, and monitor subsequent  implementation. As part of its rationale, DOD stated that technical  representatives in the services currently work with program managers to  implement the anti-tamper policy and that quarterly conferences and  seminars are ways to disseminate important information to program  managers. We believe DOD’s proposal is an improvement over the current  process given that program managers need more technical support and  guidance to identify critical technologies. However, we do not believe  DOD’s proposal is sufficient because a central review mechanism is  needed to ensure consistent critical technology identification across the  services and the agencies. Without central visibility over program  managers’ critical technology identifications, the risk exists that the same  technology is protected on some systems but not on others. Knowledge  gained from unprotected systems can expose critical technology to  compromise, which minimizes the impact of anti-tamper protection. In  addition, DOD’s dissemination of information at conferences may be  limited because conference attendance is voluntary and all program  managers may not attend and receive the information. Given the need for  consistency and a central review, we did not revise our recommendation.
DOD concurred with our remaining recommendations, but offered  alternative language for two, which we incorporated. Specifically, for our  recommendation aimed at better supporting program managers in  identifying critical technologies, DOD proposed adding language that  underscored the need for identifying technical resources and maintaining  up-to-date policies on technical support organizations’ roles and  responsibilities. While DOD has identified some resources and listed them  in several documents, it has not developed a comprehensive list of  resources to assist program managers. Therefore, we added to our  recommendation that DOD continue to identify available anti-tamper  technical resources. For our recommendation that DOD evaluate generic  anti-tamper techniques, DOD proposed language that offered greater  flexibility, which seemed reasonable and we incorporated.
section_title: Scope and Methodology
paragraphs: To determine how DOD implemented the anti-tamper policy, we collected  data and interviewed officials from 17 programs, which were identified by  DOD as having experience with implementing the policy or by us through  our review. Twelve of the 17 programs reported that their systems had  critical technologies, and most were in various stages of implementing the  anti-tamper policy. From those programs we selected six for an in-depth  review. We conducted structured interviews with the six programs that  had identified critical technologies on their systems to understand their  experiences with applying anti-tamper techniques. We selected systems  that represented a cross-section of acquisition programs and various types  of systems in different phases of development. To the extent possible,  when selecting the programs for an in-depth review, we considered factors  that may increase a system’s vulnerability and exposure to exploitation.  We also considered whether the system was approved for export by  examining the Defense Security Cooperation Agency’s data on foreign  military sales. In addition, we analyzed available program information  from the anti-tamper Executive Agent and the military focal points to  determine programs reporting critical technologies and anti-tamper plans.  DOD acknowledged that the information was incomplete, and we did not  independently verify the reliability of the data.
We supplemented the program information by interviewing the Executive  Agent, the military focal points, representatives from the intelligence  community, DOD’s Executive Committee, the Department of Energy’s  Sandia National Laboratories, the Air Force Research Laboratory, defense  contractors, and an electronic security specialist. We also discussed  DOD’s anti-tamper policy with current and former officials from the Office  of the Secretary of Defense. To observe DOD’s training of program  managers, we attended a DOD anti-tamper information workshop and a  quarterly review. We analyzed pertinent DOD policies, directives,  instructions, and guidance governing anti-tamper protection on systems.  We also conducted a literature search to obtain information on program  protection and industry practices related to anti-tamper measures.
We are sending copies of this report to interested congressional  committees; the Secretary of Defense; and the Director, Office of  Management and Budget. We will make copies available to others upon  request. In addition, this report will be available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please call  me at (202) 512-4841. Others making key contributions to this report were  Anne-Marie Lasowski, Yelena T. Harden, Gregory K. Harmon, and Holly  Ciampi.
section_title: Appendix: Comments from the Department of Defense