Defense Acquisitions: Departmentwide Direction Is Needed for Implementation of the Anti-tamper Policy
Released: 2008-01-11T00:00:00

Letter

Background

To protect its critical assets, DOD has established several protection measures for weapon systems. These measures include information assurance to protect information and information systems, software protection to prevent the unauthorized distribution and exploitation of critical software, and anti-tamper techniques to help delay exploitation of technologies through means such as reverse engineering when U.S. weapons are exported or lost on the battlefield. Examples of anti-tamper techniques include software encryption, which scrambles software instructions to make them unintelligible without first being reprocessed through a deciphering technique, and hardware protective coatings designed to make it difficult to extract or dissect components without damaging them.

In 1999, the Under Secretary of Defense for Acquisition, Technology, and Logistics (AT&L) issued a policy memorandum for implementing anti- tamper protection in acquisition programs. In the following year, AT&L issued a policy memorandum stating that technologies should be routinely assessed during the acquisition process to determine if they are critical and if anti-tamper techniques are needed to protect these technologies. In 2001, an AT&L policy memorandum designated the Air Force as the Anti- Tamper Executive Agent. The executive agent’s office, which currently has four staff, is responsible for implementing DOD’s anti-tamper policy and managing anti-tamper technology development through the Air Force Research Laboratory. The executive agent also holds periodic information sessions to educate the acquisition community about anti-tamper policy, initiatives, and technology developments. To coordinate activities, military services and defense agencies, such as the Missile Defense Agency, have an anti-tamper point of contact. Program managers are responsible for ensuring anti-tamper protection is incorporated on any weapon system with critical technologies that need protection. Since it is not feasible to protect every technology, program managers are to conduct an assessment to determine if anti-tamper protection is needed.

When assessing if anti-tamper protection is needed, program managers make several key decisions regarding the identification of critical technologies, assessment of threats and vulnerabilities, and determination of anti-tamper techniques or solutions. The process begins with determining whether or not their system’s critical program information includes any critical technologies. If it is determined that the system has no critical technologies, program managers are to document the decision and request concurrence from either the office within their component that is designated with anti-tamper responsibilities or the Anti-Tamper Executive Agent. For systems that are determined to have critical technologies, the next key steps are to identify potential threats and vulnerabilities and select anti-tamper techniques to protect those technologies. Techniques are ultimately verified and validated by a team composed of representatives from the DOD components. The program manager documents decisions in an annex of the program protection plan.

In 2004, we reported that program managers had difficulty in carrying out DOD’s anti-tamper policy on individual weapons, such as identifying critical technologies and experiencing cost increases or schedule delays when applying anti-tamper techniques—particularly when the techniques are not fully developed or when the systems are already in design or production. We made several recommendations, including increasing oversight over the identification of critical technologies across programs, improving tools and resources for program managers in identifying critical technologies, ensuring early identification of anti-tamper costs and solutions, monitoring the development of generic anti-tamper solutions and evaluating their effectiveness, and developing a business case to determine whether the current organizational structure and resources are adequate. DOD concurred or partially concurred with these recommendations. DOD has taken some steps to implement our recommendations including identifying available anti-tamper technical resources and developing a searchable spreadsheet of critical technologies, incorporating information in the Defense Acquisition Guidebook on the need for early identification of anti-tamper solutions in a weapon system, and sponsoring a study on anti-tamper techniques and their general effectiveness. While DOD has taken these steps to address parts of the recommendations, all remain open.

DOD Lacks Departmentwide Direction for Implementing its Anti- tamper Policy

DOD has recently taken several actions aimed at raising awareness about its anti-tamper policy and assisting program managers in implementing anti-tamper protection on a weapon system. Despite these actions, DOD still lacks departmentwide direction to implement its anti-tamper policy. Without such direction, DOD components are left to develop their own initiatives to assist program managers in implementing anti-tamper protection. While individual efforts are important, such as a database to track critical program information DOD-wide, their effectiveness may be limited because they have yet to be accepted and adopted across all DOD components.

DOD Has Developed Some Resources for Program Managers to Use in Implementing Anti-tamper Protection

Since our 2004 report, DOD, through the Anti-Tamper Executive Agent, has developed some resources aimed at assisting program managers as they go through the anti-tamper decision process. DOD’s resources range from providing general information about the anti-tamper policy to research on anti-tamper solutions. Specifically, DOD has developed a guidebook that includes a checklist to assist program managers in identifying security, management, and technical responsibilities when incorporating anti-tamper protection on a weapons system; developed a searchable spreadsheet to assist program managers in developed a Web site for program managers to provide general anti- tamper information, policy resources, conference briefings, implementation resources, and current events; coordinated with Defense Acquisition University to design and launch an online learning module on anti-tamper protection; funded Sandia National Laboratories to study anti-tamper techniques and their general effectiveness; and sponsored research to develop generic anti-tamper techniques through Small Business Innovation Research, a research program that funds early-stage research and development projects at small technology companies.

DOD has also updated two acquisition documents with general anti- tamper information. The first document—DOD Instruction 5000.2, Operation of a Defense Acquisition System—currently states that one of the purposes of the System Development and Demonstration phase of a weapon system is to ensure affordability and protection of critical program information by implementing appropriate solutions such as anti- tamper protection. The second document—the Defense Acquisition Guidebook—has been updated to include some basic information on the importance of implementing anti-tamper protection early in the development of a weapon system and describes program managers’ overall responsibilities for implementing the anti-tamper policy.

DOD Has Not Provided Direction to Implement Its Anti-tamper Policy

While DOD has issued broad policy memorandums that reflect the department’s desire for routinely assessing weapon systems to determine if anti-tamper protection is needed, the department has not fully incorporated the anti-tamper policy into its formal acquisition guidance. Specifically, DOD Instruction 5000.2 mentions anti-tamper protection, but the department has not provided direction for implementation of anti- tamper in a formal directive or instruction. Currently, the department is coordinating comments on a draft instruction (DOD Instruction 5200.39) on protection of critical program information that includes anti-tamper implementation. However, in commenting on the draft instruction, several DOD components have raised concerns about when and how to define critical program information that warrants protection, which have contributed to long delays in finalizing the instruction. In addition, the department has not provided specific guidance for program managers on how to implement anti-tamper protection in a DOD manual because DOD officials said this process cannot begin until the instruction is finalized. The date for finalizing the instruction has not yet been determined.

Officials from the executive agent’s office stated that departmentwide direction would give credence to the anti-tamper policy in practice. Anti- tamper points of contact told us that the policy memorandums are not sufficient to ensure that program managers are implementing anti-tamper protection on weapon systems when necessary. One service anti-tamper point of contact stated that program managers might disregard the policy memorandums because they are high-level and broad. Another service anti-tamper point of contact said that implementation is ultimately left up to the individual program manager. While a program manager’s decision should be approved by the milestone decision authority and documented in the program protection plan, some service and program officials said that programs are not always asked about anti-tamper protection during the review.

Absent Departmentwide Direction, Components Have Been Left to Develop Their Own Anti-tamper Initiatives

Lacking departmentwide direction for the anti-tamper policy, DOD components have been left to develop their own initiatives to assist program managers in anti-tamper implementation. However, the usefulness of these initiatives depends on the extent to which other components participate in these efforts.

For example, the Missile Defense Agency developed a risk assessment model to help program managers identify how much anti-tamper is needed to protect critical technologies. Specifically, the model helps program managers assess the criticality of the technology relative to the risk of exploitation. However, when the Missile Defense Agency sought comments on the initiative, the executive agent and services indicated that it was too lengthy and complex to use. The executive agent, in coordination with anti-tamper points of contact from the Missile Defense Agency and services, has taken over this effort, and it is still in development.

The Navy is also implementing an initiative: a database intended to capture the information that programs across DOD components have identified as critical. Many officials we spoke with pointed to this database as a potential tool to improve identification of critical program information across DOD components. To date, the Navy and the Army are submitting information for the database, but the Missile Defense Agency and Air Force are not. The Missile Defense Agency anti-tamper point of contact stated that its information is classified at a level above what the database can support and its program managers will not submit information for the database unless DOD requires submissions by all DOD components. However, the Missile Defense Agency does have access to the database and uses it as a cross-check to determine if it is identifying similar critical program information. The Air Force has been briefed on the initiative but does not yet have consent from all of the commands to participate. Without full participation across all DOD components, the usefulness of this database as a tool to identify critical technologies that may need anti- tamper protection will be limited.

Program Managers Face Several Challenges in Identifying Critical Technologies, Threats, and Sufficient Anti-tamper Solutions

To determine whether anti-tamper protection is needed, program managers must identify which technologies are deemed critical, determine the potential threats and vulnerabilities to these technologies, and identify sufficient anti-tamper solutions to protect the technologies. Such decisions involve a certain level of subjectivity. However, program managers lack the information or tools needed to make informed assessments at these key decision points. As a result, some technologies that need protection may not be identified or may not have sufficient protection.

Limited Information and Coordination on What Is Critical Increase the Risk That Some Technologies May Not Be Identified

Determining technologies that are critical is largely left to the discretion of the program managers. While DOD has some resources available to program managers to help identify critical technologies, they may be of limited use. For example, the executive agent’s searchable spreadsheet of critical technologies may not be comprehensive because it relies on DOD’s Militarily Critical Technologies List, which we reported in 2006 was largely out of date. Also, some program offices have used a series of questions established in a 1994 DOD manual on acquisition systems protection to help guide their discussions on what is critical. However, these questions are broad and subject to interpretation, and can result in different conclusions, depending on who is involved in the decision-making process.

In addition, identifying what is critical varies by DOD component and sometimes by program office. For example, one Air Force program office tried various approaches, including teams of subject matter experts, over 2 years to identify its list of critical program information. In contrast, the Army took the initiative to establish a research center to assist program managers in identifying critical program information, but Army officials stated that the approach used by the center has led to an underestimating of critical program information and critical technologies in programs.

At the same time, there has been limited coordination across programs on technologies that have been identified as critical—creating a stove piped process—which could result in one technology being protected under one program and not protected under another. While informal coordination can occur, programs did not have a formal mechanism for coordinating with other programs, including those within their service. For example, officials from one program office stated they had little interaction from programs within their service or other services to ensure protection of similar technologies. A program under one joint program executive office had not coordinated with other programs to identify similar technologies as critical. In addition, according to an Army official, contractors who have worked on programs across services have questioned why one service is applying anti-tamper solutions to a technology that another service has not identified as critical. Finally, one program office we spoke with identified critical program information on its system but indicated that a similar system in another service had not identified any critical program information and, therefore, had no plans to implement anti- tamper protection.

Despite the risk that some technologies that need protection may not be identified or may not be protected across programs, no formal mechanism exists within DOD to provide a horizontal view of what is critical. However, any effort to do so could be undermined by the programs’ and services’ different definitions and interpretations of “critical program information” and “critical technologies.” The Anti-Tamper Executive Agent defines critical program information as capturing all critical technologies. In contrast, the Army’s interpretation is that critical program information only includes critical technologies that are state-of-the-art. For the Navy, critical program information includes software, while hardware is part of what the Navy defines as critical technologies. One program that is part of a joint program office identified critical program information as including company proprietary information. As a result, tracking critical program information may not provide a horizontal view of all technologies services and programs have identified as needing anti-tamper protection.

Contradictory and Insufficient Intelligence Information Has Hindered Some Programs in Identifying Potential Threats

Once a program office identifies critical technologies, the next step in the anti-tamper decision process is to identify threats to those technologies. DOD’s Program Manager’s Guidebook and Checklist for Anti-tamper states that multiple threat assessments should be requested from either the service intelligence organization or counterintelligence organization. One program office we visited stated that it has requested and received multiple threat assessments from the intelligence community, which have sometimes contradicted one another, leaving the program office to decipher the information and determine the threat. According to an anti- tamper point of contact, other programs have received contradictory information—typically relating to foreign countries’ capabilities to reverse engineer. The potential impact of contradictory intelligence reports is twofold: If the threat is deemed to be low but is actually high, the technology is susceptible to reverse engineering; conversely, if the threat is deemed to be high and is actually low, the anti-tamper solution is more robust than needed.

To assist with the process of identifying threats, program offices may request threat assessments from a group within the Defense Intelligence Agency. However, this group was not able to complete assessments for approximately 6 months during 2006. While the group has resumed completing assessments, an agency official stated that it is not able to produce as many assessments as before due to limited resources. The Defense Intelligence Agency does not turn down program offices that may request assessments, but does have to put them in a queue and provide them with previous assessments, if they exist, until it can complete a full assessment for the program office. One program office indicated that it took 6 to 9 months for the agency to complete its assessment.

Program Managers Need Tools to Assist in Designing Effective Anti- tamper Solutions and Estimating Related Costs

Program managers also lack the tools needed to identify the optimal anti- tamper solutions for those critical technologies that are vulnerable to threats. Most notably, program managers lack a risk model to assess the relative strengths of different anti-tamper solutions and a tool to help estimate their costs.

According to National Security Agency officials, who are available to provide support to program managers considering or implementing anti- tamper protection, program managers and contractors sometimes have difficulty determining appropriate solutions. Four of five programs we spoke with that had experience in this area of the anti-tamper decision process had difficulty identifying how much anti-tamper protection was enough to protect a critical technology. For example, one program official told us that an anti-tamper solution developed for one of the program’s critical technologies may not be sufficient to prevent reverse engineering. Another program office stated that it is difficult to choose between competing contractors without knowing how to determine the appropriate level of anti-tamper protection needed. An anti-tamper point of contact said that program managers need a tool to help them assess the criticality of a technology versus the types of threats to that technology.

Implementing a suboptimal anti-tamper solution can have cost and performance implications for the program. Specifically, if the solution provides less anti-tamper protection than is needed, the program may have to retrofit additional anti-tamper protection to allow for a more robust solution. Not only can such retrofitting add to a program’s costs, it can compromise performance.

Given limited resources and tools for determining anti-tamper solutions, some program office officials told us that to satisfy anti-tamper solutions they relied on other protection measures. For example, officials in one program office stated that anti-tamper protection and information assurance were interchangeable and indicated that following the National Security Agency’s information assurance requirements—which number in the hundreds—should be sufficient as an anti-tamper solution for this system. This same program was not aware of anti-tamper resources and did not coordinate with an anti-tamper validation and verification team on its solutions. Also, an official from another program office indicated that anti-tamper protection and information assurance are similarly defined. While DOD and service officials agreed that some information assurance and anti-tamper measures may overlap, fulfilling information assurance requirements does not guarantee a sufficient anti-tamper solution.

Conclusions

In establishing various policies to protect its critical assets, DOD saw anti- tamper as a key way to preserve U.S. investment in critical technologies while operating in an environment of coalition warfare and a globalized industry. Program managers are ultimately responsible for implementing DOD’s anti-tamper policy. However, a lack of direction, information, and tools from DOD to implement its policy has created significant challenges for program managers. Further, this policy can compete with the demands of meeting program cost and schedule objectives, particularly when the optimal anti-tamper solution is identified late in the schedule. Until DOD establishes a formal directive or instruction for implementing its policy departmentwide and equips program managers with adequate implementation tools, program managers will continue to face difficulties in identifying critical technologies and implementing anti-tamper protection.

Recommendations for Executive Action

As DOD examines its policies for protecting critical assets, we are recommending that the Secretary of Defense direct the Under Secretary of Acquisition, Technology, and Logistics, in coordination with the Anti- Tamper Executive Agent and the Under Secretary of Defense for Intelligence, to issue or be involved in developing and providing departmentwide direction for application of its anti-tamper policy that prescribes how to carry out the policy and establishes definitions for critical program information and critical technologies.

To help ensure the effectiveness of anti-tamper implementation, we also recommend that the Secretary of Defense direct the Anti-Tamper Executive Agent to identify and provide additional tools to assist program managers in the anti-tamper decision process.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD concurred with our recommendation that the Secretary of Defense direct the Anti-Tamper Executive Agent to identify additional tools to assist program managers in the anti-tamper decision process. DOD stated that the Anti-Tamper Executive Agent is drafting Anti-Tamper Standard Guidelines to facilitate proper implementation of anti-tamper protection across the department.

DOD did not concur with our recommendation that the Secretary of Defense direct the Under Secretary of Defense (AT&L) in coordination with the Anti-Tamper Executive Agent and the Under Secretary of Defense (Intelligence) to issue departmentwide direction for application of its anti- tamper policy that prescribes how to carry out the policy and establishes definitions for critical program information and critical technologies. DOD stated that the Under Secretary of Defense (Intelligence) has primary responsibility for DOD Directive 5200.39, a security and counterintelligence support directive to acquisition programs, and its successor, DOD Instruction 5200.39 regarding protection of critical program information. The Under Secretary of Defense (Intelligence) is currently coordinating an update to this directive. Once it is issued, the department plans to update DOD 5200.1-M, which provides the execution standards and guidelines to meet the DOD Instruction 5200.39 policy.

While DOD has issued broad policy memorandums beginning in 1999 that reflect the department’s desire for routinely assessing weapon systems to determine if anti-tamper protection is needed, the department has not fully incorporated anti-tamper policy into its formal acquisition guidance. As we have reported, service officials indicated collectively that these policy memorandums are high-level, broad, and leave implementation ultimately up to the individual program manager. DOD did not indicate when the update of DOD Directive 5200.39 might be complete and guidance on anti- tamper implementation issued. We continue to believe that such direction is currently needed and that the Under Secretary of Defense for Acquisition, Technology, and Logistics, who issued the policy memorandums and is responsible for anti-tamper policy, should be involved in developing and providing the appropriate direction whether it be the update to DOD Directive 5200.39 or another vehicle. That direction should include how to implement the anti-tamper policy and how critical program information and critical technologies are defined. We continue to believe that the direction, which has been lacking since the policy was initiated in 1999, should not be further delayed. If DOD continues to experience delays in updating DOD Directive 5200.39, it should consider interim measures to meet the immediate need for anti-tamper direction.

DOD’s letter is reprinted in appendix II.

We are sending copies of this report to interested congressional committees, as well as the Secretary of Defense; the Director, Office of Management and Budget; and the Assistant to the President for National Security Affairs. In addition, this report will be made available at no charge on the GAO Web site at http://www.gao.gov.

Please contact me at (202) 512-4841 or calvaresibarra@gao.gov if you or your staff have any questions concerning this report. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Others making key contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To identify actions the Department of Defense (DOD) has taken to implement its anti-tamper policy since 2004, we reviewed DOD policies and guidance governing anti-tamper protection on weapon systems and obtained documents on various initiatives. We interviewed officials from the Anti-Tamper Executive Agent, military services, and other DOD components such as the Missile Defense Agency; Acquisition, Technology and Logistics; Defense Intelligence Agency; National Security Agency; and the Air Force Research Laboratory about initiatives or actions taken regarding anti-tamper. Through these interviews and documents, we also determined the status of our 2004 anti-tamper report recommendations. We interviewed DOD officials from Networks and Information Integration, Science and Technology, and Counterintelligence to discuss anti-tamper protection and how it relates to other program protection measures.

To determine how program managers implemented DOD’s anti-tamper policy, we interviewed officials from 14 program offices. We are not identifying the names of the programs due to classification concerns. We conducted structured interviews with 7 of the 14 program offices to discuss and obtain documents about their experiences with implementing the anti-tamper decision process and identify any challenges they faced. We selected 6 of these programs from a list of weapon systems identified in Anti-Tamper Executive Agent, services, and component documents as considering and/or implementing anti-tamper protection and a seventh program considering anti-tamper that we identified during the course of our fieldwork. Systems we selected represented a cross section of acquisition programs and various types of systems in different phases of development. For the remaining programs, we interviewed 7 not identified by the Anti-Tamper Executive Agent or the services as considering and/or implementing anti-tamper to obtain their viewpoints on DOD’s anti-tamper policy and implementation. We selected these programs by identifying lists of DOD acquisition programs and comparing them to the Anti-Tamper Executive Agent’s, services’, and components’ lists of program considering and/or implementing anti-tamper. We did not evaluate whether programs had implemented sufficient anti-tamper protection.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Anne-Marie Lasowski (Assistant Director), Gregory Harmon, Molly Whipple, Karen Sloan, John C. Martin, and Alyssa Weir made major contributions to this report.

Related GAO Products

High-Risk Series: An Update. GAO-07-310. Washington, D.C.: January 2007.

Export Controls: Challenges Exist in Enforcement of an Inherently Complex System. GAO-07-265. Washington, D.C.: December 20, 2006.

Defense Technologies: DOD’s Critical Technologies Lists Rarely Inform Export Control and Other Policy Decisions. GAO-06-793. Washington, D.C.: July 28, 2006.

President’s Justification of the High Performance Computer Control Threshold Does Not Fully Address National Defense Authorization Act of 1998 Requirements. GAO-06-754R. Washington, D.C.: June 30, 2006.

Export Controls: Improvements to Commerce’s Dual-Use System Needed to Ensure Protection of U.S. Interests in the Post-9/11 Environment. GAO-06-638. Washington, D.C.: June 26, 2006.

Defense Trade: Enhancements to the Implementation of Exon-Florio Could Strengthen the Law’s Effectiveness. GAO-05-686. Washington, D.C.: September 28, 2005.

Industrial Security: DOD Cannot Ensure Its Oversight of Contractors under Foreign Influence Is Sufficient. GAO-05-681. Washington, D.C.: July 15, 2005.

Defense Trade: Arms Export Control Vulnerabilities and Inefficiencies in the Post-9/11 Security Environment. GAO-05-468R. Washington, D.C.: April 7, 2005.

Defense Trade: Arms Export Control System in the Post-9/11 Environment. GAO-05-234. Washington, D.C.: February 16, 2005.

Defense Acquisitions: DOD Needs to Better Support Program Managers’ Implementation of Anti-Tamper Protection. GAO-04-302. Washington, D.C.: March 31, 2004.

Defense Trade: Better Information Needed to Support Decisions Affecting Proposed Weapons Transfers. GAO-03-694. Washington, D.C.: July 11, 2003.