Defense Acquisitions: Knowledge of Software Suppliers Needed to Manage Risks
Released: 2004-05-25T00:00:00

Letter

Background

To protect the security of the United States, DOD relies on a complex array of computer-dependent and mutually supportive organizational components, including the military services and defense agencies. It also relies on a broad array of computer systems, including weapon systems, command and control systems, financial systems, personnel systems, payment systems, and others. These systems are, in turn, connected with other systems operated by contractors, other government agencies, and international organizations. In addition, performance requirements for weapon systems have become increasingly demanding, and breakthroughs in software capability have led to a greater reliance on software to provide more weapon capability when hardware limitations are reached. As such, DOD weapon systems are subject to many risks that arise from exploitable software vulnerabilities. Software code that is poorly developed or purposely injected with malicious code could be used to disrupt these and other DOD information systems, and potentially others connected to the DOD systems.

DOD has reported that countries hostile to the United States are focusing resources on developing information warfare strategies. For example, a DSS report noted that in 2001 there was a significant increase in suspicious attempts by foreign entities to access U.S. technology and information and that trend is expected to continue. Information systems technology was the most sought after militarily critical technology by these entities. Forty-four countries were associated with attempts at accessing U.S. information technology, with 33 percent of the activity coming from foreign government-sponsored or affiliated entities. Because the U.S. defense industry is at the forefront of advanced design and development of weapon systems that include militarily critical technologies, access is sought after for industrial and financial purposes. Access to these technologies by potential adversaries could enhance the performance of their military systems and may be used to counter U.S. capabilities. DSS specifically noted a concern with exploitation or insertion of malicious code with the use of foreign research facilities and software development companies located outside the United States working on commercial projects related to classified or sensitive programs.

Multiple requirements and guidance are in place to ensure the protection of U.S. national security interests. They generally acknowledge the inherent risk associated with foreign access to classified and export-controlled information and technology by establishing procedures to manage such access. For example, the National Industrial Security Program Operation Manual establishes mandatory procedures for the safeguarding of classified information that is released to U.S. government contractors. It generally limits access to U.S. citizens with appropriate security clearances and establishes eligibility policies for U.S. contractors determined to have foreign ownership, control, or influence. Further, an additional DOD directive requires programs containing classified military information to have controls to prevent the unauthorized release of this information to foreign recipients. In addition, the International Traffic in Arms Regulations (ITAR) controls foreign access to defense articles and services through the establishment of the export license and authorization process. U.S. entities, including defense contractors, may apply to the Department of State for authorization to export controlled information and technology to qualified foreign recipients, which is determined through the approval or denial of license requests.

DOD estimates that it spends about 40 percent of its Research, Development, Test, and Evaluation budget on software—$21 billion for fiscal year 2003. Furthermore, DOD and industry experience indicates that about $8 billion of that amount may be spent on reworking software because of quality-related issues. Carnegie Mellon University’s Software Engineering Institute (SEI), recognized for its expertise in developing best practices for software processes, has developed models and methods that define and determine organizations’ software process maturity. Better software development practices are seen as a way to reduce the number of software defects and therefore improve overall software quality, but alone the practices cannot be expected to address malicious software development activities intended to breach security. To underscore the importance of securing software-related products, the Office of the Assistant Secretary of Defense (Networks and Information Integration) and the Federal Aviation Administration Chief Information Office are co- sponsoring, with the involvement of the Department of Energy, the National Aeronautics and Space Administration, and SEI, a project aimed at developing ways to provide safety and security assurance extensions to integrated software maturity models.

DOD’s Approach to Software Security Does Not Fully Address Risks from Foreign Suppliers

DOD’s approach to software development and acquisition generally focuses on improving overall quality, leaving decision making on software suppliers and security with individual program managers. Despite the risks associated with foreign access to defense systems, DOD acquisition policy does not require program managers to identify and manage the amount of foreign involvement for software development in weapon systems. DOD information system security requirements focus on operational software threats, rather than potential threats posed by software developers. While recent DOD initiatives could increase DOD’s focus on software security, efforts to date have not translated into greater knowledge for program managers about foreign software development activities.

DOD Acquisition Policy Allows Discretion in Managing Foreign Software Suppliers

DOD acquisition policy allows program managers discretion in managing foreign suppliers used for software development. This policy consists of general guidance for meeting overall acquisition management principles and instructs program managers to use systems engineering practices, when applicable, that focus on cost, schedule, and performance of the system. For software acquisition, program managers are encouraged to develop open software systems architectures, use COTS computer system products, and allow incremental improvements based on reusable software. All of these practices, while having the potential to benefit cost and schedule for weapon programs, could result in greater software vulnerabilities by introducing potentially malicious code from unknown software development sources. While DOD acquisition policy requires major weapon programs to maintain information about the software project’s size, effort, schedule, and quality to track the cost-related implications of software development, it does not require program managers to identify and manage suppliers or the potential security risks from foreign suppliers.

On October 30, 2002, DOD issued the Interim Defense Acquisition Guidebook, which contained the following security considerations to be used when foreign nationals participate in software development.

The change control process shall indicate whether foreign nationals, in any way, participated in software development, modification, or remediation.

Foreign nationals employed by contractors or subcontractors to develop, modify, or remediate software code specifically for DOD shall each have a security clearance commensurate with the level of the program in which the software is being used.

Primary vendors on DOD contracts may have subcontractors who employ cleared foreign nationals that work only in a certified or accredited environment.

DOD software with coding done in foreign environments or by foreign nationals shall be reviewed by software quality assurance personnel for malicious code.

Vendors of COTS software that demonstrate efforts to minimize the security risks associated with foreign nationals that have developed, modified, or remediated the COTS software being offered shall be given preference during the contracting process in product selection or evaluation.

Software quality assurance personnel shall check software sent to locations not directly controlled by DOD or its contractors for malicious code when returned to the DOD contractors’ facilities.

While this guidance acknowledges the additional risks from using foreign nationals in software development, it is not mandatory and, according to the Guidebook, is to be used at the discretion of acquisition program managers as best practices or lessons learned. Even if the suggested guidance was implemented, the procedures for addressing software security are generally for use after software suppliers have been selected, and do not provide the program manager the opportunity to evaluate whether the risks associated with using those suppliers for software development are acceptable. Further, several of these procedures would not apply when contractors use foreign nationals to develop unclassified portions of software programs. In support of DOD guidance, Air Force, Army, and Navy regulations implement DOD-wide acquisition policies. As such, they defer to DOD guidance and do not specifically address software security issues and related risks that may be inherent with foreign software development.

Information Assurance Focuses on Mitigating Operational Software Security Risks, Leaving Internal Software Development Vulnerable

Laws, requirements, and policies that are intended to provide information assurance for operational security do not fully address risks during software development. Under the Federal Information Security Management Act of 2002, all executive agencies, including DOD, are required to ensure that information security policies, procedures, and practices are adequate. In this regard, DOD is required to carry out an information assurance program that includes the development of essential information assurances technologies and programs. Generally, this includes a review of security features and information technology system safeguards. For example, DOD’s information assurance policy establishes procedures to maintain the integrity of DOD information systems. It sets out a process for all DOD information systems to achieve, among other things, an appropriate level of confidentiality, integrity, knowledge of threats and vulnerabilities, trustworthiness of users and interconnecting systems, and cost effectiveness. These procedures are intended to mitigate system vulnerabilities from operational threats, such as external hacking and unauthorized access to information systems. However, they do not apply to internal threats that could affect the integrity of the software, such as the insertion of malicious code during software development. In implementing its information assurance policy, DOD also relies on other governmentwide policies and standards. For example, DOD requires the evaluation and validation of information assurance software products, such as firewalls and intrusion detection systems, in accordance with National Security Telecommunications and Information Systems Security Policy No. 11. This policy requires the use of one of three nationally recognized evaluation and validation standards. However, these policies and standards do not include criteria to specifically identify and manage the use of foreign software suppliers.

To assist systems in meeting information assurance requirements, DOD has developed the Defense Information Technology Security Certification and Accreditation Process (DITSCAP) as a standardized evaluation process. This process includes a review by a designated approving official who certifies that the security features and information technology system safeguards will maintain information assurance through the life cycle of a system. The process results in an agreement between the program manager, the intended user of the system being certified, and the approval authorities that defines critical schedule, budget, security, functionality, and performance issues. The process includes a requirement for a threat assessment, but does not articulate how this information should be developed or reported. As such, it does not direct program managers to consider foreign involvement in software development as a risk or threat that needs to be addressed for information system security. In addition, while the process is mandatory, the implementation details may be tailored and, in some cases, integrated with other acquisition activities and documentation. According to DOD Software Assurance Program officials, the DITSCAP approving authority is not expected to evaluate whether the risks have been identified appropriately, only that the process will mitigate the risks identified. If the program manager has not identified risks from foreign suppliers, the process cannot be expected to mitigate them.

DOD also requires weapon programs to protect certain types of information during transfer of technology to foreign entities. For example, documents such as the Technology Assessment/Control Plan and the Program Protection Plan address the risks associated with the potential release of information to foreign governments through cooperative programs and foreign military sales, but the documents do not provide information on specific suppliers who will be performing work, such as software developers. The Technology Assessment/Control Plan establishes planning requirements for the potential release of sensitive information to foreign entities involved in cooperative programs or purchasing military equipment. It evaluates the risk of releasing critical military capability or sensitive information and technology against the benefit of the sale to the United States. The plan also outlines the security requirements to prevent compromise. The purpose of the Program Protection Plan is to identify measures to protect Critical Program Information from hostile collection efforts and unauthorized disclosure during the acquisition process.

Initiatives to Address Software Concerns Have Yet to be Implemented

While DOD has taken steps to strengthen software acquisition practices, it has yet to implement practices to better manage software development security risks in weapon programs. Currently, each of the military services is developing plans for improving software acquisition. The improvement plans are each at varying stages of development and include practices such as pilot programs for providing information on software metrics, additional training programs, and teaming arrangements with SEI for improved overall software management. DOD has also begun policy-level initiatives focused on better software management and on identifying and specifying software security processes and technologies to protect systems and network capabilities from various internal and external threats. Specific initiatives include the following: The Tri-Service Assessment Initiative began in 1999 to strengthen software acquisition and development as well as address repeated performance shortfalls attributed to software. Task forces conducted detailed assessments of software-intensive programs to identity potential improvements in overall software acquisition processes.

A source selection criteria working group is tasked with clarifying the policy on source selection criteria for software intensive systems and the application of software product maturity measures. Another working group is tasked with developing a proposal for a centralized clearinghouse of software best practices, but DOD has not approved any proposals.

In October 2003, the Office of the Assistant Secretary of Defense (Networks and Information Integration) established the position of Deputy Director for Software Assurance that is, as part of its function, to coordinate software security efforts with other existing initiatives concerning software protection, antitamper technologies, and software producibility. In addition, since its inception, the office has initiated working groups intended to focus on mitigating software risks and improving software security. To date, while these initiatives have presented top level findings and recommendations within DOD and in public forums, they have not externally published reports or obtained funding for implementing the recommendations.

While these plans and initiatives may help to increase DOD’s focus on software security and may lead to the development and identification of several potential software security best practices, DOD software assurance officials acknowledge that significant effort remains to adequately mitigate software risks to weapon systems.

Program Officials Generally Did Not Manage Risks from Foreign-Developed Software

While DOD initiatives have begun to recognize potential risks from foreign software suppliers, this is not always the case within the weapon programs where software is developed or acquired. Program officials for most of the systems we reviewed did not make foreign involvement in software development a specific element of their risk identification and mitigation efforts. As a result, program officials’ knowledge of the foreign-developed software included in their weapon systems varied. In addition, risk mitigation efforts emphasized program level risks, such as meeting program cost and schedule goals, instead of software security risks. Further, program managers often delegated risk mitigation and source selection to their prime contractors who tended to be concerned with software functionality and quality assurance, rather than specifically addressing software development risks associated with foreign suppliers.

Program Office Knowledge of Software Suppliers Varied Across Weapon Systems

Knowledge of the extent of foreign involvement in software development varied greatly for the 16 weapon system programs we reviewed. Overall, the knowledge program managers had was based on the function of the software being developed, the manner in which it was acquired, and the specific handling requirements. While none of these programs could fully identify all foreign-developed software for their systems, six program offices had significant knowledge of foreign software developers.

Knowledge of software developed for weapon systems can vary based on the different functions needed to meet mission requirements. Program offices were most knowledgeable about the foreign-developed software for the onboard portions of their programs, with 4 program offices able to identify all the software produced by foreign suppliers, and 11 program offices able to identify at least some of the software produced by foreign suppliers. Onboard software is that which actually runs the weapon system, even if that software is not located on the main component (aircraft, missile, satellite, etc). For example, onboard software for a missile system could include software located on a remote platform used to guide the missile toward its target. Because onboard software is the most critical for meeting mission requirements and other program goals, program managers placed greater emphasis on the quality, functionality, and usually, the security of this software. In 9 of the systems we reviewed, either prime contractors or major subcontractors conducted software development for the onboard systems. However, in meeting with prime contractors, we found that while this increased their knowledge about foreign software suppliers, the information was not always shared with government program managers, and therefore was not available for them to use to make risk management decisions to address software security.

Program offices and contractors reported very little knowledge about the level of foreign involvement for offboard software. This software, sometimes referred to as ground based, interacts with the onboard system to provide updated information in support of operational activity. For example, one program uses standard mission planning software that interacts with the onboard flight software to provide information used for navigation and targeting. In addition, offboard software is often used to check for errors or malicious code and to produce, maintain, or verify onboard software. Program officials from 10 of the programs we reviewed indicated they had very little knowledge of the developers for their offboard software, including those portions that may have been developed by foreign suppliers.

As DOD is attempting to find new ways of reducing the time and money it takes to develop software code, the increased use of COTS software may introduce additional risks and further limit visibility into the existence of foreign-developed software. Officials for 13 of the programs reviewed had almost no insight into the use of foreign developers for any COTS software placed on their systems. Even when procured directly from a known supplier, program officials could not guarantee whether additional subcontractors were used for software development. According to DOD and program officials, visibility into COTS software is limited by the willingness of the producers of that software to share information on how the code was developed. For one program we reviewed, a substantial portion of the system was a commercial acquisition. As such, the software product was not originally developed for DOD and therefore the program office had no knowledge of software development suppliers because it did not purchase the rights to the software. Further, officials from five programs told us that the cost of identifying and managing foreign software suppliers, especially for COTS software, could be substantial. Officials from two programs said that even if available, this information would not offer significant software security improvements in light of the cost required for identifying foreign suppliers of COTS software. DOD and program officials have indicated that commercial software producers often demand a cost premium to share software and source code information that would be required to determine this information.

Similar to COTS, software from other applications and embedded software is often accepted without full knowledge of the source of development. Legacy, or reused code, is most prevalent when software programs are updated into newer versions or when software just requires editing and enhancing the older code rather than developing new code. When asked to produce software similar to what they have developed previously, manufacturers can use all or part of the legacy code as a basis for developing or modifying the new code. Ten of the programs reviewed accepted legacy software without fully identifying the sources of development. Software is also developed as part of the hardware components it is tasked with managing. This embedded software is used to control other electronic hardware products, either onboard or offboard the weapon system, and is often purchased from lower tier subcontractors. While we did not specifically ask for information on foreign suppliers responsible for embedded software, officials for two of the programs reviewed stated that this software tends to include more reused and COTS software. These officials indicated that this could limit the visibility of the software suppliers because acceptance testing was usually only performed to prove functionality and the software was not further evaluated to determine the actual source that developed the software code. Further, DOD and prime contractor officials told us that it is sometimes difficult to determine whether the actual hardware subcontractor developed embedded software, or if it was done by a software developer hired by that subcontractor, thus further reducing visibility of software suppliers.

While most of the program offices we reviewed did not specifically track foreign software suppliers for security purposes, almost all of the programs had opportunities to gain such knowledge through practices designed to collect other information. One way program offices obtained information on software suppliers was through requirements in their prime contracts. Many of the program offices in our review were able to obtain some information on software suppliers, either directly or indirectly, because it was contractually required. In some cases, this information was available early in the proposal process when bidders were required to identify suppliers they intended to use in the development and manufacturing phases, including potential foreign suppliers and components they were expected to provide. For example, officials from one program office said they were effective in determining software suppliers at the prime and subcontractor levels because they were intimately involved with source selection and contract negotiation. However, only five of the prime contractors reported that they were required to notify the program office concerning their decisions on software subcontracting. Once a winning bidder is selected, more information is often available to the program manager. For example, contracts for 12 programs contained a requirement that the contractors provide a software development plan that included information on some of their planned suppliers, development risks, and action plans for the contract period. In at least two cases, program software managers became aware of foreign software suppliers while collecting information requested for this review.

Program officials also said that some knowledge of foreign software development was available as a result of procedures in place because their programs contained classified or technical program information. For example, of the six programs that had significant knowledge of foreign software suppliers, four reported they had very few foreign suppliers because handling restrictions for classified information precluded the involvement of such suppliers, not because they were specifically managing software development risks. Similarly, contractors for 12 programs had information available from the export license process. While limited information on the supplier and location of foreign entities performing software work was available from export licenses, contractors request approval directly from the State Department, which may not refer the application to DOD or the individual program offices. Consequently, program managers likely did not have this information available for use in software risk management decisions.

Program Risk Mitigation Efforts Are Focused on Meeting Performance Requirements and Are Often Delegated to Contractors

The relative importance of software security in risk mitigation efforts also varied greatly across the systems we reviewed. For 11 of the 16 systems, program managers have not identified foreign supplier involvement in software development as a significant risk to the security of their weapon systems. Instead, program managers concerned with completing their programs on budget and on schedule generally focused risk mitigation efforts on program level risks associated with the performance of system components, not on internal software development security risks. When specifically identified, software risks are usually defined by their impact when integrated with these system level risks and do not specifically focus on foreign suppliers used in software development. Software generally only becomes a concern for program managers as it begins to affect the cost or schedule of the program. For example, one of the programs we reviewed lists “software executibility” as a program level risk. The risk is based on potential cost and schedule overruns should the software not function as needed to allow related system components to meet mission requirements, rather than potential vulnerabilities from foreign suppliers. For these programs, security risks have generally been implemented to prevent unauthorized access to classified or technical program information, provide security at contractor facilities, and limit access to export-controlled technical information in accordance with ITAR license requirements, rather than specifically for software security. In addition, 12 of the programs used the Technology Assessment/Control Plan and the Program Protection Plan to ensure that risks associated with foreign participation on the program were addressed.

Programs that identified software security as a risk focused on limiting foreign access to software development facilities and denying foreign access to software code. In addition, these programs employed various measures to address software security consistent with information assurance requirements. These measures included the use of password protection, firewalls, or encrypted software, but they did not always focus on risks from foreign involvement in software development. Further, 11 programs mentioned using DITSCAP as a means for addressing general software security. However, interpretation and implementation of this requirement can vary across programs. For example, according to officials from two programs, the current DITSCAP requirements do not govern contractors in cases where the requirements were not included as part of the original contract. Representatives from two of the programs we reviewed noted that guidance for implementing DITSCAP was confusing and that they were uncertain whether the process applied to their programs. Program officials responsible for software development on one other program indicated that they had no knowledge of this process. In cases where DITSCAP is being implemented, the certification and accreditation process requirements are determined based on the program manager’s assessment of risks. If the program manager has not identified foreign software development as a program risk or threat, it will not be addressed by the process.

Because security and software risks are generally defined in terms of programmatic elements, program managers often delegate the identification and mitigation responsibility to the contractors who are developing the system and are therefore more knowledgeable about what functions are needed from the software. Officials from eight of the programs we reviewed said they expect contractors to ensure quality and security on their systems because their software development processes are more mature than those required by DOD, and that such practices could indirectly address foreign software risks. For example, contractors for these eight programs were presumed to be addressing software security because they were employing practices such as peer review and software testing consistent with SEI development models. Peer review is recognized as a best practice for improving software development and is generally performed to improve the quality and functionality of software code. In terms of security, peer review can reduce the likelihood that an individual programmer can insert malicious or other harmful code. Through dedicated software testing, teams assess the quality of the software to uncover gaps and make it as defect-free as possible. However, on eight of the programs we reviewed, decisions on the amount of software code to test were made based upon the risks and benefits to the functionality of the system to be tested, not on the benefits to security. DOD and SEI officials said that the amount of effort needed to comprehensively test every line of code to ensure complete security could be physically impossible and would require immense resources.

Because contractors for the weapon systems we reviewed had not received specific direction from program managers to address risks from foreign suppliers in software development, they tended to focus on development efforts aimed at meeting stated requirements, such as software quality and functionality. In fact, officials from the 15 contractors that responded to our review indicated this was the focus of their software development activities. While SEI representatives told us that rigorous software development could help improve the quality and functionality of software by decreasing the number of errors in software code, they also said that alone their models should never be expected to completely address software security risks. Officials from one contractor we interviewed that employs practices consistent with SEI’s highest level indicated that unless software security is a specific contract requirement, they would not modify their practices to address associated risks. SEI experts confirmed that the models they have developed do not include a security element and are not intended to certify that improved processes will address risks related to software security. In fact, it is possible that using software development practices to increase efficiency could lead to an increase in security vulnerability by encouraging the use of legacy and COTS software, unless risks are managed appropriately.

For several programs we reviewed, contractors made risk identification and mitigation decisions for business reasons and to avoid additional resource burdens (i.e., cost and access) associated with incorporating foreign suppliers necessary for software development, as opposed to being done for security reasons. For example, prime contractors for two programs did not use foreign subcontractors for economic reasons; namely the company wanted to maintain the software expertise within the company. While restricting foreign access solely for economic reasons could result in a decrease in software development security risks, it might also preclude foreign suppliers that could offer new capability or lower costs to the government. For example, contractor officials for three weapon systems told us that they restrict foreign involvement in software development because it costs too much to develop and monitor security procedures to separate foreign nationals from classified and sensitive information, not because they feel their involvement is a risk to the program. In yet another case, the contractor did not want to create dedicated partitions, such as firewalls, required to prevent employees from a foreign subcontractor from accessing unauthorized information in the system design database and instead contracted with a domestic supplier. Finally, software managers for one program told us that when foreign nationals modify or update software they had developed, it was necessary to isolate test facilities to meet security requirements, which resulted in increased cost and delays to other test activities. The officials said that, in similar cases, contracting decisions might sometimes be made in favor of U.S. suppliers to avoid costs and delays.

Conclusions

Because software is increasingly responsible for advances in weapon system capabilities, it is essential that DOD and program managers take appropriate steps to identify and manage software-related risks. While DOD has made improvements to system engineering and software development practices that can reduce the likelihood of defects in software code, current methods of testing focus on the quality of software and related functionality and failures, which will not necessarily uncover malicious intent. As the amount of software on weapon systems increases, it becomes more difficult and costly to test every line of code. Further, DOD cannot afford to monitor all worldwide software development facilities or provide clearances for all potential software developers, especially for COTS software. Given the global nature of the software industry, which offers benefits to software cost and functionality needed by weapon systems, DOD also cannot afford to exclude all foreign suppliers from its programs. While program managers should be allowed discretion in managing their acquisitions, they are responsible for knowing more about who is developing software and where and for working with DOD’s software assurance resources, and other organizations as necessary, so that risks can be identified and assessed accordingly. Unless this is done early in the software acquisition process, it cannot be included as part of software source selection and risk mitigation decisions and could result in increased cost and less effective security measures if risks have to be addressed later in the acquisition process.

Recommendations for Executive Action

We have previously made recommendations to DOD to adopt more effective software development practices and to increase oversight of software intensive systems to improve acquisition outcomes. While DOD attempts to better its software acquisition policies and implement new initiatives, it must take steps to ensure that security is an integral element in decision making and that program managers mitigate risks accordingly. We recommend that the Secretary of Defense take the following three actions to address risks attributable to software vulnerabilities and threats: Require program managers, working with software assurance experts, acquisition personnel, and other organizations as necessary, to specifically define software security requirements, including those for identifying and managing software suppliers. These requirements should then be communicated as part of the prime development contract, to be used as part of the criteria to select software suppliers.

Based on defined software security requirements, require program managers to collect and maintain information on software suppliers, including software from foreign suppliers. This information should be evaluated periodically to assess changes in the status of suppliers and adjustments to program security requirements.

Require the Office of the Assistant Secretary of Defense for Networks and Information Integration and the Office of the Undersecretary of Defense for Acquisition Technology and Logistics, as part of their role to review, oversee, and formulate security and acquisition practices, to work with other organizations as necessary to ensure that weapon program risk assessments include specific attention to software development risks and threats, including those from foreign suppliers. For example, certification and accreditation processes, such as DITSCAP, should include verification that software development practices contain adequate security measures to address identified risks and threats.

Agency Comments and Our Evaluation

In written comments on a draft of this report, DOD agreed with our findings that malicious code is a threat that is not adequately addressed in current acquisition policies and security procedures and stated that the department is working to strengthen software related risk management activities. DOD also noted the need to enhance its risk management processes to factor in vulnerabilities analysis of proposed software products, security risks of suppliers’ processes, and counterintelligence threat information. DOD partially concurred with our three recommendations based on the concern that they place too much responsibility for risk mitigation on program managers. Although the draft report recognized that software assurance experts from the Office of the Assistant Secretary of Defense for Networks and Information Integration were necessary to support program managers in risk mitigation efforts, we broadened two of our recommendations to include acquisition and other organizations to address this concern. DOD also provided separate technical comments that we incorporated into the report as appropriate. DOD’s letter is reprinted in appendix II.

DOD agreed that software security risks should be defined for DOD weapon programs, but noted that program managers should not be solely responsible for defining security requirements, including those for identifying and managing software suppliers. Instead, DOD stated that program managers should be able to rely on external resources to gain threat information on suppliers and that formulation and oversight of security practices should be a collaborative function among several offices within DOD. While we continue to believe that program managers and software assurance experts play a critical role in defining software security requirements, we do see the value of involving other DOD resources to provide coordinated evaluation of broader security concerns. As such, we modified our recommendation to reflect the inclusion of acquisition personnel and other organizations as necessary.

DOD also agreed that information on software suppliers, including foreign suppliers, should be collected and that this information should be periodically assessed to determine if adjustments to security requirements are needed. However, DOD indicated that centralized information on software suppliers is necessary because the cost of collecting and maintaining this information would require resources and assets beyond those of individual program managers. DOD indicated its intent to develop a database to identify, track, and maintain information on security risks from specific software suppliers, which could be used by program managers across various weapon and other programs for developing acquisition strategies, plans, requests for proposals, and contracts. While we agree that such a database would be helpful to program managers in collecting and maintaining information on software suppliers, we made no change to the recommendation because the program managers should be responsible for collecting this information until such a database is developed and for directing the collection of information from the database once it is completed.

Finally, DOD agreed that it should ensure that program risk assessments include specific attention to software development risks, including those from foreign suppliers. However, DOD suggested that this might be best accomplished through collaboration between the Office of the Assistant Secretary of Defense for Networks and Information Integration, the Office of the Undersecretary of Defense for Acquisition Technology and Logistics, and the Office of the Undersecretary of Defense for Intelligence. This seemed reasonable, and we adjusted our recommendation to reflect the inclusion of other organizations. Further, DOD agreed certification activities such as DITSCAP can assist in addressing insider threats in software development, but that additional guidance is necessary to ensure that software security risks are addressed during system design and development or when selecting software sources.

We are sending copies of this report to interested congressional committees; the Secretary of Defense; the Secretaries of the Air Force, Army, and Navy; the Commandant of the Marine Corps; and the Director, Office of Management and Budget. We will also make copies available to others upon request. In addition, this report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512- 4841. Staff acknowledgments are listed in appendix III.

Appendix I: Scope and Methodology

To determine how the Department of Defense (DOD) measures the extent of foreign involvement in software development in weapon systems and how risks associated with using foreign suppliers for software development are measured and mitigated, we reviewed relevant DOD guidance, policies, regulations, and procedures. In addition, we spoke with DOD officials from the Office of the Under Secretary of Defense (Acquisition Technology & Logistics), the National Security Agency, the Defense Information Systems Agency, the Defense Advanced Research Projects Agency, the Office of the Assistant Secretary of Defense (Networks and Information Integration), the Department of the Army, the Department of the Air Force, and the Department of the Navy. We met with software experts at the Software Engineering Institute of Carnegie Mellon University to obtain information on software development practices and risk identification and mitigation techniques used by the software industry. Additionally, we met with the Association of Old Crows (The Electronic Warfare and Information Operations Association) whose membership includes individuals and companies involved in the design and development of software used in DOD weapon programs.

To document and analyze how programs specifically measure and manage their use of foreign-developed software, we identified 16 DOD weapon systems and solicited information from each program office and prime contractor. We selected these weapon systems based on recommendations from DOD officials and on our internal knowledge of the systems. While our selection of programs cannot be generalized to the population of all DOD systems, the systems selected varied by product type, represented each of the military services, and represented a range of DOD contractors. The systems reviewed were the Abrams System Enhancement Package, AH-64D Apache, Bradley Upgrade, C-130 Avionics Modernization Program, C-130 J Hercules, C-17 Globe Master, Comanche Reconnaissance Attack Helicopter (RAH-66), F/A-18 Super Hornet, F/A-22 Raptor, Future Combat Systems, Global Hawk Unmanned Aerial Vehicle, Joint Helmet Mounted Cueing System, Joint Strike Fighter, Patriot Missile System, Tactical Tomahawk Missile, and Wideband Gapfiller Satellites.

Using their respective command liaisons to initially contact each office, we distributed a structured set of questions to solicit information from software managers designated by individual program managers to respond to our inquiry. To further determine how programs manage and mitigate their use of foreign-developed software, we then tailored follow-up questions to solicit information and documentation in areas such as program risk identification and management practices, security policies and procedures, and software contracting management practices. To learn more about program practices for managing and mitigating the use of foreign-developed software, we solicited information and documentation from the prime contractor for each system using contacts provided by program office officials. Information requested from contractors included government guidance for software practices, company software development and security practices, software risk mitigation efforts, software testing procedures, and software sourcing decision processes. We received information from 15 of the prime contractors through written responses, on-site interviews, and other means such as telephone conversations. We also obtained several security and software related documents such as the Program Protection Plan, the Software Development Plan, and other program specific documents, such as the contract, for the systems we reviewed.

Appendix II: Comments from the Department of Defense

Appendix III: Staff Acknowledgments

Acknowledgments

John Neumann, Brian Mullins, Delores Cohen, Shelby S. Oakley, Christopher Miller, Gary Middleton, and Marie Ahearn made key contributions to this report.