Defense Acquisitions: Knowledge of Software Suppliers Needed to Manage Risks
Published: 2004-05-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) is increasingly reliant on software and information systems for its weapon capabilities, and DOD prime contractors are subcontracting more of their software development. The increased reliance on software and a greater number of suppliers results in more opportunities to exploit vulnerabilities in defense software. In addition, DOD has reported that countries hostile to the United States are focusing resources on information warfare strategies. Therefore, software security, including the need for protection of software code from malicious activity, is an area of concern for many DOD programs. GAO was asked to examine DOD's efforts to (1) identify software development suppliers and (2) manage risks related to foreign involvement in software development on weapon systems.
section_title: What GAO Found
paragraphs: DOD acquisition and software security policies do not fully address the risk of using foreign suppliers to develop weapon system software. The current acquisition guidance allows program officials discretion in managing foreign involvement in software development, without requiring them to identify and mitigate such risks. Moreover, other policies intended to mitigate information system vulnerabilities focus mostly on operational software security threats, such as external hacking and unauthorized access to information systems, but not on insider threats, such as the insertion of malicious code by software developers. Recent DOD initiatives may provide greater focus on these risks, but to date have not been adopted as practice within DOD. While DOD has begun to recognize potential risks from foreign software content, this is not always the case within the weapon programs where software is developed or acquired. Program officials for the systems in this review did not make foreign involvement in software development a specific element of their risk identification and mitigation efforts. As a result, program officials' knowledge of the foreign developed software included in their weapon systems varied. In addition, risk mitigation efforts emphasized program level risks, such as meeting program cost and schedule goals, instead of software security risks. Further, program officials often delegated risk mitigation and source selection to contractors who are primarily concerned with software functionality and quality assurance, rather than specifically addressing software security for development risks associated with foreign suppliers. Unless program officials provide specific guidance, contractors may favor business considerations over potential software development security risks associated with using foreign suppliers. As the amount of software on weapon systems increases, it becomes more difficult and costly to test every line of code. Further, DOD cannot afford to monitor all worldwide software development facilities or provide clearances for all potential software developers. Therefore, the program manager must know more about who is developing software and where early in the software acquisition process, so that it can be included as part of software source selection and risk mitigation decisions.

section_title: Letter
paragraphs: The Department of Defense (DOD) is experiencing significant and  increasing reliance on software and information systems for its weapon  capabilities, while at the same time traditional DOD prime contractors are  subcontracting more of their software development to lower tier and  sometimes nontraditional defense suppliers. Because of economic  incentives and business relationships, these suppliers are using offshore  locations and foreign companies to complete various software  development and support tasks. DOD is also using more commercial-off- the-shelf (COTS) software to reduce development costs and allow for  frequent technology updates, which further increases the number of  software developers with either direct or indirect access to weapon  system software. The increased dependence on software capability,  combined with an exposure to a greater variety of suppliers, results in  more opportunities to exploit vulnerabilities in defense software. Software  security, including the protection of software code from hacking and other  malicious tampering, is therefore an area of concern for many DOD  systems.
As DOD’s need for software increases, knowledge about foreign suppliers  in software development is critical for identifying and mitigating risks.  Multiple national security policies and other guidance recognize the  inherent risks associated with foreign access to sensitive information and  technology. For 2001, the Defense Security Service (DSS) reported a  significant increase in suspicious attempts by foreign entities to access  U.S. technology and information, with one-third of that activity coming  from foreign government-sponsored or affiliated entities. While both   U. S.- and foreign-developed software are vulnerable to malicious  tampering, DSS specifically noted a concern with the potential  exploitation of software developed in foreign research facilities and  software companies located outside the United States working on  commercial projects related to classified or sensitive programs. As foreign  companies and individuals play a more significant role in software  development activities, the need for knowledge to manage associated risks  also increases. At your request, we (1) examined DOD’s efforts to identify  and address risks associated with foreign involvement in software  development in weapon systems and (2) determined how weapon system  program offices manage these risks in individual programs.
To perform our work, we collected information from various DOD officials  and software development experts and reviewed relevant guidance and  procedures for software development security. We identified 16 DOD  weapon systems of varying age and software capability and solicited  information from system program offices and prime contractors. Complete  details of our scope and methodology are located in appendix I. We  performed our work from April 2003 to May 2004 in accordance with  generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: To protect the security of the United States, DOD relies on a complex  array of computer-dependent and mutually supportive organizational  components, including the military services and defense agencies. It also  relies on a broad array of computer systems, including weapon systems,  command and control systems, financial systems, personnel systems,  payment systems, and others. These systems are, in turn, connected with  other systems operated by contractors, other government agencies, and  international organizations. In addition, performance requirements for  weapon systems have become increasingly demanding, and breakthroughs  in software capability have led to a greater reliance on software to provide  more weapon capability when hardware limitations are reached. As such,  DOD weapon systems are subject to many risks that arise from exploitable  software vulnerabilities. Software code that is poorly developed or  purposely injected with malicious code could be used to disrupt these and  other DOD information systems, and potentially others connected to the  DOD systems.
DOD has reported that countries hostile to the United States are focusing  resources on developing information warfare strategies. For example, a  DSS report noted that in 2001 there was a significant increase in  suspicious attempts by foreign entities to access U.S. technology and  information and that trend is expected to continue. Information systems  technology was the most sought after militarily critical technology by  these entities. Forty-four countries were associated with attempts at  accessing U.S. information technology, with 33 percent of the activity  coming from foreign government-sponsored or affiliated entities. Because  the U.S. defense industry is at the forefront of advanced design and  development of weapon systems that include militarily critical  technologies, access is sought after for industrial and financial purposes.  Access to these technologies by potential adversaries could enhance the  performance of their military systems and may be used to counter U.S.  capabilities. DSS specifically noted a concern with exploitation or  insertion of malicious code with the use of foreign research facilities and  software development companies located outside the United States  working on commercial projects related to classified or sensitive  programs.
Multiple requirements and guidance are in place to ensure the protection  of U.S. national security interests. They generally acknowledge the  inherent risk associated with foreign access to classified and   export-controlled information and technology by establishing procedures  to manage such access. For example, the National Industrial Security  Program Operation Manual establishes mandatory procedures for the  safeguarding of classified information that is released to U.S. government  contractors. It generally limits access to U.S. citizens with appropriate  security clearances and establishes eligibility policies for U.S. contractors  determined to have foreign ownership, control, or influence. Further, an  additional DOD directive requires programs containing classified military  information to have controls to prevent the unauthorized release of this  information to foreign recipients. In addition, the International Traffic in  Arms Regulations (ITAR) controls foreign access to defense articles and  services through the establishment of the export license and authorization  process.  U.S. entities, including defense contractors, may apply to the  Department of State for authorization to export controlled information  and technology to qualified foreign recipients, which is determined  through the approval or denial of license requests.
DOD estimates that it spends about 40 percent of its Research,  Development, Test, and Evaluation budget on software—$21 billion for  fiscal year 2003. Furthermore, DOD and industry experience indicates that  about $8 billion of that amount may be spent on reworking software  because of quality-related issues. Carnegie Mellon University’s Software  Engineering Institute (SEI), recognized for its expertise in developing best  practices for software processes, has developed models and methods that  define and determine organizations’ software process maturity. Better  software development practices are seen as a way to reduce the number of  software defects and therefore improve overall software quality, but alone  the practices cannot be expected to address malicious software  development activities intended to breach security. To underscore the  importance of securing software-related products, the Office of the  Assistant Secretary of Defense (Networks and Information Integration)  and the Federal Aviation Administration Chief Information Office are co- sponsoring, with the involvement of the Department of Energy, the  National Aeronautics and Space Administration, and SEI, a project aimed  at developing ways to provide safety and security assurance extensions to  integrated software maturity models.
section_title: DOD’s Approach to Software Security Does Not Fully Address Risks from Foreign Suppliers
paragraphs: DOD’s approach to software development and acquisition generally  focuses on improving overall quality, leaving decision making on software  suppliers and security with individual program managers. Despite the risks  associated with foreign access to defense systems, DOD acquisition policy  does not require program managers to identify and manage the amount of  foreign involvement for software development in weapon systems. DOD  information system security requirements focus on operational software  threats, rather than potential threats posed by software developers. While  recent DOD initiatives could increase DOD’s focus on software security,  efforts to date have not translated into greater knowledge for program  managers about foreign software development activities.
subsections: section_title: DOD Acquisition Policy Allows Discretion in Managing Foreign Software Suppliers
paragraphs: DOD acquisition policy allows program managers discretion in managing  foreign suppliers used for software development. This policy consists of  general guidance for meeting overall acquisition management principles  and instructs program managers to use systems engineering practices,  when applicable, that focus on cost, schedule, and performance of the  system. For software acquisition, program managers are encouraged to  develop open software systems architectures, use COTS computer system  products, and allow incremental improvements based on reusable  software. All of these practices, while having the potential to benefit cost  and schedule for weapon programs, could result in greater software  vulnerabilities by introducing potentially malicious code from unknown  software development sources. While DOD acquisition policy requires  major weapon programs to maintain information about the software  project’s size, effort, schedule, and quality to track the cost-related  implications of software development, it does not require program  managers to identify and manage suppliers or the potential security risks  from foreign suppliers.
On October 30, 2002, DOD issued the Interim Defense Acquisition  Guidebook, which contained the following security considerations to be  used when foreign nationals participate in software development.
The change control process shall indicate whether foreign nationals, in  any way, participated in software development, modification, or  remediation.
Foreign nationals employed by contractors or subcontractors to  develop, modify, or remediate software code specifically for DOD shall  each have a security clearance commensurate with the level of the  program in which the software is being used.
Primary vendors on DOD contracts may have subcontractors who  employ cleared foreign nationals that work only in a certified or  accredited environment.
DOD software with coding done in foreign environments or by foreign  nationals shall be reviewed by software quality assurance personnel for  malicious code.
Vendors of COTS software that demonstrate efforts to minimize the  security risks associated with foreign nationals that have developed,  modified, or remediated the COTS software being offered shall be  given preference during the contracting process in product selection or  evaluation.
Software quality assurance personnel shall check software sent to  locations not directly controlled by DOD or its contractors for  malicious code when returned to the DOD contractors’ facilities.
While this guidance acknowledges the additional risks from using foreign  nationals in software development, it is not mandatory and, according to  the Guidebook, is to be used at the discretion of acquisition program  managers as best practices or lessons learned. Even if the suggested  guidance was implemented, the procedures for addressing software  security are generally for use after software suppliers have been selected,  and do not provide the program manager the opportunity to evaluate  whether the risks associated with using those suppliers for software  development are acceptable. Further, several of these procedures would  not apply when contractors use foreign nationals to develop unclassified  portions of software programs. In support of DOD guidance, Air Force,  Army, and Navy regulations implement DOD-wide acquisition policies. As  such, they defer to DOD guidance and do not specifically address software  security issues and related risks that may be inherent with foreign  software development.
section_title: Information Assurance Focuses on Mitigating Operational Software Security Risks, Leaving Internal Software Development Vulnerable
paragraphs: Laws, requirements, and policies that are intended to provide information  assurance for operational security do not fully address risks during  software development. Under the Federal Information Security  Management Act of 2002, all executive agencies, including DOD, are  required to ensure that information security policies, procedures, and  practices are adequate. In this regard, DOD is required to carry out an  information assurance program that includes the development of essential  information assurances technologies and programs. Generally, this  includes a review of security features and information technology system  safeguards. For example, DOD’s information assurance policy establishes  procedures to maintain the integrity of DOD information systems. It sets  out a process for all DOD information systems to achieve, among other  things, an appropriate level of confidentiality, integrity, knowledge of  threats and vulnerabilities, trustworthiness of users and interconnecting  systems, and cost effectiveness. These procedures are intended to mitigate  system vulnerabilities from operational threats, such as external hacking  and unauthorized access to information systems. However, they do not  apply to internal threats that could affect the integrity of the software,  such as the insertion of malicious code during software development. In  implementing its information assurance policy, DOD also relies on other  governmentwide policies and standards. For example, DOD requires the  evaluation and validation of information assurance software products,  such as firewalls and intrusion detection systems, in accordance with  National Security Telecommunications and Information Systems Security  Policy No. 11. This policy requires the use of one of three nationally  recognized evaluation and validation standards. However, these policies  and standards do not include criteria to specifically identify and manage  the use of foreign software suppliers.
To assist systems in meeting information assurance requirements, DOD  has developed the Defense Information Technology Security Certification  and Accreditation Process (DITSCAP) as a standardized evaluation  process. This process includes a review by a designated approving  official who certifies that the security features and information technology  system safeguards will maintain information assurance through the life  cycle of a system. The process results in an agreement between the  program manager, the intended user of the system being certified, and the  approval authorities that defines critical schedule, budget, security,  functionality, and performance issues. The process includes a requirement  for a threat assessment, but does not articulate how this information  should be developed or reported. As such, it does not direct program  managers to consider foreign involvement in software development as a  risk or threat that needs to be addressed for information system security.  In addition, while the process is mandatory, the implementation details  may be tailored and, in some cases, integrated with other acquisition  activities and documentation. According to DOD Software Assurance  Program officials, the DITSCAP approving authority is not expected to  evaluate whether the risks have been identified appropriately, only that  the process will mitigate the risks identified. If the program manager has  not identified risks from foreign suppliers, the process cannot be expected  to mitigate them.
DOD also requires weapon programs to protect certain types of  information during transfer of technology to foreign entities. For example,  documents such as the Technology Assessment/Control Plan and the  Program Protection Plan address the risks associated with the potential  release of information to foreign governments through cooperative  programs and foreign military sales, but the documents do not provide  information on specific suppliers who will be performing work, such as  software developers. The Technology Assessment/Control Plan establishes  planning requirements for the potential release of sensitive information to  foreign entities involved in cooperative programs or purchasing military  equipment. It evaluates the risk of releasing critical military capability or  sensitive information and technology against the benefit of the sale to the  United States. The plan also outlines the security requirements to prevent  compromise. The purpose of the Program Protection Plan is to identify  measures to protect Critical Program Information from hostile collection  efforts and unauthorized disclosure during the acquisition process.
section_title: Initiatives to Address Software Concerns Have Yet to be Implemented
paragraphs: While DOD has taken steps to strengthen software acquisition practices, it  has yet to implement practices to better manage software development  security risks in weapon programs. Currently, each of the military services  is developing plans for improving software acquisition. The improvement  plans are each at varying stages of development and include practices  such as pilot programs for providing information on software metrics,  additional training programs, and teaming arrangements with SEI for  improved overall software management. DOD has also begun policy-level  initiatives focused on better software management and on identifying and  specifying software security processes and technologies to protect  systems and network capabilities from various internal and external  threats. Specific initiatives include the following:    The Tri-Service Assessment Initiative began in 1999 to strengthen  software acquisition and development as well as address repeated  performance shortfalls attributed to software. Task forces conducted  detailed assessments of software-intensive programs to identity  potential improvements in overall software acquisition processes.
A source selection criteria working group is tasked with clarifying the  policy on source selection criteria for software intensive systems and  the application of software product maturity measures. Another  working group is tasked with developing a proposal for a centralized  clearinghouse of software best practices, but DOD has not approved  any proposals.
In October 2003, the Office of the Assistant Secretary of Defense  (Networks and Information Integration) established the position of  Deputy Director for Software Assurance that is, as part of its function,  to coordinate software security efforts with other existing initiatives  concerning software protection, antitamper technologies, and software  producibility. In addition, since its inception, the office has initiated  working groups intended to focus on mitigating software risks and  improving software security. To date, while these initiatives have  presented top level findings and recommendations within DOD and in  public forums, they have not externally published reports or obtained  funding for implementing the recommendations.
While these plans and initiatives may help to increase DOD’s focus on  software security and may lead to the development and identification of  several potential software security best practices, DOD software  assurance officials acknowledge that significant effort remains to  adequately mitigate software risks to weapon systems.
section_title: Program Officials Generally Did Not Manage Risks from Foreign-Developed Software
paragraphs: While DOD initiatives have begun to recognize potential risks from foreign  software suppliers, this is not always the case within the weapon programs  where software is developed or acquired. Program officials for most of the  systems we reviewed did not make foreign involvement in software  development a specific element of their risk identification and mitigation  efforts. As a result, program officials’ knowledge of the foreign-developed  software included in their weapon systems varied. In addition, risk  mitigation efforts emphasized program level risks, such as meeting  program cost and schedule goals, instead of software security risks.  Further, program managers often delegated risk mitigation and source  selection to their prime contractors who tended to be concerned with  software functionality and quality assurance, rather than specifically  addressing software development risks associated with foreign suppliers.
subsections: section_title: Program Office Knowledge of Software Suppliers Varied Across Weapon Systems
paragraphs: Knowledge of the extent of foreign involvement in software development  varied greatly for the 16 weapon system programs we reviewed. Overall,  the knowledge program managers had was based on the function of the  software being developed, the manner in which it was acquired, and the  specific handling requirements. While none of these programs could fully  identify all foreign-developed software for their systems, six program  offices had significant knowledge of foreign software developers.
Knowledge of software developed for weapon systems can vary based on  the different functions needed to meet mission requirements. Program  offices were most knowledgeable about the foreign-developed software  for the onboard portions of their programs, with 4 program offices able to  identify all the software produced by foreign suppliers, and 11 program  offices able to identify at least some of the software produced by foreign  suppliers. Onboard software is that which actually runs the weapon  system, even if that software is not located on the main component  (aircraft, missile, satellite, etc). For example, onboard software for a  missile system could include software located on a remote platform used  to guide the missile toward its target. Because onboard software is the  most critical for meeting mission requirements and other program goals,  program managers placed greater emphasis on the quality, functionality,  and usually, the security of this software. In 9 of the systems we reviewed,  either prime contractors or major subcontractors conducted software  development for the onboard systems. However, in meeting with prime  contractors, we found that while this increased their knowledge about  foreign software suppliers, the information was not always shared with  government program managers, and therefore was not available for them  to use to make risk management decisions to address software security.
Program offices and contractors reported very little knowledge about the  level of foreign involvement for offboard software. This software,  sometimes referred to as ground based, interacts with the onboard system  to provide updated information in support of operational activity. For  example, one program uses standard mission planning software that  interacts with the onboard flight software to provide information used for  navigation and targeting. In addition, offboard software is often used to  check for errors or malicious code and to produce, maintain, or verify  onboard software. Program officials from 10 of the programs we reviewed  indicated they had very little knowledge of the developers for their  offboard software, including those portions that may have been developed  by foreign suppliers.
As DOD is attempting to find new ways of reducing the time and money it  takes to develop software code, the increased use of COTS software may  introduce additional risks and further limit visibility into the existence of  foreign-developed software. Officials for 13 of the programs reviewed had  almost no insight into the use of foreign developers for any COTS software  placed on their systems. Even when procured directly from a known  supplier, program officials could not guarantee whether additional  subcontractors were used for software development. According to DOD  and program officials, visibility into COTS software is limited by the  willingness of the producers of that software to share information on how  the code was developed. For one program we reviewed, a substantial  portion of the system was a commercial acquisition. As such, the software  product was not originally developed for DOD and therefore the program  office had no knowledge of software development suppliers because it did  not purchase the rights to the software. Further, officials from five  programs told us that the cost of identifying and managing foreign  software suppliers, especially for COTS software, could be substantial.  Officials from two programs said that even if available, this information  would not offer significant software security improvements in light of the  cost required for identifying foreign suppliers of COTS software. DOD and  program officials have indicated that commercial software producers  often demand a cost premium to share software and source code  information that would be required to determine this information.
Similar to COTS, software from other applications and embedded software  is often accepted without full knowledge of the source of development.  Legacy, or reused code, is most prevalent when software programs are  updated into newer versions or when software just requires editing and  enhancing the older code rather than developing new code. When asked to  produce software similar to what they have developed previously,  manufacturers can use all or part of the legacy code as a basis for  developing or modifying the new code. Ten of the programs reviewed  accepted legacy software without fully identifying the sources of  development. Software is also developed as part of the hardware  components it is tasked with managing. This embedded software is used to  control other electronic hardware products, either onboard or offboard  the weapon system, and is often purchased from lower tier  subcontractors. While we did not specifically ask for information on  foreign suppliers responsible for embedded software, officials for two of  the programs reviewed stated that this software tends to include more  reused and COTS software. These officials indicated that this could limit  the visibility of the software suppliers because acceptance testing was  usually only performed to prove functionality and the software was not  further evaluated to determine the actual source that developed the  software code. Further, DOD and prime contractor officials told us that it  is sometimes difficult to determine whether the actual hardware  subcontractor developed embedded software, or if it was done by a  software developer hired by that subcontractor, thus further reducing  visibility of software suppliers.
While most of the program offices we reviewed did not specifically track  foreign software suppliers for security purposes, almost all of the  programs had opportunities to gain such knowledge through practices  designed to collect other information. One way program offices obtained  information on software suppliers was through requirements in their  prime contracts. Many of the program offices in our review were able to  obtain some information on software suppliers, either directly or  indirectly, because it was contractually required. In some cases, this  information was available early in the proposal process when bidders  were required to identify suppliers they intended to use in the  development and manufacturing phases, including potential foreign  suppliers and components they were expected to provide. For example,  officials from one program office said they were effective in determining  software suppliers at the prime and subcontractor levels because they  were intimately involved with source selection and contract negotiation.  However, only five of the prime contractors reported that they were  required to notify the program office concerning their decisions on  software subcontracting. Once a winning bidder is selected, more  information is often available to the program manager. For example,  contracts for 12 programs contained a requirement that the contractors  provide a software development plan that included information on some  of their planned suppliers, development risks, and action plans for the  contract period. In at least two cases, program software managers became  aware of foreign software suppliers while collecting information requested  for this review.
Program officials also said that some knowledge of foreign software  development was available as a result of procedures in place because their  programs contained classified or technical program information. For  example, of the six programs that had significant knowledge of foreign  software suppliers, four reported they had very few foreign suppliers  because handling restrictions for classified information precluded the  involvement of such suppliers, not because they were specifically  managing software development risks. Similarly, contractors for 12  programs had information available from the export license process. While  limited information on the supplier and location of foreign entities  performing software work was available from export licenses, contractors  request approval directly from the State Department, which may not refer  the application to DOD or the individual program offices. Consequently,  program managers likely did not have this information available for use in  software risk management decisions.
section_title: Program Risk Mitigation Efforts Are Focused on Meeting Performance Requirements and Are Often Delegated to Contractors
paragraphs: The relative importance of software security in risk mitigation efforts also  varied greatly across the systems we reviewed. For 11 of the 16 systems,  program managers have not identified foreign supplier involvement in  software development as a significant risk to the security of their weapon  systems. Instead, program managers concerned with completing their  programs on budget and on schedule generally focused risk mitigation  efforts on program level risks associated with the performance of system  components, not on internal software development security risks. When  specifically identified, software risks are usually defined by their impact  when integrated with these system level risks and do not specifically focus  on foreign suppliers used in software development. Software generally  only becomes a concern for program managers as it begins to affect the  cost or schedule of the program. For example, one of the programs we  reviewed lists “software executibility” as a program level risk. The risk is  based on potential cost and schedule overruns should the software not  function as needed to allow related system components to meet mission  requirements, rather than potential vulnerabilities from foreign suppliers.  For these programs, security risks have generally been implemented to  prevent unauthorized access to classified or technical program  information, provide security at contractor facilities, and limit access to  export-controlled technical information in accordance with ITAR license  requirements, rather than specifically for software security. In addition,   12 of the programs used the Technology Assessment/Control Plan and the  Program Protection Plan to ensure that risks associated with foreign  participation on the program were addressed.
Programs that identified software security as a risk focused on limiting  foreign access to software development facilities and denying foreign  access to software code. In addition, these programs employed various  measures to address software security consistent with information  assurance requirements. These measures included the use of password  protection, firewalls, or encrypted software, but they did not always focus  on risks from foreign involvement in software development. Further,   11 programs mentioned using DITSCAP as a means for addressing general  software security. However, interpretation and implementation of this  requirement can vary across programs. For example, according to officials  from two programs, the current DITSCAP requirements do not govern  contractors in cases where the requirements were not included as part of  the original contract. Representatives from two of the programs we  reviewed noted that guidance for implementing DITSCAP was confusing  and that they were uncertain whether the process applied to their  programs. Program officials responsible for software development on one  other program indicated that they had no knowledge of this process. In  cases where DITSCAP is being implemented, the certification and  accreditation process requirements are determined based on the program  manager’s assessment of risks. If the program manager has not identified  foreign software development as a program risk or threat, it will not be  addressed by the process.
Because security and software risks are generally defined in terms of  programmatic elements, program managers often delegate the  identification and mitigation responsibility to the contractors who are  developing the system and are therefore more knowledgeable about what  functions are needed from the software. Officials from eight of the  programs we reviewed said they expect contractors to ensure quality and  security on their systems because their software development processes  are more mature than those required by DOD, and that such practices  could indirectly address foreign software risks. For example, contractors  for these eight programs were presumed to be addressing software  security because they were employing practices such as peer review  and  software testing consistent with SEI development models. Peer review is  recognized as a best practice for improving software development and is  generally performed to improve the quality and functionality of software  code. In terms of security, peer review can reduce the likelihood that an  individual programmer can insert malicious or other harmful code.  Through dedicated software testing, teams assess the quality of the  software to uncover gaps and make it as defect-free as possible. However,  on eight of the programs we reviewed, decisions on the amount of  software code to test were made based upon the risks and benefits to the  functionality of the system to be tested, not on the benefits to security.  DOD and SEI officials said that the amount of effort needed to  comprehensively test every line of code to ensure complete security could  be physically impossible and would require immense resources.
Because contractors for the weapon systems we reviewed had not  received specific direction from program managers to address risks from  foreign suppliers in software development, they tended to focus on  development efforts aimed at meeting stated requirements, such as  software quality and functionality. In fact, officials from the 15 contractors  that responded to our review indicated this was the focus of their software  development activities. While SEI representatives told us that rigorous  software development could help improve the quality and functionality of  software by decreasing the number of errors in software code, they also  said that alone their models should never be expected to completely  address software security risks. Officials from one contractor we  interviewed that employs practices consistent with SEI’s highest level  indicated that unless software security is a specific contract requirement,  they would not modify their practices to address associated risks. SEI  experts confirmed that the models they have developed do not include a  security element and are not intended to certify that improved processes  will address risks related to software security. In fact, it is possible that  using software development practices to increase efficiency could lead to  an increase in security vulnerability by encouraging the use of legacy and  COTS software, unless risks are managed appropriately.
For several programs we reviewed, contractors made risk identification  and mitigation decisions for business reasons and to avoid additional  resource burdens (i.e., cost and access) associated with incorporating  foreign suppliers necessary for software development, as opposed to being  done for security reasons. For example, prime contractors for two  programs did not use foreign subcontractors for economic reasons;  namely the company wanted to maintain the software expertise within the  company. While restricting foreign access solely for economic reasons  could result in a decrease in software development security risks, it might  also preclude foreign suppliers that could offer new capability or lower  costs to the government. For example, contractor officials for three  weapon systems told us that they restrict foreign involvement in software  development because it costs too much to develop and monitor security  procedures to separate foreign nationals from classified and sensitive  information, not because they feel their involvement is a risk to the  program. In yet another case, the contractor did not want to create  dedicated partitions, such as firewalls, required to prevent employees from  a foreign subcontractor from accessing unauthorized information in the  system design database and instead contracted with a domestic supplier.  Finally, software managers for one program told us that when foreign  nationals modify or update software they had developed, it was necessary  to isolate test facilities to meet security requirements, which resulted in  increased cost and delays to other test activities. The officials said that, in  similar cases, contracting decisions might sometimes be made in favor of  U.S. suppliers to avoid costs and delays.
section_title: Conclusions
paragraphs: Because software is increasingly responsible for advances in weapon  system capabilities, it is essential that DOD and program managers take  appropriate steps to identify and manage software-related risks. While  DOD has made improvements to system engineering and software  development practices that can reduce the likelihood of defects in  software code, current methods of testing focus on the quality of software  and related functionality and failures, which will not necessarily uncover  malicious intent. As the amount of software on weapon systems increases,  it becomes more difficult and costly to test every line of code. Further,  DOD cannot afford to monitor all worldwide software development  facilities or provide clearances for all potential software developers,  especially for COTS software. Given the global nature of the software  industry, which offers benefits to software cost and functionality needed  by weapon systems, DOD also cannot afford to exclude all foreign  suppliers from its programs. While program managers should be allowed  discretion in managing their acquisitions, they are responsible for knowing  more about who is developing software and where and for working with  DOD’s software assurance resources, and other organizations as  necessary, so that risks can be identified and assessed accordingly. Unless  this is done early in the software acquisition process, it cannot be included  as part of software source selection and risk mitigation decisions and  could result in increased cost and less effective security measures if risks  have to be addressed later in the acquisition process.
section_title: Recommendations for Executive Action
paragraphs: We have previously made recommendations to DOD to adopt more  effective software development practices and to increase oversight of  software intensive systems to improve acquisition outcomes. While DOD  attempts to better its software acquisition policies and implement new  initiatives, it must take steps to ensure that security is an integral element  in decision making and that program managers mitigate risks accordingly.  We recommend that the Secretary of Defense take the following three  actions to address risks attributable to software vulnerabilities and  threats:    Require program managers, working with software assurance experts,  acquisition personnel, and other organizations as necessary, to  specifically define software security requirements, including those for  identifying and managing software suppliers. These requirements  should then be communicated as part of the prime development  contract, to be used as part of the criteria to select software suppliers.
Based on defined software security requirements, require program  managers to collect and maintain information on software suppliers,  including software from foreign suppliers. This information should be  evaluated periodically to assess changes in the status of suppliers and  adjustments to program security requirements.
Require the Office of the Assistant Secretary of Defense for Networks  and Information Integration and the Office of the Undersecretary of  Defense for Acquisition Technology and Logistics, as part of their role  to review, oversee, and formulate security and acquisition practices, to  work with other organizations as necessary to ensure that weapon  program risk assessments include specific attention to software  development risks and threats, including those from foreign suppliers.  For example, certification and accreditation processes, such as  DITSCAP, should include verification that software development  practices contain adequate security measures to address identified  risks and threats.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, DOD agreed with our  findings that malicious code is a threat that is not adequately addressed in  current acquisition policies and security procedures and stated that the  department is working to strengthen software related risk management  activities. DOD also noted the need to enhance its risk management  processes to factor in vulnerabilities analysis of proposed software  products, security risks of suppliers’ processes, and counterintelligence  threat information. DOD partially concurred with our three  recommendations based on the concern that they place too much  responsibility for risk mitigation on program managers. Although the draft  report recognized that software assurance experts from the Office of the  Assistant Secretary of Defense for Networks and Information Integration  were necessary to support program managers in risk mitigation efforts, we  broadened two of our recommendations to include acquisition and other  organizations to address this concern. DOD also provided separate  technical comments that we incorporated into the report as appropriate.  DOD’s letter is reprinted in appendix II.
DOD agreed that software security risks should be defined for DOD  weapon programs, but noted that program managers should not be solely  responsible for defining security requirements, including those for  identifying and managing software suppliers. Instead, DOD stated that  program managers should be able to rely on external resources to gain  threat information on suppliers and that formulation and oversight of  security practices should be a collaborative function among several offices  within DOD. While we continue to believe that program managers and  software assurance experts play a critical role in defining software  security requirements, we do see the value of involving other DOD  resources to provide coordinated evaluation of broader security concerns.  As such, we modified our recommendation to reflect the inclusion of  acquisition personnel and other organizations as necessary.
DOD also agreed that information on software suppliers, including foreign  suppliers, should be collected and that this information should be  periodically assessed to determine if adjustments to security requirements  are needed. However, DOD indicated that centralized information on  software suppliers is necessary because the cost of collecting and  maintaining this information would require resources and assets beyond  those of individual program managers. DOD indicated its intent to develop  a database to identify, track, and maintain information on security risks  from specific software suppliers, which could be used by program  managers across various weapon and other programs for developing  acquisition strategies, plans, requests for proposals, and contracts. While  we agree that such a database would be helpful to program managers in  collecting and maintaining information on software suppliers, we made no  change to the recommendation because the program managers should be  responsible for collecting this information until such a database is  developed and for directing the collection of information from the  database once it is completed.
Finally, DOD agreed that it should ensure that program risk assessments  include specific attention to software development risks, including those  from foreign suppliers. However, DOD suggested that this might be best  accomplished through collaboration between the Office of the Assistant  Secretary of Defense for Networks and Information Integration, the Office  of the Undersecretary of Defense for Acquisition Technology and  Logistics, and the Office of the Undersecretary of Defense for Intelligence.  This seemed reasonable, and we adjusted our recommendation to reflect  the inclusion of other organizations. Further, DOD agreed certification  activities such as DITSCAP can assist in addressing insider threats in  software development, but that additional guidance is necessary to ensure  that software security risks are addressed during system design and  development or when selecting software sources.
We are sending copies of this report to interested congressional  committees; the Secretary of Defense; the Secretaries of the Air Force,  Army, and Navy; the Commandant of the Marine Corps; and the Director,  Office of Management and Budget. We will also make copies available to  others upon request. In addition, this report will be available at no charge  on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512- 4841. Staff acknowledgments are listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine how the Department of Defense (DOD) measures the extent  of foreign involvement in software development in weapon systems and  how risks associated with using foreign suppliers for software  development are measured and mitigated, we reviewed relevant DOD  guidance, policies, regulations, and procedures. In addition, we spoke with  DOD officials from the Office of the Under Secretary of Defense  (Acquisition Technology & Logistics), the National Security Agency, the  Defense Information Systems Agency, the Defense Advanced Research  Projects Agency, the Office of the Assistant Secretary of Defense  (Networks and Information Integration), the Department of the Army, the  Department of the Air Force, and the Department of the Navy. We met  with software experts at the Software Engineering Institute of Carnegie  Mellon University to obtain information on software development  practices and risk identification and mitigation techniques used by the  software industry. Additionally, we met with the Association of Old Crows  (The Electronic Warfare and Information Operations Association) whose  membership includes individuals and companies involved in the design  and development of software used in DOD weapon programs.
To document and analyze how programs specifically measure and manage  their use of foreign-developed software, we identified 16 DOD weapon  systems and solicited information from each program office and prime  contractor. We selected these weapon systems based on recommendations  from DOD officials and on our internal knowledge of the systems. While  our selection of programs cannot be generalized to the population of all  DOD systems, the systems selected varied by product type, represented  each of the military services, and represented a range of DOD contractors.  The systems reviewed were the Abrams System Enhancement Package,  AH-64D Apache, Bradley Upgrade, C-130 Avionics Modernization Program,  C-130 J Hercules, C-17 Globe Master, Comanche Reconnaissance Attack  Helicopter (RAH-66), F/A-18 Super Hornet, F/A-22 Raptor, Future Combat  Systems, Global Hawk Unmanned Aerial Vehicle, Joint Helmet Mounted  Cueing System, Joint Strike Fighter, Patriot Missile System, Tactical  Tomahawk Missile, and Wideband Gapfiller Satellites.
Using their respective command liaisons to initially contact each office,  we distributed a structured set of questions to solicit information from  software managers designated by individual program managers to respond  to our inquiry. To further determine how programs manage and mitigate  their use of foreign-developed software, we then tailored follow-up  questions to solicit information and documentation in areas such as  program risk identification and management practices, security policies  and procedures, and software contracting management practices. To learn  more about program practices for managing and mitigating the use of  foreign-developed software, we solicited information and documentation  from the prime contractor for each system using contacts provided by  program office officials. Information requested from contractors included  government guidance for software practices, company software  development and security practices, software risk mitigation efforts,  software testing procedures, and software sourcing decision processes.  We received information from 15 of the prime contractors through written  responses, on-site interviews, and other means such as telephone  conversations. We also obtained several security and software related  documents such as the Program Protection Plan, the Software  Development Plan, and other program specific documents, such as the  contract, for the systems we reviewed.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: John Neumann, Brian Mullins, Delores Cohen, Shelby S. Oakley,  Christopher Miller, Gary Middleton, and Marie Ahearn made key  contributions to this report.