Information Technology: DOD Needs to Fully Implement Program for Piloting Open Source Software
Released: 2019-09-10T00:00:00

Letter

Background

OSS is software distributed under a license that provides broad rights to use, modify, and redistribute the original source code. Open source licenses impose certain obligations on users who exercise these rights. Specific obligations vary among the many different open source licenses. Common obligations include making the source code available, publishing a copyright notice, or giving any recipient of the program a copy of the license. Certain restrictive open source licenses allow users to copy, modify and distribute software provided that modified versions (i.e., derivatives) are subject to the same license terms and conditions as the original code. This is intended to prevent software that is derived from or contains code issued under such a license from becoming a closed- source product that can be marketed and sold exclusively.

The reuse of OSS is viewed as a promising means to reduce development costs while improving software quality. According to software experts, software reuse has the potential to: increase reliability because systems will be developed with thoroughly tested and proven components, increase productivity by reducing the time and effort needed to develop software, reduce costs by enabling the sharing of knowledge and practices needed to develop and maintain software, and establish a more standard and consistent approach to software development and maintenance by using common components and procedures.

OMB Memorandum on Federal Source Code Policy

In August 2016, OMB issued a memorandum to the heads of departments and agencies to ensure that new custom-developed source code be made available for reuse across the federal government. The memorandum was intended to improve the way federal agencies buy, build, and deliver information technology and software, and required that all agencies establish a pilot program under which at least 20 percent of new custom-developed code would be released as OSS for 3 years. OMB also required that agencies develop a metric to calculate the percentage of code released as OSS to gauge its progress on implementing the pilot program.

OMB’s memorandum also identified four supporting requirements, among others, needed to implement an OSS pilot program: Issue an OSS policy that ensures code is available for government- wide reuse.

Conduct an OSS three-step software solutions analysis that includes: (1) a strategic analysis and analysis of alternatives, (2) consideration of existing commercial solutions, and (3) consideration of custom development. In addition, agencies must consider several factors throughout each of the three-steps of the analysis such as cloud computing and open standards.

Secure data rights to government-wide reuse and inventory new custom code, in accordance with the guidance provided by the code.gov website.

Facilitate the OSS community by developing and releasing the code in a manner that (1) fosters communities around shared challenges; (2) improves the ability of the OSS community to provide feedback on, and make contributions to, the source code; and (3) encourages federal employees and contractors to contribute back to the broader OSS community by adding value to existing projects. In doing so, agencies should comply with the following principles: (1) leverage existing communities, (2) engage in open development, (3) adopt a regular release schedule, (4) consider code contributions, and (5) document source code to facilitate use and adoption.

DOD Implementation of OMB’s Open Source Software Requirements

DOD’s CIO is responsible for implementing OMB’s requirements for the department’s OSS pilot program. The CIO reports directly to the Secretary of Defense, and is responsible for the department’s information technology (including national security systems and defense business systems), information resources management, and efficiencies. The CIO is also responsible for developing strategies and policy on the operation and protection of all of the department’s information technology and information systems. Other responsibilities include maintaining a consolidated inventory of mission critical and mission essential information systems, evaluating and monitoring performance measurements, and other duties to manage the information environment throughout the department.

In addition, the Defense Digital Service is responsible for assisting the CIO in implementing the OSS pilot program, among other initiatives. The Defense Digital Service is composed of commercially experienced software developers, software designers, product managers, and problem solvers within DOD. The organization works on specific projects or programs in support of the DOD in a hands-on way to materially improve digital services across the department. The Defense Digital Service also works with the CIO to monitor the identified programs, facilitate the process to open source the code, and populate the source code inventory located on Code.mil.

DOD’s Efforts to Increase Use of Open Source Software

In June 2018, DOD’s CIO issued a report to Congress as directed by section 875(b) of National Defense Authorization Act for Fiscal Year 2018 (Public Law 115-91). The report provided Congress with the department’s plan to implement the OSS pilot program established by OMB’s memorandum. In the report, the CIO committed to sharing its unclassified, custom-developed source code as widely as possible in four ways: (1) review and select software programs that have self-identified to the Defense Digital Service as ready to open source its code; (2) query its contracts database to identify contracts that contain appropriate data rights language; (3) determine if source contained within existing source code repositories can be made available; and (4) issue a department- wide data call to identify and select programs where new, custom code is being developed.

The CIO also reported that the department would prioritize and assess identified software programs and work with components to develop mechanisms to report progress. The report included selection criteria for identifying candidate software programs: (1) programs with contractually secured government data rights; (2) programs with contractual rights to enable, support, and enforce DOD and government-wide sharing and reuse of custom-developed code; (3) programs that have appropriate administration to ensure that government’s rights are maintained; and (4) programs that utilize best practices to ensure custom-developed code, documentation, and other associated materials are delivered in a reusable manner.

The CIO also reported that the Defense Digital Service would assist programs. Specifically, the Defense Digital Service is to develop guidelines, processes, and answers to frequently asked questions regarding the use of OSS.

In October 2018, the CIO issued a memorandum to the Chief Management Officer, secretaries of the military departments, Chairman of the Joint Chiefs of Staff, under secretaries of Defense, chiefs of the military services, general counsel, Director of Cost Assessment and Program Evaluation, Director of Operational Test and Evaluation, and the Assistant Secretary of Defense for Legislative Affairs notifying them of the need to implement an OSS pilot program in accordance with OMB’s 2016 memorandum. The CIO required these organizations to take four actions within 30 days of issuing the memorandum: Identify all unclassified custom-developed source code created or paid for by the department—regardless of data rights and open source status—created on or after August 2016 and provide the CIO information required by the guidelines on the code.gov website; Identify and provide a point of contact that can participate in open source efforts;

Direct authorizing officials to rapidly review and approve unclassified code for public, open source release after appropriate security, code, and policy review; and

Direct contracting officers to secure the least restrictive data rights to custom-developed source code in all future contracts in accordance with DOD federal acquisition regulations.

DOD Has Not Fully Implemented an Open Source Software Pilot Program and Related OMB Requirements

DOD was mandated by law to initiate the OSS pilot program established by OMB memorandum M-16-21 which required (1) releasing at least 20 percent of newly custom-developed code each year for the term of the pilot program, and (2) collecting additional data concerning new custom software to inform measures to gauge the performance of the pilot program. Further, the OMB memorandum also required DOD to (1) issue an OSS policy, (2) conduct an OSS analysis, (3) secure data rights and inventory custom code, and (4) facilitate the OSS community.

As of late April 2019, DOD had not fully implemented the OSS pilot program mandated by law. DOD had partially implemented the requirement of releasing at least 20 percent of newly custom-developed code as OSS. Specifically, as of July 2019, the Code.gov website reported that the department had released less than 10 percent of its custom developed code. The department was in the early stages of its pilot program and had not determined when the pilot would be fully implemented. The CIO reported that the size of the department makes it nearly impossible to inventory all of its source code custom developed since August 2016. As such, the CIO stated that it would be difficult to meet the OMB memorandum’s goal of releasing at least 20 percent of its new custom code as OSS.

In addition, DOD had not implemented the requirement to develop a consistent measure to gauge the performance of the department’s pilot program. DOD had not developed such a measure due to a lack of consensus in the department about what data should be collected. According to the CIO, if the measure is “lines of code,” then it unfairly discounts projects that invest a significant amount on research, but are small otherwise. If the measure is “project hours,” then it discounts those projects that came about from sparks of innovation that took little time to develop. If the measure is “project count,” then it ignores the other two possible measures. The CIO noted that since the components will be expected to collect and report the required data, the CIO office plans to reach out to them to facilitate consensus around data needs and what measure should be used to calculate and monitor performance.

Regarding the four OMB memorandum requirements supporting the implementation of the pilot program, the department partially implemented three and did not implement the remaining one. Table 1 describes the extent to which DOD implemented the OMB supporting requirements, and is followed by a description of DOD’s efforts on each requirement.

Issue OSS policy. DOD had not implemented the requirement to issue an OSS policy. According to DOD’s CIO, the department has existing acquisition policies applicable to OSS, such as the 5000 series and a memorandum issued in October 2009 that clarifies OSS. However, according to DOD officials, these policies are outdated and do not comply with OMB’s memorandum. For example, while the department’s policies indicated that programs must conduct an analysis of alternatives, trade studies, or a business case analysis prior to initiating any technology acquisition or custom code development, they did not require programs to consider the value of publishing custom code as OSS and negotiate data rights reflective of its value.

The department acknowledged that it does not have a policy that addresses the OMB memorandum’s requirement. According to the CIO, the department had been slow to develop a policy because these types of changes require significant resources, coordination, and buy-in across the department that will take additional time to address.

The CIO also stated that the department plans to update its existing OSS memorandum by the end of the 2019 calendar year and issue it as policy. In particular, DOD intends to work with acquisition and program management officials to define approaches, processes, and best practices to expand software reuse. If the department effectively implements this intended step consistent with the OMB memorandum, DOD should be able to fully address this requirement.

Conduct OSS analyses. DOD had partially implemented the requirement to conduct analyses to consider alternative software solutions. According to the DOD CIO, of the three elements required by OMB for a three-step analysis, the department’s current 5000 series policy addresses some of these elements. For example, as previously mentioned, the policy required programs to conduct an analysis of alternatives, trade studies, or a business case analysis prior to initiating any technology acquisition or custom code development. However, according to the CIO, DOD’s policy did not require programs to consider the value of publishing custom code as OSS and negotiate data rights reflective of its value-consideration. According to the CIO, the department has plans to address gaps in its existing policy by the end of the 2019 calendar year. If the department effectively implements this intended step consistent with the OMB memorandum, DOD should be able to fully address this requirement.

Secure data rights and inventory new custom code. DOD had partially implemented this requirement by initiating the process to secure data rights for government-wide reuse and inventory its new custom code. Specifically, the October 2018 memorandum called for defense organizations to direct contracting officers to secure the least restrictive data rights for custom-developed source code in all future contracts, and identify all unclassified custom-developed source code created after August 2016. However, when we discussed this matter with DOD information technology and software officials responsible for the management of software in December 2018—2 months after the October 2018 memorandum had been issued— seven of 11 officials were unaware of the statutory mandate to initiate the pilot program in compliance with the OMB memorandum. Further, DOD has not provided a milestone for when it expects its inventory to be completed.

Facilitate OSS community. DOD had partially implemented the requirement to establish an OSS community. According to the DOD CIO’s Custom Developed Source Code Data Call memorandum, dated October 2018, the DOD CIO is working with Defense Digital Service to develop guidelines and processes on when and how to open source code. In February 2017, DOD announced the launch of Code.mil, an open source initiative led by Defense Digital Service, that allows software developers around the world to collaborate on unclassified code written by federal employees in support of DOD projects. Finally, the Defense Information Systems Agency established a website (Forge.mil) where community members can collaborate on open source and DOD community source software. The Forge.mil website also enables collaborative development through services such as software version control, requirements management, discussion forums and document repositories.

However, DOD had not yet fully engaged in open source code development, established a regular release schedule for its software code, or fully documented its source code to facilitate use and adoption department-wide. According to the CIO, the department is in the early stages of implementing the OSS pilot program and had not yet published a revision to the existing OSS policy memorandum. The CIO stated that the office plans to request collaboration and input from organizations throughout DOD for improvement initiatives and identifying specific processes and expectations for improving custom-developed software within the Department. However, DOD has not provided a milestone for when the requirements will be fully implemented and stated that achieving 100 percent compliance is not a realistic expectation.

Until DOD fully implements the OSS pilot program mandated in the National Defense Authorization Act for Fiscal Year 2018 including the requirements of OMB memorandum M-16-21, the department will likely miss opportunities to achieve related cost savings and efficiencies. Further, the department will not be effectively positioned to ensure management oversight and implementation of the pilot program.

DOD Officials Shared Views of Expected Benefits and Risks Associated with the Use of Open Source Software

DOD Officials Agree That Using Open Source Software Could Result in Financial Benefits and Increased Efficiency

DOD officials representing 11 components reported that OSS can potentially yield financial benefits and increase efficiency. Officials provided the following examples of financial benefits:

Officials in the office of the Navy Chief Information Officer, the Army Communications-Electronics Command, the Office of the Under Secretary of Defense for Acquisition and Sustainment, and the Office of the Assistant Secretary of the Air Force for Acquisition, Technology, and Logistics stated that OSS is generally less expensive than commercial off-the-shelf (COTS) leading to cost savings.

An Air Force official we spoke with stated that the increased use of OSS may potentially result in cost savings. Further, the official noted general criteria used by the Air Force to identify software projects that may potentially be appropriate for the use of OSS. Specifically, OSS should be considered if, among other things, the required maintenance would not result in a reduction of cost savings or efficiency if the maintenance is performed in-house.

An official we spoke with at the Defense Advanced Research Projects Agency stated that OSS has positive benefits in terms of reducing costs by reducing duplicative efforts. In addition, this official also stated that OSS allows institutions of any size and budget to partake in shared investments providing access to software capabilities at a much lower cost.

A program manager from the Defense Information Systems Agency reported that the agency had identified an OSS solution that provided more functionality at less cost than the commercial solution provided through a vendor. The program manager explained that when the agency implemented the new OSS solution, it realized $20 million in annual savings over the commercial solution that had been maintained by a vendor.

Officials also shared examples of how OSS can increase efficiency in software development. For example,

Officials from the offices of the Navy CIO and the US Marine Corps CIO stated that OSS solutions may increase efficiency by providing a rapid resolution to the needs and requirements of users. In contrast, rapid development efforts are not conducive for COTS solutions because of the long process required to obtain solutions that are needed quickly.

Similarly, an Air Force official noted that the increased use of OSS may result in increased efficiency by providing rapid responses to user requirements.

A program manager from the Defense Information Systems Agency reported that the selection of an OSS solution rather than a COTS solution contracted through a vendor had resulted in increased efficiency. The official explained that the use of the OSS solution allowed the agency to develop and maintain in-house skills that would not have been available had they opted to contract with a vendor providing a skilled workforce.

DOD Officials Expressed Differing Views on the Cybersecurity Risk Posed by Open Source Software

Officials from the 11 components expressed mixed views on managing cybersecurity risks that could be posed by using OSS. Specifically, officials from three components expressed their views that security concerns and the lack of a cybersecurity governance process could result in the sporadic use of OSS. For example:

A Navy CIO official viewed insider threats, such as a disgruntled employee embedding malicious code, as a factor that could significantly limit the use and sharing of OSS. According to Navy officials, without a process to verify that the software is free of malicious code, the Navy would risk the assurance it requires to increase the use of OSS. The official said that, in contrast, such concerns are not an issue when it comes to COTS software because of the test and verification process to ensure it is free from malicious code.

An official in the office of the Marine Corps CIO stated that OSS is used sporadically in their software development efforts because some cybersecurity officials within the Marine Corps discourage its use due to security concerns.

An official from the Army’s Communications-Electronics Command noted that DOD lacks a governance process once the originating entity releases the source code as open source. The originating entity no longer retains control over redistributed versions of the source code. According to Communications-Electronics Command officials, Army project managers may be hesitant to utilize OSS because of this perceived security risk.

On the other hand, DOD officials from eight components stated that the potential cybersecurity risks posed by the use of OSS were manageable and that the use of OSS should not be limited. For example:

The policy advisor from the Office of the Under Secretary of Defense for Acquisition and Sustainment noted that scanning tools to analyze and identify safe and reliable open source code are not being used. Employing available scanning tool options could result in discovering available OSS. The policy advisor also noted that building security into software operations, rather than through the development of software, would enable users to know if code has been subverted and to react appropriately.

A program management official from the Office of the Under Secretary of Defense for Acquisition and Sustainment suggested that security concerns may be mitigated by establishing a secure repository for trusted code.

An official in the Office of the Assistant Secretary of the Air Force for Acquisition, Technology, and Logistics reported that, as long as OSS is properly vetted to ensure it is secure and free from malware, it offers an opportunity for the department to achieve cost savings and efficiencies.

Conclusions

Pilot testing the use of OSS is an important way to ascertain and improve the way DOD buys, builds, and delivers information technology and software solutions. However, the department is in the early stages of implementing its pilot program and had not determined when the pilot would be fully implemented. Specifically, DOD has not made 20 percent of its new code available for reuse nor has it identified a measure to gauge the performance of its pilot program. Moreover, DOD has not yet established milestones for securing data rights and conducting an inventory or facilitating community. Until DOD fully implements its pilot program and establishes milestones for completing the OMB requirements, the department will not be positioned to take advantage of significant cost savings and efficiencies.

Recommendations for Executive Action

We are making the following four recommendations to DOD:

The Secretary of Defense should ensure the department implements the pilot program by releasing at least 20 percent of newly custom- developed code as OSS. (Recommendation 1)

The Secretary of Defense should ensure the department identifies a measure to calculate the percentage of code released to gauge its progress on implementing the pilot program. (Recommendation 2)

The Secretary of Defense should ensure the department establishes milestones for completing the requirements of OMB memorandum M- 16-21 of securing data rights and conducting an inventory. (Recommendation 3)

The Secretary of Defense should ensure the department establishes a milestone for completing the OMB memorandum’s requirement of facilitating an OSS community. (Recommendation 4)

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report, which are reproduced in appendix II. In its comments, the department did not concur with our first recommendation, partially concurred with our second recommendation, and concurred with the third and fourth.

DOD did not concur with the first recommendation on ensuring that the department implements the pilot program by releasing at least 20 percent of newly custom-developed code as OSS. The department stated that it does not believe that the pilot program as described in the OMB memorandum is implementable as proposed. For example, DOD asserts that most of DOD’s custom developed software is created for weapons systems and releasing the associated code is sensitive for national security reasons. In addition, the size and complexity of DOD presents unique challenges for the department compared to other federal agencies such as inventorying all software development projects to establish a baseline. DOD added, however, that the OMB memorandum explicitly states that national security exceptions do not apply to the pilot program. DOD also stated that it recognizes the value of collaborative software development and has plans to release additional guidance on releasing OSS and procedures for maintaining its inventory. Once DOD establishes a baseline inventory of custom-developed software and the procedures for maintaining it, the department states it will be able to determine if the 20 percent is an appropriate goal.

We understand the potential constraints DOD faces and that national security considerations are to be factored into decisions DOD will need to make about which custom developed software to include in the pilot. However, DOD is mandated by law to implement the OSS pilot program established by OMB memorandum M-16-21. Further, the OMB memorandum instructs agencies to refrain from selecting code for release that would fall under exceptions such as national security risk. As such, DOD has flexibility on making decisions about which custom-developed code to include in the pilot. While we agree that a baseline inventory is needed, DOD must include at least 20 percent of new custom-developed code each year for the term of the program to satisfy the mandate.

DOD partially concurred with the second recommendation on ensuring that the department identifies a measure to calculate the percentage of code released to gauge its progress on implementing the pilot program. Specifically, the department stated that the additional guidance it plans to release before the end of 2019 on OSS will include measures to gauge how much code has been developed and how much has been released. In addition, DOD noted that these measures will support good management of the overall portfolio of information technology, even in the absence of the mandated pilot program established by the OMB memorandum. We believe that the measure to calculate the percentage of code should be used to assist the department in meeting the OMB memorandum’s requirements. We also agree with the benefits of developing a measure to manage the portfolio of information technology.

DOD concurred with the third and fourth recommendations related to establishing milestones for completing the OMB memorandum’s requirements of securing data rights, conducting an inventory, and facilitating an OSS community. According to the department, it has issued a memorandum that directed contracting officers to secure the least restrictive data rights to custom-developed source code, in furtherance of the OMB requirements, and also included a data call that forms an initial basis of an inventory of custom-developed software. Regarding facilitating an OSS community, DOD stated that it has formed a community of practice called DevSecOps that is open to all software development organizations in the department and plans to use this forum to facilitate collaboration on the use of OSS.

We are sending copies of this report to the appropriate congressional requesters and the Secretary of Defense. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-4456 or at harriscc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to: (1) assess the extent to which the Department of Defense (DOD) has implemented the open source software (OSS) pilot program and other related requirements established by the Office of Management and Budget (OMB), and (2) describe the views of responsible DOD officials on the use of OSS to achieve efficiency at the department.

To address the first objective, we selected six requirements from the OMB memorandum titled the Federal Source Code Policy: Achieving Efficiency, Transparency, and Innovation through Reusable and Open Source Software (M-16-21, Aug. 8, 2016) as criteria to assess the extent to which DOD has implemented the OSS pilot program. Two requirements establish the OSS pilot program: (1) releasing at least 20 percent of newly custom-developed code each year for the term of the pilot program, and (2) developing a metric to gauge the performance of the pilot program. The other four requirements support the implementation of the pilot program: (1) issuing an OSS policy, (2) conducting an OSS analysis, (3) securing data rights and inventorying custom code, and (4) facilitating the OSS community. We met with officials from OMB to collect background information on the selection of requirements for the pilot program established in memorandum M-16-21. We also met with officials from the Office of the DOD Chief Information Officer (CIO) and the Defense Digital Service to discuss the status of the department’s implementation of the OSS pilot program.

We reviewed DOD’s June 8, 2018 report to Congress and its October 2018 memorandum that details the department’s plans to implement the pilot program and compared them to the six requirements. To determine the extent to which the pilot program had been implemented, we evaluated DOD’s efforts to address each of the requirements using a 3- stage gradient scale (implemented, partially implemented, and not implemented). The requirement was assessed to be fully implemented if DOD provided us with sufficient evidence that the requirement had been fully met. We assessed a requirement to be partially implemented if DOD provided us with documentation of initial plans or had initiated action towards implementing the requirement. We determined that a requirement was not implemented when DOD did not provide us with documentation of planned or initiated actions to implement the requirement.

To address the second objective on views of various responsible DOD officials, using professional judgement, we selected components across the department responsible for the management and development of OSS. The scope of stakeholders selected represent department-wide nongeneralizable views including military components, defense agencies, and other offices. At least one representative was selected from the following components: (1) Office of the Under Secretary of Defense for Acquisition and Sustainment, (2) Office of the DOD CIO, (3) office of a military service CIO, (4) Military Service Software Center or Command Center, (5) the Defense Information Systems Agency, and (6) the Defense Advanced Research Projects Agency. We conducted interviews with DOD officials from the following entities: Office of the Under Secretary of Defense for Acquisition and Sustainment; Office of the DOD CIO; Offices of the Navy and Marine Corps CIOs; Office of the Air Force Chief Technology Officer; Army Communications Electronics Command; the Defense Information Systems Agency; the Defense Advanced Research Projects Agency; and the Office of the Assistant Secretary of the Air Force for Acquisition, Technology, and Logistics.

In order to summarize and report the views of the responsible DOD officials, we conducted structured interviews with representatives from the selected components. Each interview consisted of the same discussion topics based on the pilot program requirements established in OMB’s memorandum. The scope of this objective represents individual thoughts, views, and opinions and is not intended to convey an official or department response.

Prior to each interview, participants were provided with OSS discussion topics, and the OMB memorandum, titled the Federal Source Code Policy: Achieving Efficiency, Transparency, and Innovation through Reusable and Open Source Software (M-16-21, Aug. 8, 2016). The contents of each interview were reviewed and summarized to identify the general views of OSS and on the anticipated implementation of the pilot program requirements established in OMB’s memorandum. We noted similarities and differences in the responses provided by the officials in the use of OSS including, but not limited to, potential benefits of using OSS, managing associated risk, and opinions on implementing the pilot program in compliance with the OMB memorandum.

Discussions were split into two topic areas: practices on the use of OSS, and OMB’s memorandum to establish an OSS pilot program. Specifically, the discussion topics were presented during each meeting as follows: Part I: Practices on the use of OSS

Your experience or your organization’s history with the practice of using OSS as a means to achieve cost reduction or efficiencies when buying, building, or delivering information technology and software solutions.

The processes or practices that you or your organization perform to leverage open source code for projects that require the acquisition or development of custom source code.

The extent to which you or your organization shares or uses open source software. For example, do you share or use open source software: (1) within your organization only, (2) across the DOD with other military services or defense agencies, (3) with other federal agencies, or (4) outside the federal government with the public. Also, how is the source code shared, leveraged, catalogued, stored, and accessed.

Policies or guidance currently in use for OSS.

General views and opinions on the use of open source code.

Part II: OMB’s Memorandum to Establish an OSS Pilot Program

When and how you or your organization became aware of OMB’s memorandum on Federal Source Code Policy.

Your opinions and views about the OMB memorandum.

Any specific concerns or reservations about the requirements contained in the OMB memorandum.

The extent to which you or your organization may already be performing the steps in OMB’s proposed Three-Step Software Solutions Analysis.

Discuss the feasibility of the pilot program requirement to release at least 20 percent of new custom-developed code as OSS.

We conducted this performance audit from August 2018 to September 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Eric Winter (Assistant Director), John Ortiz (Analyst-in-Charge), Rebecca Eyler, Franklin Jackson, and Kate Nielsen made key contributions to this report.