Congress has incrementally expanded the use and scope of “other transaction” authority since first authorizing its use more than a decade ago. In 1989, Congress gave DOD, acting through the Defense Advanced Research Projects Agency (DARPA), authority to temporarily use “other transactions” for basic, applied, and advanced research projects. In 1991, Congress made this authority permanent and extended it to the military services. In 1993, Congress enacted Section 845 of the National Defense Authorization Act for Fiscal Year 1994, which provided DARPA with authority to use, for a 3-year period, “other transactions” to carry out prototype projects directly relevant to weapons or weapon systems proposed to be acquired or developed by DOD. The National Defense Authorization Act for Fiscal Year 1997 temporarily extended DARPA’s Section 845 authority and provided similar authority to the military services and defense agencies. Congress subsequently extended this authority’s expiration date until September 30, 2004. In an era of a shrinking defense industrial base and new threats, DOD views “other transaction” prototype authority as a key to attracting nontraditional defense contractors. Section 803 of the Floyd D. Spence National Defense Authorization Act for Fiscal Year 2001 defined a nontraditional defense contractor as an entity that has not, for at least a period of one year prior to the date of entering into or performing an “other transaction,” entered into or performed (1) any contract subject to full coverage under the cost accounting standards or (2) any other contract in excess of $500,000 to carry out prototype projects or to perform basic, applied, or advanced research projects for federal agencies. DOD also views Section 845 authority as a way to test creative procurement strategies—such as the use of teaming and consortia—with traditional defense contractors and in industry areas not normally associated with government contracts. Under this authority, new business relationships, which could involve changes in traditional business processes or intellectual property rights agreements, are created to leverage commercial investments and to permit DOD to influence the design, development, and availability of commercial technologies to address national security needs. In fiscal year 2001, the most recent year for which complete data are available, DOD awarded 61 Section 845 agreements, totaling $392 million in federal government funds. Contractors contributed another $97 million in cost-sharing funds. Figure 1 shows these agreements by awarding organization. DOD is required to submit an annual report to Congress addressing both research and prototype “other transaction” agreements awarded in the preceding fiscal year. The report, which is prepared and signed by the Director, Defense Research and Engineering, includes input from the Director of Defense Procurement on Section 845 agreements. The report is to address (1) the technology areas in which the work was focused; (2) the extent of cost sharing among federal and nonfederal sources; and (3) how “other transactions” contributed to a broadening of the technology and industrial base and fostered new relationships and practices that support U.S. national security interests. In December 2000, the Under Secretary of Defense for Acquisition and Technology issued a revised guide that sets out the conditions and framework for using Section 845 agreements. The guide is effective for all solicitations issued after January 5, 2001, and provides a useful framework for tailoring the terms and conditions appropriate for each agreement. DOD agreements officers view the new guide as a significant improvement over the prior version. Several key improvements are as follows: The previous guide contained very limited information on the terms and conditions to be tailored when crafting a Section 845 agreement. The current guide provides additional details on the appropriate use of terms and conditions such as intellectual property, accounting systems, and cost sharing. It instructs agreements officers not to view previously issued agreements as a template or model, but to rely on their skill and experience and to consider Federal Acquisition Regulation clauses and commercial business practices, as well as prior “other transactions,” when formulating agreements. The current guide also requires an acquisition strategy that identifies and discusses the rationale for using a Section 845 agreement. The previous guide did not define “nontraditional” contractors. The current guide defines the term, based in part on the definition in Section 803 of the National Defense Authorization Act for Fiscal Year 2001. It also requires that information on these entities be collected. DOD considers nontraditional defense contractors to be “business units,” which can be any segment of an organization or an entire business organization that is not divided into segments. The previous guide listed eight examples of benefits to be considered under Section 845 agreements, including attracting business entities that normally do not do business with the government. However, it did not identify a specific metric that should be used on all Section 845 agreements. The current guide clearly states that DOD will track, as a metric, the participation of nontraditional defense contractors. DOD also included a draft audit policy in the revised guide. According to DOD officials, the impetus for including a draft audit policy came from two DOD Inspector General reports. The first, a 1997 report, questioned the adequacy of audit coverage on DARPA’s “other transactions” for research. Although “other transactions” agreements for research included an audit clause, the report noted that agency officials intended to require audits only if they suspected fraud. The Inspector General argued that without final cost audits, agency officials could not ensure compliance with the statutory requirement pertaining to cost-sharing provisions. In a 1999 follow-up study on cost-sharing, the Inspector General raised similar concerns about prototype projects and included recommendations regarding audit policy for “other transactions” for prototype projects. On August 27, 2002, DOD issued a final rule codifying the definition of a nontraditional defense contractor and setting forth the conditions for using Section 845 agreements consistent with Section 803 of the National Defense Authorization Act for Fiscal Year 2001. The notice accompanying the final rule stated that the audit policy is being discussed and will be addressed by a separate rule. After exploring a number of performance indicators for Section 845 agreements, DOD selected one quantitative performance metric—the extent of nontraditional contractor participation—which is tracked by the Office of Defense Procurement. Officials believe that this metric is key because involving firms that do not traditionally do business with DOD increases DOD’s opportunity to leverage commercial technology investments and to take advantage of commercial business processes, such as using an integrated team approach rather than a traditional prime- subcontractor structure. Congress also has encouraged the participation of commercial firms in the development of defense systems and has recognized the critical contributions of nontraditional participants in areas such as biotechnology and pharmaceuticals in today’s national security environment. DOD contracted with RAND, a nonprofit institution, for a study to assess the overall effectiveness of the Section 845 acquisition approach and to explore the possibility of using additional metrics. In addition to this effort, a DOD working group, composed of officials from across DOD, considered the types of metrics that could be used to assess the effectiveness of Section 845 agreements. These two efforts identified several difficulties, as follows: Traditional metrics—such as cost growth, schedule slips, and performance shortfalls—are inappropriate for Section 845 projects that are inherently risky. A “path not taken” cannot be measured; that is, when a Section 845 agreement is used rather than a procurement contract, a statistical comparison between the two acquisition approaches cannot be made. Too many variables and too few Section 845 agreements would limit the results of a quantitative analysis. Few Section 845 projects have been completed, limiting the results to date. RAND concluded that important new technological capabilities—a desirable benefit of “other transaction” agreements—mostly come from segments of major firms that formerly focused on commercial projects but are now willing to apply their skills to the development of military prototypes. RAND also pointed out that there are other benefits associated with the flexibility inherent in this authority. For example, the flexibility to change project plans based on mutual agreement between DOD and industry managers, with minimal documentation or administrative burden, provides more powerful opportunities to cope with the problems and opportunities that occur when developing new systems and components. However, RAND emphasized the difficulties in developing quantifiable metrics that would be accepted as credible. In its effort to focus on collecting information on nontraditional contractors, DOD uses the Report of Other Transactions for Prototype Projects (DD Form 2759), which is completed by the agreements officer. (App. II contains a sample form.) According to the DOD guide, when funding actions are taken, the agreements officer must record information on whether the prime or subcontractor awardees are traditional contractors, nontraditional defense contractors, or non-profit organizations. The agreements officer also must record the names and addresses of significant nontraditional defense contractors. The summary information is sent to DOD’s Office of Defense Procurement, where it is aggregated. According to the DD 2759 reports for Section 845 agreements awarded in fiscal year 2001, 16 nontraditional prime contractors and 29 significant nontraditional subcontractors participated in a total of 61 agreements. Nontraditional participants included commercial business units of U.S. traditional firms as well as foreign corporations. Congress requires DOD to report annually on all “other transaction” projects—for research as well as prototypes—awarded in the preceding fiscal year. While the Section 845 portion of the report addresses the issues set forth in the congressional reporting requirement, it does not present the number of nontraditional contractors in a clear, straightforward format, such as a summary table. Because information on nontraditional participants—DOD’s key performance metric—is not summarized, it is difficult for Congress to assess how successful DOD has been in achieving this metric. The annual report includes a 1- or 2-page summary of each project that discusses (1) government and contractor cost contributions, (2) the reason for using the Section 845 authority, and (3) how the agreement contributed to a broadening of the technology base or fostered relationships and practices that support U.S. national security interests. In the fiscal year 2001 report, these individual summaries totaled 152 pages. In a fiscal year 2000 supplemental report to Congress, DOD did present a narrative summary of the number of nontraditional contractors; however, this was the only occasion when the information was clearly imparted. DOD officials stated that they are reluctant to add another reporting element and that the current report format meets congressional requirements. They added that they view the number of nontraditional contractors as secondary to the agreement-level information presented in the report. DOD also is not regularly reporting on or assessing the benefits derived from completed Section 845 agreements. In 1996, the Under Secretary of Defense for Acquisition and Technology requested a comparison of the benefits and drawbacks of completed agreements with the expected benefits at the time of award. However, this attempt to compile “lessons learned” was abandoned because many DOD officials believed that the results were parochial and not useful across the department. A draft version of the current Section 845 guide included a requirement for an assessment of completed agreements, but the requirement was not incorporated in the final version because DOD officials believe that another reporting requirement was not likely to produce a meaningful assessment of Section 845 results. DOD officials commented that the law only requires them to report on projects awarded in the previous fiscal year. They acknowledged, however, that periodic assessments of the benefits derived from completed agreements could be useful. By updating the Section 845 guide and requiring the number of nontraditional contractors to be measured as a performance metric, DOD has implemented our April 2000 recommendations. However, the reporting on the benefits derived from this alternative acquisition approach could be improved. A summary table in the annual report to Congress, setting forth the number of nontraditional contractors brought in under Section 845 agreements during the preceding year, would provide a clear picture of the extent to which DOD’s performance metric is being achieved. The current report format, consisting of summaries of each agreement, requires the reader to review each summary sheet in order to determine how the Section 845 authority was used—including the number of nontraditional contractors participating in the agreement. Thus, its usefulness to Congress is limited. Further, in the absence of regular assessments of the benefits derived from completed projects, DOD and the Congress lack vital information on the results the government is deriving from this flexible procurement strategy. The experience that DOD has gained from the use of Section 845 authority can be useful to Congress as it makes decisions about subsequent extensions of this authority to DOD and in future congressional deliberations. We recommend that the Secretary of Defense incorporate in the annual report to Congress summary information on the extent of nontraditional contractor participation and periodically report to Congress the results of studies on the benefits derived from completed Section 845 projects, including how key private sector participants contributed to the results. In written comments on a draft of this report, DOD agreed to incorporate in the summary of the annual report to Congress information on the number of new agreements and to break out the reasons for using the authority. However, DOD did not agree with our recommendation to include the number of nontraditional contractors in the annual report, stating that a raw count does not necessarily indicate the extent of nontraditional contractor participation and that it is secondary information derived from a separate reporting system. We agree that a raw number alone can be misleading. However, we do not understand why DOD is reluctant to publish the total number of nontraditional contractors—along with the other information to be reported—when those numbers are being internally collected and when this is the key performance metric DOD has established. Including the number of nontraditional contractors, along with the other information DOD has agreed to provide, would give Congress a more complete basis on which to assess the achievements gained through the use of Section 845 authority. DOD concurred with our second recommendation but stated that it would oppose the establishment of a regular reporting requirement. We are not advocating a new reporting requirement; however, we believe that periodic assessments of completed Section 845 projects would provide Congress useful information on the benefits the department is deriving from use of this authority. To assess the comprehensiveness of DOD’s new Section 845 guide, we compared it to the November 1998 guide that was in effect during our prior review. To determine the adequacy and usefulness of the revised guide and the performance metrics used, we interviewed officials in the Office of Defense Procurement and in the Office of Acquisition Initiatives—Office of the Under Secretary of Defense for Acquisition, Technology and Logistics; Washington Headquarters Services’ Directorate for Information Operations and Reports; the headquarters offices of the Army, Navy, and Air Force; DARPA; and NIMA. We also reviewed reports prepared by DOD’s Office of the Inspector General, RAND, and GAO. In addition, we reviewed various directives, memorandums, publications, correspondence, and legislation concerning Section 845 authority. To determine the number and value of fiscal year 2001 Section 845 agreements and the number of agreements having nontraditional defense contractors, we analyzed data compiled by the Office of Defense Procurement. In addition, at each military service and DARPA, we reviewed the Reports of Other Transactions for Prototype Projects (DD Form 2759) for all agreements awarded in fiscal year 2001. We did not validate or verify the information reported on these forms, including whether the cited nontraditional defense contractors met the definition in Section 803 of the National Defense Authorization Act for Fiscal Year 2001. To determine whether Congress is receiving adequate information on the number of nontraditional defense contractors participating in Section 845 agreements and whether DOD is assessing the benefits derived from completed projects, we reviewed the Section 845 portion of the annual reports for fiscal years 1999 through 2001 and the supplemental reports provided to Congress in fiscal years 1999 and 2000. We also reviewed DOD’s guidance and memorandums and held discussions with officials from the Office of Defense Procurement. We conducted our review between April and August 2002 in accordance with generally accepted government auditing standards. We are sending copies of this report to interested congressional committees; the Secretaries of Defense, the Army, the Navy, and the Air Force; the Director, DARPA; the Director, NIMA; and the Director, Office of Management and Budget. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. Please contact me at (202) 512-4841 or Michele Mackin at (202) 512-4309 if you have any questions regarding this report. Other major contributors to this report were William M. McPhail, Rosa M. Johnson, and Kenneth E. Patton.

Summary: In April 2000, GAO reported on the Department of Defense's (DOD) use of Section 845 agreements, also referred to as "other transactions" for prototype projects. These are transactions other than contracts, grants, or cooperative agreements that generally are not subject to federal laws and regulations applicable to procurement contracts. In December 2000, DOD revised its Section 845 guide. The guide specifies when Section 845 agreements may be used and provides criteria for tailoring terms and conditions for each agreement. Officials from the military services and defense agencies have found the new guide useful and a significant improvement over the prior version. The Secretary of Defense has required a metric--the number of participating nontraditional defense contractors--which is measurable and directly related to each agreement. This metric is tracked and reported internally. DOD explored additional metrics, but concluded that the number of nontraditional contractors was the only one that was quantifiable and tied directly to Section 845 outcomes. DOD's annual report to Congress on Section 845 agreements consists of summaries on each agreement. However, the key metric--the number of nontraditional contractors--is not clearly presented in these reports, making it difficult to gauge DOD's progress in achieving success on this objective. Further, DOD is not regularly assessing reporting on the benefits derived from completed Section 845 projects. In the absence of such assessments, congressional and DOD decision makers lack a vital piece of information that would help them determine whether this flexible procurement authority is achieving expecting results.