prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: The Department of Defense (DOD) obligates more than $300 billion annually to pay for goods and services (including research and development)  Most of these acquisitions are governed by numerous statutes and regulations found in Title 10 of the United States Code, the Federal Acquisition Regulation (FAR), and the Defense Federal Acquisition Regulation Supplement (DFARS)  DOD can also enter into certain transactions without triggering most of the standard acquisition statutes and regulations by using other transaction (OT) authorities  In recent years, Congress has expanded these authorities and DOD is increasingly using OTs for research, prototyping, and production  This report examines (1) how OTs work, (2) why they were established, (3) potential benefits and risks of using OTs, and (4) whether there are data available against which to measure their effectiveness  Appendix A provides a legislative history of DOD's other transaction authorities  On October 4, 1957, the Soviet Union triggered a space race with the United States when it successfully launched Sputnik I into orbit, becoming the first nation to send a man-made satellite into space  Congress, concerned that the United States was falling behind in space, held a series of hearings on an "emergency" effort to respond to the Soviet launch of Sputnik  At the same time, a bill was introduced in the Senate to create an agency with the means to quickly and efficiently develop a national space program  These efforts led to passage of the National Aeronautics and Space Act of 1958 (Space Act, P L  85-568) in July 1958, which established the National Aeronautics and Space Administration (NASA)  In an effort to give the new agency "the necessary freedom to carry on research, development, and exploration     to insure the full development of these peaceful and defense uses without unnecessary delay," the Space Act granted NASA broad authority to "enter into and perform such contracts, leases, cooperative agreements, or other transactions as may be necessary" to accomplish its mission of research and exploration (emphasis added)  Congress extended different variations of OT authorities to other select agencies, granting the authority to DOD in the FY1990 & 1991 National Defense Authorization Act (NDAA, P L  101-189 )  For an analysis of which non-DOD agencies can use OTs and how the authorities differ by agency, see Appendix B   Other transactions are legally binding contracts that are generally exempt from federal procurement laws and regulations such as the Competition in Contracting Act and the Federal Acquisition Regulation  In contrast, traditional procurement contracts must adhere to the procurement rules set forth in statute and regulation  Generally, DOD can use other transaction authorities for three purposes: 1  conduct research, 2  develop prototypes, or 3  contract for follow-on production of a successful prototype project  DOD's other transaction authorities are found in two sections of law: 10 U   S   C   2371 grants DOD the authority to use other transactions to carry out basic, applied, and advanced research projects  DOD regulations treat these projects as financial assistance instruments, not as contracts  10 U   S   C   2371b permits the use of other transactions to conduct prototype projects and follow-on production  OTs can only be used for prototypes if one of the following applies: at least one nontraditional defense contractor significantly participating in the project; all significant participants are small businesses or nontraditional defense contractors; at least one-third of the total cost of the prototype project is provided by nongovernment participants; or the senior procurement acquisition official provides in writing an explanation of the exceptional circumstances justifying an OT  Follow-on production can only be conducted when the underlying prototype OT was competitively awarded, and the prototype project was successfully completed  10 U   S   C   2373 , while generally not considered an o ther t ransaction authority , allows DOD to buy certain items and designs for experimental or test purposes without having to adhere to the procurement laws set forth in Chapter 137 of Title 10  OT authorities grant government officials the flexibility to include, amend, or exclude contract clauses and requirements that are mandatory in traditional procurements (e g , termination clauses, payments, audit requirements, intellectual property, and contract disputes)  OTs can be structured in numerous ways, including a direct relationship between a single government agency and a single provider; joint ventures; partnerships; multiple agencies joining together to fund an agreement encompassing multiple providers; or through a consortium  One common application of OTs is to forge an agreement with a consortium  A consortium is an organized group— it can consist of nonprofits, academic institutions, or contractors— focusing on a specific technology area  Generally, a lead entity coordinates and directs a consortium's activities  Consortia have consisted of a handful to as many as 1,000 members  Consortia can act to facilitate multiparty agreements whereby each member is akin to its own co-prime contractor with the government  In such a case, the government articulates the need or problem it is trying to solve, and the various members of the consortium can submit white papers for consideration  In this scenario, OTs can serve as an efficient way for all members to send unsolicited technology suggestions and solutions to solve a defined challenge  Consortia can also be used to develop an ecosystem of entities working together on a project, whereby members of a consortium pool resources and collaborate with DOD  A number of analysts argue that using consortia in this way gives the federal government a unique ability to leverage and pool the technological expertise and innovation of multiple entities in a particular sector, thereby strengthening and advancing a sector of the industrial base that may have defense applications  Seen like this, OTs can be considered a mechanism to promote defense technology and the defense industrial base, with the potential added benefit of advancing the domestic commercial technology base  Some analysts, however, have argued that many of today's consortia do not operate as collaborative organizations, but function more like managed multiple award task order contracts  These analysts argue that DOD should seek to foster more collaboration in consortia  Some analysts have argued that consortia reduce competition, since only members of the consortium under contract can participate in the project or submit white papers for consideration and funding  Others counter that even when an OT is signed with a single consortium, competition could be increased, since all members of the consortium are notified of the opportunity (expanding the pool of potential competitors aware of the opportunity), and the OT will foster internal competition among consortium members  OTs are not procurement contracts and thus are exempt from numerous procurement statutes and regulations, including the statutes in Chapter 137 of Title 10 ( Procurement Generally )  They are, however, bound by standard contract and other select laws and regulations  Examples of laws that do not apply include the Truth in Negotiations Act, Competition in Contracting Act, Cost Accounting Standards, Contract Disputes Act, and select intellectual property statutes such as the Bayh-Dole Act  A number of these laws are aimed at oversight and protecting the interests of taxpayers  While the Competition in Contracting Act does not apply, 10 U S C  2371b requires that research and prototype projects be competed "to the maximum extent practicable " DOD policy mirrors the statutory language, stating "[C]ompetition is a good thing  It helps keep prices low, quality high, and gives the government leverage in negotiations " To the extent that OTs induce nontraditional contractors to work with DOD, OTs can be viewed as promoting competition among (and within) entities that would not normally compete for DOD contracts  However, the lack of explicitly defined competition requirements could result in less competition for certain OTs  OTs are not free from all legislative and regulatory requirements  Generally, statutes and regulations that are not procurement-specific apply, including the Trade Secrets Act, (18 U S C  1905); the Economic Espionage Act (18 U S C  1831-39); elements of the Freedom of Information Act (5 U S C  552); and fiscal and property laws, such as the Anti-deficiency Act (31 U S C  1341); and the Tucker Act (28 U S C  1491)  As can be seen by these citations, many of these statutes are not located in the procurement titles of Title 10 or Title 41  The Government Accountability Office (GAO) has held that OTs are not procurement contracts, and therefore it will not review protests of such an award or solicitation  However, GAO will review "a timely protest that an agency is improperly using its other transaction authority " OT contracts can be protested to the Court of Federal Claims, although there is debate as to the extent of the court's jurisdiction  The limited protest jurisdiction of GAO is appealing to many government procurement officials  One senior Air Force official reportedly stated that OTs are "just so much faster and so much more attuned to getting something that we want today and not have to spend a couple of years going through a protest, going through this huge process to get something we wanted two years ago " It generally does not take that long to go through a GAO protest  By statute, GAO must issue an opinion on a protest within 100 days of the protest being filed, and 70% of cases are resolved in less than 60 days  While exempt from GAO bid protests, OTs are not exempt from GAO audits  Generally, OTs that include government payments exceeding $5 million are required to include a clause granting GAO the right to examine the records of any related party  However, this requirement has a number of limitations  Used properly, OTs can provide significant benefits to DOD  Along with the potential benefits come certain risks  Some of the potential benefits to OTs highlighted by analysts and officials include providing a mechanism to pool R&D resources with industry to facilitate development of, and obtain "the latest state-of-the-art, dual use technologies"; attracting nontraditional contractors with promising technological capabilities to work with DOD; lowering costs by eliminating requirements associated with the Federal Acquisition Regulations (i e , costs associated with required reporting and administrative activities) or sharing costs with industry; and "speeding up" the acquisition process  A number of experts argue that OTs provide a unique mechanism for DOD to invest in, and influence the direction of, technology development even when the end result is not directly tied to a military capability  As one observer noted, the real benefit to DOD may be that [t]he R&D has been accomplished and is available to the technical and scientific communities  As a result, a subsequent phase of research can begin or a particular approach can be demonstrated to be of no value  Some argue that OTs have particular import today  Drawing parallels to the space race, these analysts argue that DOD is engaged in a defense technology race  According to the 809 Panel: DoD is now in a period during which the time a particular technology is a dominant force on the battlefield is getting increasingly shorter, disruptive technologies are emerging at a faster pace, and these technologies are more widely dispersed…In a world with rapidly changing technology, time is a valuable resource that must not be taken for granted  It is difficult to predict what capabilities DoD will need 5 to 10 years from now—biotechnology, nanotechnology, artificial intelligence, robotics, or a new technology area not even known today  It also is unclear on what plane the military will conduct warfare—traditional battlefields, space, cyberspace, or some other domain  The current acquisition system lacks the agility needed to adapt to new paradigms  These analysts argue that OTs and similar rapid acquisition authorities are critical for DOD to compete in such a fast-paced global environment where technology and innovation are no longer driven by DOD, but by industry and foreign competitors  In 1960, the United States accounted for 69% of global R&D, with U S  defense-related R&D alone accounting for more than one-third of global R&D  The federal government funded approximately twice as much R&D as U S  business  However, from 1960 to 2016, the U S  share of global R&D fell to 28%, and the federal government's share of total U S  R&D fell from 65% to 24%, while business's share more than doubled from 33% to 67%  As a result of these global, national, and federal trends, federal defense R&D's share of total global R&D fell to 3 7% in 2016  Given the shift in the global R&D landscape, and the diminishing influence of DOD as a market mover, analysts suggest that the current procurement system is overburdened by regulations and bureaucratic processes that slow the system, increase costs, and dissuade companies from doing business with DOD  In contrast, OTs are viewed as faster, attracting companies that would otherwise forgo working with DOD and promoting broader investment in critical defense capabilities  As one analyst wrote: OTAs are currently the only way to remove the barriers necessary to get these nontraditional sources of innovation to do business with the military  Properly constructed, OTAs help speed up the process, respect a company's IP through negotiation rather than regulatory fiat, and result in contracting under commercial terms and conditions  Congress also appears to have shifted its view on appropriate use of other transactions  In the FY1999 conference report, Congress stated that [OT] authority should only be used in the exceptional cases where it can be clearly demonstrated that a normal contract or grant will not allow sufficient access to affordable technologies  By comparison, in the FY2018 NDAA, Congress expanded OT authorities and stated the following: In the execution of science and technology and prototyping programs, the Secretary of Defense shall establish a preference, to be applied in circumstances determined appropriate by the Secretary, for using transactions other than contracts, cooperative agreements, and grants  Along with the potential benefits come potential risks, including that of diminished oversight and exemption from laws and regulations designed to protect government and taxpayer interests  Some analysts, while acknowledging the important role of OTs, raise concerns over transparency and how these agreements are being employed  As one industry official stated, OTs are a contracting method, not a substitute for good acquisition practices  Discussing a particular OT for cloud services that was protested and ultimately cancelled by DOD, one observer argued The cloud contract provides a teachable moment for procurement reform-minded officials in the Pentagon and Capitol Hill  The problem was not with the OTA mechanism, which remains an essential element of reforming Pentagon procurement  Rather, the problem was with a lack of transparency with how the mechanism was employed  Scott Amey, general counsel of the Project on Government Oversight, cautioned We have to seriously consider how we are using [OTs]; whether we are using them as intended, whether we are getting the goods and services that we really want and need, whether we are getting them at the best cost and process, and we are using this procurement vehicle as a way to just circumvent the rules and have contractors not have the administration and oversight they need to hold them accountable  I'm just afraid this is going to result in a lot of waste, fraud, and abuse in the future  Congress has expressed repeated concerns that OTs could be used to circumvent congressional intent  In the FY1999 NDAA, the committees emphasized that the authority should only be used in a limited manner  The conference report stated the following: The conferees are especially concerned that such authority not be used to circumvent the appropriate management controls in the standard acquisition and budgeting process  Congress echoed a similar concern in the FY2019 NDAA  According to the House report: The committee also urges the Department to reiterate through established guidelines that OTA is not a means for circumventing appropriate use of the Federal Acquisition Regulations, and that full and open competition should be used to the maximum extent possible to maintain a sense of integrity, fairness, and credibility in the Federal Procurement process  Other transactions are also exempt from many of the socioeconomic policies put in place by Congress to promote public policies, including some Buy America requirements  Some analysts have raised concerns that OTs are a way to circumvent many of the public policies enshrined in the acquisition process  A number of analysts and officials have raised concerns that if DOD uses OTs in ways not intended by Congress—or is perceived to abuse the authority—Congress could clamp down on the authority  Under Secretary of the Army Ryan McCarthy reportedly stated that the military department is "trying to be very judicious about this authority so we don't lose it " Some analysts argue that Congress is already clamping down on the use of OTs  These analysts point to language in the FY2019 NDAA and FY2019 appropriations legislation ( P L  115-245 ) requiring additional reporting and notification (see Appendix A )  Such notification and reporting requirements, however, are not new; when Congress expanded OT authorities in the past, reporting and notification requirements were commonly included  The reporting requirements may also be a result of congressional frustration with a lack of transparency and data on how DOD uses OTs  DOD lacks authoritative data that can be used to assess OT effectiveness and better understand broader trends associated with these agreements  The most frequently cited source for such data is the Federal Procurement Data System-Next Generation (FPDS-NG), which is the primary source for tracking data on contract obligations, including other transactions for prototypes and follow-on production  FPDS-NG is configured to track data on cost-sharing, other transaction award type, and prevalence of nontraditional contractors  Obligations connected to OTs for research are tracked by the Defense Assistance Awards Data System, which is primarily used to track grants and cooperative agreements  This bifurcation of how OT data are tracked makes it more difficult to get a consolidated view of OT data  According to DOD, all OT data will be reported through FPDS-NG starting in late 2019  The procurement data in FPDS-NG are not fully reliable  There are quality issues relating to accuracy, completeness, and timeliness of data  CRS reviewed FPDS-NG data for prototype OT agreements signed or modified between FY2015 and FY2017 and found similar data inconsistencies  DOD officials acknowledge that they do not have sufficiently reliable data upon which to conduct analysis on the use of OTs and are taking steps to try to improve the data  The analyses below reflect CRS's effort to analyze DOD's use of other transaction authority based on the best available data  According to FPDS-NG, in FY2017, DOD obligated $2 1 billion—and received $360 million in cost-share contributions—on prototype other transaction agreements, representing less than 1% of DOD's total FY2017 contract obligations (approximately $320 billion)  Despite the small percentage of obligations, OTs are growing quickly and are expected to continue to grow at a rapid pace  From FY2013 to FY2017, the number of new prototype agreements increased from 12 to 94, an increase of over 650% (see Table 1 )  DOD's Defense Innovation Unit (DIU, formerly known as the Defense Innovation Unit Experimental, or DIUx) was involved with approximately half of the prototype agreements executed in 2017  DOD officials have stated their intent to further increase the department's use of OTs  Officials say that this increase is due to Congress expanding the statutory authority  The Army executed more than 66% of the prototype OT agreements between FY2013 and FY2017, often on behalf of other military departments and components (see Table 2 )  Army Contracting Command-New Jersey at Picatinny Arsenal executes many of these agreements  DIU currently uses Picatinny Arsenal to execute all of its other transaction agreements  A number of private sector companies do not pursue federal government contracts because they are unwilling to forfeit intellectual property rights or adhere to some of the procurement regulations  One of the goals of OTs is to expand the defense marketplace by creating a mechanism for access to technologies and services of companies that would not otherwise work with DOD, particularly startups and companies developing innovative technology  As one industry representative stated: Because they are "outside" the FAR, OT agreements do not require such cumbersome oversight and audit requirements such as those imposed by the Truth in Negotiations Act, cost and pricing data or an expensive Cost Accounting System (CAS) qualified financial system  CAS compliant financial systems can cost a company millions of dollars to implement and maintain — and are therefore a significant, if not potentially fatal, barrier to government market entry for startups and small, innovative companies  These requirements tend to reinforce the "legacy advantage" of large traditional contractors, who can afford to hire and staff these requirements with large staffs of accountants and lawyers  A number of nontraditional companies told CRS that they are more likely to work with DOD because of the department's other transaction authorities  Despite these claims, some observers question whether OTs are effectively bringing nontraditional contractors into the defense marketplace  A DOD Inspector General report examining other transactions from FY1994 to FY2001 found that OTs did not attract significant numbers of nontraditional defense contractors to do business with DOD  The report found that of the 209 prototype agreements examined, traditional defense contractors received 95% of the $5 7 billion in funds awarded  A recent analysis of FPDS-NG data by Federal News Network had similar findings  According to the report, from FY2015 to FY2017, while nontraditional defense contractors were awarded most of the new OTs (66% vs  33% for traditional defense contractors), the dollar value of the OTs favored traditional contractors ($20 8 billion vs  $7 4 billion for nontraditional contractors)  Some observers have questioned the accuracy of the data published by Federal News Network  According to Charlie McBride, president of Consortium Management Group (which manages two consortia working with DOD through OTs), 88% of the total dollar value of awards to CMG Group has gone to nontraditional prime contractors, and nontraditional entities have participated in the remaining 12%  This debate highlights the lack of authoritative data on OTs  The currently available data may not accurately reflect the extent to which nontraditional contractors are engaged in OT agreements  FPDS-NG does not collect data regarding subcontractors or consortia composition, making it difficult to determine the nature and extent to which nontraditional defense contractors and entities may be working under OT agreements with DOD directly or as subcontractors  When Congress extended OT authority to DOD, it authorized inserting a clause requiring a person or entity to make payments to DOD as a condition of receiving support under the agreement  Such funds were to be merged into an account dedicated to support DARPA advanced research projects  The intent of this provision was to permit DARPA to "recoup the fruits of such arrangements, when there is a 'dual use' potential for commercial application" and reinvest the funds to develop other technologies  In addition to the recoupment authority, DOD can share costs with other parties under an OT  Using this approach, the amount of each party's share is negotiated and incorporated into the agreement  Congress believed that OTs and cooperative agreements were ideal vehicles for promoting DOD-industry collaboration in developing dual-use technologies  For example, the FY1992 & 1993 NDAA ( P L  102-190 ) authorized DOD to enter into cooperative and other transaction agreements to develop critical dual-use technologies as set forth in the Defense Critical Technologies Plan  According to the Senate report:     the United States tends to underinvest in dual-use technologies  National security requirements alone often do not justify major DOD support, and market prospects alone often appear to be too long-term or high risk to justify US industry carrying the entire development burden     The committee encourages use of cooperative agreements and other transactions in lieu of grants or contracts     The provision would require that at least 50 percent of funding over the life of a partnership derive from non-federal sources but would allow for a smaller industry share at the start  In the 1990s, some DARPA OTs required participants to share costs because the types of work completed generally involved R&D that was mutually beneficial to government needs and industry commercial goals  In certain instances, present-day OTs have cost-sharing requirements that foster collaborative research  Some analysts believe that DOD is not always realizing all the benefits that OTs have to offer, such as sufficiently leveraging private capital or forming true consortia of multiple parties pooling resources  Some of these analysts believe that DOD does not sufficiently use consortia to leverage private investment through the pursuit of collaborative, mutually beneficial, dual-use technologies  Some observers argue that as the legislation on OTs has evolved, the cost-sharing provision for prototype projects has come to create an unfair playing field biased against traditional defense contractors  For prototype projects, traditional contractors generally are required to assume one-third of costs whereas nontraditional defense contractors and small businesses generally do not have to cost share  From a fairness perspective, these observers argue that traditional contractors should not have a mandatory cost share  These observers also point out that some nontraditional defense contractors are companies with billions of dollars of revenue that should not be granted a competitive cost advantage  Putting traditional defense contractors at a competitive disadvantage could deny DOD access to those companies with the most experience working on defense products, potentially depriving the military of access to leading defense-related research and technology  Other observers argue that the cost share as currently structured is appropriate: traditional defense contractors hold a significant competitive edge in their understanding of, and have the systems in place to manage, traditional contracts  In contrast, nontraditional and small businesses, which generally cannot compete with the traditional defense contractors, need the exemption from the cost-share requirement to be able to work with DOD  These observers also argue that traditional contractors are awarded the majority of dollars obligated to OTs, proving how difficult it is for nontraditional suppliers to break into the defense marketplace  Additionally, traditional contractors could avoid the cost-sharing requirement by teaming with a nontraditional contractor  DOD has not effectively tracked data on cost sharing  A 2017 report to Congress indicated that in FY2016 DOD obligated $1 billion for prototype agreements and received $68 million in cost-share contributions  A CRS review found numerous concerns with the data underlying the report, and with DOD's analytical conclusions  See Appendix C for further discussion  A number of analysts and industry officials have raised concerns that DOD could use OTs to avoid competitions, as OTs are exempt from the Competition in Contracting Act  According to statute, follow-on production using other transaction authority can only be awarded if the underlying R&D agreement was competed  In addition, 10 U S C  2371b states that "to the maximum extent practicable" OTs must be competed  Determining whether OTs are being used to circumvent competition requires a two-step analysis: 1  Are OTs competed less often that traditional contracts? 2  Is there a benefit to these OTs being competed? Available FPDS-NG data suggest that DOD is broadly complying with 10 U S C  2371b's competition mandate  Between FY2013 and FY2017, approximately 89% of all new OT prototype agreements were competed in some fashion  Many observers and analysts believe that OT agreements can be executed substantially faster, sometimes in a matter of weeks, compared to the months or years it typically takes to execute traditional contracts  Based in part on this belief, some officials and analysts are touting OTs as a new model for conducting acquisitions and the answer to many of the problems in defense acquisition  These analysts argue that in a world of increasingly fast technology development, DOD acquisitions must go faster or risk being left behind  Many acquisition professionals argue that OT contracts are not inherently faster than traditional contracting; instead, they are executed faster because they are not encumbered by the reviews, protests, and bureaucratic layers that have been overlaid on traditional contracting  According to these officials, OTs take just as long as traditional contracts if the same execution and oversight processes are applied  And because all terms are negotiable, complex negotiations could make OTs take longer to execute than traditional contracts that have required, nonnegotiable conditions  The Other Transactions Guide states The OT award process will not always be faster than the traditional procurement processes and sometimes can be as long or longer  The speed of award is tied to many factors, many of which are internal to the organization  DOD has not tracked data on the relative time it takes to execute OTs vs  traditional contracts, making it impossible to objectively assess these claims  Analysts and officials generally agree that the workforce plays a critical role in determining the success or failure of an acquisition  Because there are fewer predefined requirements, OTs can be more difficult to negotiate than traditional contracts, putting DOD at greater risk of not getting what it wants at a reasonable price  The complexity and difficulty of negotiations is particularly high when there are intellectual property/patent rights issues, as is the case with most OTs  Given these challenges, OTs often require more experienced and capable government representatives to ensure implementation of agreements that are in the government's best interest  Some analysts question the extent to which the workforce is sufficiently trained and equipped to negotiate OTs  In response to this concern, in the FY2018 NDAA, Congress required workforce education and training for OTs, and required DOD to establish a cadre of intellectual property experts to advise, assist, and provide resources to program offices that are developing intellectual property strategies for contracts and agreements (see Appendix A )  DOD officials acknowledge that more training and education is required  Given the complexity of OTs and the limited extent to which they are used, some analysts and industry officials suggested that there may be a benefit to establishing a centralized office within DOD responsible for executing or overseeing all other transaction agreements  Such a structure could help ensure that those members of the acquisition workforce engaged in other transactions are sufficiently experienced, trained, and qualified  A number of analysts have argued that DOD should take steps to improve its use of OTs  Many of these analysts have suggested that data are not consistently and accurately tracked, regulations and guidance on when and how to use OTs are vague or insufficient, and the workforce is not sufficiently prepared to effectively use OTs  A number of officials have acknowledged these shortcomings and DOD is reportedly taking steps to address them  For example, in December 2018, DOD issued an updated Other Transactions Guide , a comprehensive guide containing best practices, case studies, and a clarification of myths related to other transaction authorities  In addition, Defense Acquisition University developed new course materials addressing OTs and is working to expand its offerings of relevant training and classes  However, numerous acquisition officials question whether it is possible, or even desirable, to try to quickly implement training aimed at preparing the thousands of DOD acquisition officials to execute OTs  Some of these officials have suggested it might be appropriate to only allow a limited and vetted number of acquisition professionals to be OT agreements officers  Some argue that OTs are just one "tool in the tool box," appropriate for only specific types of contracts, and should not be used to avoid the statutory and regulatory framework or to try to accelerate the process just for the sake of speed  Others have suggested that OTs should be used to cut through bureaucracy, speed up the acquisition process, avoid regulations and bid protests, and perhaps eventually supplant the regular FAR-based contracting process  Given the benefits and risks associated with OTs, questions for Congress include the following: 1  To what extent and in what circumstances do the potential benefits of OTs in terms of cost, schedule, and added capabilities outweigh concerns over potential fraud, waste, abuse, diminished oversight, and other public policy objectives? 2  Should OT authorities be extended further, curtailed, or maintained? The FY2019 NDAA required DOD to submit a report annually through 2021, summarizing DOD's use of OTs, including organizations involved; number of transactions; amounts of payments; and purpose, description, and status of projects  The NDAA also required the Defense Innovation Unit to submit a report to Congress, to include the number of traditional and nontraditional defense contractors with DOD contracts or other transactions resulting directly from the unit's initiatives  The conference report for the FY2019 defense appropriations bill included language expressing the conferees' "[concern] with the lack of transparency surrounding the employment of OTA, particularly for follow-on production " The conferees directed DOD to provide quarterly reports to the House and Senate appropriations committees listing each active OT, and to include additional information for each agreement  The conferees also directed GAO to review DOD's use of OTs to determine whether the "employment of this authority conforms to applicable statutes and guidelines, to include the identification of any potential conflicts " GAO was also required to report on the extent to which OTs have been used since FY2016  The multitude of reporting requirements, and the questionable reliability of the available data, raise a number of questions that Congress may wish to explore, such as the following: 1  To what extent, if any, should the current reporting requirements be consolidated to create a more streamlined and consistent flow of information to Congress? 2  What specific data does Congress need in these reports to effectively conduct oversight? For example, what percentage of research OTs result in prototype projects and follow-on production? 3  To what extent are the data sufficiently reliable, and will such data be easily retrievable in the future, to allow Congress to conduct effective, timely, and ongoing oversight? If the data are not sufficiently reliable or accessible in the future, what other data collection and tracking methods could Congress mandate to ensure ongoing access to reliable data? 4  How are OTs being used? Where and when in the acquisition lifecycle is the authority being used? To what extent is the requirements process being circumvented when DOD awards an OT follow-on production contract for a major system? One analyst suggested that "no efficiency is lost if only the most able personnel are authorized to procure and administer" OT agreements and further argued that expanding the use of OTs would increase the training costs by expanding the number of people who can "weave complex agreements in a relatively unstructured environment " Congress may consider whether DOD should establish an acquisition innovation lab or center of excellence responsible for overseeing, executing, and approving all OTs across the department  Such a lab or center could be staffed and supported by a cadre of professionals with experience across the acquisition lifecycle who have a willingness and ability to embrace new ideas and rethink existing practices  Alternatively, such labs or centers could be established in the military departments  Having centers in each organization could allow for consideration of the different missions and business approaches of the departments and help educate the workforce on a more systematic basis  Proponents argue that such an office would help ensure that only experienced and capable officials, with the appropriate training, use OT authorities  Such an office could also help protect against layering internal DOD policies and bureaucracies onto OTs by placing OTs outside of the traditional bureaucratic acquisition process  Proponents could further argue that such an office could better propagate best practices and ensure that OTs are used appropriately and are consistent with guidance and legislation  Having a single office responsible for executing or approving all OTs could also help ensure more timely and accurate information, giving Congress more visibility into DOD's use of other transaction authorities  To the extent that a single, high-level official is responsible for managing and overseeing all OTs, Congress might wish to consider repealing or modifying existing statutory approval requirements  If such an office was able to provide timely and accurate information, Congress might also consider some of the current reporting requirements unnecessary, and may choose to repeal some of the reporting requirements  Opponents of such a proposal argue that centralizing OTs would have the opposite effect, increasing bureaucracy by adding yet another office within DOD  Opponents also argue that such an office could make it more time-consuming to get a project underway and may discourage program offices from attempting or suggesting OTs  Some also argue that a single office may not have the resources to execute and approve agreements in a timely manner, and would inhibit spreading expertise on how to execute OT agreements more broadly across the acquisition workforce  Even proponents who might in theory support establishing such an office could raise significant concerns regarding how such an office would function in practice  A number of alternative options could be pursued to address concerns raised by opponents of establishing a centralized office  Some of these alternative options include the following: Granting such an office primary, but not exclusive, authority to execute OTs  For example, agreement officers specifically authorized to do so could execute OTs, with the centralized office conducting a peer review  Under this construct, the office could also be charged with providing information and expertise/consulting services on the use of OTs to program offices contemplating using the authorities  Establishing centers within each military department, with a designated office in OSD serving a coordinating function (with nondelegable approval authority residing in the military department office designated for OTs)  Creating the office as a pilot program for three years, to help DOD manage OTs until such time as the workforce becomes more experienced and proficient in using these agreements  Appendix A  Legislative History Other transaction authority first appeared in the National Aeronautics and Space Act of 1958  Since then, Congress has extended OT authorities to 11 federal agencies and a number of other federal offices (see Appendix B for information on other federal entities with similar authorities)  This appendix traces the legislative history of OT authorities and select related statutes applicable to DOD  To read the full text of the three statutory provisions related to OTs (10 U S C  2371, 2371b, and 2373), see Appendix D   National Aeronautics and Space Act of 1958 (P L  85-568) Creation of Other Transaction Authority On July 29, 1958 President Dwight D  Eisenhower signed into law the National Aeronautics and Space Act (P L  85-568), which established the National Aeronautics and Space Administration (NASA)  The purpose of the act included the expansion of human knowledge, preservation of the role of the United States as a leader in space science and technology, and pursuing the most effective utilization of the scientific and engineering resources of the United States  Section 203(b)(5) of the Space Act provided NASA the authority (emphasis added) to enter into and perform such contracts, leases, cooperative agreements, or other transactions as may be necessary in the conduct of its work and on such terms as it may deem appropriate, with any instrumentality of the United States     or with any person, firm, association, corporation, or educational institution  To the maximum extent practicable and consistent with the accomplishments of the purpose of this Act, such contracts, leases, agreements, and other transactions shall be allocated by the Administrator in a manner which will enable small-business concerns to participate equitably and proportionately in the conduct of the work of the Administration  Intellectual Property Rights The Space Act specifically addressed NASA's "property rights in inventions " Section 305 stated that any invention made in the performance of any work under any contract is the exclusive property of the United States "unless the Administrator waives all or any part of the rights " This was true even when the person who created the invention "was not employed or assigned to perform research, development, or exploratory work, but the invention is nevertheless related to the contract" and was made during working hours, or with a contribution of the government  The act granted the Administrator wide latitude to "waive all or any part of the rights of the United States under this section" if doing so was deemed to be in the best interests of the United States  When such rights were waived, NASA retained an irrevocable, nonexclusive, nontransferable royalty-free license by or on behalf of the United States  National Defense Authorization Act for FY1990 & FY1991 ( P L  101-189 ) The FY1990 & FY1991 NDAA granted DARPA temporary authority to enter into "cooperative agreements and other transactions" for the purpose of conducting advanced research projects  The statute clarified that OTs should only be used when "the use of standard contracts or grants is not feasible or appropriate " Congress restricted funding for OTs and cooperative agreements to $25 million of appropriated funds for FY1990 and FY1991, and set the authority to expire on September 30, 1991  Cost Sharing The FY1990 & FY1991 NDAA permitted OTs (or cooperative agreements) to include a clause requiring a person or entity to make payments to DOD as a condition of receiving support under the agreement  Such funds were to be merged into an account dedicated to supporting DARPA advanced research projects using cooperative agreements and other transactions  The act required, to the extent practicable, that funds provided by the government not exceed the total amount provided by the other parties to the project  According to the Senate report, one of the intents of the cost-sharing provision was to permit DARPA to "recoup the fruits of such arrangements, when there is a 'dual use' potential for commercial application" and to reinvest the funds to develop other technologies  Reporting Requirements The FY1990 & FY1991 NDAA required DOD to submit an annual report on the use of OTs and cooperative agreements, to include a description of each agreement and the technologies involved, the potential military and commercial utility of the technology, the reasons a contract or grant was not feasible to support the research, and the amount of payments, if any, received by the federal government under the agreement  National Defense Authorization Act for FY1991 ( P L  101-510 ) Section 244 increased the funds authorized for cooperative agreements and OTs from $25 million to $50 million  However, no such funding was appropriated  Reporting and Notification Requirements The conference report required DOD to submit to Congress a report listing the cooperative agreements and consortia intended to be used in FY1991-1992  The conference report also required DOD to provide the armed services and appropriations committees 30 days' notice prior to DOD signing a cooperative agreement or agreement with a consortia under OT authority  The Senate report focused on consortia as a method to pool resources, share research among numerous participants, and promote critical dual-use technology  Department of Defense Appropriations Act, 1992 ( P L  102-172 ) Limitations on the Use of OTs Section 8113A of P L  102-172 placed temporary limitations on the use of agreements undertaken pursuant to 10 U S C  2371: Section 8113A limited the use of OTs and cooperative agreements exclusively to DARPA (to the exclusion of the rest of DOD) for FY1992  Section 8113A limited DARPA to obligating or expending no more than $37 5 million in FY1992 for cooperative agreements or OTs undertaken pursuant to 10 U S C  2371  Section 8113A further established that no more than $75 million could be obligated or expended by DARPA in FY1992 for DOD dual-use critical technology partnerships  National Defense Authorization Act for FY1992 & FY1993 ( P L  102-190 ) Expanded Authority Section 826 extended other transaction authority to the military departments, and established separate fund accounts in each department for cost sharing  Section 826 also repealed the sunset for cooperative agreements and OTs, making the authorities permanent  Section 821 authorized DOD to enter into cooperative and other transaction agreements to develop critical dual-use technologies as set forth in the Defense Critical Technologies Plan  According to the Senate report     the United States tends to underinvest in dual-use technologies  National security requirements alone often do not justify major DOD support, and market prospects alone often appear to be too long-term or high risk to justify US industry carrying the entire development burden     The committee encourages use of cooperative agreements and other transactions in lieu of grants or contracts     The provision would require that at least 50 percent of funding over the life of a partnership derive from non-federal sources but would allow for a smaller industry share at the start  The conference report stated that the partnerships should focus on programs that fit into the security needs within DARPA  The conference report also stated that OTs are appropriate for those cases where the "regulations applicable to the allocation of patent and data rights under the procurement statutes may not be appropriate to partnership arrangements in certain cases " National Defense Authorization Act for FY1993 ( P L  102-484 ) Cost Sharing Section 4221 established 10 U S C  2511, which required DOD to establish cooperative arrangements with industry, educational institutions, federal labs, and other entities, to pursue research, development, and application of dual-use technologies  The section authorized DOD to use grants, contracts, cooperative agreements, or OTs to create these partnerships, and that the Federal government should not contribute more than 50% of the costs related to projects under this authority  National Defense Authorization Act for FY1994 ( P L  103-160 ) Expanded Authority Section 827 established 10 U S C  2358, which gave the Secretary of Defense and the Secretaries of the military departments the authority to conduct basic, advanced, and applied research through the use of contracts, cooperative agreements, grants, and OTs  Previously, OTs were only authorized for advanced research  Prototype Authorities Section 845 granted DARPA the authority to use OTs for prototype projects directly related to weapons or weapon systems proposed to be acquired by DOD  Section 845 required that "to the maximum extent practicable," prototypes be competitively awarded  This authority was set to terminate after three years  Section 845 remained as a note to 10 U S C  2371 until separately codified as 10 U S C  2371b in the FY2016 NDAA ( P L  114-92 )  Federal Acquisition Streamlining Act of 1994 ( P L  103-355 ) Section 1301 redesignated the language in 10 U S C  2358 (granting the authority to use OTs) to 10 U S C  2371  Reporting Requirements Section 1301 also required DOD to submit an annual report to the armed services committees, to include a general description of the other transactions, including the technologies involved in the research, the potential military and, if any, commercial utility of such technologies, the reasons for not using a contract or grant to provide support for such research, and the amount of payments, if any, received during the fiscal year pursuant to a clause in the other transactions and to what accounts such payments were credited  National Defense Authorization Act for FY1997 ( P L  104-201 ) Section 203 required that a senior DOD official be designated in OSD, and that the officials' sole responsibility be developing policy related to, and ensuring implementation of, DOD's dual-use technology program  This section authorized DOD to use OTs (as well as contracts, cooperative agreements, and grants) for dual-use projects only if the project "is entered into through the use of competitive procedures " Section 743 granted DOD the authority to use OTs to conduct research on Gulf War Syndrome, to determine its relationship to possible exposures of members of the Armed Forces to chemical warfare agents and hazardous materials, and the use of inoculations and new drugs  Expanded Prototype Authorities Section 804 amended Section 845 of the FY1994 NDAA by extending to the military departments and officials designated by the Secretary of Defense, the authority to use OTs for certain prototype projects  This authority, originally granted solely to DARPA and set to expire after three years, was given a new termination date of September 30, 1999  Reporting Requirements Section 267 modified elements of the annual report to the armed services committees  National Defense Authorization Act for FY1998 ( P L  105-85 ) Section 832 amended 10 U S C  2371 by clarifying that certain information submitted to DOD (i e  a proposal, business plan, technical information) be protected from disclosure pursuant to 5 U S C  552 for a period of five years  Strom Thurmond National Defense Authorization Act for FY1999 ( P L  105-261 ) Section 241 extended the sunset day for the authority to use OTs for prototypes from September 30, 1999, to September 30, 2001  Section 817 amended Section 2371 of Title 10, United States Code, clarifying that information submitted by outside parties in cooperative agreements for basic, applied, and advanced research is protected from disclosure under Section 552 of Title 5, United States Code  Department of Defense Appropriations Act, 1999 ( P L  105-262 ) While the enacted FY1999 defense appropriations bill ( P L  105-262 ) did not include legislative language addressing OTs, H Rept  105-591 , which accompanied the House-reported version of H R  4103 , included language expressing the House Appropriations Committee's "serious reservations" regarding the Air Force's then-proposed use of an OT agreement—instead of a contract— to develop the Evolved Expendable Launch Vehicle (EELV) program  The committee noted that "under [OTs] traditional safeguards which protect the government's interest in large acquisition programs are largely absent," and required the Under Secretary of Defense for Acquisition, Technology, and Logistics (now the Under Secretary of Defense for Acquisition and Sustainment) and the DOD Inspector General to certify to the congressional defense committees that the use of an OT was appropriate for the EELV program, and that "adequate safeguards exist[ed] to protect the government's interest and monitor program performance " National Defense Authorization Act for FY2000 ( P L  106-65 ) Section 801 required that for prototypes using OT authorities, DOD ensure that GAO, under its audit authority, have access to records relating to other transaction prototype agreements exceeding $5 million  Section 801 allowed for a waiver to GAO access and exempted entities that over the last year have not entered into an agreement with DOD that provided for audit access by a government entity  According to the Senate report: Senior DOD officials have sought legislation to extend other transaction authority to production contracts  Under current authority, there is some debate about whether GAO has audit access to other transactions  As the size, costs, and complexity of programs being funded using other transactions increases, the committee wants to ensure that the GAO has audit access in relation to the higher levels of spending and added risks  Reporting Requirements The Senate report also addressed reporting requirements and congressional intent to review the use of OTs  The report stated the following: The committee is assessing the utility of other transaction prototype authority  The statement of managers accompanying the Strom Thurmond National Defense Authorization Act of 1999 directed the Secretary of Defense to report on the use of this authority to the congressional defense committees, no later than March 1, 1999  In addition, both the Department of Defense Inspector General and the General Accounting Office are reviewing the use of other transaction prototype authority and will report to Congress in the coming year  The committee is interested in the extent that new commercial firms are entering the DOD marketplace through the use of other transaction authority, as well as the degree of cost sharing between the government and non-federal government parties  The committee is also interested in any lessons learned from the broad exemptions to federal law provided by other transaction authority  For example, other transactions are exempt from the Competition in Contracting Act, Truth in Negotiations Act, Contract Disputes Act, Antikickback Act of 1986, Procurement Integrity Act, Service Contract Act, Buy American Act, and chapter 137 of title 10, United States Code  Questions have been raised about whether the government's interest is adequately protected in the absence of the applicability of these statutes  Conversely, advocates of the view that the government should take advantage of the flexibility of other transactions have championed proposals to extend other transaction authority to production  The committee directs the Secretary of Defense to provide a new report that updates information in the March 1999 report on the use of other transaction prototype authority to the congressional defense committees by February 1, 2000  National Defense Authorization Act for FY2001 ( P L  106-398 ) Limitations on the Use of OTs Section 803 limited the use of OTs for prototype projects to only those circumstances when at least one nontraditional defense contractor significantly participates in the project, one-third of the total cost of the project is paid out of funds provided by parties to the transaction other than the federal government, or the senior procurement executive determines in writing that exceptional circumstances justify use of an OT  Nontraditional defense contractor was defined as an entity that for a period of one year has not entered into or performed "any contract that is subject to full coverage under the cost accounting standards" or "any other contract in excess of $500,000 to carry out prototype projects or to perform basic, applied, or advanced research projects for a Federal agency, that is subject to the Federal Acquisition Regulation '' Section 803 also extended the authority to use OTs for prototypes from September 30, 2001, to September 30, 2004  According to the Senate report, the intent of using OTs for prototypes is to attract companies that typically do not do business with the Department of Defense and encourage cost sharing and experimentation in potentially more efficient ways of doing business with traditional defense contractors  Other transaction authority is an important acquisition tool that can facilitate the incorporation of commercial technology into military weapon systems  In an environment where, in many areas, commercial technology is now more advanced than defense technology, it is imperative that the Department continue to have the flexibility to use innovative contractual instruments that provide access to this technology  There are, however, improvements that can be made in managing and overseeing these contractual arrangements  Section 804 clarified the extent of GAO's access to records in instances where the party in question has only done business with the government in the preceding year through an OT or cooperative agreement  National Defense Authorization Act for FY2002 ( P L  107-107 ) Expanded Authority—Follow-on Production Section 822 of the FY2002 NDAA granted DOD the authority to award a follow-on production contract for prototype projects when at least one-third of the total cost of the prototype project is to be paid out of funds provided by non-federal government sources  Under this authority, such a follow-on contract could be awarded without competition if the prototype project was successfully completed, the number of units in the production contract does not exceed the number of units specified in the underlying prototype agreement, and the price for each unit does not exceed the price specified in the underlying transaction  Department of Defense and Emergency Supplemental Appropriations for Recovery from and Response to Terrorist Attacks on the United States Act, 2002 ( P L  107-117 ) Establishment of Army Venture Capital Initiative (AVCI) Section 8150 designated $25 million of the FY2002 funds made available for Army Research, Development, Test, and Evaluation (RDT&E) to be made available to the Secretary of the Army for the purpose of funding a venture capital investment corporation established pursuant to 10 U S C  2371  A 2014 RAND report stated that OT authorities were used only to form the AVCI, and were not used to acquire products or services: "While OT authorities were used to form [AVCI] itself, any volume of the Army's purchase of products and services from [AVCI] companies is conducted under the FAR " National Defense Authorization Act for FY2004 ( P L  108-136 ) Expanded Authority Section 847 of the FY2004 NDAA extended the authority to use an OT for developing prototypes to improve weapons or weapon systems currently in use by the Armed Forces  Previously, such authority was restricted to prototypes "directly relevant to weapons or weapon systems proposed to be acquired or developed" by DOD  Section 847 also established a pilot program for transitioning prototypes to follow-on contracts for production for nontraditional defense contractors  Under the pilot program, such a follow-on contract could be treated as a commercial item or an item developed with both federal and private sector funds (for purposes of negotiating intellectual property rights)  The pilot program was restricted to contracts with nontraditional defense contractors, where the value of the contract does not exceed $50 million (approximately $70 million in FY2018 dollars), and that are firm-fixed price or fixed price with economic adjustment  The pilot program was set to sunset September 30, 2008  Section 1441 authorized any agency that engages in basic, applied, or advanced research and development projects that facilitated defense against or recovery from terrorism or nuclear, biological, chemical, or radiological attack to exercise the same general authority given to DOD as found in 10 U S C  2371 (including for prototype projects)  Reporting Requirements Section 1031 sunset the annual reporting requirement for OT after the report covering FY2006 was submitted  National Defense Authorization Act for FY2006 ( P L  109-163 ) Restricted Authority and Notification Requirements Section 212 of the FY2006 NDAA directed the Army to procure the Future Combat System using contract procedures set forth in part 15 of the Federal Acquisition Regulation, in lieu of an OT  Section 823 extended ethics requirements to OT prototype authority and required the congressional defense committees be notified in writing at least 30 days before such authority is exercised  Certification Requirements Section 823 also amended Section 845 of the FY1994 NDAA, requiring a written determination by a senior procurement executive for other transaction prototype projects estimated between $20 million and $100 million, and for a written determination by the Under Secretary of Defense for Acquisition, Technology, and Logistics for prototype projects that exceed $100 million  According to the Senate report: Section 845 was intended to be used for limited prototype projects, particularly those in which the Department seeks to engage nontraditional defense contractors that may be averse to the requirements imposed by a standard Department procurement contract  For this reason, the statement of managers accompanying the Strom Thurmond National Defense Authorization Act for Fiscal Year 1999 (Public Law 105–261) states: The conferees continue to believe that the section 845 authority should only be used in the exceptional cases where it can be clearly demonstrated that a normal contract or grant will not allow sufficient access to affordable technologies  The conferees are especially concerned that such authority not be used to circumvent the appropriate management controls in the standard acquisition and budgeting process  …  The committee does not believe that the $20 9 billion agreement entered between the Army and the Lead Systems Integrator for the FCS program is consistent with the language and intent of section 845 authority  Section 845 authority is intended to be used for limited prototype projects, particularly those in which the Department of Defense seeks to engage nontraditional defense contractors that may be averse to the requirements imposed by a standard Department contract  Department of Defense Appropriations Act, 2007 ( H R  5631 ) Reporting Requirement The FY2007 defense appropriations bill ( P L  109-289 ) did not include language addressing OTs  The conference report ( H Rept  109-676 ) included language expressing the conferees' "[concern] with the continued use of OTA contracts by the Missile Defense Agency," as such contracts "lack the customary safeguards found under FAR-based contracts for organizational conflict of interest, truth in negotiations and submission of cost and pricing data " The conferees "strongly encourage[d]" the Missile Defense Agency to convert "large development and procurement contracts using OTA to FAR-based contracts," and directed the Missile Defense Agency to submit a report to the congressional defense committees on the use of OTs, to include the number, value, and justification for the use of such agreements  National Defense Authorization Act for FY2008 ( P L  110-181 ) Section 823 extended the authority for prototype projects for five more years, from September 30, 2008, to September 30, 2013  National Defense Authorization Act for FY2009 ( P L  110-417 ) Section 822 required DOD to issue guidance on rights in technical data under non-FAR agreements, including OTs  Section 822 also required that appropriate provisions relating to rights in technical data be included in non-FAR agreements, consistent with policy guidance  This requirements is in statute at 10 U S C  2320 note  Section 824 expanded the scope of the pilot program for transition to follow-on contracts for certain prototype projects to include research projects carried out under 10 U S C  2371  Authority to use the pilot program, set to expire September 30, 2008, was extended to September 30, 2010  Section 874 required OT data be included in the Federal Procurement Data System  National Defense Authorization Act for FY2011 ( P L  111-383 ) Section 866 changed the definition of nontraditional defense contractor, conforming the definition to that found in 10 U S C  2302(9)  National Defense Authorization Act for FY2013 ( P L  112-239 ) Section 863 extended the authority for using OTs for prototype projects from September 30, 2013, to September 30, 2018  Carl Levin and Howard P  "Buck" McKeon National Defense Authorization Act for FY2015 ( P L  113-291 ) Expanded Authority Section 812 expanded the authority to use OT for prototypes, to include those "directly related to enhancing the mission effectiveness of military personnel and the supporting platforms, systems, components, or materials proposed to be acquired or developed by the Department of Defense, or to improvement of platforms, systems, components, or materials in use by the Armed Forces " Prior to the FY2015 NDAA, OTs could only be used for prototypes relating to weapons or weapon systems proposed to be developed, or for the improvement of weapons or weapon systems currently in use  Reporting Requirements Section 1071 repealed the reporting requirement language found in 10 U S C  2371, relating to OTs for research projects  National Defense Authorization Act for FY2016 ( P L  114-92 ) Until the FY2016 NDAA, the prototyping and follow-on production authority established in Section 845 of the FY1994 NDAA (as amended) was found in 10 U S C  2371 note  Section 815 of the FY2016 NDAA simultaneously repealed Section 845 of the FY1994 NDAA and put the repealed language into the newly created 10 U S C  2371b  The FY2016 NDAA also modified 2371b by making the authority permanent  Expanded Authority and Small Business The authorities in Section 2371b were expanded to allow their use when "all significant participants in the transaction other than the Federal government are small businesses or nontraditional contractors" and when the agency determines that using an OT would expand the defense supply base in a manner that could not be accomplished through a contract  Section 815 eased the restriction on follow-on production contracts or transactions  Section 815 amended the definition of a nontraditional defense contractor found in 10 U S C  2302 to be an entity that is not currently performing, and for one year prior to an OT has not performed on any contract or subcontract that is subject to full coverage under the cost accounting standards pursuant to Section 1502 of Title 41, U S C  Section 815 also required DOD to update its guidance to reflect changes in the statute  The conference report stated that Congress believed OTs are an attractive option for firms and organizations that do not usually participate in government contracting due to typical overhead burdens and the "one size fits all" rules governing defense acquisition  The report also stated that OTs could support DOD's effort to access new sources of technological innovation, specifically with Silicon Valley startup firms and small commercial firms  National Defense Authorization Act for FY2018 ( P L  115-91 ) Expanded Authority Section 216 of the FY2018 NDAA authorized nonprofit research institutions to enter into OTs with DOD for prototype projects  Section 862 amended 10 U S C  2358, granting the Secretary of Defense and the military departments the authority to pursue basic research, applied research, advanced research, and development projects under the OT authorities granted in Sections 2371 and 2371b of Title 10  Workforce Section 802 required DOD to establish a cadre of intellectual property experts to advise, assist, and provide resources to program offices who are developing intellectual property strategies for contracts and agreements  Section 863 required training and education for personnel involved in OTs and other innovative contracting methods  Certification Requirements and Small Business Section 864 adjusted the language of the statute to state that the dollar threshold relates to the specific transaction for a prototype project and not for the value of the entire project  Section 864 defined a transaction for follow-on production to include "all individual prototype sub-projects awarded under the transaction to a consortium of United States industry and academic institutions " Section 864 also increased the dollar thresholds for required approvals and defined the term small business to include small businesses under Section 9 of the Small Business Act to ensure that companies participating in the Small Business Innovation Research and Small Business Technology Transfer programs were considered small businesses for the purposes of the cost-sharing requirements  Miscellaneous Section 867 required the Secretary of Defense to establish a preference for OTs in the "execution of science and technology and prototyping programs " Section 1711 required DOD to carry out a pilot program to "assess the feasibility and advisability of increasing the manufacturing capability of the defense industrial base " Pursuant to the pilot, Section 1711 authorized DOD to use OTs to support production capabilities in small and medium-sized manufacturers  John S  McCain National Defense Authorization Act for FY2019 ( P L  115-232 ) Section 211 of the FY2019 NDAA clarified that follow-on production of a prototype or subproject within a consortium may occur if the individual prototype or subproject is complete; all projects associated with the consortium do not need to be completed before follow-on production of a specific prototype  Expanded Authority The FY2019 NDAA authorized the use of OTs to develop enhanced personal protective equipment (Section 226) and to carry out research under the Explosive Ordnance Disposal Defense Program (Section 311)  Reporting Requirements Section 244 required the Defense Innovation Unit to submit a report to Congress, to include the number of traditional and nontraditional defense contractors with DOD contracts or other transactions resulting directly from the unit's initiatives  Section 873 required DOD to submit an annual report through 2021, summarizing DOD's use of OTs, including organizations involved; number of transactions; amounts of payments; and purpose, description, and status of projects  Department of Defense Appropriations Act, 2019 ( P L  115-245 ) Reporting Requirements While the enacted FY2019 defense appropriations bill did not include legislative language addressing OTs, the conference report included language expressing the conferees' "[concern] with the lack of transparency surrounding the employment of OTA, particularly for follow-on production " The conferees directed DOD to provide quarterly reports to the House and Senate appropriations committees listing each active OT, and to include the following information on each agreement: funding military service or DOD component; major command (if applicable); contracting activity; appropriation title; budget line item; minimum and maximum award value; vendor; obligations and expenditures to date; product service code; period of performance; and indication if the OT agreement included an option for follow-on production (with a description of the scope of anticipated follow-on production)  The conferees also directed GAO to review DOD's use of OTs to determine whether the "employment of this authority conforms to applicable statutes and guidelines, to include the identification of any potential conflicts " GAO was also required to report on the extent to which OTs have been used since FY2016  Notification Requirements The House report to accompany H R  6157 acknowledged OTs as an "important tool to provide flexibility and agility for cutting-edge research and development projects and prototypes " However, the report stated its concern "with the lack of transparency on the use of OTA authority for follow-on production procurements," and directed that no funds could be obligated or expended for a follow-on production contract or a transaction carried out under 10 U S C  2371b, until 30 days after the Secretary of Defense provides the congressional defense committees with a notification of the proposed contract or transaction, to include a justification of why an OT is being used for production  Appendix B  Non-DOD Federal Agencies with Agency-Wide OT or Related Authorities A number of agencies have varying other transaction or similar authorities, as reflected in Table B-1   The table below is not a comprehensive or definitive listing of every federal government entity with OT or related authorities  In some instances, offices, agencies, commissions, and other federal government entities have OT or related authorities that are only associated with certain programs or projects, such as the National Institutes of Health (which has OT authority for such specific activities such as the National Heart, Blood Vessel, Lung, and Blood Diseases and Blood Resources Program [42 U S C  §285b-3] and the Cures Acceleration Network [42 U S C  §287a])  Appendix C  Reliability of Data on Other Transactions All data have imperfections and limitations  FPDS-NG data can be used to identify broad trends and produce rough estimates, or to gather information about specific contracts  Some observers say that despite their shortcomings, FPDS-NG data are substantially more comprehensive than what is available in most other countries in the world  Understanding the limitations of government procurement data—including knowing when, how, and to what extent to rely on data—can help policymakers incorporate FPDS-NG data more effectively into their decisionmaking process  FPDS-NG OT Data Quality and Accuracy Issues Decisionmakers should be cautious when using data from FPDS-NG to develop policy or otherwise draw conclusions, especially with respect to OTs  In some cases, the data themselves may not be reliable  In other instances, a query for particular data may return differing results, depending on the parameters and timing of the analysis  In particular, all DOD data entered into FPDS-NG are subject to a 90-day delay, and updates to "data, including new actions, modifications, and corrections are made on a regular basis," which could result in changes to "data     for current and/or prior fiscal years " Inconsistencies in FPDS-NG Data Within FPDS-NG, two primary collections of obligation data exist: one associated with standard government procurement contracts or modifications to such contracts, and the other associated with prototype OT agreements or modifications to such agreements  FPDS-NG's collection of prototype OT data allows for the input of additional data elements—such as nongovernment dollars associated with cost-share prototype OT agreements—not included in FPDS-NG's collection of standard government procurement contract data  More consequentially, FPDS-NG's prototype OT data include two similar data elements that allow users to identify the fiscal year a prototype OT agreement was signed or modified  One, labeled in the database as "Fiscal Year," appears to allow users entering data into the system to manually assign a fiscal year to a transaction  FPDS-NG users entering data into the system appear to have interpreted this data element in various, conflicting ways  For example, a Department of the Air Force OT agreement was signed in February 2016 for the development of rocket propulsion system prototypes under the Evolved Expendable Launch Vehicle (EELV) program  FPDS-NG records an obligation of $115 million in FY2020 for this agreement  Fiscal law bars DOD from obligating money now for future fiscal years that have not yet occurred  The second, labeled "Contract Fiscal Year," appears to be based on the date the prototype OT agreement was signed or modified  See Table C-1 for a comparison of the "Fiscal Year" and "Contract Fiscal Year" elements for selected new prototype OT agreements signed between FY2013 and FY2017  If a user selects the Fiscal Year data element when attempting to review high-level data on recent trends in the use of prototype OT agreements within DOD—such as the total amount obligated for prototype OT agreements on an annual basis—that user will obtain a substantially different result than if he or she selects the Contract Fiscal Year data element for a similar analysis  See Table C-2 for a comparison of action obligations and nongovernment contributions using the Fiscal Year and Contract Fiscal Year data elements for new prototype OT agreements signed between FY2013 and FY2017  DOD OT Data Analysis Methodological Issues in Congressional Reports A March 2017 report to Congress entitled "An Assessment of Cost-Sharing in Other Transaction Agreements for Prototype Projects," completed by the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics indicated that In FY2016, DOD obligated (Note: Two outliers in 2016 excluded) $1 billion in section 2371b awards and received $68 million in cost-share contributions after excluding two significant trend outliers  DOD cited FPDS-NG as the source for its analysis, and defined the "significant trend outliers" excluded as "two $40 million OTAs with total cost-share of $270 million," likely referring to two DARPA prototype OT agreements conducted on a cost-share basis initiated in FY2016  However, a CRS analysis of the same FPDS-NG data identified numerous inconsistencies in DOD's methodological approach  Recreation of DOD Methodology Specifically, DOD appears to have conducted its analysis using the "Fiscal Year" data element referenced in this report's discussion of FPDS-NG OT data quality and accuracy issues, which would attribute some prototype OT activities to the wrong fiscal year for the purposes of comparative trend analysis  DOD also compared two disparate transaction types: the reported "$1 billion in Section 2371b awards" includes all action obligations associated with ongoing prototype OT activities in FY2016, including transactions associated with prototype OT indefinite delivery contracts, while the "$68 million in cost-share contributions" includes only cost-share contributions associated with new prototype OT agreements  If DOD used the "Contract Fiscal Year" data element, excluded the identified "trend outliers," and focused on all action obligations and cost-share contributions associated with ongoing prototype OT activities, it would have found instead that the department obligated $1 4 billion for prototype OT agreements in FY2016, with an additional $313 5 million in cost-share contributions from the private sector  On the other hand, if DOD used the "Contract Fiscal Year" data element, excluded the identified "trend outliers," and focused on only action obligations and cost-share contributions associated with new prototype OT agreements, it would have found instead that the department obligated $400 million for prototype OT agreements in FY2016, with an additional $272 1 million in cost-share contributions from the private sector  Appendix D  Other Transaction Authority Statutes 10 U S C  §2371  Research projects: transactions other than contracts and grants (a) ADDITIONAL FORMS OF TRANSACTIONS AUTHORIZED — The Secretary of Defense and the Secretary of each military department may enter into transactions (other than contracts, cooperative agreements, and grants) under the authority of this subsection in carrying out basic, applied, and advanced research projects  The authority under this subsection is in addition to the authority provided in Section 2358 of this title to use contracts, cooperative agreements, and grants in carrying out such projects  (b) EXERCISE OF AUTHORITY BY SECRETARY OF DEFENSE — In any exercise of the authority in subsection (a), the Secretary of Defense shall act through the Defense Advanced Research Projects Agency or any other element of the Department of Defense that the Secretary may designate  (c) ADVANCE PAYMENTS — The authority provided under subsection (a) may be exercised without regard to Section 3324 of Title 31  (d) RECOVERY OF FUNDS — (1) A cooperative agreement for performance of basic, applied, or advanced research authorized by Section 2358 of this title and a transaction authorized by subsection (a) may include a clause that requires a person or other entity to make payments to the Department of Defense or any other department or agency of the Federal Government as a condition for receiving support under the agreement or other transaction  (2) The amount of any payment received by the Federal Government pursuant to a requirement imposed under paragraph (1) may be credited, to the extent authorized by the Secretary of Defense, to the appropriate account established under subsection (f)  Amounts so credited shall be merged with other funds in the account and shall be available for the same purposes and the same period for which other funds in such account are available  (e) CONDITIONS — (1) The Secretary of Defense shall ensure that- (A) to the maximum extent practicable, no cooperative agreement containing a clause under subsection (d) and no transaction entered into under subsection (a) provides for research that duplicates research being conducted under existing programs carried out by the Department of Defense; and (B) to the extent that the Secretary determines practicable, the funds provided by the Government under a cooperative agreement containing a clause under subsection (d) or a transaction authorized by subsection (a) do not exceed the total amount provided by other parties to the cooperative agreement or other transaction  (2) A cooperative agreement containing a clause under subsection (d) or a transaction authorized by subsection (a) may be used for a research project when the use of a standard contract, grant, or cooperative agreement for such project is not feasible or appropriate  (f) SUPPORT ACCOUNTS — There is hereby established on the books of the Treasury separate accounts for each of the military departments and the Defense Advanced Research Projects Agency for support of research projects and development projects provided for in cooperative agreements containing a clause under subsection (d) and research projects provided for in transactions entered into under subsection (a)  Funds in those accounts shall be available for the payment of such support  (g) EDUCATION AND TRAINING —The Secretary of Defense shall— (1) ensure that management, technical, and contracting personnel of the Department of Defense involved in the award or administration of transactions under this section or other innovative forms of contracting are afforded opportunities for adequate education and training; and (2) establish minimum levels and requirements for continuous and experiential learning for such personnel, including levels and requirements for acquisition certification programs  (h) REGULATIONS — The Secretary of Defense shall prescribe regulations to carry out this section  (i) Protection of Certain Information From Disclosure -(1) Disclosure of information described in paragraph (2) is not required, and may not be compelled, under Section 552 of Title 5 for five years after the date on which the information is received by the Department of Defense  (2)(A) Paragraph (1) applies to information described in subparagraph (B) that is in the records of the Department of Defense if the information was submitted to the Department in a competitive or noncompetitive process having the potential for resulting in an award, to the party submitting the information, of a cooperative agreement for performance of basic, applied, or advanced research authorized by Section 2358 of this title or another transaction authorized by subsection (a)  (B) The information referred to in subparagraph (A) is the following: (i) A proposal, proposal abstract, and supporting documents  (ii) A business plan submitted on a confidential basis  (iii) Technical information submitted on a confidential basis  10 U S C  §2371b  Authority of the Department of Defense to carry out certain prototype projects (a) AUTHORITY — (1) Subject to paragraph (2), the Director of the Defense Advanced Research Projects Agency, the Secretary of a military department, or any other official designated by the Secretary of Defense may, under the authority of Section 2371 of this title, carry out prototype projects that are directly relevant to enhancing the mission effectiveness of military personnel and the supporting platforms, systems, components, or materials proposed to be acquired or developed by the Department of Defense, or to improvement of platforms, systems, components, or materials in use by the armed forces  (2) The authority of this section- (A) may be exercised for a transaction (for a prototype project) that is expected to cost the Department of Defense in excess of $100,000,000 but not in excess of $500,000,000 (including all options) only upon a written determination by the senior procurement executive for the agency as designated for the purpose of Section 1702(c) of Title 41, or, for the Defense Advanced Research Projects Agency or the Missile Defense Agency, the director of the agency that- (i) the requirements of subsection (d) will be met; an d (ii) the use of the authority of this section is essential to promoting the success of the prototype project; and (B) may be exercised for a transaction (for a prototype project) that is expected to cost the Department of Defense in excess of $500,000,000 (including all options) only if- (i) the Under Secretary of Defense for Acquisition, Technology, and Logistics determines in writing that- (I) the requirements of subsection (d) will be met; an d (II) the use of the authority of this section is essential to meet critical national security objectives; and (ii) the congressional defense committees are notified in writing at least 30 days before such authority is exercised  (3) The authority of a senior procurement executive or director of the Defense Advanced Research Projects Agency or Missile Defense Agency under paragraph (2)(A), and the authority of the Under Secretary of Defense for Acquisition, Technology, and Logistics under paragraph (2)(B), may not be delegated  (b) EXERCISE OF AUTHORITY — (1) Subsections (e)(1)(B) and (e)(2) of such Section 2371 shall not apply to projects carried out under subsection (a)  (2) To the maximum extent practicable, competitive procedures shall be used when entering into agreements to carry out projects under subsection (a)  (c) COMPTROLLER GENERAL ACCESS TO INFORMATION — (1) Each agreement entered into by an official referred to in subsection (a) to carry out a project under that subsection that provides for payments in a total amount in excess of $5,000,000 shall include a clause that provides for the Comptroller General, in the discretion of the Comptroller General, to examine the records of any party to the agreement or any entity that participates in the performance of the agreement  (2) The requirement in paragraph (1) shall not apply with respect to a party or entity, or a subordinate element of a party or entity, that has not entered into any other agreement that provides for audit access by a Government entity in the year prior to the date of the agreement  (3) (A) The right provided to the Comptroller General in a clause of an agreement under paragraph (1) is limited as provided in subparagraph (B) in the case of a party to the agreement, an entity that participates in the performance of the agreement, or a subordinate element of that party or entity if the only agreements or other transactions that the party, entity, or subordinate element entered into with Government entities in the year prior to the date of that agreement are cooperative agreements or transactions that were entered into under this section or Section 2371 of this title  (B) The only records of a party, other entity, or subordinate element referred to in subparagraph (A) that the Comptroller General may examine in the exercise of the right referred to in that subparagraph are records of the same type as the records that the Government has had the right to examine under the audit access clauses of the previous agreements or transactions referred to in such subparagraph that were entered into by that particular party, entity, or subordinate element  (4) The head of the contracting activity that is carrying out the agreement may waive the applicability of the requirement in paragraph (1) to the agreement if the head of the contracting activity determines that it would not be in the public interest to apply the requirement to the agreement  The waiver shall be effective with respect to the agreement only if the head of the contracting activity transmits a notification of the waiver to Congress and the Comptroller General before entering into the agreement  The notification shall include the rationale for the determination  (5) The Comptroller General may not examine records pursuant to a clause included in an agreement under paragraph (1) more than three years after the final payment is made by the United States under the agreement  (d) APPROPRIATE USE OF AUTHORITY — (1) The Secretary of Defense shall ensure that no official of an agency enters into a transaction (other than a contract, grant, or cooperative agreement) for a prototype project under the authority of this section unless one of the following conditions is met: (A) There is at least one nontraditional defense contractor or nonprofit research institution participating to a significant extent in the prototype project  (B) All significant participants in the transaction other than the Federal Government are small businesses (including small businesses participating in a program described under Section 9 of the Small Business Act (15 U S C  638)) or nontraditional defense contractors  (C) At least one third of the total cost of the prototype project is to be paid out of funds provided by sources other than other than the Federal Government  (D) The senior procurement executive for the agency determines in writing that exceptional circumstances justify the use of a transaction that provides for innovative business arrangements or structures that would not be feasible or appropriate under a contract, or would provide an opportunity to expand the defense supply base in a manner that would not be practical or feasible under a contract  (2) (A) Except as provided in subparagraph (B), the amounts counted for the purposes of this subsection as being provided, or to be provided, by a party to a transaction with respect to a prototype project that is entered into under this section other than the Federal Government do not include costs that were incurred before the date on which the transaction becomes effective  (B) Costs that were incurred for a prototype project by a party after the beginning of negotiations resulting in a transaction (other than a contract, grant, or cooperative agreement) with respect to the project before the date on which the transaction becomes effective may be counted for purposes of this subsection as being provided, or to be provided, by the party to the transaction if and to the extent that the official responsible for entering into the transaction determines in writing that- (i) the party incurred the costs in anticipation of entering into the transaction; and (ii) it was appropriate for the party to incur the costs before the transaction became effective in order to ensure the successful implementation of the transaction  (e) Definitions —In this section: (1) The term "nontraditional defense contractor" has the meaning given the term under Section 2302(9) of this title  (2) The term "small business" means a small business concern as defined under Section 3 of the Small Business Act (15 U S C  632)  (f) FOLLOW-ON PRODUCTION CONTRACTS OR TRANSACTIONS — (1) A transaction entered into under this section for a prototype project may provide for the award of a follow-on production contract or transaction to the participants in the transaction  A transaction includes all individual prototype subprojects awarded under the transaction to a consortium of United States industry and academic institutions  (2) A follow-on production contract or transaction provided for in a transaction under paragraph (1) may be awarded to the participants in the transaction without the use of competitive procedures, notwithstanding the requirements of Section 2304 of this title, if - (A) competitive procedures were used for the selection of parties for participation in the transaction; and (B) the participants in the transaction successfully completed the prototype project provided for in the transaction  (3) Contracts and transactions entered into pursuant to this subsection may be awarded using the authority in subsection (a), under the authority of Chapter 137 of this title, or under such procedures, terms, and conditions as the Secretary of Defense may establish by regulation  (g) AUTHORITY TO PROVIDE PROTOTYPES AND FOLLOW-ON PRODUCTION ITEMS AS GOVERNMENT-FURNISHED EQUIPMENT — An agreement entered into pursuant to the authority of subsection (a) or a follow-on contract or transaction entered into pursuant to the authority of subsection (f) may provide for prototypes or follow-on production items to be provided to another contractor as Government-furnished equipment  (h) APPLICABILITY OF PROCUREMENT ETHICS REQUIREMENTS — An agreement entered into under the authority of this section shall be treated as a Federal agency procurement for the purposes of Chapter 21 of Title 41  10 U S C  §2373  Procurement for experimental purposes (a) AUTHORITY — The Secretary of Defense and the Secretaries of the military departments may each buy ordnance, signal, chemical activity, transportation, energy, medical, space-flight, and aeronautical supplies, including parts and accessories, and designs thereof, that the Secretary of Defense or the Secretary concerned considers necessary for experimental or test purposes in the development of the best supplies that are needed for the national defense  (b) PROCEDURES — Purchases under this section may be made inside or outside the United States and by contract or otherwise  Chapter 137 of this title applies only when such purchases are made in quantities greater than necessary for experimentation, technical evaluation, assessment of operational utility, or safety or to provide a residual operational capability 


Report 2: A high-quality, reliable cost estimate is a key tool for budgeting, planning, and managing the 2020 Census  According to OMB, programs must maintain current and well-documented estimates of program costs, and these estimates must encompass the full life-cycle of the program  Among other things, OMB states that generating reliable program cost estimates is a critical function necessary to support OMB’s capital programming process  Without this capability, agencies are at risk of experiencing program cost overruns, missed deadlines, and performance shortfalls  A reliable cost estimate is critical to the success of any federal government program  With the information from reliable estimates, managers can: make informed investment decisions, allocate program resources, measure program progress, proactively correct course when warranted, and ensure overall accountability for results  To be considered reliable, a cost estimate must meet the criteria for each of the four characteristics outlined in our Cost Estimating and Assessment Guide  According to our analysis, a cost estimate is considered reliable if the overall assessment ratings for each of the four characteristics are substantially or fully met  If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable  Those characteristics are: Well-documented: An estimate is thoroughly documented, including source data and significance, clearly detailed calculations and results, and explanations of why particular methods and references were chosen  Data can be traced to their source documents  Accurate: An estimate is unbiased, the work is not overly conservative or overly optimistic, and is based on an assessment of most likely costs  Few, if any, mathematical mistakes are present  Credible: Any limitations of the analysis because of uncertainty or bias surrounding data or assumptions are discussed  Major assumptions are varied, and other outcomes are recomputed, to determine how sensitive they are to changes in the assumptions  Risk and uncertainty analysis is performed to determine the level of risk associated with the estimate  The estimate’s results are cross- checked, and an independent cost estimate (ICE) is conducted to see whether other estimation methods produce similar results  Comprehensive: An estimate has enough detail to ensure that cost elements are neither omitted nor double counted  All cost-influencing ground rules and assumptions are detailed in the estimate’s documentation  Meeting best practices outlined in our Cost Estimating and Assessment Guide for a reliable cost estimate has been a long-standing challenge for the Bureau  In 2008 we reported that the 2010 Census cost estimate was not reliable because it lacked documentation and was not comprehensive, accurate, or credible  For example, in our 2008 report on the Bureau’s cost estimation process, Bureau officials were unable to provide documentation that supported the assumptions for the initial 2001 life-cycle cost estimate as well as the updates  Consequently, we recommended that the Bureau establish guidance, policies, and procedures for estimating costs that would meet best practices criteria  The Bureau agreed with the recommendation and said at the time that it already had efforts underway to improve its future cost estimation methods and systems  Moreover, weaknesses in the life-cycle cost estimate were one reason we designated the 2010 Census a GAO High- Risk Area in 2008  In 2012 we reported that, while the Bureau was taking steps to strengthen its life-cycle cost estimates, it had not yet established guidance for developing cost estimates  We recommended that the Bureau finalize its guidance, policies, and procedures for cost estimation in accordance with best practices  The Bureau agreed with the overall theme of the report but did not comment on the recommendation  During this review we found that the Bureau took steps to address this recommendation, which is discussed later in this report  Such guidance can help to institutionalize best practices and ensure consistent processes and operations for producing reliable estimates  In a 2016 report we found that the October 2015 version of the Bureau’s life-cycle cost estimate for the 2020 Census was not reliable  Overall, we reported that the 2020 Census life-cycle cost estimate partially met two of the characteristics of a reliable cost estimate (comprehensive and accurate) and minimally met the other two (well-documented and credible)  We recommended that the Bureau take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate  The Bureau agreed with this recommendation, and took steps to improve the reliability of its cost estimate, which we focus on later in this report  Consequently, an unreliable life-cycle cost estimate is one of the reasons we designated the 2020 Census a GAO High-Risk Area in 2017  In October 2015, the Bureau estimated the cost of the 2020 Census to be $12 3 billion  According to the Bureau, the October 2015 version was the Bureau’s first attempt to model the life-cycle cost of its planned 2020 Census, in contrast to its earlier 2011 estimate, which the Bureau said was intended to produce an approximation of potential savings and to begin developing the methodology for producing decennial life-cycle cost estimates covering all phases of the decennial life cycle  To help control costs while maintaining accuracy, the Bureau introduced significant change to how it conducts the decennial census in 2020  Its planned innovations include reengineering how it builds its address list, improving self-response by encouraging the use of the Internet and telephone, using administrative records to reduce field work, and reengineering field operations using technology to reduce manual effort and improve productivity  In contrast to the estimated $12 3 billion in 2015, the 2020 Census would cost $17 8 billion in constant 2020 dollars if the Bureau repeated the 2010 Census design and methods, according to the Bureau’s estimates  In October 2017, Commerce announced that it had updated the October 2015 life-cycle cost estimate, projecting the life-cycle cost of the 2020 Census to be $15 6 billion, an increase of over $3 billion (27 percent) over its 2015 estimate  (See figure 1 ) In developing the 2017 version of the cost estimate, Bureau cost estimators identified cost inputs, their ranges for possible outcomes, and overall cost estimating relationships (i e , logical or mathematical formulas, or both)  To identify cost inputs and the ranges of potential outcomes, the Bureau worked with subject matter experts and used historical data to support assumptions and generate inputs  The Bureau’s cost estimation team used a software tool to generate the cost estimate  Because cost estimates predict future program costs, uncertainty is always associated with them  For example, data from the past (such as fuel prices) may not always be relevant in the future  Risk and uncertainty refer to the fact that because a cost estimate is a forecast, there is always a chance that the actual cost will differ from the estimate  One way to determine whether a program is realistically budgeted is to perform an uncertainty analysis, so that the probability associated with achieving its point estimate can be determined, usually relying on simulations such as those of Monte Carlo methods  This can be particularly useful in portraying the uncertainty implications of various cost estimates  Consistent with cost estimation practices outlined in our Cost Estimating and Assessment Guide, the estimate was compared with two independent cost estimates (ICE), developed by Commerce’s Office of Acquisition Management (OAM) and the Bureau’s Office of Cost Estimation, Analysis, and Assessment  The offices producing the ICEs and the cost estimate team worked together to examine the process each used, an effort known as the reconciliation process  Through this reconciliation, the Bureau identified areas where discrepancies existed and elements that could require additional review and possible improvement  According to Bureau documentation the estimate will be updated as the program meets milestones and to reflect changes in technical or program assumptions  Figure 2 details the Bureau’s cost estimation process  OAM was involved extensively in the development of the 2017 estimate, an increased involvement compared to 2015, according to Bureau officials  OAM participated in regular review meetings throughout the development of the estimate and also developed an independent cost estimate, as shown in the figure below  End-to-end system testing activities for the 2020 Census are currently underway in Providence, Rhode Island  According to the Bureau, information collected from the test, such as overall response rates and the use of administrative records to inform census records, will inform future versions of the life-cycle cost estimate  Some updates from the test will be incorporated into the next cost estimate, which will be available in the first quarter of the coming fiscal year  Since our June 2016 report, in which we reviewed the Bureau’s 2015 version of the cost estimate, the Bureau has made significant progress  For example, the Bureau has put into place a work breakdown structure (WBS) that defines the work, products, activities, and resources necessary to accomplish the 2020 Census and is standardized for use in budget planning, operational planning, and cost estimation  However, the Bureau’s October 2017 cost estimate for the 2020 Census does not fully reflect characteristics of a high-quality estimate as described in our Cost Estimating and Assessment Guide and cannot be considered reliable  Our Cost Estimating and Assessment Guide describes best practices for developing reliable cost estimates  For our reporting needs, we collapsed these best practices into four characteristics for sound cost estimating— comprehensive, well-documented, accurate, and credible—and identified specific best practices for each characteristic  To be considered reliable, an organization must meet or substantially meet each characteristic  Our review found the Bureau met or substantially met three out of the four characteristics of a reliable cost estimate, while it partially met one characteristic: well-documented  When compared to the October 2015 estimate, the 2017 estimate shows considerable improvement  (See figure 3 below ) Cost estimates are considered valid if they are well-documented to the point they can be easily repeated or updated and can be traced to original sources through auditing, according to best practices  The Bureau only partially met the criteria for well-documented, as set forth in our Cost Estimating and Assessment Guide  A cost estimate that does not fully meet the criteria for well-documented cannot be used by management to make informed and effective implementation decisions  The well-documented characteristic comprises five best practices  The Bureau substantially met two out of five best practices (as shown in figure 4)  First, the estimate describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost, and the cost estimate had been reviewed by management  Since cost estimates can inform key decisions and budget requests, it is vital that management review and understand how the estimate was developed, including risks associated with the underlying data and methods  The cost estimate only partially met three best practices for the characteristic of being well-documented  In general, some documentation was missing, inconsistent, or difficult to understand  First, we found that source data did not always support the information described in the basis of estimate document or could not be found in the files provided for two of the Bureau’s largest field operations: Address Canvassing and Non- Response Follow-Up (NRFU)  For example, the cost estimate documentation referred to actual data from the 2010 Census and information obtained from experts as sources for address canvassing rework rates  However, the folder source documents provided as support for the basis of estimate did not include this information  Next, in several cases, we could not replicate calculations, such as for mileage costs, using the description provided  Lastly, we found that some of the cost elements did not trace clearly to supporting spreadsheets and assumption documents  Failure to document an estimate in enough detail makes it more difficult to replicate calculations, or to detect possible errors in the estimate; reduces transparency of the estimation process; and can undermine the ability to use the information to improve future cost estimates or even to reconcile the estimate with another independent cost estimate  The Bureau told us it would continue to make improvements to ensure the estimate is well- documented  For the estimate to be considered well-documented, the Bureau will need to address these issues  An accurate cost estimate supports measurement of program progress by providing unbiased and correct data, which can help management ensure accountability for scheduled results  We found the Bureau’s cost estimate substantially met the criteria for accuracy  As shown in figure 5, and in line with best practices outlined in our Cost Estimating and Assessment Guide, the estimate was not overly optimistic; appeared to be free of errors; was based on historical data or input from subject matter experts; and, according to Bureau officials, is updated regularly as information becomes available  The Bureau can enhance the accuracy of their estimate by increasing the level of detail included in the documentation, such as detail on specific inflation indices used, and by monitoring actual costs against estimates  We identified areas for improvement, which, according to Bureau officials, will be addressed as part of its ongoing efforts  For example, while the basis of estimate document describes different inflation indexes, it was not clear exactly which indexes were applied to the various cost elements in the estimate  Also, evidence of how variances between estimated costs and actual expenses would be tracked over time was not available at the time of our analysis  Tools to track variance enable management to measure progress against planned outcomes  Bureau officials stated that they already have systems in place that can be adapted for tracking estimated and actual costs  All estimates include a certain amount of informed judgment about the future  Assumptions made at the start of a program can turn out to be inaccurate  Credible cost estimates identify limitations due to uncertainty or bias surrounding data or assumptions, and control for these uncertainties by identifying and quantifying cost elements that represent the most risk  We found that the Bureau’s cost estimate substantially met the criteria for credible, as shown in figure 6 below  The Bureau’s cost estimate clearly identifies risks and uncertainties, and describes approaches taken to mitigate them  In line with best practices outlined in our Cost Estimating and Assessment Guide, the Bureau did the following: Sensitivity analysis  The Bureau conducted sensitivity analysis to identify possible changes to estimated costs for the 2020 Census based on varying major assumptions, parameters, and data inputs  For example, the Bureau calculated the likely cost implications for a range of possible response rates to identify a range of projected costs and to calculate appropriate reserves for risk  Bureau officials stated that they also identified the estimate input parameters that contributed the most to estimate uncertainty  Risk and uncertainty analysis  A cost estimate is a forecast, and as such, there is always a chance that the actual cost will differ from the estimate  Uncertainty is the indefiniteness about the outcome of a situation  Uncertainty is assessed in cost estimate models to estimate the risk (or probability) that a specific funding level will be exceeded  We found the Bureau performed an uncertainty analysis on a portion of the estimate to determine whether estimated costs were realistic and to establish the probability of achieving projections outlined in the estimate  The Bureau used a combination of modeling based on Monte Carlo analysis and allocations of funding for risks  The Monte Carlo simulation was performed on a portion of the estimate to account for uncertainty around various operational parameters for which a range of outcomes was possible, including Internet response rates and the extent to which data collection issues might be resolved using administrative records  To account for the inherent uncertainty of assumptions included within the life-cycle cost estimate, the Bureau added funding to the cost estimate totaling approximately $292 million to account for risks based on the results of the Monte Carlo analysis  For other risks, such as acquisition lead time and the possibility of delays in information technology (IT) development, contingency funding was added to the estimate to reflect the potential cost of resolving these issues, through use of a backup system or an alternative approach  These are described as “special risks” in the Bureau’s basis of estimate, and total approximately $171 million  Based on additional sensitivity analysis, the Bureau added approximately $965 million to the cost estimate to reflect discrete risks outlined in the risk register as well as those associated with (1) variability in self-response rates, (2) the effect of fluctuations in the size and wage rate of the temporary workforce on the cost of field operations, and (3) the potential need to reduce the enumerator-to- manager staffing ratio in case expected efficiencies in field operations are not realized  In addition to these provisions, the Secretary of Commerce added a contingency amount of about $1 2 billion to account for what the Bureau refers to as unknown-unknowns  Bureau documentation states that conducting a decennial census is an extremely complex, high-risk operation  In order to mitigate some of the risk, contingency funding must be available to initiate ad hoc activities necessary to overcome unforeseen issues  According to Bureau documentation these include such risks as natural disasters or cyber-attacks  The Bureau provides a description of how the general risk contingency is calculated  However, this description does not clearly link calculated amounts to the risks themselves  In our June 2016 report we reported the Bureau had not properly accounted for risk and recommended the Bureau, in part; improve control over how risk and uncertainty are accounted for  We continue to believe the prior recommendation from our June 2016 report remains valid and should be addressed: that the Bureau properly account for risk in the 2020 Census cost estimate, among other things  As such, risks need to be linked to the $1 2 billion general risk contingency fund  Independent cost estimate  According to best practices outlined in our Cost Estimating and Assessment Guide, an independent cost estimate should be performed to determine whether alternate estimate approaches produce similar results  The Bureau compared their estimate with two independent cost estimates, developed by Commerce’s Office of Acquisition Management and the Bureau’s Office of Cost Estimation and Assessment  As part of their process for finalizing the cost estimate, Bureau officials reconciled differences between the estimates in discussions with the two offices, resulting in more conservative assumptions by the Bureau around risk and uncertainty in both cases  In addition to implementing our recommendation to properly account for risk, going forward, while the Bureau substantially met the credibility characteristic it will be important for them to also integrate regular cross-checks of methodology into their cost estimation process  In our analysis we observed that no specific cross-checks of cost methodology were performed  According to the Bureau, cross- checks were not performed because the Bureau considered the independent cost estimates as overall cross-checks on the reliability of their methodology and did not conduct additional cross-checks  The main purpose of cross-checking is to determine whether alternative methods for specific cost elements within the cost estimate could produce similar results  An independent cost estimate, though important for the credibility of an estimate, does not fulfill the same function as a targeted cross-check of individual elements  Comprehensive estimates have enough detail to ensure that cost elements are neither omitted nor double-counted, all cost-influencing assumptions are detailed in the estimate’s documentation, and a work breakdown structure is defined  Our analysis of the 2017 cost estimate demonstrates improvement over the 2015 cost estimate when the Bureau’s cost estimate only partially met the criteria for comprehensive  We found the Bureau met or substantially met all four best practices for the comprehensive characteristic, as shown in figure 7  For example, all life-cycle costs are included in the estimate along with a complete description of the 2020 Census program and current schedule  We also found that the Bureau substantially met criteria for documenting cost influencing ground rules and assumptions  A standardized WBS (as detailed in table 1) with supporting dictionary outlines the major work of the program and describes the activities and deliverables at the project level where costs are tracked  In 2016, the Bureau’s WBS did not contain sufficient detail and we found significant differences in the presentation of the work between sources  In 2017, based on our review of Bureau documentation and interviews with Bureau officials, we found that the WBS is standardized and cost elements are presented in detail  The WBS is a necessary program management tool because it provides a basic framework for a variety of related tasks like estimating costs, developing schedules, identifying resources, determining where risks may occur, and providing the means for measuring program status  Although the Bureau’s updated life-cycle cost estimate reflects three of the four characteristics of a reliable cost estimate, we are not making any new recommendations to the Bureau in this report  We continue to believe the prior recommendation, made in 2016, remains relevant: that the Secretary of Commerce ensure that the Bureau finalizes the steps needed to fully meet the characteristics of a high-quality estimate, most notably in the well-documented area  The Bureau told us it has used our best practices for cost estimation to develop their cost estimate, and will focus on those best practices that require attention moving forward  Without a reliable cost estimate, the Bureau is limited in its ability to make informed decisions about program resources and to effectively measure progress against operational objectives  OMB, in its guidance for preparing and executing agency budgets, cites that credible cost estimates are vital for sound management decision making and for any program or capital project to succeed  A well- developed cost estimate serves as a tool for program development and oversight, supporting management to make informed decisions  According to the Bureau, the 2020 Census cost estimate is used as a management tool to guide decision making  Bureau officials stated the cost estimate is used to examine the cost impact of program changes  For example, the cost estimate served as the basis for the fiscal year 2019 funding request developed by the Bureau  The Bureau also said it used the 2020 Census life-cycle cost estimate to establish cost controls during budget formulation activities and to monitor spending levels for fiscal year 2019 activities  According to the Bureau, as detailed operational and implementation plans are defined, the 2020 Census life- cycle cost estimate has been and will continue to be used to support ongoing “what-if” analyses in determining the cost impacts of design decisions  Specifically, using the cost estimate to model the impact of changes on overall cost, the Bureau adjusted the scope of the Census Enterprise Data Collection and Processing (CEDCaP) operation

ground_truth: The Department of Defense (DOD) obligates more than $300 billion annually to buy goods and services, and to support research and development. Most of these acquisitions are governed by procurement statutes and regulations found in Title 10 (and parts of other select titles) of the United States Code, the Federal Acquisition Regulation (FAR), and the Defense Federal Acquisition Regulation Supplement. Under certain circumstances, DOD can enter into an other transaction (OT) agreement instead of a traditional contract. OT agreements are generally exempt from federal procurement laws and regulations. These exemptions grant government officials the flexibility to include, amend, or exclude contract clauses and requirements that are mandatory in traditional procurements (e.g., termination clauses, cost accounting standards, payments, audit requirements, intellectual property, and contract disputes). OT authorities also grant more flexibility to structure agreements in numerous ways, including joint ventures; partnerships; consortia; or multiple agencies joining together to fund an agreement encompassing multiple providers. Other transaction agreements are legally binding contracts; they are referred to as agreements to distinguish them from the traditional procurement contracts governed by the FAR and procurements laws. Other transaction authorities are set forth in two sections of law: 10 U.S.C. 2371—granting authority to use OTs for basic, applied, and advanced research projects. 10 U.S.C. 2371b—granting authority to use OTs for prototype projects and follow-on production. Under this authority, a prototype project can only be conducted if at least one nontraditional defense contractor significantly participates in the project; all significant participants are small businesses or nontraditional defense contractors; at least one-third of the total cost of the prototype project is provided by nongovernment participants; or the senior procurement acquisition official provides a written justification for using an OT. Follow-on production can only be conducted when the underlying prototype OT was competitively awarded, and the prototype project was successfully completed. OTs have the potential to provide significant benefits to DOD, including attracting nontraditional contractors with promising technological capabilities to work with DOD, establishing a mechanism to pool resources with other entities to facilitate development of, and obtain, state-of-the-art dual-use technologies, and offering a unique mechanism for DOD to invest in, and influence the direction of, technology development. A number of analysts warn that along with the potential benefits come significant risks, including potentially diminished oversight and exemption from laws and regulations designed to protect government and taxpayer interests. In FY2017, DOD obligated $2.1 billion on prototype OT agreements, representing less than 1% of contract obligations for the year. However, the use of OTs is expected to grow at a rapid pace, due in part to recent statutory changes expanding other transaction authorities. A number of analysts and officials have raised concerns that if DOD uses OTs in ways not intended by Congress—or is perceived to abuse the authority—Congress could clamp down on the authority. Generally, DOD lacks authoritative data that can be used to measure and evaluate the use of other transaction authorities.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.



Report 2:  The processes for developing and updating estimates are designed to inform management about program progress and the use of program resources, supporting cost-driven planning efforts and well-informed decision making  Our work has identified a number of best practices for use in developing guidance related to cost estimation and analysis that are the basis of effective program cost estimating and should result in reliable and valid cost estimates that management can use for making informed decisions  In 2012 we reported that the Bureau had not yet established guidance for developing cost estimates  We recommended that the Bureau establish guidance, policies, and procedures for developing cost estimates that would meet best practice criteria  The Bureau agreed with the theme of the report but did not specifically agree with the recommendation  Moreover, in June 2016, we also reported that the cost estimation team did not record how and why it changed assumptions that were provided to it and did not document the sources of all data it used  The documentation of these changes to assumptions did not happen because the Bureau lacked written guidance and procedures for the cost estimation team to follow  During this review we found the Bureau has since established reliable guidance, processes, and policies for developing cost estimates and managing the cost estimation process  The following documents, shown in table 2, establish roles and responsibilities for oversight and approval of cost estimation processes, provide a detailed description of the steps taken to produce a high-quality cost estimate, and clarify the process for updating the cost estimate and associated documents over the life of a project  The Decennial Census Program’s Cost Estimate and Analysis Process, which provides a detailed description of the steps taken to produce a high-quality estimate, is reliable as it met the criteria for 8 steps and substantially met the criteria for 4 steps of the 12 best steps outlined in our Cost Estimating and Assessment Guide, as shown below in figure 8  To avoid cost overruns and to support high performance, it will be important for the Bureau to abide by their newly developed policies and guidance and continue to use the life-cycle cost estimate as a management tool  The 2017 life-cycle cost estimate includes significantly higher costs than those included in the 2015 estimate  In 2015, the Bureau estimated that they could conduct the operation at a cost of $12 3 billion in constant 2020 dollars  The Bureau’s latest cost estimate, announced in October 2017, reflects the same design, but at an expected cost of $15 6 billion  Figure 9 below shows the change in cost by WBS category for 2015 and 2017  The largest increases occurred in the Response, Managerial Contingency, and Census/Survey Engineering categories  Increased costs of $1 3 billion in the response category (costs related to collecting, maintaining, and processing survey response data) were in part due to reduced assumptions for self-response rates, leading to increases in the amount of data collected in the field, which is more costly to the Bureau  Contingency allocations increased overall from $1 35 billion in 2015 to $2 6 billion in 2017, as the Bureau gained a greater understanding of risks facing the 2020 Census  Increases of $838 million in the Census/Survey Engineering category were due mainly to the cost of an IT contract for integrating decennial survey systems that was not included in the 2015 cost estimate  Bureau officials attribute a decrease of $551 million in estimated costs for Program Management to changes in the categorization of costs associated with risks: In the 2017 version of the estimate, estimated costs related to program risks were allocated to their corresponding WBS element  More generally, factors that contributed to cost fluctuations between the 2015 and 2017 cost estimates include: changes in assumptions for census operations, improved ability to anticipate and quantify risk, an overall increase in IT costs, and more defined contract requirements  Several assumptions for the implementation of the 2020 Census have changed since the 2015 cost estimate  Some assumptions contributing to cost changes, mainly in the Response (related to collecting and processing response data) and Frame (the mapping and collecting addresses to frame enumeration activities) categories, include the following: Self-response rates  Changes in assumptions for expected self- response rates contributed to increases in the response category, as the assumed rate decreased from 63 5 percent in 2015 to 60 5 percent in 2017, thereby increasing the anticipated percentage and associated cost of nonresponse follow-up  When the Bureau does not receive responses by mail, phone, or Internet, census enumerators visit each nonresponding household to obtain data  Thus, reduced self-response rates lead to increases in the amount of data collected in the field, which is more costly to the Bureau  Bureau officials attributed this decrease to a forecasted reduction in Internet response due to added authentication steps at log in and the elimination of the function allowing users to save their responses and return later to complete the survey  Productivity rates  The productivity of enumerators collecting data for NRFU is another variable in the cost estimate that was updated, contributing to cost increases in the response category  Expected productivity rates for NRFU decreased from the 2015 estimate of 4 attempts per hour to 2 9  According to Bureau documentation, this more conservative estimate is based on historical data, rather than research and test data  In-office address canvassing rates  The Bureau will not go door-to- door to conduct in-field address canvassing across the country to update address and map information for every housing unit, as it has in prior decennial censuses  Rather, some areas would only need a review of their address and map information using computer imagery and third-party data sources—what the Bureau calls “in-office” address canvassing procedures  However, in March 2017, citing budget uncertainty the Bureau decided to discontinue one of the phases of in-office review address canvassing for the 2020 Census  The cancellation of that phase of in-office review is expected to increase the number of housing units canvassed in-field by 5 percent (from 25 to 30 percent of all canvassed housing units)  In-field canvassing is more labor intensive compared to in office procedures  The 2017 version of the cost estimate reflects this increase in workload for in-field address canvassing, though overall changes in estimated costs for the Frame category, of which Address Canvassing is a part, were minimal  Staffing  Updated analysis resulted in changes to several staffing assumptions, which resulted in decreases across WBS categories  Changes included reduced pay rates for field data collection staff based on current labor market conditions and reductions in the length of staff engagement  In general, contingency allocations increased overall from $1 35 billion in 2015 to $2 6 billion in 2017  This increase in contingency can be attributed, in part, to the Bureau gaining a clearer understanding of risk and uncertainty in the 2020 Census as it approaches  The Bureau developed some of its contingency based on proven risk management techniques, including Monte Carlo analysis and allocated funding for known risk scenarios  The 2017 estimate includes close to $1 4 billion in estimated costs for these risks, almost three times the amount included in the 2015 estimate  The basis of estimate contains detail on the various risks and the process for calculating the associated contingency  The 2017 version also includes a contingency amount of $1 2 billion for general risks, or unknown-unknowns, such as natural disasters and cyber-attacks  Contingency amounts were reallocated within the WBS to more closely reflect the nature of the risk: Bureau officials attribute a decrease from the 2015 estimate of $551 million in estimated costs for program management to changes in the categorization of costs associated with risks  Officials stated that, in 2015, discrete program risks were previously consolidated as program management costs  In 2017, these discrete costs were reallocated to associate risks with the appropriate WBS element  For example, contingency amounts related to the likelihood of achieving a certain response rate previously included in the program management work breakdown category are now a part of the “response” work breakdown category  Increases in IT costs, totaling $1 59 billion, represented almost 50 percent of the total cost increase from 2015 to 2017  The total share of IT costs as a percentage of total census costs increased from 28 percent in 2015 to 32 percent in 2017, or from $3 41 billion to approximately $5 billion  Increases in IT costs are spread across seven cost categories  Figure 10 shows the IT and non-IT cost by WBS for the 2017 cost estimate  IT costs in infrastructure, response data, and census/survey WBSs account for the majority of the approximately $5 billion  The Bureau’s October 2015 cost estimate included IT costs for, among other things, system engineering, test and evaluation, and infrastructure, as well as for a portion of the Census Enterprise Data Collection and Processing (CEDCaP) program  The 2017 estimated IT cost increases were due, in large part, to the Bureau (1) updating the cost estimate for CEDCaP; (2) including an estimate for technical integration services that contributed to increases in the Census and Survey Engineering category; and (3) updating costs related to other major contracts (such as mobile device as a service, field IT services, and payroll systems)  Bureau documents described an overall improvement in the Bureau’s ability to define and specify contract requirements  This resulted in updated estimates for several contracts, including for the Census Questionnaire Assistance (CQA) contract  Assumptions regarding call volume to the CQA were increased by 5 percent to account for expected response by phone after the elimination of the option to save Internet responses and return to complete the form later  The Bureau also cited updated cost data and the results of reconciliation with independent cost estimates as factors contributing to the increased costs of other major contracts, including for the procurement of data collection devices  The Secretary of Commerce provided comments on a draft of this report on August 2, 2018  The comments are reprinted in appendix II  The Department of Commerce generally agreed with our findings regarding the improvements the Census Bureau has made in its cost estimates  However, Commerce did not agree with our assessment that the Bureau’s 2017 lifecycle cost estimate is “not reliable ” Commerce noted that it had conducted two independent cost analyses and was satisfied that the cost estimate was reliable  The Bureau also provided technical comments that we incorporated, as appropriate  We maintain that, to be considered reliable, a cost estimate must meet or substantially meet the criteria for each of the four characteristics outlined in our Cost Estimating and Assessment Guide  These characteristics are derived from measures consistently applied by cost estimating organizations throughout the federal government and industry and are considered best practices for the development of reliable cost estimates  Without a reliable cost estimate, the Bureau is limited in its ability to make informed decisions about program resources and to effectively measure progress against operational objectives  Thus, while the Bureau has made considerable progress in all four of the characteristics, it has only partially met the criteria for the characteristic of being well-documented  Until the Bureau meets or substantially meets the criteria for this characteristic, the cost estimate cannot be considered reliable  As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date  At that time, we will send copies of the report to the appropriate congressional committees, the Secretary of Commerce, the Under Secretary of Economic Affairs, the Acting Director of the U S  Census Bureau, and other interested parties  In addition, this report is available at no charge on the GAO website at http://gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-2757 or goldenkoffr@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report  GAO staff who made key contributions to this report are listed in appendix III  The purpose of our review was to evaluate the reliability of the Census Bureau’s (Bureau) life-cycle cost estimate using our Cost Estimating and Assessment Guide  We reviewed (1) the extent to which the Bureau’s life-cycle cost estimate and associated guidance met our best practices for cost estimation using documentation and information obtained in discussions with the Bureau related to the 2020 life-cycle cost estimate and (2) compared the 2015 and 2017 life-cycle cost estimates to describe key drivers of cost growth  For both objectives we reviewed documentation from the Bureau on the 2020 life-cycle cost estimate and interviewed Bureau and Department of Commerce officials  For the first objective, we relied on our Cost Estimating and Assessment Guide as criteria  Our cost specialists assessed measures consistently applied by cost-estimating organizations throughout the federal government and industry and considered best-practices for developing reliable cost estimates  We analyzed the cost estimating practices used by the Bureau against these best practices and evaluated them in four categories: comprehensive, well-documented, accurate, and credible  Comprehensive  The cost estimate should include both government and contractor costs of the program over its full life-cycle, from inception of the program through design, development, deployment, and operation and maintenance to retirement of the program  It should also completely define the program, reflect the current schedule, and be technically reasonable  Comprehensive cost estimates should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted  Specifically, the cost estimate should be based on a product-oriented work breakdown structure (WBS) that allows a program to track cost and schedule by defined deliverables, such as hardware or software components  Finally, where information is limited and judgments are made, the cost estimate should document all cost-influencing assumptions  Well-documented  A good cost estimate—while taking the form of a single number—is supported by detailed documentation that describes how it was derived and how the expected funding will be spent in order to achieve a given objective  Therefore, the documentation should capture in writing such things as the source data used, the calculations performed and their results, and the estimating methodology used to derive each WBS element’s cost  Moreover, this information should be captured in such a way that the data used to derive the estimate can be traced back to, and verified against, their sources so that the estimate can be easily replicated and updated  The documentation should also discuss the technical baseline description and how the data were normalized  Finally, the documentation should include evidence that the cost estimate was reviewed and accepted by management  Accurate  The cost estimate should provide for results that are unbiased, and it should not be overly conservative or optimistic  An estimate is accurate when it is based on an assessment of most likely costs; adjusted properly for inflation; and contains few, if any, minor mistakes  In addition, a cost estimate should be updated regularly to reflect significant changes in the program—such as when schedules or other assumptions change—and actual costs, so that it is always reflecting current status  During the update process, variances between planned and actual costs should be documented, explained, and reviewed  Among other things, the estimate should be grounded in a historical record of cost estimating and actual experiences on other comparable programs  Credible  The cost estimate should discuss any limitations of the analysis because of uncertainty or biases surrounding data or assumptions  Major assumptions should be varied, and other outcomes recomputed to determine how sensitive they are to changes in the assumptions  Risk and uncertainty analysis should be performed to determine the level of risk associated with the estimate  Further, the estimate’s cost drivers should be cross-checked, and an independent cost estimate conducted by a group outside the acquiring organization should be developed to determine whether other estimating methods produce similar results  If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high- quality estimate and cannot be considered reliable  We also analyzed the Bureau’s cost estimation and analysis guidance and evaluated them against a 12-step process outlined in our Cost Estimation and Assessment Guide  A high-quality cost estimating process integrates the following: 1  Define estimate’s purpose  2  Develop estimating plan  3  Define program characteristics  4  Determine estimating structure  5  Identify ground rules and assumptions  6  Obtain data  7  Develop point estimate and compare it to an independent cost estimate  8  Conduct sensitivity analysis  9  Conduct risk and uncertainty analysis  10  Document the estimate  11  Present estimate to management for approval  12  Update the estimate to reflect actual costs and changes  These 12 steps, when followed correctly, should result in reliable and valid cost estimates that management can use for making informed decisions  If any of the steps in the Bureau’s process do not meet, minimally meet, or partially meet the 12 steps, then the cost estimate guidance does not fully reflect best practices for developing a high-quality estimate and cannot be considered reliable  Lastly, to describe key drivers of cost growth, we compared cost information included in the 2015 and 2017 cost estimates  We analyzed both summary and detailed cost information to assess key changes in totals overall, by WBS category, and by information technology (IT) vs  Non-IT costs  We used this analysis in conjunction with information received from the Bureau during interviews and through document transfers to describe overall changes in the cost estimate from 2015 to 2017  We conducted this performance audit from December 2017 to August 2018 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  In addition to the contact named above, Lisa Pearson (Assistant Director), Karen Cassidy (Analyst in Charge), Brian Bothwell, Jackie Chapin, Ann Czapiewski, Jason Lee, Ty Mitchell, Kayla Robinson, and Tim Wexler made significant contributions to this report 
Report 2:  Japan's NYK line, a large ship owner, recently announced that it had reached an agreement with three Japanese utilities to add LNG bunkering to ports in Western Japan  Asian countries, together with Australia and the United Arab Emirates, currently have around 10 coastal ports offering LNG bunkering, with another 15 projects in development  Some LNG bunkering operations in Europe and Asia are associated with existing LNG marine terminals, which already have LNG storage and port infrastructure in place  However, many smaller operations—including most of the projects in development—employ trucking, dedicated bunkering vessels, on-site liquefaction, and other means to extend LNG availability beyond the ports with major LNG terminals  LNG bunkering is not so advanced in South America, although with nine operating LNG marine terminals (one for export), and another six in development, South America also could support significant LNG bunkering operations in the near future  LNG bunkering in the United States currently takes place in two locations—Jacksonville, FL, and Port Fourchon, LA—with a third bunkering facility under development in Tacoma, WA  The LNG facilities in these ports serve the relatively small U S -flag domestic market  Bunkering of LNG-fueled cruise ships also is planned for Port Canaveral, FL  However, ports in North America have significant potential to expand the nation's LNG bunkering capability  Jacksonville is the largest LNG bunkering operation at a U S  port  One bunkering facility at the port, developed by JAX LNG, initially began truck-to-ship refueling operations in 2016 for two LNG-capable container ships  (The LNG is sourced from a liquefaction plant in Macon, GA  ) In August 2018, upon delivery of the Clean Jacksonville bunker barge, the facility began to replace truck-to-ship bunkering with ship-to-ship bunkering  In the future, the barge plans to source LNG from a new, small-scale liquefaction plant which JAX LNG is currently constructing at the port  A second facility at Jacksonville's port, operated by Eagle LNG, provides LNG bunkering sourced from a liquefaction plant in West Jacksonville  Eagle LNG also is constructing an on-site liquefaction and vessel bunkering facility in another part of the port, expected to begin service in 2019  Taken together, the JAX LNG and Eagle LNG facilities is expected to establish Jacksonville as a significant LNG-bunkering location with the capability to serve not only the domestic fleet but larger international vessels as well  In 2015, Harvey Gulf International Marine (Harvey) began LNG bunkering operations in the Gulf of Mexico to fuel its small fleet of LNG-powered offshore supply vessels serving offshore oil rigs  Harvey has since constructed a $25 million facility at its existing terminal in Port Fourchon to store and bunker LNG sourced from liquefaction plants in Alabama and Texas  The facility can provide truck-to-ship bunkering services for LNG-fueled offshore supply vessels, tank barges, and other vessels  A Harvey subsidiary has ordered two LNG bunkering barges to enable ship-to-ship fueling in the future  Puget Sound Energy has proposed an LNG liquefaction and bunkering facility at the Port of Tacoma, WA  Vessels traveling between Washington and Alaska typically spend the entire journey within the 200-mile North America ECA  Consequently, vessel owners operating along these routes have been interested in LNG as bunker fuel  TOTE Maritime, for example, a ship owner involved in trade between Alaska and the lower 48 states, has begun the process of retrofitting the engines of two of its container ships to be LNG-compatible  The proposed Tacoma LNG facility would be capable of producing up to 500,000 gallons of LNG per day and would include an 8 million gallon storage tank  The facility would serve the dual purposes of providing fuel for LNG-powered vessels and providing peak-period natural gas supplies for the local gas utility system  Its total construction cost reportedly is expected to be $310 million  Community and environmental concerns have slowed the progress of the proposal, which is still under regulatory review  Puget Sound Energy originally planned to put the LNG facility into service in late 2019; however, permitting issues appear likely to delay its opening until 2020 or later—if it is eventually approved  Q-LNG Transport, a company 30% owned by Harvey, has placed orders for two LNG bunkering barges to provide ship-to-ship LNG fueling as well as "ship-to-shore transfers to small scale marine distribution infrastructure in the U S  Gulf of Mexico and abroad " Q-LNG's first barge initially is expected to provide fuel to new LNG-fueled cruise ships based in Port Canaveral (and, potentially, Miami), while service from its second barge is still uncommitted  Initial plans are for the LNG to be sourced from the Elba Island LNG import/export terminal near Savannah, GA—approximately 230 nautical miles away—although the company may seek to develop an on-site LNG storage facility in the future  As noted above, U S  LNG bunkering activities thus far have been limited to a handful of vessels in domestic trade and tourism  LNG bunkering for the much larger fleet of foreign-flag ships carrying U S  imports and exports is still to be developed  As in Europe and Asia, domestic ports located near major LNG import or export terminals may serve as anchors for expanded LNG bunkering operations  Figure 4 shows existing LNG import and export terminals in North America  LNG can be liquefied from pipeline natural gas (or imported natural gas) and stored in large quantities at these facilities  The LNG can then be bunkered on site or transported to bunkering facilities elsewhere in the region by truck, rail, or barge  As discussed above, the distance between Port Canaveral and Elba Island in Q-LNG's bunker sourcing plan is 230 nautical miles  Taking this distance as a measure of how far away LNG can be sourced and barged economically, it is possible to extrapolate which U S  ports are within reach of a potential supply of LNG for vessel bunkering  Table 1 lists the top 20 U S  container shipment ports in the United States and their proximity to existing LNG import/export terminals  Of these top 20 ports, 12 are less than 230 nautical miles from an operating LNG terminal  Distances between LNG terminals and the other East Coast ports are not much greater, suggesting that LNG for vessel bunkering could be within reach of every U S  port along the Eastern Seaboard and in the Gulf of Mexico  On the West Coast, the ports of Los Angeles and Long Beach—the two largest U S  ports—are relatively close to the Costa Azul LNG import terminal in Ensenada, MX  Seattle and Tacoma are far from Ensenada, but would be served by the proposed Tacoma LNG bunkering project, if constructed  LNG bunkering for Seattle and Tacoma alternatively could be sourced from an existing LNG port facility around 100 nautical miles north in Vancouver, BC, which is expanding to provide LNG bunkering services to international carriers  Alaska's existing LNG export terminal currently is inactive, but potentially could supply LNG bunker fuel in the Pacific Northwest as well  Although existing LNG import or export terminals in North America could supply LNG for regional bunkering operations, such activities would require additional investment for infrastructure such as LNG transfer facilities and bunker barges  CRS is not aware of any public announcements among the LNG terminals above to develop bunkering operations  However, at least one LNG terminal owner, Cheniere Energy, which operates LNG terminals in Louisiana and Texas, identifies vessel bunkering as one source of future LNG demand growth worldwide  The IMO adopted safety standards for ships using natural gas as a bunker fuel in 2015  The standards, which took effect in 2017, apply to all new ships and conversions of ships (except LNG tankers, which have their own standards)  The IMO standards address engine design, LNG storage tanks, distribution systems, and electrical systems  They also establish new training requirements for crews handling LNG and other low flashpoint fuels  As is the case for the sulfur standards, the IMO LNG safety standards apply to all IMO member nations, including the United States  In addition, a number of U S  federal agencies, especially the Coast Guard and the Federal Energy Regulatory Commission, have jurisdiction over specific aspects of domestic LNG storage infrastructure and bunkering operations  The Coast Guard has the most prominent role in LNG bunkering, given its general authority over port operations and waterborne shipping  In 2015, the Coast Guard issued two guidelines for the handling of LNG fuel and for waterfront facilities conducting bunkering operations  In 2017, the Coast Guard issued additional guidelines to Captains of the Port, the local Coast Guard officials responsible for port areas, for conducting safe LNG bunkering simultaneously with other port operations  The guidelines advise on quantitative risk assessment of facilities bunkering LNG, which allows Captains of the Port to assess the risks posed to crews and facilities  The Federal Energy Regulatory Commission (FERC) plays a role in LNG bunkering due to its jurisdiction over the siting of LNG import and export terminals under the Natural Gas Act of 1938  Specifically, FERC asserts approval authority over the place of entry and exit, siting, construction, and operation of new LNG terminals as well as modifications or extensions of existing LNG terminals  Notwithstanding this siting authority, FERC reportedly does not intend to assert jurisdiction over the permitting of LNG bunkering facilities, but it may require amendment of permits it has issued for LNG import or export terminals to account for bunkering operations added afterwards  In addition to the Coast Guard and FERC, other federal agencies may have jurisdiction over specific aspects of LNG bunkering operations in U S  ports under a range of statutory authorities  For example, the Pipeline and Hazardous Materials Safety Administration within the Department of Transportation regulates the safety of natural gas pipelines and certain associated LNG storage facilities (e g , peak-shaving plants)  LNG facilities also may need to comply with the Occupational Safety and Health Administration's regulations for Process Safety Management of Highly Hazardous Chemicals  Other federal agencies, including the Environmental Protection Agency, the U S  Army Corps of Engineers, and the Transportation Security Administration, may regulate other aspects of LNG bunkering projects  CRS is not aware of new regulations to date among these agencies specifically addressing LNG bunkering  World production of LNG has been rising rapidly over the last few years, driven by growth in the natural gas sector in new regions—especially Australia and the United States  According to one industry analysis ( Figure 5 ), global LNG supply is expected to increase from 300 to 400 million metric tons per annum (MMtpa) from 2017 to 2021 based on new LNG liquefaction projects already operating or under development  An additional 150 MMtpa appears likely to come online after that  Collectively, LNG supply from these new liquefaction projects could exceed projections of demand, which would put downward pressure on LNG prices  While increases in the global supply of LNG do not necessarily translate directly into an increase in LNG available for bunkering, such increases could provide options for LNG bunkering in more ports  Estimating potential demand for LNG in the maritime sector is complicated and uncertain  One study of future LNG demand for bunkering, specifically, projects that LNG-powered vessels in operation and under construction as of June 2018 will require between 1 2 and 3 0 MMt of LNG per year  The study's review of several LNG consumption forecasts in the maritime sector shows a consensus projection between 20 to 30 MMt per year by 2030  This level of demand growth implies an increase in LNG-powered vessel construction from the current rate of around 120 ships per year to between 400 and 600 new builds per year  If these levels were reached, they could create a significant new market for LNG suppliers  Assuming a Henry Hub spot market price of $4/MMBtu in 2030, the annual market for LNG in shipping could be worth $2 9 billion to $5 8 billion, before accounting for liquefaction and transportation charges  Some studies have projected the LNG bunkering market to be even larger and to grow more quickly  However, key variables—such as the prices of Henry Hub natural gas and crude oil, the number of new vessel orders, and the future costs of emissions technology—are notoriously hard to predict with accuracy  Thus, it is not assured that natural gas consumption in the maritime sector will absorb more than a small amount of the global liquefaction capacity in development  The IMO sulfur standards apply to ship owners globally, as does the development of new LNG supply and bunkering infrastructure  In addition to these factors, domestic LNG bunkering also may be influenced by considerations more specific to the United States  These considerations include growth of the U S  natural gas supply, domestic shipbuilding opportunities, and LNG safety and security  Because of its leading role in global natural gas production, the United States has a particular interest in any new source of natural gas demand  According to the Energy Information Administration, the United States has been the world's top producer of natural gas since 2009, when it surpassed Russia  In 2017, increases in production outstripped increases in domestic gas consumption, leading to the United States becoming a net exporter of natural gas for the first time in nearly 60 years  As discussed above, North America (primarily the United States) is expected to add the most new LNG production capacity through 2030 when including projects that are operating, under construction, and likely (according to investment analysts)  Past increases in U S  LNG exports were driven by greater throughput at the Sabine Pass LNG export terminal—the only operating U S  LNG export terminal in 2017  In March 2018, the Cove Point terminal in Maryland became the second operating U S  LNG export terminal  Four additional projects under construction or commissioning are set to nearly triple U S  liquefaction by the end of 2019  This increase in liquefaction capacity likely will motivate LNG producers to secure new buyers  Figure 6 shows estimated LNG prices for various locations around the world as of October 2018  As the figure shows, LNG prices are substantially lower in North America than in Asia, Europe, and South America  Even after adding $1 00 to $2 00/MMBtu to transport the LNG to overseas ports, LNG produced in the United States is globally competitive at these prices  If LNG from the new liquefaction capacity coming online can be produced and delivered with similar economics, the cost advantage may create an opportunity for U S  LNG in bunker supply  There are over 400 petroleum fuel bunkering ports in the world, but 60% of bunkering in recent years has happened in six countries: Singapore, the United States, China, the United Arab Emirates, South Korea, and the Netherlands  Of these countries, only the United States is a significant LNG producer  Therefore, the United States could be a favorable source of LNG for domestic bunkering and for bunkering at the other major ports  While the LNG industry historically has had a good safety record, there are unique safety risks associated with LNG in vessel operations  Leakage of LNG during LNG shipping or bunkering can pose several hazards  LNG is stored at temperatures below -162 °C (-260 °F), far below the -20°C at which the carbon steels typically used in shipbuilding become brittle  Consequently, extreme care must be taken to ensure that LNG does not drip or spill onto ship hulls or decking because it could lead to brittle fracture, seriously damaging a ship or bunkering barge  LNG spilled onto water can pose a more serious hazard as it will rapidly and continuously vaporize into natural gas, which could ignite  The resulting "pool fire" would spread as the LNG spill expands away from its source and continues evaporating  A pool fire is intense, far hotter and burning far more rapidly than oil or gasoline fires, and it cannot be extinguished; all the LNG must be consumed before it goes out  Because an LNG pool fire is so hot, its thermal radiation may injure people and damage vessels or property a considerable distance from the fire itself  Many experts agree that a large pool fire, especially on water, is the most serious LNG hazard  Leaks of boil-off gas (the small amount of LNG that vaporizes in storage) can also release natural gas into a port area and cause fires or explosions  Major releases of LNG from large LNG carriers would be most dangerous within 500 meters of the spill and would pose some risk at distances up to 1,600 meters from the spill  While a bunkering barge or a vessel using LNG for fuel contains far less LNG than large LNG carriers, LNG spills in bunkering operations could still be a significant concern  Risks associated with bunkering LNG are complicated in ports seeking to engage in "simultaneous operations" during the bunkering process  Simultaneous operations entail loading and unloading cargo and personnel from a ship, maintenance, and other logistical operations performed while a ship is bunkering  Accidents that occur during such operations (for example, the operation of heavy machinery near pipes transporting LNG) can result in a spill of LNG which can threaten workers positioned near the site of operations  LNG tankers, bunkering vessels, and land-based facilities could be vulnerable to terrorism  Bunkering tanks or vessels might be physically attacked to destroy the LNG they hold—and vessels might be commandeered for use as weapons against port or coastal targets  Potential terrorist attacks on LNG terminals or tankers in the United States have long been a key concern of the public and policymakers in the context of large scale LNG imports or exports because such attacks could cause catastrophic fires in ports and nearby populated areas  For example, a 2007 report by the Government Accountability Office stated that, "the ship-based supply chain for energy commodities," specifically including LNG, "remains threatened and vulnerable, and appropriate security throughout the chain is essential to ensure safe and efficient delivery " Affected communities and federal officials continue to express concern about the security risks of LNG  The potential risks from terrorism to LNG bunkering infrastructure may be different than those of larger LNG import or export operations due to smaller quantities of LNG involved, but the risks may become more widespread if LNG bunkering operations are established in more locations  The Maritime Transportation Security Act (MTSA, P L  107-295 ) and the International Ship and Port Facility Security Code give the Coast Guard far-ranging authority over the security of hazardous materials in maritime shipping  The Coast Guard has developed port security plans addressing how to deploy federal, state, and local resources to prevent terrorist attacks  Under the MTSA, the Coast Guard has assessed the overall vulnerability of marine vessels, their potential to transport terrorists or terror materials, and their use as potential weapons  The Coast Guard has employed these assessments to augment port security as necessary and to develop maritime security standards for LNG port facilities  The IMO's overall framework for controlling vessels emissions (MARPOL Annex VI) has been in place since 2005  While the United States, as an IMO member, is subject to the IMO's 2020 sulfur standards, the international standards apply equally to all parties and all vessels  The impacts of sulfur standards on bunker fuel have been an important consideration, but IMO member nations have agreed to the standards independent of any particular energy policies  Moreover, MARPOL Annex VI preceded the U S  shale gas boom, so commitment to that initial IMO framework could not have anticipated United States' current role as a dominant energy producer  Any changes within the international shipping fleet to install sulfur scrubbers, fuel engines with LNG, or switch to other low sulfur fuels, are being driven primarily by market forces in fuel supply, shipbuilding, and shipping—not by any particular push to favor one fuel over another  Nonetheless, given its particular status, the question arises whether the standards may create an economic opportunity for the United States, in energy or otherwise  More specifically, could international adoption of LNG as a bunker fuel create an important new market for U S  natural gas producers, shipbuilders, or infrastructure developers? As discussed above, depending upon the adoption of LNG bunkering in the global fleet, the LNG bunker fuel market could grow to several billion dollars by 2030  If U S  LNG producers were to supply a significant share of this market—on the strength of comparatively low LNG production costs—LNG bunkering could increase demand for U S  natural gas production, transportation, and liquefaction  Opportunities in LNG-related shipbuilding might be more limited, as most of this occurs overseas, with the exception of Jones Act vessels  In the latter case, demand for domestically-constructed LNG bunkering barges could be one significant area of economic growth  Engineering and construction firms could benefit from new opportunities to develop new port infrastructure for LNG storage and transfer  While likely limited in number, such port facilities could be complex, high value projects costing tens or hundreds of millions of dollars to complete  Such projects could create jobs in engineering, construction, and operation, which could be important to local communities  Although LNG bunkering could present the United States with new economic opportunities, it may pose challenges as well  Rising demand for LNG in the maritime sector could increase natural gas prices for domestic consumers  In addition to being the world's largest natural gas producer, as of 2018, the United States is also the world's largest producer of crude oil and the second largest bunkering hub  Consequently, while vessel conversion to LNG bunkering may increase demand for U S -produced natural gas, it could be partially offset by reduced demand for U S -produced crude oil or refined products  Exactly how changing demand in one sector could affect the other is unclear  Furthermore, while LNG can reduce pollutant emissions from vessels, emissions and environmental impacts from increased natural gas production and transportation could increase overall emissions  Much of the net environmental impact depends upon practices in the natural gas industry, which are the subject of ongoing study and debate  Although new LNG bunkering infrastructure can create jobs, as the Tacoma LNG projects shows, the construction of such port facilities can be controversial for reasons of safety, security, and environmental impact  Overarching the considerations above is uncertainty about how the global shipping fleet will adapt to the IMO sulfur standards over time  This uncertainty complicates decisions related to both private investment and public policy  LNG-fueled ships still account for only a fraction of the U S  and global fleets, and it may take several decades for significant benefits of LNG-powered vessels to be realized  It is also possible that alternative ship fuels, including biofuels, electric engines, and hybrid engines, will become more economically viable in coming years  Given the uncertainty surrounding the future of LNG as a ship fuel, it is hard to predict the potential benefits or costs that LNG bunkering may provide to the United States  Until now, the private sector has added LNG-fueled vessels to fleets in the United States in a piecemeal manner under existing federal statutes and regulation  Congress could encourage the growth of LNG bunkering by various means, such as providing tax incentives to support the construction of LNG bunkering facilities and vessels, addressing any statutory or regulatory barriers to bunkering facility siting or operations, and providing funding for technical support to domestic carriers seeking to adopt LNG technology  Alternatively, Congress could seek to encourage competing bunker fuel options, such as biofuels, by incentivizing them in similar ways  In addition, Congress could also affect growth in LNG bunkering through policies affecting the LNG industry or domestic shipping industry as a whole  Changes in federal regulation related to natural gas production, or changes to the Jones Act, for example, while not directed at LNG bunkering, could nonetheless affect its economics  Therefore, evaluating the potential implications on LNG bunkering of broader energy, environmental, or economic objectives may become an additional consideration in congressional oversight and legislative initiatives  If LNG bunkering expands significantly in U S  ports, Congress also may examine the adequacy of existing measures to ensure the safety and security of LNG vessels, storage, and related facilities 

ground_truth: In October 2017, the Department of Commerce (Commerce) announced that the projected life-cycle cost of the 2020 Census had climbed to $15.6 billion, a more than $3 billion (27 percent) increase over its 2015 estimate. A high-quality, reliable cost estimate is a key tool for budgeting, planning, and managing the 2020 Census. Without this capability, the Bureau is at risk of experiencing program cost overruns, missed deadlines, and performance shortfalls. GAO was asked to evaluate the reliability of the Bureau's life-cycle cost estimate. This report evaluates the reliability of the Bureau's revised life-cycle cost estimate for the 2020 Census and the extent to which the Bureau is using it as a management tool, and compares the 2015 and 2017 cost estimates to describe key drivers of cost growth. GAO reviewed documentary and testimonial evidence from Bureau officials responsible for developing the 2020 Census cost estimate and used its cost assessment guide ( GAO-09-3SP ) as criteria. Since 2015, the Census Bureau (Bureau) has made significant progress in improving its ability to develop a reliable cost estimate. While improvements have been made, the Bureau's October 2017 cost estimate for the 2020 Census does not fully reflect all the characteristics of a reliable estimate. (See figure.) Specifically, for the characteristic of being well-documented, GAO found that some of the source data either did not support the information described in the cost estimate or was not in the files provided for two of its largest field operations. In GAO's assessment of the 2015 version of the 2020 Census cost estimate, GAO recommended that the Bureau take steps to ensure that each of the characteristics of a reliable cost estimate is met. The Bureau agreed and has taken steps, but has not fully implemented this recommendation. A reliable cost estimate serves as a tool for program development and oversight, helping management make informed decisions. During this review, GAO found the Bureau used the cost estimate to inform decision making. Factors that contributed to cost fluctuations between the 2015 and 2017 cost estimates include: Changes in assumptions. Among other changes, a decrease in the assumed rate for self-response from 63.5 percent in 2015 to 60.5 percent in 2017 increased the cost of collecting responses from nonresponding housing units. Improved ability to anticipate and quantify risk. In general, contingency allocations designed to address the effects of potential risks increased overall from $1.3 billion in 2015 to $2.6 billion in 2017. An overall increase in information technology (IT) costs. IT cost increases, totaling $1.59 billion, represented almost 50 percent of the total cost increase from 2015 to 2017. GAO is not making any new recommendations but maintains its earlier recommendation—that the Secretary of Commerce direct the Bureau to take specific steps to ensure its cost estimate meets the characteristics of a high-quality estimate. In its response to this report, Commerce generally agreed with the findings related to cost estimation improvements, but disagreed that the cost estimate was not reliable. However, until GAO's recommendation is fully implemented the cost estimate cannot be considered reliable.