prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 1.

Report 1: The Department of Defense (DOD) obligates more than $300 billion annually to pay for goods and services (including research and development)  Most of these acquisitions are governed by numerous statutes and regulations found in Title 10 of the United States Code, the Federal Acquisition Regulation (FAR), and the Defense Federal Acquisition Regulation Supplement (DFARS)  DOD can also enter into certain transactions without triggering most of the standard acquisition statutes and regulations by using other transaction (OT) authorities  In recent years, Congress has expanded these authorities and DOD is increasingly using OTs for research, prototyping, and production  This report examines (1) how OTs work, (2) why they were established, (3) potential benefits and risks of using OTs, and (4) whether there are data available against which to measure their effectiveness  Appendix A provides a legislative history of DOD's other transaction authorities  On October 4, 1957, the Soviet Union triggered a space race with the United States when it successfully launched Sputnik I into orbit, becoming the first nation to send a man-made satellite into space  Congress, concerned that the United States was falling behind in space, held a series of hearings on an "emergency" effort to respond to the Soviet launch of Sputnik  At the same time, a bill was introduced in the Senate to create an agency with the means to quickly and efficiently develop a national space program  These efforts led to passage of the National Aeronautics and Space Act of 1958 (Space Act, P L  85-568) in July 1958, which established the National Aeronautics and Space Administration (NASA)  In an effort to give the new agency "the necessary freedom to carry on research, development, and exploration     to insure the full development of these peaceful and defense uses without unnecessary delay," the Space Act granted NASA broad authority to "enter into and perform such contracts, leases, cooperative agreements, or other transactions as may be necessary" to accomplish its mission of research and exploration (emphasis added)  Congress extended different variations of OT authorities to other select agencies, granting the authority to DOD in the FY1990 & 1991 National Defense Authorization Act (NDAA, P L  101-189 )  For an analysis of which non-DOD agencies can use OTs and how the authorities differ by agency, see Appendix B   Other transactions are legally binding contracts that are generally exempt from federal procurement laws and regulations such as the Competition in Contracting Act and the Federal Acquisition Regulation  In contrast, traditional procurement contracts must adhere to the procurement rules set forth in statute and regulation  Generally, DOD can use other transaction authorities for three purposes: 1  conduct research, 2  develop prototypes, or 3  contract for follow-on production of a successful prototype project  DOD's other transaction authorities are found in two sections of law: 10 U   S   C   2371 grants DOD the authority to use other transactions to carry out basic, applied, and advanced research projects  DOD regulations treat these projects as financial assistance instruments, not as contracts  10 U   S   C   2371b permits the use of other transactions to conduct prototype projects and follow-on production  OTs can only be used for prototypes if one of the following applies: at least one nontraditional defense contractor significantly participating in the project; all significant participants are small businesses or nontraditional defense contractors; at least one-third of the total cost of the prototype project is provided by nongovernment participants; or the senior procurement acquisition official provides in writing an explanation of the exceptional circumstances justifying an OT  Follow-on production can only be conducted when the underlying prototype OT was competitively awarded, and the prototype project was successfully completed  10 U   S   C   2373 , while generally not considered an o ther t ransaction authority , allows DOD to buy certain items and designs for experimental or test purposes without having to adhere to the procurement laws set forth in Chapter 137 of Title 10  OT authorities grant government officials the flexibility to include, amend, or exclude contract clauses and requirements that are mandatory in traditional procurements (e g , termination clauses, payments, audit requirements, intellectual property, and contract disputes)  OTs can be structured in numerous ways, including a direct relationship between a single government agency and a single provider; joint ventures; partnerships; multiple agencies joining together to fund an agreement encompassing multiple providers; or through a consortium  One common application of OTs is to forge an agreement with a consortium  A consortium is an organized group— it can consist of nonprofits, academic institutions, or contractors— focusing on a specific technology area  Generally, a lead entity coordinates and directs a consortium's activities  Consortia have consisted of a handful to as many as 1,000 members  Consortia can act to facilitate multiparty agreements whereby each member is akin to its own co-prime contractor with the government  In such a case, the government articulates the need or problem it is trying to solve, and the various members of the consortium can submit white papers for consideration  In this scenario, OTs can serve as an efficient way for all members to send unsolicited technology suggestions and solutions to solve a defined challenge  Consortia can also be used to develop an ecosystem of entities working together on a project, whereby members of a consortium pool resources and collaborate with DOD  A number of analysts argue that using consortia in this way gives the federal government a unique ability to leverage and pool the technological expertise and innovation of multiple entities in a particular sector, thereby strengthening and advancing a sector of the industrial base that may have defense applications  Seen like this, OTs can be considered a mechanism to promote defense technology and the defense industrial base, with the potential added benefit of advancing the domestic commercial technology base  Some analysts, however, have argued that many of today's consortia do not operate as collaborative organizations, but function more like managed multiple award task order contracts  These analysts argue that DOD should seek to foster more collaboration in consortia  Some analysts have argued that consortia reduce competition, since only members of the consortium under contract can participate in the project or submit white papers for consideration and funding  Others counter that even when an OT is signed with a single consortium, competition could be increased, since all members of the consortium are notified of the opportunity (expanding the pool of potential competitors aware of the opportunity), and the OT will foster internal competition among consortium members  OTs are not procurement contracts and thus are exempt from numerous procurement statutes and regulations, including the statutes in Chapter 137 of Title 10 ( Procurement Generally )  They are, however, bound by standard contract and other select laws and regulations  Examples of laws that do not apply include the Truth in Negotiations Act, Competition in Contracting Act, Cost Accounting Standards, Contract Disputes Act, and select intellectual property statutes such as the Bayh-Dole Act  A number of these laws are aimed at oversight and protecting the interests of taxpayers  While the Competition in Contracting Act does not apply, 10 U S C  2371b requires that research and prototype projects be competed "to the maximum extent practicable " DOD policy mirrors the statutory language, stating "[C]ompetition is a good thing  It helps keep prices low, quality high, and gives the government leverage in negotiations " To the extent that OTs induce nontraditional contractors to work with DOD, OTs can be viewed as promoting competition among (and within) entities that would not normally compete for DOD contracts  However, the lack of explicitly defined competition requirements could result in less competition for certain OTs  OTs are not free from all legislative and regulatory requirements  Generally, statutes and regulations that are not procurement-specific apply, including the Trade Secrets Act, (18 U S C  1905); the Economic Espionage Act (18 U S C  1831-39); elements of the Freedom of Information Act (5 U S C  552); and fiscal and property laws, such as the Anti-deficiency Act (31 U S C  1341); and the Tucker Act (28 U S C  1491)  As can be seen by these citations, many of these statutes are not located in the procurement titles of Title 10 or Title 41  The Government Accountability Office (GAO) has held that OTs are not procurement contracts, and therefore it will not review protests of such an award or solicitation  However, GAO will review "a timely protest that an agency is improperly using its other transaction authority " OT contracts can be protested to the Court of Federal Claims, although there is debate as to the extent of the court's jurisdiction  The limited protest jurisdiction of GAO is appealing to many government procurement officials  One senior Air Force official reportedly stated that OTs are "just so much faster and so much more attuned to getting something that we want today and not have to spend a couple of years going through a protest, going through this huge process to get something we wanted two years ago " It generally does not take that long to go through a GAO protest  By statute, GAO must issue an opinion on a protest within 100 days of the protest being filed, and 70% of cases are resolved in less than 60 days  While exempt from GAO bid protests, OTs are not exempt from GAO audits  Generally, OTs that include government payments exceeding $5 million are required to include a clause granting GAO the right to examine the records of any related party  However, this requirement has a number of limitations  Used properly, OTs can provide significant benefits to DOD  Along with the potential benefits come certain risks  Some of the potential benefits to OTs highlighted by analysts and officials include providing a mechanism to pool R&D resources with industry to facilitate development of, and obtain "the latest state-of-the-art, dual use technologies"; attracting nontraditional contractors with promising technological capabilities to work with DOD; lowering costs by eliminating requirements associated with the Federal Acquisition Regulations (i e , costs associated with required reporting and administrative activities) or sharing costs with industry; and "speeding up" the acquisition process  A number of experts argue that OTs provide a unique mechanism for DOD to invest in, and influence the direction of, technology development even when the end result is not directly tied to a military capability  As one observer noted, the real benefit to DOD may be that [t]he R&D has been accomplished and is available to the technical and scientific communities  As a result, a subsequent phase of research can begin or a particular approach can be demonstrated to be of no value  Some argue that OTs have particular import today  Drawing parallels to the space race, these analysts argue that DOD is engaged in a defense technology race  According to the 809 Panel: DoD is now in a period during which the time a particular technology is a dominant force on the battlefield is getting increasingly shorter, disruptive technologies are emerging at a faster pace, and these technologies are more widely dispersed…In a world with rapidly changing technology, time is a valuable resource that must not be taken for granted  It is difficult to predict what capabilities DoD will need 5 to 10 years from now—biotechnology, nanotechnology, artificial intelligence, robotics, or a new technology area not even known today  It also is unclear on what plane the military will conduct warfare—traditional battlefields, space, cyberspace, or some other domain  The current acquisition system lacks the agility needed to adapt to new paradigms  These analysts argue that OTs and similar rapid acquisition authorities are critical for DOD to compete in such a fast-paced global environment where technology and innovation are no longer driven by DOD, but by industry and foreign competitors  In 1960, the United States accounted for 69% of global R&D, with U S  defense-related R&D alone accounting for more than one-third of global R&D  The federal government funded approximately twice as much R&D as U S  business  However, from 1960 to 2016, the U S  share of global R&D fell to 28%, and the federal government's share of total U S  R&D fell from 65% to 24%, while business's share more than doubled from 33% to 67%  As a result of these global, national, and federal trends, federal defense R&D's share of total global R&D fell to 3 7% in 2016  Given the shift in the global R&D landscape, and the diminishing influence of DOD as a market mover, analysts suggest that the current procurement system is overburdened by regulations and bureaucratic processes that slow the system, increase costs, and dissuade companies from doing business with DOD  In contrast, OTs are viewed as faster, attracting companies that would otherwise forgo working with DOD and promoting broader investment in critical defense capabilities  As one analyst wrote: OTAs are currently the only way to remove the barriers necessary to get these nontraditional sources of innovation to do business with the military  Properly constructed, OTAs help speed up the process, respect a company's IP through negotiation rather than regulatory fiat, and result in contracting under commercial terms and conditions  Congress also appears to have shifted its view on appropriate use of other transactions  In the FY1999 conference report, Congress stated that [OT] authority should only be used in the exceptional cases where it can be clearly demonstrated that a normal contract or grant will not allow sufficient access to affordable technologies  By comparison, in the FY2018 NDAA, Congress expanded OT authorities and stated the following: In the execution of science and technology and prototyping programs, the Secretary of Defense shall establish a preference, to be applied in circumstances determined appropriate by the Secretary, for using transactions other than contracts, cooperative agreements, and grants  Along with the potential benefits come potential risks, including that of diminished oversight and exemption from laws and regulations designed to protect government and taxpayer interests  Some analysts, while acknowledging the important role of OTs, raise concerns over transparency and how these agreements are being employed  As one industry official stated, OTs are a contracting method, not a substitute for good acquisition practices  Discussing a particular OT for cloud services that was protested and ultimately cancelled by DOD, one observer argued The cloud contract provides a teachable moment for procurement reform-minded officials in the Pentagon and Capitol Hill  The problem was not with the OTA mechanism, which remains an essential element of reforming Pentagon procurement  Rather, the problem was with a lack of transparency with how the mechanism was employed  Scott Amey, general counsel of the Project on Government Oversight, cautioned We have to seriously consider how we are using [OTs]; whether we are using them as intended, whether we are getting the goods and services that we really want and need, whether we are getting them at the best cost and process, and we are using this procurement vehicle as a way to just circumvent the rules and have contractors not have the administration and oversight they need to hold them accountable  I'm just afraid this is going to result in a lot of waste, fraud, and abuse in the future  Congress has expressed repeated concerns that OTs could be used to circumvent congressional intent  In the FY1999 NDAA, the committees emphasized that the authority should only be used in a limited manner  The conference report stated the following: The conferees are especially concerned that such authority not be used to circumvent the appropriate management controls in the standard acquisition and budgeting process  Congress echoed a similar concern in the FY2019 NDAA  According to the House report: The committee also urges the Department to reiterate through established guidelines that OTA is not a means for circumventing appropriate use of the Federal Acquisition Regulations, and that full and open competition should be used to the maximum extent possible to maintain a sense of integrity, fairness, and credibility in the Federal Procurement process  Other transactions are also exempt from many of the socioeconomic policies put in place by Congress to promote public policies, including some Buy America requirements  Some analysts have raised concerns that OTs are a way to circumvent many of the public policies enshrined in the acquisition process  A number of analysts and officials have raised concerns that if DOD uses OTs in ways not intended by Congress—or is perceived to abuse the authority—Congress could clamp down on the authority  Under Secretary of the Army Ryan McCarthy reportedly stated that the military department is "trying to be very judicious about this authority so we don't lose it " Some analysts argue that Congress is already clamping down on the use of OTs  These analysts point to language in the FY2019 NDAA and FY2019 appropriations legislation ( P L  115-245 ) requiring additional reporting and notification (see Appendix A )  Such notification and reporting requirements, however, are not new; when Congress expanded OT authorities in the past, reporting and notification requirements were commonly included  The reporting requirements may also be a result of congressional frustration with a lack of transparency and data on how DOD uses OTs  DOD lacks authoritative data that can be used to assess OT effectiveness and better understand broader trends associated with these agreements  The most frequently cited source for such data is the Federal Procurement Data System-Next Generation (FPDS-NG), which is the primary source for tracking data on contract obligations, including other transactions for prototypes and follow-on production  FPDS-NG is configured to track data on cost-sharing, other transaction award type, and prevalence of nontraditional contractors  Obligations connected to OTs for research are tracked by the Defense Assistance Awards Data System, which is primarily used to track grants and cooperative agreements  This bifurcation of how OT data are tracked makes it more difficult to get a consolidated view of OT data  According to DOD, all OT data will be reported through FPDS-NG starting in late 2019  The procurement data in FPDS-NG are not fully reliable  There are quality issues relating to accuracy, completeness, and timeliness of data  CRS reviewed FPDS-NG data for prototype OT agreements signed or modified between FY2015 and FY2017 and found similar data inconsistencies  DOD officials acknowledge that they do not have sufficiently reliable data upon which to conduct analysis on the use of OTs and are taking steps to try to improve the data  The analyses below reflect CRS's effort to analyze DOD's use of other transaction authority based on the best available data  According to FPDS-NG, in FY2017, DOD obligated $2 1 billion—and received $360 million in cost-share contributions—on prototype other transaction agreements, representing less than 1% of DOD's total FY2017 contract obligations (approximately $320 billion)  Despite the small percentage of obligations, OTs are growing quickly and are expected to continue to grow at a rapid pace  From FY2013 to FY2017, the number of new prototype agreements increased from 12 to 94, an increase of over 650% (see Table 1 )  DOD's Defense Innovation Unit (DIU, formerly known as the Defense Innovation Unit Experimental, or DIUx) was involved with approximately half of the prototype agreements executed in 2017  DOD officials have stated their intent to further increase the department's use of OTs  Officials say that this increase is due to Congress expanding the statutory authority  The Army executed more than 66% of the prototype OT agreements between FY2013 and FY2017, often on behalf of other military departments and components (see Table 2 )  Army Contracting Command-New Jersey at Picatinny Arsenal executes many of these agreements  DIU currently uses Picatinny Arsenal to execute all of its other transaction agreements  A number of private sector companies do not pursue federal government contracts because they are unwilling to forfeit intellectual property rights or adhere to some of the procurement regulations  One of the goals of OTs is to expand the defense marketplace by creating a mechanism for access to technologies and services of companies that would not otherwise work with DOD, particularly startups and companies developing innovative technology  As one industry representative stated: Because they are "outside" the FAR, OT agreements do not require such cumbersome oversight and audit requirements such as those imposed by the Truth in Negotiations Act, cost and pricing data or an expensive Cost Accounting System (CAS) qualified financial system  CAS compliant financial systems can cost a company millions of dollars to implement and maintain — and are therefore a significant, if not potentially fatal, barrier to government market entry for startups and small, innovative companies  These requirements tend to reinforce the "legacy advantage" of large traditional contractors, who can afford to hire and staff these requirements with large staffs of accountants and lawyers  A number of nontraditional companies told CRS that they are more likely to work with DOD because of the department's other transaction authorities  Despite these claims, some observers question whether OTs are effectively bringing nontraditional contractors into the defense marketplace  A DOD Inspector General report examining other transactions from FY1994 to FY2001 found that OTs did not attract significant numbers of nontraditional defense contractors to do business with DOD  The report found that of the 209 prototype agreements examined, traditional defense contractors received 95% of the $5 7 billion in funds awarded  A recent analysis of FPDS-NG data by Federal News Network had similar findings  According to the report, from FY2015 to FY2017, while nontraditional defense contractors were awarded most of the new OTs (66% vs  33% for traditional defense contractors), the dollar value of the OTs favored traditional contractors ($20 8 billion vs  $7 4 billion for nontraditional contractors)  Some observers have questioned the accuracy of the data published by Federal News Network  According to Charlie McBride, president of Consortium Management Group (which manages two consortia working with DOD through OTs), 88% of the total dollar value of awards to CMG Group has gone to nontraditional prime contractors, and nontraditional entities have participated in the remaining 12%  This debate highlights the lack of authoritative data on OTs  The currently available data may not accurately reflect the extent to which nontraditional contractors are engaged in OT agreements  FPDS-NG does not collect data regarding subcontractors or consortia composition, making it difficult to determine the nature and extent to which nontraditional defense contractors and entities may be working under OT agreements with DOD directly or as subcontractors  When Congress extended OT authority to DOD, it authorized inserting a clause requiring a person or entity to make payments to DOD as a condition of receiving support under the agreement  Such funds were to be merged into an account dedicated to support DARPA advanced research projects  The intent of this provision was to permit DARPA to "recoup the fruits of such arrangements, when there is a 'dual use' potential for commercial application" and reinvest the funds to develop other technologies  In addition to the recoupment authority, DOD can share costs with other parties under an OT  Using this approach, the amount of each party's share is negotiated and incorporated into the agreement  Congress believed that OTs and cooperative agreements were ideal vehicles for promoting DOD-industry collaboration in developing dual-use technologies  For example, the FY1992 & 1993 NDAA ( P L  102-190 ) authorized DOD to enter into cooperative and other transaction agreements to develop critical dual-use technologies as set forth in the Defense Critical Technologies Plan  According to the Senate report:     the United States tends to underinvest in dual-use technologies  National security requirements alone often do not justify major DOD support, and market prospects alone often appear to be too long-term or high risk to justify US industry carrying the entire development burden     The committee encourages use of cooperative agreements and other transactions in lieu of grants or contracts     The provision would require that at least 50 percent of funding over the life of a partnership derive from non-federal sources but would allow for a smaller industry share at the start  In the 1990s, some DARPA OTs required participants to share costs because the types of work completed generally involved R&D that was mutually beneficial to government needs and industry commercial goals  In certain instances, present-day OTs have cost-sharing requirements that foster collaborative research  Some analysts believe that DOD is not always realizing all the benefits that OTs have to offer, such as sufficiently leveraging private capital or forming true consortia of multiple parties pooling resources  Some of these analysts believe that DOD does not sufficiently use consortia to leverage private investment through the pursuit of collaborative, mutually beneficial, dual-use technologies  Some observers argue that as the legislation on OTs has evolved, the cost-sharing provision for prototype projects has come to create an unfair playing field biased against traditional defense contractors  For prototype projects, traditional contractors generally are required to assume one-third of costs whereas nontraditional defense contractors and small businesses generally do not have to cost share  From a fairness perspective, these observers argue that traditional contractors should not have a mandatory cost share  These observers also point out that some nontraditional defense contractors are companies with billions of dollars of revenue that should not be granted a competitive cost advantage  Putting traditional defense contractors at a competitive disadvantage could deny DOD access to those companies with the most experience working on defense products, potentially depriving the military of access to leading defense-related research and technology  Other observers argue that the cost share as currently structured is appropriate: traditional defense contractors hold a significant competitive edge in their understanding of, and have the systems in place to manage, traditional contracts  In contrast, nontraditional and small businesses, which generally cannot compete with the traditional defense contractors, need the exemption from the cost-share requirement to be able to work with DOD  These observers also argue that traditional contractors are awarded the majority of dollars obligated to OTs, proving how difficult it is for nontraditional suppliers to break into the defense marketplace  Additionally, traditional contractors could avoid the cost-sharing requirement by teaming with a nontraditional contractor  DOD has not effectively tracked data on cost sharing  A 2017 report to Congress indicated that in FY2016 DOD obligated $1 billion for prototype agreements and received $68 million in cost-share contributions  A CRS review found numerous concerns with the data underlying the report, and with DOD's analytical conclusions  See Appendix C for further discussion  A number of analysts and industry officials have raised concerns that DOD could use OTs to avoid competitions, as OTs are exempt from the Competition in Contracting Act  According to statute, follow-on production using other transaction authority can only be awarded if the underlying R&D agreement was competed  In addition, 10 U S C  2371b states that "to the maximum extent practicable" OTs must be competed  Determining whether OTs are being used to circumvent competition requires a two-step analysis: 1  Are OTs competed less often that traditional contracts? 2  Is there a benefit to these OTs being competed? Available FPDS-NG data suggest that DOD is broadly complying with 10 U S C  2371b's competition mandate  Between FY2013 and FY2017, approximately 89% of all new OT prototype agreements were competed in some fashion  Many observers and analysts believe that OT agreements can be executed substantially faster, sometimes in a matter of weeks, compared to the months or years it typically takes to execute traditional contracts  Based in part on this belief, some officials and analysts are touting OTs as a new model for conducting acquisitions and the answer to many of the problems in defense acquisition  These analysts argue that in a world of increasingly fast technology development, DOD acquisitions must go faster or risk being left behind  Many acquisition professionals argue that OT contracts are not inherently faster than traditional contracting; instead, they are executed faster because they are not encumbered by the reviews, protests, and bureaucratic layers that have been overlaid on traditional contracting  According to these officials, OTs take just as long as traditional contracts if the same execution and oversight processes are applied  And because all terms are negotiable, complex negotiations could make OTs take longer to execute than traditional contracts that have required, nonnegotiable conditions  The Other Transactions Guide states The OT award process will not always be faster than the traditional procurement processes and sometimes can be as long or longer  The speed of award is tied to many factors, many of which are internal to the organization  DOD has not tracked data on the relative time it takes to execute OTs vs  traditional contracts, making it impossible to objectively assess these claims  Analysts and officials generally agree that the workforce plays a critical role in determining the success or failure of an acquisition  Because there are fewer predefined requirements, OTs can be more difficult to negotiate than traditional contracts, putting DOD at greater risk of not getting what it wants at a reasonable price  The complexity and difficulty of negotiations is particularly high when there are intellectual property/patent rights issues, as is the case with most OTs  Given these challenges, OTs often require more experienced and capable government representatives to ensure implementation of agreements that are in the government's best interest  Some analysts question the extent to which the workforce is sufficiently trained and equipped to negotiate OTs  In response to this concern, in the FY2018 NDAA, Congress required workforce education and training for OTs, and required DOD to establish a cadre of intellectual property experts to advise, assist, and provide resources to program offices that are developing intellectual property strategies for contracts and agreements (see Appendix A )  DOD officials acknowledge that more training and education is required  Given the complexity of OTs and the limited extent to which they are used, some analysts and industry officials suggested that there may be a benefit to establishing a centralized office within DOD responsible for executing or overseeing all other transaction agreements  Such a structure could help ensure that those members of the acquisition workforce engaged in other transactions are sufficiently experienced, trained, and qualified  A number of analysts have argued that DOD should take steps to improve its use of OTs  Many of these analysts have suggested that data are not consistently and accurately tracked, regulations and guidance on when and how to use OTs are vague or insufficient, and the workforce is not sufficiently prepared to effectively use OTs  A number of officials have acknowledged these shortcomings and DOD is reportedly taking steps to address them  For example, in December 2018, DOD issued an updated Other Transactions Guide , a comprehensive guide containing best practices, case studies, and a clarification of myths related to other transaction authorities  In addition, Defense Acquisition University developed new course materials addressing OTs and is working to expand its offerings of relevant training and classes  However, numerous acquisition officials question whether it is possible, or even desirable, to try to quickly implement training aimed at preparing the thousands of DOD acquisition officials to execute OTs  Some of these officials have suggested it might be appropriate to only allow a limited and vetted number of acquisition professionals to be OT agreements officers  Some argue that OTs are just one "tool in the tool box," appropriate for only specific types of contracts, and should not be used to avoid the statutory and regulatory framework or to try to accelerate the process just for the sake of speed  Others have suggested that OTs should be used to cut through bureaucracy, speed up the acquisition process, avoid regulations and bid protests, and perhaps eventually supplant the regular FAR-based contracting process  Given the benefits and risks associated with OTs, questions for Congress include the following: 1  To what extent and in what circumstances do the potential benefits of OTs in terms of cost, schedule, and added capabilities outweigh concerns over potential fraud, waste, abuse, diminished oversight, and other public policy objectives? 2  Should OT authorities be extended further, curtailed, or maintained? The FY2019 NDAA required DOD to submit a report annually through 2021, summarizing DOD's use of OTs, including organizations involved; number of transactions; amounts of payments; and purpose, description, and status of projects  The NDAA also required the Defense Innovation Unit to submit a report to Congress, to include the number of traditional and nontraditional defense contractors with DOD contracts or other transactions resulting directly from the unit's initiatives  The conference report for the FY2019 defense appropriations bill included language expressing the conferees' "[concern] with the lack of transparency surrounding the employment of OTA, particularly for follow-on production " The conferees directed DOD to provide quarterly reports to the House and Senate appropriations committees listing each active OT, and to include additional information for each agreement  The conferees also directed GAO to review DOD's use of OTs to determine whether the "employment of this authority conforms to applicable statutes and guidelines, to include the identification of any potential conflicts " GAO was also required to report on the extent to which OTs have been used since FY2016  The multitude of reporting requirements, and the questionable reliability of the available data, raise a number of questions that Congress may wish to explore, such as the following: 1  To what extent, if any, should the current reporting requirements be consolidated to create a more streamlined and consistent flow of information to Congress? 2  What specific data does Congress need in these reports to effectively conduct oversight? For example, what percentage of research OTs result in prototype projects and follow-on production? 3  To what extent are the data sufficiently reliable, and will such data be easily retrievable in the future, to allow Congress to conduct effective, timely, and ongoing oversight? If the data are not sufficiently reliable or accessible in the future, what other data collection and tracking methods could Congress mandate to ensure ongoing access to reliable data? 4  How are OTs being used? Where and when in the acquisition lifecycle is the authority being used? To what extent is the requirements process being circumvented when DOD awards an OT follow-on production contract for a major system? One analyst suggested that "no efficiency is lost if only the most able personnel are authorized to procure and administer" OT agreements and further argued that expanding the use of OTs would increase the training costs by expanding the number of people who can "weave complex agreements in a relatively unstructured environment " Congress may consider whether DOD should establish an acquisition innovation lab or center of excellence responsible for overseeing, executing, and approving all OTs across the department  Such a lab or center could be staffed and supported by a cadre of professionals with experience across the acquisition lifecycle who have a willingness and ability to embrace new ideas and rethink existing practices  Alternatively, such labs or centers could be established in the military departments  Having centers in each organization could allow for consideration of the different missions and business approaches of the departments and help educate the workforce on a more systematic basis  Proponents argue that such an office would help ensure that only experienced and capable officials, with the appropriate training, use OT authorities  Such an office could also help protect against layering internal DOD policies and bureaucracies onto OTs by placing OTs outside of the traditional bureaucratic acquisition process  Proponents could further argue that such an office could better propagate best practices and ensure that OTs are used appropriately and are consistent with guidance and legislation  Having a single office responsible for executing or approving all OTs could also help ensure more timely and accurate information, giving Congress more visibility into DOD's use of other transaction authorities  To the extent that a single, high-level official is responsible for managing and overseeing all OTs, Congress might wish to consider repealing or modifying existing statutory approval requirements  If such an office was able to provide timely and accurate information, Congress might also consider some of the current reporting requirements unnecessary, and may choose to repeal some of the reporting requirements  Opponents of such a proposal argue that centralizing OTs would have the opposite effect, increasing bureaucracy by adding yet another office within DOD  Opponents also argue that such an office could make it more time-consuming to get a project underway and may discourage program offices from attempting or suggesting OTs  Some also argue that a single office may not have the resources to execute and approve agreements in a timely manner, and would inhibit spreading expertise on how to execute OT agreements more broadly across the acquisition workforce  Even proponents who might in theory support establishing such an office could raise significant concerns regarding how such an office would function in practice  A number of alternative options could be pursued to address concerns raised by opponents of establishing a centralized office  Some of these alternative options include the following: Granting such an office primary, but not exclusive, authority to execute OTs  For example, agreement officers specifically authorized to do so could execute OTs, with the centralized office conducting a peer review  Under this construct, the office could also be charged with providing information and expertise/consulting services on the use of OTs to program offices contemplating using the authorities  Establishing centers within each military department, with a designated office in OSD serving a coordinating function (with nondelegable approval authority residing in the military department office designated for OTs)  Creating the office as a pilot program for three years, to help DOD manage OTs until such time as the workforce becomes more experienced and proficient in using these agreements  Appendix A  Legislative History Other transaction authority first appeared in the National Aeronautics and Space Act of 1958  Since then, Congress has extended OT authorities to 11 federal agencies and a number of other federal offices (see Appendix B for information on other federal entities with similar authorities)  This appendix traces the legislative history of OT authorities and select related statutes applicable to DOD  To read the full text of the three statutory provisions related to OTs (10 U S C  2371, 2371b, and 2373), see Appendix D   National Aeronautics and Space Act of 1958 (P L  85-568) Creation of Other Transaction Authority On July 29, 1958 President Dwight D  Eisenhower signed into law the National Aeronautics and Space Act (P L  85-568), which established the National Aeronautics and Space Administration (NASA)  The purpose of the act included the expansion of human knowledge, preservation of the role of the United States as a leader in space science and technology, and pursuing the most effective utilization of the scientific and engineering resources of the United States  Section 203(b)(5) of the Space Act provided NASA the authority (emphasis added) to enter into and perform such contracts, leases, cooperative agreements, or other transactions as may be necessary in the conduct of its work and on such terms as it may deem appropriate, with any instrumentality of the United States     or with any person, firm, association, corporation, or educational institution  To the maximum extent practicable and consistent with the accomplishments of the purpose of this Act, such contracts, leases, agreements, and other transactions shall be allocated by the Administrator in a manner which will enable small-business concerns to participate equitably and proportionately in the conduct of the work of the Administration  Intellectual Property Rights The Space Act specifically addressed NASA's "property rights in inventions " Section 305 stated that any invention made in the performance of any work under any contract is the exclusive property of the United States "unless the Administrator waives all or any part of the rights " This was true even when the person who created the invention "was not employed or assigned to perform research, development, or exploratory work, but the invention is nevertheless related to the contract" and was made during working hours, or with a contribution of the government  The act granted the Administrator wide latitude to "waive all or any part of the rights of the United States under this section" if doing so was deemed to be in the best interests of the United States  When such rights were waived, NASA retained an irrevocable, nonexclusive, nontransferable royalty-free license by or on behalf of the United States  National Defense Authorization Act for FY1990 & FY1991 ( P L  101-189 ) The FY1990 & FY1991 NDAA granted DARPA temporary authority to enter into "cooperative agreements and other transactions" for the purpose of conducting advanced research projects  The statute clarified that OTs should only be used when "the use of standard contracts or grants is not feasible or appropriate " Congress restricted funding for OTs and cooperative agreements to $25 million of appropriated funds for FY1990 and FY1991, and set the authority to expire on September 30, 1991  Cost Sharing The FY1990 & FY1991 NDAA permitted OTs (or cooperative agreements) to include a clause requiring a person or entity to make payments to DOD as a condition of receiving support under the agreement  Such funds were to be merged into an account dedicated to supporting DARPA advanced research projects using cooperative agreements and other transactions  The act required, to the extent practicable, that funds provided by the government not exceed the total amount provided by the other parties to the project  According to the Senate report, one of the intents of the cost-sharing provision was to permit DARPA to "recoup the fruits of such arrangements, when there is a 'dual use' potential for commercial application" and to reinvest the funds to develop other technologies  Reporting Requirements The FY1990 & FY1991 NDAA required DOD to submit an annual report on the use of OTs and cooperative agreements, to include a description of each agreement and the technologies involved, the potential military and commercial utility of the technology, the reasons a contract or grant was not feasible to support the research, and the amount of payments, if any, received by the federal government under the agreement  National Defense Authorization Act for FY1991 ( P L  101-510 ) Section 244 increased the funds authorized for cooperative agreements and OTs from $25 million to $50 million  However, no such funding was appropriated  Reporting and Notification Requirements The conference report required DOD to submit to Congress a report listing the cooperative agreements and consortia intended to be used in FY1991-1992  The conference report also required DOD to provide the armed services and appropriations committees 30 days' notice prior to DOD signing a cooperative agreement or agreement with a consortia under OT authority  The Senate report focused on consortia as a method to pool resources, share research among numerous participants, and promote critical dual-use technology  Department of Defense Appropriations Act, 1992 ( P L  102-172 ) Limitations on the Use of OTs Section 8113A of P L  102-172 placed temporary limitations on the use of agreements undertaken pursuant to 10 U S C  2371: Section 8113A limited the use of OTs and cooperative agreements exclusively to DARPA (to the exclusion of the rest of DOD) for FY1992  Section 8113A limited DARPA to obligating or expending no more than $37 5 million in FY1992 for cooperative agreements or OTs undertaken pursuant to 10 U S C  2371  Section 8113A further established that no more than $75 million could be obligated or expended by DARPA in FY1992 for DOD dual-use critical technology partnerships  National Defense Authorization Act for FY1992 & FY1993 ( P L  102-190 ) Expanded Authority Section 826 extended other transaction authority to the military departments, and established separate fund accounts in each department for cost sharing  Section 826 also repealed the sunset for cooperative agreements and OTs, making the authorities permanent  Section 821 authorized DOD to enter into cooperative and other transaction agreements to develop critical dual-use technologies as set forth in the Defense Critical Technologies Plan  According to the Senate report     the United States tends to underinvest in dual-use technologies  National security requirements alone often do not justify major DOD support, and market prospects alone often appear to be too long-term or high risk to justify US industry carrying the entire development burden     The committee encourages use of cooperative agreements and other transactions in lieu of grants or contracts     The provision would require that at least 50 percent of funding over the life of a partnership derive from non-federal sources but would allow for a smaller industry share at the start  The conference report stated that the partnerships should focus on programs that fit into the security needs within DARPA  The conference report also stated that OTs are appropriate for those cases where the "regulations applicable to the allocation of patent and data rights under the procurement statutes may not be appropriate to partnership arrangements in certain cases " National Defense Authorization Act for FY1993 ( P L  102-484 ) Cost Sharing Section 4221 established 10 U S C  2511, which required DOD to establish cooperative arrangements with industry, educational institutions, federal labs, and other entities, to pursue research, development, and application of dual-use technologies  The section authorized DOD to use grants, contracts, cooperative agreements, or OTs to create these partnerships, and that the Federal government should not contribute more than 50% of the costs related to projects under this authority  National Defense Authorization Act for FY1994 ( P L  103-160 ) Expanded Authority Section 827 established 10 U S C  2358, which gave the Secretary of Defense and the Secretaries of the military departments the authority to conduct basic, advanced, and applied research through the use of contracts, cooperative agreements, grants, and OTs  Previously, OTs were only authorized for advanced research  Prototype Authorities Section 845 granted DARPA the authority to use OTs for prototype projects directly related to weapons or weapon systems proposed to be acquired by DOD  Section 845 required that "to the maximum extent practicable," prototypes be competitively awarded  This authority was set to terminate after three years  Section 845 remained as a note to 10 U S C  2371 until separately codified as 10 U S C  2371b in the FY2016 NDAA ( P L  114-92 )  Federal Acquisition Streamlining Act of 1994 ( P L  103-355 ) Section 1301 redesignated the language in 10 U S C  2358 (granting the authority to use OTs) to 10 U S C  2371  Reporting Requirements Section 1301 also required DOD to submit an annual report to the armed services committees, to include a general description of the other transactions, including the technologies involved in the research, the potential military and, if any, commercial utility of such technologies, the reasons for not using a contract or grant to provide support for such research, and the amount of payments, if any, received during the fiscal year pursuant to a clause in the other transactions and to what accounts such payments were credited  National Defense Authorization Act for FY1997 ( P L  104-201 ) Section 203 required that a senior DOD official be designated in OSD, and that the officials' sole responsibility be developing policy related to, and ensuring implementation of, DOD's dual-use technology program  This section authorized DOD to use OTs (as well as contracts, cooperative agreements, and grants) for dual-use projects only if the project "is entered into through the use of competitive procedures " Section 743 granted DOD the authority to use OTs to conduct research on Gulf War Syndrome, to determine its relationship to possible exposures of members of the Armed Forces to chemical warfare agents and hazardous materials, and the use of inoculations and new drugs  Expanded Prototype Authorities Section 804 amended Section 845 of the FY1994 NDAA by extending to the military departments and officials designated by the Secretary of Defense, the authority to use OTs for certain prototype projects  This authority, originally granted solely to DARPA and set to expire after three years, was given a new termination date of September 30, 1999  Reporting Requirements Section 267 modified elements of the annual report to the armed services committees  National Defense Authorization Act for FY1998 ( P L  105-85 ) Section 832 amended 10 U S C  2371 by clarifying that certain information submitted to DOD (i e  a proposal, business plan, technical information) be protected from disclosure pursuant to 5 U S C  552 for a period of five years  Strom Thurmond National Defense Authorization Act for FY1999 ( P L  105-261 ) Section 241 extended the sunset day for the authority to use OTs for prototypes from September 30, 1999, to September 30, 2001  Section 817 amended Section 2371 of Title 10, United States Code, clarifying that information submitted by outside parties in cooperative agreements for basic, applied, and advanced research is protected from disclosure under Section 552 of Title 5, United States Code  Department of Defense Appropriations Act, 1999 ( P L  105-262 ) While the enacted FY1999 defense appropriations bill ( P L  105-262 ) did not include legislative language addressing OTs, H Rept  105-591 , which accompanied the House-reported version of H R  4103 , included language expressing the House Appropriations Committee's "serious reservations" regarding the Air Force's then-proposed use of an OT agreement—instead of a contract— to develop the Evolved Expendable Launch Vehicle (EELV) program  The committee noted that "under [OTs] traditional safeguards which protect the government's interest in large acquisition programs are largely absent," and required the Under Secretary of Defense for Acquisition, Technology, and Logistics (now the Under Secretary of Defense for Acquisition and Sustainment) and the DOD Inspector General to certify to the congressional defense committees that the use of an OT was appropriate for the EELV program, and that "adequate safeguards exist[ed] to protect the government's interest and monitor program performance " National Defense Authorization Act for FY2000 ( P L  106-65 ) Section 801 required that for prototypes using OT authorities, DOD ensure that GAO, under its audit authority, have access to records relating to other transaction prototype agreements exceeding $5 million  Section 801 allowed for a waiver to GAO access and exempted entities that over the last year have not entered into an agreement with DOD that provided for audit access by a government entity  According to the Senate report: Senior DOD officials have sought legislation to extend other transaction authority to production contracts  Under current authority, there is some debate about whether GAO has audit access to other transactions  As the size, costs, and complexity of programs being funded using other transactions increases, the committee wants to ensure that the GAO has audit access in relation to the higher levels of spending and added risks  Reporting Requirements The Senate report also addressed reporting requirements and congressional intent to review the use of OTs  The report stated the following: The committee is assessing the utility of other transaction prototype authority  The statement of managers accompanying the Strom Thurmond National Defense Authorization Act of 1999 directed the Secretary of Defense to report on the use of this authority to the congressional defense committees, no later than March 1, 1999  In addition, both the Department of Defense Inspector General and the General Accounting Office are reviewing the use of other transaction prototype authority and will report to Congress in the coming year  The committee is interested in the extent that new commercial firms are entering the DOD marketplace through the use of other transaction authority, as well as the degree of cost sharing between the government and non-federal government parties  The committee is also interested in any lessons learned from the broad exemptions to federal law provided by other transaction authority  For example, other transactions are exempt from the Competition in Contracting Act, Truth in Negotiations Act, Contract Disputes Act, Antikickback Act of 1986, Procurement Integrity Act, Service Contract Act, Buy American Act, and chapter 137 of title 10, United States Code  Questions have been raised about whether the government's interest is adequately protected in the absence of the applicability of these statutes  Conversely, advocates of the view that the government should take advantage of the flexibility of other transactions have championed proposals to extend other transaction authority to production  The committee directs the Secretary of Defense to provide a new report that updates information in the March 1999 report on the use of other transaction prototype authority to the congressional defense committees by February 1, 2000  National Defense Authorization Act for FY2001 ( P L  106-398 ) Limitations on the Use of OTs Section 803 limited the use of OTs for prototype projects to only those circumstances when at least one nontraditional defense contractor significantly participates in the project, one-third of the total cost of the project is paid out of funds provided by parties to the transaction other than the federal government, or the senior procurement executive determines in writing that exceptional circumstances justify use of an OT  Nontraditional defense contractor was defined as an entity that for a period of one year has not entered into or performed "any contract that is subject to full coverage under the cost accounting standards" or "any other contract in excess of $500,000 to carry out prototype projects or to perform basic, applied, or advanced research projects for a Federal agency, that is subject to the Federal Acquisition Regulation '' Section 803 also extended the authority to use OTs for prototypes from September 30, 2001, to September 30, 2004  According to the Senate report, the intent of using OTs for prototypes is to attract companies that typically do not do business with the Department of Defense and encourage cost sharing and experimentation in potentially more efficient ways of doing business with traditional defense contractors  Other transaction authority is an important acquisition tool that can facilitate the incorporation of commercial technology into military weapon systems  In an environment where, in many areas, commercial technology is now more advanced than defense technology, it is imperative that the Department continue to have the flexibility to use innovative contractual instruments that provide access to this technology  There are, however, improvements that can be made in managing and overseeing these contractual arrangements  Section 804 clarified the extent of GAO's access to records in instances where the party in question has only done business with the government in the preceding year through an OT or cooperative agreement  National Defense Authorization Act for FY2002 ( P L  107-107 ) Expanded Authority—Follow-on Production Section 822 of the FY2002 NDAA granted DOD the authority to award a follow-on production contract for prototype projects when at least one-third of the total cost of the prototype project is to be paid out of funds provided by non-federal government sources  Under this authority, such a follow-on contract could be awarded without competition if the prototype project was successfully completed, the number of units in the production contract does not exceed the number of units specified in the underlying prototype agreement, and the price for each unit does not exceed the price specified in the underlying transaction  Department of Defense and Emergency Supplemental Appropriations for Recovery from and Response to Terrorist Attacks on the United States Act, 2002 ( P L  107-117 ) Establishment of Army Venture Capital Initiative (AVCI) Section 8150 designated $25 million of the FY2002 funds made available for Army Research, Development, Test, and Evaluation (RDT&E) to be made available to the Secretary of the Army for the purpose of funding a venture capital investment corporation established pursuant to 10 U S C  2371  A 2014 RAND report stated that OT authorities were used only to form the AVCI, and were not used to acquire products or services: "While OT authorities were used to form [AVCI] itself, any volume of the Army's purchase of products and services from [AVCI] companies is conducted under the FAR " National Defense Authorization Act for FY2004 ( P L  108-136 ) Expanded Authority Section 847 of the FY2004 NDAA extended the authority to use an OT for developing prototypes to improve weapons or weapon systems currently in use by the Armed Forces  Previously, such authority was restricted to prototypes "directly relevant to weapons or weapon systems proposed to be acquired or developed" by DOD  Section 847 also established a pilot program for transitioning prototypes to follow-on contracts for production for nontraditional defense contractors  Under the pilot program, such a follow-on contract could be treated as a commercial item or an item developed with both federal and private sector funds (for purposes of negotiating intellectual property rights)  The pilot program was restricted to contracts with nontraditional defense contractors, where the value of the contract does not exceed $50 million (approximately $70 million in FY2018 dollars), and that are firm-fixed price or fixed price with economic adjustment  The pilot program was set to sunset September 30, 2008  Section 1441 authorized any agency that engages in basic, applied, or advanced research and development projects that facilitated defense against or recovery from terrorism or nuclear, biological, chemical, or radiological attack to exercise the same general authority given to DOD as found in 10 U S C  2371 (including for prototype projects)  Reporting Requirements Section 1031 sunset the annual reporting requirement for OT after the report covering FY2006 was submitted  National Defense Authorization Act for FY2006 ( P L  109-163 ) Restricted Authority and Notification Requirements Section 212 of the FY2006 NDAA directed the Army to procure the Future Combat System using contract procedures set forth in part 15 of the Federal Acquisition Regulation, in lieu of an OT  Section 823 extended ethics requirements to OT prototype authority and required the congressional defense committees be notified in writing at least 30 days before such authority is exercised  Certification Requirements Section 823 also amended Section 845 of the FY1994 NDAA, requiring a written determination by a senior procurement executive for other transaction prototype projects estimated between $20 million and $100 million, and for a written determination by the Under Secretary of Defense for Acquisition, Technology, and Logistics for prototype projects that exceed $100 million  According to the Senate report: Section 845 was intended to be used for limited prototype projects, particularly those in which the Department seeks to engage nontraditional defense contractors that may be averse to the requirements imposed by a standard Department procurement contract  For this reason, the statement of managers accompanying the Strom Thurmond National Defense Authorization Act for Fiscal Year 1999 (Public Law 105–261) states: The conferees continue to believe that the section 845 authority should only be used in the exceptional cases where it can be clearly demonstrated that a normal contract or grant will not allow sufficient access to affordable technologies  The conferees are especially concerned that such authority not be used to circumvent the appropriate management controls in the standard acquisition and budgeting process  …  The committee does not believe that the $20 9 billion agreement entered between the Army and the Lead Systems Integrator for the FCS program is consistent with the language and intent of section 845 authority  Section 845 authority is intended to be used for limited prototype projects, particularly those in which the Department of Defense seeks to engage nontraditional defense contractors that may be averse to the requirements imposed by a standard Department contract  Department of Defense Appropriations Act, 2007 ( H R  5631 ) Reporting Requirement The FY2007 defense appropriations bill ( P L  109-289 ) did not include language addressing OTs  The conference report ( H Rept  109-676 ) included language expressing the conferees' "[concern] with the continued use of OTA contracts by the Missile Defense Agency," as such contracts "lack the customary safeguards found under FAR-based contracts for organizational conflict of interest, truth in negotiations and submission of cost and pricing data " The conferees "strongly encourage[d]" the Missile Defense Agency to convert "large development and procurement contracts using OTA to FAR-based contracts," and directed the Missile Defense Agency to submit a report to the congressional defense committees on the use of OTs, to include the number, value, and justification for the use of such agreements  National Defense Authorization Act for FY2008 ( P L  110-181 ) Section 823 extended the authority for prototype projects for five more years, from September 30, 2008, to September 30, 2013  National Defense Authorization Act for FY2009 ( P L  110-417 ) Section 822 required DOD to issue guidance on rights in technical data under non-FAR agreements, including OTs  Section 822 also required that appropriate provisions relating to rights in technical data be included in non-FAR agreements, consistent with policy guidance  This requirements is in statute at 10 U S C  2320 note  Section 824 expanded the scope of the pilot program for transition to follow-on contracts for certain prototype projects to include research projects carried out under 10 U S C  2371  Authority to use the pilot program, set to expire September 30, 2008, was extended to September 30, 2010  Section 874 required OT data be included in the Federal Procurement Data System  National Defense Authorization Act for FY2011 ( P L  111-383 ) Section 866 changed the definition of nontraditional defense contractor, conforming the definition to that found in 10 U S C  2302(9)  National Defense Authorization Act for FY2013 ( P L  112-239 ) Section 863 extended the authority for using OTs for prototype projects from September 30, 2013, to September 30, 2018  Carl Levin and Howard P  "Buck" McKeon National Defense Authorization Act for FY2015 ( P L  113-291 ) Expanded Authority Section 812 expanded the authority to use OT for prototypes, to include those "directly related to enhancing the mission effectiveness of military personnel and the supporting platforms, systems, components, or materials proposed to be acquired or developed by the Department of Defense, or to improvement of platforms, systems, components, or materials in use by the Armed Forces " Prior to the FY2015 NDAA, OTs could only be used for prototypes relating to weapons or weapon systems proposed to be developed, or for the improvement of weapons or weapon systems currently in use  Reporting Requirements Section 1071 repealed the reporting requirement language found in 10 U S C  2371, relating to OTs for research projects  National Defense Authorization Act for FY2016 ( P L  114-92 ) Until the FY2016 NDAA, the prototyping and follow-on production authority established in Section 845 of the FY1994 NDAA (as amended) was found in 10 U S C  2371 note  Section 815 of the FY2016 NDAA simultaneously repealed Section 845 of the FY1994 NDAA and put the repealed language into the newly created 10 U S C  2371b  The FY2016 NDAA also modified 2371b by making the authority permanent  Expanded Authority and Small Business The authorities in Section 2371b were expanded to allow their use when "all significant participants in the transaction other than the Federal government are small businesses or nontraditional contractors" and when the agency determines that using an OT would expand the defense supply base in a manner that could not be accomplished through a contract  Section 815 eased the restriction on follow-on production contracts or transactions  Section 815 amended the definition of a nontraditional defense contractor found in 10 U S C  2302 to be an entity that is not currently performing, and for one year prior to an OT has not performed on any contract or subcontract that is subject to full coverage under the cost accounting standards pursuant to Section 1502 of Title 41, U S C  Section 815 also required DOD to update its guidance to reflect changes in the statute  The conference report stated that Congress believed OTs are an attractive option for firms and organizations that do not usually participate in government contracting due to typical overhead burdens and the "one size fits all" rules governing defense acquisition  The report also stated that OTs could support DOD's effort to access new sources of technological innovation, specifically with Silicon Valley startup firms and small commercial firms  National Defense Authorization Act for FY2018 ( P L  115-91 ) Expanded Authority Section 216 of the FY2018 NDAA authorized nonprofit research institutions to enter into OTs with DOD for prototype projects  Section 862 amended 10 U S C  2358, granting the Secretary of Defense and the military departments the authority to pursue basic research, applied research, advanced research, and development projects under the OT authorities granted in Sections 2371 and 2371b of Title 10  Workforce Section 802 required DOD to establish a cadre of intellectual property experts to advise, assist, and provide resources to program offices who are developing intellectual property strategies for contracts and agreements  Section 863 required training and education for personnel involved in OTs and other innovative contracting methods  Certification Requirements and Small Business Section 864 adjusted the language of the statute to state that the dollar threshold relates to the specific transaction for a prototype project and not for the value of the entire project  Section 864 defined a transaction for follow-on production to include "all individual prototype sub-projects awarded under the transaction to a consortium of United States industry and academic institutions " Section 864 also increased the dollar thresholds for required approvals and defined the term small business to include small businesses under Section 9 of the Small Business Act to ensure that companies participating in the Small Business Innovation Research and Small Business Technology Transfer programs were considered small businesses for the purposes of the cost-sharing requirements  Miscellaneous Section 867 required the Secretary of Defense to establish a preference for OTs in the "execution of science and technology and prototyping programs " Section 1711 required DOD to carry out a pilot program to "assess the feasibility and advisability of increasing the manufacturing capability of the defense industrial base " Pursuant to the pilot, Section 1711 authorized DOD to use OTs to support production capabilities in small and medium-sized manufacturers  John S  McCain National Defense Authorization Act for FY2019 ( P L  115-232 ) Section 211 of the FY2019 NDAA clarified that follow-on production of a prototype or subproject within a consortium may occur if the individual prototype or subproject is complete; all projects associated with the consortium do not need to be completed before follow-on production of a specific prototype  Expanded Authority The FY2019 NDAA authorized the use of OTs to develop enhanced personal protective equipment (Section 226) and to carry out research under the Explosive Ordnance Disposal Defense Program (Section 311)  Reporting Requirements Section 244 required the Defense Innovation Unit to submit a report to Congress, to include the number of traditional and nontraditional defense contractors with DOD contracts or other transactions resulting directly from the unit's initiatives  Section 873 required DOD to submit an annual report through 2021, summarizing DOD's use of OTs, including organizations involved; number of transactions; amounts of payments; and purpose, description, and status of projects  Department of Defense Appropriations Act, 2019 ( P L  115-245 ) Reporting Requirements While the enacted FY2019 defense appropriations bill did not include legislative language addressing OTs, the conference report included language expressing the conferees' "[concern] with the lack of transparency surrounding the employment of OTA, particularly for follow-on production " The conferees directed DOD to provide quarterly reports to the House and Senate appropriations committees listing each active OT, and to include the following information on each agreement: funding military service or DOD component; major command (if applicable); contracting activity; appropriation title; budget line item; minimum and maximum award value; vendor; obligations and expenditures to date; product service code; period of performance; and indication if the OT agreement included an option for follow-on production (with a description of the scope of anticipated follow-on production)  The conferees also directed GAO to review DOD's use of OTs to determine whether the "employment of this authority conforms to applicable statutes and guidelines, to include the identification of any potential conflicts " GAO was also required to report on the extent to which OTs have been used since FY2016  Notification Requirements The House report to accompany H R  6157 acknowledged OTs as an "important tool to provide flexibility and agility for cutting-edge research and development projects and prototypes " However, the report stated its concern "with the lack of transparency on the use of OTA authority for follow-on production procurements," and directed that no funds could be obligated or expended for a follow-on production contract or a transaction carried out under 10 U S C  2371b, until 30 days after the Secretary of Defense provides the congressional defense committees with a notification of the proposed contract or transaction, to include a justification of why an OT is being used for production  Appendix B  Non-DOD Federal Agencies with Agency-Wide OT or Related Authorities A number of agencies have varying other transaction or similar authorities, as reflected in Table B-1   The table below is not a comprehensive or definitive listing of every federal government entity with OT or related authorities  In some instances, offices, agencies, commissions, and other federal government entities have OT or related authorities that are only associated with certain programs or projects, such as the National Institutes of Health (which has OT authority for such specific activities such as the National Heart, Blood Vessel, Lung, and Blood Diseases and Blood Resources Program [42 U S C  §285b-3] and the Cures Acceleration Network [42 U S C  §287a])  Appendix C  Reliability of Data on Other Transactions All data have imperfections and limitations  FPDS-NG data can be used to identify broad trends and produce rough estimates, or to gather information about specific contracts  Some observers say that despite their shortcomings, FPDS-NG data are substantially more comprehensive than what is available in most other countries in the world  Understanding the limitations of government procurement data—including knowing when, how, and to what extent to rely on data—can help policymakers incorporate FPDS-NG data more effectively into their decisionmaking process  FPDS-NG OT Data Quality and Accuracy Issues Decisionmakers should be cautious when using data from FPDS-NG to develop policy or otherwise draw conclusions, especially with respect to OTs  In some cases, the data themselves may not be reliable  In other instances, a query for particular data may return differing results, depending on the parameters and timing of the analysis  In particular, all DOD data entered into FPDS-NG are subject to a 90-day delay, and updates to "data, including new actions, modifications, and corrections are made on a regular basis," which could result in changes to "data     for current and/or prior fiscal years " Inconsistencies in FPDS-NG Data Within FPDS-NG, two primary collections of obligation data exist: one associated with standard government procurement contracts or modifications to such contracts, and the other associated with prototype OT agreements or modifications to such agreements  FPDS-NG's collection of prototype OT data allows for the input of additional data elements—such as nongovernment dollars associated with cost-share prototype OT agreements—not included in FPDS-NG's collection of standard government procurement contract data  More consequentially, FPDS-NG's prototype OT data include two similar data elements that allow users to identify the fiscal year a prototype OT agreement was signed or modified  One, labeled in the database as "Fiscal Year," appears to allow users entering data into the system to manually assign a fiscal year to a transaction  FPDS-NG users entering data into the system appear to have interpreted this data element in various, conflicting ways  For example, a Department of the Air Force OT agreement was signed in February 2016 for the development of rocket propulsion system prototypes under the Evolved Expendable Launch Vehicle (EELV) program  FPDS-NG records an obligation of $115 million in FY2020 for this agreement  Fiscal law bars DOD from obligating money now for future fiscal years that have not yet occurred  The second, labeled "Contract Fiscal Year," appears to be based on the date the prototype OT agreement was signed or modified  See Table C-1 for a comparison of the "Fiscal Year" and "Contract Fiscal Year" elements for selected new prototype OT agreements signed between FY2013 and FY2017  If a user selects the Fiscal Year data element when attempting to review high-level data on recent trends in the use of prototype OT agreements within DOD—such as the total amount obligated for prototype OT agreements on an annual basis—that user will obtain a substantially different result than if he or she selects the Contract Fiscal Year data element for a similar analysis  See Table C-2 for a comparison of action obligations and nongovernment contributions using the Fiscal Year and Contract Fiscal Year data elements for new prototype OT agreements signed between FY2013 and FY2017  DOD OT Data Analysis Methodological Issues in Congressional Reports A March 2017 report to Congress entitled "An Assessment of Cost-Sharing in Other Transaction Agreements for Prototype Projects," completed by the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics indicated that In FY2016, DOD obligated (Note: Two outliers in 2016 excluded) $1 billion in section 2371b awards and received $68 million in cost-share contributions after excluding two significant trend outliers  DOD cited FPDS-NG as the source for its analysis, and defined the "significant trend outliers" excluded as "two $40 million OTAs with total cost-share of $270 million," likely referring to two DARPA prototype OT agreements conducted on a cost-share basis initiated in FY2016  However, a CRS analysis of the same FPDS-NG data identified numerous inconsistencies in DOD's methodological approach  Recreation of DOD Methodology Specifically, DOD appears to have conducted its analysis using the "Fiscal Year" data element referenced in this report's discussion of FPDS-NG OT data quality and accuracy issues, which would attribute some prototype OT activities to the wrong fiscal year for the purposes of comparative trend analysis  DOD also compared two disparate transaction types: the reported "$1 billion in Section 2371b awards" includes all action obligations associated with ongoing prototype OT activities in FY2016, including transactions associated with prototype OT indefinite delivery contracts, while the "$68 million in cost-share contributions" includes only cost-share contributions associated with new prototype OT agreements  If DOD used the "Contract Fiscal Year" data element, excluded the identified "trend outliers," and focused on all action obligations and cost-share contributions associated with ongoing prototype OT activities, it would have found instead that the department obligated $1 4 billion for prototype OT agreements in FY2016, with an additional $313 5 million in cost-share contributions from the private sector  On the other hand, if DOD used the "Contract Fiscal Year" data element, excluded the identified "trend outliers," and focused on only action obligations and cost-share contributions associated with new prototype OT agreements, it would have found instead that the department obligated $400 million for prototype OT agreements in FY2016, with an additional $272 1 million in cost-share contributions from the private sector  Appendix D  Other Transaction Authority Statutes 10 U S C  §2371  Research projects: transactions other than contracts and grants (a) ADDITIONAL FORMS OF TRANSACTIONS AUTHORIZED — The Secretary of Defense and the Secretary of each military department may enter into transactions (other than contracts, cooperative agreements, and grants) under the authority of this subsection in carrying out basic, applied, and advanced research projects  The authority under this subsection is in addition to the authority provided in Section 2358 of this title to use contracts, cooperative agreements, and grants in carrying out such projects  (b) EXERCISE OF AUTHORITY BY SECRETARY OF DEFENSE — In any exercise of the authority in subsection (a), the Secretary of Defense shall act through the Defense Advanced Research Projects Agency or any other element of the Department of Defense that the Secretary may designate  (c) ADVANCE PAYMENTS — The authority provided under subsection (a) may be exercised without regard to Section 3324 of Title 31  (d) RECOVERY OF FUNDS — (1) A cooperative agreement for performance of basic, applied, or advanced research authorized by Section 2358 of this title and a transaction authorized by subsection (a) may include a clause that requires a person or other entity to make payments to the Department of Defense or any other department or agency of the Federal Government as a condition for receiving support under the agreement or other transaction  (2) The amount of any payment received by the Federal Government pursuant to a requirement imposed under paragraph (1) may be credited, to the extent authorized by the Secretary of Defense, to the appropriate account established under subsection (f)  Amounts so credited shall be merged with other funds in the account and shall be available for the same purposes and the same period for which other funds in such account are available  (e) CONDITIONS — (1) The Secretary of Defense shall ensure that- (A) to the maximum extent practicable, no cooperative agreement containing a clause under subsection (d) and no transaction entered into under subsection (a) provides for research that duplicates research being conducted under existing programs carried out by the Department of Defense; and (B) to the extent that the Secretary determines practicable, the funds provided by the Government under a cooperative agreement containing a clause under subsection (d) or a transaction authorized by subsection (a) do not exceed the total amount provided by other parties to the cooperative agreement or other transaction  (2) A cooperative agreement containing a clause under subsection (d) or a transaction authorized by subsection (a) may be used for a research project when the use of a standard contract, grant, or cooperative agreement for such project is not feasible or appropriate  (f) SUPPORT ACCOUNTS — There is hereby established on the books of the Treasury separate accounts for each of the military departments and the Defense Advanced Research Projects Agency for support of research projects and development projects provided for in cooperative agreements containing a clause under subsection (d) and research projects provided for in transactions entered into under subsection (a)  Funds in those accounts shall be available for the payment of such support  (g) EDUCATION AND TRAINING —The Secretary of Defense shall— (1) ensure that management, technical, and contracting personnel of the Department of Defense involved in the award or administration of transactions under this section or other innovative forms of contracting are afforded opportunities for adequate education and training; and (2) establish minimum levels and requirements for continuous and experiential learning for such personnel, including levels and requirements for acquisition certification programs  (h) REGULATIONS — The Secretary of Defense shall prescribe regulations to carry out this section  (i) Protection of Certain Information From Disclosure -(1) Disclosure of information described in paragraph (2) is not required, and may not be compelled, under Section 552 of Title 5 for five years after the date on which the information is received by the Department of Defense  (2)(A) Paragraph (1) applies to information described in subparagraph (B) that is in the records of the Department of Defense if the information was submitted to the Department in a competitive or noncompetitive process having the potential for resulting in an award, to the party submitting the information, of a cooperative agreement for performance of basic, applied, or advanced research authorized by Section 2358 of this title or another transaction authorized by subsection (a)  (B) The information referred to in subparagraph (A) is the following: (i) A proposal, proposal abstract, and supporting documents  (ii) A business plan submitted on a confidential basis  (iii) Technical information submitted on a confidential basis  10 U S C  §2371b  Authority of the Department of Defense to carry out certain prototype projects (a) AUTHORITY — (1) Subject to paragraph (2), the Director of the Defense Advanced Research Projects Agency, the Secretary of a military department, or any other official designated by the Secretary of Defense may, under the authority of Section 2371 of this title, carry out prototype projects that are directly relevant to enhancing the mission effectiveness of military personnel and the supporting platforms, systems, components, or materials proposed to be acquired or developed by the Department of Defense, or to improvement of platforms, systems, components, or materials in use by the armed forces  (2) The authority of this section- (A) may be exercised for a transaction (for a prototype project) that is expected to cost the Department of Defense in excess of $100,000,000 but not in excess of $500,000,000 (including all options) only upon a written determination by the senior procurement executive for the agency as designated for the purpose of Section 1702(c) of Title 41, or, for the Defense Advanced Research Projects Agency or the Missile Defense Agency, the director of the agency that- (i) the requirements of subsection (d) will be met; an d (ii) the use of the authority of this section is essential to promoting the success of the prototype project; and (B) may be exercised for a transaction (for a prototype project) that is expected to cost the Department of Defense in excess of $500,000,000 (including all options) only if- (i) the Under Secretary of Defense for Acquisition, Technology, and Logistics determines in writing that- (I) the requirements of subsection (d) will be met; an d (II) the use of the authority of this section is essential to meet critical national security objectives; and (ii) the congressional defense committees are notified in writing at least 30 days before such authority is exercised  (3) The authority of a senior procurement executive or director of the Defense Advanced Research Projects Agency or Missile Defense Agency under paragraph (2)(A), and the authority of the Under Secretary of Defense for Acquisition, Technology, and Logistics under paragraph (2)(B), may not be delegated  (b) EXERCISE OF AUTHORITY — (1) Subsections (e)(1)(B) and (e)(2) of such Section 2371 shall not apply to projects carried out under subsection (a)  (2) To the maximum extent practicable, competitive procedures shall be used when entering into agreements to carry out projects under subsection (a)  (c) COMPTROLLER GENERAL ACCESS TO INFORMATION — (1) Each agreement entered into by an official referred to in subsection (a) to carry out a project under that subsection that provides for payments in a total amount in excess of $5,000,000 shall include a clause that provides for the Comptroller General, in the discretion of the Comptroller General, to examine the records of any party to the agreement or any entity that participates in the performance of the agreement  (2) The requirement in paragraph (1) shall not apply with respect to a party or entity, or a subordinate element of a party or entity, that has not entered into any other agreement that provides for audit access by a Government entity in the year prior to the date of the agreement  (3) (A) The right provided to the Comptroller General in a clause of an agreement under paragraph (1) is limited as provided in subparagraph (B) in the case of a party to the agreement, an entity that participates in the performance of the agreement, or a subordinate element of that party or entity if the only agreements or other transactions that the party, entity, or subordinate element entered into with Government entities in the year prior to the date of that agreement are cooperative agreements or transactions that were entered into under this section or Section 2371 of this title  (B) The only records of a party, other entity, or subordinate element referred to in subparagraph (A) that the Comptroller General may examine in the exercise of the right referred to in that subparagraph are records of the same type as the records that the Government has had the right to examine under the audit access clauses of the previous agreements or transactions referred to in such subparagraph that were entered into by that particular party, entity, or subordinate element  (4) The head of the contracting activity that is carrying out the agreement may waive the applicability of the requirement in paragraph (1) to the agreement if the head of the contracting activity determines that it would not be in the public interest to apply the requirement to the agreement  The waiver shall be effective with respect to the agreement only if the head of the contracting activity transmits a notification of the waiver to Congress and the Comptroller General before entering into the agreement  The notification shall include the rationale for the determination  (5) The Comptroller General may not examine records pursuant to a clause included in an agreement under paragraph (1) more than three years after the final payment is made by the United States under the agreement  (d) APPROPRIATE USE OF AUTHORITY — (1) The Secretary of Defense shall ensure that no official of an agency enters into a transaction (other than a contract, grant, or cooperative agreement) for a prototype project under the authority of this section unless one of the following conditions is met: (A) There is at least one nontraditional defense contractor or nonprofit research institution participating to a significant extent in the prototype project  (B) All significant participants in the transaction other than the Federal Government are small businesses (including small businesses participating in a program described under Section 9 of the Small Business Act (15 U S C  638)) or nontraditional defense contractors  (C) At least one third of the total cost of the prototype project is to be paid out of funds provided by sources other than other than the Federal Government  (D) The senior procurement executive for the agency determines in writing that exceptional circumstances justify the use of a transaction that provides for innovative business arrangements or structures that would not be feasible or appropriate under a contract, or would provide an opportunity to expand the defense supply base in a manner that would not be practical or feasible under a contract  (2) (A) Except as provided in subparagraph (B), the amounts counted for the purposes of this subsection as being provided, or to be provided, by a party to a transaction with respect to a prototype project that is entered into under this section other than the Federal Government do not include costs that were incurred before the date on which the transaction becomes effective  (B) Costs that were incurred for a prototype project by a party after the beginning of negotiations resulting in a transaction (other than a contract, grant, or cooperative agreement) with respect to the project before the date on which the transaction becomes effective may be counted for purposes of this subsection as being provided, or to be provided, by the party to the transaction if and to the extent that the official responsible for entering into the transaction determines in writing that- (i) the party incurred the costs in anticipation of entering into the transaction; and (ii) it was appropriate for the party to incur the costs before the transaction became effective in order to ensure the successful implementation of the transaction  (e) Definitions —In this section: (1) The term "nontraditional defense contractor" has the meaning given the term under Section 2302(9) of this title  (2) The term "small business" means a small business concern as defined under Section 3 of the Small Business Act (15 U S C  632)  (f) FOLLOW-ON PRODUCTION CONTRACTS OR TRANSACTIONS — (1) A transaction entered into under this section for a prototype project may provide for the award of a follow-on production contract or transaction to the participants in the transaction  A transaction includes all individual prototype subprojects awarded under the transaction to a consortium of United States industry and academic institutions  (2) A follow-on production contract or transaction provided for in a transaction under paragraph (1) may be awarded to the participants in the transaction without the use of competitive procedures, notwithstanding the requirements of Section 2304 of this title, if - (A) competitive procedures were used for the selection of parties for participation in the transaction; and (B) the participants in the transaction successfully completed the prototype project provided for in the transaction  (3) Contracts and transactions entered into pursuant to this subsection may be awarded using the authority in subsection (a), under the authority of Chapter 137 of this title, or under such procedures, terms, and conditions as the Secretary of Defense may establish by regulation  (g) AUTHORITY TO PROVIDE PROTOTYPES AND FOLLOW-ON PRODUCTION ITEMS AS GOVERNMENT-FURNISHED EQUIPMENT — An agreement entered into pursuant to the authority of subsection (a) or a follow-on contract or transaction entered into pursuant to the authority of subsection (f) may provide for prototypes or follow-on production items to be provided to another contractor as Government-furnished equipment  (h) APPLICABILITY OF PROCUREMENT ETHICS REQUIREMENTS — An agreement entered into under the authority of this section shall be treated as a Federal agency procurement for the purposes of Chapter 21 of Title 41  10 U S C  §2373  Procurement for experimental purposes (a) AUTHORITY — The Secretary of Defense and the Secretaries of the military departments may each buy ordnance, signal, chemical activity, transportation, energy, medical, space-flight, and aeronautical supplies, including parts and accessories, and designs thereof, that the Secretary of Defense or the Secretary concerned considers necessary for experimental or test purposes in the development of the best supplies that are needed for the national defense  (b) PROCEDURES — Purchases under this section may be made inside or outside the United States and by contract or otherwise  Chapter 137 of this title applies only when such purchases are made in quantities greater than necessary for experimentation, technical evaluation, assessment of operational utility, or safety or to provide a residual operational capability 


Report 2: , support for general aviation airports)  But there are significant disagreements about the appropriate degree of federal participation in airport development and finance and about the specific types of expenditure that should be given priority within AIP  Airline and airport operators tend to view the fully authorized funding of the program as a good thing  An alternative view, however, is that too much has been spent on AIP, particularly at smaller airports that do not play a significant role in commercial aviation  The assessment of airport capital needs is fundamental to determining the appropriate federal support needed to foster a safe and efficient national airport system  The federal government's interest goes beyond capacity issues to include implementation of federal safety and noise policies  Both FAA and the Airports Council International-North America (ACI-NA) have issued projections of airports' long-term financial needs  In its most recent NPIAS report, FAA estimated that the national system's capital needs for FY2019-FY2023 will total $35 1 billion (an annual average of approximately $7 billion)  The ACI-NA infrastructure needs survey resulted in an estimate of $128 1 billion over the same years (an annual average of approximately $25 6 billion)  The main reason for the widely differing estimates was disparate views on what kinds of airport projects to include  The NPIAS report was based on information taken from airport master plans and state system plans, but FAA planners screened out planned projects not justified by aviation activity forecasts or not eligible for AIP grants  Only designated NPIAS airports were included in the FAA study  Implicit in this methodology is that the planning has been carried through to the point where financing is identified  Not all projects used to develop the NPIAS estimates are actually completed, or in some cases even begun, within the range of years covered in the NPIAS estimates  ACI-NA argues that the NPIAS underestimates AIP eligible needs because not all such needs will be in the current airport plans  The ACI-NA study reflects the broader business view of major airport operators and casts a substantially wider net  It includes projects funded by PFCs, bonds, or state or local funding; airport-funded air traffic control facilities; airport- or TSA-funded security projects; "necessary" AIP-ineligible projects such as parking facilities, hangars, revenue portions of terminals, and off-airport roads/transit facilities; and AIP-eligible projects not reported to FAA in the belief that there would be a low probability of receiving additional AIP funding  Its 2019-2023 infrastructure needs survey, for example, included major airport terminal projects that are ineligible for AIP grants  The ACI-NA study also includes projects without identified funding sources  The ACI-NA estimate is higher than the FAA estimate because of the wider net it casts and because it is adjusted for projected inflation  The estimates are important because the primary AIP reauthorization issue is the program's appropriate level of funding  Because the ACI-NA airport needs projection includes much that is not eligible for AIP grants, its accuracy may not be as critical in evaluating appropriate AIP funding levels as that of the NPIAS projections  On the other hand, the broader ACI-NA estimate may be more significant for policy choices related to bond issuance and PFCs, since these sources fund a broader range of projects than AIP  In 2004, then-FAA Administrator Marion C  Blakey stated that the agency's goal was to increase total capacity at the top 35 U S  airports by 30% over a 10-year period  FAA's Operational Evolution Plan (OEP) is intended to increase the capacity and efficiency of the National Airspace System (NAS) over a 10-year period to keep up with the expected growth in demand for air travel and air cargo  In support of that goal, FAA released a study focused on the 35 busiest airports, Capacity Needs in the National Airspace System: An Analysis of Airport and Metropolitan Area Demand and Operational Capacity in the Future (also referred to as FACT1)  The study projected 18 airports would need additional capacity by 2020  In 2007, FACT1 was updated by a second study, FACT2  FACT2 expanded the study to include 21 non-OEP airports that were identified as having the potential to be capacity constrained or were in capacity-constrained metropolitan areas  The study examined airports that would need capacity increases and also projected which airports would need capacity increases in 2015 and 2025  It identified four airports plus the New York metropolitan area that needed additional capacity in 2007  It further identified 14 hub airports as likely to be capacity-constrained in 2025  FACT2 found that, in comparison to FACT1, many non-OEP airports "    have higher capacities than originally presumed and thus less need for additional capacity " A further update, FACT3, was released in January 2015  FACT3 forecasted that the 2007-2009 recession, volatile fuel costs, airline consolidation, and replacement of many 50-seat regional jets with larger aircraft would result in 32% fewer operations and about 23% fewer enplanements in 2025 at the 30 core airports than forecast in FACT2  It projected that airport delays would remain concentrated at a few major hub airports, notably the three New York City-area airports, Philadelphia International Airport, and Hartsfield-Jackson Atlanta International Airport  This study may have implications for the reauthorization of AIP  The large runway projects that are the focus of the OEP can require long lead times—10 or more years from concept to initial construction is not unusual  At large and medium hub airports, runway projects are usually paid for, in part, by AIP funds  Therefore, some projects needed by 2025 may require AIP funding in earlier years  Because large and medium airports that levy PFCs must forgo either 50% or 75% of their AIP formula entitlement funds, most federal funding for their runway projects would probably need to take the form of AIP discretionary funds  The pool of discretionary funds is primarily the remainder of annual funding after the entitlement formula requirements are satisfied  Of the forgone PFC funds, 87 5% are reserved for the small airport fund and are also not available for OEP airports  If the AIP budget is constrained in the future, either under a reauthorization bill or during the annual appropriations process, and the entitlement formulas remain as they are, the discretionary portion of the AIP budget may be squeezed, limiting large airports' ability to draw on AIP funds for major capacity expansion projects  Many of the attributes of AIP's programmatic structure are similar to those of the 1982 act that created the program  Over the years these attributes have been modified based on perceived needs and on the practical politics of passing the periodic FAA reauthorization bills that contain the AIP provisions  These considerations make a major overhaul of the AIP structure unlikely, but may leave room for programmatic adjustments in the distribution of apportionments  One such adjustment might shift AIP funds to enhancing capacity at large and medium hub airports  There are several ways Congress might accomplish this  One would be to eliminate the requirement that large and medium hub airports that impose the maximum PFCs forgo 75% of their entitlements  This change would give larger airports a greater share of entitlement funding, but at the cost of reducing AIP grants to small airports  Alternatively, changes in the statutory set-asides of discretionary funds could give FAA more flexibility to use that money for capacity enhancement, but might reduce funding for noise mitigation and other purposes  Changes in the last several FAA authorization acts increased entitlements and broadened the range of landside projects eligible for AIP grants  These changes generally benefitted airports smaller than medium hub size  In particular, the increased amount of apportioned funds has limited the availability of funds for discretionary grants at major airports  Further changes giving airports increased flexibility in the use of their entitlements might benefit smaller airports not served by commercial aviation, in line with the national goal of having an "extensive" national airport system, but this use of funds might conflict with the goal of reducing congestion at major commercial airports  The current apportionment system relies on a $3 2 billion funding level trigger mechanism to lift most of the apportionments to twice their formula level  This has been in place for two reauthorization cycles  Should that trigger be breached, entitlements for all airports would be reduced drastically  The entitlement formulas may not be sustainable, without depleting discretionary funds, in the absence of additional funding for AIP  One way to reduce the amount of trust fund revenue needed for AIP would be to allow large and medium hub airports to opt out of AIP and rely exclusively on PFCs to finance capital projects  This would require raising or eliminating the federal cap on PFCs  These "defederalized" airports could then be released from some or all of the AIP grant assurances under which they now operate, such as land use requirements and airport revenue use restrictions  If airports exit the program, AIP spending could be reduced or could be redirected to other NPIAS airports  Airport privatization denotes a change in ownership from a public entity (such as a local government or an airport authority established by a state government) to a private one  In a number of countries, such as Great Britain, government-owned airports have been privatized by sale to private owners  In the United States, some airports have allowed private ownership of certain on-airport facilities or management functions, but the ownership of all major airports remains in the hands of government entities  The Airport Privatization Pilot Program (49 U S C  §47134; Section 149 of the Federal Aviation Reauthorization Act of 1996, P L  104-264 , as amended) authorizes FAA to exempt up to 10 airports from certain federal restrictions on the use of airport revenue off-airport  Participating airports may be also exempted from certain requirements on the repayment of federal grants  Privatized airports may still participate in the AIP, but at a lower federal share (70%)  The pilot program was renamed the Airport Investment Partnership Program (AIPP) in the 2018 FAA reauthorization act and expanded to admit more than 10 airports  The AIPP provides that at primary airports, the airport sponsor may only recover from the sale or lease an amount approved by at least 65% of the scheduled air carriers serving the airport, as well as by both scheduled and unscheduled air carriers that together account for 65% of the total landed weight at the airport for the year  The requirement that air carriers approve the use of airport revenue for nonairport purposes, such as profit distribution, may have served to limit interest in the program  To date, only two airports have completed the privatization process established under the provisions of the AIPP  One of those, Stewart International Airport in New York State, subsequently reverted to public ownership when it was purchased by the Port Authority of New York and New Jersey  Luis Muñoz Marín International Airport in San Juan, PR, is now the only commercial service airport operating under private management after privatization under the APPP  As of 2018, there are three applicants under active FAA consideration: Hendry County Airglades Airport in Clewiston, FL; Westchester Airport in White Plains, NY; and St  Louis Lambert International Airport in St  Louis, MO  There is no certainty that any AIP cost savings from privatization would be retained for use by the other AIP-eligible airports  AIP spending is determined by the authorization and appropriations process, and Congress could choose to use any savings to reduce the program size, to marginally assist in deficit reduction, to reduce general fund portions of FAA operations funding, or to make money available for spending elsewhere  Debate over FAA reauthorization generally brings forth proposals to alter the AIP grant assurances, such as ensuring that workers on airport construction projects receive prevailing wages set under the Davis-Bacon Act and pledging to use airport revenue solely for spending on airport operations and capital costs  If AIP spending remains constrained, critics are likely to argue that the grant assurances raise the cost of projects to increase airport capacity and complicate the closure and reuse of underutilized airports or airports that are locally unpopular due to noise or safety concerns  Historically, a basic funding issue is whether to change the existing discretionary fund set-aside for noise mitigation and abatement  The noise set-aside, however, has been increased in previous reauthorization acts and is now 35% of discretionary funding  Demand to use AIP funds for noise mitigation could increase if Congress grants FAA the flexibility to fund noise mitigation projects that are outside the DNL 65 decibel (dB) noise impact area, but this could divert resources from capacity and safety projects  A related issue is whether to make the planning for noise-mitigating air traffic control procedures at individual airports eligible for AIP funding  The central issue related to PFCs is whether to raise the $4 50 per enplaned passenger ceiling or to eliminate the ceiling altogether  In general, airports argue for increasing or eliminating the ceiling, whereas most air carriers and some passenger advocates oppose higher limits on the PFCs  A 2015 GAO study analyzed the effects by raising the PFC cap under three scenarios: setting the cap at $6 47, $8 00, or $8 50  The study found raising PFC would significantly increase airport funding, but could also marginally slow passenger growth and therefore the growth in revenues to the trust fund  PFC supporters feel that the PFC is more reliable than AIP funding, which is subject to the authorization and appropriations process  They also argue that PFCs are procompetitive, helping airports build gates and facilities that both encourage new entrant carriers and allow incumbent carriers to expand  Advocates of an increase in the cap also argue that over time, the value of the PFC has been eroded by inflation and an adjustment is therefore necessary  The permissible uses of revenues are an ongoing point of contention  Airport operators, in particular, would like more freedom to use PFC funds for off-airport projects, such as transportation access projects, and want the process of obtaining FAA approval to be streamlined  Carriers, on the other hand, often complain that airports use PFC funds to finance proposals of dubious value, especially outside airport boundaries, instead of high-priority projects that offer meaningful safety or capacity enhancements  The major air carriers are also unhappy with their limited influence over project decisions, as airports are required only to consult with resident air carriers instead of having to get their agreement on PFC-funded projects  Unlike interest income from governmental bonds, which is not subject to the alternative minimum tax (AMT), interest from private activity bonds is still subject to the AMT  ACI-NA has proposed broadening the definition of governmental airport bonds to, in effect, include either all airport bonds or at least those bonds issued for public-use projects that meet AIP or PFC eligibility requirements  Opponents of such changes express concerns that these changes would reduce U S  Treasury revenues  Some also argue it would make more sense to change the AMT as part of a tax bill rather than including a specific exemption for income on airport bonds in an FAA reauthorization bill  In either case, such a change would not be under the jurisdiction of the congressional committees that will have jurisdiction over most reauthorization provisions  Changes to the AMT would be under the jurisdiction of the congressional tax-writing committees, the House Committee on Ways and Means and the Senate Committee on Finance  Appendix A  Legislative History of Federal Grants-in-Aid to Airports Prior to World War II the federal government viewed airports as a local responsibility  During the 1930s, it spent about $150 million a year on airports through work relief agencies such as the Works Progress Administration (WPA)  The first federal support for airport construction was granted during World War II  After the war, the Federal Airport Act of 1946 (P L  79-377) created the Federal Aid to Airports Program, using funds appropriated annually from the general fund  Initially much of this spending supported conversion of military airports to civilian use  In the 1960s substantial funding went to upgrade and extend runways for use by commercial jets  By the end of the 1960s, congestion, both in the air and on the ground at U S  airports, was seen as evidence that airport capacity was inadequate  Airport and Airway Development and Revenue Acts of 1970 (P L  91-258) In 1970, Congress responded to the capacity concerns by passing two acts  The first, the Airport and Airway Development Act (Title I of P L  91-258), established the Airport Development Aid Program (ADAP) and the Planning Grant Program (PGP), and set forth the programs' grant criteria, distribution guidelines, and authorization of grant-in-aid funding for the first five years of the program  The second, the Airport and Airway Revenue Act of 1970 (Title II of P L  91-258), established the Airport and Airway Trust Fund  Revenues from levies on aviation users and fuel were dedicated to the fund  Under the 1970 acts the trust fund could pay capital costs and, when excess funds existed, could also help cover FAA's administrative and operations costs  Airport and Airway Development and Revenue Acts Amendments of 1971 (P L  92-174) The Nixon Administration's FAA budget requests for FY1971 and FY1972 under the new trust fund system brought it into immediate conflict with Congress over the budgetary treatment of trust fund revenues  The Administration treated the new financing system as a user-pay system, whereas many Members of Congress viewed the trust fund as primarily a capital fund  The 1971 Amendments Act made the trust fund a capital-only account (although only through FY1976), disallowing the use of trust fund revenues for FAA operations  Airport and Airway Development Amendments Act of 1976 ( P L  94-353 ) The 1976 act made a number of adjustments to the ADAP and reauthorized the Airport and Airway Trust Fund through FY1980  The act again allowed the use of trust fund resources for the costs of air navigation services (a part of operations and maintenance)  However, in an attempt to assure adequate funding of airport grants, the act included "cap and penalty" provisions which placed an annual cap on spending for costs of air navigation systems and a penalty that reduced these caps if airport grants were not funded each year at the airport program's authorized levels  This cap was altered multiple times in reauthorization acts in the following decades  ADAP grants totaled about $4 1 billion from 1971 through 1980  Congress did not pass authorizing legislation for ADAP during FY1981 and FY1982, during which the aviation trust fund lapsed, although spending for airport grants continued  Airport and Airway Improvement Act of 1982 ( P L  97-248 ) The 1982 act created the current AIP and reactivated the Airport and Airway Trust Fund  It altered the funding distribution among the newly defined categories of airports, extending aid eligibility to privately owned general aviation airports, increasing the federal share of eligible project costs, and earmarking 8% of total funding for noise abatement and compatibility planning  The act also required the Secretary of Transportation to publish a national plan for the development of public-use airports in the United States  This biannual publication, the National Plan of Integrated Airport Systems (NPIAS) , identifies airports that are considered important to the national aviation system  For an airport to receive AIP funds it must be listed in the NPIAS  Although the 1982 act was amended often in the 1980s and early 1990s, the general structure of AIP remained the same  The Airport and Airway Safety and Capacity Expansion Act of 1987 ( P L  100-223 ; 1987 act) authorized significant spending increases for AIP and added a cargo service apportionment  It also included provisions to encourage full funding of AIP at the authorized level  Title IX of P L  101-508 , the Omnibus Budget Reconciliation Act of 1990 (OBRA1990), included the Aviation and Airway Safety and Capacity Act of 1990, which allowed airports, under certain conditions, to levy a Passenger Facility Charge (PFC) to raise revenue and also established the Military Airport Program (MAP), which provided AIP funding for capacity and/or conversion-related projects at joint-use or former military airports  The Airport Noise and Capacity Act of 1990 (OBRA 1990, Title IX, Subtitle D) set a national aviation noise policy  OBRA1990 included the Revenue Reconciliation Act of 1990, which reauthorized the Aviation Trust Fund and adjusted some of the aviation taxes  The Federal Aviation Reauthorization Act of 1994 ( P L  103-305 ) reauthorized AIP for two more years and again made modifications in the cap and penalty provisions  Federal Aviation Reauthorization Act of 1996 ( P L  104-264 ) The 1996 reauthorization of the AIP made a number of adjustments to entitlement funding and discretionary set-aside provisions  It also included directives concerning intermodal planning, cost reimbursement rules, letters of intent, and the small airport fund  A demonstration airport privatization program and a demonstration program for innovative financing techniques were established  The demonstration status of the state block grant program was removed  The act did not reauthorize the taxes that supported the Airport and Airway Trust Fund  This was done by the Taxpayer Relief Act of 1997 ( P L  105-34 ), which extended, subject to a number of modifications, the existing aviation trust fund taxes through September 30, 2007  The Wendell H  Ford Aviation Investment and Reform Act for the 21 st Century of 2000 (AIR21; P L  106-181 ) The enactment of AIR21 was the culmination of two years of legislative effort to pass a multiyear FAA reauthorization bill  The initial debate focused on provisions to take the aviation trust fund off-budget or erect budgetary "firewalls" to assure that all trust fund revenues and interest would be spent each year for aviation purposes  These proposals, however, never emerged from the conference committee  Instead, the enacted legislation included a so-called "guarantee" that all of each year's receipts and interest credited to the trust fund would be made available annually for aviation purposes  AIR21 did not make major changes in the structure or functioning of AIP  It did, however, greatly increase the amount available for airport development projects  The AIP funding authorization rose from $1 9 billion in FY2000 to $3 4 billion in FY2003  The formula funding and minimums for primary airports were doubled starting in FY2001  The state apportionment for general aviation airports was increased from 18 5% to 20%  The noise set-aside was increased from 31% to 34% of discretionary funding and a reliever airport discretionary set-aside of 0 66% was established  AIR21 also increased the PFC maximum to $4 50 per boarding passenger  In return for imposing a PFC above the $3 level, large and medium hub airports would forgo 75% of their AIP formula funds  This had the effect of making a greater share of AIP funding available to smaller airports  Vision 100: Century of Aviation Reauthorization Act of 2003 ( P L  108-176 ; H Rept  108-334 ) Vision 100, signed by President George W  Bush on December 12, 2003, included significant changes to AIP  The law codified the AIR21 spending "guarantees" through FY2007  It increased the discretionary set-aside for noise compatibility projects from 34% to 35%  It increased the amount that an airport participating in the Military Airport Program (MAP) could receive to $10 million for FY2004 and FY2005, but in FY2006 and FY2007 it returned the maximum funding level to $7 million  The act allowed nonprimary airports to use their entitlements for revenue-generating aeronautical support facilities, including fuel farms and hangars, if the Secretary of Transportation determines that the sponsor has made adequate provisions for the airside needs of the airport  The law permitted AIP grants at small airports to be used to pay interest on bonds issued to finance airport projects  The act included a trial program to test procedures for authorizing small airports to impose PFCs  Vision 100 repealed the authority to use AIP or PFC funds for most airport security purposes  FAA Modernization and Reform Act of 2012 ( P L  112-95 ) The 2012 FAA reauthorization act funded AIP for four years from FY2012 to FY2015 at an annual level of $3 35 billion  A new provision, Section 138, permitted small airports reclassified as medium hubs due to increased passenger volumes to retain eligibility for up to a 90% federal share for a two-year transition period  This provision also allows certain economically distressed communities receiving subsidized air service to be eligible for up to a 95% federal share of project costs  The 2012 act maintained the $4 50 PFC cap, with a maximum charge of $18 per round-trip flight  It included a provision that instructed GAO to study alternative means for collecting PFCs  The act also expanded the number of airports that could participate in the airport privatization pilot program from 5 to 10  This law was extended through July 15, 2016  The FAA Extension, Safety, and Security Act of 2016 ( P L  114-190 ) The 2016 FAA extension act funded AIP through FY2017 at an annual level of $3 35 billion  A new provision, Section 2303, provided temporary relief to small airports that had 10,000 or more passenger boardings in 2012 but had fewer than 10,000 during the calendar year used to calculate the AIP apportionment for FY2017  This provision allowed such airports to receive apportionment for FY2017 an amount based on the number of passenger boardings at the airport during calendar year 2012  The FAA Reauthorization Act of 2018 ( P L  115-254 ) The 2018 FAA reauthorization act funded AIP for five years from FY2019 through FY2023 at an annual level of $3 35 billion  It also authorized supplemental annual funding from the general fund to the AIP discretionary funds—$1 02 billion in FY2019, $1 04 billion in FY2020, $1 06 billion in FY2021, $1 09 billion in FY2022, and $1 11 billion in FY2023—and required at least 50% of the additional discretionary funds to be available to nonhub and small hub airports  The act included a provision permitting eligible projects at small airports (including those in the State Block Grant Program) to receive 95% federal share of project costs (otherwise capped at 90%), if such projects are determined to be successive phases of a multiphase construction project that received a grant in FY2011  The 2018 reauthorization expanded the number of states that could participate in the State Block Grant Program from 10 to 20 and also expanded the existing airport privatization pilot program (now renamed the Airport Investment Partnership Program) to include more than 10 airports  The law included a provision that forbids states or local governments from levying or collecting taxes on a business on an airport that "is not generally imposed on sales or services by that State, political subdivision, or authority unless wholly utilized for airport or aeronautical purposes " Appendix B  Definitions of Airports Included in the NPIAS Commercial Service Airports Publicly owned airports that receive scheduled passenger service and board at least 2,500 passengers each year (506 airports)  Primary Airports   Airports that board more than 10,000 passengers each year  There are four subcategories: Large Hub Airports   Board 1% or more of system -wide boardings ( 30 airports, 72 % of all enplanements)Medium Hub Airports   Board 0 25% but less than 1% (31 airports, 16% of all enplanements) Small Hub Airports   Board 0 05% but less than 0 25% (72 airports, 8% of all enplanements) Non hub Primary Airports   Board more than 10,000 but less than 0 05% (247 airports, 3% of all enplanements) Non primary Commercial Service Airports   Board at least 2,500 but no more than 10,000 passengers each year (126 airports, 0 1% of all enplanements)  Other Airports General Aviation Airports   General aviation airports do not receive scheduled commercial or military service but typically do support business, personal, and instructional flying; agricultural spraying; air ambulances; on-demand air-taxies; and/or charter aircraft service (2,554 airports)  Reliever Airports   Airports designated by FAA to relieve congestion at commercial airports and provide improved general aviation access (261 airports)  Cargo Service Airports   Airports served by aircraft that transport cargo only and have a total annual landed weight of over 100 million pounds  An airport may be both a commercial service and a cargo service airport  New Airports Seven airports are anticipated to be built between 2019 and 2023  They include two primary airports, two nonprimary commercial service airports, and three general aviation airports 

ground_truth: The Department of Defense (DOD) obligates more than $300 billion annually to buy goods and services, and to support research and development. Most of these acquisitions are governed by procurement statutes and regulations found in Title 10 (and parts of other select titles) of the United States Code, the Federal Acquisition Regulation (FAR), and the Defense Federal Acquisition Regulation Supplement. Under certain circumstances, DOD can enter into an other transaction (OT) agreement instead of a traditional contract. OT agreements are generally exempt from federal procurement laws and regulations. These exemptions grant government officials the flexibility to include, amend, or exclude contract clauses and requirements that are mandatory in traditional procurements (e.g., termination clauses, cost accounting standards, payments, audit requirements, intellectual property, and contract disputes). OT authorities also grant more flexibility to structure agreements in numerous ways, including joint ventures; partnerships; consortia; or multiple agencies joining together to fund an agreement encompassing multiple providers. Other transaction agreements are legally binding contracts; they are referred to as agreements to distinguish them from the traditional procurement contracts governed by the FAR and procurements laws. Other transaction authorities are set forth in two sections of law: 10 U.S.C. 2371—granting authority to use OTs for basic, applied, and advanced research projects. 10 U.S.C. 2371b—granting authority to use OTs for prototype projects and follow-on production. Under this authority, a prototype project can only be conducted if at least one nontraditional defense contractor significantly participates in the project; all significant participants are small businesses or nontraditional defense contractors; at least one-third of the total cost of the prototype project is provided by nongovernment participants; or the senior procurement acquisition official provides a written justification for using an OT. Follow-on production can only be conducted when the underlying prototype OT was competitively awarded, and the prototype project was successfully completed. OTs have the potential to provide significant benefits to DOD, including attracting nontraditional contractors with promising technological capabilities to work with DOD, establishing a mechanism to pool resources with other entities to facilitate development of, and obtain, state-of-the-art dual-use technologies, and offering a unique mechanism for DOD to invest in, and influence the direction of, technology development. A number of analysts warn that along with the potential benefits come significant risks, including potentially diminished oversight and exemption from laws and regulations designed to protect government and taxpayer interests. In FY2017, DOD obligated $2.1 billion on prototype OT agreements, representing less than 1% of contract obligations for the year. However, the use of OTs is expected to grow at a rapid pace, due in part to recent statutory changes expanding other transaction authorities. A number of analysts and officials have raised concerns that if DOD uses OTs in ways not intended by Congress—or is perceived to abuse the authority—Congress could clamp down on the authority. Generally, DOD lacks authoritative data that can be used to measure and evaluate the use of other transaction authorities.

prompt_front: You are given two reports by the government agency.



Question: Write a one-page summary of the report 2.



Report 2: GPRAMA significantly enhances GPRA, the centerpiece of a statutory framework that Congress put in place during the 1990s to help resolve longstanding performance and management problems in the federal government and provide greater accountability for results  Congress passed GPRAMA in 2010 to address a number of persistent federal performance challenges, including focusing attention on crosscutting issues and enhancing the use and usefulness of performance information  OMB and agencies are to establish various government-wide and agency-specific performance goals, in line with GPRAMA requirements or OMB guidance  These include the following: Cross-agency priority (CAP) goals: CAP goals are crosscutting and include outcome-oriented goals covering a limited number of policy areas as well as goals for management improvements needed across the government  OMB is to coordinate with agencies to establish CAP goals at least every 4 years  OMB is also required to coordinate with agencies to develop annual federal government performance plans to, among other things, define the level of performance to be achieved toward the CAP goals  Strategic objectives: A strategic objective is the outcome or impact the agency is intending to achieve through its various programs and initiatives  Agencies establish strategic objectives in their strategic plans and may update the objectives during the annual update of performance plans  Agency priority goals (APG): At the agency level, every 2 years, GPRAMA requires that the heads of certain agencies, in consultation with OMB, identify a subset of agency performance goals as APGs  These goals are to reflect the agencies’ highest priorities  They should be informed by the CAP goals as well as consultations with relevant congressional committees and other interested parties  In a schedule established by GPRAMA, OMB and agencies are to develop and publish new CAP goals, APGs, and strategic plans (with updated strategic objectives) in February 2018  GPRAMA and related OMB guidance require agencies to regularly assess their progress in achieving goals and objectives through performance reviews  Data-driven reviews: Agency leaders and managers are to use regular meetings, at least quarterly, to review data and drive progress toward key performance goals and other management-improvement priorities  For each APG, GPRAMA requires agency leaders to conduct reviews at least quarterly to assess progress toward the goal, determine the risk of the goal not being met, and develop strategies to improve performance  Similarly, the Director of OMB, with relevant parties, is to review progress toward each CAP goal  Strategic reviews: OMB guidance directs agency leaders to annually assess progress toward achieving each strategic objective using a broad range of evidence  GPRAMA establishes certain senior leadership positions and a council, as described below  Chief Operating Officer (COO): The deputy agency head, or equivalent, is designated COO, with overall responsibility for improving agency management and performance  Performance Improvement Officer (PIO): Agency heads are to designate a senior executive within the agency as the PIO  The PIO reports directly to the COO and assists the agency head and COO with various performance management activities  Goal leaders: Goal leaders are responsible for developing strategies to achieve goals, managing execution, and regularly reviewing performance  GPRAMA requires goal leaders for CAP goals and agency performance goals, including APGs  OMB guidance directs agencies to designate goal leaders for strategic objectives  Performance Improvement Council (PIC): The PIC is charged with assisting OMB to improve the performance of the federal government and achieve the CAP goals  The PIC is chaired by the Deputy Director for Management at OMB and includes agency PIOs from each of the 24 CFO Act agencies as well as other PIOs and individuals designated by the chair  Among its responsibilities, the PIC is to work to resolve government-wide or crosscutting performance issues, and facilitate the exchange among agencies of practices that have led to performance improvements within specific programs, agencies, or across agencies  GPRAMA includes several provisions related to providing the public and Congress with information, as described below  Performance gov: GPRAMA calls for a single, government-wide performance website to communicate government-wide and agency performance information  Among other things, the website— implemented by OMB as Performance gov—is to include (1) quarterly progress updates on CAP goals and APGs; (2) an inventory of all federal programs; and (3) agency strategic plans, annual performance plans, and annual performance reports  Reporting burden: GPRAMA establishes a process to reexamine the usefulness of certain existing congressional reporting requirements  Specifically, GPRAMA requires an annual review (including congressional consultation), based on OMB guidance, of agencies’ reporting requirements to Congress  Additionally, OMB is to include in the budget a list of plans and reports determined to be outdated or duplicative and may submit legislation to eliminate or consolidate such plans or reports  In early 2017, the administration announced several efforts that are intended to improve government performance  The 2018 Budget Blueprint states that the President’s Management Agenda will seek to improve the federal government’s effectiveness by using evidence-based approaches, balancing flexibility with accountability to better achieve results, improving mission support functions, and developing and monitoring critical performance measures  In addition, OMB issued several memoranda detailing the administration’s plans to improve government performance by reorganizing the government, reducing the federal workforce, and reducing federal agency burden  A number of these efforts, which are to leverage GPRAMA and our past work, have the potential to further progress in addressing key governance challenges  As part of reorganization efforts, OMB and agencies are developing government-wide and agency reform plans, respectively, that are to leverage various GPRAMA provisions  For example, an April 2017 memorandum states that OMB intends to monitor implementation of the reform plans using CAP goals, APGs, annual strategic reviews, and Performance gov  The government-wide plan also is to include crosscutting reform proposals, such as merging agencies or programs that have similar missions  To that end, the memorandum states agencies should consider our reports, including our work on fragmentation, overlap, and duplication, as well as inspectors general reports  Many of the meaningful results that the federal government seeks to achieve, such as those related to ensuring public health, providing homeland security, and promoting economic development, require the coordinated efforts of more than one federal agency, level of government, or sector  For more than 2 decades, we have reported on agencies’ missed opportunities for improved collaboration through the effective implementation of GPRA and, more recently, GPRAMA  Our reports also have demonstrated that collaboration across agencies is critical to address issues of fragmentation, overlap, and duplication as well as many of the areas on our High-Risk List  Fragmentation, Overlap, and Duplication: Since 2011, our annual reports have identified 133 crosscutting areas that require the coordinated effort of more than one federal organization, level of government, or sector  For instance, for the area of federal grant awards, we found in January 2017 that the National Park Service (NPS), Fish and Wildlife Service, Food and Nutrition Service, and Centers for Disease Control and Prevention (CDC) had not established guidance and formal processes to ensure their grant-management staff review applications for potential duplication and overlap among grants in their agencies before awarding  We recommended that these agencies do so, and they agreed  As of August 2017, these agencies had taken several actions to address the recommendation  For example, the Department of the Interior (Interior) provided documentation showing that the Fish and Wildlife Service now requires discretionary grant applicants to provide a statement that addresses whether there is any overlap or duplication of proposed projects or activities to be funded by the grant  Fish and Wildlife also updated its guidance to grant awarding offices instructing them to perform a potential overlap and duplication review of all selected applicants prior to award  Our Action Tracker provides details on the status of actions from our annual reports  Within the 133 crosscutting areas, since 2011 we have identified 315 targeted actions where opportunities exist to better manage fragmentation, overlap, and duplication, including 29 new actions in our most recent report issued in April 2017  We found that the executive branch and Congress addressed 145 (46 percent) of the 315 actions  For example, in November 2014, we recommended that the U S  Coast Guard and Consumer Product Safety Commission establish a formal approach to coordination (such as a memorandum of understanding) to facilitate information sharing; better leverage their resources; and address challenges, including those related to fragmentation and overlap that we identified  In response to this recommendation, the two agencies signed a formal policy document to govern their coordination in May 2015  This policy document outlined procedures for determining jurisdictional authority for recreational boat-associated equipment and marine safety items  Specifically, the procedures clarified that upon receiving notice of a possible defect, the agency receiving such notice shall determine whether the item properly falls within its jurisdiction, and if not, initiate discussions to determine the appropriate jurisdiction  These new procedures should help the agencies share information and leverage each other’s resources so they can better ensure that recreational boat-associated equipment and marine safety items are fully regulated  However, more work is needed on the remaining 170 actions (54 percent) that have not been fully addressed  For example, in July 2016, we reported that four federal agencies—the Departments of Defense, Education, Health and Human Services, and Justice—manage at least 10 efforts to collect data on sexual violence, which differ in target population, terminology, measurements, and methodology  We found that data collection efforts use 23 different terms to describe sexual violence  Data collection efforts also differed in how they categorized particular acts of sexual violence, the context in which data were collected, data sources, units of measurement, and time frames  We recommended that OMB convene an interagency forum to better manage fragmentation of efforts to collect sexual violence data  In commenting on that report, OMB stated it would consider implementing the action in the future but did not believe it was the most effective use of resources at that time, in part because the agencies were not far enough along in their research  In response, we stated that given the number of federal data collection efforts, the range of differences across them, and the potential for causing confusion, it would be beneficial for agencies to discuss these differences and determine whether they are, in fact, necessary  As of July 2017, OMB had not provided an update on the status of this recommendation  High-Risk List: Since the early 1990s, our high-risk program has focused attention on government operations with greater vulnerabilities to fraud, waste, abuse, and mismanagement or that are in need of transformation to address economy, efficiency, or effectiveness challenges  As of February 2017, there were 34 high-risk areas covering a wide range of issues including human capital management, modernizing the U S  financial regulatory system, and ensuring the security of federal information systems and cyber critical infrastructure  Many of these high- risk areas require a coordinated response from more than one branch of government, agency, or sector  In the time between our 2015 and 2017 High-Risk Updates, many of these high-risk areas on our list demonstrated solid progress  During that period, 15 high-risk areas fully met at least one of the five criteria required for removal from the High-Risk List  In many cases, progress was possible through the joint efforts of Congress and leadership and staff in agencies  For example, Congress passed over a dozen laws following our 2015 High-Risk Update to help address high-risk issues  In addition, in 2017, we removed one high-risk area on managing terrorism-related information, because significant progress had been made to strengthen how intelligence on terrorism, homeland security, and law enforcement is shared among federal, state, local, tribal, international, and private sector partners  Despite this progress, continued oversight and attention is also warranted given the issue’s direct relevance to homeland security as well as the constant evolution of terrorist threats and changing technology  Our February 2017 High-Risk Update also highlighted a number of long- standing high-risk areas that require additional attention  We also added three new crosscutting areas to incorporate the management of federal programs that serve tribes and their members, the government’s environmental liabilities, and the 2020 decennial census  Based on our body of work on federal programs that serve tribes and their members, we concluded that federal agencies had (1) ineffectively administered Indian education and health care programs and (2) inefficiently fulfilled their responsibilities for managing the development of Indian energy resources  For example, we identified numerous challenges facing Interior’s Bureau of Indian Education (BIE) and Bureau of Indian Affairs, and the Department of Health and Human Services’ (HHS) Indian Health Service (IHS), in administering education and health care services  We concluded that these challenges put the health and safety of American Indians served by these programs at risk  In May 2017, we issued two additional reports on accountability for school construction and safety at schools funded by BIE  Although these agencies have taken some actions to address recommendations we made related to Indian programs, about 50 recommendations have yet to be fully resolved  We are monitoring federal efforts to address the unresolved recommendations  We also are reviewing IHS’s workforce, and tribal nations’ management and use of their energy resources  Many of the crosscutting areas highlighted by our annual reports on fragmentation, overlap, and duplication and designated as high-risk would benefit from enhanced collaboration among the federal agencies involved in them  GPRAMA establishes a framework aimed at taking a more crosscutting and integrated approach to focusing on results and improving government performance  Our survey results and past work demonstrate that agencies continue to face difficulties when working together on crosscutting issues, but also that implementing certain GPRAMA requirements can have a positive effect on collaboration  An item related to coordination in our survey of federal managers is statistically significantly lower in 2017, relative to our previous survey in 2013 and our initial survey in 1997  In 2017, an estimated 43 percent of managers agreed that they use information obtained from performance measurement to a great or very great extent when coordinating program efforts with internal or external organizations (compared to an estimated 50 percent in 2013 and an estimated 57 percent in 1997)  Moreover, our past work has found that agencies face a variety of challenges when working across organizational boundaries to deliver programs and improve performance  For example, our work has found that interagency groups have, at times, encountered difficulty clarifying roles and responsibilities or developing shared outcomes and performance measures  In contrast, our past work demonstrates that implementing GPRAMA provisions can improve collaboration  For example, in May 2016, we found that OMB and the PIC updated the governance structure for CAP goals to include both agency-level and Executive Office of the President goal leaders and held regular, senior-level reviews on CAP goal progress  Moreover, CAP goal teams told us that the CAP goal designation increased leadership attention and improved interagency collaboration on their crosscutting issues  Furthermore, our prior work has found that priority goals and related data-driven reviews have also been used to help manage crosscutting issues and enhance collaboration  Various GPRAMA requirements are aimed at improving agencies’ coordination of efforts to address crosscutting issues  As with our 2013 survey, our 2017 survey continues to show that CAP goals, APGs, and related data-driven reviews—also called quarterly performance reviews (QPR)—are associated with reported higher levels of collaboration with internal and external stakeholders  For example, our 2017 survey data indicate that about half of federal managers (an estimated 54 percent) reported they were somewhat or very familiar with CAP goals  Among these individuals, those who viewed their programs as contributing to CAP goals to a great or very great extent (36 percent) were more likely to report collaborating outside their program to a great or very great extent to help achieve CAP goals (62 percent), as shown in figure 2  Our analysis shows a similar pattern exists for APGs and QPRs  Our past work also has highlighted ways in which OMB and agencies could better implement GPRAMA’s crosscutting provisions—many of which have been addressed  A continued focus on fully and effectively implementing these provisions will be important as OMB and agencies establish new CAP goals and APGs, and assess progress toward them through related QPRs  Cross-agency priority (CAP) goals: In May 2012 and June 2013, we found that OMB had not always identified relevant agencies and program activities as contributors to the initial set of CAP goals  OMB took actions in response to our recommendations to include relevant contributors  Our most recent review, in May 2016, found that all relevant contributors had been identified for a subsequent set of CAP goals  In that report, we also found that OMB and the PIC had improved implementation of the CAP goals, in part, by helping agencies build their capacity to contribute to implementing the goals  Appendix II summarizes our past recommendations related to GPRAMA and the actions agencies have taken to address them  Agency priority goals (APGs): In April 2013, we found that agencies did not fully explain the relationship between their APGs and crosscutting efforts  Identify contributors: Similar to OMB’s responsibilities with the CAP goals, agencies are to identify the various organizations and programs that contribute to each of their performance goals, including APGs  We found that agencies identified internal contributors for their APGs, but did not list external contributors in some cases  We recommended that the Director of OMB ensure that agencies adhere to OMB’s guidance for website updates by providing complete information about the organizations, program activities, regulations, tax expenditures, policies, and other activities—both within and external to the agency— that contribute to each APG  In response, in April 2015, OMB asked agencies to identify organizations, program activities, regulations, policies, tax expenditures, and other activities contributing to their 2014-2015 APGs  Based on an analysis of the final quarterly updates for those APGs, published in December 2015, we found that agencies made progress in identifying external organizations and programs for their APGs  Describe how agency goals contribute to CAP goals: Agencies generally did not identify how their APGs contributed to CAP goals  We recommended that OMB direct agencies to describe in their performance plans how the agency’s performance goals—including APGs—contribute to any of the CAP goals as required by GPRAMA  In response, in July 2013, OMB updated its guidance directing agencies to include a list of the CAP goals to which the agency contributes and explain the agency’s contribution to them in their strategic plans, performance plans, and performance reports  Data-driven reviews: For their data-driven reviews of agency priority goals, agencies are to include, as appropriate, relevant personnel within and outside the agency who contribute to the accomplishment of each goal  However, in February 2013, we found that most Performance Improvement Officers (PIO) we surveyed (16 of 24) indicated that there was little to no involvement in these reviews from external officials who contribute to achieving agency goals  We recommended that OMB and the PIC help agencies extend their QPRs to include, as relevant, representatives from outside organizations that contribute to achieving their APGs  OMB staff told us that they generally concurred with the recommendation, but believed it would not always be appropriate to regularly include external representatives in agencies’ data-driven reviews, which they considered to be internal management meetings  In a subsequent review, we found in July 2015 that PIOs at 21 of the 22 agencies we surveyed said that their data-driven reviews had a positive effect on collaboration among officials from different offices or programs within the agency  Despite the positive effects, most agency PIOs (17) indicated that there continued to be little to no involvement in the reviews from external officials who contribute to achieving agency goals  In May 2016, OMB and PIC staff reported that, in response to our earlier recommendation, they were working with agencies to identify examples where agencies included representatives from outside organizations in data-driven reviews, and to identify promising practices based on those experiences  PIC staff told us they would disseminate any promising practices identified through the PIC Internal Reviews Working Group and other venues  In August 2017, OMB staff told us they plan to hold a summit with agencies later in the year to discuss implementing various performance management requirements, which could include agencies highlighting experiences and promising practices related to involving external officials in their data-driven reviews  We continue to believe data- driven reviews should include any relevant contributors from outside organizations and will continue to monitor progress  Despite the important role priority goals and related reviews can play in addressing crosscutting issues and enhancing collaboration, OMB recently removed the priority status of the current sets of priority goals  According to OMB staff, removing the priority designation from CAP goals and APGs returned them to regular performance goals, which are not subject to quarterly data-driven reviews or updates on the results of those reviews on Performance gov  In a June 2017 memorandum, OMB stated that CAP goals and APGs are intended to focus efforts toward achieving the priorities of current political leadership, and therefore reporting on the priority goals of the previous administration on Performance gov was discontinued for the remainder of the period covered by the goals (through September 30, 2017, the end of fiscal year 2017)  The memorandum further noted that agencies and teams working on those goals should continue working on the current goals where they align with the priorities of the current administration  Moreover, the memorandum states that agencies have flexibility in structuring their data-driven reviews, but they should continue such reviews focused on agency priorities  When asked about these actions, OMB staff told us that they believed they were working in line with the intentions of GPRAMA, which realigned the timing of goal setting with presidential terms, to better take into account changes in priorities  This is the first presidential transition since GPRAMA was enacted, and OMB staff told us they thought the act was unclear on how to handle priority goals during the changes in administrations and priorities  They stated that it was not practical to continue reporting on the priority goals of the prior administration as agencies worked to develop new strategic plans and priority goals for publication in February 2018  Hence, they told us OMB ended the current round of CAP goals and directed agencies to remove the priority designation from the APGs, returning them to regular performance goals  OMB staff further told us that although the guidance was published in a June 2017 memorandum, these decisions had been made and previously communicated to agencies during the transition in administrations  Therefore, reporting on the fiscal year 2014-2017 CAP goals, fiscal year 2016-2017 APGs, and related reviews stopped much earlier in the year, well before goal cycles were planned to be completed on September 30, 2017  OMB staff further stated that although the goals no longer had priority designations, work towards them largely continued in 2017  For example, one of the prior administration’s CAP goals was to modernize the federal permitting and review process for major infrastructure projects  OMB staff told us that they and agencies have continued many of the activities intended to achieve that goal, but they are no longer subject to quarterly data-driven reviews or updates on the results of these reviews on Performance gov  Moreover, they expect most of this work will continue towards a new and refocused CAP goal on infrastructure permitting modernization  OMB staff reaffirmed to us their intentions to resume implementation of CAP goals, APGs, and related data-driven reviews when the new planning and reporting cycle begins in February 2018  This is in line with stated plans to leverage various GPRAMA provisions to track progress of proposed government-wide and agency-specific reforms, as outlined in OMB’s April 2017 memorandum on the reform plans  In addition, OMB’s July 2017 update to its guidance for implementing GPRAMA similarly focuses on continued implementation of the act  Additional aspects of GPRAMA implementation could similarly help improve the management of crosscutting issues  Strategic reviews: OMB’s 2012 guidance implementing GPRAMA established a process in which agencies, beginning in 2014, were to conduct leadership-driven, annual reviews of their progress toward achieving each strategic objective established in their strategic plans  As we found in July 2015, effectively implementing strategic reviews could help identify opportunities to reduce, eliminate, or better manage instances of fragmentation, overlap, and duplication  Under OMB’s guidance, agencies are to identify the various organizations, program activities, regulations, tax expenditures, policies, and other activities that contribute to each objective, both within and outside the agency  Where progress in achieving an objective is lagging, the reviews are intended to identify strategies for improvement, such as strengthening collaboration to better address crosscutting challenges, or using evidence to identify and implement more effective program designs  If successfully implemented in a way that is open, inclusive, and transparent—to Congress, delivery partners, and a full range of stakeholders—this approach could help decision makers assess the relative contributions of various programs to a given objective  Successful strategic reviews could also help decision makers identify and assess the interplay of public policy tools that are being used to ensure that those tools are effective and mutually reinforcing, and that results are being efficiently achieved  In July 2017, OMB released guidance which updated the status of the 2017 strategic reviews  Because agencies are currently developing new strategic goals and objectives, OMB stated that agencies may forego the reporting and categorization requirements for any current strategic objectives that an agency determines will be substantively different or no longer aligned with the current administration’s policy, legislative, regulatory, or budgetary priorities  In addition, OMB stated that while there will be no formal meetings between OMB and the agencies to discuss findings and related progress from the 2017 strategic reviews, it expects that agencies will continue to conduct strategic reviews or assess progress made toward strategic goals and objectives aligned with administration policy  Furthermore, OMB stated that during this transition year, updates of progress on agency strategic objectives will only be published in the agency’s annual performance report and will not be reported to Performance gov  Full reporting through Performance gov is to resume after new agency strategic plans are published in February 2018  Agencies are to include a progress update for strategic objectives as part of their progress update in their fiscal year 2017 annual performance reports  Agencies also must address next steps for performance improvement as part of their fiscal year 2019 annual performance plans  Program inventories: GPRAMA requires OMB to publish a list of all federal programs, along with related budget and performance information, on a central government-wide website  Such a list could help decision makers and the public fully understand what the federal government does, how it does it, and how well it is doing  An inventory of federal programs could also be a critical tool to help decision makers better identify and manage fragmentation, overlap, and duplication across the federal government  Agencies developed initial program inventories in May 2013, but since then have not updated or more fully implemented these inventories  In October 2014, we found several issues limited the completeness, comparability, and usefulness of the May 2013 program inventories  OMB and agencies did not take a systematic approach to developing comprehensive inventories  For example, OMB’s guidance in Circular No  A-11 presented five possible approaches agencies could take to define their programs and noted that agencies could use one or more of those approaches in doing so  We found that because the agencies used inconsistent approaches to define their programs, the comparability of programs was limited within agencies as well as government-wide  In addition, we found that the inventories had limited usefulness for decision making, as they did not consistently provide the program and related budget and performance information required by GPRAMA  Moreover, we found that agencies did not solicit feedback on their inventories from external stakeholders—which can include Congress, state and local governments, third party service providers, and the public  Doing so would have provided OMB and agencies an opportunity to ensure they were presenting useful information for stakeholder decision making  We concluded that the ability to tag and sort information about programs through a more dynamic, web-based presentation could make the inventory more useful  In October 2014, we made several recommendations to OMB to update relevant guidance to help develop a more coherent picture of all federal programs and to better ensure relevant information is useful for decision makers  For example, we recommended that OMB revise its guidance to direct agencies to consult with relevant congressional committees and stakeholders on their approach to defining and identifying programs when developing or updating their inventories  OMB staff generally agreed with these recommendations, but have not yet taken any actions to implement them  OMB’s guidance for the program inventory has largely remained unchanged since 2014, when OMB postponed further development of the program inventory and eliminated portions of the guidance  For example, the guidance no longer describes, or provides directions for agencies to meet, GPRAMA’s requirements for presenting related budget or performance information for each program  OMB decided to postpone implementing a planned May 2014 update to the program inventory in order to coordinate with the implementation of the public spending reporting required by the Digital Accountability and Transparency Act of 2014 (DATA Act)  OMB subsequently stated that it would not begin implementing the program inventory until after the DATA Act was implemented in May 2017, despite requirements for regular updates to the program inventory to reflect current budget and performance information  The DATA Act is now being implemented, but OMB has postponed resuming the development of the program inventory  In July 2017, OMB staff told us that they are now considering how to align GPRAMA’s program inventory provisions with future implementation of the Program Management Improvement Accountability Act (PMIAA)  This was reflected in OMB’s July 2017 update to its guidance, which states that OMB is working with agencies to determine the right strategy to merge the implementation of the DATA Act and PMIAA with GPRAMA’s program inventory requirements to the extent possible to avoid duplicating efforts  For example, PMIAA requires OMB to coordinate with agency Program Management Improvement Officers to conduct portfolio reviews of agency programs to assess the quality and effectiveness of program management  GPRAMA requires OMB to issue guidance for implementing the program inventory requirements, among other things  Moreover, federal internal control standards state that organizations should clearly define what is to be achieved, who is to achieve it, how it will be achieved, and the time frames for achievement  As described above, OMB’s current guidance for the program inventory lacks some of those details—such as describing and providing direction to meet GPRAMA’s requirements for budget and performance information—in part because OMB is working with agencies to determine a strategy for implementation  Ensuring all GPRAMA requirements are covered and taking action on our past recommendations would help OMB improve its guidance to more fully implement the program inventory and improve its usefulness  To that end, in a report issued earlier this month, we identified a series of iterative steps that OMB could use in directing agencies to develop a useful inventory, as described in figure 3  A useful inventory would consist of all programs identified, information about each program, and the organizational structure of the programs  Our work showed that the principles and practices of information architecture—a discipline focused on organizing and structuring information—offer an approach for developing such an inventory to support a variety of uses, including increased transparency for federal programs  Such a systematic approach to planning, organizing, and developing the inventory that centers on maximizing the use and usefulness of information could help OMB ensure the inventory is implemented in line with GPRAMA requirements and meets the needs of decision makers and the public, among others  OMB’s guidance also lacks specific time frames, with associated milestones for resuming implementation of the program inventory requirements  As part of PMIAA’s requirements, OMB is to issue standards, policies, and guidelines for program and project management for agencies by December 2017  OMB staff told us that, within a year after that, they expect to issue further guidance on moving forward with resuming the program inventory  However, that general time frame was not reflected in the July 2017 update to OMB’s guidance  Providing specific time frames and associated milestones would bring the program inventory guidance in line with other portions of OMB’s guidance for implementing GPRAMA requirements, which contains a timeline of various performance planning and reporting requirements, including specific dates for meeting those requirements and related descriptions of required actions  For example, OMB’s July 2017 guidance identifies over 30 actions agencies should take between June 2017 and December 2018 to implement various GPRAMA provision  More specific time frames and milestones related to the program inventory requirements would help agencies prepare for resumed implementation by allowing them to know what actions they would be expected to take and by when  Moreover, publicly disclosing planned implementation time frames and associated milestones also would help ensure that external stakeholders are prepared to engage with agencies as they develop and update their program inventories  Effectively implementing various GPRAMA tools could help inform assessments of the performance of tax expenditures, which are reductions in tax liabilities that result from preferential provisions (figure 4)  In fiscal year 2016, tax expenditures represented an estimated $1 4 trillion in forgone revenue, an amount greater than total discretionary spending that year  Despite the magnitude of these investments, our work has also shown that little has been done to determine how well specific tax expenditures work to achieve their stated purposes and how their benefits and costs compare to those of spending programs with similar goals  GPRAMA requires OMB to identify tax expenditures that contribute to the CAP goals  In addition, OMB guidance directs agencies to identify tax expenditures that contribute to their strategic objectives and APGs  However, our past work reviewing GPRAMA implementation found that OMB and agencies rarely identified tax expenditures as contributors to these goals  Fully implementing our recommendation to identify how tax expenditures contribute to various goals could help the federal government establish a process for evaluating the performance of tax expenditures  To that end, in May 2017, we provided the Director of OMB with three priority recommendations that require attention: Develop framework for reviewing performance: In June 1994, and again in September 2005, we recommended that OMB develop a framework for reviewing tax expenditure performance  We explained that the framework should (1) outline leadership responsibilities and coordination among agencies with related responsibilities, (2) set a review schedule, (3) identify review methods and ways to address the lack of credible tax expenditure performance information, and (4) identify resources needed for tax expenditure reviews  Since their initial efforts in 1997 and 1999 to outline a framework for evaluating tax expenditures and preliminary performance measures, OMB and the Department of the Treasury (Treasury) have ceased to make progress and retreated from setting a schedule for evaluating tax expenditures  Inventory tax expenditures: In October 2014, we found that OMB had not included tax expenditures in the federal program inventory, and therefore was missing an opportunity to increase the transparency of tax expenditures and the outcomes to which they contribute  We recommended that OMB should designate tax expenditures as a program type in relevant guidance, and develop, in coordination with the Secretary of the Treasury, a tax expenditure inventory that identifies each tax expenditure and provides a description of how the tax expenditure is defined, its purpose, and related budget and performance information  OMB staff said they neither agreed nor disagreed with these recommended actions  As noted earlier, OMB has not resumed updates to the program inventory  Therefore, OMB had not taken any actions in response to this recommendation, according to OMB staff as of July 2017  Identify contributions to agency goals: In July 2016, we found that agencies had made limited progress identifying tax expenditures’ contribution to agency goals, as directed by OMB guidance  As of January 2016, 7 of the 24 CFO Act agencies identified tax expenditures as contributing to their missions or goals  The 11 tax expenditure they identified—out of the 169 tax expenditures included in the President’s Budget for Fiscal Year 2017—represented approximately $31 9 billion of the $1 2 trillion in estimated forgone revenues for fiscal year 2015  (See figure 5 ) To help address this issue, we recommended that OMB, in collaboration with the Department of the Treasury, work with agencies to identify which tax expenditures contribute to their agency goals, as appropriate  In particular, we recommended that they identify which specific tax expenditures contribute to specific strategic objectives and APGs  In July 2017, OMB staff said they had taken no actions to address the recommendation  Our July 2016 report also identified options for policymakers to further incorporate tax expenditures into federal budgeting processes, several of which options align with the recommendations discussed above  These options could help achieve various benefits, but we also reported that policymakers would need to consider challenges and tradeoffs in deciding whether or how to implement them  For example, one option was to require that all tax expenditures, or some subset of them, expire after a finite period  This option could result in greater oversight, requiring policymakers to explicitly decide whether to extend more or all tax expenditures  One consideration with this option is that it could lead to frequent changes in the tax code, such as from extended or expired tax expenditures, which can create uncertainty and make tax planning more difficult  Our previous work has shown that using performance information in decision making is essential to improving results  Performance information can be used across a range of management activities, such as setting priorities, allocating resources, or identifying problems to be addressed  However, our work continues to show that agencies can better use performance information in decision making, as shown in the example in the text box below  Department of Justice (DOJ) Could Better Analyze Performance Information to Reduce Backlog in Immigration Courts In June 2017, we found that the case backlog—cases pending from previous years that remain open at the start of a new fiscal year—at DOJ’s Executive Office for Immigration Review (EOIR) courts more than doubled from fiscal years 2006 through 2015  Stakeholders identified various factors that potentially contributed to the backlog, including continuances—temporary case adjournments until a different day or time  Our analysis of continuance records showed that the use of continuances increased by 23 percent from fiscal years 2006 through 2015  We found that EOIR collects continuance data but does not systematically assess them  Systematically analyzing the use of continuances could provide EOIR officials with valuable information about challenges the immigration courts may be experiencing, such as with operational issues like courtroom technology malfunctions, or areas that may merit additional guidance for immigration judges  Further, using this information to potentially address operational challenges could help that office meet its goals for completing cases in a timely manner  We recommended that the Director of EOIR systematically analyze immigration court continuance data to identify and address any operational challenges faced by courts or areas for additional guidance or training  EIOR agreed with this recommendation  EOIR stated that it supports conducting additional analysis of immigration court continuance data and recognizes that additional guidance or training regarding continuances may be beneficial to ensure that immigration judges use continuances appropriately in support of EOIR’s mission to adjudicate immigration cases in a careful and timely manner  We will monitor EOIR’s progress in taking these actions  Our 2017 survey of federal managers shows little change in their reported use of performance information  Using a set of survey questions, we previously developed an index that reflects the extent to which managers reported that their agencies used performance information for various management activities and decision making  The index suggests that government-wide use of performance information did not change significantly between 2013 and 2017, and it is statistically significantly lower relative to our 2007 survey, when we created the index  Figure 6 shows the questions included in the index and the government-wide results  In regard to individual survey items, in 2017 federal managers reported no changes or decreases in their use of performance information when compared to our last survey and when those survey items were first introduced  These results are generally consistent with our last few surveys  For example, in 2008 we found that there had been little change in federal managers’ reported use of performance information government-wide from 1997 to our 2007 survey  Citing those results, the Senate Committee on Homeland Security and Governmental Affairs report accompanying the bill that would become GPRAMA stated that agencies were not consistently using performance information to improve their management and results  The report further stated that provisions in GPRAMA are intended to address those findings and increase the use of performance information to improve performance and results  However, five items that were highlighted in our 2008 statement on the 2007 survey results generally show no improvement when compared to the 2017 results, as shown in figure 7  The one exception is for managers’ reported use of performance information to refine program performance measures  While this item was statistically significantly higher in 2013 relative to 2007—an estimated 46 percent to 53 percent—the 2017 result (43 percent) is a statistically significant decrease relative to 2013 and is not statistically different from the 2007 results  Another item, the use of performance information to adopt new program approaches or change work processes, also was statistically significantly lower in 2017 (47 percent) when compared to 2007 and 2013 (53 and 54 percent, respectively)  This is of particular concern as agencies are developing their reform plans  Moreover, when compared to our 1997 survey, the 2017 results show four of the five items are statistically significantly lower, and the remaining item—allocating resources—has not changed  Similarly, we found there was no improvement in 2017 for more recent survey items on other uses of performance information compared to the years in which they were introduced, as shown in figure 8  Although one item, on the use of performance information to develop program strategy, was statistically significantly higher in 2013 relative to 2007 (an estimated 58 and 51 percent, respectively), the 2017 result (53 percent) does not represent a statistically significant change from either of those years  Another item, on the use of performance information to streamline programs to reduce duplicative activities, is statistically significantly lower relative to 2013, when it was introduced (from 44 to 33 percent in 2017)  This is especially concerning because streamlining and reducing duplication are to be key parts of agencies’ reform plans  There is one area in the survey where we saw improvement: an estimated 46 percent of managers agreed to a great or very great extent that employees who report to them pay attention to their agency’s use of performance information in management decision making  That is statistically significantly higher relative to 2013 (40 percent), as well as when compared to when the item was introduced in 2007 (37 percent)  For a new and related item in the 2017 survey that asked managers the amount of attention their employees pay to the use of performance information in decision making when compared to 3 years ago, we found an estimated 48 percent reported that employees pay about the same 33 percent reported that employees pay somewhat or a great deal more attention  In September 2005, we identified five practices that agencies can apply to enhance the use of performance information in their decision making and improve results: demonstrating management commitment; communicating performance information frequently and efficiently; improving the usefulness of performance information, such as by ensuring the accessibility of the information; developing the capacity to use performance information; and aligning agency-wide goals, objectives, and measures  Many of the requirements put in place by GPRAMA reinforce the importance of these practices  Our 2017 survey of federal managers includes a number of items related to these practices  However, the 2017 results suggest that managers have not effectively adopted them  In the following sections, we examine several of the practices to enhance the use of performance information and their related survey items further  In doing so, we also highlight a subset of six survey items related to these practices that, while separate from those in our use of performance information index, we found in September 2014 to have a statistically significant and positive relationship with it  The commitment of agency leaders to results-oriented management is critical to increased use of performance information for policy and program decisions  GPRAMA requires top leadership involvement in performance management, including leading data-driven performance reviews  However, we have previously reported that improvements are needed to strengthen leadership’s commitment to use performance information, as discussed in the text box below  Department of Defense Should Strengthen Leadership Responsibilities for Using Performance Information In January 2005, we designated the Department of Defense’s (DOD) approach to business transformation as high-risk because DOD had not taken the necessary steps to achieve and sustain business reform on a broad, strategic, department-wide, and integrated basis  In the February 2017 update to our High-Risk List, we found that DOD had taken some positive steps to improve its business transformation efforts continuing to hold business function leaders accountable for diagnosing performance problems and identifying strategies for improvement, and leading regular DOD performance reviews regarding transformation goals and associated metrics and ensuring that business function leaders attend these reviews to facilitate problem solving  In July 2017, DOD officials told us that the department’s performance reviews have been put on hold until after the new Agency Strategic Plan is issued  We will review DOD’s updated Agency Strategic Plan when it is issued (expected in February 2018, as required by GPRAMA) to see if it addresses continuing to hold business function leaders accountable for diagnosing performance problems and identifying strategies for improvement  We will continue to monitor the status of these actions  GAO, High-Risk Series: Progress on Many High-Risk Areas, While Substantial Efforts Needed on Others, GAO-17-317 (Washington, D C : Feb  15, 2017)  Results from our 2017 survey show no statistically significant difference relative to 2013 in managers’ perceptions of leaders’ and supervisors’ attention and commitment to the use of performance information  (See figure 9 ) Three items are statistically significantly different from the years when they were introduced  Two items increased between 1997 and 2017: changes by management to my program(s) are based on results-oriented information (from an estimated 16 to 25 percent), and the individual I report to periodically reviews with me the outcomes of my program(s) (from 42 to 54 percent)  For the third item, top leadership demonstrates a strong commitment to using performance information to guide decision making, results decreased from 49 percent in 2007 to 42 percent in 2017  New items in the 2017 survey show some improvement in management commitment to the use of performance information in decision making  An estimated 36 percent of federal managers reported that, when compared to 3 years ago, the individual they report to pays somewhat or a great deal more attention to the use of performance information in decision making, while 46 percent said they pay about the same amount of attention  Additionally, an estimated 21 percent of federal managers said that, when compared to 3 years ago, the head of their agency pays somewhat or a great deal more attention to the use of performance information in decision making, while 33 percent said they pay about the same amount of attention  Communicating performance information frequently and effectively throughout an agency can help to achieve the agency’s goals  GPRAMA includes requirements for communicating performance information, such as reporting progress updates for APGs at least quarterly  However, our prior work has found that some agencies could continue to improve in the communication of performance information, as illustrated by the example in the text box below  Department of Education (Education) Could Better Share Effective Practices across States in Grant Program Education awards 21st Century Community Learning Centers grants to states, which in turn competitively award funds to local organizations that use them to offer academic enrichment and other activities to improve students’ academic and behavioral outcomes  In April 2017, we found that states are experiencing substantial difficulty in sustaining their programs after 21st Century funding ends  We further found that Education was missing opportunities in its monitoring efforts to collect information on states’ strategies and practices for program sustainability—information that could be useful for sharing promising practices across states  We recommended that Education use the information it collects from its monitoring visits and ongoing interactions with states to share effective practices across states for sustaining their 21st Century programs once program funding ends  Education neither agreed nor disagreed with the recommendation but outlined steps it is taking to address it  We will continue to monitor progress on the implementation of this recommendation  There is no difference for two survey items on federal managers communicating performance information relative to 2013 or since those items were introduced in 2007  In 2017, we estimate that 44 percent of federal managers agreed to a great or very great extent that agency managers at their level effectively communicate performance information on a routine basis  In addition, 34 percent agreed to a great or very great extent that managers at their level use performance information to share effective program approaches with others  Our 2017 survey data also indicate that agencies may not be effectively communicating to their employees about contributions to CAP goals or progress toward achieving APGs  Of the estimated 54 percent of federal managers who indicated they were familiar with CAP goals, 23 percent reported that their agency has communicated to its employees on those goals to a great or very great extent  Of the 74 percent of federal managers who indicated familiarity with APGs, 44 percent reported that their agency has communicated on progress toward achieving those goals to great or very great extent  Our prior work has shown that agencies should consider users’ differing needs—for accessibility, accuracy, completeness, consistency, ease of use, timeliness, and validity, among others things—to ensure that performance information will be both useful and used  GPRAMA introduced several requirements that could help to address aspects of usefulness, such as requiring agencies to disclose more information about the accuracy and validity of their performance data and actions to address limitations to the data  However, agencies face challenges in ensuring their performance information is useful, with one instance from our past work described in the text box below  The Environmental Protection Agency (EPA) Could Improve Usefulness of Information in Planned Grantee Portal EPA monitors performance reports and program-specific data from grantees to ensure that grants achieve environmental and other program results  However, in July 2016, we found that EPA’s 2014 internal analysis of its grants management business processes identified improvements that, if implemented into EPA’s planned web-based portal, could improve the accessibility and usefulness of information in grantee performance reports for EPA, grantees, and other users  We recommended, among other actions, that EPA incorporate expanded search capability features, such as keyword searches, into its proposed web- based portal for collecting and accessing performance reports to improve their accessibility  EPA agreed with our recommendation but stated that it is a long- term initiative, subject to the agency’s budget process and replacement of its existing grants management system  As of May 2017, EPA officials said that they have not begun work on the web-based portal project, which is subject to the availability of funds  Federal managers generally responded similarly in 2017 on a variety of survey items related to usefulness, relative to earlier surveys  On a broadly worded item, less than half of managers agreed to a great or very great extent that agency managers at their level take steps to ensure that performance information is useful and appropriate  At an estimated 43 percent in 2017, this represents no statistically significant change compared to our last surveys in 2013 or 2007, when the item was introduced  Responses to four survey items indicate no changes in hindrances related to the usefulness of performance information  There is no statistically significant change in managers reporting hindrances compared to 1997 or 2013, as shown in figure 10  In addition, there was a statistically significant increase when compared to 2013 on only one of six items about managers’ views on the usefulness of performance information, as shown in figure 11  As the figure shows, approximately one-third to half of managers agreed to a great or very great extent on each item related to the usefulness of performance information  Although less than half of managers reported having sufficient information on validity of performance data used to make decisions, this represents a statistically significant increase to an estimated 42 percent in 2017 compared to 36 percent in 2013, and from 28 percent in 2000, when this item was introduced  This is a notable improvement because our September 2014 report found that the strongest driver of the use of performance information was whether federal managers had confidence in its validity  Our analysis suggests that easy access to performance information is related to the effective communication of performance information  Of the estimated 49 percent of federal managers in 2017 who agreed to a great or very great extent that performance information is easily accessible to managers at their level, 63 percent also agreed that agency managers at their level effectively communicate performance information on a routine basis to a great or very great extent  Conversely, of the 20 percent that agreed to a small or no extent that performance information is easily accessible to managers at their level, 12 percent also agreed that agency managers at their level effectively communicate performance information on a routine basis to a great or very great extent  Our prior work has shown that building capacity—including analytical tools and staff expertise—is critical to using performance information in a meaningful manner  GPRAMA lays out specific requirements that reinforce the importance of staff capacity to use performance information  GPRAMA directed the Office of Personnel Management (OPM) to take certain actions to support agency hiring and training of performance management staff  Specifically, by January 2012, OPM was to identify skills and competencies needed by government personnel for setting goals, evaluating programs, and analyzing and using performance information for improving government efficiency and effectiveness  By January 2013, OPM was to incorporate these skills and competencies into relevant position classifications and to work with each agency to incorporate the identified skills into employee training  In April 2013, we found that OPM had completed its work on the first two responsibilities and taken steps to work with agencies to incorporate performance management staff competencies into training  However, OPM did not assess competency gaps among agency performance management staff to inform its work  Without this information, OPM, working with the PIC, was not well-positioned to focus on the most- needed resources and help other agencies use them  We recommended that the Director of OPM, in coordination with the PIC and the Chief Learning Officer Council, work with agencies to take the following three actions: 1  Identify competency areas needing improvement within agencies  2  Identify agency training that focuses on needed performance management competencies  3  Share information about available agency training on competency areas needing improvement  In July 2017, PIC staff stated they have not focused on identifying competency areas because the competencies do not resonate strongly with the performance community  Instead, staff said they identified a need for introductory training on performance management, which they have developed and piloted  They said that they are not sure when they will implement the training, since the PIC is reviewing priorities with its new executive director  We continue to believe that identifying the competency areas would be useful, and will monitor the PIC’s efforts to identify and share training  The need for performance management training is further highlighted by our survey results  Our 2017 survey shows no statistically significant change in managers’ responses about the availability of training on various performance management activities relative to 2013, including the use of performance information to make decisions  However, the response to each of the six questions related to specific training is statistically significantly higher relative to the year in which it was introduced, as shown in figure 12  Similarly, in 2017 there was no statistically significant change on four survey items related to agencies’ analysis and evaluation tools and staff’s skills and competencies when compared to 2013 or when these items were introduced  We estimate that in 2017 29 percent of managers agreed to a great or very great extent that their agencies were investing in resources to improve the agencies’ capacity to use performance information; 28 percent of managers agreed to a great or very great extent that their agencies were investing the resources needed to ensure that performance data are of sufficient quality; 33 percent of managers reported that they agreed to a great or very great extent that their agencies have sufficient analytical tools for managers at their levels to collect, analyze, and use performance information; and 33 percent of managers reported that they agree to a great or very great extent that the programs they are involved with have sufficient staff with the knowledge and skills needed to analyze performance information  Performance reviews can serve as a strategy to bring leadership and other responsible parties together to review performance information and identify important opportunities to drive performance improvements  Our prior work has examined how different types of performance reviews—strategic reviews, data-driven reviews, and retrospective regulatory reviews—can contribute to agencies assessing progress toward desired results  Strategic reviews: As previously mentioned, in implementing GPRAMA, OMB established a review process in which agencies are to annually assess their progress in achieving each strategic objective in their strategic plans, known as strategic reviews  Given the long-term and complex nature of many outcomes, the strategic review should be informed by a variety of evidence regarding the implementation of strategies and their effectiveness in achieving outcomes  OMB’s guidance states that the strategic review process should consider multiple perspectives and sources of evidence to understand the progress made on each strategic objective  It further states that the results of these reviews should inform many of the decision-making processes at the agency, as well as decision making by the agency’s stakeholders, in areas such as long-term strategy, budget formulation, and risk management  In 2017, agencies are completing their fourth round of these reviews  Our prior work has identified ways in which agencies can effectively conduct these reviews and leverage the results that come from them  In July 2015, we identified seven practices federal agencies can employ to facilitate effective strategic reviews  (See sidebar ) In addition, earlier this month we reported on selected agencies’ experiences in implementing these reviews  Specifically, we found that (1) strategic reviews helped direct leadership attention to progress on strategic objectives, (2) agencies used existing management and performance processes to conduct the reviews, and (3) agencies refined their reviews by capturing lessons learned  Data-driven reviews: GPRAMA requires agencies to review progress toward APGs at least once a quarter  The Senate Committee on Homeland Security and Governmental Affairs report accompanying the bill that would become GPRAMA stated that this approach is aimed at increasing the use of performance information to improve performance and results  In February 2013, we identified nine leading practices to promote successful data-driven performance reviews in the federal government  (See sidebar ) In July 2015, we found that most of the 24 CFO Act agencies were conducting their reviews in line with GPRAMA requirements and our leading practices  Moreover, agencies reported that their data-driven performance reviews had positive effects on progress toward agency goals, collaboration between agency officials, the ability to hold officials accountable for progress, and efforts to improve the efficiency of operations  Our 2017 survey shows that federal managers remain largely unfamiliar with their agency’s data-driven performance reviews, also known as quarterly performance reviews (QPRs)  An estimated 35 percent of managers reported familiarity with their agency’s QPRs  Survey results show that a greater percentage of Senior Executive Service (SES) managers than non-SES managers reported that they were familiar with QPRs  Approximately 50 percent of SES managers reported being somewhat or very familiar with QPRs; 34 percent of non-SES reported the same  However, for the estimated 35 percent of managers who reported familiarity with QPRs, the more they viewed their programs being subject to a QPR, the more likely they were to report their agency’s QPRs were driving results and conducted in line with our leading practices  Figure 13 shows several illustrative examples of these survey items  For example, of the estimated 48 percent of federal managers who reported their programs being subject to QPRs to a great or very great extent, 83 percent also reported their agencies use QPRs to identify problems or opportunities associated with agency performance goals  Conversely, for the 24 percent of managers who reported their programs were subject to QPRs to a small or no extent, 22 percent also reported the reviews were used for these purposes to a great or very great extent  Being subject to a QPR is also positively related to viewing QPRs as having led to similar meetings at lower levels  An estimated 62 percent of federal managers who reported being subject to QPRs to a great or very great extent also reported their agencies have similar meetings at lower levels to a great or very great extent  An estimated 16 percent of federal managers subject to QPRs to a small or no extent reported the same  Despite the reported benefits of and results achieved through QPRs, as found by our past work and survey data, these reviews are not necessarily widespread  GPRAMA requires agencies to conduct QPRs for APGs, which represent a small subset of goals—generally 2 to 8 priority goals at each designated agency, with approximately 100 total government-wide  Moreover, these required reviews are at the department (or major independent agency) level  These reasons may explain why most managers reported they were not familiar with the reviews  As was described previously, our 2017 survey data show that the reported use of performance information in decision making generally has not improved and in some cases is lower than it was 20 years ago  Survey data also show that managers generally have not reported increases in their employment of practices that further promote the use of performance information in decision making  This suggests that agencies could increase the use of performance information in decision making and the likelihood of achieving desired results by going beyond the specific GPRAMA requirements and expanding their use of data-driven performance reviews—in line with leading practices—to more broadly cover other agency-wide performance goals, as well as goals at lower levels within the agency  For example, such reviews at the program level could help inform the previously mentioned portfolio reviews required by the Program Management Improvement Accountability Act (PMIAA)  We have already suggested expanding reviews to other performance goals  Our management agenda for the presidential and congressional transition includes a key action to expand the use of data-driven performance reviews to assess progress toward meeting agency performance goals  Our prior work has stated that although GPRAMA’s requirements apply at the agency-wide level, they can also serve as leading practices at other organizational levels, such as component agencies, offices, programs, and projects  In addition, federal internal control standards call for the design of appropriate control activities, such as top-level reviews of actual performance and reviews by management at the functional or activity level  The standards also recommend that management design control activities at the appropriate levels in the organizational structure  The July 2017 update to OMB’s guidance states that agency leaders, including various chief officer positions, are to conduct frequent data- driven reviews to drive improvements on various management functions  For example, the agency Chief Human Capital Officer is to conduct quarterly data-driven reviews (known as HRStat) to monitor the progress of human capital goals and measures contained in the human capital operating plan  Beyond these management areas, OMB’s guidance also states that agencies may expand quarterly progress reviews beyond APGs to include other goals and priorities  However, OMB’s guidance does not identify practices for agencies to expand the use of these reviews to other goals, such as other agency-wide performance goals or those at lower levels within the agency  As mentioned previously, one of the responsibilities of the Performance Improvement Council (PIC) is to facilitate the exchange among agencies of practices that have led to performance improvements within specific programs, agencies, or across agencies  By working with the PIC to identify and share among agencies practices to expand the use of data- driven reviews, OMB could help agencies increase the use of performance information in decision making and achieve results  Retrospective regulatory reviews: In retrospective reviews, agencies evaluate how existing regulations are working in practice and whether they are achieving expected outcomes  GPRAMA requires agencies to identify and assess how their various program activities and other activities, including regulations, contribute to APGs  However, in April 2014, we found that agencies reported mixed experiences linking retrospective analyses to APGs  We recommended that OMB strengthen these reviews by issuing guidance for agencies to take actions to ensure that contributions made by regulations toward achieving APGs are properly considered, and improve how retrospective regulatory reviews can be used to help inform assessments of progress toward these APGs  OMB staff agreed with this recommendation and stated that the agency was working on strategies to help facilitate agencies’ ability to use retrospective reviews to inform APGs  To that end, in April 2017, OMB issued guidance to agencies that, among other things, emphasized the importance of performance measures related to evaluating and improving the net benefits of their respective regulatory programs  OMB included explicit references to section 6 of Executive Order 13563, which directed agencies’ efforts to conduct retrospective regulatory reviews  Specifically, the updated guidance encourages agencies to establish and report “meaningful performance indicators and goals for the purpose of evaluating and improving the net benefits of their respective regulatory programs ” The guidance further states that agencies’ efforts to improve such net benefits may be conducted as part of developing agency strategic and performance plans and priority goals  In July 2017, OMB confirmed that the updated guidance was issued, in part, to address our April 2014 recommendation  For several years, OMB has encouraged agencies to expand their use of evidence—performance measures, program evaluation results, and other relevant data analytics and research studies—in budget, management, and policy decisions with the goal of improving government effectiveness  In particular, OMB has encouraged agencies to strengthen their program evaluations—systematic studies that use research methods to address specific questions about program performance  Evaluation is closely related to performance measurement and reporting  Evaluations can be designed to better isolate the causal impact of programs from other external economic or environmental conditions in order to assess a program’s effectiveness  Thus, an evaluation study can provide a valuable supplement to ongoing performance reporting by measuring results that are too difficult or expensive to assess annually, explaining the reasons why performance goals were not met, or assessing whether one approach is more effective than another  Despite the valuable insights and information that program evaluations can provide, we continue to find that most federal managers lack access to or awareness of such studies  Our 2017 survey shows that an estimated 40 percent of managers reported that an evaluation had been completed within the past 5 years of any program, operation, or project in which they were involved—comparable to the results in our 2013 survey, when questions about program evaluations were added  In recent years, OMB has encouraged agencies to explore evidence-based tools to strengthen agency and grantee evaluation capacity, consider the effectiveness of their programs, and foster innovation rooted in research and rigorous evaluation  During the past 2 years, we examined several of those tools, as described below  Pay for success: Also known as social impact bonds, pay for success is a contracting mechanism under which investors provide the capital the government uses to provide a social service  The government specifies performance outcomes in pay for success contracts and generally includes a requirement that a program’s impact be independently evaluated  The evaluators also are to regularly review performance data, while those managing and investing in a project focus on performance and accountability, as shown in the figure 14  In September 2015, we found that the federal government’s involvement in pay for success had been limited  In addition, a formal mechanism for federal agencies to collaborate on pay for success did not exist  We concluded that, given the evolving nature of pay for success, a mechanism for federal agencies to collaborate would increase access to leading practices  We therefore recommended that OMB establish a formal means for federal agencies to collaborate on pay for success  OMB concurred and, in February 2016, announced that it had developed the Pay for Success Interagency Learning Network with representatives from 10 federal agencies to share lessons, hone policy, and strengthen implementation  Tiered evidence grants: Tiered evidence grants seek to incorporate evidence of effectiveness into grant making  Federal agencies establish tiers of grant funding based on the level of evidence grantees provide on their approaches to deliver social, educational, health, or other services  (See figure 15 ) Smaller awards are used to test new and innovative approaches, while larger awards are used to scale up approaches that have strong evidence of effectiveness  This creates incentives for grantees to use approaches supported by evidence and helps them build the capacity to conduct evaluations  In September 2016, we found that interagency collaboration had helped federal agencies that administer tiered evidence grants address challenges and share lessons learned  At that time, such collaborative efforts relied on informal networks  We recommended that OMB establish a formal means for agencies to collaborate on tiered evidence grants  OMB had no comment on the recommendation  In July 2017, OMB staff told us that they had established an interagency working group and other mechanisms to facilitate collaboration and disseminate information on tiered evidence grants  Performance partnerships: Performance partnerships allow federal agencies to provide grant recipients flexibility in how they use funding across two or more programs along with additional flexibilities  In exchange, the recipient commits to improve and assess progress toward agreed-upon outcomes  Figure 16 provides an overview of the performance partnership model  In April 2017, we examined two performance partnership initiatives authorized by Congress: the Environmental Protection Agency’s Performance Partnership Grants and the Performance Partnership Pilots for Disconnected Youth, which allows funding from multiple programs across multiple agencies to be combined into pilot programs serving disconnected youth  For the Performance Partnership Pilots for Disconnected Youth, we found that the agencies involved in the initiative had not fully identified the key financial and staff resources each agency would need to contribute over the lifetime of the initiative in line with leading practices for interagency collaboration  This was because agencies primarily had been focused on meeting near-term needs to support design and implementation  We also found that agencies had not developed criteria to help determine whether, how, and when to implement the flexibilities tested by the pilots in a broader context  (This is known as scalability ) Officials involved in the pilots told us it was too early in pilot implementation to determine such criteria  However, by not identifying these criteria while designing the pilots, they were risking not collecting needed data during pilot implementation  We recommended that OMB coordinate with federal agencies to identify (1) agency resource contributions needed for the lifetime of the pilots and (2) criteria and related data for assessing scalability  OMB neither agreed nor disagreed with these recommendations  We continue to monitor progress on these recommendations  In 2003, we identified nine key practices for effective performance management that collectively create a “line of sight” between individual performance and organizational success  (See sidebar on next page ) Our recent work and the results of our 2017 survey of federal managers highlight areas where agencies have made progress but could take additional action to better reflect several of these practices, thereby better instilling results-oriented cultures  Align individual performance expectations with organizational goals: Our 2003 report found that high-performing organizations use their performance management systems to help individuals see the connection between their daily activities and organizational goals  The executive branch has taken several steps to link individual and organizational results  For example, in October 2000, OPM issued guidance to link SES performance expectations with GPRA-required goals  In January 2012, OPM and OMB released a government-wide SES performance appraisal system that provided agencies with a standard framework to manage the performance of SES members  However, our work continues to identify areas for improvement  Goal leaders and deputy goal leaders are responsible for achieving APGs, but our July 2014 review found that the performance plans for a sample of goal and deputy goal leaders generally did not link their individual performance and the broader goal  We recommended that OMB ensure that those plans demonstrate a clear connection with APGs  OMB staff generally agreed with our recommendation  In July 2017, OMB staff stated that components of both OMB and OPM guidance support accountability for agency priority goals  Despite this, we continue to believe that ensuring an explicit connection in performance plans to APGs will improve accountability, and that additional action is needed to do so  In May 2016, we found that the Federal Emergency Management Agency (FEMA) had not aligned Federal Disaster Recovery Coordinators’ performance expectations with its organizational goals for implementing the National Disaster Recovery Framework  We concluded that without this linkage, FEMA could not evaluate how effectively the coordinators performed in implementing the framework  We recommended that FEMA align performance expectations consistent with leading practices  The Department of Homeland Security concurred with our recommendation  In July 2017, FEMA stated that it is preparing the Field Leader Manual, which will define the core competencies and duties of coordinators  We will continue to monitor FEMA’s actions to implement this recommendation  Our 2017 survey also shows that this linkage could be improved for other federal employees  An estimated 58 percent of federal managers reported using performance information to a great or very great extent in setting expectations for employees they manage or supervise  The 2017 responses do not represent a statistically significant change when compared to our last survey in 2013 (62 percent) or to 1997 (61 percent), the year this survey item was introduced  Address organizational priorities: Our prior work showed that, by requiring and tracking follow-up actions on performance gaps, high- performing organizations underscore the importance of holding individuals accountable for making progress on their priorities  Our past and 2017 surveys have identified differences in responses between SES and non-SES managers reporting being held accountable for results  For example, in 2017, our survey results indicate that there was a statistically significant difference between SES and non-SES managers reporting to a great or very great extent that they were held accountable for results of the programs for which they are responsible  However, our 2017 survey shows no change compared to our last survey in either SES or non-SES managers reporting they were held accountable for results  There are statistically significant increases when compared to 1997, when these survey items were introduced  For example, an estimated 79 percent of SES managers and 64 percent of non-SES managers reported being held accountable to a great or very great extent for results of the programs for which they are responsible in 2017  This does not represent a statistically significant change from our 2013 survey (80 percent and 67 percent, respectively), but it is statistically significantly higher than the 62 percent of SES managers and 54 percent of non-SES managers in 1997  (See figure 17 ) Similarly, as shown in figure 18, an estimated 71 percent of SES managers reported being held accountable to a great or very great extent for accomplishing agency strategic goals in 2017  This represents no statistical change since 2013 (73 percent), but it is a statistically significant increase compared to when this item was introduced in 2003 (61 percent)  Additionally, as figure 18 shows, a gap between being held accountable for strategic goals and having the decision-making authority needed to help accomplish those goals has nearly closed, due to an increase in the latter survey item  The estimated 69 percent of SES managers who reported having such authority to a great or very great extent in 2017 is a statistically significant increase relative to both 2013 (61 percent) and 1997 (51 percent)  As noted earlier, GPRAMA requires goal leaders for CAP goals and APGs  Our past work has generally found that they are in place  GPRAMA also requires agencies to identify an agency official responsible for resolving major management challenges, which can help ensure accountability  (See sidebar ) However, in June 2016 we found that 17 of the 24 CFO Act agencies had not identified an agency official responsible for resolving each of their challenges, partly because OMB guidance was not clear that major management challenges should be identified in agency performance plans  We recommended that the 17 agencies identify such officials in their performance plans, and that OMB clarify its guidance  OMB revised its guidance accordingly in July 2016, and, as of July 2017, 7 of the 17 agencies had identified officials responsible for resolving major management challenges  Link pay to individual and organizational performance: High- performing organizations seek to create pay, incentive, and reward systems that clearly link employee knowledge, skills, and contributions to organizational results  Our work has found that agencies have made progress in this area  For example, in July 2013, we found that the Securities and Exchange Commission (SEC) lacked mechanisms to monitor how supervisors used its performance management system to recognize and reward performance  To help enhance the credibility of SEC’s performance management system, we recommended that it create mechanisms to monitor how supervisors use the performance management system  In a subsequent (December 2016) report, we found that, in response to our recommendation, SEC began monitoring how supervisors provide feedback, recognize and reward staff, and address poor performance  However, federal managers generally reported no change on three items related to recognizing and rewarding employee performance since our last survey in 2013 (figure 19)  One of those items—managers agreeing to a great or very great extent that employees in their agency receive positive recognition for helping the agency to accomplish its strategic goals—had a statistically significant increase between 1997 and 2017 (from an estimated 26 percent to 46 percent)  Make meaningful distinctions in performance: Effective performance management requires the organization’s leadership to meaningfully distinguish between acceptable and outstanding performance of individuals and to appropriately reward those who perform at the highest level  For example, in January 2015, we found disparities in performance ratings for SES among agencies  Across the 24 CFO Act agencies, the percent of SES rated at the highest level ranged from about 22 percent to 95 percent in fiscal year 2013  To help address these disparities, we recommended that the Director of OPM consider the need to refine the performance certifications guidelines addressing distinctions in performance  To address this recommendation, OPM informed us, in June 2015, that it had convened a cross-agency working group that developed a standard template for agencies to complete and post on a website to more transparently justify their SES ratings distributions  In May 2016, we found that about 74 percent of non-SES employees under a five-level appraisal system—the most commonly used system— were rated in the top two of five performance categories in 2013  We explored this issue further in our December 2016 review of human capital challenges at the Veterans Health Administration (VHA), which illustrates the importance of making meaningful distinctions in performance for non- SES employees  We found that in fiscal year 2014, about 73 percent of VHA employees were rated in the top two of five performance categories  This may have been due, in part, to a policy that did not require standards to be defined for each level of performance  We recommended that VHA ensure that meaningful distinctions are being made in employee performance ratings by reviewing and revising performance management policies consistent with leading practices, among other actions  The Department of Veterans Affairs partially concurred with our recommendation  In May 2017, the department stated that it had begun piloting a new performance management process and would analyze results at the end of fiscal year 2017  One key aspect of connecting daily operations to results is aligning program performance measures to agency-wide goals and objectives  However, in 2017, an estimated 50 percent of federal managers agreed to a great or very great extent that managers at their level took steps to create such an alignment  There has been no statistically significant change since this item was introduced in 2007  In addition, GPRAMA calls for agencies to develop a balanced set of performance measures, which reinforces the need for agencies to have a variety of measures across program areas  Our 2017 survey shows that managers have not reported any difference in the availability of performance measures for their programs when compared to the 2013 results  However, the 2017 result (an estimated 87 percent) represents a statistically significant increase when compared to 1997 (76 percent)  When asked about the availability of certain types of performance measures, three of the five types (outcome, output, and efficiency) were statistically significantly higher in 2017 when compared to our initial 1997 survey  However, when comparing 2017 results to those in 2013, two of the five types (output and quality) showed a statistically significant decrease, and the other types did not change  These are illustrated in figure 20  Beyond the survey results, our work has found that some agencies had not developed or used outcome measures, but have taken steps to do so  Agencies have been responsible for measuring program outcomes since GPRA was enacted in 1993  The text box below describes two illustrative examples from our past work  Examples of Agencies That Did Not Develop or Use Outcome Measures Patient access to electronic health information: In March 2017, we found that the Department of Health and Human Services (HHS) had invested over $35 billion since 2009 to enhance patient access to electronic health information, among other things  HHS had not developed outcome measures to gauge the effectiveness of these efforts, which meant the department did not have information to determine whether the efforts were contributing to its overall goals  We recommended that HHS develop relevant outcome measures and HHS concurred  Safety interventions: According to the Federal Motor Carrier Safety Administration (FMCSA), between 2011 and 2015, over 4,000 people died in crashes involving motor carriers each year  GAO, Motor Carriers: Better Information Needed to Assess Effectiveness and Efficiency of Safety Interventions, GAO-17-49 (Washington, D C : Oct  27, 2016)  Further OMB actions could also help agencies make progress in measuring the performance of different program types  In our June 2013 report on initial GPRAMA implementation, we found that agencies experienced common issues in measuring the performance of various types of programs, such as contracts and grants  We recommended that OMB work with the PIC to develop a detailed approach to examine those difficulties  Although they took some actions, OMB and the PIC have not yet developed a comprehensive and detailed approach to address these issues  We concluded that, without such an approach, it would be difficult for the PIC and agencies to fully understand these measurement issues and develop a crosscutting approach to help address them  In August 2017, OMB staff stated that efforts related to the future implementation of the Program Management Improvement Accountability Act (PMIAA) could help address this recommendation  As highlighted in table 1, our work continues to show why it is important for OMB and the PIC to take actions to more fully address our recommendation  Congress has passed legislation to increase the transparency and accessibility of federal performance and financial data  For example, GPRAMA modernized agency reporting requirements to ensure that they make timely, relevant data available to inform decision making by Congress and agency officials as well as improve transparency for the public  Results of our 2017 survey, however, show the need for improvements in the public availability of agency performance information  An estimated 17 percent of managers reported that their agency’s performance information is easily accessible to the public to a great or very great extent, the same percentage as in 2013  Moreover, of the 87 percent of managers that reported there are performance measures for the programs they are involved in, 25 percent reported that they use information obtained from performance measurement when informing the public about how programs are performing to a great or very great extent  This is not statistically different from the 30 percent estimated in 2013  The DATA Act, enacted in 2014, built on previous transparency legislation by expanding what federal agencies are required to report regarding their spending  The act significantly increases the types of data that must be reported, requires government-wide data standards, and regular reviews of data quality to help improve the transparency and accountability of federal spending data  OMB provides websites and guidance to make agency performance and financial information available to the public; however, our prior work has identified a number of areas related to Performance gov and the DATA Act where OMB action is needed to improve the transparency and accessibility of this information  Performance gov: Since 2013, our work has identified a number of issues with Performance gov, the website intended to serve as a central source of information on the federal government’s goals and performance  Over time, we have recommended that OMB take a number of specific actions to improve the website  For example, in June 2013, we found that the website offered an inconsistent user experience and presented accessibility and navigation challenges  To clarify the purpose of the website and enhance its usability, we recommended that OMB take steps to systematically collect customer input  In August 2016, we reported that OMB was not meeting all of the reporting requirements for Performance gov, and did not have a plan to develop and improve the website  We recommended that OMB ensure that information presented on Performance gov consistently complies with reporting requirements and develop a plan for the website that includes, among other things, a customer outreach plan  OMB agreed with these recommendations and, in July 2017, OMB staff informed us that they will be partnering with a vendor to redesign Performance gov to improve the accessibility of information on the website  To inform this redesign, OMB staff said that they will consider our previous recommendations and plan to engage a wide group of stakeholders, including Congress, agency staff, and interested members of the public and outside organizations  OMB staff anticipated releasing updated agency reporting guidance in the fall of 2017 and the redesigned website in February 2018  Under GPRAMA, OMB is required to make available, through Performance gov, quarterly updates on progress toward CAP goals and APGs  As described earlier, in June 2017 OMB announced that reporting to Performance gov has been discontinued through the end of fiscal year 2017 as agencies develop new priority goals  However, Performance gov does not state that it will not be updated, nor does it provide the location of the final progress updates for these goals  OMB’s guidance states that agencies should report the results of progress on their previous APGs in their annual performance reports for fiscal year 2017  Moreover, OMB staff told us that the existing updates on Performance gov for CAP goals, last updated in December 2016, represent the final updates on those goals, although they are not labeled as such on the website  As a result, those interested in progress updates and reported results for the previous priority goals may not know where they will be able to find this information, limiting the transparency and accessibility of those results for decision makers and the public  DATA Act: The DATA Act requires federal agencies to disclose their spending and link this to program activities so that policymakers and the public can more effectively track federal spending  The act has the potential to improve the accuracy and transparency of federal spending information and increase its usefulness for government decision making and oversight  Since the DATA Act became law, OMB and Treasury have taken significant steps to make more complete and accurate federal spending data available  These have included standardizing data element definitions to make it easier to compare different federal agencies’ financial information, and issuing guidance to help agencies submit required data  In May 2017, federal agencies started to report data under the standardized definitions developed under the act  We have made a number of recommendations to address challenges that could affect the consistency and quality of the data  Addressing these recommendations could help ensure that financial data are provided to the public in a transparent and useful manner  For example, in January 2016, we found some standardized data element definitions were imprecise or ambiguous, which could result in inconsistent or potentially misleading reporting  We recommended that OMB provide agencies with additional guidance to address potential issues with the clarity, consistency, and quality of reported data  OMB released guidance in May and November 2016, but in April 2017 we found that additional guidance was needed to help agencies implement certain data definitions to produce data that would be consistent and comparable across agencies  We are in the process of examining the quality of the data that was submitted by agencies in May 2017 and was made available to the public on an early version of the USAspending gov website  We expect to issue the results of this work in fall 2017  Our past work also identified a number of actions agencies need to take to make performance information more transparent  Increasing the accessibility of this information could enhance oversight and accountability of agency performance and results  CAP goals: In May 2016, we found that while selected CAP goal teams were working to develop performance measures to track progress, they were not consistently reporting on their efforts to develop these measures  We recommended that OMB report on Performance gov the actions that CAP goal teams are taking to develop performance measures and quarterly targets to help ensure that measures are aligned with major activities, and ensure that it is possible to track teams’ progress toward establishing measures  While OMB agreed with this recommendation, it did not address it before reporting on the CAP goals was discontinued, as discussed earlier  Customer service standards: As we described earlier, in 2017, an estimated 48 percent of federal managers that indicated they have performance measures for the programs they are involved in also agreed to a great or very great extent that they have customer service performance measures  There has been no statistically significant change relative to our last survey in 2013, or the initial survey in 1997  Relatedly, in October 2014, we reviewed customer service standards at five federal agencies  Customer service standards inform customers about what they have a right to expect when they request services, and the standards should include goals for the quality and timeliness of a service an agency provides to its customers  They should also be easily available to the public so that customers know what to expect, when to expect it, and from whom  In our review of standards at five agencies, however, we found that only Customs and Border Protection had standards that were easily available to the public  We recommended the other four agencies—the United States Forest Service, Federal Student Aid, the National Park Service (NPS), and the Veterans Benefits Administration (VBA)—make their standards more easily accessible to the public  As of July 2017, only VBA had done so  Major management challenges: In June 2016, we found that 14 of the 24 CFO Act agencies did not describe their major management challenges in their performance plans, as required by GPRAMA  Furthermore, 22 of the 24 agencies reviewed did not report complete performance information for each of their major management challenges, including performance goals, milestones, indicators, and planned actions that they have developed to address such challenges  As a result, it was not always transparent what these agencies considered to be their major management challenges or how they planned to resolve these challenges  We recommended that the 22 agencies describe their major management challenges in their agency performance plans and include goals, measures, milestones, and information on planned actions and responsible officials  As of August 2017, 8 agencies—the U S  Agency for International Development, Small Business Administration, Nuclear Regulatory Commission, OPM, National Aeronautics and Space Administration (NASA), and the Departments of Education, State, and Veterans Affairs—had fully implemented our recommendations; the other 14 agencies had not  Quality of performance information: In September 2015, we found that six selected agencies reported limited information on the actions they are taking to ensure the quality of their performance information for selected APGs, as required by GPRAMA  We recommended that all six of the agencies work with OMB to fully report this information  In response, the Department of Homeland Security and NASA described how they ensure reliable performance information is reported to external audiences  As of June 2017, the Departments of Agriculture, Defense, the Interior, and Labor had not yet taken actions to address this recommendation by providing more specific explanations of how they ensure reliable performance information is reported for their APGs  Unnecessary reports: GPRAMA requires that OMB guide an annual review of agencies’ plans and reports for Congress and include in the President’s budget a list of those plans and reports determined to be outdated or duplicative  However, in July 2017, we found that OMB did not implement the report review process on an annual basis, as required  We also found that OMB published the list of agency plans and reports on Performance gov, rather than in the President’s annual budget, where they may be more visible and useful to congressional decision makers and others  Therefore, we recommended that OMB instruct agencies to identify outdated or duplicative reports on an annual basis and submit or reference the list of identified plans and reports with the President’s annual budget  OMB agreed with these recommendations  In July 2017, OMB stated it would include a list of report modification proposals in the President’s fiscal year 2019 budget as required by GPRAMA  For all of the unimplemented recommendations described above, we will continue to monitor agencies’ actions  In addition to providing access to performance and financial information, federal agencies can directly engage and collaborate with citizens, nonprofits, academic institutions, and other levels of government using open innovation strategies  Open innovation involves using various tools and approaches to harness the ideas, expertise, and resources of those outside an organization to address an issue or achieve specific goals  In October 2016, we found that in recent years agencies had frequently used five open innovation strategies—singularly or in combination—to collaborate with citizens and encourage their participation in agency initiatives  (See figure 21 ) Our October 2016 report found that agencies can use these strategies for a variety of purposes  To develop new ideas, solutions to specific problems, or new products: For example, from April 2015 to November 2016, the Department of Energy held a prize competition to create more efficient devices that would double the energy captured from ocean waves  According to the competition’s website, the winning team achieved a five-fold improvement  To enhance collaboration and agency capacity by leveraging external resources, knowledge, and expertise: For example, every 2 years since 2009, the Federal Highway Administration has regularly engaged stakeholders to identify and implement innovative ideas that have measurably improved the execution of highway construction projects  To collect the perspectives and preferences of a broad group of citizens and external stakeholders: For example, the Food and Drug Administration used in-person and online dialogue to engage outside stakeholders in the development of an online platform designed to make key datasets easily accessible to the public  Subsequently, in June 2017, we found that OMB, the Office of Science and Technology Policy (OSTP), and the General Services Administration (GSA) developed resources to support the use of open innovation strategies by federal agencies  These resources included guidance, staff to assist agencies in implementing initiatives, and websites to improve access to relevant information  For example, GSA developed a step-by-step implementation guide, program management team, and website to help agency staff carry out prize competitions and challenges  Agencies have also developed their own resources, including guidance, staff positions, and websites, to reach specific audiences and to provide tailored support for open innovation strategies they use frequently  For example, NASA’s Solve website provides a central location for the public to find the agency’s challenges and citizen science projects, as well as links to relevant resources  We also evaluated key government-wide guidance for the five strategies listed above to determine the extent to which the guidance reflects leading practices for effectively implementing open innovation initiatives  We identified these practices in our October 2016 report  We found that the guidance for each strategy reflected these practices to differing extents, as shown in figure 22  We made 22 recommendations to GSA, OMB, and OSTP to enhance the guidance  GSA and OMB generally agreed with these recommendations and OSTP neither agreed nor disagreed  We will monitor their progress toward implementing these recommendations  GPRAMA provides important tools that can help decision makers better achieve results and address the federal government’s significant and long-standing governance challenges  Although OMB and agencies have made progress in improving implementation of the act over the years, our work has highlighted numerous opportunities for further improvements  In 2017, OMB removed the priority designation of CAP goals and APGs  For those goals, this action stopped related data-driven reviews and quarterly updates of progress on Performance gov until new priority goals are published next year  What OMB considers to be the final results of CAP goals for fiscal years 2014 to 2017 already are on Performance gov (although not labeled as such)  In addition, agencies may report on their former APGs in their annual fiscal year 2017 performance reports  However, Performance gov does not state that it will not be updated or provide the location of the final progress updates for these goals, limiting transparency and its value to the public  OMB has stated its plans to restart implementation of those provisions in February 2018, with the start of a new goal cycle  We believe it is critical for OMB to do so, given the important role those tools play in addressing key governance challenges and the results we have seen in better managing crosscutting areas and driving performance improvements across the government  In addition, OMB has postponed implementation of the federal program inventory  To date, the inventory has only been developed once, in 2013, despite requirements for regular updates to reflect current budget and performance information  OMB has given a variety of reasons for the delays over the past 4 years—most recently, to determine the right strategy to merge implementation of the DATA Act and PMIAA with GPRAMA’s program inventory requirements  Although OMB staff told us that they expect to issue guidance by the end of 2018 to resume implementation of the program inventory requirements, they have not provided more specific time frames and milestones related to the program inventory requirements  Doing so would help agencies prepare for resumed implementation  Moreover, publicly disclosing planned implementation time frames and associated milestones would help ensure that interested stakeholders, such as federal decision makers and the public, are prepared to engage with agencies as they develop and update their program inventories, which in turn could help ensure the inventories meet stakeholders’ needs  A well-developed inventory would provide key program, budget, and performance information in one place to help federal decision makers better understand the federal investment and results in given policy areas, and better identify and manage fragmentation, overlap, and duplication  Information architecture offers one approach to developing an inventory  As OMB determines a strategy for implementing the program inventory and develops its guidance, considering such a systematic approach to planning, organizing, and developing the inventory that centers on maximizing the use and usefulness of information could help it ensure the inventory meets GPRAMA requirements as well as the needs of decision makers and the public  Moreover, such an approach could also help OMB implement our past recommendations related to the program inventory, which are intended to ensure the inventory provides more complete information and is useful to various stakeholders  Our survey of federal managers continues to generally show no improvement in their reported use of performance information in decision making, nor in the employment of practices that can enhance such use  One area where our survey data and past work show promise is through the use of regular, leadership-driven reviews of performance data at agencies, especially when conducted in line with related leading practices  However, GPRAMA only requires these data-driven reviews for APGs, which represent a small subset of goals, both within individual agencies as well as across the government  This is probably why most federal managers were not familiar with the reviews  Identifying and sharing practices for expanding the use of those reviews—such as for additional agency-wide performance goals and at lower levels within agencies—could significantly enhance the use of performance information and drive to better and greater results  We are making the following four recommendations to OMB: The Director of OMB should update Performance gov to explain that quarterly reporting on the fiscal year 2014 through 2017 CAP goals and fiscal year 2016 and 2017 APGs was suspended, and provide the location of final progress updates for these goals  (Recommendation 1) The Director of OMB should revise and publicly issue OMB guidance— through an update to its Circular No  A-11, a memorandum, or other means—to provide time frames and associated milestones for implementing the federal program inventory  (Recommendation 2) The Director of OMB should consider—as OMB determines its strategy for resumed implementation of the federal program inventory—using a systematic approach, such as the information architecture framework, to help ensure that GPRAMA requirements and our past recommendations for the inventory are addressed  (Recommendation 3) The Director of OMB should work with the Performance Improvement Council to identify and share among agencies practices for expanding the use of data-driven performance reviews beyond APGs, such as for other performance goals and at lower levels within agencies, that have led to performance improvements  (Recommendation 4) We provided a draft of this report to the Director of the Office of Management and Budget for review and comment  In comments provided orally and via email, OMB staff agreed with the recommendations in this report  OMB staff also asked us to (1) consider revising the draft title of the report, to better reflect progress in GPRAMA implementation, and (2) clarify our recommendations on issuing guidance for implementing the federal program inventory and expanding the use of data-driven performance reviews, by describing possible actions that could be taken to implement them  We agreed and made revisions accordingly  We are sending copies of this report to interested congressional committees, the Director of the Office of Management and Budget, and other interested parties  This report will also be available at no charge on the GAO website at http://www gao gov  If you or your staff have any questions about this report, please contact me at (202) 512-6806 or mihmj@gao gov  Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of our report  Key contributors to this report are listed in appendix III  The GPRA Modernization Act (GPRAMA) includes a statutory provision for us to periodically evaluate implementation of the act  Since 2012, we have issued over 30 products in response to this provision; this is the third summary report  This report assesses how implementation of GPRAMA has affected the federal government’s progress in resolving key governance challenges in (1) addressing crosscutting issues, (2) ensuring performance information is useful and used in decision making, (3) aligning daily operations with results, and (4) building a more transparent and open government  We reviewed relevant statutory requirements, related Office of Management and Budget (OMB) guidance, and our recent work related to GPRAMA implementation and the four key governance challenges included in our reporting objectives  Specifically, since our last summary report in September 2015, we examined various aspects of GPRAMA implementation in 12 products that covered 35 agencies, including the 24 agencies covered under the Chief Financial Officers (CFO) Act of 1990, as amended (identified in table 2)  We interviewed OMB and Performance Improvement Council staff to obtain (1) their perspectives on GPRAMA implementation and progress on the four governance challenges, and (2) updates on the status of our past recommendations  We also received updates from other agencies on the status of our past recommendations to them related to GPRAMA implementation  To supplement this review, we administered our periodic survey of federal managers on organizational performance and management issues from November 2016 through March 2017  This survey is comparable to five previous surveys we conducted in 1997, 2000, 2003, 2007, and 2013  We selected a stratified random sample of 4,395 people from a population of approximately 153,779 mid-level and upper-level civilian managers and supervisors working in the 24 executive branch agencies covered by the CFO Act, as shown in table 2  We obtained the sample from the Office of Personnel Management’s (OPM) Enterprise Human Resources Integration (EHRI) database as of September 30, 2015, which was the most recent fiscal year data available at the time  We used file designators indicating performance of managerial and supervisory functions  In reporting survey data, we use the term “government-wide” and the phrases “across the government” or “overall” to refer to the 24 CFO Act executive branch agencies  We use the terms “federal managers” and “managers” to collectively refer to both managers and supervisors  We designed the questionnaire to obtain the observations and perceptions of respondents on various aspects of results-oriented management topics  These topics include the presence and use of performance measures, any hindrances to measuring performance and using performance information, agency climate, and program evaluation use  To assess implementation of GPRAMA, the questionnaire included questions to collect respondents’ views on various provisions of GPRAMA, such as cross-agency priority goals, agency priority goals, and related quarterly performance reviews  Similar to the five previous surveys, the sample was stratified by agency and by whether the manager or supervisor was a member of the Senior Executive Service (SES)  The management levels covered general schedule (GS) or equivalent schedules at levels comparable to GS-13 through GS-15 and career SES or equivalent  Stratifying the sample in this way ensured that the population from which we sampled covered at least 90 percent of all mid- to upper-level managers and supervisors at the departments and agencies we surveyed  Most of the items on the questionnaire were closed-ended, meaning that depending on the particular item, respondents could choose one or more response categories or rate the strength of their perception on a 5-point extent scale ranging from “no extent” to “very great extent ” On most items, respondents also had an option of choosing the response category “no basis to judge/not applicable ” A few items had other options, such as “yes,” “no,” or “do not know,” or a 3-point familiarity scale (“not familiar,” “somewhat familiar,” and “very familiar”)  We asked many of the items on the questionnaire in our earlier surveys, though we introduced a number of new items in 2013, including the sections about GPRAMA and program evaluations  For 2017, we added a new question on use of performance information (question 12) and a new question on program evaluation (question 24)  Before administering the survey, questions were reviewed by our staff, including subject matter experts, a survey specialist, and a research methodologist  We also conducted pretests of the new questions with federal managers in several of the 24 CFO Act agencies  We changed the wording of subquestions or added clarifying examples based on pretester feedback  To administer the survey, we e-mailed managers in the sample to notify them of the survey’s availability on our website and we included instructions on how to access and complete the survey  To follow up with managers in the sample who did not respond to the initial notice, we emailed or called multiple times to encourage survey participation or provide technical assistance, as appropriate  Similar to our last survey, we worked with OPM to obtain the names of the managers and supervisors in our sample, except for those within selected subcomponents whose names were withheld from the EHRI database  Since Foreign Service officials from the Department of State (State) are not in the EHRI database, we drew a sample for that group with the assistance from State  We worked with officials at the Department of Homeland Security (DHS) and the Department of the Treasury (Treasury) to gain access to these individuals to maintain continuity of the population of managers surveyed from previous years  The Department of Justice (DOJ) was concerned about providing identifying information (e g , names, e-mail addresses, and phone numbers) of federal agents to us, so we administered the current survey to DOJ managers in our sample through DOJ officials  To identify the sample of managers whose names were withheld from the EHRI database, we provided DOJ with the last four digits of Social Security numbers, the subcomponent, duty location, and pay grade information  To ensure that DOJ managers received the same survey administration process as the rest of the managers in our sample to the extent possible, we provided DOJ with text for the survey activation and reminder e-mails similar to ones we emailed to managers at other agencies  DOJ administered the survey to these managers and emailed them one reminder to complete the survey  To help determine the reliability and accuracy of the EHRI data elements used to draw our sample of federal managers, we checked the data for reasonableness and the presence of any obvious or potential errors in accuracy and completeness and reviewed past analyses of the reliability of this database  For example, we identified cases where the managers’ names were withheld and contacted OPM to discuss this issue  We also checked the names of the managers in our selected sample provided by OPM with the applicable agency contacts to verify these managers were still employed with the agency  We noted discrepancies when they occurred and excluded them from our population of interest, as applicable  On the basis of these procedures, we believe the data we used from the EHRI database are sufficiently reliable for the purpose of the survey  Of the 4,395 managers selected for the 2017 survey, we found that 388 of the sampled managers had retired, separated, or otherwise left the agency or had some other reason that excluded them from the population of interest  These exclusions included managers that the agency could not locate, and therefore we were unable to request that they participate in the survey  We received usable questionnaires from 2,726 sample respondents, for a weighted response rate of about 67 percent of the remaining eligible sample  The weighted response rate across 23 of the 24 agencies ranged from 57 percent to 82 percent, while DOJ had a weighted response rate of 36 percent  See the supplemental material for each agency’s response rate  We conducted a nonresponse bias analysis using information from the survey and sampling frame as available  The analysis confirmed discrepancies in the tendency to respond to the survey related to agency and SES status  The analysis also revealed some differences in response propensity by age and GS level; however, the direction and magnitude of the differences on these factors were not consistent across agencies or strata  Our data may be subject to bias from unmeasured sources for which we cannot control  Results, and in particular estimates from agencies with low response rates such as DOJ, should be interpreted with caution because these estimates are associated with a higher level of uncertainty  The overall survey results are generalizable to the government-wide population of managers as described above  The responses of each eligible sample member who provided a usable questionnaire were weighted in the analyses to statistically account for all members of the population  All results are subject to some uncertainty or sampling error as well as nonsampling error, including the potential for nonresponse bias as noted above  Because we followed a probability procedure based on random selections, our sample is only one of a large number of samples that we might have drawn  The magnitude of sampling error will vary across the particular surveys, groups, or items being compared because we (1) used complex survey designs that differed in the underlying sample sizes, usable sample respondents, and associated variances of estimates, and (2) conducted different types of statistical analyses  For example, the 2000 and 2007 surveys were designed to produce agency-level estimates and had effective sample sizes of 2,510 and 2,943, respectively  However, the 1997 and 2003 surveys were designed to obtain government-wide estimates only, and their sample sizes were 905 and 503, respectively  Consequently, in some instances, a difference of a certain magnitude may be statistically significant  In other instances, depending on the nature of the comparison being made, a difference of equal or even greater magnitude may not achieve statistical significance  Because each sample could have provided different estimates, we express our confidence in the precision of our particular sample’s results as a 95 percent confidence interval  This is the interval that would contain the actual population value for 95 percent of the samples we could have drawn  The percentage estimates presented in this report based on our sample for the 2017 survey have 95 percent confidence intervals within plus or minus 5 5 percentage points of the estimate itself, unless otherwise noted  We also note in this report when we are 95 percent confident that changes from 1997 or 2013 relative to 2017 are statistically significant  Online supplemental material shows the questions asked on the survey along with the percentage estimates and associated 95 percent confidence intervals for each item for each agency and government-wide  In a few instances, we report estimates with larger margins of error because we deemed them reliable representations of given findings due to the statistical significance of larger differences between comparison groups  In all cases, we report the applicable margins of error  In addition to sampling errors, the practical difficulties of conducting any survey may also introduce other types of errors, commonly referred to as nonsampling errors  For example, difficulties in how a particular question is interpreted, in the sources of information available to respondents, or in how the data were entered into a database or analyzed can introduce unwanted variability into the survey results  With this survey, we took a number of steps to minimize these nonsampling errors  For example, our staff with subject matter expertise designed the questionnaire in collaboration with our survey specialists  As noted earlier, the new questions added to the survey were pretested to ensure they were relevant and clearly stated  When the data were analyzed, a second independent analyst on our staff verified the analysis programs to ensure the accuracy of the code and the appropriateness of the methods used for the computer-generated analysis  Since this was a web-based survey, respondents entered their answers directly into the electronic questionnaire, thereby eliminating the need to have the data keyed into a database, thus avoiding a source of data entry error  To supplement descriptive analysis of the survey questions, we generated an index to gauge government-wide use of performance information  The index, which was identical to one we reported in 2014, averaged manager’s responses to 11 questions deemed to relate to the concept of performance information use  The index runs from 1 (corresponding to an average value of “to no extent”) to 5 (corresponding to an average value of “to a very great extent”)  We used Cronbach’s alpha to assess the internal consistency of the scale  Our government- wide index score weights each agency’s contribution equally, and provides a relative measure of the use of performance information over time rather than an absolute indicator of the government-wide level of use of performance information  We conducted this performance audit from January 2016 to September 2017 in accordance with generally accepted government auditing standards  Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives  We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives  The Office of Management and Budget (OMB) and agencies have taken some actions to address our recommendations related to implementation of the GPRA Modernization Act of 2010 (GPRAMA); however, the majority of recommendations remain open  Since GPRAMA was enacted in January 2011, we have made 100 recommendations in 18 reports to OMB and agencies aimed at improving the act’s implementation (table 3)  Of those 100, OMB and the agencies have implemented 42 recommendations  Fifty-eight recommendations require additional action  Nearly half (47) of our recommendations are directed to OMB  For the 23 recommendations that OMB has implemented, many represent revisions to guidance to better reflect GPRAMA’s requirements or to enhance implementation  Many of the 24 recommendations to OMB that are not implemented deal with long-standing or complex challenges, on which OMB has taken limited action to date  Of those, we have designated 3 as priorities for OMB to address  Agencies have also taken some action on our recommendations, implementing 19 of the 53 recommendations we have made  The following tables present each of the 100 recommendations along with a summary of actions taken to address it  Tables 4 and 5 provide information about our recommendations to OMB that are implemented and not implemented, respectively  Tables 6 and 7 provide information about our recommendations to other agencies that are implemented and not implemented, respectively  In addition to the above contact, Benjamin T  Licht (Assistant Director) and Shannon Finnegan (Assistant Director) supervised this review and the development of the resulting report  Leah Q  Nash (Assistant Director), Elizabeth Fan (Analyst-in-Charge), and Adam Miles (Analyst-in- Charge) supervised the development and administration of the Federal Managers Survey and the resulting supplemental material  Peter Beck, Valerie Caracelli, Karin Fangman, Steven Flint, Robert Gebhart, Ricky Harrison Jr , John Hussey, Jill Lacey, Won Lee, Krista Loose, Meredith Moles, Anna Maria Ortiz, Steven Putansu, Alan Rozzi, Cindy Saunders, Stephanie Shipman, Shane Spencer, Andrew J  Stephens, and Brian Wanlass also made key contributions  Ann Czapiewski and Donna Miller developed the graphics for this report  John Ahern, Divya Bali, Jeff DeMarco, Alexandra Edwards, Ellen Grady, Jyoti Gupta, Erinn L  Sauer, and Katherine Wulff verified the information presented in this report  Managing for Results: Implementation of GPRA Modernization Act Has Yielded Mixed Progress in Addressing Pressing Governance Challenges  GAO-15-819  Washington, D C : September 30, 2015  Managing For Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges  GAO-13-518  Washington, D C : June 26, 2013  Supplemental Material for GAO-17-775: 2017 Survey of Federal Managers on Organizational Performance and Management Issues  GAO-17-776SP  Washington, D C : September 29, 2017  Program Evaluation: Annual Agency-wide Plans Could Enhance Leadership Support for Program Evaluations  GAO-17-743  Washington, D C : September 29, 2017  Managing for Results: Agencies’ Trends in the Use of Performance Information to Make Decisions  GAO-14-747  Washington, D C : September 26, 2014  Managing for Results: Executive Branch Should More Fully Implement the GPRA Modernization Act to Address Pressing Governance Challenges  GAO-13-518  Washington, D C : June 26, 2013  Managing for Results: 2013 Federal Managers Survey on Organizational Performance and Management Issues, an E-supplement to GAO-13-518  GAO-13-519SP  Washington, D C : June 26, 2013  Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation in Program Management and Policy Making  GAO-13-570  Washington, D C : June 26, 2013  Government Performance: Lessons Learned for the Next Administration on Using Performance Information to Improve Results  GAO-08-1026T  Washington, D C : July 24, 2008  Government Performance: 2007 Federal Managers Survey on Performance and Management Issues, an E-supplement to GAO-08-1026T  GAO-08-1036SP  Washington, D C : July 24, 2008  Results-Oriented Government: GPRA Has Established a Solid Foundation for Achieving Greater Results  GAO-04-38  Washington, D C : March 10, 2004  Managing for Results: Federal Managers’ Views on Key Management Issues Vary Widely Across Agencies  GAO-01-592  Washington, D C : May 25, 2001  Managing for Results: Federal Managers’ Views Show Need for Ensuring Top Leadership Skills  GAO-01-127  Washington, D C : October 20, 2000  The Government Performance and Results Act: 1997 Governmentwide Implementation Will Be Uneven  GAO/GGD-97-109  Washington, D C : June 2, 1997  Federal Programs: Information Architecture Offers a Potential Approach for Inventory Development  GAO-17-739  Washington, D C : September 28, 2017  Managing for Results: Selected Agencies’ Experiences in Implementing Strategic Reviews  GAO-17-740R  Washington, D C : September 7, 2017  Federal Reports: OMB and Agencies Should More Fully Implement the Process to Streamline Reporting Requirements  GAO-17-616  Washington, D C : July 14, 2017  Open Innovation: Executive Branch Developed Resources to Support Implementation, but Guidance Could Better Reflect Leading Practices  GAO-17-507  Washington, D C : June 8, 2017  Performance Partnerships: Agencies Need to Better Identify Resource Contributions to Sustain Disconnected Youth Pilot Programs and Data to Assess Pilot Results  GAO-17-208  Washington, D C : April 18, 2017  Open Innovation: Practices to Engage Citizens and Effectively Implement Federal Initiatives  GAO-17-14  Washington, D C : October 13, 2016  Tiered Evidence Grants: Opportunities Exist to Share Lessons from Early Implementation and Inform Future Federal Efforts  GAO-16-818  Washington, D C : September 21, 2016  Performance gov: Long-Term Strategy Needed to Improve Website Usability  GAO-16-693  Washington, D C : August 30, 2016  Tax Expenditures: Opportunities Exist to Use Budgeting and Agency Performance Processes to Increase Oversight  GAO-16-622  Washington, D C : July 7, 2016  Managing for Results: Agencies Need to Fully Identify and Report Major Management Challenges and Actions to Resolve them in their Agency Performance Plans  GAO-16-510  Washington, D C : June 15, 2016  Managing for Results: OMB Improved Implementation of Cross-Agency Priority Goals, But Could Be More Transparent About Measuring Progress  GAO-16-509  Washington, D C : May 20, 2016  Managing for Results: Greater Transparency Needed in Public Reporting on the Quality of Performance Information for Selected Agencies’ Priority Goals  GAO-15-788  Washington, D C : September 10, 2015  Pay for Success: Collaboration among Federal Agencies Would Be Helpful as Governments Explore New Financing Mechanisms  GAO-15-646  Washington, D C : September 9, 2015  Managing for Results: Practices for Effective Agency Strategic Reviews  GAO-15-602  Washington, D C : July 29, 2015  Managing for Results: Agencies Report Positive Effects of Data-Driven Reviews on Performance but Some Should Strengthen Practices  GAO-15-579  Washington, D C : July 7, 2015  Program Evaluation: Some Agencies Reported that Networking, Hiring, and Involving Program Staff Help Build Capacity  GAO-15-25  Washington, D C : November 13, 2014  Government Efficiency and Effectiveness: Inconsistent Definitions and Information Limit the Usefulness of Federal Program Inventories  GAO-15-83  Washington, D C : October 31, 2014  Managing for Results: Selected Agencies Need to Take Additional Efforts to Improve Customer Service  GAO-15-84  Washington, D C : October 24, 2014  Managing for Results: Enhanced Goal Leader Accountability and Collaboration Could Further Improve Agency Performance  GAO-14-639  Washington, D C : July 22, 2014  Managing for Results: OMB Should Strengthen Reviews of Cross-Agency Goals  GAO-14-526  Washington, D C : June 10, 2014  Managing for Results: Implementation Approaches Used to Enhance Collaboration in Interagency Groups  GAO-14-220  Washington, D C : February 14, 2014  Managing for Results: Leading Practices Should Guide the Continued Development of Performance gov  GAO-13-517  Washington, D C : June 6, 2013  Managing for Results: Agencies Should More Fully Develop Priority Goals under the GPRA Modernization Act  GAO-13-174  Washington, D C : April 19, 2013  Managing for Results: Agencies Have Elevated Performance Management Roles, but Additional Training Is Needed  GAO-13-356  Washington, D C : April 16, 2013  Managing for Results: Data-Driven Performance Reviews Show Promise But Agencies Should Explore How to Involve Other Relevant Agencies  GAO-13-228  Washington, D C : February 27, 2013  Managing for Results: A Guide for Using the GPRA Modernization Act to Help Inform Congressional Decision Making  GAO-12-621SP  Washington, D C : June 15, 2012  Managing for Results: GAO’s Work Related to the Interim Crosscutting Priority Goals under the GPRA Modernization Act  GAO-12-620R  Washington, D C : May 31, 2012  Managing for Results: Opportunities for Congress to Address Government Performance Issues  GAO-12-215R  Washington, D C : December 9, 2011 
Report 2: , support for general aviation airports)  But there are significant disagreements about the appropriate degree of federal participation in airport development and finance and about the specific types of expenditure that should be given priority within AIP  Airline and airport operators tend to view the fully authorized funding of the program as a good thing  An alternative view, however, is that too much has been spent on AIP, particularly at smaller airports that do not play a significant role in commercial aviation  The assessment of airport capital needs is fundamental to determining the appropriate federal support needed to foster a safe and efficient national airport system  The federal government's interest goes beyond capacity issues to include implementation of federal safety and noise policies  Both FAA and the Airports Council International-North America (ACI-NA) have issued projections of airports' long-term financial needs  In its most recent NPIAS report, FAA estimated that the national system's capital needs for FY2019-FY2023 will total $35 1 billion (an annual average of approximately $7 billion)  The ACI-NA infrastructure needs survey resulted in an estimate of $128 1 billion over the same years (an annual average of approximately $25 6 billion)  The main reason for the widely differing estimates was disparate views on what kinds of airport projects to include  The NPIAS report was based on information taken from airport master plans and state system plans, but FAA planners screened out planned projects not justified by aviation activity forecasts or not eligible for AIP grants  Only designated NPIAS airports were included in the FAA study  Implicit in this methodology is that the planning has been carried through to the point where financing is identified  Not all projects used to develop the NPIAS estimates are actually completed, or in some cases even begun, within the range of years covered in the NPIAS estimates  ACI-NA argues that the NPIAS underestimates AIP eligible needs because not all such needs will be in the current airport plans  The ACI-NA study reflects the broader business view of major airport operators and casts a substantially wider net  It includes projects funded by PFCs, bonds, or state or local funding; airport-funded air traffic control facilities; airport- or TSA-funded security projects; "necessary" AIP-ineligible projects such as parking facilities, hangars, revenue portions of terminals, and off-airport roads/transit facilities; and AIP-eligible projects not reported to FAA in the belief that there would be a low probability of receiving additional AIP funding  Its 2019-2023 infrastructure needs survey, for example, included major airport terminal projects that are ineligible for AIP grants  The ACI-NA study also includes projects without identified funding sources  The ACI-NA estimate is higher than the FAA estimate because of the wider net it casts and because it is adjusted for projected inflation  The estimates are important because the primary AIP reauthorization issue is the program's appropriate level of funding  Because the ACI-NA airport needs projection includes much that is not eligible for AIP grants, its accuracy may not be as critical in evaluating appropriate AIP funding levels as that of the NPIAS projections  On the other hand, the broader ACI-NA estimate may be more significant for policy choices related to bond issuance and PFCs, since these sources fund a broader range of projects than AIP  In 2004, then-FAA Administrator Marion C  Blakey stated that the agency's goal was to increase total capacity at the top 35 U S  airports by 30% over a 10-year period  FAA's Operational Evolution Plan (OEP) is intended to increase the capacity and efficiency of the National Airspace System (NAS) over a 10-year period to keep up with the expected growth in demand for air travel and air cargo  In support of that goal, FAA released a study focused on the 35 busiest airports, Capacity Needs in the National Airspace System: An Analysis of Airport and Metropolitan Area Demand and Operational Capacity in the Future (also referred to as FACT1)  The study projected 18 airports would need additional capacity by 2020  In 2007, FACT1 was updated by a second study, FACT2  FACT2 expanded the study to include 21 non-OEP airports that were identified as having the potential to be capacity constrained or were in capacity-constrained metropolitan areas  The study examined airports that would need capacity increases and also projected which airports would need capacity increases in 2015 and 2025  It identified four airports plus the New York metropolitan area that needed additional capacity in 2007  It further identified 14 hub airports as likely to be capacity-constrained in 2025  FACT2 found that, in comparison to FACT1, many non-OEP airports "    have higher capacities than originally presumed and thus less need for additional capacity " A further update, FACT3, was released in January 2015  FACT3 forecasted that the 2007-2009 recession, volatile fuel costs, airline consolidation, and replacement of many 50-seat regional jets with larger aircraft would result in 32% fewer operations and about 23% fewer enplanements in 2025 at the 30 core airports than forecast in FACT2  It projected that airport delays would remain concentrated at a few major hub airports, notably the three New York City-area airports, Philadelphia International Airport, and Hartsfield-Jackson Atlanta International Airport  This study may have implications for the reauthorization of AIP  The large runway projects that are the focus of the OEP can require long lead times—10 or more years from concept to initial construction is not unusual  At large and medium hub airports, runway projects are usually paid for, in part, by AIP funds  Therefore, some projects needed by 2025 may require AIP funding in earlier years  Because large and medium airports that levy PFCs must forgo either 50% or 75% of their AIP formula entitlement funds, most federal funding for their runway projects would probably need to take the form of AIP discretionary funds  The pool of discretionary funds is primarily the remainder of annual funding after the entitlement formula requirements are satisfied  Of the forgone PFC funds, 87 5% are reserved for the small airport fund and are also not available for OEP airports  If the AIP budget is constrained in the future, either under a reauthorization bill or during the annual appropriations process, and the entitlement formulas remain as they are, the discretionary portion of the AIP budget may be squeezed, limiting large airports' ability to draw on AIP funds for major capacity expansion projects  Many of the attributes of AIP's programmatic structure are similar to those of the 1982 act that created the program  Over the years these attributes have been modified based on perceived needs and on the practical politics of passing the periodic FAA reauthorization bills that contain the AIP provisions  These considerations make a major overhaul of the AIP structure unlikely, but may leave room for programmatic adjustments in the distribution of apportionments  One such adjustment might shift AIP funds to enhancing capacity at large and medium hub airports  There are several ways Congress might accomplish this  One would be to eliminate the requirement that large and medium hub airports that impose the maximum PFCs forgo 75% of their entitlements  This change would give larger airports a greater share of entitlement funding, but at the cost of reducing AIP grants to small airports  Alternatively, changes in the statutory set-asides of discretionary funds could give FAA more flexibility to use that money for capacity enhancement, but might reduce funding for noise mitigation and other purposes  Changes in the last several FAA authorization acts increased entitlements and broadened the range of landside projects eligible for AIP grants  These changes generally benefitted airports smaller than medium hub size  In particular, the increased amount of apportioned funds has limited the availability of funds for discretionary grants at major airports  Further changes giving airports increased flexibility in the use of their entitlements might benefit smaller airports not served by commercial aviation, in line with the national goal of having an "extensive" national airport system, but this use of funds might conflict with the goal of reducing congestion at major commercial airports  The current apportionment system relies on a $3 2 billion funding level trigger mechanism to lift most of the apportionments to twice their formula level  This has been in place for two reauthorization cycles  Should that trigger be breached, entitlements for all airports would be reduced drastically  The entitlement formulas may not be sustainable, without depleting discretionary funds, in the absence of additional funding for AIP  One way to reduce the amount of trust fund revenue needed for AIP would be to allow large and medium hub airports to opt out of AIP and rely exclusively on PFCs to finance capital projects  This would require raising or eliminating the federal cap on PFCs  These "defederalized" airports could then be released from some or all of the AIP grant assurances under which they now operate, such as land use requirements and airport revenue use restrictions  If airports exit the program, AIP spending could be reduced or could be redirected to other NPIAS airports  Airport privatization denotes a change in ownership from a public entity (such as a local government or an airport authority established by a state government) to a private one  In a number of countries, such as Great Britain, government-owned airports have been privatized by sale to private owners  In the United States, some airports have allowed private ownership of certain on-airport facilities or management functions, but the ownership of all major airports remains in the hands of government entities  The Airport Privatization Pilot Program (49 U S C  §47134; Section 149 of the Federal Aviation Reauthorization Act of 1996, P L  104-264 , as amended) authorizes FAA to exempt up to 10 airports from certain federal restrictions on the use of airport revenue off-airport  Participating airports may be also exempted from certain requirements on the repayment of federal grants  Privatized airports may still participate in the AIP, but at a lower federal share (70%)  The pilot program was renamed the Airport Investment Partnership Program (AIPP) in the 2018 FAA reauthorization act and expanded to admit more than 10 airports  The AIPP provides that at primary airports, the airport sponsor may only recover from the sale or lease an amount approved by at least 65% of the scheduled air carriers serving the airport, as well as by both scheduled and unscheduled air carriers that together account for 65% of the total landed weight at the airport for the year  The requirement that air carriers approve the use of airport revenue for nonairport purposes, such as profit distribution, may have served to limit interest in the program  To date, only two airports have completed the privatization process established under the provisions of the AIPP  One of those, Stewart International Airport in New York State, subsequently reverted to public ownership when it was purchased by the Port Authority of New York and New Jersey  Luis Muñoz Marín International Airport in San Juan, PR, is now the only commercial service airport operating under private management after privatization under the APPP  As of 2018, there are three applicants under active FAA consideration: Hendry County Airglades Airport in Clewiston, FL; Westchester Airport in White Plains, NY; and St  Louis Lambert International Airport in St  Louis, MO  There is no certainty that any AIP cost savings from privatization would be retained for use by the other AIP-eligible airports  AIP spending is determined by the authorization and appropriations process, and Congress could choose to use any savings to reduce the program size, to marginally assist in deficit reduction, to reduce general fund portions of FAA operations funding, or to make money available for spending elsewhere  Debate over FAA reauthorization generally brings forth proposals to alter the AIP grant assurances, such as ensuring that workers on airport construction projects receive prevailing wages set under the Davis-Bacon Act and pledging to use airport revenue solely for spending on airport operations and capital costs  If AIP spending remains constrained, critics are likely to argue that the grant assurances raise the cost of projects to increase airport capacity and complicate the closure and reuse of underutilized airports or airports that are locally unpopular due to noise or safety concerns  Historically, a basic funding issue is whether to change the existing discretionary fund set-aside for noise mitigation and abatement  The noise set-aside, however, has been increased in previous reauthorization acts and is now 35% of discretionary funding  Demand to use AIP funds for noise mitigation could increase if Congress grants FAA the flexibility to fund noise mitigation projects that are outside the DNL 65 decibel (dB) noise impact area, but this could divert resources from capacity and safety projects  A related issue is whether to make the planning for noise-mitigating air traffic control procedures at individual airports eligible for AIP funding  The central issue related to PFCs is whether to raise the $4 50 per enplaned passenger ceiling or to eliminate the ceiling altogether  In general, airports argue for increasing or eliminating the ceiling, whereas most air carriers and some passenger advocates oppose higher limits on the PFCs  A 2015 GAO study analyzed the effects by raising the PFC cap under three scenarios: setting the cap at $6 47, $8 00, or $8 50  The study found raising PFC would significantly increase airport funding, but could also marginally slow passenger growth and therefore the growth in revenues to the trust fund  PFC supporters feel that the PFC is more reliable than AIP funding, which is subject to the authorization and appropriations process  They also argue that PFCs are procompetitive, helping airports build gates and facilities that both encourage new entrant carriers and allow incumbent carriers to expand  Advocates of an increase in the cap also argue that over time, the value of the PFC has been eroded by inflation and an adjustment is therefore necessary  The permissible uses of revenues are an ongoing point of contention  Airport operators, in particular, would like more freedom to use PFC funds for off-airport projects, such as transportation access projects, and want the process of obtaining FAA approval to be streamlined  Carriers, on the other hand, often complain that airports use PFC funds to finance proposals of dubious value, especially outside airport boundaries, instead of high-priority projects that offer meaningful safety or capacity enhancements  The major air carriers are also unhappy with their limited influence over project decisions, as airports are required only to consult with resident air carriers instead of having to get their agreement on PFC-funded projects  Unlike interest income from governmental bonds, which is not subject to the alternative minimum tax (AMT), interest from private activity bonds is still subject to the AMT  ACI-NA has proposed broadening the definition of governmental airport bonds to, in effect, include either all airport bonds or at least those bonds issued for public-use projects that meet AIP or PFC eligibility requirements  Opponents of such changes express concerns that these changes would reduce U S  Treasury revenues  Some also argue it would make more sense to change the AMT as part of a tax bill rather than including a specific exemption for income on airport bonds in an FAA reauthorization bill  In either case, such a change would not be under the jurisdiction of the congressional committees that will have jurisdiction over most reauthorization provisions  Changes to the AMT would be under the jurisdiction of the congressional tax-writing committees, the House Committee on Ways and Means and the Senate Committee on Finance  Appendix A  Legislative History of Federal Grants-in-Aid to Airports Prior to World War II the federal government viewed airports as a local responsibility  During the 1930s, it spent about $150 million a year on airports through work relief agencies such as the Works Progress Administration (WPA)  The first federal support for airport construction was granted during World War II  After the war, the Federal Airport Act of 1946 (P L  79-377) created the Federal Aid to Airports Program, using funds appropriated annually from the general fund  Initially much of this spending supported conversion of military airports to civilian use  In the 1960s substantial funding went to upgrade and extend runways for use by commercial jets  By the end of the 1960s, congestion, both in the air and on the ground at U S  airports, was seen as evidence that airport capacity was inadequate  Airport and Airway Development and Revenue Acts of 1970 (P L  91-258) In 1970, Congress responded to the capacity concerns by passing two acts  The first, the Airport and Airway Development Act (Title I of P L  91-258), established the Airport Development Aid Program (ADAP) and the Planning Grant Program (PGP), and set forth the programs' grant criteria, distribution guidelines, and authorization of grant-in-aid funding for the first five years of the program  The second, the Airport and Airway Revenue Act of 1970 (Title II of P L  91-258), established the Airport and Airway Trust Fund  Revenues from levies on aviation users and fuel were dedicated to the fund  Under the 1970 acts the trust fund could pay capital costs and, when excess funds existed, could also help cover FAA's administrative and operations costs  Airport and Airway Development and Revenue Acts Amendments of 1971 (P L  92-174) The Nixon Administration's FAA budget requests for FY1971 and FY1972 under the new trust fund system brought it into immediate conflict with Congress over the budgetary treatment of trust fund revenues  The Administration treated the new financing system as a user-pay system, whereas many Members of Congress viewed the trust fund as primarily a capital fund  The 1971 Amendments Act made the trust fund a capital-only account (although only through FY1976), disallowing the use of trust fund revenues for FAA operations  Airport and Airway Development Amendments Act of 1976 ( P L  94-353 ) The 1976 act made a number of adjustments to the ADAP and reauthorized the Airport and Airway Trust Fund through FY1980  The act again allowed the use of trust fund resources for the costs of air navigation services (a part of operations and maintenance)  However, in an attempt to assure adequate funding of airport grants, the act included "cap and penalty" provisions which placed an annual cap on spending for costs of air navigation systems and a penalty that reduced these caps if airport grants were not funded each year at the airport program's authorized levels  This cap was altered multiple times in reauthorization acts in the following decades  ADAP grants totaled about $4 1 billion from 1971 through 1980  Congress did not pass authorizing legislation for ADAP during FY1981 and FY1982, during which the aviation trust fund lapsed, although spending for airport grants continued  Airport and Airway Improvement Act of 1982 ( P L  97-248 ) The 1982 act created the current AIP and reactivated the Airport and Airway Trust Fund  It altered the funding distribution among the newly defined categories of airports, extending aid eligibility to privately owned general aviation airports, increasing the federal share of eligible project costs, and earmarking 8% of total funding for noise abatement and compatibility planning  The act also required the Secretary of Transportation to publish a national plan for the development of public-use airports in the United States  This biannual publication, the National Plan of Integrated Airport Systems (NPIAS) , identifies airports that are considered important to the national aviation system  For an airport to receive AIP funds it must be listed in the NPIAS  Although the 1982 act was amended often in the 1980s and early 1990s, the general structure of AIP remained the same  The Airport and Airway Safety and Capacity Expansion Act of 1987 ( P L  100-223 ; 1987 act) authorized significant spending increases for AIP and added a cargo service apportionment  It also included provisions to encourage full funding of AIP at the authorized level  Title IX of P L  101-508 , the Omnibus Budget Reconciliation Act of 1990 (OBRA1990), included the Aviation and Airway Safety and Capacity Act of 1990, which allowed airports, under certain conditions, to levy a Passenger Facility Charge (PFC) to raise revenue and also established the Military Airport Program (MAP), which provided AIP funding for capacity and/or conversion-related projects at joint-use or former military airports  The Airport Noise and Capacity Act of 1990 (OBRA 1990, Title IX, Subtitle D) set a national aviation noise policy  OBRA1990 included the Revenue Reconciliation Act of 1990, which reauthorized the Aviation Trust Fund and adjusted some of the aviation taxes  The Federal Aviation Reauthorization Act of 1994 ( P L  103-305 ) reauthorized AIP for two more years and again made modifications in the cap and penalty provisions  Federal Aviation Reauthorization Act of 1996 ( P L  104-264 ) The 1996 reauthorization of the AIP made a number of adjustments to entitlement funding and discretionary set-aside provisions  It also included directives concerning intermodal planning, cost reimbursement rules, letters of intent, and the small airport fund  A demonstration airport privatization program and a demonstration program for innovative financing techniques were established  The demonstration status of the state block grant program was removed  The act did not reauthorize the taxes that supported the Airport and Airway Trust Fund  This was done by the Taxpayer Relief Act of 1997 ( P L  105-34 ), which extended, subject to a number of modifications, the existing aviation trust fund taxes through September 30, 2007  The Wendell H  Ford Aviation Investment and Reform Act for the 21 st Century of 2000 (AIR21; P L  106-181 ) The enactment of AIR21 was the culmination of two years of legislative effort to pass a multiyear FAA reauthorization bill  The initial debate focused on provisions to take the aviation trust fund off-budget or erect budgetary "firewalls" to assure that all trust fund revenues and interest would be spent each year for aviation purposes  These proposals, however, never emerged from the conference committee  Instead, the enacted legislation included a so-called "guarantee" that all of each year's receipts and interest credited to the trust fund would be made available annually for aviation purposes  AIR21 did not make major changes in the structure or functioning of AIP  It did, however, greatly increase the amount available for airport development projects  The AIP funding authorization rose from $1 9 billion in FY2000 to $3 4 billion in FY2003  The formula funding and minimums for primary airports were doubled starting in FY2001  The state apportionment for general aviation airports was increased from 18 5% to 20%  The noise set-aside was increased from 31% to 34% of discretionary funding and a reliever airport discretionary set-aside of 0 66% was established  AIR21 also increased the PFC maximum to $4 50 per boarding passenger  In return for imposing a PFC above the $3 level, large and medium hub airports would forgo 75% of their AIP formula funds  This had the effect of making a greater share of AIP funding available to smaller airports  Vision 100: Century of Aviation Reauthorization Act of 2003 ( P L  108-176 ; H Rept  108-334 ) Vision 100, signed by President George W  Bush on December 12, 2003, included significant changes to AIP  The law codified the AIR21 spending "guarantees" through FY2007  It increased the discretionary set-aside for noise compatibility projects from 34% to 35%  It increased the amount that an airport participating in the Military Airport Program (MAP) could receive to $10 million for FY2004 and FY2005, but in FY2006 and FY2007 it returned the maximum funding level to $7 million  The act allowed nonprimary airports to use their entitlements for revenue-generating aeronautical support facilities, including fuel farms and hangars, if the Secretary of Transportation determines that the sponsor has made adequate provisions for the airside needs of the airport  The law permitted AIP grants at small airports to be used to pay interest on bonds issued to finance airport projects  The act included a trial program to test procedures for authorizing small airports to impose PFCs  Vision 100 repealed the authority to use AIP or PFC funds for most airport security purposes  FAA Modernization and Reform Act of 2012 ( P L  112-95 ) The 2012 FAA reauthorization act funded AIP for four years from FY2012 to FY2015 at an annual level of $3 35 billion  A new provision, Section 138, permitted small airports reclassified as medium hubs due to increased passenger volumes to retain eligibility for up to a 90% federal share for a two-year transition period  This provision also allows certain economically distressed communities receiving subsidized air service to be eligible for up to a 95% federal share of project costs  The 2012 act maintained the $4 50 PFC cap, with a maximum charge of $18 per round-trip flight  It included a provision that instructed GAO to study alternative means for collecting PFCs  The act also expanded the number of airports that could participate in the airport privatization pilot program from 5 to 10  This law was extended through July 15, 2016  The FAA Extension, Safety, and Security Act of 2016 ( P L  114-190 ) The 2016 FAA extension act funded AIP through FY2017 at an annual level of $3 35 billion  A new provision, Section 2303, provided temporary relief to small airports that had 10,000 or more passenger boardings in 2012 but had fewer than 10,000 during the calendar year used to calculate the AIP apportionment for FY2017  This provision allowed such airports to receive apportionment for FY2017 an amount based on the number of passenger boardings at the airport during calendar year 2012  The FAA Reauthorization Act of 2018 ( P L  115-254 ) The 2018 FAA reauthorization act funded AIP for five years from FY2019 through FY2023 at an annual level of $3 35 billion  It also authorized supplemental annual funding from the general fund to the AIP discretionary funds—$1 02 billion in FY2019, $1 04 billion in FY2020, $1 06 billion in FY2021, $1 09 billion in FY2022, and $1 11 billion in FY2023—and required at least 50% of the additional discretionary funds to be available to nonhub and small hub airports  The act included a provision permitting eligible projects at small airports (including those in the State Block Grant Program) to receive 95% federal share of project costs (otherwise capped at 90%), if such projects are determined to be successive phases of a multiphase construction project that received a grant in FY2011  The 2018 reauthorization expanded the number of states that could participate in the State Block Grant Program from 10 to 20 and also expanded the existing airport privatization pilot program (now renamed the Airport Investment Partnership Program) to include more than 10 airports  The law included a provision that forbids states or local governments from levying or collecting taxes on a business on an airport that "is not generally imposed on sales or services by that State, political subdivision, or authority unless wholly utilized for airport or aeronautical purposes " Appendix B  Definitions of Airports Included in the NPIAS Commercial Service Airports Publicly owned airports that receive scheduled passenger service and board at least 2,500 passengers each year (506 airports)  Primary Airports   Airports that board more than 10,000 passengers each year  There are four subcategories: Large Hub Airports   Board 1% or more of system -wide boardings ( 30 airports, 72 % of all enplanements)Medium Hub Airports   Board 0 25% but less than 1% (31 airports, 16% of all enplanements) Small Hub Airports   Board 0 05% but less than 0 25% (72 airports, 8% of all enplanements) Non hub Primary Airports   Board more than 10,000 but less than 0 05% (247 airports, 3% of all enplanements) Non primary Commercial Service Airports   Board at least 2,500 but no more than 10,000 passengers each year (126 airports, 0 1% of all enplanements)  Other Airports General Aviation Airports   General aviation airports do not receive scheduled commercial or military service but typically do support business, personal, and instructional flying; agricultural spraying; air ambulances; on-demand air-taxies; and/or charter aircraft service (2,554 airports)  Reliever Airports   Airports designated by FAA to relieve congestion at commercial airports and provide improved general aviation access (261 airports)  Cargo Service Airports   Airports served by aircraft that transport cargo only and have a total annual landed weight of over 100 million pounds  An airport may be both a commercial service and a cargo service airport  New Airports Seven airports are anticipated to be built between 2019 and 2023  They include two primary airports, two nonprimary commercial service airports, and three general aviation airports 

ground_truth: Full implementation of GPRAMA could facilitate efforts to reform the federal government and make it more effective. GPRAMA includes a provision for GAO to review the act's implementation. This report assesses how GPRAMA implementation has affected the federal government's progress in resolving key governance challenges in (1) addressing cross-cutting issues, (2) ensuring performance information is useful and used, (3) aligning daily operations with results, and (4) building a more transparent and open government. To address these objectives, GAO reviewed statutory requirements, OMB guidance, and GAO's recent work related to GPRAMA implementation and the key governance challenges. GAO also interviewed OMB staff and surveyed a stratified random sample of 4,395 federal managers from 24 agencies on various performance and management topics. With a 67 percent response rate, the survey results are generalizable to the government-wide population of managers. The Office of Management and Budget (OMB) and agencies have made some progress in more fully implementing the GPRA Modernization Act (GPRAMA), but GAO's work and 2017 survey of federal managers highlight numerous areas where improvements are needed. Cross-cutting issues: Various GPRAMA provisions are aimed at addressing cross-cutting issues, such as cross-agency and agency priority goals and related data-driven reviews of progress towards those goals. To ensure alignment with the current administration's priorities, OMB's 2017 guidance removed the priority status of those goals, which stopped quarterly data-driven reviews and related public progress reports until new goals are published. OMB plans to resume implementation of these provisions in February 2018. GPRAMA also requires OMB and agencies to implement an inventory of federal programs, which could help decision makers better identify and manage fragmentation, overlap, and duplication. OMB and agencies implemented the inventory once, in May 2013. In October 2014, GAO found several issues limited the usefulness of that inventory. Since then, OMB has postponed updating the inventory, citing among other reasons the passage of subsequent laws. OMB has yet to develop a systematic approach for resuming implementation of the inventory and specific time frames for doing so. A systematic approach to developing the inventory could help ensure it provides useful information for decision makers and the public. Performance information: Survey results show federal managers generally reported no improvements in their use of performance information in decision making for various management activities, or practices that can enhance such use, since GAO's 2013 survey. For example, the use of performance information to streamline programs to reduce duplicative activities (an estimated 33 percent in 2017) is statistically significantly lower relative to 2013 (44 percent). In contrast, managers who were familiar with and whose programs were subject to quarterly data-driven reviews reported that those reviews were used to make progress toward agency priority goals. Identifying and sharing practices to expand the use of such reviews—for other performance goals and at lower levels within agencies—could lead to increased use of performance information. Daily operations: Agencies have made progress in developing results-oriented cultures but need to take additional actions. GAO's past work found that high-performing organizations use performance management systems to help individuals connect their daily activities to organizational goals. In 2017, about half of federal managers reported using performance information when setting expectations with employees (no change from GAO's last survey in 2013). Transparent and open government: GAO's past work identified a number of needed improvements to Performance.gov, the central government-wide website required by GPRAMA. The site is to provide quarterly updates on priority goals in effect through September 2017, but those updates stopped in December 2016. According to OMB, the existing information for cross-agency priority goals is the final update, and agencies should publish final updates on their priority goals in annual performance reports. Performance.gov does not provide users with this information, thereby limiting the transparency and accessibility of those results. In addition to following through on plans to resume implementation of key GPRAMA provisions, GAO recommends that OMB (1) consider a systematic approach to developing the program inventory, (2) revise guidance to provide specific time frames for inventory implementation, (3) identify and share practices for expanding the use of data-driven reviews, and (4) update Performance.gov to explain that reporting on priority goals was suspended and provide the location of final progress updates. OMB staff agreed with these recommendations.