text: Background This section describes DOE’s M&O contracts, incentives in those contracts, general requirements for DOE’s M&O contractor performance evaluation processes, and contracting and performance challenges involving DOE’s M&O contracts that have been identified by previous reporting. DOE Uses M&O Contracts Since the Manhattan Project produced the first atomic bomb during World War II, DOE and its predecessor agencies have depended on the expertise of private firms, universities, and others with the scientific, manufacturing, and engineering expertise needed to carry out research and development work and manage and operate the government-owned, contractor-operated facilities where the bulk of the department’s mission activities are carried out. DOE relies on contracts in general, and M&O contracts in particular, to do this work. According to DOE’s Fiscal Year 2017 Agency Financial Report, the department spends approximately 90 percent of its annual budget on contracts, and in fiscal year 2016 DOE managed contracts valued at more than $24 billion. Of that amount, DOE spent approximately 80 percent on its M&O contracts. The work is closely related to the agency’s mission and is of a long-term or continuing nature, and there is a need to ensure its continuity and for protection covering the orderly transition of personnel and work in the event of a change in contractors. and sites that are contaminated from decades of nuclear weapons production and nuclear energy research. The Office of Fossil Energy (FE) manages the nation’s Strategic Petroleum Reserve, which is an emergency stockpile of oil stored in underground salt caverns in Texas and Louisiana. NNSA, a separately organized agency within DOE, is responsible for maintaining and enhancing the safety, reliability, and performance of the nation’s nuclear weapons stockpile, promoting international nuclear safety and nonproliferation, and supporting U.S. leadership in science and technology, among other things. The Office of Nuclear Energy’s (NE) primary mission is to advance nuclear power as a resource capable of making major contributions in meeting the nation’s clean energy supply and energy security needs. The Office of Science (SC) supports scientific research for energy and the physical sciences both by supporting (1) such research, and (2) the development, construction, and operation of scientific user facilities. These DOE offices use M&O contracts to carry out their research and development, nuclear weapons production, and other missions. For example, for research and development, DOE is the nation’s single largest funding source for basic physical sciences research, supporting research in energy sciences, advanced scientific computing, physics, and other fields. For weapons production, NNSA uses production sites to maintain, evaluate, repair, and dismantle both the nuclear and non- nuclear components for nuclear weapons; manufacture weapons components; and process tritium, a key isotope used to enhance the power of nuclear weapons. DOE also uses M&O contracts for sites dedicated to other types of missions, including nuclear waste disposal and an emergency stockpile of oil. Figure 1 and appendix II provide additional information on DOE’s M&O contracts. In August 2016, we identified three key attributes associated with DOE’s M&O contracts. First, M&O contracts have a limited competitive environment—we found that about half of DOE’s fiscal year 2015 M&O contract spending was on contracts awarded noncompetitively or that received a single offer at the time they were competed. In addition, M&O contracts include longer terms than other federal contracts, so they are competed less frequently. Second, DOE M&O contracts have broad scopes of work that cover nearly all aspects of work at a site. In particular, though mission activities of M&O contractors can be highly technical, mission support activities generally accounted for about 25 to 50 percent of contractors’ total costs in fiscal year 2015, and encompassed such things as managing infrastructure, facilities, and grounds; security; and the internal audit function. Third, M&O contracts and DOE management practices contribute to a closer relationship between contractors and the government. For example, M&O contractors are generally more integrated with DOE in how they are paid and in their accounting systems than other types of contractors. With regard to payment, rather than traditional bill payment methods including invoices, payment approval and authorization, and disbursement of funds, M&O contractors can draw funds directly from federal accounts through “letter of credit financing.” With regard to accounting systems, as we reported in August 2016, DOE requires M&O contractors to follow DOE’s Accounting Handbook and integrate their costs and liabilities in DOE’s accounts each month. DOE officials said that this provides visibility into contractor accounts and allows DOE to monitor the appropriateness of the contractors’ withdrawal of funds in near real time. According to DOE officials, this integration carries over into how the value of contracts are determined—rather than establishing the cost of the contract at the time of contract award, the value of the M&O contract is determined by the amount annually obligated on the contract by DOE, consistent with DOE’s annual congressional appropriations. Incentives in M&O Contracts Cost-reimbursement type contracts allow the agency to contract for work when circumstances do not allow the agency to sufficiently define its requirements or estimate its costs to allow for a fixed-price contract. Under a fixed-price contract, a contractor accepts responsibility for completing a specified amount of work for a fixed price. In contrast, under cost-reimbursement contracts, the government reimburses a contractor for allowable costs incurred, to the extent prescribed by the contract. The government may also pay a fee that is either fixed at the outset of the contract or adjustable based on performance criteria set out in the contract. In September 2009, we reported that cost-reimbursement contracts are considered high risk for the government because of the potential for cost escalation and because the government pays a contractor’s costs of performance regardless of whether the work is completed. As such, cost-reimbursement contracts are suitable only when (1) circumstances do not allow the agency to define its requirements sufficiently to allow for a fixed-price type contract; or (2) uncertainties involved in contract performance do not permit costs to be estimated with sufficient accuracy to use any type of fixed-price contract. One major reason for the inability to accurately estimate costs is the lack of knowledge of the work needed to meet the requirements of the contract, such as with research contracts, which necessarily involve substantial uncertainties. The DOE Acquisition Regulation (DEAR) states that cost-plus-award-fee (cost reimbursement) contracts are generally the appropriate contract type for M&O contracts and that the agency can choose among a number of different contract types for its M&O contracts. Under the FAR, cost-reimbursement contracts may include specific incentives, such as arrangements intended to improve contractor efforts and discourage inefficiency and waste. Table 1 provides definitions of incentives commonly included in DOE’s M&O contracts. Generally, according to DOE officials, award fees and incentive fees are intended to motivate M&O contractor performance on an annual basis, as outlined in annual performance evaluation plans. All DOE M&O contracts GAO analyzed also include “conditional payment of fee” clauses that permit the agency to reduce an otherwise earned fee if it determines that the contractor’s performance did not meet minimum requirements, such as those related to safety, health, or the environment. Under the award term incentive, contractors can earn one additional year of performance under the contract for each year they exceed certain thresholds in their annual performance evaluations. (See apps. III through VIII for additional information on the incentives included in each M&O contract, by DOE office.) In addition, other elements of contract administration or oversight, while not formally incentives, can influence contractor performance. For example, option periods—which are established in the contract—enable the government to unilaterally extend the performance period and performance of services. According to DOE officials, other potentially important influences on contractor behavior include public reputation and the ability to compete for follow-on DOE or other government contracts. Performance Evaluation and Award Fee Requirements The FAR, DEAR, DOE’s Acquisition Guide, and DOE policies provide requirements and guidance for DOE’s annual performance evaluations of contractor performance. Under the FAR, all contracts providing for award fees must be supported by an award fee plan that establishes procedures for evaluating award fees and an Award Fee Board to conduct award fee evaluations. A Fee Determining Official makes the final determination regarding the amount of award fee the contractor earns during the evaluation period. Additionally, the FAR generally calls for entities that administer contracts providing award fees to use a set of ratings from Excellent to Unsatisfactory, which include performance descriptions and associated available award fee percentages (see Table 2 below). Award fee ratings are associated with a range of percentages of the total available award fee that DOE offices may award to a contractor based on the contractor’s assessed performance. DOE offices develop two primary documents to guide and report assessments of contractors’ performance for each fiscal year: a Performance Evaluation and Measurement Plan (PEMP) and a Performance Evaluation Report (PER). The PEMP is to be developed at the beginning of each fiscal year—which is the beginning of the evaluation period—and is to establish expectations for contractor performance and describe how the responsible DOE office will evaluate and measure performance against those expectations. The PEMP provides the blueprint for what performance is expected of contractors, how contractors’ performance will be evaluated, and how the evaluations will be used to determine award fees, award terms, and any other incentives. The PER is to be developed at the end of each evaluation period—which typically is the end of the fiscal year—and is the responsible DOE office’s evaluation of contractor performance, in which DOE documents the performance rating and, in some cases, the fees and other incentives that will be awarded to the contractor. Figure 2 shows the general steps of DOE’s performance evaluation of contractors. Further, under the FAR and DOE policy, the department is to consider technical, administrative, and cost performance during acquisition planning. The FAR provides that, for M&O contracts, replacement of an incumbent contractor is largely based on an expectation of meaningful improvement in performance or cost; thus, an agency or department should consider three categories of performance—technical, administrative, and cost—when deciding whether to extend or compete a contract at the end of the contract’s term. According to DOE officials, the annual performance evaluation process and the related PER are important sources of information for making these decisions. Thus, the PER should include relevant information on an M&O contractor’s technical, administrative, and cost performance. For DOE, the M&O contract PER is also important because DOE uses information from the PER to update a contractor’s past performance information in the Contractor Performance Assessment Reporting System (CPARS), which DOE and other agencies use to understand a contractor’s performance history and to inform their evaluations of future contract proposals. Contracting and Performance Challenges Involving DOE’s M&O Contracts A number of commissions, task forces, and other outside groups have identified challenges involving DOE’s M&O contracts. For example, two independent commissions—the Augustine-Mies Panel and CRENEL— have reported on related contract management challenges. The 2014 Augustine-Mies report focused on NNSA and made numerous recommendations for comprehensive reforms, including addressing dysfunctional government-M&O contractor relationships, improving oversight of M&O contractors, and reforming award fee and performance incentive structures. CRENEL, taking a broader view of all 17 national laboratories across DOE, in 2015 found a similar erosion of trust between DOE and some of its M&O contractors while noting that some laboratories, in particular those under SC, had better, more effective relationships. The CRENEL report recommended reforms to the management and oversight of M&O contractors and performance incentive structures. In addition to challenges, CRENEL also noted that SC’s annual performance evaluation and planning processes were robust and suggested that they be adapted by other DOE offices. NNSA’s and EM’s contract management remains on our High-Risk List for government operations vulnerable to fraud, waste, abuse, and mismanagement. In addition, since 2005 we have identified a variety of project and program outcomes associated with deficiencies in DOE’s management and oversight of its M&O contractors. We have also identified improvements needed in core processes and functions DOE relies on to oversee its M&O contractors and assess their performance. These reports include the following examples: Since 2005, during various reviews, we found that cost accounting practices used by NNSA’s M&O contractors have varied, making it difficult for NNSA to compare costs across its sites or accurately identify the total costs across its nuclear security enterprise and to obtain reliable cost data. In January 2017, we reported on the importance of reliable enterprise-wide cost information to effective management and oversight and found that the plan NNSA submitted to Congress in 2016 to improve and integrate its financial management, as required by Congress in 2013, did not provide a useful road map for guiding NNSA’s efforts. We recommended that NNSA develop a plan for producing cost information that fully incorporates leading planning practices. NNSA agreed, and we are monitoring implementation of the recommendation. In October 2014, we reported on actions taken to address challenges with the Uranium Processing Facility under construction at the NNSA Production Office Sites (specifically at the Y-12 National Security Complex), which is managed by the M&O contractor at that site. A challenge with this facility was that in July 2012 the M&O contractor concluded that required equipment would not fit into the facility as designed and that addressing this issue would cost an additional $540 million. NNSA’s analysis of the factors that contributed to this issue identified several causes, including project oversight deficiencies— specifically, failure to ensure that requests and directives from NNSA to the contractor were implemented. In May 2015, we reported on NNSA’s use of contractor assurance systems to conduct oversight and evaluate the performance of M&O contractors. Contractor assurance systems are designed and used by M&O contractors to oversee their own performance and to self- identify and correct potential problems. We found that NNSA had not fully established policies or guidance for using information from these systems to conduct oversight of M&O contractors and that NNSA therefore did not have standards for ensuring that contractors are overseen consistently. We recommended that NNSA establish policies and guidance for using information from contractor assurance systems for the oversight of M&O contractors; NNSA concurred with the recommendations and has taken some steps to establish policies and guidance, though it has not yet fully addressed our recommendations. In March 2017, we reported that DOE needed quality data to manage its risk of fraud and recommended that DOE require contractors to maintain sufficiently detailed transaction-level cost data that are reconcilable with amounts charged to the government. DOE did not concur with the recommendation and has not taken steps to implement it. Because DOE does not require its contractors to maintain sufficiently detailed transaction-level cost data that are reconcilable with amounts charged to DOE, it is not well positioned to employ data analytics as a fraud detection tool. As a result, DOE is missing an opportunity to develop, refine, and improve its data analytics and better meet requirements of the Fraud Reduction and Data Analytics Act. DOE Offices Use Different Approaches to Evaluate Contractor Performance, and all but NNSA Have Documented Their Approaches In fiscal years 2006 through 2016, the six DOE offices generally used one of three different approaches to evaluate M&O contractor performance. All but one of these offices have documented their approaches in policies and procedures; NNSA has a broad policy but does not have procedures for implementing it, in particular for collecting and using performance information. In the absence of documented procedures, NNSA may not consistently collect and use performance information in evaluating contractor performance. DOE Offices Use Three Approaches That Differ in Their Criteria and Methodologies for Ratings and Incentives According to DOE officials, DOE does not have a department-wide performance evaluation process and offices developed their approaches to performance evaluation based on their varying missions and performance evaluation priorities. We identified the following three general approaches: The Science and Energy Lab approach (used by SC, EERE, and NE) uses broad, office-wide performance criteria and a detailed process and web-based tool to collect performance information and determine ratings and incentives. The NNSA approach also uses broad, office-wide performance criteria, but ratings and incentives are determined through a series of management meetings. The Site Specific approach (used by FE and EM) uses more detailed performance criteria specific to each contract and makes rating and incentive determinations in ways that vary based on the individual criteria. These approaches generally differ in their (1) performance criteria, (2) methodologies used to determine contractor ratings, and (3) methodologies used to determine incentives. Appendixes III through VIII provide additional information on each office’s performance evaluation approach. Performance Criteria Based on our review of DOE documents, the three approaches all use a combination of what PEMPs describe as subjective and objective performance criteria. The Science and Energy Lab and NNSA approaches use primarily subjective criteria, and the Site Specific approach uses primarily objective criteria. Subjective criteria are generally qualitative statements that describe desired contractor performance, according to DOE officials. For example, a subjective criterion that SC used during fiscal year 2016 was for contractors to “provide effective and efficient strategic planning and stewardship of scientific capabilities and program vision.” In contrast, DOE officials explained that objective criteria generally describe performance that may be measured on a “pass/fail” or quantitative basis. For example, FE used objective criteria such as developing a strategic plan by a specific date or ensuring that all phases of construction were mechanically complete regarding the conversion of a tank. Performance criteria under the Science and Energy Lab and NNSA approaches share a similar structure of three tiers of criteria: goals, objectives, and notable outcomes (called key outcomes under NNSA’s approach). The criteria are also mostly subjective and broad enough to be consistent across all the contracts of the responsible DOE office. Based on our review of DOE documents and information, SC and EERE have used the Science and Energy Lab approach since fiscal year 2006 and NE since fiscal year 2007. NNSA used the NNSA approach in fiscal years 2013 through 2016. Under the Science and Energy Lab and NNSA approaches, goals are general overarching statements of the desired outcomes for each major performance area under the contract and constitute the highest performance criteria used to evaluate contractor performance. Based on documentation describing these approaches, goals are to be composed of at least two objectives, which are statements of desired results for an organization or activity and that discuss specific actions the contractor will undertake to accomplish a goal. Each office uses its respective goals and objectives consistently for each of its M&O contracts (EERE and NE each have only one site) and generally cover the same functional areas across the offices, though some NNSA goals focus specifically on NNSA’s nuclear weapons and national security missions. For complete lists of goals and objectives used by the offices using the Science and Energy Lab and NNSA approaches, see appendixes III, VI, VII, and VIII. The third tier performance criteria used to evaluate contractor performance is the notable outcome, which, according to agency documents, is intended to focus the contractor on specific items that officials identified as the most important initiatives or highest risk issues the contractor must address. According to DOE documents, notable outcomes differ from goals and objectives in that they (1) are usually objective, (2) are specific to each contractor, and (3) change from year to year. However, not all goals and objectives have associated notable outcomes. Figure 3 provides an example of the relationship between a goal and its related objectives and notable outcomes for SC’s Brookhaven National Laboratory contractor for fiscal year 2016. Our review of agency documents found that the Site Specific approach consists primarily of objective performance criteria that are specific to each contract, as well as a few broader, objective criteria. This is in contrast to the other two DOE approaches to performance evaluation, which primarily rely on broad, subjective criteria and a few objective criteria. Based on our discussions with agency officials, both EM and FE have generally used this Site Specific approach since fiscal year 2006. For both offices, objective performance criteria are defined based on quantifiable metrics (e.g., a contractor’s demonstrated waste processing rate) and milestones (e.g., whether a contractor completed a task on or before a scheduled date). For example, one of FE’s fiscal year 2016 objective performance criteria for the Strategic Petroleum Reserve M&O contract is whether facilities and systems functioned at a level adequate to meet program requirements based on average scores from its Maintenance Performance Appraisal Rating tool. Further, our review of agency documents showed that the Site Specific approach uses subjective performance criteria for aspects of performance that may be difficult to capture objectively—such as determining how effectively measures a contractor has taken have prevented harm to workers, the general public, and the environment. (See apps. IV and V for examples of the objective and subjective criteria EM and FE use.) Prior to fiscal year 2013, NNSA also used the Site Specific approach, and it had specific, objective performance criteria that varied by contract. Based on our review of agency documents, NNSA’s performance criteria were generally divided into four performance areas: (1) mission, (2) operations, (3) business, and (4) multi-site. According to NNSA officials, as a result of “lessons learned” efforts, NNSA updated this approach to its current one to provide more succinct, structured, and consistent reporting by ensuring that all NNSA M&O contractors have identical goals and objectives. Methodologies Used to Determine Contractor Ratings Based on our review of DOE documents, rating methodologies vary across the three approaches—the Science and Energy Lab approach uses a detailed, formulaic methodology; the NNSA approach determines ratings at a series of management meetings; and in the Site Specific approach, ratings depend primarily on whether the contractor accomplishes specific tasks. Based on our review of agency documents, under the Science and Energy Lab approach, stakeholders—including officials from headquarters, field offices, and internal and external customers— generally evaluate contractor performance against the criteria for each objective and notable outcome (“lab customers” evaluate objectives under science and technology goals only). Their evaluations, in the form of narratives and numerical scores, are entered into a web-based information collection tool that aggregates the scores using a series of calculations and weights to generate ratings that are then approved by the Fee Determining Official for the responsible DOE office. For example, for SC, once individual stakeholders enter objectives’ scores into the Laboratory Rating Tool, those scores are then weighted and added together through a predetermined formula to provide an overall rating of contractor performance for each goal. Under this approach, the Laboratory Rating Tool aggregates the objective scores into numerical goal ratings and corresponding letter grades from 4.3 (A+) to 0 (F) for the contractor. Notable outcomes are rated on a “pass/fail” basis, meaning that the contractor either met or did not meet them. Receiving a passing rating for the notable outcome is required for the contractor to earn a B+ or better for the notable outcome’s associated objective. Thus, although notable outcomes are not given their own numerical score or letter grade, they can have a significant effect on a contractor’s objective ratings and, ultimately, goal ratings. (See apps. III, VII, and VIII for examples of the weighting and calculations involved in aggregating ratings for EERE, NE, and SC M&O contractors.) Based on our review of agency information, the methodology for the NNSA approach to determine contractor ratings entails officials holding a series of meetings to review various internally developed periodic reports and other inputs (e.g., contractor self-assessments and inspection reports). The participants in these meetings include field office managers, program managers, and NNSA executive leadership who collaboratively review contractor performance and determine ratings. According to NNSA officials, at these meetings NNSA collaboratively reviews all M&O contracts across the NNSA complex, thereby allowing officials to weigh and compare performance. The Fee Determining Official determines the final performance ratings for each M&O contractor using rating categories from the FAR: Excellent, Very Good, Good, Satisfactory, and Unsatisfactory. NNSA does not use numerical calculations to score and weigh individual objectives or goals. Instead, NNSA officials use professional judgment to determine overall goal ratings. Based on our review of agency information, under the Site Specific approach, field office officials rate contractor performance against objective performance criteria quantitatively or pass/fail and rate subjective performance criteria using FAR award fee categories. That is, they evaluate performance against objective performance criteria as completed or not completed—for example, whether the contractor packaged 10 waste drums during the fiscal year. For the subjective performance criteria, officials assign ratings using the FAR rating categories in a similar manner to the NNSA approach. Methodologies Used to Determine Incentives Based on our review of DOE documents, the three performance evaluation approaches also use different methodologies for determining award and incentive fees, and two offices use similar methods to determine whether the contractor receives award term. Based on our review of agency documents, under the Science and Energy Lab approach, once ratings are determined, several additional detailed calculations determine how much of the available award fee is provided to the contractor. Precisely how ratings are weighted to determine fee differs by DOE office, but generally performance in technical areas is more important in determining the amount of fee the contractor earns. For example, SC determines award fees based on the contractor’s final science and technology area rating and adjusts that fee if the final management and operations area rating is 3.0 (grade B) or below. (See app. VIII for additional information on SC’s fee determination, app. III for EERE, and app. VII for NE.) Based on our review of agency information, under the NNSA approach, officials assign goals specific portions of the available award fee for each contract at the beginning of the fiscal year. At the end of the fiscal year, officials determine ratings and fees at the same time in the collaborative meeting with NNSA leadership. For example, for the Los Alamos National Laboratory contractor in fiscal year 2016, the nuclear weapons goal was 30 percent of fee, and the operations and infrastructure goal was 35 percent. As discussed earlier, the Fee Determining Official makes the final determination on the ratings and also determines how much fee to provide the contractor within the range defined by the FAR rating (Excellent, Very Good, Good, Satisfactory, Unsatisfactory). In fiscal year 2016, NNSA awarded the Los Alamos National Laboratory M&O contractor an “Excellent” rating for the nuclear weapons goal, which is associated with the contractor earning from 91 to 100 percent of the available fee for that goal. To determine the overall award fee for the contract, NNSA adds up the award fees for all of its goals. (See app. VI for an example of a NNSA fee determination letter.) Our review of DOE documents showed that the Site Specific approach has a different process for determining incentive and award fees, depending on whether the fee is tied to objective or subjective performance criteria. According to agency officials and documents, the Site Specific approach generally provides more money toward incentive fees tied to objective criteria than to award fees tied to subjective criteria—about 60 to 75 percent of available fee money goes to incentive fees. Incentive fees tied to objective performance criteria are awarded based on completion of the specific tasks or quantitative targets defined by the performance criteria. For example, one of the objective performance criteria for EM’s Waste Isolation Pilot Plant (WIPP) M&O contractor in fiscal year 2016 was to develop a maintenance and engineering program, called the Material Condition and Aging Management Program, and complete certain program activities. EM set a maximum incentive fee of $500,000 in the PEMP to be awarded upon completion of the activities. In regard to award fees that are tied to subjective performance criteria under the Site Specific approach, offices using this approach take a similar method to the NNSA approach, in that they determine ratings and fees simultaneously. Specific portions of an available award fee are assigned to subjective performance criteria at the beginning of the fiscal year and documented in the PEMP, and officials then determine the percentage of fee to award and corresponding ratings from the FAR award fee categories for each subjective performance criterion. The final decision on the percentage of the available fee awarded for subjective performance criteria is made by the Fee Determining Official, who is generally an on-site official. The overall fee awarded is the sum of the individual objective incentive fees and subjective award fees. (See apps. IV and V for examples of how fee is assigned to specific criteria under the Site Specific Approach.) With regard to award term, for the SC and NNSA contracts that had award term as an incentive, the contracts defined the conditions for receiving it, and those conditions generally included meeting certain rating thresholds, based on our review of documents from those offices. For SC, the contractor (1) was to earn at least a 3.5 (A-) science & technology area rating and a 3.1(B+) management & operations area rating, and (2) have no individual goal ratings below 3.1(B+) for science & technology area goals and 2.5 (B-) for management & operations area goals. The contracting officer is to prepare and submit a standardized document along with an annual contractor performance evaluation presentation for review through program officials, and the Director of the Office of Science is to make the final award term determination. For NNSA contracts, the contractor generally must (1) earn a rating of “Very Good” or better in four of the six goals and receive no rating of “Satisfactory” or lower in any goal, and (2) meet any additional requirements as specified in the contract. All DOE Offices but NNSA Have Clearly Documented Approaches, Which May Lead NNSA to Inconsistently Collect and Use Performance Information in Contractor Evaluations All of DOE’s offices have documented policies outlining their performance evaluation approaches, and all but NNSA have documented how information is to be collected and used to make rating determinations. SC, EERE, NE, FE, and EM have included in their documented policies and performance evaluation plans detailed procedures for collecting information on contractors’ performance that outline, among other things, how officials are to gather input from internal and external stakeholders and how the officials are to use that information in making rating determinations. For example, under SC’s Laboratory Performance Appraisal Process and PEMP Preparation Guidance (SC’s Appraisal Guidance), stakeholders are to provide evaluations using SC’s web- based information collection tool, the Laboratory Rating Tool, to provide scores and narratives on contractor performance. As a result, SC’s contractor performance evaluation approach clearly traces where performance information comes from and how the information is used in determining contractors’ final ratings. Similarly, EM and FE document how officials are to collect information and use it in PEMPs or other performance evaluation plans. For example, EM’s PEMP for the WIPP M&O contract provides step-by-step procedures for how field office officials are to assess contractor performance against each performance criterion. These procedures guide the flow of information from contractor to field office officials, who are to check and validate the information and provide rating and fee recommendations to the on-site Fee Determining Official. Similarly, field office officials at EM’s Savannah River Site and FE’s Strategic Petroleum Reserve also have detailed procedures for assessing and distributing information regarding performance. Such detailed written procedures can provide better assurance to agencies that officials are consistently gathering and using performance evaluation information and that one can trace the ultimate performance rating in the PER to the underlying performance information. In contrast to the detailed documented policies of other DOE offices, during the period of our review NNSA’s documented policy did not always match its performance evaluation approach, and the policy did not contain procedures for how officials should collect and use information so that one can trace the performance rating to the underlying performance information. As noted above, NNSA changed from using the Site Specific performance evaluation approach that focused on objective performance criteria to the agency’s current approach in fiscal year 2013. However, NNSA did not update its policy to reflect this change until December 2016. Thus, in fiscal years 2013 through 2016, NNSA was using a policy intended to evaluate site-specific objective performance criteria and incentive fees rather than the broad, office-wide subjective performance criteria that NNSA was using during those 4 fiscal years. NNSA brought its policy into alignment with its performance evaluation approach in December 2016 by issuing its Corporate Performance Evaluation Process for Management and Operating Contractors policy (NAP-4C). NAP-4C provides a general framework under which NNSA officials provide input into the contractor performance evaluation process; the policy also provides a general schedule for implementing the performance evaluation approach, as well as general references to information collection. However, NAP-4C does not include detailed procedures for how performance information should be collected and used, and according to NNSA officials, individual NNSA offices and officials determine how they collect and distribute information. This means information may be collected inconsistently across the agency, depending on individual offices’ preferences. For example, NAP-4C states that officials should “leverag information from contractor assurance systems . . . to monitor performance” but does not discuss how and when officials should use this information to ensure performance information is traceable to rating determinations. In May 2015, we reported on the importance of tracing performance information from contractor assurance systems to performance evaluations. We reported that a senior NNSA official told us NNSA could not track the extent to which information from contractor assurance systems was used in evaluating contractor performance because it could be difficult to identify the sources of information used in performance evaluations. We recommended that NNSA revise policy, guidance, and procedures on performance evaluation to fully address how and under what circumstances those responsible for evaluating M&O contractors’ performance should use information from contractor assurance systems for this purpose. NNSA concurred with our recommendation and issued revised policy for contractor oversight but has not yet developed guidance or procedures for how to use information from contractor assurance systems in its performance evaluation process. We continue to follow up on this recommendation. In addition to NAP-4C, NNSA’s Fee Determining Official issued implementation guidance for the fiscal year 2016 performance evaluation cycle. This implementation guidance directs relevant NNSA officials to follow a series of templates for interim reports to the contractor and provides the format of the final PER and specific dates for those reports. The guidance does not include procedures as to how officials throughout NNSA are to collect or use information to create the content for those templates. For example, the guidance’s Interim Feedback Report schedule states that the “program/functional offices provide input to field offices.” There is no discussion of how the program/functional office is to provide such input, what types of input are important, or how the input is to be used. Similarly, NNSA’s PEMPs also do not discuss how officials should collect or use performance information. In the absence of documented, detailed procedures, NNSA may not consistently collect and use performance information from program managers and field office officials for contracts in a given fiscal year and may therefore inconsistently apply NNSA’s evaluation process. For example, we identified two instances in which the NNSA Fee Determining Official made handwritten changes to proposed award fee amounts during fiscal year 2012 without documenting in the PER the basis for the changes, such as by identifying the performance information that would support the handwritten changes to create traceability between the award fee amounts and its supporting performance documentation. These changes awarded (1) Los Alamos National Laboratory’s contractor a year of award term, even though the contractor had not met the established rating threshold for award term, and (2) Lawrence Livermore National Laboratory’s contractor a higher award fee that also qualified the contractor for award term it otherwise would not have received. With these changes, these contractors received award terms and fees in a manner inconsistent with how award terms and fees were assessed for other M&O contractors. According to NNSA officials, this type of action would not happen currently because the agency’s approach is rooted in a policy (NAP-4C) and implementation guide that is supported by a more collaborative decision-making process. However, even under the new policy, because NNSA does not have clearly documented procedures specifying how officials are to collect or use performance information, NNSA leadership cannot have assurance that there is clear traceability between the contractor evaluation and its underlying support. Federal standards for internal control state that management should design control activities to achieve objectives and respond to risks, such as by clearly documenting internal control in management directives, administrative policies, or operating manuals. NNSA has a documented policy, but this policy does not clearly specify how to collect and use contractor performance information to evaluate contractor performance. NNSA officials stated that in their opinion their policy was still effective and robust without detailed procedures for its implementation. However, without developing and documenting clear procedures for implementing NAP-4C that specify the process for collecting contractor performance information and how officials are to ensure this information can be traced to rating determinations, NNSA leadership does not have reasonable assurance that the agency is consistently evaluating contractor performance and that it is using relevant performance information as intended. Evaluation Reports Could Better Assess M&O Contractors’ Cost Performance DOE’s Fiscal Year 2016 M&O Contractor Performance Evaluation Reports Provided Less Information on Contractors’ Cost Performance than on Other Types of Performance We found that DOE offices’ fiscal year 2016 PERs provided less information on M&O contractors’ cost performance—evaluations of the contractor’s spending, budgeting, strategic sourcing, and costs, including the contractor’s cost-effectiveness—and provided more information regarding technical and administrative areas of performance. Specifically, the PERs were 67 pages long on average and contained about 1 page of cost performance-related information overall. In contrast, information on contractors’ technical and administrative performance included in-depth descriptions of contractors’ scientific discoveries and production progress that spanned numerous pages. Figure 4 provides typical examples of the type of technical, administrative, and cost performance descriptions that we found in our review of fiscal year 2016 M&O contract PERs. In addition, in our review of the number of performance descriptions in DOE’s 2016 PERs, we found about 24 percent (179 of 737) of the performance descriptions in the PERs provided information on cost performance; about 71 percent (524 of 737) provided information on administrative performance (evaluations of contractor’s performance on mission support activities, such as information technology, human resources, legal activities, environmental safety and health, property management, risk management, and leadership activities); and about 53 percent (390 of 737) provided information on technical performance (evaluation of contractor’s performance on mission- related activities such as research and development, production, storage, clean-up, and construction). Cost Performance Information Included in DOE’s Performance Evaluation Reports is of Limited Use for Acquisition Decision-Making In addition to providing less information on M&O contractors’ cost performance than on other areas of performance, the cost information contained in DOE offices’ PERs is of limited use for acquisition decision- making. DOE’s Information Quality Guidelines define quality, in part, as information that is useful to DOE and the public. We examined whether the PERs included such useful information that would permit an overall assessment of contractor cost performance. FAR and DOE policy call for such an overall assessment, which therefore is useful to DOE for acquisition decision-making and to the public generally. Our analysis showed that the information on contractors’ cost performance in the PERs did not permit such an assessment of contractor cost performance for two primary reasons. First, the information consisted of statements that lacked detail, such as “within budget,” and did not address the significance of the performance described. For example, cost performance-related statements such as “over/under budget” and “cost savings/cost overrun” did not commonly provide information on the amount saved or lost, making it difficult to identify the significance of what was reported. Information on cost effectiveness was also rare—cost-effectiveness information was included in about 11 percent of the instances in which cost performance was discussed (48 of 441 instances). Second, cost performance information commonly applied to specific activities under the contract, such as construction activities, rather than to achievement of overall operating efficiencies. When cost performance information is limited to specific activities, it is not possible to assess a contractor’s overall cost performance based on information in the PER. We identified one reason and DOE officials identified three additional reasons why more cost performance information was not provided in DOE’s fiscal year 2016 PERs. We believe all of these contribute to why the cost performance information that was included was often not useful for acquisition decision-making: DOE offices’ policies and PEMPs did not specifically require PERs to include cost performance information and did not discuss how to ensure that cost information is useful for acquisition decision-making. Based on our review, DOE offices’ policies did not specifically require that PERs include cost performance information, nor did they discuss information quality. In addition, DOE offices’ PEMPs—which serve as a general blueprint for the type of performance information that offices should include in the corresponding PER—generally did not include specific cost performance criteria or explicitly call for evaluations of contractors’ cost performance. In contrast, DOE offices’ fiscal year 2016 PEMPs commonly included explicit technical and administrative performance criteria such as: “provide S&T results with meaningful impact on the field” (technical) and “provide an efficient and effective worker health and safety program” (administrative). There were three exceptions in which PEMPs included specific cost performance criteria: EM’s WIPP M&O contract, NE’s Idaho National Laboratory, and FE’s Strategic Petroleum Reserve M&O contract. Although SC does not have explicit cost performance goals or objectives, according to SC officials, cost performance is listed as a factor to consider in SC’s PEMPs’ descriptions of how to evaluate certain performance criteria. However, SC officials told us that PER performance descriptions may not include cost information for these criteria unless there were notable cost overruns or the contractor was doing an exceptionally good job in these areas. SC officials stated this is, in part, to keep PERs shorter and streamlined. However, when PERs are silent on cost performance, there is no formal documented record of M&O contractor cost performance. M&O contract missions made it difficult for DOE to assess contractor cost performance, resulting in less cost performance information in PERs. According to DOE officials, it is difficult to assess the costs of the scientific and research missions covered by many M&O contracts. For example, according to DOE officials, it is difficult to develop cost estimates for research activities because it is not always certain when scientific breakthroughs will occur or how long they will take. DOE uses cost-reimbursement contracts for its M&O contracts, in part because it is not possible to know with certainty and in advance how much research and development efforts will cost or what level of effort will be required. While we agree that assessing cost performance for scientific and research activities may be difficult, M&O contractors also carry out a variety of other activities for which costs may be more readily assessed. For example, a sizeable portion of the costs under M&O contracts are for administrative or mission support and other business operations activities, such as personnel, business processes, human resources, procurement, and security. In our previous work, we found that such administrative and support activities accounted for about 25 to 50 percent of M&O contractor costs in fiscal year 2015.Similarly, SC’s fiscal year 2016 annual laboratory plans identify areas, such as infrastructure and information systems, as the major cost drivers for that year. We have found that other agencies assess cost performance for contractors performing such administrative activities. DOE officials we interviewed agreed that measuring cost performance in these areas would be more feasible than measuring it for its scientific and research missions. The M&O contract type made it difficult to some degree for DOE to assess contractor cost performance. According to DOE officials, certain aspects of how DOE implements cost-reimbursement M&O contracts create challenges to evaluating cost performance. Some officials described these challenges as the result of “the budget-based nature” of M&O contracts. Specifically, according to DOE officials, M&O contract budgets (the amount contractors are allowed to spend) are not set up front in the original contract. Rather, according to DOE officials, M&O contract budgets are commonly determined by the amount DOE obligates to the contract on an annual basis, based mostly on annual congressional appropriations to the relevant DOE programs. Further, these officials noted, because much of DOE’s appropriated funds are available until expended rather than expiring at the end of the fiscal year for which they were appropriated, M&O contractors may be able to carry over those funds to spend in future fiscal years. According to DOE officials, DOE reviews M&O contractor estimates when developing its budget request, including determining how much work is required by its contractors to execute the program scope outlined in the budget request. Agency officials also noted that, with regard to cost reimbursement contracts, the federal government is legally required to reimburse contractors for all allowable costs up to the approved budget amount. We have previously reported that cost-reimbursement contracts carry a high risk for the federal government, resulting in the potential for cost escalation, as some expenditures may be allowable under the contract but may not be cost effective. We recognize that M&O contracts are unique in many ways. Nevertheless, the manner in which DOE allocates funds to the contract, and the requirement to reimburse contractors for allowable costs do not, by themselves, affect DOE’s ability to assess contractor cost performance. Some cost performance evaluation conducted outside of the annual performance evaluation process is not included in PERs. DOE officials told us they perform some activities related to contractor cost performance outside the performance evaluation process for M&O contracts, though information on these activities is not always included in PERs. For example, according to DOE officials, some M&O contractors participate in group purchasing efforts, where contractors coordinate purchases to drive up competition and drive down costs. Also, DOE offices generally monitor M&O contractor indirect costs to ensure they do not escalate without reason. In particular, SC’s M&O contractors include a “Cost of Doing Business” section in their annual laboratory plans, in which SC contractors report on indirect costs. According to SC officials, SC also uses its reviews of the Cost of Doing Business sections as opportunities to discuss options to reduce operational costs. SC officials stated that an internal process in which SC’s laboratories compete and are awarded work, in part, also serves to control costs. According to DOE officials, efforts such as group purchasing and indirect cost monitoring and reporting are not commonly included in PERs because the agency considers its existing performance criteria to be sufficiently broad to assess contractor performance. Though these efforts may be important to address contractor costs and information from the efforts could inform assessments of cost performance, they do not, on their own, represent DOE office’s evaluation of contractor’s cost performance. In addition, PERs are important records of DOE offices’ evaluations of contractor performance because, according to agency officials, DOE uses the PERs to inform acquisition decisions and help form the basis for a contractor’s performance record. We and the DOE Inspector General have identified how important it is for DOE to obtain quality cost information and use it to evaluate cost performance. For example, for more than a decade, we have reported that some DOE offices have experienced challenges obtaining quality information that could enable the offices to make better-informed decisions about programs’, and therefore DOE’s, budgetary needs. Furthermore, we reported in July 2012 that NNSA based much of its congressional budget request on contractor-generated budget proposals, which the agency often did not thoroughly evaluate. More recently, according to a 2017 DOE Inspector General report, challenges in evaluating cost performance have contributed to NNSA’s and its M&O contractors’ difficulty in demonstrating the anticipated cost savings for the NNSA Production Office Sites contract. DOE created this contract by consolidating the contracts for the Y-12 National Security Complex and the Pantex Plant into a single contract for the explicit purpose of saving costs. While collecting quality information on, measuring, and reporting on cost performance for M&O contracts may be challenging, this information is important for two reasons. First, the FAR, DOE policy, and CPARS highlight the importance of information on contractor’s cost performance for acquisition decision-making. As we previously noted, the FAR and DOE policy provide that decisions to extend or compete an M&O contract be based on an expectation of meaningful improvement in performance or cost, including consideration of a contractor’s technical, administrative, and cost performance. In addition, according to DOE officials, they largely copy information from PERs into the federal government database on contractors’ past performance, CPARS, which agencies use to inform their awarding of contracts. CPARS has several performance criteria that agencies are required to complete, one of which is “cost control.” This is challenging to address, according to DOE officials, because PERs do not typically include an explicit evaluation of cost performance even though, also according to DOE officials, PERs are the primary source of information entered into CPARs. Second, as we reported in 2009, there are inherent risks to the government from cost-reimbursement contracts such as DOE’s M&O contracts, particularly with cost escalation because the government is required to pay the contractor’s allowable costs regardless of whether the contractor completes the work. Because of these risks, we found that these types of contracts involve significantly more government oversight than do fixed-price contracts. This is, in part, because the agency needs to monitor contractor costs to provide a reasonable assurance that efficient methods and effective cost controls are used. As we previously noted, FAR, DOE guidance and policy, and CPARS highlight the importance of quality information on contractor’s cost performance. In addition, federal standards for internal control state that management should design control activities to achieve objectives and respond to risks, such as by clearly documenting internal control in management directives, administrative policies, or operating manuals. DOE offices have policies on contractor performance evaluation, but these do not specifically require that PERs include quality cost performance information that can be used to make an overall assessment of cost performance. By updating policies to require inclusion of quality cost performance information in PERs to enable an overall assessment of a contractor’s cost performance, DOE offices could strengthen their oversight of M&O contractor costs. For example, DOE offices could better inform acquisition decisions such as whether to extend or compete a contract, complete CPARS with greater ease, inform incentives for contractor performance, and uncover opportunities for federal cost savings. This is particularly important given that these cost- reimbursement type contracts carry risks of cost escalation. DOE Generally Awarded M&O Contractors High Ratings and Most Available Performance Incentives, Except in Cases of Significant Safety or Security Incidents In reviewing DOE’s M&O contractor performance evaluations for fiscal years 2006 through 2016, we found the results of the evaluations to generally include high performance ratings and most available performance incentives, including a median of 94 percent of available award and incentive fees. During this time frame, administrative performance sometimes had lower ratings—though these were balanced out in overall ratings by strong performance elsewhere—and some safety issues and accidents resulted in additional fee reductions outside the performance evaluation process. In fiscal years 2006 through 2016, three contractors received 50 percent or less of available award fee due to two significant incidents—a safety and security issue and a major accident. M&O Contractors Generally Received High Ratings in Fiscal Years 2006 through 2016 and more than 90 Percent of Available Performance Incentives For the 239 annual M&O contractor evaluations from the 24 DOE contract rating sites we reviewed, in fiscal years 2006 through 2016, DOE offices provided award and incentive fees equivalent to the FAR rating categories of Excellent or Very Good 94 percent of the time. Contractors at more than half of the 24 contract rating sites (17 of 24) received award and incentive fee percentages consistent with only Excellent or Very Good ratings for all fiscal years from 2006 through 2016. As discussed above, while the precise approaches for determining ratings and fees vary by DOE office, ratings and fees are directly linked in all three approaches: Fee is either determined through a formula based on ratings, or DOE offices determine ratings and fees at the same time. Differences between rating methodologies across offices and changes in performance evaluation approaches over time mean directly comparing ratings requires some caution; however, even acknowledging those differences, there is a clear trend of a high percentage of award and incentive fees awarded and high equivalent performance ratings across sites and years. From fiscal years 2006 through 2016, DOE also provided its M&O contractors with a median of 94 percent of their available award and incentive fees. See Table 3 for the results by FAR award fee rating category for each contract rating site for this period, and Table 4 for an analysis of average and median percentages of fees awarded by site. The amount of fee available, fee as a share of total contract spending, and the use of other incentives have varied across sites, yet performance results have been generally similar. Appendixes III through VIII provide additional details by DOE office. Contract rating site by DOE office Office of Energy Efficiency and Renewable Energy Savannah River Site–Environmental Management Waste Isolation Pilot Plant Office of Fossil Energy Strategic Petroleum Reserve Office Office of Nuclear Energy National Nuclear Security Administration Kansas City National Security Campus Lawrence Livermore National Laboratory Los Alamos National Laboratory Nevada National Security Site NNSA Production Office Sites Pantex Plant Y-12 National Security Complex Sandia National Laboratories Savannah River Site–National Nuclear Security Administration Thomas Jefferson National Accelerator Facility 94 in annual performance evaluation plans. Fixed fees are set at the inception of the contract and do not vary for performance. Of further note from our analysis of the extent to which contractors earned fees in fiscal years 2006 through 2016: Contractors for the 24 M&O contract rating sites that included award fees earned approximately $4.3 billion in total fees over this time. About three-quarters ($3.4 billion) of the $4.3 billion in fees were award fees and incentive fees, and the remaining amount was fixed fees. NNSA’s M&O contracts represent 68 percent of the fees paid and 55 percent of the total M&O contract spending over this period. As discussed above, DOE offices provided a median of 90 to 95 percent of available annual award fee to 18 of 24 M&O contract rating sites. However, six rating sites, all conducting work for NNSA, had median award fee percentages below 90 percent. Several NNSA sites had fixed fees in addition to award fees. When including those fixed fees, the percentage of total fee awarded rises, with median fee percentages rising above 90 percent for three of the sites. Contract rating sites rarely received less than 75 percent of available award fee. In addition to awarding contractors high percentages of available fees, DOE offices generally awarded M&O contractors most of the available award term incentives. Several DOE and contractor officials we interviewed noted that award term is perhaps the most valuable incentive from a contractor perspective because an extra year of work on the contract represents much more revenue for them than fees. SC and NNSA—the two offices with contracts that had award term—awarded 92 percent of award term years available, or 76 out of 83 possible award term years. Specifically, SC included award term in seven contracts and awarded M&O contractors with 95 percent of potential award term years, and NNSA included award term in four contracts and awarded contractors with 83 percent of potential award term years (see Table 5 below). Three of the unearned award term years are attributable to the contractor at Los Alamos National Laboratory, which also had a fourth award term year that NNSA revoked retroactively. According to NNSA officials, upon not earning an award term for the fourth time, Los Alamos’s contractor—in accordance with the terms of the contract—had all of its award terms revoked, and NNSA decided to recompete the contract. Administrative Performance Sometimes Had Lower Ratings, with Some Issues Resulting in Fee Reductions Outside of the Performance Evaluation Process Within the pattern of high overall performance ratings, ratings for administrative performance have generally been lower than ratings for technical performance, and some administrative performance issues— particularly safety issues and accidents—resulted in fee reductions outside the performance evaluation process, as noted in table 6 below. For example, since fiscal year 2013, when NNSA adopted common performance goals across its contract rating sites, about 83 percent of possible goal ratings (134 of 162) had been rated Very Good or better. Of the 28 goal ratings below Very Good, 22 (79 percent) were in administrative goals. In many cases, incidents that led to lower ratings involved site operations issues, such as in safety and security. Similarly, the contractors at the 10 SC contract rating sites and one NE contract rating site also showed generally higher technical performance ratings with 9 of 11 contract rating sites having higher average technical area scores than administrative area scores (the two other contract rating sites had average technical area scores that were about equal to the average administrative scores). From our review of DOE documents and discussions with officials, one factor that may be an important influence in the difference between technical and administrative scores at SC and NE rating sites is that the Science and Energy Lab performance evaluation approach does not incentivize administrative performance above a B+. As discussed above, contractors generally receive additional award fee for higher ratings, but under the Science and Energy Lab approach, in the administrative area, all scores of B+ and above lead to the same amount of award fee. Therefore, a contractor whose only difference was an administrative score of B+ versus A+ would receive the same amount of award fee. According to DOE officials, this structure is meant to encourage contractors to reinvest cost savings into technical performance rather than improving administrative systems that already meet expectations. Relatively low performance in certain areas can be balanced out in overall ratings by strong performance ratings elsewhere. Of nine occasions since fiscal year 2013 that an NNSA contractor received at least one Satisfactory goal rating (below 50 percent), the overall rating for the contractor remained Good or Very Good, and contractors were provided the majority of their fees in all but one case (the contractor for Los Alamos National Laboratory in fiscal year 2014, which we discuss further below). For example, following the break-in of trespassers and related security lapses at Y-12 in 2012, NNSA provided the M&O contractor with Satisfactory ratings in operations in fiscal years 2012 and 2013. However, Very Good and Excellent ratings in other areas meant NNSA provided an overall rating of Good to the contractor in those years, and the contractor received more than 50 percent of available award fees. For SC, in the five occasions since fiscal year 2006 in which a contractor received at least one goal rating of C (2.0) or below, overall area scores remained As and Bs and fees above 75 percent, except for one instance. On that occasion, Princeton Plasma Physics Laboratory in 2016 received multiple goal ratings of C, which led to a technical score of C+ and a fee of 68 percent. This 2016 rating for Princeton Plasma Physics Laboratory is also the only case from fiscal years 2006 through 2016 of a Satisfactory-level goal rating in a technical area goal, as the others were all in the administrative areas of site operations or leadership. The extent to which a single area of performance affects overall ratings is influenced by the broad scope of activities under an M&O contract, the broad types of performance required under the contract, and the weights used to determine overall ratings and incentives. According to DOE officials, one way the Science and Energy Lab approach addresses these factors is to include all the ratings provided by each stakeholder and for each objective in the PER. In this way, while a C from one stakeholder or objective may be weighted out overall, the grade and the feedback associated with it are still provided to the M&O contractor and clearly visible to readers of the reports. Another way that DOE offices have addressed individual performance deficiencies that may get balanced out in overall ratings is through additional fee reductions. Most offices have reduced fees outside the performance evaluation process to address specific performance deficiencies—generally administrative concerns, such as safety issues. In particular, all offices except EERE have reduced fees that would have been provided from performance evaluation results, relying on contract clauses that allow for fee reductions. Such clauses allow DOE offices to unilaterally reduce fees for the evaluation period if, for example, the contractor fails to meet performance requirements of the contract relating to environment, safety, and health. For example, NE used such clauses in 7 of the 11 years we reviewed to reduce the fee provided to the Idaho National Laboratory M&O contractor. FE has also frequently used fee reductions to address issues outside its predominantly objective performance criteria. SC, NNSA, and EM have also occasionally used additional fee reductions outside the performance evaluation process. For all offices, fee reductions generally resulted from administrative performance issues—safety issues and accidents—rather than technical performance. These fee reductions ranged from $10,000 to $35 million, and while the fee received by the contractor was lowered, the original ratings were not revised. In most cases, these reductions were for 10 percent or less of award and incentive fees provided and less than $1 million dollars; however, they represented large portions of contractors’ fees in a few cases. See Table 6 below for a list of fee reductions. From Fiscal Years 2006 through 2016, Three Contractors Received 50 Percent or Less of Available Award and Incentive Fees Due to Significant Safety and Security Incidents Three times in fiscal years 2006 through 2016, M&O contractors received 50 percent or less of available award and incentive fees due to a safety and security issue at the Lawrence Livermore National Laboratory (LLNL) in fiscal year 2008 and a major accident involving the WIPP in Carlsbad, New Mexico, and the Los Alamos National Laboratory (LANL) in fiscal year 2014. LLNL, 2008. LLNL’s M&O contractor received 50 percent of the available award and incentive fees—$15,795,584 out of $31,879,519—due to weaknesses in environmental management, security, and management/performance improvement that resulted in Satisfactory ratings in those respective areas and an overall Satisfactory rating in operations. In particular, an April 2008 inspection and force-on-force exercise conducted by DOE’s Office of Health, Safety, and Security found significant weaknesses in protective force and classified matter protection and control programs that led to an Unsatisfactory rating in security. The performance evaluation also reported issues with contractor assurance system progress, staffing, and “unacceptable” losses of key personnel. LLNL’s contractor received overall ratings of Outstanding in mission and Good in institutional management. In addition, the contractor received $21,862,651 in fixed fees, for a total fee award of $37,658,235. WIPP, 2014. WIPP’s contractor received 6.9 percent—$561,266 out of $8,192,895—of the fees available under its contract in fiscal year 2014 due to two unrelated accidents, a truck fire and a waste drum explosion, that resulted in the suspension of waste disposal at the site—the nation’s only facility for disposal of transuranic waste. The 6.9 percent of fees awarded represented an additional reduction of fees from the amounts the contractor earned for meeting a portion of its objective criteria targets and receiving Satisfactory ratings in all four subjective criteria. WIPP did not resume waste disposal operations until 2017. LANL, 2014. LANL’s contractor received none of the available award fee, and no DOE fixed fee, in fiscal year 2014 due to its improper oversight and packaging of the waste drum that exploded at WIPP. Of $63,406,380 in available fee, LANL’s contractor received about $6.3 million in fixed fee associated with work completed under contract with other federal agencies that, according to NNSA officials, could not be revoked. Similar to WIPP’s contractor, this represented an additional reduction of fees from the amounts that would have resulted from an overall Satisfactory rating (including an Unsatisfactory for operations and infrastructure; Satisfactory for science, technology and engineering; Satisfactory for leadership; and Very Good for the two mission goals). In addition to losing fee and award term, the waste portion of the LANL contract was withdrawn from the M&O contract and contracted out separately by EM. In all three cases, in the year following the 50 percent or less in award and incentive fees, performance ratings returned to at least Good levels and contractors received at least three-quarters of available award and incentive fees. With regard to the WIPP accident involved in two of the three cases, efforts to recover from the waste drum incident and return to full operations have cost hundreds of millions to date and are estimated to cost more than $600 million in total, all of which will be costs to the taxpayer. The combined unearned and reduced fee for both contractors amounted to $64,788,464, or about 10 percent of total estimated costs to the government. In addition to fee reductions, NNSA officials stated that the WIPP accident played a significant role in NNSA’s decision to not exercise the last 7 years of possible award term on the LANL contract and thus recompete the contract in 2018. According to NNSA, those 7 years translate into approximately $17 billion in work and up to $500 million in fee the LANL contractor could have earned. Also, with regard to additional actions EM took after the accidents at WIPP, according to DOE officials, EM modified the contract terms from having a single 5-year option period to five 1-year option periods. Conclusions While there are differences in how DOE’s offices approach performance evaluation of M&O contractors, all of the offices use the annual performance evaluations of the contractors and the associated rating and fee determinations to evaluate the extent to which contractors are operating sites as intended and accomplishing mission work, and to justify incentives such as fee and additional contract term. These annual performance evaluations also provide valuable information for contract management and acquisition decisions, such as whether to renew or compete expiring M&O contracts. DOE also recognizes the importance of improving performance evaluation and oversight of contractors. All of DOE’s offices except NNSA have clearly documented procedures on how to collect and use information to make rating determinations. NNSA provides a general framework for its performance evaluations in its NAP-4C policy but leaves how to collect, distribute, and document information to the discretion of individual offices and officials. In the past, NNSA officials have made changes to incentives awarded without underlying performance documentation to support the change. Without developing and documenting clear procedures for implementing NAP-4C that specify the process for collecting contractor performance information and how officials are to ensure this information can be traced to rating determinations, NNSA leadership does not have reasonable assurance that it is consistently evaluating contractor performance and that it is using relevant performance information as intended. The cost performance information included in DOE offices’ fiscal year 2016 PERs is of limited use for acquisition decision-making in that this information does not permit making an overall assessment of M&O contractors’ cost performance. DOE offices have not required specific assessment of cost performance in their performance evaluation policies, nor discussed how to ensure that cost information is useful for acquisition decision-making. However, the PERs are important sources of information for contract management—particularly for acquisition decisions and oversight of spending on cost-reimbursement contracts. DOE officials identified challenges in evaluating M&O contractors’ cost performance and ways this evaluation may occur outside of the annual performance evaluation process. These challenges contribute to why there is less cost performance-related information in PERs than for other types of performance. While collecting, measuring, and reporting quality cost performance information may be challenging, such information is important for fully assessing contractor performance and managing the inherent risks of cost-reimbursement contracts. By updating their policies to require quality cost performance information in PERs to enable an overall assessment of M&O contractor cost performance, the six DOE offices with M&O contracts could strengthen their oversight of costs for contracts worth about $20 billion a year and use this information to improve acquisition decision-making. Recommendations for Executive Action We are making seven recommendations to DOE offices: The Administrator for the National Nuclear Security Administration should develop and document clear procedures for implementing NAP-4C, specifying the process for collecting contractor performance information and describing how officials are to ensure this information can be traced to rating determinations. (Recommendation 1) The Assistant Secretary for the Office of Energy Efficiency and Renewable Energy should update its policy to require that Performance Evaluation Reports include quality information on cost performance to enable an overall assessment of Management and Operating contractor cost performance. (Recommendation 2) The Assistant Secretary for the Office of Environmental Management should update its policy to require that Performance Evaluation Reports include quality information on cost performance to enable an overall assessment of Management and Operating contractor cost performance. (Recommendation 3) The Assistant Secretary for the Office of Fossil Energy should update its policy to require that Performance Evaluation Reports include quality information on cost performance to enable an overall assessment of Management and Operating contractor cost performance. (Recommendation 4) The Administrator for the National Nuclear Security Administration should update its policy to require that Performance Evaluation Reports include quality information on cost performance to enable an overall assessment of Management and Operating contractor cost performance. (Recommendation 5) The Assistant Secretary for the Office of Nuclear Energy should update its policy to require that Performance Evaluation Reports include quality information on cost performance to enable an overall assessment of Management and Operating contractor cost performance. (Recommendation 6) The Director of the Office of Science should update its policy to require that Performance Evaluation Reports include quality information on cost performance to enable an overall assessment of Management and Operating contractor cost performance. (Recommendation 7) Agency Comments and Our Evaluation We provided a draft of this report to DOE for comment. DOE provided us with written comments, as well as technical comments, which we incorporated as appropriate. In its written comments, reproduced in appendix IX, DOE agreed with four of our seven recommendations and partially agreed with the others. DOE partially agreed with our recommendations that three DOE offices— EERE, NE, and SC—update their policies to require that PERs include quality information on cost performance to enable an overall assessment of M&O contractor cost performance. In its written comments, DOE said that the three offices have concerns that (1) our report gives the impression that DOE does not review cost performance of their respective national laboratories in an adequate manner, and (2) by focusing on the annual PERs, our report does not capture the cost performance reviews conducted in day-to-day contract oversight, the annual laboratory planning process, and contract extend/compete decisions. In its comments, DOE stated that since EERE, NE, and SC conduct cost performance reviews in normal operations and at the year-end annual evaluation process, adequate information is available to assess whether the contractor cost performance is acceptable to the department. In the report, we note that DOE conducts some cost performance evaluation activities outside of the annual performance evaluation process, although we did not assess these efforts. While there may be adequate information available, DOE does not commonly document this information or assessments from such activities in the PERs. We continue to believe that the PERs are important sources of information for contract management—particularly for acquisition decisions and oversight of spending on cost-reimbursement contracts—and that action is needed to improve these formal records of contractor performance. By not including quality information on overall cost performance and assessments in PERs, DOE offices are missing a valuable opportunity to better document contractors’ cost performance, improve acquisition decision-making, and strengthen oversight of billions of dollars in contracting. We continue to believe that it is important for EERE, NE, and SC to implement the recommendations and that by doing so, these offices would have better assurance that M&O performance evaluations fully address required elements. We are sending copies of this report to the appropriate congressional committees, the Secretary of Energy, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-3841 or bawdena@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix X. Appendix I: Objectives, Scope, and Methodology This report reviews the Department of Energy’s (DOE) performance management of its management and operating (M&O) contracts. Specifically, it examines (1) how DOE offices evaluated M&O contractor performance in fiscal years 2006 through 2016 and the extent to which these offices have documented their evaluation approaches; (2) the extent to which DOE’s fiscal year 2016 M&O contractor performance evaluation reports provide information on contractors’ technical, administrative, and cost performance; and (3) the results of DOE’s M&O contractor performance evaluations from fiscal years 2006 through 2016. For all three objectives, we reviewed performance evaluation documentation—performance evaluation plans, performance evaluation reports (PERs), fee determinations, award term determinations, and option term determinations—for 21 of the 22 DOE M&O contracts in place as of fiscal year 2016, the most recently completed contract year at the time we initiated our review. We also reviewed documentation for Bettis and Knolls Atomic Power Laboratories’ M&O contract but excluded it from our analysis because the contract does not have annual reviews and ratings comparable to the other DOE M&O contracts. The Bettis and Knolls contract does not have an award fee and thus NNSA’s Office of Naval Reactors—the office responsible for overseeing the M&O contract—does not produce annual PERs similar to those of the other offices. In addition, we did not include in our scope the DOE contract for the cleanup of the West Valley Demonstration Project in upstate New York because it was not an M&O contract in fiscal year 2016; according to DOE officials, it switched from being an M&O to a non-M&O contract in fiscal year 2007. In addition, we also interviewed DOE officials to gain a further understanding of the department’s performance evaluation processes and results, including officials at DOE headquarters and at several field offices that are responsible for providing day-to-day oversight of the activities of M&O contractors. To provide additional perspective, we interviewed officials at the Department of Defense, the National Aeronautics and Space Administration, and the Department of Homeland Security, which we selected because they also manage government-owned, contractor- operated laboratories and sponsor work at DOE laboratories, sometimes contributing views incorporated into DOE performance evaluations. To examine how DOE offices have evaluated M&O contractor performance, we reviewed DOE’s and DOE offices’ policies and procedures for performance evaluations, as well as annual performance evaluation and measurement plans and PERs from fiscal years 2006 through 2016. We also compared each office’s policies and procedures for conducting performance evaluations against federal standards for internal control, as well as the Federal Acquisition Regulation (FAR), DOE’s Acquisition Guide, and the Department of Energy Acquisition Regulations. In addition, to examine the extent to which these offices have documented their evaluation approaches, we discussed the evaluation approaches and processes with DOE officials and compared those approaches with documented policies and procedures. To evaluate the extent to which PERs provided information on each of the performance areas outlined in the FAR—technical, administrative, and cost—we performed a content analysis of 22 DOE fiscal year 2016 PERs for M&O contractors. We developed operationalized definitions of each of the three areas with input from DOE’s offices. Broadly, the operationalized definition of technical performance included mission- related activities, the operationalized definition of administrative performance included mission support activities, and the operationalized definition of cost performance included spending-related activities. Mission-related activities included, for example, research and development, production, storage, clean-up, and construction. Mission support activities included, for example, information technology, human resources, legal activities, environmental safety and health, property management, risk assessment, and leadership activities. Cost-related activities included, for example, spending, budgeting, strategic sourcing, and costs, including the contractor’s cost-effectiveness. In identifying information related to cost performance, we considered all evaluative statements related to cost, including broad terms such as saving, cost, spending, and budget. Then we categorized performance descriptions under these three performance areas and counted the number of performance descriptions that included information in the M&O contracts’ PERs related to each of the areas. A performance description could be categorized as related to one, two, or all three areas. Two analysts independently reviewed each PER and then met to agree on the categorizations. When differences arose, we included a third analyst to arrive at a consensus. For the vast majority of M&O contracts, we analyzed the performance descriptions at the level of objectives—where most performance descriptions were found—and included notable outcomes described under those objectives. In a few instances, we used other comparable units of analysis, such as goals, for some National Nuclear Security Administration (NNSA) M&O contracts (in which performance information was provided by goals, not objectives) and criteria for Office of Environmental Management (EM) and Office of Fossil Energy (FE) (in which performance information was provided under numerous subjective and objective criteria). Based on our analysis, we reported the total number of performance descriptions for each area, as well as the percentage of performance descriptions that contained information related to each area. Because performance criteria descriptions could contain information related to more than one area, the percentages total more than 100 percent. To determine the extent of cost performance-related information in DOE’s fiscal year 2016 PERs for its M&O contracts, we performed a content analysis. From our analysis, we reported the total number of pages the cost performance-related information represented, compared with the average number of total report pages. To determine the number of pages, we counted the number of pages of each PER. In addition, to evaluate the quality of cost performance-related information, we reviewed DOE Information Quality Guidelines, which apply to information DOE offices make available to the public. We then performed a content analysis of DOE fiscal year 2016 PERs based on the definition of quality in the guidelines, which includes that information generated for DOE and the public be useful. We further analyzed and categorized the types of cost performance-related information. Types of cost information included, for example, within budget, over budget, cost savings, cost overrun, and cost effectiveness. We defined cost effectiveness as good value for money spent. To examine the results of DOE’s M&O contractor performance evaluations from fiscal years 2006 through 2016, we analyzed performance ratings and incentives awarded in PERs, fee determination letters, and other performance evaluation documents. Throughout the report, we analyzed and provided information by “contract rating sites” rather than individual contractors or physical sites, because the individual contractors and how certain sites align with the contracts have changed over time. We analyzed 24 distinct contract rating sites covered by 21 M&O contracts in place as of fiscal year 2016. There are three more contract rating sites than the number of contracts in 2016: two additional contract rating sites because two individual contracts were consolidated into one contract during the period we covered—we analyzed the two individual contracts from prior to 2014 separately from the current consolidated contract—and one additional contract rating site because two DOE offices separately evaluated the performance of a single contractor that performed activities for each of those offices. To summarize the results of DOE’s annual contract performance evaluations, we analyzed overall annual percentages of available award and incentive fees provided at each contract rating site and presented the corresponding FAR rating categories. We reviewed performance evaluation ratings from 239 performance evaluations at the 24 contract ratings sites. We also did not include ratings from the EM portion of the Savannah River Site contract for fiscal years 2006 through 2008 because, according to EM officials and award fee documents, it had multi-year award fee targets that did not align with individual fiscal years. We conducted this performance audit from October 2016 through February 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Additional Information on the Department of Energy’s Management and Operating Contracts Table 7 provides additional information on the Department of Energy’s 22 management and operating contracts, contractors, contract award and end year, and total spending through these contracts. Table 8 presents the spending data adjusted for inflation. Appendix III: Additional Information on the Office of Energy Efficiency and Renewable Energy’s Performance Evaluations The Office of Energy Efficiency and Renewable Energy (EERE) focuses on aiding the development and implementation of renewable energy technologies and improving energy efficiency across various sectors. EERE administers its management and operating (M&O) contract at the National Renewable Energy Laboratory (NREL), in Golden, Colorado. As we describe in our report, EERE follows a Science and Energy Lab approach to evaluate its M&O contractor’s performance that uses broad, office-wide performance criteria, which are mostly subjective. Table 9 provides the full list of the goals and objectives EERE used to evaluate its M&O contractor performance in fiscal year 2016. For the most part, these performance criteria remained unchanged from fiscal year 2006 through fiscal year 2016. As we describe in our report, EERE uses detailed methodologies to determine ratings and incentives. To illustrate the detailed formulas and calculations involved, Figure 5 provides an example of how ratings and fees are calculated. Table 11 shows the rating scores the contractors earned for Mission and Operations goals. Figure 6 shows the annual total fee (both award fee and fixed fee) EERE provided to its M&O contractors for fiscal years 2006 through 2016. Table 12 provides the percentage of available award and incentive fees provided to the M&O contractors for fiscal years 2006 through 2016 by contract rating site. Appendix IV: Additional Information on the Office of Environmental Management’s Performance Evaluations The Department of Energy’s Office of Environmental Management (EM) is responsible for decontaminating and decommissioning facilities and sites that are contaminated from decades of nuclear weapons production and nuclear energy research. EM has two management & operating (M&O) contract sites: the Savannah River Site (SRS) in Aiken, South Carolina; and the Water Isolation Pilot Plant (WIPP) in Carlsbad, New Mexico. As we describe in our report, EM follows a Site Specific approach to evaluate its M&O contractors that uses detailed performance criteria specific to each contract. Under this approach, most performance criteria we reviewed are objective criteria, and a few are broader, subjective criteria. Tables 13 through 16 provide examples of some of the specific criteria EM used at each site. We provide examples rather than a full list because each site has numerous individual metrics, which are often quite technical. Specifically, Tables 13 and 14 provide examples of EM’s objective performance criteria, which are defined based on quantifiable metrics (e.g., a contractor’s demonstrated waste processing rate) and milestones (e.g., whether a contractor completed a task on or before a scheduled date). Table 13 includes 3 of the 6 objective performance criteria EM used to evaluate the SRS contractor’s performance during fiscal year 2016. Table 14 contains examples of 3 of the 9 objective criteria EM used to evaluate the WIPP contractor in fiscal year 2016. Tables 15 and 16 provide examples of EM’s subjective criteria, which are used for aspects of performance that may be difficult to capture objectively. Table 15 provides examples of 3 of the 12 subjective criteria for evaluating the SRS M&O contractor’s performance during fiscal year 2016, while Table 16 contains the fiscal year 2016 subjective criteria for evaluating the WIPP M&O contractor’s performance. The following tables and figure provide details on the incentives available to and earned by EM’s M&O contractors from fiscal year 2006 through fiscal year 2016. Table 17 shows the performance incentives that EM included in its M&O contracts. We use the term “contract rating sites” rather than individual contractors or physical sites, because the individual contractors and how certain sites align with the contracts may have changed over time. Figure 7 shows the annual total fee (both award fee and fixed fee) provided to EM M&O contractors for fiscal years 2006 through 2016. Because EM and National Nuclear Security Administration activities at the Savannah River Site are rated separately, only the EM portion of fees is represented below. Table 18 provides the percentage of available award and incentive fees EM’s M&O contractors earned for fiscal years 2006 through 2016. Appendix V: Additional Information on the Office of Fossil Energy’s Performance Evaluations The Department of Energy’s Office of Fossil Energy (FE) manages the nation’s Strategic Petroleum Reserve (SPR), which consists of salt caverns storing crude oil in Texas and Louisiana. As we describe in our report, FE follows a Site Specific approach to evaluate its M&O contractors that uses detailed performance criteria specific to each contract. Under this approach, most performance criteria we reviewed are objective criteria, and a few are broader, subjective criteria. Table 19 provides examples of FE’s objective performance criteria, which are defined based on quantifiable metrics (e.g., the contractor’s demonstrated oil drawdown rate) and performance targets (e.g., whether a contractor completed a task on or before a scheduled date). Table 19 includes 4 of the 11 objective performance criteria FE used to evaluate the Strategic Petroleum Reserve Office (SPRO) contractor’s performance during fiscal year 2016. We provide examples rather than a full list because there were numerous individual metrics, which are often quite technical. Table 20 contains the full list of FE’s subjective performance criteria— which FE uses for aspects of performance that may be difficult to capture objectively—for evaluating the SPRO M&O contractor’s performance during fiscal year 2016. Table 21 shows the performance incentives that FE included in its M&O contract. Figure 8 shows the annual total fee (both award fee and fixed fee) FE provided to its M&O contractors for fiscal years 2006 through 2016. Table 22 provides the percentage of available award and incentive fees provided to M&O contractors for fiscal years 2006 through 2016. Appendix VI: Additional Information on the National Nuclear Security Administration’s Performance Evaluations The National Nuclear Security Administration (NNSA), a separately organized agency within DOE, is responsible for maintaining and enhancing the safety, reliability, and performance of the nation’s nuclear weapons stockpile, promoting international nuclear safety and nonproliferation, and supporting U.S. leadership in science and technology. NNSA administers management and operating (M&O) contracts at eight national laboratories, plants, and sites: Bettis and Knolls Atomic Power Laboratory in West Mifflin, Pennsylvania, and Niskayuna and West Milton, New York Kansas City National Security Campus in Kansas City, Missouri Lawrence Livermore National Laboratory in Livermore, California Los Alamos National Laboratory in Los Alamos, New Mexico Nevada National Security Site near Las Vegas, Nevada NNSA Production Office Sites Pantex Plant in Amarillo, Texas Y-12 National Security Complex in Oak Ridge, Tennessee Sandia National Laboratories in Albuquerque, New Mexico Savannah River Site in Aiken, South Carolina As we describe in our report, NNSA follows an approach to evaluate its M&O contractors that uses broad, office-wide performance criteria that are mostly subjective. Table 23 provides the full list of the goals and objectives NNSA used to evaluate its M&O contractors’ performance in fiscal year 2016. While there have been some language amendments, overall, goals and objectives have remained the same from fiscal year 2013 through fiscal year 2016. As we describe in our report, under the NNSA approach, goals are assigned specific portions of the available award fee for each contract at the beginning of the fiscal year—and at the end of the fiscal year, officials determine ratings and fees at the same time in a collaborative meeting with NNSA leadership. Figure 9 provides an example of award fee amounts assigned to individual goals. Table 24 shows the performance incentives that NNSA included in its M&O contracts. We use the term “contract rating sites” rather than individual contractors or physical sites, because the individual contractors and how certain sites align with the contracts have changed over time. Specifically, NNSA consolidated its Y-12 National Security Complex and Pantex Plant contracts into the National Production Office Sites contract in fiscal year 2014, and NNSA and the Office of Environmental Management separately evaluated their respective activities carried out by the Savannah River Site contractor. Table 25 provides annual performance ratings by goal for fiscal years 2013 through 2016 for each NNSA contract rating site. Figure 10 shows the annual total fee (both award fee and fixed fee) provided to NNSA M&O contractors for fiscal years 2006 through 2016 by contract rating site. Table 26 provides the percentage of available award and incentive fees provided to M&O contractors for fiscal years 2006 through 2016 by contract rating site. Appendix VII: Additional Information on the Office of Nuclear Energy’s Performance Evaluations The Office of Nuclear Energy’s (NE) primary mission is to advance nuclear power as a resource capable of making major contributions in meeting U.S. energy supply, environmental, and energy security needs. NE administers its management and operating (M&O) contract at the Idaho National Laboratory (INL), in Idaho Falls, Idaho. As we describe in our report, NE follows a Science and Energy Lab approach to evaluate its M&O contractor that uses broad, office-wide performance criteria that are mostly subjective. Table 27 provides the full list of the goals and objectives NE used to evaluate its M&O contractor performance in fiscal year 2016. For the most part, these performance criteria have remained unchanged from fiscal year 2007 through fiscal year 2016. As discussed above, NE uses detailed methodologies to determine ratings and incentives. To illustrate the detailed formulas and calculations involved, Figure 11 provides an excerpt from a fee determination letter as an example of how ratings and fees are calculated. Table 28 shows the performance incentives that NE included in its M&O contract. Table 29 shows the rating scores the contractor earned for Mission and Operations goals. Figure 12 shows the annual total fee (both award fee and fixed fee) provided to NE’s M&O contractor for fiscal years 2006 through 2016. Table 30 provides the percentage of available award and incentive fees provided to M&O contractor for fiscal years 2006 through 2016. Appendix VIII: Additional Information on the Office of Science’s Performance Evaluations The Office of Science (SC) supports scientific research for energy and the physical sciences both by directly supporting such research, for example, through grants to and cooperative agreements with universities, and by supporting the development, construction, and operation of scientific user facilities. SC administers management and operating (M&O) contracts at 10 national laboratory sites: Ames Laboratory in Ames, Iowa Argonne National Laboratory in Argonne, Illinois Brookhaven National Laboratory in Upton, New York Fermi National Accelerator Laboratory in Batavia, Illinois Lawrence Berkeley National Laboratory in Berkeley, California Oak Ridge National Laboratory, in Oak Ridge, Tennessee Pacific Northwest National Laboratory in Richland, Washington Princeton Plasma Physics Laboratory in Princeton, New Jersey SLAC National Accelerator Laboratory in Stanford, California Thomas Jefferson National Accelerator Facility in Newport News, As we describe in our report, SC follows a Science and Energy Lab approach to evaluate its M&O contractors that uses broad, office-wide performance criteria that are mostly subjective. Table 31 provides the full list of the goals and objectives SC used to evaluate its M&O contractors’ performance in fiscal year 2016. Generally, these performance criteria remained mostly unchanged from fiscal year 2006 through fiscal year 2016. As discussed above, SC uses detailed methodologies to determine ratings and incentives. To illustrate the detailed formulas and calculations involved, Figure 13 provides excerpts from a performance evaluation report as an example of how ratings and fees are calculated. The following tables and figure provide details on the incentives available to and earned by SC’s M&O contractors from fiscal year 2006 through 2016. Table 32 shows the performance incentives that SC included in its M&O contracts. We use the term “contract rating sites” rather than individual contractors or physical sites, because the individual contractors and how certain sites align with the contracts may have changed over time. Table 33 shows the rating scores the contractor earned for the Science and Technology goals and Maintenance and Operations goals, by contract rating site. Figure 14 shows the annual total fee (both award fee and fixed fee) SC M&O contractors earned for fiscal years 2006 through 2016 by contract rating site. Table 34 provides the percentage of available award and incentive fees SC’s M&O contractors earned for fiscal years 2006 through 2016 by contract rating site. Under the award term incentive, some SC M&O contractors are able to earn one additional year of performance under the contract for each year they exceed certain thresholds in their annual performance evaluations. Table 36 shows award term results for fiscal years 2006 through 2016 by contract rating site. Appendix IX: Comments from the Department of Energy Appendix X: GAO Contact and Staff Acknowledgments GAO Contact: Staff Acknowledgements: In addition to the contact named above, Quindi Franco (Assistant Director), Ryan Gottschall (Analyst in Charge), Danny Baez, and Diantha Garms made key contributions to this report. Also contributing to this report were John Delicath, Brenna Derritt, Cindy Gilbert, Timothy Guinane, Rich Johnson, Danny Royer, Kiki Theodoropoulos, and Tatiana Winger.
target: In fiscal years 2006 through 2016, the federal government spent almost $193 billion on DOE's M&O contracts—a form of contract that traces its origins to the Manhattan Project. Six DOE offices use M&O contracts to manage and operate federally owned sites that perform work to fulfill DOE's diverse missions, such as conducting scientific research and maintaining nuclear weapons. GAO was asked to review DOE's performance management of its M&O contracts. This report examines, among other things, (1) how DOE offices evaluated M&O contractor performance in fiscal years 2006 through 2016; (2) the extent to which DOE's fiscal year 2016 M&O contractor PERs provide information on contractors' technical, administrative, and cost performance; and (3) the results of DOE's M&O contractor performance evaluations for fiscal years 2006 through 2016. GAO reviewed performance evaluation documents for 21 of the 22 DOE M&O contracts; analyzed DOE policies, procedures, and guidelines, and federal regulations; analyzed technical, administrative, and cost aspects of M&O contracts' 2016 PERs; and interviewed DOE officials. In fiscal years 2006 through 2016, six offices within the Department of Energy (DOE) generally used one of three different approaches to evaluate management and operating (M&O) contractor performance. Although these approaches varied in the performance criteria and methodologies used for determining contractor ratings and incentives, all the offices annually set expectations for contractors and assessed performance. In analyzing DOE's fiscal year 2016 Performance Evaluation Reports (PER), GAO found that these reports provided less information on M&O contractors' cost performance than on contractors' technical and administrative performance. The cost information provided in the PERs often was not detailed, did not indicate the significance of the performance being described, and applied only to specific activities. Further, the information is of limited use for acquisition decision-making, such as deciding whether to extend the length of a contract, because it does not permit an overall assessment of cost performance. A key reason PERs did not include more cost performance information is that the DOE offices' policies do not require specific assessments of cost performance or discuss how to ensure cost information is useful for future acquisition decision-making. By updating policies to require inclusion of quality cost performance information in PERs, DOE offices could better assess M&O contractors' costs, improve acquisition decision-making, and ensure performance evaluations fully address required elements. Based on GAO's review of DOE M&O contractor performance evaluations from fiscal years 2006 through 2016, DOE generally provided high performance ratings and more than 90 percent of available performance incentives (see figure). Ratings for some areas of contractor performance, as well as ratings for contractor performance at specific DOE sites, varied from this trend. For example, three times during this period contractors received 50 percent or less of available award and incentive fees due to a major accident and safety and security issues.