Nuclear Waste Cleanup: DOE Could Improve Program and Project Management by Better Classifying Work and Following Leading Practices
Released: 2019-03-05T00:00:00

Letter

Background

EM’s cleanup sites and areas of cleanup work,

EM’s status as a program, the history of EM’s requirements for operations activities, and key EM offices and DOE oversight bodies for EM’s cleanup work.

EM Cleanup Sites and Areas of Cleanup Work

EM has a headquarters office and 16 sites at which the agency oversees cleanup work. Figure 1 shows the EM sites where cleanup work remains.

EM divides its cleanup work into six work areas. These areas, described below, sometimes include both operations activities and capital asset projects: 1. spent nuclear fuel stabilization and disposition, including safe shipping, receipt, storage, and disposition of spent nuclear fuel and heavy water; 2. nuclear materials stabilization and disposition, including the management, disposition, safe surveillance, and maintenance of nuclear materials; 3. radioactive liquid waste stabilization and disposition, including treatment, management, and permanent disposal of radioactive liquid waste stored in storage tanks; 4. nuclear facility decontamination and decommissioning, including the deactivation, decontamination, and decommissioning of EM-owned nuclear, radioactive, and industrial buildings and structures; 5. solid waste stabilization and disposition, including receipt, treatment, storage, and disposal of legacy and newly generated low-level waste, mixed low-level waste, transuranic waste, hazardous waste, and sanitary waste; and 6. soil and water remediation, including cleanup of waste regulated under the Resource Conservation and Recovery Act and the Comprehensive Environmental Response, Compensation, and Liability Act.

EM’s Status as a Program

EM refers to itself as a program, and EM’s organization and mission fit PMI’s definition of a program. According to PMI, programs include multiple program components, such as sub-programs (in EM’s case, each cleanup site is a sub-program) and projects (in EM’s case, the cleanup work at each site), which are interrelated and managed in a coordinated way to obtain benefits not available from managing them individually. According to PMI officials, organizations often use the terms “program” and “project” interchangeably, but the two terms have different meanings and apply to different levels of management. Programs are a means of executing a strategy and achieving organizational goals and objectives. A program may continue indefinitely. In contrast, a project is a temporary endeavor undertaken to create a unique product, service, or result. Projects are executed to improve the efficient implementation of a program. The relationship between a program and a project is illustrated in figure 2 below.

History of EM’s Requirements for Operations Activities

In June 2009, EM developed the category of work that EM calls operations activities to differentiate this work from capital asset projects. Until then, EM managed all of its cleanup work as projects under Order 413.3B. EM documentation from that time explained that EM decided to differentiate its cleanup work so that it could quickly make use of an infusion of $6 billion for EM under the American Recovery and Reinvestment Act of 2009 (Recovery Act). EM officials stated that EM could not use the funds quickly at that time if the work had to follow the project management requirements in Order 413.3B.

In 2010, shortly after the initiation of the Recovery Act work, EM decided to make the approach of managing part of its work as operations activities permanent. EM officials could not provide any documentation from the time supporting this decision, which was not consistent with EM findings from 2009. In particular, according to EM documentation from 2009, executing all cleanup work under Order 413.3B had served EM well in defining and controlling the technical scope, project and life-cycle costs, completion dates, and risks of its cleanup work, and had helped EM improve its overall performance and become more efficient. EM began managing operations activities based on a memorandum developed by EM leadership.

In 2012, EM developed the operations activities protocol, which superseded the 2010 memorandum for managing operations activities. This protocol stated that although operations activities are not subject to DOE’s Order 413.3B requirements, EM will apply the appropriate project management principles from this order using a “graded approach.” We reviewed the 2012 operations activities protocol in October 2012 and found that it contained less stringent requirements for operation activities than Order 413.3B for capital asset projects. We also found that EM did not have a clear classification policy that set out under what conditions EM should consider particular cleanup work to be an operations activity or a capital asset project. In the absence of such a policy, EM classified as operations activities certain cleanup work that DOE’s Office of Project Management considered to be capital asset projects. We recommended that EM provide DOE’s Office of Project Management with information on EM’s classification decisions. In 2012, DOE agreed with our recommendation, and EM officials stated in August 2018 that they are developing guidance.

In July 2017, EM developed a cleanup policy that applies to both operations activities and capital asset projects. For managing capital asset projects, this policy supplements Order 413.3B. For managing operations activities, this policy supersedes the 2012 operations activities protocol. The 2017 cleanup policy states that EM will apply DOE’s project management principles described in Order 413.3B to its operations activities in a tailored way.

At the time of our review, EM had developed 11 standard operating policies and procedures that are associated with the 2017 cleanup policy and that provide guidance on areas such as program performance reporting, assessing contractors’ performance against contract requirements, and what officials have approval authority at major steps in the contract process. However, according to EM officials, the standard operating policies and procedures are not requirements.

Key EM Offices and DOE Oversight Bodies for EM’s Cleanup Work

The EM program is executed by two main components: EM headquarters, which serves as the program manager for the EM program, and 16 cleanup sites, which serve as sub-programs. The following EM headquarters and site officials are key to managing and overseeing EM’s operations activities, according to the 2017 cleanup policy:

The Assistant Secretary for Environmental Management serves as the head of EM and is responsible for the execution of EM’s mission. In December 2017, the Assistant Secretary for EM began reporting to the DOE Undersecretary of Science, who in turn reports to the DOE Deputy Secretary of Energy. The Assistant Secretary for Environmental Management, among other things, provides leadership and develops mission strategies, policy, and guidance for the EM cleanup program.

The Principal Deputy Assistant Secretary for Environmental Management serves as the EM management official responsible for operations, including coordination, oversight, and leadership on scope, cost, and schedule elements. Under the 2017 cleanup policy, this official has approval authority for contracts equal to or greater than $200 million. This official is also responsible for conducting periodic contract reviews for contracts with a total estimated cost equal to or greater than $200 million.

The Associate Principal Deputy Assistant Secretary for Field Operations provides leadership and develops mission strategies, policy, and guidance for site operations. This official is responsible for, among other things, meeting monthly with each site individually to discuss the status of cleanup work there.

The EM Deputy Assistant Secretary for Acquisition and Project Management is responsible for providing independent oversight and reports to the Associate Principal Deputy Assistant Secretary for Corporate Services. Under the 2017 cleanup policy, this official is responsible for programmatic peer reviews that review cleanup activities at each site. This official is also responsible for the implementation of Order 413.3B and review of capital asset projects.

At each of the 16 cleanup sites, the EM site manager is responsible and accountable for management and integration of all EM site-level activities. Under the 2017 cleanup policy, site managers have approval authority over contracts under $200 million. The site manager is also required to conduct periodic contract reviews for contracts with a total estimated cost of less than $200 million.

Outside of EM, two DOE bodies play a role in the oversight of EM’s capital asset projects, but not of operations activities:

DOE’s Office of Project Management has served as DOE’s enterprise project management organization since July 2015, when the Secretary of Energy gave it this responsibility as part of an initiative to improve DOE’s program and project management. As such, DOE states that this office—as an enterprise project management organization—is responsible for providing leadership and assistance in developing and implementing DOE-wide policies, procedures, programs, and management systems pertaining to project management, as well as for independently monitoring, assessing, and reporting on project execution performance. Officials from this office are experts in project management, especially as it relates to capital asset projects, and oversee the implementation of DOE’s Order 413.3B. This office also validates project performance baselines— scope, cost, and schedule—for the department’s capital asset projects, including EM’s.

The Project Management Risk Committee reviews and provides advice on capital asset projects with a total project cost of $100 million or more. The Risk Committee’s purpose is to assess the risks associated with projects across DOE and advise DOE senior leaders on project management, including on cost, schedule, and technical issues. The committee includes nine senior DOE officials from across the department, including top project management officials from the National Nuclear Security Administration, the Office of Science, and EM.

DOE’s EM Program Manages Most of Its Multibillion-Dollar Cleanup Work as Operations Activities, Posing Cost and Schedule Risks

DOE’s EM program manages most of its cleanup work as operations activities, posing cost and schedule risks. These risks stem from EM’s management of such work using less stringent requirements than for capital asset projects even though EM spends billions of dollars annually on operations activities. Site managers have the discretion to classify cleanup work as operations activities, even if the work has characteristics of capital asset projects, because DOE and EM have not established requirements for classifying EM’s cleanup work. In addition, EM has not addressed concerns raised by DOE project management experts that some operations activities should be classified as capital asset projects.

DOE’s EM Program Manages Most of Its Cleanup Work as Operations Activities, under Less Stringent Requirements Than Capital Asset Projects

EM manages its cleanup work under different requirements, depending on whether it classifies the work as a capital asset project or an operations activity, with operations activities having less stringent requirements. EM currently manages most of its work as operations activities. EM’s work is divided into 77 operations activities and 20 capital asset projects. In the fiscal year 2019 budget, operations activities accounted for 77 percent of EM’s approximately $7.2 billion budget— about $5.5 billion—while capital asset projects accounted for 18 percent of EM’s budget—about $1.3 billion. Figure 3 illustrates how EM classified and funded its work during fiscal year 2019.

For capital asset projects, EM manages the work in accordance with the requirements in DOE’s Order 413.3B, which is DOE’s project management order. This order contains numerous, detailed requirements that describe the steps and project management best practices to follow throughout the life of a project. The DOE Secretary strengthened this order in May 2016 by adding more stringent requirements, based in part on our prior recommendations. Examples of the requirements included in this order include:  A capital asset project with a total project cost over $50 million must undergo rigorous reviews outside the project’s management line. Different types of reviews are to be conducted by an independent body within the program for capital asset projects over $50 million, DOE’s Office of Project Management and the Project Management Risk Committee for capital asset projects over $100 million, and the Energy Systems Acquisition Advisory Board for capital asset projects over $750 million. Review and approval are to be received from the Under Secretary for capital asset projects over $100 million, and the Deputy Secretary for capital asset projects over $750 million.

A capital asset project must complete its original scope of work within 110 percent of the original cost baseline to be considered successful.

The program must conduct a root cause analysis to determine the underlying contributing causes of cost overruns, schedule delays, and performance shortcomings, if the program, the project manager or independent oversight offices realize a capital asset project can no longer meet its established scope, cost or schedule baseline.

Contingency to cover potential risks that might appear during the life of a project must be included as part of the total project cost estimate included in the performance baseline.

All cost and schedule estimates developed during the life of the project must follow GAO best practices.

For operations activities, EM follows the requirements in its 2017 cleanup policy, which has fewer, less detailed, and less stringent requirements than Order 413.3B. For example, in contrast to the more stringent requirements in Order 413.3B, under EM’s 2017 cleanup policy:

The highest level of review an operations activity must receive is by EM’s top management for contracts equal to or greater than $200 million.

For an operations activity to be considered successful, it must be completed within 110 percent of the current cost and scope baseline—not the original baseline established at the beginning of cleanup work.

There is no requirement to conduct a root cause analysis for operations activities.

EM does not fund contingency for operations activities.

Cost and schedule estimates made before EM authorizes execution of a contract are to follow GAO best practices, but the policy does not include a requirement to follow best practices for cost estimates developed during contract execution. Figure 4 below illustrates how operations activities are managed under less stringent requirements than capital asset projects.

EM project management officials in charge of developing the 2017 cleanup policy stated that EM intentionally wrote this policy at a high level because EM planned to develop standard operating policies and procedures that would establish more detailed steps to implement the policy. As noted earlier, these standard operating policies and procedures provide guidance but are not requirements.

DOE and EM Have Not Established Requirements for Classifying EM’s Cleanup Work or Addressed Concerns That Some Operations Activities Should Be Capital Asset Projects

Neither DOE nor EM has a policy on how to classify cleanup work as either operations activities or capital asset projects. According to DOE Office of Project Management officials, DOE does not have a department- wide policy on how to classify cleanup work. Instead, these officials stated that DOE’s general management approach is to let its individual programs, such as EM, decide how to classify their work. EM officials explained that EM allows each site manager to determine independently how to classify cleanup work because according to EM’s 2017 cleanup policy, the site manager is responsible and accountable for the planning and execution of all site-level activities.

DOE project management experts on the Project Management Risk Committee and in DOE’s Office of Project Management have raised concerns related to EM’s 2017 cleanup policy and the classification of cleanup work since 2015. These officials have stated that some current operations activities should be classified as capital asset projects. Specifically: In November 2015, EM approached DOE’s Project Management Risk Committee with a proposal for a new cleanup policy, which later became EM’s 2017 cleanup policy. In comments on the proposal, the committee’s members expressed concerns that the proposed policy did not address how EM would classify cleanup work, noting that if programs or sites get to decide on what is a capital asset project and what is not—which in turn drives the level of DOE oversight—then this approach was not an appropriate governance model. The committee’s members also questioned why EM chose not to use the already available requirements in Order 413.3B. EM did not respond to the committee’s concerns. Instead, according to the committee’s meeting minutes, the DOE Undersecretary for Management and Performance, who at the time oversaw EM, informed the committee in November 2015 that EM was proceeding with drafting its new cleanup policy.

In late 2016, DOE’s Office of Project Management officials drafted an appendix to Order 413.3B that sought to define operations activities and capital asset projects. Under the classification proposal in the draft appendix, some of the work now classified as operations activities would have become capital asset projects and subject to more stringent requirements. For example, under the appendix, the cleanup of radioactive liquid waste tanks and solid waste exhumation and disposition would have been designated as capital asset projects. However, EM officials informed officials from the DOE Office of Project Management that EM would continue to develop its own policy, which it issued in July 2017. This 2017 cleanup policy did not reclassify any of the operations activities that, in the opinion of DOE’s Office of Project Management, should be capital asset projects.

Officials from DOE’s Office of Project Management we interviewed said that continuing to classify and manage most of EM’s cleanup work as operations activities poses significant risks to DOE. According to these officials, managing the work this way poses cost and schedule risks for the following reasons, among others:

Because the review of operations activities is conducted entirely within EM, DOE does not have information on how EM manages operations activities and cannot hold EM accountable for cost- effective and timely completion of this cleanup work, which represents a $5.5 billion investment by taxpayers in operations activities in fiscal year 2019 (see fig. 3).

Operations activities are not required to go through a thorough upfront planning process to determine the scope of work to be completed. Therefore, these activities are more subject to scope creep, cost overruns, and schedule delays, which can detract from EM’s credibility with Congress and other stakeholders.

Because EM does not set aside contingency funds to cover risks for its operations activities—a project management best practice and requirement under Order 413.3B—if risks are realized, EM must either reduce or delay scope to later years, which increases costs, causes schedule delays, and undermines EM’s ability to budget for activities across the EM program.

Officials from DOE’s Office of Project Management stated that EM did not respond to their concerns that EM’s approach to classification of cleanup work poses unwarranted cost and schedule risks. Officials in EM told us they view the role of DOE’s Office of Project Management and the Project Management Risk Committee as limited to reviewing Order 413.3B requirements and overseeing capital asset projects. However, since July 2015, DOE’s Office of Project Management has served as DOE’s enterprise project management organization, with department-wide responsibilities for overseeing project management. As previously noted, DOE states that this office is responsible for, among other things, independently monitoring, assessing, and reporting on project execution performance. Therefore, review of classification of cleanup work that constitutes projects is within the scope of the office’s responsibilities. Until EM works together with DOE’s Office of Project management to (1) establish requirements for classifying cleanup work as capital asset projects or operations activities and (2) assess EM’s ongoing operations activities to determine if they should be reclassified as capital asset projects based on the newly established requirements, the department may incur more project management risk of cost increases and schedule delays than it should for hundreds of billions of dollars of remaining work.

EM’s Cleanup Policy Does Not Follow Most Selected Program and Project Management Leading Practices

EM’s 2017 cleanup policy, which governs the EM program and its operations activities, does not follow most selected leading practices for program and project management. More specifically, EM’s 2017 cleanup policy does not follow any of 9 selected program management leading practices related to scope, cost, schedule performance, and independent reviews. Further, EM’s 2017 cleanup policy follows 3 of 12 selected project management leading practices related to these areas; it does not follow the remaining 9. Figure 5 shows the percentage of selected program and project management leading practices that DOE’s Office of Environmental Management’s 2017 cleanup policy follows.

EM’s Cleanup Policy Does Not Follow Any of Nine Selected Leading Program Management Practices

EM’s 2017 cleanup policy does not follow (i.e., does not meet, minimally meets, or partially meets) the nine leading practices for program management related to scope, cost, schedule performance, and independent reviews that we selected based on PMI’s standards. More specifically, the policy partially met two of the leading practices, minimally met four others, and did not meet three, as discussed below:

Having a program management plan and a roadmap that are updated regularly. (Minimally meets.) EM’s policy does not require an overarching program management plan or strategic plan that encompasses the work at all sites. The policy does require that each site maintain a life-cycle baseline based on the scope, cost, and schedule of work, which are components of a program management plan. However, the requirement is specific to each site and not the entire EM program.

Having a reliable, integrated, comprehensive life-cycle cost estimate that is updated on a regular basis. (Partially meets.) EM’s policy requires an integrated life-cycle cost estimate for the entire EM program but does not state that the cost estimate must be reliable or updated on a regular basis.

Having a reliable, integrated master schedule that is updated on a regular basis. (Does not meet.) EM’s policy does not require an integrated master schedule at the program level.

Measuring performance against both a program’s life-cycle cost and integrated master schedule baselines. (Does not meet.) EM’s policy does not require that EM track and monitor all high-level program components against a program’s life-cycle cost and integrated master schedule baselines for the entire EM program.

Completing performance reporting and analysis in a way that provides a clear picture of program performance. (Minimally meets.) EM’s policy requires performance reporting to the EM headquarters management level, but it does not require that performance information be analyzed to give a clear picture of program performance.

Having a lessons learned database. (Partially meets.) EM’s policy requires that EM collect and disseminate lessons learned, but the policy does not specify a framework, such as a database, for how the lessons learned should be collected and shared.

Conducting program risk management throughout the life of the program. (Does not meet.) EM’s policy does not require EM to conduct risk management throughout the life of the program.

Monitoring and controlling the program, including conducting root cause analyses and developing corrective action plans. (Minimally meets.) EM’s policy does not have any requirements related to monitoring and controlling activities at a program level when there is evidence that the program’s cost or schedule baseline will not be met. It does require some monitoring and controlling activities at the site level.

Having an independent oversight body that conducts periodic reviews of the progress of the program in delivering its expected benefits. (Minimally meets.) EM’s policy does not require any independent entity outside EM to review the performance of the EM program as a whole in delivering its expected benefits. The policy requires EM’s Office of Project Management to conduct a periodic Programmatic Peer Review of cleanup work at each site, but this review is not independent of EM.

EM officials stated that even though EM’s policy does not follow these program management leading practices, EM officials may take some actions that address these leading practices. For example, to address the leading practice of having a lessons learned database, EM officials explained that EM’s Office of Project Management generates and distributes across EM a monthly lessons-learned bulletin on a topic of its choosing, and these lessons learned are uploaded on a site accessible to everyone within EM. They also explained that officials across EM could enter lessons learned in a DOE-wide lessons-learned database managed by DOE’s Office of Environment, Health, Safety, and Security. In addition, to address the leading practice of monitoring and controlling the program, including conducting root cause analyses and developing corrective action plans, the new Assistant Secretary for Environmental Management requested the development of a root cause analysis and a corrective action plan for the EM program in August 2018. To address the Assistant Secretary’s request, EM officials stated that in November 2018 they identified nine improvement areas for the EM program, for which they are developing corrective measures.

However, when we reviewed the actions EM officials cited they took to address the selected leading practices, we found that they fell short of following leading practices. For example, the lessons learned listed in the bulletins we reviewed were related only to capital asset projects, and the database cited by EM officials is not used often by EM; it contains a total of six entries on EM-related issues from 2005 to 2017. In addition, EM officials stated they do not apply key practices that can be used to identify and apply lessons learned. Further, EM officials in charge of developing a root cause analysis and a corrective action plan stated that EM does not have a process for doing so and that EM has not prepared such an analysis or plan since 2011. They also stated that EM does not intend to publish this document and that EM will not develop a root cause analysis to show the problems these corrective measures are supposed to address.

The selected leading practices help ensure that a program achieves its goals and intended benefits and that it optimizes scope, cost, and schedule performance, and independent review of performance. Without documenting such leading practices in policy, EM officials may not be aware of expectations to carry them out and may not do so consistently. Under federal standards for internal control, management should design control activities to achieve objectives and respond to risks, such as by clearly documenting internal control in management directives, administrative policies, or operating manuals. Furthermore, these standards state that management periodically reviews policies, procedures, and related control activities for continued relevance and effectiveness in achieving the entity’s objectives or addressing related risks. Until EM reviews and revises its cleanup policy to include program management leading practices related to scope, cost, schedule performance, and independent review, the EM program is at risk of continued uncontrolled changes to the program’s scope, exceeding its cost estimate and schedule, failing to meet its programmatic goals, and increasing DOE’s environmental liabilities.

EM’s Cleanup Policy Does Not Follow Most Selected Project Management Leading Practices

EM’s 2017 cleanup policy, which applies to operations activities, follows (i.e., substantially or fully meets) 3 and does not follow (i.e., does not meet, minimally meets, or partially meets) 9 of the 12 leading practices for project management related to scope, cost, schedule performance, and independent reviews that we selected based on PMI’s standards. Specifically, the policy follows these three selected leading practices:

Establishing a performance baseline and tracking it from the beginning to the end of the project. (Substantially meets.) EM’s policy requires that a contractor must establish a cost baseline and complete key performance measures within 110 percent of the approved, current cost baseline. The policy also requires that managers in charge of the work be responsible for successfully executing work within the approved performance baseline.

Conducting monitoring and controlling activities to measure performance at regular intervals. (Fully meets.) EM’s policy requires periodic project reviews from various levels, from the federal cleanup director in charge of the operations activity and site manager, all the way to EM senior leadership.

Using an EVM system that is independently certified and continuously monitored to assess project performance. (Substantially meets.) EM’s policy requires the implementation at the contract level of a work control system, either an EVM system or an approved alternative. EM guidance suggests that the EVM system be surveilled regularly, although EM does not require the EVM system to be independently certified.

The policy did not follow the other 9 selected project management leading practices; specifically, it partially met 5, while the remaining 4 were minimally or not met, as explained below:

Establishing a project execution plan with policies and procedures to manage and control project planning. (Does not meet.) EM’s policy does not require a plan to establish policies and procedures to manage and control project planning.

Clearly and completely defining the scope of a project so that its performance can be measured. (Partially meets.) EM’s policy requires that the scope be defined for a segment—typically a 5- to 10- year contract—at the beginning of the work. However, EM’s policy also states that the segment’s scope may be reduced to free up funding to cover risks. When risks occur and the scope is reduced, the segment’s performance may not be accurately and fully measured.

Developing a cost estimate using GAO best practices. (Partially meets.) EM’s policy requires that EM follow our best practices for cost estimating prior to starting the execution of a segment. However, once the contractor begins executing the segment, the policy does not require EM to follow our best practices, even when independent cost estimates are developed during a baseline change process.

Developing and maintaining an integrated master schedule using GAO best practices. (Minimally meets.) EM’s policy requires that the contract specify the schedule for the segment, which could be an input to an overall integrated master schedule for that segment. The policy does not require that an integrated master schedule be developed and maintained in accordance with GAO best practices.

Conducting risk assessments throughout the life cycle of the project; prioritizing risks in a risk register; developing risk mitigation strategies; and determining the appropriate amount of contingency. (Minimally meets.) EM’s policy does not require a risk management plan for projects. In addition, the policy states that EM will not fund contingency to cover risks that may occur for operations activities.

Capturing lessons learned throughout the continuum of a project in a database and disseminating them among projects. (Partially meets.) EM’s policy requires the EM Deputy Assistant Secretary for Acquisition and Project Management to collect and disseminate lessons learned, but the policy does not specify that this process should be done throughout the continuum of a project or that lessons learned should be disseminated among operations activities.

Developing a root cause analysis and corrective action plan to identify and address the underlying causes of cost overruns, schedule delays, and performance shortcomings when a cost or schedule overrun occurs. (Does not meet.) The policy does not contain any information on the steps that EM will take, such as developing a root cause analysis and corrective action plan, once management becomes aware that a cost or schedule overrun is probable for an operations activity.

Conducting a variety of independent reviews throughout the life of a project, including at key decision points, and on multiple aspects of the project, such as the mission need, cost, earned- value management system, and baseline review. (Partially meets.) EM’s policy requires reviews of segments conducted or organized by EM’s Office of Project Management. However, there are no requirements for any independent reviews conducted by DOE offices or other entities outside EM.

Establishing project-reporting systems/databases to provide a clear picture of project performance to management and to keep the contractor accountable. (Partially meets.) EM’s policy established a requirement that performance information be reported in the Integrated Planning, Accountability, and Budgeting System database for each operations activity. However, EM’s policy does not address how this performance information will provide a clear picture of performance and how it will be used to keep the contractor accountable.

Our findings on the inclusion of project management leading practices in EM’s 2017 cleanup policy are consistent with concerns raised by DOE’s Project Management Risk Committee. According to meeting minutes from December 2015, the committee expressed concerns that EM’s proposed cleanup policy (adopted in July 2017) appeared to run counter to the Secretary’s initiative to apply best practices to oversight of project management. In committee meeting minutes from November 2015, the committee expressed concern with the level of rigor that would be applied to independent cost analysis, project reviews, general oversight, and risk mitigation under the new cleanup policy. According to PMI, effective project management is key to implementing an organization’s strategy, and has a dramatic impact on the bottom line; organizations that invest in proven project management practices—such as these selected leading practices—continue to experience greater success than their underperforming counterparts.

In addition, under federal standards for internal control, management periodically reviews policies, procedures, and related control activities for continued relevance and effectiveness in achieving the entity’s objectives or addressing related risks. Until EM reviews and revises its policy to include project management leading practices related to scope, cost, and schedule performance, and independent reviews, EM’s operations activities are at risk of scope creep or uncontrolled changes to scope, exceeding their initial budget and schedule, and failing to meet their goals.

EM’s Performance Measures for Operations Activities Do Not Provide a Clear Picture of Overall Performance

EM uses three tools to measure the overall performance of operations activities, but these tools do not provide a clear picture of overall performance. These tools are earned value management, performance metrics, and milestones, according to EM documentation and officials. However, EM has not followed best practices for its contractors’ EVM systems; EM’s performance metrics do not link performance to cost; and EM postpones milestones when they are at risk of missing them and does not consistently track or report those milestone changes over time. Figure 6 summarizes our findings on these three performance measures and how they affect EM’s ability to effectively manage the cleanup effort.

EM Relies on Three Tools to Measure Performance of Its Operations Activities

To measure the overall performance of its operations activities, EM relies primarily on EVM data, supplemented by program-wide performance metrics and cleanup milestones, according to EM documentation and officials. EVM is a management tool used to measure the value of work accomplished in a given period and compare it with the planned value of work scheduled for the same period and with the actual cost of the work accomplished. EVM data can alert project managers to potential problems sooner than expenditures alone can. The use of EVM as a management tool is considered a best practice for conducting cost and schedule performance analysis for projects.

EM’s 2017 cleanup policy requires that contractors use an EVM system or an approved alternative for monitoring and controlling work at the contract level. We reviewed all 20 EM contracts covering operations activities and found that EM requires its contractors to maintain EVM systems for 17 of all 20 contracts. EM paid contractors for maintaining these systems and providing EVM reports to EM. For example, EM has paid one contractor $1 million annually to maintain its EVM system, and EM has paid contractors anywhere from $10,000 to $235,000 annually to receive their EVM reports, according to EM responses to our information request.

EVM by itself may not be sufficient to measure the progress of operations activities, according to EM’s 2012 operations activities protocol. The second tool EM uses to measure performance is performance metrics. EM developed 17 program-wide performance metrics for its cleanup work. The goal of these metrics is to measure progress toward completing the scope of work for the contract and the entire life of an operations activity. EM headquarters collects information from the sites monthly to measure how each activity has performed against a goal set at the beginning of each year. Examples of EM’s performance metrics include the number of cleanup sites being eliminated, the cubic meters of transuranic waste being disposed of, the number of containers of high-level waste packaged for final disposition, and the number of closed radioactive liquid waste tanks.

The EM cleanup sites set targets for these metrics annually. According to EM officials, many operations activities have one or more of these performance metrics associated with them, but some do not. Appendix II contains the full list of EM’s performance metrics.

The third tool EM uses to measure performance are cleanup milestones. Cleanup milestones represent deadlines for various cleanup-related activities derived from agreements DOE enters into with its regulators, including the Environmental Protection Agency and states. There are many different types of milestones, including enforceable and planning milestones. Generally, an enforceable milestone has a fixed, mandatory due date that is subject to the availability of appropriated funds while a planning milestone is not enforceable and usually represents a placeholder for shorter term work.

EM collects program-wide performance information from the three performance measures tools in a centralized database known as the Integrated Planning, Accountability, and Budgeting System. These performance data are used by EM to manage its program and to provide information to DOE management, Congress, and other stakeholders. According to DOE’s Office of Inspector General and EM officials, this database was developed as a program management tool to provide information to

EM headquarters officials, to ensure effective overall program performance;

DOE’s Chief Financial Officer, for inclusion in DOE-wide reports;

Congress and taxpayers, to identify the remaining environmental cleanup liability and to provide transparency regarding contractor performance; and stakeholders, to make sure the work reported is accurate, timely, complete, and in accordance with agreements.

EM Has Not Ensured That EVM Systems Are Comprehensive, Provide Reliable Data, or Are Used by Leadership for Decision-Making

EM relies on contractors’ EVM systems to measure the performance of its contractors’ operations activities, but EM has not followed (i.e., has not met, has minimally met, or has partially met) best practices to ensure that these systems are (1) comprehensive, (2) provide reliable data, and (3) are used by EM leadership for decision-making—which are the three characteristics of a reliable EVM system. Moreover, EM has allowed the contractors to categorize a large portion of their work in a way that limits the usefulness of the EVM data.

EM Has Not Followed Best Practices for Its Contractors’ EVM Systems

Our analysis of EM contractors’ EVM systems for operations activities found that EM has not followed (i.e., has not met, has minimally met, or has partially met) best practices, as discussed below. As a result, EM has not ensured that these systems are: (1) comprehensive, (2) provide reliable data, and (3) used by EM leadership for decision-making—which are the three characteristics of a reliable EVM system. (See app. III for more specific information on EM’s performance on each best practice considered and app. IV for information on how each contract followed each best practice.)

Comprehensive: Best practices to ensure EVM systems are comprehensive are: (1) requiring the contractor’s EVM systems be certified to meet guidelines established by the Earned Value Management Systems EIA-748-D Intent Guide; (2) conducting an integrated baseline review to ensure that all work is accurately captured in the performance measurement baseline; and (3) performing regular surveillance to ensure the contractors continue to maintain their EVM systems in a way to meet the EIA-748-D guidelines. We found that 17 out of 20 contractors’ EVM systems were certified to be compliant with the EIA-748-D guidelines, but of these 17, 4 contractors had self-certified their EVM systems. However, only about half of the EVM systems met the best practices for conducting integrated baseline reviews and performing ongoing surveillance. Among those, many of the reviews were not rigorous enough to ensure that the performance measurement baseline captured all of the work. In November 2017, EM issued a standard operating policy and procedure, which suggests that EVM systems be surveilled regularly. However, we discovered that EM officials were not performing thorough surveillance reviews to ensure that EVM systems were in alignment with the EIA-748-D guidelines and that the data being reported by the EVM systems were reliable.

Provide reliable data: Best practices to ensure that the contractors’ EVM systems provide reliable data are (1) the EVM data do not contain any anomalies and (2) estimates at completion— the expected total cost of completing all work based on the contractor’s performance to date—are realistic. The EVM data for contracts covering operations activities contained numerous, unexplained anomalies in all the months we reviewed, including missing or negative values for some of the completed work to date. Negative values should occur rarely, if ever, in EVM reporting because they imply the undoing of previously scheduled or performed work. In addition, we found problems with the estimate at completion listed in all 20 contractors’ EVM systems. More specifically, we found (1) many instances where the actual costs exceeded the estimates at completion even though there was still a lot of work remaining; (2) several occasions where the estimates at completion were less than half of the original budget at the beginning of the project; and (3) several contractors reported estimates at completion of zero dollars when their original budgets were for hundreds of millions of dollars. These problems indicated that the EVM systems were not being updated in a timely manner or were not well monitored since the estimate at completion values were too optimistic and highly unlikely.

Used by EM leadership for decision-making. Best practices to ensure that the data from the contractors’ EVM systems are used by EM leadership for decision-making are: (1) reviewing EVM data, including cost and schedule variances, on a regular basis; (2) ensuring that EM management use EVM data to develop corrective action plans; and (3) ensuring that the performance measurement baseline is updated to reflect changes. We reviewed monthly reports EM sites present to EM headquarters management for review. We found that none of the sites adequately reported EVM variances to EM headquarters management; they were all missing some EVM information such as trend data or the estimate at completion. In addition, many of the sites’ monthly reports did not include corrective action plans for addressing variances, if any, between planned and actual performance. We also reviewed monthly reports that the EM Office of Project Management started to present to EM headquarters senior leadership in October 2017, and found that these reports included most of the EVM indicators for all 15 contracts on which EM Office of Project Management reported. However, EM Office of Project Management officials stated that they have only started suggesting corrective action to EM headquarters senior leadership since early 2018; it is too soon to tell how EM headquarters senior leadership is using this information to determine which contracts need the most attention and which corrective actions management will develop and take. Moreover, this monthly report uses unreliable EVM data, as we found in the prior characteristic. Finally, regarding the third best practice, EM provided evidence that 17 out of 20 contractors had a formal process in place for updating the budget baseline. However, the extent to which contractors followed their processes was questionable given the problems we found with the estimates at completion, as discussed in the prior characteristic.

Even though EM requires most of its contractors for operations activities to maintain EVM systems and pays them for doing so, EM’s 2017 policy generally does not require that EVM systems be maintained and used in a way that follow EVM best practices. Until EM updates its cleanup policy to require that EVM systems be maintained and used in a way that follow EVM best practices, EM leadership may not have access to reliable performance data to make informed decisions in managing its cleanup work and to provide to Congress and other stakeholders on billions of dollars’ worth of cleanup work every year.

Much of the Cleanup Work Is Categorized in a Way That Limits the Usefulness of the EVM Data

Compounding the limitations with the EVM systems currently in place, EM has categorized a large portion of its work in a way that limits the usefulness of the EVM data. Specifically, a sizable amount of the work is categorized as level of effort for all 14 contracts for which we could identify the percentage of the level-of-effort work (in dollars). Work that is categorized as level of effort does not have defined deliverables or physical products. Progress for level-of-effort work is measured by the passage of time, but is not measured against a scheduled amount, so no schedule variance occurs. The effectiveness of EVM systems, which are designed to measure performance against cost and schedule targets, will be limited if there is a high amount of level-of-effort work, according to our best practices. Thus, according to best practices, categorizing work as level of effort should be minimized to the extent possible if EVM is being used to measure performance, and contracts with level-of-effort work over 15 percent should be subject to additional scrutiny. As shown in figure 7 below, the range for EM’s contracts on operations activities is between 36 and 83 percent. (We used letters for each contract, rather than identifying the site or contractor).

According to EM officials, at least half of the level-of-effort work conducted under the cleanup contracts consists of recurring activities necessary to maintain the sites, which EM refers to as “minimum safety” work. According to EM officials, examples of such work include physical security, health and radiation protection and services, or critical facility and infrastructure maintenance for safe conditions. These officials said that minimum safety work makes up 30 to 60 percent of individual sites’ budgets, for a total of at least $2.7 billion, or 42 percent, of EM’s $6.4 billion fiscal year 2018 budget. The Assistant Secretary for EM noted in September 2018 that much of DOE’s environmental cost liability has to do with the management of the minimum safety work. The Assistant Secretary also noted that significant potential cost savings could result from reducing minimum safety work and planned to start an initiative in fiscal year 2019 to examine how EM can reduce this work.

EM officials agreed that some of the contractor’s work currently categorized as level of effort could in fact be measured discretely. According to an ANSI guideline, only work not measurable or for which measurement is impractical may be categorized as level of effort. EM officials we interviewed stated that EM relies on its contractors to categorize work as discrete or as level of effort, and EM approves these decisions during the integrated baseline review. According to EM officials, there is no EM policy or guidance on what circumstances justify categorizing work as level of effort. Federal standards for internal controls state that management should design control activities to achieve objectives and respond to risks, such as by clearly documenting internal control in management directives, administrative policies, or operating manuals. Until EM develops a policy that ensures that work is categorized as level of effort only in appropriate, specified circumstances, such as when work is not measurable or when measurement is impractical, it may not have reliable performance data to help it achieve its objective of reducing risks and costs associated with billions of dollars’ worth of cleanup work every year.

Performance Metrics and Milestones for EM Cleanup Work Do Not Provide a Clear Picture of Performance

We found that EM’s 17 performance metrics for its cleanup work measure the scope of work accomplished in a specific year but do not link that work to the cost of completing it. For example, EM reported in the Integrated Planning, Accountability, and Budgeting System database eliminating 72,000 gallons of radioactive liquid waste out of a target of 342,000 gallons for fiscal year 2017 at the Savannah River Site, and disposing of 1,734 cubic meters of low-level waste out of a target of 360 cubic meters at the Idaho site. However, in neither case did EM indicate how much that work cost to accomplish. According to officials from DOE’s Office of Project Management, the scope of work accomplished is not a good indicator of performance by itself because it does not allow the project manager to know whether EM received good value from the contractor. In contrast, EVM systems allow managers to measure the value of work accomplished in a given period. As discussed above, EM collects EVM data, but EM’s performance metrics do not link to the EVM data. According to federal standards for internal control, management should use quality information to achieve an entity’s objectives and the quality information must be complete, among other things. In EM’s case, its objective, as stated in its mission, includes completing its cleanup work in a way that reduces associated risks and costs. By integrating reliable EVM data into EM’s performance metrics for operations activities, EM could provide a clearer picture of performance and better indicate whether EM is achieving its objective of reducing risks and costs.

With regard to cleanup milestones, we found in February 2019 that EM has hundreds of milestones, but the exact number cannot be determined because of inconsistencies in tracking and defining milestones between sites and EM headquarters, and sites have the discretion to send updated milestone data to EM headquarters when they choose. As a result, some sites track milestones differently than EM headquarters does. We also found that EM does not consistently define or track met, missed, or postponed cleanup-related milestones at selected sites, and EM’s milestone reporting to Congress is inconsistent. EM sites renegotiate milestone dates with their regulators before they are missed, and EM does not track the history of these changes. This is because once milestones are changed, sites are not required to maintain or track the original milestone dates. As a result, the new milestones become the new agreed-upon time frame, essentially resetting the deadline. Further, in its report to Congress on enforceable milestones’ status, EM reports the most recently renegotiated milestone dates with no indication of whether or how often those milestones have been missed or postponed. Thus, the EM program is unable to use milestone data to provide a clear, reliable picture of its performance. Furthermore, EM officials at headquarters and selected sites said they had not conducted root cause analyses on missed or postponed milestones. Thus, EM cannot address systemic problems and consider them when renegotiating milestones with regulators. In addition, without such analysis, EM and its cleanup regulators lack information to set more realistic and achievable milestones. As a result, future milestones are likely to continue to be pushed back, further delaying the cleanup work and likely increasing cleanup costs. In this same report, we recommended, among other things, that EM should establish a standard definition of milestones across the cleanup sites, track changes to the milestones, report annually to Congress on the status of its milestones, and conduct root cause analyses of performance shortcomings that lead to missed or postponed milestones.

Conclusions

DOE’s EM program has the challenging mission of safely cleaning up radioactive waste, spent nuclear fuel, and environmental contamination from 50 years of federal nuclear weapons production and energy research, while working to reduce associated risks and costs within the established regulatory framework. Since its mission began in 1989, EM has spent more than $164 billion on its cleanup work, and it faces future cleanup costs of more than $377 billion—the federal government’s single largest environmental liability.

To improve management of projects undertaken within the department, including EM, DOE established its Office of Project Management and strengthened project management requirements in Order 413.3B for managing capital asset projects. However, since 2009, when EM created a new category of cleanup work called operations activities, EM has opted not to apply DOE’s project management requirements to almost 80 percent of its cleanup work. From fiscal years 2011 to 2018, EM’s environmental liability increased by about $214 billion. DOE’s Office of Project Management officials have raised concerns about how EM classifies this work. Until EM works together with DOE’s Office of Project management (1) to establish requirements for classifying cleanup work as capital asset projects or operations activities and (2) to assess EM’s ongoing operations activities to determine if they should be reclassified as capital asset projects based on the newly established requirements, the department may incur more project management risk of cost increases and schedule delays than it should for hundreds of billions of dollars of remaining work.

In July 2017, EM released a new cleanup policy containing requirements for managing its program and its operations activities, but this policy does not follow most of the selected program and project management leading practices we identified related to management of scope, cost, and schedule performance, and independent review of performance. Until EM reviews and revises its cleanup policy to include program and project management leading practices related to scope, cost, schedule performance, and independent reviews, the EM program is at risk of uncontrolled changes to scope, exceeding its cost estimates and schedule, failing to meet its goals, and increasing DOE’s environmental liabilities.

The new Assistant Secretary for the Office of Environmental Management has acknowledged the importance of improving EM’s performance in addressing the department’s large and growing environmental liabilities. However, the three tools that EM uses to measure its overall program performance and contractors’ performance on operations activities— earned value management, performance metrics, and cleanup milestones—do not provide a clear, reliable picture of performance for EM leadership, Congress, and other stakeholders. In particular, EM’s EVM systems for operations activities are not comprehensive, do not provide reliable data, and are not used by EM leadership to measure overall performance of the EM program. Furthermore, a large portion of the work performed by contractors is categorized as level of effort, limiting the usefulness of the EVM data. In addition, EM’s performance metrics are not linked to the costs of the work performed. Until EM updates its cleanup policy to require that EVM systems be maintained and used in a way that follows EVM best practices, EM leadership may not have access to reliable performance data to make informed decisions in managing its cleanup work and to provide to Congress and other stakeholders on billions of dollars’ worth of cleanup work every year. Moreover, until EM develops a policy that ensures that work is categorized as level of effort only in appropriate, specified circumstances, such as when work is not measurable or when measurement is impractical, it may not have reliable performance data to help it achieve its objective of reducing risks and costs associated with billions of dollars’ worth of cleanup work every year. Finally, by integrating reliable EVM data into EM’s performance metrics for operations activities, EM could provide a clearer picture of performance and better indicate whether EM is achieving its objective of reducing risks and costs.

Recommendations for Executive Action

We are making the following seven recommendations to DOE:

The Secretary of Energy should direct the Director of the Office of Project Management and the Assistant Secretary of the Office of Environmental Management to work together to establish requirements for classifying cleanup work as capital asset projects or operations activities. (Recommendation 1)

The Secretary of Energy should direct the Director of the Office of Project Management and the Assistant Secretary of the Office of Environmental Management to work together to asses EM’s ongoing operations activities to determine if they should be reclassified as capital asset projects based on the newly established requirements. (Recommendation 2)

The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to review and revise EM’s 2017 cleanup policy to include program management leading practices related to scope, cost, schedule performance, and independent reviews. (Recommendation 3)

The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to review and revise EM’s 2017 cleanup policy to include project management leading practices related to scope, cost, schedule performance, and independent reviews. (Recommendation 4)

The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to update its cleanup policy to require that EVM systems be maintained and used in a way that follows EVM best practices. (Recommendation 5)

The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to develop a policy to ensure that work is categorized as level of effort only in appropriate, specified circumstances, such as when work is not measurable or when measurement is impractical. (Recommendation 6)

The Secretary of Energy should direct the Assistant Secretary of the Office of Environmental Management to integrate EVM data into EM’s performance metrics for operations activities. (Recommendation 7)

Agency Comments and Our Evaluation

We provided DOE with a draft of this report for its review and comment. In its written comments, reproduced in appendix V, DOE generally agreed with the findings in the report and its recommendations and described actions that it intends to take in response to our recommendations. More specifically, of the seven recommendations, DOE concurred with four and partially concurred with three.

DOE partially concurred with our recommendations that the Director of the Office of Project Management and the Assistant Secretary for the Office of Environmental Management (EM) work together to (1) establish requirements for classifying cleanup work as capital asset projects or operations activities, and (2) assess EM’s ongoing operations activities to determine if they should be reclassified as capital asset projects based on the newly established requirements. DOE stated that the department commits (1) to reviewing its methodology for categorizing work and revising it, as appropriate, as well as (2) to determining the appropriate application of any revisions to the work classification methodology to new and existing work. DOE also stated that the Assistant Secretary for EM is ultimately responsible for the proper classification of work and will consult with the Office of Project Management. We appreciate DOE’s commitment to addressing these two recommendations. As we stated in our report, in July 2015, the Secretary of Energy gave DOE’s Office of Project Management responsibility to serve as DOE’s enterprise project management organization. As such, DOE states that this office is responsible for providing leadership and assistance in developing and implementing DOE-wide policies, procedures, programs, and management systems pertaining to project management, as well as for independently monitoring, assessing, and reporting on project execution performance. Officials from this office are experts in project management, especially as it relates to capital asset projects. Given (1) the high-risk posed by EM’s cleanup work and the high environmental liability, which may continue to grow; (2) the difference in the stringency of requirements between managing and overseeing operations activities and capital asset projects; and (3) the concerns raised by DOE top project management experts that some current operations activities should be classified as capital asset projects, we encourage the Secretary to direct EM not only to consult with DOE’s Office of Project Management but to take advantage of the office’s role and expertise and direct EM to work with this office to come to an agreement about proper classification requirements and classification of current and future cleanup work. It is in DOE’s interest to ensure its cleanup work is classified and managed appropriately, regardless of which office is ultimately responsible for the proper classification of work.

DOE concurred with our recommendations to review and revise EM’s 2017 cleanup policy to include program and project management leading practices related to scope, cost, schedule performance, and independent reviews and to require that EVM systems be maintained and used in a way that follows EVM best practices. DOE also concurred with our recommendation to develop a policy to ensure that work is categorized as level of effort only in appropriate, specified circumstances, such as when work is not measurable or when measurement is impractical. DOE also partially concurred with our recommendation to integrate EVM data into EM’s performance metrics for operations activities. For all these recommendations, DOE stated that EM is already in the process of reviewing the EM cleanup policy for necessary updates, revisions, and modifications. DOE further stated that EM will consider and incorporate changes relative to these recommendations, as appropriate, during this process, and EM will also consider any necessary changes to related guidance or policies and procedures.

DOE also provided technical comments, which we incorporated in our report as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 14 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Energy, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made significant contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

Our report examined: (1) how the EM program manages its cleanup work, (2) the extent to which EM’s cleanup policy follows selected program and project management leading practices, and (3) how EM measures the overall performance of its operations activities.

To examine how the EM program manages its cleanup work, we reviewed various DOE documents, including DOE’s Order 413.3B, EM’s 2012 operations activities protocol, EM’s 2017 cleanup policy, standard operating policies and procedures associated with this cleanup policy, EM’s mission and functions document, EM’s draft 45-day review documentation, meeting minutes from the Project Management Risk Committee, draft appendix to Order 413.3B developed by DOE’s Office of Project Management, and documents received from cleanup sites. We also interviewed DOE officials from the Office of Project Management and members of the Project Management Risk Committee, and EM officials from headquarters, such as the Associate Principal Deputy Assistant Secretary for Field Operations, the Deputy Assistant Secretary for Acquisition and Project Management, officials from EM’s Office of Project Management, Office of Budget and Planning, Office of Program Planning, officials in charge of managing the Integrated Planning, Accountability, and Budgeting System database that collects monthly performance information from the sites, and officials from 5 of EM’s 16 cleanup sites. (We contacted all sites and interviewed 5 sites over the phone that responded to our request for an interview.) We then decided to conduct site visits. We visited two of these sites—Savannah River and Idaho— because they are among the sites with the highest number of operations activities and the most diverse types of and highest-cost cleanup work remaining. Our findings from these 5 sites are not generalizable to all EM sites, but they help explain the delineation of roles between the site managers and EM headquarters in managing and classifying cleanup work. We also attended an EM internal training session in which EM headquarters officials introduced the 2017 cleanup policy to officials at the Hanford site and attended EM cleanup public conferences. Moreover, we reviewed the role of DOE’s Office of Project Management in EM’s cleanup work. More specifically, we examined whether this office played a role in the development of EM’s 2017 cleanup policy and classification of EM’s cleanup work, consistent with its designation as DOE’s enterprise project management organization. To assess the reliability of EM’s fiscal year 2019 budget data, we requested information about EM’s Financial Integration System module of the Integrated Planning, Accountability, and Budgeting System database, from which these data were provided. Based on the responses from officials in charge of this database, we determined the data to be sufficiently reliable for our purposes.

To examine the extent to which EM’s cleanup policy follows selected program and project management leading practices, we selected two sets of criteria for program and project management leading practices using leading practices from the Project Management Institute, which are generally recognized as the top leading practices for program and project management. To select program management leading practices, we first reviewed the Project Management Institute’s The Standard for Program Management—Third Edition (2013). We identified 9 program management leading practices based on PMI’s standards related to a program’s management of scope, cost, schedule performance, and independent review of performance. To select project management leading practices, we first identified 12 project management leading practices listed in DOE’s Order 413.3B related to a project’s management of scope, cost, schedule performance, and independent review of performance. We then compared these 12 project management leading practices to PMI’s A Guide to the Project Management Body of Knowledge–Fifth Edition, which includes PMI’s standards for project management, to make sure these leading practices align with PMI’s standards for project management. To select these leading practices, (1) two GAO analysts separately examined the PMI and DOE documentation, then, (2) a GAO specialist independent of the team producing this report reviewed the leading practices we selected. All three GAO staff agreed on these selected leading practices. To validate our selection of program and project management leading practices, we shared these selected leading practices with PMI representatives and incorporated their feedback, as appropriate. PMI representatives agreed with the program and project management leading practices that we selected.

We then compared EM’s 2017 cleanup policy and the 11 associated standard operating policies and procedures developed by EM by the time of our analysis (by May 2018) with the 9 program management and 12 project management leading practices we selected. We included these standard operating policies and procedures in our analysis because EM officials stated that EM intentionally wrote this policy at a high level because EM planned to develop standard operating policies and procedures that would establish more detailed steps to implement the policy. We analyzed the extent to which the policy and the 11 standard operating policies and procedures follow these leading practices. We also interviewed EM headquarters and site officials to learn more about the 2017 cleanup policy. We used a 5-point scoring system to determine the extent to which EM’s cleanup policy follows selected program and project management leading practices. We used the following 5-point scoring system: “fully met” means that complete evidence was provided that satisfied the leading practice; “substantially met” means that evidence was provided that satisfied a large portion of the leading practice; “partially met” means that evidence was provided that satisfied about half of the leading practice; “minimally met” means that evidence was provided that satisfied a small portion of the leading practice; and “did not meet” means that no evidence was provided that satisfied the leading practice. If the score for each leading practice was “fully met” or “substantially met,” we concluded that EM’s cleanup policy and its associated standard operating policies and procedures followed the leading practice. In contrast, if the score was “partially met,” “minimally met,” or “not met,” we concluded that EM’s policy did not follow the leading practice. To determine this score, two GAO analysts separately examined EM’s policy document and then agreed on a final score for each of the leading practices.

To examine how EM measures the performance of its operations activities, we analyzed EM’s use of the three measures of performance that EM policy identified: earned value management (EVM); performance metrics; and cleanup milestones. To evaluate EM’s EVM systems, we compared EM’s use of EVM with 8 of the 10 best practices for earned value management found in our Cost Estimating and Assessment Guide, which draws best practices from federal cost-estimating organizations and industry. Specifically, we reviewed the use of EVM systems in the 21 contracts EM uses to execute its operations activities and compared this review’s results with EVM best practices. To gather this information, we submitted a data collection instrument to all 16 sites to ascertain whether or not they follow these best practices for each contract containing operations activities. We also requested documentation, such as EVM system certification information or surveillance reports, supporting their answers. We relied mainly on the sites’ responses but, when available, also reviewed the documentation we received to check the sites’ answers for accuracy and completeness.

To determine whether information on EVM is reported to EM senior leadership, we also reviewed (1) monthly progress reports EM sites presented to EM headquarters management that ranged from April 2017 to April 2018 depending on the site and (2) monthly reports that EM Office of Project Management presents to EM headquarters senior leadership; specifically the April 2018 Cleanup Program Monthly Performance and the EM Segment Activity Portfolio Summary, or “Quad Chart,” reports, which were the most recent reports available at the time of this analysis.

In addition, as part of our analysis, we analyzed EM headquarters’ EVM data on operations activities from October 2016 through September 2017 (the most recent data available at the time of our review) to determine whether or not the EVM data were reliable. We checked for data anomalies, such as missing or negative values for each of those months. We also reviewed DOE and EM documents—such as monthly progress reports submitted by the 16 sites to EM headquarters for review or the monthly reviews prepared by an EM headquarters office for senior management—to see what EVM data senior management used for decision-making.

To provide a score for our analysis, we used the following 5-point scoring system to score the answer for each contract for each best practice: “fully met” means that complete evidence was provided that satisfied the best practice; “substantially met” means that evidence was provided that satisfied a large portion of the best practice; “partially met” means that evidence was provided that satisfied about half of the best practice; “minimally met” means that evidence was provided that satisfied a small portion of the best practice; and, “did not meet” means that no evidence was provided that satisfied the best practice. For each best practice, we color-coded the assessment at the contract level. Contracts that fully met or substantially met the criteria were coded green, those that partially met the criteria were coded yellow, and those that did not or minimally meet the criteria were coded red. We then assigned a score for each color: 1 for red, 3 for yellow, and 5 for green. We determined the overall score for each best practice by taking the average across the 20 contracts we reviewed. After scoring each best practice individually, we then used these scores to develop an average score for the three EVM characteristics: whether EM has ensured that these EVM systems are (1) comprehensive, (2) provide reliable data, and (3) are used by EM leadership for decision-making.

To examine EM’s use of performance metrics data, we reviewed annual performance metrics collected by EM headquarters for every operations activity from 2010 to 2017. We chose this period because 2010 is the time when EM started classifying work as operations activities while 2017 was the most recent available data at the time of our analysis. We reviewed relevant documentation, and interviewed agency officials knowledgeable about those data, among other things. Specifically, we interviewed DOE and EM officials at headquarters and from the five cleanup sites (including in-person interviews at the Savannah River and Idaho sites). We also reviewed our prior work in GAO-19-207 related to EM’s cleanup agreements and milestones.

We conducted this performance audit from April 2017 to February 2019, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: EM’s Program-wide Performance Metrics Presented to Congress, as of the end of Fiscal Year 2017

Appendix II: EM’s Program-wide Performance Metrics Presented to Congress, as of the end of Fiscal Year 2017 The information in this table is from DOE’s fiscal year 2019 budget request, which was the most recent request presented to Congress. DOE, Department of Energy: FY 2019 Congressional Budget Request for Environmental Management, DOE/CF-0142, Vol. 5 (Washington, D.C.: March 2018).

Appendix III: GAO Assessment of How Earned Value Management Systems Used for EM’s Operations Activities Met Best Practices

GAO assessment of individual best practice Substantially met. Seventeen out of 20 contracts we reviewed had a certified EVM system, of which 4 self- certified. EM officials reported that the remaining three contracts were not certified or were not required to be certified.

Partially met. Thirteen out of the 20 contracts we reviewed had conducted or planned to conduct an integrated baseline review to ensure that the performance measurement baseline provides reliable cost and schedule data for managing the program and projecting accurate estimates at completion. However, many of these reviews were not rigorous enough to ensure that the performance measurement baseline captured all of the work. Not assessed.

Partially met. Eleven out of the 20 contracts fully met this best practice, and contractors performed self- assessments or conducted annual reviews for 5 additional contracts. However, EM field and headquarters officials were not performing thorough reviews to check whether the EVM systems were in alignment with the EIA-748-D guidelines to ensure that the data being reported by the systems were reliable.

Partially met. The EVM data for operations activities contracts contained numerous, unexplained anomalies in all the months we reviewed—including missing or negative values for some of the completed work to date. Having anomalies in the EVM data occurring each month can cause potential distortions resulting in inaccurate projections of estimates at completion. Not assessed.

Characteristic / Score

Does EM’s use of EVM systems follow characteristic?

GAO assessment of individual best practice Minimally met. We found problems with the estimate at completion in all of the 20 contracts we analyzed. For example, we found instances where the estimates at completion were either (1) less than half the original budget, (2) higher than expected, or 3) zero when the original budget was for hundreds of millions of dollars. These problems indicated that the EVM systems were not being updated in a timely manner or were not well monitored since the estimate at completion values were too optimistic and highly unlikely.

Partially met. We reviewed two sources of information on earned value management reporting to EM senior leadership for this best practice. 1) When reviewing the monthly reports EM sites present to EM headquarters management, we found that none of the sites adequately reported EVM data. 2) When reviewing the new monthly report format that EM’s Office of Project Management presents to EM headquarters senior leadership since October 2017, we found that EM reported on the performance of 15 out of the 20 contracts. We found that these reports included most of EVM indicators for all 15 contracts on which EM Office of Project Management reported. However, this monthly report uses unreliable EVM data, as we found in the prior characteristic.

Partially met. We reviewed two sources of information on earned value management reporting to EM senior leadership for this best practice. 1) When reviewing the monthly reports EM sites present to EM headquarters management, we found that they contained corrective action plans for only 3 contracts. 2) When reviewing the new monthly reports that EM’s Office of Project Management present to EM headquarters senior leadership since October 2017, EM Office of Project Management officials stated that they have only started suggesting corrective action to EM headquarters senior leadership since early 2018; it is too soon to tell how EM headquarters senior leadership is using this information to determine which contracts need the most attention and which corrective actions management will develop and take.

Substantially met. EM provided evidence that 17 out of 20 contractors had a formal process in place for updating the budget baseline. However, the extent to which contractors followed their processes was questionable given the problems we found with the estimates at completion, as discussed in the prior characteristic above.

Appendix IV: EM’s Earned Value Management Systems Used by Contracts Containing Operations Activities

B

Not reviewed Assessment for each best practice: Not met—provided no evidence that satisfies any of the best practice; Minimally met—provided evidence that satisfies a small portion of the best practice; Partially met—provided evidence that satisfies about half of the best practice; Substantially met—provided evidence that satisfies a large portion of the best practice; and Met – provided complete evidence that satisfies the entire best practice. We determined the overall score for each best practice by taking the average across the 20 contracts we reviewed.

We did not evaluate the following two best practices: (1) the schedule reflects the work breakdown structure, the logical sequencing of activities, and the necessary resources and (2) EVM data are consistent among various reporting formats. We excluded these two best practices because we examined the use of EVM by contractors at a higher program level and did not conduct in-depth analysis of each contractor’s EVM system.

EM uses 21 contracts for its operations activities. We reviewed the use of EVM systems in 20 of these contracts because one contract (contract K) is a fixed price contract, which does not require the use of EVM.

Appendix V: Comments from the Department of Energy

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Nico Sloss (Assistant Director), Cristian Ion (Analyst in Charge), Nathan Anderson, Margot Bolon, Jenny Chow, Jennifer Echard, Juan Garay, Cindy Gilbert, Katherine Nicole Laubacher, Cynthia Norris, Karen Richey, Dan C. Royer, Kiki Theodoropoulos, and David Wishard made key contributions to this report.