NUCLEAR WASTE

Waste Isolation Pilot 

Plant Recovery 

Demonstrates Cost 

and Schedule 

Requirements 

Needed for DOE 

Cleanup Operations

Report to Congressional Committees

August 2016

GAO-16-608

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-16-608, a report to 

congressional committees

August 2016

NUCLEAR WASTE

Waste Isolation Pilot Plant Recovery 

Demonstrates Cost and Schedule 

Requirements Needed for DOE Cleanup 

Operations

What GAO Found

The Department of Energy (DOE) did not meet its initial cost and schedule 

estimates for restarting nuclear waste disposal operations at the Waste Isolation 

Pilot Plant (WIPP), resulting in a cost increase of about $64 million and a delay 

of nearly 9 months. DOE incurred this cost increase and delay partly because it 

did not follow all best practices in developing the cost and schedule estimates. In 

particular, DOE’s schedule did not include extra time, or contingency, to account 

for known project risks. Instead, DOE estimated it would restart waste operations 

in March 2016 based on a schedule with no contingency that gave DOE less 

than a 1 percent chance of meeting its restart date. In January 2016, DOE 

approved new estimates that added 8.5 months to the schedule, extending the 

restart to December 2016; increased the estimated cost of recovery by $2 

million; and resulted in an additional $61.6 million in costs for operating WIPP in 

fiscal year 2016. DOE’s WIPP operations activity manager said the revised 

schedule included contingency. However, according to DOE officials, they did 

not follow other best practices. For example, DOE did not provide evidence of 

having an independent cost estimate to validate the revised estimate. DOE did 

not follow all best practices for cost and schedule estimates in part because DOE 

does not require that its cleanup operations, such as WIPP, follow these 

practices. Therefore, DOE cannot have confidence that its estimates are reliable. 

In contrast, DOE established new requirements in June 2015 that its capital 

asset projects, such as the new ventilation system at WIPP, follow these best 

practices. By also requiring cleanup operations to follow them, DOE would have 

more confidence in the estimates for cleanup operations and capital asset 

projects.

DOE did not follow all best practices in analyzing and selecting an alternative for 

the new ventilation system at WIPP. As a result, DOE’s analysis was not reliable 

and DOE cannot be confident that the alternative it selected in December 2015 

will best provide the needed capabilities at WIPP. The analysis of alternatives 

(AOA) process entails identifying, analyzing, and selecting a preferred alternative 

to best meet the mission need. Of the four categories of best practices for AOAs, 

DOE’s process fully met the category for identifying alternatives. For example, 

DOE identified a broad range of ventilation alternatives. However, DOE only 

partially or minimally met the other three categories: general principles, analyzing 

alternatives, and selecting the preferred alternative. DOE did not follow the best 

practice to select the preferred alternative based on a cost-benefit analysis that 

assesses the difference between the life-cycle costs and benefits of each 

alternative. In addition, an independent review that DOE commissioned 

consistent with best practices found that DOE's AOA did not adequately 

document a cost-benefit analysis and that, as a result, the selection of the 

preferred alternative was not supported by compelling information. The 

independent review recommended that DOE conduct a cost-benefit analysis 

consistent with best practices. However, DOE did not conduct the recommended 

analysis and document it before selecting the final alternative because there was 

no requirement to do so. In June 2015, the Secretary of Energy directed DOE to 

develop guidance for conducting AOAs consistent with AOA best practices. A 

DOE official said the department expected to issue the new guidance by 

December 2016.

View GAO-16-608. For more information, 

contact David C. Trimble at (202) 512-3841 

or trimbled@gao.gov. 

Why GAO Did This Study

DOE’s WIPP is the only deep geologic 

repository for the disposal of U.S. 

defense-related nuclear waste. In 

February 2014, waste operations were 

suspended following a truck fire and an 

unrelated radiological release. DOE 

estimated in February 2015 that it 

would complete recovery activities and

restart limited waste operations by 

March 2016. To resume full operations, 

DOE planned to build a new ventilation 

system at WIPP. DOE completed an 

AOA to identify the best solution for 

this system in December 2015.

The Senate Report accompanying a 

bill for the National Defense 

Authorization Act for Fiscal Year 2015 

included a provision for GAO to review 

WIPP operations. This report examines 

the extent to which DOE (1) met its 

initial cost and schedule estimates for 

restarting waste disposal operations, 

and (2) followed best practices in 

analyzing and selecting an alternative 

for the new ventilation system. GAO 

examined documentation on the WIPP 

recovery estimates. GAO compared 

DOE’s February 2015 cost and 

schedule estimates and AOA with best 

practices GAO published.

What GAO Recommends

GAO recommends that DOE require 

cleanup operations to follow best 

practices for cost and schedule 

estimates and require projects, 

including the WIPP ventilation system, 

to implement recommendations from 

independent AOA reviews or document 

the reasons for not doing so. DOE 

concurred with the recommendations.
Page i GAO-16-608 DOE WIPP Recovery

Letter 1

Background 5

DOE Did Not Meet Its Initial Cost and Schedule Estimates for 

Restarting Waste Disposal Operations 13

DOE Did Not Follow All Best Practices in Analyzing and Selecting 

an Alternative for the New Ventilation System at WIPP 20

Conclusions 26

Recommendations for Executive Action 28

Agency Comments and Our Evaluation 28

Appendix I Objectives, Scope, and Methodology 32

Appendix II Assessment of DOE’s Cost Estimate for the WIPP Recovery 

Project Compared with Best Practices 36

Appendix III Assessment of DOE’s Schedule for the WIPP Recovery Project 

Compared with Best Practices 39

Appendix IV Assessment of DOE’s Analysis of Alternatives for the WIPP 

Ventilation System Project Compared with Best Practices 41

Appendix V Comments from the Department of Energy 46

Appendix VI GAO Contact and Staff Acknowledgments 49

Tables

Table 1: Alternatives Proposed by Contractors to DOE for Further 

Evaluation for the New Permanent Ventilation System at 

the Waste Isolation Pilot Plant 22

Contents
Page ii GAO-16-608 DOE WIPP Recovery

Table 2: Assessment of DOE’s February 2015 Cost Estimate for 

the WIPP Recovery Project Compared with Industry Best 

Practices 36

Table 3: Assessment of DOE’s February 2015 Schedule for the 

WIPP Recovery Project Compared with Industry Best 

Practices 39

Table 4: Assessment of DOE’s Analysis of Alternatives (AOA) for 

the WIPP Permanent Ventilation System Project 

Compared with AOA Best Practices 42

Figure

Figure 1: The Waste Isolation Pilot Plant’s Surface Facilities, 

Shafts, and Underground Facilities and Location of the 

Two Incidents in February 2014 6

Abbreviations

AOA analysis of alternatives 

cfm cubic feet of air per minute

DOE Department of Energy 

EM Office of Environmental Management 

NWP Nuclear Waste Partnership LLC 

Trinity Trinity Engineering Associates

WIPP Waste Isolation Pilot Plant

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-16-608 DOE WIPP Recovery

441 G St. N.W.

Washington, DC 20548

August 4, 2016

Congressional Committees

The Department of Energy’s (DOE) Waste Isolation Pilot Plant (WIPP) 

near Carlsbad, New Mexico, serves as the only deep geologic repository 

in the U.S. for the disposal of defense-related nuclear waste. WIPP is 

designed to safely dispose of a specific type of defense waste, referred to 

as transuranic waste,1 generated by DOE’s nuclear weapons research, 

production, and cleanup activities at sites across the country. The waste 

is disposed of in underground rooms mined out of an ancient salt 

formation more than 2,000 feet below the earth’s surface. Since WIPP 

began to accept waste in 1999, DOE has depended on WIPP’s capability 

to accept transuranic waste shipments. In February 2014, waste 

operations at WIPP were suspended following two accidents 

underground: a fire on a salt-hauling truck and an unrelated radiological 

release from a waste container that contaminated portions of the facility 

underground and released a small amount of radiation into the 

environment above ground. The suspension of WIPP’s operations has 

impaired DOE’s ability to meet its cleanup and national security missions 

as well as regulatory cleanup milestones agreed to with states that host 

DOE sites.

The department has made it a top priority to restart WIPP’s waste 

disposal operations. In September 2014, DOE issued a WIPP recovery 

plan.2 This plan describes the major activities needed to restart 

operations, first on a limited basis and then moving to full operations. 

WIPP’s full operations include waste disposal operations concurrent with 

mining and mine maintenance activities. A key set of activities in the 

 1

The word “transuranic” is used for elements that have atomic numbers greater than that 

of uranium. Transuranic waste is defined in the WIPP Land Withdrawal Act of 1992 as 

waste containing more than 100 nanocuries of alpha-emitting transuranic isotopes per 

gram of waste, with half-lives greater than 20 years, except for—(A) high-level radioactive 

waste; (B) waste that the Secretary of Energy has determined, with the concurrence of the 

Administrator of the Environmental Protection Agency, does not need the degree of 

isolation required by the disposal regulations; or (C) waste that the Nuclear Regulatory 

Commission has approved for disposal on a case-by-case basis in accordance with part 

61 of title 10, Code of Federal Regulations. Pub L. No. 102-579, § 2 (1992).

2

Department of Energy, Waste Isolation Pilot Plant Recovery Plan (Sept. 30, 2014). 

Letter
Page 2 GAO-16-608 DOE WIPP Recovery

recovery plan is to increase ventilation airflow underground, which was 

425,000 cubic feet of air per minute (cfm) before the WIPP accidents. As 

a result of the radioactive contamination underground, DOE has 

continually run WIPP’s existing ventilation system in a filtered mode, 

which is not the normal operating mode and has reduced the airflow to 

60,000 cfm. Running in filtered mode severely restricts the number of 

people that can be underground, as well as the activities that can be 

conducted there, such as mine repair activities using diesel-powered 

equipment that creates fumes and dust. To increase the airflow to restart 

limited waste disposal operations, DOE planned two upgrades to the 

existing ventilation system: an interim system that would increase airflow 

to about 114,000 cfm, and a supplemental system that would increase 

airflow to about 180,000 cfm. To resume full operations, DOE planned to 

build a new permanent ventilation system to replace the existing one and 

to provide airflow of about 540,000 cfm, which DOE estimates is sufficient 

to safely support full and concurrent waste operations, mining, and mine 

maintenance activities. From October 2014 through December 2015, 

DOE conducted an analysis of alternatives (AOA) to identify the best 

solution for the new permanent ventilation system. The AOA is a key first 

step in DOE’s procedures for the acquisition of a capital asset such as 

WIPP’s new permanent ventilation system.3 The process entails 

identifying, analyzing, and selecting a preferred alternative to best meet 

the mission need by comparing the operational effectiveness, costs, and 

risks of potential alternatives.

In February 2015, DOE approved a project management baseline that 

included the estimated cost and schedule for completing the activities 

described in the WIPP recovery plan. DOE estimated that it would need 

$242 million to complete the recovery activities needed to restart limited 

waste disposal operations by March 2016. DOE also estimated a cost 

range of $77 million to $309 million to complete the new permanent 

ventilation system by the end of December 2018, paving the way for 

resumption of full operations. In the recovery plan, DOE described its 

schedule as aggressive and noted that it did not include extra time to 

account for unanticipated delays. In May 2015, a DOE assessment found 

that pressure to achieve the March 2016 deadline contributed to poor 

 3

DOE defines capital assets as land, structures, equipment, and intellectual property that 

are used by the federal government and have an estimated useful life of 2 years or more. 
Page 3 GAO-16-608 DOE WIPP Recovery

safety practices in WIPP recovery efforts.4 In July 2015, DOE announced 

that it experienced delays in implementing the project baseline, including 

delays related to procuring equipment and delays related to correcting 

deficiencies in safety practices. As a result of these delays, the 

department announced that it would revise the WIPP project 

management baseline with the goal of developing a more realistic 

schedule. In January 2016, DOE approved the revised estimates for the 

project cost and schedule to restart limited waste operations. 

Nonetheless, the department still faces challenges in completing the 

recovery. For example, in March 2016, the Defense Nuclear Facilities 

Safety Board, which oversees DOE’s nuclear facilities such as WIPP, 

reported that DOE had made progress in revising its nuclear safety plans 

at WIPP but additional work remained to address safety concerns to 

prevent a recurrence of the February 2014 radiological accident.5 As we 

found in our most recent High-Risk Series update, DOE has a history of 

exceeding its cost and schedule estimates and then creating new 

baselines.6 We have also found that when DOE creates new baselines 

during the execution of projects, it can be challenging to independently 

assess DOE’s performance in managing these projects.7

The Senate Armed Services Committee report accompanying a bill for the 

National Defense Authorization Act for Fiscal Year 2015 included a 

provision for us to review recovery operations at WIPP.8 Our report 

examines the extent to which DOE (1) met its initial cost and schedule 

estimates for restarting waste disposal operations at WIPP and (2) 

followed best practices in analyzing and selecting an alternative for the 

new ventilation system.

 4

Department of Energy, Office of Enterprise Assessments, Memorandum: Office of 

Enterprise Assessments Operational Analysis of Safety Trends at the Waste Isolation Pilot 

Plant, May 2014 - May 2015 (Oct. 15, 2015). 

5

Defense Nuclear Facilities Safety Board, Staff Issue Report: Waste Isolation Pilot Plant 

Documented Safety Analysis (Washington, D.C.: Mar. 28, 2016). 

6

GAO, High Risk Series: An Update, GAO-15-290 (Washington, D.C.: Feb. 11, 2015).

7

GAO, Department of Energy: Better Information Needed to Determine If Nonmajor 

Projects Meet Performance Targets, GAO-13-129 (Washington, D.C.: Dec. 19, 2012). 

8

S. Rep. No. 113-176, at 286 (2014). 
Page 4 GAO-16-608 DOE WIPP Recovery

To address both objectives above, we conducted a site visit to WIPP in 

January 2015. During the site visit we obtained documentation and 

interviewed officials from DOE’s Office of Environmental Management 

(EM), which is responsible for oversight of WIPP and exercises this 

responsibility primarily through its Carlsbad Field Office. We also 

interviewed representatives at WIPP from Nuclear Waste Partnership 

LLC (NWP), which is the private contractor that manages and operates 

WIPP for DOE. To examine the extent to which DOE met its initial 

(February 2015) cost and schedule estimates for restarting waste 

disposal operations, we compared DOE’s initial cost and schedule 

estimates to restart limited waste disposal operations contained in its 

February 2015 WIPP recovery project management baseline with DOE’s 

revised estimates in its January 2016 revised baseline. We reviewed 

DOE’s reports on the reasons it exceeded the initial estimates and the 

risk management plans for WIPP recovery prepared by NWP which were 

used in developing the initial estimates. We compared the initial cost and 

schedule estimates from the February 2015 recovery baseline with the 

best practices described in our cost and schedule guides.9 We reviewed 

documentation on the revised (January 2016) baseline and interviewed 

DOE and NWP on the approach followed to develop the revised cost and 

schedule estimates to restart limited waste disposal operations in the 

baseline, but we did not assess the revised estimates against the best 

practices because of the time frame of our review. To examine the extent 

to which DOE followed best practices in analyzing and selecting an 

alternative for the new ventilation system, we reviewed documentation on 

the process DOE followed and compared the process with best practices 

for conducting an AOA that we published in December 2014.10 During the 

review, we also interviewed officials from DOE’s Office of Enterprise 

Assessments, which provides internal oversight of DOE facilities; the 

 9

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009); and 

GAO Schedule Assessment Guide: Best Practices for Project Schedules, GAO-12-120G

(Washington, D.C.: May 2012).

10GAO, DOE and NNSA Project Management: Analysis of Alternatives Could Be 

Improved by Incorporating Best Practices, GAO-15-37 (Washington, D.C.: Dec. 11, 2014). 

We compiled the best practices identified in this report by reviewing AOA policies and 

guidance used by seven public and private-sector entities with experience in the AOA 

process. GAO verified these practices with subject-matter experts. We updated these 

AOA best practices in an October 2015 report: Amphibious Combat Vehicle: Some 

Acquisition Activities Demonstrate Best Practices; Attainment of Amphibious Capability to 

be Determined, GAO-16-22 (Washington, D.C.: Oct. 28, 2015). 
Page 5 GAO-16-608 DOE WIPP Recovery

Defense Nuclear Facilities Safety Board, which provides external 

oversight of DOE defense nuclear facilities; the U.S. Mine Safety and 

Health Administration, which provides external oversight of mining 

activities at WIPP; and the U.S. Environmental Protection Agency and 

New Mexico Environment Department, both of which provide external 

regulation of WIPP. (See app. I for further information on the scope and 

methodology of our review.)

We conducted this performance audit from November 2014 to August

2016 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

This section describes (1) WIPP’s layout and base operations, (2) the 

February 2014 truck fire and radiological release accident investigation 

reports, (3) DOE’s requirements for operations activities and for project 

management applicable to the WIPP Recovery effort, (4) best practices 

for cost and schedule estimating, (5) best practices for conducting an 

AOA, (6) Revisions in 2014 and 2015 to DOE’s requirements for AOAs, 

and (7) DOE’s AOA process for WIPP’s new permanent ventilation 

system.

As shown in figure 1, WIPP’s layout consists of surface facilities, 

underground facilities, and four shafts that connect the surface with the 

underground. WIPP’s underground facilities currently include seven waste 

disposal units or panels where waste containers are placed for final 

disposal, an area designated for constructing future disposal units, and an 

area for experimental research. The four shafts connecting the surface 

and underground facilities are (1) the air intake shaft, which is the primary 

air supply to the underground; (2) the exhaust shaft, through which all air 

exits the underground; (3) the salt handling shaft, which is used to 

remove mined salt from the underground and also supplies air to the 

underground; and (4) the waste handling shaft which is used to transport 

waste containers to the underground. Figure 1 also shows the locations of 

the February 2014 accidents that led to the suspension of WIPP’s 

operations.

Background

WIPP Facility Layout and 

Base Operations
Page 6 GAO-16-608 DOE WIPP Recovery

Figure 1: The Waste Isolation Pilot Plant’s Surface Facilities, Shafts, and Underground Facilities and Location of the Two 

Incidents in February 2014

Under normal operating conditions, base operations at WIPP include 

waste handling to receive waste from generator sites and prepare and 

place the waste, ongoing maintenance of the WIPP surface facilities and 

underground, as well as support functions such as program management, 

engineering, quality assurance, safety and security, and environmental 

management. In addition, NWP reviews and certifies waste containers on 

DOE’s behalf at DOE’s transuranic waste generator sites. During 

implementation of the WIPP recovery plan, DOE continued aspects of 
Page 7 GAO-16-608 DOE WIPP Recovery

these base operations such as maintenance of the surface facilities and 

program management. In addition, NWP continued to review and certify 

waste packages at select sites.

The salt truck fire, which occurred on February 5, 2014, created 

substantial smoke and soot that damaged key equipment and facilities in 

the underground. On February 14, 2014, the radiological release occurred 

when a transuranic waste container was breached. The breach was 

caused by a chemical reaction inside the container between materials 

that DOE later determined should not have been packaged together. The 

reaction generated enough heat to increase pressure in the container. 

The pressure forced open the container’s lid and propelled its radioactive 

waste contents, combustible gases, and other materials into the air and 

on to adjacent waste containers. The radioactive contents, gases, and 

other materials ignited and triggered a fire in the disposal room by igniting 

other materials in the room. WIPP’s ventilation system failed to contain all 

of the airborne radiological material underground and allowed a small 

amount to enter the environment. As a result of the release, portions of 

the WIPP underground and the existing ventilation system were 

radiologically contaminated.

DOE issued an accident investigation report on the salt truck fire in March 

2014 and an accident investigation report on the radiological release in 

April 2014 called “phase one report.”11 The phase one report focused on 

the release to the environment and the response at WIPP to the release. 

One year later, in April 2015, DOE issued a phase two report that focused 

on the radiological release from inside the waste container.12 The reports 

determined that the causes of the accidents included the degraded 

condition of critical equipment, inadequacies in the design and operations 

of WIPP’s ventilation system, and deficiencies in the management of 

WIPP’s safety programs that are intended to control and discipline 

operations to protect workers, the public, and the environment from 

radiological and other hazards. In total, the three reports made 100

 11Department of Energy, Accident Investigation Report: Underground Salt Haul Truck Fire 

at the Waste Isolation Pilot Plant on February 5, 2014 (Washington, D.C.: Mar. 13, 2014) 

and Accident Investigation Report: Phase 1 Radiological Release Event at the Waste 

Isolation Pilot Plant on February 14, 2014 (Apr. 22, 2014). 

12Department of Energy, Accident Investigation Report: Phase 2 Radiological Release 

Event at the Waste Isolation Pilot Plant on February 14, 2014 (Apr. 15, 2015). 

WIPP Truck Fire and 

Radiological Release 

Accidents and 

Investigation Reports
Page 8 GAO-16-608 DOE WIPP Recovery

recommendations to DOE and NWP to complete corrective actions.

13 For 

example, the phase one report identified a number of weaknesses with 

the WIPP nuclear safety documentation and recommended that this 

documentation be revised prior to resuming limited waste disposal 

operations to ensure the repository can be operated safely with respect to 

workers, the public, and the environment.14

DOE is managing the recovery activities to restart limited waste disposal 

operations at WIPP and to design and build the new permanent 

ventilation system capital asset project following two separate project 

management requirements. DOE requires that the recovery activities to 

restart limited waste disposal operations and overall base operations of 

WIPP be managed following EM’s protocol for operations activities. This 

protocol defines operations activities to include activities that are projectlike with defined start and end dates and reoccurring facility or 

environmental operations.15 The new permanent ventilation system 

project must follow DOE’s Order 413.3B, which governs project 

management for the acquisition of capital assets.16 The following is a 

description of the two different requirements:

 13In addition to the accident investigations, in February 18, 2016, DOE’s Office of 

Enterprise Assessments’ Office of Enforcement concluded an investigation and issued a 

preliminary notice of violation to NWP for violations of worker safety and health and 

nuclear safety requirements in connection with the accidents. In its notice, DOE proposed 

no civil penalties for the worker safety and health violations cited because DOE’s Carlsbad 

Field Office had previously reduced NWP’s contract fee in response to the salt haul truck 

fire and radiological release accidents. 

14The documentation, called a documented safety analysis, is an analysis of the extent to 

which a nuclear facility can be operated safely with respect to workers, the public, and the 

environment, including a description of the conditions, safe boundaries, and hazard 

controls that provide the basis for ensuring safety. The phase one report recommended 

that DOE revise the WIPP Documented Safety Analysis, Revision 4, dated November 

2013, DOE/WIPP-07-3372.

15Department of Energy, Office of Environmental Management, Policy and Protocol for 

Office of Environmental Management Operations Activities (Mar. 15, 2012). According to 

the protocol, operations activities include treatment, stabilization, packaging, storage, 

transportation, and disposition of waste and nuclear materials; operations of 

environmental remediation systems; long-term environmental stewardship; and facility 

shutdown and deactivation activities.

16Department of Energy, Program and Project Management for the Acquisition of Capital 

Assets, DOE Order 413.3B (Washington, D.C.: Nov. 29, 2010). 

DOE’s Requirements for 

Operations Activities and 

for Project Management 

Applicable to the WIPP 

Recovery Effort
Page 9 GAO-16-608 DOE WIPP Recovery

• EM operations activities protocol. EM’s protocol establishes a 

framework for managing and reporting the progress of cleanup 

operations by requiring, among other things, that project performance 

be measured objectively and that management actions be taken to 

mitigate risks and manage costs. EM’s protocol directs EM sites to 

develop performance baselines—or estimates—for cost, schedule, 

and scope to use in assessing the project’s performance over the 

fiscal year, multiyear contract period of performance, and the life cycle 

of the project.17 The protocol directs that the project develop 

estimates but does not include specific requirements that must be 

followed, such as the steps that must be followed to develop cost and 

schedule estimates.

• DOE Order 413.3B. DOE’s order provides program and project 

management direction for the acquisition of capital assets, with the 

stated goal of delivering fully capable projects within the planned cost, 

schedule, and performance baseline. The order establishes five 

critical decision processes of project development that each end with 

a major approval milestone—or “critical decision” point—that cover 

the life of a project. The order specifies the requirements that must be 

met, including for developing and managing project cost and schedule 

estimates to move a project past each critical decision milestone. 

DOE also provides suggested approaches for meeting the 

requirements contained in the order through a series of guides, such 

as a guide for cost estimating. In June 2015, the Secretary of Energy 

directed that this order be revised to require that cost and schedule 

estimates be developed, maintained, and documented in a manner 

consistent with methods and the best practices identified in GAO’s 

cost and schedule assessment guides and other published standards 

and best practices, such as the Federal Acquisition Regulation and 

Office of Management and Budget guidance.18

 17According to EM’s operations activity protocol, a contract period of performance 

baseline is only applicable if required by contract.

18Department of Energy, Secretary of Energy, Memorandum for Heads of All Department 

Elements: Project Management Policies and Principles (Washington, D.C.: June 8, 2015). 
Page 10 GAO-16-608 DOE WIPP Recovery

The GAO Cost Estimating and Assessment Guide and the GAO Schedule 

Assessment Guide compiled best practices corresponding to the 

characteristics of high-quality and reliable cost and schedule estimates.19

A high-quality, reliable cost estimate has the following four characteristics:

• comprehensive (e.g., has enough detail to ensure that cost elements 

are neither omitted nor double counted),

• well-documented (e.g., allows for data it contains to be traced to 

source documents),

• accurate (e.g., is based on an assessment of most likely costs and 

has been adjusted properly for inflation), and

• credible (e.g., discusses any limitations because of uncertainty or 

bias surrounding data or assumptions).

Similarly, a high-quality, reliable schedule has four characteristics:

• comprehensive (e.g., captures all government and contractor 

activities necessary to accomplish a project’s objectives),

• well-constructed (e.g., sequences all activities using the most 

straightforward logic possible),

• controlled (e.g., is updated periodically to realistically forecast dates 

for activities), and

• credible (e.g., uses data about risks to predict the level of confidence 

in meeting a completion date and necessary schedule contingency 

and high-priority risks are identified based on conducting a robust 

schedule risk analysis).

The characteristics of a high-quality and reliable cost estimate are 

supported by best practices listed in appendix II, and the characteristics 

of a high-quality and reliable schedule are supported by best practices 

listed in appendix III.

 19GAO-09-3SP and GAO-12-120G. 

GAO’s Best Practices for 

Cost and Schedule 

Estimating
Page 11 GAO-16-608 DOE WIPP Recovery

In a December 2014 report we compiled 24 best practices for conducting 

an AOA.20 As noted above, the AOA is a key first step in acquisition of a 

capital asset such as WIPP’s new permanent ventilation system. The 

process entails identifying, analyzing, and selecting a preferred 

alternative. Conforming to these best practices helps ensure that the 

preferred alternative selected is the one that best meets the agency’s 

mission needs. We grouped these 24 best practices into the following four 

categories:

• General principles. This category includes best practices to be 

applied in planning, conducting, and documenting the AOA, such as 

defining functional requirements based on the mission need and 

conducting the analysis without a predetermined solution.

• Identifying alternatives. The identifying alternatives category 

includes best practices that help ensure the alternatives to be 

analyzed are sufficient, diverse, and viable.

• Analyzing alternatives. The analyzing alternatives category contains 

best practices related to estimating the costs and benefits of each 

alternative over its life cycle.

• Selecting a preferred alternative. The selecting a preferred 

alternative category includes best practices to help ensure a preferred 

alternative is selected that best meets the mission need by comparing 

alternatives based on their costs and benefits and independently 

reviewing the AOA process. Appendix IV lists the 24 best practices 

organized by the four categories.

DOE’s procedures for conducting an AOA have changed twice since DOE 

began the AOA process for the WIPP permanent ventilation system. DOE 

started the AOA process for this system in October 2014. At that time, 

DOE was operating under AOA requirements defined in Order 413.3B. 

The AOA process, as defined in the order, occurs during the span of the 

first two critical decisions—Critical Decision-0 and Critical Decision-1 

(Approve Alternative Selection and Cost Range). In December 2014, the 

Secretary of Energy established a new requirement that for projects 

estimated to cost $50 million or more, the responsible program office is to 

 20GAO-15-37.

GAO’s Best Practices for 

Analysis of Alternatives

Revisions in 2014 and 

2015 to DOE’s 

Requirements for Analysis 

of Alternatives
Page 12 GAO-16-608 DOE WIPP Recovery

conduct an AOA independent of the contractor responsible for the 

project.21 Later, in June 2015, the Secretary of Energy directed the 

department to develop guidance for conducting AOAs consistent with the 

AOA best practices that GAO has published.22 The Secretary further 

required that AOAs be conducted and documented in a manner 

consistent with the guidance when it is complete. A DOE official told us in 

June 2016 that the department expected the guidance to be completed by 

December 2016. The Secretary’s action followed a recommendation that 

we made in December 2014 that DOE revise its Order 413.3B to adopt 

AOA best practices.23

DOE’s changes in its requirements for the AOA process under Order 

413.3B took effect in the midst of DOE’s AOA process for WIPP’s new 

permanent ventilation system. DOE’s process was conducted as follows:

• DOE EM approved the mission need for the new ventilation system 

project in October 2014.

• Also in October 2014, NWP started an initial AOA and completed it in 

January 2015 shortly after the Secretary’s December 2014 directive 

on the independence of AOAs and prior to the Secretary’s June 2015 

directive about the use of the AOA best practices; therefore the new 

requirements did not apply to NWP’s AOA.

• To respond to the Secretary’s December 2014 directive, DOE 

contracted in spring 2015 with a contractor unaffiliated with WIPP—

Trinity Engineering Associates (Trinity)—to do a second AOA. DOE 

specifically directed Trinity in its contract to implement all of the AOA 

best practices that we compiled in our December 2014 report. The 

 21Department of Energy, Secretary of Energy, Memorandum for Heads of All Department 

Elements: Improving the Department’s Management of Projects (Washington, D.C.: Dec. 

1, 2014).

22Secretary of Energy, Memorandum, June 8, 2015. 

23GAO-15-37. Specifically, in our December 2014 report, we found that DOE’s 

requirements for the AOA process in Order 413.3B conformed to only 1 of the 24 best 

practices for conducting an AOA and when the requirements were combined with 

associated guidance—which includes nonmandatory approaches for meeting 

requirements—we found that they conformed to 9 of the 24 best practices. We 

recommended that DOE update its Order 413.B requirements to incorporate best 

practices for conducting an AOA. 

DOE’s AOA Process for 

WIPP’s New Permanent 

Ventilation System
Page 13 GAO-16-608 DOE WIPP Recovery

WIPP program manager in the Carlsbad Field Office told us that he 

included the best practices in the contract to provide more formality to 

their process. Trinity completed its AOA in October 2015.

• DOE evaluated the preferred alternatives proposed by NWP and 

Trinity, and it selected and approved a final alternative to complete 

Critical Decision-1 in December 2015.

DOE did not meet its initial cost and schedule estimates for the efforts 

needed to restart WIPP disposal operations, resulting in about $64 million 

in added costs and a delay of nearly 9 months. Two primary factors 

contributed to the cost increase and schedule delay. First, DOE only 

partially followed best practices in developing its initial cost and schedule 

estimates, which made them unreliable and increased the likelihood that 

they would be exceeded. Second, DOE did not successfully manage key 

project risks it had identified that had potential to cause delay.

DOE incurred a cost increase of about $64 million and a nearly 9-month 

schedule delay in its efforts to restart WIPP waste disposal operations. 

Specifically, in January 2016, approximately a year after approving the 

initial project baseline, DOE approved a new project management 

baseline that increased the estimated costs for the recovery project by $2 

million (from $242 million to $244 million) and added 8.5 months to the 

project schedule, extending the date when limited waste disposal 

operations might begin from March 2016 to December 2016. According to

DOE officials, the project was unable to meet its estimated completion 

date due to delays, including delays associated with procuring 

components for the interim ventilation system upgrade that were found to 

be faulty, and delays associated with DOE’s decision to require the 

project to adhere to new nuclear safety requirements. According to the 

operations activity manager for WIPP in the Carlsbad Field Office, the 

recovery project only exceeded the $242 million baseline by $2 million in 

part because DOE overestimated the cost of some project activities and 

spent contingency funds to make up for cost increases in certain parts of 

the project. For example, DOE overestimated the cost for 

decontaminating the WIPP underground by more than $6 million, which 

offset a $5 million cost increase caused by delays associated with the 

faulty ventilation system components. However, we identified an 

additional $61.6 million cost increase in base operations that was 

attributable to the delay in completing the recovery project. Specifically, 

DOE’s justification supporting the President’s fiscal year 2016 budget 

DOE Did Not Meet Its 

Initial Cost and 

Schedule Estimates 

for Restarting Waste 

Disposal Operations

DOE Incurred A Cost 

Increase of about $64 

Million and a Nearly 9-

month Delay in Its Efforts 

to Restart WIPP 

Operations 
Page 14 GAO-16-608 DOE WIPP Recovery

request to Congress from February 2015 estimated about $130.6 million 

for WIPP’s base operations in fiscal year 2016. According to DOE 

officials, as DOE revised its cost and schedule estimates for the WIPP 

recovery project in the fall of 2015, DOE also revised its cost estimate for 

WIPP’s base operations and provided the updated information to the 

congressional appropriations committees. In December 2015, DOE 

received $192.1 million for fiscal year 2016 base operations, an increase 

of $61.6 million (47 percent) over the initial estimate for base operations 

in fiscal year 2016. According to the operations activity manager for WIPP 

in the Carlsbad Field Office and representatives from NWP, this increase 

to base operations occurred because DOE estimated the cost of base 

operations on the assumption of restarting WIPP in March 2016 following 

its initial schedule estimate. The delay in the recovery efforts required 

DOE to keep base operations running alongside the recovery efforts for a 

longer period of time than initially planned.

DOE did not meet its initial schedule and cost estimates for restarting 

waste disposal operations at WIPP, in part because DOE did not develop 

the estimates following all best practices, rendering the estimates 

unreliable and increasing the likelihood that they would be exceeded. 

More specifically, of the four characteristics of a high-quality and reliable 

cost estimate, DOE’s initial cost estimate substantially met two 

characteristics—comprehensive and well-documented—but partially met 

or minimally met the other two characteristics—accurate and credible. We 

made the following observations:

• DOE substantially met best practices for a comprehensive cost 

estimate by, for example, including a majority of life-cycle costs and 

identifying the cost estimating ground rules and assumptions.

• DOE substantially met best practices for a well-documented 

estimate by including the actual sources for the cost data and 

documenting that the cost estimate was reviewed and accepted by 

DOE management.

• DOE partially met best practices for an accurate estimate by, for 

example, only documenting a portion of the process they used to 

adjust costs to account for inflation.

• DOE minimally met best practices for a credible estimate by, for 

instance, not developing an independent cost estimate. Had DOE 

completed an independent cost estimate, the department would have 

DOE’s Initial Cost and 

Schedule Estimates for 

Restarting Operations at 

WIPP Were Unreliable
Page 15 GAO-16-608 DOE WIPP Recovery

had an unbiased and objective benchmark to assess whether the cost 

estimate prepared by NWP could be achieved, and thus would have 

been positioned to reduce the risk that the project would proceed 

underfunded.

Regarding DOE’s initial schedule for restarting waste disposal operations, 

DOE substantially met two of the four characteristics of a high-quality 

schedule—comprehensive and controlled—but partially met the 

characteristics—well-constructed and credible—resulting in a schedule 

that was also unreliable and unrealistic. We made the following 

observations:

• DOE substantially met best practices for a comprehensive schedule 

because, for example, the schedule reflected the activities in the 

recovery work breakdown structure, which defined the work 

necessary to accomplish the project’s objectives. In addition, work 

scope was assigned in the schedule as the responsibility of NWP or 

DOE.

• DOE substantially met best practices for a controlled schedule by, 

for example, regularly updating its master schedule and using the 

schedule as the basis for measuring performance.

• DOE partially met best practices for a well-constructed schedule 

because, for instance, a significant number of activities in the 

schedule had incorrect or missing logic relationships that are 

important for determining how delays or accelerations in one activity 

would affect the start or finish of other activities later in the schedule.

• DOE partially met best practices for a credible schedule, in 

particular, because its schedule did not include extra time, or 

contingency, to account for known project risks. As noted above, DOE

acknowledged in its WIPP recovery plan that the schedule did not 

include contingency that may be needed due to unanticipated 

difficulties or delays with the project. However, DOE did not 

acknowledge in the plan that the schedule also did not include 

contingency for the occurrence of known and quantified risks that had 

been anticipated in its project risk analysis—such as the risks that 

were the primary causes of the project delays. Notably, DOE’s risk 

analysis predicted that the March 2016 date to restart operations had 

a less than 1 percent confidence level, meaning in effect that DOE 

had a less than 1 percent chance of meeting the March 2016 

deadline. EM officials said the department used a less than 1 percent 

confidence level because it wanted to have an aggressive goal for 
Page 16 GAO-16-608 DOE WIPP Recovery

restarting operations. According to EM officials, when EM managers 

presented this schedule to senior DOE decision-makers for approval, 

they described the schedule as “aggressive” but did not clarify that 

there was less than a 1 percent chance of meeting the schedule. 

According to these officials, they did not think that senior managers 

would understand the project management terminology regarding 

confidence levels.

In January 2016, DOE approved revised cost and schedule estimates for 

restarting WIPP’s limited waste disposal operations.24 The operations 

activity manager for WIPP in the Carlsbad Field Office said the revised 

schedule included contingency and the revised restart date of December 

2016 had an 80 percent confidence level. However, according to DOE 

officials, they did not follow other best practices. For example, DOE did 

not provide evidence of having an independent cost estimate which would 

have provided DOE an unbiased and objective benchmark to validate the 

estimates prepared by NWP. As noted above, we did not assess DOE’s 

revised estimates against all of the best practices. The full results of our 

analysis of the initial cost and schedule estimates can be found in 

appendixes II and III.

DOE did not follow all best practices in developing the initial cost and 

schedule estimates for the WIPP recovery project or in developing new 

estimates because, unlike DOE’s requirements for capital asset projects 

under Order 413.3B, DOE’s EM operations activities protocol that 

governs cleanup operations such as WIPP recovery does not require the 

use of best practices in developing such estimates. As mentioned above, 

DOE EM’s protocol requires that EM sites develop or approve baselines 

for each project’s cost and schedule to judge project performance. 

However, the protocol does not specify any best practices in terms of the 

steps to follow in developing the cost and schedule estimates in these 

baselines. The absence of a requirement to follow best practices for cost 

and schedule estimating is in contrast to a decision in June 2015 by the 

Secretary of Energy to require that all capital asset projects with an 

estimated cost of $10 million or more follow such practices under Order 

 24DOE revised its estimates by establishing an integrated baseline for WIPP that 

combined recovery activities, WIPP’s life-cycle base operations, and capital asset 

projects. According to NWP, the integrated baseline primarily focused on fiscal years 2016 

and 2017 because those years complete the current base contract between DOE and 

NWP.
Page 17 GAO-16-608 DOE WIPP Recovery

413.3B. Operations activities such as the WIPP recovery activities are not 

considered capital asset projects and therefore are not required to follow 

these requirements even though EM’s operations activities share many of 

the same characteristics as capital asset projects. Without requiring EM’s 

operations activities such as WIPP to follow all best practices when 

developing cost and schedule estimates DOE cannot have confidence 

that it is producing reliable baselines needed to monitor its performance in 

managing these activities. Without reliable baselines, DOE will also 

continue to be at risk of cost overruns and delays in achieving its cleanup 

missions such as the permanent disposal of transuranic waste from sites 

across the country.

DOE has reported progress in completing the activities needed to restart 

limited waste operations using its revised cost and schedule estimates, 

although it still faces challenges and it remains unclear whether DOE 

identified and analyzed the risks associated with these challenges in 

revising the estimates. In terms of progress, for example, in December 

2015, NWP submitted revisions to WIPP’s nuclear safety documentation 

to EM for review. In addition, DOE reported in March 2016 that NWP 

completed most of the construction of the interim ventilation system 

upgrade in the underground. However, as noted above, in March 2016, 

the Defense Nuclear Facilities Safety Board reported that additional work 

was needed to revise WIPP’s nuclear safety documentation to prevent 

recurrence of a radiological release accident.25 In addition, in April 2016, 

DOE’s Office of Enterprise Assessments, which provides independent 

internal oversight of DOE’s management of safety, issued two reports that 

found that although NWP had made improvements in its operational 

safety and emergency management programs and procedures, significant 

challenges remained to fully meet DOE requirements and effectively plan 

and implement these programs and procedures.26 As we noted above, 

DOE officials acknowledged that the department did not follow all best 

practices in developing the revised estimates to restart WIPP operations. 

Without having followed all best practices, including having an 

independent cost estimate conducted to validate the estimates, DOE 

cannot be confident that NWP sufficiently accounted for these challenges 

 25Defense Nuclear Facilities Safety Board, Staff Issue Report, Mar. 28, 2016.

26Department of Energy, Office of Enterprise Assessments, Assessment of Selected 

Conduct of Operations Processes at the Waste Isolation Pilot Plant (April 2016); and 

Office of Enterprise Assessments, Emergency Management Assessment of the Waste 

Isolation Pilot Plant (April 2016).
Page 18 GAO-16-608 DOE WIPP Recovery

in revising its risk analysis and that DOE set an appropriate allowance for 

contingency to reduce the risk of cost overruns and delays in restarting 

WIPP’s operations.

DOE did not successfully manage key risks that it had identified and that 

contributed to the project’s cost increase and schedule delay but has 

taken steps to revise the risk management process for the WIPP recovery 

project. DOE’s EM operations activities protocol requires the use of a risk 

management process to identify and mitigate risks to completing a project 

within its baseline cost and schedule estimates. For the WIPP recovery 

project, DOE and NWP developed a project risk register to support the 

initial project management baseline for WIPP recovery. The risk register 

listed the risks that could increase costs or delay completion. DOE and 

NWP assessed the likelihood and consequence of each risk and 

identified a mitigation plan for each risk that described the actions to 

reduce the impact on the project if the risk were realized.

In a public statement explaining the need to revise its baseline for the 

recovery project, DOE cited problems with three activities that the 

department had previously identified in its project risk register: (1) 

completing revisions to WIPP’s nuclear safety documentation; (2) 

installing the interim ventilation system upgrade; and (3) completing the 

corrective actions that address the recommendations in the phase two 

accident investigation report. DOE had identified risks related to each of 

these activities in the project risk register but was not able to effectively 

mitigate them, as discussed below:

• WIPP nuclear safety documentation. NWP identified the risk of a 

potential delay of up to 6 months in restarting operations if DOE 

decided that the WIPP nuclear safety documentation should be 

revised according to standards that were in the process of being 

updated rather than according to the existing standards.27 According 

to DOE, when NWP started to revise the safety documentation in the 

 27Department of Energy, DOE Standard: Preparation of Nonreactor Nuclear Facility 

Documented Safety Analysis, DOE-STD-3009-2014 (Washington, D.C.: November 2014). 

The DOE standard describes a method for preparing a documented safety analysis that is 

acceptable to DOE for nonreactor nuclear facilities such as WIPP. The November 2014 

standard—which had not yet been issued when NWP conducted its risk analysis—is an 

update to the prior version issued in 1994. According to the document, the 2014 update 

clarifies and updates facility safety requirements. 

DOE Did Not Successfully 

Manage WIPP Recovery 

Project Risks That 

Contributed to the Project 

Cost Increase and 

Schedule Delay but Has 

Revised Its Risk 

Management Process
Page 19 GAO-16-608 DOE WIPP Recovery

summer of 2014, the contractor assumed it should follow the existing 

standard. NWP added this risk to its project risk register to account for 

the possibility that EM could require NWP to revise the nuclear safety 

documentation according to the new standards. To mitigate this risk, 

NWP’s plan was to ensure they had concurrence with DOE to start 

the revisions using the existing standards. DOE issued the new 

standards in November 2014 and notified NWP in December 2014 

that the WIPP safety documentation would need to comply with them. 

According to an NWP recovery project manager, this change in policy 

resulted in a 7-month delay in NWP’s schedule for revising the WIPP 

safety documentation.

• Interim ventilation system. NWP identified the risk of a potential 

delay of 3 months to complete the installation of the interim ventilation 

system upgrade if NWP faced difficulties acquiring the components for 

the system. To mitigate this risk, NWP’s plan was to identify difficultto-procure equipment early in the project, ensure it had a valid and 

up-to-date list of qualified suppliers, and monitor and review 

equipment purchases. According to NWP officials, this plan was 

partially effective. Specifically, NWP officials said that they discovered 

that components for the interim ventilation system were damaged 

when they inspected the components before formally accepting them. 

In addition, they said that the costs to return the components to the 

manufacturer and repair them were paid for by the manufacturer. 

However, the NWP officials said that as a result of receiving the faulty 

equipment, the installation of the interim ventilation system was 

delayed about 4 months because NWP needed to send the 

components back to the manufacturer and make corrections to its 

shipping process to prevent a reoccurrence of these issues.

• Corrective actions from the phase two accident investigation 

report. NWP identified the risk of potential delay of up to 6 weeks 

regarding the length of time DOE needed to complete the phase two 

accident investigation. In the summer and fall of 2014, when NWP 

was developing the initial project management baseline, DOE’s phase 

two accident investigation was still underway. According to NWP 

officials, they needed to wait for DOE to complete the accident 

investigation and issue the final report with recommendations before 

they could begin developing and implementing corrective actions. 

DOE issued the final report in April of 2015. According to the 

operations activity manager for WIPP in the Carlsbad Field Office, 

completing the corrective actions in response to the report’s 

recommendations resulted in a 3-month delay.
Page 20 GAO-16-608 DOE WIPP Recovery

NWP officials said that in recognition of the need for a more effective risk 

management process, they revised their process as part of revising the 

recovery cost and schedules estimates. These officials explained, for 

example, the revised process now involves more frequent discussions 

and updating the status of potential risks with managers at WIPP who 

oversee the departments where the project risks are generated. We did 

not conduct an assessment of the effectiveness of this new risk 

management process.

DOE did not follow all best practices in analyzing and selecting an 

alternative for the new ventilation system at WIPP. As a result, DOE’s 

analysis was not reliable and DOE cannot be confident that the selected 

alternative will best provide the needed capabilities. Of the four categories 

of the best practices, DOE fully met the category for identifying 

alternatives. However, DOE partially or minimally met the other three 

categories of best practices—general principles, analyzing alternatives, 

and selecting the preferred alternative—because of key limitations. (See 

app. IV for the detailed results of our analysis of DOE’s AOA process.)

DOE’s AOA process fully met the best practices in the identifying 

alternatives category. To identify alternatives, DOE relied on analyses 

conducted by NWP, the contractor responsible for the project, and the 

second contractor—Trinity. These analyses were completed in January 

and October 2015, respectively. NWP and Trinity identified a broad range 

of alternatives. Specifically, they identified nine ventilation system 

alternatives that the contractors determined should be analyzed because 

the alternatives could meet the mission need and other screening criteria 

that they defined: NWP identified six and Trinity identified three. The 

alternatives included continued use of the existing exhaust shaft as well 

as constructing a new shaft or more than one shaft, such as a new 

exhaust shaft and a new air intake shaft. In addition, the studies identified 

different ventilation filtration capacities and modes of operation that 

included continuous filtering of all exhaust air or filtering air only when a 

radiological release is detected underground.

DOE’s AOA process partially met the best practices in the general 

principles category, which covers how the AOA process is planned, 

conducted, and documented. In particular, DOE partially or minimally 

followed the best practices of defining the mission need for the project, 

DOE Did Not Follow 

All Best Practices in 

Analyzing and 

Selecting an 

Alternative for the 

New Ventilation 

System at WIPP

DOE Fully Met Best 

Practices in the Identifying

Alternatives Category

DOE Partially Met Best 

Practices in the General 

Principles Category
Page 21 GAO-16-608 DOE WIPP Recovery

analyzing alternatives without a predetermined solution, and defining the 

functional requirements based on the mission need. As the AOA related 

to the best practices of defining the mission need for the project and 

analyzing alternatives without a predetermined solution, DOE did not 

define the mission need for the new ventilation system by focusing only 

on the capabilities needed for the project, but instead defined the need for 

the system to include a particular solution, which included constructing a 

new ventilation exhaust shaft. Because DOE defined the mission need by 

mentioning a new exhaust shaft, both contractors appeared to have 

analyzed alternatives with a preference for the alternatives that included 

constructing a new shaft. Specifically, NWP completed its analysis of the 

six alternatives by proposing two for further consideration by DOE, one 

that did not include a new shaft and one that did. NWP’s alternative that 

did not include a new shaft was its highest-scoring alternative and the 

alternative that included a new shaft was its third-highest scoring 

alternative. NWP officials told us that they proposed the second 

alternative along with the highest-scoring one because they believed the 

mission need statement discussed a new shaft. Regarding Trinity, in its 

final AOA report, Trinity stated that the mission need statement endorsed 

a new exhaust shaft. Trinity then analyzed three alternatives, one of 

which included using the existing shaft, and proposed an alternative to 

DOE that included constructing a new shaft. In its final assessment, DOE 

assessed the two alternatives proposed by NWP and the one proposed 

by Trinity and selected NWP’s alternative that included constructing a 

new shaft, shown as the second of the three alternatives in table 1. 

According to best practices, the AOA process should be an unbiased 

inquiry into the costs, benefits, and capabilities of all alternatives. 28 By 

conducting the AOA with a predetermined solution, DOE undermined the 

credibility of its final decision.

 28GAO-16-22.
Page 22 GAO-16-608 DOE WIPP Recovery

Table 1: Alternatives Proposed by Contractors to DOE for Further Evaluation for the New Permanent Ventilation System at the 

Waste Isolation Pilot Plant

Contractor that

proposed the alternative Description of alternative

Nuclear Waste Partnership LLC • Use existing exhaust shaft and add new filtration capacity sufficient for 

full mining and waste operations.

Nuclear Waste Partnership LLC • Use existing exhaust shaft and add new filtration capacity sufficient for 

waste operations and build new exhaust shaft for mining operations 

without filtration.

Trinity Engineering Associates • Build new exhaust shaft with filtration capacity sufficient for full mining 

and waste operations.

Source: Department of Energy (DOE) documents. | GAO-16-608

Note: Nuclear Waste Partnership LLC completed an analysis of six alternatives in January 2015 and 

proposed two to DOE for further evaluation. Trinity Engineering Associates completed an analysis of 

three alternatives in October 2015 and proposed one to DOE for further evaluation.

Another limitation under the general principles category was that DOE 

partially followed the best practice of defining the functional requirements 

that the ventilation system would need to satisfy. DOE did not consistently 

define two key functional requirements of the project—(1) the rate of 

airflow needed to support full operations at WIPP, and (2) the expected 

operational life of the new system—which limited the reliability of the 

overall AOA process as follows:

• Regarding the estimated airflow, DOE did not specify a functional 

requirement in its mission need statement for the rate of airflow that 

the new ventilation system must be capable of providing for full 

operations at WIPP. Instead, the mission need statement described 

the minimum airflow required by WIPP’s hazardous waste facility

permit, which was 260,000 cfm, and the actual airflow at WIPP before 

the accidents, which was 425,000 cfm. In conducting its AOA, NWP 

calculated its own airflow that it thought would be necessary for full 

operations at WIPP, which they estimated was about 540,000 cfm. 

NWP estimated the higher airflow based on its analysis of 

underground operations and used this amount to analyze a range of 

alternatives. In contrast, Trinity used the airflow associated with the 

WIPP hazardous waste facility permit, 260,000 cfm, as the estimated

airflow in conducting its AOA. The federal project director in the 

Carlsbad Field Office told us that he did not direct Trinity to redo its 

analysis using NWP’s higher airflow because he believed that airflow 

was needed for examining the scale of the system and would be 

defined lower during design. Moreover, he said that he wanted to 

maintain Trinity’s independence from NWP to be consistent with the 

December 2014 DOE requirement for conducting independent AOAs. 
Page 23 GAO-16-608 DOE WIPP Recovery

However, as a result of not defining the same functional airflow 

requirement for both contractors, DOE officials explained that they 

needed to conduct additional analysis to compare the alternatives 

proposed by NWP and Trinity.

• Regarding the estimated operational life of the new ventilation system, 

DOE did not specify a functional requirement in its mission need

statement for the estimated operational life of WIPP. Without such a 

requirement, NWP used 2030 as the estimated end date for WIPP 

operations—this date was based on DOE’s approved life-cycle plan 

for ending WIPP’s operations current at the time NWP conducted its 

analysis. NWP completed its AOA in January 2015 when it presented 

two alternatives to DOE for further evaluation—as described above, 

NWP had proposed these two alternatives from an initial group of six 

alternatives. In February 2015, EM revised the estimated operational 

life of WIPP, extending it from 2030 to 2050 to more accurately reflect 

DOE’s schedules for transuranic waste cleanup at DOE sites. Trinity 

did not start its AOA process until after DOE had revised the 

estimated operational life of WIPP to 2050 and therefore used the 

2050 date in conducting its AOA. In analyzing NWP’s two proposed 

alternatives, DOE used the new estimated operational life of 2050 and 

did not reassess the four previously eliminated alternatives using the 

revised date.

By not following the best practice of specifying functional requirements at 

the start of the AOA process, including specifying the airflow that the 

system needed to deliver and the estimated end date of operations for 

WIPP, it is unclear whether the AOAs conducted by NWP and Trinity and 

DOE’s subsequent analyses of these AOAs allowed DOE to select the 

alternative that best meets mission needs.
Page 24 GAO-16-608 DOE WIPP Recovery

DOE’s AOA process minimally met the best practices in the analyzing 

alternatives category. Significant limitations in this category included DOE 

and its contractors not consistently examining life-cycle costs for each 

alternative and not quantifying their benefits.29 Regarding examining lifecycle costs:

• NWP did not examine full life-cycle cost estimates for all of the 

alternatives it examined. Specifically, NWP’s AOA completed in 

January 2015 examined the estimated costs for the design and 

construction of each of the six alternatives, but did not examine the 

full life-cycles of these alternatives. As a result, NWP eliminated four 

of its six principal alternatives from further evaluation before 

examining each of them in terms of full life-cycle costs. According to 

the NWP project manager who led NWP’s AOA team, the team did 

not have sufficient time to examine the full life-cycle costs of each 

alternative. In addition, according to DOE officials, NWP was not 

required to examine the full life-cycle costs of each alternative under 

DOE Order 413.3B because they completed the analysis in January 

2015, which was several months before the Secretary’s June 2015 

directive to incorporate AOA best practices in the order. According to 

AOA best practices, the team conducting the AOA should be given 

enough time to complete the AOA process to ensure a robust and 

complete analysis.

• NWP and Trinity used different assumptions in developing life-cycle 

cost estimates, which prevented DOE from directly comparing the lifecycle estimates for the three proposed alternatives. We found that 

DOE relied on NWP and Trinity to define the assumptions they used 

rather than providing the contractors consistent guidance or direction. 

As a result, NWP assumed certain costs should be excluded in 

estimating the life-cycle costs of its two proposed alternatives. These 

costs included the costs for major equipment replacements or 

upgrades which NWP assumed would not be needed during the 30-

year operational life of the system; costs for providing facility security, 

quality assurance, and nuclear safety that are provided to all facilities 

at WIPP; and costs for the final closure of the new exhaust shaft after 

 29Under DOE Order 413.3B life-cycle costs cover all direct, indirect, recurring, 

nonrecurring, and other related costs incurred or estimated to be incurred in the planning, 

design, development, procurement, production, operations and maintenance, support, 

recapitalization, and final disposition of real property over its anticipated life span for every 

aspect of the program, regardless of funding source.

DOE Minimally Met Best 

Practices in the Analyzing 

Alternatives Category
Page 25 GAO-16-608 DOE WIPP Recovery

2050 when the system will be decommissioned. In contrast, Trinity 

assumed these costs should be included in estimating the life-cycle 

costs of its alternatives. As a result of including these additional costs, 

Trinity’s estimated life-cycle cost was substantially more than NWP’s 

estimate—Trinity’s estimate for its proposed alternative was $3.45 

billion and NWP’s estimates for its two proposed alternatives were 

$368.8 million and $467.6 million.

Regarding quantifying benefits, DOE and its contractors did not provide 

any measures of the benefits or effectiveness for each of the alternatives 

in their analyses. By not ensuring that the life-cycle costs of each 

alternative were developed in a consistent manner and not ensuring that 

the benefits or effectiveness of each alternative were quantified, DOE did 

not have an accurate and complete picture of all the alternatives to make 

reliable comparisons between them.

DOE’s AOA process partially met the best practices in the selecting a 

preferred alternative category which covers selecting an alternative that 

best meets the mission need and also independently reviewing the overall 

AOA process. Most notably, DOE and its contractors did not follow the 

best practice to select the alternatives based on a cost-benefit analysis 

and only partially followed the best practice to independently review the 

AOA process, as follows:

• Regarding cost-benefit analysis, DOE and its contractors did not 

select the alternatives based on such an analysis which compares the 

life-cycle costs and benefits or effectiveness of each alternative.30 As 

noted above, DOE and its contractors did not consistently examine 

life-cycle cost estimates for each alternative and did not quantify their 

benefits. Therefore, DOE did not produce the information needed for a 

cost-benefit analysis. By not selecting the alternatives based on a 

cost-benefit analysis, DOE did not adequately justify that the selected 

final alternative would best provide the capabilities needed at WIPP.

 30Specifically, a cost-benefit analysis compares alternatives using the net present value of 

the estimated life-cycle costs and benefits or effectiveness of each alternative. The 

present value of an estimate reflects the time value of money, the concept that a dollar in 

the future is worth less than a dollar today because the dollar today can be invested and 

earn interest. The present value of an estimate is calculated using an interest rate called a 

discount rate. 

DOE Partially Met Best 

Practices in the Selecting 

a Preferred Alternative 

Category
Page 26 GAO-16-608 DOE WIPP Recovery

• Regarding independently reviewing the AOA process, DOE conducted 

an independent review of the AOA process (which assessed whether 

best practices were followed) but did not implement the 

recommendation identified by the review. Specifically, an independent 

review conducted by DOE’s Office of Project Management Oversight 

and Assessments found that the project team did not adequately 

document a cost-benefit analysis and that, as a result, the selection of 

the preferred alternative was not supported by compelling information. 

The review recommended that DOE perform a cost-benefit analysis to 

be consistent with best practices and support the selection of the final 

alternative. However, DOE and NWP did not conduct the 

recommended cost-benefit analysis and document it before DOE 

selected the final alternative. According to the DOE officials who led 

the independent review of the AOA process, the project team was not 

required by DOE’s Order 413.3B to implement the recommendation or 

justify and document the reasons for not doing so. Therefore, DOE 

EM approved the selection of the preferred alternative for the 

permanent ventilation system project in December 2015 without 

implementing the independent review’s recommendation.

Conducting an independent review of the AOA is a best practice because 

it is one of the most reliable means to ensure that bias does not influence 

the AOA process and that the AOA is sufficiently thorough to ensure that 

a preferred solution is chosen and not a favored solution. DOE’s 

independent review was an internal control to help DOE meet its goal for 

the capital asset project, in this case selecting the alternative that will best 

provide the capabilities to meet the mission need. Under federal 

standards for internal control, management is to evaluate and document 

internal control issues and determine appropriate corrective actions.31 By 

not implementing the recommendation from the independent review to do 

a cost-benefit analysis or not justifying and documenting the reason for 

not doing so, DOE cannot provide assurance that the final alternative 

selected would best provide the capabilities needed at WIPP.

Restarting WIPP’s waste disposal operations is a top priority for DOE, 

and it has made progress in its efforts to restart limited waste disposal 

operations. However, DOE did not meet its initial cost and schedule 

 31GAO, Standards for Internal Control in the Federal Government, GAO-14-704G

(Washington, D.C.: Sept. 10, 2014).

Conclusions
Page 27 GAO-16-608 DOE WIPP Recovery

estimates to restart operations and incurred a cost increase of about $64 

million—$2 million to the recovery project and $61.6 million in base 

operations—and a delay of nearly 9 months. DOE incurred the cost 

increase and schedule delay, in part, because DOE’s initial estimates did 

not follow all best practices and were therefore unreliable. Notably, DOE 

did not include any contingency in its schedule, giving itself less than a 1 

percent chance of success—said another way DOE gave itself a 99 

percent chance of failure in meeting its schedule to restart operations. 

Moreover, when DOE revised its cost and schedule estimates for WIPP 

recovery it still did not follow all best practices. DOE did not develop its 

estimates for WIPP recovery following best practices, in part because

DOE does not require its cleanup operations activities, such as WIPP, to 

follow them. This lack of a requirement is in contrast to a new policy put 

into effect in June 2015 by the Secretary of Energy, which established the 

requirements that DOE develop cost and schedule estimates for its 

capital asset projects following best practices—such projects include the 

new permanent ventilation system at WIPP. Without similar requirements 

for EM operations activities to follow best practices, DOE cannot have 

confidence that it is producing reliable baselines needed to monitor its 

performance in managing these activities and reduce the risk of cost 

overruns and delays in achieving its cleanup missions, such as the 

permanent disposal of transuranic waste from DOE sites across the 

country. By also requiring cleanup operations to follow best practices, 

DOE would have more confidence in the estimates for its cleanup 

operations activities and its capital asset projects.

In selecting the preferred alternative for the new permanent ventilation 

system at WIPP, DOE relied on an AOA process that did not follow all 

best practices. In particular, DOE only partially followed or did not follow 

best practices to define functional requirements based on the mission 

need, compare alternatives using a cost-benefit analysis, or 

independently review the AOA process. By not following these and other 

practices, DOE’s AOA process was not reliable. Notably, DOE conducted 

an independent review of the process which found that the project team 

did not adequately document a cost-benefit analysis and that, as a result, 

the selection of the preferred alternative was not supported by compelling 

information. Nonetheless, DOE did not implement the recommendation of 

the review to conduct a cost-benefit analysis before the department 

selected the preferred alternative. DOE officials explained that the project 

team was not required by DOE’s Order 413.3B to implement the 

recommendation. By not implementing the recommendation of the 

independent review or not justifying and documenting the reason for not 

doing so, DOE cannot provide assurance that the final selected 
Page 28 GAO-16-608 DOE WIPP Recovery

alternative would best provide the capabilities needed at WIPP. More 

broadly, without requiring in its Order 413.3B that recommendations from 

independent reviews of AOAs be implemented or that the reason for not 

doing so be justified and documented, DOE cannot have assurance that it 

selects, for all capital asset projects, preferred alternatives from its AOAs 

that best meet the mission need.

To help ensure that DOE develops and uses reliable cost and schedule 

estimates and AOAs, we recommend that the Secretary of Energy take 

the following three actions:

• Direct EM to revise its protocol governing cleanup operations activities 

to require use of best practices in developing cost and schedule 

estimates.

• Direct EM to implement the recommendation made by DOE’s Office of 

Project Management Oversight and Assessments in its independent 

review of the AOA for WIPP’s new permanent ventilation system to 

perform a cost-benefit analysis consistent with best practices for 

conducting an AOA, or justify and document why the office does not 

intend to do so.

• Direct DOE to revise its Order 413.3B to require that DOE offices 

implement any recommendations from an independent review of the 

extent to which an AOA followed best practices, or justify and 

document the rationale for not doing so.

We provided a draft of this report to DOE for review and comment. In 

written comments, reproduced in appendix V, DOE concurred with the 

report’s recommendations. DOE stated that our recommendations were 

consistent with its commitment to continuous improvement in project 

management. In addition, DOE stated that it had two issues that the 

department believed needed to be addressed. DOE also provided 

technical comments that were incorporated, as appropriate. 

DOE identified the following two issues:

• DOE stated that WIPP recovery activities were under way concurrent 

with DOE-wide efforts to improve project management, including 

revisions to its project management order (DOE Order 413.B). DOE

stated that, consequently, EM project management practices were 

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
Page 29 GAO-16-608 DOE WIPP Recovery

separate from evolving department project management guidance

and that it is now in the process of bringing EM practices into 

conformance with the department’s project management guidance. 

• DOE stated that the report needs to be clear, in each instance, 

regarding the $64 million cost increase for WIPP attributed to 

schedule delays in recovery activities, to indicate that $2 million of the 

cost increase was WIPP recovery project-related and that $61.6 

million was due to reexamination and assessment of the cost of base 

operations. 

We do not believe changes are needed to the report for either of these 

issues. The report explains that WIPP recovery activities were based on 

EM’s operations activities protocol and that actions were under way within 

the department to improve project management, including revisions to 

DOE’s project management order. In addition, the report includes a 

breakout of the cost increase. We did, however, revise the conclusions to 

reflect this breakout. 

As noted above, DOE concurred with the report’s three 

recommendations. Regarding the first recommendation—that EM revise 

its protocol governing cleanup operations activities to require use of best 

practices in developing cost and schedule estimates—DOE stated in its 

written comments that it concurred with clarification. DOE stated that EM 

is transitioning from the operations activities protocol to a new directive 

that is expected to include a key decision approving a cost and schedule 

baseline. As EM develops the guidance for this key decision, it will 

include the use of cost and schedule best practices. DOE stated that EM

plans to finalize this new directive by September 2016 and seek 

departmental approval by December 2016. 

Regarding the second recommendation— that EM implement the 

recommendation made by DOE in its independent review of the AOA for 

WIPP’s new ventilation system to perform a cost-benefit analysis 

consistent with best practices, or justify and document why it does not 

intend to do so—DOE stated in its written comments that it concurred with 

clarification. DOE stated that in accordance with GAO best practices, 

further cost-benefit analysis will be conducted on the project prior to 

approval of Critical Decision-2 (Approve Performance Baseline). DOE 

stated that several alternatives remain to be evaluated including the size 

of the ventilation system and the location of the exhaust shaft. 
Page 30 GAO-16-608 DOE WIPP Recovery

Regarding the third recommendation—that DOE revise its project 

management (Order 413.3B) to require that DOE offices implement 

recommendations from independent reviews of the extent to which an 

AOA followed best practices, or justify and document the rationale for not 

doing so—DOE concurred with the recommendation. DOE stated that it

will prepare a project management policy on how DOE offices should 

respond to recommendations from independent reviews. DOE stated that 

it will prepare this policy by December 2016 and update DOE Order 

413.3B at the next available opportunity. 

We are sending copies of this report to the appropriate congressional 

committees, the Secretary of Energy, and other interested parties. In 

addition, the report is available at no charge on the GAO website at 

http://www.gao.gov.

If you or your staff have any questions about this report, please contact 

me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made key contributions to this report 

are listed in appendix VI.

David C. Trimble

Director, Natural Resources and Environment
Page 31 GAO-16-608 DOE WIPP Recovery

List of Committees

The Honorable John McCain

Chairman

The Honorable Jack Reed

Ranking Member

Committee on Armed Services

United States Senate

The Honorable Lamar Alexander

Chairman

The Honorable Dianne Feinstein

Ranking Member

Subcommittee on Energy and Water Development

Committee on Appropriations

United States Senate

The Honorable Mac Thornberry

Chairman

The Honorable Adam Smith

Ranking Member

Committee on Armed Services

House of Representatives

The Honorable Mike Simpson

Chairman

The Honorable Marcy Kaptur

Ranking Member

Subcommittee on Energy and Water Development

Committee on Appropriations

House of Representatives
Appendix I: Objectives, Scope, and 

Methodology

Page 32 GAO-16-608 DOE WIPP Recovery

Our report examined the extent to which the Department of Energy (DOE)

(1) met its initial cost and schedule estimates for restarting waste disposal 

operations at the Waste Isolation Pilot Plant (WIPP) and (2) followed best 

practices in analyzing and selecting an alternative for the new ventilation 

system. To address both objectives, we conducted a site visit to WIPP in 

January 2015. During the site visit we obtained documentation and 

interviewed officials from DOE’s Office of Environmental Management 

(EM), which is responsible for oversight of WIPP and exercises this 

responsibility primarily through its Carlsbad Field Office. We also 

interviewed representatives at WIPP from Nuclear Waste Partnership 

LLC (NWP), which is the private contractor that manages and operates 

WIPP for DOE.

To examine the extent to which DOE met its initial cost and schedule 

estimates for restarting waste disposal operations, we compared DOE’s 

initial cost and schedule estimates to restart limited waste disposal 

operations contained in its February 2015 WIPP recovery project 

management baseline with DOE’s revised estimates in its January 2016 

integrated project management baseline for WIPP. We reviewed DOE’s 

budget justification supporting the President’s fiscal year 2016 budget 

request to Congress from February 2015 and the amounts DOE received 

for WIPP in the 2016 Consolidated Appropriations Act.1 We reviewed 

DOE’s reports on the reasons it exceeded the initial estimates for 

restarting waste disposal operations and DOE’s risk management plans 

for WIPP recovery prepared by NWP and used in developing the 

February 2015 estimates. We also compared DOE’s initial cost and 

schedule estimates to restart limited waste disposal operations from the 

February 2015 WIPP recovery project management baseline with the 

best practices described in our cost and schedule guides that identified 

the characteristics of high-quality, reliable cost and schedule estimates 

because these were the approved estimates when we conducted our 

analysis.2 Specifically, we compared DOE’s initial WIPP recovery cost 

estimate presented in DOE’s project management baseline document 

and supporting documents and data with the best practices in our Cost 

 1

Pub. L. No. 114-113 (2015).

2

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009) and 

GAO Schedule Assessment Guide: Best Practices for Project Schedules, GAO-12-120G

(Washington, D.C.: May 2012). 

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 33 GAO-16-608 DOE WIPP Recovery

Estimating and Assessment Guide. In addition, we compared DOE’s 

initial WIPP recovery schedule presented in DOE’s project management 

baseline document and supporting documents and data with the best 

practices in our Schedule Assessment Guide. We interviewed the 

Carlsbad Field Office officials who oversee the recovery project and 

NWP’s cost estimator and scheduler. We provided a draft of our cost and 

schedule assessments to the Carlsbad Field Office and NWP and revised 

the draft, as appropriate, after discussing our assessment with the federal 

officials and the contractor. We reviewed documentation on the revised 

(January 2016) baseline and interviewed DOE and NWP officials on the 

approach followed to develop the revised cost and schedule estimates to 

restart WIPP’s waste disposal operations in the baseline, but we did not 

assess the revised estimates against the best practices because of the 

time frame of our review.

To examine the extent to which DOE followed best practices in analyzing 

and selecting an alternative for WIPP’s new ventilation system, we used 

as criteria the best practices for conducting an AOA identified in 

GAO-15-37 issued in December 2014.3 GAO developed the best 

practices identified in this report by reviewing AOA policies and guidance 

used by seven public and private-sector entities with experience in the 

AOA process, and verified these practices with subject-matter experts. 

DOE’s AOA process for the WIPP ventilation system consisted of three 

elements: NWP’s January 2015 AOA that resulted in the selection of two 

preferred alternatives proposed for further analysis, a second AOA 

completed by Trinity Engineering Associates (Trinity) in October 2015 

which resulted in a single preferred alternative proposed to DOE, and 

DOE’s final alternative evaluation process led by the Carlsbad Field 

Office that considered the three alternatives proposed by the initial 

studies. Our analysis assessed the overall AOA process considering each 

element. We compared the process with the best practices and 

determined a score for the overall process. We reviewed project 

documentation from the Carlsbad Field Office, NWP, and Trinity and 

interviewed Carlsbad Field Office officials and NWP representatives in 

charge of the AOA. In addition, in October 2015, as we were conducting 

our engagement, EM and DOE’s Office of Project Management Oversight 

and Assessments completed separate assessments of the WIPP AOA for 

 3

GAO, DOE and NNSA Project Management: Analysis of Alternatives Could Be Improved 

by Incorporating Best Practices, GAO-15-37 (Washington, D.C.: Dec. 11, 2014). 
Appendix I: Objectives, Scope, and 

Methodology

Page 34 GAO-16-608 DOE WIPP Recovery

the new ventilation system. We reviewed documentation of these reviews 

and interviewed the DOE officials who worked on them. We examined the 

extent that the independent assessments followed the best practice to 

have an independent entity assess the extent that a project’s AOA 

followed all best practices.

To score DOE’s AOA process, a GAO analyst examined the AOA 

documentation received from the agency and then assigned a score for 

each of the 24 best practices. Following this, a GAO AOA specialist 

independent of the engagement team reviewed the AOA documentation 

and the scores assigned by the analyst for accuracy and cross-checked 

the scores in all the analyses for consistency. We used a five-point 

scoring system to determine the extent to which DOE’s AOA process 

conformed to the best practices.4 After determining a score for each 

individual best practice, we calculated the score for each category—(1) 

general principles, (2) identifying alternatives, (3) analyzing alternatives, 

and (4) selecting a preferred alternative—by calculating the average of 

the scores for the best practices that fall under each category. If the score 

for each best practice and the average score for each category was “fully 

met” or “substantially met,” we concluded that the AOA process 

conformed to best practices and therefore could be considered reliable. In 

contrast, if the score was “partially met,” “minimally met,” or “not met,” we 

concluded that the AOA process did not conform to best practices and 

therefore could not be considered reliable. We shared our analysis with 

DOE officials and representatives from NWP for review and incorporated 

their technical comments and any additional evidence they provided in 

our analysis, as appropriate. We also interviewed officials from DOE’s 

 4

The five-point scoring system was based on the scoring system we have used to assess 

other federal programs and projects’ cost and schedule estimates against best practices 

published in our Cost and Schedule guides. The scoring system does not weight cost 

estimating, scheduling, and AOA best practices because it is not possible to quantitatively 

determine the relative weights of each of the criteria. Weighting has the potential to vary 

across programs, as well as due to where a program may be in its different stages of its 

life cycle. Therefore, the fairest standardized methodology to evaluate programs is to have 

all criteria weighted equally. The system we used was as follows: “fully met” (i.e., five 

points) means that DOE’s documentation demonstrated that DOE completely met the best 

practice; “substantially met” (i.e. four points) means that DOE’s documentation 

demonstrated that DOE met a large portion of the best practice; “partially met” (i.e., three 

points) means that DOE’s documentation demonstrated that DOE met about half of the 

best practice; “minimally met” (i.e., two points) means that DOE’s documentation 

demonstrated that DOE met a small portion of the of the best practice; and “did not meet”

(i.e., one point) means that DOE’s documentation did not demonstrate that DOE met the 

best practice. 
Appendix I: Objectives, Scope, and 

Methodology

Page 35 GAO-16-608 DOE WIPP Recovery

Office of Enterprise Assessments, which provides internal oversight of 

DOE facilities; the Defense Nuclear Facilities Safety Board, which 

provides external oversight of DOE defense nuclear facilities; the U.S. 

Mine Safety and Health Administration, which provides external oversight 

of mining activities at WIPP; as well as the U.S. Environmental Protection 

Agency and the New Mexico Environment Department, both of which 

provide external regulation of WIPP.

We conducted this performance audit from November 2014 to August

2016 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Assessment of DOE’s Cost 

Estimate for the WIPP Recovery Project 

Compared with Best Practices

Page 36 GAO-16-608 DOE WIPP Recovery

Table 2 below assesses the Department of Energy’s (DOE) initial cost 

estimate in its February 2015 project management baseline for the Waste 

Isolation Pilot Plant (WIPP) recovery project compared to best practices 

for cost estimating published in the GAO Cost Estimating and 

Assessment Guide.1 Overall, of four characteristics of a high-quality and 

reliable cost estimate, DOE’s initial cost estimate substantially met two 

characteristics—comprehensive and well-documented—but partially met 

or minimally met the other two characteristics—accurate and credible.

Table 2: Assessment of DOE’s February 2015 Cost Estimate for the WIPP Recovery Project Compared with Industry Best 

Practices

Best practice characteristic 

and overall assessment Best practice Detailed assessmenta

Comprehensive:

Substantially met

The cost estimate includes all life-cycle 

costs. 

Substantially met.

The cost estimate prepared by the Department of Energy’s 

(DOE) Waste Isolation Pilot Plant (WIPP) contractor—

Nuclear Waste Partnership LLC (NWP)—on DOE’s behalf, 

was not a life-cycle cost estimate. However, the estimate 

covered the costs for all activities in DOE’s WIPP recovery 

plan to restart limited waste disposal operations and also 

covered the order of magnitude cost range for construction 

of the new permanent ventilation system needed to return 

WIPP to full operations. In addition, DOE issued a directive 

to NWP to update the life-cycle cost estimate for WIPP to 

account for increased operating costs at WIPP.

The cost estimate completely defines 

the program, reflects the current 

schedule, and is technically reasonable. 

Substantially met. The cost estimate was based on DOE’s 

WIPP recovery plan document, prepared on its behalf by 

NWP, and which established the technical baseline for the 

work. Evidence was provided to show Carlsbad Field Office 

approval of the estimate, but none was shown to support 

approval by DOE headquarters.

The cost estimate work breakdown 

structure is product-oriented, traceable 

to the statement of work/objective, and 

at an appropriate level of detail to 

ensure that cost elements are neither 

omitted nor double-counted. 

Substantially met. The work breakdown structure showed 

the work of the project, and used an appropriate level of 

detail to ensure that no costs were omitted. However, the 

work breakdown structure may not have adequately 

captured sustainment costs.

The estimate documents all costinfluencing ground rules and 

assumptions. 

Fully met. Ground rules and assumptions were 

documented in the cost estimate. For example, the 

estimate documented assumptions such as funding support 

and constraints as well as technical and programmatic 

risks.

 1

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 

Appendix II: Assessment of DOE’s Cost 

Estimate for the WIPP Recovery Project 

Compared with Best Practices
Appendix II: Assessment of DOE’s Cost 

Estimate for the WIPP Recovery Project 

Compared with Best Practices

Page 37 GAO-16-608 DOE WIPP Recovery

Best practice characteristic 

and overall assessment Best practice Detailed assessmenta

Well-documented:

Substantially met

The documentation captures the source 

data used, the reliability of the data, and 

how the data were normalized. 

Substantially met. The cost estimate was based on 

historical data for NWP work at WIPP as well as vendor 

quotes for recovery tasks. However, it was not possible to 

trace all vendor quotes to the estimate.

The documentation describes in 

sufficient detail the calculations 

performed and the estimating 

methodology used to derive each 

element’s cost. 

Partially met. Some portions of the estimating 

methodology were broken out and documented, but there 

were gaps in the documentation. For instance, in some 

cases, material costs were not broken out separately but 

were instead rolled up into an overall cost number.

The documentation describes, step by 

step, how the estimate was developed 

so that a cost analyst unfamiliar with the 

program could understand what was 

done and replicate it. 

Partially met. The documentation provided included 

narrative descriptions, an executive summary, introduction, 

risk and uncertainty analysis, and steps taken to estimate 

costs. But it did not explain whether a sensitivity analysis 

was conducted or how the estimate would be updated to 

reflect actual costs and changes. Also, the documentation 

provided was not at a level of detail that would allow for a 

full replication of the estimate.

The documentation discusses the 

technical baseline description and the 

data in the baseline are consistent with 

the estimate. 

Fully met. The technical baseline description came from 

DOE’s WIPP Recovery Plan and the estimate was 

consistent with this document.

The documentation provides evidence 

that the cost estimate was reviewed and 

accepted by management. 

Substantially met. Documentation was provided that 

showed management approval of the estimate, including 

resolution of comments on earlier drafts of the estimate. 

However, the documentation did not include estimate 

details and cost contingency.

Accurate:

Partially met

The cost estimate results are unbiased, 

not overly conservative or optimistic, 

and based on an assessment of most 

likely costs. 

Minimally met. A clear link between the risk and 

uncertainty analysis and the cost estimate was not 

presented, and therefore we were unable to determine 

what the level of confidence was for the point estimate.

The estimate has been adjusted 

properly for inflation. 

Partially met. Costs were developed in constant year 

dollars and then escalated to future year dollars. 

Documentation was not sufficient to determine if historical 

data were adjusted for inflation and normalized to constant 

year dollars.

The estimate contains few, if any, minor 

mistakes. 

Partially met. We found inconsistencies between the final 

documented estimate and the electronic cost model. As a 

result, we were unable to fully assess whether the estimate 

contained any errors. Spot checks did not specifically 

reveal any double-counting or omissions but these 

problems could not be ruled out given the inability to trace 

the estimate from the cost model.

The cost estimate is regularly updated 

to reflect significant changes in the 

program so that it always reflects 

current status. 

Substantially met. We found that the cost estimate had 

been updated to reflect changes and program 

assumptions. In addition, an earned value management 

system is in the process of being implemented, which will 

assist with keeping the estimate current.
Appendix II: Assessment of DOE’s Cost 

Estimate for the WIPP Recovery Project 

Compared with Best Practices

Page 38 GAO-16-608 DOE WIPP Recovery

Best practice characteristic 

and overall assessment Best practice Detailed assessmenta

Variances between planned and actual 

costs are documented, explained, and 

reviewed. 

Partially met. Documentation was provided which 

described cost variances and contained information on the 

impact to the project as well as corrective actions. 

However, in some cases, the documentation did not allow 

us to directly link specific project problems with their 

corresponding cost variance.

The estimate is based on a historical 

record of cost estimating and actual 

experiences from other comparable 

programs. 

Partially met. Most of the raw historical data used to 

develop the estimate were not provided. Some data were 

provided from WIPP operations prior to the shutdown of 

operations in February 2014. However, these data were not 

readily reviewable by a third party

The estimating technique for each cost 

element was used appropriately. 

Not met. NWP did not have documentation that would 

have allowed us to determine if data provided by subjectmatter experts were properly adjusted for bias, which 

prevented us from determining whether historical data were 

adjusted correctly to reflect current circumstances.

Credible:

Minimally met

The cost estimate includes a sensitivity 

analysis that identifies a range of 

possible costs based on varying major 

assumptions, parameters, and data 

inputs. 

Not met. A sensitivity analysis was not conducted for this 

cost estimate. 

A risk and uncertainty analysis was 

conducted that quantified the 

imperfectly understood risks and 

identified the effects of changing key 

cost driver assumptions and factors. 

Substantially met. Separate risk analyses were conducted 

for technical and programmatic risk, estimate uncertainty, 

and schedule uncertainty. However, probabilities for the 

nonrisk-adjusted point estimate were not reported.

Major cost elements were crosschecked to see whether results were 

similar. 

Not met. Major cost elements were not cross checked with 

alternative methodologies to see if results are similar.

An independent cost estimate was 

conducted by a group outside the 

acquiring organization to determine 

whether other estimating methods 

produce similar results. 

Minimally met. An independent cost estimate was not 

conducted. Documentation was provided showing a review 

from another team within the same contractor organization. 

However, the work of this other team was not independent 

and did not develop a separate estimate using the same 

ground rules and assumptions.

Source: GAO analysis of the WIPP recovery project management baseline cost estimate approved by DOE in February 2015. | GAO-16-608

a

The ratings we used in this analysis are as follows: “Fully met” means DOE provided complete 

evidence that satisfies the entire best practice. “Substantially met” means DOE provided evidence 

that satisfies a large portion of the best practice. “Partially met” means DOE provided evidence that 

satisfies about half of the best practice. “Minimally met” means DOE provided evidence that satisfies 

a small portion of the best practice. “Not met” means DOE provided no evidence that satisfies the 

best practice.
Appendix III: Assessment of DOE’s Schedule 

for the WIPP Recovery Project Compared with 

Best Practices

Page 39 GAO-16-608 DOE WIPP Recovery

Table 3 below assesses the Department of Energy’s (DOE) initial 

schedule in its February 2015 project management baseline for the 

Waste Isolation Pilot Plant (WIPP) recovery project compared to best

practices for developing a schedule estimate published in the GAO 

Schedule Assessment Guide.1 Overall, DOE’s schedule substantially met 

two of the four characteristics of a high-quality schedule—comprehensive 

and controlled—but partially met the characteristics of well-constructed 

and credible.

Table 3: Assessment of DOE’s February 2015 Schedule for the WIPP Recovery Project Compared with Industry Best Practices

Best practice characteristic 

and overall assessment Best practice Detailed assessmenta

Comprehensive:

Substantially met

Capturing all activities. Substantially met. The Department of Energy’s (DOE) 

schedule, prepared by its Waste Isolation Pilot Plant (WIPP) 

contractor—Nuclear Waste Partnership LLC (NWP)—reflected 

the activities in the work breakdown structure—which defines 

the work necessary to accomplish a project’s objectives. Key 

milestones were captured and identified and work scope was 

assigned as the responsibility of either NWP or DOE. 

However, we found many instances of repeated activity names 

and descriptions in the schedule which may obscure the 

required work for each activity.

Assigning resources to all activities. Partially met. The schedule included costs and the total cost 

given in the schedule was largely aligned with the total 

budgeted cost. However less than 10 percent of the remaining 

activities in the schedule had resources assigned to them. No 

evidence was provided that the schedule was used to explore 

possible resource availability conflicts.

Establishing the durations of all 

activities.

Substantially met. Durations were established for activities 

using the most likely estimate, according to NWP officials. Our 

analysis found that 90 percent of the activities in the recovery 

schedule were less than 2 working months in duration, 

representing manageable pieces of work.

Well-constructed:

Partially met

Sequencing all activities. Partially met. We found that the schedule predominately used 

finish-to-start logic, which clearly indicated the activities that 

were supposed to finish before others begin, but we also found 

a significant number of missing logic links between activities. 

Without these links, activities that slip early in the schedule 

would not transmit delays to activities that should depend on 

them. 

 1

GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules, 

GAO-12-120G (Washington, D.C.: May 2012). 

Appendix III: Assessment of DOE’s Schedule 

for the WIPP Recovery Project Compared 

with Best Practices
Appendix III: Assessment of DOE’s Schedule 

for the WIPP Recovery Project Compared with 

Best Practices

Page 40 GAO-16-608 DOE WIPP Recovery

Best practice characteristic 

and overall assessment Best practice Detailed assessmenta

Confirming that the critical path is 

valid.

Partially met. A critical path was identified for the schedule. 

According to NWP officials, it was used to track the progress of 

the project. The critical path contained some long duration 

activities which should have been reevaluated to determine if 

they could have been broken up into more manageable 

pieces. Additionally, due to the problems with logic links 

between activities discussed in the best practice above (i.e., 

sequencing all activities), we were unable to validate the 

reliability of the critical path.

Ensuring reasonable total float. Partially met. The schedule included large total float values 

that probably did not represent the actual degree of flexibility in 

the schedule. NWP officials stated they would address the 

schedule logic and the large total float values as the work 

progressed and more was known about the activities.

Credible:

Partially met

Verifying that the schedule is 

traceable horizontally and vertically.

Partially met. We found that the schedule was traceable 

vertically, with all levels of detail of the schedule derived from 

the same integrated master schedule. However, the schedule 

was not fully traceable horizontally, in particular because of the 

activities with missing logic links discussed in the above best 

practice. 

Conducting a schedule risk analysis. Minimally met. A schedule risk analysis was performed. 

However, according to NWP officials, it relied on a prior 

version of the schedule. Therefore it was outdated. In addition, 

the analysis focused on the activities on the critical path only, 

whereas it should have focused on all work activities. DOE 

also did not use the risk analysis to determine a risk-adjusted 

project completion date. Specifically, the completion date it 

selected, March 31, 2016, did not include contingency to 

account for the known risks that DOE analyzed. The 

completion date, in fact, had a less than 1 percent confidence 

level, or chance of success, in being achieved.

Controlled:

Substantially met

Updating the schedule with actual 

progress logic.

Substantially met. NWP officials stated that many activities 

had their status checked either daily or weekly and that the 

critical path was updated weekly by trained project schedulers. 

Officials provided an example of a monthly schedule report, 

but it did not address logic or critical path changes.

Maintaining a baseline schedule. Substantially met. The documentation provided showed that 

the baseline schedule was being maintained through a change 

control process and used as the basis for measuring 

performance. However, NWP officials told us that they do not 

have the schedule basis documentation that is essential for 

validating and defending the baseline schedule.

Source: GAO analysis of the WIPP recovery project schedule estimate approved by DOE in February 2015. | GAO-16-608

a

The ratings we used in this analysis are as follows: “Fully met” means DOE provided complete 

evidence that satisfies the entire best practice. “Substantially met” means DOE provided evidence 

that satisfies a large portion of the best practice. “Partially met” means DOE provided evidence that 

satisfies about half of the best practice. “Minimally met” means DOE provided evidence that satisfies 

a small portion of the best practice. “Not met” means DOE provided no evidence that satisfies the 

best practice.
Appendix IV: Assessment of DOE’s Analysis of 

Alternatives for the WIPP Ventilation System 

Project Compared with Best Practices

Page 41 GAO-16-608 DOE WIPP Recovery

Table 4 below compares the Department of Energy’s (DOE) analysis of 

alternatives (AOA) process completed in December 2015 for the new 

permanent ventilation system at the Waste Isolation Pilot Plant (WIPP) to 

best practices for conducting an AOA published in a December 2014 

GAO report.1 DOE’s overall AOA process consisted of an initial AOA by 

DOE’s WIPP contractor, Nuclear Waste Partnership LLC, completed in 

January 2015; a second AOA by a contractor unaffiliated with WIPP, 

Trinity Engineering Associates, completed in October 2015; and DOE’s 

final evaluation of the three preferred alternatives proposed by the 

contractors’ analyses. Overall, DOE’s AOA process fully met the category 

for identifying alternatives and partially or minimally met the other three 

categories of best practices—general principles, analyzing alternatives, 

and selecting the preferred alternative.

 1

GAO, DOE and NNSA Project Management: Analysis of Alternatives Could Be Improved 

by Incorporating Best Practices, GAO-15-37 (Washington, D.C.: Dec. 11, 2014). GAO 

developed the best practices identified in this report by reviewing AOA policies and 

guidance used by seven public and private-sector entities with experience in the AOA 

process, and verified these practices with subject-matter experts. 

Appendix IV: Assessment of DOE’s Analysis 

of Alternatives for the WIPP Ventilation 

System Project Compared with Best 

Practices
Appendix IV: Assessment of DOE’s Analysis of 

Alternatives for the WIPP Ventilation System 

Project Compared with Best Practices

Page 42 GAO-16-608 DOE WIPP Recovery

Table 4: Assessment of DOE’s Analysis of Alternatives (AOA) for the WIPP Permanent Ventilation System Project Compared 

with AOA Best Practices

Best practice

category and scorea Best practice Detailed assessmenta

General principles:

Partially met

1. The customer defines the mission

need without a predetermined 

solution.

Partially met. The Department of Energy’s (DOE) mission need 

memorandum for the Waste Isolation Pilot Plant (WIPP) new 

permanent ventilation system defined the capabilities needed for 

the system’s equipment without specifying a particular 

configuration. However, the memorandum suggested that a new 

shaft was needed. 

2. The customer defines functional 

requirements based on the 

mission need and without a 

predetermined solution.

Partially met. DOE’s mission need memorandum defined highlevel functions but DOE did not develop a functional requirements 

document with detailed definitions of the requirements and analysis 

of how they were based on the mission need. DOE did not clearly 

define two requirements in its mission need memorandum: the total 

airflow needed to support full WIPP operations and end date for 

WIPP’s operating life. As a result, DOE’s AOA teams—one 

organized by its WIPP contractor, Nuclear Waste Partnership LLC 

(NWP) and a second organized by a contractor unaffiliated with 

WIPP, Trinity Engineering Associates (Trinity)—did not use the 

same airflows to analyze alternatives and did not consistently use 

the same end date for WIPP’s operating life in analyzing all 

alternatives. DOE’s final conceptual design report defined the 

safety-related functional requirements based on the mission need 

that would apply to all three preferred alternatives.

3. The customer provides the team 

conducting the AOA with enough 

time to complete the AOA process

to ensure a robust and complete 

analysis.

Minimally met. DOE started the AOA process in October 2014 and 

completed it in December 2015. However, DOE did not ensure that 

its AOA teams had the time to estimate both the life-cycle costs 

and benefits of each alternative. In addition, DOE did not take the 

time to implement recommendations made by its own independent 

review of the AOA process. 

4. The team includes members with 

diverse areas of expertise 

including, at a minimum, subjectmatter expertise, project 

management, cost estimating, and 

risk management.

Partially met. DOE’s AOA teams assembled by its contractors 

provided documentation that indicated they included expertise in 

relevant technical and regulatory subject matters such as nuclear 

safety, mine safety, and mine and nuclear ventilation. The final 

AOA reports did not include resumes to show the level of expertise. 

NWP’s AOA did not include expertise in cost estimating and risk 

management.

5. The team creates a plan, including 

proposed methodologies, for 

identifying, analyzing, and 

selecting alternatives, before 

beginning the AOA process.

Partially met. DOE’s AOA teams did not have detailed planning 

documents before starting the AOA process that identified the 

methodologies to be followed for identifying, analyzing, and 

selecting the alternatives. DOE did not have a documented plan for 

integrating the two initial AOAs by its contractors and conducting 

the final evaluation process.

6. The team documents all steps 

taken to identify, analyze, and 

select alternatives in a single 

document. 

Partially met. DOE’s AOA reports, including its final conceptual 

design report, documented many steps completed to identify, 

analyze, and select the final alternative. Certain key steps were not 

well-documented such as the WIPP contractor’s analysis that 

supported selection of two preferred alternatives in its January 

2015 AOA report and the selection criteria developed by DOE for 

choosing the final alternative.
Appendix IV: Assessment of DOE’s Analysis of 

Alternatives for the WIPP Ventilation System 

Project Compared with Best Practices

Page 43 GAO-16-608 DOE WIPP Recovery

Best practice

category and scorea Best practice Detailed assessmenta

7. The team documents and justifies 

all assumptions and constraints 

used in the analysis. 

Partially met. DOE’s AOA teams defined and justified many 

assumptions and constraints such as regulatory requirements. 

DOE did not ensure that specific assumptions were consistently 

defined or justified for all alternatives analyzed. For example, 

DOE’s AOA teams did not consistently use 2050 as WIPP’s 

mission end date for analyzing all alternatives and did not 

consistently justify the total airflow needed to support full 

operations. 

8. The team conducts the analysis 

without a predetermined solution. 

Minimally met. Because DOE defined the mission need by 

mentioning a new exhaust shaft, DOE’s AOA teams appeared to 

have analyzed alternatives with a preference for the alternatives 

that included constructing a new shaft. After DOE’s AOA teams 

proposed a total of three alternatives to be evaluated for a final 

selection, DOE appeared to have a preference for one of NWP’s 

proposed alternatives. For example, DOE did not take the time to 

conduct and document a cost-benefit analysis, as was 

recommended by DOE’s independent assessment of the AOA 

process, to justify the selection of the final alternative.

Identifying alternatives:

Fully met

9. The team identifies and considers 

a diverse range of alternatives to 

meet the mission need.

Fully met. DOE’s AOA teams identified nine principal ventilation 

alternatives that were analyzed. These included using the existing 

shaft(s) or drilling new shaft(s), splitting or combining airflows for 

mining and waste disposal operations, different ventilation filtration 

capacities, and different modes of operation. 

10. The team describes alternatives in 

sufficient detail to allow for robust 

analysis. 

Fully met. DOE’s AOA teams described the main features, 

including nuclear safety features, of the nine principal alternatives, 

their major equipment and components, and operating modes. 

NWP developed diagrams for each alternative showing the path of 

airflows. The descriptions of alternatives were primarily qualitative.

11. The team includes one alternative 

representing the status quo to 

provide a basis of comparison 

among alternatives. 

Fully met. DOE’s AOA teams included the existing WIPP 

ventilation system with the interim and supplemental upgrades 

being completed as part of the recovery project as the alternative 

representing the status quo.

12. The team screens the list of 

alternatives before proceeding, 

eliminates those that are not 

viable, and documents the 

reasons for eliminating any 

alternatives. 

Fully met. DOE’s AOA teams screened a number of alternatives 

that they determined were not viable because they would not meet 

the mission need capabilities for nuclear safety or full operations 

before conducting their analyses of the nine principal alternatives.

Analyzing alternatives:

Minimally met 

13. The team develops a life-cycle 

cost estimate for each alternative, 

including all costs from inception 

of the project through design, 

development, deployment, 

operation, maintenance, and 

retirement.

Minimally met. DOE did not ensure that its AOA teams developed 

life-cycle cost estimates for all principal alternatives, or that they 

were calculated in a consistent manner. NWP did not develop lifecycle cost estimates for four of the six principal alternatives it 

analyzed. Trinity developed life-cycle cost estimates for its three 

principal alternatives. DOE did not ensure that life-cycle cost 

estimates for the final three alternatives were calculated in a 

consistent manner to allow reliable comparison. 
Appendix IV: Assessment of DOE’s Analysis of 

Alternatives for the WIPP Ventilation System 

Project Compared with Best Practices

Page 44 GAO-16-608 DOE WIPP Recovery

Best practice

category and scorea Best practice Detailed assessmenta

14. The team presents the life-cycle 

cost estimate for each alternative 

as a range or with a confidence 

interval, and not solely as a point 

estimate.

Minimally met. DOE did not ensure that its AOA teams developed 

life-cycle cost estimate ranges for all principal alternatives. 

15. The team expresses the life-cycle 

cost estimate in present value 

terms and explains why it chose 

the specific discount rate used. 

Partially met. DOE did not ensure that its AOA teams expressed 

life-cycle cost estimates in present value terms for all of the nine 

principal alternatives that were analyzed and that present values 

were calculated in a consistent manner. For example, DOE did not 

ensure that its AOA teams used the same discount rates in 

calculating present values. NWP used Office of Management and 

Budget guidance for selecting the discount rate. Trinity used a 

different rate but did not explain the reason for the discount rate.

16. The team uses a standard 

process to quantify the 

benefits/effectiveness of each 

alternative and documents this 

process.

Not met. DOE’s AOA teams did not use a standard process to 

quantify benefits or effectiveness for each alternative.

17. The team quantifies the 

benefits/effectiveness resulting 

from each alternative over that 

alternative’s full life cycle, if 

possible.

Not met. DOE’s AOA teams did not quantify measures of benefits 

or effectiveness for each alternative as part of the AOA

18. The team explains how each 

measure of benefit/effectiveness 

supports the mission need.

Minimally met. DOE’s AOA teams described pros and cons or 

qualitative benefits for each alternative in relation to the mission 

need but did not describe quantitative measures of benefits or 

effectiveness for each alternative.

19. The team identifies and 

documents the significant risks 

and mitigation strategies for each 

alternative.

Minimally met. DOE’s AOA teams identified and documented 

qualitative risks and mitigation strategies for each alternative. DOE 

developed risk and uncertainty analysis for two of the nine 

alternatives that were analyzed. The risk and uncertainty analysis 

only covered the design and construction phase of the alternatives 

and not the full life cycle of the system.

20. The team tests and documents 

the sensitivity of both the cost and 

benefit/effectiveness estimates for 

each alternative to risks and 

changes in key assumptions.

Not met. DOE’s AOA teams did not conduct a sensitivity analysis 

of the cost or benefits or effectiveness estimates for the 

alternatives. 
Appendix IV: Assessment of DOE’s Analysis of 

Alternatives for the WIPP Ventilation System 

Project Compared with Best Practices

Page 45 GAO-16-608 DOE WIPP Recovery

Best practice

category and scorea Best practice Detailed assessmenta

Selecting a preferred 

alternative:

Partially met

21. The team or the decision maker

defines selection criteria based on 

the mission need. 

Partially met. DOE’s AOA teams identified selection criteria but did 

not develop detailed definitions of the criteria. The criteria 

established by NWP and Trinity reflected elements of project 

management—technical, safety, regulatory, cost and schedule—

and not specific capabilities needed to meet the mission need or 

functional requirements. DOE developed final selection criteria to 

evaluate the final three alternatives but did not document how the 

criteria aligned with the mission need.

22. The team or the decision maker 

weights the selection criteria to 

reflect the relative importance of 

each criterion.

Partially met. DOE’s AOA teams did not weight selection criteria 

consistently. For example, the teams weighted similar criteria 

differently (e.g., schedule and technical criteria). NWP changed 

how it weighted the selection criteria resulting in selection of two 

preferred alternatives during its January 2015 AOA. 

23. The team or the decision maker 

compares alternatives using net 

present value, if possible.

Not met. DOE’s AOA teams did not conduct cost-benefit analysis 

to compare alternatives using net present value and did not 

document reasons it was not possible. As discussed above, DOE’s 

AOA teams did not consistently examine life-cycle cost estimates 

for each alternative and did not quantify benefits of alternatives. 

Therefore, DOE did not produce the information needed for 

meeting the best practice. 

24. An entity independent of the AOA 

process reviews the extent to 

which all best practices have been 

followed (for certain projects, 

additional independent reviews 

may be necessary at earlier 

stages of the process such as for 

reviewing the study plan or for 

reviewing the identification of 

viable alternatives). 

Partially met. DOE completed two separate independent reviews 

of the AOA process. The first review was by DOE’s Office of 

Project Management Oversight and Assessments. The second was 

by the Office of Environmental Management (EM). The first review 

found that DOE did not justify the selection of the chosen 

alternative and recommended that DOE conduct a cost-benefit 

analysis to justify the final alternative. DOE did not fully address the 

recommendation because the project team was not required to. 

The second review by EM did not fully and accurately examine the 

extent that best practices were followed.

Source: GAO analysis of information from DOE, Nuclear Waste Partnership LLC, and Trinity Engineering Associates. | GAO-16-608

a

The score for each category represents the average scores of the best practices included in that 

category. An overall score of “fully met” or “substantially met” means that the AOA was reliable. The 

five-point scoring system that we used was as follows: “Fully met” means that DOE’s documentation 

demonstrated that it completely met the best practice; “Substantially met” means that DOE’s 

documentation demonstrated that it met a large portion of the best practice; “Partially met” means that 

DOE’s documentation demonstrated that it met about half of the best practice; “Minimally met” means 

that the DOE’s documentation demonstrated that it met a small portion of the best practice; and “Not 

met” means that DOE’s documentation did not demonstrate that it met the best practice.
Appendix V: Comments from the Department 

of Energy

Page 46 GAO-16-608 DOE WIPP Recovery

Appendix V: Comments from the Department 

of Energy
Appendix V: Comments from the Department 

of Energy

Page 47 GAO-16-608 DOE WIPP Recovery
Appendix V: Comments from the Department 

of Energy

Page 48 GAO-16-608 DOE WIPP Recovery
Appendix VI: GAO Contact and Staff 

Acknowledgments

Page 49 GAO-16-608 DOE WIPP Recovery

David C. Trimble, (202) 512-3841 or trimbled@gao.gov.

In addition to the contact named above, Daniel Feehan, Assistant 

Director; Cheryl Arvidson; Mark Braza; Richard P. Burkard; Jennifer 

Echard; Brian M. Friedman; Carly Gerbig; Jason Lee; Eli Lewine; Cynthia 

Norris; and Katrina Pekar-Carpenter made key contributions to this report.

Appendix VI: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(361607)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. 

Listen to our Podcasts and read The Watchblog.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.