Nuclear Waste: Waste Isolation Pilot Plant Recovery Demonstrates Cost and Schedule Requirements Needed for DOE Cleanup Operations
Released: 2016-08-04T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DOE's WIPP is the only deep geologic repository for the disposal of U.S. defense-related nuclear waste. In February 2014, waste operations were suspended following a truck fire and an unrelated radiological release. DOE estimated in February 2015 that it would complete recovery activities and restart limited waste operations by March 2016. To resume full operations, DOE planned to build a new ventilation system at WIPP. DOE completed an AOA to identify the best solution for this system in December 2015.

The Senate Report accompanying a bill for the National Defense Authorization Act for Fiscal Year 2015 included a provision for GAO to review WIPP operations. This report examines the extent to which DOE (1) met its initial cost and schedule estimates for restarting waste disposal operations, and (2) followed best practices in analyzing and selecting an alternative for the new ventilation system. GAO examined documentation on the WIPP recovery estimates. GAO compared DOE's February 2015 cost and schedule estimates and AOA with best practices GAO published.

section_title: What GAO Found
paragraphs: The Department of Energy (DOE) did not meet its initial cost and schedule estimates for restarting nuclear waste disposal operations at the Waste Isolation Pilot Plant (WIPP), resulting in a cost increase of about $64 million and a delay of nearly 9 months. DOE incurred this cost increase and delay partly because it did not follow all best practices in developing the cost and schedule estimates. In particular, DOE's schedule did not include extra time, or contingency, to account for known project risks. Instead, DOE estimated it would restart waste operations in March 2016 based on a schedule with no contingency that gave DOE less than a 1 percent chance of meeting its restart date. In January 2016, DOE approved new estimates that added 8.5 months to the schedule, extending the restart to December 2016; increased the estimated cost of recovery by $2 million; and resulted in an additional $61.6 million in costs for operating WIPP in fiscal year 2016. DOE's WIPP operations activity manager said the revised schedule included contingency. However, according to DOE officials, they did not follow other best practices. For example, DOE did not provide evidence of having an independent cost estimate to validate the revised estimate. DOE did not follow all best practices for cost and schedule estimates in part because DOE does not require that its cleanup operations, such as WIPP, follow these practices. Therefore, DOE cannot have confidence that its estimates are reliable. In contrast, DOE established new requirements in June 2015 that its capital asset projects, such as the new ventilation system at WIPP, follow these best practices. By also requiring cleanup operations to follow them, DOE would have more confidence in the estimates for cleanup operations and capital asset projects.

DOE did not follow all best practices in analyzing and selecting an alternative for the new ventilation system at WIPP. As a result, DOE's analysis was not reliable and DOE cannot be confident that the alternative it selected in December 2015 will best provide the needed capabilities at WIPP. The analysis of alternatives (AOA) process entails identifying, analyzing, and selecting a preferred alternative to best meet the mission need. Of the four categories of best practices for AOAs, DOE's process fully met the category for identifying alternatives. For example, DOE identified a broad range of ventilation alternatives. However, DOE only partially or minimally met the other three categories: general principles, analyzing alternatives, and selecting the preferred alternative. DOE did not follow the best practice to select the preferred alternative based on a cost-benefit analysis that assesses the difference between the life-cycle costs and benefits of each alternative. In addition, an independent review that DOE commissioned consistent with best practices found that DOE's AOA did not adequately document a cost-benefit analysis and that, as a result, the selection of the preferred alternative was not supported by compelling information. The independent review recommended that DOE conduct a cost-benefit analysis consistent with best practices. However, DOE did not conduct the recommended analysis and document it before selecting the final alternative because there was no requirement to do so. In June 2015, the Secretary of Energy directed DOE to develop guidance for conducting AOAs consistent with AOA best practices. A DOE official said the department expected to issue the new guidance by December 2016.

section_title: What GAO Recommends
paragraphs: GAO recommends that DOE require cleanup operations to follow best practices for cost and schedule estimates and require projects, including the WIPP ventilation system, to implement recommendations from independent AOA reviews or document the reasons for not doing so. DOE concurred with the recommendations.

section_title: Letter
paragraphs: The Department of Energy’s (DOE) Waste Isolation Pilot Plant (WIPP)  near Carlsbad, New Mexico, serves as the only deep geologic repository  in the U.S. for the disposal of defense-related nuclear waste. WIPP is  designed to safely dispose of a specific type of defense waste, referred to  as transuranic waste, generated by DOE’s nuclear weapons research,  production, and cleanup activities at sites across the country. The waste  is disposed of in underground rooms mined out of an ancient salt  formation more than 2,000 feet below the earth’s surface. Since WIPP  began to accept waste in 1999, DOE has depended on WIPP’s capability  to accept transuranic waste shipments. In February 2014, waste  operations at WIPP were suspended following two accidents  underground: a fire on a salt-hauling truck and an unrelated radiological  release from a waste container that contaminated portions of the facility  underground and released a small amount of radiation into the  environment above ground. The suspension of WIPP’s operations has  impaired DOE’s ability to meet its cleanup and national security missions  as well as regulatory cleanup milestones agreed to with states that host  DOE sites.

The department has made it a top priority to restart WIPP’s waste  disposal operations. In September 2014, DOE issued a WIPP recovery  plan. This plan describes the major activities needed to restart  operations, first on a limited basis and then moving to full operations.  WIPP’s full operations include waste disposal operations concurrent with  mining and mine maintenance activities. A key set of activities in the  recovery plan is to increase ventilation airflow underground, which was  425,000 cubic feet of air per minute (cfm) before the WIPP accidents. As  a result of the radioactive contamination underground, DOE has  continually run WIPP’s existing ventilation system in a filtered mode,  which is not the normal operating mode and has reduced the airflow to  60,000 cfm. Running in filtered mode severely restricts the number of  people that can be underground, as well as the activities that can be  conducted there, such as mine repair activities using diesel-powered  equipment that creates fumes and dust. To increase the airflow to restart  limited waste disposal operations, DOE planned two upgrades to the  existing ventilation system: an interim system that would increase airflow  to about 114,000 cfm, and a supplemental system that would increase  airflow to about 180,000 cfm. To resume full operations, DOE planned to  build a new permanent ventilation system to replace the existing one and  to provide airflow of about 540,000 cfm, which DOE estimates is sufficient  to safely support full and concurrent waste operations, mining, and mine  maintenance activities. From October 2014 through December 2015,  DOE conducted an analysis of alternatives (AOA) to identify the best  solution for the new permanent ventilation system. The AOA is a key first  step in DOE’s procedures for the acquisition of a capital asset such as  WIPP’s new permanent ventilation system. The process entails  identifying, analyzing, and selecting a preferred alternative to best meet  the mission need by comparing the operational effectiveness, costs, and  risks of potential alternatives.

In February 2015, DOE approved a project management baseline that  included the estimated cost and schedule for completing the activities  described in the WIPP recovery plan. DOE estimated that it would need  $242 million to complete the recovery activities needed to restart limited  waste disposal operations by March 2016. DOE also estimated a cost  range of $77 million to $309 million to complete the new permanent  ventilation system by the end of December 2018, paving the way for  resumption of full operations. In the recovery plan, DOE described its  schedule as aggressive and noted that it did not include extra time to  account for unanticipated delays. In May 2015, a DOE assessment found  that pressure to achieve the March 2016 deadline contributed to poor  safety practices in WIPP recovery efforts. In July 2015, DOE announced  that it experienced delays in implementing the project baseline, including  delays related to procuring equipment and delays related to correcting  deficiencies in safety practices. As a result of these delays, the  department announced that it would revise the WIPP project  management baseline with the goal of developing a more realistic  schedule. In January 2016, DOE approved the revised estimates for the  project cost and schedule to restart limited waste operations.  Nonetheless, the department still faces challenges in completing the  recovery. For example, in March 2016, the Defense Nuclear Facilities  Safety Board, which oversees DOE’s nuclear facilities such as WIPP,  reported that DOE had made progress in revising its nuclear safety plans  at WIPP but additional work remained to address safety concerns to  prevent a recurrence of the February 2014 radiological accident. As we  found in our most recent High-Risk Series update, DOE has a history of  exceeding its cost and schedule estimates and then creating new  baselines. We have also found that when DOE creates new baselines  during the execution of projects, it can be challenging to independently  assess DOE’s performance in managing these projects.

The Senate Armed Services Committee report accompanying a bill for the  National Defense Authorization Act for Fiscal Year 2015 included a  provision for us to review recovery operations at WIPP. Our report  examines the extent to which DOE (1) met its initial cost and schedule  estimates for restarting waste disposal operations at WIPP and (2)  followed best practices in analyzing and selecting an alternative for the  new ventilation system.

To address both objectives above, we conducted a site visit to WIPP in  January 2015. During the site visit we obtained documentation and  interviewed officials from DOE’s Office of Environmental Management  (EM), which is responsible for oversight of WIPP and exercises this  responsibility primarily through its Carlsbad Field Office. We also  interviewed representatives at WIPP from Nuclear Waste Partnership  LLC (NWP), which is the private contractor that manages and operates  WIPP for DOE. To examine the extent to which DOE met its initial  (February 2015) cost and schedule estimates for restarting waste  disposal operations, we compared DOE’s initial cost and schedule  estimates to restart limited waste disposal operations contained in its  February 2015 WIPP recovery project management baseline with DOE’s  revised estimates in its January 2016 revised baseline. We reviewed  DOE’s reports on the reasons it exceeded the initial estimates and the  risk management plans for WIPP recovery prepared by NWP which were  used in developing the initial estimates. We compared the initial cost and  schedule estimates from the February 2015 recovery baseline with the  best practices described in our cost and schedule guides. We reviewed  documentation on the revised (January 2016) baseline and interviewed  DOE and NWP on the approach followed to develop the revised cost and  schedule estimates to restart limited waste disposal operations in the  baseline, but we did not assess the revised estimates against the best  practices because of the time frame of our review. To examine the extent  to which DOE followed best practices in analyzing and selecting an  alternative for the new ventilation system, we reviewed documentation on  the process DOE followed and compared the process with best practices  for conducting an AOA that we published in December 2014. During the  review, we also interviewed officials from DOE’s Office of Enterprise  Assessments, which provides internal oversight of DOE facilities; the  Defense Nuclear Facilities Safety Board, which provides external  oversight of DOE defense nuclear facilities; the U.S. Mine Safety and  Health Administration, which provides external oversight of mining  activities at WIPP; and the U.S. Environmental Protection Agency and  New Mexico Environment Department, both of which provide external  regulation of WIPP. (See app. I for further information on the scope and  methodology of our review.)

We conducted this performance audit from November 2014 to August  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: This section describes (1) WIPP’s layout and base operations, (2) the  February 2014 truck fire and radiological release accident investigation  reports, (3) DOE’s requirements for operations activities and for project  management applicable to the WIPP Recovery effort, (4) best practices  for cost and schedule estimating, (5) best practices for conducting an  AOA, (6) Revisions in 2014 and 2015 to DOE’s requirements for AOAs,  and (7) DOE’s AOA process for WIPP’s new permanent ventilation  system.
subsections: section_title: WIPP Facility Layout and Base Operations
paragraphs: As shown in figure 1, WIPP’s layout consists of surface facilities,  underground facilities, and four shafts that connect the surface with the  underground. WIPP’s underground facilities currently include seven waste  disposal units or panels where waste containers are placed for final  disposal, an area designated for constructing future disposal units, and an  area for experimental research. The four shafts connecting the surface  and underground facilities are (1) the air intake shaft, which is the primary  air supply to the underground; (2) the exhaust shaft, through which all air  exits the underground; (3) the salt handling shaft, which is used to  remove mined salt from the underground and also supplies air to the  underground; and (4) the waste handling shaft which is used to transport  waste containers to the underground. Figure 1 also shows the locations of  the February 2014 accidents that led to the suspension of WIPP’s  operations.

Under normal operating conditions, base operations at WIPP include  waste handling to receive waste from generator sites and prepare and  place the waste, ongoing maintenance of the WIPP surface facilities and  underground, as well as support functions such as program management,  engineering, quality assurance, safety and security, and environmental  management. In addition, NWP reviews and certifies waste containers on  DOE’s behalf at DOE’s transuranic waste generator sites. During  implementation of the WIPP recovery plan, DOE continued aspects of  these base operations such as maintenance of the surface facilities and  program management. In addition, NWP continued to review and certify  waste packages at select sites.

section_title: WIPP Truck Fire and Radiological Release Accidents and Investigation Reports
paragraphs: The salt truck fire, which occurred on February 5, 2014, created  substantial smoke and soot that damaged key equipment and facilities in  the underground. On February 14, 2014, the radiological release occurred  when a transuranic waste container was breached. The breach was  caused by a chemical reaction inside the container between materials  that DOE later determined should not have been packaged together. The  reaction generated enough heat to increase pressure in the container.  The pressure forced open the container’s lid and propelled its radioactive  waste contents, combustible gases, and other materials into the air and  on to adjacent waste containers. The radioactive contents, gases, and  other materials ignited and triggered a fire in the disposal room by igniting  other materials in the room. WIPP’s ventilation system failed to contain all  of the airborne radiological material underground and allowed a small  amount to enter the environment. As a result of the release, portions of  the WIPP underground and the existing ventilation system were  radiologically contaminated.

DOE issued an accident investigation report on the salt truck fire in March  2014 and an accident investigation report on the radiological release in  April 2014 called “phase one report.” The phase one report focused on  the release to the environment and the response at WIPP to the release.  One year later, in April 2015, DOE issued a phase two report that focused  on the radiological release from inside the waste container. The reports  determined that the causes of the accidents included the degraded  condition of critical equipment, inadequacies in the design and operations  of WIPP’s ventilation system, and deficiencies in the management of  WIPP’s safety programs that are intended to control and discipline  operations to protect workers, the public, and the environment from  radiological and other hazards. In total, the three reports made 100  recommendations to DOE and NWP to complete corrective actions. For  example, the phase one report identified a number of weaknesses with  the WIPP nuclear safety documentation and recommended that this  documentation be revised prior to resuming limited waste disposal  operations to ensure the repository can be operated safely with respect to  workers, the public, and the environment.

section_title: DOE’s Requirements for Operations Activities and for Project Management Applicable to the WIPP Recovery Effort
paragraphs: DOE is managing the recovery activities to restart limited waste disposal  operations at WIPP and to design and build the new permanent  ventilation system capital asset project following two separate project  management requirements. DOE requires that the recovery activities to  restart limited waste disposal operations and overall base operations of  WIPP be managed following EM’s protocol for operations activities. This  protocol defines operations activities to include activities that are project- like with defined start and end dates and reoccurring facility or  environmental operations. The new permanent ventilation system  project must follow DOE’s Order 413.3B, which governs project  management for the acquisition of capital assets. The following is a  description of the two different requirements:

EM operations activities protocol. EM’s protocol establishes a  framework for managing and reporting the progress of cleanup  operations by requiring, among other things, that project performance  be measured objectively and that management actions be taken to  mitigate risks and manage costs. EM’s protocol directs EM sites to  develop performance baselines—or estimates—for cost, schedule,  and scope to use in assessing the project’s performance over the  fiscal year, multiyear contract period of performance, and the life cycle  of the project. The protocol directs that the project develop  estimates but does not include specific requirements that must be  followed, such as the steps that must be followed to develop cost and  schedule estimates.

DOE Order 413.3B. DOE’s order provides program and project  management direction for the acquisition of capital assets, with the  stated goal of delivering fully capable projects within the planned cost,  schedule, and performance baseline. The order establishes five  critical decision processes of project development that each end with  a major approval milestone—or “critical decision” point—that cover  the life of a project. The order specifies the requirements that must be  met, including for developing and managing project cost and schedule  estimates to move a project past each critical decision milestone.  DOE also provides suggested approaches for meeting the  requirements contained in the order through a series of guides, such  as a guide for cost estimating. In June 2015, the Secretary of Energy  directed that this order be revised to require that cost and schedule  estimates be developed, maintained, and documented in a manner  consistent with methods and the best practices identified in GAO’s  cost and schedule assessment guides and other published standards  and best practices, such as the Federal Acquisition Regulation and  Office of Management and Budget guidance.

section_title: GAO’s Best Practices for Cost and Schedule Estimating
paragraphs: The GAO Cost Estimating and Assessment Guide and the GAO Schedule  Assessment Guide compiled best practices corresponding to the  characteristics of high-quality and reliable cost and schedule estimates.  A high-quality, reliable cost estimate has the following four characteristics: comprehensive (e.g., has enough detail to ensure that cost elements  are neither omitted nor double counted), well-documented (e.g., allows for data it contains to be traced to accurate (e.g., is based on an assessment of most likely costs and  has been adjusted properly for inflation), and credible (e.g., discusses any limitations because of uncertainty or  bias surrounding data or assumptions).

Similarly, a high-quality, reliable schedule has four characteristics: comprehensive (e.g., captures all government and contractor  activities necessary to accomplish a project’s objectives), well-constructed (e.g., sequences all activities using the most  straightforward logic possible), controlled (e.g., is updated periodically to realistically forecast dates  for activities), and credible (e.g., uses data about risks to predict the level of confidence  in meeting a completion date and necessary schedule contingency  and high-priority risks are identified based on conducting a robust  schedule risk analysis).

The characteristics of a high-quality and reliable cost estimate are  supported by best practices listed in appendix II, and the characteristics  of a high-quality and reliable schedule are supported by best practices  listed in appendix III.

section_title: GAO’s Best Practices for Analysis of Alternatives
paragraphs: In a December 2014 report we compiled 24 best practices for conducting  an AOA. As noted above, the AOA is a key first step in acquisition of a  capital asset such as WIPP’s new permanent ventilation system. The  process entails identifying, analyzing, and selecting a preferred  alternative. Conforming to these best practices helps ensure that the  preferred alternative selected is the one that best meets the agency’s  mission needs. We grouped these 24 best practices into the following four  categories:

General principles. This category includes best practices to be  applied in planning, conducting, and documenting the AOA, such as  defining functional requirements based on the mission need and  conducting the analysis without a predetermined solution.

Identifying alternatives. The identifying alternatives category  includes best practices that help ensure the alternatives to be  analyzed are sufficient, diverse, and viable.

Analyzing alternatives. The analyzing alternatives category contains  best practices related to estimating the costs and benefits of each  alternative over its life cycle.

Selecting a preferred alternative. The selecting a preferred  alternative category includes best practices to help ensure a preferred  alternative is selected that best meets the mission need by comparing  alternatives based on their costs and benefits and independently  reviewing the AOA process. Appendix IV lists the 24 best practices  organized by the four categories.

section_title: Revisions in 2014 and 2015 to DOE’s Requirements for Analysis of Alternatives
paragraphs: DOE’s procedures for conducting an AOA have changed twice since DOE  began the AOA process for the WIPP permanent ventilation system. DOE  started the AOA process for this system in October 2014. At that time,  DOE was operating under AOA requirements defined in Order 413.3B.  The AOA process, as defined in the order, occurs during the span of the  first two critical decisions—Critical Decision-0 and Critical Decision-1  (Approve Alternative Selection and Cost Range). In December 2014, the  Secretary of Energy established a new requirement that for projects  estimated to cost $50 million or more, the responsible program office is to  conduct an AOA independent of the contractor responsible for the  project. Later, in June 2015, the Secretary of Energy directed the  department to develop guidance for conducting AOAs consistent with the  AOA best practices that GAO has published. The Secretary further  required that AOAs be conducted and documented in a manner  consistent with the guidance when it is complete. A DOE official told us in  June 2016 that the department expected the guidance to be completed by  December 2016. The Secretary’s action followed a recommendation that  we made in December 2014 that DOE revise its Order 413.3B to adopt  AOA best practices.

section_title: DOE’s AOA Process for WIPP’s New Permanent Ventilation System
paragraphs: DOE’s changes in its requirements for the AOA process under Order  413.3B took effect in the midst of DOE’s AOA process for WIPP’s new  permanent ventilation system. DOE’s process was conducted as follows:

DOE EM approved the mission need for the new ventilation system  project in October 2014.

Also in October 2014, NWP started an initial AOA and completed it in  January 2015 shortly after the Secretary’s December 2014 directive  on the independence of AOAs and prior to the Secretary’s June 2015  directive about the use of the AOA best practices; therefore the new  requirements did not apply to NWP’s AOA.

To respond to the Secretary’s December 2014 directive, DOE  contracted in spring 2015 with a contractor unaffiliated with WIPP— Trinity Engineering Associates (Trinity)—to do a second AOA. DOE  specifically directed Trinity in its contract to implement all of the AOA  best practices that we compiled in our December 2014 report. The  WIPP program manager in the Carlsbad Field Office told us that he  included the best practices in the contract to provide more formality to  their process. Trinity completed its AOA in October 2015.

DOE evaluated the preferred alternatives proposed by NWP and  Trinity, and it selected and approved a final alternative to complete  Critical Decision-1 in December 2015.

section_title: DOE Did Not Meet Its Initial Cost and Schedule Estimates for Restarting Waste Disposal Operations
paragraphs: DOE did not meet its initial cost and schedule estimates for the efforts  needed to restart WIPP disposal operations, resulting in about $64 million  in added costs and a delay of nearly 9 months. Two primary factors  contributed to the cost increase and schedule delay. First, DOE only  partially followed best practices in developing its initial cost and schedule  estimates, which made them unreliable and increased the likelihood that  they would be exceeded. Second, DOE did not successfully manage key  project risks it had identified that had potential to cause delay.
subsections: section_title: DOE Incurred A Cost Increase of about $64 Million and a Nearly 9- month Delay in Its Efforts to Restart WIPP Operations
paragraphs: DOE incurred a cost increase of about $64 million and a nearly 9-month  schedule delay in its efforts to restart WIPP waste disposal operations.  Specifically, in January 2016, approximately a year after approving the  initial project baseline, DOE approved a new project management  baseline that increased the estimated costs for the recovery project by $2  million (from $242 million to $244 million) and added 8.5 months to the  project schedule, extending the date when limited waste disposal  operations might begin from March 2016 to December 2016. According to  DOE officials, the project was unable to meet its estimated completion  date due to delays, including delays associated with procuring  components for the interim ventilation system upgrade that were found to  be faulty, and delays associated with DOE’s decision to require the  project to adhere to new nuclear safety requirements. According to the  operations activity manager for WIPP in the Carlsbad Field Office, the  recovery project only exceeded the $242 million baseline by $2 million in  part because DOE overestimated the cost of some project activities and  spent contingency funds to make up for cost increases in certain parts of  the project. For example, DOE overestimated the cost for  decontaminating the WIPP underground by more than $6 million, which  offset a $5 million cost increase caused by delays associated with the  faulty ventilation system components. However, we identified an  additional $61.6 million cost increase in base operations that was  attributable to the delay in completing the recovery project. Specifically,  DOE’s justification supporting the President’s fiscal year 2016 budget  request to Congress from February 2015 estimated about $130.6 million  for WIPP’s base operations in fiscal year 2016. According to DOE  officials, as DOE revised its cost and schedule estimates for the WIPP  recovery project in the fall of 2015, DOE also revised its cost estimate for  WIPP’s base operations and provided the updated information to the  congressional appropriations committees. In December 2015, DOE  received $192.1 million for fiscal year 2016 base operations, an increase  of $61.6 million (47 percent) over the initial estimate for base operations  in fiscal year 2016. According to the operations activity manager for WIPP  in the Carlsbad Field Office and representatives from NWP, this increase  to base operations occurred because DOE estimated the cost of base  operations on the assumption of restarting WIPP in March 2016 following  its initial schedule estimate. The delay in the recovery efforts required  DOE to keep base operations running alongside the recovery efforts for a  longer period of time than initially planned.

section_title: DOE’s Initial Cost and Schedule Estimates for Restarting Operations at WIPP Were Unreliable
paragraphs: DOE did not meet its initial schedule and cost estimates for restarting  waste disposal operations at WIPP, in part because DOE did not develop  the estimates following all best practices, rendering the estimates  unreliable and increasing the likelihood that they would be exceeded.  More specifically, of the four characteristics of a high-quality and reliable  cost estimate, DOE’s initial cost estimate substantially met two  characteristics—comprehensive and well-documented—but partially met  or minimally met the other two characteristics—accurate and credible. We  made the following observations:

DOE substantially met best practices for a comprehensive cost  estimate by, for example, including a majority of life-cycle costs and  identifying the cost estimating ground rules and assumptions.

DOE substantially met best practices for a well-documented  estimate by including the actual sources for the cost data and  documenting that the cost estimate was reviewed and accepted by  DOE management.

DOE partially met best practices for an accurate estimate by, for  example, only documenting a portion of the process they used to  adjust costs to account for inflation.

DOE minimally met best practices for a credible estimate by, for  instance, not developing an independent cost estimate. Had DOE  completed an independent cost estimate, the department would have  had an unbiased and objective benchmark to assess whether the cost  estimate prepared by NWP could be achieved, and thus would have  been positioned to reduce the risk that the project would proceed  underfunded.

Regarding DOE’s initial schedule for restarting waste disposal operations,  DOE substantially met two of the four characteristics of a high-quality  schedule—comprehensive and controlled—but partially met the  characteristics—well-constructed and credible—resulting in a schedule  that was also unreliable and unrealistic. We made the following  observations:

DOE substantially met best practices for a comprehensive schedule  because, for example, the schedule reflected the activities in the  recovery work breakdown structure, which defined the work  necessary to accomplish the project’s objectives. In addition, work  scope was assigned in the schedule as the responsibility of NWP or  DOE.

DOE substantially met best practices for a controlled schedule by,  for example, regularly updating its master schedule and using the  schedule as the basis for measuring performance.

DOE partially met best practices for a well-constructed schedule  because, for instance, a significant number of activities in the  schedule had incorrect or missing logic relationships that are  important for determining how delays or accelerations in one activity  would affect the start or finish of other activities later in the schedule.

DOE partially met best practices for a credible schedule, in  particular, because its schedule did not include extra time, or  contingency, to account for known project risks. As noted above, DOE  acknowledged in its WIPP recovery plan that the schedule did not  include contingency that may be needed due to unanticipated  difficulties or delays with the project. However, DOE did not  acknowledge in the plan that the schedule also did not include  contingency for the occurrence of known and quantified risks that had  been anticipated in its project risk analysis—such as the risks that  were the primary causes of the project delays. Notably, DOE’s risk  analysis predicted that the March 2016 date to restart operations had  a less than 1 percent confidence level, meaning in effect that DOE  had a less than 1 percent chance of meeting the March 2016  deadline. EM officials said the department used a less than 1 percent  confidence level because it wanted to have an aggressive goal for  restarting operations. According to EM officials, when EM managers  presented this schedule to senior DOE decision-makers for approval,  they described the schedule as “aggressive” but did not clarify that  there was less than a 1 percent chance of meeting the schedule.  According to these officials, they did not think that senior managers  would understand the project management terminology regarding  confidence levels.

In January 2016, DOE approved revised cost and schedule estimates for  restarting WIPP’s limited waste disposal operations. The operations  activity manager for WIPP in the Carlsbad Field Office said the revised  schedule included contingency and the revised restart date of December  2016 had an 80 percent confidence level. However, according to DOE  officials, they did not follow other best practices. For example, DOE did  not provide evidence of having an independent cost estimate which would  have provided DOE an unbiased and objective benchmark to validate the  estimates prepared by NWP. As noted above, we did not assess DOE’s  revised estimates against all of the best practices. The full results of our  analysis of the initial cost and schedule estimates can be found in  appendixes II and III.

DOE did not follow all best practices in developing the initial cost and  schedule estimates for the WIPP recovery project or in developing new  estimates because, unlike DOE’s requirements for capital asset projects  under Order 413.3B, DOE’s EM operations activities protocol that  governs cleanup operations such as WIPP recovery does not require the  use of best practices in developing such estimates. As mentioned above,  DOE EM’s protocol requires that EM sites develop or approve baselines  for each project’s cost and schedule to judge project performance.  However, the protocol does not specify any best practices in terms of the  steps to follow in developing the cost and schedule estimates in these  baselines. The absence of a requirement to follow best practices for cost  and schedule estimating is in contrast to a decision in June 2015 by the  Secretary of Energy to require that all capital asset projects with an  estimated cost of $10 million or more follow such practices under Order  413.3B. Operations activities such as the WIPP recovery activities are not  considered capital asset projects and therefore are not required to follow  these requirements even though EM’s operations activities share many of  the same characteristics as capital asset projects. Without requiring EM’s  operations activities such as WIPP to follow all best practices when  developing cost and schedule estimates DOE cannot have confidence  that it is producing reliable baselines needed to monitor its performance in  managing these activities. Without reliable baselines, DOE will also  continue to be at risk of cost overruns and delays in achieving its cleanup  missions such as the permanent disposal of transuranic waste from sites  across the country.

DOE has reported progress in completing the activities needed to restart  limited waste operations using its revised cost and schedule estimates,  although it still faces challenges and it remains unclear whether DOE  identified and analyzed the risks associated with these challenges in  revising the estimates. In terms of progress, for example, in December  2015, NWP submitted revisions to WIPP’s nuclear safety documentation  to EM for review. In addition, DOE reported in March 2016 that NWP  completed most of the construction of the interim ventilation system  upgrade in the underground. However, as noted above, in March 2016,  the Defense Nuclear Facilities Safety Board reported that additional work  was needed to revise WIPP’s nuclear safety documentation to prevent  recurrence of a radiological release accident. In addition, in April 2016,  DOE’s Office of Enterprise Assessments, which provides independent  internal oversight of DOE’s management of safety, issued two reports that  found that although NWP had made improvements in its operational  safety and emergency management programs and procedures, significant  challenges remained to fully meet DOE requirements and effectively plan  and implement these programs and procedures. As we noted above,  DOE officials acknowledged that the department did not follow all best  practices in developing the revised estimates to restart WIPP operations.  Without having followed all best practices, including having an  independent cost estimate conducted to validate the estimates, DOE  cannot be confident that NWP sufficiently accounted for these challenges  in revising its risk analysis and that DOE set an appropriate allowance for  contingency to reduce the risk of cost overruns and delays in restarting  WIPP’s operations.

section_title: DOE Did Not Successfully Manage WIPP Recovery Project Risks That Contributed to the Project Cost Increase and Schedule Delay but Has Revised Its Risk Management Process
paragraphs: DOE did not successfully manage key risks that it had identified and that  contributed to the project’s cost increase and schedule delay but has  taken steps to revise the risk management process for the WIPP recovery  project. DOE’s EM operations activities protocol requires the use of a risk  management process to identify and mitigate risks to completing a project  within its baseline cost and schedule estimates. For the WIPP recovery  project, DOE and NWP developed a project risk register to support the  initial project management baseline for WIPP recovery. The risk register  listed the risks that could increase costs or delay completion. DOE and  NWP assessed the likelihood and consequence of each risk and  identified a mitigation plan for each risk that described the actions to  reduce the impact on the project if the risk were realized.

In a public statement explaining the need to revise its baseline for the  recovery project, DOE cited problems with three activities that the  department had previously identified in its project risk register: (1)  completing revisions to WIPP’s nuclear safety documentation; (2)  installing the interim ventilation system upgrade; and (3) completing the  corrective actions that address the recommendations in the phase two  accident investigation report. DOE had identified risks related to each of  these activities in the project risk register but was not able to effectively  mitigate them, as discussed below:

WIPP nuclear safety documentation. NWP identified the risk of a  potential delay of up to 6 months in restarting operations if DOE  decided that the WIPP nuclear safety documentation should be  revised according to standards that were in the process of being  updated rather than according to the existing standards. According  to DOE, when NWP started to revise the safety documentation in the  summer of 2014, the contractor assumed it should follow the existing  standard. NWP added this risk to its project risk register to account for  the possibility that EM could require NWP to revise the nuclear safety  documentation according to the new standards. To mitigate this risk,  NWP’s plan was to ensure they had concurrence with DOE to start  the revisions using the existing standards. DOE issued the new  standards in November 2014 and notified NWP in December 2014  that the WIPP safety documentation would need to comply with them.  According to an NWP recovery project manager, this change in policy  resulted in a 7-month delay in NWP’s schedule for revising the WIPP  safety documentation.

Interim ventilation system. NWP identified the risk of a potential  delay of 3 months to complete the installation of the interim ventilation  system upgrade if NWP faced difficulties acquiring the components for  the system. To mitigate this risk, NWP’s plan was to identify difficult- to-procure equipment early in the project, ensure it had a valid and  up-to-date list of qualified suppliers, and monitor and review  equipment purchases. According to NWP officials, this plan was  partially effective. Specifically, NWP officials said that they discovered  that components for the interim ventilation system were damaged  when they inspected the components before formally accepting them.  In addition, they said that the costs to return the components to the  manufacturer and repair them were paid for by the manufacturer.  However, the NWP officials said that as a result of receiving the faulty  equipment, the installation of the interim ventilation system was  delayed about 4 months because NWP needed to send the  components back to the manufacturer and make corrections to its  shipping process to prevent a reoccurrence of these issues.

Corrective actions from the phase two accident investigation  report. NWP identified the risk of potential delay of up to 6 weeks  regarding the length of time DOE needed to complete the phase two  accident investigation. In the summer and fall of 2014, when NWP  was developing the initial project management baseline, DOE’s phase  two accident investigation was still underway. According to NWP  officials, they needed to wait for DOE to complete the accident  investigation and issue the final report with recommendations before  they could begin developing and implementing corrective actions.  DOE issued the final report in April of 2015. According to the  operations activity manager for WIPP in the Carlsbad Field Office,  completing the corrective actions in response to the report’s  recommendations resulted in a 3-month delay.

NWP officials said that in recognition of the need for a more effective risk  management process, they revised their process as part of revising the  recovery cost and schedules estimates. These officials explained, for  example, the revised process now involves more frequent discussions  and updating the status of potential risks with managers at WIPP who  oversee the departments where the project risks are generated. We did  not conduct an assessment of the effectiveness of this new risk  management process.

section_title: DOE Did Not Follow All Best Practices in Analyzing and Selecting an Alternative for the New Ventilation System at WIPP
paragraphs: DOE did not follow all best practices in analyzing and selecting an  alternative for the new ventilation system at WIPP. As a result, DOE’s  analysis was not reliable and DOE cannot be confident that the selected  alternative will best provide the needed capabilities. Of the four categories  of the best practices, DOE fully met the category for identifying  alternatives. However, DOE partially or minimally met the other three  categories of best practices—general principles, analyzing alternatives,  and selecting the preferred alternative—because of key limitations. (See  app. IV for the detailed results of our analysis of DOE’s AOA process.)
subsections: section_title: DOE Fully Met Best Practices in the Identifying Alternatives Category
paragraphs: DOE’s AOA process fully met the best practices in the identifying  alternatives category. To identify alternatives, DOE relied on analyses  conducted by NWP, the contractor responsible for the project, and the  second contractor—Trinity. These analyses were completed in January  and October 2015, respectively. NWP and Trinity identified a broad range  of alternatives. Specifically, they identified nine ventilation system  alternatives that the contractors determined should be analyzed because  the alternatives could meet the mission need and other screening criteria  that they defined: NWP identified six and Trinity identified three. The  alternatives included continued use of the existing exhaust shaft as well  as constructing a new shaft or more than one shaft, such as a new  exhaust shaft and a new air intake shaft. In addition, the studies identified  different ventilation filtration capacities and modes of operation that  included continuous filtering of all exhaust air or filtering air only when a  radiological release is detected underground.

section_title: DOE Partially Met Best Practices in the General Principles Category
paragraphs: DOE’s AOA process partially met the best practices in the general  principles category, which covers how the AOA process is planned,  conducted, and documented. In particular, DOE partially or minimally  followed the best practices of defining the mission need for the project,  analyzing alternatives without a predetermined solution, and defining the  functional requirements based on the mission need. As the AOA related  to the best practices of defining the mission need for the project and  analyzing alternatives without a predetermined solution, DOE did not  define the mission need for the new ventilation system by focusing only  on the capabilities needed for the project, but instead defined the need for  the system to include a particular solution, which included constructing a  new ventilation exhaust shaft. Because DOE defined the mission need by  mentioning a new exhaust shaft, both contractors appeared to have  analyzed alternatives with a preference for the alternatives that included  constructing a new shaft. Specifically, NWP completed its analysis of the  six alternatives by proposing two for further consideration by DOE, one  that did not include a new shaft and one that did. NWP’s alternative that  did not include a new shaft was its highest-scoring alternative and the  alternative that included a new shaft was its third-highest scoring  alternative. NWP officials told us that they proposed the second  alternative along with the highest-scoring one because they believed the  mission need statement discussed a new shaft. Regarding Trinity, in its  final AOA report, Trinity stated that the mission need statement endorsed  a new exhaust shaft. Trinity then analyzed three alternatives, one of  which included using the existing shaft, and proposed an alternative to  DOE that included constructing a new shaft. In its final assessment, DOE  assessed the two alternatives proposed by NWP and the one proposed  by Trinity and selected NWP’s alternative that included constructing a  new shaft, shown as the second of the three alternatives in table 1.  According to best practices, the AOA process should be an unbiased  inquiry into the costs, benefits, and capabilities of all alternatives.  By  conducting the AOA with a predetermined solution, DOE undermined the  credibility of its final decision.

Another limitation under the general principles category was that DOE  partially followed the best practice of defining the functional requirements  that the ventilation system would need to satisfy. DOE did not consistently  define two key functional requirements of the project—(1) the rate of  airflow needed to support full operations at WIPP, and (2) the expected  operational life of the new system—which limited the reliability of the  overall AOA process as follows:

Regarding the estimated airflow, DOE did not specify a functional  requirement in its mission need statement for the rate of airflow that  the new ventilation system must be capable of providing for full  operations at WIPP. Instead, the mission need statement described  the minimum airflow required by WIPP’s hazardous waste facility  permit, which was 260,000 cfm, and the actual airflow at WIPP before  the accidents, which was 425,000 cfm. In conducting its AOA, NWP  calculated its own airflow that it thought would be necessary for full  operations at WIPP, which they estimated was about 540,000 cfm.  NWP estimated the higher airflow based on its analysis of  underground operations and used this amount to analyze a range of  alternatives. In contrast, Trinity used the airflow associated with the  WIPP hazardous waste facility permit, 260,000 cfm, as the estimated  airflow in conducting its AOA. The federal project director in the  Carlsbad Field Office told us that he did not direct Trinity to redo its  analysis using NWP’s higher airflow because he believed that airflow  was needed for examining the scale of the system and would be  defined lower during design. Moreover, he said that he wanted to  maintain Trinity’s independence from NWP to be consistent with the  December 2014 DOE requirement for conducting independent AOAs.

However, as a result of not defining the same functional airflow  requirement for both contractors, DOE officials explained that they  needed to conduct additional analysis to compare the alternatives  proposed by NWP and Trinity.

Regarding the estimated operational life of the new ventilation system,  DOE did not specify a functional requirement in its mission need  statement for the estimated operational life of WIPP. Without such a  requirement, NWP used 2030 as the estimated end date for WIPP  operations—this date was based on DOE’s approved life-cycle plan  for ending WIPP’s operations current at the time NWP conducted its  analysis. NWP completed its AOA in January 2015 when it presented  two alternatives to DOE for further evaluation—as described above,  NWP had proposed these two alternatives from an initial group of six  alternatives. In February 2015, EM revised the estimated operational  life of WIPP, extending it from 2030 to 2050 to more accurately reflect  DOE’s schedules for transuranic waste cleanup at DOE sites. Trinity  did not start its AOA process until after DOE had revised the  estimated operational life of WIPP to 2050 and therefore used the  2050 date in conducting its AOA. In analyzing NWP’s two proposed  alternatives, DOE used the new estimated operational life of 2050 and  did not reassess the four previously eliminated alternatives using the  revised date.

By not following the best practice of specifying functional requirements at  the start of the AOA process, including specifying the airflow that the  system needed to deliver and the estimated end date of operations for  WIPP, it is unclear whether the AOAs conducted by NWP and Trinity and  DOE’s subsequent analyses of these AOAs allowed DOE to select the  alternative that best meets mission needs.

section_title: DOE Minimally Met Best Practices in the Analyzing Alternatives Category
paragraphs: DOE’s AOA process minimally met the best practices in the analyzing  alternatives category. Significant limitations in this category included DOE  and its contractors not consistently examining life-cycle costs for each  alternative and not quantifying their benefits. Regarding examining life- cycle costs:

NWP did not examine full life-cycle cost estimates for all of the  alternatives it examined. Specifically, NWP’s AOA completed in  January 2015 examined the estimated costs for the design and  construction of each of the six alternatives, but did not examine the  full life-cycles of these alternatives. As a result, NWP eliminated four  of its six principal alternatives from further evaluation before  examining each of them in terms of full life-cycle costs. According to  the NWP project manager who led NWP’s AOA team, the team did  not have sufficient time to examine the full life-cycle costs of each  alternative. In addition, according to DOE officials, NWP was not  required to examine the full life-cycle costs of each alternative under  DOE Order 413.3B because they completed the analysis in January  2015, which was several months before the Secretary’s June 2015  directive to incorporate AOA best practices in the order. According to  AOA best practices, the team conducting the AOA should be given  enough time to complete the AOA process to ensure a robust and  complete analysis.

NWP and Trinity used different assumptions in developing life-cycle  cost estimates, which prevented DOE from directly comparing the life- cycle estimates for the three proposed alternatives. We found that  DOE relied on NWP and Trinity to define the assumptions they used  rather than providing the contractors consistent guidance or direction.  As a result, NWP assumed certain costs should be excluded in  estimating the life-cycle costs of its two proposed alternatives. These  costs included the costs for major equipment replacements or  upgrades which NWP assumed would not be needed during the 30- year operational life of the system; costs for providing facility security,  quality assurance, and nuclear safety that are provided to all facilities  at WIPP; and costs for the final closure of the new exhaust shaft after  2050 when the system will be decommissioned. In contrast, Trinity  assumed these costs should be included in estimating the life-cycle  costs of its alternatives. As a result of including these additional costs,  Trinity’s estimated life-cycle cost was substantially more than NWP’s  estimate—Trinity’s estimate for its proposed alternative was $3.45  billion and NWP’s estimates for its two proposed alternatives were  $368.8 million and $467.6 million.

Regarding quantifying benefits, DOE and its contractors did not provide  any measures of the benefits or effectiveness for each of the alternatives  in their analyses. By not ensuring that the life-cycle costs of each  alternative were developed in a consistent manner and not ensuring that  the benefits or effectiveness of each alternative were quantified, DOE did  not have an accurate and complete picture of all the alternatives to make  reliable comparisons between them.

section_title: DOE Partially Met Best Practices in the Selecting a Preferred Alternative Category
paragraphs: DOE’s AOA process partially met the best practices in the selecting a  preferred alternative category which covers selecting an alternative that  best meets the mission need and also independently reviewing the overall  AOA process. Most notably, DOE and its contractors did not follow the  best practice to select the alternatives based on a cost-benefit analysis  and only partially followed the best practice to independently review the  AOA process, as follows:

Regarding cost-benefit analysis, DOE and its contractors did not  select the alternatives based on such an analysis which compares the  life-cycle costs and benefits or effectiveness of each alternative. As  noted above, DOE and its contractors did not consistently examine  life-cycle cost estimates for each alternative and did not quantify their  benefits. Therefore, DOE did not produce the information needed for a  cost-benefit analysis. By not selecting the alternatives based on a  cost-benefit analysis, DOE did not adequately justify that the selected  final alternative would best provide the capabilities needed at WIPP.

Regarding independently reviewing the AOA process, DOE conducted  an independent review of the AOA process (which assessed whether  best practices were followed) but did not implement the  recommendation identified by the review. Specifically, an independent  review conducted by DOE’s Office of Project Management Oversight  and Assessments found that the project team did not adequately  document a cost-benefit analysis and that, as a result, the selection of  the preferred alternative was not supported by compelling information.  The review recommended that DOE perform a cost-benefit analysis to  be consistent with best practices and support the selection of the final  alternative. However, DOE and NWP did not conduct the  recommended cost-benefit analysis and document it before DOE  selected the final alternative. According to the DOE officials who led  the independent review of the AOA process, the project team was not  required by DOE’s Order 413.3B to implement the recommendation or  justify and document the reasons for not doing so. Therefore, DOE  EM approved the selection of the preferred alternative for the  permanent ventilation system project in December 2015 without  implementing the independent review’s recommendation.

Conducting an independent review of the AOA is a best practice because  it is one of the most reliable means to ensure that bias does not influence  the AOA process and that the AOA is sufficiently thorough to ensure that  a preferred solution is chosen and not a favored solution. DOE’s  independent review was an internal control to help DOE meet its goal for  the capital asset project, in this case selecting the alternative that will best  provide the capabilities to meet the mission need. Under federal  standards for internal control, management is to evaluate and document  internal control issues and determine appropriate corrective actions. By  not implementing the recommendation from the independent review to do  a cost-benefit analysis or not justifying and documenting the reason for  not doing so, DOE cannot provide assurance that the final alternative  selected would best provide the capabilities needed at WIPP.

section_title: Conclusions
paragraphs: Restarting WIPP’s waste disposal operations is a top priority for DOE,  and it has made progress in its efforts to restart limited waste disposal  operations. However, DOE did not meet its initial cost and schedule  estimates to restart operations and incurred a cost increase of about $64  million—$2 million to the recovery project and $61.6 million in base  operations—and a delay of nearly 9 months. DOE incurred the cost  increase and schedule delay, in part, because DOE’s initial estimates did  not follow all best practices and were therefore unreliable. Notably, DOE  did not include any contingency in its schedule, giving itself less than a 1  percent chance of success—said another way DOE gave itself a 99  percent chance of failure in meeting its schedule to restart operations.  Moreover, when DOE revised its cost and schedule estimates for WIPP  recovery it still did not follow all best practices. DOE did not develop its  estimates for WIPP recovery following best practices, in part because  DOE does not require its cleanup operations activities, such as WIPP, to  follow them. This lack of a requirement is in contrast to a new policy put  into effect in June 2015 by the Secretary of Energy, which established the  requirements that DOE develop cost and schedule estimates for its  capital asset projects following best practices—such projects include the  new permanent ventilation system at WIPP. Without similar requirements  for EM operations activities to follow best practices, DOE cannot have  confidence that it is producing reliable baselines needed to monitor its  performance in managing these activities and reduce the risk of cost  overruns and delays in achieving its cleanup missions, such as the  permanent disposal of transuranic waste from DOE sites across the  country. By also requiring cleanup operations to follow best practices,  DOE would have more confidence in the estimates for its cleanup  operations activities and its capital asset projects.

In selecting the preferred alternative for the new permanent ventilation  system at WIPP, DOE relied on an AOA process that did not follow all  best practices. In particular, DOE only partially followed or did not follow  best practices to define functional requirements based on the mission  need, compare alternatives using a cost-benefit analysis, or  independently review the AOA process. By not following these and other  practices, DOE’s AOA process was not reliable. Notably, DOE conducted  an independent review of the process which found that the project team  did not adequately document a cost-benefit analysis and that, as a result,  the selection of the preferred alternative was not supported by compelling  information. Nonetheless, DOE did not implement the recommendation of  the review to conduct a cost-benefit analysis before the department  selected the preferred alternative. DOE officials explained that the project  team was not required by DOE’s Order 413.3B to implement the  recommendation. By not implementing the recommendation of the  independent review or not justifying and documenting the reason for not  doing so, DOE cannot provide assurance that the final selected  alternative would best provide the capabilities needed at WIPP. More  broadly, without requiring in its Order 413.3B that recommendations from  independent reviews of AOAs be implemented or that the reason for not  doing so be justified and documented, DOE cannot have assurance that it  selects, for all capital asset projects, preferred alternatives from its AOAs  that best meet the mission need.

section_title: Recommendations for Executive Action
paragraphs: To help ensure that DOE develops and uses reliable cost and schedule  estimates and AOAs, we recommend that the Secretary of Energy take  the following three actions:

Direct EM to revise its protocol governing cleanup operations activities  to require use of best practices in developing cost and schedule  estimates.

Direct EM to implement the recommendation made by DOE’s Office of  Project Management Oversight and Assessments in its independent  review of the AOA for WIPP’s new permanent ventilation system to  perform a cost-benefit analysis consistent with best practices for  conducting an AOA, or justify and document why the office does not  intend to do so.

Direct DOE to revise its Order 413.3B to require that DOE offices  implement any recommendations from an independent review of the  extent to which an AOA followed best practices, or justify and  document the rationale for not doing so.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOE for review and comment. In  written comments, reproduced in appendix V, DOE concurred with the  report’s recommendations. DOE stated that our recommendations were  consistent with its commitment to continuous improvement in project  management. In addition, DOE stated that it had two issues that the  department believed needed to be addressed. DOE also provided  technical comments that were incorporated, as appropriate.

DOE identified the following two issues:

DOE stated that WIPP recovery activities were under way concurrent  with DOE-wide efforts to improve project management, including  revisions to its project management order (DOE Order 413.B). DOE  stated that, consequently, EM project management practices were  separate from evolving department project management guidance  and that it is now in the process of bringing EM practices into  conformance with the department’s project management guidance.

DOE stated that the report needs to be clear, in each instance,  regarding the $64 million cost increase for WIPP attributed to  schedule delays in recovery activities, to indicate that $2 million of the  cost increase was WIPP recovery project-related and that $61.6  million was due to reexamination and assessment of the cost of base  operations.

We do not believe changes are needed to the report for either of these  issues. The report explains that WIPP recovery activities were based on  EM’s operations activities protocol and that actions were under way within  the department to improve project management, including revisions to  DOE’s project management order. In addition, the report includes a  breakout of the cost increase. We did, however, revise the conclusions to  reflect this breakout.

As noted above, DOE concurred with the report’s three  recommendations. Regarding the first recommendation—that EM revise  its protocol governing cleanup operations activities to require use of best  practices in developing cost and schedule estimates—DOE stated in its  written comments that it concurred with clarification. DOE stated that EM  is transitioning from the operations activities protocol to a new directive  that is expected to include a key decision approving a cost and schedule  baseline. As EM develops the guidance for this key decision, it will  include the use of cost and schedule best practices. DOE stated that EM  plans to finalize this new directive by September 2016 and seek  departmental approval by December 2016.

Regarding the second recommendation— that EM implement the  recommendation made by DOE in its independent review of the AOA for  WIPP’s new ventilation system to perform a cost-benefit analysis  consistent with best practices, or justify and document why it does not  intend to do so—DOE stated in its written comments that it concurred with  clarification. DOE stated that in accordance with GAO best practices,  further cost-benefit analysis will be conducted on the project prior to  approval of Critical Decision-2 (Approve Performance Baseline). DOE  stated that several alternatives remain to be evaluated including the size  of the ventilation system and the location of the exhaust shaft.

Regarding the third recommendation—that DOE revise its project  management (Order 413.3B) to require that DOE offices implement  recommendations from independent reviews of the extent to which an  AOA followed best practices, or justify and document the rationale for not  doing so—DOE concurred with the recommendation. DOE stated that it  will prepare a project management policy on how DOE offices should  respond to recommendations from independent reviews. DOE stated that  it will prepare this policy by December 2016 and update DOE Order  413.3B at the next available opportunity.

We are sending copies of this report to the appropriate congressional  committees, the Secretary of Energy, and other interested parties. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.

If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix VI.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our report examined the extent to which the Department of Energy (DOE)  (1) met its initial cost and schedule estimates for restarting waste disposal  operations at the Waste Isolation Pilot Plant (WIPP) and (2) followed best  practices in analyzing and selecting an alternative for the new ventilation  system. To address both objectives, we conducted a site visit to WIPP in  January 2015. During the site visit we obtained documentation and  interviewed officials from DOE’s Office of Environmental Management  (EM), which is responsible for oversight of WIPP and exercises this  responsibility primarily through its Carlsbad Field Office. We also  interviewed representatives at WIPP from Nuclear Waste Partnership  LLC (NWP), which is the private contractor that manages and operates  WIPP for DOE.

To examine the extent to which DOE met its initial cost and schedule  estimates for restarting waste disposal operations, we compared DOE’s  initial cost and schedule estimates to restart limited waste disposal  operations contained in its February 2015 WIPP recovery project  management baseline with DOE’s revised estimates in its January 2016  integrated project management baseline for WIPP. We reviewed DOE’s  budget justification supporting the President’s fiscal year 2016 budget  request to Congress from February 2015 and the amounts DOE received  for WIPP in the 2016 Consolidated Appropriations Act. We reviewed  DOE’s reports on the reasons it exceeded the initial estimates for  restarting waste disposal operations and DOE’s risk management plans  for WIPP recovery prepared by NWP and used in developing the  February 2015 estimates. We also compared DOE’s initial cost and  schedule estimates to restart limited waste disposal operations from the  February 2015 WIPP recovery project management baseline with the  best practices described in our cost and schedule guides that identified  the characteristics of high-quality, reliable cost and schedule estimates  because these were the approved estimates when we conducted our  analysis. Specifically, we compared DOE’s initial WIPP recovery cost  estimate presented in DOE’s project management baseline document  and supporting documents and data with the best practices in our Cost  Estimating and Assessment Guide. In addition, we compared DOE’s  initial WIPP recovery schedule presented in DOE’s project management  baseline document and supporting documents and data with the best  practices in our Schedule Assessment Guide. We interviewed the  Carlsbad Field Office officials who oversee the recovery project and  NWP’s cost estimator and scheduler. We provided a draft of our cost and  schedule assessments to the Carlsbad Field Office and NWP and revised  the draft, as appropriate, after discussing our assessment with the federal  officials and the contractor. We reviewed documentation on the revised  (January 2016) baseline and interviewed DOE and NWP officials on the  approach followed to develop the revised cost and schedule estimates to  restart WIPP’s waste disposal operations in the baseline, but we did not  assess the revised estimates against the best practices because of the  time frame of our review.

To examine the extent to which DOE followed best practices in analyzing  and selecting an alternative for WIPP’s new ventilation system, we used  as criteria the best practices for conducting an AOA identified in  GAO-15-37 issued in December 2014. GAO developed the best  practices identified in this report by reviewing AOA policies and guidance  used by seven public and private-sector entities with experience in the  AOA process, and verified these practices with subject-matter experts.  DOE’s AOA process for the WIPP ventilation system consisted of three  elements: NWP’s January 2015 AOA that resulted in the selection of two  preferred alternatives proposed for further analysis, a second AOA  completed by Trinity Engineering Associates (Trinity) in October 2015  which resulted in a single preferred alternative proposed to DOE, and  DOE’s final alternative evaluation process led by the Carlsbad Field  Office that considered the three alternatives proposed by the initial  studies. Our analysis assessed the overall AOA process considering each  element. We compared the process with the best practices and  determined a score for the overall process. We reviewed project  documentation from the Carlsbad Field Office, NWP, and Trinity and  interviewed Carlsbad Field Office officials and NWP representatives in  charge of the AOA. In addition, in October 2015, as we were conducting  our engagement, EM and DOE’s Office of Project Management Oversight  and Assessments completed separate assessments of the WIPP AOA for  the new ventilation system. We reviewed documentation of these reviews  and interviewed the DOE officials who worked on them. We examined the  extent that the independent assessments followed the best practice to  have an independent entity assess the extent that a project’s AOA  followed all best practices.

To score DOE’s AOA process, a GAO analyst examined the AOA  documentation received from the agency and then assigned a score for  each of the 24 best practices. Following this, a GAO AOA specialist  independent of the engagement team reviewed the AOA documentation  and the scores assigned by the analyst for accuracy and cross-checked  the scores in all the analyses for consistency. We used a five-point  scoring system to determine the extent to which DOE’s AOA process  conformed to the best practices. After determining a score for each  individual best practice, we calculated the score for each category—(1)  general principles, (2) identifying alternatives, (3) analyzing alternatives,  and (4) selecting a preferred alternative—by calculating the average of  the scores for the best practices that fall under each category. If the score  for each best practice and the average score for each category was “fully  met” or “substantially met,” we concluded that the AOA process  conformed to best practices and therefore could be considered reliable. In  contrast, if the score was “partially met,” “minimally met,” or “not met,” we  concluded that the AOA process did not conform to best practices and  therefore could not be considered reliable. We shared our analysis with  DOE officials and representatives from NWP for review and incorporated  their technical comments and any additional evidence they provided in  our analysis, as appropriate. We also interviewed officials from DOE’s  Office of Enterprise Assessments, which provides internal oversight of  DOE facilities; the Defense Nuclear Facilities Safety Board, which  provides external oversight of DOE defense nuclear facilities; the U.S.  Mine Safety and Health Administration, which provides external oversight  of mining activities at WIPP; as well as the U.S. Environmental Protection  Agency and the New Mexico Environment Department, both of which  provide external regulation of WIPP.

We conducted this performance audit from November 2014 to August  2016 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix II: Assessment of DOE’s Cost Estimate for the WIPP Recovery Project Compared with Best Practices
paragraphs: Table 2 below assesses the Department of Energy’s (DOE) initial cost  estimate in its February 2015 project management baseline for the Waste  Isolation Pilot Plant (WIPP) recovery project compared to best practices  for cost estimating published in the GAO Cost Estimating and  Assessment Guide. Overall, of four characteristics of a high-quality and  reliable cost estimate, DOE’s initial cost estimate substantially met two  characteristics—comprehensive and well-documented—but partially met  or minimally met the other two characteristics—accurate and credible.

section_title: Appendix III: Assessment of DOE’s Schedule for the WIPP Recovery Project Compared with Best Practices
paragraphs: Table 3 below assesses the Department of Energy’s (DOE) initial  schedule in its February 2015 project management baseline for the  Waste Isolation Pilot Plant (WIPP) recovery project compared to best  practices for developing a schedule estimate published in the GAO  Schedule Assessment Guide. Overall, DOE’s schedule substantially met  two of the four characteristics of a high-quality schedule—comprehensive  and controlled—but partially met the characteristics of well-constructed  and credible.

section_title: Appendix IV: Assessment of DOE’s Analysis of Alternatives for the WIPP Ventilation System Project Compared with Best Practices
paragraphs: Table 4 below compares the Department of Energy’s (DOE) analysis of  alternatives (AOA) process completed in December 2015 for the new  permanent ventilation system at the Waste Isolation Pilot Plant (WIPP) to  best practices for conducting an AOA published in a December 2014  GAO report. DOE’s overall AOA process consisted of an initial AOA by  DOE’s WIPP contractor, Nuclear Waste Partnership LLC, completed in  January 2015; a second AOA by a contractor unaffiliated with WIPP,  Trinity Engineering Associates, completed in October 2015; and DOE’s  final evaluation of the three preferred alternatives proposed by the  contractors’ analyses. Overall, DOE’s AOA process fully met the category  for identifying alternatives and partially or minimally met the other three  categories of best practices—general principles, analyzing alternatives,  and selecting the preferred alternative.

section_title: Appendix V: Comments from the Department of Energy

section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: David C. Trimble, (202) 512-3841 or trimbled@gao.gov.

section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Daniel Feehan, Assistant  Director; Cheryl Arvidson; Mark Braza; Richard P. Burkard; Jennifer  Echard; Brian M. Friedman; Carly Gerbig; Jason Lee; Eli Lewine; Cynthia  Norris; and Katrina Pekar-Carpenter made key contributions to this report.


Question-summary annotations: questions: question: What is WIPP?
answer: DOE's WIPP is the only deep geologic repository for the disposal of U.S. defense-related nuclear waste.

question: Why were waste operations suspended?
answer: In February 2014, waste operations were suspended following a truck fire and an unrelated radiological release.

question: What did the DOE state as their goal in February 2015?
answer: DOE estimated in February 2015 that it would complete recovery activities and restart limited waste operations by March 2016.
child_questions: question: How did they plan to achieve their goal?
answer: To resume full operations, DOE planned to build a new ventilation system at WIPP. DOE completed an AOA to identify the best solution for this system in December 2015.

questions: question: What was included in the Senate Report?
answer: The Senate Report accompanying a bill for the National Defense Authorization Act for Fiscal Year 2015 included a provision for GAO to review WIPP operations.
child_questions: question: What did the report examine?
answer: This report examines the extent to which DOE (1) met its initial cost and schedule estimates for restarting waste disposal operations, and (2) followed best practices in analyzing and selecting an alternative for the new ventilation system.

question: What did GAO examine?
answer: GAO examined documentation on the WIPP recovery estimates.
child_questions: question: What did the GAO do when examining WIPP recovery estimates?
answer: GAO compared DOE's February 2015 cost and schedule estimates and AOA with best practices GAO published.