Nuclear Waste: DOE Needs to Improve Cost Estimates for Transuranic Waste Projects at Los Alamos
Published: 2015-02-18T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Nuclear weapons activities at LANL have generated large quantities of TRU waste that must be disposed of properly. To address a 2005 cleanup agreement with the state of New Mexico requiring DOE to close LANL's TRU waste site, NNSA is to oversee two TRU waste projects. The first is to remove the waste stored at LANL and ship it to WIPP for permanent disposal. The second is to construct a facility—the TWF—to provide new capabilities for managing newly generated TRU waste at LANL. NNSA has developed cost estimates for both projects.
GAO was asked to review cost estimates for the TRU waste projects at LANL. This report examines (1) the extent to which NNSA's TRU waste removal project at LANL has met its cost estimates and (2) the extent to which NNSA's cost estimate for the TWF met best practices for a reliable estimate. GAO reviewed spending data for the TRU waste removal project for fiscal years 2006 through 2014 and the cost estimates for both projects, compared the cost estimate for the TWF with best practices, and interviewed agency officials.
section_title: What GAO Found
paragraphs: The National Nuclear Security Administration's (NNSA) project to remove transuranic (TRU) waste—primarily discarded equipment and soils contaminated with certain radioactive material—at Los Alamos National Laboratory (LANL) did not meet its cost estimates. At the end of fiscal year 2014, NNSA had spent about $931 million on the project, exceeding its 2006 estimate of $729 million by $202 million. Under current plans, the project is also expected to exceed its 2009 estimate. NNSA did not meet its cost estimates, in part, because they were based on aggressive funding assumptions designed to meet the completion dates agreed to in a 2005 cleanup agreement, which the Department of Energy (DOE) did not fully fund. At the time of GAO's review, NNSA was developing a new project completion cost estimate of about $1.6 billion, with completion projected for October 2022. NNSA had not revised the project's cost estimate since 2009 because the agency was reluctant to approve an estimate with a completion date that conflicted with the 2005 cleanup agreement. However, according to an NNSA official, NNSA's new estimate may not reflect current conditions—partly because of uncertainty created by funding and the indefinite suspension of shipments of TRU waste to the permanent repository at DOE's Waste Isolation Pilot Plant (WIPP) after a radioactive release closed WIPP in February 2014. By revising the estimate to include the current understanding of project conditions, including the uncertainty at WIPP, NNSA program managers can, for example, more accurately identify cost overruns.
NNSA's cost estimate for the TRU Waste Facility (TWF), which consisted of separate cost estimates for completing construction and for operations and maintenance, partially reflected each of the four characteristics of a reliable estimate (comprehensive, well-documented, accurate, and credible) as established by best practices. For example, NNSA's estimate was partially well-documented by clearly documenting the data sources and methodology used to develop the construction estimate. However, NNSA did not sufficiently document the approach used to develop the operations and maintenance estimate, which represented about 74 percent of the TWF's life-cycle costs, because DOE's project management order does not require these costs to be documented when a project is approved to request funding from Congress for construction. As a result, GAO could not determine whether the cost-estimating approach was appropriate. In addition, NNSA's estimate was partially credible because NNSA completed an independent cost estimate (ICE) that provided an unbiased cross-check of the construction estimate consistent with best practices, but it did not include the operations and maintenance costs in the ICE because it was not required by DOE's project management order. Moreover, NNSA did not conduct a sensitivity analysis to quantify variations in the TWF's cost estimates due to changes in key assumptions because it was not required by DOE, which also affected the estimate's credibility. Doing a sensitivity analysis increases the chance that decisions for the TWF will focus on the elements that have the greatest effect on cost, according to best practices. Updating the TWF's cost estimate to include all life-cycle costs and needed analyses, would provide NNSA more reliable information for better managing the TWF as it prepares for the start of operations, which NNSA expects could be as early as April 2016.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOE revise the cost estimate for the TRU waste removal project to reflect the current understanding of project conditions and update the TWF's cost estimate to allow better management of the project's life-cycle costs going forward. DOE generally agreed with GAO's recommendations.

section_title: Letter
paragraphs: Since the federal government established Los Alamos National  Laboratory (LANL) in New Mexico in 1943, nuclear weapons research  and development activities there have contributed to the nation’s defense  but have also created large quantities of radioactive waste. This waste  includes transuranic (TRU) solid waste, which typically consists of  discarded rags, tools, equipment, soil, or other solid materials that have  been contaminated by certain man-made radioactive elements,  particularly plutonium. Exposure to TRU waste is dangerous. In  particular, internal exposure to plutonium through inhaling or swallowing  is an extremely serious health hazard. Because plutonium generally stays  in the body for decades, it exposes organs and tissues to radiation,  increasing the risk of cancer. TRU waste remains radioactive for  extremely long periods: hundreds of thousands of years, in some cases.  To reduce the long-term risks of exposure to this waste, TRU waste must  be disposed of properly.
Today, LANL is one of the three national laboratories that support the  National Nuclear Security Administration’s (NNSA) mission to maintain a  safe, secure, and reliable nuclear weapons deterrent. NNSA—a  semiautonomous agency established within the Department of Energy  (DOE)—is responsible for managing the nation’s nuclear weapons. LANL  houses most of NNSA’s capabilities for conducting plutonium research  and for producing plutonium “pits,” which are the central cores of nuclear  weapons. Each year, these activities generate new TRU waste at LANL.
For many years, the TRU waste generated at LANL has been stored and  processed at a location called Area G. The activities that generated TRU  waste at LANL also generated hazardous waste that requires  environmental cleanup. Under a 2005 consent order agreement with the  New Mexico Environment Department, DOE is required to, among other  things, clean up the hazardous waste contamination at Area G and then  close the site by December 6, 2015. To close Area G, DOE must first  complete two TRU waste projects: (1) a TRU waste removal project; and  (2) a TRU Waste Facility (TWF) construction project.
The goal of the TRU waste removal project is to safely and securely  transfer the TRU waste stored at Area G to the DOE Waste Isolation Pilot  Plant (WIPP) near Carlsbad, New Mexico.  DOE must complete the TRU  waste removal project before it can complete the hazardous waste  cleanup actions required to close Area G. Most of the TRU waste stored  at Area G is categorized as legacy waste, which NNSA defines as waste  generated at LANL before fiscal year 1999, as opposed to newly  generated waste, which was generated starting in 1999. As the office  within DOE responsible for legacy waste cleanup operations, DOE’s  Office of Environmental Management (EM) receives funding for cleanup  projects, including the TRU waste removal project, and has established  the project management protocols that apply to the project. Unlike at  other DOE sites where EM directly oversees cleanup projects, NNSA has  directly overseen the management and operating contractor at LANL  responsible for the project using EM’s Operations Activity Protocol. EM  and NNSA have collaborated to develop the cost estimates for the  project. In 2006, EM and NNSA estimated that the TRU waste removal  project would cost $729 million to complete by 2012. After encountering  problems maintaining the schedule established as part of the 2006  estimate, EM and NNSA revised the estimate in 2009 to a cost range of  $848 million to $1.2 billion, with a completion date of 2018. However, in  May 2014, DOE placed the TRU waste removal project on hold  indefinitely in response to an incident in February 2014, at WIPP that has  halted all TRU waste shipments to WIPP. The incident involved a TRU  waste container shipped from LANL that ruptured while in storage,  releasing radioactive material. In September 2014, the Secretary of  Energy directed EM and NNSA to develop a plan to transition the direct  federal oversight of the legacy environmental cleanup work at LANL,  including the TRU waste removal project, from NNSA to EM in order to  align the focus and accountability of the cleanup work with EM and  enable NNSA’s management and operating contractor at LANL to focus  on the core national security missions at the lab.
The goal of the TWF construction project is to replace LANL’s capabilities  that currently reside in Area G for storing and certifying newly generated  TRU waste containers for shipment to WIPP. DOE cannot close Area G  until the replacement facility is operational because LANL needs  continuous TRU waste capabilities to support the ongoing nuclear  weapons mission. NNSA is overseeing the TWF construction project  using DOE’s project management order for capital asset acquisition  projects. In February 2013, NNSA approved the baseline cost estimate  of $99.2 million to complete the TWF’s construction and prepare it for  operations. According to NNSA’s current plan, it expects to complete  construction of the TWF between April 30, 2016, and January 31, 2018.  NNSA also estimated that, once completed, the facility will cost $300  million to operate and maintain for its projected useful life of 50 years,  spanning 2018 through 2068.
Since 1990, DOE’s management of contracts and projects, including  those executed by NNSA, has been on our list of areas at high risk for  fraud, waste, abuse, and mismanagement. In particular, DOE and NNSA  have experienced long-standing difficulties in preparing reliable cost  estimates for cleanup projects and capital asset construction projects, as  well as in meeting their cost estimates. In our 2013 High-Risk Update, to  acknowledge progress DOE, including NNSA, has made in managing  nonmajor projects (i.e., those costing less than $750 million), we  narrowed the focus of DOE’s high-risk designation to major contracts and  projects (i.e., those costing $750 million or greater) but noted that we  would continue to monitor nonmajor projects to ensure that progress in  this area continues and is sustained. In March 2009, we issued the GAO  Cost Estimating and Assessment Guide (Cost Guide), which identifies the  four characteristics of a high-quality, reliable cost estimate based on best  practices used throughout government and industry: comprehensive,  well-documented, accurate, and credible.
You asked us to review issues related to the cost estimates for removing  the TRU waste stored at LANL and completion of the TWF at LANL. This  report examines the extent to which (1) NNSA’s TRU waste removal  project at LANL has met its cost estimates and (2) NNSA’s cost estimate  for the TWF project at LANL met best practices for a reliable cost  estimate.
To examine the extent to which NNSA’s TRU waste removal project at  LANL has met its cost estimates, we reviewed documentation on the cost  estimates to complete the project that were set in 2006 and 2009. We  compared the cost estimates with data provided to us by NNSA on the  dollars spent for fiscal years 2006 through 2013 for the project, as well as  NNSA’s estimate of fiscal year 2014 year-end spending for the project.  We also reviewed NNSA’s fiscal year work plans for 2012 through 2014,  the most recent work plans after NNSA adopted EM’s Operations  Activities Protocol for managing the project at LANL. We interviewed  NNSA and EM officials to understand changes in the project’s cost and  schedule assumptions and to clarify the cost-estimating requirements for  DOE cleanup projects. To examine the extent to which NNSA’s cost  estimate for the TWF project at LANL met best practices for a reliable  cost estimate, we focused on the estimated life-cycle costs of the TWF.  These estimated costs consisted of NNSA’s performance baseline cost  estimate for completing the TWF Phase B- construction project  (construction estimate) approved in February 2013 and NNSA’s estimate  of the TWF’s operations and maintenance costs for the projected 50-year  useful life of the facility. NNSA reported the construction estimate and the  operations and maintenance estimate in its construction project data  sheet for the TWF project in DOE’s fiscal year 2014 congressional budget  justification documents. We analyzed NNSA’s cost-estimating approach  against the best practices found in the GAO Cost Guide. Additional  details on our objectives, scope, and methodology can be found in  appendix I.
We conducted this performance audit from July 2013 to February 2015 in  accordance with generally accepted government standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
subsections: section_title: Background
paragraphs: This section describes (1) DOE’s project management requirements for  capital asset projects and cleanup projects, (2) TRU waste operations at  Area G, (3) the TRU waste removal project, (4) the TWF construction  project, and (5) GAO’s Cost Guide.
subsections: section_title: DOE’s Project Management Requirements for Capital Asset Projects and Cleanup Projects
paragraphs: DOE has established separate project management requirements for its  capital asset projects and certain cleanup projects defined as operations  activities. DOE’s project management order for capital asset projects,  Order 413.3B, establishes the requirements for managing capital asset  projects, and EM’s Operations Activities Protocol establishes the  requirements for managing cleanup projects defined as operations  activities. The TRU waste removal project is a cleanup operations  activity and is subject to EM’s Operations Activities Protocol, whereas the  TWF is a capital asset construction project and must be carried out in  accordance with Order 413.3B.
subsections: section_title: DOE’s Project Management Order 413.3B
paragraphs: DOE’s project management order for capital asset projects, Order  413.3B, applies to all capital asset projects with a total project cost  greater than or equal to $50 million. Capital asset projects include  construction projects that build large complexes that often house unique  equipment and technologies such as those that process TRU waste or  other radioactive material. DOE’s order establishes a process for NNSA  and other DOE offices to manage projects, from identification of need  through project completion, with the goal of delivering projects within the  original performance baseline that are fully capable of meeting mission  performance and other requirements, such as environmental, safety, and  health standards. In particular, the order defines five major milestones— or Critical Decision (CD) points—that span the life of a project:
CD-0: Approve mission need. DOE identifies a credible  performance gap between its current capabilities and capacities and  those required to achieve the goals defined in its strategic plan. The  mission need translates this gap into functional requirements. DOE  formally establishes a project and begins the process of conceptual  planning and identifying a range of alternative approaches to meet the  identified need.
CD-1: Approve alternative selection and cost range. DOE  completes the conceptual design and selects its preferred approach  based on analysis of life-cycle costs, and approves the project’s  preliminary cost range to complete the project’s design and  construction.
CD-2: Approve the performance baseline. A project’s performance  baseline consists of key cost, scope, schedule, and performance  parameter targets. The project’s scope defines the technical goals  and requirements that the project is to deliver at completion. The  performance baseline cost includes the entire project budget, or total  project cost, and represents DOE’s commitment to Congress. At this  milestone, DOE completes its preliminary design and develops a  definitive cost estimate that is a point estimate and no longer a range.
CD-3: Approve the start of construction. Design and engineering  are essentially complete and have been reviewed, and project  construction or implementation begins.
CD-4: Approve the start of operations or project completion. For  construction projects, at this milestone, DOE completes the project  and begins the transition to operations.
DOE’s project management order for capital asset projects specifies the  requirements that must be met, including for developing project cost  estimates, along with the documentation necessary, to move a project  past each CD point. In addition, the order requires senior management to  review the supporting documentation and decide whether to approve the  project at each CD point. DOE also provides suggested approaches for  meeting the requirements contained in the order through a series of  guides, such as guides for cost estimating and project reviews.
section_title: EM’s Operations Activities Protocol
paragraphs: The life-cycle cost estimate data are maintained in EM’s Integrated Planning,  Accountability, and Budgeting System and are used to develop DOE’s reported  environmental liability. performance and, if practical, how that performance affects the life- cycle cost estimate and contract period of performance baseline.
section_title: TRU Waste Operations at Area G
paragraphs: Before 1970, TRU waste generated at LANL was managed as low-level  radioactive waste and was disposed of at Area G in pits and trenches  along with hazardous waste. In 1970, in response to concerns that TRU  waste remained radioactive for an extremely long time and scientific  research recommending deep geologic disposal for this waste, the Atomic  Energy Commission––a DOE predecessor––directed sites that generated  TRU waste to begin segregating it from other waste and storing it in  retrievable packages for an interim period, pending disposal in a  repository. As a result of the directive, starting in the early 1970s, the  TRU waste generated at LANL was stored in segregated TRU waste pits  and trenches and aboveground in fabric domes so that it could be more  easily retrieved when a permanent repository site opened. (see fig. 1).
Today, Area G serves as LANL’s primary location for storing and  processing TRU waste. Both legacy and newly generated TRU waste are  stored and processed for shipping to WIPP at facilities in Area G. TRU  waste operations at Area G include the following processes: packaging waste into 55-gallon drums or other approved containers  following DOE standards, called waste acceptance criteria, to protect  human health, safety, and the environment during the waste’s  transport to and disposal in WIPP;  repackaging containers if they are found to not meet WIPP’s waste  acceptance criteria;  resizing large waste using methods such as cutting it into smaller  pieces so that it can be placed into approved containers; characterizing the waste by using specialized scanning equipment to  assess the contents of each waste container and the amount of  radioactivity it contains; and certifying the waste to declare that it meets WIPP’s waste acceptance  criteria.
section_title: TRU Waste Removal Project
paragraphs: Starting in 2011, NNSA and the New Mexico Environment Department  agreed to significant changes in the strategy for completing the TRU  waste removal project. In that year, a wildfire occurred near Area G,  resulting in increased public concern about the risk posed by the TRU  waste stored aboveground at Area G. To address this risk, in 2012,  NNSA and the New Mexico Environment Department reached a voluntary  agreement, called the Framework Agreement, which established a June  2014 deadline for the accelerated removal of 3,706 cubic meters of  aboveground TRU waste at a high risk from wildfires. To meet this  deadline, NNSA initiated an effort know as the “3706 Campaign.” To  facilitate this campaign, the New Mexico Environment Department and  DOE agreed to extend other deadlines established under the 2005  Consent Order governing hazardous waste cleanup activities for locations  across LANL. With these deadlines extended, NNSA was able to  reallocate EM funding for environmental cleanup activities at LANL to  focus on the 3706 Campaign. Using the additional funds, NNSA  increased the TRU waste processing capacity at LANL by constructing  more facilities for repackaging waste and hiring additional contractors to  operate the facilities 7 days a week. Under the Framework Agreement,  NNSA also agreed to remove the remaining aboveground TRU waste at  Area G and developed a schedule for removing the belowground TRU  waste at Area G by September 30, 2018. According to NNSA officials,  part of the plan in establishing the Framework Agreement was that, once  the 3706 Campaign was completed, NNSA and the New Mexico  Environment Department would discuss renegotiating the final completion  date for the Consent Order.
On May 30, 2014, DOE announced that NNSA had completed the  removal of about 93 percent of the TRU waste included in the 3706  Campaign but would not meet the June 2014 deadline because of the  department’s decision to halt the TRU waste removal project. As noted  previously, DOE halted the project at LANL in February 2014, in response  to an incident at WIPP that involved a LANL TRU waste container that  ruptured and leaked radioactive material while in storage.
section_title: TWF Construction Project
paragraphs: The TWF project is to replace NNSA’s existing capabilities that reside at  Area G for storage, characterization, and certification of newly generated  TRU waste at LANL. The TWF design includes multiple buildings for  waste storage, waste characterization, and operational support. The  facility will also have space and utility hookups for three mobile trailers to  be provided by WIPP that will contain additional characterization  capabilities needed to certify that TRU waste containers meet WIPP  waste acceptance criteria (see fig. 2).
The TWF is being designed and  constructed as a high-hazard nuclear facility, which must meet nuclear  safety standards for storage and handling of nuclear waste. Nuclear  safety design features of the TWF include a barrier to prevent large  vehicles from crashing into the facility, a seismic power cutoff switch  designed to reduce possible sources of fire that could result from an  earthquake, and a tank to store water to help suppress any earthquake- initiated fire.
NNSA’s Office of Defense Programs, which is the program office  responsible for maintaining the nation’s nuclear weapons stockpile, is  sponsoring the TWF project. The office provides the annual funding for  planning and construction and approves the project’s milestones. NNSA’s  Office of Acquisition and Project Management is responsible for  overseeing the construction of the TWF within NNSA’s approved cost and  schedule estimates. To do so, the office provides direction and oversight  of NNSA’s management and operating contractor at LANL.
NNSA divided the TWF project’s design and construction into two  subprojects: site development and facilities construction. NNSA  completed the site development activities, which included relocation of  utility lines, as well as excavation and grading to prepare the site for the  facility’s construction, in December 2012, at a cost of $7.7 million. In  February 2013, NNSA approved the project’s CD-2 performance baseline  estimate of $99.2 million to construct the TWF with a completion date  between April 30, 2016, and January 31, 2018. As mentioned  previously, NNSA also estimated that the facility will cost $300 million to  operate and maintain for its projected useful life of 50 years, spanning  2018 through 2068. In combination, NNSA’s estimated life-cycle costs for  the TWF when it approved the project’s performance baseline to  complete construction at CD-2 totaled about $406.9 million.
section_title: GAO’s Cost Guide
paragraphs: Drawing from federal cost-estimating organizations and industry, the Cost  Guide provides best practices about the processes, procedures, and  practices needed for ensuring development of high-quality—that is,  reliable cost estimates.characteristics of a high-quality, reliable cost estimate:   The Cost Guide identifies the following four
Comprehensive when it accounts for all life-cycle costs associated  with a project, is based on a completely defined and technically  reasonable plan, and it contains a cost estimating structure in  sufficient detail to ensure that costs are neither omitted nor double- counted;
Well-documented when supporting documentation explains the  process, sources, and methods used to create the estimate and  contains the underlying data used to develop the estimate;
Accurate when it is not overly conservative or too optimistic and is  based on an assessment of the costs most likely to be incurred; and
Credible when a sensitivity analysis has been conducted, the level of  confidence associated with the point estimate has been identified  through the use of risk and uncertainty analysis, and the point  estimate has been cross-checked with an independent cost estimate  (ICE).
To develop a cost estimate that embodies these four characteristics, our  Cost Guide lays out best practice steps. For example, one step in  developing an accurate estimate is to identify and document ground rules  that establish a common set of agreed-on estimating standards and solid  assumptions that are measurable, specific, and consistent with historical  data. According to the Cost Guide, it is imperative that cost estimators  brief management on the ground rules and assumptions used for an  estimate so that management understands the conditions the estimate  was structured on and can avoid overly optimistic assumptions.
section_title: NNSA Has Not Met Its Cost Estimates to Complete the TRU Waste Removal Project at LANL
paragraphs: NNSA’s TRU waste removal project at LANL did not meet its 2006 cost  estimate and is not expected to meet the 2009 cost estimate established  for the completion of the project. During our review, NNSA and EM were  in the process of developing a new cost estimate for the project. The TRU  waste removal project has not met its past cost estimates, partly because  the 2006 and 2009 cost estimates were based on aggressive funding  assumptions to meet the deadlines of the Consent Order. In addition,  because NNSA did not maintain or use two of the three project baselines  outlined in its cleanup project requirements, it could not measure the  progress of the total project.
As of the end of fiscal year 2014, NNSA had spent about $931 million on  the project, which exceeded the 2006 cost estimate of $729 million by  $202 million (see fig. 3). The amount expended by the end of fiscal year  2014 did not exceed the $1.2 billion upper range of the 2009 cost  estimate, but it did exceed the $848 million lower range of the estimate by  $83 million. As of July 2014, the most recent date for which data were  available, NNSA had removed approximately 79 percent of the TRU  waste at LANL; however, the remaining 21 percent includes the waste  buried belowground, which will be the most difficult and expensive to  address, according to NNSA officials. The new fiscal year 2015 draft  estimate, currently under review by NNSA and EM, projects that the final  project costs will be approximately $1.6 billion, or $400 million above the  upper range of the 2009 cost estimate.
NNSA’s TRU waste removal project exceeded the 2006 cost estimate  and is not expected to meet the 2009 cost estimate, in part, because  NNSA and EM developed the cost estimates using aggressive project  funding assumptions that were based on the need to meet the Consent  Order requirement for closing Area G by 2015, according to NNSA and  EM officials. For the 2006 cost estimate, NNSA officials overseeing the  TRU waste removal project developed the parameters of the project  estimate based on the need to remove almost all of the TRU waste by  2012. In particular, to meet these deadlines, NNSA based its cost  estimate on funding projections provided by EM that were consistent with  meeting the Consent Order deadline and that assumed that EM would  increase the yearly funding for environmental cleanup projects at LANL.  According to NNSA and EM officials, they recognized that the funding  assumptions used in the cost estimate were aggressive and that  significant funding shortfalls would inhibit the TRU waste removal  project’s ability to remain on schedule. From fiscal year 2006 through  fiscal year 2008, EM provided approximately $457 million—$284 million  (38 percent) less than the amount requested by NNSA officials for all  cleanup activities at LANL—and, as a result, it was not possible to fund  the TRU waste removal project at the levels established in the 2006  estimate. According to NNSA officials, EM was unable to increase the  funding for LANL cleanup projects due to limited budget flexibility and  competing demands from cleanup projects at other DOE sites. In a 2008  report on LANL’s cleanup efforts, DOE’s Inspector General found that EM  did not have enough money to address all the milestones in the   NNSA officials said that, by  environmental agreements they signed.2009, the TRU waste removal project had fallen behind schedule and  could not be completed by 2012, in part, because of the shortfall in  funding. The extension of the project’s completion date beyond 2012  resulted in additional costs, which contributed to the total cost of the  project exceeding the 2006 estimate.
In 2009, NNSA and EM developed a new cost estimate for the TRU  waste removal project with a completion date in 2018 but again used  aggressive funding assumptions to complete the project as close to the  Consent Order’s 2015 deadline as possible. Similar to the 2006 estimate,  the 2009 cost estimate used funding projections provided by EM that  assumed an increase in the yearly levels of funding for LANL cleanup  activities that were necessary for NNSA to complete the TRU waste  removal project by July of 2018—2 and a half years after the Consent  Order deadline in 2015. However, due to the same budget restrictions  that affected cleanup project funding previously, actual funding levels  provided by EM for fiscal years 2009 to 2012 for all cleanup projects at  LANL again came in below the levels requested by NNSA officials at  LANL. Specifically, LANL received approximately $1 billion over these  years, which was $240 million (19 percent) less than the levels requested  by NNSA officials at LANL for cleanup projects at the site. As a result,  funding for the TRU waste removal project was reduced, and this  reduction caused the project to fall behind the schedule set in the 2009  cost estimate, according to NNSA officials. Moreover, the 2009 cost  estimate was never officially approved by NNSA and EM because the  date estimated for project completion was not consistent with the  requirements of the Consent Order. According to NNSA officials, they  could not formally approve the 2009 estimate because it included a 2018  estimated completion date for the TRU waste removal project, which  conflicted with the required 2015 closure date established in the Consent  Order.
In addition to developing estimates using aggressive funding  assumptions, NNSA did not maintain or use two of the three project  baselines outlined in the Operations Activity Protocol, so the agency  could not measure the progress of the total project. As discussed  previously, NNSA was to manage the TRU waste removal project using  EM’s Operations Activities Protocol, which is intended to provide the  framework for managing and reporting on the progress of cleanup  projects through the use of three performance baselines: life-cycle cost,  contract period of performance, and fiscal year work plan. However, for  the TRU waste removal project, because NNSA did not have an updated  life-cycle cost baseline and did not establish a contract period of  performance baseline, it only used the fiscal year work plan baseline to  manage the project, as discussed below:
Life-cycle cost baseline. NNSA has not updated the life-cycle cost  baseline for the project since 2009, even though agency officials told  us they were aware that the estimate has been out-of-date since  about 2012. Since that year, the project has undergone significant  changes that affected its estimated cost and completion date. For  example, by initiating the 3706 Campaign in 2012, NNSA altered the  scope of work from what was planned in the 2009 cost estimate.  NNSA and EM officials told us that, although they recognized in 2012  that the 2009 estimate was no longer valid, they did not see the  purpose in completing a new estimate before completion of the  campaign and the expected renegotiation of the Consent Order  deadlines. According to these officials, a new cost estimate for the  project would have either needed to use unreasonably high funding  assumptions to achieve project completion by 2015 or, if it used more  reasonable funding assumptions, it could not have been approved  because of the political issues associated with a completion date  beyond the 2015 Consent Order deadline. The Operations Activities  Protocol requires that EM or NNSA Site Office Managers develop cost  estimates that cover the full life-cycle of a cleanup project, but it  leaves the Site Office Manager discretion to determine whether an  update to the life-cycle cost baseline is required. Because NNSA’s  LANL Site Office Manager, in consultation with EM officials, decided  not to update the life-cycle cost baseline, NNSA could not measure  project performance to determine the impact of management actions.  For example, from fiscal year 2012 to 2014, NNSA used additional  funding that was reallocated to the TRU waste removal project to  increase the pace of TRU waste packaging and removal; however,  because they did not have an updated cost estimate to measure  against, NNSA managers were unable to identify the effect these  actions had on the total cost of the project.
Contract period of performance baseline. NNSA has not managed  the TRU waste removal project using a contract period of  performance because, according to NNSA officials, the project, like  other projects at LANL, is being conducted through NNSA’s  management and operating contract. According to the Operations  Activities Protocol, a contract period of performance baseline is  required for those cleanup projects that are executed through a  contract that establishes a performance baseline for cost and scope  over the duration of the contract. The management and operating  contract covers all work performed at LANL, but it does not establish a  cost estimate for specific projects such as the TRU waste removal  project, according to NNSA officials. In contrast, for cleanup projects  at EM-managed sites, the scope of the contract is typically limited to  the cleanup project and would not include other site activities that  were unrelated to the project. According to NNSA officials, executing  the TRU waste removal project through the management and  operating contract does not provide the baseline necessary for  establishing a contract period of performance, so NNSA is not  required to manage to a contract period of performance baseline for  this project. According to EM officials, DOE determined that as part of  EM’s transition to direct oversight of the legacy cleanup work at LANL,  EM will transition away from using a management and operations  contract to manage the remaining cleanup work. When this change in  contract type is completed, the officials stated that a contract period of  performance baseline will be available for monitoring performance of  the work, including the TRU waste removal project.
Fiscal year work plan baseline. NNSA has used the fiscal year work  plans outlined in the Operations Activities Protocol to monitor the  performance of portions of the TRU waste removal project and to  manage and assess the performance of the entire TRU waste  removal project in the absence of an accurate life-cycle cost estimate  baseline or a contract period of performance baseline. According to  NNSA officials, a new fiscal year work plan is developed each year for  the TRU waste removal project using an integrated priorities list for  remaining TRU removal work and the projected funding amount  allocated for LANL cleanup. However, while NNSA and EM site and  headquarters officials monitored progress against the fiscal year work  plan, the agency was unable to evaluate the performance of the entire  TRU waste removal project and identify potential cost overruns  because its life-cycle cost estimate was out-of-date, and it did not  manage to a contract period of performance. The Operations  Activities Protocol requires that the Site Office Manager report yearly  on any variances between total project costs to date and the  estimated costs for the entire project as part of the fiscal year work  plans. However, because NNSA did not have an accurate cost  estimate for the TRU waste removal project, the 2012, 2013, and  2014 fiscal year work plans used the outdated 2009 cost estimate to  report on the variances between the current and estimated costs for  the project. As a result, these fiscal year work plans did not provide  accurate information on project performance to date to help managers  measure total project performance and manage costs.
At the time of our review, NNSA and EM were in the process of  developing a new cost estimate for the TRU waste removal project that  they expect to complete in fiscal year 2015; however, this estimate may  quickly become inaccurate due to assumptions related to funding and the  status of WIPP that could be invalidated. As discussed previously, the  new draft cost estimate increases the estimated total cost of the project to  $1.6 billion, with a completion date in fiscal year 2023 (i.e., October  2022). According to an NNSA official, this draft estimate uses a more  conservative approach than past estimates by expanding the scope of the  project estimate to include the costs to remove additional TRU waste that  was not included in past estimates.concerning when WIPP will reopen and the need to address the  impending Consent Order deadlines in 2015 that have not yet been  renegotiated with New Mexico, the same NNSA official told us that  several of the new estimate’s assumptions are no longer valid or may not  be valid after a few years. For example, the funding assumptions in the  new estimate may already be invalid. With the TRU waste removal project  on hold as a result of the WIPP closure, the New Mexico Environment  Department is no longer providing deadline extensions for some of  LANL’s hazardous waste cleanup work and, as a result, NNSA officials  managing the cleanup work have had to reprioritize their funding to  attempt to meet those deadlines. Restoring funding to hazardous waste  cleanup projects reduces the funding available for TRU waste removal,  which would invalidate the funding assumptions used in the new cost  estimate. In addition, the NNSA official told us that the new cost estimate  also assumes that LANL will resume TRU waste shipments to WIPP in  fiscal year 2017 based on an unofficial estimate from the WIPP manager  that WIPP will reopen between 18 and 30 months after the initial   However, in light of the uncertainty  assessment of the incident.shipments from LANL in this time frame, the TRU waste removal project  at LANL would be delayed, resulting in additional costs not accounted for  in the new estimate, such as the costs for maintaining the project  workforce longer than anticipated.
If WIPP does not start accepting TRU waste  To objectively measure the performance of the TRU waste removal  project and take action to manage project costs, NNSA managers of the  project need an updated cost estimate. EM’s Operations Activity Protocol  leaves it to the discretion of the Site Office Manager to update cleanup  project estimates, and NNSA’s LANL Site Office Manager chose to delay  revising the outdated 2009 estimate. As a result, DOE does not have an  estimate of the total cost or completion date of the TRU waste removal  project that uses updated assumptions based on the current  understanding of project conditions. NNSA and EM are developing a new  cost estimate for the project; however, the new estimate may quickly  become inaccurate because of changes in funding, and the status of the  WIPP may soon invalidate its assumptions. According to best practices  for cost estimating, maintaining an updated cost estimate is critical so that  officials making decisions about the future management of a project have  accurate information for assessing their alternatives. By revising the TRU  waste removal project’s estimate to include the current understanding of  project conditions, NNSA program managers could more accurately  identify cost overruns.
section_title: NNSA’s Cost Estimate for the Transuranic Waste Facility Project at LANL Partially Met Best Practices
paragraphs: NNSA’s cost estimate for the TWF partially met best practices. More  specifically, NNSA’s cost estimate—which consisted of separate cost  estimates for completing construction and for operations and  maintenance, as the TWF’s life-cycle costs—partially reflected each of  the four characteristics of a reliable estimate (comprehensive, well- documented, accurate, and credible) as established by best practices.  In developing the construction estimate, NNSA took several steps that  conformed to best practices, such as validating the construction estimate  by completing an ICE. Although DOE’s project management order for  capital asset projects does not require the use of all best practices in  developing a cost estimate, DOE’s related cost-estimating guidance,  which is optional, describes most of them. In contrast, in developing its  operations and maintenance cost estimate, NNSA did not take steps that  conformed to best practices. In particular, NNSA did not sufficiently  document the approach (i.e., data sources and methodologies) used to  develop the estimate, even though operations and maintenance costs  represented about 74 percent of the total life-cycle costs of the facility.  The reason was that operations and maintenance cost estimates for a  construction project do not need to be updated and documented at CD-2  under DOE’s project management order, as funds for these costs do not  need to be specifically requested from Congress to complete the project.  By not sufficiently documenting the approach used to develop the  estimate, NNSA may not have reliable information to support budgetary  decisions for funding the TWF’s operations and maintenance in the  future. For example, the contractor’s representatives told us that they  estimated the operations and maintenance costs to be $6 million annually  from 2018 through 2068, for a total of $300 million but did not use an  inflation rate in the calculations. Thus, although $6 million may be an  accurate estimate for the first year, without documentation of the  approach used to develop the estimate, we could not determine its  reliability for the first or future years.
Appendix II provides a summary description of our assessment of  NNSA’s cost estimates for the TWF project’s construction and operations  and maintenance. The following are examples from our assessment, by  best practice characteristic:
Comprehensive. The TWF estimates partially reflected the  characteristics of comprehensive cost estimates. For example, NNSA  partially followed the best practice for completely defining the  program, as NNSA’s contractor based the TWF construction cost  estimate on a mature design plan that detailed the technical  requirements and characteristics for the TWF. In contrast, for the TWF  operations and maintenance cost estimate, NNSA’s contractor was  not able to provide us with definitions of the technical requirements  and characteristics that would have formed the basis of the estimate,  although the $300 million estimate represented a substantial change  from a $642 million estimate that NNSA’s contractor produced in June  2010 for CD-1 (approve alternative selection and cost range). As a  result, we could not determine whether the $300 million estimate  reflected the most recent TWF design approved at CD-2 to complete  construction. NNSA officials told us they did not develop an updated  and documented basis to support the operations and maintenance  cost estimate because DOE’s project order does not require updated  and documented estimates of all life-cycle costs at CD-2. Instead, at  CD-2, the order focuses on the need for the baseline cost estimate to  complete construction because funding for construction needs to be  specifically requested from Congress to complete the project.  According to best practices, clearly defining the technical  requirements would help to ensure that managers have an adequate  understanding of the facility and where information was limited and  assumptions were made in developing the estimate.
Well-documented. The TWF estimates partially reflected the  characteristics of well-documented cost estimates. For example,  regarding the construction estimate, NNSA’s contractor documented  the data sources and the methodology used to calculate the  construction estimate so that a cost analyst unfamiliar with the project  could understand what was done and replicate the estimate. In  contrast, NNSA did not document the approach (data sources and  methodologies) used to develop the operations and maintenance  estimate, even though the operations and maintenance costs  represented about 74 percent of the TWF’s life-cycle costs. As  mentioned previously, DOE’s project management order does not  require documentation of the operations and maintenance costs at  CD-2. NNSA was required to report the operations and maintenance  cost estimate by following a DOE budget formulation guidance that  did not specify requirements for documenting the estimate. Because  NNSA did not document the approach used, we could not determine  whether it was appropriate for developing the operations and  maintenance estimate; whether NNSA management reviewed the  estimate, including its risks and uncertainties, or whether NNSA  management approved the estimate.
Accurate. The TWF estimates partially reflected the characteristics of  accurate cost estimates. For example, NNSA’s contractor partially  followed the best practice for properly adjusting the estimates for  inflation. Regarding the construction estimate, NNSA’s contractor  developed the estimate using pricing data that were adjusted for  inflation. However, the contractor did not then normalize the data to  remove the effects of inflation. According to representatives from the  contractor, they did not believe data normalization was an applicable  step for the TWF construction estimate because the data set was too  small. According to cost-estimating best practices, data normalization  is often necessary to ensure comparability of data sets because data  can be gathered from a variety of sources and in different forms that  need to be adjusted before being used. Regarding the operations and  maintenance estimate, NNSA’s contractor did not properly adjust the  estimate for inflation over the 50-year useful life of the TWF.  Specifically, the contractor’s representatives told us that they  estimated the operations and maintenance costs to be $6 million  annually from 2018 through 2068, for a total of $300 million, but they  did not use an inflation rate in these calculations. Adjusting for  inflation is an important step in developing an estimate because if the  inflation amount is not correct, the estimate is not accurate. Applying  the wrong inflation rate will either result in a higher cost estimate or  estimated costs that are not sufficient to keep pace with inflation.
Credible. The TWF estimates partially reflected the characteristics of  credible cost estimates. For example, NNSA followed the best  practice to have an ICE completed in January 2013 to validate the  TWF construction costs. However, because DOE’s project  management order does not define the operations and maintenance  costs as project costs, NNSA was not required to include these costs  in the ICE. By not including the TWF operations and maintenance  costs in the ICE, NNSA managers may lack insight into these future  costs. According to best practices, an ICE can provide NNSA  managers with additional insight into the TWF’s potential operations  and maintenance costs—in part, because ICEs frequently use  different methods and are less burdened with organizational bias.  Therefore, according to best practices, an ICE can be used as a  benchmark to assess the reasonableness of the contractor’s  proposed operations and maintenance costs, improving NNSA  management’s ability to make sound investment decisions, and  accurately assess the contractor’s performance. Moreover, because  DOE’s project management order does not require it, NNSA’s  contractor did not follow the best practice to complete a sensitivity  analysis to quantify the extent to which either the construction or  operations and maintenance cost estimates could vary because of  changes in key assumptions and ground rules. Such an analysis is a  best practice because uncertainty cannot be avoided and, therefore, it  is necessary to identify the cost elements that represent the most risk  and, if possible, quantify them. According to cost-estimating best  practices, doing a sensitivity analysis increases the chance that  decisions that influence the design, production, and operation of the  TWF will be made with a focus on the elements that have the greatest  effect on cost.
According to NNSA officials who oversee the TWF project, combining the  TWF construction estimate with the operations and maintenance estimate  to reflect the life-cycle costs and assessing the combined estimate  obscured the positive steps NNSA took in developing the construction  estimate. We agree that the TWF construction estimate conformed to  several best practices. Examples are as follows:
NNSA developed the construction estimate based on a mature design  plan for the facility to ensure that it was based on the best available  information at the time,
NNSA documented the data sources and the methodologies used to  calculate the construction estimate so that a cost analyst unfamiliar  with the project could understand what was done and replicate it, and
NNSA had an ICE completed to provide an unbiased test of the  reasonableness of the TWF construction costs.
Nonetheless, as described above, and in appendix II, we also identified  examples where NNSA’s construction estimate did not conform to best  practices. Because NNSA did not follow all best practices for cost  estimating, particularly for the TWF operations and maintenance cost  estimate, NNSA may not have reliable information to support budgetary  decisions for funding the TWF’s operations and maintenance. NNSA  expects the TWF may be ready to start operations as early as April 2016,  and when it does, NNSA will need to balance funding for the TWF with  the operations and maintenance costs for other nuclear infrastructure and  facilities at LANL and other NNSA sites that make up the national nuclear  security enterprise. In December 2013, we found that, as the facilities and  infrastructure that support the nuclear security enterprise continue to age,  maintenance costs are likely to grow. In that report, NNSA officials said  that deferred maintenance projects will have to compete against  programmatic priorities for funding within the overall pool of maintenance  funds available. By having reliable information on the TWF’s costs,  including operations and maintenance, well before the project starts  operations, NNSA managers would be better able to plan for, and  manage the costs of the TWF in balance with other infrastructure in the  national nuclear security enterprise.
GAO, Project and Program Management: DOE Needs to Revise Requirements and  Guidance for Cost Estimating and Related Reviews, GAO-15-29 (Washington, D.C.: Nov.  25, 2015). Specifically regarding DOE’s cost-estimating guide, we found, in November  2014, that for two best practice steps—determining the estimating structure and  conducting sensitivity analysis—the guide only partially or minimally describes those  steps. operations and maintenance estimate until the project is constructed and  undergoes operational readiness reviews in preparation for CD-4 (to  approve the start of operations or project completion). Updating the  TWF’s cost estimate to include all life-cycle costs and needed analyses  would provide NNSA more reliable information for better managing the  TWF as it prepares for the start of operations.
section_title: Conclusions
paragraphs: Safely removing the TRU waste stored at LANL is a critical part of  NNSA’s efforts to clean up the legacy environmental contamination from  decades of nuclear weapons activities. NNSA has made progress in  removing the TRU waste from LANL’s Area G and has monitored the  project’s recent performance through fiscal year work plans. But NNSA  has consistently used cost estimates for completing the project that it  could not meet because the estimates were developed based on  aggressive and unrealized funding assumptions, and the agency chose to  delay revising the 2009 estimate when it was determined to be outdated,  which was not consistent with the intent of DOE’s cleanup project  requirements for maintaining an updated life-cycle cost baseline. While  NNSA and EM are developing a new cost estimate for the project, the  new estimate may quickly become inaccurate because changes in  funding and the status of the WIPP may soon invalidate its assumptions.  By revising the estimate to include the current understanding of project  conditions, including the uncertainty at WIPP, NNSA program managers  can more accurately identify cost overruns consistent with best practices.
When completed, NNSA expects the TWF to provide TRU waste  capabilities at LANL to support NNSA’s nuclear weapons mission for the  next 50 years and has taken several steps that conformed to best  practices in developing the TWF’s construction estimate. However, NNSA  has not developed a reliable estimate of its operations and maintenance  costs by, for example, not sufficiently documenting its approach and not  using an inflation rate in its calculations because DOE’s project  management order for capital asset projects does not require the use of  all cost-estimating best practices in developing estimates of all life-cycle  costs. Thus, although the operations and maintenance costs were  estimated to be $6 million annually from 2018 through 2068, for a total of  $300 million, without documenting the approach used to develop the  estimate, and not using an inflation rate in these calculations, we could  not determine the reliability of the estimate for future years. DOE agreed  with the recommendations in our November 2014 report to revise its order  to require that DOE, NNSA, and its contractors develop cost estimates in  accordance with all best practices. However, opportunities exist currently  to enhance the reliability of the TWF cost estimate. Updating the TWF’s  cost estimate to include all life-cycle costs, as well as needed analyses,  would provide NNSA more reliable information for better managing the  TWF as it prepares for the start of operations, which NNSA expects could  be as early as April 2016.
section_title: Recommendations for Executive Action
paragraphs: To develop reliable cost estimates for the TRU waste removal project and  for the TWF construction project at LANL, we recommend that the  Secretary of Energy take the following two actions:
Direct NNSA and EM to revise the cost estimate for the TRU waste  removal project to ensure that it uses updated assumptions based on  the current understanding of project conditions, such as the status of  WIPP.
Direct NNSA to revise and update the TWF project’s cost estimate by  following all best practices for developing a reliable cost estimate that  covers all life-cycle costs for better managing the project going  forward.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided DOE with a draft of this report for its review and comment. In  written comments, reproduced in appendix III, NNSA provided a joint  response to our draft report for itself and DOE’s EM, which generally  agreed with both of the report’s recommendations. In its comments,  NNSA stated that it will update its cost estimates for both the TRU waste  removal project at Area G and the TWF’s operations and maintenance  and provided details of the specific actions planned or taken to address  both recommendations and timelines for completing these actions. NNSA  also provided general and technical comments that we incorporated into  the report, as appropriate.
In regard to the report’s first recommendation to revise the cost estimate  for the TRU waste removal project to ensure that it uses updated  assumptions based on the current understanding of project conditions,  NNSA stated actions have been taken to address this recommendation.  Specifically, a comprehensive life-cycle baseline revision was submitted  and reviewed, and EM is currently taking steps to revise and finalize this  new baseline cost estimate for the project in light of realistic out-year  funding profiles to support a planned renegotiation of the Consent Order  with the New Mexico Environment Department. In addition, the pending  changes to the type of contract used to manage the legacy cleanup work  at LANL will be factored into the baseline revision. DOE plans to complete  the revised cost estimate by September 30, 2015. We are pleased that  DOE plans to address this recommendation and has actions under way to  do so.
In regard to the report’s second recommendation to revise and update the  TWF project’s cost estimate by following all best practices for developing  a reliable cost estimate that covers all life-cycle costs, NNSA stated in its  written comments that it will update the TWF’s operations and  maintenance cost estimate to ensure effective management of the facility  once it is operational. Regarding the TWF’s construction estimate, as we  described in the report and in appendix II, NNSA’s TWF construction  estimate conformed to several but not all best practices. In particular,  NNSA validated the project team’s estimate through an ICE to provide an  unbiased test of the reasonableness of the TWF construction costs.  Regarding the TWF’s operations and maintenance estimate, NNSA  stated it will prepare the updated estimate as part of the programming  process for the fiscal year 2017 budget, which takes place in fiscal year  2015, to support postconstruction activities and operations. Further,  NNSA stated that the estimate will reflect operational costs for a 7-year  window and incorporate applicable best practices, including  documentation of any significant deviations and uncertainties impacting  the estimate. The estimated completion date for these activities is March  30, 2015.
We are encouraged by NNSA’s planned actions to update the TWF  operations and maintenance estimate using applicable best practices.  However, it will be particularly important for NNSA to document its  decisions on which best practices are being followed and the reasons  practices not being followed are not applicable. As we noted in the report,  one of the key weaknesses we found was that NNSA did not document  the approach (data sources and methodologies) used to develop the  $300 million operations and maintenance estimate for the TWF, even  though the operations and maintenance costs represented about 74  percent of the TWF’s life-cycle costs. Because NNSA did not document  the approach used, we could not determine whether it was appropriate for  developing the operations and maintenance estimate. With regard to the  time frames covered by this estimate, NNSA plans to update the TWF  operations and maintenance cost estimate to cover a 7-year period.  Given the need now for reliable information on the estimated costs for  operations, NNSA’s plan to update the TWF operations and maintenance  cost estimate following applicable best practices, and covering a 7-year  period, by March 30, 2015, would provide it more reliable information for  managing the facility as it prepares for the start of operations. As we  noted in the report, however, NNSA expects the TWF to provide TRU  waste capabilities at LANL to support NNSA’s nuclear weapons mission  for the next 50 years. By having a reliable and updated life-cycle estimate  for the TWF that covers the estimated useful life of the facility, NNSA  managers would be better able to plan for the TWF costs in balance with  other infrastructure in the national nuclear security enterprise.
As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of Energy, and other interested  parties. In addition, this report will be available at no charge on the GAO  website at http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made major contributions to  this report are listed in appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to examine (1) the extent to which the National  Nuclear Security Administration (NNSA) has met its cost targets for the  transuranic (TRU) waste removal project at Los Alamos National  Laboratory (LANL) and (2) the extent to which NNSA’s cost estimate for  the TRU Waste Facility (TWF) project at LANL met best practices for a  reliable cost estimate.
To examine the extent to which NNSA’s TRU waste removal project at  LANL has met its cost estimates, we reviewed documentation of NNSA’s  total project cost estimates from 2006 and 2009. We focused on these  cost estimates because they were developed after the establishment of  the Consent Order in 2005, and they were the most recently completed  estimates for the total cost of the project. During our review, NNSA was in  the process of developing a new cost estimate, the draft fiscal year 2015  cost estimate, for the TRU waste removal project. We interviewed the  NNSA officials and representatives from its LANL contractor who were  working on the draft estimate to understand the cost estimation process  and preliminary results. We reviewed data provided to us by NNSA from  the Department of Energy’s (DOE) Integrated Planning, Accountability,  and Budgeting System on the annual dollars spent for fiscal years 2006  through 2013 for the TRU waste removal project, as well as NNSA’s  estimate of fiscal year 2014 year-end spending for the project. We  compared the project spending data with the 2006 and 2009 cost  estimates, and with the new draft 2015 cost estimate. We also reviewed  data provided to us by NNSA from LANL’s Waste Compliance and  Tracking System on total volumes of TRU waste removed from LANL  from 1999 through July 2014, as well as NNSA’s estimate of the total  volume of TRU waste remaining. We assessed the reliability of the project  data we reviewed and analyzed, and we determined that the data for this  period were sufficiently reliable to examine the extent to which NNSA’s  TRU waste removal project at LANL has met its cost targets. To assess  the reliability of the project data, we reviewed information provided by  NNSA on the data systems used for managing and reporting the data,  including the systems’ controls and checks that ensure the accuracy and  completeness of the data, as well as procedures that were in place to  review and certify the reliability of the data such as inspector general or  internal audit reports of the quality of the data. In addition, we reviewed  NNSA’s fiscal year work plans for the years after NNSA adopted the  Operations Activities Protocol for managing the TRU waste removal  project, fiscal years 2012 through 2014. We compared the total project  cost estimates and the fiscal year work plans to the requirements for  developing and using cost estimates found in DOE’s Operations Activities  Protocol, which sets the requirements for managing cleanup projects at  DOE defined as operations activities. To refine our analysis, we  interviewed officials from NNSA’s Office of Environment, Health, and  Safety in headquarters, the Environmental Project’s Office in NNSA’s  LANL Site Office, and the DOE Office of Environmental Management’s  (EM) Office of Disposal Operations. We also met with the contractors  working on the TRU waste removal project at LANL during a visit to the  site where we toured Area G and the buildings conducting TRU waste  processing. average of the individual assessment ratings to determine the overall  assessment rating for each of the four characteristics as follows: Not Met  = 1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 to 3.4,  Substantially Met = 3.5 to 4.4, and Fully Met = 4.5 to 5.0. We consider a  cost estimate reliable if the overall assessment ratings for each of the four  characteristics are substantially or fully met. If any of the characteristics  are not met, minimally met, or partially met, then the cost estimate does  not fully reflect the characteristics of a high-quality estimate and cannot  be considered reliable.
We conducted this performance audit from July 2013 to February 2015 in  accordance with generally accepted government standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: GAO’s Assessment of NNSA’s Cost Estimates for the TWF Construction and Operations and Maintenance, as of June 2014 Compared with Industry Best Practices
subsections: section_title: Best practice characteristic and overall assessment Comprehensive: Partially met
paragraphs: Best practice  The cost estimate includes all life-cycle costs.
Detailed assessment Partially met. The life-cycle cost estimate consisted  of a design and construction (construction) estimate  and a 50-year operations and maintenance estimate.  The operations and maintenance cost estimate was  not updated at Critical Decision (CD)-2 when the  construction estimate was approved. The life-cycle  estimate did not include retirement of the facility.
The cost estimate completely defines the program,  reflects the current schedule, and is technically  reasonable.
Partially met. The construction estimate was based  on the technical requirements considered 90 percent  mature. The operations and maintenance estimate  was not supported by definitions of the technical  requirements that would have formed the basis of the  estimate.
The cost estimate work breakdown structure is  product-oriented, traceable to the statement of  work/objective, and at an appropriate level of detail  to ensure that cost elements are neither omitted nor  double-counted.
Partially met. The construction estimate work  breakdown structure covered the major work for the  end product of the TWF project. The work  breakdown structure did not present all cost  elements at a clear level of detail, and it was not  standardized so that cost data can be collected and  used for estimating future programs. The operations  and maintenance estimate did not reflect a  documented work breakdown structure.
The estimate documents all cost-influencing ground  rules and assumptions.
Partially met. The construction estimate included  ground rules and assumptions, such as technical  specifications, vendor quotes, and registered risks.  The operations and maintenance estimate did not  document ground rules and assumptions.
The documentation captures the source data used,  the reliability of the data, and how the data were  normalized.
Partially met. The construction estimate documented  the data sources used but not data reliability and  how the data were normalized. The operations and  maintenance estimate was not supported by detailed  documentation.
The documentation describes in sufficient detail the  calculations performed and the estimating  methodology used to derive each element’s cost.
Partially met. The construction estimate used an  engineering buildup approach for individual cost  elements. The operations and maintenance estimate  did not include documentation on how cost elements  were derived.
The documentation describes, step by step, how the  estimate was developed so that a cost analyst  unfamiliar with the program could understand what  was done and replicate it.
Partially met. The construction estimate  documentation explained how work breakdown  structure elements were estimated and the  documentation was mathematically sensible and  logical. The documentation explains how  management reserve and contingency were  calculated and was composed of cost and schedule  uncertainty. The operations and maintenance  estimate did not include documentation that detailed  how the estimate was developed.
section_title: Best practice characteristic and overall assessment
paragraphs: Best practice  The documentation discusses the technical baseline  description, and the data in the baseline is consistent  with the estimate.
Detailed assessment Partially met. The documentation for the construction  estimate matched the technical requirements  document. The operations and maintenance  estimate was not supported by a technical baseline  document.
The documentation provides evidence that the cost  estimate was reviewed and accepted by  management.
Partially met. NNSA approved the construction  estimate. The approval memo did not detail  recommendations for changes, feedback, and the  level of contingency reserves decided upon to reach  a desired level of confidence. The operations and  maintenance estimate did not document  management review and approval.
The cost estimate results are unbiased, not overly  conservative or optimistic, and based on an  assessment of most likely costs.
Partially met. The construction estimate included risk  and uncertainty analysis, the results included S curve  cumulative probabilities. The risk and uncertainty is  quantified as management reserve and contingency.  The cost estimate was in range when compared with  metrics that benchmark the TWF estimate to similar  nuclear projects. The operations and maintenance  estimate did not include documentation to help  determine whether it was unbiased and not overly  conservative or optimistic.
The estimate has been adjusted properly for inflation.  Partially met. The construction estimate was  adjusted for inflation for the period of the construction  schedule, but the cost data were not normalized. The  operations and maintenance estimate was not  properly adjusted for inflation.
The estimate contains few, if any, minor mistakes.
Partially met. The construction estimate included  minor calculation errors in the cost summary table.  We were not able to perform random sampling to  check calculations for accuracy because the  electronic cost model provided to us did not identify  the formulas for calculations. The operations and  maintenance estimate did not include detailed  calculations that we could use to check its accuracy.
The cost estimate is regularly updated to reflect  significant changes in the program so that it always  reflects current status.
Minimally met. The construction estimate was  updated to reflect changes in technical or program  assumptions at the CD-3 (approve the start of  construction) milestone but is not regularly updated  with actual costs on an ongoing basis. The  operations and maintenance estimate is not regularly  updated to reflect changes in the project and has not  been updated since June 2010.
Variances between planned and actual costs are  documented, explained, and reviewed.
Minimally met. The construction estimate did not  explain variances between planned and actual costs.  The CD-2 estimate included a summary level  reconciliation with the CD-1 estimate.
section_title: Best practice characteristic and overall assessment
paragraphs: Best practice  The estimate is based on a historical record of cost  estimating and actual experiences from other  comparable programs.
Detailed assessment Partially met. The construction estimate was based  on the contractor’s market price data for the scope of  work required to complete the project. The estimate  also uses metrics from similar nuclear projects but  did not include documentation on the reliability of the  metrics data. The operations and maintenance  estimate did not document whether it was based on  historical or other data.
The estimating technique for each cost element was  used appropriately.
Partially met. The construction estimate was based  on engineering buildup approaches appropriate to  each cost element. The operations and maintenance  estimate did not include documentation of the  techniques used for each cost element.
The cost estimate includes a sensitivity analysis that  identifies a range of possible costs based on varying  major assumptions, parameters, and data inputs.
Minimally met. The construction estimate identified  and examined key cost drivers but did not include a  formal sensitivity analysis. The operations and  maintenance estimate did not include a sensitivity  analysis.
A risk and uncertainty analysis was conducted that  quantified the imperfectly understood risks and  identified the effects of changing key cost driver  assumptions and factors.
Partially met. The construction estimate included risk  and uncertainty analysis for cost and schedule and  quantified the cost of these risks as management  reserve and contingency reserve. The estimate did  not document how correlation of cost elements was  accounted for in the risk and uncertainty analysis.  The operations and maintenance estimate did not  include risk and uncertainty analysis.
Major cost elements were cross-checked to see  whether results were similar.
Partially met. The construction estimate included  cross-checks with metrics that benchmark the TWF  to similar nuclear projects. The operations and  maintenance estimate did not include documentation  of cross-checks.
An independent cost estimate (ICE) was conducted  by a group outside the acquiring organization to  determine whether other estimating methods  produce similar results.
Partially met. An ICE for the project’s construction  phase was performed by DOE’s Office of Acquisition  and Project Management. The ICE appears to have  been based on a similar technical baseline to the  program office estimate. However, the program  estimate was 13 percent higher than the ICE. NNSA  did not document how it reconciled the two  estimates. The ICE did not cover the operations and  maintenance costs of the facility.
The ratings we used in this analysis are as follows: “Not met” means the cost estimate provided no  evidence that satisfies the best practice. “Minimally met” means the cost estimate provided evidence  that satisfies a small portion of the best practice. “Partially met” means the cost estimate provided  evidence that satisfies about half of the best practice. “Substantially met” means the cost estimate  provided evidence that satisfies a large portion of the best practice. “Fully met” means the cost  estimate provided complete evidence that satisfies the entire best practice.
section_title: Appendix III: Comments from the Department of Energy
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Diane LoFaro, Assistant  Director; Mark Braza; Richard P. Burkard; Brian M. Friedman; Abishek  Krupanand; Eli Lewine; Cynthia Norris; Katrina Pekar-Carpenter; and  Karen Richey made key contributions to this report.