Nuclear Waste: DOE Needs to Improve Cost Estimates for Transuranic Waste Projects at Los Alamos
Released: 2015-03-23T00:00:00

Letter

Background

This section describes (1) DOE’s project management requirements for capital asset projects and cleanup projects, (2) TRU waste operations at Area G, (3) the TRU waste removal project, (4) the TWF construction project, and (5) GAO’s Cost Guide.

DOE’s Project Management Requirements for Capital Asset Projects and Cleanup Projects

DOE has established separate project management requirements for its capital asset projects and certain cleanup projects defined as operations activities. DOE’s project management order for capital asset projects, Order 413.3B, establishes the requirements for managing capital asset projects, and EM’s Operations Activities Protocol establishes the requirements for managing cleanup projects defined as operations activities. The TRU waste removal project is a cleanup operations activity and is subject to EM’s Operations Activities Protocol, whereas the TWF is a capital asset construction project and must be carried out in accordance with Order 413.3B.

DOE’s Project Management Order 413.3B

DOE’s project management order for capital asset projects, Order 413.3B, applies to all capital asset projects with a total project cost greater than or equal to $50 million. Capital asset projects include construction projects that build large complexes that often house unique equipment and technologies such as those that process TRU waste or other radioactive material. DOE’s order establishes a process for NNSA and other DOE offices to manage projects, from identification of need through project completion, with the goal of delivering projects within the original performance baseline that are fully capable of meeting mission performance and other requirements, such as environmental, safety, and health standards. In particular, the order defines five major milestones— or Critical Decision (CD) points—that span the life of a project:

CD-0: Approve mission need. DOE identifies a credible performance gap between its current capabilities and capacities and those required to achieve the goals defined in its strategic plan. The mission need translates this gap into functional requirements. DOE formally establishes a project and begins the process of conceptual planning and identifying a range of alternative approaches to meet the identified need.

CD-1: Approve alternative selection and cost range. DOE completes the conceptual design and selects its preferred approach based on analysis of life-cycle costs, and approves the project’s preliminary cost range to complete the project’s design and construction.

CD-2: Approve the performance baseline. A project’s performance baseline consists of key cost, scope, schedule, and performance parameter targets. The project’s scope defines the technical goals and requirements that the project is to deliver at completion. The performance baseline cost includes the entire project budget, or total project cost, and represents DOE’s commitment to Congress. At this milestone, DOE completes its preliminary design and develops a definitive cost estimate that is a point estimate and no longer a range.

CD-3: Approve the start of construction. Design and engineering are essentially complete and have been reviewed, and project construction or implementation begins.

CD-4: Approve the start of operations or project completion. For construction projects, at this milestone, DOE completes the project and begins the transition to operations.

DOE’s project management order for capital asset projects specifies the requirements that must be met, including for developing project cost estimates, along with the documentation necessary, to move a project past each CD point. In addition, the order requires senior management to review the supporting documentation and decide whether to approve the project at each CD point. DOE also provides suggested approaches for meeting the requirements contained in the order through a series of guides, such as guides for cost estimating and project reviews.

EM’s Operations Activities Protocol

The life-cycle cost estimate data are maintained in EM’s Integrated Planning, Accountability, and Budgeting System and are used to develop DOE’s reported environmental liability. performance and, if practical, how that performance affects the life- cycle cost estimate and contract period of performance baseline.

TRU Waste Operations at Area G

Before 1970, TRU waste generated at LANL was managed as low-level radioactive waste and was disposed of at Area G in pits and trenches along with hazardous waste. In 1970, in response to concerns that TRU waste remained radioactive for an extremely long time and scientific research recommending deep geologic disposal for this waste, the Atomic Energy Commission––a DOE predecessor––directed sites that generated TRU waste to begin segregating it from other waste and storing it in retrievable packages for an interim period, pending disposal in a repository. As a result of the directive, starting in the early 1970s, the TRU waste generated at LANL was stored in segregated TRU waste pits and trenches and aboveground in fabric domes so that it could be more easily retrieved when a permanent repository site opened. (see fig. 1).

Today, Area G serves as LANL’s primary location for storing and processing TRU waste. Both legacy and newly generated TRU waste are stored and processed for shipping to WIPP at facilities in Area G. TRU waste operations at Area G include the following processes: packaging waste into 55-gallon drums or other approved containers following DOE standards, called waste acceptance criteria, to protect human health, safety, and the environment during the waste’s transport to and disposal in WIPP; repackaging containers if they are found to not meet WIPP’s waste acceptance criteria; resizing large waste using methods such as cutting it into smaller pieces so that it can be placed into approved containers; characterizing the waste by using specialized scanning equipment to assess the contents of each waste container and the amount of radioactivity it contains; and certifying the waste to declare that it meets WIPP’s waste acceptance criteria.

TRU Waste Removal Project

Starting in 2011, NNSA and the New Mexico Environment Department agreed to significant changes in the strategy for completing the TRU waste removal project. In that year, a wildfire occurred near Area G, resulting in increased public concern about the risk posed by the TRU waste stored aboveground at Area G. To address this risk, in 2012, NNSA and the New Mexico Environment Department reached a voluntary agreement, called the Framework Agreement, which established a June 2014 deadline for the accelerated removal of 3,706 cubic meters of aboveground TRU waste at a high risk from wildfires. To meet this deadline, NNSA initiated an effort know as the “3706 Campaign.” To facilitate this campaign, the New Mexico Environment Department and DOE agreed to extend other deadlines established under the 2005 Consent Order governing hazardous waste cleanup activities for locations across LANL. With these deadlines extended, NNSA was able to reallocate EM funding for environmental cleanup activities at LANL to focus on the 3706 Campaign. Using the additional funds, NNSA increased the TRU waste processing capacity at LANL by constructing more facilities for repackaging waste and hiring additional contractors to operate the facilities 7 days a week. Under the Framework Agreement, NNSA also agreed to remove the remaining aboveground TRU waste at Area G and developed a schedule for removing the belowground TRU waste at Area G by September 30, 2018. According to NNSA officials, part of the plan in establishing the Framework Agreement was that, once the 3706 Campaign was completed, NNSA and the New Mexico Environment Department would discuss renegotiating the final completion date for the Consent Order.

On May 30, 2014, DOE announced that NNSA had completed the removal of about 93 percent of the TRU waste included in the 3706 Campaign but would not meet the June 2014 deadline because of the department’s decision to halt the TRU waste removal project. As noted previously, DOE halted the project at LANL in February 2014, in response to an incident at WIPP that involved a LANL TRU waste container that ruptured and leaked radioactive material while in storage.

TWF Construction Project

The TWF project is to replace NNSA’s existing capabilities that reside at Area G for storage, characterization, and certification of newly generated TRU waste at LANL. The TWF design includes multiple buildings for waste storage, waste characterization, and operational support. The facility will also have space and utility hookups for three mobile trailers to be provided by WIPP that will contain additional characterization capabilities needed to certify that TRU waste containers meet WIPP waste acceptance criteria (see fig. 2).

The TWF is being designed and constructed as a high-hazard nuclear facility, which must meet nuclear safety standards for storage and handling of nuclear waste. Nuclear safety design features of the TWF include a barrier to prevent large vehicles from crashing into the facility, a seismic power cutoff switch designed to reduce possible sources of fire that could result from an earthquake, and a tank to store water to help suppress any earthquake- initiated fire.

NNSA’s Office of Defense Programs, which is the program office responsible for maintaining the nation’s nuclear weapons stockpile, is sponsoring the TWF project. The office provides the annual funding for planning and construction and approves the project’s milestones. NNSA’s Office of Acquisition and Project Management is responsible for overseeing the construction of the TWF within NNSA’s approved cost and schedule estimates. To do so, the office provides direction and oversight of NNSA’s management and operating contractor at LANL.

NNSA divided the TWF project’s design and construction into two subprojects: site development and facilities construction. NNSA completed the site development activities, which included relocation of utility lines, as well as excavation and grading to prepare the site for the facility’s construction, in December 2012, at a cost of $7.7 million. In February 2013, NNSA approved the project’s CD-2 performance baseline estimate of $99.2 million to construct the TWF with a completion date between April 30, 2016, and January 31, 2018. As mentioned previously, NNSA also estimated that the facility will cost $300 million to operate and maintain for its projected useful life of 50 years, spanning 2018 through 2068. In combination, NNSA’s estimated life-cycle costs for the TWF when it approved the project’s performance baseline to complete construction at CD-2 totaled about $406.9 million.

GAO’s Cost Guide

Drawing from federal cost-estimating organizations and industry, the Cost Guide provides best practices about the processes, procedures, and practices needed for ensuring development of high-quality—that is, reliable cost estimates.characteristics of a high-quality, reliable cost estimate: The Cost Guide identifies the following four

Comprehensive when it accounts for all life-cycle costs associated with a project, is based on a completely defined and technically reasonable plan, and it contains a cost estimating structure in sufficient detail to ensure that costs are neither omitted nor double- counted;

Well-documented when supporting documentation explains the process, sources, and methods used to create the estimate and contains the underlying data used to develop the estimate;

Accurate when it is not overly conservative or too optimistic and is based on an assessment of the costs most likely to be incurred; and

Credible when a sensitivity analysis has been conducted, the level of confidence associated with the point estimate has been identified through the use of risk and uncertainty analysis, and the point estimate has been cross-checked with an independent cost estimate (ICE).

To develop a cost estimate that embodies these four characteristics, our Cost Guide lays out best practice steps. For example, one step in developing an accurate estimate is to identify and document ground rules that establish a common set of agreed-on estimating standards and solid assumptions that are measurable, specific, and consistent with historical data. According to the Cost Guide, it is imperative that cost estimators brief management on the ground rules and assumptions used for an estimate so that management understands the conditions the estimate was structured on and can avoid overly optimistic assumptions.

NNSA Has Not Met Its Cost Estimates to Complete the TRU Waste Removal Project at LANL

NNSA’s TRU waste removal project at LANL did not meet its 2006 cost estimate and is not expected to meet the 2009 cost estimate established for the completion of the project. During our review, NNSA and EM were in the process of developing a new cost estimate for the project. The TRU waste removal project has not met its past cost estimates, partly because the 2006 and 2009 cost estimates were based on aggressive funding assumptions to meet the deadlines of the Consent Order. In addition, because NNSA did not maintain or use two of the three project baselines outlined in its cleanup project requirements, it could not measure the progress of the total project.

As of the end of fiscal year 2014, NNSA had spent about $931 million on the project, which exceeded the 2006 cost estimate of $729 million by $202 million (see fig. 3). The amount expended by the end of fiscal year 2014 did not exceed the $1.2 billion upper range of the 2009 cost estimate, but it did exceed the $848 million lower range of the estimate by $83 million. As of July 2014, the most recent date for which data were available, NNSA had removed approximately 79 percent of the TRU waste at LANL; however, the remaining 21 percent includes the waste buried belowground, which will be the most difficult and expensive to address, according to NNSA officials. The new fiscal year 2015 draft estimate, currently under review by NNSA and EM, projects that the final project costs will be approximately $1.6 billion, or $400 million above the upper range of the 2009 cost estimate.

NNSA’s TRU waste removal project exceeded the 2006 cost estimate and is not expected to meet the 2009 cost estimate, in part, because NNSA and EM developed the cost estimates using aggressive project funding assumptions that were based on the need to meet the Consent Order requirement for closing Area G by 2015, according to NNSA and EM officials. For the 2006 cost estimate, NNSA officials overseeing the TRU waste removal project developed the parameters of the project estimate based on the need to remove almost all of the TRU waste by 2012. In particular, to meet these deadlines, NNSA based its cost estimate on funding projections provided by EM that were consistent with meeting the Consent Order deadline and that assumed that EM would increase the yearly funding for environmental cleanup projects at LANL. According to NNSA and EM officials, they recognized that the funding assumptions used in the cost estimate were aggressive and that significant funding shortfalls would inhibit the TRU waste removal project’s ability to remain on schedule. From fiscal year 2006 through fiscal year 2008, EM provided approximately $457 million—$284 million (38 percent) less than the amount requested by NNSA officials for all cleanup activities at LANL—and, as a result, it was not possible to fund the TRU waste removal project at the levels established in the 2006 estimate. According to NNSA officials, EM was unable to increase the funding for LANL cleanup projects due to limited budget flexibility and competing demands from cleanup projects at other DOE sites. In a 2008 report on LANL’s cleanup efforts, DOE’s Inspector General found that EM did not have enough money to address all the milestones in the NNSA officials said that, by environmental agreements they signed.2009, the TRU waste removal project had fallen behind schedule and could not be completed by 2012, in part, because of the shortfall in funding. The extension of the project’s completion date beyond 2012 resulted in additional costs, which contributed to the total cost of the project exceeding the 2006 estimate.

In 2009, NNSA and EM developed a new cost estimate for the TRU waste removal project with a completion date in 2018 but again used aggressive funding assumptions to complete the project as close to the Consent Order’s 2015 deadline as possible. Similar to the 2006 estimate, the 2009 cost estimate used funding projections provided by EM that assumed an increase in the yearly levels of funding for LANL cleanup activities that were necessary for NNSA to complete the TRU waste removal project by July of 2018—2 and a half years after the Consent Order deadline in 2015. However, due to the same budget restrictions that affected cleanup project funding previously, actual funding levels provided by EM for fiscal years 2009 to 2012 for all cleanup projects at LANL again came in below the levels requested by NNSA officials at LANL. Specifically, LANL received approximately $1 billion over these years, which was $240 million (19 percent) less than the levels requested by NNSA officials at LANL for cleanup projects at the site. As a result, funding for the TRU waste removal project was reduced, and this reduction caused the project to fall behind the schedule set in the 2009 cost estimate, according to NNSA officials. Moreover, the 2009 cost estimate was never officially approved by NNSA and EM because the date estimated for project completion was not consistent with the requirements of the Consent Order. According to NNSA officials, they could not formally approve the 2009 estimate because it included a 2018 estimated completion date for the TRU waste removal project, which conflicted with the required 2015 closure date established in the Consent Order.

In addition to developing estimates using aggressive funding assumptions, NNSA did not maintain or use two of the three project baselines outlined in the Operations Activity Protocol, so the agency could not measure the progress of the total project. As discussed previously, NNSA was to manage the TRU waste removal project using EM’s Operations Activities Protocol, which is intended to provide the framework for managing and reporting on the progress of cleanup projects through the use of three performance baselines: life-cycle cost, contract period of performance, and fiscal year work plan. However, for the TRU waste removal project, because NNSA did not have an updated life-cycle cost baseline and did not establish a contract period of performance baseline, it only used the fiscal year work plan baseline to manage the project, as discussed below:

Life-cycle cost baseline. NNSA has not updated the life-cycle cost baseline for the project since 2009, even though agency officials told us they were aware that the estimate has been out-of-date since about 2012. Since that year, the project has undergone significant changes that affected its estimated cost and completion date. For example, by initiating the 3706 Campaign in 2012, NNSA altered the scope of work from what was planned in the 2009 cost estimate. NNSA and EM officials told us that, although they recognized in 2012 that the 2009 estimate was no longer valid, they did not see the purpose in completing a new estimate before completion of the campaign and the expected renegotiation of the Consent Order deadlines. According to these officials, a new cost estimate for the project would have either needed to use unreasonably high funding assumptions to achieve project completion by 2015 or, if it used more reasonable funding assumptions, it could not have been approved because of the political issues associated with a completion date beyond the 2015 Consent Order deadline. The Operations Activities Protocol requires that EM or NNSA Site Office Managers develop cost estimates that cover the full life-cycle of a cleanup project, but it leaves the Site Office Manager discretion to determine whether an update to the life-cycle cost baseline is required. Because NNSA’s LANL Site Office Manager, in consultation with EM officials, decided not to update the life-cycle cost baseline, NNSA could not measure project performance to determine the impact of management actions. For example, from fiscal year 2012 to 2014, NNSA used additional funding that was reallocated to the TRU waste removal project to increase the pace of TRU waste packaging and removal; however, because they did not have an updated cost estimate to measure against, NNSA managers were unable to identify the effect these actions had on the total cost of the project.

Contract period of performance baseline. NNSA has not managed the TRU waste removal project using a contract period of performance because, according to NNSA officials, the project, like other projects at LANL, is being conducted through NNSA’s management and operating contract. According to the Operations Activities Protocol, a contract period of performance baseline is required for those cleanup projects that are executed through a contract that establishes a performance baseline for cost and scope over the duration of the contract. The management and operating contract covers all work performed at LANL, but it does not establish a cost estimate for specific projects such as the TRU waste removal project, according to NNSA officials. In contrast, for cleanup projects at EM-managed sites, the scope of the contract is typically limited to the cleanup project and would not include other site activities that were unrelated to the project. According to NNSA officials, executing the TRU waste removal project through the management and operating contract does not provide the baseline necessary for establishing a contract period of performance, so NNSA is not required to manage to a contract period of performance baseline for this project. According to EM officials, DOE determined that as part of EM’s transition to direct oversight of the legacy cleanup work at LANL, EM will transition away from using a management and operations contract to manage the remaining cleanup work. When this change in contract type is completed, the officials stated that a contract period of performance baseline will be available for monitoring performance of the work, including the TRU waste removal project.

Fiscal year work plan baseline. NNSA has used the fiscal year work plans outlined in the Operations Activities Protocol to monitor the performance of portions of the TRU waste removal project and to manage and assess the performance of the entire TRU waste removal project in the absence of an accurate life-cycle cost estimate baseline or a contract period of performance baseline. According to NNSA officials, a new fiscal year work plan is developed each year for the TRU waste removal project using an integrated priorities list for remaining TRU removal work and the projected funding amount allocated for LANL cleanup. However, while NNSA and EM site and headquarters officials monitored progress against the fiscal year work plan, the agency was unable to evaluate the performance of the entire TRU waste removal project and identify potential cost overruns because its life-cycle cost estimate was out-of-date, and it did not manage to a contract period of performance. The Operations Activities Protocol requires that the Site Office Manager report yearly on any variances between total project costs to date and the estimated costs for the entire project as part of the fiscal year work plans. However, because NNSA did not have an accurate cost estimate for the TRU waste removal project, the 2012, 2013, and 2014 fiscal year work plans used the outdated 2009 cost estimate to report on the variances between the current and estimated costs for the project. As a result, these fiscal year work plans did not provide accurate information on project performance to date to help managers measure total project performance and manage costs.

At the time of our review, NNSA and EM were in the process of developing a new cost estimate for the TRU waste removal project that they expect to complete in fiscal year 2015; however, this estimate may quickly become inaccurate due to assumptions related to funding and the status of WIPP that could be invalidated. As discussed previously, the new draft cost estimate increases the estimated total cost of the project to $1.6 billion, with a completion date in fiscal year 2023 (i.e., October 2022). According to an NNSA official, this draft estimate uses a more conservative approach than past estimates by expanding the scope of the project estimate to include the costs to remove additional TRU waste that was not included in past estimates.concerning when WIPP will reopen and the need to address the impending Consent Order deadlines in 2015 that have not yet been renegotiated with New Mexico, the same NNSA official told us that several of the new estimate’s assumptions are no longer valid or may not be valid after a few years. For example, the funding assumptions in the new estimate may already be invalid. With the TRU waste removal project on hold as a result of the WIPP closure, the New Mexico Environment Department is no longer providing deadline extensions for some of LANL’s hazardous waste cleanup work and, as a result, NNSA officials managing the cleanup work have had to reprioritize their funding to attempt to meet those deadlines. Restoring funding to hazardous waste cleanup projects reduces the funding available for TRU waste removal, which would invalidate the funding assumptions used in the new cost estimate. In addition, the NNSA official told us that the new cost estimate also assumes that LANL will resume TRU waste shipments to WIPP in fiscal year 2017 based on an unofficial estimate from the WIPP manager that WIPP will reopen between 18 and 30 months after the initial However, in light of the uncertainty assessment of the incident.shipments from LANL in this time frame, the TRU waste removal project at LANL would be delayed, resulting in additional costs not accounted for in the new estimate, such as the costs for maintaining the project workforce longer than anticipated.

If WIPP does not start accepting TRU waste To objectively measure the performance of the TRU waste removal project and take action to manage project costs, NNSA managers of the project need an updated cost estimate. EM’s Operations Activity Protocol leaves it to the discretion of the Site Office Manager to update cleanup project estimates, and NNSA’s LANL Site Office Manager chose to delay revising the outdated 2009 estimate. As a result, DOE does not have an estimate of the total cost or completion date of the TRU waste removal project that uses updated assumptions based on the current understanding of project conditions. NNSA and EM are developing a new cost estimate for the project; however, the new estimate may quickly become inaccurate because of changes in funding, and the status of the WIPP may soon invalidate its assumptions. According to best practices for cost estimating, maintaining an updated cost estimate is critical so that officials making decisions about the future management of a project have accurate information for assessing their alternatives. By revising the TRU waste removal project’s estimate to include the current understanding of project conditions, NNSA program managers could more accurately identify cost overruns.

NNSA’s Cost Estimate for the Transuranic Waste Facility Project at LANL Partially Met Best Practices

NNSA’s cost estimate for the TWF partially met best practices. More specifically, NNSA’s cost estimate—which consisted of separate cost estimates for completing construction and for operations and maintenance, as the TWF’s life-cycle costs—partially reflected each of the four characteristics of a reliable estimate (comprehensive, well- documented, accurate, and credible) as established by best practices. In developing the construction estimate, NNSA took several steps that conformed to best practices, such as validating the construction estimate by completing an ICE. Although DOE’s project management order for capital asset projects does not require the use of all best practices in developing a cost estimate, DOE’s related cost-estimating guidance, which is optional, describes most of them. In contrast, in developing its operations and maintenance cost estimate, NNSA did not take steps that conformed to best practices. In particular, NNSA did not sufficiently document the approach (i.e., data sources and methodologies) used to develop the estimate, even though operations and maintenance costs represented about 74 percent of the total life-cycle costs of the facility. The reason was that operations and maintenance cost estimates for a construction project do not need to be updated and documented at CD-2 under DOE’s project management order, as funds for these costs do not need to be specifically requested from Congress to complete the project. By not sufficiently documenting the approach used to develop the estimate, NNSA may not have reliable information to support budgetary decisions for funding the TWF’s operations and maintenance in the future. For example, the contractor’s representatives told us that they estimated the operations and maintenance costs to be $6 million annually from 2018 through 2068, for a total of $300 million but did not use an inflation rate in the calculations. Thus, although $6 million may be an accurate estimate for the first year, without documentation of the approach used to develop the estimate, we could not determine its reliability for the first or future years.

Appendix II provides a summary description of our assessment of NNSA’s cost estimates for the TWF project’s construction and operations and maintenance. The following are examples from our assessment, by best practice characteristic:

Comprehensive. The TWF estimates partially reflected the characteristics of comprehensive cost estimates. For example, NNSA partially followed the best practice for completely defining the program, as NNSA’s contractor based the TWF construction cost estimate on a mature design plan that detailed the technical requirements and characteristics for the TWF. In contrast, for the TWF operations and maintenance cost estimate, NNSA’s contractor was not able to provide us with definitions of the technical requirements and characteristics that would have formed the basis of the estimate, although the $300 million estimate represented a substantial change from a $642 million estimate that NNSA’s contractor produced in June 2010 for CD-1 (approve alternative selection and cost range). As a result, we could not determine whether the $300 million estimate reflected the most recent TWF design approved at CD-2 to complete construction. NNSA officials told us they did not develop an updated and documented basis to support the operations and maintenance cost estimate because DOE’s project order does not require updated and documented estimates of all life-cycle costs at CD-2. Instead, at CD-2, the order focuses on the need for the baseline cost estimate to complete construction because funding for construction needs to be specifically requested from Congress to complete the project. According to best practices, clearly defining the technical requirements would help to ensure that managers have an adequate understanding of the facility and where information was limited and assumptions were made in developing the estimate.

Well-documented. The TWF estimates partially reflected the characteristics of well-documented cost estimates. For example, regarding the construction estimate, NNSA’s contractor documented the data sources and the methodology used to calculate the construction estimate so that a cost analyst unfamiliar with the project could understand what was done and replicate the estimate. In contrast, NNSA did not document the approach (data sources and methodologies) used to develop the operations and maintenance estimate, even though the operations and maintenance costs represented about 74 percent of the TWF’s life-cycle costs. As mentioned previously, DOE’s project management order does not require documentation of the operations and maintenance costs at CD-2. NNSA was required to report the operations and maintenance cost estimate by following a DOE budget formulation guidance that did not specify requirements for documenting the estimate. Because NNSA did not document the approach used, we could not determine whether it was appropriate for developing the operations and maintenance estimate; whether NNSA management reviewed the estimate, including its risks and uncertainties, or whether NNSA management approved the estimate.

Accurate. The TWF estimates partially reflected the characteristics of accurate cost estimates. For example, NNSA’s contractor partially followed the best practice for properly adjusting the estimates for inflation. Regarding the construction estimate, NNSA’s contractor developed the estimate using pricing data that were adjusted for inflation. However, the contractor did not then normalize the data to remove the effects of inflation. According to representatives from the contractor, they did not believe data normalization was an applicable step for the TWF construction estimate because the data set was too small. According to cost-estimating best practices, data normalization is often necessary to ensure comparability of data sets because data can be gathered from a variety of sources and in different forms that need to be adjusted before being used. Regarding the operations and maintenance estimate, NNSA’s contractor did not properly adjust the estimate for inflation over the 50-year useful life of the TWF. Specifically, the contractor’s representatives told us that they estimated the operations and maintenance costs to be $6 million annually from 2018 through 2068, for a total of $300 million, but they did not use an inflation rate in these calculations. Adjusting for inflation is an important step in developing an estimate because if the inflation amount is not correct, the estimate is not accurate. Applying the wrong inflation rate will either result in a higher cost estimate or estimated costs that are not sufficient to keep pace with inflation.

Credible. The TWF estimates partially reflected the characteristics of credible cost estimates. For example, NNSA followed the best practice to have an ICE completed in January 2013 to validate the TWF construction costs. However, because DOE’s project management order does not define the operations and maintenance costs as project costs, NNSA was not required to include these costs in the ICE. By not including the TWF operations and maintenance costs in the ICE, NNSA managers may lack insight into these future costs. According to best practices, an ICE can provide NNSA managers with additional insight into the TWF’s potential operations and maintenance costs—in part, because ICEs frequently use different methods and are less burdened with organizational bias. Therefore, according to best practices, an ICE can be used as a benchmark to assess the reasonableness of the contractor’s proposed operations and maintenance costs, improving NNSA management’s ability to make sound investment decisions, and accurately assess the contractor’s performance. Moreover, because DOE’s project management order does not require it, NNSA’s contractor did not follow the best practice to complete a sensitivity analysis to quantify the extent to which either the construction or operations and maintenance cost estimates could vary because of changes in key assumptions and ground rules. Such an analysis is a best practice because uncertainty cannot be avoided and, therefore, it is necessary to identify the cost elements that represent the most risk and, if possible, quantify them. According to cost-estimating best practices, doing a sensitivity analysis increases the chance that decisions that influence the design, production, and operation of the TWF will be made with a focus on the elements that have the greatest effect on cost.

According to NNSA officials who oversee the TWF project, combining the TWF construction estimate with the operations and maintenance estimate to reflect the life-cycle costs and assessing the combined estimate obscured the positive steps NNSA took in developing the construction estimate. We agree that the TWF construction estimate conformed to several best practices. Examples are as follows:

NNSA developed the construction estimate based on a mature design plan for the facility to ensure that it was based on the best available information at the time,

NNSA documented the data sources and the methodologies used to calculate the construction estimate so that a cost analyst unfamiliar with the project could understand what was done and replicate it, and

NNSA had an ICE completed to provide an unbiased test of the reasonableness of the TWF construction costs.

Nonetheless, as described above, and in appendix II, we also identified examples where NNSA’s construction estimate did not conform to best practices. Because NNSA did not follow all best practices for cost estimating, particularly for the TWF operations and maintenance cost estimate, NNSA may not have reliable information to support budgetary decisions for funding the TWF’s operations and maintenance. NNSA expects the TWF may be ready to start operations as early as April 2016, and when it does, NNSA will need to balance funding for the TWF with the operations and maintenance costs for other nuclear infrastructure and facilities at LANL and other NNSA sites that make up the national nuclear security enterprise. In December 2013, we found that, as the facilities and infrastructure that support the nuclear security enterprise continue to age, maintenance costs are likely to grow. In that report, NNSA officials said that deferred maintenance projects will have to compete against programmatic priorities for funding within the overall pool of maintenance funds available. By having reliable information on the TWF’s costs, including operations and maintenance, well before the project starts operations, NNSA managers would be better able to plan for, and manage the costs of the TWF in balance with other infrastructure in the national nuclear security enterprise.

GAO, Project and Program Management: DOE Needs to Revise Requirements and Guidance for Cost Estimating and Related Reviews, GAO-15-29 (Washington, D.C.: Nov. 25, 2015). Specifically regarding DOE’s cost-estimating guide, we found, in November 2014, that for two best practice steps—determining the estimating structure and conducting sensitivity analysis—the guide only partially or minimally describes those steps. operations and maintenance estimate until the project is constructed and undergoes operational readiness reviews in preparation for CD-4 (to approve the start of operations or project completion). Updating the TWF’s cost estimate to include all life-cycle costs and needed analyses would provide NNSA more reliable information for better managing the TWF as it prepares for the start of operations.

Conclusions

Safely removing the TRU waste stored at LANL is a critical part of NNSA’s efforts to clean up the legacy environmental contamination from decades of nuclear weapons activities. NNSA has made progress in removing the TRU waste from LANL’s Area G and has monitored the project’s recent performance through fiscal year work plans. But NNSA has consistently used cost estimates for completing the project that it could not meet because the estimates were developed based on aggressive and unrealized funding assumptions, and the agency chose to delay revising the 2009 estimate when it was determined to be outdated, which was not consistent with the intent of DOE’s cleanup project requirements for maintaining an updated life-cycle cost baseline. While NNSA and EM are developing a new cost estimate for the project, the new estimate may quickly become inaccurate because changes in funding and the status of the WIPP may soon invalidate its assumptions. By revising the estimate to include the current understanding of project conditions, including the uncertainty at WIPP, NNSA program managers can more accurately identify cost overruns consistent with best practices.

When completed, NNSA expects the TWF to provide TRU waste capabilities at LANL to support NNSA’s nuclear weapons mission for the next 50 years and has taken several steps that conformed to best practices in developing the TWF’s construction estimate. However, NNSA has not developed a reliable estimate of its operations and maintenance costs by, for example, not sufficiently documenting its approach and not using an inflation rate in its calculations because DOE’s project management order for capital asset projects does not require the use of all cost-estimating best practices in developing estimates of all life-cycle costs. Thus, although the operations and maintenance costs were estimated to be $6 million annually from 2018 through 2068, for a total of $300 million, without documenting the approach used to develop the estimate, and not using an inflation rate in these calculations, we could not determine the reliability of the estimate for future years. DOE agreed with the recommendations in our November 2014 report to revise its order to require that DOE, NNSA, and its contractors develop cost estimates in accordance with all best practices. However, opportunities exist currently to enhance the reliability of the TWF cost estimate. Updating the TWF’s cost estimate to include all life-cycle costs, as well as needed analyses, would provide NNSA more reliable information for better managing the TWF as it prepares for the start of operations, which NNSA expects could be as early as April 2016.

Recommendations for Executive Action

To develop reliable cost estimates for the TRU waste removal project and for the TWF construction project at LANL, we recommend that the Secretary of Energy take the following two actions:

Direct NNSA and EM to revise the cost estimate for the TRU waste removal project to ensure that it uses updated assumptions based on the current understanding of project conditions, such as the status of WIPP.

Direct NNSA to revise and update the TWF project’s cost estimate by following all best practices for developing a reliable cost estimate that covers all life-cycle costs for better managing the project going forward.

Agency Comments and Our Evaluation

We provided DOE with a draft of this report for its review and comment. In written comments, reproduced in appendix III, NNSA provided a joint response to our draft report for itself and DOE’s EM, which generally agreed with both of the report’s recommendations. In its comments, NNSA stated that it will update its cost estimates for both the TRU waste removal project at Area G and the TWF’s operations and maintenance and provided details of the specific actions planned or taken to address both recommendations and timelines for completing these actions. NNSA also provided general and technical comments that we incorporated into the report, as appropriate.

In regard to the report’s first recommendation to revise the cost estimate for the TRU waste removal project to ensure that it uses updated assumptions based on the current understanding of project conditions, NNSA stated actions have been taken to address this recommendation. Specifically, a comprehensive life-cycle baseline revision was submitted and reviewed, and EM is currently taking steps to revise and finalize this new baseline cost estimate for the project in light of realistic out-year funding profiles to support a planned renegotiation of the Consent Order with the New Mexico Environment Department. In addition, the pending changes to the type of contract used to manage the legacy cleanup work at LANL will be factored into the baseline revision. DOE plans to complete the revised cost estimate by September 30, 2015. We are pleased that DOE plans to address this recommendation and has actions under way to do so.

In regard to the report’s second recommendation to revise and update the TWF project’s cost estimate by following all best practices for developing a reliable cost estimate that covers all life-cycle costs, NNSA stated in its written comments that it will update the TWF’s operations and maintenance cost estimate to ensure effective management of the facility once it is operational. Regarding the TWF’s construction estimate, as we described in the report and in appendix II, NNSA’s TWF construction estimate conformed to several but not all best practices. In particular, NNSA validated the project team’s estimate through an ICE to provide an unbiased test of the reasonableness of the TWF construction costs. Regarding the TWF’s operations and maintenance estimate, NNSA stated it will prepare the updated estimate as part of the programming process for the fiscal year 2017 budget, which takes place in fiscal year 2015, to support postconstruction activities and operations. Further, NNSA stated that the estimate will reflect operational costs for a 7-year window and incorporate applicable best practices, including documentation of any significant deviations and uncertainties impacting the estimate. The estimated completion date for these activities is March 30, 2015.

We are encouraged by NNSA’s planned actions to update the TWF operations and maintenance estimate using applicable best practices. However, it will be particularly important for NNSA to document its decisions on which best practices are being followed and the reasons practices not being followed are not applicable. As we noted in the report, one of the key weaknesses we found was that NNSA did not document the approach (data sources and methodologies) used to develop the $300 million operations and maintenance estimate for the TWF, even though the operations and maintenance costs represented about 74 percent of the TWF’s life-cycle costs. Because NNSA did not document the approach used, we could not determine whether it was appropriate for developing the operations and maintenance estimate. With regard to the time frames covered by this estimate, NNSA plans to update the TWF operations and maintenance cost estimate to cover a 7-year period. Given the need now for reliable information on the estimated costs for operations, NNSA’s plan to update the TWF operations and maintenance cost estimate following applicable best practices, and covering a 7-year period, by March 30, 2015, would provide it more reliable information for managing the facility as it prepares for the start of operations. As we noted in the report, however, NNSA expects the TWF to provide TRU waste capabilities at LANL to support NNSA’s nuclear weapons mission for the next 50 years. By having a reliable and updated life-cycle estimate for the TWF that covers the estimated useful life of the facility, NNSA managers would be better able to plan for the TWF costs in balance with other infrastructure in the national nuclear security enterprise.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Energy, and other interested parties. In addition, this report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to examine (1) the extent to which the National Nuclear Security Administration (NNSA) has met its cost targets for the transuranic (TRU) waste removal project at Los Alamos National Laboratory (LANL) and (2) the extent to which NNSA’s cost estimate for the TRU Waste Facility (TWF) project at LANL met best practices for a reliable cost estimate.

To examine the extent to which NNSA’s TRU waste removal project at LANL has met its cost estimates, we reviewed documentation of NNSA’s total project cost estimates from 2006 and 2009. We focused on these cost estimates because they were developed after the establishment of the Consent Order in 2005, and they were the most recently completed estimates for the total cost of the project. During our review, NNSA was in the process of developing a new cost estimate, the draft fiscal year 2015 cost estimate, for the TRU waste removal project. We interviewed the NNSA officials and representatives from its LANL contractor who were working on the draft estimate to understand the cost estimation process and preliminary results. We reviewed data provided to us by NNSA from the Department of Energy’s (DOE) Integrated Planning, Accountability, and Budgeting System on the annual dollars spent for fiscal years 2006 through 2013 for the TRU waste removal project, as well as NNSA’s estimate of fiscal year 2014 year-end spending for the project. We compared the project spending data with the 2006 and 2009 cost estimates, and with the new draft 2015 cost estimate. We also reviewed data provided to us by NNSA from LANL’s Waste Compliance and Tracking System on total volumes of TRU waste removed from LANL from 1999 through July 2014, as well as NNSA’s estimate of the total volume of TRU waste remaining. We assessed the reliability of the project data we reviewed and analyzed, and we determined that the data for this period were sufficiently reliable to examine the extent to which NNSA’s TRU waste removal project at LANL has met its cost targets. To assess the reliability of the project data, we reviewed information provided by NNSA on the data systems used for managing and reporting the data, including the systems’ controls and checks that ensure the accuracy and completeness of the data, as well as procedures that were in place to review and certify the reliability of the data such as inspector general or internal audit reports of the quality of the data. In addition, we reviewed NNSA’s fiscal year work plans for the years after NNSA adopted the Operations Activities Protocol for managing the TRU waste removal project, fiscal years 2012 through 2014. We compared the total project cost estimates and the fiscal year work plans to the requirements for developing and using cost estimates found in DOE’s Operations Activities Protocol, which sets the requirements for managing cleanup projects at DOE defined as operations activities. To refine our analysis, we interviewed officials from NNSA’s Office of Environment, Health, and Safety in headquarters, the Environmental Project’s Office in NNSA’s LANL Site Office, and the DOE Office of Environmental Management’s (EM) Office of Disposal Operations. We also met with the contractors working on the TRU waste removal project at LANL during a visit to the site where we toured Area G and the buildings conducting TRU waste processing. average of the individual assessment ratings to determine the overall assessment rating for each of the four characteristics as follows: Not Met = 1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 to 3.4, Substantially Met = 3.5 to 4.4, and Fully Met = 4.5 to 5.0. We consider a cost estimate reliable if the overall assessment ratings for each of the four characteristics are substantially or fully met. If any of the characteristics are not met, minimally met, or partially met, then the cost estimate does not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable.

We conducted this performance audit from July 2013 to February 2015 in accordance with generally accepted government standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: GAO’s Assessment of NNSA’s Cost Estimates for the TWF Construction and Operations and Maintenance, as of June 2014 Compared with Industry Best Practices

Best practice characteristic and overall assessment Comprehensive: Partially met

Best practice The cost estimate includes all life-cycle costs.

Detailed assessment Partially met. The life-cycle cost estimate consisted of a design and construction (construction) estimate and a 50-year operations and maintenance estimate. The operations and maintenance cost estimate was not updated at Critical Decision (CD)-2 when the construction estimate was approved. The life-cycle estimate did not include retirement of the facility.

The cost estimate completely defines the program, reflects the current schedule, and is technically reasonable.

Partially met. The construction estimate was based on the technical requirements considered 90 percent mature. The operations and maintenance estimate was not supported by definitions of the technical requirements that would have formed the basis of the estimate.

The cost estimate work breakdown structure is product-oriented, traceable to the statement of work/objective, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted.

Partially met. The construction estimate work breakdown structure covered the major work for the end product of the TWF project. The work breakdown structure did not present all cost elements at a clear level of detail, and it was not standardized so that cost data can be collected and used for estimating future programs. The operations and maintenance estimate did not reflect a documented work breakdown structure.

The estimate documents all cost-influencing ground rules and assumptions.

Partially met. The construction estimate included ground rules and assumptions, such as technical specifications, vendor quotes, and registered risks. The operations and maintenance estimate did not document ground rules and assumptions.

The documentation captures the source data used, the reliability of the data, and how the data were normalized.

Partially met. The construction estimate documented the data sources used but not data reliability and how the data were normalized. The operations and maintenance estimate was not supported by detailed documentation.

The documentation describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost.

Partially met. The construction estimate used an engineering buildup approach for individual cost elements. The operations and maintenance estimate did not include documentation on how cost elements were derived.

The documentation describes, step by step, how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it.

Partially met. The construction estimate documentation explained how work breakdown structure elements were estimated and the documentation was mathematically sensible and logical. The documentation explains how management reserve and contingency were calculated and was composed of cost and schedule uncertainty. The operations and maintenance estimate did not include documentation that detailed how the estimate was developed.

Best practice characteristic and overall assessment

Best practice The documentation discusses the technical baseline description, and the data in the baseline is consistent with the estimate.

Detailed assessment Partially met. The documentation for the construction estimate matched the technical requirements document. The operations and maintenance estimate was not supported by a technical baseline document.

The documentation provides evidence that the cost estimate was reviewed and accepted by management.

Partially met. NNSA approved the construction estimate. The approval memo did not detail recommendations for changes, feedback, and the level of contingency reserves decided upon to reach a desired level of confidence. The operations and maintenance estimate did not document management review and approval.

The cost estimate results are unbiased, not overly conservative or optimistic, and based on an assessment of most likely costs.

Partially met. The construction estimate included risk and uncertainty analysis, the results included S curve cumulative probabilities. The risk and uncertainty is quantified as management reserve and contingency. The cost estimate was in range when compared with metrics that benchmark the TWF estimate to similar nuclear projects. The operations and maintenance estimate did not include documentation to help determine whether it was unbiased and not overly conservative or optimistic.

The estimate has been adjusted properly for inflation. Partially met. The construction estimate was adjusted for inflation for the period of the construction schedule, but the cost data were not normalized. The operations and maintenance estimate was not properly adjusted for inflation.

The estimate contains few, if any, minor mistakes.

Partially met. The construction estimate included minor calculation errors in the cost summary table. We were not able to perform random sampling to check calculations for accuracy because the electronic cost model provided to us did not identify the formulas for calculations. The operations and maintenance estimate did not include detailed calculations that we could use to check its accuracy.

The cost estimate is regularly updated to reflect significant changes in the program so that it always reflects current status.

Minimally met. The construction estimate was updated to reflect changes in technical or program assumptions at the CD-3 (approve the start of construction) milestone but is not regularly updated with actual costs on an ongoing basis. The operations and maintenance estimate is not regularly updated to reflect changes in the project and has not been updated since June 2010.

Variances between planned and actual costs are documented, explained, and reviewed.

Minimally met. The construction estimate did not explain variances between planned and actual costs. The CD-2 estimate included a summary level reconciliation with the CD-1 estimate.

Best practice characteristic and overall assessment

Best practice The estimate is based on a historical record of cost estimating and actual experiences from other comparable programs.

Detailed assessment Partially met. The construction estimate was based on the contractor’s market price data for the scope of work required to complete the project. The estimate also uses metrics from similar nuclear projects but did not include documentation on the reliability of the metrics data. The operations and maintenance estimate did not document whether it was based on historical or other data.

The estimating technique for each cost element was used appropriately.

Partially met. The construction estimate was based on engineering buildup approaches appropriate to each cost element. The operations and maintenance estimate did not include documentation of the techniques used for each cost element.

The cost estimate includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs.

Minimally met. The construction estimate identified and examined key cost drivers but did not include a formal sensitivity analysis. The operations and maintenance estimate did not include a sensitivity analysis.

A risk and uncertainty analysis was conducted that quantified the imperfectly understood risks and identified the effects of changing key cost driver assumptions and factors.

Partially met. The construction estimate included risk and uncertainty analysis for cost and schedule and quantified the cost of these risks as management reserve and contingency reserve. The estimate did not document how correlation of cost elements was accounted for in the risk and uncertainty analysis. The operations and maintenance estimate did not include risk and uncertainty analysis.

Major cost elements were cross-checked to see whether results were similar.

Partially met. The construction estimate included cross-checks with metrics that benchmark the TWF to similar nuclear projects. The operations and maintenance estimate did not include documentation of cross-checks.

An independent cost estimate (ICE) was conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results.

Partially met. An ICE for the project’s construction phase was performed by DOE’s Office of Acquisition and Project Management. The ICE appears to have been based on a similar technical baseline to the program office estimate. However, the program estimate was 13 percent higher than the ICE. NNSA did not document how it reconciled the two estimates. The ICE did not cover the operations and maintenance costs of the facility.

The ratings we used in this analysis are as follows: “Not met” means the cost estimate provided no evidence that satisfies the best practice. “Minimally met” means the cost estimate provided evidence that satisfies a small portion of the best practice. “Partially met” means the cost estimate provided evidence that satisfies about half of the best practice. “Substantially met” means the cost estimate provided evidence that satisfies a large portion of the best practice. “Fully met” means the cost estimate provided complete evidence that satisfies the entire best practice.

Appendix III: Comments from the Department of Energy

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Diane LoFaro, Assistant Director; Mark Braza; Richard P. Burkard; Brian M. Friedman; Abishek Krupanand; Eli Lewine; Cynthia Norris; Katrina Pekar-Carpenter; and Karen Richey made key contributions to this report.