Plutonium Disposition Program: DOE Needs to Analyze the Root Causes of Cost Increases and Develop Better Cost Estimates
Released: 2014-02-13T00:00:00

Letter

Background

The end of the cold war left the United States with a surplus of weapons- grade plutonium. Much of this material is found in a key nuclear weapon component known as a pit. In 1997, DOE announced a plan to dispose of surplus, weapons-grade plutonium through an approach that included fabrication of plutonium into MOX fuel for use in domestic commercial nuclear reactors. In 2000, the United States and Russia entered into a Plutonium Management and Disposition Agreement, in which each country pledged to dispose of at least 34 metric tons of surplus, weapons- grade plutonium. Through a protocol to the agreement signed in 2010, the United States and Russia reaffirmed their commitment to dispose of surplus, weapons-grade plutonium as MOX fuel in nuclear reactors, and the agreement entered into force in 2011.

The MOX facility is designed to remove impurities from plutonium feedstock obtained from nuclear weapon pits, form the plutonium into MOX fuel pellets, and fabricate pellets into fuel assemblies for use in a reactor. The MOX facility is a reinforced concrete structure measuring about 600,000 square feet (including support buildings) and, when complete, will include about 300 separate process systems using approximately 23,000 instruments; 85 miles of process piping; 500,000 linear feet of conduit; 3,600,000 linear feet of power and control cable; and 1,000 tons of heating, ventilation, and air conditioning duct work. The WSB will be a 33,000 square foot reinforced concrete structure and will include tanks, evaporators, and solidification equipment to process radioactive liquid waste streams from the MOX facility into solid waste forms suitable for disposal at DOE sites in New Mexico and Nevada. Figure 1 shows aerial views of construction progress for the MOX facility and WSB as of June 2013 and July 2013, respectively.

In addition to the MOX facility and WSB, NNSA’s plans for the U.S. Plutonium Disposition program include the following two additional components:

MOX Irradiation, Feedstock, and Transportation (MIFT). Among other activities, this component includes: (1) production of plutonium feedstock for the MOX facility, (2) qualification of MOX fuel for use in commercial nuclear reactors, and (3) procurement and maintenance of shipping containers for plutonium feedstock and MOX fuel.

Plutonium Disposition and Infrastructure Program (PDIP). This component includes overall management and integration of the MOX facility and WSB projects and integration of the projects with activities falling under MIFT; preparation of environmental impact statements and records of decision for the program in accordance with the National Environmental Policy Act; support for infrastructure at the Savannah River Site, such as site roads; and other activities.

NNSA’s plans for producing plutonium feedstock previously included design and construction of a stand-alone Pit Disassembly and Conversion Facility (PDCF) at the Savannah River Site. As we reported in March 2010, NNSA never established a definitive cost and schedule estimate for the PDCF, but NNSA estimated in January 2011 that the cost of the facility could range from $4.5 billion to $4.8 billion. NNSA canceled the PDCF in January 2012 and, instead, proposed in a July 2012 draft environmental impact statement to meet the feedstock requirements for the MOX facility through existing facilities at DOE’s Los Alamos National Laboratory and the Savannah River Site.cycle cost estimate for the Plutonium Disposition program, NNSA spent $730.1 million on the PDCF prior to its cancellation.

According to NNSA’s draft life- In July 2012, NNSA also announced its preferred alternative for disposition of 13.1 metric tons of surplus plutonium not already included in the 34 metric tons planned for disposal as MOX fuel. The additional plutonium included pits declared excess to national defense needs, as well as surplus non-pit plutonium. According to NNSA officials, the preferred alternative would increase the amount of plutonium disposed as MOX fuel to about 42 metric tons. As of December 2013, DOE had not issued a final supplemental environmental impact statement or record of decision on the facilities to be used to meet plutonium feedstock requirements for the MOX facility or on the disposition pathway for the 13.1 metric tons of surplus plutonium.

NNSA’s Office of Defense Nuclear Nonproliferation provides policy direction for the Plutonium Disposition program, develops and manages annual budgets and the life-cycle cost estimate for the overall program, and manages the MIFT and PDIP components of the program.

NNSA’s Office of Acquisition and Project Management is responsible for managing construction of the MOX facility and WSB projects within approved cost and schedule estimates. To do so, the office manages teams of federal project directors and federal staff that provide direction and oversight of the contractors for both projects, report monthly on the projects’ cost and schedule performance, and evaluate contractors’ performance in areas such as management of subcontractors. The office also conducts reviews of the construction projects to evaluate technical, cost, scope, and other aspects of the projects so that any necessary course corrections can be made. DOE’s project management order requires that such reviews be conducted at least once per year.

NNSA entered into cost-reimbursable contracts for construction of the MOX facility and WSB. A cost-reimbursable contract provides for payment of a contractor’s allowable incurred costs to the extent prescribed in the contract. Agencies may use cost-reimbursable contracts when uncertainties in the scope of work or cost of services prevent the use of contract types in which prices are fixed, known as fixed-price contracts. The MOX and WSB contracts included fees with payment tied to meeting or exceeding preestablished requirements or withholding of fees for any requirements not met, thereby reducing contractors’ profits. Under the MOX contract, NNSA provided four types of fees that the contractor could earn: (1) incentive fees—a type of fee specifically tied to meeting a project’s cost and schedule estimate; (2) milestone fees tied to on-time completion of construction milestones; (3) award fees, which are generally intended to motivate performance in areas other than cost and schedule, such as safety; and (4) fixed fees, a set amount a contractor receives for contract performance. In contrast, NNSA included only one type of fee for the WSB—a performance incentive fee under the contract for management and operation of the Savannah River Site, which included construction of the WSB. In order to provide the contractor performance incentives specifically related to construction of the WSB, NNSA established various performance measures, such as meeting the project’s cost and schedule and completing construction milestones, and allocated portions of the fee to each performance measure.

The contractors for the MOX facility and WSB work with subcontractors to construct the facilities. For example, the WSB contractor entered into a subcontract that included all construction activities for the WSB with the exception of early site work, such as installation of underground utilities.

Once the construction subcontractor completes its work, the WSB contractor is responsible for start-up testing and operation of the facility.

Under DOE’s project management order, the Deputy Secretary of Energy is the senior DOE official accountable for all of the department’s project acquisitions. In addition, the Deputy Secretary approves cost and schedule estimates for all major construction projects—defined as those with values of at least $750 million, which includes the MOX facility—and approves any cost increase over $100 million for a major or nonmajor project. The DOE Office of Acquisition and Project Management conducts external independent reviews to validate estimates prior to approval by the Deputy Secretary. Once estimates have been approved, this office monitors projects’ cost and schedule performance and reports to the Deputy Secretary on a monthly basis. Figure 2 depicts the roles of NNSA, DOE, and contractors in managing the Plutonium Disposition program.

The GAO Cost Estimating and Assessment Guide and the GAO Schedule Assessment Guidethe four characteristics of high-quality, reliable cost and schedule estimates, respectively: compiled best practices corresponding to

The characteristics of a high-quality, reliable cost estimate are comprehensive, well-documented, accurate, and credible. For example, (1) a comprehensive estimate has enough detail to ensure that cost elements are neither omitted nor double counted, (2) a well- documented estimate allows for data it contains to be traced to source documents, (3) an accurate estimate is based on an assessment of most likely costs and has been adjusted properly for inflation, and (4) a credible estimate discusses any limitations because of uncertainty or bias surrounding data or assumptions. Our cost estimating guide also lays out 12 key steps that should result in high- quality cost estimates. For example, one of the steps is to conduct an independent cost estimate––that is, one generated by an entity that has no stake in approval of the project but uses the same detailed technical information as the project estimate. Having an independent entity perform such a cost estimate and comparing it with a project team’s estimate provides an unbiased test of whether a project team’s estimate is reasonable.

The four characteristics of a high-quality, reliable schedule are comprehensive, well-constructed, credible, and controlled. For example, (1) a comprehensive schedule includes all government and contractor activities necessary to accomplish a project’s objectives, (2) a well-constructed schedule sequences all activities using the most straightforward logic possible, (3) a credible schedule uses data about risks and opportunities to predict a level of confidence in meeting the completion date, and (4) a controlled schedule is updated periodically to realistically forecast dates for activities.

NNSA Identified Various Cost Drivers for the Plutonium Disposition Program’s Construction Projects

NNSA identified various drivers of the cost increases for the MOX facility and WSB. NNSA’s budget request for fiscal year 2014 summarized the cost drivers that NNSA considered to be most significant. In addition, NNSA identified some of these drivers in earlier documents, including in reports of project reviews conducted in 2011 and 2012, in monthly status reports for the projects, and, for the WSB, in the document requesting approval for a cost increase. NNSA and contractor officials provided additional details on these drivers during interviews with us.

Key drivers NNSA identified for the cost increase for the MOX facility included the following:

DOE’s approval of the cost and schedule before design was complete. The head of NNSA’s Office of Acquisition and Project Management told us that, judging from the MOX contractor’s design costs during construction of the MOX facility, the overall design was about 58 percent complete when DOE approved the project’s cost and schedule estimate in April 2007. In contrast, according to DOE’s project management order, to support the development of a cost estimate, the design of complex nuclear processing facilities needs to be closer to 100 percent complete than the design of basic facilities, such as administrative buildings and general purpose laboratories. NNSA’s budget request for fiscal year 2014 stated that the cost of critical system components for the MOX facility averaged 60 percent higher than estimated as a result of approval of these estimates before design was complete. According to NNSA and MOX contractor officials, after the contractor completed designs for critical system components, such as the gloveboxes used in the facility for handling plutonium and related infrastructure, equipment suppliers submitted higher bids than the contractor anticipated. For example, according to the contractor’s Vice President of Operations, a vendor submitted a bid in 2008 that was four times the amount the same vendor had estimated in 2005.

Higher-than-anticipated costs to install equipment. For example, the MOX contractor estimated in its September 2012 proposal to increase the cost of the facility that the labor hours to install each foot of the approximately 85 miles of piping in the facility increased by as much as 26 percent and that, as facility designs became more definitive, the total amount of pipe increased by close to 33 percent over the previous estimate. In addition, according to NNSA, the number of safety systems needed to meet Nuclear Regulatory Commission (NRC) requirements was greater than anticipated, further adding to equipment installation costs. According to NNSA officials, NNSA and the contractor did not have a good understanding of the cost of designing the facility to meet NRC requirements related to demonstrating the ability to withstand an earthquake. The officials explained that the facility’s design is based on a similar facility in France but that NRC regulatory requirements differ from those in France.

The contractor’s difficulty identifying suppliers and subcontractors able to fabricate and install equipment meeting nuclear quality assurance criteria.project was experiencing the same issues identifying qualified suppliers and subcontractors as other nuclear projects across DOE. These issues included a higher than expected effort associated with attracting qualified vendors and, after vendors were selected, responding to questions or correcting noncompliance with requirements. For example, according to NNSA and the MOX contractor, the contractor needed to station quality assurance personnel at supplier and subcontractor locations to oversee activities.

According to NNSA’s review of the MOX project in 2011, the

Greater-than-expected turnover of engineering and technical staff. In particular, the project lost staff to other nuclear industry projects, including projects in neighboring states, resulting in a nearly complete turnover of construction management personnel over a period of several years and the need to provide training to replacement personnel. NNSA identified this driver in its budget request for fiscal year 2012. Specifically, the budget request stated that over 15 percent of the project’s engineering and technical personnel had left for other nuclear industry jobs in the previous year with pay increases of at least 25 percent. The budget request further stated that finding experienced replacements had become difficult and expensive. According to the budget requests for fiscal years 2013 and 2014, the loss of experienced engineering and technical staff to other nuclear industry projects has continued.

Change in scope of the project to add capability to the MOX facility to produce plutonium feedstock. As part of its decision to cancel plans for a stand-alone PDCF and to instead meet feedstock requirements through existing facilities, NNSA directed the MOX contractor to include feedstock capability in its September 2012 proposal to increase the cost of the facility. The contractor’s proposal included an estimate of $262.3 million to add feedstock capability.

In identifying these drivers of the cost increase for the MOX facility, NNSA did not identify the dollar amount associated with each cost driver. An NNSA official said that the MOX contractor’s system for tracking and reporting on cost and schedule performance could potentially be used to determine dollar amounts that each driver added to the overall cost increase—which is one possible use of such a system—but that doing so would be time-consuming and difficult. As a result, NNSA officials could not substantiate the relative importance of the cost drivers. For example, NNSA officials said they had not conducted a formal analysis to back up an estimate, which they had made when we first discussed the cost drivers with them, that lack of design maturity of critical system components accounted for more than half of the increase. In reviewing the MOX contractor’s system, we found that, as NNSA officials stated, using the system to determine the dollar amounts each driver added to the cost increase would be difficult—for example, because the system’s identification of cost increases at a summary level, such as site construction support, did not correspond to the cost drivers identified by NNSA.

Key cost drivers NNSA identified for the WSB included the following:

Higher-than-anticipated bids for the construction subcontract.

According to the NNSA federal project director for the WSB, the WSB contractor received two bids in 2009 from prospective construction subcontractors that both came in at about $26 million higher than the contractor’s estimate. NNSA officials did not explain the reason for the difference, stating that the bidders were not required to provide details of their estimates. The federal project director said that NNSA supported the WSB contractor awarding the construction subcontract, despite the higher cost, in order to maintain the schedule for completing the WSB in time to support the start-up of the MOX facility. According to NNSA officials, the project applied cost savings from earlier work to cover part of the increased cost of the construction subcontract and had sufficient contingency—the portion of a project’s budget that is available to account for uncertainties in the project’s scope—to absorb the remainder of the increase. Consequently, however, contingency to absorb further cost increases as construction progressed was reduced.

Design errors, omissions, and inconsistencies. According to the NNSA federal project director, the WSB contractor and subcontractor made hundreds of design changes, which led to an additional cost increase in the construction subcontract. According to NNSA’s log of design changes, as of August 2013, design changes increased the cost of the construction subcontract by about $15 million, from $91.5 million to $106.5 million. The federal project director said that, unlike the design of the MOX facility, the design of the WSB was about 90 percent complete at the start of construction. A September 2008 report of NNSA’s independent review of the WSB prior to approval of the cost and schedule estimate found that the design was essentially complete. Nevertheless, according to the federal project director, design changes were needed because of constructability issues, such as equipment that met specifications in design documents not being available by the time the project reached construction.

Schedule delays resulting from the construction subcontractor not meeting required targets. According to the NNSA federal project director’s feedback on the WSB contractor’s performance in September 2009, NNSA had concerns related to the project schedule and the ability to meet the completion date in part because of a delayed start in the construction subcontract. By the time NNSA approved the cost increase for the WSB in December 2012, schedule delays in the construction subcontract had grown to 15 months. The approved cost increase included about $30 million in the contractor’s delay-related costs because NNSA’s contract for the WSB is cost- reimbursable.

The actual cost attributable to the WSB may be even higher depending on the outcome of a lawsuit filed by the subcontractor against the WSB contractor related to design changes and schedule delays that increased the subcontractor’s costs in excess of the amount specified in its fixed- price subcontract. The approved cost increase for the WSB included contingency to account for the possibility of higher costs incurred by the construction subcontractor.

NNSA Has Not Analyzed Root Causes of the Cost Increases for the Plutonium Disposition Program’s Construction Projects

NNSA has not analyzed the underlying, or root, causes of the close to $3 billion in construction cost increases for the MOX facility and WSB. DOE’s project management order requires that lessons learned be captured throughout a project to allow for the exchange of information within DOE in the context of project management and to benefit future endeavors. However, the project management order does not include a requirement for a root cause analysis of projects experiencing significant cost increases or schedule delays. NNSA officials said that they decide on a case-by-case basis whether to conduct a root cause analysis. In contrast, under the Weapon Systems Acquisition Reform Act of 2009, the Department of Defense must perform a root cause analysis of a cost increase that exceeds a certain threshold.

Documentation NNSA provided to us on the cost drivers for the MOX facility and WSB do not provide clear details about the causes of the cost increases. Such details can be found in a root cause analysis, which would help address questions about why the drivers identified by NNSA occurred and help inform lessons learned. Key questions about the cause of the key drivers include the following:

DOE’s reasons for approving a cost and schedule estimate for the MOX facility before the design was complete, even though a July 2006 review of the project found that the cost estimate’s basis on portions of the design that were less than 50 percent complete posed a risk to the project. Similarly, a root cause analysis would address why one of the drivers of the cost increase for the WSB identified by NNSA was design errors, omissions, and inconsistencies, given that a review prior to approval of the project’s cost and schedule estimate found that most of the design was ready for construction.

The extent to which NNSA and its contractors shared responsibility for cost drivers, such as the greater-than-anticipated number of safety systems needed in the MOX facility to meet NRC requirements. According to NNSA officials, the department hired the MOX contractor because it considered the contractor to be well-qualified to engineer and estimate all of the safety systems for the facility, taking into account NRC requirements. However, the record for DOE’s approval of the cost and schedule estimate for the facility shows that DOE was aware of complexities in adapting MOX technology to comply with NRC requirements. Specifically, the minutes from DOE’s July 2006 meeting to request approval of the estimate stated that these complexities had already contributed to a $1.1 billion increase in the estimated cost.

The sufficiency of measures DOE took to ensure that the cost estimate for the MOX facility it approved in 2007 reflected an awareness of market conditions, such as the availability of suppliers and subcontractors with the ability and experience to meet nuclear quality assurance criteria. As required under the MOX contract, in October 2006—before DOE approved the cost and schedule estimate for the facility—the contractor submitted a construction market analysis report, which stated that the contractor had experienced trouble obtaining qualified suppliers and that the subcontractor pool using nuclear quality standards had been decreasing due to inactivity in the nuclear industry. However, the report provided limited detail and did not include recommendations to address availability of qualified suppliers.

The thoroughness of DOE’s review, required under DOE’s project management order, to ensure that the WSB contractor’s system for tracking and reporting on cost and schedule performance provided accurate information. DOE recertified the contractor’s system in December 2011 after identifying and closing out several corrective actions and continuous improvement opportunities. However, DOE found additional problems with the system after January 2012, when the WSB contractor informed NNSA that schedule delays for the project were greater than the contractor previously revealed. Based in part on the contractor’s revelations, DOE reexamined the contractor’s system and suspended its certification in November 2012.

The corrective actions NNSA and its contractors took after periodic project reviews identified problems, including problems cited by NNSA as drivers of cost increases for the MOX facility and WSB. For example, multiple reviews of the MOX facility found that costs to install equipment were underestimated. A July 2006 review found that installation for electrical; piping; and heating, ventilation, and air- conditioning equipment were underestimated by close to $160 million and nearly 3 million labor hours. NNSA’s project reviews of the facility in 2011 and 2012 continued to raise concerns about unrealistic installation rates.

The responsiveness of NNSA project managers to emerging cost and schedule issues. Without a review of the timing of NNSA initiating the process of increasing the projects’ cost and schedule estimates, it is not clear whether NNSA acted in a timely manner or whether project cost and schedule indicators warranted earlier action. For example, an NNSA review of the MOX facility in the spring of 2011 found that the most significant risk to delivering the project within cost centered on the ability of the project team to identify about $364 million in savings to offset expected cost growth, but NNSA did not initiate the process of increasing the project’s cost and schedule estimates until January 2012.

Without a root cause analysis, it is uncertain whether NNSA will be able to accurately identify underlying causes of the cost increases for the MOX facility and WSB in order to identify and implement corrective measures and identify lessons learned to share with and apply to other DOE construction projects.

NNSA Has Taken Steps to Hold Contractors Accountable for Cost Increases by Withholding Fees

After determining that the performance of the contractors for the MOX facility and WSB contributed to the projects’ construction cost increases, NNSA took steps to hold the contractors accountable for their performance by withholding fees specified under the contracts. Specifically, NNSA withheld portions of two of the four types of the MOX contractor’s fees and 41 percent of the WSB contractor’s fees.

NNSA Withheld Portions of Two of the Four Types of the MOX Contractor’s Fees

NNSA withheld portions of two of the four types of fees that the MOX contractor could earn under the contract for construction of the facility— incentive fees and award fees. In total, NNSA withheld $45.1 million or close to one-third of all fees the contractor could earn as of November 2013. Under the terms of the MOX contract, the contractor could still earn incentive fees that have been withheld, but only if it completes the overall project within cost and schedule. Table 1 summarizes fees paid to and withheld from the contractor as of November 2013.

Details of fees NNSA withheld and paid under the MOX contract include the following: Incentive fees. NNSA did not pay $36.5 million or over half of the $65.6 million in incentive fees that the MOX contractor could earn from fiscal year 2008, when construction began, through fiscal year 2013. Of the $29.1 million in incentive fees paid to the contractor, $21.6 million remains provisional, meaning that NNSA can require that the fees be paid back as a result of the project not being completed within cost. The amount not paid represented the contractor’s entire incentive fees for fiscal years 2011 through 2013. Specifically, under the terms of the MOX contract, NNSA can withhold quarterly payments of incentive fees if an increase in the projected cost to complete the MOX facility exceeds $200 million. NNSA began withholding incentive fees for the first quarter of fiscal year 2011 when, for the first time, the increase in the projected cost to complete the facility exceeded this threshold. NNSA memos for subsequent quarters in fiscal year 2011 noted that the project’s cost and schedule metrics continued to worsen, reducing the likelihood of resumption of payments. In a July 2011 letter to the contractor explaining its rationale for not resuming payments, NNSA stated that it was sensitive to the potential impacts of the “nuclear renaissance”—the contractor’s term for the resurgence of U.S. nuclear engineering and manufacturing capability after being dormant for more than 20 years, which the contractor stated limited the availability of qualified suppliers and subcontractors and led to staff turnover and higher-than- anticipated costs to install equipment. However, NNSA stated that such impacts would not necessarily overcome other evidence showing that the contractor was not meeting the overarching goal of the incentive fees, which is that the facility be completed within cost.

Award fees. NNSA withheld $8.6 million or about a quarter of the $32.6 million in award fees that the MOX contractor could earn from fiscal year 2008 through fiscal year 2012. The amount withheld included about half of the fees the contractor was eligible to earn in fiscal year 2012. NNSA’s award fee evaluation for fiscal year 2012 cited various factors, such as poor construction planning; less than optimal coordination of work; and overly conservative specifications for installation of fire doors, resulting in delays and unnecessary costs. In contrast, NNSA paid $24.0 million in award fees for performance in other areas, such as maintaining a high level of worker safety—an area in which the contractor has consistently performed well, according to NNSA’s award fee evaluations.

Milestone fees. NNSA did not withhold any milestone fees and instead paid milestone fees of $30.8 million for tasks with deadlines ranging from February 2009 to March 2014. Examples of tasks for which NNSA paid milestone fees (some of which the MOX contractor completed early) included completing the roof, installing the first glovebox, constructing a technical support building, and completing a start-up plan for the facility. According to the NNSA officials, although NNSA did not withhold milestone fees, NNSA stopped paying any of the $30.2 million in remaining milestones fees as part of an understanding with the contractor to renegotiate the amount of and conditions for earning milestone fees.

Fixed fees. According to the contracting officer, NNSA did not withhold any of the $15.7 million in fixed fees—the total amount of fixed fees for construction-related work under the MOX contract. NNSA included these fees in the contract to reward the contractor for work performed during contract negotiations, when other fees had not yet been negotiated.

NNSA Withheld 41 Percent of the WSB Contractor’s Fees through Fiscal Year 2012

In a March 2013 analysis of the WSB contractor’s performance, the NNSA contracting officer for the WSB recommended that the contractor should be held accountable for performance failures that contributed to the project’s cost increase. For example, the analysis stated that the contractor did not require the subcontractor to add crews or take other steps to correct delays until almost 2 years after the federal project director began expressing concerns about the delays. In accordance with this assessment, NNSA withheld $7.7 million or about 40 percent of the $18.9 million in performance incentive fees that the WSB contractor could earn from fiscal year 2009, when construction began, through fiscal year 2012, for the portion of fees allocated to construction of the WSB under the management and operation contract for the Savannah River Site. Most of the fees withheld were for the contractor’s performance in fiscal years 2011 and 2012 (see table 2). In particular, NNSA withheld $3.3 million of the $6.9 million in fees the contractor could earn in fiscal year 2011 and $3.9 million of the $4.0 million in fees the contractor could earn in fiscal year 2012. The fees withheld were tied to various performance measures, which DOE acquisition regulations require be established prior to the start of each evaluation period. Performance measures NNSA established included meeting the schedule for testing various types of equipment, providing engineering support to and coordinating with the construction subcontractor, and maintaining the project within pre- established cost and schedule metrics.

The $3.3 million in fees withheld for fiscal year 2011 included $2 million that NNSA took back—that is, was paid back by the contractor—after making its fee determination for the contractor. Specifically, according to a December 2012 letter from the NNSA contracting officer to the contractor, the fiscal year 2011 fee determination was premised on the contractor’s statements that schedule delays were recoverable and that the project would be completed within the approved cost estimate. Shortly after NNSA made its fee determination, however, the contractor notified NNSA that the project was further behind schedule than previously represented and that cost factors not included in the contractor’s system for tracking and reporting on cost and schedule performance would result in a cost overrun. The contracting officer’s letter stated that NNSA would have reduced the contractor’s fee if it had known the extent of delays and cost overruns when it made its fee determination, and NNSA required the contractor to repay $4 million. In May 2013, NNSA agreed to a settlement with the contractor to reduce the amount taken back to $2 million after the contractor appealed NNSA’s initial demand.

In addition to withholding fees, in a June 2012 letter to the contractor, NNSA’s contracting officer questioned why she should not conclude that the contractor’s actions rose to the level of gross negligence or willful misconduct, warranting disallowance of costs, meaning that the contractor would bear part of the cost increase resulting from the project’s schedule delays. For example, the letter stated that the contractor’s system for tracking and reporting on cost and schedule performance did not meet industry standards and impeded NNSA’s ability to understand the potential impact of delays in construction of various segments of the project on the final delivery date. According to NNSA officials, NNSA is waiting until after completion of WSB construction, and total construction costs are known, to determine unallowable costs.

Estimates for the Plutonium Disposition Program’s Life-cycle Cost and Construction Projects Did Not Meet All Best Practices for Reliability

NNSA’s most recent cost and schedule estimates for the Plutonium Disposition program did not fully reflect the characteristics of high-quality, reliable estimates as established by best practices used throughout government and industry and documented in the GAO Cost Estimating and Assessment Guide and GAO Schedule Assessment Guide. Specifically, (1) NNSA’s draft April 2013 life-cycle cost estimate for the overall program was partially comprehensive, partially well-documented, and partially accurate but did not meet any of the best practices for a credible estimate; (2) the MOX contractor’s September 2012 proposal for increasing the cost of the MOX facility was substantially comprehensive but partially well-documented and accurate and minimally credible; and (3) the WSB contractor’s February 2013 monthly update to its schedule estimate was minimally well-constructed and partially met the other three characteristics of a reliable schedule—comprehensive, credible, and controlled.

NNSA Did Not Follow All Key Steps for Developing a Reliable Life-cycle Cost Estimate for the Plutonium Disposition Program

In developing its draft April 2013 life-cycle cost estimate of $24.2 billion for the Plutonium Disposition program, NNSA followed several of the 12 key steps for developing high-quality cost estimates, including defining the estimate’s purpose, defining the program’s characteristics, and obtaining the data. NNSA did not follow other key steps, however, such as conducting an independent cost estimate. As a result, the estimate was not reliable. In particular, NNSA’s draft life-cycle cost was partially comprehensive, partially well-documented, and partially accurate but did not meet any of the best practices for a credible estimate.

Table 3 summarizes the major components of NNSA’s draft April 2013 life-cycle cost estimate. The estimate assumed that the MOX facility would start operations in November 2019 and that it would take approximately 15 years to complete the mission to dispose of 34 metric tons of surplus weapons-grade plutonium.

Table 4 lists the steps, or best practices, necessary for developing a high- quality cost estimate. Appendix II summarizes our assessment of NNSA’s process for developing its draft life-cycle cost estimate against the steps that should result in the four characteristics of a high-quality cost estimate.

Our assessment of NNSA’s process for developing its draft life-cycle cost estimate included the following observations:

Comprehensive. The draft life-cycle cost estimate was partially comprehensive because work breakdown structures were developed for the MOX and WSB projects and other components of the program, but NNSA had not formalized a program-level work breakdown structure. A typical work breakdown structure provides a clear picture of what needs to be accomplished, how the work will be done, and a basis for identifying resources and tasks for developing a cost estimate. Without a program-level work breakdown structure, NNSA cannot ensure that its life-cycle cost estimate captures all relevant costs, which can mean cost overruns.

Well-documented. The draft life-cycle cost estimate was partially well- documented because NNSA defined the estimate’s purpose and the program’s characteristics, but it did not develop a single document to describe data sources and steps taken in developing the estimate— such as applying escalation rates to account for inflation—so that the estimate could be replicated by someone other than those who prepared it. In addition, NNSA stated that a document identified the estimate’s ground rules and assumptions but that the assumptions have changed frequently, hindering development of a life-cycle cost estimate. Examples of changes in assumptions not reflected in NNSA’s draft April 2013 estimate included the slowdown of activities during the assessment of alternative plutonium disposition strategies and NNSA’s plans to increase the amount of plutonium disposed of as MOX fuel.

Accurate. The draft life-cycle cost estimate was partially accurate in that NNSA followed the best practice for developing a point estimate—a best guess at a cost estimate usually falling between best and worst case extremes. NNSA also updated the estimate periodically to include actual costs and changes to program and project requirements. However, NNSA did not use a formal system for tracking and reporting on cost and schedule performance to update the estimate, limiting the ability of someone other than those who prepared the estimate to check the estimate’s accuracy and to identify when, how much, and why the program cost more or less than planned.

Credible. The draft life-cycle cost estimate was not credible because NNSA did not conduct an independent cost estimate to provide an unbiased test of whether its estimate was reasonable, a formal sensitivity analysis to examine the effects of changing assumptions and ground rules, or a risk and uncertainty analysis to assess variability in point estimates due to factors such as errors and cost estimators’ inexperience or biases. NNSA conducted such analyses for portions of its life-cycle cost estimate, but not for the entire estimate. For example, NNSA’s Plutonium Disposition program office arranged for another office within NNSA to conduct an independent assessment of the MOX facility’s operations costs, but not for the program’s entire life-cycle cost.

NNSA did not follow all key steps for developing high-quality cost estimates in part because it did not have a requirement to develop its life- cycle cost estimate. According to NNSA officials, DOE’s project management order includes requirements for development of cost and schedule estimates for a project, such as the MOX facility or WSB, but does not specify equivalent requirements for a program like Plutonium Disposition, which includes multiple projects, as well as supporting activities. As a result, when developing the life-cycle cost estimate for the Plutonium Disposition program, NNSA officials used an ad hoc approach to adapt requirements for managing projects in DOE’s project management order. NNSA officials also said that its April 2013 life-cycle cost estimate did not include all the steps of a high-quality, reliable estimate in part because NNSA considered the estimate to be draft and, therefore, had not fully implemented plans for developing it.

In the absence of a specific requirement in DOE’s project management order for developing a life-cycle cost estimate for a program, NNSA officials said they developed a life-cycle cost estimate for the Plutonium Disposition program for several reasons. According to these officials, these reasons included that the cost of the program is largely made up of capital projects, such as the MOX facility, and that requirements for congressional budget submissions specify that the full life-cycle cost of such projects be presented. In addition, each year NNSA must submit to Congress its estimated expenditures covering the fiscal year with respect to which the budget is submitted and at least the four succeeding fiscal years. NNSA officials said that, to accurately estimate expenditures for this 5-year period, they needed to develop a life-cycle cost estimate for the overall Plutonium Disposition program. An NNSA official noted that NNSA plans to use a version of its life-cycle cost estimate as a basis for evaluating alternative strategies to dispose of surplus weapons-grade plutonium.

Contractor’s Proposed Estimate for the MOX Facility Did Not Meet Most Best Practices for Reliability

The MOX contractor’s September 2012 proposal for increasing the cost of the MOX facility was substantially comprehensive but was partially well- documented, partially accurate, and minimally credible. The contractor’s estimate did not fully reflect the characteristics of a high-quality, reliable estimate in part because it was a proposal, as opposed to an approved cost estimate. For example, one of the best practices for a well- documented estimate—and a requirement of DOE’s project management order—is that a cost estimate be reviewed and accepted by management. Because DOE had not approved it and instead postponed its review and approval pending the outcome of NNSA’s assessment of alternative plutonium disposition strategies, the contractor’s estimate partially met this best practice. This best practice would be met by DOE’s completion of its review and approval of a new estimate for the MOX facility, assuming the assessment of alternative plutonium disposition strategies maintains the current strategy of disposing plutonium as MOX fuel.

Though the contractor’s September 2012 estimate did not fully reflect the characteristics of a high-quality estimate and cannot be considered reliable, the MOX contractor began using it as a provisional baseline for purposes of monthly reporting on the project’s cost and schedule performance. Specifically, as directed by NNSA, the contractor began a transition in June 2012 to report its monthly performance against the contractor’s proposed estimate of $7.7 billion. The contractor completed the transition and ceased any reporting of performance against the previously approved estimate early in 2013. Managing projects that no longer have an approved cost and schedule estimate is a challenge because cost and schedule estimates provide a baseline for measuring progress. At a July 2013 hearing, the Deputy Secretary of Energy noted that not having such a baseline is the point of maximum risk of unrestricted cost growth on a project.

Appendix III summarizes our assessment of how well the MOX contractor’s proposal met the characteristics of a high-quality estimate. Our assessment included the following observations:

Comprehensive. The proposal was substantially comprehensive in that it included all construction costs, as defined by the statement of work under the MOX contract. The proposal was not fully comprehensive, however, because it only partially met certain best practices for a comprehensive estimate, such as documenting all cost-influencing ground rules and assumptions. The proposal partially met this best practice because it did not provide justifications for some assumptions, such as not more than 10 percent of the supports for piping systems being nonstandard and requiring unique designs.

Well-documented. The proposal was partially well-documented because it described in sufficient detail the calculations performed and the estimating methodology used to derive the cost of each element in the work breakdown structure. However, it did not provide all types of information specified in best practices for a well-documented estimate, such as how data on labor and travel costs were normalized. Data normalization is often necessary to ensure comparability because data can be gathered from a variety of sources and in different forms that need to be adjusted before being used.

Accurate. The proposal was partially accurate in that it appeared to adjust cost elements for inflation and contained only a few minor mistakes, but the contractor did not update its proposal with actual costs incurred after it developed the proposal and submitted it to NNSA in September 2012. NNSA and contractor officials agreed that the estimate was no longer an accurate reflection of the cost to complete construction—for example, because the proposal assumed a higher level of funding than the project received in fiscal year 2013. The officials said that, if the MOX project continues, the contractor would need to prepare a new proposal that includes costs for work conducted after the initial proposal was developed.

Credible. The proposal was minimally credible because DOE halted its independent cost estimate of the proposal pending the outcome of NNSA’s assessment of alternative plutonium disposition strategies. Moreover, the proposal did not include a formal sensitivity analysis to examine the effects of changing assumptions and ground rules, and it provided no evidence that major cost elements were cross-checked to determine whether alternative cost-estimating methods produced similar results. Finally, the proposal included an analysis of risks, such as difficulty attracting and retaining workers, and uncertainty in estimating materials and other costs. On the basis of this analysis, the proposal included a total of $713.1 million to account for risks and uncertainty—$641.4 million for the original scope of the MOX facility and $71.7 million for the addition of plutonium feedstock capability (see table 5). However, the contractor did not properly conduct or clearly document all steps in the analysis to determine the amount of funding to account for risks and uncertainty that could increase the cost of the project.

Contractor’s Schedule Estimate for the WSB Did Not Meet Most Best Practices for Reliability

The WSB contractor’s February 2013 monthly update to its schedule estimate did not fully reflect the characteristics of a high-quality, reliable schedule estimate as established by best practices. Specifically, the contractor’s schedule estimate was minimally well-constructed and partially met the other three characteristics of a reliable, high-quality schedule as measured against best practices—comprehensive, credible, and controlled. Table 6 shows the characteristics of a high-quality schedule estimate and corresponding best practices. Appendix IV summarizes our assessment of how well the WSB contractor’s February 2013 schedule estimate met the characteristics of a high-quality estimate.

Our assessment of the WSB contractor’s February 2013 schedule estimate included the following observations:

Comprehensive. The estimate was partially comprehensive in that it captured and established the durations of contractor and government activities to complete the project but did not capture the remaining detailed work to be performed by the construction subcontractor. Specifically, it reduced the subcontractor’s 3,851 activities to complete its portion of the work to one placeholder activity. According to the NNSA federal project director, the WSB contractor reduced the subcontractor’s activities to a placeholder because the subcontractor submitted unreliable schedules with repeated changes in the estimated completion date for its portion of work.

Well-constructed. The estimate was minimally well-constructed in that it sequenced activities in ways that can obscure a schedule’s earliest completion date. In addition, the sequencing of activities included “merge points”—the convergence of many parallel activities into a single successor activity, which decreased the probability of successor activities starting on time. For example, performance of an assessment of readiness to operate the WSB was preceded by 212 activities. NNSA officials explained that the merge points resulted from the need to complete activities in parallel to meet requirements set forth in DOE’s project management order.

Credible. The estimate was partially credible in that the WSB contractor conducted a schedule risk analysis to determine the amount of schedule contingency—a reserve of extra time to account for risks and ensure completion of the project on time. However, a DOE review conducted prior to approval of an increase in the project’s cost and a delay in the start of operations found that the results of the contractor’s analysis were unreliable—for example, because project team members were not consulted regarding risk inputs. As a result, the schedule risk analysis did not clearly support the 12 months of schedule contingency included in the approved cost increase and schedule delay.

Controlled. The estimate was partially controlled in that, according to project officials, the schedule was updated weekly and used to measure performance, but no narrative accompanied weekly updates to provide decision makers with a log of changes and their effect, if any, on the schedule time frame. In addition, project officials did not provide documentation enabling the schedule to be validated, such as documentation describing sequencing of activities or assumptions used in developing the schedule.

The NNSA federal project director and contractor’s project leader said that the contractor had begun to correct problems in the contractor’s schedule estimate—for example, by replacing the placeholder for the subcontractor’s activities with a schedule of more detailed activities independently developed by the contractor. However, delays on the project continued after the contractor began correcting the problems. Notably, according to DOE’s October 2013 monthly report on the WSB, continuing delays in completion of the construction subcontract—one of the key drivers NNSA identified for the WSB cost increase—already used up about 10 of the 12 months of schedule contingency, placing the project’s completion date in jeopardy.

Conclusions

NNSA has identified drivers of the close to $3 billion increase in the projected cost to complete the MOX facility and WSB and has taken steps to hold the MOX and WSB contractors accountable for their role in the cost increases by withholding and taking back fees. However, the various drivers identified by NNSA, such as DOE’s approval of the cost and schedule estimate for the MOX facility before design was complete, do not provide the level of detail that can be found in a root cause analysis. In addition, DOE’s project management order requires that lessons learned be captured throughout a project but does not include a requirement for a root cause analysis when a project exceeds its cost estimate, even when a project exceeds its cost estimate by billions of dollars. The decision whether to conduct such an analysis is instead made on a case-by-case basis. Because NNSA has not conducted a root cause analysis to identify the underlying causes of the cost increases for the MOX facility and WSB, it cannot provide assurance that it has correctly identified the underlying causes to ensure that they will not lead to further cost increases as the projects move forward. Further, without a root cause analysis, NNSA’s ability to identify recommended solutions and lessons learned that could be applied to other projects is lessened. Conducting a root cause analysis of the cost increases for the MOX facility and WSB could help NNSA address its long-standing difficulties in completing projects within cost and on schedule, which has led to NNSA’s project management remaining on GAO’s list of areas at high risk of fraud, waste, abuse, and mismanagement.

NNSA has drafted a life-cycle cost estimate of $24.2 billion for the Plutonium Disposition program—an important step toward presenting the full cost of NNSA’s current strategy to dispose of surplus weapons-grade plutonium as MOX fuel. A cost estimate that presents the full cost of NNSA’s current plutonium disposition strategy is essential to inform NNSA’s ongoing evaluation of alternative plutonium disposition strategies and provide Congress with a complete picture of the cost of the program. NNSA developed its life-cycle cost estimate even though neither DOE nor NNSA required the estimate. In particular, DOE’s project management order does not explicitly require that life-cycle cost estimates be developed for programs like the Plutonium Disposition program that include both construction projects and other efforts and activities not related to construction, such as producing plutonium feedstock for the MOX facility. In the absence of such a requirement, NNSA followed several of the 12 key steps described in the GAO Cost Estimating and Assessment Guide for developing high-quality, reliable cost estimates, but it did not follow other key steps. Because NNSA did not follow all of the steps, the life-cycle estimate for the Plutonium Disposition program is not reliable. Similarly, the contractors’ cost and schedule estimates for the MOX facility and WSB did not meet all best practices compiled in GAO’s guides for preparing high-quality, reliable cost and schedule estimates. Not meeting these best practices increased the risk of further cost increases and delays for the projects and, because the projects are components of NNSA’s life-cycle cost estimate, for the overall Plutonium Disposition program.

Recommendations for Executive Action

We are making six recommendations in this report to the Secretary of Energy. To identify lessons learned from and provide assurance of preventing recurrence of cost increases for the MOX facility and WSB, and to develop reliable cost estimates for the Plutonium Disposition program, we recommend that the Secretary of Energy direct the DOE and NNSA Offices of Acquisition and Project Management and the NNSA office responsible for managing the Plutonium Disposition program, as appropriate, to take the following four actions:

Conduct an analysis of the root causes of the cost increases for the MOX facility and WSB, such as the causes of the design changes that led to cost increases, and identify and prioritize recommended solutions.

Revise and update the program’s life-cycle cost estimate following the 12 key steps described in the GAO Cost Estimating and Assessment Guide for developing high-quality cost estimates, such as conducting an independent cost estimate to provide an objective and unbiased assessment of whether the estimate can be achieved.

Ensure that the MOX contractor revises its proposal for increasing the cost of the MOX facility to meet all best practices for a high-quality, reliable cost estimate—for example, by cross-checking major cost elements to determine whether alternative estimating methods produce similar results.

Ensure that the approved cost increase for the WSB is based on a schedule that the contractor has revised to meet all best practices for a high-quality, reliable schedule estimate, such as reflecting all activities (both government and contractor) needed to complete construction.

To ensure that future DOE projects benefit from lessons learned that reflect the underlying causes of cost increases or schedule delays experienced by other projects, and that Congress and DOE have life- cycle cost estimates for DOE programs that include individual construction projects, we further recommend that the Secretary of Energy take the following two actions to revise DOE’s project management order or otherwise implement a departmentwide requirement:

Require a root cause analysis of all projects that experience cost increases or schedule delays exceeding a certain threshold established by DOE.

Require life-cycle cost estimates covering the full cost of programs that include both construction projects and other efforts and activities not related to construction.

Agency Comments and Our Evaluation

We provided a draft of this product to DOE for comment. In written comments, reproduced in appendix V, NNSA stated that the agency and DOE generally agreed with our recommendations. In particular, NNSA concurred with four of our six recommendations and partially concurred with the other two. NNSA described actions it planned to take to implement the recommendations with which it concurred and time frames for taking these actions. NNSA also provided technical comments, which we incorporated into the report as appropriate.

We are pleased that NNSA concurred with our recommendation to conduct an analysis of the root causes of the cost increases for the MOX facility and WSB and stated that it is planning to conduct such an analysis, which was not mentioned during the course of our review. NNSA also concurred with our recommendation to revise and update the Plutonium Disposition program’s life-cycle cost estimate and stated that it would do so after a decision was made on the path forward for the program. The path forward could involve the use of alternative strategies to dispose of surplus weapons-grade plutonium.

NNSA also concurred with our recommendation to ensure that the MOX contractor revises its proposal for increasing the cost of the MOX facility to meet all best practices for a high-quality, reliable cost estimate. In its comment letter, NNSA stated that it is working with the contractor to ensure that the cost estimating processes and procedures are updated such that the best practices are met.

In addition, NNSA concurred with our recommendation to ensure that the approved cost increase for the WSB is based on a schedule that the contractor has revised to meet all best practices for a high-quality, reliable schedule estimate. In its comment letter, NNSA stated that it has revised the schedule since we reviewed it and that it now reflects all activities needed to complete construction. We did not review the update to the WSB contractor’s schedule to confirm that it captured all activities to complete construction, which is one of the best practices associated with the characteristics of a high-quality schedule. Moreover, as detailed in appendix IV, the schedule we reviewed only partially or minimally met 7 of the other 9 best practices. To fully implement our recommendation, NNSA would need to ensure that the contractor has revised its schedule to meet all best practices for a high-quality, reliable schedule estimate. In its comment letter, NNSA stated that during the next project review, which is expected to occur by December 31, 2014, NNSA will review the schedule against best practices.

NNSA partially concurred with our fifth recommendation that DOE require a root cause analysis of all projects that experience cost increases or schedule delays exceeding a certain threshold established by the department. In its comment letter, NNSA stated that DOE program offices currently perform tailored root cause analyses as part of the baseline change proposal process outlined in the department’s project management order for increasing a project’s cost and schedule estimates. NNSA stated that, as a result, the department does not believe that an update to the project management order is required. NNSA further stated that the department will review the lessons learned from NNSA's root cause analyses for the MOX and WSB projects to see what best practices may be of benefit to other projects. However, as we stated in the report, DOE’s project management order does not include a requirement for a root cause analysis of projects experiencing significant cost increases or schedule delays, and NNSA officials said that they decide on a case-by-case basis whether to conduct a root cause analysis. Moreover, the order does not define what a root cause analysis is, how or when a root cause analysis should be conducted, or what is meant by a tailored analysis. In addition, NNSA’s written comments did not provide information on the conditions that would trigger a root cause analysis. Leaving root cause analyses to an informal and undefined process within DOE program offices could result in such analyses not being conducted, not being conducted consistently, or not accurately identifying underlying causes of cost increases in order to identify and implement corrective measures and apply lessons learned to other DOE projects. We continue to believe that a root cause analysis should be conducted for all projects that experience cost increases or schedule delays above a threshold established by the department. We note that our recommendation is consistent with a requirement in the Weapon Systems Acquisition Reform Act of 2009; under the act, the Department of Defense must perform a root cause analysis of a cost increase that exceeds a certain threshold.

NNSA partially concurred with our final recommendation that DOE require life-cycle cost estimates covering the full cost of programs that include both construction projects and other efforts and activities not related to construction. In its comment letter, NNSA stated that the department’s project management order requires a comprehensive life-cycle cost analysis as part of the alternative selection process and that no further update to the order is required to address this recommendation. The intent of our recommendation goes beyond that of preparing a life-cycle cost estimate at the stage of selecting an alternative for a new capital asset project. Instead, the recommendation applies to departmental programs that include capital asset projects to meet the overall program need. As we stated in the report, NNSA did not follow all key steps for developing high-quality cost estimates in developing its draft April 2013 life-cycle cost estimate for the Plutonium Disposition program, which currently includes the MOX facility and WSB capital asset projects, in part because it did not have a requirement to develop it. NNSA’s response to our recommendation suggests that the life-cycle cost estimates for the MOX and WSB projects that were required to be prepared years ago, when the projects were selected from among other alternatives, are the only life-cycle cost estimates needed to manage the Plutonium Disposition program. Furthermore, NNSA’s response contradicts the fact that it concurred with our recommendation to revise and update the life- cycle cost estimate for the overall Plutonium Disposition program in accordance with cost estimating best practices. We continue to believe that our recommendation that the department require life-cycle cost estimates covering the full cost of programs that include construction projects should be implemented.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 7 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Energy, the NNSA Administrator, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

To assess drivers of the construction cost increases for the Mixed Oxide (MOX) Fuel Fabrication Facility and Waste Solidification Building (WSB) that the National Nuclear Security Administration (NNSA) identified, we reviewed the Department of Energy’s (DOE) budget request for NNSA for fiscal year 2014, which provided a summary of the cost drivers for both projects. To assess cost drivers in further detail, we reviewed the MOX contractor’s September 2012 proposal for increasing the project’s cost, which discussed drivers from the contractor’s perspective. We also reviewed DOE’s December 2012 document approving an increase in the estimated cost of the WSB and a delay in the start of operations, which summarized cost drivers and their impact on the project’s cost and schedule. We visited the Savannah River Site to observe construction progress for both projects and interviewed NNSA and contractor officials responsible for managing the projects. We also interviewed officials from the NNSA Office of Fissile Materials Disposition, the NNSA Office of Acquisition and Project Management, and the DOE Office of Acquisition and Project Management. Separately, to understand how, if at all, cost drivers for the MOX facility were related to Nuclear Regulatory Commission (NRC) regulation and licensing of the construction and operation of the facility, we reviewed NRC construction inspection reports and related documents, and we interviewed NRC officials responsible for overseeing the facility’s construction. In order to understand the components of cost growth for the MOX facility, which represented most of the Plutonium Disposition program’s construction cost increase, we also analyzed the MOX contractor’s earned value management (EVM) system that the contractor used to track and report on cost and schedule performance, including data from the EVM system on cumulative cost and schedule variance trends from July 2011 through April 2012 and the contractor’s variance report for April 2012.

To determine the extent to which NNSA analyzed underlying causes of the cost increases, we reviewed documents providing context for cost drivers. The documents we reviewed included NNSA Office of Acquisition and Project Management project review reports and monthly status reports; DOE Office of Acquisition and Project Management monthly status reports; DOE documents related to approval of the previous cost and schedule estimates for the MOX facility and WSB in April 2007 and December 2008, respectively; and documents related to specific cost drivers identified by NNSA, such as the MOX contractor’s October 2006 report on construction markets and DOE reports related to its suspension of the WSB contractor’s system for tracking and reporting cost and schedule performance in November 2012. We also interviewed NNSA officials to determine the extent to which they had conducted or planned any analyses to identify underlying causes of cost increases for the Plutonium Disposition program’s construction projects.

To determine steps NNSA took to hold contractors accountable for their role in the cost increases for the Plutonium Disposition program’s construction projects, we reviewed the contracts for the MOX facility and WSB, fees specified under the contracts, and NNSA’s fee evaluations and other documentation supporting its fee determinations. We also interviewed NNSA contracting officers who were responsible for administering the MOX and WSB contracts regarding the terms of the contracts, fees specified under the contracts, and actions NNSA took or planned to take to hold contractors accountable for their role in the cost increases. We obtained NNSA data on fees it paid to and withheld from the contractors, and we assessed the reliability of the data by checking for obvious errors in accuracy and completeness; comparing the data with other sources of information, such as NNSA’s fee determinations; and interviewing NNSA contracting officers who had knowledge of the data. We determined that NNSA’s data on fees were sufficiently reliable for reporting on the fees paid to and withheld from the contractors.

To assess the extent to which NNSA’s most recent estimates of the Plutonium Disposition program’s life-cycle cost and of the cost and schedule for completing the program’s construction projects met best practices we have compiled in guides identifying the characteristics of high-quality, reliable cost and schedule estimates, we tailored our methodology to the differing stages of NNSA’s development and approval of each estimate:

NNSA’s life-cycle cost estimate for the Plutonium Disposition program. Because NNSA had not finalized a life-cycle cost estimate, we assessed NNSA’s most recent available estimate—spreadsheets dated April 2013 representing NNSA’s draft life-cycle cost estimate. In particular, we assessed the process NNSA used to develop the estimate against the 12 key steps described in the GAO Cost Estimating and Assessment Guide that should result in a high-quality, reliable cost estimate. To provide information on NNSA’s process, NNSA officials responsible for developing the estimate filled out a data collection instrument we developed. The data collection instrument summarized each of the 12 key steps and provided space for NNSA officials to describe actions they had taken to meet the criteria for each step. To review the information provided by NNSA, we checked NNSA’s April 2013 estimate for obvious errors in accuracy and completeness and compared it with previous versions of the life-cycle cost estimate provided by NNSA. In addition, we interviewed NNSA officials to determine what requirements, if any, they followed for developing the estimate, their purpose for developing it, and their plans for presenting it for management approval. Finally, we interviewed NNSA officials from the Office of Analysis and Evaluation, which the Plutonium Disposition program had tasked with conducting an independent assessment of the MOX facility’s operating costs.

NNSA’s estimate to complete the MOX facility. Because NNSA had not approved a revised cost and schedule estimate for the MOX facility, we assessed the MOX contractor’s September 2012 proposal for increasing the project’s cost, which NNSA had directed the MOX contractor to use as a provisional baseline for purposes of monthly reporting. We compared data presented in various tables of the proposal for consistency and reviewed additional documents, including the technical baseline providing a detailed description of the MOX facility. We provided a draft of our assessment to NNSA and revised the draft, as appropriate, after discussing our assessment with NNSA program officials and the contractor.

NNSA’s estimate to complete the WSB. We assessed the WSB schedule estimate that the cost increase for the project approved in December 2012 was based on because, as described in the GAO Schedule Assessment Guide, a reliable schedule can contribute to an understanding of the cost impact if a project does not finish on time. Specifically, we compared the contractor’s February 2013 monthly update to its schedule estimate, which was the most recent available update when we conducted our analysis, with the 10 best practices associated with the characteristics of a high-quality schedule. As part of our assessment, we reviewed documents related to the project’s schedule, including NNSA’s project execution plan for the WSB, the project’s work breakdown structure, and the project’s February 2013 update to the document showing the longest path to project completion. In addition, we interviewed the NNSA federal project director for the WSB and the WSB contractor’s project leader and scheduler. We provided a draft of our assessment to NNSA and revised the draft, as appropriate, after discussing our assessment with NNSA program officials and the contractor.

We conducted this performance audit from November 2012 to February 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Assessment of NNSA’s Process for Developing a Life-cycle Cost Estimate for the Plutonium Disposition Program

Best practice characteristic and overall assessment Comprehensive: Partially met

Detailed assessment Partially met. NNSA assigned a team to develop and update the estimate but did not have a written plan for developing it.

Partially met. Work breakdown structures to define in detail the work necessary to accomplish objectives were developed for the MOX and WSB projects and other components of the program, but NNSA had not formalized a program-level work breakdown structure.

Substantially met. NNSA officials described the purpose as supporting annual budget requests, which include requirements that NNSA (1) present the full life-cycle cost of capital projects, such as the MOX facility, and (2) estimate expenditures for the fiscal year with respect to which the budget is submitted and at least the four succeeding fiscal years.

Substantially met. NNSA developed a program requirements document to identify the scope, functions, and requirements of the Plutonium Disposition program. NNSA documented performance characteristics for program components in contracts, technical baselines, and execution plans.

Partially met. NNSA identified ground rules and assumptions, but NNSA officials said that assumptions for the program change frequently, hindering development of a life-cycle cost estimate.

Substantially met. NNSA collected data at the project level, where, according to NNSA, data were documented in contractor systems and estimates were developed by teams of knowledgeable staff using historical information, current cost and pricing information, engineering and vendor quotes, cost guides, and current material and labor costs.

Minimally met. NNSA documented the estimate on spreadsheets, but it did not develop a single document to describe data sources and steps taken in developing the estimate so that it could be replicated by someone other than those who prepared it.

Not met. NNSA considered the estimate to be draft and predecisional, and NNSA officials said they did not have plans to present an estimate to management for approval until NNSA completes its reevaluation of its strategy for disposing of surplus weapons-grade plutonium.

Partially met. NNSA developed a point estimate, but it did not use a program-level work breakdown structure to do so because it had not formalized such a structure.

Partially met. NNSA updated the estimate periodically to include actual costs and changes to program and project requirements, but it did not clearly document how changes affected the estimate.

Best practice characteristic and overall assessment Credible: Not met

Detailed assessment Not met. NNSA did not conduct an independent cost estimate for the overall program’s life-cycle cost estimate, and it had not completed independent cost estimates for the program’s two construction projects.

Not met. NNSA did not conduct a formal sensitivity analysis at the program level.

Not met. NNSA did not conduct a risk and uncertainty analysis at the program level.

Appendix III: Assessment of the MOX Contractor’s Proposed Cost Estimate Compared with Industry Best Practices

Best practice characteristic and overall assessment Comprehensive: Substantially met

Best practice The cost estimate includes all life-cycle costs.

Detailed assessment Met. The estimate covered construction and startup costs; at NNSA’s direction, the estimate excluded operation and maintenance costs.

The cost estimate completely defines the program, reflects the current schedule, and is technically reasonable.

Substantially met. The estimate was based on NNSA’s statement of work and the contractor’s technical baseline for the original scope of the MOX facility.

The cost estimate work breakdown structure is product-oriented, traceable to the statement of work/objective, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted.

Partially met. The work breakdown structure clearly outlined the end product and major work of the project, but some cost elements were missing from the work breakdown structure.

The estimate documents all cost- influencing ground rules and assumptions.

Partially met. The estimate documented that it was based on a profile of NNSA’s projected annual funding to complete the project but did not provide justifications for some assumptions, such as not more than a set amount of work being nonstandard.

The documentation captures the source data used, the reliability of the data, and how the data were normalized.

Partially met. The estimate was based on actual costs through May 2012 and used a database of labor and other costs, but it did not state whether or how all data had been normalized to ensure data comparability.

The documentation describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost.

Met. The estimate used a combination of expert opinion and extrapolation from actual data to develop estimates for and sum up individual cost elements of the work breakdown structure.

The documentation describes, step by step, how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it.

Partially met. The estimate used quantities of materials and labor hours to develop estimates for individual cost elements but did not document how these quantities were estimated.

The documentation discusses the technical baseline description and the data in the baseline is consistent with the estimate.

Partially met. The estimate agreed with NNSA’s statement of work and the contractor’s technical baseline for the original scope of the MOX facility, but the technical baseline did not cover the addition of capability to supply plutonium feedstock.

The documentation provides evidence that the cost estimate was reviewed and accepted by management.

Partially met. DOE began a review of the proposed estimate but did not approve it.

The cost estimate results are unbiased, not overly conservative or optimistic, and based on an assessment of most likely costs.

Minimally met. The estimate was higher than needed to achieve an 85 percent confidence level—the level directed by NNSA—that the final cost would be less than the estimate.

The estimate has been adjusted properly for inflation.

Substantially met. The estimate appeared to adjust cost elements for inflation, but adjustments were not well-documented.

Best practice characteristic and overall assessment

Best practice The estimate contains few, if any, minor mistakes.

Detailed assessment Met. The estimate contained few minor mistakes, and calculations within the estimate were internally consistent.

The cost estimate is regularly updated to reflect significant changes in the program so that it always reflects current status.

Partially met. The estimate was based on actual costs through May 2012 and did not reflect updated costs from the contractor’s system for tracking and reporting cost and schedule performance.

Variances between planned and actual costs are documented, explained, and reviewed.

Minimally met. The estimate explained variances between planned and actual costs at a high level but not at the cost element level.

The estimate is based on a historical record of cost estimating and actual experiences from other comparable programs.

Partially met. The estimate did not explain to what extent it was based on historical data from other similar programs or facilities.

The estimating technique for each cost element was used appropriately.

Substantially met. The estimating method used— developing the estimate at the lowest level of the work breakdown structure, one piece at a time, with the sum of the pieces becoming the estimate—was appropriate for a project under way.

The cost estimate includes a sensitivity analysis that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs.

Not met. The estimate did not include a sensitivity analysis.

A risk and uncertainty analysis was conducted that quantified the imperfectly understood risks and identified the effects of changing key cost driver assumptions and factors.

Partially met. The estimate included a risk and uncertainty analysis but did not properly conduct or clearly document all steps in the analysis.

Major cost elements were cross-checked to see whether results were similar.

Not met. The estimate provided no evidence that major cost elements were cross-checked.

An independent cost estimate was conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results.

Not met. DOE halted its independent cost estimate of the contractor’s proposed estimate as part of DOE’s decision to reevaluate its strategy for disposing of surplus weapons-grade plutonium.

Appendix IV: Assessment of the Waste Solidification Building’s Schedule Estimate Compared with Industry Best Practices

Best practice characteristic and overall assessment Comprehensive: Partially met

Detailed assessment Minimally met. The schedule estimate’s 2,429 activities to complete the project included one summary activity in place of the construction subcontractor’s 3,851 activities and, therefore, did not capture the remaining detailed work to be performed by the subcontractor.

Partially met. The schedule estimate assigned resources, such as labor and materials, to only about half of the remaining 2,429 activities.

Substantially met. The schedule estimate included activity durations that were generally short enough to be consistent with the needs of effective planning.

Minimally met. The schedule estimate sequenced activities in ways that decreased the probability of activities starting on time and contained activities that were not properly tied with the start or end date of other activities, potentially obscuring the critical path determining the project’s earliest completion date.

Partially met. Changes to the critical path were evaluated monthly and tracked in monthly status reports, but constraints in scheduled dates of certain activities convoluted the critical path.

Ensuring reasonable total float Minimally met. The schedule estimate included high total float values—the amount of time by which an activity can slip without affecting a completion date—potentially resulting in an inaccurate assessment of the project’s completion date.

Substantially met. The schedule estimate was traceable horizontally (i.e., across sequenced activities) and vertically (i.e., between activities and subactivities).

Minimally met. The contractor conducted a schedule risk analysis, but the results of the analysis were unreliable for determining the likelihood of the project’s completion date and did not align with DOE’s revised cost and schedule estimate.

Partially met. According to project officials, the schedule was updated weekly, but no narrative accompanied the weekly updates.

Maintaining a baseline schedule Minimally met. Project officials stated that they used the schedule to measure performance, but they did not provide thorough documentation enabling the schedule to be validated, such as a narrative providing a log of changes and their effects.

Appendix V: Comments from the National Nuclear Security Administration

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Daniel Feehan, Assistant Director; Remmie Arnold; Antoinette Capaccio; Juaná S. Collymore; Joseph Cook; Tisha Derricotte; Emile Ettedgui; Cristian Ion; Alison O’Neill; Cheryl Peterson; and Karen Richey made key contributions to this report.