Plutonium Disposition Program: DOE Needs to Analyze the Root Causes of Cost Increases and Develop Better Cost Estimates
Released: 2014-02-13T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: NNSA, a separately organized agency within DOE, manages the Plutonium Disposition program to dispose of surplus weapons-grade plutonium by burning it as MOX fuel—a mixture of plutonium and uranium oxides—in specially modified commercial nuclear reactors. In 2012, DOE forecasted cost increases of close to $3 billion over the previous estimates for the program's two construction projects, the MOX facility and the WSB for disposing of waste from the MOX facility.

GAO was asked to review these cost increases and the life-cycle cost estimate. This report examines: (1) drivers NNSA identified for the cost increases; (2) the extent to which NNSA analyzed underlying causes of the cost increases; (3) steps NNSA took to hold construction contractors accountable for their role, if any, in the cost increases; and (4) the extent to which NNSA's most recent estimates met cost- and schedule-estimating best practices. GAO reviewed NNSA's draft life-cycle cost estimate and contractor estimates of the MOX project's cost and WSB schedule, compared the estimates with cost- and schedule-estimating best practices, and interviewed DOE and NNSA officials.

section_title: What GAO Found
paragraphs: The Department of Energy's (DOE) National Nuclear Security Administration (NNSA) identified various drivers for the close to $3 billion increase in the estimated cost of the Plutonium Disposition program's two construction projects—the Mixed Oxide (MOX) Fuel Fabrication Facility and the Waste Solidification Building (WSB). These drivers included DOE's approval of the MOX facility's cost and schedule estimates before design was complete and schedule delays in construction of the WSB. According to NNSA, the cost of critical system components for the MOX facility averaged 60 percent higher than estimated as a result of approval of estimates before design was complete.

NNSA has not analyzed the underlying, or root, causes of the Plutonium Disposition program construction cost increases to help identify lessons learned and help address the agency's difficulty in completing projects within cost and schedule, which has led to NNSA's management of major projects remaining on GAO's list of areas at high risk of fraud, waste, abuse, and mismanagement. DOE's project management order requires that lessons learned be captured throughout a project to, among other things, benefit future endeavors. NNSA officials said that, because the order does not require a root cause analysis of cost increases, NNSA decides on a case-by-case basis whether to conduct one. Unlike a root cause analysis, the cost drivers NNSA identified provided few details about why the drivers existed, such as DOE's reasons for approving the MOX facility's cost and schedule estimates before the design was complete. Without a root cause analysis, it is uncertain whether NNSA will be able to accurately identify underlying causes of the increases to identify and implement corrective measures and identify lessons learned to apply to other projects.

After determining that the performance of the contractors for the MOX facility and WSB contributed to cost increases, NNSA took steps to hold the contractors accountable by withholding fees specified under the contracts. In particular, as of November 2013, NNSA withheld $45.1 million or close to one-third of the MOX contractor's fees, including fees tied to meeting the MOX project's cost and schedule estimates. In addition, NNSA withheld $7.7 million or about 40 percent of the WSB contractor's fees tied to various performance measures for the WSB, such as completing construction milestones.

NNSA's most recent estimates for the Plutonium Disposition program did not fully reflect all the characteristics of reliable cost estimates (e.g., credible) and schedule estimates (e.g., well-constructed) as established by best practices for cost- and schedule-estimating, placing the program at risk of further cost increases. For example: (1) NNSA's draft April 2013 life-cycle cost estimate of $24.2 billion for the overall program was not credible because NNSA did not conduct an independent cost estimate to provide an unbiased test of whether the estimate was reasonable. (2) Because the MOX contractor's September 2012 proposal for increasing the cost of the MOX facility did not include a formal analysis to examine the effects of changing assumptions, it was minimally credible. (3) The WSB contractor's February 2013 monthly update to its schedule estimate was minimally well-constructed in that it contained activities that were not properly tied with the start or end date of other activities, which could potentially obscure the critical path determining the project's completion date.

section_title: What GAO Recommends
paragraphs: GAO is recommending, among other things, that DOE conduct a root cause analysis of the Plutonium Disposition program's cost increases and ensure that future estimates of the program's life-cycle cost and cost and schedule for the program's construction projects meet all best practices for reliable estimates. DOE generally agreed with GAO's recommendations.

section_title: Letter
paragraphs: Plutonium is a man-made, radioactive element that poses a danger of  nuclear weapons proliferation and a risk to human health and the  environment. Under an agreement signed in 2000, the United States and  Russia have each pledged to dispose of at least 34 metric tons of surplus  weapons-grade plutonium no longer needed for defense purposes by  burning it as mixed oxide (MOX) fuel in specially modified commercial  nuclear reactors. Once used and removed from a reactor, the plutonium  can no longer be readily used to make a nuclear weapon. The National  Nuclear Security Administration (NNSA), a separately organized agency  within the Department of Energy (DOE), manages the Plutonium  Disposition program. As part of this program, NNSA entered into separate  contracts for design and construction services for two facilities at DOE’s  Savannah River Site in South Carolina: (1) the MOX Fuel Fabrication  Facility for producing MOX fuel for nuclear reactors and (2) the Waste  Solidification Building (WSB) for disposing of liquid waste from the MOX  facility.

DOE, Program and Project Management for the Acquisition of Capital Assets, DOE  Order 413.3B (Washington, D.C.: Nov. 29, 2010). management order establishes a process for DOE and NNSA to review  and approve a project’s construction cost and schedule estimates,  including changes to the estimates; report monthly on a project’s cost and  schedule performance; and conduct project reviews at least once per  year.

In February of 2013, NNSA’s contract administration and project  management were again included on GAO’s list of areas at high risk of  fraud, waste, abuse, and mismanagement. In particular, NNSA has  experienced long-standing difficulties in completing major projects within  cost and on schedule. As we testified in March 2013, DOE forecasted an  increase in its estimated cost and schedule to complete the MOX facility  and WSB. Specifically, in 2012, DOE forecasted a close to $3 billion  increase in the estimated cost to complete the two facilities:

MOX facility. In April 2007, DOE approved a cost estimate for the  MOX facility of $4.8 billion and start of operations in September  2016. Construction began in August 2007. In 2012, at NNSA’s  direction to update the estimate, the MOX contractor submitted a  proposal to increase the cost of the facility to $7.7 billion with the start  of operations delayed to November 2019. DOE began evaluating the  proposal and, pending the outcome of its evaluation, directed the  contractor to use its proposed cost and schedule estimate as a  provisional baseline for purposes of monthly reporting.

WSB. In December 2008, NNSA approved start of construction of the  WSB and a cost estimate of $344.5 million and start of operations in  September 2013. In February 2012, NNSA directed the WSB contractor to develop a plan to address potential cost and schedule  overruns. In December 2012, NNSA approved an increase in the  estimated cost to $414.1 million and a delay in the start of operations  to August 2015.

NNSA has not finalized a life-cycle cost estimate for the Plutonium  Disposition program—that is, an estimate of all costs to complete the  mission to dispose of surplus weapons-grade plutonium—but, in April  2013, it completed a draft estimate of $24.2 billion. In addition to  construction costs, the estimate included operation and maintenance of  the MOX facility and WSB, as well as other components of the program,  such as production of plutonium feedstock for the MOX facility. The life- cycle cost estimate also included $5.2 billion in actual costs through fiscal  year 2012: $3.4 billion for the MOX facility, $265 million for the WSB, and  $1.5 billion for other components of the program.

In 2013, in light of the cost increases for the Plutonium Disposition  program’s construction projects, the fiscal year 2014 budget request for  NNSA stated that converting plutonium to MOX fuel may be unaffordable.  The budget request announced that, as a result, NNSA would slow down  activities associated with the current plutonium disposition strategy during  an assessment of alternative plutonium disposition strategies. As of  January 2014, this assessment had not yet been completed.

To provide assistance to federal agencies in preparing cost and schedule  estimates, we have compiled best practices used throughout government  and industry. In March 2009 and May 2012, we issued guides identifying  the characteristics of high-quality, reliable cost and schedule estimates,  respectively. Specifically, the four characteristics of a high-quality cost  estimate are comprehensive, well-documented, accurate, and credible,  and the four characteristics of a high-quality schedule estimate are  comprehensive, well-constructed, credible, and controlled.

You asked us to review issues related to the construction cost increases  for the MOX facility and WSB and the life-cycle cost of the overall  Plutonium Disposition program. This report examines: (1) drivers of the  cost increases that NNSA identified for the Plutonium Disposition  program’s construction projects, (2) the extent to which NNSA analyzed  underlying causes of the cost increases, (3) steps NNSA took under the  contracts for the construction projects to hold the contractors accountable  for their role, if any, in the cost increases, and (4) the extent to which  NNSA’s most recent estimates for the program’s life-cycle cost and the  cost and schedule for completing the program’s construction projects met  best practices for reliable cost and schedule estimates.

To assess drivers of the cost increases, we reviewed the MOX  contractor’s September 2012 proposal for increasing the project’s cost,  the change to the WSB project’s cost approved in December 2012, and  other documents. In addition, we visited the Savannah River Site to  observe construction progress for both projects and interviewed NNSA  and contractor officials. Because the MOX facility represents most of the  program’s construction cost increase, we also analyzed the MOX  contractor’s system for tracking and reporting on cost and schedule  performance. To determine the extent to which NNSA analyzed  underlying causes of the cost increases, we reviewed DOE and NNSA  documentation of the cost increases and interviewed NNSA officials. To  determine steps taken to hold contractors accountable for their role in the  cost increases, we reviewed the contracts for both projects, fees specified  under the contracts, and NNSA’s fee evaluations and other  documentation supporting its fee determinations. In addition, we obtained  NNSA data on fees it paid to and withheld from the MOX and WSB  contractors. We assessed the reliability of the data by interviewing NNSA  contracting officers responsible for administering the contracts and other  means, and we determined that the data were sufficiently reliable for  reporting on the fees NNSA paid and withheld.

To assess the extent to which NNSA’s most recent estimates of the  program’s life-cycle cost and the cost and schedule for completing the  program’s construction projects met best practices, we tailored our  methodology to the differing stages of NNSA’s development and approval  of each estimate. Specifically, because NNSA had not finalized a life- cycle cost estimate for the program or a revised cost and schedule  estimate for the MOX facility, we assessed the most recent available  estimates—spreadsheets dated April 2013 representing NNSA’s draft life- cycle cost estimate and the MOX contractor’s September 2012 proposal  for increasing the project’s cost. We assessed the WSB schedule  estimate because, as described in the GAO Schedule Assessment Guide,  a reliable schedule can contribute to an understanding of the cost impact  if a project does not finish on time. Specifically, we compared the  contractor’s February 2013 monthly update to its schedule estimate,  which was the most recent available update when we conducted our  analysis. Additional details on our objectives, scope, and methodology  can be found in appendix I.

We conducted this performance audit from November 2012 to February  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: The end of the cold war left the United States with a surplus of weapons- grade plutonium. Much of this material is found in a key nuclear weapon  component known as a pit. In 1997, DOE announced a plan to dispose of  surplus, weapons-grade plutonium through an approach that included  fabrication of plutonium into MOX fuel for use in domestic commercial  nuclear reactors. In 2000, the United States and Russia entered into a  Plutonium Management and Disposition Agreement, in which each  country pledged to dispose of at least 34 metric tons of surplus, weapons- grade plutonium. Through a protocol to the agreement signed in 2010, the  United States and Russia reaffirmed their commitment to dispose of  surplus, weapons-grade plutonium as MOX fuel in nuclear reactors, and  the agreement entered into force in 2011.

The MOX facility is designed to remove impurities from plutonium  feedstock obtained from nuclear weapon pits, form the plutonium into  MOX fuel pellets, and fabricate pellets into fuel assemblies for use in a  reactor. The MOX facility is a reinforced concrete structure measuring  about 600,000 square feet (including support buildings) and, when  complete, will include about 300 separate process systems using  approximately 23,000 instruments; 85 miles of process piping; 500,000  linear feet of conduit; 3,600,000 linear feet of power and control cable;  and 1,000 tons of heating, ventilation, and air conditioning duct work. The  WSB will be a 33,000 square foot reinforced concrete structure and will  include tanks, evaporators, and solidification equipment to process  radioactive liquid waste streams from the MOX facility into solid waste  forms suitable for disposal at DOE sites in New Mexico and Nevada.  Figure 1 shows aerial views of construction progress for the MOX facility  and WSB as of June 2013 and July 2013, respectively.

In addition to the MOX facility and WSB, NNSA’s plans for the U.S.  Plutonium Disposition program include the following two additional  components:

MOX Irradiation, Feedstock, and Transportation (MIFT). Among other  activities, this component includes: (1) production of plutonium  feedstock for the MOX facility, (2) qualification of MOX fuel for use in  commercial nuclear reactors, and (3) procurement and maintenance  of shipping containers for plutonium feedstock and MOX fuel.

Plutonium Disposition and Infrastructure Program (PDIP). This  component includes overall management and integration of the MOX  facility and WSB projects and integration of the projects with activities  falling under MIFT; preparation of environmental impact statements  and records of decision for the program in accordance with the  National Environmental Policy Act; support for infrastructure at the  Savannah River Site, such as site roads; and other activities.

NNSA’s plans for producing plutonium feedstock previously included  design and construction of a stand-alone Pit Disassembly and Conversion  Facility (PDCF) at the Savannah River Site. As we reported in March  2010, NNSA never established a definitive cost and schedule estimate for  the PDCF, but NNSA estimated in January 2011 that the cost of the  facility could range from $4.5 billion to $4.8 billion. NNSA canceled the  PDCF in January 2012 and, instead, proposed in a July 2012 draft  environmental impact statement to meet the feedstock requirements for  the MOX facility through existing facilities at DOE’s Los Alamos National  Laboratory and the Savannah River Site.cycle cost estimate for the Plutonium Disposition program, NNSA spent  $730.1 million on the PDCF prior to its cancellation.

According to NNSA’s draft life- In July 2012, NNSA also announced its preferred alternative for  disposition of 13.1 metric tons of surplus plutonium not already included  in the 34 metric tons planned for disposal as MOX fuel. The additional  plutonium included pits declared excess to national defense needs, as  well as surplus non-pit plutonium. According to NNSA officials, the  preferred alternative would increase the amount of plutonium disposed as  MOX fuel to about 42 metric tons. As of December 2013, DOE had not  issued a final supplemental environmental impact statement or record of  decision on the facilities to be used to meet plutonium feedstock  requirements for the MOX facility or on the disposition pathway for the  13.1 metric tons of surplus plutonium.

NNSA’s Office of Defense Nuclear Nonproliferation provides policy  direction for the Plutonium Disposition program, develops and manages  annual budgets and the life-cycle cost estimate for the overall program,  and manages the MIFT and PDIP components of the program.

NNSA’s Office of Acquisition and Project Management is responsible for  managing construction of the MOX facility and WSB projects within  approved cost and schedule estimates. To do so, the office manages  teams of federal project directors and federal staff that provide direction  and oversight of the contractors for both projects, report monthly on the  projects’ cost and schedule performance, and evaluate contractors’  performance in areas such as management of subcontractors. The office  also conducts reviews of the construction projects to evaluate technical,  cost, scope, and other aspects of the projects so that any necessary  course corrections can be made. DOE’s project management order  requires that such reviews be conducted at least once per year.

NNSA entered into cost-reimbursable contracts for construction of the  MOX facility and WSB. A cost-reimbursable contract provides for  payment of a contractor’s allowable incurred costs to the extent  prescribed in the contract. Agencies may use cost-reimbursable  contracts when uncertainties in the scope of work or cost of services  prevent the use of contract types in which prices are fixed, known as  fixed-price contracts. The MOX and WSB contracts included fees with  payment tied to meeting or exceeding preestablished requirements or  withholding of fees for any requirements not met, thereby reducing  contractors’ profits. Under the MOX contract, NNSA provided four types  of fees that the contractor could earn: (1) incentive fees—a type of fee  specifically tied to meeting a project’s cost and schedule estimate;  (2) milestone fees tied to on-time completion of construction milestones;  (3) award fees, which are generally intended to motivate performance in  areas other than cost and schedule, such as safety; and (4) fixed fees, a  set amount a contractor receives for contract performance. In contrast,  NNSA included only one type of fee for the WSB—a performance  incentive fee under the contract for management and operation of the  Savannah River Site, which included construction of the WSB. In order to  provide the contractor performance incentives specifically related to  construction of the WSB, NNSA established various performance  measures, such as meeting the project’s cost and schedule and  completing construction milestones, and allocated portions of the fee to  each performance measure.

The contractors for the MOX facility and WSB work with subcontractors to  construct the facilities. For example, the WSB contractor entered into a  subcontract that included all construction activities for the WSB with the  exception of early site work, such as installation of underground utilities.

Once the construction subcontractor completes its work, the WSB  contractor is responsible for start-up testing and operation of the facility.

Under DOE’s project management order, the Deputy Secretary of Energy  is the senior DOE official accountable for all of the department’s project  acquisitions. In addition, the Deputy Secretary approves cost and  schedule estimates for all major construction projects—defined as those  with values of at least $750 million, which includes the MOX facility—and  approves any cost increase over $100 million for a major or nonmajor  project. The DOE Office of Acquisition and Project Management conducts  external independent reviews to validate estimates prior to approval by  the Deputy Secretary. Once estimates have been approved, this office  monitors projects’ cost and schedule performance and reports to the  Deputy Secretary on a monthly basis. Figure 2 depicts the roles of NNSA,  DOE, and contractors in managing the Plutonium Disposition program.

The GAO Cost Estimating and Assessment Guide and the GAO  Schedule Assessment Guidethe four characteristics of high-quality, reliable cost and schedule  estimates, respectively:   compiled best practices corresponding to

The characteristics of a high-quality, reliable cost estimate are  comprehensive, well-documented, accurate, and credible. For  example, (1) a comprehensive estimate has enough detail to ensure  that cost elements are neither omitted nor double counted, (2) a well- documented estimate allows for data it contains to be traced to source  documents, (3) an accurate estimate is based on an assessment of  most likely costs and has been adjusted properly for inflation, and  (4) a credible estimate discusses any limitations because of  uncertainty or bias surrounding data or assumptions. Our cost  estimating guide also lays out 12 key steps that should result in high- quality cost estimates. For example, one of the steps is to conduct an  independent cost estimate––that is, one generated by an entity that  has no stake in approval of the project but uses the same detailed  technical information as the project estimate. Having an independent  entity perform such a cost estimate and comparing it with a project  team’s estimate provides an unbiased test of whether a project team’s  estimate is reasonable.

The four characteristics of a high-quality, reliable schedule are  comprehensive, well-constructed, credible, and controlled. For  example, (1) a comprehensive schedule includes all government and  contractor activities necessary to accomplish a project’s objectives,  (2) a well-constructed schedule sequences all activities using the  most straightforward logic possible, (3) a credible schedule uses data  about risks and opportunities to predict a level of confidence in  meeting the completion date, and (4) a controlled schedule is updated  periodically to realistically forecast dates for activities.

section_title: NNSA Identified Various Cost Drivers for the Plutonium Disposition Program’s Construction Projects
paragraphs: NNSA identified various drivers of the cost increases for the MOX facility  and WSB. NNSA’s budget request for fiscal year 2014 summarized the  cost drivers that NNSA considered to be most significant. In addition,  NNSA identified some of these drivers in earlier documents, including in  reports of project reviews conducted in 2011 and 2012, in monthly status  reports for the projects, and, for the WSB, in the document requesting  approval for a cost increase. NNSA and contractor officials provided  additional details on these drivers during interviews with us.

Key drivers NNSA identified for the cost increase for the MOX facility  included the following:

DOE’s approval of the cost and schedule before design was  complete. The head of NNSA’s Office of Acquisition and Project  Management told us that, judging from the MOX contractor’s design  costs during construction of the MOX facility, the overall design was  about 58 percent complete when DOE approved the project’s cost  and schedule estimate in April 2007. In contrast, according to DOE’s  project management order, to support the development of a cost  estimate, the design of complex nuclear processing facilities needs to  be closer to 100 percent complete than the design of basic facilities,  such as administrative buildings and general purpose laboratories.  NNSA’s budget request for fiscal year 2014 stated that the cost of  critical system components for the MOX facility averaged 60 percent  higher than estimated as a result of approval of these estimates  before design was complete. According to NNSA and MOX contractor  officials, after the contractor completed designs for critical system  components, such as the gloveboxes used in the facility for handling  plutonium and related infrastructure, equipment suppliers submitted  higher bids than the contractor anticipated. For example, according to  the contractor’s Vice President of Operations, a vendor submitted a  bid in 2008 that was four times the amount the same vendor had  estimated in 2005.

Higher-than-anticipated costs to install equipment. For example, the  MOX contractor estimated in its September 2012 proposal to increase  the cost of the facility that the labor hours to install each foot of the  approximately 85 miles of piping in the facility increased by as much  as 26 percent and that, as facility designs became more definitive, the  total amount of pipe increased by close to 33 percent over the  previous estimate. In addition, according to NNSA, the number of  safety systems needed to meet Nuclear Regulatory Commission  (NRC) requirements was greater than anticipated, further adding to  equipment installation costs. According to NNSA officials, NNSA  and the contractor did not have a good understanding of the cost of  designing the facility to meet NRC requirements related to  demonstrating the ability to withstand an earthquake. The officials  explained that the facility’s design is based on a similar facility in  France but that NRC regulatory requirements differ from those in  France.

The contractor’s difficulty identifying suppliers and subcontractors able  to fabricate and install equipment meeting nuclear quality assurance  criteria.project was experiencing the same issues identifying qualified  suppliers and subcontractors as other nuclear projects across DOE.  These issues included a higher than expected effort associated with  attracting qualified vendors and, after vendors were selected,  responding to questions or correcting noncompliance with  requirements. For example, according to NNSA and the MOX  contractor, the contractor needed to station quality assurance  personnel at supplier and subcontractor locations to oversee  activities.

According to NNSA’s review of the MOX project in 2011, the

Greater-than-expected turnover of engineering and technical staff. In  particular, the project lost staff to other nuclear industry projects,  including projects in neighboring states, resulting in a nearly complete  turnover of construction management personnel over a period of  several years and the need to provide training to replacement  personnel. NNSA identified this driver in its budget request for fiscal  year 2012. Specifically, the budget request stated that over 15 percent  of the project’s engineering and technical personnel had left for other  nuclear industry jobs in the previous year with pay increases of at  least 25 percent. The budget request further stated that finding  experienced replacements had become difficult and expensive.  According to the budget requests for fiscal years 2013 and 2014, the  loss of experienced engineering and technical staff to other nuclear  industry projects has continued.

Change in scope of the project to add capability to the MOX facility to  produce plutonium feedstock. As part of its decision to cancel plans  for a stand-alone PDCF and to instead meet feedstock requirements  through existing facilities, NNSA directed the MOX contractor to  include feedstock capability in its September 2012 proposal to  increase the cost of the facility. The contractor’s proposal included an  estimate of $262.3 million to add feedstock capability.

In identifying these drivers of the cost increase for the MOX facility, NNSA  did not identify the dollar amount associated with each cost driver. An  NNSA official said that the MOX contractor’s system for tracking and  reporting on cost and schedule performance could potentially be used to  determine dollar amounts that each driver added to the overall cost  increase—which is one possible use of such a system—but that doing so  would be time-consuming and difficult. As a result, NNSA officials could  not substantiate the relative importance of the cost drivers. For example,  NNSA officials said they had not conducted a formal analysis to back up  an estimate, which they had made when we first discussed the cost  drivers with them, that lack of design maturity of critical system  components accounted for more than half of the increase. In reviewing  the MOX contractor’s system, we found that, as NNSA officials stated,  using the system to determine the dollar amounts each driver added to  the cost increase would be difficult—for example, because the system’s  identification of cost increases at a summary level, such as site  construction support, did not correspond to the cost drivers identified by  NNSA.

Key cost drivers NNSA identified for the WSB included the following:

Higher-than-anticipated bids for the construction subcontract.

According to the NNSA federal project director for the WSB, the WSB  contractor received two bids in 2009 from prospective construction  subcontractors that both came in at about $26 million higher than the  contractor’s estimate. NNSA officials did not explain the reason for the  difference, stating that the bidders were not required to provide details  of their estimates. The federal project director said that NNSA  supported the WSB contractor awarding the construction subcontract,  despite the higher cost, in order to maintain the schedule for  completing the WSB in time to support the start-up of the MOX facility.  According to NNSA officials, the project applied cost savings from  earlier work to cover part of the increased cost of the construction  subcontract and had sufficient contingency—the portion of a project’s  budget that is available to account for uncertainties in the project’s  scope—to absorb the remainder of the increase. Consequently,  however, contingency to absorb further cost increases as construction  progressed was reduced.

Design errors, omissions, and inconsistencies. According to the  NNSA federal project director, the WSB contractor and subcontractor  made hundreds of design changes, which led to an additional cost  increase in the construction subcontract. According to NNSA’s log of  design changes, as of August 2013, design changes increased the  cost of the construction subcontract by about $15 million, from $91.5  million to $106.5 million. The federal project director said that, unlike  the design of the MOX facility, the design of the WSB was about 90  percent complete at the start of construction. A September 2008  report of NNSA’s independent review of the WSB prior to approval of  the cost and schedule estimate found that the design was essentially  complete. Nevertheless, according to the federal project director,  design changes were needed because of constructability issues, such  as equipment that met specifications in design documents not being  available by the time the project reached construction.

Schedule delays resulting from the construction subcontractor not  meeting required targets. According to the NNSA federal project  director’s feedback on the WSB contractor’s performance in  September 2009, NNSA had concerns related to the project schedule  and the ability to meet the completion date in part because of a  delayed start in the construction subcontract. By the time NNSA  approved the cost increase for the WSB in December 2012, schedule  delays in the construction subcontract had grown to 15 months. The  approved cost increase included about $30 million in the contractor’s  delay-related costs because NNSA’s contract for the WSB is cost- reimbursable.

The actual cost attributable to the WSB may be even higher depending  on the outcome of a lawsuit filed by the subcontractor against the WSB  contractor related to design changes and schedule delays that increased  the subcontractor’s costs in excess of the amount specified in its fixed- price subcontract. The approved cost increase for the WSB included  contingency to account for the possibility of higher costs incurred by the  construction subcontractor.

section_title: NNSA Has Not Analyzed Root Causes of the Cost Increases for the Plutonium Disposition Program’s Construction Projects
paragraphs: NNSA has not analyzed the underlying, or root, causes of the close to $3  billion in construction cost increases for the MOX facility and WSB.  DOE’s project management order requires that lessons learned be  captured throughout a project to allow for the exchange of information  within DOE in the context of project management and to benefit future  endeavors. However, the project management order does not include a  requirement for a root cause analysis of projects experiencing significant  cost increases or schedule delays. NNSA officials said that they decide  on a case-by-case basis whether to conduct a root cause analysis. In  contrast, under the Weapon Systems Acquisition Reform Act of 2009, the  Department of Defense must perform a root cause analysis of a cost  increase that exceeds a certain threshold.

Documentation NNSA provided to us on the cost drivers for the MOX  facility and WSB do not provide clear details about the causes of the cost  increases. Such details can be found in a root cause analysis, which  would help address questions about why the drivers identified by NNSA  occurred and help inform lessons learned. Key questions about the cause  of the key drivers include the following:

DOE’s reasons for approving a cost and schedule estimate for the  MOX facility before the design was complete, even though a July  2006 review of the project found that the cost estimate’s basis on  portions of the design that were less than 50 percent complete posed  a risk to the project. Similarly, a root cause analysis would address  why one of the drivers of the cost increase for the WSB identified by  NNSA was design errors, omissions, and inconsistencies, given that a  review prior to approval of the project’s cost and schedule estimate  found that most of the design was ready for construction.

The extent to which NNSA and its contractors shared responsibility for  cost drivers, such as the greater-than-anticipated number of safety  systems needed in the MOX facility to meet NRC requirements.  According to NNSA officials, the department hired the MOX contractor  because it considered the contractor to be well-qualified to engineer  and estimate all of the safety systems for the facility, taking into  account NRC requirements. However, the record for DOE’s approval  of the cost and schedule estimate for the facility shows that DOE was  aware of complexities in adapting MOX technology to comply with  NRC requirements. Specifically, the minutes from DOE’s July 2006  meeting to request approval of the estimate stated that these  complexities had already contributed to a $1.1 billion increase in the  estimated cost.

The sufficiency of measures DOE took to ensure that the cost  estimate for the MOX facility it approved in 2007 reflected an  awareness of market conditions, such as the availability of suppliers  and subcontractors with the ability and experience to meet nuclear  quality assurance criteria. As required under the MOX contract, in  October 2006—before DOE approved the cost and schedule estimate  for the facility—the contractor submitted a construction market  analysis report, which stated that the contractor had experienced  trouble obtaining qualified suppliers and that the subcontractor pool  using nuclear quality standards had been decreasing due to inactivity  in the nuclear industry. However, the report provided limited detail and  did not include recommendations to address availability of qualified  suppliers.

The thoroughness of DOE’s review, required under DOE’s project  management order, to ensure that the WSB contractor’s system for  tracking and reporting on cost and schedule performance provided  accurate information. DOE recertified the contractor’s system in  December 2011 after identifying and closing out several corrective  actions and continuous improvement opportunities. However, DOE  found additional problems with the system after January 2012, when  the WSB contractor informed NNSA that schedule delays for the  project were greater than the contractor previously revealed. Based in  part on the contractor’s revelations, DOE reexamined the contractor’s  system and suspended its certification in November 2012.

The corrective actions NNSA and its contractors took after periodic  project reviews identified problems, including problems cited by NNSA  as drivers of cost increases for the MOX facility and WSB. For  example, multiple reviews of the MOX facility found that costs to  install equipment were underestimated. A July 2006 review found that  installation for electrical; piping; and heating, ventilation, and air- conditioning equipment were underestimated by close to $160 million  and nearly 3 million labor hours. NNSA’s project reviews of the facility  in 2011 and 2012 continued to raise concerns about unrealistic  installation rates.

The responsiveness of NNSA project managers to emerging cost and  schedule issues. Without a review of the timing of NNSA initiating the  process of increasing the projects’ cost and schedule estimates, it is  not clear whether NNSA acted in a timely manner or whether project  cost and schedule indicators warranted earlier action. For example,  an NNSA review of the MOX facility in the spring of 2011 found that  the most significant risk to delivering the project within cost centered  on the ability of the project team to identify about $364 million in  savings to offset expected cost growth, but NNSA did not initiate the  process of increasing the project’s cost and schedule estimates until  January 2012.

Without a root cause analysis, it is uncertain whether NNSA will be able  to accurately identify underlying causes of the cost increases for the MOX  facility and WSB in order to identify and implement corrective measures  and identify lessons learned to share with and apply to other DOE  construction projects.

section_title: NNSA Has Taken Steps to Hold Contractors Accountable for Cost Increases by Withholding Fees
paragraphs: After determining that the performance of the contractors for the MOX  facility and WSB contributed to the projects’ construction cost increases,  NNSA took steps to hold the contractors accountable for their  performance by withholding fees specified under the contracts.  Specifically, NNSA withheld portions of two of the four types of the MOX  contractor’s fees and 41 percent of the WSB contractor’s fees.
subsections: section_title: NNSA Withheld Portions of Two of the Four Types of the MOX Contractor’s Fees
paragraphs: NNSA withheld portions of two of the four types of fees that the MOX  contractor could earn under the contract for construction of the facility— incentive fees and award fees. In total, NNSA withheld $45.1 million or  close to one-third of all fees the contractor could earn as of November  2013. Under the terms of the MOX contract, the contractor could still earn  incentive fees that have been withheld, but only if it completes the overall  project within cost and schedule. Table 1 summarizes fees paid to and  withheld from the contractor as of November 2013.

Details of fees NNSA withheld and paid under the MOX contract include  the following:  Incentive fees. NNSA did not pay $36.5 million or over half of the  $65.6 million in incentive fees that the MOX contractor could earn  from fiscal year 2008, when construction began, through fiscal year  2013. Of the $29.1 million in incentive fees paid to the contractor,  $21.6 million remains provisional, meaning that NNSA can require  that the fees be paid back as a result of the project not being  completed within cost. The amount not paid represented the  contractor’s entire incentive fees for fiscal years 2011 through 2013.  Specifically, under the terms of the MOX contract, NNSA can withhold  quarterly payments of incentive fees if an increase in the projected  cost to complete the MOX facility exceeds $200 million. NNSA began  withholding incentive fees for the first quarter of fiscal year 2011  when, for the first time, the increase in the projected cost to complete  the facility exceeded this threshold. NNSA memos for subsequent  quarters in fiscal year 2011 noted that the project’s cost and schedule  metrics continued to worsen, reducing the likelihood of resumption of  payments. In a July 2011 letter to the contractor explaining its  rationale for not resuming payments, NNSA stated that it was  sensitive to the potential impacts of the “nuclear renaissance”—the  contractor’s term for the resurgence of U.S. nuclear engineering and  manufacturing capability after being dormant for more than 20 years,  which the contractor stated limited the availability of qualified suppliers  and subcontractors and led to staff turnover and higher-than- anticipated costs to install equipment. However, NNSA stated that  such impacts would not necessarily overcome other evidence  showing that the contractor was not meeting the overarching goal of  the incentive fees, which is that the facility be completed within cost.

Award fees. NNSA withheld $8.6 million or about a quarter of the  $32.6 million in award fees that the MOX contractor could earn from  fiscal year 2008 through fiscal year 2012. The amount withheld  included about half of the fees the contractor was eligible to earn in  fiscal year 2012. NNSA’s award fee evaluation for fiscal year 2012  cited various factors, such as poor construction planning; less than  optimal coordination of work; and overly conservative specifications  for installation of fire doors, resulting in delays and unnecessary costs.  In contrast, NNSA paid $24.0 million in award fees for performance in  other areas, such as maintaining a high level of worker safety—an  area in which the contractor has consistently performed well,  according to NNSA’s award fee evaluations.

Milestone fees. NNSA did not withhold any milestone fees and instead  paid milestone fees of $30.8 million for tasks with deadlines ranging  from February 2009 to March 2014. Examples of tasks for which  NNSA paid milestone fees (some of which the MOX contractor  completed early) included completing the roof, installing the first  glovebox, constructing a technical support building, and completing a  start-up plan for the facility. According to the NNSA officials, although  NNSA did not withhold milestone fees, NNSA stopped paying any of  the $30.2 million in remaining milestones fees as part of an  understanding with the contractor to renegotiate the amount of and  conditions for earning milestone fees.

Fixed fees. According to the contracting officer, NNSA did not  withhold any of the $15.7 million in fixed fees—the total amount of  fixed fees for construction-related work under the MOX contract.  NNSA included these fees in the contract to reward the contractor for  work performed during contract negotiations, when other fees had not  yet been negotiated.

section_title: NNSA Withheld 41 Percent of the WSB Contractor’s Fees through Fiscal Year 2012
paragraphs: In a March 2013 analysis of the WSB contractor’s performance, the  NNSA contracting officer for the WSB recommended that the contractor  should be held accountable for performance failures that contributed to  the project’s cost increase. For example, the analysis stated that the  contractor did not require the subcontractor to add crews or take other  steps to correct delays until almost 2 years after the federal project  director began expressing concerns about the delays. In accordance with  this assessment, NNSA withheld $7.7 million or about 40 percent of the  $18.9 million in performance incentive fees that the WSB contractor could  earn from fiscal year 2009, when construction began, through fiscal year  2012, for the portion of fees allocated to construction of the WSB under  the management and operation contract for the Savannah River Site.  Most of the fees withheld were for the contractor’s performance in fiscal  years 2011 and 2012 (see table 2). In particular, NNSA withheld $3.3  million of the $6.9 million in fees the contractor could earn in fiscal year  2011 and $3.9 million of the $4.0 million in fees the contractor could earn  in fiscal year 2012. The fees withheld were tied to various performance  measures, which DOE acquisition regulations require be established prior  to the start of each evaluation period. Performance measures NNSA  established included meeting the schedule for testing various types of  equipment, providing engineering support to and coordinating with the  construction subcontractor, and maintaining the project within pre- established cost and schedule metrics.

The $3.3 million in fees withheld for fiscal year 2011 included $2 million  that NNSA took back—that is, was paid back by the contractor—after  making its fee determination for the contractor. Specifically, according to  a December 2012 letter from the NNSA contracting officer to the  contractor, the fiscal year 2011 fee determination was premised on the  contractor’s statements that schedule delays were recoverable and that  the project would be completed within the approved cost estimate. Shortly  after NNSA made its fee determination, however, the contractor notified  NNSA that the project was further behind schedule than previously  represented and that cost factors not included in the contractor’s system  for tracking and reporting on cost and schedule performance would result  in a cost overrun. The contracting officer’s letter stated that NNSA would  have reduced the contractor’s fee if it had known the extent of delays and  cost overruns when it made its fee determination, and NNSA required the  contractor to repay $4 million. In May 2013, NNSA agreed to a settlement  with the contractor to reduce the amount taken back to $2 million after the  contractor appealed NNSA’s initial demand.

In addition to withholding fees, in a June 2012 letter to the contractor,  NNSA’s contracting officer questioned why she should not conclude that  the contractor’s actions rose to the level of gross negligence or willful  misconduct, warranting disallowance of costs, meaning that the  contractor would bear part of the cost increase resulting from the project’s  schedule delays. For example, the letter stated that the contractor’s  system for tracking and reporting on cost and schedule performance did  not meet industry standards and impeded NNSA’s ability to understand  the potential impact of delays in construction of various segments of the  project on the final delivery date. According to NNSA officials, NNSA is  waiting until after completion of WSB construction, and total construction  costs are known, to determine unallowable costs.

section_title: Estimates for the Plutonium Disposition Program’s Life-cycle Cost and Construction Projects Did Not Meet All Best Practices for Reliability
paragraphs: NNSA’s most recent cost and schedule estimates for the Plutonium  Disposition program did not fully reflect the characteristics of high-quality,  reliable estimates as established by best practices used throughout  government and industry and documented in the GAO Cost Estimating  and Assessment Guide and GAO Schedule Assessment Guide.  Specifically, (1) NNSA’s draft April 2013 life-cycle cost estimate for the  overall program was partially comprehensive, partially well-documented,  and partially accurate but did not meet any of the best practices for a  credible estimate; (2) the MOX contractor’s September 2012 proposal for  increasing the cost of the MOX facility was substantially comprehensive  but partially well-documented and accurate and minimally credible; and  (3) the WSB contractor’s February 2013 monthly update to its schedule  estimate was minimally well-constructed and partially met the other three  characteristics of a reliable schedule—comprehensive, credible, and  controlled.
subsections: section_title: NNSA Did Not Follow All Key Steps for Developing a Reliable Life-cycle Cost Estimate for the Plutonium Disposition Program
paragraphs: In developing its draft April 2013 life-cycle cost estimate of $24.2 billion  for the Plutonium Disposition program, NNSA followed several of the 12  key steps for developing high-quality cost estimates, including defining  the estimate’s purpose, defining the program’s characteristics, and  obtaining the data. NNSA did not follow other key steps, however, such  as conducting an independent cost estimate. As a result, the estimate  was not reliable. In particular, NNSA’s draft life-cycle cost was partially  comprehensive, partially well-documented, and partially accurate but did  not meet any of the best practices for a credible estimate.

Table 3 summarizes the major components of NNSA’s draft April 2013  life-cycle cost estimate. The estimate assumed that the MOX facility  would start operations in November 2019 and that it would take  approximately 15 years to complete the mission to dispose of 34 metric  tons of surplus weapons-grade plutonium.

Table 4 lists the steps, or best practices, necessary for developing a high- quality cost estimate. Appendix II summarizes our assessment of NNSA’s  process for developing its draft life-cycle cost estimate against the steps  that should result in the four characteristics of a high-quality cost  estimate.

Our assessment of NNSA’s process for developing its draft life-cycle cost  estimate included the following observations:

Comprehensive. The draft life-cycle cost estimate was partially  comprehensive because work breakdown structures were developed  for the MOX and WSB projects and other components of the program,  but NNSA had not formalized a program-level work breakdown  structure. A typical work breakdown structure provides a clear picture  of what needs to be accomplished, how the work will be done, and a  basis for identifying resources and tasks for developing a cost  estimate. Without a program-level work breakdown structure, NNSA  cannot ensure that its life-cycle cost estimate captures all relevant  costs, which can mean cost overruns.

Well-documented. The draft life-cycle cost estimate was partially well- documented because NNSA defined the estimate’s purpose and the  program’s characteristics, but it did not develop a single document to  describe data sources and steps taken in developing the estimate— such as applying escalation rates to account for inflation—so that the  estimate could be replicated by someone other than those who  prepared it. In addition, NNSA stated that a document identified the  estimate’s ground rules and assumptions but that the assumptions  have changed frequently, hindering development of a life-cycle cost  estimate. Examples of changes in assumptions not reflected in  NNSA’s draft April 2013 estimate included the slowdown of activities  during the assessment of alternative plutonium disposition strategies  and NNSA’s plans to increase the amount of plutonium disposed of as  MOX fuel.

Accurate. The draft life-cycle cost estimate was partially accurate in  that NNSA followed the best practice for developing a point  estimate—a best guess at a cost estimate usually falling between best  and worst case extremes. NNSA also updated the estimate  periodically to include actual costs and changes to program and  project requirements. However, NNSA did not use a formal system for  tracking and reporting on cost and schedule performance to update  the estimate, limiting the ability of someone other than those who  prepared the estimate to check the estimate’s accuracy and to identify  when, how much, and why the program cost more or less than  planned.

Credible. The draft life-cycle cost estimate was not credible because  NNSA did not conduct an independent cost estimate to provide an  unbiased test of whether its estimate was reasonable, a formal  sensitivity analysis to examine the effects of changing assumptions  and ground rules, or a risk and uncertainty analysis to assess  variability in point estimates due to factors such as errors and cost  estimators’ inexperience or biases. NNSA conducted such analyses  for portions of its life-cycle cost estimate, but not for the entire  estimate. For example, NNSA’s Plutonium Disposition program office  arranged for another office within NNSA to conduct an independent  assessment of the MOX facility’s operations costs, but not for the  program’s entire life-cycle cost.

NNSA did not follow all key steps for developing high-quality cost  estimates in part because it did not have a requirement to develop its life- cycle cost estimate. According to NNSA officials, DOE’s project  management order includes requirements for development of cost and  schedule estimates for a project, such as the MOX facility or WSB, but  does not specify equivalent requirements for a program like Plutonium  Disposition, which includes multiple projects, as well as supporting  activities. As a result, when developing the life-cycle cost estimate for the  Plutonium Disposition program, NNSA officials used an ad hoc approach  to adapt requirements for managing projects in DOE’s project  management order. NNSA officials also said that its April 2013 life-cycle  cost estimate did not include all the steps of a high-quality, reliable  estimate in part because NNSA considered the estimate to be draft and,  therefore, had not fully implemented plans for developing it.

In the absence of a specific requirement in DOE’s project management  order for developing a life-cycle cost estimate for a program, NNSA  officials said they developed a life-cycle cost estimate for the Plutonium  Disposition program for several reasons. According to these officials,  these reasons included that the cost of the program is largely made up of  capital projects, such as the MOX facility, and that requirements for  congressional budget submissions specify that the full life-cycle cost of  such projects be presented. In addition, each year NNSA must submit to  Congress its estimated expenditures covering the fiscal year with respect  to which the budget is submitted and at least the four succeeding fiscal  years. NNSA officials said that, to accurately estimate expenditures for  this 5-year period, they needed to develop a life-cycle cost estimate for  the overall Plutonium Disposition program. An NNSA official noted that  NNSA plans to use a version of its life-cycle cost estimate as a basis for  evaluating alternative strategies to dispose of surplus weapons-grade  plutonium.

section_title: Contractor’s Proposed Estimate for the MOX Facility Did Not Meet Most Best Practices for Reliability
paragraphs: The MOX contractor’s September 2012 proposal for increasing the cost of  the MOX facility was substantially comprehensive but was partially well- documented, partially accurate, and minimally credible. The contractor’s  estimate did not fully reflect the characteristics of a high-quality, reliable  estimate in part because it was a proposal, as opposed to an approved  cost estimate. For example, one of the best practices for a well- documented estimate—and a requirement of DOE’s project management  order—is that a cost estimate be reviewed and accepted by management.  Because DOE had not approved it and instead postponed its review and  approval pending the outcome of NNSA’s assessment of alternative  plutonium disposition strategies, the contractor’s estimate partially met  this best practice. This best practice would be met by DOE’s completion  of its review and approval of a new estimate for the MOX facility,  assuming the assessment of alternative plutonium disposition strategies  maintains the current strategy of disposing plutonium as MOX fuel.

Though the contractor’s September 2012 estimate did not fully reflect the  characteristics of a high-quality estimate and cannot be considered  reliable, the MOX contractor began using it as a provisional baseline for  purposes of monthly reporting on the project’s cost and schedule  performance. Specifically, as directed by NNSA, the contractor began a  transition in June 2012 to report its monthly performance against the  contractor’s proposed estimate of $7.7 billion. The contractor completed  the transition and ceased any reporting of performance against the  previously approved estimate early in 2013. Managing projects that no  longer have an approved cost and schedule estimate is a challenge  because cost and schedule estimates provide a baseline for measuring  progress. At a July 2013 hearing, the Deputy Secretary of Energy noted  that not having such a baseline is the point of maximum risk of  unrestricted cost growth on a project.

Appendix III summarizes our assessment of how well the MOX  contractor’s proposal met the characteristics of a high-quality estimate.  Our assessment included the following observations:

Comprehensive. The proposal was substantially comprehensive in  that it included all construction costs, as defined by the statement of  work under the MOX contract. The proposal was not fully  comprehensive, however, because it only partially met certain best  practices for a comprehensive estimate, such as documenting all  cost-influencing ground rules and assumptions. The proposal partially  met this best practice because it did not provide justifications for some  assumptions, such as not more than 10 percent of the supports for  piping systems being nonstandard and requiring unique designs.

Well-documented. The proposal was partially well-documented  because it described in sufficient detail the calculations performed and  the estimating methodology used to derive the cost of each element in  the work breakdown structure. However, it did not provide all types of  information specified in best practices for a well-documented  estimate, such as how data on labor and travel costs were  normalized. Data normalization is often necessary to ensure  comparability because data can be gathered from a variety of sources  and in different forms that need to be adjusted before being used.

Accurate. The proposal was partially accurate in that it appeared to  adjust cost elements for inflation and contained only a few minor  mistakes, but the contractor did not update its proposal with actual  costs incurred after it developed the proposal and submitted it to  NNSA in September 2012. NNSA and contractor officials agreed that  the estimate was no longer an accurate reflection of the cost to  complete construction—for example, because the proposal assumed  a higher level of funding than the project received in fiscal year 2013.  The officials said that, if the MOX project continues, the contractor  would need to prepare a new proposal that includes costs for work  conducted after the initial proposal was developed.

Credible. The proposal was minimally credible because DOE halted  its independent cost estimate of the proposal pending the outcome of  NNSA’s assessment of alternative plutonium disposition strategies.  Moreover, the proposal did not include a formal sensitivity analysis to  examine the effects of changing assumptions and ground rules, and it  provided no evidence that major cost elements were cross-checked to  determine whether alternative cost-estimating methods produced  similar results. Finally, the proposal included an analysis of risks, such  as difficulty attracting and retaining workers, and uncertainty in  estimating materials and other costs. On the basis of this analysis, the  proposal included a total of $713.1 million to account for risks and  uncertainty—$641.4 million for the original scope of the MOX facility  and $71.7 million for the addition of plutonium feedstock capability  (see table 5). However, the contractor did not properly conduct or  clearly document all steps in the analysis to determine the amount of  funding to account for risks and uncertainty that could increase the  cost of the project.

section_title: Contractor’s Schedule Estimate for the WSB Did Not Meet Most Best Practices for Reliability
paragraphs: The WSB contractor’s February 2013 monthly update to its schedule  estimate did not fully reflect the characteristics of a high-quality, reliable  schedule estimate as established by best practices. Specifically, the  contractor’s schedule estimate was minimally well-constructed and  partially met the other three characteristics of a reliable, high-quality  schedule as measured against best practices—comprehensive, credible,  and controlled. Table 6 shows the characteristics of a high-quality  schedule estimate and corresponding best practices. Appendix IV  summarizes our assessment of how well the WSB contractor’s February  2013 schedule estimate met the characteristics of a high-quality estimate.

Our assessment of the WSB contractor’s February 2013 schedule  estimate included the following observations:

Comprehensive. The estimate was partially comprehensive in that it  captured and established the durations of contractor and government  activities to complete the project but did not capture the remaining  detailed work to be performed by the construction subcontractor.  Specifically, it reduced the subcontractor’s 3,851 activities to complete  its portion of the work to one placeholder activity. According to the  NNSA federal project director, the WSB contractor reduced the  subcontractor’s activities to a placeholder because the subcontractor  submitted unreliable schedules with repeated changes in the  estimated completion date for its portion of work.

Well-constructed. The estimate was minimally well-constructed in that  it sequenced activities in ways that can obscure a schedule’s earliest  completion date. In addition, the sequencing of activities included  “merge points”—the convergence of many parallel activities into a  single successor activity, which decreased the probability of  successor activities starting on time. For example, performance of an  assessment of readiness to operate the WSB was preceded by 212  activities. NNSA officials explained that the merge points resulted  from the need to complete activities in parallel to meet requirements  set forth in DOE’s project management order.

Credible. The estimate was partially credible in that the WSB  contractor conducted a schedule risk analysis to determine the  amount of schedule contingency—a reserve of extra time to account  for risks and ensure completion of the project on time. However, a  DOE review conducted prior to approval of an increase in the project’s  cost and a delay in the start of operations found that the results of the  contractor’s analysis were unreliable—for example, because project  team members were not consulted regarding risk inputs. As a result,  the schedule risk analysis did not clearly support the 12 months of  schedule contingency included in the approved cost increase and  schedule delay.

Controlled. The estimate was partially controlled in that, according to  project officials, the schedule was updated weekly and used to  measure performance, but no narrative accompanied weekly updates  to provide decision makers with a log of changes and their effect, if  any, on the schedule time frame. In addition, project officials did not  provide documentation enabling the schedule to be validated, such as  documentation describing sequencing of activities or assumptions  used in developing the schedule.

The NNSA federal project director and contractor’s project leader said  that the contractor had begun to correct problems in the contractor’s  schedule estimate—for example, by replacing the placeholder for the  subcontractor’s activities with a schedule of more detailed activities  independently developed by the contractor. However, delays on the  project continued after the contractor began correcting the problems.  Notably, according to DOE’s October 2013 monthly report on the WSB,  continuing delays in completion of the construction subcontract—one of  the key drivers NNSA identified for the WSB cost increase—already used  up about 10 of the 12 months of schedule contingency, placing the  project’s completion date in jeopardy.

section_title: Conclusions
paragraphs: NNSA has identified drivers of the close to $3 billion increase in the  projected cost to complete the MOX facility and WSB and has taken steps  to hold the MOX and WSB contractors accountable for their role in the  cost increases by withholding and taking back fees. However, the various  drivers identified by NNSA, such as DOE’s approval of the cost and  schedule estimate for the MOX facility before design was complete, do  not provide the level of detail that can be found in a root cause analysis.  In addition, DOE’s project management order requires that lessons  learned be captured throughout a project but does not include a  requirement for a root cause analysis when a project exceeds its cost  estimate, even when a project exceeds its cost estimate by billions of  dollars. The decision whether to conduct such an analysis is instead  made on a case-by-case basis. Because NNSA has not conducted a root  cause analysis to identify the underlying causes of the cost increases for  the MOX facility and WSB, it cannot provide assurance that it has  correctly identified the underlying causes to ensure that they will not lead  to further cost increases as the projects move forward. Further, without a  root cause analysis, NNSA’s ability to identify recommended solutions  and lessons learned that could be applied to other projects is lessened.  Conducting a root cause analysis of the cost increases for the MOX  facility and WSB could help NNSA address its long-standing difficulties in  completing projects within cost and on schedule, which has led to NNSA’s  project management remaining on GAO’s list of areas at high risk of  fraud, waste, abuse, and mismanagement.

NNSA has drafted a life-cycle cost estimate of $24.2 billion for the  Plutonium Disposition program—an important step toward presenting the  full cost of NNSA’s current strategy to dispose of surplus weapons-grade  plutonium as MOX fuel. A cost estimate that presents the full cost of  NNSA’s current plutonium disposition strategy is essential to inform  NNSA’s ongoing evaluation of alternative plutonium disposition strategies  and provide Congress with a complete picture of the cost of the program.  NNSA developed its life-cycle cost estimate even though neither DOE nor  NNSA required the estimate. In particular, DOE’s project management  order does not explicitly require that life-cycle cost estimates be  developed for programs like the Plutonium Disposition program that  include both construction projects and other efforts and activities not  related to construction, such as producing plutonium feedstock for the  MOX facility. In the absence of such a requirement, NNSA followed  several of the 12 key steps described in the GAO Cost Estimating and  Assessment Guide for developing high-quality, reliable cost estimates,  but it did not follow other key steps. Because NNSA did not follow all of  the steps, the life-cycle estimate for the Plutonium Disposition program is  not reliable. Similarly, the contractors’ cost and schedule estimates for the  MOX facility and WSB did not meet all best practices compiled in GAO’s  guides for preparing high-quality, reliable cost and schedule estimates.  Not meeting these best practices increased the risk of further cost  increases and delays for the projects and, because the projects are  components of NNSA’s life-cycle cost estimate, for the overall Plutonium  Disposition program.

section_title: Recommendations for Executive Action
paragraphs: We are making six recommendations in this report to the Secretary of  Energy. To identify lessons learned from and provide assurance of  preventing recurrence of cost increases for the MOX facility and WSB,  and to develop reliable cost estimates for the Plutonium Disposition  program, we recommend that the Secretary of Energy direct the DOE and  NNSA Offices of Acquisition and Project Management and the NNSA  office responsible for managing the Plutonium Disposition program, as  appropriate, to take the following four actions:

Conduct an analysis of the root causes of the cost increases for the  MOX facility and WSB, such as the causes of the design changes that  led to cost increases, and identify and prioritize recommended  solutions.

Revise and update the program’s life-cycle cost estimate following the  12 key steps described in the GAO Cost Estimating and Assessment  Guide for developing high-quality cost estimates, such as conducting  an independent cost estimate to provide an objective and unbiased  assessment of whether the estimate can be achieved.

Ensure that the MOX contractor revises its proposal for increasing the  cost of the MOX facility to meet all best practices for a high-quality,  reliable cost estimate—for example, by cross-checking major cost  elements to determine whether alternative estimating methods  produce similar results.

Ensure that the approved cost increase for the WSB is based on a  schedule that the contractor has revised to meet all best practices for  a high-quality, reliable schedule estimate, such as reflecting all  activities (both government and contractor) needed to complete  construction.

To ensure that future DOE projects benefit from lessons learned that  reflect the underlying causes of cost increases or schedule delays  experienced by other projects, and that Congress and DOE have life- cycle cost estimates for DOE programs that include individual  construction projects, we further recommend that the Secretary of Energy  take the following two actions to revise DOE’s project management order  or otherwise implement a departmentwide requirement:

Require a root cause analysis of all projects that experience cost  increases or schedule delays exceeding a certain threshold  established by DOE.

Require life-cycle cost estimates covering the full cost of programs  that include both construction projects and other efforts and activities  not related to construction.

section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this product to DOE for comment. In written  comments, reproduced in appendix V, NNSA stated that the agency and  DOE generally agreed with our recommendations. In particular, NNSA  concurred with four of our six recommendations and partially concurred  with the other two. NNSA described actions it planned to take to  implement the recommendations with which it concurred and time frames  for taking these actions. NNSA also provided technical comments, which  we incorporated into the report as appropriate.

We are pleased that NNSA concurred with our recommendation to  conduct an analysis of the root causes of the cost increases for the MOX  facility and WSB and stated that it is planning to conduct such an  analysis, which was not mentioned during the course of our review.  NNSA also concurred with our recommendation to revise and update the  Plutonium Disposition program’s life-cycle cost estimate and stated that it  would do so after a decision was made on the path forward for the  program. The path forward could involve the use of alternative strategies  to dispose of surplus weapons-grade plutonium.

NNSA also concurred with our recommendation to ensure that the MOX  contractor revises its proposal for increasing the cost of the MOX facility  to meet all best practices for a high-quality, reliable cost estimate. In its  comment letter, NNSA stated that it is working with the contractor to  ensure that the cost estimating processes and procedures are updated  such that the best practices are met.

In addition, NNSA concurred with our recommendation to ensure that the  approved cost increase for the WSB is based on a schedule that the  contractor has revised to meet all best practices for a high-quality, reliable  schedule estimate. In its comment letter, NNSA stated that it has revised  the schedule since we reviewed it and that it now reflects all activities  needed to complete construction. We did not review the update to the  WSB contractor’s schedule to confirm that it captured all activities to  complete construction, which is one of the best practices associated with  the characteristics of a high-quality schedule. Moreover, as detailed in  appendix IV, the schedule we reviewed only partially or minimally met 7 of  the other 9 best practices. To fully implement our recommendation, NNSA  would need to ensure that the contractor has revised its schedule to meet  all best practices for a high-quality, reliable schedule estimate. In its  comment letter, NNSA stated that during the next project review, which is  expected to occur by December 31, 2014, NNSA will review the schedule  against best practices.

NNSA partially concurred with our fifth recommendation that DOE require  a root cause analysis of all projects that experience cost increases or  schedule delays exceeding a certain threshold established by the  department. In its comment letter, NNSA stated that DOE program offices  currently perform tailored root cause analyses as part of the baseline  change proposal process outlined in the department’s project  management order for increasing a project’s cost and schedule  estimates. NNSA stated that, as a result, the department does not believe  that an update to the project management order is required. NNSA further  stated that the department will review the lessons learned from NNSA's  root cause analyses for the MOX and WSB projects to see what best  practices may be of benefit to other projects. However, as we stated in  the report, DOE’s project management order does not include a  requirement for a root cause analysis of projects experiencing significant  cost increases or schedule delays, and NNSA officials said that they  decide on a case-by-case basis whether to conduct a root cause analysis.  Moreover, the order does not define what a root cause analysis is, how or  when a root cause analysis should be conducted, or what is meant by a  tailored analysis. In addition, NNSA’s written comments did not provide  information on the conditions that would trigger a root cause analysis.  Leaving root cause analyses to an informal and undefined process within  DOE program offices could result in such analyses not being conducted,  not being conducted consistently, or not accurately identifying underlying  causes of cost increases in order to identify and implement corrective  measures and apply lessons learned to other DOE projects. We continue  to believe that a root cause analysis should be conducted for all projects  that experience cost increases or schedule delays above a threshold  established by the department. We note that our recommendation is  consistent with a requirement in the Weapon Systems Acquisition Reform  Act of 2009; under the act, the Department of Defense must perform a  root cause analysis of a cost increase that exceeds a certain threshold.

NNSA partially concurred with our final recommendation that DOE require  life-cycle cost estimates covering the full cost of programs that include  both construction projects and other efforts and activities not related to  construction. In its comment letter, NNSA stated that the department’s  project management order requires a comprehensive life-cycle cost  analysis as part of the alternative selection process and that no further  update to the order is required to address this recommendation. The  intent of our recommendation goes beyond that of preparing a life-cycle  cost estimate at the stage of selecting an alternative for a new capital  asset project. Instead, the recommendation applies to departmental  programs that include capital asset projects to meet the overall program  need. As we stated in the report, NNSA did not follow all key steps for  developing high-quality cost estimates in developing its draft April 2013  life-cycle cost estimate for the Plutonium Disposition program, which  currently includes the MOX facility and WSB capital asset projects, in part  because it did not have a requirement to develop it. NNSA’s response to  our recommendation suggests that the life-cycle cost estimates for the  MOX and WSB projects that were required to be prepared years ago,  when the projects were selected from among other alternatives, are the  only life-cycle cost estimates needed to manage the Plutonium  Disposition program. Furthermore, NNSA’s response contradicts the fact  that it concurred with our recommendation to revise and update the life- cycle cost estimate for the overall Plutonium Disposition program in  accordance with cost estimating best practices. We continue to believe  that our recommendation that the department require life-cycle cost  estimates covering the full cost of programs that include construction  projects should be implemented.

As agreed with your offices, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 7 days from the  report date. At that time, we will send copies to the appropriate  congressional committees, the Secretary of Energy, the NNSA  Administrator, and other interested parties. In addition, the report will be  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made key contributions to  this report are listed in appendix VI.

section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: To assess drivers of the construction cost increases for the Mixed Oxide  (MOX) Fuel Fabrication Facility and Waste Solidification Building (WSB)  that the National Nuclear Security Administration (NNSA) identified, we  reviewed the Department of Energy’s (DOE) budget request for NNSA for  fiscal year 2014, which provided a summary of the cost drivers for both  projects. To assess cost drivers in further detail, we reviewed the MOX  contractor’s September 2012 proposal for increasing the project’s cost,  which discussed drivers from the contractor’s perspective. We also  reviewed DOE’s December 2012 document approving an increase in the  estimated cost of the WSB and a delay in the start of operations, which  summarized cost drivers and their impact on the project’s cost and  schedule. We visited the Savannah River Site to observe construction  progress for both projects and interviewed NNSA and contractor officials  responsible for managing the projects. We also interviewed officials from  the NNSA Office of Fissile Materials Disposition, the NNSA Office of  Acquisition and Project Management, and the DOE Office of Acquisition  and Project Management. Separately, to understand how, if at all, cost  drivers for the MOX facility were related to Nuclear Regulatory  Commission (NRC) regulation and licensing of the construction and  operation of the facility, we reviewed NRC construction inspection reports  and related documents, and we interviewed NRC officials responsible for  overseeing the facility’s construction. In order to understand the  components of cost growth for the MOX facility, which represented most  of the Plutonium Disposition program’s construction cost increase, we  also analyzed the MOX contractor’s earned value management (EVM)  system that the contractor used to track and report on cost and schedule  performance, including data from the EVM system on cumulative cost and  schedule variance trends from July 2011 through April 2012 and the  contractor’s variance report for April 2012.

To determine the extent to which NNSA analyzed underlying causes of  the cost increases, we reviewed documents providing context for cost  drivers. The documents we reviewed included NNSA Office of Acquisition  and Project Management project review reports and monthly status  reports; DOE Office of Acquisition and Project Management monthly  status reports; DOE documents related to approval of the previous cost  and schedule estimates for the MOX facility and WSB in April 2007 and  December 2008, respectively; and documents related to specific cost  drivers identified by NNSA, such as the MOX contractor’s October 2006  report on construction markets and DOE reports related to its suspension  of the WSB contractor’s system for tracking and reporting cost and  schedule performance in November 2012. We also interviewed NNSA  officials to determine the extent to which they had conducted or planned  any analyses to identify underlying causes of cost increases for the  Plutonium Disposition program’s construction projects.

To determine steps NNSA took to hold contractors accountable for their  role in the cost increases for the Plutonium Disposition program’s  construction projects, we reviewed the contracts for the MOX facility and  WSB, fees specified under the contracts, and NNSA’s fee evaluations  and other documentation supporting its fee determinations. We also  interviewed NNSA contracting officers who were responsible for  administering the MOX and WSB contracts regarding the terms of the  contracts, fees specified under the contracts, and actions NNSA took or  planned to take to hold contractors accountable for their role in the cost  increases. We obtained NNSA data on fees it paid to and withheld from  the contractors, and we assessed the reliability of the data by checking  for obvious errors in accuracy and completeness; comparing the data with  other sources of information, such as NNSA’s fee determinations; and  interviewing NNSA contracting officers who had knowledge of the data.  We determined that NNSA’s data on fees were sufficiently reliable for  reporting on the fees paid to and withheld from the contractors.

To assess the extent to which NNSA’s most recent estimates of the  Plutonium Disposition program’s life-cycle cost and of the cost and  schedule for completing the program’s construction projects met best  practices we have compiled in guides identifying the characteristics of  high-quality, reliable cost and schedule estimates, we tailored our  methodology to the differing stages of NNSA’s development and approval  of each estimate:

NNSA’s life-cycle cost estimate for the Plutonium Disposition  program. Because NNSA had not finalized a life-cycle cost estimate,  we assessed NNSA’s most recent available estimate—spreadsheets  dated April 2013 representing NNSA’s draft life-cycle cost estimate. In  particular, we assessed the process NNSA used to develop the  estimate against the 12 key steps described in the GAO Cost  Estimating and Assessment Guide that should result in a high-quality,  reliable cost estimate. To provide information on NNSA’s process,  NNSA officials responsible for developing the estimate filled out a  data collection instrument we developed. The data collection  instrument summarized each of the 12 key steps and provided space  for NNSA officials to describe actions they had taken to meet the  criteria for each step. To review the information provided by NNSA,  we checked NNSA’s April 2013 estimate for obvious errors in  accuracy and completeness and compared it with previous versions of  the life-cycle cost estimate provided by NNSA. In addition, we  interviewed NNSA officials to determine what requirements, if any,  they followed for developing the estimate, their purpose for developing  it, and their plans for presenting it for management approval. Finally,  we interviewed NNSA officials from the Office of Analysis and  Evaluation, which the Plutonium Disposition program had tasked with  conducting an independent assessment of the MOX facility’s  operating costs.

NNSA’s estimate to complete the MOX facility. Because NNSA had  not approved a revised cost and schedule estimate for the MOX  facility, we assessed the MOX contractor’s September 2012 proposal  for increasing the project’s cost, which NNSA had directed the MOX  contractor to use as a provisional baseline for purposes of monthly  reporting. We compared data presented in various tables of the  proposal for consistency and reviewed additional documents,  including the technical baseline providing a detailed description of the  MOX facility. We provided a draft of our assessment to NNSA and  revised the draft, as appropriate, after discussing our assessment with  NNSA program officials and the contractor.

NNSA’s estimate to complete the WSB. We assessed the WSB  schedule estimate that the cost increase for the project approved in  December 2012 was based on because, as described in the GAO  Schedule Assessment Guide, a reliable schedule can contribute to an  understanding of the cost impact if a project does not finish on time.  Specifically, we compared the contractor’s February 2013 monthly  update to its schedule estimate, which was the most recent available  update when we conducted our analysis, with the 10 best practices  associated with the characteristics of a high-quality schedule. As part  of our assessment, we reviewed documents related to the project’s  schedule, including NNSA’s project execution plan for the WSB, the  project’s work breakdown structure, and the project’s February 2013  update to the document showing the longest path to project  completion. In addition, we interviewed the NNSA federal project  director for the WSB and the WSB contractor’s project leader and  scheduler. We provided a draft of our assessment to NNSA and  revised the draft, as appropriate, after discussing our assessment with  NNSA program officials and the contractor.

We conducted this performance audit from November 2012 to February  2014 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

section_title: Appendix II: Assessment of NNSA’s Process for Developing a Life-cycle Cost Estimate for the Plutonium Disposition Program
subsections: section_title: Best practice characteristic and overall assessment Comprehensive: Partially met
paragraphs: Detailed assessment  Partially met. NNSA assigned a team to develop and update the  estimate but did not have a written plan for developing it.

Partially met. Work breakdown structures to define in detail the  work necessary to accomplish objectives were developed for the  MOX and WSB projects and other components of the program,  but NNSA had not formalized a program-level work breakdown  structure.

Substantially met. NNSA officials described the purpose as  supporting annual budget requests, which include requirements  that NNSA (1) present the full life-cycle cost of capital projects,  such as the MOX facility, and (2) estimate expenditures for the  fiscal year with respect to which the budget is submitted and at  least the four succeeding fiscal years.

Substantially met. NNSA developed a program requirements  document to identify the scope, functions, and requirements of the  Plutonium Disposition program. NNSA documented performance  characteristics for program components in contracts, technical  baselines, and execution plans.

Partially met. NNSA identified ground rules and assumptions, but  NNSA officials said that assumptions for the program change  frequently, hindering development of a life-cycle cost estimate.

Substantially met. NNSA collected data at the project level, where,  according to NNSA, data were documented in contractor systems  and estimates were developed by teams of knowledgeable staff  using historical information, current cost and pricing information,  engineering and vendor quotes, cost guides, and current material  and labor costs.

Minimally met. NNSA documented the estimate on spreadsheets,  but it did not develop a single document to describe data sources  and steps taken in developing the estimate so that it could be  replicated by someone other than those who prepared it.

Not met. NNSA considered the estimate to be draft and  predecisional, and NNSA officials said they did not have plans to  present an estimate to management for approval until NNSA  completes its reevaluation of its strategy for disposing of surplus  weapons-grade plutonium.

Partially met. NNSA developed a point estimate, but it did not use  a program-level work breakdown structure to do so because it had  not formalized such a structure.

Partially met. NNSA updated the estimate periodically to include  actual costs and changes to program and project requirements,  but it did not clearly document how changes affected the estimate.

section_title: Best practice characteristic and overall assessment Credible: Not met
paragraphs: Detailed assessment  Not met. NNSA did not conduct an independent cost estimate for  the overall program’s life-cycle cost estimate, and it had not  completed independent cost estimates for the program’s two  construction projects.

Not met. NNSA did not conduct a formal sensitivity analysis at the  program level.

Not met. NNSA did not conduct a risk and uncertainty analysis at  the program level.

section_title: Appendix III: Assessment of the MOX Contractor’s Proposed Cost Estimate Compared with Industry Best Practices
subsections: section_title: Best practice characteristic and overall assessment Comprehensive: Substantially met
paragraphs: Best practice  The cost estimate includes all life-cycle  costs.

Detailed assessment  Met. The estimate covered construction and startup  costs; at NNSA’s direction, the estimate excluded  operation and maintenance costs.

The cost estimate completely defines the  program, reflects the current schedule,  and is technically reasonable.

Substantially met. The estimate was based on NNSA’s  statement of work and the contractor’s technical  baseline for the original scope of the MOX facility.

The cost estimate work breakdown  structure is product-oriented, traceable to  the statement of work/objective, and at an  appropriate level of detail to ensure that  cost elements are neither omitted nor  double-counted.

Partially met. The work breakdown structure clearly  outlined the end product and major work of the project,  but some cost elements were missing from the work  breakdown structure.

The estimate documents all cost- influencing ground rules and  assumptions.

Partially met. The estimate documented that it was  based on a profile of NNSA’s projected annual funding  to complete the project but did not provide justifications  for some assumptions, such as not more than a set  amount of work being nonstandard.

The documentation captures the source  data used, the reliability of the data, and  how the data were normalized.

Partially met. The estimate was based on actual costs  through May 2012 and used a database of labor and  other costs, but it did not state whether or how all data  had been normalized to ensure data comparability.

The documentation describes in sufficient  detail the calculations performed and the  estimating methodology used to derive  each element’s cost.

Met. The estimate used a combination of expert  opinion and extrapolation from actual data to develop  estimates for and sum up individual cost elements of  the work breakdown structure.

The documentation describes, step by  step, how the estimate was developed so  that a cost analyst unfamiliar with the  program could understand what was  done and replicate it.

Partially met. The estimate used quantities of materials  and labor hours to develop estimates for individual  cost elements but did not document how these  quantities were estimated.

The documentation discusses the  technical baseline description and the  data in the baseline is consistent with the  estimate.

Partially met. The estimate agreed with NNSA’s  statement of work and the contractor’s technical  baseline for the original scope of the MOX facility, but  the technical baseline did not cover the addition of  capability to supply plutonium feedstock.

The documentation provides evidence  that the cost estimate was reviewed and  accepted by management.

Partially met. DOE began a review of the proposed  estimate but did not approve it.

The cost estimate results are unbiased,  not overly conservative or optimistic, and  based on an assessment of most likely  costs.

Minimally met. The estimate was higher than needed  to achieve an 85 percent confidence level—the level  directed by NNSA—that the final cost would be less  than the estimate.

The estimate has been adjusted properly  for inflation.

Substantially met. The estimate appeared to adjust  cost elements for inflation, but adjustments were not  well-documented.

section_title: Best practice characteristic and overall assessment
paragraphs: Best practice  The estimate contains few, if any, minor  mistakes.

Detailed assessment  Met. The estimate contained few minor mistakes, and  calculations within the estimate were internally  consistent.

The cost estimate is regularly updated to  reflect significant changes in the program  so that it always reflects current status.

Partially met. The estimate was based on actual costs  through May 2012 and did not reflect updated costs  from the contractor’s system for tracking and reporting  cost and schedule performance.

Variances between planned and actual  costs are documented, explained, and  reviewed.

Minimally met. The estimate explained variances  between planned and actual costs at a high level but  not at the cost element level.

The estimate is based on a historical  record of cost estimating and actual  experiences from other comparable  programs.

Partially met. The estimate did not explain to what  extent it was based on historical data from other  similar programs or facilities.

The estimating technique for each cost  element was used appropriately.

Substantially met. The estimating method used— developing the estimate at the lowest level of the work  breakdown structure, one piece at a time, with the sum  of the pieces becoming the estimate—was appropriate  for a project under way.

The cost estimate includes a sensitivity  analysis that identifies a range of possible  costs based on varying major  assumptions, parameters, and data  inputs.

Not met. The estimate did not include a sensitivity  analysis.

A risk and uncertainty analysis was  conducted that quantified the imperfectly  understood risks and identified the effects  of changing key cost driver assumptions  and factors.

Partially met. The estimate included a risk and  uncertainty analysis but did not properly conduct or  clearly document all steps in the analysis.

Major cost elements were cross-checked  to see whether results were similar.

Not met. The estimate provided no evidence that major  cost elements were cross-checked.

An independent cost estimate was  conducted by a group outside the  acquiring organization to determine  whether other estimating methods  produce similar results.

Not met. DOE halted its independent cost estimate of  the contractor’s proposed estimate as part of DOE’s  decision to reevaluate its strategy for disposing of  surplus weapons-grade plutonium.

section_title: Appendix IV: Assessment of the Waste Solidification Building’s Schedule Estimate Compared with Industry Best Practices
subsections: section_title: Best practice characteristic and overall assessment Comprehensive: Partially met
paragraphs: Detailed assessment  Minimally met. The schedule estimate’s 2,429 activities to  complete the project included one summary activity in place of  the construction subcontractor’s 3,851 activities and, therefore,  did not capture the remaining detailed work to be performed by  the subcontractor.

Partially met. The schedule estimate assigned resources, such  as labor and materials, to only about half of the remaining 2,429  activities.

Substantially met. The schedule estimate included activity  durations that were generally short enough to be consistent with  the needs of effective planning.

Minimally met. The schedule estimate sequenced activities in  ways that decreased the probability of activities starting on time  and contained activities that were not properly tied with the start  or end date of other activities, potentially obscuring the critical  path determining the project’s earliest completion date.

Partially met. Changes to the critical path were evaluated  monthly and tracked in monthly status reports, but constraints in  scheduled dates of certain activities convoluted the critical path.

Ensuring reasonable total float  Minimally met. The schedule estimate included high total float  values—the amount of time by which an activity can slip without  affecting a completion date—potentially resulting in an  inaccurate assessment of the project’s completion date.

Substantially met. The schedule estimate was traceable  horizontally (i.e., across sequenced activities) and vertically  (i.e., between activities and subactivities).

Minimally met. The contractor conducted a schedule risk  analysis, but the results of the analysis were unreliable for  determining the likelihood of the project’s completion date and  did not align with DOE’s revised cost and schedule estimate.

Partially met. According to project officials, the schedule was  updated weekly, but no narrative accompanied the weekly  updates.

Maintaining a baseline schedule  Minimally met. Project officials stated that they used the  schedule to measure performance, but they did not provide  thorough documentation enabling the schedule to be validated,  such as a narrative providing a log of changes and their effects.

section_title: Appendix V: Comments from the National Nuclear Security Administration

section_title: Appendix VI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact

section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Daniel Feehan, Assistant  Director; Remmie Arnold; Antoinette Capaccio; Juaná S. Collymore;  Joseph Cook; Tisha Derricotte; Emile Ettedgui; Cristian Ion; Alison  O’Neill; Cheryl Peterson; and Karen Richey made key contributions to this  report.


Question-summary annotations: questions: question: What has NNSA still not done regarding the program construct cost increases which may be essential to do going forward?
answer: NNSA has not analyzed the underlying, or root, causes of the Plutonium Disposition program construction cost increases to help identify lessons learned and help address the agency's difficulty in completing projects within cost and schedule, which has led to NNSA's management of major projects remaining on GAO's list of areas at high risk of fraud, waste, abuse, and mismanagement.

question: How does the NNSA currently decide whether to conduct a root cause analysis of cost increases?
answer: NNSA officials said that, because the order does not require a root cause analysis of cost increases, NNSA decides on a case-by-case basis whether to conduct one.
child_questions: question: Without a root cause analysis, what struggles will the NNSA continue to grapple with?
answer: Without a root cause analysis, it is uncertain whether NNSA will be able to accurately identify underlying causes of the increases to identify and implement corrective measures and identify lessons learned to apply to other projects.

question: How does the current information NNSA has found differ from information that would be found in a root cause analysis?
answer: Unlike a root cause analysis, the cost drivers NNSA identified provided few details about why the drivers existed, such as DOE's reasons for approving the MOX facility's cost and schedule estimates before the design was complete.

questions: question: What did NNSA learn contributed to cost increases?
answer: After determining that the performance of the contractors for the MOX facility and WSB contributed to cost increases, NNSA took steps to hold the contractors accountable by withholding fees specified under the contracts.

question: In 2013, how did NNSA hold MOX contractor accountable?
answer: In particular, as of November 2013, NNSA withheld $45.1 million or close to one-third of the MOX contractor's fees, including fees tied to meeting the MOX project's cost and schedule estimates.
child_questions: question: How did NNSA hold WSB contractor accountable?
answer: In addition, NNSA withheld $7.7 million or about 40 percent of the WSB contractor's fees tied to various performance measures for the WSB, such as completing construction milestones.

questions: question: Generally, what shortcomings were there in NNSA's most recent estimates?
answer: NNSA's most recent estimates for the Plutonium Disposition program did not fully reflect all the characteristics of reliable cost estimates (e.g., credible) and schedule estimates (e.g., well-constructed) as established by best practices for cost- and schedule-estimating, placing the program at risk of further cost increases.
child_questions: question: What example does this article provide of a specific shortcoming in the 2012 proposal?
answer: (2) Because the MOX contractor's September 2012 proposal for increasing the cost of the MOX facility did not include a formal analysis to examine the effects of changing assumptions, it was minimally credible.

question: Why was the NNSA's draft April 2013 life-cycle cost not credible?
answer: (1) NNSA's draft April 2013 life-cycle cost estimate of $24.2 billion for the overall program was not credible because NNSA did not conduct an independent cost estimate to provide an unbiased test of whether the estimate was reasonable.

question: How did the monthly update from the WSB contractor obscure the determination of the project's completion date?
answer: (3) The WSB contractor's February 2013 monthly update to its schedule estimate was minimally well-constructed in that it contained activities that were not properly tied with the start or end date of other activities, which could potentially obscure the critical path determining the project's completion date.

questions: question: What was the GAO asked to do in response?
answer: GAO was asked to review these cost increases and the life-cycle cost estimate.

question: What four aspects does their report examine?
answer: This report examines: (1) drivers NNSA identified for the cost increases; (2) the extent to which NNSA analyzed underlying causes of the cost increases; (3) steps NNSA took to hold construction contractors accountable for their role, if any, in the cost increases; and (4) the extent to which NNSA's most recent estimates met cost- and schedule-estimating best practices.
child_questions: question: What methodology did the GAO undertake in order to go about creating this report?
answer: GAO reviewed NNSA's draft life-cycle cost estimate and contractor estimates of the MOX project's cost and WSB schedule, compared the estimates with cost- and schedule-estimating best practices, and interviewed DOE and NNSA officials.