text: Background This section discusses (1) the history and status of the MOX project; (2) the roles of DOE, NNSA, and the contractor in managing and overseeing the MOX project; (3) project management lessons learned; and (4) DOE’s and NNSA’s recurring project management problems. History and Status of the MOX Project DOE began the MOX project over 20 years ago, in 1997, as part of a strategy to manage the disposition of large quantities of surplus, weapons-grade plutonium no longer needed for defense purposes. This strategy, now undertaken through NNSA’s Plutonium Disposition program, originally planned to dispose of the plutonium through a dual approach—(1) conversion into mixed-oxide fuel and (2) immobilization in glass or ceramic material—but NNSA later cancelled the immobilization approach in favor of the approach for only mixed-oxide fuel. In 1999, DOE awarded a contract to design, construct, and operate a MOX facility to the contractor consortium of Duke, Cogema, Stone & Webster, LLC— now called MOX Services, LLC (MOX Services). In February 2002, NNSA reported to Congress that the construction of the MOX project would begin in fiscal year 2004, with operations set to begin in fiscal year 2007, and cost nearly $1 billion to design and construct. However, as figure 1 shows, construction of the MOX project did not begin until 2007 after DOE formally approved the project’s estimated cost of about $4.8 billion and estimated completion date of September 2016. In December 2008, DOE approved a revised cost estimate for completing construction of the MOX project of $4.9 billion and a 1-month delay in the start of operations to October 2016. From 2009 through 2011, the estimated cost to complete construction of the MOX project remained at $4.9 billion. However, the MOX project’s cost and schedule estimate changed significantly in 2012. That year, at NNSA’s direction to update the estimate, the MOX contractor submitted a proposal to increase the cost of the facility to about $7.7 billion—an increase of about $2.8 billion from the 2008 estimate—with the start of operations delayed by about 3 years, to November 2019. After receiving the MOX contractor’s revised estimate that indicated significant cost increases and schedule delays to the project, NNSA stated in its fiscal year 2014 budget request that pursuing the MOX approach might be unaffordable and proposed to slow down construction while the agency assessed alternative approaches for plutonium disposition. After a series of reviews, DOE ultimately concluded that pursuing an alternative disposition approach—referred to as “dilute and dispose”—could significantly reduce the life-cycle cost of the Plutonium Disposition program, compared with continuing the program using the MOX approach. Following the identification of a potentially less costly approach to plutonium disposition, in February 2016, DOE’s fiscal year 2017 budget request proposed terminating the MOX project in favor of pursuing the dilute and dispose approach. Congress appropriated funding for the MOX project for fiscal years 2017 and 2018 and directed DOE to continue work on the project. In August 2016, DOE issued a revised cost estimate of approximately $17.2 billion to complete construction of the MOX project by 2048. In the face of this significant cost increase, the National Defense Authorization Act for Fiscal Year 2018 authorized the Secretary of Energy to terminate the MOX project if, among other things, he could certify that the remaining life-cycle cost for an alternative option for carrying out plutonium disposition would be less than approximately half of the estimated remaining life-cycle cost of carrying out the MOX project. In May 2018, DOE completed this certification and notified Congress of its intention to terminate construction of the MOX project and to instead pursue the dilute and dispose option. The Secretary of Energy reported that the life-cycle cost estimate was $19.9 billion for the dilute and dispose option compared to $49.4 billion for the MOX project. In October 2018, NNSA terminated the project. Additional information on the history and status of the MOX project is in appendix II. Roles of DOE, NNSA, and the Contractor in Managing the MOX Project DOE and NNSA are responsible for providing overall direction to, and oversight of, the contractor for the MOX project. The contractor, MOX Services, is responsible for the design, construction, and operation of the MOX facility. DOE. The Office of Project Management participates in a number of the MOX project’s oversight activities. In particular, the office has led independent reviews of the MOX project to validate its cost and schedule estimates and has conducted certification and surveillance reviews of the MOX contractor’s earned value management (EVM) system. NNSA. Subsequent to its establishment in 2000, several NNSA offices have provided overall direction to, and oversight of, the contractor for the MOX project, including the Office of Fissile Materials Disposition and the Office of Defense Nuclear Nonproliferation. In November 2011, after starting to place increased emphasis on improving its management of projects, the newly created Office of Acquisition and Project Management began providing overall direction to, and oversight of, the contractor for the MOX project. In March 2013, the Office of Acquisition and Project Management established the NNSA MOX Project Management Office at the Savannah River Site to lead the onsite project and contract management direction, administration, and oversight of the MOX project. MOX Services. As the contractor for the MOX project, MOX Services is responsible for designing, constructing, and operating the MOX facility. MOX Services has also subcontracted work to complete certain construction activities, such as the fabrication of specific types of equipment, including the complex gloveboxes needed for handling plutonium and the heating, ventilation, and air conditioning systems. Figure 2 depicts the roles of, and interrelation among and between, DOE, NNSA, and the MOX contractor in overseeing the MOX project. Project Management Lessons Learned According to key practices that we and others have identified for both program and project management, it is important to identify and apply lessons learned from programs, projects, and missions to limit the chance of recurrence of previous failures or difficulties. As such, the use of lessons learned—such as project management lessons learned—is a principal component of an organizational culture committed to continuous improvement. Lessons learned, therefore, serve to communicate knowledge more effectively and to ensure that beneficial information is factored into planning, work processes, and activities. They also provide a powerful method of sharing ideas for improving work processes, facility or equipment design and operation, quality, and cost-effectiveness. Moreover, as we and others have previously found, agencies can learn lessons from an event and make decisions about when and how to use that knowledge to change behavior. Key practices of a lessons-learned process include collecting, analyzing, saving or archiving, and sharing and disseminating information and knowledge gained on positive and negative experiences (see fig. 3). DOE and NNSA Have Faced Recurring Project Management Problems For more than 2 decades, we and others have reported on the recurring nature of the problems affecting DOE’s and NNSA’s ability to manage contracts and projects effectively. Many of these problems have related to DOE’s and NNSA’s struggles with managing projects, such as the MOX project, within their initial cost and schedule estimates, including the following: In 1999, the National Academy of Science’s National Research Council reported that recurring problems with project management had raised questions about the credibility of DOE’s conceptual designs and cost estimates. In a March 2007 report, we found that 9 of 12 major projects we reviewed—including the MOX project—had exceeded their original cost estimates, schedule estimates, or both, principally because of ineffective project oversight and contractor management. In a November 2014 report, the Congressional Advisory Panel on the Governance of the Nuclear Security Enterprise (Augustine-Mies Panel) stated that NNSA’s inability to estimate costs and execute projects according to plan has been a major source of dissatisfaction among the national leadership and had significantly undermined NNSA’s credibility. Further, in April 2015, we found that NNSA has had a long history of identifying corrective actions for problems and declaring them successfully resolved, only to then identify additional actions needed to address the problems. As we found, the recurrence of such problems suggests that NNSA did not have a full understanding of the root causes of its contract- and project-management challenges. Moreover, our 2017 high-risk report found that DOE had taken several important steps that demonstrate its commitment to improving contract and project management, but that DOE’s efforts had not fully addressed several areas where the department continues to have shortcomings. Areas with shortcomings include acquisition planning for major contracts and the quality of enterprise-wide cost information available to DOE managers and key stakeholders. Additional information on our prior work highlighting selected DOE and NNSA project management problems is in appendix III. NNSA Recognized Certain Indicators of Cost and Schedule Problems after Strengthening Its Oversight in 2010 and 2011 Prior to 2011, NNSA project staff had failed to recognize and fully resolve certain cost and schedule problems that indicated that the MOX project would not be completed on time or within its approved cost estimates. However, after taking actions to strengthen its project management oversight in late 2010 and 2011, NNSA recognized indicators of a number of problems with the MOX project that contributed to NNSA’s decision to terminate the project. NNSA Failed to Recognize and Fully Resolve Certain Cost and Schedule Problems Affecting the MOX Project Prior to 2011 Prior to 2011, NNSA’s staff responsible for overseeing the MOX project failed to recognize and fully resolve certain cost and schedule problems that indicated that the project would not be completed on time or within its approved cost estimates. The NNSA staff responsible for overseeing the MOX project at that time were generally inexperienced in overseeing complex nuclear construction projects. From 2007 through 2011, staff overseeing the MOX project were primarily familiar with large programmatic initiatives and operations but had little experience in managing large, complex first-of-a-kind nuclear construction projects, according to a May 2014 root cause analysis. Although information available to the NNSA staff showed that there were cost and schedule problems that indicated the increasing likelihood that the project would not be completed within its approved total cost estimate of $4.9 billion, the staff did not recognize and fully resolve four key problems. First, information about the contractor’s use of inaccurate rates to estimate the time needed to complete certain construction activities— commonly referred to as unit rates or planned production rates—indicated that the project would not be completed within its approved cost estimate. These rates are used to reflect levels of productivity during construction and to help develop projects’ cost and schedule estimates, including updates to annual forecasted estimates. Following the start of construction in August 2007, the MOX contractor began to experience lower-than-estimated productivity rates for key construction activities, according to the May 2014 root cause analysis report. Despite this issue, the contractor did not incorporate more realistic assumptions regarding the unit and production rates, such as by updating the estimated costs and time needed to complete specific construction activities, when developing the contractor’s annual forecasted estimates of the project’s total cost for 2008 through 2011. MOX contractor representatives told us that the unit rates they used to develop cost and schedule estimates were realistic based on assumptions at that time and that DOE was involved in the development of the unit rates. In addition, the MOX contractor’s representatives told us that expected improvements in unit rates did not materialize because of higher than expected levels of worker turnover. NNSA staff overseeing the project at that time did not recognize that the unit rates for calculating and updating unit rate estimates should be realistic and reflect levels of productivity during construction, as called for in project management principles, or resolve the issues. As a result, the staff did not take action to resolve the MOX contractor’s continued use of unrealistic unit rates that did not reflect actual construction progress being made. Furthermore, NNSA staff did not recognize the extent to which decreased productivity by the contractor created future cost increases and schedule delays or resolve the issue. Consequently, from 2008 to 2011, the MOX contractor continued to use its overly optimistic and unrealistic unit rate estimates when developing its annual forecasted cost estimates. Second, the MOX contractor’s annual forecasted estimates for the project consistently increased from 2008 through 2011, and the level of confidence in those estimates decreased, indicating that the project would not be completed within its approved cost estimate. Beginning in 2008, the MOX contractor submitted an annual update to its forecasted estimate for the project. These estimates increased each year, rising by about $140 million to $280 million annually, with the estimated total project cost increasing from about $4.1 billion in 2008 to about $4.7 billion in 2011 (an increase of about 15 percent). The MOX contractor’s representatives said they attempted to mitigate the increases, such as by identifying cost savings on the project. Additionally, as the May 2014 root-cause analysis report stated, the level of confidence for completing the MOX project within the approved $4.9 billion total project cost estimate declined each year, from an 85 percent likelihood of completing the project within the estimate in 2009 to 45 percent in 2011. Both the annual increases in forecasted estimates and the annual decline in level of confidence illustrated the increasing likelihood that the MOX contractor would not complete the project for $4.9 billion. As a result of inexperience, the NNSA staff overseeing the project at that time did not adequately examine the potential consequences of such cost performance trends over the future schedule and through project completion or resolve the issues. As the May 2014 root-cause analysis report stated, NNSA staff did not fully recognize how the risks and challenges the MOX project faced negatively affected not only the project’s performance but also its cost and schedule. For example, that report found that the staff were unable to determine that there were fundamental problems with completing the MOX project’s design and with maintaining construction efficiency and progress; both of which contributed to schedule delays and cost increases. The May 2014 root- cause analysis report stated that because of inexperience in project management, NNSA staff did not direct the MOX contractor to develop a more realistic and achievable forecasted estimate for the total cost to complete the MOX project until January 2012. Third, information about procuring materials out of sequence and the resulting rework indicated that the project would not be completed on schedule or within its approved cost estimate. According to NNSA officials, the MOX contractor’s method for measuring earned value incentivized the contractor to purchase and procure materials early and, in a number of cases, out of sequence, as this helped demonstrate progress. For example, figure 4 shows outdoor “laydown yards” and an offsite warehouse storing large amounts of commodities, such as pipes and electrical panels, that NNSA officials said the MOX contractor procured earlier than needed. The May 2014 root-cause analysis report stated that between 2007 and 2011, the equipment and material procured out of sequence resulted in the need for rework in some cases because later design changes required changes to the equipment or the need to procure different items, leading to additional costs for the project. The MOX contractor’s representatives told us they disagreed with NNSA’s characterization that they procured material too early. According to the contractor representatives, they purchased materials in support of both the project schedule and planned construction end date of 2016, as well as to achieve the efficiencies through bulk pricing or reduced delivery charges from procuring larger quantities of items or multiple items at the same time. Additionally, the MOX contractor representatives disagreed that they structured the methods for measuring earned value performance to claim earned value in ways that did not reflect actual progress. In particular, the MOX contractor representatives said that NNSA staff were involved in the development of the original methods used for measuring earned value. NNSA staff did not take steps to resolve the issues with the disproportionate value earned by the MOX contractor for purchasing, procuring, and placing certain commodities until 2015 when the MOX contractor revised its methods for measuring earned value. Consequently, the reported commodity installation data based on the MOX contractor’s methods for measuring claimed earned value inflated the amount of progress being made on the construction of the MOX project compared with the amount of work completed. Fourth, information about the use of management reserve funds early in the project indicated that the project would not be completed within its approved baseline. To address cost increases experienced early in the project, the MOX contractor began to use the project’s management reserve funds. A May 2010 surveillance review of the MOX contractor’s EVM system prepared for DOE by an independent contractor identified this issue and concluded that the rate at which the MOX contractor was using its management reserve indicated that it was unlikely that there would be any reserve left to address any risks that were expected to be encountered later in the project. DOE’s June 2011 follow-up review of the MOX contractor’s EVM system found that the MOX contractor was no longer covering cost variances by using management reserve; however, the MOX contractor’s previous use of management reserve to cover cost overruns had resulted in inaccurate, inflated cost performance and understated forecasted cost estimates. The MOX contractor’s representatives told us they disagreed with the premise that the management reserve was used to obscure cost performance. Moreover, they noted that NNSA’s cost-accounting and management staff worked with the contractor on all EVM issues, including the use of management reserve. NNSA staff did not recognize and resolve issues with the contractor’s use of the management reserve to mitigate cost overruns or the effect on the project’s cost performance and forecasted cost estimates in part because, as the May 2014 root cause analysis report stated, the staff possessed little experience in project management. According to project management principles, management reserve should be prevented from being consumed too early so as to ensure that enough reserve remains available to address any problems that may arise late in the project. The inexperienced NNSA staff also did not recognize that certain problems were creating cost overruns because, as stated in the May 2010 surveillance review, the MOX contractor’s use of the management reserve to cover such overruns hid the problems and did not alleviate their root causes. As a result of not recognizing or resolving the MOX contractor’s inappropriate use of the management reserve earlier, NNSA reported inaccurate measurements of cost performance to DOE and other stakeholders. DOE’s Project Management Changes Strengthened Oversight of the MOX Project In late 2010 and 2011, DOE began to implement actions to strengthen project management across the department, including NNSA. These actions, which agency officials said were primarily undertaken in response to project management problems we and others had identified, contributed to the steps NNSA began to take to strengthen its project management and oversight of the MOX project. Changes that strengthened NNSA’s oversight of the MOX project included: (1) initiating project peer reviews and (2) making several organizational changes to improve project oversight. These changes to DOE’s and NNSA’s oversight of the MOX project contributed to the decision to terminate the project. First, in its November 2010 update to requirements for capital asset projects, DOE established a requirement to conduct peer reviews at least once a year for large or high-visibility projects with a total project cost of $100 million or greater. The update required peer reviews more frequently for complex projects or those experiencing performance challenges. According to DOE and NNSA officials, they added the requirement in response to a recommendation in our May 2008 report. According to NNSA officials, as a result of this requirement, NNSA began conducting peer reviews of the MOX project in 2011. These reviews led NNSA to identify significant cost and schedule problems at the MOX project and included a number of recommendations to improve project performance. For example, a March 2012 NNSA peer review found that the MOX project’s total cost may have been understated by anywhere from $600 million to $900 million, in part because the contractor’s estimated unit rates and planned production rates were not reflective of the actual performance at that time. Moreover, the peer review found that the estimated completion date of October 2016 was also at risk. As a result, the peer review team recommended, among other things, that the MOX contractor develop an update to its formal cost and schedule estimate. As a result of the findings and recommendations from its peer reviews, NNSA requested and the MOX contractor submitted in September 2012 a proposal that included a revised cost estimate for the MOX project of about $7.7 billion and an estimated completion date of November 2019. In response to the significant cost increases, schedule delays, and project risks captured in the MOX contractor’s updated cost and schedule estimate, NNSA proposed a slowdown of MOX project construction activities in its fiscal year 2014 budget request to begin assessing alternative plutonium disposition strategies. Second, NNSA carried out several organizational changes starting in 2011 that led to improved oversight of the MOX project in some areas and the continued identification of cost and schedule problems. Specifically, NNSA transitioned management and oversight of the MOX project from the Office of Defense Nuclear Nonproliferation to the Office of Acquisition and Project Management, an office newly created in January 2011 to improve project oversight through the application of project management principles. In 2013, the Office of Acquisition and Project Management created the MOX Project Management Office at the Savannah River Site to provide project and contract management oversight for the MOX project. After establishing the MOX Project Management Office, the Office of Acquisition and Project Management sought to better address long- standing staffing challenges. For example, a May 2006 external independent review conducted for DOE found that, among other things, NNSA understaffed the oversight of the MOX project and recommended that DOE acquire sufficient personnel with the proper skills to manage and perform oversight of the project. However, NNSA did not address this issue until after the creation of the Office of Acquisition and Project Management. The Office of Acquisition and Project Management increased the number of staff with specific project management skillsets at the MOX Project Management Office from 20 for fiscal years 2010 to 2012 to 36 (18 federal employees and 18 support service contractors) for fiscal years 2016 to 2018. As a result of the staffing changes, the NNSA MOX Project Management Office strengthened its oversight of the MOX project, which contributed to the identification of additional problems, as described below. Conducted more in-depth assessments of the MOX contractor’s EVM system. After initially certifying the MOX contractor’s EVM system in May 2008, a May 2010 surveillance review of the MOX contractor’s EVM system prepared for DOE by an independent contractor identified a number of issues. The MOX contractor addressed the issues, according to DOE’s June 2011 review, resulting in the recertification of the EVM system at that time. According to NNSA officials, NNSA’s MOX Project Management Office conducted more in-depth assessments of the MOX contractor’s EVM system starting in 2013. These assessments led NNSA staff to identify a number of concerns with the contractor’s EVM system, such as earned value data errors; overstatements of the data on the percentage of work completed in certain areas; and in one instance, about $300 million in known cost growth that was not incorporated into the MOX project’s forecasted estimate of total project cost. According to NNSA officials, in March 2016, the NNSA federal project director requested an in-depth review of the contractor’s EVM system because of the continued identification of issues with the system, and the MOX contractor not adequately addressing them. According to its October 2016 review, DOE’s Office of Project Management identified significant deficiencies representing systematic and material internal control weaknesses and concluded that the MOX contractor’s EVM system could not be relied upon to provide credible and reliable cost and schedule performance data for either the project’s current status or its forecasted cost and schedule estimates. As a result, DOE’s Office of Project Management rescinded the MOX contractor’s EVM system certification because the system was no longer in compliance with the relevant standards. Implemented a more rigorous invoice review process. According to NNSA officials, prior to 2014, NNSA did not have a rigorous process in place to review the contractor’s invoices. The officials said that NNSA staff did not review all invoices and, for the reviews that were completed, they did not always thoroughly examine the details behind the invoices, such as reviewing invoices to verify that costs were allowable under DOE regulations. The NNSA officials told us that as part of their efforts to improve oversight of the MOX contractor’s invoice submissions, NNSA’s MOX Project Management Office staff developed a more rigorous invoice review process that resulted in a September 2014 guide. In addition, the NNSA MOX Project Management Office assigned an additional staff member to (1) help conduct invoice reviews due to the volume of work needed to review the MOX contractor’s invoices and (2) ensure that payments were made within the 14 days generally required by regulation. According to NNSA officials, as a result of the changes implemented by the office, NNSA identified a number of potentially unallowable costs ranging from less than $1,000 to more than $2 million. Reviewed the MOX contractor’s annual incurred costs. NNSA officials said that incurred cost audits were supposed to be conducted at least annually for the MOX project and that the Defense Contract Audit Agency was supposed to conduct the audits. However, these officials explained that due to a significant backlog, the Defense Contract Audit Agency did not complete all of the required audits. In light of the Defense Contract Audit Agency’s significant backlog—as well as a requirement prohibiting the agency from conducting non- defense agency audits—the NNSA MOX Project Management Office arranged to have a third party conduct an audit of the MOX contractor’s fiscal year 2010 incurred costs. This third-party audit identified more than $30 million in potentially unallowable costs. The significant cost and schedule problems that NNSA staff identified after strengthening its oversight of the MOX project contributed to NNSA’s decision to terminate it. Project management principles state that effective project management helps organizations to, among other things, increase the chances of success; resolve problems and issues; and identify, recover, or terminate failing projects. After NNSA’s project peer reviews and the MOX contractor’s proposed update to the project’s cost and schedule estimate showed the significant likelihood of additional cost growth and schedule delays, NNSA proposed slowing down construction of the MOX facility in 2013 and ultimately terminated the project in October 2018. DOE Has Requirements for Documenting and Sharing Lessons Learned, but They Do Not Ensure Consistent or Timely Documentation or the Evaluation of Corrective Actions As outlined in DOE Order 413.3B, DOE requires that project management staff document and share project management lessons learned on capital asset projects like MOX but does not require that all project management lessons learned from capital asset projects be documented consistently or shared in a timely manner. Moreover, DOE Order 413.3B does not require the evaluation of the results of corrective actions taken in response to lessons learned that are identified during the course of capital asset projects such as the MOX project to ensure that the problems experienced are resolved department-wide. DOE’s Requirements for Documenting and Sharing Lessons Learned for Capital Asset Projects DOE’s requirements for capital asset projects, as outlined in Order 413.3B, specify that project management lessons learned should be captured—that is, documented—throughout the continuum of a project. According to the order, there are five critical decisions (CD) that structure the life of a project. The CDs, which are summarized in figure 5, include approving: mission need (CD-0); alternative selection and cost range (CD-1); project performance baseline (CD-2); the start of construction or execution (CD-3); and the start of operations or project completion (CD- 4). DOE Order 413.3B requires project staff to submit project management lessons learned to DOE’s Office of Project Management within 90 days of two critical decision points: (1) upfront planning and design lessons learned are to be submitted within 90 days of CD-3 approval and (2) project execution and facility startup lessons learned are to be submitted within 90 days of CD-4 approval. DOE Order 413.3B also requires that lessons learned for capital asset projects be collected, analyzed, and disseminated by project management support offices. These offices consist of DOE or NNSA staff who provide support to federal project directors and are established exclusively to oversee and manage the activities associated with projects. Additionally, DOE Order 413.3B states that the Project Management Risk Committee should support project management activities within DOE by enabling the sharing of lessons learned on a routine basis. DOE and NNSA officials told us that program and project offices document and save project management lessons learned for capital asset projects in different ways. In particular, DOE and NNSA officials told us that peer reviews, which are saved in DOE’s Project Assessment and Reporting System (PARS II) database, are a primary source of project management lessons learned. The officials also said that project management lessons learned are saved through monthly project reports, monthly staff meetings, Project Management Risk Committee meeting notes, and project management workshops and training courses. In addition, DOE and NNSA officials told us that some lessons learned are shared through informal person-to-person discussions that allow lessons learned to be shared among staff. Further, the officials said that they address project management problems identified in lessons learned by making changes to DOE Order 413.3B. In addition, while not required, DOE may capture some lessons learned for projects during the project review process. For example, DOE’s standard-operating procedures for conducting external independent reviews state that the scope of such reviews can include assessing whether project teams are documenting and sharing lessons learned from their projects internally and externally. However, as noted in the standard-operating procedures, this is an example of an area that can be included as part of an external independent review, although there is no requirement to do so. DOE’s Lessons-Learned Requirements for Capital Asset Projects Do Not Ensure Consistent or Timely Documentation and Sharing or the Evaluation of Corrective Actions DOE Order 413.3B requires project management lessons learned for capital asset projects to be documented throughout the life of a project but does not specifically require lessons learned to be documented and saved in a consistent manner or shared routinely or in a timely manner. Moreover, the order does not require all corrective actions related to these lessons learned to be evaluated for effectiveness. DOE Does Not Require That Lessons Learned for Capital Asset Projects Be Documented and Saved Consistently Although DOE and NNSA use multiple means to document and save lessons learned, we found DOE and NNSA program and project offices do not document and save such lessons consistently so that they are readily accessible by other staff. For example, NNSA uses an internal database to save project management lessons learned for its projects. However, NNSA officials told us that DOE staff outside of NNSA must request access to the database before they can read and examine the lessons learned that are documented and saved in the database. Officials from DOE’s Office of Science told us that their office submits some lessons learned to the PARS II database and maintains some project management lessons-learned reports on a publicly available webpage. A senior official from DOE’s Office of Environmental Management told us that some lessons learned from its projects are sent to its staff through monthly lessons-learned bulletins, but the bulletins are not entered into PARS II. In addition, DOE and NNSA officials said that project staff can enter specific lessons learned gleaned from their project in a lessons- learned repository within PARS II. For example, as of November 2017, PARS II contained 20 entries for project management lessons learned from the MOX project. According to key practices for lessons learned identified by us and the Center for Army Lessons Learned, a central component of a successful lessons-learned process is to ensure that lessons learned are stored in a logical, organized manner. Specifically, as we have previously found, lessons learned should be stored in a manner—such as an electronic database—that allows users to perform information searches using key words and functional categories. Moreover, information in the database should be updated regularly and provide a logical system for organizing information that is easily retrievable and made available to any requester. We have also found that relying on person-to-person discussions to share lessons learned can be problematic because personal networks can dissolve—for example, through attrition or retirement—and informal information sharing does not ensure everyone is benefiting from the lessons that are gleaned. Further, by not documenting and saving all lessons learned (e.g., those shared through person-to-person exchanges), there is also generally no way to ensure the validation of the information shared. This is not consistent with the key practice from the Center for Army Lessons Learned, which states that by documenting and saving project management lessons learned in a logical, organized manner such as an electronic database, lessons learned can be archived, managed, and made available for review by other projects and applied to them at a future date. Because DOE Order 413.3B does not indicate where all project management lessons learned should be documented and saved in a consistent manner, the department cannot ensure that future capital asset projects will be able to take advantage of experiences from past projects. We found that DOE and NNSA did not document all lessons learned in a consistent manner, and DOE officials acknowledged that DOE Order 413.3B does not require documenting or saving lessons learned that are presented through various formal or informal means in a common location. By developing requirements that clearly define how and where all project management lessons learned should be documented and saved to make them readily accessible across the department, such as in a database, DOE—including NNSA—could improve the agency’s existing lessons-learned process. DOE Does Not Require That Lessons Learned for Capital Asset Projects Be Submitted and Shared Routinely or in a Timely Manner DOE Order 413.3B’s requirements for project management lessons learned do not require that all lessons learned be shared routinely or in a timely manner. In particular, the order does not require that lessons learned be submitted and shared routinely until CD-3—the start of construction. Consequently, DOE and NNSA staff are not required to submit lessons learned during the CD-0, CD-1, and CD-2 phases of a project. These earlier phases, which involve upfront planning and design for the selected project, often occur many years before the approval and start of construction. Notably, both the MOX and Uranium Processing Facility (UPF) projects took about 10 years to reach the start of construction (CD-3) and experienced cost increases and schedule delays. We and others have previously found that lessons learned should be submitted in a timely manner so as to ensure that key information is available to identify and address problems or incorporate successful activities as early and quickly in the process as possible. For example, we found that lessons-learned reports (i.e., reports documenting lessons- learned reviews) should be prepared promptly so that knowledgeable personnel are available to contribute to the reports, important details are recalled accurately, and there are no delays in the dissemination of lessons learned. Moreover, according to the Center for Army Lessons Learned, the guiding principle in executing a sharing strategy for lessons learned is to get the right information to the right person at the right time. Such a strategy can entail developing a process for creating timelines for sharing lessons learned that are tied to the urgency of the information and a means to disseminate that information. Because DOE Order 413.3B does not require lessons learned to be submitted prior to CD-3, the department is limiting its ability to promptly evaluate and address early issues with projects and apply such lessons learned to other projects department-wide. This approach could affect the successful completion of capital asset projects, particularly those that experience prolonged upfront planning and design phases similar to those the MOX and UPF projects experienced. By developing requirements for sharing project management lessons learned from early in the CD phases of projects (i.e., prior to CD-3) routinely and in a timely manner to improve the ability to identify and evaluate problematic practices and positive experiences, DOE—including NNSA—could help improve the success of future capital asset projects and avoid the problems encountered overseeing the MOX project. DOE Does Not Require the Evaluation of the Effectiveness of Corrective Actions Taken DOE Order 413.3B does not require the evaluation of the results of corrective actions taken to address project management lessons learned that are identified during the course of capital asset projects such as MOX. According to DOE guidance and statements, officials track whether lessons identified through reviews or other efforts are implemented. For example, according to DOE’s standard-operating procedures for conducting external independent reviews and officials from DOE’s Office of Project Management, DOE staff conducting external independent reviews of projects should assess whether project teams are reviewing and incorporating applicable lessons learned. In addition, DOE project management officials told us that peer review recommendations and the corrective actions to be taken to address them are tracked until the closure of each recommendation. However, DOE has not evaluated whether corrective actions taken have led to the resolution of the problematic practices identified in the lessons learned because DOE Order 413.3B does not require this type of evaluation. According to key practices for lessons learned identified by the Center for Army Lessons Learned and us, a central component of a successful lessons-learned process is to establish a means to ensure that issues are being resolved as intended. The Center for Army Lessons Learned states that while not all issues require a formal process to resolve, there should be a process in place to identify and prioritize the most important things that need to be fixed. For example, this process could entail addressing only those problems that may necessitate the need for department-wide improvements, as some issues may be narrowly focused and be specific to one project or site. The Center for Army Lessons Learned further states that an organization’s ability to change behavior by implementing a lesson is ineffective unless the organization observes changes in behavior and verifies that the lesson is learned. Additionally, we have found that if agency management decides to take action to apply an identified lesson, then it should take subsequent action to observe that the change in behavior actually occurred and collect additional information to verify that the change had the desired effect. Although DOE Order 413.3B does not require DOE to evaluate the effectiveness of corrective actions other than those associated with peer reviews, other DOE orders and guidance require the evaluation of the effectiveness of other types of corrective actions. For example, DOE Order 226.1B requires that DOE’s organizations and contractors implement oversight processes that ensure they evaluate and correct relevant quality assurance problems on a timely basis to prevent their recurrence. In addition, DOE’s order and guide for implementing an effective quality assurance program highlight the importance of undertaking corrective actions to prevent the recurrence of problems, including determining the effectiveness of the corrective actions for significant problems. By developing requirements for evaluating the effectiveness of corrective actions taken in response to project management problems in capital asset projects, particularly those that necessitate the need for department-wide improvements, DOE—including NNSA—could verify that changes made as a result of lessons learned had the intended outcome as the agency does for contractors. Conclusions DOE and NNSA made changes that strengthened oversight of large capital asset projects. These changes helped NNSA better identify cost and schedule problems affecting the MOX project and contributed to NNSA’s decision to ultimately terminate the project. DOE’s Order 413.3B includes certain requirements for documenting and sharing project management lessons learned. However, the requirements in DOE Order 413.3B do not fully incorporate several key practices for lessons learned. For example, the order does not require that DOE or NNSA document project management lessons learned for capital asset projects consistently or that such lessons learned are shared in a timely manner. By developing requirements that clearly define how and where all project management lessons learned should be documented and saved to make them readily accessible across the department, such as in a database, DOE—including NNSA—could improve the existing lessons- learned process and enable future projects across the department to take advantage of experiences from past projects. In addition, because DOE Order 413.3B does not require lessons learned for capital asset projects to be submitted prior to the start of construction (CD-3), the department is limiting its ability to promptly evaluate and address early issues with projects as well as applying such lessons learned to other projects department-wide. By developing requirements for sharing project management lessons learned from the beginning of a project routinely and in a timely manner to improve DOE’s ability to identify and evaluate problematic practices and positive experiences, DOE—including NNSA—could help improve the success of future capital asset projects and avoid the problems the agency encountered on the MOX project. Moreover, while DOE tracks the implementation of certain project management lessons learned for capital asset projects, DOE Order 413.3B does not require that DOE—including NNSA—evaluate corrective actions identified outside the peer review process and taken in response to lessons identified to verify that the changes made had the desired effect. By developing requirements for evaluating the effectiveness of corrective actions taken in response to project management problems in capital asset projects, particularly those that necessitate the need for department-wide improvements, DOE could verify that changes made as a result of lessons learned had the intended outcome as the agency does for contractors. Recommendations for Executive Action We are making the following three recommendations to DOE: The Secretary of Energy, in coordination with DOE’s Office of Project Management and NNSA’s Office of Acquisition and Project Management, should develop requirements that clearly define how and where project management lessons learned for capital asset projects should be documented and saved to make them readily accessible across the department. (Recommendation 1) The Secretary of Energy, in coordination with DOE’s Office of Project Management and NNSA’s Office of Acquisition and Project Management, should develop requirements for sharing project management lessons learned for capital asset projects from the beginning of a project (i.e., prior to the start of construction at CD-3) routinely and in a timely manner to improve DOE’s ability to identify and evaluate problematic practices and positive experiences. (Recommendation 2) The Secretary of Energy, in coordination with DOE’s Office of Project Management and NNSA’s Office of Acquisition and Project Management, should develop requirements for evaluating the effectiveness of corrective actions taken in response to project management problems for capital asset projects, with a focus on those lessons that necessitate the need for department-wide improvements. (Recommendation 3) Agency Comments, Third-Party Views, and Our Evaluation We provided a draft of this report to DOE, NNSA, and MOX Services for review and comment. In written comments, which are reproduced in full in appendix IV, DOE concurred with the report’s recommendations and described actions that it intends to take in response to our recommendations. In response to our first recommendation, DOE intends to issue a policy memorandum by December 2019 and revise DOE Order 413.3B to identify the project management lessons learned repository and outline the kinds of information the repository will collect. In response to our second recommendation, DOE intends to issue a policy memorandum by December 2019 and revise DOE Order 413.3B to collect lessons learned as part of its peer review process. Because DOE Order 413.3B requires that peer reviews for projects of $100 million or greater be conducted once between CD-0 and CD-1, annually between CD-1 and CD-2, at least annually between CD-2 and CD-4, and more frequently for the most complex projects or those experiencing performance challenges, this action is responsive to our recommendation and should help DOE begin to identify lessons learned in a more routine and timely manner. In response to our third recommendation, DOE plans to revise the Project Management Risk Committee charter by assigning it the responsibility to qualitatively evaluate the effectiveness of corrective actions taken in response to project management lessons learned from projects with a total cost greater than $750 million having department-wide implications. We are encouraged that DOE agrees with our recommendation and view this change as a positive first step. However, this action may not fully address the recommendation. For example, the planned action states that the Project Management Risk Committee would evaluate the effectiveness of corrective actions for projects with total costs of $750 million or more, but there may be some lessons learned with applicability department-wide from projects that do not meet this cost threshold. Additionally, DOE’s planned action as described in its response does not discuss who would be responsible for evaluating the effectiveness of corrective actions or a timeline for performing the assessments. The Project Management Risk Committee has typically served as a review group and has not itself performed such evaluations. DOE and MOX Services also provided technical comments, which we incorporated in our report as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Energy, the Administrator of NNSA, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V. Appendix I: Objectives, Scope, and Methodology Our report examined (1) when the National Nuclear Security Administration’s (NNSA) project management oversight processes recognized cost and schedule problems at the Mixed Oxide Fuel Fabrication Facility (MOX) project and the actions the agency took to address them and (2) the extent to which the Department of Energy (DOE) requires that project management lessons learned from MOX and other projects be documented and shared. To address both objectives, we reviewed relevant documents from DOE, NNSA, and MOX Services, LLC (MOX Services), the contractor constructing the MOX project. We reviewed past reports by GAO and the National Academy of Sciences’ National Research Council to examine previously identified weaknesses in DOE project management, contractor performance, and federal oversight of individual projects, as well as DOE’s efforts to make improvements. We also reviewed DOE reports focused on analyzing the root causes of contract- and project- management issues affecting DOE and NNSA and identifying potential corrective actions and other general improvements. We visited the Savannah River Site to tour the MOX project while it was under construction and interviewed officials from NNSA’s MOX Project Management Office, including the federal project director, and representatives from MOX Services. We also monitored the status of the MOX project. To examine when NNSA’s project management oversight processes recognized cost and schedule problems at the MOX project and the actions the agency took to address them, we identified and reviewed DOE and NNSA documents outlining the agencies’ management and oversight roles and responsibilities and the processes the agencies used to monitor the cost and schedule of the MOX project. We also examined NNSA guidance and memorandums detailing the 2011 transition of oversight responsibilities for the construction of the MOX project from NNSA’s Office of Defense Nuclear Nonproliferation to its Office of Acquisition and Project Management and the effect this change had on NNSA’s efforts to oversee the project. In addition, we reviewed DOE, NNSA, and MOX Services documents, as well as independent reviews and assessments, concerning the performance and status of the MOX project. In particular, we reviewed a May 2014 report prepared for DOE that identified and analyzed the root causes behind the cost increases that affected the MOX project through 2012, after the formal approval of its cost and schedule estimates in 2007. We also reviewed surveillance reviews and a May 2013 assessment of the MOX contractor’s earned value management (EVM) system, which the contractor and NNSA used to monitor project performance and status, including cost and schedule, after construction began. Moreover, we examined project cost and budget information that DOE, NNSA, MOX Services, and others developed—such as the contractor’s September 2012 baseline change proposal and DOE’s August 2016 revised cost and schedule estimate—to determine when they began to identify the MOX project’s cost increases and schedule delays and why such problems might have occurred. We also reviewed reports by GAO and DOE’s Office of Inspector General that identified and discussed cost and schedule problems affecting the MOX project. Additionally, we interviewed officials from DOE and NNSA to discuss how and when they identified the MOX project’s cost and schedule problems. To examine the extent to which DOE requires that project management lessons learned from MOX and other projects be documented and shared, we reviewed DOE’s Order 413.3B, which outlines the primary set of project management requirements governing DOE and NNSA capital asset projects that have a total project cost of greater than $50 million. We also reviewed DOE guidance documents, such as those related to DOE Order 413.3B, to further understand DOE’s suggested approaches for meeting its existing lessons learned requirements. Similarly, we reviewed documents from NNSA and DOE’s Offices of Environmental Management and Science, such as those found in business-operating procedures and standard-operating policies and procedures, to examine how those documents supplement the lessons learned requirements included in DOE Order 413.3B. In addition, we collected examples of capital asset project-management lessons learned from DOE and NNSA, including those from the MOX project, from a variety of sources, such as lessons-learned reports, project peer reviews, entries stored in DOE’s Project Assessment and Reporting System (PARS II) and NNSA’s internal databases, monthly lessons-learned bulletins, and presentations, among others. To better understand lessons learned and their role within project management, we reviewed reports by GAO, the U.S. Army’s Center for Army Lessons Learned, and the Project Management Institute that identify and discuss key practices for lessons learned. We selected these sources because they are widely recognized for key practices on lessons learned. We then compared the project management lessons learned requirements outlined in DOE Order 413.3B against these key practices. We also discussed project management lessons learned requirements and processes with officials from DOE’s Offices of Environmental Management, Project Management, and Science and NNSA’s Office of Acquisition and Project Management. We conducted this performance audit from May 2017 to December 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Appendix II: Timeline of Selected Information and Events Pertaining to the MOX Project Appendix II: Timeline of Selected Information and Events Pertaining to the MOX Project Details DOE announced a plan to dispose of surplus, weapons-grade plutonium through a dual approach that would include constructing a facility for the purposes of converting the plutonium into mixed-oxide fuel for use in modified commercial reactors. The initial estimate for the MOX project—that is, not an approved baseline—totaled $1.4 billion, with completion of construction expected to be in September 2004. DOE awarded the contract for designing, constructing, and operating a MOX facility to the consortium of Duke, Cogema, Stone & Webster, LLC—now MOX Services, LLC, or MOX Services. According to a December 2005 DOE Inspector General report, in 1999, an independent team reviewed the MOX contract and warned of the potential for escalating costs because the contractor had no incentives to minimize costs nor penalties for overruns or poor performance. DOE announced that it would construct the MOX project (as well as two other facilities) at the Savannah River Site located in Aiken, South Carolina. A February 2001 independent cost estimate of the MOX contractor’s preliminary cost estimate for the MOX project concluded that it would cost about $2.4 billion to construct and operate it. The independent cost estimate concluded that it would cost about $1.1 billion to construct the facility. The National Nuclear Security Administration’s (NNSA) February 2002 report to Congress on the disposition of surplus defense plutonium at the Savannah River Site concluded that the facility component of the mixed-oxide fuel option identified would cost about $2.2 billion to implement over about 20 years. According to the report, about $1 billion of these costs would be for designing and constructing the facility, with construction being completed during fiscal year 2007. According to DOE’s fiscal year 2004 budget request, a preliminary estimate of the MOX project’s total cost totaled about $1.8 billion. A July 2004 independent review found that the MOX project had experienced a cost increase of about 300 percent for the design and development phase compared to what was preliminarily planned for in 1999, in part to due to a number factors, including design changes and underestimates. Moreover, the report cited the MOX project as an example of a DOE project greater than $500 million that should have had an approved performance baseline many years prior given that it had reached critical decision (CD)-1 approval, or the approval of alternative selection and cost range, in 1997. According to a December 2005 report by DOE’s Office of Inspector General, as of July 2005, NNSA’s not-yet-validated estimate for the design and construction of the MOX project was about $3.5 billion ($2.8 billion for construction). Phases Details In February 2006, DOE’s fiscal year 2007 budget request reported a preliminary estimate for the MOX project totaling about $3.6 billion, but the department reiterated that the estimate would be finalized following the completion of the project’s performance baseline. The request also noted that design costs for the MOX project increased from $243 million to $765 million, primarily due to the decision to fund some design work for gloveboxes and enhanced aqueous polishing during the design phase as opposed to the construction phase and increased design work to adapt the facility to handle and treat several tons of pure plutonium resulting from the cancelation of plutonium immobilization, which would have entailed incorporating plutonium into a corrosion-resistant ceramic matrix and then encasing the immobilized plutonium in glass along with highly radioactive nuclear wastes that already existed at DOE sites, thereby rendering the plutonium as inaccessible and unattractive for reuse in nuclear weapons. However, NNSA canceled this approach in 2002. A July 2006 external independent review of the MOX project’s preliminary cost and schedule estimate projected the MOX project’s total cost to be about $4.7 billion, with the project expected to be completed in April 2016. The review’s estimated total project cost reflected an increase of $352 million over the proposed total project cost of $4.3 billion due to increases in the cost of some construction activities and contingency. In February 2007, DOE’s fiscal year 2008 budget request reported that the revised total cost for the MOX project totaled about $4.7 billion and that the estimate was in the final stages of validation as part of the department’s critical decision process. The request stated that the revised cost was a change from the prior not-yet-validated $3.6 billion estimate in DOE’s fiscal year 2007 budget request, with over 50 percent of the $1.1 billion cost increase attributed to an increase in contingency funds for the project during construction and cold startup. Also in February 2007, responsibility for the MOX contract was officially transferred to the Savannah River Site Office. In April 2007, DOE formally approved a cost estimate, or baseline, for the MOX project of $4.8 billion and start of operations in September 2016. In August 2007, construction of the MOX project began. In May 2008, DOE certified the MOX contractor’s earned value management (EVM) system. A July 2008 independent project review identified a number of concerns, including that only one person was dedicated to the development and upkeep of the MOX project’s procurement status information and that the project’s procurement strategy would require additional procurement and engineering staff to meet future demands. In December 2008, as a result of funding reductions for fiscal year 2008, DOE approved a revised cost estimate for the MOX project of $4.9 billion and a 1-month delay in the start of operations to October 2016. According to a July 2009 report, the MOX contractor’s 2009 annual forecasted estimate for completing the MOX project totaled approximately $4.4 billion, an increase of about $283.8 million from the 2008 annual forecasted estimate. In May 2010, an independent review of the MOX contractor’s EVM system found that the contractor’s performance data could not be used to accurately assess the cost performance of the project, in part because the contractor was inappropriately using management reserve funds to cover cost overruns. The MOX contractor began to implement a number of corrective actions in response to the report’s findings. According to an August 2010 report, the MOX contractor’s 2010 annual forecasted estimate for completing the MOX project totaled approximately $4.6 billion, an increase of about $207.1 million from the 2009 annual forecasted estimate. Phases Details In February 2011, DOE’s Office of Acquisition and Project Management—now the Office of Project Management—changed the overall status of the MOX project from green to yellow, indicating that the project was at risk of breaching its approved cost estimate (i.e., performance baseline). A May 2011 project peer review found that the MOX project faced expected cost growth and would be challenged in identifying approximately $364 million in cost savings necessary to deliver the project at its total project cost (of $4.9 billion). A June 2011 follow-on to the May 2010 independent review of the MOX contractor’s EVM system found that the project was likely to exceed the total project cost by anywhere from $104 million to $699 million, with an estimated most likely cost overrun of $493 million. Nonetheless, DOE recertified the MOX contractor’s EVM system after the MOX contractor completed a number of corrective actions. According to a July 2011 report, the MOX contractor’s 2011 annual forecasted estimate for completing the MOX project totaled approximately $4.7 billion, an increase of about $142.4 million from the 2010 annual forecasted estimate. In January 2012, NNSA directed the MOX contractor to add additional scope for plutonium metal oxidation capability and to include updates to the project’s current cost and schedule projections, with a baseline change proposal due by the end of May 2012. A March 2012 project review found that the MOX project’s cost and schedule baselines had a very low probability of being met, and estimated that the total project cost was likely underestimated by anywhere from $600 to $900 million when compared to the project’s approved total cost of $4.9 billion. The review team recommended that the project should develop an updated and more realistic baseline. Also in March 2012, DOE changed the overall status of the MOX project from yellow to red, indicating that the project was expected to breach its approved cost estimate (i.e., its performance baseline). A July 2012 project peer review found that the MOX project’s likely total project cost would fall within the range of $6.9 billion to $7.3 billion as opposed to the project’s approved total cost of $4.9 billion. In September 2012, the MOX contractor submitted its revised baseline change proposal to update the MOX project’s cost and schedule projections, including additional scope of work that would provide the MOX project with a plutonium metal oxidation capability, referred to as direct metal oxidation. According to the contractor’s proposal, it would cost about $7.4 billion to complete the MOX project without the direct metal oxidation by November 2019. The addition of the direct metal oxidation scope of work would cost an additional $262.3 million, which would be completed in June 2023 after the completion of MOX project and the start-up of operations by November 2019. Phases Details In April 2013, DOE’s fiscal year 2014 budget request proposed a slowdown of construction of the MOX project while NNSA took steps to assess alternative plutonium disposition strategies. According to the request, NNSA cited the increase to the contractor’s total estimated cost for the project and the budget environment as factors in its decision to pursue a slowdown of the MOX project while conducting an assessment of potential alternative plutonium disposition strategies. According to NNSA, a May 2013 estimate prepared by the U.S. Army Corps of Engineers estimated that, not including contractor fee, it would cost $9.4 billion to construct the MOX project by 2024 at an annual funding level of $630 million. According to NNSA, a June 2013 estimate prepared by the MOX contractor estimated that it would cost between $8.5 and $9.7 billion to construct the MOX project, with completion from 2023 to 2032 depending on whether the annual funding level totaled $350 million or $500 million. In September 2013, NNSA estimated it would cost about $10.5 billion to construct the MOX project by 2027 at an annual funding level of $500 million. According to NNSA, a November 2013 estimate prepared by the U.S. Army Corps of Engineers estimated that it would cost from $10 to $11.7 billion to construct the MOX project, with completion from 2026 to 2036 depending on whether the annual funding level totaled $350 million or $500 million. In March 2014, DOE’s fiscal year 2015 budget request stated that ongoing analysis led to the determination that the MOX project would be significantly more expensive than anticipated and concluded that, due to cost increases, the MOX approach was not viable within available resources. The request, therefore, called for placing the facility in cold stand-by so NNSA could further study more efficient options for plutonium disposition. A May 2014 root cause analysis report found that some of the cost drivers that contributed to the MOX project’s cost increases since 2007 included not having sufficiently experienced project teams in place, basing the approved cost and schedule estimates on incomplete front- end planning, not sufficiently developing designs to support the project’s fast-track procurement and construction, experiencing greater than expected inefficient execution of construction activities, not implementing effective corrective actions, and not adequately applying federal oversight to identify and address project performance issues Also in May 2014, the DOE Office of Inspector General reported continuing concerns about the achievability of the estimated cost and completion date for the MOX project. The report also noted that the MOX project no longer had an approved cost and schedule estimate and in light of the project continuing to receive significant funding, recommended that the MOX contractor develop a new cost and schedule estimate. In September 2014, in light of certain insufficient project data, NNSA directed the MOX contractor to conduct a review to determine and validate the work completion status—that is, state of completeness—for all commodities being installed in the MOX project. In December 2014, both the Carl Levin and Howard P. McKeon National Defense Authorization Act for Fiscal Year 2015 and the Consolidated and Further Continuing Appropriations Act, 2015 directed DOE to continue construction and project or program support activities related to the MOX project. However, the National Defense Authorization Act also directed DOE to report on, among other things, alternatives to the MOX project, including cost estimates for each alternative, and how such alternatives would conform to the Plutonium Management and Disposition Agreement. Phases Details In February 2015, DOE’s fiscal year 2016 budget request called for the continued construction of the MOX project, in part because all four congressional committees of jurisdiction directed that construction on the MOX project continue in fiscal year 2015 while NNSA conducted additional cost studies and technology alternative studies. In March 2015, NNSA’s MOX Project Management Office assessed the MOX contractor’s use of level of effort versus the discrete method of earned value and determined a disproportionate use of level of effort—around 56 percent—was masking the performance of the contractor’s discrete work and therefore affecting the accurate measurement of the project’s progress. In April 2015, the Aerospace Corporation completed a report on the MOX project and estimated that the MOX project’s total cost would be about $21.5 billion, with projected completion in 2045 at an annual funding level of $500 million. In June 2015, the MOX contractor finished its completeness verification review and found that it had over-reported on the results of certain commodities being installed in the MOX project. As a result of this review, the MOX contractor revised the amount of earned value claimed for these commodities to address the over-reporting and provide a more realistic accounting of the selected commodities. In February 2016, DOE’s fiscal year 2017 budget request proposed terminating the MOX project in favor of the dilute and dispose option as the path forward for the disposition of the nation’s surplus, weapons-grade plutonium. According to the request, the MOX project was found to be significantly more expensive than anticipated and would require approximately $800 million to $1 billion annually for decades. A May 2016 report prepared for the MOX contractor by High Bridge Associates, Inc., estimated that completing the construction of the MOX project could cost about $5.2 billion and be completed in 10 years, with an annual funding level of about $520 million. In July 2016, the MOX contractor submitted its annual forecasted estimate for completing construction of the MOX project and estimated the total project cost to be about $10 billion, with completion in 2029, with an annual funding level of $350 million. In August 2016, DOE issued an updated performance baseline estimating that it would cost approximately $17.2 billion to complete construction of the MOX project by 2048 assuming an annual funding level of $350 million. DOE further estimated that it would cost about $14.3 billion to complete construction of the MOX project by 2035 assuming an annual funding level of $500 million. In October 2016, DOE rescinded the MOX contractor’s EVM system certification of compliance in response to an August 2016 surveillance review that identified material non- compliances such as the overstatement of earned value and percentage complete. Phases Details A February 2017 report by the U.S. Army Corps of Engineers found that there is likely to be a substantial amount of rework at the MOX project but noted that the magnitude of the likely rework has yet to be determined. The report stated that some of the rework is attributed to design constructability issues as well as procuring, fabricating, and completing work out of sequence. In May 2017, DOE’s fiscal year 2018 budget request reiterated for the second consecutive year, a plan to terminate the MOX project in favor of pursuing the dilute and dispose option for plutonium disposition. Also in May 2017, a DOE Office of Inspector General report stated that NNSA was not aware of the total cost of rework at the MOX project because the time and cost of rework were not definitively tracked prior to fiscal year 2014. In December 2017, section 3121 of the National Defense Authorization Act for Fiscal Year 2018 authorized the Secretary of Energy to terminate the MOX project if, among other things, the Secretary certified that the remaining life-cycle cost for an alternative option for carrying out plutonium disposition would be less than approximately half of the estimated remaining life-cycle cost of carrying out the plutonium disposition approach utilizing the MOX project. In February 2018, DOE’s fiscal year 2019 budget request reiterated for the third consecutive year a plan to terminate the MOX project in favor of pursuing the dilute and dispose option for plutonium disposition. In May 2018, the Secretary of Energy waived existing requirements to continue MOX construction, but the state of South Carolina obtained an injunction in federal district court temporarily blocking the waiver in June, which NNSA subsequently appealed. In October 2018, a federal appellate court granted a stay of the federal district court’s injunction that prohibited termination of the MOX contract and cessation of construction operations. NNSA subsequently issued a notice of termination to the MOX contractor. Appendix III: Selected GAO Recommendations from Prior Reports We have made numerous agency recommendations in prior reports to improve contract and project management in the Department of Energy (DOE) and the National Nuclear Security Administration (NNSA). Some reports contain recommendations for department and agency policies, and others address project management problems for specific projects or also address other agencies besides NNSA. A description of some of our key recommendations, with the status of implementation as of December 2018, is provided below in table 2. For the most up-to-date status of these agency recommendations, see our website: http://www.gao.gov. Appendix IV: Comments from the Department of Energy Appendix V: GAO Contact and Staff Acknowledgments GAO Contact Staff Acknowledgments In addition to the contact named above, Hilary Benedict (Assistant Director), Rodney Bacigalupo, Antoinette Capaccio, Tara Congdon, Pamela Davidson, Richard P. Johnson, Eleni Orphanides, Kevin Remondini, Karen Richey, Sara Sullivan, and Tatiana Winger made key contributions to this report.
target: The MOX project, located at DOE's Savannah River Site in South Carolina and overseen by NNSA, experienced significant cost increases and schedule delays following the start of construction in 2007. After spending nearly $6 billion, NNSA terminated the project in October 2018. While DOE and NNSA have made some recent progress, they have historically struggled to complete, within their original cost and schedule estimates, other major construction projects intended to help maintain the nuclear security complex. GAO was asked to review issues related to oversight of the MOX project. This report examines (1) when NNSA's project management oversight processes recognized cost and schedule problems at the MOX project and the actions the agency took to address them and (2) the extent to which DOE requires that project management lessons learned from MOX and other projects be documented and shared. GAO reviewed agency documents, visited the MOX project, and interviewed DOE and NNSA officials and representatives of the MOX contractor. The Department of Energy's (DOE) National Nuclear Security Administration (NNSA) has strengthened its oversight of the Mixed Oxide Fuel Fabrication Facility (MOX) project since 2011 and, as a result, began recognizing cost and schedule problems. The project, begun in 1997, was intended to dispose of large quantities of weapons-grade plutonium no longer required for national security. Prior to 2011, NNSA's project staff failed to recognize signs that the project would not be completed on time or within its approved cost. An independently conducted analysis, prepared in 2014 in response to a GAO recommendation, determined that NNSA staff did not recognize early problems because they were inexperienced in project management. To strengthen oversight, NNSA in late 2010 and 2011 began actions, such as conducting additional reviews and transferring oversight of the project to a newly established office specializing in project management. NNSA continued to identify the contractor's performance problems, such as the lack of credible, reliable cost and schedule data. These continued problems contributed to NNSA's decision to terminate the project. DOE requires that project staff document and share project management lessons learned on capital asset projects like the MOX project, but not all lessons are to be documented consistently or shared in a timely manner. GAO found that DOE's and NNSA's offices document project management lessons learned differently and that not all of the documented lessons learned are readily accessible to other staff. Additionally, GAO found that DOE does not require that project staff share lessons learned for capital asset projects until the start of construction, which can occur many years after the start of the project. Under key practices, such lessons should be stored in a logical, organized manner, be easily retrievable, and be submitted in a timely manner (see fig.). By developing requirements that clearly define how and where project management lessons learned should be documented and requiring that the lessons be shared in a timely manner, DOE could improve its lessons-learned process and help improve the success of future capital asset projects. Also, for capital asset projects, DOE does not require the evaluation of the results of all corrective actions to respond to lessons learned to ensure that problems are resolved, consistent with key practices. By developing requirements to evaluate the effectiveness of corrective actions, DOE could better verify whether the actions had the intended outcome.