DOE Project Management: NNSA Needs to Clarify Requirements for Its Plutonium Analysis Project at Los Alamos
Released: 2016-08-09T00:00:00

Letter

Background

This section describes: (1) Los Alamos’s role in NNSA’s nuclear security enterprise; (2) DOE and NNSA programs’ plutonium analysis needs at Los Alamos; (3) DOE and NNSA project management orders and policies; and (4) best practices for project cost and schedule estimating.

Los Alamos’s Role in NNSA’s Nuclear Security Enterprise

NNSA is responsible for managing the nation’s nuclear security missions: ensuring a safe, secure, and reliable nuclear deterrent; achieving designated reductions in the nuclear weapons stockpile; and supporting the nation’s nuclear nonproliferation efforts. NNSA directs these missions but relies on management and operating contractors to carry them out and manage day-to-day operations at each of its eight sites that comprise the agency’s nuclear security enterprise. These sites include laboratories, production plants, and a test site. Together, these sites implement NNSA’s stockpile stewardship program. Specifically, under this program, NNSA annually assesses the nation’s nuclear weapons stockpile and (1) determines which components, including the pit, will need refurbishment to extend each weapon’s life; (2) designs and produces the necessary components; (3) installs components in the weapons; and (4) certifies that the changes do not adversely affect the safety and reliability of the weapons. The 2010 Nuclear Posture Review—which outlines U.S. nuclear policy, strategy, capabilities, and force posture— identified long-term stockpile modernization goals for NNSA that include sustaining a safe, secure, and effective nuclear arsenal through life extension programs and investing in a modern infrastructure. NNSA has identified the revised CMRR project as critical to its infrastructure modernization efforts.

Of NNSA’s eight sites, the Los Alamos National Laboratory in New Mexico houses most of the nation’s capabilities for plutonium analysis in support of its nuclear weapons mission. Los Alamos also has a broader plutonium-related research and analysis mission. For example, Los Alamos conducts basic and applied research in the chemistry of plutonium and other radiological materials for the study of nuclear materials, including nuclear materials separation, processing, and recovery.

Currently there are three main facilities at Los Alamos that support plutonium analysis:

Chemistry and Metallurgy Research facility. This facility became operational in 1952 and houses equipment for performing plutonium analysis. With this equipment, NNSA conducts analysis activities that support: the production, development, and testing of nuclear weapon pits, programs to extend the life of nuclear weapons in the stockpile, and efforts to dismantle surplus nuclear weapons. NNSA continues to operate some plutonium analysis equipment in the facility, but the agency has committed to ending plutonium operations in the facility by the end of 2019.

Plutonium Facility 4. This facility began operations in 1978 and is the nation’s only high-hazard, high-security, fully operational plutonium facility that produces pits. Plutonium Facility 4 is also used to support the production of plutonium-238 heat sources used for NASA spacecraft missions as well as the development of methods for fabricating advanced nuclear fuels.

Radiological Laboratory Utility Office Building (radiological lab).

NNSA began operations in the radiological lab in 2014. NNSA built the radiological lab—consisting of office space, training areas, utilities, and laboratory space for analysis—to complement the previously approved CMRR nuclear facility. The radiological lab has the capacity to handle small plutonium samples for use in analytical chemistry analysis to support plutonium program missions in Plutonium Facility 4.

DOE and NNSA Plutonium Analysis Needs at Los Alamos

Several DOE and NNSA program offices conduct plutonium analysis in the three main plutonium facilities at Los Alamos. The following are the primary users of plutonium analysis capabilities at Los Alamos.

NNSA Office of Defense Programs

This office is responsible for maintaining the reliability, security, and safety of the nuclear weapons stockpile by assessing the reliability of existing nuclear weapon pits and producing new pits to replace those destroyed in the testing process, among other activities. The majority of funding and scope associated with plutonium work at Los Alamos supports the Office of Defense Programs’ missions. Pit production relies on plutonium analysis to produce war reserve pits—pits that are certified for inclusion in the nuclear weapons stockpile—in support of life extension programs. Los Alamos has used its analytical chemistry equipment to support pit production more than to support all other program activities combined. From 2007 to 2012, NNSA produced a limited number of war reserve pits at Los Alamos; according to NNSA, by 2011 the agency had demonstrated a production rate of up to 10 pits per year. In 2008, the Nuclear Weapons Council—a body that serves as the focal point of the Department of Defense and DOE interagency activities to maintain the nation’s nuclear weapons stockpile—established the requirement for NNSA to develop pit production capabilities of 50 to 80 pits per year.

We found in March 2016 that modernizing NNSA’s pit production capacity, including implementing the revised CMRR project, is fundamental to supporting a multi-billion-dollar life extension program.

NNSA’s Fiscal Year 2016 Stockpile Stewardship and Management Plan contains the agency’s plan for meeting the weapons council’s requirement and supporting life extension programs. According to the fiscal year 2016 plan, NNSA will develop the capability to produce at Los Alamos an increasing number of new pits over time. These pits will be of a different type than the pits produced earlier, so Los Alamos has started a development process to establish a pit production capability for a new pit type. Beginning in fiscal year 2024, NNSA plans to be able to produce 10 war reserve pits per year, increasing to 30 war reserve pits per year in fiscal year 2026, and 50 to 80 war reserve pits per year by 2030 (see table 1).

NNSA’s ARIES supports the disposition of surplus weapons-grade plutonium by disassembling pits and converting the plutonium into a plutonium oxide. NNSA established ARIES in Plutonium Facility 4 as a technology development and demonstration project for pit disassembly and conversion and has used plutonium analysis equipment there to analyze plutonium samples from the oxide converted from existing pits. According to NNSA, ARIES currently conducts most of its plutonium analysis that supports disassembly and conversion at NNSA’s Savannah River Site in South Carolina. However, ARIES will likely need more analysis equipment in Plutonium Facility 4 if its mission at Los Alamos is expanded, according to NNSA.

DOE Office of Nuclear Energy’s Office of Space and Defense Power Systems

This office provides plutonium-238 heat sources for electric generators used on NASA spacecraft and for national security applications. The plutonium-238 heat sources are also used to heat critical components on NASA spacecraft. The office relies on plutonium analysis equipment to support its work. The office recently stopped operating in the Chemistry and Metallurgy Research facility and installed its plutonium analysis equipment in Plutonium Facility 4. This DOE office shares the plutonium-238 analysis equipment with NNSA to support work examining existing pits.

DOE and NNSA Project Management Orders and Policies

DOE Order 413.3B governs NNSA’s capital asset acquisition activities. The order establishes the critical decision (CD) process. This process breaks down capital asset acquisition into project phases that progress from a broad statement of mission need into requirements that guide project execution, through design and construction, and conclude with an operational facility. Each phase ends with a major approval milestone—or “critical decision”—and each critical decision requires the successful completion of the preceding phase. DOE’s capital asset acquisition process, or critical decision process, is depicted in figure 1.

DOE Order 413.3B provides direction for preparing a mission need statement at CD-0 (identify need). According to the order, a mission need statement identifies the capability gap between the current state of a program’s mission and the mission plan and is part of the first phase of identifying and executing a capital asset project. The mission need, however, should not identify a particular solution such as equipment, facility, or technology.

In addition, Order 413.3B and NNSA’s business operating procedure for creating program requirements documents for capital asset projects (NNSA’s requirements policy) provide direction for preparing and updating a program requirements document. According to Order 413.3B, requirements identified in a program requirements document are statements that define the ultimate goals a project must satisfy. According to A Guide to the Project Management Body of Knowledge, defining requirements for a project is a key step in developing a project because the requirements dictate what is included—and what is not included—in a project’s scope of work. Order 413.3B states that NNSA must prepare a program requirements document for capital asset projects at CD-0. Order 413.3B directs project managers to implement NNSA’s requirements policy when preparing the program requirements document at CD-0. According to this policy, a project’s requirements should include a specific type of requirement called a key performance parameter. Order 413.3B states that key performance parameters define how well a project will perform its functions, and appropriate parameters are those that express performance in terms of capacity, throughput (i.e., the amount of material or items passing through a system or process), or processing rate, among others. These parameters, among other things, identify vital characteristics, functions, or design bases that, if changed, would have a major impact on facility or system performance, scope, schedule, cost, or risk.

Under Order 413.3B, the first two decision points—CD-0 (identify need) and CD-1 (select alternative)—span the analysis of alternatives process. The use of the analysis of alternatives process is a key first step to help ensure that the selected alternative best meets the agency’s mission need. The majority of the analysis of alternatives process is conducted during the conceptual design phase and ends with CD-1 approval.

Best Practices for Project Cost and Schedule Estimating

To provide assistance in preparing high-quality cost and schedule estimates, we compiled best practices used throughout government and industry in two guides. In March 2009, we issued our Cost Estimating and Assessment Guide (cost guide). Drawing from federal cost estimating organizations and industry, our cost guide provides best practices about the processes, procedures, and practices needed for ensuring development of high-quality—that is, reliable—cost estimates. A high-quality cost estimate helps ensure that management is given the information it needs to make informed decisions. The cost guide identifies the following four characteristics of a high-quality cost estimate: (1) comprehensive, (2) well documented, (3) accurate, and (4) credible.

We issued our Schedule Assessment Guide (schedule guide) in May 2012 as a companion to the cost guide. A well-planned schedule is a fundamental management tool that can help government programs use public funds effectively by specifying when work will be performed in the future and measuring program performance against an approved plan. The schedule guide identifies 10 best practices for developing and maintaining a reliable, high-quality schedule. For example, the first best practice is for the schedule to capture all activities necessary to complete the project.

In May 2016, DOE published an update to Order 413.3B that institutionalizes recent policies that the Secretary of Energy had established in two memorandums, including one dated June 2015 that related to, among other things, cost and schedule estimating best practices. Specifically, the updated order incorporates policies first established in the June 2015 memorandum that state (1) cost estimates shall be developed, maintained, and documented in a manner consistent with methods and best practices identified in our cost guide, DOE guidance, and applicable acquisition regulations and Office of Management and Budget guidance; and (2) projects shall develop and maintain an integrated master schedule in a manner consistent with the methods and best practices identified in our schedule guide and in the National Defense Industrial Association’s Planning and Scheduling Excellence Guide.

NNSA Did Not Identify the Plutonium Analysis Capacity That Its Revised CMRR Project Should Provide, So the Project May Not Meet DOE and NNSA Needs NNSA’s Requirements for the Revised CMRR Project Did Not Define the Plutonium Analysis Capacity the Project Should Provide

NNSA defined a set of requirements for the revised CMRR project, but these requirements did not include key performance parameters, such as the plutonium analysis capacity that the project should provide. NNSA has initiated actions that could increase the project’s plutonium analysis capacity, but these actions create risk for meeting the project’s estimated cost and schedule. Even with the new actions that NNSA has initiated, the agency is not designing the revised CMRR project to meet all DOE and NNSA plutonium analysis needs at Los Alamos.

NNSA defined a set of requirements for the revised CMRR project, but these requirements did not include key performance parameters, as called for by NNSA’s requirements policy. Order 413.3B states that key performance parameters define how well a project will perform its functions, and appropriate parameters are those that express performance in terms of capacity or throughput, among others. According to NNSA documents, the revised CMRR project’s primary function is to provide plutonium analysis capabilities in support of NNSA’s stockpile stewardship program. The stockpile stewardship program includes producing new plutonium pits, but the revised CMRR project’s requirements document did not define the annual rate of pit production that the project should support in a key performance parameter. NNSA’s Fiscal Year 2016 Stockpile Stewardship and Management Plan stated that the agency will increase its capability to produce new pits over time to support life extension programs: 10 pits per year in 2024, 30 pits per year in 2026, and 50 to 80 pits per year by 2030. According to NNSA documents, the agency needs adequate quantities of plutonium analysis equipment, namely analytical chemistry equipment, and the space in which to house the equipment to support its planned pit production rates. The higher the pit production rate, the more analytical chemistry equipment will be needed. The project’s requirements document included a list of plutonium analysis equipment that the project will provide and the capacity that each piece of equipment will provide—expressed as a number of plutonium samples processed per year. This information indicates the analysis capacity the project may provide but does not indicate the extent to which the project will support broader needs, including pit production.

NNSA officials and Los Alamos contractor representatives said that the program requirements document for the revised CMRR project does not contain a pit production-related performance parameter because NNSA tasked the revised project only with replacing plutonium analysis equipment that had been located in the Chemistry and Metallurgy Research facility. They explained that NNSA did not specifically task the revised project with supporting specific pit production levels. Instead, contractor representatives told us that they created the list of analysis equipment to include in the revised project based on the capabilities that needed to be transferred from the Chemistry and Metallurgy Research facility and the amount of available space in the radiological lab and Plutonium Facility 4. NNSA officials told us they do not plan to identify key performance parameters for the revised CMRR project until CD-2 and that they do not plan to establish a key performance parameter that identifies pit production rates for the revised CMRR project to support in the near future. These officials agreed, however, that having sufficient plutonium analysis capacity is critical to supporting pit production.

Not identifying key performance parameters in the program requirements document for the revised CMRR project, including a parameter related to pit production, likely contributed to the project potentially providing insufficient plutonium analysis capacity to support planned pit production rates. In 2015, NNSA’s contractor at Los Alamos conducted an analysis, calculating the pit production rate that the revised CMRR project’s analysis equipment was expected to support, based on the amount of equipment, location, and the amount of plutonium allowed in the radiological lab and Plutonium Facility 4. The contractor’s analysis showed that the revised CMRR project, as approved at CD-1 in August 2014, may not provide sufficient analysis capacity to support a 10-pits-per-year production rate. Although NNSA previously demonstrated that it could produce up to 10 war reserve pits per year at Los Alamos, the pits were produced in Plutonium Facility 4 with support from the Chemistry and Metallurgy Research facility. However, according to contractor representatives, the radiological lab has stricter limits on the amount of plutonium allowed in it than the Chemistry and Metallurgy Research facility does, so supporting pit production in Plutonium Facility 4 with the radiological lab limits analysis capacity to support pit production.

Not identifying a key performance parameter related to pit production in the requirements document may have also contributed to different understandings of how the project will meet critical mission requirements. For example, NNSA and DOE management officials we interviewed have drawn their own, widely different conclusions about the extent to which the revised CMRR project will support pit production. One NNSA official involved in overseeing the project told us that the analysis equipment that will be installed under the revised CMRR project would provide the analysis capacity to support a production rate of 30 pits per year. A senior NNSA official said the project would provide the same capabilities as the previously approved CMRR project that included the nuclear facility but said that he was not aware of the revised CMRR project’s expected analysis capacity. A senior DOE official said that, when he reviewed the CD-1 documentation and participated in meetings regarding the August 2014 CD-1 approval for the revised CMRR project, he was not aware that the project was needed to support pit production, and another DOE official responsible for reviewing the project said the revised project would support a production rate of about 10 pits per year.

We have previously found that poorly defined, incomplete, or missing requirements make it difficult to hold projects accountable, result in programs or projects that do not meet user needs, and can result in cost and schedule growth. Notably, the memorandum approving CD-1 for the revised CMRR project states that one of the main reasons that the CMRR nuclear facility experienced significant cost and schedule growth was because of poorly defined requirements for the facility. By updating the program requirements document for the revised CMRR project to identify a key performance parameter that describes the plutonium analysis capacity that the revised CMRR project is required to provide to support specific pit production rates, NNSA could clarify the extent to which the project will support planned pit production, better enabling the agency to identify whether it will need to acquire additional plutonium analysis equipment or space by other methods.

NNSA Is Pursuing Actions to Increase Analysis Capacity

Even though NNSA did not identify pit production rates in a key performance parameter in the program requirements document for the revised CMRR project to support, the agency has taken steps since the August 2014 CD-1 approval that may increase the plutonium analysis capacity that the project will provide. For example, although the 2015 contractor analysis determined that the revised CMRR project as approved in August 2014 may not support a 10 pits-per-year production rate, the analysis also found that the project could support such a production rate at project completion in 2024 if the contractor successfully implements efforts to increase the efficiency of working in the radiological lab. These efforts involve potentially increasing the project’s analysis capacity by reducing the amount of plutonium used in each analysis, in turn allowing scientists to conduct more analyses at a time. According to contractor representatives we interviewed, they have not determined if these efforts will be successful.

NNSA has initiated another action that could increase the plutonium analysis capacity that the revised CMRR project will provide, according to contractor documents. In November 2015, DOE approved a restructuring of the revised CMRR project by splitting the Plutonium Facility 4 Equipment Installation subproject, one of the two subprojects that comprised the revised CMRR project when it was approved in August 2014, and adding a new subproject that would upgrade the radiological lab from a radiological facility to a Hazard Category 3 nuclear facility. Upgrading the radiological lab, according to NNSA and contractor documents, would involve (1) renovating the radiological lab to accommodate higher levels of plutonium, and (2) installing and operating a set of plutonium analysis equipment in the radiological lab that was otherwise slated to go into Plutonium Facility 4 as part of the revised CMRR project. In May 2015, NNSA and its contractor conducted a business case analysis of the upgrade that stated that the benefits of increasing work performed in the radiological lab instead of in Plutonium Facility 4 include increased operational efficiency and therefore increased analysis capacity. In addition, the business case analysis stated that the proposal offers the benefit of avoiding using the high-hazard, high- security space in Plutonium Facility 4 for analysis operations that do not need that level of safeguarding. According to the separate 2015 contractor analysis, conducting more plutonium analysis in the radiological lab instead of in Plutonium Facility 4—if combined with successful efforts to reduce the amount of plutonium used in each analysis—would likely increase the project’s plutonium analysis capacity enough to support a 30-pits-per-year production rate. In a best case scenario, the analysis found the project might be able to support production at 80 pits per year.

NNSA’s efforts to increase the plutonium analysis capacity of the revised CMRR project, however, may create risks for meeting the project’s estimated cost and schedule. NNSA and DOE officials told us that the agency has not increased an existing facility’s hazard category before and that NNSA is working to identify all of the steps that will be involved in upgrading the radiological lab to a Hazard Category 3 nuclear facility. Also, the May 2015 business case analysis described some of the risks associated with the upgrade, some of which carry potentially significant cost and schedule effects if realized. For example, the business case analysis stated that it is possible that a forthcoming nuclear safety analysis could require modifying the building’s ventilation system or increasing the safety rating of the gloveboxes used in the radiological lab for handling plutonium. The business case analysis found that requiring such modifications could negatively affect the upgrade’s schedule because, for example, the contractor has already started procuring gloveboxes for the radiological lab and changing their safety rating would require modifying the procurement or the gloveboxes themselves. Also, project costs could increase to the point where NNSA might decide it is not affordable to pursue the upgrade. If NNSA does not pursue the upgrade or the upgrade does not result in increasing the revised project’s analysis capacity enough to support pit production rates of 50 to 80 pits per year by 2030 to support life extension programs, NNSA officials told us they do not currently have an alternate plan for acquiring and locating the remaining analysis equipment needed to support those production rates. Given the uncertainty surrounding the revised CMRR project’s ability to support pit production rates of 50 to 80 pits per year, having a plan for providing sufficient analysis capacity may better enable NNSA to meet its pit production plans and support planned life extension programs critical to the agency’s modernization efforts.

NNSA Is Not Designing the Revised CMRR Project to Directly Meet Plutonium Analysis Needs of All DOE and NNSA Programs at Los Alamos

Even considering the actions that NNSA is pursuing to increase analysis capacity, the agency is designing the revised CMRR project to meet the Office of Defense Programs’ plutonium analysis needs for its stockpile management mission and not the analysis needs of other NNSA and DOE programs that conduct plutonium analysis work at Los Alamos. This is counter to the revised CMRR project’s requirements document for CD-1. More specifically, the first project requirement states that the revised CMRR project must provide infrastructure and equipment needed to support program offices with missions assigned to Los Alamos that involve plutonium analysis operations, including stockpile support and national security and science stewardship. As stated, this requirement appears to include all DOE and NNSA program offices with plutonium analysis needs assigned to Los Alamos, including but not limited to the needs of the program sponsor—the Office of Defense Programs—and also other program offices, including the NNSA Office of Defense Nuclear Nonproliferation’s ARIES and the DOE Office of Nuclear Energy’s Office of Space and Defense Power Systems. The revised CMRR project’s first requirement also directs readers to an appendix to the program requirements document that shows a list of programs—both within and outside of the Office of Defense Programs—that use plutonium analysis capabilities at Los Alamos mapped against specific types of plutonium analysis techniques. NNSA’s requirements policy states that mission requirements should be a comprehensive set of what the project must provide to satisfy the mission need. In addition, A Guide to the Project Management Body of Knowledge states that mission requirements provide the basis for defining a project’s scope.

According to NNSA officials and contractor representatives, however, the revised CMRR project’s scope does not include dedicated analysis equipment that specifically meets the needs of DOE and NNSA programs outside of the Office of Defense Programs. These officials and contractor representatives said that these programs’ plutonium analysis needs are not specifically included in the first project requirement for the revised CMRR project. They said that, as the project sponsor, it is Defense Programs’ requirements that will dictate the specific analysis equipment to include in the project’s scope of work. Officials we interviewed with ARIES and the Office of Space and Defense Power Systems said they are not customers of the revised CMRR project and that they did not provide input about their plutonium analysis requirements to inform the revised CMRR project’s program requirements document. The Space and Defense Power Systems officials said that they have acquired dedicated plutonium analysis equipment to meet their plutonium analysis needs separately at Los Alamos, and the ARIES officials said they have moved most of their plutonium analysis work to another site but that they may seek to increase their analysis work at Los Alamos in the future if their mission changes.

Contractor representatives explained that they included programs outside of Defense Programs in the revised CMRR project’s program requirements document to be comprehensive and show that the project may support customers beyond Defense Programs. They said that other programs may use the plutonium analysis equipment installed as part of the revised CMRR project, if and when the equipment is available. However, writing the first project requirement in a broad manner may provide the impression to senior managers and other stakeholders that the revised CMRR project will include the dedicated analysis equipment and space necessary to meet the plutonium analysis needs of programs other than Defense Programs. It also adds to the uncertainty surrounding the exact nature of the plutonium analysis capacity that the project will provide. By updating the program requirements document for the revised CMRR project to clarify whether the revised project will provide plutonium analysis equipment to meet the needs of DOE and NNSA programs other than those in the Office of Defense Programs, managers and stakeholders may have a clearer understanding of the extent to which the project will satisfy DOE and NNSA programs’ needs.

NNSA Did Not Define Key Parameters or Program Requirements for the Plutonium Modular Approach, Therefore It Is Unclear If It Will Meet Analysis Needs

In November 2015, DOE approved the mission need (CD-0) for the Plutonium Modular Approach to examine building modular nuclear facilities to support plutonium work at Los Alamos. However, it is unclear whether the Plutonium Modular Approach will help meet DOE and NNSA programs’ plutonium analysis needs at Los Alamos because NNSA did not identify key performance parameters, as with the revised CMRR project, or program-specific requirements in the mission need documentation. Further, the Plutonium Modular Approach’s statement of mission need prematurely identified a specific solution, which is counter to Order 413.3B.

NNSA Did Not Specify Key Parameters, Including Plutonium Analysis Capacity

In its mission need documentation for the Plutonium Modular Approach, NNSA adopted a mission need statement and an initial set of requirements to meet the mission need. The mission need statement defined two missions: (1) provide high-hazard, high-security laboratory space to conduct operations to support enduring stockpile stewardship and management activities and (2) enable NNSA to continue working in Plutonium Facility 4 for longer than its planned life by moving some high- hazard operations out of the facility. The Plutonium Modular Approach requirements document stated that the project is to build no less than two modular nuclear facilities at Los Alamos adjacent to Plutonium Facility 4. The requirements document also included a list of potential program missions that the site might relocate from Plutonium Facility 4 into the new modular facilities and a statement that the space should be designed to accommodate any of the potential program missions listed. The program missions included pit production, pit disassembly conducted by ARIES, and plutonium-238 work conducted in part by the Office of Space and Defense Power Systems.

As with the revised CMRR project, NNSA did not identify any key performance parameters for the Plutonium Modular Approach to meet. NNSA’s requirements policy directs NNSA to develop as many requirements as can be determined, including key performance parameters, in the program requirements document when it is first developed at CD-0. Also, NNSA did not identify any requirements that are specific to the needs of NNSA or DOE programs, including requirements related to providing additional plutonium analysis capacity that could support pit production. According to the best practices that we have identified for conducting analyses of alternatives, setting project requirements early in the analysis of alternatives process helps ensure that the selected alternative best meets the agency’s needs. The program requirements document for the Plutonium Modular Approach states that NNSA plans to identify program-specific requirements at CD-2, which is after the agency has selected the project alternative. NNSA officials and contractor representatives told us that they did not identify program-specific requirements for the Plutonium Modular Approach to meet because they expect that these requirements will change between now and project completion. They said that although additional high- hazard, high-security laboratory space is needed, the exact use for that space has not been determined. In commenting on a draft of this report, contractor representatives stated that the justification for the Plutonium Modular Approach is “somewhat nebulous” because the mission need is based on the need to extend the life of Plutonium Facility 4 rather than on meeting specific program needs. Extending the life of Plutonium Facility 4 may be worthwhile, but by defining key performance parameters and program-specific requirements before conducting the analysis of alternatives, NNSA would have better information about program-specific requirements to inform its analysis and to provide a clearer basis for selecting a project alternative at CD-1.

Conducting the analysis of alternatives for the Plutonium Modular Approach without considering program-specific requirements may negatively affect programs that currently use space in Plutonium Facility 4. For example, NNSA reported in the mission need documentation that moving plutonium-238 activities out of Plutonium Facility 4 and into a new modular facility is a key example of how NNSA could extend the life of Plutonium Facility 4 by reducing nuclear safety risks in the facility. NNSA officials and contractor representatives told us that plutonium-238 activities is a top candidate to move into a modular facility if the goal is to extend the life of Plutonium Facility 4. However, NNSA did not consult the Office of Space and Defense Power Systems, which conducts most of the plutonium-238 activities in Plutonium Facility 4, when developing the mission need and requirements for the Plutonium Modular Approach, according to officials with this office. These officials told us that they were not involved in discussions regarding the mission need for the project and that they had not heard about NNSA’s potential plan to move plutonium-238 equipment out of Plutonium Facility 4. They said that NNSA’s projected 2027 operational date for new modular facilities could directly impede the office’s ability to meet NASA’s future needs. They also said that if NNSA determines it needs to move the plutonium-238 equipment out of Plutonium Facility 4 to reduce risks, the Office of Space and Defense Power Systems may seek options for housing their equipment at another site rather than move into a new modular facility at Los Alamos.

Mission Need Prematurely Identified a Specific Solution

NNSA’s mission need statement for the Plutonium Modular Approach may not provide NNSA with the flexibility to explore a variety of alternatives without limiting potential solutions because it prematurely identified a specific solution for the project, contrary to DOE Order 413.3B. According to the order, at CD-0 NNSA must develop statements of mission need that do not identify a particular solution such as equipment, facility, or technology. However, NNSA’s mission need statement identifies a specific type of facility—high-hazard, high-security laboratory space. Also, in the program requirements document supporting CD-0, NNSA lists the Plutonium Modular Approach’s first mission requirement as delivering no less than two new nuclear facilities with full operating capability no later than 2027. The mission need statement document states that a few options appear to be available to meet the mission need that would involve upgrading existing facilities instead of building new modular facilities, and NNSA officials told us that they will consider a range of options as part of the analysis of alternatives to support CD-1. However, by narrowly defining the mission need and the first requirement in terms of building two modular nuclear facilities at Los Alamos by 2027, there is effectively no project alternative other than the modular approach that NNSA could select to satisfy this requirement and the mission need as currently documented.

According to an NNSA official who helped develop the CD-0 project documentation, NNSA included language about building two new nuclear facilities in the project documentation to acknowledge congressional direction contained in the national defense authorization acts for fiscal years 2014 and 2015 about building new modular facilities. This official characterized the mission need statement as balancing compliance with congressional direction with compliance with Order 413.3B. The fiscal year 2014 statutory language limited the funds that NNSA could spend on pursuing an alternative other than the modular facilities, and the fiscal year 2015 statutory language expressed the sense of Congress that the best choice was to build two modular structures by 2027. The language neither required NNSA to move forward with a specific strategy nor relieved NNSA of its responsibility to analyze other alternatives.

Identifying a specific solution in the mission need statement may create a potential bias against other potentially viable alternatives, limiting NNSA’s ability to explore a variety of solutions. In our December 2014 report on NNSA’s analysis of alternatives process, we found that conducting such an analysis without a predetermined solution is a best practice. In that report, DOE and NNSA officials acknowledged that an unreliable analysis of alternatives is a risk factor for major cost increases and schedule delays for NNSA projects. We recommended that NNSA incorporate best practices into its analysis of alternatives requirements to minimize the risk of developing unreliable analyses of alternatives and incurring major cost increases and schedule delays on projects. DOE agreed with our recommendation and in May 2016, the agency included language in an update to Order 413.3B directing that analyses of alternatives be conducted consistent with our published best practices. By rephrasing the Plutonium Modular Approach mission need statement and requirements so they are independent of a particular solution before conducting the analysis of alternatives, NNSA may be better positioned to objectively consider other alternatives before it makes its selection of an alternative at CD-1.

NNSA’s Revised CMRR Project Has a Lower Estimated Total Cost than the Previously Approved CMRR Project, but Agency Estimates of Cost Savings May Be Overstated

The total estimated cost for the revised CMRR project is lower than the total estimated cost for the previously approved CMRR project that included constructing the CMRR nuclear facility, but estimates of cost savings may be too high because the revised project includes less scope and is likely to provide less plutonium analysis capacity. In internal documents used to present information to senior decision makers as they considered approving CD-1 for the revised CMRR project, DOE and NNSA stated that cancelling the nuclear facility portion of the previously approved CMRR project and approving the revised CMRR project would save billions of dollars. For example, in an August 2014 slide presentation to DOE’s acquisition advisory board, NNSA reported that the total cost of its previously approved CMRR project—consisting of designing and constructing both the radiological lab and the nuclear facility, and installing equipment in both—would have reached up to approximately $7 billion. The presentation showed that since the total estimated cost of the revised CMRR project is up to approximately $3 billion—including sunk costs spent on designing, constructing, and installing equipment in the radiological lab and partially designing the nuclear facility—the agency will reduce costs by approximately $4 billion as a result of cancelling the CMRR nuclear facility. In a memorandum to the Deputy Secretary of Energy dated a day after the presentation and recommending approval of the revised CMRR project, DOE’s Office of Management stated that approving the revised CMRR project would reduce the total cost by more than $2 billion. In that memorandum and in NNSA’s budget requests for fiscal years 2016 and 2017, the agency stated that the revised CMRR project eliminated the need to construct the CMRR nuclear facility, thereby serving as its replacement.

NNSA’s cost savings calculations—whether $2 billion or $4 billion—may be overstated, however, because the revised CMRR project will likely provide less plutonium analysis capacity and less work scope than the previously approved CMRR project, which included the nuclear facility, and the calculations did not include costs associated with the Plutonium Modular Approach. More specifically:

The revised CMRR project will likely provide less analysis capacity than the previously approved CMRR project. Before NNSA deferred construction of the CMRR nuclear facility, NNSA’s contractor at Los Alamos estimated that the nuclear facility could support production of 40 pits per year in Plutonium Facility 4, and found that supporting 80 pits per year would require additional equipment and staff. NNSA reported that it expected the nuclear facility to support the production of up to 80 pits per year. In contrast, as discussed earlier, the 2015 contractor analysis determined that the revised project, as approved at CD-1 in August 2014, may not support the production of 10 pits per year and that the agency needs to successfully implement efforts to increase the project’s analysis capacity to support production rates of 10 pits per year and 30 pits per year. In a best case scenario, the project might be able to support an 80-pits-per-year rate, according to the contractor’s analysis. NNSA’s statement that the revised CMRR project replaces the CMRR nuclear facility is based on the revised project providing the same analysis “capabilities” as the planned nuclear facility but not the same analysis capacity. NNSA uses the term capability to refer to specific types of analysis techniques, such as radiochemistry. For example, according to NNSA’s contractor at Los Alamos, the revised project will install a set of plutonium analysis capabilities that is roughly comparable to the set of capabilities that was planned for the nuclear facility. However, contractor representatives added that, although the set of analysis capabilities is comparable, the analysis capacity is likely to be less because it is less efficient to conduct plutonium analysis in Plutonium Facility 4 and the radiological lab than it would have been in the CMRR nuclear facility.

The revised CMRR project includes less work scope than the previously approved CMRR project. The CMRR nuclear facility was to include, among other things: a 31,100 square foot nuclear material storage vault; a 12,800 square foot tunnel system for secure transfer of materials between Plutonium Facility 4, the nuclear facility, and the radiological lab; and 7,200 square feet for staging waste drums before they were to be shipped for disposal. The revised CMRR project does not include the storage vault, tunnel, or a new waste staging area. In its CD-1 documentation for the revised CMRR project, NNSA stated that it eliminated these scope elements because they were not needed at the time. However, NNSA also stated that it will eventually need a tunnel connecting Plutonium Facility 4 to the radiological lab and the future modular nuclear facilities and a waste staging area and that although the need for a new storage vault has been delayed, it has not necessarily been eliminated. As a result, removing these scope elements from the revised CMRR project may result in a reduction of the CMRR project’s total cost, but it may not represent a net cost savings to the agency or the taxpayer, because some scope elements that were removed will need to be constructed eventually.

NNSA’s calculations did not include costs associated with the modular facilities. NNSA stated in the revised CMRR project’s CD-1 documentation that the overall strategy for maintaining plutonium operations at Los Alamos without the CMRR nuclear facility consists of both the revised CMRR project and the Plutonium Modular Approach. The documentation mentioned that building new modular nuclear facilities will accommodate activities that are not included in the revised CMRR project or that are needed to facilitate the project. Therefore, without including some costs associated with the Plutonium Modular Approach, any cost savings calculations from cancelling the CMRR nuclear facility are likely to be overstated.

According to NNSA officials and contractor representatives, NNSA’s cancellation of the previously approved CMRR project’s nuclear facility may yield benefits in terms of an improved funding profile. These officials and representatives explained that one benefit of pursuing the revised CMRR project and the Plutonium Modular Approach lies in improving the agency’s ability to fund the two projects. They said that although the total cost to build a set of modular nuclear facilities is not necessarily less than building one large nuclear facility, acquiring smaller nuclear facilities over time may be more affordable because NNSA can spread out the facilities’ effect on the budget over a greater period of time. However, we found in our March 2016 report on NNSA’s modernization plans and the budget estimates to implement these plans that the total costs of NNSA’s plutonium infrastructure strategy at Los Alamos included in the plans were uncertain and may be underestimated because NNSA had not determined the number of modular facilities that may be required. Therefore, the actual amount of any cost savings that NNSA may realize has yet to be determined.

NNSA’s Schedule and Cost Estimates for the Revised CMRR Project Partially Met Best Practices

NNSA’s estimated schedule and costs for completing the revised CMRR project partially met best practices. Importantly, the schedule estimate did not include all work activities required for the project’s successful execution. The cost estimate exhibited two of the four characteristics of reliable cost estimates; as a result, NNSA cannot have confidence that it can meet its schedule and cost goals, including supporting the agency’s commitment to end plutonium operations in the Chemistry and Metallurgy Research facility by the end of 2019.

NNSA’s Schedule Did Not Include Most Activities Needed to Complete the Project

NNSA did not maintain a schedule for the revised CMRR project that included all activities needed to complete the project and did not sufficiently analyze risks to determine whether the project’s estimated completion dates are reasonable. In August 2014, in support of the CD-1 decision, NNSA estimated that it would complete the revised CMRR project’s first subproject—installing plutonium analysis equipment in the radiological lab—in December 2019 and that it would complete the second subproject—installing plutonium analysis equipment in Plutonium Facility 4—in February 2024. NNSA’s contractor at Los Alamos updated the schedule subsequent to CD-1 approval. We reviewed an updated project schedule from August 2015 and assessed the extent to which it reflected best practices for creating a high-quality, reliable schedule. We found that NNSA had not maintained an integrated master schedule that contained all work activities needed to complete the project consistent with best practices or conducted a schedule risk analysis to determine the likelihood of meeting completion dates. Also, the agency may not meet the December 2019 completion date for the first subproject to install analysis equipment in the radiological lab, which supports NNSA’s commitment to end plutonium operations in the Chemistry and Metallurgy Research facility.

NNSA Had Not Maintained an Integrated Master Schedule

We found that NNSA’s August 2015 updated schedule for the revised CMRR project substantially or fully met several scheduling best practices, but the schedule was limited to short-term work that ends in 2017. The project schedule estimate fully met the best practice of establishing durations for those activities that the contractor included in the schedule. This best practice allows managers to more easily track progress and ensures that the schedule realistically reflects how long each activity will take. The schedule also substantially met the best practice of being horizontally and vertically traceable, meaning that related activities were correctly linked to intended outcomes, so that changing the amount of time planned for an activity would appropriately affect forecasted dates and that varying levels of activities were properly aligned. In addition, the updated schedule also substantially met the best practices for assigning resources to activities and sequencing activities. These best practices help ensure that resources, such as labor and equipment, are available when needed and that activities are listed in the order in which they are to be carried out.

However, we found that NNSA had not maintained a schedule for the revised CMRR project that contained all the activities within the project’s scope of work, as called for by best practices. NNSA’s August 2015 updated schedule that we reviewed did not contain most of the planned work necessary to complete the project. According to best practices, a project team should develop and maintain an integrated master schedule that includes the entire scope of work required for a project’s successful execution. An integrated master schedule should include near-term and long-term activities throughout the life of the project, and a project team may account for uncertainty in future years by using a lesser level of detail to describe future activities. Among other things, an integrated master schedule provides managers with a plan for carrying out the project and a mechanism for measuring progress and identifying potential problems in meeting schedule milestones. An integrated master schedule is also important for meeting other best practices, such as assigning resources to activities and validating the critical path.

An NNSA official and contractor representatives we interviewed who are members of the CMRR project team told us that they developed an integrated master schedule to support the project’s CD-1 decision, but they did not maintain the integrated master schedule after CD-1. Rather, the August 2015 updated project schedule we reviewed was limited to work activities scheduled to occur in the short-term, including design and site preparation activities that support progress toward CD-2 and long- lead procurement of analysis equipment. According to schedule documentation, these activities represent about 7 percent of the total remaining project, by cost. Specifically, these activities were scheduled to end in 2017 at an estimated cost of approximately $134 million. In contrast, the project’s entire scope of work is not scheduled to end until 2024 at an approximate cost of up to $2 billion.

An NNSA project team member said the team did not maintain the integrated master schedule after CD-1 because, in general, they do not update a CD-1 schedule until they develop the project’s schedule baseline before CD-2. NNSA directed contractor team members to create the project’s updated schedule based on a plan—finalized in September 2014—to implement the project through CD-2. Project team members said that the updated schedule was sufficient for managing the project until CD-2. Project team members also said limiting the updated schedule to short-term work allows them to focus attention on near-term, critical activities as they manage and execute the project. At the time of the revised CMRR project’s CD-1 approval in August 2014, Order 413.3B directed agencies to develop project schedules before multiple critical decisions, but the order did not specify when, and if, projects should create and maintain an integrated master schedule. By maintaining a schedule that is limited to short-term work ending in 2017, however, NNSA and its contractor have limited insight into how current performance affects completion dates beyond 2017, including completing the first subproject to install analysis equipment in the radiological lab in December 2019—in support of ending operations in the Chemistry and Metallurgy Research facility—and completing the rest of the revised CMRR project by 2024.

Subsequent to approving CD-1 in August 2014, DOE updated its direction to project managers for developing and maintaining integrated master schedules. In June 2015, the Secretary of Energy issued a department- wide memorandum directing NNSA to, among other things, develop, maintain, and document an integrated master schedule consistent with the methods and best practices identified by our schedule guide. In May 2016, DOE incorporated these policies into an update to Order 413.3B. Project team members said that they are creating a new project schedule to support the revised CMRR project’s first CD-2 decision scheduled to occur by the end of September 2016, but according to an NNSA project team member, this schedule will be limited to activities estimated to end in 2019, primarily involving the installation of equipment in the radiological lab and, to a lesser extent, in Plutonium Facility 4. As a result, this new schedule may not meet the criteria for an integrated master schedule and may not conform to the May 2016 update to Order 413.3B. NNSA plans to approve additional schedule baselines at a second CD-2 decision in mid-2017 for the remaining portions of the project, including activities needed to support upgrading the radiological lab to a Hazard Category 3 nuclear facility and installing most of the analysis equipment planned for Plutonium Facility 4. An NNSA project team member said that the remaining portions will be added to the overall project schedule estimate once their schedule baselines are approved at the second CD-2 decision. By using this approach, however, NNSA will not develop an integrated master schedule that includes all activities until the second CD-2 decision in mid-2017. Without developing and maintaining an integrated master schedule that includes all project activities under all subprojects prior to approving the first CD-2 decision for the revised CMRR project, NNSA does not have reasonable assurance that it has a mechanism for measuring progress and identifying potential problems in meeting schedule milestones for the revised CMRR project. Appendix II contains additional information about the details supporting our assessment of the August 2015 updated project schedule.

NNSA Did Not Conduct a Schedule Risk Analysis

We found that NNSA developed the revised CMRR project schedule without sufficiently analyzing risks to determine whether the project’s estimated completion dates are reasonable, contrary to best practices.

Project teams should conduct a schedule risk analysis to examine risks and opportunities for the project, according to scheduling best practices. This analysis determines the level of confidence that the agency can have in whether it will meet its estimated completion dates, typically expressed as a percentage. For example, an agency may learn from a schedule risk analysis that it can have 70 percent confidence that it will finish the project by the estimated completion date. For the revised CMRR project, NNSA reported that it created an “aggressive” schedule to finish its first subproject by the end of 2019 to support ending plutonium operations in the Chemistry and Metallurgy Research facility. However, because NNSA did not conduct a schedule risk analysis, the agency does not have information about the level of confidence associated with meeting the first subproject’s estimated completion date, and therefore cannot determine if that date is reasonable.

Instead of a schedule risk analysis, the CMRR project team completed a different kind of analysis called a risk assessment to support the August 2014 CD-1 approval decision, but this assessment minimally met best practices for a schedule risk analysis. The risk assessment provided information about, among other things, the potential effects of selected technical and programmatic risks on the schedule but was less comprehensive than a schedule risk analysis. The project team told us that they did not conduct a schedule risk analysis because they did not have sufficient detail about activities in the schedule, including those that would occur beyond CD-2. However, according to best practices, a schedule risk analysis can be performed on a summary version of the schedule if some activities are not well defined.

We previously found weaknesses in the schedule risk analysis that NNSA conducted on the previously approved CMRR project’s schedule. In March 2012, immediately after NNSA deferred construction of the CMRR nuclear facility, we found that NNSA’s schedule risk analysis did not include all risks identified by the project team, and we recommended that the agency conduct a new schedule risk analysis when it resumed the CMRR project and before establishing the schedule baseline at CD-2. NNSA disagreed with our recommendation, stating that it would not be prudent to update the project’s schedule at that time because the agency had just deferred the project. However, NNSA did not conduct a schedule risk analysis when it later resumed the project in 2014. Project team members told us that they plan to complete separate schedule risk analyses before each CD-2 decision for the revised CMRR project. According to the Secretary of Energy’s June 2015 memorandum on project management and the May 2016 update to Order 413.3B, NNSA is to develop project schedules consistent with the methods and best practices found in our schedule guide; these best practices include conducting a schedule risk analysis on a complete project schedule. Since NNSA will not create an integrated master schedule that includes all project activities until mid-2017, the schedule risk analysis that the agency will conduct on the portion of the project that NNSA is to approve at the first CD-2 decision by the end of September 2016 will be limited and may not account for all project risks. As a result, the schedule risk analysis may provide NNSA with information about the confidence level associated with meeting the first subproject’s December 2019 completion date, but may not provide information on the likelihood of meeting the project’s overall 2024 completion date. Without conducting a comprehensive schedule risk analysis that applies to the integrated master schedule to identify the likelihood that the revised CMRR project can meet all of its estimated completion dates, NNSA does not have reasonable assurance that it can meet the project’s overall 2024 completion date.

NNSA May Not Meet Its Estimate to Complete the CMRR Subproject That Supports Ending Operations in the Chemistry and Metallurgy Research Facility

According to NNSA documentation, the agency may not meet its December 23, 2019, estimate for completing the first subproject of the revised CMRR project because the schedule for the first subproject does not include any schedule contingency to account for risks to the “aggressive” schedule that may be realized. NNSA’s risk assessment identified four risks to the subproject that could require from 19 to 20 months, or more, of schedule contingency if the risks are realized. Therefore, if NNSA needs to add more than 8 days to the schedule as contingency to mitigate risks, the subproject’s completion date extends into 2020; if NNSA needs the entire schedule contingency, the completion date extends into 2021. Project team officials told us that they set the completion date for the first subproject so that it would finish before the end of 2019 to support NNSA’s commitment to end plutonium operations in the Chemistry and Metallurgy Research facility in 2019. According to NNSA documents, the project team streamlined some activities, including glovebox design work, to support the 2019 completion date. Unless its “aggressive” schedule stays precisely on track, the agency is likely to need at least some of its schedule contingency, extending the first subproject’s completion date to beyond 2019.

NNSA officials told us that the agency remains committed to ending plutonium operations in the Chemistry and Metallurgy Research facility by the end of 2019. In commenting on a draft of this report, NNSA stated that the Office of Defense Programs may choose to accept risk or a gap in plutonium analysis capabilities to support this commitment if the first CMRR subproject is not completed by the end of 2019. One official also said NNSA is evaluating options to account for any gaps in capabilities and has worked with contractor representatives to prioritize establishing the most important capabilities first so that any gaps would involve less critical capabilities.

NNSA’s Cost Estimate Exhibited Two of Four Characteristics of Reliable Estimates

NNSA’s cost estimate for the revised CMRR project exhibited two of the four characteristics of a high-quality, reliable estimate. We reviewed and assessed NNSA’s August 2014 CD-1 cost estimate covering the entire revised CMRR project—a cost range from $1.5 billion to $2.0 billion— against the four characteristics of reliable cost estimating: (1) comprehensive, (2) well documented, (3) accurate, and (4) credible. We found that NNSA’s cost estimate was substantially comprehensive and well documented. For example, the estimate recorded all ground rules and assumptions and fully defined the project’s scope, including technical characteristics and performance, design, and quality assurance requirements, which are important for comprehensiveness. The estimate also sufficiently described the methods used to calculate the cost of each component of the estimate, which helps make an estimate well documented. Figure 2 shows our assessment of NNSA’s cost estimate and appendix III contains additional information about the details supporting our assessment.

We found that NNSA’s cost estimate was partially accurate, in part, because the project team did not determine the project’s most likely costs when calculating management reserve and the total cost range for the project. Specifically, the project team generally used appropriate estimating techniques to develop the point estimate. The team, however, then increased the point estimate by as much as 50 percent to include funds for management reserve. According to our cost guide, management reserve is typically the amount of funds the contractor may need to spend to account for uncertainty. According to best practices, managers should use analyses, such as a risk analysis, to calculate a project’s most likely costs, and the estimate should not be overly conservative or optimistic. Contractor project team members told us that they set the amount of management reserve funding based on their subject matter experts’ assessments about the uncertainty related to the project and not on a formal risk analysis.

Similarly, contractor project team members told us that they created the top end of the total cost range by applying the highest amount possible under DOE guidance rather than using a risk analysis to calculate the most likely costs. DOE guidance suggests that NNSA could have added from 20 percent to 100 percent to the point estimate to create the top end of the range, depending on the level of information available about the project’s scope of work. Project team members said they determined that the highest amount possible was appropriate based on their subject matter experts’ opinion that the project carried high levels of uncertainty. However, adding the highest amount possible above the point estimate may increase the high end of the cost range more than necessary. Because NNSA calculated management reserve and the total cost range based on assumptions rather than analyses, the high end of the cost range may overestimate the funds that will be needed and may not provide management with reliable information to inform its decisions.

Cost Estimate Was Partially Credible

We found that NNSA’s cost estimate was partially credible, in part, because NNSA did not follow the best practice to conduct a sensitivity analysis. A sensitivity analysis quantifies the extent to which the cost estimate could vary because of changes in key assumptions and ground rules. Performing such an analysis increases the chance that decisions that influence the design of the project will be made with a focus on the elements that have the greatest effect on cost. The CMRR project team created a document that it labeled a sensitivity analysis to support its cost estimate, but this analysis did not follow most of the steps for conducting a sensitivity analysis as called for by best practices. The project team members said that they plan to conduct a sensitivity analysis to support the baseline cost estimate at CD-2. This would be in keeping with the Secretary’s June 2015 memorandum and the May 2016 update to Order 413.3B, which direct project managers to develop cost estimates in accordance with best practices found in our cost guide, including sensitivity analysis.

The cost estimate also was partially credible because NNSA did not conduct an independent cost estimate of the revised CMRR project. According to best practices, an independent cost estimate is generated by an entity that has no stake in approval of the project and provides an unbiased test of whether a project estimate is reasonable. Prior to CD-1, DOE conducted an independent cost review of the project estimate, which addresses a cost estimate’s high-value, high-risk, and high-interest aspects without evaluating the remainder of the estimate, as permitted under Order 413.3B. The independent cost review identified a number of problems with NNSA’s cost estimate, such as estimation errors, and according to DOE officials, NNSA incorporated some of the independent cost review team’s comments and recommendations prior to CD-1. Nevertheless, an independent cost estimate is a more thorough analysis than an independent cost review. We have previously recommended that DOE require independent cost estimates for all major projects at CD-1 to improve the credibility of cost estimates at this stage, but DOE has made independent cost estimates optional at CD-1. Project officials told us NNSA is preparing to have an independent cost estimate conducted to support the CD-2 cost estimate, which is required by Order 413.3B.

Conclusions

NNSA has committed to improving its performance in managing major construction projects and has recently taken important steps toward achieving that end. However, NNSA’s planning for the revised CMRR project at the Los Alamos National Laboratory indicates that the agency still has work to do. Specifically, NNSA did not identify a type of requirement called a key performance parameter in the revised CMRR project’s requirements document, contrary to NNSA’s requirements policy. These performance parameters are important because they define how well a project will perform its functions, expressed in terms such as capacity or throughput. One of the revised CMRR project’s primary functions is to provide plutonium analysis to support producing new plutonium pits for life extension programs that are critical to NNSA’s nuclear modernization efforts. NNSA needs a sufficient amount of plutonium analysis capacity to support the agency’s pit production plans, but it is currently unclear how well the project will support NNSA’s plan to develop the capability to produce 50 to 80 pits per year by 2030. By updating the program requirements document for the revised CMRR project to identify a key performance parameter that describes a pit production rate, NNSA could clarify the extent to which the project will support pit production plans, better enabling the agency to identify whether it will need to acquire additional plutonium analysis equipment or space by other methods. Also, NNSA officials told us that if the revised CMRR project does not provide sufficient plutonium analysis capacity to support a 50 to 80 pits-per-year rate by 2030, they do not have an alternate plan for obtaining that capacity. Given the uncertainty associated with the revised CMRR project supporting a 50 to 80 pits-per- year rate, having a plan for providing sufficient analysis capacity if the revised CMRR project will not support this rate may better enable NNSA to meet its pit production plans and support planned life extension programs critical to the agency’s modernization efforts.

In addition, NNSA’s broadly stated first requirement for the revised CMRR project suggests that the project must meet plutonium analysis needs of all DOE and NNSA program offices with plutonium analysis operations at Los Alamos. However, the revised CMRR project’s scope is focused on exclusively meeting the needs of the Office of Defense Programs. Other DOE and NNSA offices may use the plutonium analysis equipment installed for the CMRR project, if and when it is available, but NNSA is not including plutonium analysis equipment in the scope of the project specifically to meet their needs. NNSA’s requirements policy states that the project’s requirements should be a comprehensive set of what the project must provide to satisfy the mission need. By updating the program requirements document for the revised CMRR project to clarify whether the CMRR project will provide plutonium analysis equipment to meet the needs of DOE and NNSA programs other than those in the Office of Defense Programs, managers and stakeholders may have a clearer understanding of the extent to which the project will satisfy DOE and NNSA programs’ needs.

We commend the Secretary of Energy for issuing the May 2016 update to Order 413.3B—and the June 2015 department-wide memorandum that preceded it—directing project teams to develop and maintain integrated master schedules consistent with the best practices in our schedule guide. Such a step is likely to improve schedule outcomes for the agency. However, NNSA did not maintain the integrated master schedule it created at CD-1 for the revised CMRR project, and the schedule that the agency plans to create to support the first CD-2 decision by the end of September 2016 may not meet the criteria for an integrated master schedule and, therefore, may not conform to the May 2016 update to Order 413.3B. Without developing and maintaining an integrated master schedule that includes all project activities under all subprojects prior to approving the first CD-2 decision for the revised CMRR project, consistent with current DOE project management policy and scheduling best practices, NNSA does not have reasonable assurance that it has a mechanism for measuring progress and identifying potential problems in meeting schedule milestones for the revised CMRR project. Also, waiting until the second CD-2 decision in mid-2017 to create the integrated master schedule means NNSA will not conduct a schedule risk analysis for the entire project until then. Conducting a schedule risk analysis to support the first CD-2 decision by the end of September 2016 may provide NNSA with information about the level of confidence that the agency can have in its ability to complete the first subproject by the end of 2019. However, without conducting a comprehensive schedule risk analysis that applies to the integrated master schedule to identify the likelihood that the project can meet all of its estimated completion dates, NNSA does not have reasonable assurance that it can meet the project’s overall 2024 completion date.

Regarding the Plutonium Modular Approach, NNSA has approved the mission need for another potentially multi-billion dollar project at Los Alamos. NNSA, however, prematurely identified a specific solution when it approved the mission need—contrary to Order 413.3B and best practices—by stating that the Plutonium Modular Approach would construct high-hazard, high-security laboratory space. NNSA also specifically set the project’s first requirement as building no less than two modular nuclear facilities at Los Alamos with full operating capability by 2027. By defining the mission need and requirements in terms of this specific type of laboratory space and nuclear facility, NNSA has effectively predetermined that it is the only alternative that can meet the mission need and satisfy requirements. By rephrasing the mission need statement and requirements so they are independent of a particular solution before conducting the analysis of alternatives, NNSA may be better positioned to objectively consider other alternatives before making its selection of a project alternative at CD-1.

Also, when approving the mission need for the Plutonium Modular Approach, NNSA did not identify key performance parameters or program-specific requirements in the program requirements document. NNSA’s requirements policy directs NNSA to develop requirements that include key performance parameters in the program requirements document when it is first developed, and according to the best practices that we have identified for conducting analyses of alternatives, setting requirements early in the analysis of alternatives process helps ensure that the selected alternative best meets the agency’s needs. By defining key performance parameters or program-specific requirements before conducting the analysis of alternatives, NNSA would have better information about programs’ specific needs to inform its analysis and to provide a clearer basis for selecting a project alternative at CD-1.

Recommendations for Executive Action

We are making seven recommendations to the Secretary of Energy.

To ensure that NNSA will acquire sufficient plutonium analysis equipment and space to meet its needs, including pit production to support critical life extension programs, we recommend that the Secretary direct that the Under Secretary for Nuclear Security, in his capacity as the NNSA Administrator: update the program requirements document for the revised CMRR project to identify a key performance parameter that describes the plutonium analysis capacity the CMRR project is required to provide to support specific pit production rates and specify plans for how the agency will obtain additional plutonium analysis capacity if the revised CMRR project will not provide sufficient plutonium analysis capacity to support NNSA’s pit production plans.

To ensure that NNSA will provide clear information to stakeholders about the program needs that the revised CMRR project will satisfy, we recommend that the Secretary direct the Under Secretary for Nuclear Security, in his capacity as the NNSA Administrator, to update the program requirements document for the revised CMRR project to clarify whether the project will provide plutonium analysis equipment to meet the needs of DOE and NNSA programs other than those in the Office of Defense Programs.

To ensure that NNSA’s future schedule estimates for the revised CMRR project provide the agency with reasonable assurance regarding meeting the project’s completion dates, we recommend that the Secretary direct the Under Secretary for Nuclear Security, in his capacity as the NNSA Administrator, to develop future schedules for the revised CMRR project that are consistent with current DOE project management policy and scheduling best practices. Specifically: develop and maintain an integrated master schedule that includes all project activities under all subprojects prior to approving the project’s first CD-2 decision and conduct a comprehensive schedule risk analysis that applies to the integrated master schedule to identify the likelihood the project can meet its completion dates.

To ensure that NNSA is better positioned to objectively consider alternatives before making its selection of an alternative for the Plutonium Modular Approach, we recommend that the Secretary direct the Under Secretary for Nuclear Security, in his capacity as the NNSA Administrator, before completing the analysis of alternatives, to rephrase the statement of mission need and requirements for the Plutonium Modular Approach so that they are independent of a particular solution.

Finally, to ensure that NNSA has information about program-specific needs to inform its analysis of alternatives for the Plutonium Modular Approach and to provide a clearer basis for selecting a project alternative, we recommend that the Secretary direct the Under Secretary for Nuclear Security, in his capacity as the NNSA Administrator, before completing the analysis of alternatives, to identify key performance parameters and program-specific requirements for the Plutonium Modular Approach.

Agency Comments and Our Evaluation

We provided a draft of this report to DOE and NNSA for their review and comment. In its written comments, reproduced in appendix IV, NNSA generally neither agreed nor disagreed with our recommendations, although it outlined actions taken and planned to address each of our recommendations. NNSA also provided technical comments, which we incorporated in the report as appropriate.

Concerning our findings, in its comments, NNSA stated that the report substantially misinterprets the relationship between the CMRR project and plutonium pit production. NNSA also stated that the report incorrectly implies that the driver for the CMRR project and the project's ultimate success depend on meeting plutonium pit production needs when the goal of the project is to replace specific plutonium analysis capabilities. We state in this report that the revised CMRR project’s purpose is to provide plutonium analysis capabilities to facilitate replacing the aging Chemistry and Metallurgy Research facility. However, we also state that, according to NNSA documents, one of the primary functions of these analysis capabilities is to support NNSA’s efforts to produce new plutonium pits for life extension programs that are critical to nuclear modernization efforts. As such, the revised CMRR project plays a key role in supporting NNSA’s pit production plans. We agree that the success of the CMRR project should not be measured in terms of actual pits produced, but rather in terms of the plutonium analysis capacity the project provides in support of production efforts and other stockpile stewardship needs. As a result, we continue to believe that it is important for the agency to be clear about how well the CMRR project will support NNSA’s pit production plans. In addition, NNSA stated that the report does not reflect the depth and breadth of steps NNSA has taken in the last 2 years to improve project management in accordance with the Secretary of Energy's new policies. As we state in the report, we are pleased with the steps that NNSA has recently taken to improve project management, including the Secretary’s new policies, and we are hopeful that they will lead to improved outcomes.

Concerning our recommendations, in its written comments NNSA outlined planned actions that are responsive to two of our seven recommendations. Specifically, the first of these two recommended that NNSA specify plans for how the agency will obtain additional plutonium analysis capacity if the revised CMRR project will not provide sufficient capacity to support pit production plans. In its written comments, NNSA stated that it will update its plutonium strategy to include estimates of the requirements and additional means, if needed, to achieve the required capacity. NNSA stated that this action will be completed by September 30, 2017. The second of these two recommended that NNSA update the CMRR program requirements document to clarify whether the project will provide plutonium analysis equipment to meet the needs of DOE and NNSA programs other than those in the Office of Defense Programs. NNSA stated that it will update this document to clarify that the project will not install any unique analysis equipment required solely for non-defense-related programs. NNSA estimated the completion date for this activity to be December 31, 2017, or the next planned revision of the program requirements document.

Regarding our other five recommendations, NNSA outlined actions planned or taken that we do not believe are sufficiently responsive. Specifically, regarding our recommendation that NNSA identify a key performance parameter for the CMRR project that describes the plutonium analysis capacity needed to support specific pit production rates, NNSA stated that it is important to understand how the revised CMRR project will support the agency’s overall plutonium strategy, including pit production and other NNSA programs. However, NNSA did not agree that it would be appropriate to establish a key performance parameter for the revised CMRR project related to supporting a specific pit production rate. The agency stated that incorporating such a key performance parameter would require the project to rely on factors beyond its control to demonstrate that it had met such a performance parameter before the project was completed. Instead, NNSA stated that it will perform an analysis to estimate a pit production capacity range that the project will support and include a reference to this analysis in its next revision to the program requirements document. Although conducting this analysis and including a reference to it in the program requirements document may be helpful, it may not be sufficient to describe how well the CMRR project will perform one of its primary functions—conducting plutonium analysis that is essential to supporting NNSA’s pit production activities. As a result, we continue to believe that it is important for NNSA to define this information in a key performance parameter.

Regarding our recommendation that NNSA develop and maintain an integrated master schedule that includes all project activities contained within the revised CMRR project, NNSA stated that it does not intend to create one integrated master schedule. Instead, NNSA reiterated its plan—described earlier in this report—to create a separate schedule for each subproject as it reaches CD-2 rather than create an integrated master schedule that includes all project efforts. NNSA stated that, because the subprojects are minimally interdependent, there is little value to be gained by developing an integrated master schedule for the entire project. NNSA also stated that the subproject schedules will reflect best practices defined in our schedule guide, including representing effort beyond the near-term that is less well defined as planning packages. We are pleased that DOE plans to incorporate the scheduling best practices into the subproject schedules. However, NNSA’s proposed action does not meet the criteria for an integrated master schedule because it will not result in a schedule that includes all activities needed to complete the project when NNSA approves the first schedule baseline by the end of September 2016. At that time, the project schedule will reflect work activities only through 2019. Given that the entire CMRR project schedule is anticipated to extend into the mid-2020s, we continue to believe that it is necessary for NNSA to follow the best practice of creating an integrated master schedule that reflects the entire project because it provides the transparency needed to properly manage this complex project.

Similarly, regarding our recommendation that NNSA conduct a comprehensive schedule risk analysis for the revised CMRR project, NNSA stated that, rather than conduct a risk analysis that applies to an integrated master schedule, it would conduct separate schedule risk analyses for each of the subproject schedules in accordance with our schedule guide. We are pleased that NNSA will follow best practices when conducting these analyses. Conducting individual schedule risk analyses on each subproject schedule could provide information regarding the level of confidence associated with meeting the subprojects’ estimated completion dates. However, a comprehensive risk analysis performed on an integrated master schedule—a best practice—is important to identify threats and opportunities that may affect multiple subprojects and the entire CMRR project.

Regarding our recommendation that NNSA rephrase the statement of mission need and requirements for the Plutonium Modular Approach to be independent of a particular solution, NNSA stated that it concurs with the recommendation. However, NNSA stated that it believes the conclusions articulated in our report that the agency has preselected an alternative for the Plutonium Modular Approach are incorrect and that it initiated the analysis of alternatives process in May 2016, which was always intended to be independent of a particular solution. We are encouraged by NNSA’s statement that it intends to conduct the analysis of alternatives independent of a particular solution. However, we do not believe that NNSA’s actions are responsive to this recommendation because these actions do not include rephrasing the mission need statement and requirements. Since the mission need statement provides the foundation on which the analysis of alternatives process is based, we believe that continuing to define the mission need in terms of a specific solution—constructing high-hazard, high-security laboratory space— creates a potential bias against other potentially viable alternatives.

Regarding our recommendation that NNSA identify key performance parameters and program-specific requirements before conducting the analysis of alternatives for the Plutonium Modular Approach, NNSA stated in its written comments that it plans to develop initial key performance parameters, consistent with applicable DOE policy, after it completes the analysis of alternatives. The applicable policy, however, is NNSA’s. Under NNSA policy, NNSA is directed to define these parameters prior to approving the mission need at CD-0 rather than after the agency completes the analysis of alternatives, as NNSA is proposing to do. Therefore, we continue to believe that NNSA should identify these parameters before completing the analysis of alternatives that they have already begun.

We are sending copies of this report to the appropriate congressional committees, the Secretary of Energy, the Administrator of NNSA, and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made significant contributions to the report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

In this report, we assessed: (1) the extent to which the National Nuclear Security Administration’s (NNSA) revised Chemistry and Metallurgy Research Replacement (CMRR) project is expected to meet NNSA and other Department of Energy (DOE) programs’ plutonium analysis needs at Los Alamos National Laboratory; (2) the extent to which the Plutonium Modular Approach is expected to meet plutonium analysis needs at Los Alamos; (3) how the revised CMRR project’s scope and cost compare with those of the previously approved CMRR project; and (4) the extent to which the revised CMRR project’s schedule and cost estimates reflect scheduling and cost estimating best practices.

To assess the extent to which NNSA’s revised CMRR project is expected to meet NNSA and DOE programs’ plutonium analysis needs, we first identified the plutonium analysis equipment, space for that equipment, and the analysis capacity that the project is expected to provide. This information was contained in a set of documents that NNSA and its management and operating contractor at Los Alamos developed to support the revised project’s most recent critical decision (CD) milestone in August 2014, CD-1, including the mission need statement and program requirements document. We also reviewed documents that NNSA and its management and operating contractor at Los Alamos created after the revised CMRR project’s CD-1 approval, including analyses that estimated the project’s plutonium analysis capacity. We reviewed the revised CMRR project’s mission need statement and program requirements document in light of DOE’s project management directive Order 413.3B and accompanying DOE guidance and NNSA’s business operating procedure for program requirements documents, as appropriate. We interviewed officials from the CMRR project sponsor, NNSA’s Office of Defense Programs’ Office of Major Modernization Programs, and other project stakeholders, including NNSA’s Office of Acquisition and Project Management, DOE’s Office of Project Management Oversight and Assessments, and DOE’s Office of Enterprise Assessments, about the revised CMRR project. We also interviewed Los Alamos contractor representatives involved with the revised CMRR project during a visit to the Los Alamos site in August 2015 and later by telephone. We then reviewed documentation of NNSA and DOE programs’ planned plutonium analysis needs at Los Alamos, as summarized in planning documents created by the Los Alamos contractor and as contained in the CD-1 documentation. We discussed the information in these documents with officials from NNSA’s Offices of Defense Programs and other relevant program offices, including the NNSA Office of Defense Nuclear Nonproliferation’s Advanced Recovery and Integrated Extraction System (ARIES) and the DOE Office of Nuclear Energy’s Office of Space and Defense Power Systems. We selected these program offices because after the Office of Defense Programs, they conduct the most plutonium analysis at Los Alamos.

To determine the extent to which NNSA’s Plutonium Modular Approach is expected to meet plutonium analysis needs, we reviewed documentation supporting DOE’s November 2015 approval of the mission need (CD-0), including the mission need statement and program requirements document. We reviewed the mission need statement in light of DOE Order 413.3B and accompanying DOE guidance. We interviewed NNSA officials with the Office of Defense Programs’ Office of Major Modernization Programs—the Plutonium Modular Approach sponsor— and other stakeholders, including NNSA’s Office of Acquisition and Project Management and DOE’s Office of Project Management Oversight and Assessments. We also interviewed Los Alamos contractor representatives during our August 2015 site visit and officials with the Office of Space and Defense Power Systems and ARIES to discuss their involvement with the project.

To assess how the cost and scope of NNSA’s revised CMRR project compare with those of the previously approved CMRR project, which included the construction of a nuclear facility, we reviewed the revised project’s CD-1 documentation and noted instances where the revised CMRR project CD-1 documentation discussed the difference between the cost estimates for the previously approved project and the revised project, including in the CD-1 approval memo signed by the Deputy Secretary of Energy and in a slide presentation that NNSA provided to a DOE acquisition advisory board. Regarding the projects’ scopes of work, we also reviewed the revised project’s CD-1 documentation and noted instances where it discussed the scope that NNSA planned to include in the previously approved project but that the agency eliminated from the revised project. We also reviewed an analysis conducted by the Department of Defense’s Office of Cost Assessment and Program Evaluation that included a comparison of the scope of the CMRR nuclear facility with the scope of the two subprojects that make up the revised CMRR project and the Plutonium Modular Approach, and we interviewed officials from this office about their analysis.

To assess the extent to which the revised CMRR project schedule and cost estimates reflect scheduling and cost-estimating best practices, we compared the schedule and cost estimates with best practices. Specifically, for our assessment of the revised CMRR project schedule estimate, we used the criteria in the May 2012 GAO Schedule Assessment Guide (schedule guide). The schedule guide identifies 10 best practices associated with effective scheduling: (1) capturing all activities, (2) sequencing all activities, (3) assigning resources to all activities, (4) establishing the duration of all activities, (5) verifying that the schedule can be traced horizontally and vertically, (6) confirming that the critical path is valid, (7) ensuring total reasonable float, (8) conducting a schedule risk analysis, (9) updating the schedule using actual progress and logic, and (10) maintaining a baseline schedule. We assessed an August 2015 version of the revised CMRR project schedule that NNSA’s contractor at Los Alamos had updated since the project’s August 2014 CD-1 approval. We did not assess the August 2014 project schedule estimate that NNSA developed to support CD-1 approval, because the agency did not provide us with an electronic copy of the CD-1 schedule, which was necessary to conduct our analysis. NNSA provided the August 2015 project schedule to us in two parts, one for each of the two subprojects that make up the revised CMRR project. We assessed each subproject’s schedule estimate against the criteria in our schedule guide and then combined the scores to create the overall schedule assessment. For our assessment of the revised CMRR project cost estimate, we used the criteria in the GAO Cost Estimating and Assessment Guide (cost guide). The cost guide identifies best practices for the development of reliable cost estimates and summarizes these best practices into four characteristics of high-quality cost estimating—comprehensive, well documented, accurate, and credible. We reviewed the revised CMRR project cost estimate that NNSA prepared to support the CD-1 approval in August 2014 and assessed the extent to which it exhibited the characteristics of high-quality cost estimating and the associated best practices. We also reviewed the project’s CD-1 documentation that supported the cost estimate. For the schedule and cost assessments, we interviewed members of the CMRR project team, which consisted of NNSA officials and Los Alamos contractor representatives who helped develop and maintain the estimates, to determine the policies and procedures the project team had followed to develop the estimates. We also interviewed officials with DOE’s Office of Project Management Oversight and Assessments to understand the department’s schedule and cost estimating policies and guidance, and we reviewed those policies and guidance. After conducting our initial assessments of the schedule and cost estimates, we shared our draft analyses with NNSA officials and contractor representatives and obtained comments and additional information, which we then incorporated as appropriate. We did not assess the schedule or cost estimates for the Plutonium Modular Approach because NNSA had not approved the estimates when we started our review.

We conducted this performance audit from May 2015 to August 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Assessment of NNSA’s Schedule Estimate for the Revised Chemistry and Metallurgy Research Replacement Project

When the National Nuclear Security Administration (NNSA) approved critical decision (CD) 1 (select alternative) for the revised Chemistry and Metallurgy Research Replacement (CMRR) project at Los Alamos National Laboratory in New Mexico in August 2014, the project’s future work was organized under two subprojects. The first subproject, Radiological Laboratory Utility Office Building Equipment Installation Phase 2, involved installing plutonium analysis equipment in the radiological laboratory. The second subproject, Plutonium Facility 4 Equipment Installation, involved installing new plutonium analysis equipment in Plutonium Facility 4, the only high-hazard, high-security, fully operational plutonium facility in the country for producing plutonium pits, after removing contaminated equipment in space no longer in use. For our assessment of the revised CMRR project schedule, we used the criteria in the GAO Schedule Assessment Guide (schedule guide), which identifies 10 best practices associated with effective scheduling. We assessed an August 2015 version of the revised CMRR project schedule that NNSA’s contractor at Los Alamos had updated since the project’s approval in August 2014. We did not assess the project schedule estimate that NNSA developed to support the August 2014 CD-1 approval because the agency did not provide us with an electronic copy of that schedule. NNSA provided the August 2015 project schedule to us in two parts, one for each of the subprojects. We assessed each subproject’s schedule estimate against the criteria in our schedule guide and then combined the scores to create the overall schedule assessment. Table 2 presents our assessment, including a narrative summary of the subprojects’ schedule estimates and overall scores in terms of the 10 best practices.

Appendix III: Assessment of NNSA’s Cost Estimate for the Revised Chemistry and Metallurgy Research Replacement Project

The GAO Cost Estimating and Assessment Guide (cost guide) identifies best practices for the development of reliable cost estimates. The cost guide identifies the following four characteristics of a high-quality, reliable cost estimate: comprehensive, well documented, accurate, and credible. Using the criteria in our cost guide, we assessed the extent to which the National Nuclear Security Administration’s (NNSA) August 2014 cost estimate for both subprojects of the revised Chemistry and Metallurgy Research Replacement (CMRR) project exhibited the characteristics of high-quality cost estimating and the related best practices. NNSA created the cost estimate in June 2014 to support the revised CMRR project’s August 2014 critical decision (CD) 1 (select alternative). Table 3 shows scores for both the characteristics and the best practices, including a narrative summary of our findings for each best practice.

Appendix IV: Comments from the National Nuclear Security Administration

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Dan Feehan (Assistant Director), Andrew Berglund, Antoinette Capaccio, John Delicath, Jennifer Echard, Emile Ettedgui, Keith Hornbacher, Jason Lee, Leslie Kaas Pollock, Sara Sullivan, and Kiki Theodoropoulos made significant contributions to this report.