Modernizing the Nuclear Security Enterprise: New Plutonium Research Facility at Los Alamos May Not Meet All Mission Needs
Released: 2012-03-26T00:00:00

Letter

Background

In the mid-1990s, Congress directed DOE to develop the Stockpile Stewardship Program to provide a single, highly integrated technical program for maintaining the continued safety and reliability of the nuclear weapons stockpile. Stockpile stewardship comprises activities associated with conducting nuclear weapons research, design, and development; maintaining the knowledge base and capabilities to support nuclear weapons testing; and assessing and certifying nuclear weapons safety and reliability. Stockpile stewardship includes operations associated with producing, maintaining, refurbishing, surveilling, and dismantling the nuclear weapons stockpile. The Stockpile Stewardship Program’s objectives were updated as a result of the 2010 Nuclear Posture Review, which establishes the U.S. nuclear policy for the next 5 to 10 years, including the nation’s nuclear weapons stockpile requirements. The Nuclear Posture Review and the Stockpile Stewardship Program reinforce the New Strategic Arms Reduction Treaty between the United States and Russia. As part of this treaty, the United States has agreed to reduce the size of its strategic nuclear weapons stockpile from a maximum of 2,200 to 1,550 weapons, with the remaining weapons in the stockpile continuing to be an essential element of U.S. defense strategy.

Nuclear stockpile requirements include a pit production capacity that is defined by estimating the number of pits NNSA needs to manufacture annually to effectively support the nuclear weapons stockpile. The demand for pits has fluctuated over the past decade for various reasons. Until 2005, NNSA planned to produce pits in a large-scale manufacturing plant to be built called the Modern Pit Facility, which would have increased pit production capacity per year to a range of 125 to 450 pits. This project was terminated and, at around the same time, NNSA began to study a new approach for modernizing the stockpile, called the Reliable Replacement Warhead program, which would have produced 50 pits per year and which was also short-lived. Through this program, NNSA would have designed new weapon components, including pits, to be safer and easier to manufacture, maintain, dismantle, and certify without nuclear testing. Since 2008, NNSA’s guidance has established pit capacity for future production at about 20 pits per year, with an upper range limit of 80 pits per year. In addition, NNSA has recently determined that pit lifetimes are longer than anticipated and that it may increase the reuse of existing pits, reducing the demand for newly manufactured pits. Currently, pit capacity requirements are uncertain and still in flux. Demand may again fluctuate as a result of the Nuclear Posture Review and changes to the Stockpile Stewardship Program. For example, there are still unknowns in implementing the Nuclear Posture Review and modernization work on each nuclear weapon type may require a varied number of new pits.

To execute the activities to maintain and refurbish the nation’s existing nuclear weapons stockpile, NNSA oversees eight sites that comprise its nuclear security enterprise—formerly known as the nuclear weapons complex—which includes three national weapons laboratories, four production plants, and a test site, all of which carry out missions to support NNSA’s programs. One of these sites, Los Alamos National Laboratory, plays a crucial role in carrying out NNSA’s maintenance of the nuclear weapons stockpile, including (1) production of weapons components, (2) assessment and certification of the nuclear weapons stockpile, (3) surveillance of weapons components and weapon systems, (4) assurance of the safe and secure storage of strategic materials, and (5) management of excess plutonium inventories. Los Alamos was established in 1943 during the Manhattan Project in northern New Mexico. It is a multidisciplinary, multipurpose institution primarily engaged in theoretical and experimental research and development. A significant portion of Los Alamos’ work is focused on ensuring that nuclear weapons stockpile needs are met. Since 2000, pit production has been established within the Plutonium Facility Complex at Los Alamos’s Technical Area 55, and certified pits have been produced over the past 5 years in that facility.

A particularly important facility at Los Alamos within Technical Area 55 is the nearly 60-year-old Chemistry and Metallurgy Research facility. The facility has unique capabilities for performing analytical chemistry, material characterization, and research and development related to plutonium. This includes activities that support the manufacturing, development, and surveillance of nuclear weapons pits; programs to extend the life of nuclear weapons in the stockpile; and nuclear weapon dismantlement efforts. This pit production mission support work was first assigned to Los Alamos in 1996. NNSA also currently maintains some plutonium-related research capabilities at other facilities, such as Livermore’s Superblock facility. These capabilities are necessary components of NNSA’s overall stockpile management strategy. NNSA and DOE also use the unique plutonium-related capabilities located at Los Alamos and Livermore to support the plutonium-related research needs of other national security missions and activities outside of the nuclear weapons stockpile work, including nuclear nonproliferation activities; homeland security activities, such as nuclear forensics and nuclear counterterrorism; waste management; and material recycle and recovery programs.

The Chemistry and Metallurgy Research facility was initially designed and constructed to comply with building codes in effect during the late 1940s and early 1950s. In 1992, recognizing that some of the utility systems and structural components were aging, outmoded, and generally deteriorating, DOE began upgrading the facility. These upgrades addressed specific safety, reliability, consolidation, and security issues with the intent of extending the useful life of the facility for an additional 20 to 30 years. However, beginning in about 1997 and continuing to the present, a series of additional operational and safety concerns have surfaced. In particular, a 1998 seismic study identified two small parallel faults beneath the northern portion of the Chemistry and Metallurgy Research facility. The presence of these faults raised concerns about the structural integrity of the building in the event of an earthquake. DOE and NNSA determined that, over the long term, Los Alamos could not continue to operate the mission-critical support capabilities in the existing Chemistry and Metallurgy Research facility at an acceptable level of risk to worker safety and health. To ensure that NNSA can fulfill its national security mission for the next 50 years in a safe, secure, and environmentally sound manner, NNSA decided in 2004 to construct a replacement facility, known as the CMRR.

Federal agencies, including DOE and NNSA, have experienced long- standing difficulties in completing major projects within cost and on schedule. To provide assistance in preparing high-quality cost and schedule estimates, we compiled best practices used throughout government and industry and, in March 2009, issued a guide outlining the criteria for high-quality cost and schedule estimates. Specifically, our guide identified four characteristics of a high-quality, reliable cost estimate: (1) credible, (2) well-documented, (3) accurate, and (4) comprehensive.should result in high-quality cost estimates and hundreds of best practices drawn from across industry and government for carrying out these steps. For example, one of the key steps includes conducting an independent cost estimate––that is, one generated by an entity that has no stake in the approval of the project but uses the same detailed technical information as the project estimate. Having an independent entity perform such a cost estimate and comparing it to the project team’s estimate provides an unbiased test of whether the project team’s cost estimate is reasonable.

In addition, our cost guide lays out 12 key steps that Our guide also identified nine best practices for effectively estimating schedules: (1) capturing key activities, (2) sequencing key activities, (3) assigning resources to key activities, (4) establishing the duration of key activities, (5) integrating key activities horizontally and vertically, (6) establishing the critical path for key activities, (7) identifying total float (i.e., the time that activities can slip before the delay affects the completion date), (8) performing a risk analysis of the schedule, and (9) updating the schedule using logic and durations to determine dates. Many of these practices have also been incorporated into DOE’s recent guidance for establishing performance baselines.

GAO-09-3SP.

CMRR’s Initial Cost Estimate Has Significantly Increased and Its Schedule Has Been Delayed

The estimated cost to construct the CMRR, according to estimates prepared in April 2010, is nearly six times higher than the project’s initial cost estimate that was prepared in 2005. The project’s estimated completion date has also been delayed by at least 8 to 12 years. Our review of these most recent detailed cost and schedule estimates for the CMRR project found that the estimates generally reflect best practices, but are not yet entirely reliable.

Estimated Cost to Construct CMRR Has Increased by Almost Six- Fold, and Its Scheduled Completion Has Been Delayed

Since CMRR was first proposed, its costs have risen significantly, and its schedule has been repeatedly delayed. Specifically, in 2005, when DOE developed initial plans for CMRR, it estimated that the project would cost from $745 million to $975 million and would be completed between 2013 and 2017. This estimate was prepared using preliminary information— before a detailed project design was substantially under way—and was therefore considered by DOE to be a rough estimate. In April 2010, NNSA estimated that the CMRR will cost between $3.7 and $5.8 billion—a nearly six-fold increase from the initial estimate—and that construction will be complete by 2020—a 3- to 7-year delay. In February 2012, after we had provided NNSA with a draft of this report for its comments, NNSA announced that it had decided to defer CMRR construction by at least an additional 5 years, bringing the total delay from NNSA’s original plans to 8 to 12 years.

NNSA officials explained that the majority of the cost increases occurred because of changes to the facility’s design and because of project delays. Specifically,

Modifications to the facility’s design. To address concerns about seismic activity, the project design was modified to strengthen the facility to withstand a potential earthquake. For example, significant design changes resulted from the need to thicken the concrete walls to satisfy increasingly stringent seismic requirements. In addition, to proceed to final design, project officials had to evaluate the potential effects of an earthquake on the facility’s complex ventilation system. This effort included several studies, consultations with vendors and other designers, and an assessment of the availability of equipment that would meet seismic requirements. Overall, Los Alamos estimates the seismic related design changes increased the project costs by almost $500 million.

Delays in the construction start date and longer overall project duration. CMRR construction was originally expected to begin in 2008, but was first delayed until 2013 and is now not expected to begin before 2018. The initial delay in starting construction from 2008 to 2013 had varying causes, including facility design changes described previously as well as the additional time needed for NNSA to determine where and how to consolidate plutonium operations in the nuclear security enterprise, according to project officials. This delay starting construction pushed the estimated construction completion date from between 2013 and 2017 to 2020—3 to 7 years later than initially expected. At the time, the facility was expected to be operational in 2022. These delays further increased costs, partly because inflation meant that equipment and materials became more expensive as time passed. In addition, the longer project duration also contributes to increases in the cost of workers’ wages and salaries. Overall, project officials estimate that about $1.2 billion in additional costs resulted from these schedule delays. In February 2012, NNSA announced another significant project delay—at least an additional 5- year deferral in starting the construction of the CMRR—resulting in a total of an 8 to 12 year delay from NNSA’s original plans. However, NNSA has not yet determined the impact to the project’s costs as a result of this additional delay.

NNSA’s Most Recent Cost and Schedule Estimates Generally Meet Industry Best Practices, but Are Not Yet Entirely Reliable

Our review of NNSA’s most recent cost and schedule estimates for the CMRR construction project found that the estimates were generally well prepared but that important weaknesses remain. Specifically, we found that the CMRR cost estimate prepared in April 2010 exhibits most of the characteristics of high-quality, reliable cost estimates. As identified by the professional cost-estimating community and documented in our cost- estimating guide, a high-quality cost estimate is comprehensive, well- documented, accurate, and credible. Our review of the CMRR cost estimate found that the cost estimate exhibits three of the four characteristics of a high-quality estimate by being substantially comprehensive, well documented, and accurate, but only partially credible, as shown in table 1. Appendix II contains additional information about each of the four general best practice characteristics and our assessment of the estimate compared to detailed best practices.

The CMRR cost estimate only partially met industry best practices for credibility because project officials did not use alternate methods to crosscheck major cost elements to see whether the results were similar under different estimating methods. In addition, according to our guide, there are varying methods of validating an estimate, but the most rigorous method is the independent cost estimate that is generated by an entity that has no stake in the approval of the project. Conducting an independent cost estimate is especially important at major milestones because it provides senior decision makers with a more objective assessment of the likely cost of a project. A second, less rigorous method for validating a project’s cost estimate—an independent cost review— focuses on examining the estimate’s supporting documentation and interviewing relevant staff. Independent cost reviews address only the cost estimate’s high-value, high-risk, and high-interest aspects without evaluating the remainder of the estimate. An independent cost review on the entire CMRR project was initiated in 2011, but the more rigorous method of validating—conducting an independent cost estimate—has only been used on a small portion of the project representing about 6 percent of the project’s total costs. According to NNSA officials, DOE orders do not require NNSA to seek an independent cost estimate until just prior to establishing the project baseline, and project officials told us NNSA is preparing to have one conducted before the project baseline is established. However, until a quality independent cost estimate is completed on the entire project or another means of validating the estimate for the project, DOE and NNSA officials cannot be confident that the current cost estimate is completely credible.

With regard to CMRR’s schedule, the project’s schedule estimate fully met two and substantially met six out of nine best practices for a high- quality schedule as identified by our guide and minimally met one. For example, two of the best practices the estimate fully met concerned how well it (1) captured all of the project’s activities, including design, construction, and other tasks that collectively form a comprehensive schedule, and (2) is successfully kept up-to-date. Table 2 lists best practices along with our assessment of the extent to which the project’s schedule met each best practice.

The CMRR schedule estimate minimally met industry best practices for conducting a schedule risk analysis. Namely, according to our guide, a high-quality schedule requires a schedule risk analysis that uses already identified risks, among other things, to predict the level of confidence in meeting a project’s completion date and the amount of contingency time needed to cover unexpected delays. CMRR project officials identified and documented hundreds of risks to the project, but these risks were not used in preparing a schedule risk analysis. For example, project officials identified the following three risks that are likely to occur: (1) a necessary electrical system upgrade that might not be completed in time for construction activities, (2) uncertainties associated with the flow of simultaneous design changes, and (3) noncompliance with certain quality assurance standards for nuclear facilities. These risks could cause delays, ranging anywhere from 1 to 5 years. Nevertheless, the project’s schedule risk analysis identified only a 1-year schedule contingency for the entire project. If NNSA is unable to successfully mitigate these risks and if they occur together, there is a high likelihood that the 1-year contingency that NNSA established may be exceeded. As a result, project officials cannot be certain the schedule estimate contains all identified risks in its risk analysis. Project officials told us that, before the project baseline is established, they expect to have a schedule risk analysis that includes identified risks and that they are in the early stages of developing a plan to do so.

NNSA is taking steps to mitigate the risks that have been identified and, because the project is still in early stages, many risks may be resolved. For example, to mitigate the risk that the electrical system upgrade would not be completed in time to avoid a delay in construction activities, project officials have identified specific steps to help ensure that the upgrade is performed in a timely manner. However, without a schedule risk analysis that contains risks identified by CMRR project officials, NNSA cannot be fully confident, once it decides to resume CMRR construction plans, that sufficient schedule contingency is established to ensure that the project will be completed on time and within estimated costs. As a result, overall project costs could potentially exceed NNSA’s April 2010 estimate of between $3.7 billion and $5.8 billion and NNSA had not yet determined the impact to the project’s costs of its recent decision to defer CMRR construction for at least 5 years. Appendix III contains additional information on each practice and our assessment of the estimate compared to best practices.

NNSA Considered Several Options to Preserve Plutonium- Related Research Capabilities, but Ultimately Chose to Build a Minimally Sized Facility at Los Alamos

To replace the plutonium-related research capabilities in Los Alamos’s deteriorating Chemistry and Metallurgy Research facility, NNSA considered several options. In the end, NNSA decided to build a minimally sized CMRR facility at Los Alamos with a broad suite of capabilities to meet nuclear weapons stockpile needs over the long-term. These capabilities would also be used to support plutonium-related research needs of other departmental missions. NNSA evaluated these options based on their expected effect on cost, schedule, risk, and ability to meet the plutonium-related research needs of the nuclear weapons stockpile stewardship program.

NNSA first focused on identifying and replacing the capabilities necessary to maintain and modernize the nuclear weapons stockpile. Specifically, these capabilities included those necessary to study the chemical and metallurgic properties of plutonium pits to ensure that they are properly produced, certified, and monitored over time so they remain safe and reliable. For example, to ensure that a nuclear weapon will function as intended, the plutonium inside of the pits needs to meet strict specifications. Meeting these specifications requires having the capability to analyze and characterize the plutonium’s chemistry and material properties. The specifications require NNSA to measure several chemical attributes, including chemical composition and impurities, as well as the pit’s structural attributes, such as the metal’s microscopic grain size, its texture, any potential defects, and its weld characteristics. NNSA identified at least 58 distinct capabilities that will be required in the new facility to allow it to conduct the analyses necessary to build at least one pit of every type currently in the stockpile.many as 79 capabilities may be required if NNSA needs to manufacture a larger quantity of pits—up to its high estimate of 80 pits per year, which is the Department of Defense’s published military requirement for pit production. In addition to research capabilities, NNSA determined that the new facility would need to provide other capabilities to support research operations. In particular, long-term plutonium storage space is needed to support plutonium-related research at CMRR.

To house these needed capabilities, NNSA assessed three potential sizes for a new facility—22,500 square feet, 31,500 square feet, and 40,500 square feet. The 40,500 square foot option included about 10,500 square feet of unequipped space—known as contingency space—to allow for program changes, such as increased pit manufacturing. In addition, this contingency space could accommodate users outside Los Alamos, such as researchers from Livermore. However, in 2004, NNSA chose the smallest and least expensive option—22,500 square feet. NNSA officials told us that cost was the primary driver of this decision.

NNSA’s choice to build a minimally sized facility was questioned in two studies conducted subsequent to NNSA’s decision in 2004. Specifically, a Los Alamos study conducted in 2006 found that increasing CMRR’s size by 9,000 square feet—to a total of 31,500 square feet—would be the best option based on cost, schedule, risk, and the facility’s ability to meet plutonium-related research needs.independent study prepared for NNSA in 2006 determined that adding 9,000 square feet to CMRR would lower risk and increase facility flexibility Furthermore, a separate but could cost an additional $179 million.told us that a smaller sized facility had the best chance of minimizing costs. NNSA officials acknowledge that the smaller size option poses more risk because the facility will include no contingency space. This space may be necessary, for example, to respond to potential increases in pit production needs if in the future they unexpectedly approach or exceed 80 pits per year. If this occurs, and no contingency space is available, other plutonium-related research beyond that required for the nuclear weapons stockpile will also likely be affected. According to NNSA and Los Alamos officials, these risks could be mitigated by conducting some nonnuclear weapons plutonium-related research at other facilities, such as Los Alamos’s PF-4 pit production facility. However, PF-4 also has ongoing laboratory and storage limitations and may not be able to accommodate these other nonweapons plutonium activities.

Subsequent to its 2004 decision to build CMRR at Los Alamos, NNSA continued to study other locations for consolidating plutonium-related research within the nuclear security enterprise. Specifically, as part of its development of a complexwide strategy to modernize nuclear research, development, and production facilities that support the nuclear weapons stockpile, NNSA studied consolidating the nation’s plutonium-related research capabilities at Los Alamos, the Pantex Plant in Texas, the Nevada National Security Site in Nevada, the Savannah River Site in South Carolina, and the Y-12 National Security Complex in Tennessee. In December 2008, NNSA decided to consolidate plutonium research at Los Alamos and reaffirmed its earlier 2004 decision to locate the new CMRR at Los Alamos. Consolidating plutonium-related research capabilities at Los Alamos presented several advantages, including lower costs and risks when compared to other locations. For example, colocating plutonium analytical capabilities with Los Alamos’s pit manufacturing capabilities reduced the costs and risks of protecting plutonium from potential theft. As part of NNSA’s decision to consolidate plutonium research at Los Alamos, NNSA also decided that the CMRR would be used to support plutonium-related research needs of other non-weapons activities, including nuclear nonproliferation activities; homeland security activities, such as nuclear forensics and nuclear counterterrorism; waste management; and material recycle and recovery programs. However, the size of the planned CMRR facility—22,500 square feet—has not changed since NNSA’s initial 2004 decision, which calls into question the facility’s ability to support the needs of these other activities.

CMRR May Meet Nuclear Weapons Stockpile Requirements but May Not Accommodate Other Plutonium-Related Research Needs

NNSA’s plans to construct the CMRR focused on meeting changing nuclear weapons stockpile requirements. However, CMRR may not be able to accommodate all stockpile and other plutonium-related research needs, particularly as other NNSA facilities reduce or end their plutonium research activities as a result of broader NNSA plans to consolidate its plutonium activities.

NNSA’s CMRR Plans Focus on Meeting Nuclear Weapons Stockpile Requirements, But Some Plutonium-Related Work for the Stockpile May Not Be Accommodated

NNSA’s plans to construct the CMRR primarily focus on maintaining plutonium-related research capabilities that are necessary for meeting nuclear weapons stockpile requirements. NNSA designed the CMRR to support the capabilities necessary for maintaining the safety and reliability of the nuclear stockpile––namely, the testing, manufacturing, and certification of the pits––and, in particular, plutonium-related research capabilities, such as analytical chemistry and materials characterization, and associated special nuclear materials vault storage. More specifically, in designing the CMRR, NNSA analyzed detailed data on past nuclear weapons activities conducted at Los Alamos, including information on the frequency of plutonium samples analyzed over time and the expected annual requirement for manufacturing new pits to determine the plutonium-related research capabilities the new facility would need to meet NNSA weapons program requirements. For example, NNSA studied the number of plutonium samples that had been processed in 2007 at the old Chemistry and Metallurgy Research facility for analytical chemistry and materials characterization work and used the number as an average representation in assuming future workloads. In addition, NNSA considered the numbers of specific pieces of equipment and the associated square footage of laboratory space needed to conduct specific analytical chemistry and material characterization work.

In its planning, NNSA considered how plutonium-related capabilities in the CMRR could meet changing stockpile requirements, including NNSA’s established upper limit of producing 80 pits per year. NNSA designed the facility to ensure that it can meet the pit production requirements regardless of the specific number of pits produced—or, in other words, the number of pits produced each year will not significantly affect the capabilities NNSA will need in the new facility, although capacity limits cap the quantity of new pits at 80 pits per year. For example, NNSA’s 2009 CMRR Program Requirements document states that the new facility will have laboratory spaces designed in a way that is flexible and modular to accommodate changes in the mission and the dynamic conditions associated with normal processing and maintenance activities in a laboratory environment.

NNSA officials indicated that they are confident that the CMRR will generally meet nuclear weapons activities needs and accommodate changes in the nuclear weapons stockpile requirements, including the ability to produce up to 80 pits per year. However, some weapons activities capabilities that currently exist at other NNSA sites may no longer be available to the nuclear security enterprise because of broader NNSA modernization plans to consolidate plutonium activities. As part of NNSA’s plan to consolidate plutonium related work at Los Alamos, the CMRR was designed to absorb some plutonium-related research from other facilities as those other facilities reduce or end their weapons activities work. For example, Livermore’s Superblock facility is equipped with the necessary systems to safely work with plutonium and to support extending the life of certain warheads in the nuclear weapons stockpile. Under NNSA’s strategy to consolidate plutonium work at Los Alamos, the majority of Livermore’s plutonium is scheduled to be removed in 2012, and some of this research will be discontinued at Superblock. NNSA plans to have the CMRR take on much of this work; however, Livermore officials told us they believe that NNSA may still lose some plutonium- related capabilities once some research is discontinued at Superblock. For example, NNSA may face a gap in the plutonium-related capabilities necessary to help improve nuclear warhead surety—that is, safety, security, and use control. NNSA has not planned for another facility to take over this work, and NNSA officials told us that the CMRR has not been designed to support this surety research. Furthermore, NNSA and Los Alamos officials told us that NNSA may also lose some pit testing capabilities that only take place in the Superblock at Livermore and are expected to be discontinued there in 2013. Pit testing includes thermal, vibration, and other environmental tests on pits that ensure that the weapon can successfully function from the time it is in the stockpile until it is deployed and reaches a target. Livermore officials told us that CMRR will not accommodate pit environmental testing because the systems used to conduct the environmental tests could cause vibrations through the rest of the facility. This could disrupt other work that requires precision instrumentation. Livermore officials also told us that these pit environmental testing capabilities are necessary to help meet nuclear weapons stockpile requirements. Because the CMRR was not intended to support all of these capabilities, NNSA will need to find another location if this plutonium-related work currently being conducted at Livermore is to be continued. NNSA has begun studying the extent to which the environmental pit testing capabilities will be needed, and if so, where they will be located. However, NNSA currently has no final plans for relocating them elsewhere.

CMRR May Not Meet Other Plutonium-Related Research and Storage Needs As DOE and NNSA Have Not Fully Analyzed Programs Outside of Nuclear Weapons Stockpile Work

DOE and NNSA conduct important plutonium-related research in other mission areas outside of nuclear weapons stockpile work, and it is unclear whether the CMRR as designed will be large enough to accommodate these nonweapons activities because they have not comprehensively studied their long-term research and storage needs. A NNSA record of decision states that the CMRR will support other national security missions involving plutonium-related research, including nonproliferation, nuclear forensics, and nuclear counterterrorism programs. For example, NNSA plans to use analytical chemistry capabilities in CMRR to perform nuclear forensics work that would be needed to, among other things, identify the source of and individuals responsible for any planned or actual use of a nuclear device.

However, DOE and NNSA have not comprehensively studied the long- term plutonium-related research and storage needs of programs outside of NNSA’s nuclear weapons stockpile work and therefore cannot be sure that the CMRR can accommodate them. In particular, DOE does not have important information on departmentwide analytical chemistry and material characterization research and storage needs, which can be helpful in making fully informed planning decisions about its long-term infrastructure and consolidation plans for the nuclear security enterprise.

As we have previously reported, conceptual planning for a building—a process by which an organization’s facility needs are identified and understood—is the critical phase of any successful building project development. This conceptual planning results in a building design that should be well defined according to an organization’s needs and include input from all key stakeholders before it is designed. NNSA and Los Alamos officials told us that the programs supporting mission areas outside of the nuclear stockpile work—including NNSA’s Office of National Technical Nuclear Forensics and Office of Fissile Materials Disposition—were generally not involved in planning the CMRR. Los Alamos officials said that they thought that there was too much time before the new facility would be operating for other mission areas to know their specific needs. However, by not including input from all the mission areas during the design of CMRR, NNSA has risked not knowing all of the potential needs and uses for the new facility to complement its important missions outside of the nuclear weapons stockpile work.

NNSA and Los Alamos have considered using space in Los Alamos’ PF-4 plutonium facility to handle additional plutonium-related research. However, NNSA officials told us that operating at this high pit production range would also likely use all of PF-4’s capacity. As a result, NNSA would have to consider reducing or eliminating other mission work currently supported in PF-4 or modify CMRR to incorporate additional needed space at additional cost. support nonweapons activity needs only if additional capacity remains after all weapons-related activities are supported. If additional capacity is not available, NNSA may face the prospect of not being able to use the new facility for one of its intended purposes of supporting certain plutonium-related research for missions outside of nuclear weapons stockpile work. A 2004 NNSA study suggested that this could effectively result in national security, nonproliferation, and environmental management programs potentially not performing in a cost-effective, compliant, and timely manner.

In addition, the CMRR has been designed to support Los Alamos and NNSA’s mission need to store significant quantities of nuclear material associated with the plutonium operations in a safe and secure manner using vault storage. Specifically, NNSA plans to shift all of Los Alamos’ current vault storage materials from its existing chemistry and metallurgy facility and overflow inventory from the PF-4 facility to the CMRR. However, Los Alamos officials told us that Los Alamos may not have enough storage space even after the CMRR is complete. NNSA plans to first use the newly available vault space in the CMRR for short-term, daily storage of nuclear materials being used for programmatic work and then use any remaining space for long-term storage. NNSA designed the CMRR without much long-term vault storage because these materials were initially planned to be shipped offsite for disposal. However, due to broader departmental challenges with other NNSA sites receiving materials for disposal, Los Alamos may not be able to ship its nuclear material off-site. If this is the case, Los Alamos officials told us that they may have to find additional long-term vault storage. This could also potentially affect Los Alamos’ ability to receive nuclear materials from other sites under NNSA’s consolidation strategy. In addition, Los Alamos officials told us that NNSA is still considering facility layout options that would allow for vault storage space to be configured for other operations and lab space. If this space is used for functional laboratory space rather than storage, less space will be available for short-term vault storage than NNSA originally thought.

Los Alamos officials told us that one of the major uses of CMRR storage space will be to relieve vault storage space at its plutonium facility that has already reached its available storage capacity.

Conclusions

Once NNSA resumes the CMRR project and constructs the facility, CMRR will play an important role in ensuring the continued safety and reliability of the U.S. nuclear weapons stockpile. The CMRR can potentially offer NNSA the opportunity to improve efficiency, save costs, and reduce safety hazards for workers. Because of the facility’s importance to the stockpile, multibillion dollar price tag, the inherent challenges in building facilities that can safely and securely store plutonium, and NNSA’s ongoing difficulties managing large projects, it is critical that NNSA and Congress have accurate estimates of the project’s costs and schedules, particularly when the CMRR project is resumed. After facing a nearly six-fold increase in estimated cost and schedule delays, NNSA’s most recent cost and schedule estimates generally meet industry best practices, but there are important weaknesses that call these estimates’ reliability into question. For example, an independent cost estimate—the most rigorous method to validate major cost elements that is performed by an entity that has no stake in the approval of the project—has not yet been conducted. To its credit, NNSA plans to have an independent cost estimate conducted prior to the completion of CMRR’s project baseline once the project is resumed. With regard to the project’s schedule estimate, however, NNSA cannot yet provide high assurance that all project risks are fully accounted for in the project’s schedule risk analysis that is used for updating the project’s schedule contingency estimates. As a result, NNSA cannot yet be fully confident that, once it decides to resume the CMRR project, the project will meet its estimated completion date, which could lead to further delays and additional costs.

However, reliable cost and schedule estimates for CMRR that fully meet industry best practices are of little use if DOE’s and NNSA’s mission needs are not met. Constructing CMRR is an important part of NNSA’s strategy to modernize its nuclear weapons facilities into a smaller and more responsive, efficient, and secure infrastructure to meet the changing requirements of the nuclear weapons stockpile. The CMRR was intended to support the plutonium-related research and storage needs of other DOE and NNSA national security missions and activities outside of the nuclear weapons stockpile work, including homeland security and nuclear nonproliferation activities; but because NNSA decided early in the project to reduce the size of the proposed facility to save money, CMRR may now lack the ability to accommodate these other research needs. In particular, the planned removal of most plutonium from Livermore presents NNSA with a dilemma in that the primary benefit of consolidating plutonium at Los Alamos—lower security costs—may be offset by the need to replace Lawrence Livermore National Laboratory’s plutonium research, storage, and environmental testing capabilities. Importantly, when NNSA decided to consolidate plutonium operations at Los Alamos, it did not fully consider whether planned or existing facilities at Los Alamos would be capable of continuing plutonium work being conducted elsewhere. For example, CMRR was not intended to accommodate the thermal, vibration, and other environmental pit testing that Livermore currently conducts because the vibrations this type of testing creates could disrupt other work at CMRR that requires precision instrumentation. Nevertheless, this type of testing is necessary to meet nuclear weapons stockpile requirements and so must be conducted somewhere. The full extent of the potential shortfall in plutonium research capabilities is not well-understood because DOE and NNSA have not comprehensively assessed their plutonium-related research, storage, and environmental testing needs. Plutonium research for the nuclear weapons stockpile and for other missions may have to compete for limited laboratory and storage space in CMRR and other facilities at Los Alamos, especially if the demand for newly manufactured pits unexpectedly increases. As a result, expansion of CMRR or construction of costly additional plutonium research, storage, and testing facilities at Los Alamos or elsewhere may be needed sometime in the future.

Recommendations for Executive Action

To strengthen cost and schedule estimates for the CMRR and ensure needed plutonium research needs are sufficiently accommodated, we recommend that the Secretary of Energy take the following three actions: 1. Once NNSA resumes the CMRR project and prior to establishing a new cost and schedule baseline, incorporate all key risks identified by CMRR project officials into the project’s schedule risk analysis, and ensure that this information is then used to update schedule contingency estimates, as appropriate. 2. Conduct a comprehensive assessment of needed plutonium-related research, storage, and environmental testing needs for nuclear weapons stockpile activities as well as other missions currently conducted at other NNSA and DOE facilities, with particular emphasis on mitigating the consequences associated with eliminating plutonium research, storage, and environmental testing capabilities from NNSA’s Lawrence Livermore National Laboratory. 3. Using the results of this assessment, report to Congress detailing any modifications to existing or planned facilities or any new facilities that will be needed to support plutonium-related research, storage, and environmental testing needs for nuclear weapons stockpile activities as well as other missions conducted by NNSA and DOE.

Agency Comments and Our Evaluation

We provided NNSA with a draft of this report for its review and comment. In its written comments, reproduced in appendix IV, NNSA generally agreed with our recommendations to conduct a comprehensive assessment of needed plutonium-related research, storage, and environmental testing needs and to report to Congress on any modifications to existing or planned facilities or any new facilities that will be needed to support these needs. However, NNSA disagreed with our recommendation to incorporate all key risks identified by project officials into the project’s schedule risk analysis.

Specifically, NNSA stated that, subsequent to receiving our draft report for its comments, the President’s budget request for fiscal year 2013 was released and resulted in several changes to the funding and execution of the CMRR project. In particular, construction of the CMRR is now to be deferred for at least 5 years. Therefore, NNSA stated that it is conducting additional analysis to determine the most effective way to provide analytical chemistry, materials characterization, and storage capabilities that were originally intended for the CMRR through the use of existing infrastructure. As part of this analysis, NNSA stated that it will evaluate options to use existing facilities at other sites. We believe this is consistent with our recommendation that NNSA conduct a comprehensive assessment of needed plutonium-related research, storage, and environmental testing needs and that NNSA’s decision to defer construction of the CMRR will give it sufficient time to conduct this assessment.

NNSA also commented that it will continue to work with Congress and other stakeholders as it adjusts its plutonium strategy. In our view, this is also consistent with our recommendation to report to Congress on any modifications to existing or planned facilities or any new facilities that will be needed to support plutonium-related research, storage, and environmental testing needs for nuclear weapons stockpile activities as well as other missions conducted by NNSA and DOE.

With regard to our recommendation to incorporate all key risks identified by CMRR project officials into the project’s schedule risk analysis, NNSA commented that spending project money to update the CMRR project’s schedule would not be prudent because of the construction delay. Therefore, NNSA disagreed with the recommendation. NNSA stated that its efforts in the near term would be focused on closing out the current design and that any future efforts will require updated cost and schedule estimates. We agree with NNSA that it is not necessary to update the project’s schedule at this time because of the recently announced construction delay; however, we maintain that it is important that all project risks are fully accounted for in the CMRR’s schedule once the project is resumed. Therefore, we clarified our recommendation to specify that NNSA should take action to ensure that the CMRR’s schedule risk analysis is appropriately revised to account for all project risks when NNSA resumes the project and before it establishes a new cost and schedule baseline.

We are sending copies of this report to the Secretary of Energy; the Administrator of NNSA; the Director, Office of Management and Budget; the appropriate congressional committees; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to examine (1) changes in the cost and schedule estimates for the construction of the facility and the extent to which its most recent estimates reflect best practices, (2) options the National Nuclear Security Administration (NNSA) considered to ensure that plutonium-related research activities could continue as needed, and (3) the extent to which NNSA’s plans to construct the Chemistry and Metallurgy Research Replacement Nuclear Facility (CMRR) and its consideration of options reflected changes in nuclear weapons stockpile requirements and other plutonium-related research needs.

To examine the project’s cost and schedule estimates and the extent to which its current estimates reflect best practices, we reviewed relevant NNSA documents and met with agency and contractor officials on the changes that have occurred to date and the reasons for them. We compared NNSA’s most recent detailed cost and schedule estimates with industry best practices contained in our cost estimating and assessment guide and discussed them with project officials to give them the opportunity to provide feedback on our assessment. Our review examined specifically those NNSA cost estimates that were prepared in April 2010 and schedule estimates, which at the time of our review were updated as of May 2011 or more recent for some portions of the schedule. As such, the cost and schedule estimates we reviewed do not reflect NNSA’s 5- year construction deferral recently announced in February 2012 and NNSA has not yet determined the potential long-term cost impact of this delay.

To examine the options NNSA considered to continue plutonium-related analytical work, we reviewed NNSA and contractor documents on plutonium research needs and the various options available to meet those needs. We also met with NNSA and contractor officials to better understand how these options were analyzed to determine the best approach to fulfill NNSA’s mission. While NNSA evaluated options on how to best meet its mission needs, it may have also evaluated alternatives based on the environmental impact of building the CMRR. As such, our review examined the options NNSA assessed to maintain the capabilities for plutonium-related analytical chemistry, material characterization, and storage and did not address NNSA’s compliance with requirements of the National Environmental Policy Act. We also met with NNSA and contractor officials to gain a better understanding of how these options were analyzed to determine the best approach to fulfill NNSA’s mission.

To determine the extent to which NNSA’s plans reflect changes in nuclear weapons stockpile requirements, we reviewed NNSA analyses that were used to support CMRR project decisions and met with NNSA officials to determine if these analyses were comprehensive and reflected up-to-date nuclear weapons stockpile requirements. We also visited Los Alamos and Lawrence Livermore National Laboratories. To ensure the data we used were sufficiently reliable, we compared information gathered from a variety of data sources. For example, we interviewed officials from both Los Alamos and Lawrence Livermore National Laboratories to obtain separate and independent perspectives on CMRR project plans. We determined the data were sufficiently reliable for our purposes.

We conducted this performance audit from February 2011 through February 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Summary Assessment of CMRR’s Cost Estimate Compared to Industry Best Practices

Best practice characteristic Comprehensive

Detailed best practice The cost estimate includes all life cycle costs.

Detailed assessment Substantially met. The total project cost for the construction of the Nuclear Facility is $4.2 billion. Government and contractor costs are included. However, operations and retirement costs are not included. These costs were not included because there was no mandate to estimate them. The cost estimate spans from start of construction in June 2010 to completion in 2020 with a schedule contingency through 2022.

The cost estimate completely defines the program, reflects the current schedule, and is technically reasonable.

Fully met. Technical descriptions were provided in multiple documents such as the “CMRR Nuclear Facility (NF) Estimate at Complete Forecast–April 2010,” the Los Alamos CMRR Mission Need Statement, the Program Requirements Documents, the WBS dictionary, and the “Final Environmental Impact Statement for the Chemistry and Metallurgy Project.”

The cost estimate work breakdown structure is product-oriented, traceable to the statement of work/objective, and at an appropriate level of detail to ensure that cost elements are neither omitted nor double-counted.

Partially met. The work breakdown structure and work breakdown structure dictionary are product oriented and the work breakdown structure flows down to level 4 of the program, project, or task. A statement of work was provided in the form of a mission need statement; however, it is not easily reconciled with the work breakdown structure dictionary.

The estimate documents all cost- influencing ground rules and assumptions.

Fully met. Cost influencing ground rules and assumptions can be found in the CMRR Estimate Update Execution Plan. Budget constraints and escalation are addressed. A list of high-level risk drivers along with the handling costs and risk input information was provided. Exclusions to the cost estimate are noted in the documents.

The documentation captures the source data used, the reliability of the data, and how the data were normalized.

Partially met. The data was analyzed and high-level cost drivers have been addressed as well as unit rates and quantities. Source data used to develop the estimate were found. The cost estimate was based on historical data from other Department of Energy (DOE) sites and the data was normalized. However, the independent review team found inconsistencies and discrepancies of quantities (hours) and costs. In addition, the review team reported that even though the basis of estimate referred to current contract awards or proposals, no reference was made to specific contracts or proposals by date and number.

Best practice characteristic

Detailed best practice The documentation describes in sufficient detail the calculations performed and the estimating methodology used to derive each element’s cost.

Detailed assessment Substantially met. While not explicitly stating what methodology was used, the pricing approach summary indicates that the estimate was developed using a combination of the build-up method and extrapolation from pricing information and productivity rates from other DOE sites. However the calculations involved were not clearly shown.

The documentation describes, step by step, how the estimate was developed so that a cost analyst unfamiliar with the program could understand what was done and replicate it.

Substantially met. The documentation for the estimate contains a summary narrative about the project as well as high-level cost summaries. The documentation discusses risk and contingency reserve. However, it does not address sensitivity although a sensitivity analysis was performed. Narrative on how the sensitivity analysis was conducted was not provided.

The documentation discusses the technical baseline description and the data in the baseline is consistent with the estimate.

Substantially met. There are technical descriptions discussed in the documentation that are consistent with the basis of estimate and the work outlined in the detail cost estimate spreadsheets. However, we are unable to map specific technical descriptions as outlined in the requirements document to cost elements in the high-level or detailed cost estimates. During the site visit, project officials showed us how the scope of work in the work breakdown structure dictionary was written in a way to illustrate how the scope of work was captured.

The documentation provides evidence that the cost estimate was reviewed and accepted by management.

Partially met. Los Alamos policy states that reviews shall be performed. According to project officials, these reviews typically include an integrated project team review, functional manager review, directorate review, and in the case of projects of high complexity or risk, an external corporate review and/or DOE Los Alamos Site Office review. A CMRR functional review was held March 12, 2010, and the review of the current estimate was listed on the meeting agenda. However, without further documentation we are unable to determine whether or not a briefing was given to management that clearly explains the detail of the cost estimate— including presentation of lifecycle costs, ground rules and assumptions, estimating methods and data sources as they relate to each work breakdown structure element, results of sensitivity analysis, risk and uncertainty analysis, and if a desired level of confidence was reached. Additionally, it is not clear that an affordability analysis, contingency reserve, conclusions, or recommendations were discussed with management. The documentation also does not show management’s acceptance of the cost estimate.

Best practice characteristic Accurate

Detailed best practice The cost estimate results are unbiased, not overly conservative or optimistic, and based on an assessment of most likely costs.

Detailed assessment Substantially met. Risk and uncertainty analyses were performed providing an 84 percent confidence level. There are three components that contribute to the total contingency value established for the project—schedule, estimate, and technical and programmatic risk analysis.

The estimate has been adjusted properly for inflation.

Substantially met. The documentation contained information on escalation rates. However, it is unclear how the cost estimate data were normalized. For example, costs are listed but are not labeled as constant or then-year dollars. Detailed calculations on how escalation was applied to the cost estimate are not documented.

The estimate contains few, if any, minor mistakes.

Substantially met. The numbers shown in the estimate at complete document and the cost estimate spreadsheet are accurate and the independent review team found only one minor mistake in their review of the estimate. However, we were not provided access to the detailed calculations behind the spreadsheet to check that the estimate was calculated correctly.

The cost estimate is regularly updated to reflect significant changes in the program so that it always reflects current status.

Substantially met. The CMRR Project Control Plan outlines a formal change control process that is to be executed in accordance with the Los Alamos Project Management and Site Services Directorate as well as the CMRR Baseline Change Control Board. These documents provide an approach to document, communicate, and approve potential changes to scope, cost, and schedule, and they provide the basis for incorporating changes into the project baseline and/or the forecast estimate at completion. These documents also describe the activities and responsibilities for making changes to the baseline.

Any variances between planned and actual costs are documented, explained, and reviewed.

Substantially met. Earned value is entered for each work package based on the earned value method indicated for that work package. Progress is reported in terms of percent complete by work package and is verified, analyzed, and reported to the project controls team. This information is then analyzed by the project controls team and control account managers and reviewed with CMRR management as the final reports are completed and published. However, there is no evidence of the cost estimate being updated to capture variances from the earned value system.

Best practice characteristic

Detailed best practice The estimate is based on a historical record of cost estimating and actual experiences from other comparable programs.

Detailed assessment Substantially met. Part of the estimate was developed using the engineering build up method which includes historical data from other DOE/NNSA sites (Waste Treatment Plant, Mixed Oxide Fuel Fabrication Facility, and two chemical demilitarization facilities). The reliability of the data is documented where confidence levels associated with quantity, productivity, labor, and nonlabor pricing are addressed. However, for some of the data, the sources were not provided and there was no evidence that earned value data was used to develop or update the estimate.

The cost estimate includes a sensitivity analysis—a technique that identifies a range of possible costs based on varying major assumptions, parameters, and data inputs.

Substantially met. CMRR conducted some sort of sensitivity analysis. No documentation was given providing a narrative on how the sensitivity analysis was conducted—including whether high percentages of cost were determined and how their parameters and assumptions were examined. Additionally, it cannot be determined whether the outcomes were evaluated for parameters most sensitive to change or how this analysis was applied to the estimate. However, during a site visit, Los Alamos officials provided a copy of a report that shows how a sensitivity analysis was applied to the nuclear facility cost estimate. For this assessment, a high and low range was determined. Some of the factors that were varied included overhead and General and Administrative rates, and escalation.

Best practice characteristic

Detailed best practice A risk and uncertainty analysis was conducted that quantified the imperfectly understood risks and identified the effects of changing key cost driver assumptions and factors.

Detailed assessment Substantially met. The cost estimate includes contingency costs for schedule ($99 million), cost estimate ($508 million) and technical and programmatic risks ($404 million). While a schedule risk analysis was performed that identified $99 million in schedule contingency, it is not clear how this analysis was done as no supporting documentation was provided. An independent review team assessed the schedule risk analysis and found that the risk model did not contain enough detail to allow specific risk events to be associated with the schedule activities they affect. Documentation supporting the cost estimate ($508 million) risk and uncertainty analysis was conducted via a Monte Carlo simulation which established an 84 percent confidence level for cost estimate uncertainty. The process by which this analysis was done is well documented and includes the contingency level range results. However, this risk and uncertainty analysis only reviewed classic cost estimate contingency and did not assess technical, programmatic or schedule risks. In addition, the independent review team found that the cost risk uncertainty analysis was done at a summary level so it does not fully reflect the uncertainty of the design costs associated with uncertainty related to quantities or prices listed.

Major cost elements were crossed checked to see whether results were similar.

Partially met. Documentation was provided that shows comparison of selected CMRR cost elements against cost estimates of other sites.

An independent cost estimate was conducted by a group outside the acquiring organization to determine whether other estimating methods produce similar results.

Partially met. An independent cost estimate was not conducted by a group outside of the acquiring organization. However, an independent cost review was performed by the U.S. Army Corps of Engineers in conjunction with an experienced contractor. This independent cost review resulted in the identification of key findings which require a Corrective Action Plan. The independent cost review focused on engineering design, and nuclear facility special facility equipment engineering design. The independent cost review team had 24 key findings and recommendations.

The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no evidence that satisfies any of the practice. “Minimally met” means the CMRR provided evidence that satisfies a small portion of the practice. “Partially met” means the CMRR provided evidence that satisfies about half of the practice. “Substantially met” means the CMRR provided evidence that satisfies a large portion of the practice. “Fully met” means the CMRR provided evidence that completely satisfies the practice.

Appendix III: Summary Assessment of CMRR’s Schedule Estimate Compared to Industry Best Practices

Explanation The schedule should reflect all activities as defined in the program’s work breakdown structure, to include activities to be performed by both the government and its contractors.

Detailed assessment Fully met. The schedule integrates all of the effort of NNSA, its contractor, and its major subcontractors.

The schedule should be planned so that it can meet critical program dates. To meet this objective, key activities need to be logically sequenced in the order that they are to be carried out. In particular, activities that must finish before the start of other activities (i.e., predecessor activities) as well as activities that cannot begin until other activities are completed (i.e., successor activities) should be identified. By doing so, interdependencies among activities that collectively lead to the accomplishment of events or milestones can be established and used as a basis for guiding work and measuring progress.

Substantially met. While we found that about 16 percent of the activities were missing predecessors and successors, or had constraints, lags, and leads, the majority (84 percent) of the activities were logically sequenced. There are more than 2,400 activities (5 percent) with missing or dangling predecessors or successors. There are summary tasks linked with logic (3 percent), but we have determined that they do not affect the credibility of the schedule. There are 123 activities (less than 1 percent) with start-to-finish logic. There are 460 activities (less than 1 percent) that have 10 predecessors or more. There are 590 activities (1 percent) scheduled with constraints, in addition to or substituting for complete logic.

The schedule should reflect what resources (i.e., labor, material, and overhead) are needed to do the work, whether all required resources will be available when they are needed, and whether any funding or time constraints exist.

Substantially met. Not all activities in the project schedule are resource loaded—only 3,757 activities (8 percent) out of the 45,429 activities with positive remaining duration have resources assigned in the schedule we received. However, there is credible evidence that the program and Los Alamos manage resources in various ways outside the project schedule and that their resource solutions are fed back to the project schedule so that it is feasible given resource limits.

The schedule should realistically reflect how long each activity will take to execute. In determining the duration of each activity, the same rationale, data, and assumptions used for cost estimating should be used. Further, these durations should be as short as possible and they should have specific start and end dates. Excessively long periods needed to execute an activity should prompt further decomposition of the activity so that shorter execution durations will result.

Substantially met. There are 1,642 activities (4 percent) with durations 44 days or greater, which means that the majority of the activities (96 percent) have activities that are of short duration. Contributing to this is the rolling wave approach to the schedule, where the near-term activities are detailed while activities further in the future are left in large planning packages until they become near-term, at which point they are broken down into their component activities.

The schedule should be horizontally integrated, meaning that it should link the products and outcomes associated with already sequenced activities. These links are commonly referred to as handoffs and serve to verify that activities are arranged in the right order to achieve aggregated products or outcomes. The schedule should also be vertically integrated, meaning that traceability exists among varying levels of activities and supporting tasks and subtasks. Such mapping or alignment among levels enables different groups to work to the same master schedule.

Substantially met. As discussed previously in the “sequencing all activities,” there are activities missing predecessor and successor logic as well as the presence of constraints, lags, and leads that call into question the adequacy of horizontal traceability. Vertical traceability was confirmed. The schedule hierarchy includes five levels, increasing in detail and specificity from top to bottom.

Explanation Using scheduling software, the critical path—the longest duration path through the sequenced list of key activities—should be identified. The establishment of a program’s critical path is necessary for examining the effects of any activity slipping along this path. Potential problems that may occur on or near the critical path should also be identified and reflected in the scheduling of the time for high-risk activities.

The schedule should identify float so that schedule flexibility can be determined. As a general rule, activities along the critical path typically have the least amount of float.

Detailed assessment Substantially met. This schedule’s critical path has 5,479 activities with zero or negative total float. There are so many critical activities because of a number of constraints on intermediate milestones which is causing negative float on paths to those activities. However, these activities do not all drive the final delivery. Los Alamos officials said that when they baseline the schedule, they plan to remove many of the constraints that are causing negative float. Many of these constraints are there to enable Los Alamos to monitor status of intermediate milestones. Substantially met. Of the remaining activities, 22 percent have unexplained large positive and large negative total float values. Even with agency review, these were present in the schedule. The total float values in many cases are several years long. There are 4,611 activities (10 percent) that have total float over 1,000 days or about 3.8 years. These high total float values are likely related to the incomplete logic described in the “sequencing all activities” best practice.

A schedule risk analysis should be performed using a schedule built using a good critical path method and data about project schedule risks, as well as statistical analysis techniques (such as Monte Carlo) to predict the level of confidence in meeting a program’s completion date. This analysis focuses not only on critical path activities but also on activities near the critical path, since they can potentially affect program status.

Minimally met. There is no evidence that a risk analysis has been conducted on this schedule or any summary schedule derived from this schedule. Los Alamos officials said that they have conducted a risk analysis using Monte Carlo simulation based on a prior and more concise schedule a full year before the version we reviewed was developed. The version we reviewed contained 90,000 activities and was developed in the Spring of 2010—a full year after Los Alamos conducted its risk analysis and Monte Carlo simulation. Los Alamos did not conduct a risk analysis on this more recent schedule, nor did it prepare and simulate a summary schedule based on this more recent schedule. The summary schedule that Los Alamos simulated was based on critical and near critical paths. This schedule comprised the main, secondary and tertiary critical paths. As a result, we believe that the schedule did not cover the entire work of the project, and therefore may have excluded some activities or paths that have risk sufficient to affect the finish date. Instead, Los Alamos selected about 2,100 activities based on total float, but this practice is risky because they may not have included all of the activities that risks in the risk register may affect.

Explanation The schedule should use logic and durations in order to reflect realistic start and completion dates for program activities. The schedule should be continually monitored to determine when forecasted completion dates differ from the planned dates, which can be used to determine whether schedule variances will affect downstream work. Maintaining the integrity of the schedule logic is not only necessary to reflect true status, but is also required before conducting a schedule risk analysis.

Detailed assessment Fully met. The CMRR schedule is updated at least monthly, although much of it is updated weekly. The schedule integrity is checked after each update and metrics are compiled on problems to determine if the schedule’s integrity is improving with each update. There are no activities in the past that lack the designation of actual start or actual finish. There are some activities on or after the data date that have actual start or finish designations, but that may be because there are 15 schedules combined in the Integrated Master Schedule and some were updated somewhat after May 9, 2011.

The ratings we used in this analysis are as follows: “Not met” means the CMRR provided no evidence that satisfies any part of the practice. “Minimally met” means the CMRR provided evidence that satisfies a small portion of the practice. “Partially met” means the CMRR provided evidence that satisfies about half of the practice. “Substantially met” means the CMRR provided evidence that satisfies a large portion of the practice. “Fully met” means the CMRR provided evidence that completely satisfies the practice.

Appendix IV: Comments from the National Nuclear Security Administration

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the contact named above, Ryan T. Coles, Assistant Director; John Bauckman; Jennifer Echard; Eugene Gray; David T. Hulett; Jonathan Kucskar; Alison O’Neill; Christopher Pacheco; Tim Persons; Karen Richey; Stacey Steele; Vasiliki Theodoropoulos; and Mary Welch made key contributions to this report.