Department of Energy: Better Information Needed to Determine If Nonmajor Projects Meet Performance Targets
Released: 2013-01-22T00:00:00

Letter

Background

NNSA—a separately organized agency within DOE—has primary responsibility for ensuring the safety, security, and reliability of the nation’s nuclear weapons stockpile. NNSA carries out these activities at eight government-owned, contractor-operated sites: three national laboratories, four production plants, and one test site (see fig. 1). These sites, taken together, have been a significant component of U.S. national security since the 1940s. Contractors operate these sites under management and operations (M&O) contracts.the contractor with broad discretion in carrying out the mission of the particular contract, but grant the government the option to become much more directly involved in day-to-day M&O. Currently, NNSA’s workforce is made up of about 34,000 M&O contractor employees across the eight sites, and about 2,400 federal employees directly employed by NNSA in its Washington headquarters, at offices located at each of the eight sites, and at its Albuquerque, New Mexico, complex.

Some Projects Have Met or Are Expected to Meet Their Performance Targets, but EM and NNSA Did Not Clearly Document Information Needed to Determine the Performance of Others

Of the 71 EM and NNSA nonmajor projects we reviewed that were completed or ongoing for fiscal years 2008 to 2012, we were able to determine performance for 44 projects. Among these 44 projects, 21 have met or are expected to meet all three of their performance targets for the scope of work delivered, cost, and completion date, while 23 have not met or are not expected to meet one or more of their three targets. The remaining 27 of the 71 projects we reviewed had insufficiently documented performance targets or had modified scope targets, among other things, which prevented us from determining whether they met or were expected to meet their performance targets, according to our analysis of DOE data. Determining whether projects fully met or partially met performance targets was difficult because EM and NNSA did not always follow DOE requirements for documenting these targets. DOE has taken steps to ensure that EM and NNSA more clearly document performance targets for their projects, but some problems persist.

EM and NNSA Have Met or Are Expected to Meet All Performance Targets for Some Nonmajor Projects

Of the 71 nonmajor projects we reviewed, 44 projects—17 EM projects and 27 NNSA projects—had documented targets for scope, cost, and completion date, enabling us to determine their performance. Table 2 shows the expected or completed performance of these 44 EM and NNSA nonmajor projects.

As the table shows, of the 44 projects for which we were able to determine performance, 21 projects met or are expected to meet their performance targets for scope, cost, and completion date. Specifically, 17 completed projects—5 EM and 12 NNSA projects—met all three of their performance targets. These projects included a $22 million EM project to expand an existing waste disposal facility at the Oak Ridge Reservation in Tennessee and a $469 million NNSA project to construct chemical, electrical, and other laboratories and workspaces at the Sandia National Laboratories in New Mexico. In addition, as of August 29, 2012, 4 ongoing projects—1 EM and 3 NNSA projects—were expected to meet all three of their performance targets, according to DOE estimates. These projects included a $77 million EM project to construct two disposal units for storing waste at the Savannah River Site in South Carolina and a $199 million NNSA project to equip the Radiological Laboratory/Utility/ Office Building at the Los Alamos National Laboratory in New Mexico to make it suitable for performing programmatic work.

Table 2 also shows that 23 EM and NNSA projects did not meet or are not expected to meet one or more of their three performance targets. Of these 23 projects, 13 projects met or are expected to meet two of their three performance targets, and eight met or are expected to meet one of their three performance targets (see apps. II and III for more details). In addition, one project did not meet any of its performance targets. Specifically, EM’s project to decontaminate and decommission the Main Plant Process Building in West Valley, New York, did not complete all of its planned scope of work when it was completed in October 2011, almost 4 months after its completion date target and more than $50 million over its cost target of $46 million. EM cancelled the remaining project— Uranium-233 Disposition project, at the Oak Ridge Reservation in Tennessee—in December 2011 after spending approximately $225 million. (See apps. II and III for more details.)

In assessing whether projects had achieved their scope, cost, and completion date targets, we followed DOE and Office of Management and Budget performance metrics.must be completed within their original scope target. For cost targets, For the scope, DOE states that projects Office of Management and Budget guidance and DOE performance metrics regard projects completed at less than 10 percent above their original cost targets as having achieved satisfactory performance. Regarding completion date, DOE’s performance metrics do not address targets for completion. However, because Office of Management and Budget guidance includes performance standards for project schedule, we considered projects to be on time if they were or are expected to be completed at less than 10 percent past their original completion date targets.

Inconsistent Documentation of Scope Targets, Among Other Things, Made Assessment of the 44 Projects’ Performance Difficult

We encountered two major problems in assessing the performance of the 44 projects described above, which made it more difficult for us and DOE to independently assess project performance. First, EM and NNSA did not consistently follow DOE requirements for documenting scope targets and tracking these targets using DOE’s performance database. Second, EM did not always establish credible completion date targets or conduct required independent reviews when it restructured its PBSs in 2010.

Establishing a clearly defined target for scope is critical for an agency to accurately track and assess a project’s overall performance. In particular, a project’s scope of work directly affects estimates of the project’s cost and completion date. If the scope target is too broad or vaguely stated, it can be difficult to track whether or to what extent certain aspects of project scope were reduced or eliminated between CD-2 (when the baseline was established) and CD-4 (when the project was completed), potentially affecting cost and completion date targets. Accordingly, since 2003, DOE Order 413.3 has required that information on scope targets be documented in a project execution plan as part of CD-2. The current order also requires a project’s acquisition executive to sign a memorandum approving CD-2 that contains this information. In addition, the order requires that information on scope targets, as well as other critical performance information, be entered into DOE’s centralized database on project performance—the Project Assessment and Reporting System (PARS).OAPM, is used to track and report on project performance.

This database, which is administered by However, EM and NNSA did not always follow the order’s requirement on documenting scope targets, and we encountered the following problems when we attempted to identify scope targets for EM and NNSA projects:

Key project documents associated with CD-2 and identified in Order 413.3—project execution plans and CD-2 approval memorandums— often did not contain information on scope targets. Instead, we had to obtain and review a variety of other project documents to try and locate this information. For example, we obtained information on scope targets for several EM and NNSA projects from briefing slides (i.e., a PowerPoint presentation) prepared for a DOE advisory board involved in reviewing and approving projects. We also used independent project review reports, documents describing the functional and operational requirements of the projects, and contractor documents that provided detailed descriptions of a project’s scope of work, among other documents. (See app. I for more details on our scope and methodology.)

The additional project documents that EM and NNSA provided to verify scope targets were often dated several months (or more) before or after the approval of CD-2. Because these documents were often not contemporaneous with the date when CD-2 was approved, we had difficulty determining whether any scope targets had changed during the interval. For example, we obtained information on scope targets for two EM projects from a project execution plan that was signed and dated almost 9 months after the CD-2 approval memorandum had been signed. (If documents were not dated within 1 year of CD-2 approval, we did not consider them sufficient and reliable for purposes of determining scope targets.)

EM and NNSA often did not clearly or uniformly identify scope targets in their documents, instead providing this information in a variety of ways. For example, a NNSA project provided this information in briefing slides for a project advisory board under the headings “programmatic requirements summary” and “physical design summary.” In cases where we were unable to clearly identify scope targets, we relied on EM and NNSA officials to identify the information that they considered to represent scope targets.

OAPM has encountered similar problems in trying to identify scope targets for projects at CD-2 and track how well completed projects had met these targets at CD-4. Specifically, OAPM officials told us that DOE program offices may have documented a project’s scope targets in a variety of project documents, such as a project execution plan or an acquisition strategy plan, rather than in an approval memorandum at CD- 2. As a result, OAPM officials said they occasionally must reconstruct a project’s scope targets from other contemporaneous planning documents near the time of the CD-2 approval. In a few instances, they said that no audit trail exists to compare scope targets established at CD-2 with the scope targets cited in a project’s CD-4 approval memorandum. In addition, an OAPM official told us that OAPM has not completed the process of locating and entering these data into PARS and continues to work with EM and NNSA to reconstruct project scope targets near the time of CD-2 approval.

We found two problems with the performance baselines of many projects that EM restructured from its portfolio of PBS activities in 2009 and 2010. First, several projects did not have a credible completion date target. For example, EM’s April 2010 memorandum approving CD-2 for the Zone 1 Remedial Actions project, located in Oak Ridge, Tennessee, gave a target completion date of fiscal year 2017, which meant that the target completion date was at the end of the fiscal year (September 30, 2017), according to EM officials. However, EM approved the formal completion of this project, via a CD-4 approval memorandum, on September 30, 2011—6 years ahead of the completion date target identified in the CD-2 approval memorandum. In explaining this difference, EM officials stated that they had linked the target completion date for this project to the end of the existing PBS near-term baseline, which had been established before the restructuring process. According to EM officials, EM used this method because it already had a contract in place to conduct work activities associated with the PBS near-term baseline. Since the end of the PBS near-term baseline was to coincide with the end of the contract period, EM officials said that they did not think it would be appropriate to change the target completion dates. In addition, EM officials told us that they were more focused on finishing the scope of work for a given project within a specific dollar amount, and that it was not worth the additional expenditure of time and dollars to modify contracts to change the completion date. EM used this same methodology in establishing target completion dates for several other projects we examined.

When we found that EM’s practice of establishing target completion dates did not provide a meaningful benchmark for assessing project performance, we had to locate additional documentation to establish a more credible completion date target. For example, for the Zone 1 Remedial Actions project, we reviewed additional documentation and decided that a more credible completion date target was December 15, 2011. As a result, for this and other projects, the completion date targets we used to evaluate the performance of some projects are different than the ones that DOE uses in its PARS database.

Second, EM often established a new performance baseline and approved CD-2 for a project without having the baseline reviewed by an independent team of experts, as DOE Order 413.3 requires. Among other things, the review team is responsible for examining a project’s cost and completion date targets to ensure that they are credible and valid. However, EM did not conduct such reviews when it restructured some of the projects we examined. Instead, according to EM officials, EM relied on independent reviews conducted in the 2007 to 2008 time frame as part of the CD-2 approval process for PBS activities. In addition, EM officials said that it was not worth the additional expenditure of time and dollars to conduct new independent reviews for these projects.

If EM had conducted new reviews as part of its restructuring process, it is possible that these reviews would have uncovered problems with some of the performance targets that EM had to correct later. Specifically, we identified two projects—the decontamination and decommissioning of the Paducah Gaseous Diffusion Plant in Paducah, Kentucky, and the Main Plant Process Building in West Valley, New York—for which EM had to significantly increase the cost targets it had approved 10 and 5 months earlier, respectively. In both cases, these cost increases were due to errors that EM project officials made in calculating total project cost. For example, for the Paducah Gaseous Diffusion Plant project, the project team did not incorporate additional project costs, including funds for contingencies and the contractor’s fee, into the project’s cost estimate. This omission resulted in underestimating the project’s cost target by about $8 million, or about 21 percent. In both cases, these errors increased the projects’ cost targets and caused EM to miss the original cost targets, according to our assessment of performance.

Limited Documentation or Changing Performance Baselines Complicates Evaluation of 27 Projects’ Performance

We were unable to determine the extent to which 27 of the 71 nonmajor projects that EM and NNSA completed or had under way from fiscal year 2008 through fiscal year 2012 had met their scope, cost, and completion date targets for four reasons. First, EM and NNSA did not establish a performance baseline for eight projects. Second, EM and NNSA did not provide documentation that fully identified one or more performance targets—including targets for scope, cost, and completion date—for eight projects. Third, NNSA did not fully document a final project cost or a current completion date for three projects. Fourth, EM and NNSA modified the scope targets of eight projects after CD-2, rendering the original performance targets unusable for purposes of assessing performance.

EM and NNSA did not establish a performance baseline for a total of 8 of the 71 nonmajor projects we reviewed that were completed or ongoing for fiscal years 2008 to 2012. Without a performance baseline, a project’s performance cannot be assessed. Specifically, we found the following:

EM. EM did not establish a performance baseline for 6 of the 30 EM nonmajor projects we reviewed. According to EM documentation, when the office established near-term baselines for its PBS activities in the 2007 time frame, it decided that it would not establish a baseline for a few projects that were near completion or for which physical work was essentially complete, and remaining costs were low. Because EM had essentially completed all physical work before fiscal year 2008 on the 6 projects we identified, EM never established a performance baseline for these projects, according to EM officials. However, we included these projects in our review because EM did not formally approve the completion of these projects (via a CD-4 approval memorandum) until the 2010 to 2011 time frame, which meant that these projects would have been ongoing until that time. The combined cost of these 6 projects is approximately $1.5 billion. (See app. II for more details.)

NNSA. NNSA did not establish a performance baseline for 2 of the 41 NNSA nonmajor projects we reviewed. According to NNSA documents and project officials, after a May 2000 wildfire damaged lands and buildings at the Los Alamos National Laboratory, NNSA formally authorized two emergency recovery efforts in July 2000. Because this authorization was granted outside of the critical decision process, NNSA did not establish formal performance targets for these projects. The combined cost of these two projects is $145 million. (See app. III for more details.)

For 8 of the 71 nonmajor construction projects we reviewed, EM and NNSA did not fully define and document one or more performance targets for scope, cost, and completion date when they established and approved performance baselines for these projects at CD-2. Specifically, we found the following:

For EM, 2 of the 30 EM nonmajor projects we reviewed did not have clearly defined and documented targets for scope and completion date, 1 project did not have a clearly defined and documented scope target, and 1 project did not have a clearly defined and documented completion date target. The combined cost of these 4 projects, 2 of which are ongoing, is estimated to be at least $182 million. (See app. II for more details.)

For NNSA, 4 of the 41 NNSA nonmajor projects we reviewed did not have clearly defined and documented scope targets. The combined cost of these 4 projects, 2 of which are ongoing, is estimated to be $122 million. (See app. III for more details.)

For 2 of the 71 nonmajor projects we reviewed, NNSA did not fully document the final project cost at CD-4. The final cost has not yet been settled for these 2 projects due to pending litigation with the contractor. NNSA estimated the combined cost of these 2 projects, both of which have been completed, to be $195 million. In addition, for the Nuclear Materials Safeguards and Security Upgrades, Phase II project at the Los Alamos National Laboratory, NNSA and contractor officials have determined that the project’s remaining construction costs will exceed the existing funds for the project and have halted work on the project. As a result, NNSA has not determined what the project’s revised completion date target will be.

EM and NNSA modified the scope targets of 8 of the 71 projects we reviewed after approving them at CD-2. EM and NNSA used procedures to control and approve these modifications but did not establish new CD-2 performance targets. As a result, the scope modifications rendered the original CD-2 performance targets unusable for assessing project performance. We consider a project’s scope target to have been modified if, among other things, EM or NNSA increased the scope of the project after approving it at CD-2 or reduced the scope for programmatic reasons and provided a sound justification for this reduction. In contrast, if EM or NNSA reduced project scope solely to meet a project’s cost target, we did not consider the scope target to have been modified; rather, we considered the scope target not to have been met. (See apps. II and III for more information.)

Projects for which EM or NNSA had modified the scope target sometimes exceeded expectations. For example, an NNSA project to build a highway at the Nevada National Security Site had an original scope target of 19.2 miles of highway and a cost target of about $14 million, but the project team completed an additional 12 miles of highway at an incremental cost of about $4 million, ahead of the original completion date target. However, because the scope target changed, the total cost of the project also changed, which made it unfair to judge the project’s performance against its original cost target.

Table 3 provides an example of a NNSA project for which we consider the scope target to have been modified.

We have previously reported on problems with the way DOE documents and tracks the scope of its projects, and DOE has taken actions to address this issue. In a 2008 report on DOE’s Office of Science, we noted concerns within DOE that projects sometimes had overly broad definitions of scope, making it difficult to determine the effects of a change in project To address this issue, we recommended that DOE consider scope.whether it could strengthen its project management guidance to help ensure that each project’s scope is clearly and sufficiently defined. DOE generally agreed with our recommendation and revised Order 413.3 in November 2010 to establish clearer requirements for identifying and documenting project scope at CDs 2 and 4. Specifically, the revised order requires a project’s acquisition executive to clearly identify the scope target in the documentation approving CD-2. In the documentation approving CD-4, when a project is declared complete, this official must clearly identify the scope accomplished and compare this scope with the target established at CD-2.

To determine whether NNSA and EM had improved their documentation of scope targets since DOE revised Order 413.3, we identified two nonmajor projects for which NNSA and EM established performance targets in 2011. These projects are NNSA’s Sanitary Effluent Reclamation Facility Expansion project at the Los Alamos National Laboratory and EM’s Purification Area Vault project at the Savannah River Site. In reviewing project documentation, we found that both NNSA and EM had provided information on targets for scope, cost, and completion date in their memorandums approving CD-2. In particular, NNSA’s approval memorandum identified the following scope targets: (1) expand the existing Sanitary Effluent Reclamation Facility capacity to treat 300 gallons per minute of product water in an 18-hour day; (2) provide a 400,000-gallon product water storage tank, which provides a consistent supply of water to the cooling towers in the event the facility is off-line for maintenance; and (3) provide additional evaporation capacity. These scope targets provide a quantitative measure of how the project is to perform at completion, as required by DOE’s order.

However, we found problems with the way EM documented the scope target for its Purification Area Vault project. EM’s approval memorandum provided a high-level description of the project’s scope, stating that the project will modify an existing portion of the K-Area Complex at the Savannah River Site in South Carolina to accommodate a vault; implement passive and active fire protection features as identified in the project fire hazards analysis; and install a new heating, ventilation, and air conditioning system. However, the scope target cited in the approval memorandum did not provide sufficient detail for measuring scope performance at project completion and, therefore, it may be difficult for an independent reviewer to accurately assess project performance. Specifically, we found the following:

The first part of the scope target—construct a secure storage location for holding at least 500 containers—provides a quantitative measure of how the project is to perform at completion, as required by DOE’s order. The second part of the scope target—attain CD-4 approval for storage of containers—does not provide a quantitative measure and instead reflects a stage in DOE’s critical decision framework. Therefore, only one part of the scope target can be used to independently measure project performance regarding scope.

The scope target only captures some of the elements of scope contained in the high-level scope description. As a result, the effect of any changes to these other elements of scope on project performance is unclear. For example, if EM decided not to fully implement the fire protection features identified in its hazards analysis, it is not clear whether EM or an independent reviewer would consider the project to have met its scope target.

Only the scope target—as opposed to the other elements of scope in the high-level scope description—is currently being tracked in DOE’s centralized database for project performance. Given the other issues we identified with this scope target, an independent reviewer relying solely on information in DOE’s database may not have enough information to assess the project’s performance accurately.

Several Factors Affect EM and NNSA in Managing Nonmajor Projects

Several factors affected EM and NNSA in managing their nonmajor projects that were completed or ongoing from fiscal years 2008 to 2012. According to our interviews with project officials, these factors included the suitability of the acquisition strategy, contractor performance, and adherence to project management requirements.

Acquisition Strategy

Because EM and NNSA carry out their work primarily through agreements with private contractors, a project’s acquisition strategy is a critical factor that affects the ability of these offices to properly manage their nonmajor projects. According to DOE guidance, an acquisition strategy is the high-level business management approach chosen to achieve project objectives within specified resource constraints. The acquisition strategy is the framework for planning, organizing, staffing, controlling, and leading a project. As part of this framework, agency officials have to choose the most appropriate contract alternative for a given project. Alternatives can include the use of multiple contractors to perform different tasks or the use of a prime contractor (such as the M&O contractor at a DOE site), who would be responsible for awarding subcontracts for different tasks. In addition, agency officials should identify the use of special procedures, such as the use of a “design-build” contract, whereby a single contract is awarded for both design work and construction, or the use of a “design-bid-build” contract, whereby separate contracts are awarded for the design and construction.

Some EM and NNSA officials told us that the acquisition strategy was an important factor in the successful management of their projects. For example, EM retained a prime contractor to manage the Soil and Water Remediation–2012 project at the Idaho National Laboratory using a contract containing incentives based on cost and schedule performance. According to project officials, the fee structure under this acquisition strategy is relatively simple and gives the project team flexibility to tie incentives to different performance milestones across the multiple subprojects within the contract. These officials said that this contract structure has been a very effective tool in achieving performance goals. EM expects this project to meet its cost target of $743 million and its completion date target of September 2012, and officials said that they expect the contractor to receive its incentive fee, as called for in the contract.

Other EM and NNSA officials cited the existence of an inadequate acquisition strategy as having a negative effect on the performance of their projects. For example, according to NNSA project officials at the Los Alamos site office, the M&O contractor at the Los Alamos National Laboratory decided to construct the Radiological Laboratory/Utility/Office Building project using a design-build acquisition strategy with a single prime subcontractor responsible for both design and construction. This approach was chosen based on the M&O contractor’s experience with constructing office buildings. However, the project also involved the construction of a radiological laboratory, which entailed the use of rigorous documentation standards to show that the project can meet nuclear quality assurance standards, among other things. Officials of the NNSA site office said that one of their key lessons learned would be to use a design-bid-build acquisition strategy if they had to manage a similar project in the future. The use of a design-bid-build contract would have offered several advantages over a design-build contract, according to these officials. First, it would have allowed NNSA staff more time to develop more robust project specifications and a more mature project design before having contractors bid on the construction of that design. Second, NNSA staff might have had more time to evaluate bids from contractors to see if they had the skills to construct the project. Third, with a more mature design, NNSA might have been able to reduce the number of federal staff and the time spent overseeing the project. This project was completed in June 2010, a few months after its completion date target. Its cost target was $164 million; however, the final cost of this project has not been determined because of ongoing litigation. According to the officials of the site office, NNSA withheld the M&O contractor’s performance incentive fee as a result of less than desirable contractor and subcontractor management during the design and construction of the facility.

DOE has previously identified ineffective acquisition strategies as being among its top 10 management challenges. Specifically, in its 2008 root cause analysis, DOE reported that its acquisition strategies and plans were often ineffective.acquisition planning early enough in the process or devote the time and resources to do it well.

Contractor Performance

Because contractors carry out the work associated with EM and NNSA nonmajor projects, contractor performance is a fundamental factor affecting EM’s and NNSA’s management of these projects. According to EM and NNSA project officials, poor contractor performance was a significant factor impeding their ability to successfully manage nonmajor projects. Among other things, officials cited concerns with finding qualified contractors that understood DOE’s nuclear safety requirements and maintained adequate internal control processes. Examples are as follows:

Nuclear Facility Decontamination & Decommissioning – High Flux Beam Reactor Project, Brookhaven National Laboratory, New York: This project was completed in December 2010, more than a year ahead of its completion date target, and at a cost of $16 million, which was 31 percent higher than its cost target of $12 million. EM officials stated that the major factor increasing costs was that the contractor did not properly prepare for and pass internal safety reviews, which were necessary to demonstrate the contractor’s readiness to begin removal and disposal of key reactor components. Because the contractor required more time than originally planned to prepare for and pass internal safety reviews, work on the project was delayed, and the total project cost increased. Officials did not explain why the project was completed well ahead of its completion date target despite the delays encountered. Officials stated that one of the most important lessons learned was to better ensure earlier in the process that the contractor had a rigorous process in place (e.g., procedures and training) to demonstrate that their personnel were ready to perform the decontamination and decommissioning work. Because EM’s cleanup work at Brookhaven National Laboratory is performed under the Laboratory Management and Operations Contract under the purview of DOE’s Office of Science, EM officials said that they have provided information to the Office of Science to be included in the contractor’s overall performance evaluation.

Process Research Unit Project, Niskayuna, New York: EM established a cost target of $79 million for this ongoing project. In September 2010, a contamination incident occurred while the contractor was performing open air demolition of a building at the site. According to DOE’s incident report, the contamination incident had two root causes: (1) the contractor failed to fully understand, characterize, and control the radiological hazard; and (2) the contractor failed to implement a work control process that ensured facility conditions supported proceeding with the work. As a result of this incident, as well as weather-related issues, the project has exceeded its cost target, and the project’s final cost and completion date depend on the outcome of negotiations between DOE and the contractor, according to project officials.

Nuclear Materials Safeguards and Security Upgrades Project, Phase II, Los Alamos National Laboratory, New Mexico: This ongoing project is expected to meet its cost target of $245 million but not its completion date target of January 2013. NNSA used a design-bid- build acquisition strategy for this project, with one contractor responsible for designing the project, and another contractor responsible for construction activities. According to project officials, during the construction phase, the building contractor had to stop work when it discovered errors with the design of the project. Specifically, officials told us the designs contained an erroneous elevation drawing that did not adequately account for the presence of a canyon and a pipeline containing radioactive liquid waste on the north side of the project site. In addition, other construction subcontractors, whose work was to be performed in sequence, had to wait to begin their work. As a result of these problems, the design contractor spent considerable time redesigning the project, according to project officials, and NNSA has had to award additional funding and schedule time to the construction contractors to compensate for the inadequate design. All told, officials told us the additional costs resulting from redesign and the delay of construction ranged from $15 million to $20 million. In addition, NNSA and contractor officials recently determined that the project’s remaining construction costs will exceed the existing funds for the project and have halted work. As a result, NNSA has not determined what the project’s revised completion date target will be.

Adherence to Project Management Requirements

Effective project management also depends on having project officials consistently follow DOE’s project management requirements. Among other things, these requirements are aimed at ensuring that projects (1) have a sufficiently mature design before establishing performance targets and beginning construction activities; (2) have had their earned value management systems certified for more accurate reporting on performance; (3) undergo a review by an independent group of experts before beginning construction activities; and (4) maintain an adequate process to account for any significant changes to the project’s scope, cost, or completion date targets (known as a change control process). DOE has previously identified adherence to project management requirements as among its top 10 management challenges, stating in its 2008 root cause analysis that the agency has not ensured that these requirements are consistently followed. That is, in some instances, projects are initiated or carried out without fully complying with the processes and controls contained in DOE policy and guidance.

We found a similar problem with adherence to DOE project management requirements in some of the projects we reviewed, although these problems were more often associated with EM projects than with NNSA projects. Specifically, in half of the 10 EM projects we reviewed in depth, officials cited a lack of adherence to project requirements, particularly not having a sufficiently mature design when establishing performance targets and beginning work activities, as a significant factor impeding their ability to manage projects within the performance baseline.

For example, EM’s project to convert depleted uranium hexafluoride into a more stable chemical form at two locations—Paducah, Kentucky and Portsmouth, Ohio—was completed in November 2010, more than 2 years after its completion date target and more than $200 million over its cost target of $346 million. A lessons-learned report, completed in 2009 at the request of DOE, concluded that DOE’s critical decision process had become a “mere rubber stamp of approval.”Project had results consistent with its level of definition at the time of project commitment and execution start. Future DOE projects will likely demonstrate similar performance unless they are better defined at the start of detailed design and they follow not only the letter of DOE’s [critical decision] process, but also its spirit.” According to EM officials, EM withheld the construction contractor’s incentive fee due to its poor performance.

It stated: “In the end, the … In contrast, among the 10 NNSA projects we reviewed in depth, several NNSA project managers credited adherence to project management processes as contributing positively to project performance. The advantages of adhering to project management processes are illustrated by one of the projects we reviewed—the Ion Beam Laboratory project at Sandia National Laboratories, New Mexico. This project—to use ion beams to qualify electronics and other nonnuclear weapon components for use in the nuclear stockpile—was completed ahead of schedule in September 2011 at a cost of $31 million, which was 22 percent lower than its cost target of $40 million. Project officials stated that implementing a procedure to control any changes to the performance baseline and a Baseline Change Control Board served as the foundation to manage all changes to ensure that cost, schedule, and technical aspects were evaluated to meet the mission of the project. In addition, project officials made active use of earned value management data, with several officials noting that applying earned value management principles on a regular basis assisted the project team in taking management actions to keep the project on track. For example, the Sandia Project Manager provided monthly reports to Sandia senior managers and the federal project director to communicate the project’s progress, the accomplishment of milestones, financial outlays, project issues, and appropriate corrective actions. Owing to the project’s success in meeting its performance targets, NNSA did not withhold any fee from the contractor.

EM’s Workforce Plans Do Not Consistently Identify Mission- Critical Occupations and Skills or Shortfalls in These Areas

EM’s eight workforce plans for its federal workforce do not consistently identify (1) mission-critical occupations and skills and (2) current and future shortfalls in these areas. As shown in table 4, of the eight EM workforce plans, one fully identifies both mission-critical occupations and mission-critical skills; another four plans identify either mission-critical occupations or mission-critical skills, but not both; and four of the eight plans identify current and future shortfalls in mission-critical occupations.

EM’s workforce plans may not consistently identify mission-critical occupations and skills and shortfalls in these areas in part because EM’s Office of Human Capital has not established a consistent set of terms that all EM sites use to define and describe mission-critical occupations and skills, according to our analysis. Instead, the five EM workforce plans that identified or partially identified these occupations and skills (as shown in table 4) used different terms to identify them. The plans also differed in the number and type of occupations or skills identified as mission-critical. For example, two plans identified three such occupations and skills, while another identified 20 different job series associated with 40 different position titles. Table 5 shows the variations in mission-critical occupations and skills identified in these five EM workforce plans.

When we brought the issue of inconsistent terminology to the attention of EM officials, they agreed that it would be useful to establish a consistent set of terms for mission-critical occupations and skills and told us that they plan to address this issue in the fiscal year 2013 planning cycle. However, we note that EM’s guidance to its site offices already instructed them to describe shortfalls and surpluses in the skills most critical to site performance; nonetheless, not all site offices did so.

Notwithstanding the variations in terms for mission-critical occupations and skills in EM’s workforce plans, many of the plans indicate that EM’s federal workforce may soon face shortfalls in a number of important areas, including project and contract management. Examples are as follows:

The Portsmouth/Paducah Project Office’s plan states that the office will need more staffing, including in project management and contracting, to meet mission needs in future years. Specifically, the plan notes that 31 percent of its current federal workforce could retire by fiscal year 2017, including up to 67 percent of its contract specialists and up to 64 percent of its general engineers.

The workforce plan for EM headquarters, issued in July 2011, stated that 26 percent of its federal workforce was currently eligible to retire, with an additional 22 percent of the workforce projected to become eligible for retirement by fiscal year 2015. The EM headquarters plan projected that 60 percent of contracting officers would be eligible for retirement by fiscal year 2015.

The Idaho Operations Office workforce plan states that a significant number of federal employees in leadership and mission-critical positions were already eligible for retirement at the end of fiscal year 2011, but the plan does not specify the number or positions of these employees.

The Carlsbad Field Office workforce plan indicates that both of that office’s “contract/procurement specialists” will be eligible to retire by fiscal year 2017, along with 10 of its 15 general engineers.

The workforce plan for the Office of River Protection, which manages the storage, retrieval, treatment, and disposal of tank waste at the Hanford Site in Washington State, projects that the office will face a 61 percent shortfall in “contracting” and a 53 percent shortfall in “project management” by fiscal year 2017.

EM officials said that they recognize the need to better identify mission- critical occupations and skills and shortfalls in these areas, and that they have taken a number of steps to address these issues. For example, officials in EM’s Office of Human Capital told us that they conducted a skills assessment in 2010 that helped EM identify key occupational series to target in its succession planning efforts. In addition, officials in this office told us that they are actively engaged in mitigating the risk of having a large number of EM federal employees retire in the near future by developing a voluntary separation incentive plan and voluntary early retirement plan. If employees eligible for retirement participate in this plan, EM could fill vacated positions with younger employees who could develop their skills in future years. Moreover, officials in EM’s Office of Acquisition and Project Management told us that to ensure that each project team has the skilled staff it needs to meet project goals, they consult with the EM officials in charge of each project team, consider the project’s execution plan, and use DOE staffing guidance as a tool to inform staffing decisions.

EM officials also said that EM sites serve diverse functions and that, therefore, some variation in the workforce plans and their descriptions of mission-critical occupations and competencies is to be expected. Nonetheless, without a workforce plan or summary document presenting a consistent set of occupations and skills that are critical to every site office’s mission, such as project and contract management, it is difficult for DOE and us to understand EM’s most critical current and future human capital needs.

Conclusions

The 71 nonmajor projects that we reviewed cost an estimated $10.1 billion and are critical to DOE’s mission to secure the nation’s nuclear weapons stockpile and manage the radioactive waste and contamination that resulted from the production of such weapons. EM and NNSA are making some progress in managing these projects. For example, we identified some NNSA and EM nonmajor projects that used sound project management practices, such as the application of effective acquisition strategies, to help ensure the successful completion of these projects. However, some contract and project management problems persist. Specifically, both EM and NNSA have approved the start of construction and cleanup activities for some nonmajor projects without clearly defining and documenting performance targets for scope, cost, or completion date in the appropriate CD-2 documentation, as required by DOE’s project management order. In addition, EM and NNSA have not consistently tracked project performance, particularly for scope, in DOE’s centralized database for tracking and reporting project performance, as required by DOE’s project management order. Moreover, EM has approved new performance targets for projects without ensuring that these targets are reviewed by an independent team of experts, as required by DOE’s project management order. Without clearly defining and documenting a project’s performance targets and tracking performance against these targets through project completion, and without ensuring that projects are independently reviewed, neither DOE nor we can determine whether the department is truly delivering on its commitments when its contractors complete work on its projects.

Problems also persist regarding DOE’s workforce—specifically, its current and potential shortfalls in federal personnel with the skills necessary to manage its contracts and projects, an issue that has received attention in our high-risk list. EM recognizes the need to address this issue and has taken steps to do so, such as conducting succession planning based on an assessment of key skills, as well as having EM’s Office of Project Management consult with EM’s project teams to ensure that the project teams have the skilled personnel they need to execute projects successfully. However, EM does not consistently identify in its workforce plans the occupations and skills most critical to the agency’s mission, as well as current and future shortfalls in these areas. This issue is compounded by EM’s decentralized planning process, in which site offices produce their own workforce plans that do not use consistent terminology and are not aggregated centrally by EM headquarters into a single workforce plan or summary document. Some variation among site- specific workforce plans is to be expected, but EM officials have stated that it would be useful to establish a consistent set of terms for mission- critical occupations and skills and told us that they plan to address this issue in the fiscal year 2013 planning cycle. That said, previous EM guidance for workforce planning specified that the plans describe shortfalls and surpluses in the skills most critical to site performance, but not all of EM’s plans did so. Without a summary document or single workforce plan presenting a consistent set of occupations and skills that are critical to every site office’s mission, such as project and contract management, using consistent terms, it is difficult for DOE or us to understand EM’s most critical current and future human capital needs.

Recommendations for Executive Action

To ensure that DOE better tracks information on its nonmajor projects, including the extent to which these projects meet their performance targets, and that EM consistently identifies mission-critical occupations and skills, as well as any current and future shortfalls in these areas, in its workforce plans, we recommend that the Secretary of Energy take the following five actions:

Ensure that the department clearly defines performance targets— including targets for scope, cost, and completion date—for each of its projects and documents the targets in appropriate CD-2 documentation, as is required by DOE’s project management order.

Ensure that the department tracks the performance of its projects using the performance targets, particularly scope, it establishes for its projects, as is required by DOE’s project management order.

Ensure that each project is reviewed by an independent team of experts before the department approves performance targets, as is required by DOE’s project management order.

Direct EM to develop a summary document or a single workforce plan that contains information on mission-critical occupations and skills, as well as current and potential future shortfalls in these areas, for all EM sites.

Ensure that EM follows through on its plan to address the use of consistent terms across all EM sites for mission-critical occupations and skills.

Agency Comments

We provided a draft of this report to DOE for review and comment. In written comments, DOE agreed with our recommendations. DOE’s written comments are reprinted in appendix IV. DOE also provided technical clarifications, which we incorporated as appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Secretary of Energy, the appropriate congressional committees, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or trimbled@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V.

Appendix I: Scope and Methodology

To determine the extent to which the Department of Energy’s (DOE) Office of Environmental Management (EM) and National Nuclear Security Administration (NNSA) nonmajor projects have met their scope, cost, and completion date targets, we obtained performance information on 71 nonmajor projects. These 71 nonmajor projects included 30 EM projects and 41 NNSA projects that were either: (1) completed (i.e., reached critical decision 4) from fiscal year 2008 to fiscal year 2011 or (2) ongoing from fiscal year 2008 to fiscal year 2011 and for which EM and NNSA had established performance baselines at critical decision 2. We also collected performance information for ongoing projects for fiscal year 2012. The total estimated cost of these 71 projects is approximately $10.1 billion. The names and locations of these 71 projects are provided in apps. II and III. We excluded the following types of projects from our review: (1) major projects, or those projects that each cost more than $750 million; (2) EM projects funded entirely by the American Recovery and Reinvestment Act of 2009 because of a separate GAO review looking at these projects; (3) information technology acquisitions; and (4) We identified these projects using DOE’s Project operational activities.Assessment and Reporting System (PARS). To assess the reliability of PARS data, we interviewed officials about the system and reviewed relevant documents. On the basis of this information, we determined that the system has adequate and sound controls for entering and maintaining data. We also conducted electronic testing on the specific data fields of interest, including cost, schedule, and scope targets. We determined that the cost and schedule data were complete and sufficiently reliable for our purposes; however, we found the scope data to be incomplete. Through interviews with officials, we ascertained that the scope data were not missing because of a system or data entry problem; instead, because EM and NNSA had not consistently identified and documented scope targets for the 71 projects we reviewed, these data could not be entered into PARS. Therefore, we obtained data on project scope, cost, and schedule directly from EM and NNSA officials.

For the 71 nonmajor projects, we reviewed selected documents providing information about the projects’ targets for scope, cost, and completion date. We relied on DOE Order 413.3 for requirements on (1) specifying the scope, cost, and schedule targets for a project’s performance baseline and (2) documenting the performance baseline.requirements, we reviewed the relevant documentation (including critical decision memoranda and project execution plans) and compared the performance targets established for scope, cost, and schedule with the actual performance of completed projects and the expected performance of ongoing projects. For completed projects, we compared the performance targets for scope, cost, and schedule—as documented in critical decision 2 (CD-2) approval memorandum and project execution plans—with the completed scope, actual costs, and approval dates as documented in critical decision 4 (CD-4) approval memorandum. For ongoing projects, we compared the performance targets for scope, cost, and schedule with DOE project performance reports; we also had officials from EM, NNSA, and DOE’s Office of Acquisition and Project Management review performance information as of August 29, 2012. In cases where key project documents—including the CD-2 and CD-4 approval memoranda and project execution plans—did not identify all three performance targets for scope, cost, and completion, we requested and reviewed alternative project documents. These included, among other things: independent project review reports; briefing slides prepared for DOE advisory boards; contractor work packages; DOE documents listing the functional and operational requirements of projects; memoranda used to request approval of changes to project baselines; final acceptance reports documenting that contractors delivered project requirements; and DOE quarterly and monthly status reports on ongoing projects. When reviewing alternative project documents, we requested documents dated as close to CD-2 and CD-4 as possible. If documents were not dated within 1 year of CD-2 approval, we did not consider them sufficient and reliable for purposes of determining scope targets.

In keeping with our prior work, and in recognition of Office of Management and Budget guidance and DOE’s project performance goals, we characterized nonmajor projects that met or exceeded (or are expected to meet or exceed) their cost and schedule targets by less than 10 percent as completed within budget and on time, whereas we considered projects that exceeded (or will exceed) their targets by 10 percent or more to be over cost or late.whether a project had successfully met its scope target. Projects that reduced their scope target to meet their cost targets were considered not to have met their scope targets. In a few cases, EM and NNSA increased the scope of work associated with a project after establishing performance targets at CD-2; in these cases, we noted that these projects had been modified and did not calculate whether they had met or exceeded their original cost and schedule targets.

In addition, we considered To evaluate factors affecting EM’s and NNSA’s management of nonmajor projects, we selected a nongeneralizable sample of 20 out of the 71 projects—including 10 EM projects and 10 NNSA projects—for more detailed review. The names of these 20 projects are provided in apps II and III. Results from nonprobability samples, including our sample of 20 projects, cannot be used to make inferences about EM’s and NNSA’s overall project performance or generalized to projects we did not include in our sample. We were interested in gathering information on the selected projects to identify material factors that may not exist across all projects but could help us understand EM’s and NNSA’s organization strengths and potential challenges. We selected these 20 projects to ensure that our sample included completed and ongoing projects, with a wide range of project costs. Together, the 20 projects represented about $4.1 billion, or approximately 41 percent, of the total value of the 71 projects.

For these 20 projects, we developed a structured interview template to identify the key factors that affected the management of these projects. We used three primary sources in developing this structured interview template—GAO’s cost guide, DOE’s Order 413.3, and DOE’s guidance document on conducting project reviews. The structured interview template focused on certain aspects of project management, such as the preparation of project designs, risk estimates, and cost and schedule targets, as well as the adherence to DOE project management requirements. We pretested the structured interview template during a site visit to the Y-12 National Security Complex and the Oak Ridge Reservation near Oak Ridge, Tennessee. At each site, we selected six projects and interviewed relevant EM and NNSA federal project directors and other knowledgeable staff using the structured interview template. Based on our pretesting, we revised the structured interview template and conducted 20 interviews with the relevant EM and NNSA federal project directors and other knowledgeable staff to gather their perspectives on their projects’ performance and reasons for it.

To evaluate the extent to which EM’s workforce plans identify mission- critical occupations and skills and any current and future shortfalls in these areas, we examined EM’s strategic workforce plans for its headquarters and site office staff, DOE’s corrective action plan for contract and project management, and the Office of Personnel Management’s Human Capital Assessment and Accountability Framework. Specifically, we obtained the eight EM workforce plans, prepared by EM headquarters, the Consolidated Business Center, the Richland Operations Office and Office of River Protection (which manage operations at the EM site in Hanford, Washington), the Portsmouth/Paducah Site Office, the Savannah River Operations Office, the Idaho Operations Office, the Carlsbad Field Office, and the Oak Ridge Office. We reviewed these plans in their entirety, and also searched for relevant terms, to determine the extent to which the plans identified mission-critical occupations and skills and any current and future shortfalls in these areas. In addition to our document review, we interviewed DOE and EM officials with knowledge of EM’s practices in workforce planning, including officials in EM’s Office of Acquisition and Project Management and Office of Human Capital and Corporate Services. We conducted these interviews to determine how EM develops its workforce plans and to obtain EM officials’ points of view regarding the state of the EM workforce.

We conducted this performance audit from June 2011 to December 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Summary of Office of Environmental Management Nonmajor Projects Reviewed

We obtained and reviewed performance information on 30 EM nonmajor projects that were either completed or ongoing from fiscal year 2008 through fiscal year 2012. Table 6 summarizes this information for 17 EM projects for which we could determine performance. Table 7 summarizes this information for 6 projects for which EM did not establish performance targets. Table 8 summarizes this information for 4 EM projects with incomplete documentation of their performance targets, which meant that we could not determine performance. Table 9 summarizes this information for 3 projects for which EM modified the scope after establishing performance targets for these projects, rendering the original performance targets unusable for purposes of assessing performance.

Appendix III: Summary of National Nuclear Security Administration Nonmajor Projects Reviewed

We obtained and reviewed performance information on 41 NNSA nonmajor projects that were either completed or ongoing from fiscal year 2008 through fiscal year 2012. Table 10 summarizes this information for 27 NNSA projects for which we could determine performance. Table 11 summarizes this information for 2 projects for which NNSA did not establish performance targets. Table 12 summarizes this information for 7 NNSA projects with incomplete documentation of their performance targets or final cost, which meant that we could not determine performance. Table 13 summarizes this information for 5 projects for which NNSA modified the scope after establishing performance targets for these projects, rendering the original performance targets unusable for purposes of assessing performance.

Appendix IV: Comments from the Department of Energy

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Dan Feehan, Assistant Director; Sandra Davis; Robert Grace; and Jason Holliday made key contributions to this report. Also contributing to this report were John Bauckman; Jennifer Echard; Cindy Gilbert; Steven Lozano; Minette Richardson; Cheryl Peterson; and Carol Hernnstadt Shulman.