Department of Energy: Actions Needed to Improve DOE and NNSA Oversight of Management and Operating Contractors
Released: 2015-06-12T00:00:00

Letter

Background

Since 2000, we have identified problems ranging from significant cost and schedule overruns on major projects to ineffective oversight of safety and security at NNSA and EM sites, indicating that DOE and NNSA continue to face challenges in ensuring the effectiveness of their oversight efforts. Examples, in chronological order, of the problems on which we have reported and where ineffective oversight was identified as a cause include the following:  2004: a suspension—or stand-down—of operations at one NNSA laboratory to address systemic safety and security concerns identified after an undergraduate student was partially blinded in a laser accident and two classified computer disks were reported missing;  2006: the discovery of a large number of classified documents and electronic files that had been unlawfully removed from an NNSA laboratory as a result of a drug raid on a private residence;  2007: nearly 60 serious accidents or near misses including worker exposure to radiation, inhalation of toxic vapors, and electrical shocks at three nuclear weapons laboratories from 2000 through 2007;  2008: the identification of significant protective force weaknesses (i.e., 13 specific deficiencies) during an independent physical security evaluation of an NNSA laboratory that included a force-on-force exercise to simulate an attack on a sensitive facility;  2012: 11 public hearings held since 2002 to address concerns about DOE’s safety practices by the Defense Nuclear Facilities Safety Board—an independent executive branch agency created by Congress to independently assess safety conditions and operations at defense nuclear facilities at DOE’s sites, including NNSA and EM;  2012: a serious security breach at an NNSA production plant—the Y- 12 National Security Complex (Y-12) near Oak Ridge, Tennessee—in which three trespassers gained access to the protected area directly adjacent to one of the nation’s most critically important nuclear weapon-related facilities before being interrupted by the security measures in place, resulting in the identification of multiple and unprecedented security system failures; and  2014: operations were shut down at WIPP following an underground fire involving a vehicle and, 9 days later, an unrelated radiological event occurred when a nuclear waste container breached underground at WIPP, contaminating a portion of the WIPP facility and releasing a small amount of contamination into the environment.

DOE and NNSA policies and orders concerning oversight of M&O contractors have evolved over time and now require that each DOE M&O contractor—including those overseen by NNSA and EM—have a CAS. In April 2002, in an internal memorandum, DOE outlined an approach for improving contract performance and promoting greater contractor accountability by, among other things, moving from an oversight approach focused on compliance with requirements contained in DOE orders and directives to relying on contractor management information provided through CAS to establish accountability and drive improvement. In 2005, DOE issued DOE Policy 226.1, Department of Energy Oversight Policy, and followed it with an associated implementing order requiring that assurance systems be implemented by DOE M&O contractors, among others, to encompass all aspects of the activities designed to identify deficiencies and opportunities for improvement. The focus of this DOE policy and order was to drive continuous improvement through contractor self-assurance and effective federal oversight. In March 2010, the Deputy Secretary of Energy announced a reform effort to revise DOE’s safety and security directives and modify the department’s oversight approach to “provide contractors with the flexibility to tailor and implement safety without excessive federal oversight or overly prescriptive departmental requirements.” In the memorandum announcing this effort, the Deputy Secretary noted that oversight of contractors’ activities at DOE and NNSA sites had become excessive and that burdensome safety requirements were affecting the productivity of work at DOE’s sites. The memorandum stated that reducing this burden on contractors would lead to measurable productivity improvement. In February 2011, NNSA issued a policy (NAP-21) with the purpose of providing further direction to NNSA officials and M&O contractors about the framework for the oversight model. Later in 2011, DOE issued Policy and Order 226.1B, which updated DOE’s oversight policy. While the previous DOE oversight policy and order were focused on driving continuous improvement, the 2011 versions—which are still in use—focus on improving the efficiency and effectiveness of DOE oversight programs by leveraging the processes and outcomes of CAS to reduce direct, hands-on oversight, when appropriate. Under the oversight framework, federal overseers are to continue to give additional oversight emphasis to high-hazard and high-risk operations, but where it can be determined that risk is lower and contractor-generated information in CAS is reliable, federal oversight can rely more on information from CAS.

NAP-21, which similarly focuses on the oversight efficiencies that can be gained by appropriately leveraging information from CAS, specifically applies to NNSA and its M&O contractors and elaborates on the more general DOE oversight policy and order by (1) developing an approach for federal officials to use in determining the appropriate mix of oversight activities for different contractor-performed functions and (2) by establishing a process by which NNSA would affirm the effectiveness of both CAS implementation by the contractor and the federal oversight approach at each site in the nuclear security enterprise.

First, NAP-21 describes a spectrum of approaches that can be employed by NNSA officials to oversee M&O contractors. On one side of the spectrum is “transaction-based oversight,” or direct, hands-on oversight activities to test or observe contractors’ performance through such mechanisms as on-site reviews, facility inspections, and other actions that involve direct evaluation of contractor activities. On the other side of the spectrum is “systems-based oversight,” where federal overseers rely on contractors’ processes and information from their CAS. NAP-21 calls for NNSA to use a mix of systems-based and transaction-based oversight approaches in overseeing contractors’ performance and provides a framework for determining the appropriate mix of these approaches based on the results of a three-pronged evaluation: (1) a risk assessment that analyzes the likelihood that an event will occur that adversely affects the achievement of mission or program objectives or harms human health or the environment; (2) a CAS maturity assessment that establishes the level of confidence NNSA officials have in the adequacy of performance information developed by the contractor and the ability of the contractor to effectively identify and address performance weaknesses; and (3) an assessment that considers the contractors’ prior performance for a specific activity. NNSA’s oversight framework allows for the oversight for any particular activity to range from primarily transaction-based oversight to primarily systems-based, or anywhere in between based on the outcome of these three assessments. Figure 1 shows the factors—as described in NAP-21—that should be considered by NNSA officials in determining an appropriate oversight approach. NAP-21 anticipates that, over time, as contractors’ CAS mature, NNSA officials will use transaction-based oversight primarily for areas of highest risk and hazard, and systems-based oversight for lower risk and hazard activities where they can rely more heavily on a contractor’s CAS.

Second, NAP-21 includes a process, known as “affirmation,” designed for a federal assessment review team—composed of staff from NNSA program offices and field offices—to review each field office’s mix of oversight approaches and practices, as well as implementation of each M&O contractor’s CAS. The goal of the review is to affirm that each contractor has a fully implemented and reliable CAS and that each field office’s approach to oversight is appropriate. According to senior agency officials, these affirmation reviews were envisioned as a crucial element in ensuring the effectiveness of NNSA’s overall approach to contractor oversight across the nuclear security enterprise.

In addition, as the nation’s only permanent disposal site for certain types of defense nuclear waste, key nuclear weapons missions depend on the availability of DOE’s WIPP in order to continue their own operations. WIPP’s operations were suspended in February 2014 following the underground vehicle fire and unrelated radiological event. In April 2015, DOE formally determined that the nuclear waste container that breached resulting in the radiological event at WIPP was packaged at NNSA’s Los Alamos National Laboratory (LANL). At the time of the 2014 incident at WIPP, waste packaging operations at LANL, were overseen by NNSA’s Los Alamos Field Office in coordination with EM’s CBFO, which provided additional verification by certifying packaged radioactive waste containers to ensure they met criteria set by CBFO for disposal in WIPP. At WIPP itself, oversight of the M&O contractor is led by EM through the CBFO.

NNSA Has Not Fully Established Policy or Guidance for Using Information from CAS to Conduct Contractor Oversight

In our May 2015 report, we found that NNSA has not fully established policy or guidance to support determining appropriate approaches to overseeing its M&O contractors, including for using information from CAS. Specifically, we found that NNSA does not have complete policy or guidance to support the assessments NAP-21 requires for determining an effective approach to overseeing M&O contractors at each site. NAP-21 outlines the three-pronged evaluation framework NNSA officials are responsible to carry out in determining an appropriate mix of oversight approaches based on assessments of risk, CAS maturity, and past performance. However, NAP-21 does not provide detailed or comprehensive guidance to NNSA officials on how to conduct these assessments, and NNSA headquarters has not issued any additional guidance for this purpose. We found that DOE and NNSA have some policies and guidance that are relevant to conducting risk assessments for security and safety and, in some cases, for large construction projects. We did not, however, identify any headquarters-level policy or guidance for assessing CAS maturity, for assessing contractors’ past performance to inform an oversight approach, or for assessing risk in other areas. Without such policy or guidance, NNSA officials responsible for conducting assessments may do so inconsistently, and thus treat similar risks differently. Further, we found that NNSA did not complete a chapter of NAP-21, which appears in the policy’s table of contents with the title Requirements Analysis Process, but the corresponding page in the document simply notes that the details of the chapter would be developed at a later date. NNSA officials told us the content of this chapter was intended to establish a process for NNSA to identify requirements in DOE and NNSA orders and directives essential to support safe and effective mission accomplishment and that this identification could assist M&O contractors in identifying key performance measures that could be tracked in CAS to help contractors ensure their compliance with requirements.

As a result of our findings, we recommended that NNSA establish comprehensive oversight polices including for using information from CAS to conduct oversight of M&O contractors and describing how to conduct assessment of risk, CAS maturity, and the level of the contractor’s past performance in determining an appropriate oversight approach. NNSA concurred with our recommendation and stated that it will cancel NAP-21 and instead issue a new corporate policy that will form a comprehensive framework for CAS in the context of ensuring safe, secure, and high- quality mission delivery. NNSA estimated it will complete this policy by September 30, 2015.

NNSA’s Field Offices Developed Procedures for Determining Appropriate Oversight Approaches, but the Procedures Are Not Always Complete and Differ

In the absence of sufficiently detailed and comprehensive guidance from NNSA headquarters for determining an appropriate mix of oversight approaches, NNSA field offices responsible for day-to-day oversight of M&O contractors reported developing their own procedures for this purpose. As described in our May 2015 report, these officials reported that their field office procedures for assessing risk were complete, but that their procedures for assessing CAS maturity and past performance in determining an appropriate oversight approach were not always complete. In addition, we found substantial differences among the procedures field offices had that may affect NNSA’s ability to ensure consistent oversight of its contractors. For example, the five field offices that reported having complete procedures for assessing CAS maturity used different processes and scales for rating maturity. While each of these procedures may be effective for each field office’s purposes, these differences could affect the consistency with which NNSA’s field offices are determining an appropriate mix of oversight approaches across the nuclear security enterprise. We recommended that NNSA work with field office managers to establish field office procedures consistent with headquarters policy and guidance to support assessment practices for determining appropriate oversight approaches. NNSA concurred with our recommendation and stated that field offices will develop new or modify existing procedures, as appropriate, to support the new requirements and estimated the completion date for these activities is September 2016.

Furthermore, field office officials have raised concerns that staffing levels and the mix of staff skills may not be adequate to conduct appropriate oversight in the near future and that this may result in overreliance on information from CAS without the ability to ensure that this information is sufficiently reliable. For example, in response to our survey of field offices conducted for our May 2015 report, six of NNSA’s seven field offices responded that having fewer staff to implement NAP-21’s approach to oversight is a challenge. Furthermore, five of seven field offices noted that not having certain subject matter experts is a challenge for oversight that could be exacerbated in the future as senior field office staff are expected to become retirement eligible. In a January 2013 report to DOE’s Federal Technical Capability Panel, one field office reported that its staffing levels were less than the number required to perform the oversight identified as necessary. This field office noted that staffing shortages were offset through support from other offices and increased reliance on contractor- generated information from CAS. The 2013 report did not indicate if the field office’s increased reliance on information from CAS for oversight was supported by the field office’s analysis of the risk of the activity, the maturity level of the contractor’s CAS, and contractor performance in the area. We found that NNSA has not assessed whether it has sufficient, qualified personnel to implement the oversight framework described in NAP-21.

We recommended that NNSA assess staffing needs to determine whether it has sufficient qualified personnel to conduct oversight activities consistent with comprehensive polices and guidance, including use of information for CAS. NNSA concurred with our recommendation and stated that it will assess staffing needs by December 2016, to allow for field level policies and procedures to be considered in the development of the staffing strategy.

NNSA Discontinued the Process for Headquarters Review of Field Offices’ Oversight Approaches

We also found that NNSA headquarters discontinued affirmation reviews (the process established by NAP-21 for reviewing the effectiveness of contractors’ CAS implementation and field offices’ oversight approaches) effectively eliminating the primary internal control activity that NAP-21 included for the agency to evaluate oversight effectiveness across the nuclear security enterprise. Prior to discontinuing this process, NNSA conducted affirmation reviews at three sites—Sandia National Laboratories, the Nevada National Security Site, and the Y-12 National Security Complex—and all three reviews resulted in affirmations of the effective implementation of the contractor’s CAS and of the federal oversight approach. However, following the 2012 security incident at Y- 12—which occurred after NNSA affirmed the implementation and reliability of the contractor’s CAS and the effectiveness of the Y-12 field office’s mix of oversight approaches—NNSA discontinued its affirmation review process. According to NNSA officials, after investigating the root causes for the security lapse at Y-12, NNSA determined that its affirmation reviews focused too heavily on affirming that a CAS existed and covered the five required attributes of a CAS as outlined in NAP-21. According to NNSA officials, the affirmation reviews did not focus enough on evaluating the effectiveness of either the contractor’s CAS or the field office’s approach to determining the appropriate mix of systems- and transaction-based oversight.

After discontinuing the affirmation reviews, NNSA initiated an Oversight Improvement Project to focus on evaluating the effectiveness of contractors’ CAS and field offices’ oversight approaches. However, a senior NNSA official told us the project was never completed, and NNSA has not developed another process in lieu of affirmation reviews. Discontinuing affirmation reviews without replacing them with another form of validation eliminates the internal control activity in NAP-21 to provide NNSA with assurance of oversight effectiveness across the nuclear security enterprise. Further, continuing a process to review the effectiveness of oversight approaches would have provided information allowing for oversight practices to be compared across field offices and for differences among them to be evaluated. According to NNSA headquarters and field officials, there is no current mechanism for this to occur.

We recommended that NNSA reestablish a process for reviewing the effectiveness of field offices’ oversight approaches, including their determinations for how and when to use information from CAS. NNSA concurred with our recommendation and stated that its new corporate policy and guidance will outline such an approach for validating the effectiveness of the field office oversight activities and estimated the completion for the effort to be March 2016.

Preliminary Observations from Ongoing Work on DOE’s Processes for Oversight of WIPP

Our preliminary observations on NNSA’s oversight of waste packaging activities at LANL parallel two of the findings from our recently released May 2015 report. Our preliminary observations are based on our review of specific sections of DOE’s Phase II accident investigation board report on the radiological release.  First, with regard to our finding that NNSA has not fully established policies or guidance for using information from CAS to conduct oversight of M&O contractors, the accident investigation board report on WIPP found that NNSA’s Los Alamos Field Office was overreliant on CAS for environmental compliance oversight. The accident investigation board report also found that the field office did not adequately conduct transactional assessments of the contractor in areas such as environmental compliance and operations of the LANL facility where the TRU waste container that breached was processed and packaged. According to the accident investigation board report, the NNSA field office’s overreliance on the contractor-generated information in CAS was not consistent with a 2011 NNSA review that observed CAS was still maturing and that a strong NNSA field office oversight presence should continue. Moreover, the accident investigation board identified specific deficiencies in CAS such as inadequate contractor self-assessments regarding waste processing and packaging and concluded that CAS was not effective in identifying weaknesses that contributed to the incident. Under the oversight framework, determining that a CAS is not fully mature would result in a heavier reliance on transactions-based approaches to overseeing LANL’s waste packaging operations.  Second, with respect to our finding that NNSA field office officials have raised concerns that staffing levels and the mix of staff skills may not be adequate to conduct appropriate oversight—which may result in overreliance on information from CAS—according to the accident investigation board report on WIPP Los Alamos Field Office officials attributed their overreliance on the information in CAS for environmental compliance oversight to a lack of resources to directly perform this oversight. The report also found that the field office did not have senior technical expertise, such as organic chemistry expertise, necessary for conducting adequate technical reviews related to the contractor’s processing of the TRU wastes, which were the source of the radiological release at WIPP.

Our preliminary observations on DOE’s processes for overseeing the contractor responsible for managing and operating WIPP indicate that EM has not outlined an EM-specific policy framework for its field office officials to use in establishing and implementing effective oversight programs beyond the 2011 DOE oversight policy and order. However incomplete, NNSA developed NAP-21 in an effort to elaborate on DOE’s policy and order by providing an NNSA-specific oversight policy framework that included a three-pronged evaluation framework for determining an appropriate oversight approach. EM headquarters officials told us that EM does not provide its field offices supplemental EM-specific policy or other formal direction on how to use the broad DOE oversight order but encourages them to use DOE’s oversight guide focused on nuclear safety that provides suggested, not mandatory, approaches to designing and implementing field office oversight programs. For example, this guide describes site-specific conditions that field offices should consider in establishing oversight priorities and allocating oversight resources, including consideration of the types of nuclear facilities and their hazards and the status and effectiveness of the contractor’s CAS. At this point in our ongoing review, we are not aware of examples of direction provided to EM field offices to oversee M&O contractors’ performance in areas other than nuclear safety, such as business operations or safeguards and security. EM headquarters officials told us that EM provides field offices with evaluation guides that they can use to develop their evaluations of specific elements of a contractor’s nuclear facility safety program, such as the WIPP’s M&O contractor’s CAS. We have not yet evaluated the DOE oversight guide, EM’s reliance on it, or the evaluation guides EM has developed for its field offices, but we will do so as we complete our work.

In conclusion, GAO has reported for years on the management challenges DOE faces, as well as specific safety and security incident such as the recent accident at WIPP. DOE’s management and oversight reform efforts have sought to address the conditions underlying safety and security failures, but recent events at WIPP show that more work is needed. Our recently released report concludes that NNSA does not have complete standards against which to measure whether oversight approaches are effective, including how information from CAS is being used for oversight. This is because NNSA does not have complete policy or guidance to implement the oversight framework and has discontinued its reviews intended to evaluate the effectiveness of field offices’ oversight approaches; also, in the absence of headquarters policy or guidance, its field offices have developed procedures that are not fully complete and differ. As a result, NNSA runs the risk of not using its oversight resources effectively, either by underutilizing information from CAS and missing opportunities for efficiency, or by overrelying on information from CAS and possibly missing contractor performance issues that put safety, security, or mission accomplishment at risk. With respect to the recent events at WIPP, these issues concern DOE as well.

Chairman Murphy, Ranking Member DeGette, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to answer any questions that you may have at this time.

GAO Contacts and Staff Acknowledgments

If you or your staff members have any questions concerning this testimony, please contact me at (202) 512-3841 or bawdena@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Other individuals who made key contributions include David Trimble, Director; Daniel Feehan, Assistant Director; David Bennett; Richard Burkard; John Delicath; Brian M. Friedman; Carly Gerbig; Christopher Pacheco; Eli Lewine; Rebecca Shea; Rajneesh Verma; and Kiki Theodoropoulos.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.