Yucca Mountain: Quality Assurance at DOE's Planned Nuclear Waste Repository Needs Increased Management Attention
Published: 2006-03-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Energy (DOE) is working to obtain a license from the Nuclear Regulatory Commission (NRC) to construct a nuclear waste repository at Yucca Mountain in Nevada. The project, which began in the 1980s, has been beset by delays. In a 2004 report, GAO raised concerns that persistent quality assurance problems could further delay the project. Then, in 2005, DOE announced the discovery of employee e-mails suggesting quality assurance problems, including possible falsification of records. Quality assurance, which establishes requirements for work to be performed under controlled conditions that ensure quality, is critical to making sure the project meets standards for protecting public health and the environment. GAO was asked to examine (1) the history of the project's quality assurance problems, (2) DOE's tracking of these problems and efforts to address them since GAO's 2004 report, and (3) challenges facing DOE as it continues to address quality assurance issues within the project.
section_title: What GAO Found
paragraphs: DOE has had a long history of quality assurance problems at the Yucca Mountain project. In the 1980s and 1990s, DOE had problems assuring NRC that it had developed adequate plans and procedures related to quality assurance. More recently, as it prepares to submit a license application for the repository to NRC, DOE has been relying on costly and time-consuming rework to resolve lingering quality assurance problems uncovered during audits and after-the-fact evaluations. DOE announced, in 2004, that it was making a commitment to continuous quality assurance improvement and that its efforts would be tracked by performance indicators that would enable it to assess progress and direct management attention as needed. However, GAO found that the project's performance indicators and other key management tools were not effective for this purpose. For example, the management tools did not target existing areas of concern and did not track progress in addressing them. The tools also had weaknesses in detecting and highlighting significant problems for management attention. DOE continues to face quality assurance and other challenges. First, DOE is engaged in extensive efforts to restore confidence in scientific documents because of the quality assurance problems suggested in the discovered e-mails between project employees, and it has about 14 million more project e-mails to review. Second, DOE faces quality assurance challenges in resolving design control problems associated with its requirements management process--the process for ensuring that high-level plans and regulatory requirements are incorporated into specific engineering details. Problems with the process led to the December 2005 suspension of certain project work. Third, DOE continues to be challenged to manage a complex program and organization. Significant personnel and project changes initiated in October 2005 create the potential for confusion over roles and responsibilities--a situation DOE found to contribute to quality assurance problems during an earlier transition.

section_title: Letter
paragraphs: The nuclear waste created as a by-product of the nuclear power process in  reactors can remain highly radioactive for hundreds of thousands of years,  and will require proper disposal to protect public health and the  environment. Over 50,000 metric tons of this waste is currently being  stored at 72 sites around the country, principally at commercial nuclear  power plants. These wastes have been accumulating for several decades in  surface storage designed to be temporary. The Nuclear Waste Policy Act of  1982 obligated the Department of Energy (DOE) to construct an  underground geological repository for permanent storage and begin  accepting these wastes by January 31, 1998. However, it was not until 2002,  after more than 15 years of scientific study, that Yucca Mountain in Nevada  was approved by Congress as a suitable location for the repository. DOE is  continuing to experience delays, and it does not currently have a schedule  for when construction of the repository will begin. The project to build and  operate a repository at Yucca Mountain is highly complex. It is also highly  controversial among some of the public, in large part, because of their  concern that the repository may not be adequate, over the long term, to  prevent the release of radioactive material to the environment. DOE has  established quality assurance procedures to ensure that its work relative to  the project and the technical information it produces are of high quality and  defensible. However, persistent problems with implementing these  procedures and resulting questions about the quality of the work have  significantly contributed to project delays. Resolving these quality issues is  essential to proceeding with construction.
To construct a repository at the Yucca Mountain site, DOE must obtain a  license from the Nuclear Regulatory Commission (NRC). As part of the  licensing process, DOE must demonstrate to NRC that its plans for the  repository will meet standards for protecting public health and the  environment from harmful exposure to the radioactive waste. The  Environmental Protection Agency (EPA) set these standards in 2001, but as  a result of a 2004 court ruling, EPA is proposing to revise the standards to  extend the protection period from 10,000 years to 1 million years.
To demonstrate that it can meet these standards, DOE has been conducting  scientific and technical studies at the Yucca Mountain site that will serve as  supporting documentation for DOE’s planned license application. For  example, it has developed mathematical models to measure the probability  that various combinations of natural and engineered (human-made)  features of a repository could safely contain waste for the long term; the  models take into account possible water infiltration through the mountain  (see fig. 1), earthquakes, volcanic action, or other scenarios. Thus, one of  DOE’s most important tasks during the licensing process will be to  demonstrate the adequacy of its data, software, and models. Accordingly,  NRC requires nuclear facilities to develop a quality assurance program that  ensures that the technical information submitted in support of a license  application—such as scientific data, models, and details on design and  construction—is well documented and defensible. More specifically, data  used to support conclusions about the safety and design of the repository  must meet transparency and traceability standards. That is, the data must  be clear in justifying and explaining any underlying assumptions,  calculations, and conclusions, and must be capable of being traced back to  original source materials.
To meet NRC’s requirements, DOE established a quality assurance program  for the Yucca Mountain project. The program establishes requirements that  scientific, design, engineering, and other work, such as procurement and  record keeping, is to be performed under controlled conditions that ensure  quality and enable the work to be verified by others. For example, the  program establishes general requirements for calibrating equipment before  conducting tests, stipulating when and how the equipment should be  calibrated and how to document the results. The project’s line  organizations, which are responsible for carrying out various functions or  aspects of the work, then create their own policies and procedures to  implement the requirements.
Project employees are required to follow such procedures to help ensure  the reliability of project information. Quality assurance auditors  periodically verify that the procedures have been followed. Project  employees, including quality assurance auditors, are required to identify  when procedures are not being followed or when they encounter problems  with the procedures. These problems can be identified in “condition  reports” under the project’s Corrective Action Program, which establishes  procedures for the prompt identification and correction of problems.  Alternatively, project employees can submit problems for resolution  through the Employee Concerns Program, which allows for submissions to  be confidential or anonymous.
Because quality assurance plays a key role in ensuring that the information  DOE uses to support its license application is of high quality and fully  defensible, problems in this area raise concerns about delays to DOE’s  submission and NRC’s review of the license application. In April 2004, for  example, we reported that recurring quality assurance problems at the  Yucca Mountain project could delay the licensing and operation of the  repository. As we noted, a 2004 NRC evaluation found quality assurance  problems such as data that could not be readily traced back to their  sources. NRC indicated that unless DOE rectified such problems before  submitting the license application, NRC could be in the position of  requesting large volumes of additional information, which could prevent it  from making a decision on the license within the time required by law.  Then, in early 2005, DOE reported it had discovered a series of e-mail  messages among some U.S. Geological Survey (USGS) employees working  on the Yucca Mountain project under a contract with DOE that appeared to  imply that workers had falsified records for scientific work. Several of  these messages, written in the late 1990s, appeared to show disdain for the  project’s quality assurance program and its requirements. As a result of  these e-mails, DOE is engaging in an extensive review of records to restore  confidence in scientific documents that will be used to support its license  application.
DOE’s recent efforts to better manage quality assurance problems include  its Management Improvement Initiatives (Initiatives), which began in 2002  and were reported completed in April 2004. The Initiatives’ purpose was to  ensure that work and products consistently met quality objectives and  were fully defensible by establishing a foundation for continuous  improvement in areas of identified management weaknesses. In our 2004  report, we concluded that, while DOE considered the Initiatives to have  been completed, it could not assess their effectiveness in addressing the  management weaknesses because its performance goals lacked objective  measures and time frames for determining success. By the end of the  Initiatives, DOE had established two tools to alert management about  quality-related and other problems: (1) a one-page summary of  performance indicators for key project activities and processes (the  summary, which DOE refers to as a “panel,” is prepared monthly for  discussion and action by project managers) and (2) quarterly trend  evaluation reports analyzing patterns and trends in problems identified  through the Corrective Action Program. Then, in October 2005, DOE  initiated planning for an aggressive series of changes to the facility design,  organization, and management of the Yucca Mountain project. This effort,  known as the “new path forward,” is intended to address quality assurance  and other challenges prior to submission of a license application.  According to the project’s Acting Director, DOE will be considering  changes in performance indicators and other management tools to better  support the new path forward.
In this context, you requested that we provide additional information on  the project’s quality assurance problems and DOE’s efforts to correct them.  As agreed with your office, this report discusses (1) the history of the  project’s quality assurance problems since its start in the 1980s, (2) DOE’s  tracking of quality problems and progress implementing quality assurance  requirements since our April 2004 report, and (3) challenges DOE faces as  it continues to address quality assurance issues at the project. In addition,  you asked for information about concerns raised in recent years through  the project’s Employee Concerns Program, which is provided in   appendix II.
To determine the history of quality assurance problems, we reviewed  previous GAO, DOE, and NRC documents, visited the project, and  interviewed officials from DOE, NRC, and Bechtel/SAIC Company, LLC  (BSC), which is DOE’s management contractor for the Yucca Mountain  project. To assess DOE’s tracking of quality-related problems and progress  in addressing them, we examined management tools and associated  documentation, such as monthly indicator panels and quarterly trend  reports, and interviewed BSC and DOE officials regarding those tools. To  identify current quality assurance and other challenges, we attended  quarterly NRC management meetings, interviewed the Acting Director and  other senior managers of the DOE project, and gathered information on  management turnover. Due to the criminal investigation under way related  to possible falsification of records implied in USGS e-mail exchanges, we  did not examine the investigated issues beyond confirming that a concern  about the e-mails had been submitted to the Employee Concerns Program.  However, to determine if concerns about other instances of potential  falsification of records had been raised by project employees, we reviewed  employee concerns filed with the project’s Employee Concerns Program  from January 2004 to December 2005. More information on our scope and  methodology is provided in appendix I. We conducted our work from July  2005 through January 2006 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: Congress enacted the Nuclear Waste Policy Act of 1982 to establish a  comprehensive policy and program for the safe, permanent disposal of  commercial spent fuel and other highly radioactive wastes in one or more  mined geologic repositories. The act charged DOE with (1) establishing  criteria for recommending sites for repositories; (2) “characterizing”  (investigating) three sites to determine each site’s suitability for a  repository (1987 amendments to the act directed DOE to investigate only  the Yucca Mountain site); (3) recommending one suitable site to the  President, who, if he considered the site qualified for a license application,  would submit a recommendation to Congress; and (4) seeking a license  from NRC to construct and operate a repository at the approved site. The  act created the Office of Civilian Radioactive Waste Management within  DOE to manage its nuclear waste program.
Since the 1980s, DOE has spent years conducting site characterization  studies at the Yucca Mountain site to determine whether it is suitable for a  high-level radioactive waste and spent nuclear fuel repository. DOE, for  example, has completed numerous scientific studies of the mountain and  its surrounding region for water flow and the potential for rock movement,  including volcanoes and earthquakes that might adversely affect the  performance of the repository. To allow scientists and engineers greater  access to the rock being studied, DOE excavated two tunnels for studying  the deep underground environment: (1) a five-mile main tunnel that loops  through the mountain, with several research areas or alcoves connected to  it; and (2) a 1.7-mile tunnel that crosses the mountain (see fig. 2). This  second tunnel allows scientists to study properties of the rock and the  behavior of water near the potential repository area. In July 2002, Congress  approved the President’s recommendation of the Yucca Mountain site for  the development of a repository.
The Yucca Mountain project is currently focused on preparing an  application to obtain a license from NRC to construct a repository. The  required application information includes both repository design work and  scientific analyses. DOE is engaged in necessary tasks such as compiling  information and writing sections of the license application, and is  conducting technical exchanges with NRC staff and addressing key  technical issues identified by NRC to ensure that sufficient supporting  information is provided. It also plans to further develop the design of the  repository, including revised designs for the repository’s surface facilities  and canisters to hold the waste. DOE is also identifying and preparing  potentially relevant documentary material that it is required to make  available on NRC’s Web-based information system, known as the Licensing  Support Network. This is a critical step because DOE is required to certify  that the documentary material has been identified and made electronically  available no later than 6 months in advance of submitting the license  application.
In February 2005, DOE announced that it does not expect the repository to  open until 2012 at the earliest, which is more than 14 years later than the  1998 goal specified by the Nuclear Waste Policy Act of 1982. More recently,  the conference report for DOE’s fiscal year 2006 appropriations observed  that further significant schedule slippages for submitting a license  application are likely. Further delays could arise from factors such as the  time needed for EPA to establish revised radiation standards for Yucca  Mountain and for DOE to revise its technical documents in response. Such  delays could be costly because nuclear utilities, which pay for most of the  disposal program through a fee on nuclear power, have sued DOE, seeking  damages for not starting the removal of spent nuclear fuel from storage at  commercial reactors by the 1998 deadline. Estimates of the potential  damages vary widely, from DOE’s estimate of about $5 billion to a nuclear  industry’s estimate of about $50 billion, but the cost for the damages will  likely rise if there are further delays to opening the repository.
Given these schedule slippages, Congress has considered other options for  managing existing and future nuclear wastes, such as centralized interim  storage at one or more DOE sites. The conference report for DOE’s fiscal  year 2006 appropriations directed DOE to develop a spent nuclear fuel  recycling plan to reuse the fuel. However, according to the policy  organization of the nuclear energy industry, no technological option  contemplated will eliminate the need to ultimately dispose of nuclear waste  in a geologic repository.
In October 2005, the project’s Acting Director issued a memorandum  calling for the development of wide-ranging plans for the “new path  forward,” DOE’s effort to address quality assurance and other challenges  prior to applying for a license. To restore confidence in scientific  documents that will support the license application, some of the plans will  address the need to review and replace USGS work products, a  requirement for USGS to certify its scientific work products, and  establishing a lead national laboratory to assist the project. Other plans are  focused on a new simplified design for the waste canisters and repository  facilities, a design that is expected to improve the safety and operation of  the repository by eliminating the need to directly handle and process the  spent fuel at the repository. Further, this aggressive effort called for  management changes, including a transition plan; more rigorous project  management, including a new baseline schedule; rescoping existing  contracts and developing new contracts; tracking project hiring actions; a  financial plan; and new reporting indicators.
After DOE submits the license application, NRC plans to take 90 days to  examine the application for completeness to determine whether DOE has  addressed all NRC requirements. One of the reviews for completeness will  include an examination of DOE’s documentation of the quality assurance  program to assess whether it addresses all NRC criteria. These criteria  include, among other things, organization, design control, document  control, corrective actions, quality assurance records, and quality audits. If  it deems any part of the application is incomplete, NRC may either reject  the application or require that DOE furnish the necessary documentation  before proceeding with the detailed technical review of the application. If it  deems the application is complete, NRC will docket the application,  indicating its readiness for a detailed technical review.
Once the application is accepted and placed on the docket, NRC will  conduct its 18-month technical review of the application to determine if the  application meets all NRC requirements, including the soundness of  scientific analyses and preliminary facility design, and NRC quality  assurance criteria. If NRC discovers problems with the technical  information used to support the application, it may conduct specific  reviews, including inspections, to determine the extent and effect of the  problem. Because the data, models, and software used in modeling  repository performance are integral parts of this technical review, quality  assurance plays a key role since it is the mechanism used to verify the  accuracy of the information DOE presents in the application. NRC may  conduct reviews, including inspections, of the quality assurance program if  technical problems are identified that are attributable to quality problems.  NRC will hold public hearings chaired by its Atomic Safety and Licensing  Board to examine specific topics. After completing the proceedings, the  board will forward its initial decision to the NRC commissioners for their  review. Finally, within 3 to 4 years from the date that NRC dockets the  application, NRC will make a decision to grant the construction  authorization, reject the application, or grant the construction  authorization with conditions. NRC will grant a construction authorization  only if it concludes from its reviews that the repository would meet its  reasonable expectation that the safety and health of workers and the public  would be protected.
section_title: DOE Has a Long History of Quality Assurance Problems at Yucca Mountain and Is Relying on Costly and Time-Consuming Measures to Correct Problems Before Submitting Its License Application for the Repository
paragraphs: DOE has repeatedly experienced quality assurance problems with its work  on the Yucca Mountain project. In the late 1980s, DOE had been challenged  to fix and develop adequate plans and procedures related to quality  assurance. By the late 1990s, audits by GAO, DOE, and others identified  recurring quality assurance problems with several aspects of key scientific  data, models, and software. Currently, in preparing to submit the license  application to NRC, DOE is relying on costly and time-consuming rework  to resolve lingering quality assurance problems with the transparency and  traceability of data and in project design and engineering documents  uncovered during audits and after-the-fact evaluations.
subsections: section_title: DOE Has Had Problems Implementing and Maintaining an Effective Quality Assurance Program
paragraphs: DOE has a long-standing history of attempting to address NRC concerns  about its quality assurance program. Although NRC will have responsibility  for regulating the construction, operation, and decommissioning (closure)  phases of the project, its regulatory and oversight role does not begin until  DOE submits a license application. As a result, NRC’s role in the project has  been limited to providing guidance to DOE to ensure an understanding of  NRC regulations and that the years of scientific and technical work will not  later be found inadequate for licensing purposes. Specifically, since 1984,  NRC has agreed to point out problems it identifies with the quality  assurance program so that DOE can take timely corrective action. Initially,  this NRC guidance was mainly focused on ensuring that DOE had the  necessary quality assurance organization, plans, and procedures.
As we reported in 1988, NRC had reviewed DOE’s quality assurance plans  and procedures comprising the principal framework of its quality  assurance program, and concluded that they were inadequate and did not  meet NRC requirements. NRC also concluded that DOE’s efforts to  independently identify and resolve weaknesses in the plans and procedures  were ineffective. After observing DOE quality assurance audits, NRC  determined that the audits were ineffective for measuring whether quality  assurance procedures were being effectively implemented. Further, NRC  identified additional concerns, during the 1980s, related to DOE  management and organizational deficiencies relating to the quality  assurance program. Specifically, among other things, NRC found the  following:  DOE had a small staff and relied heavily on contractors to provide  quality assurance oversight. Based on its experience in regulating  nuclear power plants, NRC found that these types of organizations  frequently developed major quality-related problems.
DOE had indirect project control, with administrative and functional  control over the project split between different offices. NRC found that  such project control arrangements tend to have serious quality  assurance-related problems because conflicts can arise between quality  and other organizational goals, such as cost and schedule.
During a 1984 NRC visit to Nevada, DOE project participants had  expressed the opinion that quality assurance is “unnecessary,  burdensome, and an imposition.” Further, in 1986, DOE issued a stop- work order to the USGS based on a determination that USGS staff did  not appreciate the importance of quality assurance and that USGS work  would not meet NRC expectations. NRC believed that organizational  attitudes can indicate whether a project is likely to experience problems  relating to quality assurance and found such examples troublesome.
Finally, based in part on the information obtained from its oversight  activities, NRC concluded, in 1989, that DOE and its key contractors had  yet to develop and implement an acceptable quality assurance program.
However, by March 1992, NRC came to the conclusion that DOE had made  significant progress in improving its quality assurance program. NRC noted  that DOE had addressed many of its concerns, specifically that, among  other things, (1) all of the contractor organizations had developed and were  in the process of implementing quality assurance programs that met NRC  requirements, (2) quality assurance management positions had been filled  with full-time DOE personnel with appropriate knowledge and experience,  and (3) DOE had demonstrated that it is capable of evaluating and  correcting deficiencies in the overall quality assurance program.  Nevertheless, in October 1994, NRC found problems with quality  assurance, particularly with the site contractor’s ability to effectively  implement corrective actions and DOE’s ability to oversee the site  contractor’s quality assurance program.
section_title: Recurring Issues with Project Data, Models, and Software Illustrate DOE’s Difficulties Addressing Quality Assurance Problems
paragraphs: As DOE’s quality assurance program matured, it resolved NRC concerns  about its organization, plans, and procedures, and in the late 1990s began  successfully detecting new quality assurance problems in three areas  critical to the repository’s successful performance: the adequacy of the  data sources, the validity of scientific models, and the reliability of  computer software developed at the site. These problems surfaced in 1998  when DOE began to run the initial version of its performance assessment  model. Specifically, DOE was unable to ensure that critical project data had  been collected and tracked back to the original sources. In addition, DOE  did not have a standardized process for developing scientific models used  to simulate a variety of geologic events or an effective process for ensuring  that computer software used to support the scientific models would work  properly. As required by DOE’s quality assurance procedures, the  department conducted a root cause analysis and issued a corrective action  plan in 1999. After corrective actions were taken, DOE considered the  issues resolved.
However, in 2001, similar deficiencies associated with models and software  resurfaced. DOE attributed the recurrence to ineffective procedures and  corrective actions, improper implementation of quality procedures by line  managers, and personnel who feared reprisal for expressing quality  concerns. Recognizing the need to correct these recurring problems, DOE  conducted a comprehensive root cause analysis that included reviews of  numerous past self-assessments and independent program assessments,  and identified weaknesses in management systems, quality processes, and  organization roles and responsibilities. Following the analysis, in July 2002,  DOE issued its Management Improvement Initiatives (Initiatives) that  addressed quality problems with software and models. In addition, DOE  added other corrective actions to address management weaknesses that it  found in areas such as roles and responsibilities, quality assurance  processes, written procedures, corrective action plans, and work  environment.
However, DOE continued to face difficulties in resolving quality assurance  problems concerning the data, software, and modeling to be used in  support of the licensing application:  Data management. As part of NRC’s quality assurance requirements,  data used to support conclusions about the safety and design of the  repository must be either collected under a quality assurance program  or subjected to prescribed testing procedures to ensure the data are  accurate for their intended use. In addition, the data supporting these  conclusions must also be traceable back to its original source. In 1998,  DOE identified quality assurance problems with the quality and  traceability of data—specifically that some data had not been properly  collected or tested to ensure their accuracy and that data used to  support scientific analysis could not be properly traced back to their  source. DOE again found similar problems in April and September 2003,  when a DOE audit revealed that some data sets did not have the  documentation necessary to trace them back to their sources; the  processes for data control and management were unsatisfactory; and  faulty definitions were developed, which allowed unqualified data to be  used.
Software management. DOE quality assurance procedures require that  software used to support analysis and conclusions about the  performance and safety of the repository be tested or created in such a  way to ensure that it is reliable. From 1998 to 2003, multiple DOE audits  found recurring quality assurance problems that could affect confidence  in the adequacy of software codes. For example, in 2003, DOE auditors  found problems related to software similar to those found previously in  areas such as technical reviews, software classification, planning,  design, and testing. Further, a team of industry professionals hired by  DOE to assess quality assurance problems with software reported in  February 2004 that these problems kept recurring because DOE did not  assess the effectiveness of its corrective actions and did not adequately  identify the root causes of the problems.
Model validation. Models are used to simulate natural and  environmental conditions at Yucca Mountain, and to demonstrate the  performance of the future repository over time. However, before models  can be used to support the license application, DOE must demonstrate  through a process called validation that the models are able to  accurately predict geologic events. In 1998, a team of project personnel  evaluated the models and determined that 87 percent did not comply  with the validation requirements. In 2001, and again in 2003, DOE audits  found that project personnel were not properly following procedures— specifically in the areas of model documentation, model validation, and  checking and review. Further, the 2003 audit concluded that previous  corrective actions designed to improve validation and reduce errors in  model reports were not fully implemented.
After many years of working to address these quality assurance problems  with data, software, and models, DOE had mostly resolved these problems  and closed the last of the associated condition reports by February 2005.
section_title: DOE Is Now Relying on Costly and Time-Consuming Rework to Resolve Additional Problems
paragraphs: As DOE prepares to submit the Yucca Mountain project license application  to NRC, it has relied on costly and time consuming rework to ensure that  the documents supporting the application are accurate and complete.  Specifically, DOE has relied on inspections and rework by DOE personnel  to resolve quality assurance problems with the traceability and  transparency of technical work products. These efforts to deal with quality  problems at the end, rather than effectively ensuring that work  organizations are producing quality products from the beginning, add to the  project’s cost and could potentially delay DOE’s submission of the license  application to NRC. In addition, DOE’s efforts indicate that some corrective  actions have been ineffective in resolving problems with the quality  assurance process. Further, DOE is now detecting quality assurance  problems in design and engineering work that are similar to the quality  assurance problems it experienced with its scientific work in the late  1990s.
Although DOE did not initiate its major effort to address these problems  until 2004, the department and NRC for years had known of quality  assurance problems with the traceability and transparency of technical  work products called Analysis and Model Reports (AMR). AMRs are a key  component of the license application, and contain the scientific analysis  and modeling data demonstrating the safety and performance of the  planned repository. Among other quality requirements, AMRs must be  traceable back to their original source material and data, and must also be  transparent in justifying and explaining their underlying assumptions,  calculations, and conclusions. In 2003, based in part on these problems as  well as DOE’s long-standing problems with data, software, and modeling,  NRC conducted an independent evaluation of three AMRs. The scope of the  review was to determine if the AMRs met NRC requirements for being  traceable, transparent, and technically appropriate for their use in the  license application. NRC found significant problems. First, in some cases  DOE was not transparent in explaining the basis on which it was reaching  conclusions. For example, in some circumstances, DOE selected a single  value from a range of data without sufficient justification. Other times,  DOE did not explain how a range of experimental conditions were  representative of repository conditions. Second, where DOE did  sufficiently explain the basis for a conclusion, it did not always provide the  necessary technical information, such as experimental data, analysis, or  expert judgment, to trace the support for that explanation back to source  materials. For example, DOE did not explain how information on one type  of material provided an appropriate comparison for another material.  Moreover, while DOE had identified similar problems in the past, the  actions taken to correct them did not identify and resolve other  deficiencies. NRC concluded that these findings suggested that other AMRs  possibly had similar problems, and that if not resolved, such problems  could delay NRC’s review of the license application as it would need to  conduct special inspections to resolve any issues it found with the quality  of technical information.
To address problems of traceability and transparency, DOE in the spring of  2004 initiated an effort called the Regulatory Integration Team (RIT) to  perform a comprehensive inspection and rework of the AMRs to ensure  they met NRC requirements and expectations. According to DOE officials,  the RIT involved roughly 150 full-time personnel from DOE, USGS, and  multiple national laboratories such as Sandia, Los Alamos, and Lawrence  Livermore. First, the RIT screened all of the approximately 110 AMRs and  prioritized its efforts on 89 that needed additional rework. Ten AMRs were  determined to be acceptable, and 11 were canceled because they were no  longer needed to support the license application. According to DOE  officials, approximately 8 months later, the RIT project was completed at a  cost of about $20 million, with a total of over 3,700 problems and issues  addressed or corrected. In February 2005, in a letter to DOE, the site  contractor stated that the RIT effort was successful and that the AMRs had  been revised to improve traceability and transparency.
Subsequently, however, additional problems with traceability and  transparency have been identified, requiring further inspections and  rework. For example, after the March 2005 discovery of e-mails from USGS  employees written between May 1998 and March 2000 implying that  employees had falsified documentation of their work to avoid quality  assurance standards, DOE initiated a review of additional AMRs that were  not included in the scope of the 2004 RIT review. The additional AMRs  contained scientific work performed by the USGS employees and had been  assumed by the RIT to meet NRC requirements for traceability and  transparency. However, according to DOE officials, DOE’s review  determined that these AMRs did not meet NRC’s standards, and additional  rework was required. Further, similar problems were identified as the focus  of the project shifted to the design and engineering work required for the  license application. In February 2005, the site contractor determined that in  addition to problems with AMRs, similar traceability and transparency  problems existed in the design and engineering documents that comprise  the Safety Analysis Report—the report necessary for demonstrating to  NRC how the facilities and other components of the repository site will  meet the project’s health, safety, and environmental goals and objectives. In  a root cause analysis of this problem, the site contractor noted that  additional resources were needed to inspect and rework the documents to  correct the problems.
section_title: DOE Cannot Be Certain Its Efforts to Improve Quality Assurance Have Been Effective Because of Weaknesses in Tracking Progress and Identifying Problems
paragraphs: DOE cannot be certain that it has met continuous improvement goals for  implementing its quality assurance requirements, a commitment DOE  made at the closure of its Management Improvement Initiatives (Initiatives)  in April 2004. At that time, DOE told us it expected that the progress  achieved with the initiatives would continue and that its performance  indicators would enable it to assess further progress and direct  management attention as needed. However, DOE’s performance indicators,  as well as a second management tool—trend evaluation reports—have not  been effective for this purpose. More specifically, the indicators panel did  not highlight the areas of concern covered by the initiatives and had  weaknesses in assessing progress because the indicators kept changing.
The trend evaluation reports also did not focus on tracking the concerns  covered by the Initiatives, had technical weaknesses for identifying  significant and recurring problems, had inconsistently tracked progress in  addressing problems, and could not fully analyze projectwide problems.  In addition, the trend reports’ tracking of problems for which corrective  actions were already being taken was at times overly influenced by  judgments about whether additional management action was warranted  rather than the problems’ significance.
subsections: section_title: The Panel’s Focus and Frequent Changes Hindered the Tracking of Progress with Management Concerns and Quality Problems
paragraphs: By the time that the actions called for by the Initiatives had been completed  in April 2004, project management had already developed the indicators  panel, which DOE refers to as the annunciator panel, to use at monthly  management meetings to monitor project performance. The panel was a  single page composed of colored blocks representing selected performance  indicators and their rating or level of performance. A manager viewing the  panel would be able to quickly see the color rating of each block or  indicator. For example, red indicated degraded or adverse performance  warranting significant management attention; yellow indicated  performance warranting increased management attention or acceptable  performance that could change for the worse; and green indicated good  performance. The panel represented a hierarchy of indicators in which the  highest level, or primary, indicators were shown; secondary indicators that  determined the primary indicators’ ratings were shown for some primary  indicators; but lower third- or fourth-level indicators were not shown. Our  review analyzed a subset of these indicators that DOE designated as the  indicators that best predict performance in areas affecting quality. While  we were conducting our review, DOE suspended preparation of the panel  after August 2005 while it reconsiders its use of indicators to monitor  project performance. DOE had also suspended preparation of the panel  from late 2004 to early 2005 in order to make substantial revisions. These  revisions were made, in part, to emphasize fewer, more important  indicators for management attention.
The Initiatives raised concerns about five key areas of management  weakness as adversely affecting the implementation of quality assurance  requirements: 1. Roles and responsibilities were becoming confused as the project  transitioned from scientific studies to activities supporting licensing.  The confusion over roles and responsibilities was undermining  managers’ accountability for results. The Initiatives’ objective was to  realign DOE’s project organization to give a single point of  responsibility for project functions, such as quality assurance and the  Corrective Action Program, and hold the project contractor more  accountable for performing the necessary work in accordance with  quality, schedule, and cost requirements.  2. Product quality was sometimes being achieved through inspections by  the project’s Office of Quality Assurance rather than being routinely  implemented by the project’s work organizations. As a result, the  Initiatives sought to increase work organizations’ responsibility for  being the principle means for achieving quality. 3. Work procedures were typically too burdensome and inefficient, which  impeded work. The Initiatives sought to provide new user-friendly and  effective procedures, when necessary, to allow routine compliance with  safety and quality requirements.  4. Multiple corrective action programs existed, processes were  burdensome and did not yield useful management reports, and  corrective actions were not completed in a timely manner. The  Initiatives sought to implement a single program to ensure that  problems were identified, prioritized, and documented and that timely  and effective corrective actions were taken to preclude recurrence of  problems.  5. The importance of a safety-conscious work environment that fosters  open communication about concerns was not understood by all  managers and staff, and they had not been held accountable when  inappropriately overemphasizing the work schedule, inadequately  attending to work quality, and acting inconsistently in practicing the  desired openness about concerns. Through issuing a work environment  policy, providing training on the policy, and improving the Employee  Concerns Program, the Initiatives sought to create an environment in  which employees felt free to raise concerns without fear of reprisal and  with confidence that issues would be addressed promptly and  appropriately.
As shown in table 1, the Initiatives’ effectiveness indicators for tracking  progress in addressing these management weaknesses did not have  equivalent performance indicators visible in the annunciator panel when it  was prepared for the last time, using August 2005 data.
Two of the Initiatives’ key areas of concern—(1) roles, responsibilities,  authority, and accountability; and (2) work procedures—and their  associated effectiveness indicators were not represented in the panel’s  visible or underlying indicators. The Initiatives’ effectiveness indicator for  tracking trends in recurring problems also was not represented. In other  cases, the Initiatives’ effectiveness indicators were represented in  underlying lower-level indicators that had very little impact on the rating of  the visible indicator. An example is the Initiatives’ indicator for timely  completion of employee concerns. The panel’s related visible indicator was  work environment, whose rating was based on 4 secondary and 23 tertiary  indicators. Of the third-level indicators, two were for timeliness of  completion of employee concerns, and combined they contributed 3  percent toward the rating of the work environment indicator. As a result of  the weighting of these many underlying indicators, ratings for individual  lower-level indicators could be different from the visible indicator. For  example, in August 2005, the work environment indicator showed good  performance. However, the ratings of four underlying indicators from the  project’s employee survey on the work environment—collectively  accounting for 25 percent of the work environment indicator’s score— indicated the need for increased management attention. Moreover, some of  the Initiatives’ indicators, such as the work organizations’ self- identification of significant problems, had their impact on visible indicators  diluted by the inclusion of other indicators that were not focused solely on  the detection of significant problems.
Another shortcoming of the annunciator panel was that frequent changes  to the indicators hindered the ability to identify problems for management  attention and track progress in resolving them. The indicators could  change in many ways, such as changes in their definition, calculation, or  data sources used in calculations, or from the deletion or addition of a  subindicator. When such changes were made to the indicators, progress  became less clear because changes in reported performance levels may  have been the result of the indicator changes rather than actual  performance changes. Some of the indicators for key project processes  with quality elements changed from one to five times during the 8-month  period from April 2004 through November 2004. Even after the major  revision of the panel in early 2005, most of the performance indicators  tracking quality issues continued to change over the next 6 months—that  is, from March 2005 through August 2005. As shown in table 2, only one of  the five relevant indicators did not change during this period. One indicator  was changed four times during the 6-month period, resulting in it being  different in more months than it remained the same.
Moreover, the panel was not always available to identify problems and  track progress. The panel was not created for December 2004, January  2005, and February 2005 because it was undergoing a major revision. At  that time, DOE told NRC that the performance indicators for the panel  were revised to reflect the change in the work as the project moved into the  engineering, procurement, and construction phase. DOE also reduced the  total number of visible indicators from 60 to 30 to focus on fewer, more  critical aspects of project management. Panels with the new indicators  were then produced for 6 months, starting with March 2005 and ending  after August 2005. This second interruption of the panels resulted from  another major revision to the indicators; this time, indicators are being  made congruent with project work as designated by DOE’s “new path  forward,” again to focus on fewer, more important activities. In December  2005, a senior DOE official told us that the project would begin to measure  key activities, but without use of the panel.
section_title: Trend Evaluation Reports Have Not Specifically Tracked the Initiatives’ Management Concerns and Have Had Weaknesses Tracking Significant and Recurrent Problems for Management Attention
paragraphs: According to DOE, some of the Initiatives’ areas of concern and their  associated effectiveness indicators—for example, trends in quality  problems related to roles and responsibilities—were being captured, at  least partially, in the project’s quarterly trend evaluation reports rather than  in the performance indicators. However, the trend reports are a  management tool designed more to identify emerging and unanticipated  problems than to monitor progress with already identified problems, such  as those addressed by the Initiatives. In developing these reports, trend  analysts seek to identify patterns and trends in condition reports (CR),  which document problematic conditions through the project’s Corrective  Action Program. The trend reports analyze CRs for more significant  problems (Levels A and B) and minor problems (Level C), but not at Level  D (opportunities for improvement). The trend analysis typically separates  the reported problems into categories such as organizational unit, type of  problem, and cause. These categories are intended to provide insights into  the problems. For example, analysis might reveal that most occurrences of  a particular type of problem are associated with a certain organization.
In practice, DOE missed opportunities to use trend reports to call attention  to progress in the Initiatives’ areas of concern. For example, the Initiatives  sought to clarify roles and responsibilities within and between DOE and  BSC to ensure clear accountability for project results during the project’s  transition from scientific studies to the design and engineering activities  necessary to license a repository. Similar organizational transition  problems were identified in the November 2004 trend report. While that  report attributed increases in the number of causal factors associated with  change management, supervisory methods, and work organization to  recent BSC reorganizations and changes in the project from science-based  to design and engineering activities, it did not specifically mention issues of  roles and responsibilities or that roles and responsibilities was an  Initiatives’ area of concern. However, an analysis of the cause of the  problems noted in various significant condition reports, which is  performed for certain condition reports and outside of the process of  developing trend reports, found evidence of weaknesses in the  organizational interfaces among BSC organizations, as well as between  BSC and DOE. According to this cause analysis, these organizational  interface weaknesses were associated with some manner of change and  represented weaknesses in the definition of roles and responsibilities.  Trend reports are generally based on condition reports, and problems with  roles and responsibilities seem to be identified in cause analyses rather  than in the condition reports themselves.
Similarly, DOE missed an opportunity to use trend reports to discuss the  Initiatives’ goal that the project’s line or work organizations become more  accountable for self-identifying significant problems. The August 2005  trend report briefly cited an evaluation of a CR highlighting the low rate of  self-identification of significant problems during the previous quarter and  reported the evaluation’s conclusion that it was not a problem warranting  management attention. However, the trend report did not mention that  about 35 percent of significant problems were self-identified during the  previous quarter, while the Initiatives’ goal was that 80 percent of  significant problems would be self-identified. Thus, the trend report missed  an opportunity to either raise a performance problem or pose the question  of whether the Initiatives’ goal needed to be reassessed.
Beyond whether they effectively tracked the Initiatives’ areas of concern,  trend reports face important obstacles, in general, to adequately identify  recurrent and significant problems:  Recurring or similar conditions can be difficult to clearly identify for  management’s attention and resolution. A trend report noted that there  will be few cases where recurrent conditions are obvious because each  condition slightly differs.
Trend analysis tends to focus on the number of CRs issued, but the  number of CRs does not necessarily reflect the significance of a  problem. For example, the number of CRs involving requirements  management decreased by over half from the first quarter to the second  quarter of fiscal year 2005. However, this decrease was not a clear sign  of progress. Not only did the number rise again in the third quarter, but  the May 2005 trend report also noted that the number of all condition  reports had dropped during the second quarter. According to the report,  the volume of CRs in the first quarter had been high because of reviews  of various areas, including requirements management. Another example  is the records management problem. The November 2005 trend report  stated that a records management problem identified in various CRs,  despite accounting for about 50 percent of all business administration  problems, reflected an underlying error rate of less than 1 percent and  thus was not a significant problem.
The lack of an increasing trend in the number of reported problems does  not necessarily mean the lack of a significant problem for management  attention. Knowing the appropriate level of performance, regardless of  the trend, is difficult without having clearly appropriate benchmarks  from organizations engaged in activities similar to the Yucca Mountain  project. Such benchmarks would clarify, for example, whether a  project’s percentages of human performance errors compare favorably,  regardless of whether the numbers are increasing. Similarly, the trend in  the number and types of CRs during any period is not necessarily a sign  of improvement or worsening conditions. Trends can be attributed to  various factors, including increases in the number of audits or self- assessments, which can lead to more CRs being issued.
At the time of analysis, some trend data may not be sufficiently reliable  or complete to ensure sound findings for management’s attention. For  example, although some actions were taken in December 2004 to ensure  that cause and other codes were properly assigned, a BSC audit in June  2005 again raised questions about the consistency of the coding. With  respect to completeness, the fourth quarter report for 2005 noted that 28  percent of the Level B CRs did not have a cause code at the time of the  trend analysis, and one finding was presented even though two-thirds of  the data was missing.
Due, in part, to these obstacles and changes to how the analysis is done,  trend reports have not consistently determined the significance of  problems or performed well in tracking progress in resolving problems. For  example, trend reports have questionably identified significant human  performance problems and ineffectively tracked progress in resolving the  problem because of no clearly appropriate or precise benchmark for  performance, inconsistent focus on the problem, and unreliable data on  cause codes.
The February 2004 trend report identified a human performance problem  based on Yucca Mountain project data showing the project’s proportion of  skill-based errors to all human performance errors was two times higher  than benchmark data from the Institute of Nuclear Power Operations  (INPO). The report used this comparison to suggest that the project  needed to adopt successful commercial nuclear practices for addressing  skill-based errors. However, the report cautioned that other comparisons  with these INPO data may not be appropriate because of differences in the  nature, complexity, and scope of work performed, but did not explain why  the report’s comparison of INPO data for skill-based errors to the Yucca  Mountain project should be an exception to this caution. The May 2004  trend report repeated this comparison to INPO, finding skill-based errors  three times higher than the benchmark data. However, this INPO  benchmark has not been used in subsequent reports.
The November 2004 trend report redefined the problem as the  predominance of human performance errors in general, rather than the  skill-based component of these errors—but later reports reinterpreted this  predominance as not a problem. The problem with skill-based errors was  unclear in the November 2004 report because these errors were showing a  decreasing trend, a finding that was attributed as likely the result of  unreliable assignment of cause codes. Instead, the report cited an adverse  trend based on the fact that the human performance cause category  accounted for over half of the total number of causes for condition reports  prepared during the quarter. Under the project’s trend analysis guidelines,  this large predominance of human performance causes—in contrast to  management, communication or procedure, and other cause categories— was designated an adverse trend. Nevertheless, by February 2005, trend  reports began interpreting this predominance as generally appropriate,  given the type of work done by the project. That is, the project’s work  involves mainly human efforts and little equipment, while work at nuclear  power plants involves more opportunities for errors caused by equipment.  In our view, this interpretation that a predominance of human performance  errors would be expected implies an imprecise benchmark for appropriate  performance.
Although trend reports continued to draw conclusions about human  performance problems, the February 2005 report indicated that any  conclusions were hard to justify because of data reliability problems with  cause coding. For example, the majority of problems attributed to human  performance causes are minor, or Level C, problems that receive less  rigorous cause analysis, such as not completing a form. This less rigorous  analysis tends to reveal only individual human errors—that is, human  performance problems—whereas more rigorous analysis tends to reveal  less immediately obvious problems with management and procedures.
Trend reports have also inconsistently tracked progress in resolving the  problem associated with the “flow-down” of requirements into the project’s  procedures—that is, with ensuring that program, regulatory, and statutory  requirements are identified, allocated, and assigned to the project  organizations that are responsible for applicable activities. Such  requirements management problems can result in inadequate control over  design inputs and, possibly, inputs to scientific models. Progress with this  problem was less clear because of inconsistent methods of categorizing  requirements management problems over time. Initially, based on reviews  of annual trends in condition reports, the September 2004 and November  2004 trend reports observed a systemic and continuing problem in the flow- down of requirements from BSC’s Project Requirements Document and  identified this as an adverse trend. In subsequent reports, the requirements  flow-down problem was variously treated as an aspect of requirements  management or records management, or as a latent management weakness  or weak change management. When treated as an aspect of these broader  problems, the significance of the original flow-down problem and any  progress in resolving it became diluted and less clear. The primary focus  eventually became requirements management, which the February 2005  trend report designated as a potential trend, whereas the flow-down  problem had earlier been designated an adverse trend. Consequently, as a  result of this change, the flow-down of requirements got less direct  attention and analysis—for example, receiving only a footnote in the  August 2005 trend report stating that the April 2004 condition report issued  to address the adverse trend was still overseeing implementation of  corrective actions.
In addition, because trend reports examine only condition reports issued to  BSC, they do not always assess the projectwide significance of problems  such as requirements management. When analyzing one category of  issues associated with requirements management, the November 2005  report stated that BSC and DOE shared the process problems, which  cannot be adequately addressed by just one of the organizations. However,  for a second category of these issues, the report did not analyze most of the  condition reports because 6 of the 10 relevant reports were assigned to  DOE. For a third category of issues, no analysis or recommendation was  provided because all of the reports were assigned to DOE and therefore did  not fall within the scope of the trend report.
section_title: DOE Has Not Adequately Tracked Problems Being Addressed by Ongoing Management Actions
paragraphs: The tracking of problems for which corrective actions are already being  taken appeared at times to be overly influenced by judgments, rather than  the problems’ significance, about whether additional management action is  warranted. As a result, problems might be rated as less significant, or not  tracked further.
The situation of assigning a lower rating to a problem’s significance was  apparently caused by the fact that ratings were simultaneously an  assessment of a problem’s significance and of the need for management  action. In its current formulation, DOE’s rating categories cannot  accurately represent both the assessment of a problem’s significance and a  judgment that additional actions are not needed because the designated  rating category will distort one or the other. For instance, the November  2005 trend report analyzed the four categories of requirements  management issues and designated one category that included problems  with requirements flow-down as a “monitoring trend”—defined as a small  perturbation in numbers that does not warrant action but needs to be  monitored closely. Describing this trend as a small perturbation, or a  disturbance in numbers, did not accurately describe the report’s  simultaneous recognition that significant process problems spanned both  BSC and DOE and the fact that the numbers and types of problems were  consistently identified over the previous three quarters. A more  understandable explanation for the low rating is that designating the  problem at any higher level of significance would have triggered guidelines  involving the issuance of a condition report, which, according to the  judgment expressed in the report, was not needed. Specifically, the report  indicated that existing condition reports have already identified and were  evaluating and resolving the problem, thereby eliminating the need to issue  a new condition report.
By rating the problem at the lowest level of significance and not calling for  additional actions, the trend report did not sufficiently draw management’s  attention to the problem. The trend report’s assessment did not convey that  other serious problems might have been raised by the additional condition  reports. At about the same time that the trend report judged that no new  condition reports were necessary, an Employee Concerns Program’s  investigation of requirements management resulted in 14 new condition  reports—3 at the highest level of significance and 8 at the second-highest  level of significance. For example, the Employee Concerns Program’s  investigation resulted in condition reports calling for an analysis of the  collective significance of the numerous existing condition reports and an  assessment of whether the quality assurance requirement for complete and  prompt remedial action had been met. As a result of the investigation and  a concurrent DOE root cause analysis, during the December 2005 Quarterly  Management Meeting with NRC, DOE stated that strong actions were  required to address the problems with its requirements management  system and any resulting uncertainty about the adequacy of its design  products.
Trend reports identified significant problems in the February 2005 report  but did not continue to track the problems after a separate analysis  identified ongoing improvement actions. According to the trend report,  Level B condition reports collectively indicated organizational weaknesses  associated with change management involving cross-departmental  interfaces. The trend report recommended that management focus on  these problems, and cited a condition report that would further investigate  them. The cause analysis for that condition report and a related condition  report found that the problems were well-known, in part through a BSC  review, and related to a variety of ongoing BSC improvement actions. Since  this was a broad category of problems with many initiatives under way, the  cause analysis recommended no new actions other than for management to  remain aware of the problems. However, the trend reports that followed  provided no further analyses to focus management’s awareness on these  problems or to assess progress in resolving them.
section_title: DOE’s ‘New Path Forward’ for Preparing to Submit Its License Application Faces Substantial Quality Assurance and Other Challenges
paragraphs: In October 2005, DOE announced an aggressive series of proposed changes  to the design, organization, and management of the Yucca Mountain  project, but this effort—known as the “new path forward”—will face  substantial challenges. Some key challenges facing DOE are (1)  determining the extent of problems and restoring confidence in the  documents supporting the license application after the discovery of e-mails  raising the potential of falsified records, (2) settling design issues and  associated problems with requirements management, and (3) replacing key  personnel and managing the transition of new managers and other  organizational challenges. The current Acting Director of the Office of  Civilian Radioactive Waste Management (OCRWM) stated that DOE will  not announce a schedule for submitting a license application until DOE  addresses these important quality assurance and other challenges. Since  DOE is still formulating its plans, it is too early to determine whether the  new path will resolve these challenges.
subsections: section_title: Determining the Extent of Problems with Relevant Documents Will Delay DOE’s Submission of the License Application
paragraphs: In March 2005, after announcing the discovery of USGS e-mails suggesting  the possible violation of quality assurance requirements, including the  falsification of records, DOE has taken steps to address lingering concerns  about the adequacy of the scientific work related to the flow of water into  the repository and whether similar quality assurance problems are evident  in other e-mails relevant to the licensing application. Specifically, DOE is  (1) conducting an extensive review of approximately 14 million e-mails to  determine whether these e-mails raise additional quality assurance  concerns and whether they might be relevant to the licensing process, and  (2) reworking the technical documents created by USGS personnel to  ensure that the science underlying the conclusions on water infiltration are  correct and supportable in the license application. The Acting Director of  OCRWM has stated that DOE will not submit a license application until  these efforts are complete. Consequently, given the early planning stage of  these efforts, it is unknown how long this will delay the submission of a  license application.
As part of the licensing process, DOE is required to publicly disclose all  documents relevant to the licensing application, including e-mails, by  posting them on DOE’s public Web site, which is accessible through the  NRC-sponsored, Internet-based Licensing Support Network (LSN). To  satisfy schedule requirements, DOE must certify that relevant documents  have been posted to the network and made available for public review 6  months before the submission of the license application. In preparation for  submitting the license application by December 2004, in June of that year,  DOE submitted almost 700,000 e-mails to the LSN that had been reviewed  by their original authors and determined to be relevant to the licensing  process. They were part of a group of approximately 6 million archived e- mails authored by individuals still associated with the project. However, in  August 2004, NRC’s Atomic Safety and Licensing Board ruled that DOE had  not met its regulatory obligation to make all relevant documentary material  available. Specifically, DOE had not reviewed a group of approximately 4  million archived e-mails authored by individuals no longer affiliated with  the project to determine whether the e-mails were relevant to the licensing  process. As part of its effort to address the board’s ruling, BSC began a  review of e-mails authored by employees who were not currently working  at the project. During this review, the contractor discovered and brought  forward e-mails between USGS scientists working on water infiltration  models that raised questions of the potential falsification of technical  information in order to sidestep quality assurance requirements.
Following the discovery of the e-mails, DOE conducted a search to  determine if there were similar e-mails in the approximately 1 million e- mails previously determined relevant for licensing. However, the DOE  Inspector General reported in November 2005 that there was no evidence  that the project requirements for identifying and addressing conditions  adverse to quality, such as those contained in the USGS e-mails, were  considered during the initial review of e-mails. Further, among the  approximately 10 million e-mails that had already been reviewed for the  licensing process, they found additional e-mails that identified possible  conditions adverse to quality that had not been identified by project  personnel as requiring further review. The DOE Inspector General  recommended, among other things, that DOE (1) expand the review of  archived e-mails to include both those deemed relevant and those deemed  not relevant to the licensing process, and ensure that conditions adverse to  quality are appropriately identified, investigated, reported, and resolved;  and (2) ensure that current and future e-mails are reviewed for possible  conditions adverse to quality and that such conditions are appropriately  addressed under the Corrective Action Program (CAP) system. DOE  accepted the Inspector General’s recommendations. Specifically, DOE  agreed to develop a corrective action plan to expand the review of archived  e-mails to ensure that conditions adverse to quality are appropriately  identified and processed under the CAP system. In addition to this review,  the DOE Inspector General opened a criminal investigation into the USGS  e-mails in March 2005. As of December 2005, the investigation was still in  progress.
According to NRC on-site representatives, completing these e-mail reviews  will be challenging because DOE now has to screen millions of e-mails to  ensure that records were not falsified. Further, many of these e-mails were  written by employees who no longer work at the project or may be  deceased, making it difficult to learn their true meaning and context.  Moreover, if additional e-mails are found that raise quality assurance  concerns, DOE may have to initiate further review, inspections, or rework  to address the newfound problems. NRC officials stated that it takes the  issue of potentially falsified documents by USGS employees very seriously,  wants a full understanding of the situation regarding the USGS e-mails, and  will conduct follow-up in this area. Because NRC wants DOE to submit a  high-quality license application, it has encouraged DOE to take the time  and actions necessary to fully and adequately resolve these and other  quality assurance issues.
Immediately following the discovery of the USGS e-mails, DOE undertook  a scientific investigation into the technical documents created by USGS  personnel. In October 2005, DOE began developing an action plan for  reviewing, validating, augmenting, and replacing USGS work products that  had come under scrutiny. Although the plan is not yet complete, the Acting  Director told us that the license application would not be submitted until  the USGS work is replaced and there is confidence that all requirements  have been met. In an effort to ensure that the scientific work underlying  water infiltration modeling is accurate, DOE is working to corroborate the  original work by engaging multiple agencies and organizations to rework  the models. For example, DOE has (1) had its lead project contractor work  with the Idaho National Laboratories to extensively review the software  and data used in the original science work, (2) engaged Sandia National  Laboratories to rework the model and calculations using different software  than was used originally, and (3) also asked USGS to rework the models.  Consequently, when this additional rework is completed, DOE will have  four sets of analysis (including the original scientific work) with which they  can evaluate, compare, and corroborate results. DOE will then pick one set  of scientific analysis for inclusion in the license application, and work to  explain and defend its choice.
section_title: Ongoing Design and Requirements Management Issues Could Delay DOE’s Submission of the License Application
paragraphs: In October 2005, DOE announced significant changes to the design of the  Yucca Mountain repository to simplify the project and improve its safety  and operation. However, these changes will also require additional design  and engineering work that will add uncertainty about the timing of the  submission of a license application. DOE had been considering a design  where radioactive waste would be shipped to the Yucca Mountain site,  removed from its shipping container, placed and sealed in a special  disposal container, and finally moved into the underground repository. As a  result, DOE contemplated handling the waste up to four separate times. In  late 2003, DOE engineers began identifying potential safety problems with  this approach. First, possible fissures or holes in the cladding surrounding  the spent nuclear fuel accidentally caused during the handling of the waste  could cause air to mix with the fuel and oxidize. Consequently, this  radioactive oxidized material could then leak and be dispersed into the air.  Second, DOE engineers determined that the original facility design would  not be able to adequately control the levels of radioactivity in the buildings  where the waste would be repackaged before being moved in the  repository. To address these problems, DOE researched a series of options,  including only accepting radioactive waste that had already decayed to the  point where oxidization would not be problematic, and testing the waste  shipments for oxidization and treating them at another site before they  arrived at the repository. In addition, DOE also considered changing the  design by filling the processing buildings with inert gas to prevent  oxidization and revising the electrical and ventilation systems. According  to a DOE official, these options were impractical or added complexity to  the design.
However, in October 2005, DOE proposed a new design that relies on  uniform canisters that would be filled and sealed before being shipped,  eliminating the need for direct handling of the waste prior to being placed  in the repository. As a result, DOE will not have to construct several  extremely large buildings costing millions of dollars for handling  radioactive waste. DOE believes this change will improve the safety,  operation, and long-term performance of the repository. However, this  change will also pose a challenge to the project because of the widespread  implications and the unknown time and effort required to implement it. For  example, to implement the new design, DOE will need to, among other  things,  get approval from the Energy Systems Acquisition Advisory Board for  a new project plan, which, among other things, includes details on the  conceptual design, cost estimates, risk management efforts, and  acquisition strategies;  plan, design, and produce standardized canisters for the transportation   coordinate this new approach with commercial nuclear power plants,  NRC, and government organizations that plan on shipping waste to the  project; and  revise procurement and contracting plans to support the new design.
Finally, DOE will need to perform the detailed design and engineering work  required to implement the new design, and create new technical  documents to support the license application. However, before it can  present its new plans and perform this design and engineering work, DOE  officials have stated that it will need to resolve long-standing quality  assurance problems involving requirements management. Requirements  management is the process that ensures the broad plans and regulatory  requirements affecting the project are tracked and incorporated into  specific engineering details. According to DOE’s root cause analyses, low- level documents were appropriately updated and revised to reflect high- level design changes through fiscal year 1995. However, from 1995 through  2002, many of these design documents were not adequately maintained and  updated to reflect current designs and requirements. Further, a document  that is a major component of the project’s requirements management  process was revised in July 2002, but has never been finalized or approved.  Instead, the project envisioned a transition to a new requirements  management system after the planned submission of the license  application in December 2004. However, for various reasons, the license  application was not submitted at that time, and the transition to a new  requirements management system was never implemented. As a result, the  document refers to the out-of-date NRC regulations contained in 10 CFR  part 60, and not the regulations in 10 CFR part 63 that were finalized in  October 2002.
The scope and cause of requirements management problems have been  identified in multiple DOE and NRC reviews.
Multiple condition reports issued in 2004 and 2005 have identified  problems with requirements management. Due to these condition  reports and NRC concerns that repetitive deficiencies and the failure to  implement timely corrective actions could have direct implications on  the quality of the planned license application, NRC performed a review  of Corrective Action Program documents related to the requirements  management program in the late summer of 2005. NRC determined that  these reports identified approximately 35 deficiencies related to  requirements management. Because the requirements management  documents are not current and the new requirements management  system has not been implemented, NRC concluded that there does not  appear to be a requirements management mechanism in place. Further,  based on the number of reports and other issues identified by DOE  audits, NRC concluded that the project’s Corrective Action Program was  not effective in, among other things, eliminating the repeated  identification of deficiencies relating to requirements management or  initiating the actions to identify and appropriately address the root  cause of these problems.
In September 2005, DOE began reviewing the root causes associated  with CR-6278, a condition report identifying problems with  requirements management. As part of the review, DOE personnel  analyzed 135 condition reports and other events and allegations. Among  other things, this review found that DOE expectations for requirements  management were diluted and eventually neglected, that DOE reduced  funding for requirements management due to reductions in its annual  budget, and that these and other events caused the requirements  management process to become “completely dysfunctional” from July  2002 to the time of the review in the fall of 2005. The analysis identified  the root causes of these conditions as DOE’s failure to fund, maintain,  and rigidly apply a requirements management system.
In November 2005, a team of DOE personnel concluded an investigation  into an employee’s concerns regarding requirements management. The  team substantiated all of the concerns they investigated and found  instances of failures and breakdowns in the requirements management  process. For example, among other things, the team found that no  procedure was developed to describe how requirements management  was to occur; some existing requirements management procedures were  not implemented; and project management was aware of these  conditions but corrective actions were deferred because the planned  requirements management system was expected to address the  problem. As a contributing factor, the team also observed that the  project’s lead contractor had not implemented a “traditional systems  engineering approach” as it did not have, among other things, typical  engineering management plans or a separate systems engineering  organization responsible for requirements management. As a result of  the investigation, the team initiated 14 condition reports, 13 of which  identified quality-related problems.
To address these problems, on December 19, 2005, DOE issued a stop-work  order on design and engineering for the surface facility and certain other  technical work. DOE stated that the root cause analysis for CR 6278 and  the investigation into employee concerns revealed that the project has not  maintained or properly implemented its requirements management system,  resulting in inadequacies in the design control process. This stop-work  order will be in effect until, among other things, the project’s lead  contractor improves the requirements management system; validates that  processes exist and are being followed; and requirements are appropriately  traced to implementing mechanisms and products. Further, DOE will  establish a team to take other actions necessary to prevent inadequacies in  requirements management and other management systems from recurring.
An example of the potential risks of a breakdown with requirements  management was noted during a BSC audit on the design process in March  2005. NRC on-site representatives observing this audit reported that the  audit team noted problems with inconsistencies between the design  documents of the planned fuel-handing facility that would be receiving,  preparing, and packaging the waste before it is placed in the repository.  The original set of requirements specified that no water from a fire  protection system was to be used in the fuel-handling areas of the facility  because under certain scenarios, water used for fire suppression could  facilitate an accidental nuclear reaction, a condition known as criticality.  Later, as the project began to review the design of the fuel-handling facility,  the design was changed to allow the use of water sprinklers in the fuel- handling areas of the facility to suppress possible fires. NRC noted that  personnel working on the design knew of the inconsistencies between  older and newer design documents, but no formal tracking mechanism had  been provided to ensure that those issues were rectified. According to an  NRC on-site representative in December 2005, this was an example of a  concern with requirements management, and that repetitive and  uncorrected issues associated with the requirements management process  could have direct implications on the quality of the license application.
While the project may be able to resolve these inconsistencies through an  informal process, the lack of a formal design control and requirements  management process increases the risk that not all such problems will be  addressed. These requirements management problems are potentially  significant because if the high-level engineering needs of the project are not  accurately or completely reflected in the detailed design, then the quality of  the license application may be compromised and cause delays in the  license application review process. For example, according to a 1989  speech prepared by NRC’s Office of General Counsel stressing the  importance of quality assurance, a West Coast nuclear power plant  experienced similar quality assurance problems with requirements  management. After a license was issued by NRC, power plant personnel  discovered that the wrong diagrams were used to develop design  requirements. As a result of this and other quality assurance weaknesses  identified by NRC, the license was suspended and the power plant was  required to initiate an independent program to verify the correctness of the  design. Further, NRC reopened hearings on the issue of the adequacy of the  power plant’s quality assurance program related to the plant’s design.
section_title: DOE Faces Challenges in Managing the Transition, Complexity, and Continuity of Its ‘New Path Forward’
paragraphs: In October 2005, DOE announced a “new path forward” that would create a  new project schedule and financial plan to address the completion of  scientific and engineering work in support of a license application.  However, DOE faces challenges to successfully implementing the new  path, in terms of managing the transition, program and organizational  complexities, and the continuity of management. According to DOE  managers involved with planning the new path forward, the organizational  transition could take several months to complete. It is too early to  determine whether DOE’s new effort will resolve quality assurance issues  and move the project forward to the submission of a license application.
Accountability for quality and results, which was identified as a significant  transition issue in the Initiatives, will likely pose a challenge for managing  the transition to the new path forward. The Initiatives sought to clarify  roles and responsibilities within and between DOE and contractor  organizations to ensure clear accountability for results and quality during  the transition from OCRWM’s organization, processes, procedures, and  skills supporting scientific studies to those supporting the activities  necessary to license a repository. As the project realigns organizations,  processes, procedures, and skills to support the new path forward, it will  also be faced with the challenge of ensuring that accountability is not  undermined during the transition. For instance, according one DOE  manager, transitioning project work to a lead laboratory under a direct  contract with DOE could pose a significant challenge for quality assurance  because the laboratories are currently working under BSC quality  assurance procedures and will now have to develop their own procedures.
Implicitly recognizing the importance of accountability issues, elements of  the new path forward seek to address issues that can negatively affect  quality assurance and project management in general. For instance, the  new path includes plans for developing and transmitting requirements to  USGS for the certification of scientific work. In addition, a senior project  official told us that the lead laboratory would provide a single point of  accountability that will enhance the quality of the science work. The Acting  Director indicated that OCRWM’s management structure may have to be  reorganized to have a single manager clearly accountable for each of the  new path’s major tasks in science, engineering, and licensing. Moreover, the  project is developing new performance indicators to allow the project to  assess important activities under the new path forward. Outside of the new  path, as the result of a September 2005 DOE Inspector General report on  accountability problems with managing contract incentives, OCRWM  agreed to develop a comprehensive corrective action plan to provide  clearer and more objective performance standards in the BSC contract.
Program complexity and other project characteristics are also likely to  pose challenges to managing quality assurance. Based on its experience  with licensing and regulating nuclear power plants, NRC observed in the  mid-1980s that the Yucca Mountain project’s characteristics, such as a large  and complicated program, increased the likelihood of major quality-related  problems. Although the new path is intended to simplify design, licensing,  and construction, the project remains a complicated program that seeks to  both restore confidence in its scientific studies and pursue new design and  engineering activities. As a result, the project has to manage quality  assurance issues simultaneously in both areas. Moreover, the project  involves a complicated organizational structure. The project will continue  contracting work with BSC, USGS, and the Sandia National Laboratory,  which involves working with organizations in various locations. In our 1988  report, we noted that the geographic distance between the various  organizations may hamper OCRWM’s quality assurance communication  and oversight objectives.
The project also faces management challenges related to ensuring  management continuity at the project, since DOE has experienced turnover  in 9 of 17 key management positions since 2001. To ensure the right  managers move the project forward to licensing, the project has a  recruitment effort for replacing key departing managers. In the past year,  the project has lost key managers through the departures of the director of  Project Management and Engineering, the director of the License  Application and Strategy, the director of Quality Assurance, and the  contractor’s general manager. According to NRC on-site representatives in  August and October 2005, management turnover is a concern for NRC  because it would like to see continuity of qualified managers rather than a  series of acting managers. Recruiting replacement managers can impact  project continuity, and newly acting managers may not take full rein of  project tasks. However, the Acting Director told us that the recruitment  process is an opportunity to improve project managers and staff, but  recruiting the right people is challenging for various reasons—for example,  government salaries are less than those in industry, and employment  clauses restrict subsequent employment in related industries.
Finally, since new directors sometimes give new direction to the project, a  critical issue for sustaining the current new path forward is continuity with  OCRWM’s director. This position was occupied by three individuals  between late 1999 and early 2005. The last OCRWM director assumed the  position in April 2002, started the Management Improvement Initiatives in  2002, and left the position in February 2005. The current Acting Director  began functioning in his position in the summer of 2005, and initiated the  new path forward in October 2005. DOE is currently awaiting  congressional confirmation of a nominee to take the director position.  However, the Acting Director told us he expects that the new path forward  will be sustained because it has been endorsed by the Secretary of Energy.
section_title: Conclusions
paragraphs: DOE’s Yucca Mountain project has been wrestling with quality assurance  problems for a long time. Now, after more than 20 years of project work,  DOE is again faced with substantial quality assurance and other challenges  to submit a fully defensible license application to NRC. Unless these  challenges are effectively addressed, further delays on the project are  likely. Furthermore, even as DOE faces new quality assurance challenges, it  cannot be certain it has resolved past problems, largely because the  department has not been well served by management tools—specifically,  its performance indicators and trend evaluation reports—that have not  effectively identified and tracked progress on significant and recurring  problems. First, the management tools have provided limited coverage of  the areas of concern identified in the Management Improvement Initiatives  and thus have not enabled DOE managers to effectively monitor progress  in these important areas. Second, the tools have often not reflected the full  extent or significance of problems because their scope has been limited  and not based on projectwide analysis. Third, the trend evaluation reports  have, at times, not accurately characterized problems because reliable and  complete data and appropriate performance benchmarks were not  available at the time of analysis. Fourth, frequent changes in performance  indicators and the way analysis is done have made it difficult to accurately  identify trends over time. Fifth, the tools’ rating categories have sometimes  been misleading as to the significance of problems because the ratings tend  to be skewed by the fact that corrective actions were already being taken,  without considering their effectiveness or considering the significance of  the problem on its own terms. These shortcomings with the tools limit  project managers’ ability to direct and oversee such a large and complex  undertaking as constructing an underground repository for nuclear wastes.  Further complicating DOE’s ability to manage the project are the vacancies  in key managerial positions for the quality assurance program and  elsewhere on the project. The tools become even more important for new  managers who need to quickly understand project management issues.
section_title: Recommendations for Executive Action
paragraphs: To improve the effectiveness of DOE’s efforts to monitor performance in  key areas at the Yucca Mountain project, including quality assurance, we  recommend that the Secretary of Energy direct the Director, Office of  Civilian Radioactive Waste Management, to take the following five actions  to strengthen the project’s management tools:   Reassess the coverage that the management tools provide for the areas  of concern identified in the Management Improvement Initiatives and  ensure that performance in these important areas is effectively  monitored, especially in light of the more recent condition reports and  associated cause analyses, trend reports, and other reviews indicating  continuing problems.
Base future management tools, such as the trend evaluation reports, on  projectwide analysis of problems, unless there are compelling reasons  for a lesser scope.
Establish quality guidelines for trend evaluation reports to ensure sound  analysis when reporting problems for management’s attention. Such  guidelines should address, among other things, having reliable and  complete data and appropriate benchmarks.
To the extent practicable, make analyses and indicators of performance  consistent over time so that trends or progress can be accurately  identified and, where changes to analyses or indicators are made for  compelling reasons, provide a clear history of the changes and their  impact on measuring progress.
Focus the management tools’ rating categories on the significance of the  monitored condition, not on a judgment of the need for management  action.
section_title: Agency Comments
paragraphs: We provided DOE and NRC with draft copies of this report for their review  and comment. In oral comments, DOE agreed with our recommendations  and provided technical and editorial comments that we have incorporated  in the report, as appropriate. We also incorporated, as appropriate, NRC’s  oral editorial comments, which primarily served to clarify its role.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies to interested congressional  committees and Members of Congress, the Secretary of Energy, and the  Chairman of the Nuclear Regulatory Commission. We will also make copies  available to others upon request. In addition, the report will be available at  no charge on the GAO Web site at www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-3841 or wellsj@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Objectives, Scope, and Methodology
paragraphs: The objectives of this review were to determine (1) the history of the Yucca  Mountain project’s quality assurance problems since the project’s start in  the 1980s, (2) the Department of Energy’s (DOE) tracking of quality  problems and progress implementing quality assurance requirements since  our April 2004 report, and (3) challenges that DOE faces as it continues to  address quality assurance issues within the project. In addition, we were  asked to provide information about implementation of the project’s  Employee Concerns Program and the types of concerns raised in recent  years through the program.
To determine the history of the project’s quality assurance problems, we  reviewed our prior reports and those of DOE’s Office of the Inspector  General concerning the Yucca Mountain project. We also reviewed internal  DOE evaluations and audit reports written about the quality assurance  program and Nuclear Regulatory Commission (NRC) reports and NRC- prepared summaries of NRC and DOE quarterly management meetings,  technical exchange meetings, and quality assurance meetings dating to  early 2004. In addition, we reviewed letters and communications between  DOE and NRC regarding quality assurance from the NRC Web archives  from the late 1980s. Furthermore, we reviewed plans for the Regulatory  Integration Team (RIT) and subsequent correspondence between  Bechtel/SAIC Company, LLC (BSC), DOE’s management contractor for the  Yucca Mountain project, and DOE. Moreover, we discussed quality  assurance issues with officials of DOE’s Office of Civilian Radioactive  Waste Management (OCRWM), including the Acting Director and Deputy  Director, at DOE headquarters in Washington, D.C., and at its field office in  Las Vegas. In addition, we interviewed representatives of Navarro Quality  Services, a DOE subcontractor, as well as BSC, and NRC officials in the  agency’s field office in Las Vegas, Nevada, and at its headquarters in  Rockville, Maryland.
To determine DOE’s tracking of quality problems and progress  implementing quality assurance requirements since our April 2004 report,  we interviewed OCRWM, BSC, and NRC officials about the status of these  efforts since the issuance of our prior report. We also reviewed DOE’s  Management Improvement Initiatives (2002), DOE’s Management  Improvement Initiatives Transition Approach (2003), and our 2004 report to  understand the history of the improvement efforts. To understand DOE’s  management tools to monitor problems and progress, we reviewed the  available performance indicators panels from April 2004 through August  2005, when it was last produced; the documentation on the individual  indicators applied to August 2005 data; and the quarterly trend reports from  the fourth quarter of fiscal year 2003 through the fourth quarter of fiscal  year 2005. We also reviewed information from condition reports and  examined documentation on DOE’s Quality Assurance Requirements and  Description (issued in August 2004), BSC’s Trend Evaluation and  Reporting, and DOE’s Procedure: Condition Reporting and Resolution  (issued in November 2005).
To determine challenges that DOE faces as it continues to address quality  assurance issues within the project, we reviewed information from  condition reports, NRC on-site representative reports, DOE Inspector  General reports, and an OCRWM Office of Concerns Program’s  investigative report on past quality assurance problems and DOE’s efforts  to address them. We obtained information on turnover in key management  positions at DOE and BSC since 2000. In addition, we discussed with DOE  and NRC officials DOE’s difficulties in addressing recurring quality  assurance problems and the quality assurance implications of the Yucca  Mountain project moving from the site characterization phase to design  and licensing. Also, to better understand issues and challenges, we  attended quarterly meetings held between DOE and NRC in Rockville in  September 2005 and Las Vegas in December 2005.
To identify recent employee concerns related to quality assurance, such as  falsification of records and a safety-conscious work environment, as well  as to identify the actions taken to address those concerns, we reviewed all  concerns received by the OCRWM and BSC Employee Concerns Programs  from January through November 2005. For the OCRWM program, we  reviewed all employee concerns files to identify concerns related to quality  assurance. For the BSC program, we first read summary descriptions of  each concerns file, and reviewed the concerns files for only those we  identified as related to quality assurance. We then conducted a content  analysis of all concerns files that we reviewed. Next, our three team  members reached consensus about the correct classification of a concern  as a quality assurance problem, such as potential falsification of records.  Finally, through a second review of concerns files, we verified our recorded  information for those concerns that seemed to be important illustrations of  problems. In addition, we also spot-checked a sample of OCRWM and BSC  concerns received in 2005 to verify the accuracy of their placement in  various concerns categories. We found that the concerns were generally  categorized accurately.
We performed our work from July 2005 through January 2006 in  accordance with generally accepted government auditing standards.
section_title: Yucca Mountain Project Employee Concerns Programs
paragraphs: NRC expects licensees to establish a safety conscious work environment— that is, one in which (1) employees are encouraged to raise concerns either  to their own management or to NRC without fear of retaliation and (2)  employees’ concerns are resolved in a timely and appropriate manner  according to their importance. NRC encourages but does not require  licensees to establish employee concerns programs to help achieve such a  work environment, and both DOE and BSC have established such  programs. DOE’s Employee Concerns Program is currently operated under  the requirements of DOE Order 442.1A, but the department, in anticipation  of becoming a licensee, is in the process of establishing the program to  meet NRC expectations.
DOE and contractor employees at the Yucca Mountain project may raise  concerns about quality, safety, or other work environment issues—such as  harassment, intimidation, retaliation, and discrimination—through various  means. Employees are encouraged to resolve concerns at the lowest  possible level in the organization, in the following order:  Use normal supervisory channels, such as by raising an issue to a  manager for resolution.
Initiate a condition report through the Corrective Action Program—a  process in which any employee can formally identify a problem on the  project, such as with policies, procedures, or the work environment, and  have the issue investigated and, if necessary, fixed through corrective  actions.
Submit a concern via e-mail, telephone, or in person to one of the  project’s two Employee Concerns Programs—a BSC program for BSC  employees and other subcontractors and another run by DOE for either  DOE or BSC employees.
Contact NRC directly.
The DOE and BSC concerns programs are intended to supplement rather  than replace the resolution of problems through managers or the  Corrective Action Program.
DOE and BSC Employee Concerns Programs have each established a  communication network to allow employees to register concerns. These  networks include brochures and regular newsletters on the programs and  numerous links to the program on the project’s intranet, where employees  can obtain concerns forms. Both the DOE and BSC concerns programs of  the Yucca Mountain project have four main steps: 1. Employees notify the concerns program staff about issues that they feel  should be corrected, such as safety or health issues; harassment,  intimidation, retaliation, or discrimination; concerns raised through the  Corrective Action Program; and quality assurance problems. 2. The concerns program staff document and handle the concern in  accordance with the requirements of DOE Order 442.1A. 3. The concerns program notifies the employees of the results of the  investigation and notifies management of any deficiencies.  4. Project management develops corrective actions for deficiencies, and  the program validates that the concerns have been effectively  addressed by the actions.
Under DOE Order 442.1A, concerns may be addressed through an  investigation by the concerns program staff, an independent investigation,  a referral, a transfer, or a dismissal of the concern. Employees can request  or waive confidentiality. If a concern is submitted anonymously,  interpreting the main issues and problems is left up to the concerns  program staff, and action on the concern may be limited if the submitted  information does not clearly or sufficiently define the concern.
The concerns program may conduct its own investigation of the concern.  Alternatively, it may refer the concern to another project organization for  investigation or resolution. After the results of the investigation or  resolution are reported to the concerns program within a specified period,  the concerns program accepts the results or requires additional actions. In  other cases, concerns may be transferred to another organization with the  appropriate subject matter responsibility or expertise, such as the Office of  Human Relations, Office of General Counsel, or Office of the Inspector  General.
After investigating a concern, the concerns programs determines whether  the concern is substantiated, partially substantiated, unsubstantiated, or  indeterminate. If a concern is substantiated or partially substantiated, the  investigation results are presented to the responsible senior managers. A  concern is considered indeterminate when evidence is insufficient to  substantiate a concern or allow for a conclusion to be drawn. Some  concerns can be resolved through a noninvestigative resolution, a method  to address concerns promptly when minimal effort is required for  resolution. Some resolutions involve the development of management  corrective action plans that are tracked until they are closed. In addition,  for deficiencies that identify systemic problems, the concerns programs  may file a condition report through the Corrective Action Program.  Moreover, DOE and contractor employees are required to report certain  conditions or alleged conditions to DOE’s Office of the Inspector General  under DOE Order 221.1, which covers waste, fraud, and abuse. The  concerns program handles some employee concerns in this way.
From January through November 2005, DOE’s concerns program opened  139 employee concerns for investigation, and the BSC concerns program  opened 112 concerns for investigation. DOE’s concerns program places  concerns into 14 categories, while the BSC program uses 20 categories.  For both DOE and BSC, the category receiving by far the most concerns for  calendar year 2005 was management: “management/mismanagement” for  DOE and “management practices” for BSC. According to DOE,  management concerns generally involved conditions related to  management behavior, policy practice, budget allocation, or use of  resources. According to the manager of BSC’s program, about half of the  concerns in the management practices category involve hiring and human  relations issues and the other half involve organizational policies and other  issues. The “quality” category accounts for a relatively small portion of  total concerns—18 percent of concerns for the DOE program and 4 percent  for the BSC program. Tables 3 and 4 show the concerns received by the  DOE and BSC programs for January through November 2005.
The Employee Concerns Programs, which are designed to provide an  alternative to raising issues through the Corrective Action Program and  issuing condition reports, have been playing an active and sometimes key  role in identifying and addressing quality assurance problems, as can be  seen in the following examples:   As part of an effort to identify e-mails relevant to the licensing process  and that therefore should be included in the Licensing Support Network,  BSC employees in late 2004 discovered e-mails suggesting potential  falsification of technical records. The e-mails were submitted to the  Employee Concerns Program in March 2005 and were eventually  reported to the DOE Inspector General for investigation. The quality  assurance issues raised by the e-mails have resulted in a substantial  effort by DOE to restore confidence in the quality of technical  documents that will support its license application to construct the  repository.
In mid-2005, the DOE concerns program referred to the project’s senior  management an employee’s allegation that the project’s schedule was  taking priority over quality in the review of technical documents. In this  instance, the Office of Concerns Program Manager negotiated with  senior management to address the time and resource needs for ensuring  quality assurance, rather than simply communicating to the organization  that quality should take priority over the schedule.
As the result of an employee’s concerns referred to DOE by NRC in mid-  2005, the Employee Concerns Program initiated an extensive  investigation of issues related to requirements management. That  investigation substantiated the employee’s concerns and led to the  issuance of 14 condition reports for problem resolution. Signifying the  importance of this issue, DOE discussed problems with requirements  management with NRC at their quarterly meeting in December 2005.
The Employee Concerns Programs’ role in identifying and addressing  quality assurance and other issues is dependent upon employees’  willingness to submit concerns, but the employees’ willingness has  sometimes been in doubt. A late 2004 DOE survey of project employees  indicated, for example, that less than two-thirds of employees were  confident that submitted concerns would be thoroughly investigated and  appropriately resolved. DOE recognizes the need to improve employee  trust and willingness to use the concerns program, and both the DOE and  BSC program are engaged in outreach efforts. However, employees’  willingness to submit concerns may be affected by factors outside the  programs’ control. According to a DOE manager, the project’s recent and  pending workforce reductions may account for a decreasing number of  concerns submitted to the DOE program in late 2005. Based on OCRWM  Employee Concerns Program data, the program averaged about 13  concerns a month from January through November 2005. However, the  number of monthly concerns dropped to 5 in October and 3 in November  2005.
During our review of concerns opened for investigation from January 2004  through November 2005, we did not identify any concerns alleging  problems similar to the falsification of technical records suggested by the  USGS e-mails. Although we found records of an early 2004 concern about  an instance of inappropriate management of a technical document, this  instance was resolved and did not appear to be an intentional or systematic  effort to falsify records. The manager of the BSC program told us of a  concern raised about another set of e-mails, but this concern was not about  record falsification. The manager of the DOE program told us that she had  not seen any reportable allegations of falsification of technical records  since she took her position in July 2004.
section_title: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Raymond Smith (Assistant  Director), Casey Brown, John Delicath, James Espinoza, and Terry Hanford  made key contributions to this report.