Yucca Mountain: DOE Has Improved Its Quality Assurance Program, but Whether Its Application for a NRC License Will Be High Quality Is Unclear
Published: 2007-08-02T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Nuclear power reactors generate highly radioactive waste. To permanently store this waste, the Department of Energy (DOE) has been working to submit a license application to the Nuclear Regulatory Commission (NRC) for a nuclear waste repository at Yucca Mountain about 100 miles from Las Vegas, Nevada. Although the project has been beset with delays, in part because of persistent problems with its quality assurance program, DOE stated in July 2006 that it will submit a license application with NRC by June 30, 2008. NRC states that a high-quality application needs to be complete, technically adequate, transparent by clearly justifying underlying assumptions, and traceable back to original source materials. GAO examined (1) DOE's development of its schedule for submitting a license application and the stakeholders with whom it consulted, (2) NRC's assessment of DOE's readiness to submit a high-quality application, and (3) DOE's progress in addressing quality assurance recommendations and challenges identified in GAO's March 2006 report. GAO reviewed DOE's management plan for creating the license application, reviewed correspondence and attended prelicensing meetings between DOE and NRC, and interviewed DOE managers and NRC on-site representatives for the Yucca Mountain project. In commenting on a draft of the report, both DOE and NRC agreed with the report.
section_title: What GAO Found
paragraphs: The director of DOE's Office of Civilian Radioactive Waste Management set the June 30, 2008, date for filing the license application with NRC in consultation with the DOE and contractor managers for the Yucca Mountain project. DOE officials told us that external stakeholders were not consulted because there was neither a legal requirement nor a compelling management reason to do so. According to the director, the June 2008 schedule is achievable because DOE has already completed a large amount of work, including the completion of a draft license application in 2005 that DOE decided not to submit to NRC. NRC officials believe it is likely that DOE will submit a license application by June 30, 2008, but until NRC receives the application, officials will not speculate about whether it will be high quality. NRC has not seen a draft of the license application, and NRC's long-standing practice is to maintain an objective and neutral position toward a future application until it is filed. To help ensure that DOE understands its expectations, NRC has, among other things, held periodic prelicensing management and technical meetings with DOE. DOE has made progress in resolving the quality assurance recommendations and challenges identified in GAO's March 2006 report. For example, DOE has replaced the one-page summary of performance indicators that GAO had determined was ineffective with more frequent and rigorous project management meetings. DOE has addressed the management challenges GAO identified to varying degrees. For example, regarding management continuity, DOE has worked to fill and retain personnel in key management positions, such as the director of quality assurance. However, for various reasons--including the long history of recurring problems and likely project leadership changes in January 2009 when the current administration leaves office--it is unclear whether DOE's actions will prevent these problems from recurring.

section_title: Letter
paragraphs: Nuclear power reactors generate nearly 20 percent of the nation’s  electricity, but they create waste that can remain highly radioactive for  hundreds of thousands of years and requires proper disposal to protect  public health and the environment. More than 50,000 metric tons of this  radioactive waste is stored temporarily at 72 sites around the country— primarily at commercial nuclear power plants. The Nuclear Waste Policy  Act of 1982 established the Office of Civilian Radioactive Waste  Management (OCRWM) within the Department of Energy (DOE) and  directed OCRWM to construct an underground geological repository to  permanently dispose of spent nuclear fuel and other radioactive waste.  The act also set 1998 as the target date for DOE to start accepting this  waste and required the owners of spent nuclear fuel and high-level  radioactive wastes to enter into contracts with DOE for the disposal of  these materials and to pay for the repository’s construction and  operations. In 2002, the Congress approved the President’s  recommendation of the Yucca Mountain site in Nevada for the repository.  However, construction has not yet begun. DOE currently estimates that  2017 is the earliest date that the repository could open. Since DOE was  unable to begin the acceptance of waste in 1998 as required by the statute  and contracts, the government has incurred a liability that DOE now  estimates to be approximately $7 billion. Each year of delay in the  operation of the repository beyond 2017 could increase the government’s  liability by up to $500 million.
Before construction of the repository can begin, DOE must submit a  license application to obtain a construction authorization from the Nuclear  Regulatory Commission (NRC). The act directs NRC to issue or deny  construction authorization within 3 years after receiving DOE’s license  application, unless NRC extends this period by not more than 1 year and  reports the reasons for doing so to the Secretary of Energy and the  Congress. To ensure that its license application review is completed within  the allotted 3- to 4-year time frame, NRC expects the application to be high  quality—that is, to contain the information necessary and sufficient to  support the technical positions it presents. Specifically, NRC has stated  that a high-quality license application would be complete, technically  adequate, transparent––clearly justifying and explaining any underlying  assumptions and conclusions—and traceable back to original source  materials.
As part of the licensing process, DOE must demonstrate that its repository  will meet NRC’s regulations, which implement the Environmental  Protection Agency’s (EPA) standard for protecting public health and the  environment from harmful exposure to the radioactive waste. In  preparation to file a license application, DOE has been conducting  numerous scientific and technical studies at the Yucca Mountain site that  will serve as supporting documentation to demonstrate that it can meet  these standards. DOE has also developed computer models to measure the  probability that various combinations of natural and engineered features  of the repository could safely contain waste for the long term, taking into  account possible water infiltration, waste package corrosion, earthquakes,  volcanic action, or other scenarios.
NRC requires a license applicant to support its technical analysis by  implementing a quality assurance program that ensures that the scientific,  engineering, procurement, recordkeeping, and other work at the project is  performed under controlled conditions and can be verified by others. DOE  project teams are responsible for carrying out various functions or aspects  of the work and creating their own policies and procedures to implement  the quality assurance requirements. DOE has established a quality  assurance program that, for example, contains general requirements for  calibrating equipment before conducting scientific tests, such as  stipulating when and how the equipment should be calibrated and how to  document the results.
In March 2006, we reported that DOE had experienced persistent  problems with its quality assurance program for the Yucca Mountain  project. We concluded that the project’s management tools were  ineffective for monitoring performance and detecting new quality  assurance problems. We recommended that DOE take action to strengthen  the project’s management tools to better identify problems and track  progress in addressing them. Our report also identified three substantial  management challenges facing the project. First, in March 2005, DOE  announced the discovery of e-mail messages implying that some U.S.  Geological Survey (USGS) employees responsible for analyzing water  infiltration in Yucca Mountain may have falsified scientific data and had  shown disdain for quality assurance program requirements. Subsequently,  in January 2007, we reported that DOE had spent about $20.5 million to  survey e-mail messages to determine the extent and nature of the problem,  rework the USGS analysis, and conduct quality assurance and e-mail  training. Second, DOE needs to ensure that specific engineering designs  reflected high-level plans and regulatory requirements. For example,  design changes to a spent fuel handling building led to the description of  different design requirements regarding the need for a water-based, fire- suppression system. The activation of such a system could facilitate a  nuclear reaction, if there were also an inadvertent release of spent nuclear  fuel inside the building. The requirement for the system, therefore, was  eliminated, but not all building design documents reflected the change.  Third, DOE has experienced substantial turnover in key project  management positions. NRC has stressed the importance of a continuity of  qualified managers rather than a series of acting managers, but 9 of 17 key  management positions at DOE turned over between 2001 and 2006.
Quality assurance problems are not new at the project and over time have  contributed to delays in filing a license application. In 2001, DOE  determined that, in part because of ongoing efforts to resolve quality  assurance problems, it would be unable to submit a license application to  NRC by December 2002, the target date scheduled when the Congress  approved the Yucca Mountain site. DOE was also unable to meet a  December 2004 goal for submitting a license application. Past initiatives to  address these problems and prepare for the submission of a license  application have included changes to improve the project’s organizational  culture and ability to quickly detect and resolve problems. Subsequently,  in October 2005, DOE implemented its “New Path Forward” by making  major changes to the project’s design, organization, and management. For  example, to improve accountability at the project, it reorganized project  staff to create a single manager in charge of the project’s major tasks in  science, engineering, and licensing. DOE also designated Sandia National  Laboratories as the project’s lead laboratory to integrate the scientific  work that Bechtel/SAIC Company, LLC (BSC), the project’s lead  contractor, had previously overseen.
Shortly after his Senate confirmation in May 2006, OCRWM’s director  announced a new schedule to submit DOE’s license application for the  repository to NRC by June 30, 2008. The director has also fundamentally  changed DOE’s management of the Yucca Mountain project—DOE now  directly manages the project, rather than limiting its role to overseeing  BSC’s implementation of its management and operating contract.  OCRWM’s director and deputy director now hold monthly program review  meetings with DOE and contractor project managers and routinely  participate in quality assurance management meetings with a focus on  identifying and correcting problems. OCRWM’s director has also identified  specific performance measures for each project employee to improve the  organizational culture.
In light of persistent quality assurance problems and other delays that the  Yucca Mountain project has experienced, we examined (1) DOE’s  development of its schedule for filing a license application by June 30,  2008, and the stakeholders with whom it consulted; (2) NRC’s assessment  of DOE’s readiness to file a license application that is high quality and  enables NRC to complete its evaluation within 3 to 4 years; and (3) DOE’s  progress in implementing the recommendations and resolving the  additional challenges identified in our March 2006 report and the extent to  which these challenges were considered in setting the license application  schedule.
To examine DOE’s development of its license application schedule, we  reviewed Yucca Mountain project documents, including DOE’s license  application management plan and progress reports, and interviewed senior  OCRWM managers. To obtain NRC’s assessment of DOE’s readiness to file  a high-quality license application, we attended prelicensing meetings and  other interactions between NRC and DOE and reviewed NRC  correspondence and statements regarding DOE’s planned license  application submission. We also interviewed NRC officials regarding these  interactions and their views of the planned license application. Finally, to  determine DOE’s progress in implementing the recommendations in our  March 2006 report and resolving the additional challenges we identified,  we reviewed prior GAO reports, corrective action reports, and related  Yucca Mountain project documents; interviewed senior OCRWM managers  in DOE headquarters and the Yucca Mountain project office in Las Vegas,  Nevada; and interviewed NRC officials. We conducted our work from  February 2007 through July 2007 in accordance with generally accepted  government auditing standards.
subsections: section_title: Background
paragraphs: The Nuclear Waste Policy Act of 1982, as amended, establishes a  comprehensive policy and program for the safe, permanent disposal of  commercial spent nuclear fuel and other highly radioactive wastes in one  or more geologic repositories. The act charges DOE with (1) establishing  criteria for recommending sites for repositories; (2) “characterizing”  (investigating) the Yucca Mountain site to determine its suitability for a  repository; (3) if the site is found suitable, recommending it to the  President, who would submit a recommendation to the Congress if he  agreed that the site was qualified; and (4) seeking permission from NRC to  construct and operate a repository at the approved site. Under the Nuclear  Waste Policy Act, users of nuclear-power-generated electricity pay $0.001  per kilowatt-hour into a Nuclear Waste Fund, which may be used only to  pay for the siting, licensing, and construction of a nuclear waste  repository. In fiscal year 2006, DOE reported that the fund had $19.4  billion. DOE also reported that it had spent about $11.7 billion (in fiscal  year 2006 dollars) from project inception in fiscal years 1983 through 2005  and estimated that an additional $10.9 billion (in fiscal year 2006 dollars)  would be incurred from fiscal years 2006 to 2017 to build the repository.
Since the early 1980s, DOE has studied the Yucca Mountain site to  determine whether it is suitable for a high-level radioactive waste and  spent nuclear fuel repository. For example, DOE completed numerous  scientific studies of water flow and the potential for rock movement near  the mountain, including the likelihood that volcanoes and earthquakes will  adversely affect the repository’s performance. To allow scientists and  engineers greater access to the rock being studied, DOE excavated two  tunnels for studying the deep underground environment: (1) a 5-mile main  tunnel that loops through the mountain, with several research areas or  alcoves connected to it, and (2) a 1.7-mile tunnel that crosses the  mountain, allowing scientists to study properties of the rock and the  behavior of water near the potential repository area. Since July 2002, when  the Congress approved the President’s recommendation of the Yucca  Mountain site for the development of a repository, DOE has focused on  preparing its license application.
In October 2005, DOE announced a series of changes in the management  of the project and in the design of the repository to simplify the project  and improve its safety and operation. Previously, DOE’s design required  radioactive waste to be handled at least four separate times by  transporting the waste to the Yucca Mountain site, removing the waste  from its shipping container, sealing it in a special disposal container, and  moving it into the underground repository. The new repository design  relies on uniform canisters that would be filled and sealed before being  shipped, reducing the need for direct handling of most of the waste prior  to being placed in the repository. As a result, DOE will not have to  construct several extremely large buildings costing millions of dollars for  handling radioactive waste. In light of these changes, DOE has been  working on revising the designs for the repository’s surface facilities,  developing the technical specifications for the canisters that will hold the  waste, and revising its draft license application.
In accordance with NRC regulations, before filing its license application,  DOE must first make all documentary material that is potentially relevant  to the licensing process electronically available via NRC’s Internet-based  document management system. This system, known as the Licensing  Support Network, provides electronic access to millions of documents  related to the repository project. DOE is required to initially certify to NRC  that it has made its documentary material available no later than 6 months  in advance of submitting the license application. NRC, Nevada, and other  parties in the licensing process must also certify their documentary  material was made available following DOE’s initial certification. This  information will then be available to the public and all the parties  participating in the licensing process. OCRWM currently expects to certify  its material in the Licensing Support Network by December 21, 2007. In  addition, OCRWM expects to complete the necessary designs and have the  draft license application ready for DOE management’s review by   February 29, 2008.
NRC is charged with regulating the construction, operation, and  decommissioning phases of the project and is responsible for ensuring  that DOE satisfies public health, safety, and environmental regulatory  requirements. Once DOE files the license application, NRC will begin a  four-stage process to process the application and decide whether to   (1) authorize construction of the repository, (2) authorize construction  with conditions, or (3) deny the application. As shown in figure 1, this  process includes the following steps:    Acceptance review. NRC plans to take up to 180 days to examine the  application for completeness to determine whether the license application  has all of the information and components NRC requires. If NRC  determines that any part of the application is incomplete, it may either  reject the application or require that DOE furnish the necessary  documentation. NRC will docket the application once it deems the  application complete, indicating its readiness for a detailed technical  review.
Technical review. The detailed technical review, scheduled for 18 to 24  months, will evaluate the soundness of the scientific data, computer  modeling, analyses, and preliminary facility design. The review will focus  on evaluating DOE’s conclusions about the ability of the repository  designs to limit exposure to radioactivity, both during the construction  and operation phase of the repository (known as preclosure) and during  the phase after the repository has been filled, closed, and sealed (known  as postclosure.) If NRC discovers problems with the technical information  used to support the application, it may conduct activities to determine the  extent and effect of the problem. As part of this review, NRC staff will  prepare a safety evaluation report that details staff findings and  conclusions on the license application.
Public hearings. NRC will also convene an independent panel of judges— called the Atomic Safety Licensing Board—to conduct a series of public  hearings to address contested issues raised by affected parties and review  in detail the related information and evidence regarding the license  application. Upon completion, the board will make a formal ruling (called  the initial decision) resolving matters put into controversy. This initial  decision can then be appealed to the NRC commissioners for further  review.
NRC commission review. In the likely event of an appeal, the NRC  commissioners will review the Atomic Safety Licensing Board’s initial  decision. In addition, outside of the adjudicatory proceeding, they will  complete a supervisory examination of those issues contested in the  proceeding to consider whether any significant basis exists for doubting  that the facility will be constructed or operated with adequate protection  of the public health and safety. The commissioners will also review any  issues about which NRC staff must make appropriate findings prior to the  authorization of construction, even if they were not contested in the  proceeding.
However, until DOE submits a license application, NRC’s role has involved  providing regulatory guidance; observing and gathering information on  DOE activities related to repository design, performance assessment, and  environmental studies; and verifying site characterization activities. These  prelicensing activities are intended to identify and resolve potential  licensing issues early to help ensure that years of scientific work are not   found to be inadequate for licensing purposes. DOE and NRC have  interacted since 1983 on the repository. In 1998, they entered into a  prelicensing interaction agreement that provides for technical and  management meetings, data and document reviews, and the prompt  exchange of information between NRC’s on-site representatives and DOE  project personnel. Consistent with this prelicensing interaction agreement  and NRC’s regulations, NRC staff observe and review activities at the site  and other scientific work as they are performed to allow early  identification of potential licensing issues for timely resolution at the staff  level.
EPA also has a role in the licensing process––setting radiation exposure  standards for the public outside the Yucca Mountain site. In 2001, EPA set  standards for protecting the public from inadvertent releases of  radioactive materials from wastes stored at Yucca Mountain, which are  required by law to be consistent with recommendations of the National  Academy of Sciences. In July 2004, the U.S. Court of Appeals for the  District of Columbia Circuit ruled that EPA’s standards were not  consistent with the National Academy of Sciences’ recommendations. In  response, EPA proposed a revised rule in August 2005. The director of  EPA’s Office of Air and Radiation Safety told us that EPA plans to finalize  its rule this year. In addition, NRC must develop exposure limits that are  compatible with EPA’s rule. NRC published a proposed rule which it states  is compatible with EPA’s rule, received public comments in 2005, but has  not yet finalized the rule. If EPA’s rule does not change significantly in  response to public comments, NRC’s rule would not require major  revisions either and could be finalized within months. However, if EPA’s  final rule has major changes, it could require major changes to NRC’s rule,  which could take more than a year to redraft, seek and incorporate public  comments, and finalize, according to NRC officials.
section_title: DOE’s Schedule to Submit a License Application to NRC by June 30, 2008, Was Developed in Consultation with Yucca Mountain Project Managers
paragraphs: In July 2006, DOE announced its intent to file a license application to NRC  no later than June 30, 2008. OCRWM’s director set the June 30, 2008, goal  to jump-start what he viewed as a stalled project. OCRWM’s director told  us that he consulted with DOE and contractor project managers to get a  reasonable estimate of an achievable date for submitting the license  application and asked OCRWM managers to develop a plan and schedule  for meeting the June 30, 2008, goal. OCRWM’s director believes this  schedule is achievable, noting that DOE had already performed a  significant amount of work toward developing a license application.  Specifically, DOE completed a draft license application in September 2005,  but opted not to file it with NRC to allow more time to address the USGS  e-mail issue, revise the repository’s design to simplify the project and  improve its safety and operation, and consider revising its technical  documents in response to the possibility that EPA would revise the  radiation standards for the proposed repository. Table 1 shows the  project’s major milestones.
DOE did not consult with external stakeholders in developing this  schedule because there was no legal or regulatory requirement or  compelling management reason to do so, according to senior OCRWM  officials. However, these officials noted that the NRC review process  includes extensive public hearings on the application, which will provide  stakeholders with an opportunity to comment on and challenge the  substance of the application. In addition, regarding other aspects of the  program, senior OCRWM officials noted that they have often consulted  with external stakeholders, including city and county governments near  the proposed repository site, NRC, USGS, and nuclear power companies.  OCRWM has also consulted with Nevada, the U.S. Department of the Navy,  and other DOE offices. For example, in developing its standards for the  canisters that will be used to store, transport, and place the waste in the  repository, DOE consulted with the Navy and the nuclear power plant  operators that generate the nuclear waste and will use the proposed  canisters. In addition, DOE has worked with the local city and county  governments near the repository to develop the plans for transporting the  waste to the proposed repository.
OCRWM’s director has made the submission of the license application by  June 30, 2008, the project’s top strategic objective and management  priority. Accordingly, each OCWRM office has created business plans  detailing how its work will support this objective. Furthermore, DOE has  developed a license application management plan that incorporates the  lessons learned from previous license application preparation efforts and  works to ensure that the license application meets all DOE and NRC  statutory, regulatory, and quality requirements. The plan establishes a  process whereby teams assess the statutory and regulatory requirements  for the license application, identify any gaps and inadequacies in the  existing drafts of the license application, and draft or revise these sections.  Since the license application is expected to be thousands of pages long,  the plan divides the license application into 71 subsections, each with a  team assigned specific roles and responsibilities, such as for drafting a  particular subsection or approving a particular stage of the draft. Finally,  the plan also creates new project management controls to provide  oversight of this process and manage risks. For example, the plan details  how issues that may pose risks to the schedule or quality of the license  application should be noted, analyzed, and resolved, and how the  remaining issues should be elevated to successively higher levels of  management.
section_title: NRC Officials Are Uncertain Whether DOE Will File a High- Quality License Application That Will Facilitate Completion of a Timely Review
paragraphs: NRC officials believe it is likely that DOE will submit a license application  by June 30, 2008, but will not speculate about its quality due to a long- standing practice to maintain an objective and neutral position toward  proposed license applications until they are filed with NRC. According to  NRC officials, NRC’s ability to review an application in a timely manner is  contingent on the application being high quality, which NRC officials  define as being complete and accurate, including traceable and  transparent data that adequately support the technical positions presented  in the license application. NRC has expressed concern about the lack of a  rigorous quality assurance program and the reliability of USGS scientific  work that DOE had certified before the USGS e-mails were discovered.  Based on its prelicensing review, NRC recognizes that DOE is addressing  problems with its quality assurance program and, by developing a new  water infiltration model, is restoring confidence in the reliability of its  scientific work.
When the Nuclear Waste Policy Act of 1982 gave NRC responsibility for  licensing the nuclear waste repository, NRC staff began engaging in  prelicensing activities aimed at gathering information from DOE and  providing guidance so that DOE would be prepared to meet NRC’s  statutory and regulatory requirements and NRC would be prepared to  review the license application. NRC issued high-level waste disposal  regulations containing criteria for approving the application and publicly  available internal guidance detailing the steps and activities NRC will  perform to review the application. NRC also established a site office at  OCRWM’s Las Vegas, Nevada, offices to act as NRC’s point of contact and  to facilitate prompt information exchanges. NRC officials noted that they  have also been working for several years to communicate NRC’s  expectations for a high-quality license application.
Although NRC has no formal oversight role in the Yucca Mountain project  until DOE files a license application, NRC staff observe DOE audits of its  quality assurance activities to identify potential issues and problems that  may affect licensing. The NRC staff then report their findings in quarterly  reports that summarize their work and detail any problems or issues they  identify. For example, after observing a DOE quality assurance audit at the  Lawrence Livermore National Laboratory in August 2005, NRC staff  expressed concern that humidity gauges used in scientific experiments at  the project were not properly calibrated—an apparent violation of quality  assurance requirements. Due in part to concerns that quality assurance  requirements had not been followed, BSC issued a February 7, 2006, stop- work order affecting this scientific work. In June 2007, OCRWM project  managers told us that because quality assurance rules were not followed,  DOE could not use this scientific work to support the license application.
To facilitate prelicensing interactions, NRC and DOE developed a formal  process in 1998 for identifying and documenting technical issues and  information needs. As shown in table 2, issues were grouped into nine key  technical issues focused mainly on postclosure performance of the  geologic repository. Within this framework, NRC and DOE defined 293  agreements in a series of technical exchange meetings. An agreement is  considered closed when NRC staff determines that DOE has provided the  requested information. Agreements are formally closed in public  correspondence or at public technical exchanges. As of June 2007, DOE  has responded to all 293 of the agreements. NRC considers 260 of these to  be closed. NRC considers 8 of the remaining 33 agreements to be  potentially affected by the USGS e-mail issue that emerged in 2005. Their  resolution will be addressed after NRC examines the new water  infiltration analysis. NRC considers that the remaining 25 have been  addressed but still need additional information. DOE has indicated that it  does not plan any further responses on these agreements, and that the  information will be provided in the June 2008 license application.
NRC determined that adding agreements to the original 293 was not an  efficient means to continue issue resolution during prelicensing, given  DOE’s stated intent to submit its license application, first in 2004, and now  in 2008. NRC is now using public correspondence, as well as public  technical exchanges and management meetings, to communicate  outstanding and emerging technical issues. For example, NRC’s September  2006 correspondence provided input on DOE’s proposed approach for  estimating seismic events during the postclosure period and requested  further interactions on the topic. Also, since May 2006, NRC and DOE have  conducted a series of technical exchanges to discuss such topics as DOE’s  total system performance assessment model, the seismic design of  buildings, and other DOE design changes. Other interactions are planned  to ensure that NRC has sufficient information to conduct its prelicensing  responsibilities.
section_title: DOE Has Made Progress in Implementing Our Quality Assurance Recommendations and Resolving Challenges We Identified
paragraphs: DOE is implementing the recommendations and addressing the challenges  identified in our March 2006 report, but it is unclear whether the  department’s actions will prevent similar problems from recurring.  Specifically, in response to our recommendations that DOE improve its  management tools, DOE has eliminated the one-page summary (or panel)  of performance indicators and has revised its trend evaluation reports.  DOE is supplementing these changes with more rigorous senior  management meetings that track program performance to better ensure  that new problems are identified and resolved. DOE has also begun  addressing additional management challenges by independently reworking  USGS’s water infiltration analysis, fixing problems with a design and  engineering process known as requirements management, and reducing  the high-turnover rate and large number of acting managers in key project  management positions.
subsections: section_title: DOE Has Replaced or Improved Two Previously Ineffective Management Tools
paragraphs: Our March 2006 report found that two of the project’s management tools— the panel of performance indicators and the trend evaluation reports— were ineffective in helping DOE management to monitor progress toward  meeting performance goals, detecting new quality assurance problems,  and directing management attention where needed. In response, DOE has  stopped using its panel of performance indicators and replaced them with  monthly program review meetings—chaired by OCRWM’s director and  attended by top-level OCRWM, BSC, Sandia, and USGS managers—that  review the progress of four main OCRWM projects: (1) the drafting of the  license application; (2) the effort to select and load documents and  records into NRC’s Licensing Support Network; (3) work supplementing  DOE’s environmental impact statement to reflect the October 2005  changes in repository design, which shift from direct handling of waste to  the use of canisters; and (4) the development of a system to transport  waste from where it is generated, mainly nuclear power plants, to the  repository. In addition, DOE has developed the following four new, high- level performance indicators that it evaluates and discusses at its monthly  program review meetings:    safety, including injuries and lost workdays due to accidents at the project;    quality, including efforts to improve OCRWM’s corrective action program,  which works to detect and resolve problems at the project and the  performance of the quality assurance program;   cost, including actual versus budgeted costs, staffing levels, and efforts to  recruit new employees; and    culture, including the project’s safety conscious work environment  program, which works to ensure that employees are encouraged to raise  safety concerns to their managers or to NRC without fear of retaliation  and that employees’ concerns are resolved in a timely and appropriate  manner according to their importance.
Although DOE plans to develop additional performance indicators, these  four simplified indicators have replaced about 250 performance indicators  on the previous performance indicator panel. According to a cognizant  DOE official, the previous performance indicator panel was ineffective, in  part, because it focused on what could be measured, as opposed to what  should be measured, resulting in DOE focusing its efforts on developing  the performance indicator panel instead of determining how to use this  information as a management tool. The monthly program review and the  new performance indicators are designed to be more useful to OCRWM  management by being simpler and more focused on the key mission  activities.
DOE has also revised its trend evaluation reports to create new  organizational structures and procedures that detail the processes and  steps for detecting and analyzing trends and preparing trend evaluation  reports for senior management review. DOE has appointed a trend  program manager and implemented a work group to oversee these  processes. Furthermore, as we recommended, the new trend program has  an increased focus on the significance of the monitored condition by  synthesizing trends projectwide instead of separating OCRWM’s and BSC’s  trend evaluation reports.
To improve the utility of trend evaluation reports as a management tool,  the procedures now identify the following three types of trends and  criteria for evaluating them:    Adverse trends are (1) repeated problems that involve similar tasks or  have similar causes and are determined by management to be significant  or critical to the success of the project; (2) repeated problems that are less  significant but collectively indicate a failure of the quality assurance  program, may be precursors to a more significant problem, or pose a  safety problem; and (3) patterns of problems that management determines  warrant further analysis and actions to prevent their recurrence.
Emerging trends are problems that do not meet the criteria for an adverse  trend, but require actions to ensure that they do not evolve into an adverse  trend.
Monitored trends are fluctuations in the conditions being monitored that  OCRWM management determines do not warrant action, but each  fluctuation needs close monitoring to ensure that it does not evolve into  an emerging or adverse trend.
DOE has also implemented changes to its corrective action program—the  program that provides the data that are analyzed in the trend evaluation  program. The corrective action program is the broader system for  recognizing problems and tracking their resolution. It is one of the key  elements of the project’s quality assurance framework and has been an  area of interest to NRC in its prelicensing activities. The corrective action  program consists of a computer system that project employees can use to  enter information about a problem they have identified and create a  record, known as a condition report, and a set of procedures for  evaluating the condition reports and ensuring these problems are resolved.
Regarding our broader conclusions that the OCRWM quality assurance  program needed more management attention, in spring 2006, DOE  requested a team of external quality assurance experts to review the  performance of the quality assurance program. The experts concluded that  8 of the 10 topics they studied—including the corrective action program— had not been effectively implemented. Specifically, the team found that the  corrective action program did not ensure that problems were either  quickly or effectively resolved. Furthermore, a follow-up internal DOE  study, called a root cause analysis report, concluded that the corrective  action program was ineffective primarily because senior management had  failed to recognize the significance of repeated internal and external  reviews and did not aggressively act to correct identified problems and  ensure program effectiveness.
In response, DOE has revised the corrective action program in an effort to  change organizational behaviors and provide increased management  attention. For example, DOE has restructured the condition screening  team, which previously had poor internal communication and adversarial  relationships among its members, according to a senior project manager.  Similarly, a December 2006 external review of the quality assurance  program found that OCRWM staff had focused its efforts on trying to  downgrade the significance of condition reports to deflect individual and  departmental responsibility, rather than ensuring that the underlying  causes and problems were addressed. In response, DOE (1) reorganized  the condition screening team to reduce the size of the team but include  more senior managers; (2) identified roles, responsibilities, and  management expectations for the team, including expectations for  collaborating and communicating; and (3) formalized processes and  criteria for screening and reviewing condition reports. The condition  screening team now assigns one of four significance levels to each new  condition report and assigns a manager who is responsible for  investigating the problem.
In addition, DOE has restructured the management review committee,  which oversees the corrective action program and the condition screening  team. The management review committee is charged with, among other  things, reviewing the actions of the condition screening team, particularly  regarding the condition reports identified as having the highest two levels  of significance. The management review committee also reviews draft root  cause analysis reports, and any condition reports that could affect the  license application. Whereas these functions were previously performed  by BSC, the management review committee is now sponsored by  OCRWM’s deputy director and includes senior DOE, BSC, and Sandia  managers. DOE has also created written policies to clarify the roles,  responsibilities, and expectations of the management review committee.  The goal of these changes is to refocus management attention––with  OCRWM’s deputy director serving as a champion for the corrective action  program––and ensure that problems are resolved in a timely and efficient  manner.
section_title: DOE Has Addressed Other Management Challenges
paragraphs: DOE has addressed to varying degrees three other management challenges  identified in our March 2006 report: (1) restoring confidence in USGS’s  scientific documents; (2) problems with a design and engineering process  known as requirements management; and (3) managing a changing and  complex program, particularly given the high turnover in key management  positions. Specifically:    USGS e-mail issue. DOE has taken three actions to address concerns  about the reliability of USGS’s scientific work after a series of e-mails  implied that some USGS employees had falsified scientific and quality  assurance documents and disdained DOE’s quality assurance processes.  Specifically, DOE (1) evaluated USGS’s scientific work; (2) directed  Sandia to independently develop a new water infiltration model to  compare with USGS’s model and reconstruct USGS’s technical documents;  and (3) completed a root cause analysis, including a physical review of  more than 50,000 e-mails and keyword searches of nearly 1 million other   e-mails sampled from more than 14 million e-mails. DOE’s evaluation of  USGS’s scientific work concluded that there was no evidence that the  USGS employees falsified or modified information. DOE’s root cause  analysis team concluded that there was no apparent widespread or  pervasive pattern across OCRWM of a negative attitude toward quality  assurance or willful noncompliance with quality assurance requirements.  However, the analysis found that OCRWM’s senior management had failed  to hold USGS personnel accountable for the quality of the scientific work,  fully implement quality assurance requirements, and effectively implement  the corrective action program. These internal studies and reports and  Sandia’s independent development of a new water infiltration model are  intended to restore public confidence in the water infiltration modeling  work in the license application.
Problems with design control and the requirements management  process. DOE has revised its design control and requirements management  processes to address the problems that our March 2006 report identified.  In addition, to gauge the effectiveness of these changes, DOE conducted  an internal study called a readiness review, in which it determined that the  changes in the processes were sufficient and that BSC was prepared to  resume design and engineering work. Subsequently, in January 2007,  DOE’s independent assessment of BSC and the requirements management  process concluded that the processes and controls were adequate and  provided a general basic direction for the design control process. DOE has  also contracted with Longenecker and Associates to review the project’s  engineering processes with the final report due in the summer of 2007.
Management turnover. DOE has worked to fill and retain personnel in key  management positions that had been vacant for extended periods of time,  most notably the director of quality assurance and the OCRWM project  director. In addition, as part of an effort to change the organizational  culture, OCRWM’s director has created a team to evaluate how to improve  succession planning and identify gaps in the skills or staffing levels in  OCRWM. However, DOE continues to lose key project managers, most  recently with the departure of OCRWM’s deputy director. Furthermore,  additional turnover is possible after the 2008 presidential election, when  the incoming administration is likely to replace OCRWM’s director.  Historically, new directors have tended to have different management  priorities and have implemented changes to the organizational structure  and policies. To address this concern, OCRWM’s director suggested  legislatively changing the director position by making it a long-term  appointment to reflect the long-term nature of the Yucca Mountain project.
section_title: Concluding Observations
paragraphs: The OCRWM director’s schedule for filing a repository license application  with NRC by June 30, 2008, will require a concerted effort by project  personnel. However, given the waste repository’s history since its  inception in 1983, including two prior failed efforts to file a license  application, it is unclear whether DOE’s license application will be of  sufficient quality to enable NRC to conduct a timely review of the  supporting models and data that meet the statutory time frames. DOE has  taken several important actions to change the organizational culture of the  Yucca Mountain project since the issuance of our March 2006 report.  These actions appear to be invigorating, for example, the quality assurance  program by focusing management attention on improving quality by  resolving problems. However, for a variety of reasons, it has yet to be seen  whether DOE’s actions will prevent the kinds of problems our March 2006  report identified from recurring or other challenges from developing. First,  some of DOE’s efforts, such as its efforts to reduce staff turnover, are in  preliminary or planning stages and have not been fully implemented.  Therefore, their effectiveness cannot yet be determined. Second,  improving the quality assurance program will also require changes in the  organizational behaviors of OCRWM’s staff and contractors. OCRWM’s  director told us that these types of cultural changes can be particularly  difficult and take a long time to implement. Consequently, it may be years  before OCRWM fully realizes the benefits of these efforts. Finally, as we  have previously reported, DOE has a long history of quality assurance  problems and has experienced repeated difficulties in resolving these  problems.
section_title: Agency Comments
paragraphs: We provided DOE and NRC with a draft of this report for their review and  comment. In their written responses, both DOE and NRC agreed with our  report. (See apps. I and II.) In addition, both DOE and NRC provided  comments to improve the draft report’s technical accuracy, which we have  incorporated as appropriate.
section_title: Scope and Methodology
paragraphs: To examine the development of DOE’s license application schedule, we  reviewed DOE documents related to the announcement and creation of  the license application. We also reviewed the DOE management plan for  creating the license application and other internal reports on the progress  in drafting the application. We interviewed OCRWM’s director and other  OCRWM senior management officials in DOE headquarters and its Las  Vegas project office about the process for creating the schedule, including  consultations with stakeholders. In addition, we observed meetings  covering topics related to the license application schedule between DOE  and NRC, the Advisory Committee on Nuclear Waste and Materials, and  the Nuclear Waste Technical Review Board. These meetings were held in  Rockville, Maryland; Las Vegas, Nevada; and Arlington, Virginia.
To obtain NRC’s assessment of DOE’s readiness to file a high-quality  license application, we obtained NRC documents—such as the status of  key technical issues and briefing slides on NRC’s technical exchanges with  DOE. We also attended NRC staff briefings for the Commission’s Advisory  Committee on Nuclear Waste and Materials, including a briefing on NRC’s  prelicensing activities; reviewed meeting transcripts; and observed a NRC- DOE quarterly meeting and recorded NRC’s comments. In addition, we  interviewed NRC’s project manager who is responsible for reviewing the  postclosure portion of a license application, NRC’s on-site representative  at the Las Vegas office, and other NRC regional officials. Furthermore, we  interviewed the director of EPA’s Office of Air and Radiation Safety  regarding the status of EPA’s rulemaking to set radiation exposure  standards for the public outside the Yucca Mountain site.
To determine DOE’s progress in implementing the recommendations and  resolving the additional challenges identified in our March 2006 report, we  reviewed prior GAO reports that assessed DOE’s quality assurance  process and relevant DOE corrective action reports, root cause analyses,  and other internal reviews that analyzed DOE’s efforts to improve its  management tools and its corrective action program in general. We also  reviewed related NRC documents, such as some observation audit reports.  We observed NRC and DOE management meetings and technical  exchanges in Rockville, Maryland, and Las Vegas, Nevada, that covered  related issues. We also interviewed OCRWM’s director in DOE  headquarters and senior managers at the Yucca Mountain project office in  Las Vegas about their efforts to address our recommendations. Regarding  the quality assurance challenges noted in our prior report, we reviewed a  January 2007 GAO report discussing the USGS issue and reviewed DOE  documents detailing their actions to restore confidence in the scientific  documents. We reviewed internal DOE documents regarding requirements  management and interviewed the program’s chief engineer in charge of  resolving this issue. Finally, regarding staff turnover in key management  positions, we reviewed OCRWM’s strategic objectives, business plan, and  project documents and interviewed OCRWM’s director and other senior  project managers about their efforts to improve succession planning.
As agreed with your office, unless you publicly announce the contents of  this report, we plan no further distribution of it until 30 days from the date  of this letter. At that time, we will send copies of this report to the  appropriate congressional committees, the Secretary of Energy, the  Chairman of the Nuclear Regulatory Commission, the director of the  Office of Management and Budget, and other interested parties. We will  also make copies available to others upon request. In addition, the report  will be available at no charge on the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact me  at (202) 512-3841 or gaffiganm@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Key contributors to this report were Richard Cheston, Casey  Brown, Omari Norman, Alison O’Neill, and Daniel Semick.
section_title: Appendix I: Comments from the Department of Energy
section_title: Appendix II: Comments from the Nuclear Regulatory Commission