Modernizing the Nuclear Security Enterprise: NNSA's Reviews of Budget Estimates and Decisions on Resource Trade-offs Need Strengthening
Published: 2012-07-31T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: NNSA, a semiautonomous agency within DOE, is responsible for the nations nuclear weapons, nonproliferation, and naval reactors programs. Since its inception in 2000, the agency has faced challenges in its ability to accurately identify the costs of major projects. In addition, both the DOE Inspector General, in 2003, and GAO, in 2007, reported concerns with NNSAs PPBE process, specifically in how NNSA validates budget estimates and decides on resource allocations or trade-offs.
GAO was asked to review how NNSA manages programming and budgeting through its PPBE process. GAO examined (1) the current structure of NNSAs PPBE process, (2) the extent to which NNSA reviews its budget estimates, and (3) how NNSA decides on resource trade-offs in its PPBE process. To carry out its work, GAO reviewed NNSA policies, instructions, guidance, and internal reports documenting the agencys PPBE process and interviewed NNSA, DOE, and M&O contractor officials.
section_title: What GAO Found
paragraphs: The National Nuclear Security Administrations (NNSA) planning, programming, budgeting, and evaluation (PPBE) process provides a framework for the agency to plan, prioritize, fund, and evaluate its program activities. Formal policies guide NNSA and management and operating (M&O) contractors through each of four phases of the agencys PPBE cycleplanning, programming, budgeting, and evaluation. These phases appear to be sequential, but the process is continuous and concurrent because of the amount of time required to develop priorities and review resource requirements, with at least two phases ongoing at any time.
NNSA does not thoroughly review budget estimates before it incorporates them into its proposed annual budget. Instead, NNSA relies on informal, undocumented reviews of such estimates and its own budget validation review processthe formal process for assessing budget estimates. Neither of these processes adheres to Department of Energy (DOE) Order 130.1, which defines departmental provisions for the thoroughness, timing, and documentation of budget reviews. NNSA officials said the agency does not follow the order because it expired in 2003. Nevertheless, the order is listed as current on DOEs website, and a senior DOE budget official confirmed that it remains in effect, although it is outdated in terminology and organizational structure. Additionally, according to NNSA officials, the agencys trust in its contractors minimizes the need for formal review of its budget estimates. GAO identified three key problems in NNSAs budget validation review process. First, this process does not inform NNSA, DOE, Office of Management and Budget, or congressional budget development decisions because it occurs too late in the budget cycleafter the submission of the Presidents budget to Congress. Second, this process is not sufficiently thorough to ensure the credibility and reliability of NNSAs budget because it is limited to assessing the processes used to develop budget estimates rather than the accuracy of the resulting estimates and is conducted for a small portion of NNSAs budgetapproximately 1.5 percent of which received such review in 2011. Third, other weaknesses in this process, such as no formal evaluative mechanism to determine if corrective actions were taken in response to previous findings, limit the processs effectiveness in assessing NNSAs budget estimates.
NNSA uses a variety of management tools to decide on resource trade-offs during the programming phase of the PPBE process. One of these tools, integrated priority listswhich rank program activities according to their importance for meeting mission requirementsis to provide senior managers with an understanding of how various funding scenarios would affect program activities. However, NNSA has weakened its ability to gauge the effects of resource trade-offs. For example, in 2010, NNSA disbanded its Office of Integration and Assessments, created in response to DOE Inspector General and GAO recommendations that NNSA establish an independent analysis unit to perform such functions as reviewing proposals for program activities and verifying cost estimates. NNSA agreed with these recommendations and, in 2009, instituted the office to identify, analyze, and assess options for deciding on resource trade-offs. Without an independent analytical capability, NNSA may have difficulty making the best decisions about what activities to fund and whether they are affordable.
section_title: What GAO Recommends
paragraphs: GAO recommends that, among other things, DOE update the departmental order for budget reviews, improve the formal process for reviewing budget estimates, and reinstitute an independent analytical capability. The agency agreed in principle with six recommendations but not with one to consolidate various integrated priority lists. GAO continues to believe this recommendation has merit as discussed in the report.

section_title: Letter
paragraphs: The National Nuclear Security Administration (NNSA)—a  semiautonomous agency within the Department of Energy (DOE)— manages the nation’s nuclear security programs, the primary goals of  which are to maintain the safety, security, and effectiveness of the U.S.  nuclear weapons stockpile; reduce the global threat posed by nuclear  proliferation and terrorism; and provide nuclear propulsion systems for the  U.S. Navy. To manage these programs, NNSA oversees eight separate  sites—collectively known as the nuclear security enterprise—that are  managed and operated by private contractors, known as management  and operating (M&O) contractors. Among other things, these contractors  operate and maintain the government-owned facilities and infrastructure  deemed necessary to support the nuclear security enterprise and to  support the capabilities to conduct scientific, technical, engineering, and  production activities. To fund these activities, the President requested  more than $11.5 billion for NNSA in his fiscal year 2013 budget  submission to Congress. Sound management practices, such as linking  program activities to defined objectives and ensuring the reliability of  budget estimates, can help NNSA use agency resources wisely. It is  particularly important for NNSA to embrace this approach because, as  our past work has shown, it has faced numerous challenges in accurately  estimating the costs of major projects since its inception in 2000. For  example, we reported in March of this year that the estimated cost to  construct the Chemistry and Metallurgy Research Replacement Nuclear  Facility at NNSA’s Los Alamos National Laboratory rose from between  $745 million and $975 million in 2005 to between $3.7 billion and $5.8  billion in 2010—a near sixfold increase. In addition, in February of this  year we testified that the estimated cost to construct the modern Uranium  Processing Facility at NNSA’s Y-12 National Security Complex jumped  from between $600 million and $1.1 billion in 2004 to between $4.2 billion  and $6.5 billion in 2011—a near sevenfold increase. Furthermore, in  2000 and 2009, respectively, we reported that NNSA’s efforts to extend  the operational lives of nuclear weapons experienced a $300 million cost  increase for the refurbishment of one warhead and a $70 million increase  for another.projects, we believe improved federal oversight of NNSA’s modernization  of the nuclear security enterprise is critical to ensuring that resources are  spent in as an effective and efficient manner as possible.
NNSA was created by the National Defense Authorization Act for Fiscal Year 2000 (Pub.  L. No. 106-65, § 3201 et seq. ). have reported, DOD’s PPBE process logically attempts to plan programs  and develop budgets based on high-level strategic goals, but at times it  has been poorly implemented. For example, DOD decision makers often  commit to more programs than the department can reasonably support.  Nonetheless, DOD’s process has the potential to provide comparisons of  the cost-effectiveness of program activities and to help managers develop  the kinds of detailed program and budget plans called for in the NNSA  Act. However, both the DOE Inspector General and GAO have reported  on concerns with the development and implementation of NNSA’s PPBE  process. Specifically, these reports identified deficiencies in how NNSA  ensures the validity of its budget estimates and how it decides to allocate  its resources (resource trade-offs). In our January 2007 report, we  recommended, among other things, that NNSA complete all of its PPBE  policy guidance. In response, NNSA issued PPBE policy in the form of  policy letters that identify the responsibilities of NNSA management,  program and site offices, and contractors throughout the agency’s PPBE  cycle.
In this context, you asked us to determine how NNSA manages its  programming and budgeting activities through its PPBE process. For this  report, our objectives were to examine (1) the current structure of NNSA’s  PPBE process, (2) the extent to which NNSA reviews its budget  estimates, and (3) how NNSA decides on resource trade-offs in its PPBE  process.
To address these objectives, we analyzed NNSA documentation— including policies, instructions, guidance, and internal reports—related to  the implementation of NNSA’s PPBE policy across NNSA, as well as  program-specific documentation from the Offices of Defense Programs,  Defense Nuclear Nonproliferation, and Naval Reactors; these three  offices correspond to NNSA’s primary missions and collectively account  for approximately 85 percent of the President’s fiscal year 2013 NNSA  budget submission to Congress. To examine the current structure of  NNSA’s PPBE process, we reviewed NNSA policy documents, guidance,  and instructions. To determine the extent to which NNSA reviews its  budget estimates, we reviewed NNSA’s formal budget validation review  process and associated internal NNSA reports and met with officials  involved in the development, oversight, or execution of NNSA budget  estimate reviews. To identify the tools NNSA uses to decide on resource  trade-offs, we analyzed how NNSA determines which program activities it  considers necessary to achieve NNSA goals. Additionally, to address all  these objectives, we met with officials from the NNSA Offices of the  Administrator, Management and Budget, Defense Programs, Defense  Nuclear Nonproliferation, Naval Reactors, and Acquisition and Project  Management; the site offices for Los Alamos, Sandia, and the Y-12  National Security Complex; and the Naval Reactors Laboratory Field  Office; DOE officials from the Office of Budget; and M&O contractor  officials from Los Alamos and Sandia National Laboratories, the Y-12  National Security Complex, and Bettis Atomic Power Laboratory.Because NNSA’s Office of Naval Reactors is organized as a separate  entity within NNSA reporting both to NNSA and the U.S. Navy, we also  met with Navy officials from its offices of Financial Management and  Budgeting, and Cost Engineering and Industrial Analysis. Appendix I  contains a detailed description of our objectives, scope and methodology.
We conducted this performance audit from July 2011 to July 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: NNSA oversees programs to carry out its missions for nuclear weapons,  nonproliferation, and naval nuclear propulsion, for which the President’s  budget submission to Congress requested more than $11.5 billion for  fiscal year 2013—about 42 percent of DOE’s total budget. NNSA has  primary mission responsibilities in three areas. First, it is responsible for  providing the United States with safe, secure, and reliable nuclear  weapons in the absence of underground nuclear testing and maintains  core competencies in nuclear weapons science, technology, and  engineering. Second, NNSA implements key U.S. government nuclear  security, nonproliferation, and arms control activities, including securing  vulnerable nuclear and radiological material at facilities throughout the  world; removing plutonium and highly enriched uranium from partner  countries; eliminating U.S. nuclear material declared surplus to defense  needs; negotiating and providing the technical capability to verify arms  control treaties and agreements; strengthening other countries’ capacities  to implement nonproliferation obligations; and enhancing other nations’  capabilities to deter and detect illicit movement of nuclear and radiological  materials. Third, NNSA provides the research, development, design, and  operational support for militarily effective naval nuclear propulsion plants,  as well as enriched uranium for fabrication into fuel for the Navy’s  propulsion reactors. NNSA receives four congressional appropriations to  fund its activities, three of which align with its primary missions— Weapons Activities (for Defense Programs), Defense Nuclear  Nonproliferation, Naval Reactors—and one that funds its management  activities—Office of the Administrator.
Since the Manhattan Project produced the first atomic bomb during World  War II, NNSA, DOE, and predecessor agencies have depended on the  expertise of private firms, universities, and others to carry out research  and development work and efficiently operate the government-owned,  contractor-operated facilities necessary for the nation’s nuclear defense.  NNSA conducts its activities at research and development laboratories,  production plants, and other facilities (collectively referred to as the  nuclear security enterprise). Specifically, NNSA operates three national  laboratories that design and ensure the reliability of nuclear weapons— Lawrence Livermore National Laboratory, California; Los Alamos National  Laboratory, New Mexico; and the Sandia National Laboratories, New  Mexico and California; and four nuclear weapons production sites—the  Pantex Plant, Texas; the Y-12 National Security Complex, Tennessee;  the Kansas City Plant, Missouri; and the Savannah River Site, South  Carolina; as well as the Nevada National Security Site.
NNSA’s relationship with its contractors has been formally established  over the years through its M&O contracts—contracting strategies that  give these contractors responsibility to carry out major portions of NNSA’s  missions and apply their scientific, technical, and management  expertise. M&O contractors at NNSA sites operate under NNSA’s  direction and oversight but largely independently of one another. Various  headquarters organizations within NNSA develop policies and NNSA site  offices, colocated with NNSA’s sites, conduct day-to-day oversight of the  M&O contractors, and evaluate the M&O contractors’ performance in  carrying out the sites’ missions.
section_title: NNSA Has Established a Formal PPBE Process That Includes Four Defined Phases
paragraphs: According to NNSA’s policy, the NNSA PPBE process is composed of  four phases—planning, programming, budgeting, and evaluation—and  their associated activities. The different phases of PPBE appear  sequential, but because of the amount of time required to develop and  review resource requirements, the process is continuous and concurrent  with at least two phases ongoing at any given time, including phases for  different fiscal years. Figure 1 shows the four phases and the months  during which each phase is scheduled to occur, according to NNSA  policies and guidance.
Planning. According to NNSA policy, in this phase, scheduled to begin  annually in November, NNSA is to identify the goals it needs to achieve  over the next 5 years and the program activities needed to meet these  goals. According to NNSA officials, these goals are defined in a variety of  documents, including presidential directives, policy statements, and DOE  and NNSA strategic plans. This phase begins with the issuance of  NNSA’s annual Strategic Planning Guidance, which provides any updates  to the strategic plans and identifies any emerging issues. The NNSA  program offices use this guidance to conduct their own internal planning  processes, update their multiyear plans, including revising or adding  program activities needed to meet the agency’s goals.
Programming. According to NNSA policy, in this phase, scheduled to  begin annually in February, NNSA is to determine which program  activities and funding levels it will include in its budget proposal to DOE  for the fiscal year beginning in October of the following calendar year.  This determination is based on analysis of the activities’ estimated costs,  as well as the need to meet the NNSA goals defined in the planning  process. To determine these activities, NNSA program offices are to work  with their contractors to obtain estimates for the cost of the program  activities identified in the planning phase and determine how to  accomplish these activities within anticipated funding levels, which are  defined in annual NNSA Program and Fiscal Guidance. NNSA program  offices are to then rank these activities in order of priority for meeting  program goals and document these decisions in integrated priority lists.  These lists can include proposed program activities above the anticipated  funding levels specified in NNSA guidance—these proposed activities are  known as unfunded requirements. Using these lists, as well as other  briefing materials, a group of senior NNSA officials including the heads of  all program offices—the Program Review Council—then is to meet with  the Principal Deputy Administrator to discuss and defend each program’s  proposed program activities. After reviewing the deliberations of the  Program Review Council and the associated documents provided by the  program offices, the NNSA Administrator is to decide on resources trade- offs that result in the combination of program activities that best meet  NNSA’s goals over the 5-year period covered by the Future Years  Nuclear Security Program plan. The Administrator is responsible for  issuing the Administrator’s Final Recommendations (AFR), scheduled to  be completed in May at the conclusion of the programming phase, to  document NNSA’s justification for its priorities and to serve as the basis  for the agency’s participation in DOE’s program review process, the  Strategic Resources Review.
Budgeting. According to NNSA policy, this phase is to integrate NNSA  planning and programming priorities and budget estimates into DOE’s  departmental budget process and consists of the following three parts:
Budget formulation. During formulation, which is scheduled to begin  annually in July for the fiscal year beginning in October of the  following calendar year, NNSA submits its proposed budget to DOE  and participates in the Strategic Resources Review. If DOE’s budget  deliberations result in changes to NNSA’s proposed budget, NNSA  may have to rebalance its work activities. In September each year,  DOE submits its proposed budget to the Office of Management and  Budget (OMB) for review. Depending on OMB revisions, NNSA may  need to again revise its work activities. These revisions are  incorporated into the President’s final budget request for DOE, which  is submitted to Congress in February.
Budget validation. According to NNSA guidance, the agency uses its  budget validation review process to ensure its budget request is  consistent with NNSA priorities and its budget estimating processes  are reasonable. NNSA policy calls for NNSA’s Office of PPBE to  manage a three-phase process of validating approximately 20 percent  of NNSA’s programs each year, so that 100 percent of its budget is  validated every 5 years. Programs to undergo validations are to be  determined by a combination of factors, including Program Managers’  request, Administrator direction, and/or significant external  interest/high program visibility. During Phase I of the process,  scheduled for completion in June, before the beginning of the fiscal  year in October, program officials determine if their activities conform  with strategic guidance and program plans and review their methods  for formulating budgets. In Phase II, scheduled annually for July to  September, NNSA contractors or program offices, whichever  developed the budget estimates, conduct a self-assessment of their  budget planning, formulation, and cost-estimating processes. Phase II  self-assessments are to be reviewed by a team—known as a  validation review team—that comprises NNSA headquarters and site  office staff. During Phase III, scheduled to occur from July through  August, these validation review teams also review the cost-estimating  practices used by the NNSA contractors and program offices.  Importantly, NNSA’s validation guidance emphasizes that reviews  should focus on the processes used to formulate budget plans and  derive budget estimates rather than on the accuracy of the resulting  estimates. According to this guidance, validation review teams are to  issue a report on their findings on Phases II and III in September to  inform NNSA, DOE, and OMB decisions for the following year’s  budget cycle.
Budget execution. According to NNSA policy, during this process,  DOE and NNSA are to allocate, distribute, and control funds to  achieve the priorities established in the programming phase, and to  maintain the fiscal limits set up in the budgeting phase, which are  subject to appropriation of funds by Congress. The execution  coincides with the fiscal year and commences once appropriated  funds become available—whether by appropriation or Continuing  Resolution—at the beginning of the fiscal year every October.
Evaluation. According to NNSA policy, NNSA is to employ an ongoing  cycle of evaluations to review program performance. Evaluations are to  include annual and quarterly NNSA performance reviews, performance  reviews conducted as part of the Government Performance and Results  Act, reviews conducted by OMB, and DOE oversight activities.
section_title: NNSA Does Not Thoroughly Review Budget Estimates When Developing Its Annual Budget
paragraphs: NNSA does not thoroughly review budget estimates before it incorporates  them into its annual budget request. Instead, it relies on undocumented,  informal reviews of these estimates by site and headquarters program  office officials and the formal budget validation reviews, which conclude  after the submission of the President’s budget to Congress. Neither of  these processes meets DOE provisions for ensuring the credibility and  reliability of agency budgets, as defined in DOE Order 130.1. According  to senior NNSA officials, NNSA does not comply with DOE Order 130.1  because it believes the order expired in 2003 and therefore no longer  applies to NNSA budget activities. Furthermore, they stated that the need  for a formal review of budget estimates is minimized, in part, because of  the inherent trust between NNSA and its M&O contractors. Additionally,  we identified three key problems in NNSA’s formal budget validation  review process: it occurs too late to affect budget decisions, is not  sufficiently thorough, and includes other weaknesses that limit its  effectiveness.
subsections: section_title: NNSA’s Process for Reviewing Budget Estimates Is Not Thorough or Documented
paragraphs: NNSA does not have a thorough, documented process for assessing the  validity of its budget estimates prior to their inclusion in the President’s  budget submission to Congress, thereby limiting the reliability and  credibility of the budget submission. Specifically, according to NNSA  officials from NNSA’s Offices of Management and Budget, Defense  Programs; Defense Nuclear Nonproliferation; and the site offices for Los  Alamos, Sandia, and Y-12, during the programming phase of PPBE, site  and headquarters program office officials conduct informal,  undocumented reviews of the budget estimates that M&O contractors  submitted to determine their reasonableness, though some officials noted  that the level of review may vary across site and headquarters program  offices. According to these officials, this informal review is often  conducted by comparing current budget estimates with those from  previous years because the work is largely the same from year to year. If  the estimates are similar, and no major programmatic change has taken  place, site office and headquarters program office officials said that they  generally view these budget estimates as reasonable for inclusion in  NNSA’s budget estimate. However, site office officials told us that their  ability to thoroughly review budget estimates is limited. For example,  according to NNSA officials at the Los Alamos Site Office, they do not  have the personnel needed or the time, because of other laboratory  management responsibilities, to oversee the laboratory’s budget  estimation practices. They told us that only one dedicated budget analyst  is employed at the site office and, because of insufficient personnel  resources in the office, a majority of this analyst’s time is spent  conducting work that is not directly related to budget oversight. NNSA  officials from the Y-12 Site Office also told us that they informally review  budget estimates when they initially submit them to headquarters  program offices. However, they also stated that they become more  involved in reviewing budget estimates when the agency is formulating its  final budget submission, and the M&O contractors are asked to develop  multiple iterations of budget estimates based on various hypothetical  funding scenarios. However, these officials also stated that their reviews  are not documented. NNSA officials from Defense Programs’ Office of  Analysis and Evaluation told us that the presence of certified cost  engineers—individuals with professional certification in the field of cost  assessment and program management—at the NNSA site offices could  enhance NNSA’s ability to understand how M&O contractors and  programs develop budget estimates and assess those estimates.
The practices the site and headquarters program offices follow do not  align with the criteria for thoroughness or documentation established in  DOE Order 130.1. Specifically, DOE Order 130.1 states that contractor- developed budget estimates should be thoroughly reviewed and deemed  reasonable prior to their inclusion in agency budgets and that these  reviews should be documented. Senior officials from NNSA’s Office of  Management and Budget told us that the agency does not strictly adhere  to DOE Order 130.1 because it believes that the order has expired and no  longer applies to NNSA budget activities. According to these officials, this  order expired in 2003, and they are unaware of any other DOE or federal  government requirement to conduct budget validation reviews. They  further stated that NNSA is conducting budget validation reviews only  because it considers them to be a good business practice and that NNSA  will work with DOE on updating the order if DOE initiates that process.  NNSA officials stated that, if DOE updated and reissued DOE Order  130.1, it would comply to the extent that it had the resources to do so.  However, DOE Order 130.1 remains listed on DOE’s “All Current  Directives” website, and a senior DOE budget official told us that DOE  Order 130.1 remains an active order. Additionally, this official stated that  a key principle of DOE Order 130.1—federal oversight of contractors’  practices for budget formulation—is appropriate and valid. This official  noted, however, that the order is outdated in terms of the terminology it  uses to describe DOE—it was issued in 1995, predating the 2000  establishment of NNSA—and should be updated to reflect the  department’s current organizational structure. Furthermore, in March  2009, we issued a cost-estimating guide—a compilation of cost- estimating best practices drawn from across industry and government—in  which we reported that validation is considered a best practice to ensure  that cost data are credible and reliable for use in justifying estimates to  agency management. As a result, NNSA’s site and headquarters  program office reviews of budget estimates are neither thorough nor  documented. According to the Principal Deputy Administrator, NNSA  continues to face challenges in moving away from its historical process  for developing budgets based solely on the unreviewed estimates  produced by NNSA M&O contractors and that NNSA’s practices for  understanding its program activity costs are not as sufficient as they need  to be.
In contrast, NNSA’s Office of Naval Reactors’ is jointly staffed and funded  by both NNSA and the Navy and is therefore subject to naval and DOD,  as well as NNSA, standards for reviewing contractor-developed budget  estimates. The Office of Naval Reactors conducts a semiannual  process—known as budget confirmation—to review all contractor- developed budget estimates. This review is conducted and documented  by NNSA technical experts and approved by the Director of the Office of  Naval Reactors; this director manages both NNSA and the Navy’s  activities within the office and has final budgetary decision authority.  Officials in NNSA’s Office of Management and Budget told us that the  Office of Naval Reactors’ process is much more rigorous than that used  by other NNSA program offices we reviewed. Furthermore, NNSA has  exempted the Office of Naval Reactors from NNSA’s formal budget  validation review process because of management’s confidence in the  quality of the office’s budget confirmation process.
Senior officials in NNSA’s Office of Management and Budget told us that  NNSA does not have the financial and personnel resources needed to  conduct budget estimate reviews with the same rigor as the Navy and  DOD. Furthermore, these officials said, the need for a formal review of  M&O contractor-developed budget estimates is minimized within NNSA  because site office officials have historical knowledge of work with  NNSA’s M&O contractors that allows them to assess the reasonableness  of M&O contractor-developed budget estimates without conducting a  formal review and because of the “inherent trust” between NNSA and its  M&O contractors that results from its contracting strategy with them.  Specifically, one of these officials stated that, to a large extent, only the  M&O contractors are in a position to know the scientific and engineering  details of nuclear weapons and the associated work scope and funding  necessary to ensure their safety and reliability. However, for the last 10  years, we have reported that NNSA has significant weaknesses in its  ability to control costs and effectively manage its M&O contractors. We  are therefore concerned that NNSA management continues to deny the  need for NNSA to improve its processes for developing credible and  reliable budget estimates.
section_title: NNSA’s Formal Budget Validation Review Process Occurs Too Late to Affect Budget Decisions and Is Not Sufficiently Thorough
paragraphs: We identified three key problems in NNSA’s annual budget validation  review process—its formal process for assessing M&O contractor- and  program-developed budget estimates. First, NNSA’s annual budget  validation review process occurs too late in the budget cycle to inform  NNSA, DOE, OMB, and congressional budget development or  appropriations decisions. DOE Order 130.1, which is referenced in  NNSA’s policy for its budget validation review process, states that  agencies should thoroughly review budget estimates before using these  estimates to develop budgets. However, NNSA’s Phase II and Phase III  budget validation reviews are scheduled to begin 5 months after the  President submits his budget to Congress. Additionally, during each of  the past four budget validation cycles, NNSA did not complete its budget  validation reports for at least 12 months following the President’s budget  submission to Congress and at least 4 months after the beginning of the  fiscal year for which NNSA reviewed the budget estimates. Therefore,  Congress considered the budget request for NNSA and appropriated  funds to it, and NNSA executed these funds to M&O contractors, before  NNSA had published the results of the budget validation reviews.  Because of their timing, NNSA’s budget validation reviews cannot inform  NNSA’s budget development, DOE or OMB reviews, or Congress’  appropriation processes. According to NNSA policy, the timing of NNSA’s  budget validation review process is designed to inform the NNSA, DOE,  and OMB budgeting processes for the fiscal year following that for which  the budget validation reviews were conducted. However, the timing of the  publication of the budget validation review reports for each of the last 4  years precluded even such delayed consideration because they were  issued following the OMB budget formulation process for the following  fiscal year.
Second, NNSA’s budget validation review process is not sufficiently  thorough to ensure the credibility and reliability of NNSA’s budget. DOE  Order 130.1 states that budgets should be based on budget estimates  that have been thoroughly reviewed by site and headquarters program  offices. However, NNSA’s budget validation review process is limited to  assessing the processes M&O contractors and programs used to develop  budget estimates rather than the accuracy of the resulting budget  estimates. NNSA’s 2010 budget validation review guidance states that  the agency lacks the resources and expertise needed to thoroughly  evaluate the accuracy of budget estimates on its own and therefore relies  on assessments of the reasonableness of the processes used by M&O  contractors to develop budget estimates. NNSA officials from the Los  Alamos and Y-12 Site Offices told us that they believe the budget  validation review process would benefit NNSA more if it more thoroughly  assessed the budgetary processes M&O contractors used to develop  their budget estimates. Furthermore, NNSA policy and budget validation  review guidance stipulate that 20 percent of the agency’s programs  should be reviewed annually to help ensure its validity, but NNSA’s formal  validation process actually results in significantly smaller portion of its  budget being reviewed. For example, in 2011, NNSA’s annual budget  validation guidance identified four programs subject to budget validation  review—the Engineering Campaign, Nuclear Counterterrorism Incident  Response, Global Threat Reduction Initiative, and Fissile Materials  Disposition—each of which is conducted at multiple NNSA sites.  However, NNSA conducted validation reviews at only one site for each of  these programs, which resulted in a formal validation review of  approximately 12, 21, 15, and 4 percent of each of those programs’ total  budgets, respectively, which, together, constituted 1.5 percent of NNSA’s  budget request for fiscal year 2012.
Third, other weaknesses in NNSA’s budget validation review process limit  its effectiveness as a resource to assess the validity of its budget  estimates. In particular, NNSA workgroups that reviewed the 2007 and  2008 budget validation review cycles recommended that NNSA formally  evaluate the status of recommendations made during previous budget  validation reviews. However, NNSA has not incorporated a formal  mechanism for implementing an evaluation into its budget validation  review process. NNSA officials at the Los Alamos and Y-12 site offices  also told us that not having such an evaluative mechanism was a  weakness in NNSA’s budget validation process. Without a formal  mechanism, NNSA is limited in its ability to measure (1) any progress  M&O contractors or programs have made in their processes for  estimating budgets in response to recommendations from previous  budget validation reviews and (2) the effectiveness of NNSA’s budget  validation review process. For example, a 2010 budget validation review  of the Readiness Campaign recommended that the program more  formally document its budget processes, guidance, and estimating  assumptions. Furthermore, a 2009 budget validation review of the  Elimination of Weapons Grade Plutonium Production program found that  the program could not provide documentation of its internal budget  processes. However, in both instances, NNSA did not follow up to  determine if the programs had addressed these concerns during  subsequent budget validation reviews. Additionally, budget validation  reviews do not always include recommendations to improve M&O  contractor or program processes for estimating budgets when they  identify potentially serious weaknesses in those M&O contractor’s or  programs’ ability to develop cost estimates. For example, according to a  2010 budget validation review of budget estimation activities for the  Nonproliferation and Verification Research and Development program at  Sandia National Laboratories, six of the eight projects reviewed lacked  sufficient documentation to support their cost estimates, including two that  lacked any supporting documentation. The report noted the importance  of credible cost estimates, but it did not formally recommend any remedial  improvements and rated the overall processes used to develop those cost  estimates as satisfactory. Additionally, NNSA officials in the Defense  Programs’ Office of Analysis and Evaluation told us that the cost  information used to support budget validation review reports is often  flawed or nonexistent.
section_title: NNSA Has Implemented Some Tools to Support Decision Making on Resource Trade-offs but Has Stopped Using or Developing Other Capabilities
paragraphs: During the programming phase of PPBE, NNSA uses a variety of  management tools, such as integrated priority lists and requirements and  resources assessments, to support its programming phase and assist  senior managers in making decisions on resource trade-offs. However, it  has stopped using these capabilities or developing others.
subsections: section_title: NNSA Has Developed and Implemented Some Tools to Assist Management in Deciding on Resource Trade-offs
paragraphs: NNSA uses the following management tools to decide on resource trade- offs during the programming phase of its PPBE process:  Integrated priority lists. NNSA’s policy for the programming phase  stipulates that each of NNSA’s nine program offices is to annually  develop an integrated priority list that ranks program activities according  to their importance for meeting mission requirements. These lists provide  senior NNSA and DOE managers with an understanding of how various  funding scenarios would affect program activities. Specifically, these lists  rank the priority of program activities that are within anticipated  appropriation levels—which are of the highest priority—as well as those  that NNSA would fund if the appropriation levels were sufficiently high to  do so. For example, the program activity listed last on an integrated  priority list would be the first to forgo funding if appropriation levels are  lower than anticipated. Conversely, these lists define program activities— unfunded requirements—that would be funded if appropriation levels are  higher than anticipated. NNSA instructions for the programming phase  stipulate that the agency is to combine the nine program office-developed  integrated priority lists into four that correspond to the four congressional  appropriations NNSA receives. Three of the integrated priority lists— those of the Offices of the Administrator, Defense Nuclear  Nonproliferation, and Naval Reactors—correspond directly to specific  appropriations,represent activities funded by the Weapons Activities appropriation into a   but NNSA does not combine the six others that  single integrated priority list. NNSA officials stated that these six others  represent separate and distinct mission areas and only the Administrator  can decide on the resource trade-offs among them. Of the six program  offices funded by the Weapons Activities appropriation, Defense  Programs accounts for a large majority—approximately 82 percent—of  the funding requested in NNSA’s fiscal year 2013 budget submission to  Congress. According to officials in NNSA’s Office of Management and  Budget, the Administrator is responsible for deciding on how to allocate  resources across program offices. However, these decisions are not  documented or reflected in a single integrated priority list for program  activities funded by the Weapons Activities appropriation. By not  combining these lists, NNSA is limiting the formal documentation  available to inform DOE about how various Weapons Activities  appropriation funding scenarios would affect the program activities  conducted by these six program offices. Specifically, because these six  lists are not integrated, it is unclear which program activities would be  affected by changes to appropriation levels or which programs across the  six lists are of the highest priority.
Requirements and Resources Assessments. During the 2010 and 2011  programming cycles, NNSA used its Requirements and Resources  Assessment process to independently assess the need to conduct new  program activities and unfunded requirements. According to the NNSA  handbook for this process, officials in NNSA’s Office of Management and  Budget review program offices’ budget submissions for activities that are  either above anticipated funding levels or are for new activities. For these  assessments, officials are to analyze specific factors related to these  activities, such as their need for meeting agency priorities and the  reasonableness of the assumptions used to produce their budget  estimates. The objective of this process is to ensure that new program  activities and unfunded requirements are needed to meet NNSA priorities.  For example, according to officials in NNSA’s Office of Management and  Budget, the use of the Requirements and Resources Assessment  process was a contributing factor in reducing the amount of unfunded  program activities included in NNSA’s budget from approximately $1  billion for fiscal year 2012 to approximately $80 million for fiscal year  2013. Furthermore, draft NNSA guidance states that the process has  identified inconsistencies in the quality of estimates and the level of  insight and understanding program managers have regarding the fidelity  of the estimates supporting their budgets. According to officials in NNSA’s  Office of Management and Budget, this process is a simple and effective  tool for providing management with additional information on the need to  conduct proposed new program activities or unfunded requirements.
However, these officials also stated that this process is time-consuming  and would not be practical or efficient to apply to the entirety of NNSA  program activities because it was designed to assess program  components rather than entire programs; they added that other types of  program reviews or validations would be better suited for conducting  program needs analysis on an enterprise-wide basis. Additionally,  because the NNSA Office of Integration and Assessments, which was  responsible for conducting these assessments, was dissolved in 2010,  officials in NNSA’s Office of Management and Budget told us that they  may discontinue the use of Requirements and Resources Assessment  process in future programming cycles. Furthermore, in the current austere  budget environment, they do not foresee any programs proposing  activities that are either new or above anticipated funding levels.  Therefore, the continued use of this process in future programming cycles  is uncertain. However, we believe that NNSA has demonstrated this  process can be an important tool for assessing the necessity to fund  certain activities in order to meet its mission requirements.
Enterprise Portfolio Analysis Tool. NNSA’s Office of Defense Programs is  implementing a data system—the Enterprise Portfolio Analysis Tool— designed to provide a consistent framework for managing the PPBE  process within Defense Programs, which accounts for 54 percent, or $6.2  billion, of the President’s $11.5 billion fiscal year 2013 budget request for  NNSA. As we testified in February 2012, a tool such as this could help  NNSA obtain the basic data it needs to make informed management  decisions, determine return on investment, and identify opportunities for   Currently, this tool includes a mechanism to identify when  cost saving.decisions on resource trade-offs must be made if, for example, M&O  contractor-developed budget estimates for program requirements exceed  the budget targets NNSA provided for those programs. Additionally, the  tool is to incorporate Defense Programs’ common work activity  structure—known as its work breakdown structure—to facilitate an  analysis of consistent budget data from across the NNSA enterprise.  Specifically, the tool may allow Defense Programs managers to compare  the budget estimates for analogous activities across the nuclear security  enterprise regardless of which M&O contractor or program is conducting  them. Furthermore, Defense Programs officials stated that they eventually  plan to use this tool to compare budget estimates of program activities  with the amounts the programs ultimately expended, but they said that  the introduction of this capability is not imminent. According to Defense  Programs and M&O contractor officials, the implementation of this tool is  placing an additional labor burden on NNSA M&O contractors because of  the quantity of historical budget data that need to be entered into it.  However, according to Defense Programs officials, once these initial  historical data are entered, the M&O contractors will need to annually  update the system with the most recent year’s data.
section_title: NNSA Does Not Use Capabilities That Were Previously Implemented or Partially Developed to Support Its Programming Activities
paragraphs: NNSA no longer has an independent analytical capability to perform such  functions as reviewing proposals for program activities and verifying cost  estimates. In addition, since 2009, the NNSA Administrator has not  formally documented his decisions on resource trade-offs at the close of  the programming phase in the AFR. Furthermore, NNSA has not  completed cost estimating guidance to assist NNSA program managers in  identifying reliable M&O contractor practices for estimating costs for  operations and sustainment activities. By not using these capabilities,  NNSA has reduced its ability to decide on resource trade-offs because it  has not enhanced, made formal, or implemented capabilities that it had  already, to varying degrees, developed or used.
The DOE Inspector General and GAO, recommended in 2003 and 2007,  respectively, that NNSA establish an independent analysis unit to perform  such functions as reviewing proposals for program activities and verifying  cost estimates. NNSA agreed with these recommendations and, in  2009, instituted the Office of Integration and Assessments to identify,  analyze, assess, and present to senior NNSA management options for  managing its programs and making decisions on resource trade-offs. The  specific responsibilities of this office included analyzing program  performance, evaluating programming and funding alternatives, and  assessing the implementation and effectiveness of process improvement  initiatives. Furthermore, this office managed the Requirements and  Resources Assessment process during the 2010 programming cycle.  However, NNSA disbanded the office in 2010, 18 months after it was  formally created. NNSA officials also told us that it was never properly  staffed, which thereby limited its effectiveness.
In the memorandum establishing the Office of Integration and  Assessments, NNSA stated that it expected the office, in conjunction with  DOE’s Office of Cost Analysis, to provide DOD-like analytical resources   Since then, however, DOE has also eliminated its Office  across NNSA.of Cost Analysis. With both of these offices now gone, neither NNSA nor  DOE has independent cost assessment or program evaluation  capabilities analogous to those of DOD. In contrast, because NNSA’s  Office of Naval Reactors is organized as a separate entity within NNSA  reporting both to NNSA and the U.S. Navy, unlike the rest of NNSA, it is  subject to the Navy’s independent analytical resources, such as  assessments by the Naval Sea Systems Command’s Office of Cost  Engineering and Industrial Analysis and the Naval Center for Cost  Analysis, which conduct independent reviews and analyses of program  cost estimates. Furthermore, DOD has an Office of Cost Assessment and  Program Evaluation, which has a similar function but with a purview that  extends across DOD, including the Navy. These layers of independent  review approximate NNSA’s vision for independent analysis, as described  in the memorandum establishing the Office of Integration and  Assessments. Following the dissolution of the Office of Integration and  Assessments in 2010, NNSA’s Office of Defense Programs created the  Office of Analysis and Evaluation to conduct similar program review  functions. However, the capabilities of this office are limited by several  factors. For example, because the office is positioned within Defense  Programs, it does not have purview to conduct analysis on any of NNSA’s  other programs, which, in total, constitute nearly half of the agency’s  budget request for fiscal year 2013. Additionally, according to Defense  Programs officials, this office does not have the capability to self-initiate  reviews of programs but rather is instructed by Defense Programs’  management on what activities to assess, thereby limiting the office’s  independence. Furthermore, NNSA officials from this office stated that  properly staffing the office remains a challenge because many qualified  individuals left DOE and NNSA when they eliminated the Offices of Cost  Analysis and Integration and Assessments, respectively.
Even though NNSA has had difficulty in maintaining an agencywide  independent analytical capability, NNSA’s Principal Deputy Administrator  told us that NNSA remains supportive of the concept of an independent  analytical unit to conduct assessments of programs agencywide.  However, senior NNSA officials told us that creating and developing the  capabilities of such an office would be difficult in the current budget  environment and that therefore NNSA has no current plans to institute  such a capability.
The NNSA Administrator has not formally documented his decisions on  resource trade-offs at the close of the programming phase in the AFR  since 2009, which is inconsistent with NNSA policy and instructions.  When issued, this document articulated the Administrator’s rationale and  methodology for deciding on resource trade-offs during the programming  phase of the PPBE process—which one senior official in NNSA’s Office of  Management and Budget described as an important component of the  PPBE process—to support in his budget proposal to DOE and to better  facilitate NNSA’s participation in DOE’s Strategic Resources Review.  According to senior NNSA officials, the Administrator considered the AFR  to be a useful management tool but decided to discontinue issuing it  because of concerns that its contents, which are predecisional Executive  Branch deliberative material and embargoed from public release by OMB  Circular A-11, could be leaked and thereby reduce the flexibility of DOE  and OMB in making final decisions regarding the President’s Budget.  Instead of the AFR, the Administrator now develops an internal document  called “Administrator’s Preliminary Decisions,” which is not required in  NNSA policy, guidance, or instructions; contains more generalized  information; and does not have the rationales, methodologies, and  justifications for decision making on resource trade-offs that were  previously incorporated into the AFR.
NNSA developed a draft guide—the Program Managers’ Guide to  Understanding and Reviewing Cost Estimates for Operations and  Sustainment Activities—in 2010 to assist NNSA program managers in  identifying reliable M&O contractor practices for estimating costs for  operations and sustainment activities—activities not related to  construction; according to this guide, these activities constitute  approximately 80 percent of NNSA’s annual budget. This guide was also  created to supplement the information provided in NNSA’s Business  Operating Procedure 50.005, Establishment of an Independent Cost  Estimate Policy and interim Cost Estimating Guide 50.005, which  identifies best practices for preparing cost estimates.Program Managers’ Guide to Understanding and Reviewing Cost  Estimates for Operations and Sustainment Activities was largely  completed but never finalized before NNSA dissolved the Office of  Integration and Assessments, which had drafted the guide, and NNSA  officials said the agency has no plans to complete or issue it. According to  officials in NNSA’s Office of Management and Budget, NNSA drafted this  guide because it recognized that supplemental information focused on  operations and sustainment activities cost estimates—the development of  which, according to this guide, are not governed by any specific NNSA  guidance or processes—could enhance the tools available to program  managers in evaluating cost estimates and how they are translated into  budget estimates. The objective of the guide was to provide an instructive  document to facilitate program managers’ ability to understand what  constitutes a rigorous process for ensuring quality operations and  sustainment cost estimates on an ongoing basis and evaluate the  reasonableness those estimates. This guide also defined key  components of cost estimating to clarify the responsibilities and  expectations of NNSA program managers and included instructions for  how NNSA program managers can assess the quality of budget  estimates submitted by M&O contractors. NNSA officials with Defense  Programs’ Office of Analysis and Evaluation told us that additional  guidance on how to assess the costs of operations and sustainment  activities could enhance program managers’ ability to assess the  reliability and credibility of cost estimates.
section_title: Conclusions
paragraphs: NNSA has established a formal four-phase PPBE process that uses  short- and long-term planning to define program priorities and match them  to available budgetary resources. However, DOE and NNSA have not  taken adequate steps to make this process as effective and efficient as  possible. In particular, DOE Order 130.1, which defines DOE’s provisions  for budget activities, references outdated terminology and organizations  that no longer exist within the department, leading to confusion regarding  the order’s applicability and requirement for implementation. As a result,  NNSA believes that the order has expired and that it is not required to  adhere to its provisions. By not adhering to these provisions, NNSA is  reducing the credibility of its budget proposals. Moreover, NNSA’s  process for developing budget estimates continues to rely heavily on its  M&O contractors to develop budget estimates without an effective,  thorough review of the validity of those estimates. Without thorough  reviews by site and headquarters program offices of budget estimates,  NNSA cannot have a high level of confidence in its budget estimates or in  its ability to make informed decisions on resource trade-offs and to  enhance the credibility and reliability of its budget. Furthermore, NNSA’s  formal budget validation review process does not sufficiently ensure the  credibility and reliability of NNSA’s budget, primarily because of  deficiencies in the timing of these reviews. Also, without a formal  mechanism to evaluate the status of recommendations made during  previous budget validation reviews, NNSA is limited in its ability to  measure any progress M&O contractors or programs have made to their  budget estimating processes.
NNSA has reduced its ability to decide on resource trade-offs because it  has not enhanced, made formal, or implemented capabilities that it has  already, to varying degrees, developed or used. In particular, NNSA does  not follow its instructions for preparing an integrated priority list for each  congressional appropriation, as it does not combine the six priority lists  that represent activities funded by the Weapons Activities appropriation  into a single integrated list. By not combining these lists into a single  integrated priority list, NNSA is limiting the formal documentation  available to inform DOE which program activities would be affected by  changes to this appropriation. Moreover, NNSA instituted and then  disbanded an independent analytic capability that would provide it with an  independent cost assessment or evaluation capabilities of the  reasonableness and affordability of various programs and projects  proposed by NNSA offices. By disbanding its independent analytical  capability, NNSA is losing its ability to improve its cost-estimating  capabilities and better ensure that its project cost estimates are credible  and reliable. Because of the fiscal constraints in the current budget  environment, it is all the more critical that NNSA have the capability to  conduct independent cost analyses to enhance its ability to make the  most effective and efficient resource decisions on resource trade-offs.  Despite previous recommendations that DOE’s Inspector General made  in 2003, and that we made in 2007, to institute an independent analytical  capability to assess programs throughout all of NNSA, NNSA continues to  lack such a function. Not having this capability could preclude NNSA from  making the best decisions about what activities to fund and whether they  are affordable. In addition, NNSA may cease using its Requirements and  Resources Assessment process—which is intended to provide some  independent analysis of new program activities and unfunded  requirements—in future PPBE budget cycles because it does not  anticipate program proposals for new activities or unfunded requirements.  By not retaining this process, NNSA would lose an important tool for  assessing the necessity to fund certain activities in order to meet its  mission requirements.
Furthermore, NNSA no longer follows its policy to issue the AFR. Without  a formal document, NNSA and DOE have no formal record of the  Administrator’s rationale and methodology for deciding on resource trade- offs during the programming phase of the PPBE process. We recognize  that NNSA needs to hold confidential, internal budgetary and resource  trade-off deliberations; however, we do not believe that this need  supersedes NNSA policy or the benefits provided by documented  decision making during programming, which one senior NNSA official  described to us an important component in NNSA’s PPBE process. Not  issuing the AFR (or some similarly precise documentation) places the  Administrator in conflict with official NNSA policy and with an important  PPBE precept—the importance of transparency.
Finally, NNSA developed draft guidance in 2010 to assist NNSA program  managers in identifying reliable M&O contractor practices for estimating  costs for operations and sustainment activities. Such guidance would  better equip NNSA program managers to more accurately evaluate the  reasonableness of cost estimates, but this guidance is in draft form and  NNSA has no plans to complete and issue it. Without such guidance,  NNSA program managers are limited in their ability to assess the  reliability and credibility of budget estimates.
section_title: Recommendations for Executive Action
paragraphs: To enhance NNSA’s ability to better ensure the validity of its budget  submissions, and to decide on resource trade-offs, we recommend that  the Secretary of Energy take the following seven actions:
Direct the DOE Office of Budget to formally evaluate DOE Order  130.1 and revise as necessary, and communicate any revisions to the  NNSA Administrator so that the agency will have updated provisions  for assessing the quality of its budget estimates.
Direct the Administrator of NNSA to:
Develop a formal process, or amend its budget validation review  process, to ensure that all budget estimates are thoroughly  reviewed by site and headquarters program offices, and that these  reviews are timed to inform NNSA, DOE, OMB, and congressional  budget decisions.
Once this process is developed, incorporate a formal mechanism  to evaluate the status of recommendations made during previous  budget validation reviews so that NNSA can measure M&O  contractors’ and programs’ progress in responding to deficiencies  with their budget estimates.
Combine the integrated priorities lists for each of the program  offices funded within the Weapons Activities appropriation into a  single integrated priorities list, as stipulated in NNSA instructions,  to better inform DOE which program activities would be affected  by changes to this appropriation.
Reinstitute an independent analytical capability to provide senior  decision makers with independent program reviews, including an  analysis of different options for deciding on resource trade-offs,  and facilitate NNSA making the best decisions about what  activities to fund and whether they are affordable. As part of this  capability, formally retain the Requirements and Resources  Assessment process to review proposed new activities and  unfunded requirements.
Reinstitute the issuance of the Administrator’s Final  Recommendations to document the Administrator’s rationale and  methodology for deciding on resource trade-offs to support in his  budget proposal to DOE and to better facilitate NNSA’s  participation in DOE’s budget process.
Complete and formally issue the Program Managers’ Guide to  Understanding and Reviewing Cost Estimates for Operations and  Sustainment Activities so that program managers will be better  equipped to evaluate the reasonableness of cost estimates.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided DOE with a draft of this report for its review and comment. In  its written comments, NNSA, responding on behalf of DOE, provided  observations on the report’s findings and stated that it generally agreed in  principle with six of our seven recommendations and did not concur with  one. NNSA did not concur with our recommendation to combine the  integrated priorities lists for all program offices funded by the Weapons  Activities appropriation into a single integrated priorities list, as is  stipulated by NNSA instructions for the programming phase of PPBE.
NNSA agrees that the integrated priorities lists are a useful tool to  facilitate NNSA and DOE decision-making. However, NNSA states that it  believes reaching management consensus on a single integrated  priorities list for these program offices would be a difficult, time consuming  process and that its current approach for deciding on resource trade-offs  is effective and efficient. We acknowledge that NNSA uses a variety of  tools in addition to integrated priorities lists to conduct programming  activities, but we continue to believe that combining the integrated  priorities lists for all program offices funded by the Weapons Activities  appropriation could enhance the agency’s ability to support its decisions  on resource trade-offs for DOE consideration during the Strategic  Resources Review. However, NNSA stated in its comments that it would  consider the development of more robust integrated priority lists if  circumstances require changes to its current approach.
NNSA further acknowledged that aspects of its PPBE process could be  improved but disagreed with our report’s characterization of its budget  estimate review processes as not being thorough. NNSA commented that  it believes that our conclusions overemphasize some procedural areas for  potential improvement, without accurately considering the cumulative  effectiveness of NNSA’s PPBE process as a whole. We continue to  believe that the agency’s processes for reviewing budget estimates are  not sufficiently thorough to ensure the credibility and reliability of those  estimates and do not meet the provisions defined in DOE Order 130.1.  Specifically, the reviews conducted by site and headquarters program  office officials are informal and undocumented, and NNSA’s budget  validation review process—the agency’s formal process for assessing  M&O contractor- and program-developed budget estimates—does not  assess the accuracy of budget estimates and is conducted for a small  portion of the agency’s annual budget. NNSA’s letter is reproduced in  appendix II.
NNSA also provided technical comments, which we incorporated  throughout the report as appropriate.
We are sending this report to the Secretary of Energy, the Administrator  of NNSA, the appropriate congressional committees, and other interested  parties. In addition, the report is available at no charge on the GAO  website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Office  of Congressional Relations and Public Affairs may be found on the   last page of this report. Key contributors to this report are listed in  appendix III.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to determine (1) the current structure of  the National Nuclear Security Administration’s (NNSA) planning,  programming, budgeting, and evaluation (PPBE) process; (2) the extent  to which NNSA reviews its budget estimates; and (3) how NNSA decides  on resource trade-offs in its PPBE process.
To determine the current structure of NNSA’s PPBE process, we  reviewed the NNSA policies and other headquarters-developed  instructions and guidance documents that define how the process is  designed to function. We also reviewed program-specific PPBE  documentation from the Offices of Defense Programs, Defense Nuclear  Nonproliferation, and Naval Reactors; these three offices correspond to  NNSA’s primary missions and collectively account for approximately 85  percent of the President’s fiscal year 2013 NNSA budget submission to  Congress. We also interviewed officials from NNSA’s Office of  Management and Budget, which is responsible for managing NNSA’s  PPBE process, as well as the offices of Defense Programs, Defense  Nuclear Nonproliferation, and Naval Reactors to discuss how NNSA’s  PPBE process is designed to function.
To determine the extent to which NNSA reviews its budget estimates, we  reviewed DOE Order 130.1 and NNSA policies, instructions, and  guidance that define how such reviews are to be conducted. We also  analyzed documentation of the formal budget validation reviews  conducted by NNSA for the last five review cycles, as well as the results  of two NNSA workgroups that evaluated the budget validation review  process. Furthermore, we interviewed officials involved in the  development, oversight, or execution of NNSA budget estimate reviews  from the NNSA Offices of the Administrator, Management and Budget,  Defense Programs, Defense Nuclear Nonproliferation, Naval Reactors,  and Acquisition and Project Management; the site offices for Los Alamos,  Sandia, and the Y-12 National Security Complex, and the Naval Reactors  Laboratory Field Office; DOE officials from the Office of Budget; and M&O  contractor officials from Los Alamos and Sandia National Laboratories,  the Y-12 National Security Complex, and Bettis Atomic Power  Laboratory. Because NNSA’s Office of Naval Reactors is organized as a  separate entity within NNSA reporting both to NNSA and the U.S. Navy,  we also met with Navy officials from its Offices of Financial Management  and Budgeting, and Cost Engineering and Industrial Analysis.
To determine how NNSA decides on resource trade-offs, we reviewed  NNSA policies, instructions, and guidance for its programming process.  Based on these documents, we identified the tools that NNSA uses, or  has used, to assist NNSA management in deciding on resource trade-offs  and reviewed documentation of how these tools were applied by program  offices and NNSA management during the programming phases of  previous PPBE cycles. We also interviewed officials from the NNSA  Offices of the Administrator, Management and Budget, Defense  Programs, Defense Nuclear Nonproliferation, Naval Reactors to discuss  how they decide on, and document, resource trade-offs.
We conducted this performance audit from July 2011 to July 2012 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Comments from the U.S. Department of Energy
section_title: Appendix III: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Dan Feehan (Assistant Director),  Robert Baney, Matthew Tabbert, and Eugene Wisnoski made significant  contributions to this report. Cheryl Peterson, Jeremy Sebest, Rebecca  Shea, Carol Herrnstadt Shulman, and Kiki Theodoropoulos provided  technical assistance.