Modernizing the Nuclear Security Enterprise: NNSA's Reviews of Budget Estimates and Decisions on Resource Trade-offs Need Strengthening
Released: 2012-07-31T00:00:00

Letter

Background

NNSA oversees programs to carry out its missions for nuclear weapons, nonproliferation, and naval nuclear propulsion, for which the President’s budget submission to Congress requested more than $11.5 billion for fiscal year 2013—about 42 percent of DOE’s total budget. NNSA has primary mission responsibilities in three areas. First, it is responsible for providing the United States with safe, secure, and reliable nuclear weapons in the absence of underground nuclear testing and maintains core competencies in nuclear weapons science, technology, and engineering. Second, NNSA implements key U.S. government nuclear security, nonproliferation, and arms control activities, including securing vulnerable nuclear and radiological material at facilities throughout the world; removing plutonium and highly enriched uranium from partner countries; eliminating U.S. nuclear material declared surplus to defense needs; negotiating and providing the technical capability to verify arms control treaties and agreements; strengthening other countries’ capacities to implement nonproliferation obligations; and enhancing other nations’ capabilities to deter and detect illicit movement of nuclear and radiological materials. Third, NNSA provides the research, development, design, and operational support for militarily effective naval nuclear propulsion plants, as well as enriched uranium for fabrication into fuel for the Navy’s propulsion reactors. NNSA receives four congressional appropriations to fund its activities, three of which align with its primary missions— Weapons Activities (for Defense Programs), Defense Nuclear Nonproliferation, Naval Reactors—and one that funds its management activities—Office of the Administrator.

Since the Manhattan Project produced the first atomic bomb during World War II, NNSA, DOE, and predecessor agencies have depended on the expertise of private firms, universities, and others to carry out research and development work and efficiently operate the government-owned, contractor-operated facilities necessary for the nation’s nuclear defense. NNSA conducts its activities at research and development laboratories, production plants, and other facilities (collectively referred to as the nuclear security enterprise). Specifically, NNSA operates three national laboratories that design and ensure the reliability of nuclear weapons— Lawrence Livermore National Laboratory, California; Los Alamos National Laboratory, New Mexico; and the Sandia National Laboratories, New Mexico and California; and four nuclear weapons production sites—the Pantex Plant, Texas; the Y-12 National Security Complex, Tennessee; the Kansas City Plant, Missouri; and the Savannah River Site, South Carolina; as well as the Nevada National Security Site.

NNSA’s relationship with its contractors has been formally established over the years through its M&O contracts—contracting strategies that give these contractors responsibility to carry out major portions of NNSA’s missions and apply their scientific, technical, and management expertise. M&O contractors at NNSA sites operate under NNSA’s direction and oversight but largely independently of one another. Various headquarters organizations within NNSA develop policies and NNSA site offices, colocated with NNSA’s sites, conduct day-to-day oversight of the M&O contractors, and evaluate the M&O contractors’ performance in carrying out the sites’ missions.

NNSA Has Established a Formal PPBE Process That Includes Four Defined Phases

According to NNSA’s policy, the NNSA PPBE process is composed of four phases—planning, programming, budgeting, and evaluation—and their associated activities. The different phases of PPBE appear sequential, but because of the amount of time required to develop and review resource requirements, the process is continuous and concurrent with at least two phases ongoing at any given time, including phases for different fiscal years. Figure 1 shows the four phases and the months during which each phase is scheduled to occur, according to NNSA policies and guidance.

Planning. According to NNSA policy, in this phase, scheduled to begin annually in November, NNSA is to identify the goals it needs to achieve over the next 5 years and the program activities needed to meet these goals. According to NNSA officials, these goals are defined in a variety of documents, including presidential directives, policy statements, and DOE and NNSA strategic plans. This phase begins with the issuance of NNSA’s annual Strategic Planning Guidance, which provides any updates to the strategic plans and identifies any emerging issues. The NNSA program offices use this guidance to conduct their own internal planning processes, update their multiyear plans, including revising or adding program activities needed to meet the agency’s goals.

Programming. According to NNSA policy, in this phase, scheduled to begin annually in February, NNSA is to determine which program activities and funding levels it will include in its budget proposal to DOE for the fiscal year beginning in October of the following calendar year. This determination is based on analysis of the activities’ estimated costs, as well as the need to meet the NNSA goals defined in the planning process. To determine these activities, NNSA program offices are to work with their contractors to obtain estimates for the cost of the program activities identified in the planning phase and determine how to accomplish these activities within anticipated funding levels, which are defined in annual NNSA Program and Fiscal Guidance. NNSA program offices are to then rank these activities in order of priority for meeting program goals and document these decisions in integrated priority lists. These lists can include proposed program activities above the anticipated funding levels specified in NNSA guidance—these proposed activities are known as unfunded requirements. Using these lists, as well as other briefing materials, a group of senior NNSA officials including the heads of all program offices—the Program Review Council—then is to meet with the Principal Deputy Administrator to discuss and defend each program’s proposed program activities. After reviewing the deliberations of the Program Review Council and the associated documents provided by the program offices, the NNSA Administrator is to decide on resources trade- offs that result in the combination of program activities that best meet NNSA’s goals over the 5-year period covered by the Future Years Nuclear Security Program plan. The Administrator is responsible for issuing the Administrator’s Final Recommendations (AFR), scheduled to be completed in May at the conclusion of the programming phase, to document NNSA’s justification for its priorities and to serve as the basis for the agency’s participation in DOE’s program review process, the Strategic Resources Review.

Budgeting. According to NNSA policy, this phase is to integrate NNSA planning and programming priorities and budget estimates into DOE’s departmental budget process and consists of the following three parts:

Budget formulation. During formulation, which is scheduled to begin annually in July for the fiscal year beginning in October of the following calendar year, NNSA submits its proposed budget to DOE and participates in the Strategic Resources Review. If DOE’s budget deliberations result in changes to NNSA’s proposed budget, NNSA may have to rebalance its work activities. In September each year, DOE submits its proposed budget to the Office of Management and Budget (OMB) for review. Depending on OMB revisions, NNSA may need to again revise its work activities. These revisions are incorporated into the President’s final budget request for DOE, which is submitted to Congress in February.

Budget validation. According to NNSA guidance, the agency uses its budget validation review process to ensure its budget request is consistent with NNSA priorities and its budget estimating processes are reasonable. NNSA policy calls for NNSA’s Office of PPBE to manage a three-phase process of validating approximately 20 percent of NNSA’s programs each year, so that 100 percent of its budget is validated every 5 years. Programs to undergo validations are to be determined by a combination of factors, including Program Managers’ request, Administrator direction, and/or significant external interest/high program visibility. During Phase I of the process, scheduled for completion in June, before the beginning of the fiscal year in October, program officials determine if their activities conform with strategic guidance and program plans and review their methods for formulating budgets. In Phase II, scheduled annually for July to September, NNSA contractors or program offices, whichever developed the budget estimates, conduct a self-assessment of their budget planning, formulation, and cost-estimating processes. Phase II self-assessments are to be reviewed by a team—known as a validation review team—that comprises NNSA headquarters and site office staff. During Phase III, scheduled to occur from July through August, these validation review teams also review the cost-estimating practices used by the NNSA contractors and program offices. Importantly, NNSA’s validation guidance emphasizes that reviews should focus on the processes used to formulate budget plans and derive budget estimates rather than on the accuracy of the resulting estimates. According to this guidance, validation review teams are to issue a report on their findings on Phases II and III in September to inform NNSA, DOE, and OMB decisions for the following year’s budget cycle.

Budget execution. According to NNSA policy, during this process, DOE and NNSA are to allocate, distribute, and control funds to achieve the priorities established in the programming phase, and to maintain the fiscal limits set up in the budgeting phase, which are subject to appropriation of funds by Congress. The execution coincides with the fiscal year and commences once appropriated funds become available—whether by appropriation or Continuing Resolution—at the beginning of the fiscal year every October.

Evaluation. According to NNSA policy, NNSA is to employ an ongoing cycle of evaluations to review program performance. Evaluations are to include annual and quarterly NNSA performance reviews, performance reviews conducted as part of the Government Performance and Results Act, reviews conducted by OMB, and DOE oversight activities.

NNSA Does Not Thoroughly Review Budget Estimates When Developing Its Annual Budget

NNSA does not thoroughly review budget estimates before it incorporates them into its annual budget request. Instead, it relies on undocumented, informal reviews of these estimates by site and headquarters program office officials and the formal budget validation reviews, which conclude after the submission of the President’s budget to Congress. Neither of these processes meets DOE provisions for ensuring the credibility and reliability of agency budgets, as defined in DOE Order 130.1. According to senior NNSA officials, NNSA does not comply with DOE Order 130.1 because it believes the order expired in 2003 and therefore no longer applies to NNSA budget activities. Furthermore, they stated that the need for a formal review of budget estimates is minimized, in part, because of the inherent trust between NNSA and its M&O contractors. Additionally, we identified three key problems in NNSA’s formal budget validation review process: it occurs too late to affect budget decisions, is not sufficiently thorough, and includes other weaknesses that limit its effectiveness.

NNSA’s Process for Reviewing Budget Estimates Is Not Thorough or Documented

NNSA does not have a thorough, documented process for assessing the validity of its budget estimates prior to their inclusion in the President’s budget submission to Congress, thereby limiting the reliability and credibility of the budget submission. Specifically, according to NNSA officials from NNSA’s Offices of Management and Budget, Defense Programs; Defense Nuclear Nonproliferation; and the site offices for Los Alamos, Sandia, and Y-12, during the programming phase of PPBE, site and headquarters program office officials conduct informal, undocumented reviews of the budget estimates that M&O contractors submitted to determine their reasonableness, though some officials noted that the level of review may vary across site and headquarters program offices. According to these officials, this informal review is often conducted by comparing current budget estimates with those from previous years because the work is largely the same from year to year. If the estimates are similar, and no major programmatic change has taken place, site office and headquarters program office officials said that they generally view these budget estimates as reasonable for inclusion in NNSA’s budget estimate. However, site office officials told us that their ability to thoroughly review budget estimates is limited. For example, according to NNSA officials at the Los Alamos Site Office, they do not have the personnel needed or the time, because of other laboratory management responsibilities, to oversee the laboratory’s budget estimation practices. They told us that only one dedicated budget analyst is employed at the site office and, because of insufficient personnel resources in the office, a majority of this analyst’s time is spent conducting work that is not directly related to budget oversight. NNSA officials from the Y-12 Site Office also told us that they informally review budget estimates when they initially submit them to headquarters program offices. However, they also stated that they become more involved in reviewing budget estimates when the agency is formulating its final budget submission, and the M&O contractors are asked to develop multiple iterations of budget estimates based on various hypothetical funding scenarios. However, these officials also stated that their reviews are not documented. NNSA officials from Defense Programs’ Office of Analysis and Evaluation told us that the presence of certified cost engineers—individuals with professional certification in the field of cost assessment and program management—at the NNSA site offices could enhance NNSA’s ability to understand how M&O contractors and programs develop budget estimates and assess those estimates.

The practices the site and headquarters program offices follow do not align with the criteria for thoroughness or documentation established in DOE Order 130.1. Specifically, DOE Order 130.1 states that contractor- developed budget estimates should be thoroughly reviewed and deemed reasonable prior to their inclusion in agency budgets and that these reviews should be documented. Senior officials from NNSA’s Office of Management and Budget told us that the agency does not strictly adhere to DOE Order 130.1 because it believes that the order has expired and no longer applies to NNSA budget activities. According to these officials, this order expired in 2003, and they are unaware of any other DOE or federal government requirement to conduct budget validation reviews. They further stated that NNSA is conducting budget validation reviews only because it considers them to be a good business practice and that NNSA will work with DOE on updating the order if DOE initiates that process. NNSA officials stated that, if DOE updated and reissued DOE Order 130.1, it would comply to the extent that it had the resources to do so. However, DOE Order 130.1 remains listed on DOE’s “All Current Directives” website, and a senior DOE budget official told us that DOE Order 130.1 remains an active order. Additionally, this official stated that a key principle of DOE Order 130.1—federal oversight of contractors’ practices for budget formulation—is appropriate and valid. This official noted, however, that the order is outdated in terms of the terminology it uses to describe DOE—it was issued in 1995, predating the 2000 establishment of NNSA—and should be updated to reflect the department’s current organizational structure. Furthermore, in March 2009, we issued a cost-estimating guide—a compilation of cost- estimating best practices drawn from across industry and government—in which we reported that validation is considered a best practice to ensure that cost data are credible and reliable for use in justifying estimates to agency management. As a result, NNSA’s site and headquarters program office reviews of budget estimates are neither thorough nor documented. According to the Principal Deputy Administrator, NNSA continues to face challenges in moving away from its historical process for developing budgets based solely on the unreviewed estimates produced by NNSA M&O contractors and that NNSA’s practices for understanding its program activity costs are not as sufficient as they need to be.

In contrast, NNSA’s Office of Naval Reactors’ is jointly staffed and funded by both NNSA and the Navy and is therefore subject to naval and DOD, as well as NNSA, standards for reviewing contractor-developed budget estimates. The Office of Naval Reactors conducts a semiannual process—known as budget confirmation—to review all contractor- developed budget estimates. This review is conducted and documented by NNSA technical experts and approved by the Director of the Office of Naval Reactors; this director manages both NNSA and the Navy’s activities within the office and has final budgetary decision authority. Officials in NNSA’s Office of Management and Budget told us that the Office of Naval Reactors’ process is much more rigorous than that used by other NNSA program offices we reviewed. Furthermore, NNSA has exempted the Office of Naval Reactors from NNSA’s formal budget validation review process because of management’s confidence in the quality of the office’s budget confirmation process.

Senior officials in NNSA’s Office of Management and Budget told us that NNSA does not have the financial and personnel resources needed to conduct budget estimate reviews with the same rigor as the Navy and DOD. Furthermore, these officials said, the need for a formal review of M&O contractor-developed budget estimates is minimized within NNSA because site office officials have historical knowledge of work with NNSA’s M&O contractors that allows them to assess the reasonableness of M&O contractor-developed budget estimates without conducting a formal review and because of the “inherent trust” between NNSA and its M&O contractors that results from its contracting strategy with them. Specifically, one of these officials stated that, to a large extent, only the M&O contractors are in a position to know the scientific and engineering details of nuclear weapons and the associated work scope and funding necessary to ensure their safety and reliability. However, for the last 10 years, we have reported that NNSA has significant weaknesses in its ability to control costs and effectively manage its M&O contractors. We are therefore concerned that NNSA management continues to deny the need for NNSA to improve its processes for developing credible and reliable budget estimates.

NNSA’s Formal Budget Validation Review Process Occurs Too Late to Affect Budget Decisions and Is Not Sufficiently Thorough

We identified three key problems in NNSA’s annual budget validation review process—its formal process for assessing M&O contractor- and program-developed budget estimates. First, NNSA’s annual budget validation review process occurs too late in the budget cycle to inform NNSA, DOE, OMB, and congressional budget development or appropriations decisions. DOE Order 130.1, which is referenced in NNSA’s policy for its budget validation review process, states that agencies should thoroughly review budget estimates before using these estimates to develop budgets. However, NNSA’s Phase II and Phase III budget validation reviews are scheduled to begin 5 months after the President submits his budget to Congress. Additionally, during each of the past four budget validation cycles, NNSA did not complete its budget validation reports for at least 12 months following the President’s budget submission to Congress and at least 4 months after the beginning of the fiscal year for which NNSA reviewed the budget estimates. Therefore, Congress considered the budget request for NNSA and appropriated funds to it, and NNSA executed these funds to M&O contractors, before NNSA had published the results of the budget validation reviews. Because of their timing, NNSA’s budget validation reviews cannot inform NNSA’s budget development, DOE or OMB reviews, or Congress’ appropriation processes. According to NNSA policy, the timing of NNSA’s budget validation review process is designed to inform the NNSA, DOE, and OMB budgeting processes for the fiscal year following that for which the budget validation reviews were conducted. However, the timing of the publication of the budget validation review reports for each of the last 4 years precluded even such delayed consideration because they were issued following the OMB budget formulation process for the following fiscal year.

Second, NNSA’s budget validation review process is not sufficiently thorough to ensure the credibility and reliability of NNSA’s budget. DOE Order 130.1 states that budgets should be based on budget estimates that have been thoroughly reviewed by site and headquarters program offices. However, NNSA’s budget validation review process is limited to assessing the processes M&O contractors and programs used to develop budget estimates rather than the accuracy of the resulting budget estimates. NNSA’s 2010 budget validation review guidance states that the agency lacks the resources and expertise needed to thoroughly evaluate the accuracy of budget estimates on its own and therefore relies on assessments of the reasonableness of the processes used by M&O contractors to develop budget estimates. NNSA officials from the Los Alamos and Y-12 Site Offices told us that they believe the budget validation review process would benefit NNSA more if it more thoroughly assessed the budgetary processes M&O contractors used to develop their budget estimates. Furthermore, NNSA policy and budget validation review guidance stipulate that 20 percent of the agency’s programs should be reviewed annually to help ensure its validity, but NNSA’s formal validation process actually results in significantly smaller portion of its budget being reviewed. For example, in 2011, NNSA’s annual budget validation guidance identified four programs subject to budget validation review—the Engineering Campaign, Nuclear Counterterrorism Incident Response, Global Threat Reduction Initiative, and Fissile Materials Disposition—each of which is conducted at multiple NNSA sites. However, NNSA conducted validation reviews at only one site for each of these programs, which resulted in a formal validation review of approximately 12, 21, 15, and 4 percent of each of those programs’ total budgets, respectively, which, together, constituted 1.5 percent of NNSA’s budget request for fiscal year 2012.

Third, other weaknesses in NNSA’s budget validation review process limit its effectiveness as a resource to assess the validity of its budget estimates. In particular, NNSA workgroups that reviewed the 2007 and 2008 budget validation review cycles recommended that NNSA formally evaluate the status of recommendations made during previous budget validation reviews. However, NNSA has not incorporated a formal mechanism for implementing an evaluation into its budget validation review process. NNSA officials at the Los Alamos and Y-12 site offices also told us that not having such an evaluative mechanism was a weakness in NNSA’s budget validation process. Without a formal mechanism, NNSA is limited in its ability to measure (1) any progress M&O contractors or programs have made in their processes for estimating budgets in response to recommendations from previous budget validation reviews and (2) the effectiveness of NNSA’s budget validation review process. For example, a 2010 budget validation review of the Readiness Campaign recommended that the program more formally document its budget processes, guidance, and estimating assumptions. Furthermore, a 2009 budget validation review of the Elimination of Weapons Grade Plutonium Production program found that the program could not provide documentation of its internal budget processes. However, in both instances, NNSA did not follow up to determine if the programs had addressed these concerns during subsequent budget validation reviews. Additionally, budget validation reviews do not always include recommendations to improve M&O contractor or program processes for estimating budgets when they identify potentially serious weaknesses in those M&O contractor’s or programs’ ability to develop cost estimates. For example, according to a 2010 budget validation review of budget estimation activities for the Nonproliferation and Verification Research and Development program at Sandia National Laboratories, six of the eight projects reviewed lacked sufficient documentation to support their cost estimates, including two that lacked any supporting documentation. The report noted the importance of credible cost estimates, but it did not formally recommend any remedial improvements and rated the overall processes used to develop those cost estimates as satisfactory. Additionally, NNSA officials in the Defense Programs’ Office of Analysis and Evaluation told us that the cost information used to support budget validation review reports is often flawed or nonexistent.

NNSA Has Implemented Some Tools to Support Decision Making on Resource Trade-offs but Has Stopped Using or Developing Other Capabilities

During the programming phase of PPBE, NNSA uses a variety of management tools, such as integrated priority lists and requirements and resources assessments, to support its programming phase and assist senior managers in making decisions on resource trade-offs. However, it has stopped using these capabilities or developing others.

NNSA Has Developed and Implemented Some Tools to Assist Management in Deciding on Resource Trade-offs

NNSA uses the following management tools to decide on resource trade- offs during the programming phase of its PPBE process: Integrated priority lists. NNSA’s policy for the programming phase stipulates that each of NNSA’s nine program offices is to annually develop an integrated priority list that ranks program activities according to their importance for meeting mission requirements. These lists provide senior NNSA and DOE managers with an understanding of how various funding scenarios would affect program activities. Specifically, these lists rank the priority of program activities that are within anticipated appropriation levels—which are of the highest priority—as well as those that NNSA would fund if the appropriation levels were sufficiently high to do so. For example, the program activity listed last on an integrated priority list would be the first to forgo funding if appropriation levels are lower than anticipated. Conversely, these lists define program activities— unfunded requirements—that would be funded if appropriation levels are higher than anticipated. NNSA instructions for the programming phase stipulate that the agency is to combine the nine program office-developed integrated priority lists into four that correspond to the four congressional appropriations NNSA receives. Three of the integrated priority lists— those of the Offices of the Administrator, Defense Nuclear Nonproliferation, and Naval Reactors—correspond directly to specific appropriations,represent activities funded by the Weapons Activities appropriation into a but NNSA does not combine the six others that single integrated priority list. NNSA officials stated that these six others represent separate and distinct mission areas and only the Administrator can decide on the resource trade-offs among them. Of the six program offices funded by the Weapons Activities appropriation, Defense Programs accounts for a large majority—approximately 82 percent—of the funding requested in NNSA’s fiscal year 2013 budget submission to Congress. According to officials in NNSA’s Office of Management and Budget, the Administrator is responsible for deciding on how to allocate resources across program offices. However, these decisions are not documented or reflected in a single integrated priority list for program activities funded by the Weapons Activities appropriation. By not combining these lists, NNSA is limiting the formal documentation available to inform DOE about how various Weapons Activities appropriation funding scenarios would affect the program activities conducted by these six program offices. Specifically, because these six lists are not integrated, it is unclear which program activities would be affected by changes to appropriation levels or which programs across the six lists are of the highest priority.

Requirements and Resources Assessments. During the 2010 and 2011 programming cycles, NNSA used its Requirements and Resources Assessment process to independently assess the need to conduct new program activities and unfunded requirements. According to the NNSA handbook for this process, officials in NNSA’s Office of Management and Budget review program offices’ budget submissions for activities that are either above anticipated funding levels or are for new activities. For these assessments, officials are to analyze specific factors related to these activities, such as their need for meeting agency priorities and the reasonableness of the assumptions used to produce their budget estimates. The objective of this process is to ensure that new program activities and unfunded requirements are needed to meet NNSA priorities. For example, according to officials in NNSA’s Office of Management and Budget, the use of the Requirements and Resources Assessment process was a contributing factor in reducing the amount of unfunded program activities included in NNSA’s budget from approximately $1 billion for fiscal year 2012 to approximately $80 million for fiscal year 2013. Furthermore, draft NNSA guidance states that the process has identified inconsistencies in the quality of estimates and the level of insight and understanding program managers have regarding the fidelity of the estimates supporting their budgets. According to officials in NNSA’s Office of Management and Budget, this process is a simple and effective tool for providing management with additional information on the need to conduct proposed new program activities or unfunded requirements.

However, these officials also stated that this process is time-consuming and would not be practical or efficient to apply to the entirety of NNSA program activities because it was designed to assess program components rather than entire programs; they added that other types of program reviews or validations would be better suited for conducting program needs analysis on an enterprise-wide basis. Additionally, because the NNSA Office of Integration and Assessments, which was responsible for conducting these assessments, was dissolved in 2010, officials in NNSA’s Office of Management and Budget told us that they may discontinue the use of Requirements and Resources Assessment process in future programming cycles. Furthermore, in the current austere budget environment, they do not foresee any programs proposing activities that are either new or above anticipated funding levels. Therefore, the continued use of this process in future programming cycles is uncertain. However, we believe that NNSA has demonstrated this process can be an important tool for assessing the necessity to fund certain activities in order to meet its mission requirements.

Enterprise Portfolio Analysis Tool. NNSA’s Office of Defense Programs is implementing a data system—the Enterprise Portfolio Analysis Tool— designed to provide a consistent framework for managing the PPBE process within Defense Programs, which accounts for 54 percent, or $6.2 billion, of the President’s $11.5 billion fiscal year 2013 budget request for NNSA. As we testified in February 2012, a tool such as this could help NNSA obtain the basic data it needs to make informed management decisions, determine return on investment, and identify opportunities for Currently, this tool includes a mechanism to identify when cost saving.decisions on resource trade-offs must be made if, for example, M&O contractor-developed budget estimates for program requirements exceed the budget targets NNSA provided for those programs. Additionally, the tool is to incorporate Defense Programs’ common work activity structure—known as its work breakdown structure—to facilitate an analysis of consistent budget data from across the NNSA enterprise. Specifically, the tool may allow Defense Programs managers to compare the budget estimates for analogous activities across the nuclear security enterprise regardless of which M&O contractor or program is conducting them. Furthermore, Defense Programs officials stated that they eventually plan to use this tool to compare budget estimates of program activities with the amounts the programs ultimately expended, but they said that the introduction of this capability is not imminent. According to Defense Programs and M&O contractor officials, the implementation of this tool is placing an additional labor burden on NNSA M&O contractors because of the quantity of historical budget data that need to be entered into it. However, according to Defense Programs officials, once these initial historical data are entered, the M&O contractors will need to annually update the system with the most recent year’s data.

NNSA Does Not Use Capabilities That Were Previously Implemented or Partially Developed to Support Its Programming Activities

NNSA no longer has an independent analytical capability to perform such functions as reviewing proposals for program activities and verifying cost estimates. In addition, since 2009, the NNSA Administrator has not formally documented his decisions on resource trade-offs at the close of the programming phase in the AFR. Furthermore, NNSA has not completed cost estimating guidance to assist NNSA program managers in identifying reliable M&O contractor practices for estimating costs for operations and sustainment activities. By not using these capabilities, NNSA has reduced its ability to decide on resource trade-offs because it has not enhanced, made formal, or implemented capabilities that it had already, to varying degrees, developed or used.

The DOE Inspector General and GAO, recommended in 2003 and 2007, respectively, that NNSA establish an independent analysis unit to perform such functions as reviewing proposals for program activities and verifying cost estimates. NNSA agreed with these recommendations and, in 2009, instituted the Office of Integration and Assessments to identify, analyze, assess, and present to senior NNSA management options for managing its programs and making decisions on resource trade-offs. The specific responsibilities of this office included analyzing program performance, evaluating programming and funding alternatives, and assessing the implementation and effectiveness of process improvement initiatives. Furthermore, this office managed the Requirements and Resources Assessment process during the 2010 programming cycle. However, NNSA disbanded the office in 2010, 18 months after it was formally created. NNSA officials also told us that it was never properly staffed, which thereby limited its effectiveness.

In the memorandum establishing the Office of Integration and Assessments, NNSA stated that it expected the office, in conjunction with DOE’s Office of Cost Analysis, to provide DOD-like analytical resources Since then, however, DOE has also eliminated its Office across NNSA.of Cost Analysis. With both of these offices now gone, neither NNSA nor DOE has independent cost assessment or program evaluation capabilities analogous to those of DOD. In contrast, because NNSA’s Office of Naval Reactors is organized as a separate entity within NNSA reporting both to NNSA and the U.S. Navy, unlike the rest of NNSA, it is subject to the Navy’s independent analytical resources, such as assessments by the Naval Sea Systems Command’s Office of Cost Engineering and Industrial Analysis and the Naval Center for Cost Analysis, which conduct independent reviews and analyses of program cost estimates. Furthermore, DOD has an Office of Cost Assessment and Program Evaluation, which has a similar function but with a purview that extends across DOD, including the Navy. These layers of independent review approximate NNSA’s vision for independent analysis, as described in the memorandum establishing the Office of Integration and Assessments. Following the dissolution of the Office of Integration and Assessments in 2010, NNSA’s Office of Defense Programs created the Office of Analysis and Evaluation to conduct similar program review functions. However, the capabilities of this office are limited by several factors. For example, because the office is positioned within Defense Programs, it does not have purview to conduct analysis on any of NNSA’s other programs, which, in total, constitute nearly half of the agency’s budget request for fiscal year 2013. Additionally, according to Defense Programs officials, this office does not have the capability to self-initiate reviews of programs but rather is instructed by Defense Programs’ management on what activities to assess, thereby limiting the office’s independence. Furthermore, NNSA officials from this office stated that properly staffing the office remains a challenge because many qualified individuals left DOE and NNSA when they eliminated the Offices of Cost Analysis and Integration and Assessments, respectively.

Even though NNSA has had difficulty in maintaining an agencywide independent analytical capability, NNSA’s Principal Deputy Administrator told us that NNSA remains supportive of the concept of an independent analytical unit to conduct assessments of programs agencywide. However, senior NNSA officials told us that creating and developing the capabilities of such an office would be difficult in the current budget environment and that therefore NNSA has no current plans to institute such a capability.

The NNSA Administrator has not formally documented his decisions on resource trade-offs at the close of the programming phase in the AFR since 2009, which is inconsistent with NNSA policy and instructions. When issued, this document articulated the Administrator’s rationale and methodology for deciding on resource trade-offs during the programming phase of the PPBE process—which one senior official in NNSA’s Office of Management and Budget described as an important component of the PPBE process—to support in his budget proposal to DOE and to better facilitate NNSA’s participation in DOE’s Strategic Resources Review. According to senior NNSA officials, the Administrator considered the AFR to be a useful management tool but decided to discontinue issuing it because of concerns that its contents, which are predecisional Executive Branch deliberative material and embargoed from public release by OMB Circular A-11, could be leaked and thereby reduce the flexibility of DOE and OMB in making final decisions regarding the President’s Budget. Instead of the AFR, the Administrator now develops an internal document called “Administrator’s Preliminary Decisions,” which is not required in NNSA policy, guidance, or instructions; contains more generalized information; and does not have the rationales, methodologies, and justifications for decision making on resource trade-offs that were previously incorporated into the AFR.

NNSA developed a draft guide—the Program Managers’ Guide to Understanding and Reviewing Cost Estimates for Operations and Sustainment Activities—in 2010 to assist NNSA program managers in identifying reliable M&O contractor practices for estimating costs for operations and sustainment activities—activities not related to construction; according to this guide, these activities constitute approximately 80 percent of NNSA’s annual budget. This guide was also created to supplement the information provided in NNSA’s Business Operating Procedure 50.005, Establishment of an Independent Cost Estimate Policy and interim Cost Estimating Guide 50.005, which identifies best practices for preparing cost estimates.Program Managers’ Guide to Understanding and Reviewing Cost Estimates for Operations and Sustainment Activities was largely completed but never finalized before NNSA dissolved the Office of Integration and Assessments, which had drafted the guide, and NNSA officials said the agency has no plans to complete or issue it. According to officials in NNSA’s Office of Management and Budget, NNSA drafted this guide because it recognized that supplemental information focused on operations and sustainment activities cost estimates—the development of which, according to this guide, are not governed by any specific NNSA guidance or processes—could enhance the tools available to program managers in evaluating cost estimates and how they are translated into budget estimates. The objective of the guide was to provide an instructive document to facilitate program managers’ ability to understand what constitutes a rigorous process for ensuring quality operations and sustainment cost estimates on an ongoing basis and evaluate the reasonableness those estimates. This guide also defined key components of cost estimating to clarify the responsibilities and expectations of NNSA program managers and included instructions for how NNSA program managers can assess the quality of budget estimates submitted by M&O contractors. NNSA officials with Defense Programs’ Office of Analysis and Evaluation told us that additional guidance on how to assess the costs of operations and sustainment activities could enhance program managers’ ability to assess the reliability and credibility of cost estimates.

Conclusions

NNSA has established a formal four-phase PPBE process that uses short- and long-term planning to define program priorities and match them to available budgetary resources. However, DOE and NNSA have not taken adequate steps to make this process as effective and efficient as possible. In particular, DOE Order 130.1, which defines DOE’s provisions for budget activities, references outdated terminology and organizations that no longer exist within the department, leading to confusion regarding the order’s applicability and requirement for implementation. As a result, NNSA believes that the order has expired and that it is not required to adhere to its provisions. By not adhering to these provisions, NNSA is reducing the credibility of its budget proposals. Moreover, NNSA’s process for developing budget estimates continues to rely heavily on its M&O contractors to develop budget estimates without an effective, thorough review of the validity of those estimates. Without thorough reviews by site and headquarters program offices of budget estimates, NNSA cannot have a high level of confidence in its budget estimates or in its ability to make informed decisions on resource trade-offs and to enhance the credibility and reliability of its budget. Furthermore, NNSA’s formal budget validation review process does not sufficiently ensure the credibility and reliability of NNSA’s budget, primarily because of deficiencies in the timing of these reviews. Also, without a formal mechanism to evaluate the status of recommendations made during previous budget validation reviews, NNSA is limited in its ability to measure any progress M&O contractors or programs have made to their budget estimating processes.

NNSA has reduced its ability to decide on resource trade-offs because it has not enhanced, made formal, or implemented capabilities that it has already, to varying degrees, developed or used. In particular, NNSA does not follow its instructions for preparing an integrated priority list for each congressional appropriation, as it does not combine the six priority lists that represent activities funded by the Weapons Activities appropriation into a single integrated list. By not combining these lists into a single integrated priority list, NNSA is limiting the formal documentation available to inform DOE which program activities would be affected by changes to this appropriation. Moreover, NNSA instituted and then disbanded an independent analytic capability that would provide it with an independent cost assessment or evaluation capabilities of the reasonableness and affordability of various programs and projects proposed by NNSA offices. By disbanding its independent analytical capability, NNSA is losing its ability to improve its cost-estimating capabilities and better ensure that its project cost estimates are credible and reliable. Because of the fiscal constraints in the current budget environment, it is all the more critical that NNSA have the capability to conduct independent cost analyses to enhance its ability to make the most effective and efficient resource decisions on resource trade-offs. Despite previous recommendations that DOE’s Inspector General made in 2003, and that we made in 2007, to institute an independent analytical capability to assess programs throughout all of NNSA, NNSA continues to lack such a function. Not having this capability could preclude NNSA from making the best decisions about what activities to fund and whether they are affordable. In addition, NNSA may cease using its Requirements and Resources Assessment process—which is intended to provide some independent analysis of new program activities and unfunded requirements—in future PPBE budget cycles because it does not anticipate program proposals for new activities or unfunded requirements. By not retaining this process, NNSA would lose an important tool for assessing the necessity to fund certain activities in order to meet its mission requirements.

Furthermore, NNSA no longer follows its policy to issue the AFR. Without a formal document, NNSA and DOE have no formal record of the Administrator’s rationale and methodology for deciding on resource trade- offs during the programming phase of the PPBE process. We recognize that NNSA needs to hold confidential, internal budgetary and resource trade-off deliberations; however, we do not believe that this need supersedes NNSA policy or the benefits provided by documented decision making during programming, which one senior NNSA official described to us an important component in NNSA’s PPBE process. Not issuing the AFR (or some similarly precise documentation) places the Administrator in conflict with official NNSA policy and with an important PPBE precept—the importance of transparency.

Finally, NNSA developed draft guidance in 2010 to assist NNSA program managers in identifying reliable M&O contractor practices for estimating costs for operations and sustainment activities. Such guidance would better equip NNSA program managers to more accurately evaluate the reasonableness of cost estimates, but this guidance is in draft form and NNSA has no plans to complete and issue it. Without such guidance, NNSA program managers are limited in their ability to assess the reliability and credibility of budget estimates.

Recommendations for Executive Action

To enhance NNSA’s ability to better ensure the validity of its budget submissions, and to decide on resource trade-offs, we recommend that the Secretary of Energy take the following seven actions:

Direct the DOE Office of Budget to formally evaluate DOE Order 130.1 and revise as necessary, and communicate any revisions to the NNSA Administrator so that the agency will have updated provisions for assessing the quality of its budget estimates.

Direct the Administrator of NNSA to:

Develop a formal process, or amend its budget validation review process, to ensure that all budget estimates are thoroughly reviewed by site and headquarters program offices, and that these reviews are timed to inform NNSA, DOE, OMB, and congressional budget decisions.

Once this process is developed, incorporate a formal mechanism to evaluate the status of recommendations made during previous budget validation reviews so that NNSA can measure M&O contractors’ and programs’ progress in responding to deficiencies with their budget estimates.

Combine the integrated priorities lists for each of the program offices funded within the Weapons Activities appropriation into a single integrated priorities list, as stipulated in NNSA instructions, to better inform DOE which program activities would be affected by changes to this appropriation.

Reinstitute an independent analytical capability to provide senior decision makers with independent program reviews, including an analysis of different options for deciding on resource trade-offs, and facilitate NNSA making the best decisions about what activities to fund and whether they are affordable. As part of this capability, formally retain the Requirements and Resources Assessment process to review proposed new activities and unfunded requirements.

Reinstitute the issuance of the Administrator’s Final Recommendations to document the Administrator’s rationale and methodology for deciding on resource trade-offs to support in his budget proposal to DOE and to better facilitate NNSA’s participation in DOE’s budget process.

Complete and formally issue the Program Managers’ Guide to Understanding and Reviewing Cost Estimates for Operations and Sustainment Activities so that program managers will be better equipped to evaluate the reasonableness of cost estimates.

Agency Comments and Our Evaluation

We provided DOE with a draft of this report for its review and comment. In its written comments, NNSA, responding on behalf of DOE, provided observations on the report’s findings and stated that it generally agreed in principle with six of our seven recommendations and did not concur with one. NNSA did not concur with our recommendation to combine the integrated priorities lists for all program offices funded by the Weapons Activities appropriation into a single integrated priorities list, as is stipulated by NNSA instructions for the programming phase of PPBE.

NNSA agrees that the integrated priorities lists are a useful tool to facilitate NNSA and DOE decision-making. However, NNSA states that it believes reaching management consensus on a single integrated priorities list for these program offices would be a difficult, time consuming process and that its current approach for deciding on resource trade-offs is effective and efficient. We acknowledge that NNSA uses a variety of tools in addition to integrated priorities lists to conduct programming activities, but we continue to believe that combining the integrated priorities lists for all program offices funded by the Weapons Activities appropriation could enhance the agency’s ability to support its decisions on resource trade-offs for DOE consideration during the Strategic Resources Review. However, NNSA stated in its comments that it would consider the development of more robust integrated priority lists if circumstances require changes to its current approach.

NNSA further acknowledged that aspects of its PPBE process could be improved but disagreed with our report’s characterization of its budget estimate review processes as not being thorough. NNSA commented that it believes that our conclusions overemphasize some procedural areas for potential improvement, without accurately considering the cumulative effectiveness of NNSA’s PPBE process as a whole. We continue to believe that the agency’s processes for reviewing budget estimates are not sufficiently thorough to ensure the credibility and reliability of those estimates and do not meet the provisions defined in DOE Order 130.1. Specifically, the reviews conducted by site and headquarters program office officials are informal and undocumented, and NNSA’s budget validation review process—the agency’s formal process for assessing M&O contractor- and program-developed budget estimates—does not assess the accuracy of budget estimates and is conducted for a small portion of the agency’s annual budget. NNSA’s letter is reproduced in appendix II.

NNSA also provided technical comments, which we incorporated throughout the report as appropriate.

We are sending this report to the Secretary of Energy, the Administrator of NNSA, the appropriate congressional committees, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-3841 or aloisee@gao.gov. Contact points for our Office of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to determine (1) the current structure of the National Nuclear Security Administration’s (NNSA) planning, programming, budgeting, and evaluation (PPBE) process; (2) the extent to which NNSA reviews its budget estimates; and (3) how NNSA decides on resource trade-offs in its PPBE process.

To determine the current structure of NNSA’s PPBE process, we reviewed the NNSA policies and other headquarters-developed instructions and guidance documents that define how the process is designed to function. We also reviewed program-specific PPBE documentation from the Offices of Defense Programs, Defense Nuclear Nonproliferation, and Naval Reactors; these three offices correspond to NNSA’s primary missions and collectively account for approximately 85 percent of the President’s fiscal year 2013 NNSA budget submission to Congress. We also interviewed officials from NNSA’s Office of Management and Budget, which is responsible for managing NNSA’s PPBE process, as well as the offices of Defense Programs, Defense Nuclear Nonproliferation, and Naval Reactors to discuss how NNSA’s PPBE process is designed to function.

To determine the extent to which NNSA reviews its budget estimates, we reviewed DOE Order 130.1 and NNSA policies, instructions, and guidance that define how such reviews are to be conducted. We also analyzed documentation of the formal budget validation reviews conducted by NNSA for the last five review cycles, as well as the results of two NNSA workgroups that evaluated the budget validation review process. Furthermore, we interviewed officials involved in the development, oversight, or execution of NNSA budget estimate reviews from the NNSA Offices of the Administrator, Management and Budget, Defense Programs, Defense Nuclear Nonproliferation, Naval Reactors, and Acquisition and Project Management; the site offices for Los Alamos, Sandia, and the Y-12 National Security Complex, and the Naval Reactors Laboratory Field Office; DOE officials from the Office of Budget; and M&O contractor officials from Los Alamos and Sandia National Laboratories, the Y-12 National Security Complex, and Bettis Atomic Power Laboratory. Because NNSA’s Office of Naval Reactors is organized as a separate entity within NNSA reporting both to NNSA and the U.S. Navy, we also met with Navy officials from its Offices of Financial Management and Budgeting, and Cost Engineering and Industrial Analysis.

To determine how NNSA decides on resource trade-offs, we reviewed NNSA policies, instructions, and guidance for its programming process. Based on these documents, we identified the tools that NNSA uses, or has used, to assist NNSA management in deciding on resource trade-offs and reviewed documentation of how these tools were applied by program offices and NNSA management during the programming phases of previous PPBE cycles. We also interviewed officials from the NNSA Offices of the Administrator, Management and Budget, Defense Programs, Defense Nuclear Nonproliferation, Naval Reactors to discuss how they decide on, and document, resource trade-offs.

We conducted this performance audit from July 2011 to July 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the U.S. Department of Energy

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Dan Feehan (Assistant Director), Robert Baney, Matthew Tabbert, and Eugene Wisnoski made significant contributions to this report. Cheryl Peterson, Jeremy Sebest, Rebecca Shea, Carol Herrnstadt Shulman, and Kiki Theodoropoulos provided technical assistance.