NUCLEAR 

NONPROLIFERATION

Better Information 

Needed on Results of 

National Nuclear 

Security 

Administration’s 

Research and 

Technology 

Development Projects

Report to the Committee on Armed 

Services, House of Representatives

February 2017

GAO-17-210

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-17-210, a report to the 

Committee on Armed Services, House of 

Representatives

February 2017

NUCLEAR NONPROLIFERATION

Better Information Needed on Results of 

National Nuclear Security Administration's 

Research and Technology Development 

Projects

What GAO Found

The full extent to which research and technology development projects managed 

by two programs in the National Nuclear Security Administration’s (NNSA) Office 

of Defense Nuclear Nonproliferation (DNN) have resulted in advanced 

(progressed technologies or science supporting them), transitioned (provided to 

users for further development or deployment), or deployed (used in the field) 

technologies is unclear because NNSA does not consistently track and 

document all of these outcomes. Specifically, the DNN Research and 

Development (DNN R&D) and Nonproliferation and Arms Control (NPAC) 

programs track and document technology advancements resulting from their 

projects, such as in project reports. However, NNSA officials acknowledged that 

DNN R&D—by far the larger program—does not consistently track and 

document whether its projects result in technologies transitioned to or deployed 

by end users. In contrast, the NPAC program tracks and documents these 

project outcomes. By not consistently tracking and documenting technology 

transition and deployment outcomes, NNSA is unable to demonstrate the full 

results of its projects. GAO gathered information from various sources on a 

sample of 91 projects. Of these, 88 projects advanced technologies by, for 

example, building instrument hardware or developing models for data analysis; 

the other 3 did not advance technologies but assessed potential applications of 

existing technologies. Among the 88 projects that advanced technologies, 33 

also resulted in technologies being transitioned, including software used to 

analyze nuclear detonations. Finally, of these 33 projects, 17 also resulted in 

deployed technologies, including an enrichment monitoring tool used in Iran and 

space-based nuclear detonation sensors. Reasons that some technologies in the 

sample did not move beyond the advancement or transition stage included that

the technology needed further development or evaluation before being used. 

Limitations with the clarity of the DNN R&D program’s publicly reported 

performance measures make them difficult to interpret, potentially hindering 

users’ ability to determine the program’s progress; better documentation of 

project performance against baseline targets may enhance NNSA’s performance 

assessments. The DNN R&D program’s performance measures have clarity 

limitations because they do not, for example, define measurement criteria or 

provide context justifying how the program determined that it met its performance 

targets; this may cause external users of the measures, including Congress, to

have difficulty interpreting NNSA’s assessment of performance. In a December 

2011 report, GAO recommended that NNSA clarify its publicly reported 

measures for DNN’s R&D program. GAO continues to believe that NNSA should 

do so. Regarding project-level performance, NNSA uses project plans to 

establish baseline targets for projects’ scope and completion date, and tracks 

progress by communicating with project managers and reviewing documents 

such as quarterly and final project reports. However, NNSA officials said that 

final project reports do not document their assessment of performance against 

baseline targets and that there is no common template for final project reports, 

which GAO confirmed in a review of several final project reports. Documenting 

such assessments could enhance NNSA’s ability to assess project performance 

against goals, consistent with federal internal control standards.

View GAO-17-210. For more information, 

contact Shelby S. Oakley at (202) 512-3841 or 

oakleys@gao.gov.

Why GAO Did This Study

Nuclear proliferation is a national security threat. DNN’s two research and 

technology development programs develop technical solutions to prevent 

nuclear proliferation. From fiscal years 

2012 through 2015, these programs 

obligated over $1.1 billion on 511 projects. The DNN R&D program obligated over $1 billion of this amount.

A House report included a provision for 

GAO to review the programs. This

report evaluates, among other 

objectives, (1) the extent to which 

projects managed by NNSA’s research 

and technology development programs 

resulted in advanced, transitioned, or 

deployed nonproliferation technologies 

and (2) how NNSA measures the 

performance of its research and 

technology development programs and 

projects.

GAO reviewed documents on a nongeneralizable sample of 91 research 

and technology development projects, 

selected from all program areas to 

include projects with the largest 

amount of funding and to capture a 

broad range of the programs’ technical 

functions. GAO also reviewed DNN’s 

program objectives and publicly

reported performance measures and 

interviewed NNSA officials and 

technology users at other agencies.

What GAO Recommends

GAO recommends that NNSA consistently track and document results of 

DNN R&D projects and document 

assessments of final project results 

against baseline performance targets. 

NNSA neither agreed nor disagreed 

with the first recommendation and 

partially agreed with the second, but 

agreed to take actions in response to 

both recommendations.
Page i GAO-17-210 Nuclear Nonproliferation

Letter 1

Background 5

NNSA Does Not Consistently Track and Document the Full Extent 

of Nonproliferation Technology Advancement, Transition, and 

Deployment 9

NNSA’s Publicly Reported Program Measures Lack Clarity, and 

Improved Project-level Documentation Could Enhance 

Performance Assessment 18

DNN Programs Use Similar Five-step Processes to Select 

Research and Technology Development Projects 22

Conclusions 25

Recommendations for Executive Action 27

Agency Comments and Our Evaluation 27

Appendix I: Objectives, Scope and Methodology 30

Appendix II: Selected Defense Nuclear Nonproliferation Research 

and Technology Development Projects that Resulted in 

Deployed Technologies 36

Appendix III: Selected Research and Technology Development Projects 

that Resulted in Advanced, Transitioned, and Deployed 

Technologies 37

Appendix IV: Publicly Reported National Nuclear Security Administration 

Performance Goals for Research and Technology 

Development Projects 38

Appendix V: Comments from the National Nuclear Security Administration 39

Appendix VI: GAO Contact and Staff Acknowledgments 41

Contents
Page ii GAO-17-210 Nuclear Nonproliferation

Tables

Table 1: Obligated Funding and Number of Office of Defense

Nuclear Nonproliferation (DNN) Research and 

Technology Development Projects from Fiscal Years 2012 

through 2015 8

Table 2: Number of Projects Selected for Review from Each 

National Nuclear Security Administration (NNSA) Office of 

Defense Nuclear Nonproliferation (DNN) Program and 

Functional Area 32

Table 3: Selected Defense Nuclear Nonproliferation (DNN) 

Research and Technology Development Projects that 

Resulted in Deployed Technologies 36

Table 4: Selected Projects in the Office of Defense Nuclear 

Nonproliferation’s (DNN) Research and Technology 

Development Programs that Resulted in Advanced, 

Transitioned, and Deployed Technologies 37

Figures

Figure 1: Programs, Program Areas, and Functional Areas within 

the Office of Defense Nuclear Nonproliferation that 

Manage Research and Technology Development 

Projects 7

Figure 2: Selected National Nuclear Security Administration 

Research and Technology Development Projects that 

Resulted in Advanced, Transitioned, and Deployed 

Technologies 14

Figure 3: Project Selection Process Used by the Office of Defense 

Nuclear Nonproliferation’s Research and Technology 

Development Programs 25

Figure 4: Publicly Reported National Nuclear Security 

Administration (NNSA) Performance Goals for the Office 

of Defense Nuclear Nonproliferation’s (DNN) Research 

and Technology Development Projects 38
Page iii GAO-17-210 Nuclear Nonproliferation

Abbreviations

CSA Comprehensive Safeguards Agreements

CTBT Comprehensive Nuclear-Test-Ban Treaty

CTBTO Preparatory Commission for the Comprehensive NuclearTest-Ban Treaty Organization

DHS Department of Homeland Security

DNN Office of Defense Nuclear Nonproliferation

DNN R&D DNN Research and Development program

DOD Department of Defense

DOE Department of Energy

GOR goals, objectives, and requirements

GPRA Government Performance and Results Act of 1993

GPRAMA GPRA Modernization Act of 2010

IAEA International Atomic Energy Agency

IRS Internal Revenue Service

JCPOA Joint Comprehensive Plan of Action

NDD Nuclear Detonation Detection program area

NNSA National Nuclear Security Administration

NPAC Nonproliferation and Arms Control program

NPT Treaty on the Non-Proliferation of Nuclear Weapons

OLEM online enrichment monitor

OMB Office of Management and Budget

PD Proliferation Detection program area

R&D research and development

Safeguards International Nuclear Safeguards program area

State Department of State

Verification Nuclear Verification program area

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-17-210 Nuclear Nonproliferation

441 G St. N.W.

Washington, DC 20548

February 3, 2017

The Honorable Mac Thornberry

Chairman

The Honorable Adam Smith

Ranking Member

Committee on Armed Services

House of Representatives

The threat posed by the proliferation of nuclear weapons is one of the 

most pressing national security challenges facing the United States. To 

address this challenge, the National Nuclear Security Administration 

(NNSA)—a separately organized agency within the Department of Energy 

(DOE)—implements nuclear nonproliferation programs worldwide under 

its Office of Defense Nuclear Nonproliferation (DNN).1 DNN’s mission is 

to develop and implement policy and technical solutions to eliminate 

proliferation-sensitive materials and limit or prevent the spread of 

materials, technology, and expertise related to nuclear and radiological 

weapons and programs around the world. As part of this mission, DNN 

supports programs that conduct research and develop technologies for 

use in treaty monitoring, safeguarding nuclear materials, and detecting 

proliferation-related activities and nuclear detonations. DNN’s nuclear 

nonproliferation research and development (R&D) activities have been 

implemented under different program names since the mid-1960s.

We have previously identified challenges affecting NNSA’s nuclear 

nonproliferation R&D activities. For example, in August 2002, we found 

that NNSA did not have a formal process to identify the needs of 

technology users in the U.S. government’s nonproliferation community for 

two of the three research areas managed at that time by NNSA’s 

Nonproliferation and Verification R&D Program—the predecessor of 

DNN’s current nuclear nonproliferation R&D program.2 Additionally, we 

found then that NNSA had inadequate tools to monitor project progress. 

 1NNSA was created by the National Defense Authorization Act for Fiscal Year 2000 and 

was given responsibility for the nation’s nuclear weapons, nonproliferation, and naval 

reactors programs. Pub. L. No. 106-65 §§ 3211, 3214 – 3216 (1999).

2GAO, Nonproliferation R&D: NNSA’s Program Develops Successful Technologies, but 

Project Management Can Be Strengthened, GAO-02-904 (Washington, D.C.: Aug. 23, 

2002).

Letter
Page 2 GAO-17-210 Nuclear Nonproliferation

NNSA agreed with our recommendations that the agency offer technology 

users opportunities to provide input through all phases of R&D projects 

and that it upgrade the R&D program’s project management information 

system to track progress in meeting projects’ milestones. More recently, 

we found in a December 2011 report that some DNN program 

performance measures did not include key attributes of successful 

measures, such as being clear, reliable, and balanced.3 For instance, we 

found that performance measures for DNN’s Nonproliferation and 

Verification R&D program were unclear because the measures assessed 

progress using unspecified criteria and milestones from a classified R&D 

requirements document. Because these performance measures were 

linked to an unstated secondary set of criteria, we found that a third party 

unfamiliar with the classified requirements would have difficulty 

interpreting the measures and discerning whether the criteria for 

measuring progress were appropriate, sufficient, and up to date. NNSA 

neither agreed nor disagreed with our recommendation that the agency 

clarify its performance measures for evaluating the Nonproliferation and 

Verification R&D program. We discuss this recommendation later in this 

report.

The House Armed Services Committee Report 114-102, which 

accompanied H.R. 1735, the National Defense Authorization Act for 

Fiscal Year 2016, included a provision for us to review NNSA’s 

nonproliferation R&D programs.4 This report (1) evaluates the extent to 

which projects managed by NNSA’s research and technology 

development programs have resulted in advanced, transitioned, or 

deployed nonproliferation technologies, (2) evaluates how NNSA 

measures the performance of its research and technology development 

programs and projects, and (3) describes how DNN programs decide 

which research and technology development projects to pursue.

To evaluate the extent to which NNSA’s research and technology 

development projects have resulted in advanced, transitioned, or 

deployed nonproliferation technologies, we obtained and analyzed 

information on a nongeneralizable sample of 91 research and technology 

development projects managed by the two DNN programs responsible for 

 3GAO, Nuclear Nonproliferation: Action Needed to Address NNSA’s Program 

Management and Coordination Challenges, GAO-12-71 (Washington, D.C.: Dec. 14, 

2011).

4H.R. 1735 was vetoed by the President. S. 1356 was later enacted as the National 

Defense Authorization Act for Fiscal Year 2016. Pub. L. No. 114-92, 129 Stat. 726 (2015). 
Page 3 GAO-17-210 Nuclear Nonproliferation

such projects. The information included project data from a program 

database. We selected these 91 projects—which were undertaken 

between fiscal years 2012 and 2015—for the sample based on funding 

level and project technical area, among other factors, to cover a broad 

range of program activities.5 We also interviewed officials in NNSA 

headquarters, project managers at DOE and NNSA national laboratories, 

and officials in other agencies that are end users of technologies and 

other outcomes that result from DNN’s projects.6 Specifically, we 

interviewed officials at the Departments of Defense (DOD), Homeland 

Security (DHS), and State, as well as officials at the Preparatory 

Commission for the Comprehensive Nuclear-Test-Ban Treaty 

Organization (CTBTO) and the International Atomic Energy Agency 

(IAEA).7 We also reviewed documents that establish program goals and 

federal internal control standards.8

To evaluate how NNSA measures the performance of its research and 

technology development programs and projects, we reviewed externally 

reported and internal program– and project-level measures that NNSA 

uses to assess performance in DNN’s research and technology 

development programs. While we also reviewed internal program-level 

measures, we focused our evaluation on external, publicly reported 

 5The results of this sample cannot be generalized to all projects managed by NNSA’s 

research and technology development programs, but they can provide examples of results 

from such projects.

6We use the term “project manager” to describe the lead scientist for each research and 

technology development project. Within NNSA, this person is described as the project’s 

“principal investigator.” 

7The Comprehensive Nuclear-Test-Ban Treaty (CTBT) provides for a ban on nuclear 

explosions. It opened for signature in 1996 but will not come into force unless ratified by 

44 countries that held certain nuclear technologies when the treaty was drafted. The 

CTBTO, founded in 1996, is working to build a verification regime to enforce the CTBT, 

should the treaty enter into force. This verification regime includes an on-site inspection 

capacity, as well as the International Monitoring System, which monitors the planet for 

signs of nuclear explosions by using seismic, hydroacoustic, infrasound, and radionuclide 

detection capabilities. The IAEA is an independent international organization based in 

Vienna, Austria, and is affiliated with the United Nations. IAEA’s dual missions are to 

promote peaceful uses of nuclear energy and verify that nuclear technologies and 

materials are used for peaceful purposes and are not diverted to nuclear weapons.

8GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1

(Washington, D.C.: November 1999). GAO revised and reissued Standards for Internal 

Control in the Federal Government; the revision is effective as of October 1, 2015. See 

GAO, Standards for Internal Control in the Federal Government, GAO-14-704G 

(Washington, D.C.: September 2014).
Page 4 GAO-17-210 Nuclear Nonproliferation

performance measures. Specifically, we reviewed measures developed 

for and reported on in NNSA’s budget materials in response to the GPRA 

Modernization Act of 2010 (GPRAMA), which updated the Government 

Performance and Results Act of 1993.9 For project-level measures, we 

reviewed documentation on the 91 projects we selected for our 

nongeneralizable sample; specifically, we reviewed documents that 

establish baseline targets for projects’ scope, cost, and completion date, 

as well as documents that contain information about project results. We 

also interviewed DNN officials to obtain their views about program and 

project management and performance. In addition, we reviewed 

documents that establish criteria for successful performance measures 

and federal internal control standards.10

To describe how DNN decides which projects to pursue, we reviewed 

documents that establish the nonproliferation mission needs of DOE, 

interagency, and IAEA stakeholders; those that establish program 

requirements and project selection criteria; and project proposals. We 

also interviewed NNSA and other agency officials and project managers 

at the national laboratories to understand the decision-making processes 

that DNN uses to define broad program areas and select projects for 

funding.

To assess the reliability of the data we analyzed on the projects in our 

nongeneralizable sample, we interviewed NNSA officials and reviewed 

documentation, and we determined that the data were sufficiently reliable 

for presenting the information contained in this report on NNSA project 

characteristics and funding. Additional details on our scope and 

methodology can be found in appendix I.

 9GPRA is the Government Performance and Results Act of 1993 (Pub. L. No. 103-62, 107 

Stat. 285). The GPRA Modernization Act of 2010 (GPRAMA) (Pub. L. No. 111-352, 124 

Stat. 3866 (2011)), which amended GPRA, modernizes the federal government’s 

performance management framework. Federal agencies are to report publicly on their 

performance against goals. The Office of Management and Budget (OMB) reviews 

agencies’ performance against GPRAMA measures and can give agencies some flexibility 

on the form and content of publicly reported performance information. See also GAO, 

Managing for Results: GPRA Modernization Act Implementation Provides Important 

Opportunities to Address Government Challenges, GAO-11-617T (Washington, D.C.: May 

10, 2011).

10For example, see GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing 

Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002); 

International Space Station: Measurable Performance Targets and Documentation 

Needed to Better Assess Management of National Laboratory,GAO-15-397 (Washington, 

D.C.: Apr. 27, 2015); and GAO/AIMD-00-21.3.1 and GAO-14-704G 
Page 5 GAO-17-210 Nuclear Nonproliferation

We conducted this performance audit from November 2015 to February 

2017 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

Within DNN, two programs manage research and technology 

development projects: the DNN Research and Development (DNN R&D) 

program and the Nonproliferation and Arms Control (NPAC) program. 

These two programs have established program areas to focus their 

research and technology development activities.

• DNN R&D program areas. Within the DNN R&D program, two 

program areas pursue research and technology development 

projects: Nuclear Detonation Detection and Proliferation Detection. 

The Nuclear Detonation Detection program area develops and 

provides global monitoring capabilities for detecting foreign nuclear 

weapon detonations, in order to meet both treaty monitoring and 

military needs. The Proliferation Detection program area develops 

capabilities to detect special nuclear materials (which include 

plutonium and highly enriched uranium), to detect weapons 

production and movement, and to enhance transparent nuclear 

reductions and monitoring. These two program areas each are divided 

into three functional areas.

• The Nuclear Detonation Detection program area’s three functional 

areas are:

• Space-based Nuclear Detonation Detection, which develops 

and builds space-based sensors for nuclear explosion 

detection;11

• Ground-based Nuclear Detonation Detection, which produces 

and updates modeling and analysis capabilities—such as 

 11Projects in the Space-based Nuclear Detonation Detection functional area service the 

U.S. Nuclear Detonation Detection System, the space-based portion of the U.S. Atomic 

Energy Detection System, which is an Air Force-operated system that also includes 

ground-based detection capabilities. The U.S. Nuclear Detonation Detection System 

began operating in the 1960s as a treaty monitoring system and evolved to include a 

tactical warning and attack assessment mission. It consists of satellites that locate and 

characterize the magnitude of nuclear detonations.

Background
Page 6 GAO-17-210 Nuclear Nonproliferation

software codes and algorithms that help distinguish 

underground nuclear detonations from natural seismic 

events—and provides other technical support for the nation’s 

ground-based nuclear explosion monitoring networks;12 and

• Forensics-based Nuclear Detonation Detection, which 

conducts R&D to advance capabilities in the field of nuclear 

forensics analysis.13

• The Proliferation Detection program area’s three functional areas 

are:

• Nuclear Weaponization and Material Production Detection, 

which supports the development of technology to detect and 

characterize the production of nuclear weapons and related 

materials by foreign entities;

• Nuclear Weapons and Material Security, which, among other 

things, develops tools for nuclear security, treaty monitoring 

and verification, and operational interdiction and nuclear 

security efforts across NNSA; and

• Enabling Capabilities, which develops cross-cutting 

technologies applicable to multiple NNSA and interagency 

missions.

• NPAC program areas. Within the NPAC program, two program areas 

pursue technology development projects:

• International Nuclear Safeguards (Safeguards), among other 

things, develops technologies to detect and deter undeclared 

nuclear materials and activities. Specifically, Safeguards’ activities 

include developing and transferring tools, technologies, and 

approaches to improve U.S., IAEA, and IAEA member states’

capabilities in undertaking IAEA safeguards activities, such as 

monitoring uranium enrichment levels in nuclear facilities.14

 12The Ground-based Nuclear Detonation Detection functional area began in the 1990s as 

the Ground-based Nuclear Explosion Monitoring program, with the goal of transforming 

U.S. nuclear explosion monitoring capabilities. 

13Nuclear forensics includes the analysis of nuclear or radiological materials that are 

intercepted or the radioactive debris and prompt output signals (such as gamma rays) 

produced by a nuclear event. 

14IAEA safeguards are a set of technical measures and activities by which IAEA seeks to 

verify that nuclear material subject to safeguards is not diverted for use in nuclear 

weapons or for other proscribed purposes. 
Page 7 GAO-17-210 Nuclear Nonproliferation

• Nuclear Verification (Verification), among other things, develops 

and deploys technologies to maintain the United States’ ability to 

monitor and verify nuclear reduction agreements, detect treaty 

violations, and verify other nuclear nonproliferation commitments. 

The goals of the Verification program area include developing 

technologies to verifiably and irreversibly disable nuclear facilities 

in countries of concern.

Figure 1 illustrates the structure of the DNN programs and functional 

areas that manage research and technology development projects.

Figure 1: Programs, Program Areas, and Functional Areas within the Office of Defense Nuclear Nonproliferation that Manage 

Research and Technology Development Projects

From fiscal year 2012 through fiscal year 2015, DNN R&D and NPAC 

obligated a total of more than $1.1 billion on 511 research and technology 

development projects. The DNN R&D program obligated more than $1 

billion on 420 projects, and the NPAC program obligated about $73 

million on 91 projects. According to NNSA officials we interviewed, the 
Page 8 GAO-17-210 Nuclear Nonproliferation

Space-based Nuclear Detonation Detection functional area expenditures 

were mostly for production activities that resulted in technologies 

deployed to end users, rather than for basic and applied R&D, in contrast 

with the other DNN R&D program functional areas.15 DNN R&D and 

NPAC projects are undertaken at DOE and NNSA national laboratories 

and sites; these projects generally last about 3 years from beginning to 

end, according to program officials. Table 1 shows the number of projects 

undertaken and funding obligated in each program and functional area 

from fiscal years 2012 through 2015.

Table 1: Obligated Funding and Number of Office of Defense Nuclear Nonproliferation (DNN) Research and Technology 

Development Projects from Fiscal Years 2012 through 2015

Program Area/Functional Area Total Obligated Funding

Percentage of Total 

Funding Obligated Total Projects

Percentage of 

Total Projects 

Nonproliferation and Arms Control (NPAC) Program

Safeguards $55,225,000 4.9% 72 14.1%

Verification $18,019,546 1.6% 19 3.7%

NPAC Subtotal $73,244,546 6.5% 91 17.8%

DNN Research and Development (DNN R&D) Program

Nuclear Detonation Detection 

(NDD) Directors Accounta

$850,000 0.1% 1 0.2%

NDD Program Area/Space-based 

Nuclear Detonation Detection

$313,400,240b 27.6% 45 8.8%

NDD Program Area/Ground-based 

Nuclear Detonation Detection

$59,093,992 5.2% 29 5.7%

NDD Program Area/Forensicsbased Nuclear Detonation 

Detection

$25,086,967 2.2% 21 4.1%

Nuclear Detonation Detection 

Subtotal

$398,431,199 35.1% 96 18.8%

Proliferation Detection (PD) Office 

Levela $203,333,359 17.9% 24 4.7%

PD program area/Nuclear 

Weaponization and Material 

Production Detection

$194,972,190 17.2% 92 18.0%

PD program area/Nuclear 

Weapons and Material Security

$156,789,785 13.8% 133 26.0%

PD program area/Enabling 

Capabilities

$107,097,422 9.4% 75 14.7%

 15Specifically, NNSA officials said that about 90 percent of this functional area’s 

obligations were for production activities. 
Page 9 GAO-17-210 Nuclear Nonproliferation

Program Area/Functional Area Total Obligated Funding

Percentage of Total 

Funding Obligated Total Projects

Percentage of 

Total Projects 

Nonproliferation and Arms Control (NPAC) Program

Proliferation Detection Subtotal $662,192,756 58.4% 324 63.4%

DNN R&D Subtotal $1,060,623,955 93.5% 420 82.2%

Total $1,133,868,501 511

Source: GAO analysis of DNN information │GAO-17-210

a

The Nuclear Detonation Detection Directors Account and the Proliferation Detection Office Level 

Account include projects under the direction of the directors for those program areas.

b

According to NNSA officials, about 90 percent of this amount w as obligated on production activities, 

rather than R&D activities.

According to NNSA officials, the end users of technologies resulting from 

DNN’s research and technology development programs include DOD, 

including its Defense Threat Reduction Agency, U.S. Strategic Command, 

and Air Force Technical Applications Center; DHS; State; IAEA and other 

international organizations, including the CTBTO; international 

governments; universities; and private industry, including small business. 

NNSA itself also uses some of the technologies that result from DNN’s 

research and technology development programs.

The full extent to which research and technology development projects 

managed by NNSA’s DNN R&D and NPAC programs have resulted in 

advanced, transitioned, or deployed technologies is unclear because 

NNSA does not consistently track and document all of these project 

outcomes. However, by reviewing a range of information on a 

nongeneralizable sample of 91 DNN research and technology 

development projects, as well as interviewing end users of technologies 

that resulted from some of the projects, we were able to determine the 

outcomes of these projects. More specifically, we found that 88 of the 91 

projects in the sample resulted in technologies being advanced—that is, 

the project progressed the technology itself or the scientific knowledge 

behind the technology.16 Additionally, we found that, among these 88 

projects, 33 resulted in technologies being transitioned—that is, provided 

to users outside of the project team for further development or 

deployment. Finally, we found that 17 of these 33 projects resulted in 

deployed technologies—that is, we were able to confirm that a technology 

was being actively used in the field by a federal agency or foreign partner.

 16Examples of activities that advanced technologies in the projects that we reviewed 

included laboratory experiments, field tests, reports, and publications.

NNSA Does Not 

Consistently Track

and Document the 

Full Extent of 

Nonproliferation 

Technology 

Advancement, 

Transition, and 

Deployment
Page 10 GAO-17-210 Nuclear Nonproliferation

The full extent to which NNSA’s research and technology development 

projects result in advanced, transitioned, or deployed technologies is 

unclear because NNSA does not consistently track and document all of 

these project outcomes. Both the DNN R&D and NPAC programs track 

and document projects’ results in advancing technologies. For instance, 

the DNN R&D program uses an online system, WebPMIS, to track project 

costs and maintain an archive of project-related documentation. This 

documentation includes project reports that provide information on the 

scientific and technological advancements that result from a project. 

Similarly, the NPAC program provided documentation of technology 

advancement for our review. However, the DNN R&D program, which is 

by far the larger program, does not consistently track and document 

projects that result in technologies being transitioned or deployed. NNSA 

officials acknowledged that the DNN R&D program does not use the 

WebPMIS system, or otherwise collect and maintain information, to track 

and document whether its projects result in technologies that have been 

transitioned to or deployed by end users. As a result, the DNN R&D 

program was not able to readily provide us with information on the extent 

to which its projects result in transitioned or deployed technologies. In 

contrast, we found that the NPAC program tracks and documents the 

extent to which its safeguards– and verification-related projects have 

resulted in technology transition or deployment, as well as tracking

scientific and technological advancements. For example, NNSA officials 

provided us with information on all NPAC projects for the past 5 fiscal 

years that showed whether technologies that projects developed have 

been deployed; they also provided documentation of technologies that 

have been transitioned to end users. Both the DNN R&D and NPAC 

programs list technology transition—which includes providing 

technologies to end users, who may then deploy them—as a goal in 

strategic planning documents. In addition, NPAC program officials told us 

that their program evaluates its success in part based on the extent to 

which end users deploy technologies that the NPAC program develops. 

Similarly, DNN R&D program officials said that meeting end users’

technology needs constitutes an important part of their program mission. 

Moreover, under federal internal control standards, management should 

design control activities to achieve objectives and respond to risks, and 

should record, communicate, and use quality information to enable 

managers to carry out internal control responsibilities and evaluate 

program performance in achieving key objectives.17

 17 GAO/AIMD-00-21.3.1 and GAO-14-704G. Using quality information entails obtaining 

NNSA Tracks and 

Documents 

Nonproliferation 

Technology Advancement, 

but Does Not Do So 

Consistently for 

Technology Transition and 

Deployment
Page 11 GAO-17-210 Nuclear Nonproliferation

Differing program emphases—both between the DNN R&D and NPAC 

programs and among the DNN R&D program’s functional areas—may 

account for the inconsistencies we found in NNSA’s tracking and 

documentation of transition– and deployment-related project outcomes. 

For instance, the NPAC program generally focuses on developing 

technologies at higher readiness levels to prepare them for transition and 

deployment in the field, according to NNSA officials.18 Conversely, most 

of the DNN R&D program’s functional areas conduct R&D on 

technologies that are further from being deployment-ready because they 

require more basic work at lower technology readiness levels, with some 

exceptions, NNSA officials told us. Notably, the DNN R&D program’s 

Space-based Nuclear Detonation Detection functional area has a history, 

dating to the 1960s, of researching, developing, and producing 

successive upgrades to technologies used by the U.S. Atomic Energy 

Detection System and its space-based element, the U.S. Nuclear 

Detonation Detection System, to monitor and detect nuclear detonations, 

NNSA officials stated. Officials in the DNN R&D program office said that 

well-defined DOD technology requirements and delivery schedules drive 

these research, development, and production activities; as a result, the 

Space-based Nuclear Detonation Detection functional area has a 

stronger focus on technology production and deployment than the other 

DNN R&D program functional areas, which typically do not have similarly 

specific requirements and schedules.

NNSA officials we interviewed offered other reasons for why the agency 

does not consistently track the deployment– and transition-related 

outcomes of the DNN R&D program’s projects. For example:

• End users generally are responsible for the final development and 

production steps that precede technology deployment, with the 

notable exception of the Space-based Nuclear Detonation Detection 

projects discussed above.19 DNN’s responsibility generally entails 

 relevant data from reliable internal and external sources, processing the data into quality 

information, and then considering the data to make informed decisions and evaluate 

program performance.

18The technology readiness level system is a scale of nine levels used to measure a 

technology’s progress, starting with paper studies of a basic concept and ending with a 

technology that has proven itself in actual use.

19In addition, according to DNN R&D program officials we interviewed, DNN R&D 

programs occasionally produce prototype technologies that become “de facto operational 

assets” for some end users. 
Page 12 GAO-17-210 Nuclear Nonproliferation

providing end users with tools or knowledge that are sufficiently ready 

for the end user to take the final steps necessary for the technology’s 

use in mission-specific purposes, such as adapting a detection 

algorithm or producing a field-ready detector array.

• It can be difficult to trace a link between a project and a given result—

for example, when another agency builds on a project’s work; when a 

discrete project does not lead directly to an outcome, but results in a 

“spin-off” project or is part of a related portfolio of projects that does 

lead to the desired outcome; or when there is a significant period of 

time between the end of a project and the deployment of any resulting 

technology.

• Project managers at DOE and NNSA national laboratories know the 

outcomes of projects and can provide information to NNSA officials 

when needed.

• Information about some nonproliferation technologies is classified, 

with the result that NNSA is not informed about their uses and users.

These factors notwithstanding, by not consistently tracking and 

documenting all relevant outcomes of projects across DNN’s research 

and technology development program areas, NNSA is unable to 

demonstrate to Congress and the public the full extent to which its 

projects result in technologies that are transitioned, deployed, or 

otherwise used toward fulfilling the nation’s nuclear nonproliferation goals. 

Moreover, not maintaining documentary information about project results 

may affect control activities by limiting the quality of information that 

NNSA has available to evaluate project performance; it also may reduce 

opportunities for sharing knowledge, and may make DNN vulnerable to 

the loss of institutional memory when personnel involved with a project 

retire or leave the agency.
Page 13 GAO-17-210 Nuclear Nonproliferation

By gathering information from various sources about our sample of 91 

selected DNN research and technology development projects, we were 

able to determine that some projects resulted in technologies being 

advanced, transitioned, or deployed, or that they resulted in more than 

one of these outcomes.20 More specifically, we found that nearly all of the 

projects we selected (88 of 91) advanced the technology itself or the 

scientific knowledge behind the technology.21 Of those 88 projects, we 

found that 33 also resulted in the advanced technology being transitioned 

to users outside of the project team for further development or 

deployment. Among the 33 projects that resulted in a transitioned 

technology, 17 also resulted in active technology deployment—that is, 

field use by a federal agency or foreign partner. Figure 2 below illustrates 

these results.

 20As noted in appendix I, we selected 63 projects from the DNN R&D program and 28 

projects from the NPAC program. To confirm the deployed technologies that resulted from 

projects in our sample, we used several information sources for each project, including 

interviews with several officials across several locations. DNN R&D officials provided 

some information on potential end users of their technologies but referred us to officials at 

the national laboratories to obtain more complete information on each of the projects we 

selected for our sample. The laboratory officials we interviewed who participated in or 

were familiar with these projects were able to provide information on the deployed 

technologies and other outcomes that resulted from only some of the projects. We also 

interviewed interagency stakeholders, but only in some cases could the stakeholders 

confirm that technologies they deployed resulted from the DNN projects in our sample. We 

also reviewed project documentation, including project plans; annual, quarterly, and final 

reports; and project publications. 

21The remaining 3 projects were technology assessments that considered potential 

applications of existing technologies.

Selected Projects 

Resulted in Advanced, 

Transitioned, and 

Deployed Technologies
Page 14 GAO-17-210 Nuclear Nonproliferation

Figure 2: Selected National Nuclear Security Administration Research and 

Technology Development Projects that Resulted in Advanced, Transitioned, and 

Deployed Technologies

Almost all of the selected projects in our sample (88 of 91) advanced 

technologies in some way. Examples of projects that advanced 

technologies include projects in which NNSA scientists built instrument 

hardware, made progress in developing new technology combinations or 

applications, developed an algorithm or model to further data analysis, or 

made scientific advances that improved nonproliferation analysis and 

operations.

Some of the projects in our sample that have resulted in technologies 

being advanced have not resulted in technologies being transitioned to 

end users (55 of 88 such projects). According to NNSA officials and 

scientists we interviewed, technology transition has not occurred on these 

projects for a variety of reasons, including:

• The technology is not developed enough to be transitioned. For 

example, a DNN R&D Proliferation Detection project developed a 

Eighty-eight of the 91 Projects 

in Our Sample Resulted in 

Technology Advancement
Page 15 GAO-17-210 Nuclear Nonproliferation

technology to provide new capabilities in imaging gamma rays.22 The 

imager is a laboratory prototype and needs further development, 

according to a scientist we interviewed at Oak Ridge National 

Laboratory.

• The project has not connected with an end user who wants to receive 

the technology. For example, a DNN R&D Proliferation Detection 

project developed a hand-held fast-neutron generator, but NNSA has 

yet to find an end user interested in the technology, according to the 

manager of the Nuclear Threat Reduction Program at Lawrence 

Livermore National Laboratory.23

• The project developed a technology to meet a future nonproliferation 

need rather than an existing requirement. For example, a project 

within the NPAC Verification program developed a fast-neutron 

imaging technology that could be used in stockpile warhead 

verification, should a future arms control agreement call for warhead 

inspections. The technology currently is not being used in a 

verification context, but NNSA officials said it is in use at NNSA sites 

to compile benchmark data on nuclear weapons; these data may later 

be used in verification and other national security missions.

We determined that 33 of the 91 projects in our sample resulted in 

technologies that were transitioned to end users. These 33 projects 

include 17 projects that resulted in deployed technologies, since a 

technology is transitioned to an end user before it is deployed. For the 

remaining 16 projects, we found that either the technologies had not been 

deployed by an end user (13 of the 16) or we could not confirm with end 

users that the technologies had been deployed (3 of the 16). Specifically:

• We counted 13 of the 16 projects and their resulting technologies as 

transitioned but not deployed because the end users need to take 

certain steps before the technologies can be deployed, according to 

NNSA and end user agency officials. For example, one DNN R&D 

Nuclear Detonation Detection project transitioned an instrument to a 

DOD end user. However, the instrument has not been deployed 

 22This technology, called a Compton camera, can detect gamma rays from any direction, 

as opposed to devices that must be aimed at the gamma ray source. In addition, the 

camera’s imaging capabilities produce a “picture” of the gamma ray source, potentially 

enabling users to see the precise location of the gamma ray source.

23Neutron generators are used in a variety of nonproliferation and commercial contexts. 

Detection of fast neutrons is of particular interest in the nonproliferation context because 

these neutrons can indicate the presence of special nuclear materials.

Thirty-three of the 91 Projects 

in Our Sample Resulted in 

Transitioned Technologies
Page 16 GAO-17-210 Nuclear Nonproliferation

because DOD needs to find funds to further develop the technology, 

which, according to DOD officials, was at a low level of technology 

readiness when transitioned. In another case, a DNN R&D Groundbased Nuclear Detonation Detection project transitioned software to a 

DOD end user, but the user is still evaluating the software before 

integrating it into its systems.24

• For the remaining 3 of the 16 projects, we could not confirm whether 

technologies that resulted from the projects and were transitioned to 

end users are being used.

We were able to confirm that 17 of the 91 projects in our sample resulted 

in actively deployed technologies. More specifically, we found that 13 of 

the 63 DNN R&D projects and 4 of the 28 NPAC projects in our sample 

resulted in deployed technologies.

Examples of the projects that resulted in deployed technologies include:

• An NPAC Safeguards project that developed an online enrichment 

monitor (OLEM) for use in uranium enrichment monitoring. As we 

reported in June 2016, the IAEA is using the OLEM in the Natanz Fuel 

Enrichment Plant in Iran to confirm that enrichment levels are at or 

below 3.67 percent, per Iran’s commitment under the agreement 

known as the Joint Comprehensive Plan of Action (JCPOA).25 IAEA 

has previously used enrichment monitors, but the OLEM is a newer 

technology that improves upon older monitoring systems, as we found 

in our June 2016 report.

• DNN R&D Space-based Nuclear Detonation Detection projects that 

resulted in global burst detection sensors that are in use by the Air 

Force on satellites to detect, identify, and precisely locate nuclear 

explosions.

 24According to NNSA officials, this project also developed improvements to scientific 

understanding and modeling that undergird software in use at national laboratories.

25In July 2015, multilateral talks with Iran culminated in an agreement—the Joint 

Comprehensive Plan of Action (JCPOA)—in which the United States, France, Germany, 

the United Kingdom, Russia, and China, with the High Representative of the European 

Union for Foreign Affairs and Security Policy, agreed to reciprocal commitments with Iran. 

These commitments include providing sanctions relief if Iran implements its nuclear 

commitments as laid out in the JCPOA. GAO, Iran Nuclear Agreement: The International 

Atomic Energy Agency’s Authorities, Resources, and Challenges,GAO-16-565 

(Washington, D.C.: June 9, 2016).

Seventeen of the 91 Projects in 

Our Sample Resulted in 

Deployed Technologies
Page 17 GAO-17-210 Nuclear Nonproliferation

• DNN R&D Ground-based Nuclear Detonation Detection projects that 

resulted in enhanced understanding of waves produced by 

underground nuclear explosions.26 The Air Force, the CTBTO, and 

international scientists analyze these waves to locate underground 

nuclear explosions and establish their yield.

Of the 17 projects that resulted in deployed technology, 9 were active or 

ongoing projects and 8 were retired or in closeout, meaning that all of the 

research work was completed.27 Similarly, of the 33 projects that resulted 

in transitioned technology, 15 were active or ongoing projects and 18 

were retired or in closeout. Projects resulting in deployed and transitioned 

technologies can continue to be active for the following reasons:

• In some cases, project officials provide maintenance and support to 

the technology after it has been deployed. For example, the OLEM 

unit has been deployed, but the project is still active because NNSA is 

continuing to address end user maintenance needs, according to 

agency officials.

• In other cases, the project has deployed or transitioned a technology 

but has yet to accomplish research goals. For example, a Nuclear 

Detonation Detection project has deployed a technology to a DOD 

end user, but research continues because the project has not met all 

of the research objectives.

See appendix II for a list of the projects in our sample that resulted in 

deployed technologies. Appendix III provides information on the number 

of advanced, transitioned, and deployed technologies that resulted from 

the 91 projects we selected for our sample, subdivided by DNN program 

and functional areas.

 26The program analyzes integrated data on electromagnetic, infrasound, and acoustic 

waves, which can be produced by nuclear explosions or by natural seismic events. 

According to officials in the DNN R&D program’s Ground-based Nuclear Detonation 

Detection functional area, projects in this area seek to decrease the incidence of “false 

alarms” (caused, for example, by earthquakes) and increase the accuracy of 

measurements of nuclear detonation events.

27We found no statistically significant difference between active and retired projects in 

terms of the likelihood that they resulted in a deployed technology. Of the 88 projects that 

resulted in an advanced technology, 51 were active or ongoing projects, and 37 of the 

projects were retired, in closeout, or on hold.
Page 18 GAO-17-210 Nuclear Nonproliferation

NNSA’s DNN R&D and NPAC programs use publicly reported measures 

to assess program-level performance, and both programs review project 

documentation and communicate with project managers to assess 

project-level performance. Clarity limitations with the DNN R&D program’s 

publicly reported measures, however, make them difficult to interpret, and 

improvements in NNSA’s final project documentation could enhance 

assessment of project performance.

NNSA reports publicly on the DNN R&D and NPAC programs’

performance using measures published in DOE’s annual budget 

requests, in response to requirements in GPRAMA; however, limitations 

with the clarity of the DNN R&D program’s measures may make it difficult 

for users, such as Congress, to understand the targets that NNSA has 

established for its research and technology development programs, as 

well as how NNSA has measured performance against these targets.28

The NNSA budget materials we reviewed present a total of six GPRAMA 

measures associated with the DNN R&D and NPAC programs. 

Specifically, the DNN R&D program reports on five GPRAMA measures 

covering both the Nuclear Detonation Detection and Proliferation 

Detection program areas. The NPAC program reports on one measure for 

its Safeguards program area, but does not have a GPRAMA measure for 

its Verification program area.29 The six GPRAMA measures identify 

specific performance timeframes and endpoint targets, and the publicly 

reported information summarizes NNSA’s assessment of the programs’

performance against the targets. Appendix IV shows the NPAC and DNN 

R&D performance measures as presented in NNSA’s budget request for 

fiscal year 2017, including endpoint targets and NNSA’s assessment of 

the programs’ performance against these targets.

 28In addition, NNSA officials stated that the programs use additional, internal measures to 

assess program performance. We limited our evaluation to publicly reported program 

performance measures because these measures are required by law.

29According to NNSA officials, OMB does not require all program elements to have a 

GPRAMA measure.

NNSA’s Publicly 

Reported Program 

Measures Lack 

Clarity, and Improved 

Project-level 

Documentation Could 

Enhance 

Performance 

Assessment

Clarity Limitations May 

Hinder Users’Ability to 

Interpret DNN’s Publicly 

Reported Performance

Measures
Page 19 GAO-17-210 Nuclear Nonproliferation

We found that the DNN R&D program’s performance measures are 

unclear because the program does not define targets or explain its 

assessments of performance against the targets in sufficient context to 

allow users to interpret the measures or performance assessments. For 

example, regarding three of the DNN R&D program’s measures:

• The Nuclear Detonation Detection measure tracks percentage of 

progress against an annual index summarizing the status of all NNSA 

Nuclear Detonation Detection R&D deliveries that improve the 

nation’s ability to detect nuclear detonations.

• The Nuclear Weapons and Material Security measure tracks 

cumulative percentage of progress toward demonstrating 

improvements in special nuclear material detection, warhead 

monitoring, chain-of-custody monitoring, safeguards, and 

characterization capabilities.

• The Nuclear Weaponization and Material Production Detection 

measure tracks cumulative percentage of progress toward 

demonstrating improvements in detection and characterization of 

nuclear weapons production activities.

For these three measures, the baseline criteria for assessing 

performance—the annual index and the cumulative percentages of 

progress—are not defined, and no justification is presented to clarify how 

NNSA concluded that it had met its performance targets. Without such 

context, the measures provide statements of NNSA’s assessment of their 

own performance, but do not provide users with information they can use 

to evaluate whether NNSA’s assessments are valid and justified.

Previous GAO reports have identified key attributes of successful 

performance measures, including that they be clearly stated to enable 

entities to assess their own performance and to enable stakeholders to 

determine whether a program is achieving its goals.30 In addition, our 

December 2011 report on NNSA program management and coordination 

challenges in the area of nuclear nonproliferation identified concerns with 

the clarity of performance measures used by the DNN programs we 

 30GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season 

Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002). See also GAO, 

International Space Station: Measurable Performance Targets and Documentation 

Needed to Better Assess Management of National Laboratory,GAO-15-397 (Washington, 

D.C.: Apr. 27, 2015).
Page 20 GAO-17-210 Nuclear Nonproliferation

review in this report.31 Specifically, we found that several of the DNN R&D 

program’s performance measures were linked to secondary criteria that 

are classified, official use only, or otherwise not publicly stated, making it 

difficult for third-party users to interpret the measures and discern 

whether the criteria are appropriate, sufficient, and up to date for tracking 

purposes. We recommended in our 2011 report that NNSA clarify these 

measures. NNSA neither agreed nor disagreed with this 

recommendation.

NNSA officials told us, when we interviewed them for this report, that they 

do not agree that the DNN R&D program’s performance measures are 

unclear, for two reasons.

• First, according to NNSA officials, NNSA provides supplemental 

information for Office of Management and Budget (OMB) users of the 

performance information that explains the targets and how 

performance is measured. Specifically, NNSA officials told us that the 

DNN R&D program provides additional documentation to OMB—

some of which the officials also provided to us for review—that 

includes information explaining the progress against annual targets 

under each of the program’s performance measures. However, most 

of the information that NNSA provided on the DNN R&D program is 

classified or official use only.

• Second, an NNSA official told us that the DNN R&D program 

performance measures are not unclear because the measures are 

used primarily to inform internal program managers who are familiar 

with classified requirements against which progress is assessed, 

rather than to provide information to third-party users. However, there 

may be potential external users of the performance information—such 

as congressional decision makers and other external parties—who 

are not familiar with the baseline requirements and may have difficulty 

interpreting the level of performance achieved by the DNN R&D 

program.

In the current review, the clarity issues we found with the DNN R&D 

program’s performance measures were similar to those we identified in 

December 2011.32 We continue to believe that NNSA should clarify these 

publicly reported measures, so that users of the performance 

 31GAO-12-71. At the time of this 2011 report, the DNN R&D program was known as the 

DNN Nonproliferation and Verification R&D program. 

32GAO-12-71
Page 21 GAO-17-210 Nuclear Nonproliferation

information—including Congress and the public—have a sufficient basis 

to judge the programs’ performance in meeting the United States’ nuclear 

nonproliferation technology needs.

NNSA assesses the performance of the DNN R&D and NPAC programs’

projects by reviewing project documentation and communicating with 

project managers, but improvements in NNSA’s final project 

documentation could enhance assessment of project performance. To 

assess project performance, NNSA officials said that both programs track 

progress during the project by communicating with project managers at 

the national laboratories and by reviewing documents, such as quarterly 

project reports, that provide information on project performance. 

According to NNSA documentation we reviewed and NNSA officials we 

interviewed, the DNN R&D and NPAC programs establish baseline 

targets for projects’ scope of work and completion date in project plans. In 

some cases, initial project plans are updated—for example, to incorporate 

new or modified project objectives. At the end of the project, project 

managers at the national laboratories produce final project reports for 

program managers to review.

NNSA officials stated that these final project reports describe a project’s 

technical results. However, the officials said that the final reports do not 

document their assessment of project performance against the baseline 

targets established in the initial project plans. They also said that there is 

no common template for final project reports. We confirmed this in our 

review of several final project reports.

NNSA officials provided several reasons why they do not document their

assessment of project performance against baseline targets or use a 

common template for final project reports. For example, according to 

NNSA officials:

• Program managers communicate with project managers at the 

laboratories to ensure that projects are proceeding as planned.

• Restating the baseline targets in final project reports would be 

duplicative of information contained in other sources, such as the 

WebPMIS database.

• The varying formats of the laboratories’ final reports do not affect 

program managers’ ability to understand the technical outcomes of 

the projects.

Better Documentation 

Could Enhance NNSA’s 

Assessment of Project 

Performance
Page 22 GAO-17-210 Nuclear Nonproliferation

• According to NNSA officials, initial baselines may change through 

“progressive baselining”—which entails changing the initial project 

plans when discoveries warrant or course corrections are needed, 

subject to a defined change control process and the approval of the 

federal program manager.

Federal internal control standards specify that managers should use 

quality information to make informed decisions and evaluate 

performance; that managers need to compare actual performance to 

planned or expected results; and that entities should record and 

communicate information to managers who need it to carry out their 

internal control responsibilities.33 Moreover, the standards specify that 

program managers need data to determine whether their programs are 

meeting their goals for accountability for effective and efficient use of 

resources. Documenting assessments that compare final project 

performance results against baseline targets for scope of work and 

completion date—whether through a common template for final project 

reports or by other means—could enhance NNSA’s ability to manage its 

programs in accordance with these standards.

Both the DNN R&D and NPAC programs use similar five-step processes 

to decide which research and technology development projects to pursue. 

According to NNSA officials and stakeholders we interviewed at 

technology end user agencies, NNSA collaborates throughout the 

decision-making and project selection process with the end users. In our 

review of program documents and interviews with agency officials, we 

found that the DNN R&D and NPAC programs generally take the 

following steps in their project selection processes:34

Step 1: DNN establishes project requirements. According to NNSA 

officials, DNN R&D and NPAC project requirements flow from national 

policy documents, such as the Nuclear Posture Review; additionally, 

NPAC project requirements flow from international sources such as the 

Treaty on the Non-Proliferation of Nuclear Weapons (NPT), 

 33GAO/AIMD-00-21.3.1 and GAO-14-704G

34The one exception is DNN R&D’s Space-based Nuclear Detonation Detection functional 

area. According to an NNSA official, this functional area is not included in its annual call 

for proposals because it has a different mission than the other functional areas; unlike the 

others, it is a longstanding technology developer with an additional production capacity on 

behalf of the agencies that operate the U.S. Nuclear Detonation Detection System.

DNN Programs Use 

Similar Five-step 

Processes to Select 

Research and 

Technology 

Development 

Projects
Page 23 GAO-17-210 Nuclear Nonproliferation

Comprehensive Safeguards Agreements; Additional Protocols; and the 

IAEA Department of Safeguards Long Term R&D Plan.35 To identify the 

technical capabilities that are required to meet the requirements 

established in these documents and sources, NNSA officials develop 

more detailed plans, such as the DNN R&D program’s goals, objectives, 

and requirements (GOR) documents. The GOR documents specify 

requirements for five of the DNN R&D program’s six functional areas—all 

but the cross-cutting Enabling Capabilities functional area. Additionally, 

each GOR document has a corresponding technology roadmap, which 

establishes detailed technical requirements to meet the needs identified 

in the GOR document.36 Similarly, the NPAC program develops needs 

documents that identify specific needs and corresponding capabilities to 

develop; NPAC uses these documents to guide the proposal selection 

process. NNSA consults with interagency stakeholders to inform the 

requirements specified in the GOR documents and needs documents.

Step 2: The DNN programs issue annual calls for proposals. Both 

programs issue annual calls for proposals, which specify the technical 

requirements proposals should address. The programs also require that 

those who submit proposals clearly explain how the proposed project 

relates to identified project requirements. According to NNSA officials, the 

DNN R&D program typically issues its calls for proposals in mid-October37

 35The Treaty on the Non-Proliferation of Nuclear Weapons (NPT), which entered into force 

in 1970, requires non–nuclear weapon states that are party to the treaty—countries that 

had not manufactured and detonated a nuclear device before January 1, 1967—to refrain 

from acquiring nuclear weapons and to subject all nuclear material used in peaceful 

activities to IAEA safeguards. Comprehensive Safeguards Agreements (CSAs) are 

agreements between the IAEA and each non–nuclear weapon state to accept IAEA 

safeguards on all source or special fissionable material in all peaceful nuclear activities 

within the territory of such state under its jurisdiction, or carried out under its control 

anywhere. Additional Protocols require countries to provide IAEA with a broader range of 

information on the country’s nuclear and nuclear-related activities than the country 

provides under CSAs. 

36An NNSA official explained that the Enabling Capabilities functional area does not have 

a GOR document because it pursues cross-cutting projects that may ultimately have 

applications in multiple functional areas. 

37NNSA officials informed us that although mid-October is the typical time the DNN R&D 

program issues calls for proposals, fiscal and political challenges have resulted in program 

officials’ tendency to wait before issuing the call to ensure alignment with emerging 

direction or to mitigate potential disconnects in planning. For example, NNSA officials said 

that, most recently, there were back-to-back 2-year budget deals (in fiscal years 2014-

2015 and fiscal years 2016-2017) to avoid sequestration; these budget deals had 

significant impact on the DNN R&D budget, scope, and planning implied in the call, which 

was then modified before distribution to the labs.
Page 24 GAO-17-210 Nuclear Nonproliferation

and NPAC issues its calls for proposals during the spring.38 The DNN 

R&D program’s calls for proposals specify weighted criteria against which 

proposals are reviewed, including mission relevance, scientific and 

technical merit, and budget. The NPAC program’s calls for proposals 

differ between its two program areas. The Safeguards program area 

provides explicit evaluation criteria including cost effectiveness, technical 

maturity, and deployment potential. Calls for proposals in the Verification 

program area do not provide explicit evaluation criteria, but they provide 

information on program goals and areas of technical consideration for the 

current budget year. They also require applicants to describe the 

proposed work, including activities, milestones, and deliverables.

Step 3: National laboratories submit informal project proposals. Scientists 

at the national laboratories respond to the call for proposals by submitting 

informal project proposals, known as white papers, to the NNSA program 

offices. The white papers describe proposed projects to demonstrate new 

capabilities or technologies sought in the calls for proposals. As specified 

in the DNN R&D program’s calls for proposals, the program uses a threetiered system to rank proposals—specifically, by designating proposals as 

hot, warm, or cold—and provides comments to improve the proposals or 

future white paper submissions.39 According to an NNSA official, the DNN 

R&D program returns the informal project proposals to the original 

submitters. All hot and warm proposals may be amended in response to 

program officials’ comments and then resubmitted in the next step of the 

process as formal project proposals. NNSA officials stated that cold 

proposals receive no further consideration. NNSA officials within the 

NPAC program stated that they use a ranking system that is similar to the 

DNN R&D program’s system; the officials also provided documentation 

that shows their rationale for selecting or not selecting projects for further 

consideration. Both programs seek to provide feedback within a general 

time frame. The DNN R&D program aims to provide feedback within 2 

weeks of receiving the white papers. According to NNSA officials, NPAC’s 

Safeguards program area provides feedback on the white papers about 3 

to 3 1/2 months from the date of submission, and the Verification program 

 38Specifically, NNSA officials said that the Safeguards program area generally sends out 

its call at the end of March or in the first week of April, while the Verification program area 

sends out its call in May.

39Hot white papers are those for which a follow-up proposal is highly desired; warm white 

papers are those for which a follow-up proposal could be submitted, but with modifications 

to refocus the project; and cold white papers are those for which a follow-up proposal is 

not desired.
Page 25 GAO-17-210 Nuclear Nonproliferation

area provides feedback about 2 1/2 months from the date of submission. 

According to NNSA officials and interagency stakeholders we 

interviewed, interagency stakeholders are consulted during this step of 

the process, as appropriate.

Step 4: Laboratories submit formal project proposals. Scientists at the 

national laboratories amend hot and warm white papers and resubmit 

them as formal proposals for NNSA’s consideration.

Step 5: DNN selects projects in consultation with laboratories and end 

users. NNSA officials identify the formal proposals that best meet the 

criteria and requirements specified in the calls for proposals, considering 

input provided by laboratory officials, interagency stakeholders, and 

potential end users on each proposal. According to NNSA officials, after 

discussing budgetary considerations and current, prioritized needs with 

interagency stakeholders and potential end users, program officials 

develop a final, ranked list of projects selected to receive funding and 

support. Program officials then circulate this final list to interagency 

stakeholders and potential end users.

Figure 3 shows the general five-step selection process used by DNN’s 

programs.

Figure 3: Project Selection Process Used by the Office of Defense Nuclear 

Nonproliferation’s Research and Technology Development Programs

NNSA’s research and technology development programs make vital 

contributions to national security, and the projects they support address a 

range of important proliferation detection and monitoring needs, including 

improving abilities to detect covert nuclear material production, nuclear 

explosions, and potential violations of IAEA safeguards agreements. It is 

not clear, however, how often NNSA’s research and technology 

development projects—especially those supported by the DNN R&D 

Conclusions
Page 26 GAO-17-210 Nuclear Nonproliferation

program—result in transitioned or deployed technologies because NNSA 

does not consistently track and document such information. We 

acknowledge that DNN’s research and technology development programs 

pursue a broad range of scientific tools and capabilities, with the goal of 

ensuring that the United States remains prepared to deal with 

unanticipated nonproliferation challenges as they emerge. Therefore, we 

recognize that not all R&D projects, such as projects conducted at low 

levels of technology readiness, are intended to result in a deployed 

technology, and that a higher deployment rate does not necessarily 

indicate a more successful R&D program, or vice versa. Moreover, we 

recognize that in cases where technology transition and deployment are 

project goals, certain challenges complicate DNN’s efforts to track the 

deployed technologies that result from its projects. Nevertheless, we were 

able, as part of our review of selected projects, to track projects’ end 

results, including end users’ deployment of technologies that result from 

the projects. Moreover, not tracking such end results may affect control 

activities by limiting the quality of information that NNSA has available to 

evaluate project performance. More consistently tracking and 

documenting the transitioned and deployed technologies that result from 

its projects would help ensure that NNSA maintains and uses quality 

information to evaluate its performance and achieve its objectives, in 

keeping with federal internal control standards. Doing so would also 

facilitate knowledge sharing within DNN, and it would provide a means by 

which to present valuable information to Congress and other decision 

makers about the programs’ results and their overall value.

In addition, better tracking of project results by the DNN R&D program 

may have the salutary benefit of providing information to the program that 

could allow it to develop clearer program performance measures, as we 

recommended in December 2011.40 The DNN R&D program’s 

performance measures continue to have clarity limitations similar to the 

ones we identified in the December 2011 report, because understanding 

the publicly reported performance information depends on users of the 

information having access to or being familiar with criteria in official use 

only or classified documents. We continue to believe that it is important 

for NNSA to implement our December 2011 recommendation that NNSA 

clarify the DNN R&D program’s publicly reported performance measures, 

so that Congress and other users of these measures have a sufficient 

understanding of DNN programs’ status and progress.

 40GAO-12-71
Page 27 GAO-17-210 Nuclear Nonproliferation

By documenting assessments that compare the final results of a project 

against the baseline targets for scope of work and completion date that 

are established in initial plans for each project, NNSA could improve the 

quality of the information it needs to evaluate performance at the end of 

the project. Specifically, taking these steps could enhance managers’

ability to use quality data to assess actual project performance against 

planned or expected results, consistent with federal internal control 

standards. Such assessments are essential to ensure that the DNN R&D 

and NPAC programs’ research and technology development projects are 

effectively and efficiently using resources, another key aspect of federal 

internal control standards. If changes to the initial baselines are 

necessary—which may be the case in some instances—we believe that 

they should be made in such a way that initial project targets are still 

documented and available for review in evaluating the full progression of 

a project.

We recommend that the NNSA Administrator take these two actions:

• Direct the DNN R&D program to track and document the transitioned 

and deployed technologies that result from its research and 

technology development projects, to the extent practicable.

• Direct the DNN R&D and NPAC programs to document, using a 

common template or other means, their assessment that compares 

the final results of each project against the baseline targets 

established in each project’s initial project plan.

We provided drafts of this report to NNSA, DOD, DHS, and State for 

review and comment. In emails, DOD, DHS, and State stated that they 

had no comments on the report. In NNSA’s written comments, which are 

summarized below and reproduced in appendix V, NNSA neither agreed 

nor disagreed with our first recommendation and partially agreed with our 

second recommendation; however, NNSA stated that it plans to take 

actions in response to both recommendations. NNSA also provided 

technical comments, which we incorporated as appropriate.

In its written comments, NNSA did not state whether it agreed or 

disagreed with the recommendation that, to the extent practicable, the 

DNN R&D program track and document transitioned and deployed 

technologies resulting from the program’s projects, but it described 

planned actions consistent with the recommendation. NNSA stated that 

our report did not clearly or fully disclose reasons that the DNN R&D 

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
Page 28 GAO-17-210 Nuclear Nonproliferation

program does not track and document transition and deployment of 

technologies resulting from its projects. Specifically, NNSA commented 

that the DNN R&D program’s space-based activities—unlike the other 

DNN R&D program areas—are production and deployment oriented, and 

that it would therefore be misleading to track individual projects in this 

area that are transitioned and deployed. Regarding the other DNN R&D 

program areas, NNSA commented that it does not have ongoing insight 

into transition and deployment outcomes for various reasons—for 

example, due to classification barriers regarding the transition and 

deployment of certain technologies. NNSA also expressed concern that 

information on such outcomes may not be readily available or may not be 

cost-effective to obtain. Our report discusses a number of reasons why 

the agency may not consistently track the deployment– and transitionrelated outcomes of the DNN R&D program’s projects and notes the 

production and deployment orientation of the space-based project

portfolio. However, we disagree that tracking the outcomes of these 

projects would be misleading. To the contrary, it is vital to track these 

outcomes, as the space-based program area receives the largest share 

of the DNN R&D program’s funding. Moreover, we believe that 

information on project outcomes should be readily available to NNSA 

program officials in the course of their regular interactions with the project 

managers at the national laboratories and the interagency end users of 

nonproliferation technologies—as was the case with projects in our 

sample. NNSA agreed that, where such information is readily available,

reporting of transitioned and deployed technologies can provide 

information on program successes, and it identified actions it will take 

consistent with the recommendation. Specifically, NNSA stated it will

complete an assessment by June 2017 of the DNN R&D program’s 

portfolio to determine to what extent transition and deployment data are 

readily available for specific projects and include additional information on 

those projects as part of its performance reporting. If implemented as 

planned, such an assessment would be a step toward ensuring that 

NNSA maintains and uses quality information to evaluate its performance 

and achieve its objectives, in keeping with federal internal control 

standards.

NNSA stated that it agrees in part with our second recommendation—that 

NNSA document, using a common template or other means, its 

assessment that compares the final results of each DNN R&D and NPAC 

project against the baseline targets established in each project’s initial 

project plan. NNSA stated that it supports implementation of a common 

template for documenting project closeout to provide consistency in 

information reporting, but it noted that R&D projects are dissimilar to 
Page 29 GAO-17-210 Nuclear Nonproliferation

acquisition activities and that comparing final project results to an initial 

project baseline would be misleading. Specifically, for exploratory R&D 

projects, NNSA commented that project outcomes can be “significantly 

indeterminate” at the beginning of a project and stated that it uses an 

approach known as “progressive baselining” to adjust plans and 

benchmarks as research matures, where appropriate. NNSA stated that 

final project results should be compared to the final approved plans to 

ensure consistency in project documentation and to provide a meaningful 

assessment of project performance. NNSA agreed that information on the 

project’s initial baseline is important to understand the progression of the 

project. We acknowledge that adjustments to initial project plans may be 

appropriate in some cases, provided that the initial plans remain available 

and transparent and that decisions behind such revisions are welldocumented, consistent with federal internal control standards. NNSA 

stated it will establish a common template by June 2017 and use it to 

compare projects’ final results to the final approved plans. It also stated 

that the template will require a brief description of significant changes 

between the initial and final approved project plans. If implemented as 

planned, the template could help NNSA to ensure that the DNN R&D and 

NPAC programs’ research and technology development projects are 

effectively and efficiently using resources, in keeping with federal internal 

control standards.

We are sending copies of this report to the appropriate congressional 

committees; the Administrator of NNSA; the Secretaries of Defense, 

Energy, Homeland Security, and State; and other interested parties. In 

addition, the report is available at no charge on the GAO website at 

http://www.gao.gov.

If you or your staff members have questions about this report, please 

contact me at (202) 512-3841 or oakleys@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. GAO staff who made key contributions to this 

report are listed in appendix VI.

Shelby S. Oakley

Acting Director, Natural Resources and Environment
Appendix I: Objectives, Scope and 

Methodology

Page 30 GAO-17-210 Nuclear Nonproliferation

This report (1) evaluates the extent to which projects managed by the 

National Nuclear Security Administration’s (NNSA) research and 

technology development programs have resulted in advanced, 

transitioned, or deployed nonproliferation technologies, (2) evaluates how 

NNSA measures the performance of its research and technology 

development programs and projects, and (3) describes how programs in 

NNSA’s Office of Defense Nuclear Nonproliferation (DNN) decide which 

research and technology development projects to pursue.

To determine the extent to which NNSA’s research and technology 

development programs have resulted in advanced, transitioned, or 

deployed nonproliferation technologies, we selected a nongeneralizable 

sample from research and technology development projects undertaken 

between fiscal years 2012 and 2015 by the two DNN programs 

responsible for such projects: the Office of DNN Research and 

Development (DNN R&D) and the Office of Nonproliferation and Arms 

Control (NPAC). As detailed below, we selected the projects in the 

sample based on funding level and project technical area of focus and 

status in order to cover a broad range of program activities.

In total, the DNN R&D and NPAC programs provided information on 511 

projects. Specifically, the DNN R&D program provided us with a list of 

420 projects executed between fiscal years 2012 and 2015, and the two 

NPAC program areas that manage technology development projects—

International Nuclear Safeguards (NPAC Safeguards) and Nuclear 

Verification (NPAC Verification)—provided us with a list of 91 projects that 

were executed between fiscal years 2012 and 2015.1

From this universe, we developed selection criteria to include projects 

from five of the major Department of Energy (DOE) and NNSA 

laboratories—Los Alamos National Laboratory, Oak Ridge National 

Laboratory, Sandia National Laboratories, Lawrence Livermore National 

Laboratory, and Pacific Northwest National Laboratory. Our selection 

criteria ensured variations in program areas and project outcomes.2 We 

generally limited our selection to the top 25 percent of projects in terms of 

total funding between fiscal years 2012 and 2015, and excluded all 

 1NPAC Safeguards and NPAC Verification provided lists of 72 and 19 projects, 

respectively, executed within that time frame. 

2We selected these five laboratories because, according to NNSA officials, they managed 

projects accounting for the largest share of project funding. 

Appendix I: Objectives, Scope and 

Methodology
Appendix I: Objectives, Scope and 

Methodology

Page 31 GAO-17-210 Nuclear Nonproliferation

projects that were not led by one of the five national laboratories listed 

above.3 We also excluded projects in certain categories, such as projects 

under the program directors and university accounts. After these 

exclusions, 134 projects remained in the sample.

The next step in our selection process was to group together similar 

projects, based on project status, focus area or type, age, and cost.4 We 

generally selected 2 projects from each group, to ensure that each 

combination of these characteristics was represented in our final sample.

In cases in which a group contained only one project, we included only 

that project.

• For the DNN R&D program, we treated projects with the same 

laboratory, focus area, and status as a single group, resulting in 40 

groups. For each group, we generally chose the 2 projects with the 

lowest project number (signifying the earliest start date), resulting in 

the inclusion of 63 projects.

• For the NPAC Safeguards program area, we treated projects with the 

same technology category (comparable to focus area) and project 

status as a single group, resulting in 13 groups. Because we did not 

have data to determine earliest start date, we generally chose the 2 

projects with the highest total expenditures from each group, resulting 

in the inclusion of 17 projects.

• For the NPAC Verification program area, we treated projects with the 

same R&D category (comparable to focus area) and project status as 

a single group, resulting in 8 groups. Because we did not have data to 

determine earliest start date, we chose the 2 projects with the highest 

total expenditures from each group, resulting in the inclusion of 11 

projects.

 3For NPAC Verification, we did not exclude projects based on funding due to the small 

overall number of projects. 

4Data provided by NPAC and DNN R&D used different terms to describe data categories.
Appendix I: Objectives, Scope and 

Methodology

Page 32 GAO-17-210 Nuclear Nonproliferation

This resulted in a final sample of 91 projects covering the DNN R&D and 

NPAC programs and all of their program and functional areas.5 Table 2, 

below, shows the number of projects we selected from each program and 

functional area.

Table 2: Number of Projects Selected for Review from Each National Nuclear Security Administration (NNSA) Office of 

Defense Nuclear Nonproliferation (DNN) Program and Functional Area

Program Area Functional Area

Number of Projects in our 

Selection

Nonproliferation and Arms Control (NPAC) Safeguards 17

NPAC Verification 11

NPAC Subtotal 28

DNN Research and Development (R&D)

Nuclear Detonation Detection

Forensics-based Nuclear Detonation Detection

5

DNN R&D Nuclear Detonation Detection Ground-based Nuclear Detonation Detection 8

DNN R&D Nuclear Detonation Detection Space-based Nuclear Detonation Detection 10

DNN R&D Nuclear Detonation Detection 

Subtotal 23

DNN R&D Proliferation Detection Nuclear Weapons and Material Security 13

DNN R&D Proliferation Detection Enabling Capabilities 10

DNN R&D Proliferation Detection Nuclear Weaponization and Material Production 

Detection 17

DNN R&D Proliferation Detection Subtotal 40

DNN R&D Subtotal 63

Total 91

Source: GAO analysis of NNSA information. │GAO-17-210

To determine the extent to which the 91 projects in our sample resulted in 

advanced, transitioned, or deployed technologies, we gathered project 

information from interviews and documentary sources. Specifically, we 

interviewed project officials, including the project manager, at the national 

laboratory that was the lead on the project, as well as interagency officials 

 5As discussed in the background section of this report, the DNN R&D program has two 

program areas: Nuclear Detonation Detection and Proliferation Detection. These program 

areas, in turn, each have three functional areas. The Nuclear Detonation Detection 

program area’s three functional areas are Forensics-based Nuclear Detonation Detection, 

Ground-based Nuclear Detonation Detection, and Space-based Nuclear Detonation 

Detection. The Proliferation Detection program area’s three functional areas are Nuclear 

Weapons and Material Security, Enabling Capabilities, and Nuclear Weaponization and 

Material Production Detection. 
Appendix I: Objectives, Scope and 

Methodology

Page 33 GAO-17-210 Nuclear Nonproliferation

that received technologies transitioned from the DNN R&D and NPAC 

programs.6 Our interviews included officials from the Departments of 

Defense (DOD), Homeland Security, and State; the International Atomic 

Energy Agency (IAEA); and the Preparatory Commission for the 

Comprehensive Test Ban Treaty Organization.7 We also reviewed 

program and project data and documentation, including basic project 

data—such as project names and funding information—stored in DNN 

R&D’s WebPMIS database; program funding information; project plans; 

annual, quarterly, and final reports; and research presentations and 

publications. We reviewed the evidence to determine whether each 

project in our sample resulted in advanced, transitioned, or deployed 

technology or other outcomes. The outcomes that we identified are not 

mutually exclusive for any project. For example, one project could 

transition a technology to an end user and also result in a publication, or a 

project may have resulted in a research outcome that we did not identify 

because the outcome was not indicated in the evidence we collected. 

Finally, we reviewed documents that establish program goals, including 

strategic planning documents, as well as Standards for Internal Control in 

the Federal Government, which establishes federal standards for the use 

of information in achieving an entity’s objectives.8

To evaluate how NNSA measures the performance of its research and 

technology development programs and projects, we reviewed externally 

reported and internal program– and project-level measures that NNSA 

uses to assess performance in DNN’s research and technology 

development efforts. For program-level measures, we focused our review 

on the publicly reported measures developed and reported on in NNSA’s 

budget materials in response to the GPRA Modernization Act of 2010 

 6We use the term “project manager” to describe the lead scientist for each research and 

technology development project. Within NNSA, this person is described as the project’s 

“principal investigator.” 

7Within the Department of Defense, we interviewed officials from the Air Force Technical 

Applications Center and the Defense Threat Reduction Agency. Within the Department of 

Homeland Security, we interviewed officials from the Domestic Nuclear Detection Office.

8GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1

(Washington, D.C.: November 1999). GAO has revised and reissued Standards for 

Internal Control in the Federal Government, and the new revision is effective as of 

October 1, 2015. See GAO, Standards for Internal Control in the Federal Government, 

GAO-14-704G (Washington, D.C.: September 2014).
Appendix I: Objectives, Scope and 

Methodology

Page 34 GAO-17-210 Nuclear Nonproliferation

(GPRAMA).9 Specifically, we reviewed the GPRAMA measures for the 

DNN R&D and NPAC programs that NNSA presented in its budget 

materials for fiscal years 2015 through 2017. We also reviewed classified 

and official use only materials that, according to NNSA officials, NNSA 

shares with the Office of Management and Budget (OMB) in their 

discussions of the GPRAMA measures. The materials we reviewed 

included technology roadmaps, which define the technology pathways 

that NNSA follows to address portfolio requirements and develop funding 

priorities, and final reports that NNSA presents to OMB on the completion 

of activities it undertakes in connection with the GPRAMA measures. We 

also reviewed DOE’s Annual Performance Reports for fiscal years 2014 

and 2015. In addition, we reviewed documents and interviewed NNSA 

officials regarding internal measures that DNN R&D and NPAC use to 

monitor program performance, including independent project performance 

reviews. To further evaluate DNN’s project-level performance measures, 

we reviewed information on the 91 selected projects in our 

nongeneralizable sample. Specifically, we reviewed documents that 

establish baseline targets for projects’ scope, cost, and completion date 

and documents that contain information on project results. We also 

interviewed DNN officials to obtain their views on program and project 

management and performance. Finally, we reviewed documents that 

establish criteria for successful performance measures, including our past 

reports on the key attributes of such measures, as well as federal internal 

control standards.10

To describe how DNN decides which projects to pursue, we reviewed 

documents that establish the nonproliferation mission needs of DOE and 

interagency and IAEA stakeholders in nonproliferation-related research 

and technology development. The documents we reviewed include 

 9GPRA is the Government Performance and Results Act of 1993. Pub. L. No. 103-62, 107 

Stat. 285. GPRAMA (Pub. L. No. 111-352, 124 Stat. 3866 (2011)), which amended GPRA, 

modernizes the federal government’s performance management framework. Federal 

agencies are to report publicly on their performance against goals. The Office of 

Management and Budget (OMB) reviews agencies’ performance against GPRAMA 

measures and can give agencies some flexibility on the form and content of publicly 

reported performance information. See also GAO, Managing for Results: GPRA 

Modernization Act Implementation Provides Important Opportunities to Address 

Government Challenges, GAO-11-617T (Washington, D.C.: May 10, 2011).

10For example, see GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing 

Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002); 

International Space Station: Measurable Performance Targets and Documentation 

Needed to Better Assess Management of National Laboratory, GAO-15-397 (Washington, 

D.C.: Apr. 27, 2015); and GAO/AIMD-00-21.3.1 and GAO-14-704G. 
Appendix I: Objectives, Scope and 

Methodology

Page 35 GAO-17-210 Nuclear Nonproliferation

DOD’s 2010 Nuclear Posture Review Report; the Executive Office of the 

President’s National Science and Technology Council’s Nuclear Defense 

Research and Development Roadmap, Fiscal Years 2013-2017; the 

White House’s National Security Strategy of February 2015; DOE and 

NNSA strategic plans; the IAEA Department of Safeguards’ Long-Term 

R&D Plan, 2012-2023; and IAEA’s Development and Implementation 

Support Programme for Nuclear Verification 2016-2017. In addition, we 

reviewed documents that establish program requirements, such as the 

DNN R&D program’s Goals, Objectives, and Requirements documents; 

the DNN R&D and NPAC programs’ calls for project proposals, including 

the project selection criteria provided in the proposal calls; and project 

proposals. We also interviewed NNSA and other agency officials and 

project managers at the national laboratories to obtain their views on 

program priorities and the decision-making processes used to define 

broad program areas and select projects for funding.

To assess the reliability of the data we analyzed on the projects in our 

nongeneralizable sample, we interviewed NNSA officials and reviewed 

documentation, and we determined that the data were sufficiently reliable 

for presenting the information contained in this report on NNSA project 

characteristics and funding.

We conducted this performance audit from November 2015 to February 

2017 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Selected Defense Nuclear 

Nonproliferation Research and Technology 

Development Projects that Resulted in 

Deployed Technologies

Page 36 GAO-17-210 Nuclear Nonproliferation

Table 3: Selected Defense Nuclear Nonproliferation (DNN) Research and Technology Development Projects that Resulted in 

Deployed Technologies

Project Name Program Area Functional Area

1 Online Enrichment Monitor (OLEM) Nonproliferation and Arms Control 

(NPAC)

Safeguards

2 Tamper Resistant/Indicating Aerosol 

Contaminant Extractor (TRI-ACE)

NPAC Safeguards

3 Spent Fuel NDA (five different prototype 

devices; library of spent fuel 

assemblies)

NPAC Safeguards

4 Advanced Experimental Fuel Counter 

(AEFC)

NPAC Verification

5 Improving Nuclear Data Defense Nuclear Nonproliferation 

Research & Development (DNN 

R&D) 

Forensics-based Nuclear 

Detonation Detection

6 Characterization of Nuclear Fallout DNN R&D Forensics-based Nuclear 

Detonation Detection

7 Optical Modeling of Urban NUDETs DNN R&D Forensics-based Nuclear 

Detonation Detection

8 Post-Detonation Nuclear Forensics DNN R&D Forensics-based Nuclear 

Detonation Detection

9 Signal Analysis DNN R&D Ground-based Nuclear 

Detonation Detection

10 Signal Propagation Research & 

Development for Waveform 

Technologies

DNN R&D Ground-based Nuclear 

Detonation Detection

11 CASCADES DNN R&D Ground-based Nuclear

Detonation Detection

12 FIELDED SYSTEMS DNN R&D Space-based Nuclear 

Detonation Detection

13 FIELDED SYSTEMS DNN R&D Space-based Nuclear 

Detonation Detection

14 Warhead Monitoring Planning and 

Execution

DNN R&D Nuclear Weapons and 

Material Security

15 Solution-Growth Scintillators DNN R&D Nuclear Weapons and 

Material Security

16 Remote Conversion DNN R&D Nuclear Weaponization and Material 

Production Detection

17 Integrated Approach for Upstream 

Processing Detection

DNN R&D Nuclear Weaponization and Material 

Production Detection

Source: GAO analysis of NNSA and technology end user information │GAO-17-210

Appendix II: Selected Defense Nuclear 

Nonproliferation Research and Technology 

Development Projects that Resulted in 

Deployed Technologies
Appendix III: Selected Research and 

Technology Development Projects that 

Resulted in Advanced, Transitioned, and 

Deployed Technologies

Page 37 GAO-17-210 Nuclear Nonproliferation

Table 4: Selected Projects in the Office of Defense Nuclear Nonproliferation’s (DNN) Research and Technology Development 

Programs that Resulted in Advanced, Transitioned, and Deployed Technologies

Program Area/Functional Area 

(number of projects reviewed in area)

Number of Projects 

that Resulted in 

Deployed 

Technologies

Number of Projects that 

Resulted in Transitioned 

Technologies

Number of Projects that 

Resulted in Advanced 

Technologies

Nonproliferation and Arms Control (NPAC) Program

Verification program area (11 projects) 1 3 11

Safeguards program area (17 projects) 3 5 16

NPAC subtotal (28 projects) 4 8 27

DNN Research and Development (R&D) Program

Nuclear Detonation Detection (NDD) program 

area/Forensics-based Nuclear Detonation 

Detection functional area (5 projects) 4 4 5

NDD program area/Ground-based Nuclear 

Detonation Detection functional area (8 projects) 3 8 8

NDD program area/Space-based Nuclear 

Detonation Detection functional area (10 projects) 2 6 10

DNN R&D NDD program area subtotal 

(23 projects) 9 18 23

Proliferation Detection program area/Nuclear 

Weaponization and Material Production Detection 

functional area (17 projects) 2 3 15

Proliferation Detection program area/Enabling 

Capabilities functional area (10 projects) 0 1 10

Proliferation Detection program area/Nuclear 

Weapons and Material Security functional area 

(13 projects) 2 3 13

DNN R&D Proliferation Detection program area 

subtotal (40 projects) 4 7 38

DNN R&D subtotal (63 projects) 13 25 61

Total (91 projects) 17 33 88

Source: GAO Analysis of DNN R&D and NPAC information │GAO-17-210

Note: Projects may have resulted in more than one outcome. For example, each project that resulted 

in a deployed technology w as also counted as having resulted in an advanced technology and a 

transitioned technology.

Appendix III: Selected Research and 

Technology Development Projects that 

Resulted in Advanced, Transitioned, and 

Deployed Technologies
Appendix IV: Publicly Reported National 

Nuclear Security Administration Performance 

Goals for Research and Technology 

Development Projects

Page 38 GAO-17-210 Nuclear Nonproliferation

Figure 4: Publicly Reported National Nuclear Security Administration (NNSA) 

Performance Goals for the Office of Defense Nuclear Nonproliferation’s (DNN) 

Research and Technology Development Projects

Appendix IV: Publicly Reported National 

Nuclear Security Administration Performance

Goals for Research and Technology 

Development Projects
Appendix V: Comments from the National 

Nuclear Security Administration

Page 39 GAO-17-210 Nuclear Nonproliferation

Appendix V: Comments from the National 

Nuclear Security Administration
Appendix V: Comments from the National 

Nuclear Security Administration

Page 40 GAO-17-210 Nuclear Nonproliferation
Appendix VI: GAO Contact and Staff 

Acknowledgments

Page 41 GAO-17-210 Nuclear Nonproliferation

Shelby S. Oakley, 202-512-3841 or oakleys@gao.gov

In addition to the contact named above, William Hoehn (Assistant 

Director), Danny Baez, Antoinette C. Capaccio, Penney Harwell Caramia, 

Tara Congdon, John Delicath, Rob Grace, Ben Licht, Alexandria Palmer, 

Timothy M. Persons, Steven Putansu, Ron Schwenn, Kiki 

Theodoropoulos, Jack Wang, and Tonya Woodbury made key 

contributions to this report.

Appendix VI: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(100473)
The Government Accountability Office, the audit, evaluation, and investigative 

arm of Congress, exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability of the 

federal government for the American people. GAO examines the use of public 

funds; evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through GAO’s website (http://www.gao.gov). Each weekday afternoon, GAO 

posts on its website newly released reports, testimony, and correspondence. To 

have GAO e-mail you a list of newly posted products, go to http://www.gao.gov

and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of production and 

distribution and depends on the number of pages in the publication and whether 

the publication is printed in color or black and white. Pricing and ordering 

information is posted on GAO’s website, http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, MasterCard, 

Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, LinkedIn, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.

Visit GAO on the web at www.gao.gov and read The Watchblog.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-4400, 

U.S. Government Accountability Office, 441 G Street NW, Room 7125, 

Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707

U.S. Government Accountability Office, 441 G Street NW, Room 7814, 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Strategic Planning and 

External Liaison

Please Print on Recycled Paper.