United States General Accounting Office

GAO Report to Congressional Requesters

November 1994 NATIONAL

LABORATORIES

Are Their R&D

Activities Related to

Commercial Product

Development?

GAO/PEMD-95-2

GAO United States

General Accounting Office

Washington, D.C. 20548

Program Evaluation and

Methodology Division

B-256574 

November 25, 1994

The Honorable Marilyn Lloyd

Chairman, Subcommittee on Energy

The Honorable Tim Valentine

Chairman, Subcommittee on Technology,

 Environment, and Aviation

The Honorable Tom Lewis

Ranking Minority Member, Subcommittee

 on Technology, Environment, and Aviation

Committee on Science, Space, and Technology

House of Representatives

You asked us to consider how the national laboratories of the U.S.

Department of Energy (DOE) can best be focused to help solve the

problems our nation faces during the current decade. As a beginning, we

have developed an inventory of the human and capital resources housed in

the national laboratories that will provide baseline data for future reports

on a number of DOE laboratory policy issues. This report addresses the

Committee’s interest in the current balance of the research effort in the 10

laboratories’ research programs. It examines the extent to which the

national laboratories are engaged now in basic and applied research or in

research related to commercial product development.

Background The Department of Energy’s multiprogram laboratories have had missions

that are national in scope since their inception during World War II. The

original laboratories—Lawrence Berkeley (Calif.), Los Alamos (N. Mex.),

and Oak Ridge (Tenn.)—were established as government-owned,

contractor-operated institutions to apply the productive capability of

private industry to the development of atomic weapons.1

 The

weapons-development mission continued during the cold war, and six

additional laboratories—Argonne (Ill.), Brookhaven (N.Y.), Sandia (N.

Mex. and Calif.), Idaho Engineering (Idaho), Lawrence Livermore (Calif.)

and Pacific Northwest (Wash.)—were created between 1946 and 1965 to

1

Ernest Orlando Lawrence founded the Radiation Laboratory for basic scientific research on the

University of California at Berkeley campus in 1931. It was funded under government contract in 1942.

The laboratory was renamed Lawrence Radiation Laboratory after his death in 1958 and later called

Lawrence Berkeley.

Page 1 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

foster civilian applications of nuclear technology.2 A 10th laboratory, the

Solar Energy Research Institute, was designated a national laboratory in

1991 to expand federal energy research and development (R&D) capability

in alternative energy sources, and it was renamed the National Renewable

Energy Laboratory (Colo.).3

 As a group, the 10 laboratories are known as

the national laboratories.

As the laboratories’ experience and research capability evolved, mission

emphases shifted among them. Sandia, Los Alamos, and Lawrence

Livermore acquired primary responsibility for nuclear weapons research

and development and the largest share of the laboratories’ funds.

Responsibility for research in the environmental and biological, energy,

and national security areas was distributed among all 10 laboratories to

varying degrees.4

 However, the Congress and DOE are reassessing this

mission configuration.

Since 1980, the Congress has had an active interest, expressed in a series

of laws, in seeing that more of the national laboratories’ outputs be put to

commercial uses.5

 Changing needs for defense technology resulting from

the end of the cold war and concern with maintaining U.S industry’s

competitiveness in global markets have led several members of Congress

to open a public debate and propose new legislation that addresses the

2

The Atomic Energy Act of 1946 delegated responsibility for nuclear weapons research, development,

and manufacture as well as civilian uses of nuclear energy to a civilian agency, the Atomic Energy

Commission. Historically, civilian agencies have held this responsibility in the United States and most

other countries.

3

The National Renewable Energy Laboratory was established in 1977 as the Solar Energy Research

Institute. One of DOE’s program-directed laboratories, its R&D activities focus on developing

competitive renewable energy and related technologies for the nation and on facilitating their

commercialization.

4

A brief description of the 10 national laboratories’ missions is provided in appendix I.

5

See Technology Transfer: Implementation Status of the Federal Technology Transfer Act of 1986

(GAO/RCED-89-154, May 30, 1989), Implementation of the Technology Transfer Act: A Preliminary

Assessment (GAO/T-PEMD-90-4, May 3, 1990), and Diffusing Innovations: Implementing the

Technology Transfer Act of 1986 (GAO/PEMD-91-23, May 29, 1991) for a discussion of congressional

interest in technology transfer from the national laboratories, as expressed in the Stevenson-Wydler

Technology Innovation Act of 1980 (P.L. 96-480) and the Federal Technology Transfer Act of 1986 (P.L.

99-502). The implementation of Executive Order No. 12591, “Facilitating Access to Science and

Technology,” which ordered executive departments and agencies to facilitate collaboration between

federal laboratories and other public and private sector organizations, also is considered in Diffusing

Innovations.

Note that this legislation and our reports concern all laboratories owned, leased, or otherwise used by

a federal agency. The 10 national laboratories are a subset of this population, which covers at least 10

executive branch departments and 297 laboratories. See also the National Competitiveness

Technology Transfer Act (P.L. 101-189), which in 1989 authorized DOE to establish technology transfer

as a mission of government-owned, contractor-operated laboratories, such as the national laboratories,

and to approve formation of cooperative research and development agreements (CRADAs) between

these laboratories and industry.

Page 2 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

national laboratories’ missions, structure, and cooperation with industry.6

Among the alternatives being considered in the public debate are reducing

all the laboratories’ budgets, consolidating or closing some of them, and

redirecting their weapons development mission toward commercial

product-related R&D in such areas as technology development for

environmental restoration, energy, and high-performance computing.

Underlying these discussions are questions about the type of R&D activities

the national laboratories are performing now, the nature and scope of

their outputs, and their potential for assisting industry in bringing

technology to the marketplace. This report is an effort to inform the

debate by providing an empirical base for these questions, as a starting

point for addressing the broader issues. It examines whether the balance

of laboratories’ effort is in basic and applied research or research related

to commercial product development, the distribution of the laboratories’

research outputs, and their potential for commercial application. Findings

were based on a cross-section of the laboratories’ R&D activities for the

period 1989-92. However, the objectives for most of the programs in the

study population were initiated before the national laboratories’ legislative

mandate for technology transfer in the National Competitiveness

Technology Transfer Act took effect in late 1989. In most fields of R&D,

more than 4 years are required for outputs to evolve after objectives have

been established. Therefore, the commercial product-related effort we

found is to be considered a baseline against which future activities and

outputs can be measured.

Methodology We began our work by developing a comprehensive description of current

research activities in the 10 laboratories. We chose to survey the

laboratories directly because we could find no sufficiently comprehensive

existing documentation. We collected our data through a survey of the 10

laboratories’ research programs and the facilities and equipment that

support them. The survey scope consisted of all major research programs

and facilities with costs of at least $10 million, as well as special

6

See U.S. Congress, “Department of Energy Laboratory Technology Act of 1993,” H.R.1432, sections 2,

4, 5, and 9.

Page 3 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

nominations by the laboratories themselves of other less costly programs

and facilities.7

These two criteria were designed to ensure that all large subprograms and

smaller subprograms that were important to the laboratories’ missions

would be included in our sample. This allowed us to describe the

laboratories’ major research efforts. However, findings based on these

criteria should not be considered representative of a laboratory’s entire

research effort since the proportion of programs budgeted at less than

$10 million can vary from one laboratory to another.

DOE’s Budget and Reporting System categories provided a common

classification scheme for the laboratories’ 12 research programs, which

permitted cross-laboratory comparisons of program characteristics.

Research program and subprogram names are shown in table 1. The data

we collected on these programs covered fiscal years 1989-92.

We conducted pilot tests of the survey methodology and data collection

instruments at Brookhaven and the National Renewable Energy

Laboratory. We then revised the instrument and administered one version

to the remaining eight laboratories.8

 After we processed the survey

responses, we asked each laboratory to confirm by letter that our list of

research programs and facilities was, in fact, complete.

7

A research program in the survey population is one of several broad areas of research activity taking

place within a laboratory that had a total annual budget equal to or exceeding $10 million, was funded

in fiscal year 1992, and was planned to continue in fiscal year 1993. A program that did not meet these

criteria could be nominated by the laboratory for inclusion in the survey on the basis of its uniqueness

or contribution to science and technology development. Each laboratory could nominate up to 10

programs in this category. A facility in the survey population is an entity that houses and comprises

the equipment used in conducting R&D. A facility could be a building or defined structure, some area

within a structure, or a defined area not confined to a structure (for example, a testing area). An

acquisition cost of $10 million or nomination by the laboratory for its uniqueness, world-class quality,

synergistic effect when combined with other facilities, or contribution to a given research program or

project qualified a facility for inclusion in the survey. Each laboratory could nominate up to 15

facilities that did not meet the $10-million acquisition cost criterion.

8

The national laboratory inventory data collection instrument has two parts—part I concerns the

laboratory’s research programs; part II requests information about a laboratory’s major research

facilities. This report contains data for eight of the 10 laboratories from part I of the revised data

collection instrument, which is reproduced in appendix II. Data reported for Brookhaven and the

National Renewable Energy Laboratory were collected with the pilot-test versions of the data

collection instrument, which were somewhat different from the instrument in appendix II.

Page 4 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

Table 1: Classification for National

Laboratory Research Programs Program Subprogram categorya

Energy Research Magnetic fusion (AT)

High-energy physics (KA)

Superconducting supercollider (KS)

Nuclear physics (KB)

Basic energy sciences (KC)

Biological and environmental (KP)

Conservation and Renewable

Energy

Electric energy systems (AK)

Geothermal (AM)

Solar energy (EB)

Building and community systems (EC)

Industrial Energy Conservation (ED)

Transportation (EE)

State and local programs (EF)

Environment, Safety and

Health

Environment, safety, and health (HA)

Nuclear Energy Nuclear energy R&D (AF)

Uranium enrichment (CD)

Defense Programs Weapons activities (GB) 

Verification and control technologies (GC)

Nuclear safeguards and security (GD)

Production and surveillance support to the nuclear

weapons complex (NM)

New Production Reactors New production reactors (NP)

Environmental Restoration

and Waste Management

Environmental restoration and waste

management—defense (EW, EM)

Environmental restoration and waste

management—nondefense (EX)

Fossil Energy Coal (AA)

Petroleum (AC)

Strategic petroleum reserve (SA)

Civilian Radioactive Waste

Management

Nuclear waste fund (DB)

Policy Planning and Analysis Policy, analysis, and systems studies (PE)

Intelligence Intelligence (NT)

Work for others Work for others (WFO)

aDOE’s Budget and Reporting System subprogram category codes are in parentheses. For

purposes of this project, we have used the code “WFO” to identify work for others programs.

Source: National laboratories’ institutional plans for fiscal years 1991-96.

The national laboratories engage in a wide range of defense and

nondefense R&D-related activities. These range from generating hypotheses

and testing fundamental science principles to assisting a potential user in

adapting laboratory outputs to a production or service delivery system. To

Page 5 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

analyze the extent to which the laboratories are engaged in basic and

applied research or research related to commercial product development,

we divided their activities into five categories: basic research, applied

research, development, technology transfer, and technical assistance.

Basic research is research undertaken primarily to gain fuller knowledge

or understanding of a subject and to contribute to the knowledge base in

the field of investigation. Applied research is research directed toward

the practical use of knowledge or understanding of a subject to meet a

recognized need. Development is research directed toward the

production of useful materials, devices, systems, or methods, including the

design and development of prototypes or processes. Development has

some type of product as the output goal, but may conclude with a

prototype rather than a usable good. Additional time, research, and testing

are usually required to convert the prototype to a weapon or commercially

viable product.

Because the national laboratories perform R&D only through the

development stage, additional mechanisms and arrangements are required

to achieve application of the laboratories’ outputs in the public or private

sector. These activities are technology transfer and technical assistance.

Technology transfer is the process that fosters the use of devices,

processes, “know-how,” or scientific and technical information produced

in a national laboratory by universities, private industry, or government

agencies. It includes making potential users aware of the laboratories’

research outputs, assisting in their selection or use, and collaborating with

representatives of private industry and public or nonprofit institutions to

ensure that some of the laboratories’ outputs will have commercial or

public applications. Technical assistance applies the laboratory’s

expertise to practical problems but does not involve the use of a

laboratory’s outputs. It is any form of assistance, other than financial, to a

state or local government or a business, including publications,

workshops, conferences, studies, or telephone consultation.

Development, technical assistance, and technology transfer are the three

national laboratory research activities related to commercial product

development. All five categories, already used by the laboratories but

specially grouped for our analysis, constitute a natural framework that,

together with DOE’s program classification scheme, allowed us to look at

R&D-related activity across all 10 laboratories, using expenditures as a

measure of activity.

Page 6 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

Recognizing that the laboratories do not maintain records of their R&D

expenditures in terms of our five categories, we asked managers of the

subprograms in our study population to estimate, for each subprogram

they managed, the proportion of funds expended in each of the five areas.

Our analysis of R&D activity is therefore presented as percentages, not

actual dollar values. To provide a context for considering our findings, we

present in table 2 the fiscal year 1992 budgets for subprograms in the study

population that were included in our analysis and the laboratories’ total

budgets in fiscal year 1992.

Table 2: National Laboratory Budgets

for Fiscal Year 1992 National laboratory R&D budgeta Total budgetb

Argonnne $357.8 $577.8

Lawrence Berkeley 193.3 270.6

Oak Ridge 346.6 726.9

Pacific Northwest 340.4 417.0

Idaho National Engineeringc 98.7 931.5

Lawrence Livermore 727.6 1395.5

Los Alamos 710.9 1239.6

Sandia 722.1 1389.6

Brookhaven 253.1 472.6

National Renewable Energy 113.4 131.2

Total $3,863.9 $7,552.3

aIn millions of dollars; includes salaries and wages, overhead, expendables, capital equipment,

and other factors for a laboratory’s subprograms in the study population. Construction costs are

not included. Dollar values are the total of a laboratory’s research subprograms’ budgets reported

by program managers.

bIn millions of dollars; includes salaries and wages, expendables, overhead, capital equipment,

general purpose equipment, construction, and all other factors for all of a laboratory’s research,

educational, administrative, and other activities. Dollar values are the estimated budget

authorization reported in the laboratory 5-year institutional plan.

cIdaho National Engineering Laboratory conducts energy, defense, environmental, and nuclear

reactor research; provides scientific training in nuclear reactors and waste management; and

designs, constructs, and operates defense production facilities. Idaho laboratory representatives

determined that much of their work in the environmental, nuclear energy, and defense areas did

not meet criteria for inclusion in the inventory. In addition, the total laboratory budget estimate in

the fiscal year 1992 institutional plan included $250 million for construction, which was not

included in the R&D budget for the study population.

We also examined the laboratories’ outputs. As output measures, we

selected products of laboratory R&D that were clearly identifiable to our

respondents and for which they were likely to maintain records. Since our

study objective was to examine the balance among laboratory activities

Page 7 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

rather than their impact, we focused on outputs of R&D activity that

occurred within the laboratories rather than their efforts at job creation or

increased sales. Because of great variation in the size, scope, field of

investigation and funding level of the subprograms in the study population,

both within and among laboratories, we presented our findings as simple

tabulations, rather than as standardized units. Use of a single measure for

standardizing the outputs, such as dollar of funding per output, would

have failed to account for variations among the subprograms on other

dimensions. Moreover, because of the institutional complexity this

variation represents, we interpreted our output findings very

conservatively, treating them as measures of activity rather than indicators

of performance. We looked at the outputs in two broad categories:

(1) publications and reports and (2) outputs related to commercial

product development. The outputs attributed to each category are

described in the Principal Findings section.

Finally, we looked at three other indicators—the formation of cooperative

R&D agreements, R&D effort devoted to critical technologies, and program

managers’ assessment of their on-going research—to gauge the

laboratories’ potential for commercial product development.

We conducted our review in accordance with generally accepted

government auditing standards.9

Results in Brief The national laboratories devoted slightly more than half (52.4 percent) of

their R&D funds to research related to commercial product development

during fiscal year 1992. This includes 30.9 percent for development,

14.4 percent for technical assistance, and 7 percent for technology

transfer. However, most of the 10 laboratories’ development work

(56.7 percent) was devoted to defense (which may have more limited

market opportunities) rather than nondefense research. Less than half

(44.6 percent) of the laboratories’ effort was spent on basic and applied

research: 17.4 percent on basic research and 27.2 percent on applied

research.

The 10 laboratories produced many more publications and reports

(21,593) than they did outputs related to commercial product development

(2,510) in fiscal year 1992. We expected this finding because publications

and reports are the primary mechanism for disseminating the results of all

types of R&D-related activities. Further, we found that the defense program

9

See appendix III for a more detailed discussion of survey methodology.

Page 8 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

supported most of the commercial product-related outputs, and that

Sandia, Los Alamos, and Lawrence Livermore—which are known as

weapons laboratories—produced most of these outputs. However,

because we asked the laboratories to classify these outputs—prototype

devices and materials, algorithms, software, and other commercial

products or processes that have an identified commercial use—at a time

that is still several years away from market entry, whether or to what

degree they will actually achieve commercial application is unknown.

With regard to the laboratories’ potential for commercial product

development, we found that such potential exists, based on the three

indicators we examined. Activity is increasing in the formation of

cooperative R&D agreements between the laboratories and industry. These

agreements increased from 17 in fiscal year 1989 to 196 in fiscal year 1992.

Of course, they ensure only that the laboratories and industry will

collaborate on R&D; a commercial innovation may or may not be produced.

With respect to the research emphasis selected, about three-fourths

(74.1 percent) of the 10 laboratories’ R&D expenditures were focused on

those technologies the National Critical Technologies Panel had identified

as vital to national needs.10 Here again, the potential for commercial

product development exists, but the actual outcome will not be known for

several years. Finally, over half (57.5 percent) of the managers of programs

with commercial product potential expected clear evidence of that

potential to emerge within 5 years or less from fiscal year 1992.

Principal Findings

Balance Among

R&D-Related Activities

To examine the balance of the national laboratories’ current R&D-related

activities, we analyzed the distribution of laboratory expenditures for R&D

within and among laboratories and research programs. For the 10

laboratories overall, R&D-related activity was almost evenly divided

between basic and applied research on the one hand, and research related

to commercial product development on the other. Approximately

8 percent more of the effort was devoted to R&D activities related to

commercial product development, as shown in figure 1. More applied

10The Congress established the National Critical Technologies Panel through the Fiscal Year 1990

Defense Authorization Act (P.L. 101-189), an amendment to the National Science and Technology

Policy, Organization, and Priorities Act of 1976. Congress asked the Panel to identify up to 30 areas of

technological development they consider essential for the nation’s long-term security and economic

prosperity. The Panel reports to the Congress and the President on the critical technologies biennially

through the year 2000.

Page 9 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

research than basic research was conducted: 27.2 percent versus

17.4 percent. Among research activities related to commercial product

development, most (30.9 percent) was development, but more activity was

devoted to technical assistance (14.4 percent) than technology transfer

(7 percent). Thus, R&D-related activity directly targeted on potential

commercial applications of the laboratories’ outputs currently constitutes

the smallest proportion of the laboratories’ R&D-related effort. Despite its

small size, however, this level of effort exceeds the laboratories’ minimum

statutory requirement for technology transfer activity.11

Figure 1: National Laboratory Mean

Percent Expenditures for R&D-Related

Activitiesa

52.4% • Research related to commercial

product developmentc

44.6%•

Basic and applied research

•

3.0%

Otherb

aFiscal year 1992.

bSubprogram expenditures for activities other than basic and applied research or research

related to commercial product development, such as training graduate students and postdoctoral

fellows or safety procedures.

cDevelopment, technical assistance, and technology transfer.

11The Stevenson-Wydler Act requires that each federal agency with one laboratory or more make

available at least one-half of 1 percent of its R&D budget for technology transfer activities and that

laboratories having 200 or more scientific, engineering, and technical full-time-equivalent (FTE) staff

assign at least one FTE to an Office of Research and Technology Applications, which has formal

responsibility for the laboratory’s technology transfer activities.

Page 10 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

These overall percentages, however, mask major differences among the

laboratories with regard to R&D funding distribution. (See table IV.1 in

appendix IV.) Four laboratories—Argonne, Lawrence Berkeley, Oak Ridge,

and Brookhaven—spent 25 percent or more of their research funds on

basic research. These laboratories account for over half (59.3 percent) of

the total national laboratory research budget that is spent on basic

research. (See table IV.2.) Los Alamos spent 19.4 percent of its R&D funds

to support its mission to perform “basic research in selected disciplines

that help maintain an outstanding science and technology base.” Only

about 10 percent or less of the laboratory research budget was spent on

basic research at the other laboratories. The energy research program

accounted for the greatest proportion of funds spent on basic research,

both within and among research program areas. (See tables IV.3 and IV.4.)

As table IV.1 shows, four laboratories—Oak Ridge, Pacific Northwest,

Lawrence Livermore, and Los Alamos—spent 29 percent or more of their

research funds on applied research. Among the 10 laboratories, Lawrence

Livermore and Los Alamos accounted for almost half (47.9 percent) of

applied research expenditures. (See table IV.2.) Most applied research was

supported by programs in the areas of defense, energy research, and work

for others. (See table IV.4.)

As noted earlier, most of the laboratories’ development work, the most

product-oriented of R&D activities, was devoted to defense, rather than

nondefense, research. Almost three-quarters (71.5 percent) of all the

laboratories’ development research was conducted at Lawrence

Livermore, Los Alamos, and Sandia. (See table IV.2.)12 In turn, the largest

share of development research was performed in the defense and nuclear

energy programs. (See tables IV.3 and IV.4.) Therefore, while it is true that

across the 10 laboratories, a greater proportion of research funding was

devoted to activities more closely related to commercial product

development than to basic and applied research, most of these funds

currently support defense research. To determine whether this research

will have commercial opportunities for use, we examined the national

laboratories’ outputs.

12Pacific Northwest and the National Renewable Energy Laboratory each spend almost one-third of

their research funds on development. However, these laboratories are funded at a substantially lower

level than the weapons laboratories.

Page 11 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

National Laboratories’

R&D Outputs

A second measure of the type of effort in which the national laboratories

are engaged—as between basic and applied research or research related

to commercial product development—is output. The laboratories produce

two major types of outputs: (1) publications and reports, and (2) outputs

related to commercial product development.13 Table 3 shows that, across a

4-year period, most of the laboratories’ outputs were publications and

reports. This finding was expected because reports and publications are

the primary mechanisms for diffusion of R&D findings, and they are

prepared at all stages of the R&D process. Reports, conference papers, and

published articles, which can be produced more quickly than books and

book chapters, substantially outnumber the latter.

As we discussed above, a slightly higher percentage of the laboratories’

expenditures was devoted to R&D activities related to commercial product

development than to basic and applied research; nevertheless, few of their

outputs were commercial product-related.14 Prototype devices and

materials, algorithms, and software are the largest number of outputs in

this group. These outputs tend to arise from the development stage of the

R&D process, which often occurs several years before production of a

marketable or usable good. Not all outputs of the development stage will,

of course, achieve commercial application.

Most of the prototype devices and materials, algorithms, and software, as

indicated in tables V.1 and V.2 in appendix V, were produced at the

weapons laboratories, and most were funded by DOE’s defense program.

Other outputs laboratory managers identified as commercial products or

commercial processes also tend to arise from the development stage.

Although they will require a substantial additional investment before they

are ready to market, these products or processes will more likely result in

actual commercial applications because a potential commercial use has

already been identified. Most of these outputs were produced by Los

Alamos, Sandia, and Pacific Northwest, and the defense program supports

most of the research that has led to these outputs. The point here is that

although defense-funded R&D has produced more outputs that could lead

to commercial products, whether these outputs will achieve commercial

application is still unknown.

13A third set of outputs, which are not related to these categories, is designated as “other.”

14Prototype devices and materials, algorithms, software, patents, licenses, commercial products,

commercial processes, and spin-off companies are defined as laboratory outputs related to

commercial product development. The research program managers have identified a potential

commercial use for commercial products and commercial processes, but these outputs have achieved

at most a precompetitive stage of development. We view patents, licenses, and spin-off companies as

evidence of intent to pursue a marketing strategy for a research output.

Page 12 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

Table 3: National Laboratories’ Research Program Outputs for Fiscal Years 1989-92

1992 1991 1990 1989 Total

Outputs

All 4

yearsa

Any 4

yearsb

All 4

yearsa

Any 4

yearsb

All 4

yearsa

Any 4

yearsb

All 4

yearsa

Any 4

yearsb

All 4

yearsa

Any 4

yearsb

Publications and reports

Books 88 94 120 121 181 181 120 120 509 516

Articles 6,153 6,471 6,330 6,387 5,612 5,652 5,959 5,970 24,054 24,480

Book chaptersc 355 371 269 271 321 322 280 280 1,225 1,244

Reports 6,245 6,802 5,416 5,542 4,761 4,851 4,497 4,533 20,919 21,728

Conference papers 7,237 7,855 6,812 6,931 6,498 6,572 6,421 6,449 26,968 27,807

Subtotal 20,078 21,593 18,947 19,252 17,373 17,578 17,277 17,352 73,675 75,775

Outputs related to commercial

product development

Prototypes 501 566 485 490 497 500 492 492 1,975 2,048

Algorithms 732 778 713 715 632 634 552 552 2,629 2,679

Software 508 554 417 422 350 381 314 315 1,589 1,669

Patentsc 159 237 139 199 145 203 148 166 591 805

Licenses 210 303 128 149 42 60 39 47 419 559

Commercial products 28 36 17 24 19 26 17 21 81 107

Commercial processes 22 29 13 20 11 18 14 18 60 86

Spin-off companyc 6 7 2 3 8 11 11 12 27 33

Subtotal 2,166 2,510 1,914 2,022 1,704 1,833 1,587 1,623 7,371 7,988

Other

New programsc 153 159 142 146 137 139 98 100 530 544

Invention disclosuresc 320 617 336 618 247 481 232 341 1,135 2,057

Otherd 510 523 472 483 412 421 404 414 1,798 1,841

Subtotal 983 1,299 950 1,247 796 1,041 734 855 3,463 4,442

aThese are outputs for subprograms that were in operation every year during fiscal years

1989-92.

bThese are outputs for subprograms that were initiated in any year during fiscal years 1989-92.

cResponses were not collected from Brookhaven National Laboratory and National Renewable

Energy Laboratory.

dResearch subprogram outputs other than those listed above, such as technical abstracts,

workshops for laboratory users, and an electronic bulletin board service.

Patent applications may be submitted for inventions throughout the entire

R&D process, but a license is usually acquired only when a decision to

market a technology has been made. The number of licenses awarded,

therefore, is a stronger measure of output activity related to commercial

Page 13 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

product development than the number of patents. A trend in the data

indicative of the laboratories’ production of outputs related to commercial

product development is the increase in the number of licenses awarded

during fiscal years 1989 through 1992. (See table 3.) In fiscal year 1992,

Sandia and Pacific Northwest awarded the most licenses, and most

licensed outputs were supported by defense program research. (See tables

V.1 and V.2.)

We expected to find that most commercial product-related outputs were

supported by research programs that spent most of their R&D funds for

development. However, the R&D expenditures of those programs that

supported the most outputs related to commercial product development

covered the range of R&D activities. We found that in fiscal year 1992, four

research programs—energy research, conservation and renewable energy,

defense, and work for others—supported most of the commercial

product-related outputs of all types and that, over 4 years, commercial

product-related output production had been increasing each year in three

of the programs, as shown in figure V.1. We also found that in fiscal year

1992, the largest proportion of expenditures in the defense and

conservation and renewable energy programs was for development.

As expected, the defense and conservation and renewable energy

programs supported more of the outputs specifically designated as

commercial products and processes than any of the 10 other research

programs. However, in looking more closely at these four programs, we

found some interesting differences. Work for others, which supports more

commercial product and process-type outputs than eight other programs,

devoted a slightly higher proportion of R&D expenditures to applied

research than to development. But in energy research, which supports

more commercial product- and process-related outputs than nine other

programs, the largest proportion of expenditures was for basic research.

(See table V.3.)

Laboratories’ Potential for

Commercial Product

Development

We looked at three indicators of the national laboratories’ potential for

commercial product development: (1) formation of cooperative research

and development agreements; (2) proportion of R&D expenditures in

critical technology areas; and (3) research program managers’ judgments

about their programs’ outputs. Of these three, the most frequently used

indicator of the national laboratories’ potential for commercial product

Page 14 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

development is the formation of CRADAs.

15 Here we found a major

increase in activity. The national laboratories reported that from fiscal

year 1989 through 1992, they entered into 196 CRADAs. Among programs in

the study population in operation all 4 years, the number of new CRADAs

formed increased from 17 in fiscal 1989 to 130 in fiscal 1992.

Sandia and Oak Ridge laboratories were most active in entering into

CRADAs. (See table VI.1. in appendix VI.) Most were formed for research

sponsored by programs in the defense and conservation and renewable

energy areas. (See table VI.2.) The greatest increase in CRADA formation

occurred at Sandia, where 74 CRADAs were in effect in fiscal year 1992.

Fifty-three of the CRADAs effective in fiscal year 1992 were sponsored by

the defense program technology transfer initiative at Sandia. This

subprogram was initiated in June 1990 to identify opportunities for

commercializing technologies produced by DOE-funded defense research

activities in such areas as advanced manufacturing and precision

engineering, materials and processes, advanced microelectronics and

photonics, and computer architecture and applications.

Although the national laboratories do not yet have a legislative mandate or

mission for research in the critical technologies, their research program

managers reported that 74.1 percent of R&D expenditures are devoted to

work in critical technology areas. This research was distributed over the

22 areas identified by the National Critical Technologies Panel, with the

greatest concentration in energy technologies (13.6 percent); pollution

minimization, remediation, and waste management (8.8 percent);

computer simulation and modeling (6.7 percent); and materials synthesis

and processing (6.2 percent). (See table VI.3.)

Work in these critical technology areas was distributed broadly among the

laboratories and research programs. Five laboratories—Argonne,

Lawrence Berkeley, Oak Ridge, Idaho, and Lawrence Livermore—devoted

approximately 20-30 percent of their research funds to energy

technologies. (See table VI.3.) Pacific Northwest expended the greatest

proportion of R&D funds (41.3 percent) on pollution minimization

15A CRADA is a contractual provision created to foster technology transfer from federal laboratories to

the private sector. Agreements can be formed with businesses as well as nonprofit organizations and

state and local government agencies. The National Competitiveness Technology Transfer Act of 1989

authorized government-owned, contractor-operated laboratories, such as the national laboratories, to

enter into CRADAs.

CRADA formation represents commercial product potential because it establishes the process

uniquely for conducting research related to commercial product development. The conclusion of the

period of performance does not guarantee that the research will have been completed or that a

market-ready product will have been developed.

Page 15 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

technologies. Idaho and Lawrence Livermore were most active in

computer simulation and modeling. Oak Ridge and Los Alamos devoted

the greatest percentage of effort to materials synthesis and processing. As

a group, the laboratories devoted approximately three-fourths of their R&D

expenditures to research in critical technology areas, but Sandia and Los

Alamos expended only about half of their resources on critical

technologies research. All of the research programs sponsored research in

critical technologies to some degree, with the least effort expended by

environment, safety, and health. (See table VI.4.)

Finally, laboratory research program managers’ judgments about their

research programs’ potential for commercial product development

were optimistic. Among the subset of all national laboratory programs

with a potential for commercial product development, almost 58 percent

of the program managers expected that development to occur within 5

years of fiscal 1992. (See figure VI.1.) An additional 27.6 percent reported

that their program has the potential for commercial product development

within 5-10 years.

Conclusions As of 1992, the national laboratories spent slightly more than half of their

R&D funds on research related to commercial product development.

However, most of this R&D was performed at the weapons laboratories and

was supported by the defense and nuclear energy programs. Analysis of

the outputs produced by the national laboratories indicated that

defense-funded research produced more outputs—prototype devices and

materials, algorithms, software, and other products and processes that

have an identified commercial application—that are precursors to

marketable goods, but at this point, whether they will achieve commercial

application is not known.

Moreover, three indicators of the laboratories’ potential for commercial

product development—CRADA formation, critical technology research, and

program managers’ expectations for commercial potential—showed that

some activity was occurring. CRADA formation was increasing, but these

arrangements ensure only that collaboration between the laboratories and

industry will occur, not that a commercial product will be generated.

Almost three-fourths of the laboratories’ effort was devoted to research in

critical technology areas, but achievement of commercial application will

not be known for several years. Over half of the managers of research

subprograms that have commercial product potential expected

innovations to arise within 5 years, but these expectations must be

Page 16 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

considered “best educated guesses.” While we can conclude, therefore,

that the national laboratories’ were engaged in slightly more research

related to commercial product development than basic and applied

research, it is too early to determine whether this activity will produce

technologies with commercial uses.

Agency Comments We requested comments on a draft report and received a response from

DOE and the 10 national laboratories. DOE questioned the definitions and

categories we defined to analyze the laboratories’ R&D-related activities

and our finding that the laboratories perform slightly more research

related to commercial product development than basic and applied

research. DOE also thought that this study should have examined additional

institutional factors, including the R&D activities of other agencies, and

should have used data maintained by DOE headquarters rather than

surveyed the laboratories for data.

We note that the definitions for R&D-related activities we employed are

derived from a Congressional Budget Office study of the federal R&D

enterprise, our study of the Technology Transfer Act of 1986, and expert

opinion. We also disagree with DOE’s proposed broader scope for this

study because it exceeds our study objective and would have required

additional data collection and analyses that are beyond the study scope.

Furthermore, our exploration of data available at DOE headquarters found

that it was not adequate to satisfy our information needs.

Eight laboratories agreed with the report’s objective, analyses, and

conclusions. However, one of this group, Lawrence Berkeley, thought that

the relationship of commercial product development to the broader needs

of industry and the nation should have been addressed in the study. Two

of the laboratories raised issues about study methodology. Idaho believed

that a greater proportion of the budget for its subprograms should have

been included in the study sample. Oak Ridge questioned the effect of the

study’s sampling methodology on output findings for the laboratory and

the definition of the category called outputs related to commercial product

development.

Lawrence Berkeley said that we had overlooked an important issue. The

laboratory thought that the study should have included an examination of

the relationship of the national laboratories’ role in commercial product

development to the broader needs of industry and the nation. We agree

that this issue is important to address as part of the public debate about

Page 17 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

the laboratories’ missions and structure. However, we disagree that it

should have been examined in this report, which focuses on establishing

an empirical baseline of national laboratories’ activities.

DOE’s Idaho Operations Office responded for Idaho National Engineering

Laboratory. The Idaho Operations Office said that the budget figure

reported for Idaho subprograms included in the study sample should have

been higher. We did not agree to revise Idaho’s budget figure, because to

do so would have violated the study methodology used to sample

programs at other laboratories.

Oak Ridge took the position that most of its commercial product-related

outputs were produced by subprograms that were not selected in the

study sample because they were funded at less than $10 million. The

laboratory expressed concern that the subprograms we sampled produced

only 7 percent of its commercial product-related outputs while

representing 73 percent of its overall budget. Oak Ridge based this

position on summary output data for the entire laboratory and sampled

subprograms that laboratory representatives had tabulated. Again, we

could not include the output data for Oak Ridge’s unsampled programs in

our analyses without violating the sampling methodology. We also had

some questions about the large number of outputs the Oak Ridge analysis

ascribed to unsampled programs.

Oak Ridge also thought that our definitions for these outputs equated the

laboratories’ development work with commercial product development.

We disagree. The definitions we used make it clear that the laboratories

were not expected to produce commercial products. Our conclusion

reiterates that the laboratories’ outputs related to commercial product

development are “precursors to marketable goods” and that “whether they

will achieve commercial application is not known.”

We provide a more detailed discussion of all these comments and our

response in appendixes VII through XVII.

As agreed with your offices, we plan no further distribution of this report

until 30 days from its date of issue, unless you publicly announce its

contents earlier. We will then send copies to interested parties, and we

will also make copies available to others upon request.

Page 18 GAO/PEMD-95-2 National Laboratories’ R&D Activities
B-256574 

If you have any questions or would like additional information, please call

me at (202) 512-3092. Other major contributors to this report are listed in

appendix XVIII.

Kwai-Cheung Chan

Director of Program Evaluation in

 Physical Systems Areas

Page 19 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Contents

Letter 1

Appendix I 

Description of the

National Laboratories

24

Appendix II 

National Laboratory

Inventory Research

Program Data

Collection Instrument

29

Appendix III 

Survey Methodology

48

Appendix IV 

Balance Among the

National Laboratories’

R&D Activities

50

Appendix V 

National Laboratories’

R&D Outputs

54

Appendix VI 

National Laboratories’

Potential for

Commercial Product

Development

60

Page 20 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Contents

Appendix VII 

Comments From the

Department of Energy

67

Appendix VIII 

Comments From

Argonne National

Laboratory

75

Appendix IX 

Comments From

Brookhaven National

Laboratory

76

Appendix X 

Comments From

DOE’s Idaho

Operations Office

79

Appendix XI 

Comments From

Lawrence Berkeley

Laboratory

81

Appendix XII 

Comments From

Lawrence Livermore

National Laboratory

85

Page 21 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Contents

Appendix XIII 

Comments From Los

Alamos National

Laboratory

87

Appendix XIV 

Comments From

National Renewable

Energy Laboratory

89

Appendix XV 

Comments From Oak

Ridge National

Laboratory

91

Appendix XVI 

Comments From

Pacific Northwest

Laboratory

94

Appendix XVII 

Comments From

Sandia National

Laboratories

95

Appendix XVIII 

Major Contributors to

This Report

97

Tables Table 1: Classification for National Laboratory Research

Programs

5

Table 2: National Laboratory Budgets for Fiscal Year 1992 7

Page 22 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Contents

Table 3: National Laboratories’ Research Program Outputs for

Fiscal Years 1989-92

13

Table IV.1: Mean Percent Expenditures for R&D-Related

Activities Within Laboratories

50

Table IV.2: Mean Percent Expenditures for R&D-Related

Activities Among Laboratories

51

Table IV.3: Mean Percent Expenditures for R&D-Related

Activities Within Programs

52

Table IV.4: Mean Percent Expenditures for R&D-Related

Activities Among Programs

53

Table V.1: National Laboratories’ Research Program Outputs by

Laboratory

55

Table V.2: National Laboratories’ Research Program Outputs by

Program

56

Table V.3: Proportion of Funds Spent by Programs With More

Than 100 Outputs Related to Commercial Product Development

58

Table VI.1: CRADAs in Effect at the National Laboratories 60

Table VI.2: CRADAs in Effect Within the National Laboratories by

Research Program

61

Table VI.3: Mean Percent Expenditures for Critical Technologies

Within Laboratories

62

Table VI.4: Mean Percent Expenditures for Critical Technologies

Within Programs

64

Figures Figure 1: National Laboratory Mean Percent Expenditures for

R&D-Related Activities

10

Figure V.1: Trends in Outputs Related to Commercial Product

Development

59

Figure VI.1: Program Managers’ Views of Programs’ Commercial

Product Potential

66

Abbreviations

AMTEX American Textile Partnership

AT&T American Telephone and Telegraph

CRADA Cooperative research and development agreement

DOE Department of Energy

FTE Full-time-equivalent

OSTP Office of Science and Technology Policy

PNGV Partnership for a New Generation of Vehicles

R&D Research and development

Page 23 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix I 

Description of the National Laboratories

The descriptions of the national laboratories are adapted from the 5-year

institutional plans that the laboratories update and issue annually and

from U.S. Department of Energy, Multiprogram Laboratories, 1979 to 1988,

A Decade of Change (Washington, D.C.: Apr. 1990).

Argonne National

Laboratory

Argonne was established in 1946. The University of Chicago operates the

laboratory, which develops and operates national facilities for use by

university, industry, and national laboratory groups; performs basic

research, technology-directed research and technology evaluations; and

conducts technology transfer through cooperative research, and

development agreements, sponsored research, staff exchanges, and

licensing of intellectual property or through the formation of new firms by

the laboratory’s Arch Development Corporation.

The laboratory’s basic research effort includes experimental and

theoretical research on fundamental problems in the physical, life, and

environmental sciences to advance scientific understanding and support

energy technology development. Argonne’s technology-directed research

includes conceptualization, design, and testing of advanced fission

reactors and other technologies for power applications in both the civilian

and defense sectors and investigations of strategies for overcoming

materials, chemical, and electrochemical barriers to the development of

these technologies. Argonne also supports DOE and, where appropriate,

other federal agencies in characterizing and evaluating nationally

important projects and technology options in terms of their environmental

cost or other implications.

Lawrence Berkeley

Laboratory

Lawrence Berkeley, founded in 1931 as the Radiation Laboratory by Ernest

Orlando Lawrence of the University of California at Berkeley, was one of

the original national laboratories. It was funded under government

contract in 1942. The University of California, which operates the

laboratory, renamed it the Lawrence Radiation Laboratory after his death

in 1958, and later called it Lawrence Berkeley. The laboratory conducts a

wide range of interdisciplinary research with core competencies in

biosciences and biotechnology; particle and photon beams; advanced

detector systems; characterization and synthesis of materials; chemical

dynamics, catalysis, and surface sciences; advanced techniques for energy

supply and energy efficiency; and environmental assessment and

remediation. It performs research in the energy, physical, and life sciences;

develops and operates national experimental facilities; fosters industry’s

Page 24 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix I 

Description of the National Laboratories

interactions with the laboratory’s research programs; and offers scientific

and engineering education programs.

The laboratory’s work in the energy sciences includes applied science,

such as the energy efficiency of buildings; chemical sciences, such as the

structure and reactivity of transient species; earth sciences, including

geophysical imaging methods, isotopic geochemistry and physicochemical

process investigation; and materials sciences, such as advanced ceramic,

metallic, and polymeric materials for electronic, magnetic, catalytic, and

structural applications. Accelerator and fusion research, nuclear science,

and physics are pursued in the general science area. Lawrence Berkeley’s

work in the life sciences includes cellular and molecular biology, chemical

biodynamics, and research medicine and radiation biophysics. This work

is supported by the laboratory’s scientific and technical resources in the

areas of engineering, information and computing sciences, and

occupational health.

Oak Ridge National

Laboratory

Oak Ridge was one of the original national laboratories. Now operated by

Martin Marietta Energy Systems, Oak Ridge was established in 1943. The

laboratory’s R&D activities are focused on basic and applied research,

technology development, and other technological challenges in areas that

include energy production and conservation technologies; experimental

and theoretical research in physical, chemical, materials, computational,

biomedical, earth, environmental, and social sciences; the design, building,

and operation of unique research facilities for the benefit of university,

industrial, and other federal agency and national laboratory researchers;

and the development of environmental protection and waste management

technologies. Oak Ridge also performs technology transfer and offers

educational services from the preschool through the postdoctoral level.

Pacific Northwest

Laboratory

Pacific Northwest was established in 1965. Battelle Memorial Institute now

operates the laboratory, which performs scientific research and rapid

technology development and deployment to meet national needs.

Laboratory efforts include molecular science, hazardous waste

characterization, global environmental studies, subsurface science,

biological systems, technical support for environmental policies and

procedures, federal infrastructure modernization, national security

technology, energy-efficient methods, advanced analytical methods,

materials research, magnetic fusion research, civilian nuclear waste

management, technical support for nuclear power plant operation, space

Page 25 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix I 

Description of the National Laboratories

exploration technology, fossil fuel technology, renewable energy sources,

energy policy analysis, and surveillance and oversight of operations at its

Hanford site.

Idaho National

Engineering

Laboratory

This laboratory was established in 1949. Three contractors operated the

laboratory during the time period of our study: Westinghouse Idaho

Nuclear Co., Rockwell-INEL, and EG&G Idaho. The laboratory’s areas of

primary emphasis are nuclear reactor technology R&D, defense

production-related support, waste management and environmental

restoration analysis, advanced energy production technology

development, and research and development on energy and environmental

issues, including performance testing of industry-developed electric

vehicles, small hydropower and geothermal power production, and fossil

energy research. Idaho also offers educational activities and performs

technology transfer.

Lawrence Livermore

National Laboratory

Lawrence Livermore was established in 1952. The University of California

operates the laboratory, which serves as a national resource in science

and engineering, focused on national security, energy, environment,

biomedicine, economic competitiveness, and science and mathematics

education, with a special responsibility for nuclear weapons. National

security has traditionally been a special focus of the laboratory’s research

and development effort. Lawrence Livermore’s major areas of activity have

included research, development, and testing for all phases of the nuclear

weapons life cycle; strategic defense research; arms control and treaty

verification technology; inertial confinement fusion; atomic vapor laser

isotope separation; magnetic fusion; other energy research; research in

biological, ecological, atmospheric, and geophysical sciences;

charged-particle beam and free-electron laser research; advanced laser

and optical technology applications; technology transfer; and science

education. The laboratory also participates in human genome research as

part of a nationally directed initiative.

Los Alamos National

Laboratory

Los Alamos, one of the original national laboratories, was established in

1943 and is operated by the University of California. Ensuring the nation’s

deterrence capability through nuclear weapons technology is the

laboratory’s primary focus. Los Alamos’ major R&D activities include

research, design, development, engineering, and testing of nuclear

warheads; maintenance and enhancement of the weapons technology base

Page 26 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix I 

Description of the National Laboratories

and warhead stockpile management; research, development, and testing

support for advanced nuclear directed-energy concepts; nuclear materials

R&D for the nuclear weapons program; nonnuclear strategic defense R&D

activities; advanced conventional munitions development and simulation;

verification and safeguards R&D; vulnerability, lethality, effects, and

countermeasures research; advanced defense technologies; intelligence

activities involving hardware analysis and technology security; weapons

and energy technology systems studies; and R&D in nonnuclear energy and

technology areas. The laboratory’s basic research activities in defense and

energy areas include atomic and molecular physics, bioscience, chemistry,

computational science and applied mathematics, geoscience, space

science, astrophysics, materials science, nuclear and particle physics,

plasma physics, fluids, and particle beams. Los Alamos also performs

technology transfer and offers science and engineering education

programs.

Sandia National

Laboratories

Sandia was established in 1949 under an agreement with AT&T to operate

the laboratory for the government as a public service on a nonprofit basis.

AT&T stepped out of this role in 1993. A contract was recently awarded to

Martin Marietta Corporation to operate the laboratories. Sandia’s major

areas of effort are nuclear weapons, arms control and treaty verification,

environmental restoration and waste management, energy supply and

conservation, advanced conventional military technologies, and other

programs in the national interest. The laboratories’ R&D activities in these

areas include research, development, and engineering associated with

advancing nuclear explosives to integrated, functional weapons for

Department of Defense weapon delivery systems; other defense programs,

including development of verification and control technologies to support

arms reduction and concepts and systems for the safeguarding and

security of nuclear materials; research, development, and engineering for

hazardous waste removal, minimization, and remediation; and nonnuclear

energy research in energy efficiency, recovery techniques, conversion

technologies, alternative energy sources, characterization of

environmental change phenomena, environmental restoration

technologies, and basic energy sciences. Sandia also conducts technology

transfer and offers mathematics and science education opportunities.

Brookhaven National

Laboratory

Brookhaven was established in 1947 by a group of nine universities to

facilitate their mutual access to large-scale research facilities, particularly

in nuclear science. The laboratory is operated by Associated Universities,

Page 27 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix I 

Description of the National Laboratories

a corporation governed by a board of trustees representing the original

nine universities as well as other universities, research institutions, and

industrial organizations. Brookhaven’s primary role is to conceive, design,

build, and operate large-scale, complex facilities for scientific research

and to conduct basic and applied research in energy-related physical, life,

and environmental sciences. When feasible, Brookhaven makes its

laboratory facilities available to state and federal agencies, universities,

and private industry. The laboratory’s major areas of R&D are high-energy

and nuclear physics; basic energy sciences emphasizing research on

biological, chemical, and physical phenomena underlying energy-related

transfer, conversion, and storage systems; life sciences, nuclear medicine,

and medical applications of nuclear techniques; and a broad span of

applied programs that draw on the laboratory’s unique capabilities.

Brookhaven makes all useful results and knowledge obtained from its

research activities available to private industry. Brookhaven also performs

technology transfer and offers science and engineering education

programs.

National Renewable

Energy Laboratory

The former Solar Energy Research Institute was designated a DOE national

laboratory in 1991 and renamed the National Renewable Energy

Laboratory. The focus of the laboratory’s effort is on developing

competitive renewable energy and related technologies and facilitating

their commercialization. The laboratory’s R&D activities include basic and

applied research, exploratory and advanced development and other

activities in renewable energy and related technologies; analytic studies

and technology evaluations; and collaborative R&D with universities and

industry. The laboratory also manages subcontracted R&D on behalf of DOE

and serves as a source of scientific and technical information on

renewable energy.

Page 28 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

Page 29 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

Please return the completed questionnaire in the enclosed envelope within 10 working days of receipt. In

the event that the enclosed envelope is misplaced, please mail the questionnaire to:

Nancy Briggs, Ph.D

Project Manager

U.S. General Accounting Office

Program Evaluation and Methodology Division

Room 5853

441 G Street St., N.W.

Washington, DC 20548

Thank you in advance for your cooperation and assistance in addressing an issue of such critical importance

to the nation. We will send a report on the analysis of the information to the Congress and you.

I-2

Page 30 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

DEFINITIONS

The definitions listed below are included to provide a common frame of reference for responding to the

survey.

Applied Research: A study directed toward the practical use of knowledge or understanding of a subject to meet a recognized need.

Basic Research: A study undertaken primarily to gain fuller knowledge or understanding of a subject and to

contribute to the knowledge base in the field of investigation.

Capital Equipment Budget: For a national laboratory, this includes the budget for both research equipment

and general purpose equipment.

Cooperative Research and Development Agreement (CRDA): A contractual form created for the express purpose

of fostering technology transfer from the federal domain to the private sector.

Development: Research directed toward the production of useful materials, devices, systems, or methods, including the design and development of prototypes and processes.

Facility: An entity that comprises the equipment used for research programs. A facility may include a

building or defined structure, some area within a structure, or a defined area not confined to a structure

(for example, a testing area).

Laboratory: A group of facilities owned, leased, or otherwise used by the U.S. Department of Energy to

perform research and development. A laboratory consists of land (including, but not limited to, remote

testing areas), buildings, human resources, research programs, and equipment.

Mission: The primary scientific and technical research programs that a laboratory pursues.

Operating Budget: For a national laboratory this comprises research and development program costs,

including salaries and wages, expendables and overhead.

Research and Development: Intensive, systematic study directed toward fuller scientific knowledge or

understanding of the subject under investigation; practical use of knowledge; or the production of

materials, devices, systems, or methods. Research and development includes basic research, applied

research, and development.

Research Program: One of several broad areas of research activity within a laboratory’s mission.

Technology: Devices, processes, "know how," or scientific and technical information produced through the

research and development process.

Technology Transfer: The process that fosters the use of devices, processes, "know how," or scientific and

technical information produced in a federal laboratory by universities, private industry, or government

agencies, whether national (federal, state, or local) or foreign.

Total Budget: For a national laboratory, this includes operating, capital equipment, and construction

costs.

User Facility: A federal laboratory facility available for use either free of charge or on a costreimbursed basis by investigators from private industry, academic institutions, or state and local

government agencies.

I-3

Page 31 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

PROGRAM AND SUBPROGRAM DESCRIPTION

1. Please identify the defense subprogram for which you are reporting. Report only for a subprogram funded by the laboratory’s

operating budget during fiscal year 1992 and which is on-going for fiscal year 1993. (Check only one.)

Defense

a. GB-011 Weapons RD&T

b. GB-02 Inertial Confinement Fusion (Guidance)

c. GB-02 Inertial Confinement Fusion (Required)

d. GB-05 Program Direction

e. GC Verification and Control Total

f. Other (Please specify)

___________________________________

1

U.S. Department of Energy Budget and Reporting and System code.

I-4

Page 32 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

2. Please describe the subprogram’s research objectives and the major scientific and technical areas the subprogram addresses.

___________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

__________________________________________________________________________________________________________

_________________________________________________________________________________________________________

________________________________________________________________________________________________________

________________________________________________________________________________________________________

I-5

Page 33 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

3. What is the time period of performance for this subprogram? (Write in the subprogram start date and end-date below).

Subprogram start date ________________/_________________/__________________

(month) (Day) (Year)

Subprogram end date _______________/__________________/___________________

(month) (Day) (Year)

I-6

Page 34 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

4. How important, if at all, is each of the research and development-related activities listed below to your subprogram

objectives? (Check the relevant space.)

Research and development-related

activity

Importance

Little

importance (1)

Somewhat

important (2)

Moderately

important (3)

Very

important (4)

Essential

(5)

Not part

of mission (6)

a. Basic research1

b. Applied research2

c. Development3

d. Providing technical assistance to

government agencies

e. Providing technical assistance to

universities

f. Providing technical assistance to

private firms or industrial

organizations

g. Transfer technology from this

laboratory to U.S. government

organizations

h. Transfer technology from this

laboratory to foreign government organizations

i. Transfer technology from this

laboratory to U.S.4 firms or industrial

organizations

j. Transfer technology from this

laboratory to foreign5 firms or

industrial organizations

k. Other (Please specify)

_____________________________________

_____________________________________

_____________________________________

_____________________________________

_____________________________________

1

Basic Research: A study undertaken primarily to gain fuller knowledge or understanding of a subject and to contribute to the

knowledge base in the field of investigation.

2

Applied Research: A study directed toward the practical use of knowledge or understanding of a subject to meet a recognized need.

3

Development: Research directed toward the production of useful materials, devices, systems, or methods, including the design and development of prototypes and processes.

4

Majority-owned by U.S. citizens

5

Majority-owned by non-U.S. citizens

I-7

Page 35 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

5. In your estimation, what percent, if any, of your total research subprogram budget was expended each year during fiscal years

1989-92 for each of the research and development-related activities listed below? (Annual total should equal 100 percent. Write

"N/A" if the program was not in operation in given year.)

Research and development-related

activity

1992 1991 1990 1989

a. Basic research1

b. Applied research2

c. Development3

d. Providing technical

assistance to government

agencies

e. Providing technical

assistance to universities

f. Providing technical

assistance to private firms or

industrial organizations

g. Transfer technology from

this laboratory to U.S.

government organizations

h. Transfer technology from

this laboratory to foreign

government organizations

i. Transfer technology from

this laboratory to U.S.4 firms or

industrial organizations

j. Transfer technology from

this laboratory to foreign5 firms

or industrial organizations

k. Other (Please specify)

______________________________

______________________________

______________________________

______________________________

l. Total 100% 100% 100% 100%

1

Basic Research: A study undertaken primarily to gain fuller knowledge or understanding of a subject and to contribute to the

knowledge base in the field of investigation.

2

Applied Research: A study directed toward the practical use of knowledge or understanding of a subject to meet a recognized need.

3

Development: Research directed toward the production of useful materials, devices, systems, or methods, including the design and development of prototypes and processes.

4

Majority-owned by U.S. citizens

5

Majority-owned by non-U.S. citizens

I-8

Page 36 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

6. We identified the technologies listed below using the Report of the National Critical Technologies Panel. During fiscal years 1989-92, which of these technologies did the research of this subprogram support? In your estimation what percentage of the

research subprograms budget was spent each year for research about these technologies? (Check all that apply. Write in the

estimated percentage for activities checked.)

1992 1991 1990 1989

a. Materials synthesis and

processing _________________ ________________ _______________ _______________

b. Electronic and photonic

materials _________________ ________________ _______________ _______________

c. Ceramics _________________ ________________ _______________ _______________

d. Composites _________________ ________________ _______________ _______________

e. High-performance metals and

alloys ________________ _________________ _______________ _______________

f. Flexible computer

integrated manufacturing ________________ _________________ _______________ _______________

g. Intelligent processing equipment ________________ _________________ _______________ _______________

h. Micro - and nanofabrication ________________ _________________ _______________ _______________

i. System management

technologies ________________ _________________ ______________ _______________

j. Software ________________ _________________ ______________ _______________

k. Microelectronics and optoelectronics ________________ _________________ ______________ _______________

l. High-performance computing ________________ _________________ ______________ _______________

and networking

m. High definition imaging ________________ _________________ _____________ _______________

and displays

n. Sensors and signal processing ________________ _________________ _____________ _______________

o. Data storage and

peripherals ________________ _________________ _____________ _______________

p. Computer simulation and

modeling ________________ _________________ _____________ _______________

I-9

Page 37 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

q. Applied molecular biology ________________ ________________ _____________ _______________

r. Medical technology ________________ ________________ ______________ _______________

s. Aeronautics ________________ ________________ ______________ _______________

t. Surface transportation _______________ _______________ ______________ _______________ technologies

u. Energy technologies _______________ _______________ ______________ _______________

v. Pollution minimization, remediation, and waste _______________ _______________ ______________ _______________

management

w. Other (Please specify) _______________ _______________ ______________ _______________

______________________

I-10

Page 38 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

7. How many of the research and development-related outputs listed below did your subprogram produce each year from fiscal year

1989 to 1992? Write "N/A" if the program was not in operation in a given year.)

1992 1991 1990 1989

a. Published books ____________ ___________ ___________ ___________

b. Published articles ____________ ___________ ___________ ___________

c. Published book chapters ____________ ___________ ___________ ___________

d. Technical and scientific reports/monographs for internal

use only

___________ ___________ ___________ ___________

e. Technical and scientific

reports/monographs for release

to others outside the

laboratory ___________ ____________ ____________ ____________

f. Prototype devices and materials ___________ ____________ ____________ ____________

g. Papers for presentation at professional conferences ___________ ____________ ____________ ____________

h. New program ___________ ____________ ____________ ____________

i. Algorithms ____________ ____________ ____________ ____________

j. Software ____________ ____________ ____________ ____________

k. Invention disclosures ____________ ____________ ____________ ____________

l. Patents ____________ ____________ _____________ ___________

m. Licenses ____________ ____________ ____________ ___________

n. Commercial products ____________ ____________ ____________ ____________

o. Commercial processes ____________ ____________ ____________ ____________

p. Spin-off company ____________ ____________ ____________ ____________

q. Other (Please specify) ____________ ____________ _____________ ____________

______________________

I-11

Page 39 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

8. How much royalty income was earned each year during fiscal years 1989-92 by technologies supported by this subprogram? Show

dollars in millions. (Write "N/A" if the program was not in operation in a given year.)

1992 1991 1990 1989

Royalty income earned $_______________ _______________ _______________ ______________

9. What was the total number of CRDAs and "Work for Others" contracts in effect for this subprogram each year during fiscal years

1989-92?

1992 1991 1990 1989

a. CRDAs _____________ ____________ _____________ ______________

Work for others

b. Federal agency other

than DOE _____________ ____________ ______________ ______________

c. Industry _____________ ____________ ______________ ______________

d. Foreign governments _____________ ____________ ______________ ______________

e. Other sources _____________ ____________ ______________ ______________

I-12

Page 40 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

10. During what time period, if at all, does this research subprogram have the potential for industrial application or commercial

product development (Check only one).

a. Immediate future (Less than 5 years)

b. Short-term future (Next 5-10 years)

c. Long-term future (Over 10-20 years)

d. Very long-term future (Over 20 years)

I-13

Page 41 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

11. Is any university or private industry working in cooperation with your research subprogram?

a. Yes

b. No

If yes, please provide the following information:

c. Organization name: ____________________________________________

d. Address: ______________________________________________________

e. Contact person: _______________________________________________

f. Telephone number: _____________________________________________

g. Research topic of cooperative effort: ____________________________________________________________

Please also indicate if this is a U.S. or foreign government, university or firm. (Check the relevant space.)

Country

Organization

Government Agency

(1)

University

(2)

Firm or Company

(3)

h. U.S.

i. Foreign

If there is more than one organization working in cooperation with your subprogram, please photocopy this page and provide this

information for all organizations.

I-14

Page 42 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

12. If possible, please identify up to three other U.S. or foreign government, university, or industrial research institutions that in your view are working on scientific and technical problems similar to those addressed by this research subprogram.

Name of scientific and technical

problem

Name of other government,

university or industrial R&D

institution

Contact person’s name and

telephone number

a.

________________________________

________________________________

________________________________

________________________________

a. a.

b. b.

c. c.

b.

_______________________________

_______________________________

_______________________________

_______________________________

a. a.

b. b.

c. c.

c.

_______________________________

_______________________________

_______________________________

_______________________________

a. a.

b. b.

c. c.

I-15

Page 43 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

13. If possible please identify one or two administrators of research programs in U.S. government, university, or industrial research institutions who are familiar with the scientific and technical aspects of your research subprogram as well as the

facilities and equipment that support it. Please do not list anyone who is affiliated with programs in the U.S. Department of

Energy.

a. Research administrator’s name and title

Name: _____________________________________________

Title: ____________________________________________

Organization name, address and phone number

Name: ______________________________________________

Address: ___________________________________________

Telephone no: ______________________________________

FAX no: ____________________________________________

b. Research administrator’s name and title

Name: _____________________________________________

Title: ____________________________________________

Organization name, address, and phone number

Name: ______________________________________________

Address: ___________________________________________

Telephone no: ______________________________________

FAX no: ____________________________________________

I-16

Page 44 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

PROGRAM BUDGET

(Dollars in millions)

1992 1991 1990 1989

14. What was your research subprogram’s total annual

budget during fiscal years 1989-92? Show dollars in

millions. (Write the dollar amount in each column.

Write "N/A" if the subprogram was not in operation in

a given year.)

Operating

a. Salaries and wages ________ ________ _________ ________

b. All other ________ ________ _________ ________

c. Capital equipment ________ ________ _________ ________

d. Construction ________ ________ _________ ________

e. Other (Please specify)

_______________________________

_______________________________

_______________________________ _______ ________ _________ _________

15. What percentage of the research subprogram’s total annual budget was contributed by each of the following sources during

fiscal years 1989-92? (Write "N/A" if the subprogram was not in operation in a given year. Annual total should equal 100

percent.)

1992 1991 1990 1989

a. Contracts from DOE _______ _______ _______ _______

Work for Others

b. Federal agencies other than DOE _______ _______ _______ _______

c. U.S. industries _______ _______ _______ _______

d. Foreign industry _______ _______ _______ _______

I-17

Page 45 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

15. (Continued) What percentage of the research subprogram’s total annual budget was contributed by each of the

following sources during fiscal years 1989-92 (Annual total should equal 100 percent.)

e. Foreign government _______ _______ _______ _______

f. Other sources _______ _______ _______ _______

Total 100% 100% 100% 100%

I-18

Page 46 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix II 

National Laboratory Inventory Research

Program Data Collection Instrument

PERSONNEL

16. How many workers are employed annually by your research subprogram in each of the following job categories and what is the

total employment? Please provide both the number of full-time personnel and the number of full-time equivalent (FTE) staff years

for each year during fiscal years 1989-92.

Job category

1992 1991 1990 1989

Number of

personnel

FTEs Number of

personnel

FTEs Number of

personnel

FTEs Number of

personnel

FTEs

a. Administrators (not

involved in research)

b. Scientists, engineers,

and other researchers

(including research

administrators directly

involved in research)

c. Technicians supporting

research (through testing,

inspection, maintenance, or

construction of research

equipment, computer

programming)

d. Clerical maintenance and

other support personnel

e. Other (Please specify)

________________________

________________________

________________________

________________________

________________________

________________________

f. Total employees

THANK YOU FOR YOUR PARTICIPATION

I-19

Page 47 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix III 

Survey Methodology

Survey Response The research program survey population was enumerated by applying

selection criteria to each laboratory’s research programs. After processing

the surveys, we sent the laboratories a letter requesting confirmation that

our list of research programs and subprograms was complete. In response

to our letter, the laboratories confirmed a total of 252 research

subprograms. The laboratories returned a total of 247 data collection

instruments, for a survey response rate of 98 percent.

Data Quality Issues The data contained in this report are results of analyses of national

laboratory program managers’ responses to questions 5, 6, 7, 9, and 10 in

part I of the national laboratory inventory. These responses represent

program managers’ judgments or self-reports about question elements, as

follows. We made no attempt to validate these responses through

independent sources.

Question 5 Responses are research program managers’ best estimates of the

proportion of the total program budget expended for each R&D-related

activity. Although they had our definitions for key R&D-related activities

listed in the data collection instrument, their responses also may reflect

their own understanding of terms such as basic research, applied research,

or technical assistance.

Question 6 Responses are research program managers’ best estimates of the

proportion of the total program budget expended for research in critical

technology areas. The response categories in the question are the critical

technologies identified by the National Critical Technologies Panel. Some

overlap may exist among these categories because they were not identified

for research measurement purposes. The Panel’s critical technology

categories were used in this question to determine the congruence

between research already being conducted at the national laboratories and

the research needs articulated by a congressionally mandated body.

A few responses submitted for this question summed to more than

100 percent. These responses were prorated to include them in the

calculation of mean percent expenditures for R&D in critical technologies.

Question 7 Responses are research program managers’ reports of research program

outputs. The responses concerning commercial products and commercial

Page 48 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix III 

Survey Methodology

processes are judgments made about research outputs that have reached

only the precompetitive stage of the R&D process.

Question 9 Responses are research program managers’ reports about CRADAs in effect

through the end of fiscal year 1992.

Question 10 Responses are research program managers’ judgments about potential

industrial application or commercial product development for outputs of

their research program over a 20-year planning horizon. The size of

research subprograms in the study population varied; thus, managers were

considering outputs of one or more research activities in making their

assessments.

Page 49 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IV 

Balance Among the National Laboratories’

R&D Activities

Data concerning the distribution of the national laboratories’ expenditures

among R&D-related activities, by laboratory and by program, are presented

below.

Table IV.1: Mean Percent Expenditures for R&D-Related Activities Within Laboratoriesa

R&D-related activity ANL LBL ORNL PNL INEL LLNL LANL SNL BNL NREL All labs

Basic research 29.9 39.7 25.1 9.8 2.2 7.6 19.4 4.9 53.6 6.9 17.4

Applied research 22.5 15.9 29.1 29.7 20.6 36.2 33.1 23.5 7.5 24.3 27.2

Development 17.4 7.4 17.9 31.8 25.5 43.6 25.9 47.7 13.9 31.9 30.9

Technical assistance to

Government agencies 10.8 4.3 5.9 21.9 15.8 6.2 8.6 11.9 7.8 4.5 9.7

Universities 3.5 11.0 6.2 1.0 1.6 0.6 1.4 0.9 5.7 3.3 2.5

Private firms or industrial

organizations 1.6 3.5 5.0 1.2 1.4 0.8 1.7 2.0 2.9 7.9 2.2

Technology transfer to

Government agencies 4.0 10.4 4.5 2.4 7.0 1.9 2.5 3.1 1.6 1.7 3.2

Private firms or industrial

organizations 6.2 1.7 6.1 2.1 4.6 2.1 3.3 3.9 0.9 18.1 3.8

Otherb 4.1 6.1 0.2 0 21.3 1.0 4.1 2.1 6.2 1.5 3.0

Total 100 100 100 100 100 100 100 100 100 100 100

Legend

ANL = Argonne National Laboratory

LBL = Lawrence Berkeley Laboratory

ORNL = Oak Ridge National Laboratory

PNL = Pacific Northwest Laboratory

INEL = Idaho National Engineering Laboratory

LLNL = Lawrence Livermore National Laboratory

LANL = Los Alamos National Laboratory

SNL = Sandia National Laboratories

BNL = Brookhaven National Laboratory

NREL = National Renewable Energy Laboratory

aFiscal year 1992.

bSubprogram expenditures for activities other than those listed above, such as training graduate

students and postdoctoral fellows or safety procedures.

Page 50 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IV 

Balance Among the National Laboratories’

R&D Activities

Table IV.2: Mean Percent Expenditures for R&D-Related Activities Among Laboratoriesa

R&D-related activity ANL LBL ORNL PNL INEL LLNL LANL SNL BNL NREL Total

Basic research 14.9 10.9 13.1 5.0 0.3 8.3 20.7 5.3 20.4 1.2 100

Applied research 7.2 2.8 9.7 9.6 2.0 25.3 22.6 16.3 1.8 2.7 100

Development 5.0 1.1 5.2 9.1 2.2 26.8 15.6 29.1 3.0 3.1 100

Technical assistance to

Government agencies 9.7 2.1 5.6 20.0 4.2 12.1 16.5 23.2 5.3 1.4 100

Universities 12.1 20.9 22.2 3.3 1.6 4.3 10.2 6.9 14.8 3.9 100

Private firms or industrial

organizations 6.4 7.6 20.8 4.8 1.7 7.3 14.7 17.1 9.0 10.7 100

Technology transfer to

Government agencies 10.8 15.4 12.6 6.7 5.6 11.4 14.5 18.2 3.3 1.6 100

Private firms or industrial

organizations 14.2 2.2 14.5 4.8 3.1 10.4 16.0 19.3 1.6 14.0 100

Otherb 11.9 9.6 0.7 0.0 18.1 6.4 25.2 13.1 13.6 1.5 100

All activity 8.7 4.8 9.1 8.8 2.6 19.0 18.6 18.9 6.6 3.0 100

Legend

ANL = Argonne National Laboratory

LBL = Lawrence Berkeley Laboratory

ORNL = Oak Ridge National Laboratory

PNL = Pacific Northwest Laboratory

INEL = Idaho National Engineering Laboratory

LLNL = Lawrence Livermore National Laboratory

LANL = Los Alamos National Laboratory

SNL = Sandia National Laboratories

BNL = Brookhaven National Laboratory

NREL = National Renewable Energy Laboratory

aFiscal year 1992.

bSubprogram expenditures for activities other than those listed above, such as training graduate

students and postdoctoral fellows or safety procedures.

Page 51 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IV 

Balance Among the National Laboratories’

R&D Activities

Table IV.3: Mean Percent Expenditures for R&D-Related Activities Within Programsa

R&D-related activity ER CE ES&H NE DP NPR ERWM FE CRWM PPA INT WFO

All

programs

Basic research 53.1 4.4 0 7.6 7.2 0 0.3 14.1 8.2 3.3 2.0 11.5 17.4

Applied research 15.2 25.9 8.8 29.4 32.5 7.6 32.4 31.5 52.8 43.3 50.0 26.1 27.2

Development 10.1 28.6 9.8 51.7 40.9 27.7 37.3 15.8 17.3 6.7 15.0 23.4 30.9

Technical assistance to

Government

agencies 4.0 5.9 71.8 1.6 9.7 0 10.1 10.3 11.6 46.7 25.0 25.1 9.7

Universities 6.5 4.0 2.7 0.6 0.7 0 1.2 2.9 0.4 0 0 2.8 2.5

Private firms or

industrial

organizations 2.8 11.6 0.1 0.6 0.9 0 0.7 5.8 0 0 0 2.2 2.2

Technology transfer to

Government

agencies 4.1 3.1 2.4 2.0 2.8 0 3.3 7.7 1.7 0 8.0 4.3 3.2

Private firms or

industrial

organizations 2.5 15.7 0.7 6.5 2.5 0 2.8 7.1 0 0 0 4.4 3.8

Otherb 1.7 0.7 3.7 0 2.8 64.7 11.8 4.9 8.0 0 0 0.4 3.0

Total 100 100 100 100 100 100 100 100 100 100 100 100 100

Legend

ER = Energy Research

CE = Conservation and Renewable Energy

ES&H = Environment, Safety and Health

NE = Nuclear Energy

DP = Defense Programs

NPR = New Production Reactors

ERWM = Environmental Restoration and Waste Management

FE = Fossil Energy

CRWM = Civilian Radioactive Waste Management

PPA = Policy Planning and Analysis

INT = Intelligence

WFO = Work for Others

aFiscal year 1992.

bSubprogram expenditures for activities other than those listed above, such as training graduate

students and postdoctoral fellows or safety procedures.

Page 52 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IV 

Balance Among the National Laboratories’

R&D Activities

Table IV.4: Mean Percent Expenditures for R&D-Related Activities Among Programsa

R&D-related activity ER CE ES&H NE DP NPR ERWM FE CRWM PPA INT WFO Total

Basic research 68.7 1.7 0 3.0 17.8 0 0.1 0.2 0.6 0 0 7.7 100

Applied research 12.6 6.4 0.1 7.3 51.3 0.2 7.9 0.3 2.4 0.1 0.1 11.2 100

Development 7.3 6.2 0.1 11.3 56.7 0.6 8.0 0.1 0.7 0 0 8.9 100

Technical assistance to

Government agencies 9.4 4.1 3.0 1.1 43.1 0 6.9 0.3 1.5 0.3 0.1 30.3 100

Universities 57.9 10.8 0.4 1.6 12.5 0 3.2 0.3 0.2 0 0 12.8 100

Private firms or industrial

organizations 29.1 36.3 0 1.9 17.6 0 2.3 0.7 0 0 0 12.0 100

Technology transfer to

Government agencies 28.4 6.4 0.3 4.1 36.9 0 6.9 0.6 0.6 0 0.1 15.5 100

Private firms or industrial

organizations 14.5 27.8 0.1 11.5 27.5 0 4.8 0.5 0 0 0 13.4 100

Otherb 13.0 1.6 0.5 0 39.2 14.6 25.9 0.4 3.3 0 0 1.4 100

All activity 22.5 6.8 0.4 6.8 42.9 0.7 6.6 0.3 1.2 0.1 0 11.7 100

Legend

ER = Energy Research

CE = Conservation and Renewable Energy

ES&H = Environment, Safety and Health

NE = Nuclear Energy

DP = Defense Programs

NPR = New Production Reactors

ERWM = Environmental Restoration and Waste Management

FE = Fossil Energy

CRWM = Civilian Radioactive Waste Management

PPA = Policy Planning and Analysis

INT = Intelligence

WFO = Work for Others

aFiscal year 1992.

bSubprogram expenditures for activities other than those listed above, such as training graduate

students and postdoctoral fellows or safety procedures.

Page 53 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix V 

National Laboratories’ R&D Outputs

Data concerning outputs of the national laboratories’ R&D-related activities,

by laboratory and by program, are presented below.

Page 54 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix V 

National Laboratories’ R&D Outputs

Table V.1: National Laboratories’ Research Program Outputs by Laboratorya

Outputs ANL LBL ORNL PNL INEL LLNL LANL SNL BNL NREL Total

Publications and reports

Books 8 2 16 1 3 17 14 10 14 9 94

Articles 738 1,634 955 210 79 461 1,134 534 634 92 6,471

Book chapters 71 35 154 11 5 51 25 19 b b 371

Reports 540 620 1,200 170 335 1,405 828 1,009 515 180 6,802

Conference papers 662 1,027 1,437 369 245 747 1,619 968 534 247 7,855

Subtotal 2,019 3,318 3,762 761 667 2,681 3,620 2,540 1,697 528 21,593

Outputs related to commercial

product development Prototypes 28 14 27 31 28 68 75 233 50 12 566

Algorithms 11 6 6 10 1 21 52 646 17 8 778

Software 36 11 7 3 120 57 96 193 23 8 554

Patents 14 12 12 38 14 37 33 77 b b 237

Licenses 2 3 4 83 16 8 5 145 35 2 303

Commercial products 3 2 1 7 0 1 7 8 5 2 36

Commercial processes 3 1 2 6 0 1 3 12 1 0 29

Spin-off company 1 0 0 1 1 0 2 2 b b 7

Subtotal 98 49 59 179 180 193 273 1,316 131 32 2,510

Other

New program 20 8 3 0 2 6 29 91 b b 159

Invention disclosures 43 32 54 122 54 89 11 212 b b 617

Otherc 141 0 4 0 1 96 15 0 101 165 523

Subtotal 204 40 61 122 57 191 55 303 101 165 1,299

Total 2,321 3,407 3,882 1,062 904 3,065 3,948 4,159 1,929 725 25,402

Legend

ANL = Argonne National Laboratory

LBL = Lawrence Berkeley Laboratory

ORNL = Oak Ridge National Laboratory

PNL = Pacific Northwest Laboratory

INEL = Idaho National Engineering Laboratory

LLNL = Lawrence Livermore National Laboratory

LANL = Los Alamos National Laboratory

SNL = Sandia National Laboratories

BNL = Brookhaven National Laboratory

NREL = National Renewable Energy Laboratory

aFiscal year 1992.

bResponses were not collected from Brookhaven National Laboratory and National Renewable

Energy Laboratory.

cResearch subprogram outputs other than those listed above, such as technical abstracts,

workshops for laboratory users, and an electronic bulletin board service.

Page 55 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix V 

National Laboratories’ R&D Outputs

Table V.2: National Laboratories’ Research Program Outputs by Programa

Outputsb ER CE ES&H NE DP NPR ERWM FE CRWM PPA INT WFO Total

Publications and reports

Books 46 12 0 3 23 0 1 0 0 0 0 8 93

Articles 3,951 475 5 33 1,040 0 45 12 32 2 0 666 6,261

Book chaptersc 257 35 0 2 39 0 4 5 1 0 0 17 360

Reports 1,692 582 20 393 2,407 80 296 50 84 16 0 1,012 6,632

Conference papers 3,463 663 6 219 2,022 9 215 44 70 2 0 773 7,486

Subtotal 9,409 1,767 31 650 5,531 89 561 111 187 20 0 2,476 20,832

Outputs related to

commercial product

development

Prototypes 85 39 0 20 309 0 18 1 0 0 0 63 535

Algorithms 43 16 0 10 671 0 6 1 2 0 0 19 768

Software 49 17 0 29 233 0 17 3 0 3 0 200 551

Patentsc 52 18 0 5 111 0 6 0 0 0 0 7 199

Licenses 41 22 0 0 152 0 1 0 0 0 0 4 220

Commercial products 3 7 0 1 13 0 1 0 0 0 0 4 29

Commercial processes 3 4 0 1 13 0 1 0 0 0 0 1 23

Spin-off companyc 3 0 0 0 3 0 0 0 0 0 0 0 6

Subtotal 279 123 0 66 1,505 0 50 5 2 3 0 298 2,331

Other

New programc 25 12 0 2 97 0 5 2 0 0 0 16 159

Invention disclosuresc 80 58 0 28 270 0 40 0 0 0 0 19 495

Otherd 139 168 90 0 108 0 11 6 0 0 0 1 523

Subtotal 244 238 90 30 475 0 56 8 0 0 0 36 1,177

Total 9,932 2,128 121 746 7,511 89 667 124 189 23 0 2,810 24,340

(Table notes on next page)

Page 56 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix V 

National Laboratories’ R&D Outputs

Legend

ER = Energy Research

CE = Conservation and Renewable Energy

ES&H = Environment, Safety and Health

NE = Nuclear Energy

DP = Defense Programs

NPR = New Production Reactors

ERWM = Environmental Restoration and Waste Management

FE = Fossil Energy

CRWM = Civilian Radioactive Waste Management

PPA = Policy Planning and Analysis

INT = Intelligence

WFO = Work for Others

aFiscal year 1992.

bPacific Northwest Laboratory provided information about all outputs for the laboratory as a whole

that are not included in the data presented here.

cResponses were not collected from Brookhaven National Laboratory and National Renewable

Energy Laboratory.

dResearch subprogram outputs other than those listed above, such as technical abstracts,

workshops for laboratory users, and an electronic bulletin board service.

Page 57 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix V 

National Laboratories’ R&D Outputs

Table V.3: Proportion of Funds Spent

by Programs With More Than 100

Outputs Related to Commercial

Product Developmenta

R&D-related activity ER CE DP WFO

Basic research 53.1 4.4 7.2 11.5

Applied research 15.2 25.9 32.5 26.1

Development 10.1 28.6 40.9 23.4

Technical assistance to

Government agencies 4.0 5.9 9.7 25.1

Universities 6.5 4.0 0.7 2.8

Private firms or industrial organizations 2.8 11.6 0.9 2.2

Technology transfer to

Government agencies 4.1 3.1 2.8 4.3

Private firms or industrial organizations 2.5 15.7 2.5 4.4

Otherb 1.7 0.7 2.8 0.4

Total 100 100 100 100

Legend

ER = Energy Research

CE = Conservation and Renewable Energy

DP = Defense Programs

WFO = Work for Others

aFiscal year 1992.

bSubprogram expenditures for activities other than those listed above, such as training graduate

students and postdoctoral fellows or safety procedures.

Page 58 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix V 

National Laboratories’ R&D Outputs

Figure V.1: Trends in Outputs Related

to Commercial Product Developmenta

Number of outputs

0

100

200

300

400

500

600

700

800

900

1000

1100

1200

1300

1400

1500

Energy

Research

Conservation

and Renewable

Energy

Defense

Programs

Work for Others

Fiscal year

1989

1990

1991

1992

aThe number of outputs is shown only for research subprograms in the study population that were

in operation all 4 years (fiscal years 1989-92).

Page 59 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

Data concerning the formation of cooperative research and development

agreements, expenditures for R&D in critical technologies, and the views of

national laboratory program managers on their programs’ potential for

commercial product development are presented below.

Table VI.1: CRADAs in Effect at the National Laboratoriesa

1992 1991 1990 1989

National laboratoryb

All 4

yearsc

Any 4

yearsd

All 4

yearsc

Any 4

yearsd

All 4

yearsc

Any 4

yearsd

All 4

yearsc

Any 4

yearsd

Argonne 7 7 2 2 0 0 0 0

Brookhaven 4 4 0 0 0 0 0 0

Lawrence Berkeley 4 4 0 0 0 0 0 0

Oak Ridge 55 55 38 38 20 20 9 9

Pacific Northwest 4 9 0 0 0 0 0 0

Idaho 12 12 9 9 4 4 1 1

Lawrence Livermore 13 13 3 3 1 1 1 1

Los Alamos 18 18 8 8 6 6 6 6

Sandia 13 74e 4 11 0 0 0 0

Total 130 196 64 71 31 31 17 17

aFiscal years 1989-92.

bResponses on CRADA formation were not collected from National Renewable Energy

Laboratory.

cThese are CRADAs formed for research sponsored by subprograms that were in operation every

year during fiscal years 1989-92.

dThese are CRADAs formed for research sponsored by subprograms that were initiated in any

year during fiscal years 1989-92.

eMost of the CRADAs formed in fiscal year 1992 were sponsored by the DOE defense program

technology transfer initiative at Sandia. This subprogram was initiated in June 1990 to identify

opportunities for commercializing technologies produced by DOE-funded defense research

activities.

Page 60 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

Table VI.2: CRADAs in Effect Within the National Laboratories by Research Programa

1992 1991 1990 1989

Research programb

All 4

yearsc

Any 4

yearsd

All 4

yearsc

Any 4

yearsd

All 4

yearsc

Any 4

yearsd

All 4

yearsc

Any 4

yearsd

Energy Research 41 41 13 13 4 4 1 1

Conservation and Renewable Energy 52 52 41 41 26 26 15 15

Environment, Safety and Health 0 0 0 0 0 0 0 0

Nuclear Energy 0 0 1 1 0 0 0 0

Defense Programs 26 79e 7 13 1 1 1 1

New Production Reactors 0 0 0 0 0 0 0 0

Environmental Restoration and Waste Management 1 14 1 2 0 0 0 0

Fossil Energy 3 3 0 0 0 0 0 0

Civilian Radioactive Waste Management 0 0 0 0 0 0 0 0

Policy Planning and Analysis 0 0 0 0 0 0 0 0

Intelligence 0 0 0 0 0 0 0 0

Work for Others 7 7 1 1 0 0 0 0

Total 130 196 64 71 31 31 17 17

aFiscal years 1989-92.

bResponses were not collected from National Renewable Energy Laboratory.

cThese are CRADAs formed for research sponsored by subprograms that were in operation every

year during fiscal years 1989-92.

dThese are CRADAs formed for research sponsored by subprograms that were initiated in any

year during fiscal years 1989-92.

eMost of the CRADAs formed in fiscal year 1992 were sponsored by the DOE defense program

technology transfer initiative at Sandia. This subprogram was initiated in June 1990 to identify

opportunities for commercializing technologies produced by DOE-funded defense research

activities.

Page 61 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

Table VI.3: Mean Percent Expenditures for Critical Technologies Within Laboratoriesa

Critical technologyb ANL LBL ORNL PNL INEL LLNL LANL SNL All labs

Material synthesis and processing 4.3 5.5 11.8 4.3 4.6 3.6 13.0 1.9 6.2

Electronic and photonic materials 1.1 4.4 1.7 0.2 0 1.9 1.4 2.1 1.7

Ceramics 2.2 2.2 9.8 0.5 0.8 0.7 0.7 0.4 1.8

Composites 0.2 0.4 4.0 0.6 6.9 0.4 0.9 0.2 1.0

High-performance metals and alloys 3.0 1.5 8.9 0 1.9 1.6 1.2 1.4 2.2

Flexible computer integrated

manufacturing 0.2 0 1.6 0.7 1.3 0 0.6 0.9 0.6

Intelligent processing equipment 0.2 0 0.3 1.1 3.7 0.3 1.1 0.8 0.7

Micro- and nanofabrication 0.2 0.4 1.5 0.1 0.5 0.4 1.2 0.7 0.7

System management technologies 0 0.2 0.6 5.0 0.8 0.8 0.7 1.4 1.2

Software 2.2 0.7 0.4 2.4 12.0 6.1 3.5 5.6 4.0

Microelectronics and optoelectronics 0.6 1.1 0.5 2.3 0.3 2.1 2.3 11.5 3.7

High-performance computing and

networking 1.9 0.5 3.3 0.2 3.4 5.0 2.0 1.2 2.4

High-definition imaging and displays 0.4 1.0 0.2 1.1 1.2 0.3 0.5 0.1 0.4

Sensors and signal processing 0.8 0.2 1.5 3.5 7.1 5.1 4.2 9.0 4.6

Data storage and peripherals 0.1 0.2 0 0.8 0 0.4 1.1 1.9 0.8

Computer simulation and modeling 5.7 1.4 5.0 5.4 14.6 12.7 4.5 5.0 6.7

Applied molecular biology 0.2 8.6 4.6 0.7 1.4 2.4 1.6 0 1.9

Medical technology 0.8 4.7 0.6 0.3 0.1 0.3 0.4 0 0.6

Aeronautics 0 0 0.3 1.0 0.5 0 0 0 0.2

Surface transportation technologies 1.9 0 1.2 0 2.2 0 0 0.4 0.5

Energy technologies 32.0 24.2 19.7 7.6 24.8 18.7 4.5 3.5 13.6

Pollution minimization, remediation, and

waste management 18.8 9.1 1.9 41.3 7.5 3.8 3.8 1.9 8.8

Otherc 5.3 0.1 6.6 15.3 4.3 32.9 2.1 0.4 10.2

Research funds not expended on

critical technologies 18.0 33.3 14.1 5.6 0 0.5 48.5 49.9 25.9

Total 100 100 100 100 100 100 100 100 100

(Table notes on next page)

Page 62 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

LEGEND

ANL = Argonne National Laboratory

LBL = Lawrence Berkeley Laboratory

ORNL = Oak Ridge National Laboratory

PNL = Pacific Northwest Laboratory

INEL = Idaho National Engineering Laboratory

LLNL = Lawrence Livermore National Laboratory

LANL = Los Alamos National Laboratory

SNL = Sandia National Laboratories

aFiscal year 1992.

bResponses were not collected from Brookhaven National and National Renewable Energy

Laboratory.

cSubprogram expenditures for activities other than those listed above, such as robotics, special

nuclear materials, environmental R&D, and detector technology.

Page 63 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

Table VI.4: Mean Percent Expenditures for Critical Technologies Within Programsa

Critical technologyb ER CE ES&Hc NE DP NPR ERWM FE CRWM PPA INT WFO

All

programs

Material synthesis and

processing 7.0 9.6 0 6.1 7.3 0 1.6 8.5 4.4 0 0 3.1 6.2

Electronic and photonic

materials 3.1 0.6 0 0.7 1.9 0 0 5.3 0 0 0 0.5 1.7

Ceramics 3.5 12.5 0 2.4 0.5 3.1 0.1 9.8 0 0 0 0.6 1.8

Composites 2.3 2.7 0 0.4 0.3 0 0.5 3.9 0 0 0 1.7 1.0

High-performance

metals and alloys 4.0 2.5 0 6.2 1.2 4.6 0.2 3.8 1.8 0 0 1.4 2.2

Flexible computer

integrated manufacturing 0.4 0 0 0.3 0.6 0 0 0 0 0 0 1.8 0.6

Intelligent processing

equipment 0.5 0.3 0 0.3 0.8 0 0.1 0 0 0 0 1.6 0.7

Micro- and

nanofabrication 0.6 0.4 0 0 1.0 0 0 0.6 0 0 0 0.7 0.7

System management

technologies 0.1 0.3 0 0.7 0.9 0 3.4 0 7.1 0 0 2.9 1.2

Software 2.8 0.9 0 2.3 5.3 0 0.3 7.6 5.6 5.0 0 5.6 4.0

Microelectronics and

optoelectronics 1.0 0.6 0 0.7 6.7 0 0 0 0 0 0 2.4 3.7

High-performance

computing and

networking 7.1 0 0 2.0 1.2 0 0 0 0.3 0 0 1.6 2.4

High-definition imaging

and displays 0.4 0.1 0 0.3 0.4 4.6 0 0 0 0 0 1.0 0.4

Sensors and signal

processing 0.9 0.7 5.0 3.6 7.4 0 2.9 2.4 0 0 10.0 3.3 4.6

Data storage and

peripherals 0.7 0 0 0 1.2 0 1.1 0 0 0 0 0 0.8

Computer simulation and

modeling 6.5 2.1 10.0 4.0 7.4 4.6 3.9 8.5 12.2 5.0 0 8.4 6.7

Applied molecular

biology 8.8 0.3 0 0 0.1 0 0.6 0 0 0 0 0 1.9

Medical technology 2.2 0 0 0 0 0 0 0 0 0 0 1.0 0.6

Aeronautics 0 0 0 0 0 0 0 0 0 0 0 1.3 0.2

Surface transportation

technologies 0 4.2 0 0 0.2 0 1.2 0 0 0 0 1.0 0.5

Energy technologies 12.6 39.7 0 68.6 3.6 83.1 1.0 41.3 25.8 90.0 0 12.5 13.6

Pollution minimization,

remediation, and waste

management 2.3 1.7 10.0 1.3 2.5 0 77.6 0.3 9.5 0 0 11.1 8.8

Otherd 10.5 0.4 0 0 14.5 0 0.5 8.1 3.6 0 90.0 9.3 10.2

(continued)

Page 64 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

Critical technologyb ER CE ES&Hc NE DP NPR ERWM FE CRWM PPA INT WFO

All

programs

Research funds not

expended on

critical technologies 22.7 20.3 75.0 0 34.9 0 4.8 0 29.6 0 0 27.1 25.9

Total 100 100 100 100 100 100 100 100 100 100 100 100 100

Legend

ER = Energy Research

CE = Conservation and Renewable Energy

ES&H = Environment, Safety and Health

NE = Nuclear Energy

DP = Defense Programs

NPR = New Production Reactors

ERWM = Environmental Restoration and Waste Management

FE = Fossil Energy

CRWM = Civilian Radioactive Waste Management

PPA = Policy Planning and Analysis

INT = Intelligence

WFO = Work for Others

aFiscal year 1992.

bResponses were not collected from Brookhaven National and National Renewable Energy

Laboratory.

cOnly one laboratory provided complete information for the ES&H program. ES&H activity at this

laboratory provides support for environmental protection at DOE sites in several areas, ensuring

safe facility management practices, developing and recommending radiation and chemical

protection policies and practices, and evaluating the health of DOE personnel and the public.

R&D to improve dosimetry and measurement techniques also is undertaken.

dSubprogram expenditures for activities other than those listed above, such as robotics, special

nuclear materials, environmental R&D, and detector technology.

Page 65 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VI 

National Laboratories’ Potential for

Commercial Product Development

Figure VI.1: Program Managers’ Views

of Programs’ Commercial Product

Potentiala

aTotal exceeds 100 owing to rounding.

Page 66 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See comment 1.

See comment 2.

Page 67 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

See comment 3.

See comment 4.

See comment 5.

See comment 6.

See comment 7.

See comment 8.

See comment 9.

Page 68 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

Page 69 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

The following are GAO’s comments on the September 14, 1994, letter from

DOE.

GAO Comments 1. The definitions for basic research, applied research, and development

that our study employs are derived from a Congressional Budget Office

study of the federal R&D enterprise. The definition of technology transfer is

the one used in our study of the Technology Transfer Act of 1986 and the

definition of technical assistance is based on expert opinion. Our analysis

examined the laboratories’ effort in each type of activity separately, and

grouped, in two major categories, in order to address the study objective:

to provide an empirical base for examining the extent to which the

laboratories are engaged in basic and applied research or research related

to commercial product development. Figure 1 and tables IV.1-IV.4 allow

the reader to view our findings in both the two major categories and as

separate R&D-related activities. The finding for each major category

presented in figure 1 is the sum of the findings for the corresponding

separate R&D-related activities presented in the last column of table IV.1.

DOE disagrees with the category we established for “research related to

commercial product development”—that is, that development, technical

assistance, and technology transfer are all laboratory activities related to

commercial product development—but does not question our definitions

or findings for each separate activity. We agree that DOE may decline to

accept our definition for research related to commercial product

development, but we do not agree that our finding for the sum of the three

separate activities is erroneous. This finding is based on laboratory

research managers’ estimates of the distribution of their subprograms’

expenditures that were collected, verified, and analyzed according to

generally accepted government auditing standards. We consider these

estimates, made by research managers who are closely involved with the

R&D, more accurate than estimates that may be obtained by other methods.

2. The analyses we produced were intended to establish baseline data for

addressing empirical questions underlying the public debate, rather than

to serve as a comprehensive analysis of the laboratories’ roles. To address

the study objective, we focused on the 10 laboratories as a set of

institutions, on comparing the distribution of expenditures for five types of

R&D-related activities both within and among the 10 laboratories, on the

nature and scope of their outputs, and on their potential for working with

industry to bring commercial products to market. Given this approach,

with the exception of expenditures for critical technologies and

Page 70 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

collaboration with industry, which we do examine, the other factors DOE

suggests for analysis are beyond the scope of this study. However, we

anticipate that our study might stimulate another party to undertake the

type of institutional, comparative analysis that DOE suggests.

3. We agree with DOE that the laboratories collaborate in R&D with industry

partners who then perform the additional testing and research activities

required for commercial application. We also agree that “it typically should

take years from the conclusion of a CRADA and the transfer of a technology

to a partner, to the commercialization of a product.” The explicit

definitions of terms and the discussion of CRADAs in the report make this

clear. (See pp. 6, 12, and 15.) However, we disagree that the report

attributes commercial product development work to the national

laboratories.

4. We state in the section on Methodology that we began our work with a

survey of the laboratories’ R&D activities because we could find no

sufficiently comprehensive (emphasis added) existing documentation.

To confirm that we had not overlooked an important information source

when we designed and implemented our data collection strategy, we made

inquiries about DOE’s institutional plan and research and development

databases. We found that DOE headquarters maintains only the institutional

plan database and that it includes only one of the data items, research

program budget, that we used in our report. This budget information was

available for fiscal years 1989-91 when we implemented our survey but

would not have been useful for our analyses because it is not compiled at

the same level of detail as our data.

We also found that the research and development database is not one of

DOE’s databases. It is being developed by the Critical Technologies

Institute for the Office of Science and Technology Policy (OSTP) in the

Executive Office of the President. When it is complete, it will have five

data items analogous to our data. However, this database was not

available when we developed our national laboratory inventory and is not

now available to users other than OSTP. Forty-one of the items in our report

are not included in either the institutional plan or research and

development databases.

The Laboratory Management Division in DOE’s Office of Energy Research

maintains the institutional plan database. It has research program budget

data for fiscal years 1979 to the present at the program level for 9 of the 10

national laboratories, and it has subprogram budget data for selected

Page 71 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

programs, such as energy research, defense programs, civilian radioactive

waste management, and work for others. Because they are incomplete at

the subprogram level, these data would not have been useful for our

R&D-related activities and critical technologies analyses, which required

budget data for all subprograms in our sample. Further, none of the budget

data for the National Renewable Energy Laboratory are included in the

institutional plan database. These data must be obtained from NREL’s

hardcopy institutional plan, which is available from the Office of Energy

Efficiency and Renewable Energy at headquarters.

The Critical Technologies Institute’s research and development database

will have information on laboratory expenditures for basic research,

applied research, development, and technology transfer for research

subprogram categories analogous, but not identical, to those we used, and

on CRADAs—for the national laboratories as well as for the laboratories of

several other federal agencies—when it is available to organizations other

than OSTP. The Critical Technologies Institute representative to whom we

spoke could not specify when the database will be available. However, the

research and development database will not have information comparable

to the 16 research subprogram outputs we collected from the laboratories

nor on the proportion of subprogram expenditures for the 22 critical

technologies and the proportion of expenditures for technical assistance.

We are also aware that abstracts of CRADA agreements can be obtained

through DOE headquarters from the Office of Scientific and Technical

Information, which is based in Oak Ridge, Tennessee. However, we also

found these data to be incomplete. In August 1992, we requested these

data through DOE’s Office of Technology Utilization at headquarters and

received 147 abstracts for the nine laboratories from which we collected

CRADA information—49 fewer than the total the laboratories reported to us.

Since the fiscal year was not then complete, we assumed that all CRADA

information had not yet been reported to DOE or entered into the database.

Our experience developing the survey frame, moreover, suggested that the

laboratories’ institutional plan data needed modification to address our

study requirements and that the information available from DOE was not

consistent with information available from the laboratories. We used the

list of research programs included in the institutional plans as a

preliminary frame for part I of the survey. Recognizing that the

laboratories are dynamic institutions, we asked each laboratory to confirm

the list before survey implementation. Most of the laboratories made both

deletions and additions to the list to meet our survey selection criteria.

Page 72 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

(See p. 4.) We used the lists of facilities reported in the DOE report, Capsule

Review of DOE Research and Development Laboratories and Field

Facilities, as a preliminary frame for part II of the survey. The laboratories

made deletions and additions to these lists as well and in two cases almost

completely replaced them. Changes of this magnitude confirmed the

strategy of collecting data directly from the laboratories to address our

study’s information requirements.

5. During the agency review of our draft report, two laboratories provided

us with additional CRADA information, bringing the total number of CRADAs

in effect among all programs in operation in any year from fiscal year 1989

to 1992 to 196. (See table VI.1.) This total is the number of CRADAs in effect

in fiscal year 1992, rather than “now,” to which DOE refers and which we

assume is fiscal year 1994. Moreover, we found a substantial increase in

CRADA formation in fiscal year 1992, sponsored by DOE’s defense program

technology transfer initiative at Sandia. (See tables VI.1 and VI.2.) It is

possible that the increase we found persisted and included more

laboratories, bringing the total to 1,000 in fiscal year 1994. However, such a

change would not render our finding for fiscal year 1992 inaccurate.

6. Brookhaven brought it to our attention that the number of CRADAs

formed is limited by the amount of money allocated to a laboratory and

that this amount varies widely from laboratory to laboratory. We agree

with Brookhaven that characterizing CRADA formation as the “strongest”

indicator of a laboratory’s commercial product potential is misleading for

this reason, and we have modified our discussion of CRADA findings.

Scientific user facilities and personnel exchanges will be examined in a

separate study. Licensing is described in the section on Principal Findings

of this report. (See pp. 13-14.) CRADAs are cost-shared cooperative

agreements targeted to a commercial innovation.

7. We treat laboratory outputs as measures of activity, not as measures of

impact or productivity. (See pp. 7-8.)

8. We found that the 10 laboratories produced many more publications and

reports (21,593) than they did outputs related to commercial product

development (2,510) in fiscal year 1992. This is a statement of fact,

tabulated from reports to us by the laboratories’ research managers. It

describes the laboratories’ activity. It is not intended as a criticism of the

research enterprise.

Page 73 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VII 

Comments From the Department of Energy

9. The purpose of this report was to examine the balance of R&D-related

activity across the laboratories, rather than to examine the magnitude of

the R&D investment. We used the proportion of funds expended for each

type of R&D-related activity as a measure of activity, not as a measure of

investment. (See pp. 6-7.) An examination of human resources and a

comparison of DOE’s national laboratories to those of other agencies was

beyond the scope of this study, given its focus on laboratory R&D-related

activity.

Page 74 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix VIII 

Comments From Argonne National

Laboratory

A representative of Argonne, Internal Audit, called us on July 5, 1994, to

report that the laboratory had no substantive comments on the report

draft.

Page 75 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IX 

Comments From Brookhaven National

Laboratory

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Now p. 4.

See comment 1.

Now p. 7.

See comment 2.

Now table IV.1, p. 50.

See comment 3.

Page 76 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IX 

Comments From Brookhaven National

Laboratory

Now p. 14.

See comment 4.

See comment 5.

Now p. 28.

Page 77 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix IX 

Comments From Brookhaven National

Laboratory

The following are GAO’s comments on the June 27, 1994, letter from

Brookhaven National Laboratory.

GAO Comments 1. We have added a statement to the report clarifying this difference.

2. We have evaluated the data Brookhaven submitted and, after making the

appropriate changes, added it to the database. These data have been

incorporated into the tables included in the report letter and appendixes.

3. We agree with Brookhaven’s evaluation of this response and have made

the change they requested to the database and report tables.

4. We agree with Brookhaven and have modified the discussion of CRADA

findings.

5. The information on CRADA formation Brookhaven submitted in the pilot

version of the data collection instrument has been added to the database

and the tables in appendix VI. We also have added the sentence

Brookhaven suggests to appendix I.

Page 78 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix X 

Comments From DOE’s Idaho Operations

Office

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See comment 1.

See comment 2.

Page 79 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix X 

Comments From DOE’s Idaho Operations

Office

The following are GAO’s comments on the June 29, 1994, memorandum

from DOE’s Idaho Operations Office.

GAO Comments

General Comments DOE’s Idaho Operations Office representative, who responded for Idaho,

observed that the value in the “R&D Budget” column of table 2 for Idaho

should be $275 million, rather than $98.7 million and that the Idaho

Operations Office made this determination by applying DOE headquarters’

definitions for research programs to Idaho’s research programs. The list of

Idaho research programs to which the Idaho Operations Office applied DOE

headquarters’ definitions is unspecified. We disagree with this

determination, because it violated the study methodology.

Specific Comments 1. The R&D budgets of the 10 national laboratories in table 2 were not

compared.

2. We coordinated data collection from the laboratories with DOE’s

operations office representatives, but none of them participated in any of

the technical activities involving survey implementation. Therefore, the

Idaho Operations Office representative may not have been aware that GAO

program selection criteria should have been employed to assess the “R&D

Budget” column value for Idaho in table 2 to be consistent with the

methodology employed for the other nine laboratories. The use of DOE

headquarters’ definitions for research programs to make this

determination would result in a list of subprograms that differs

substantially from the one jointly developed by GAO and Idaho.

Subprograms included in the survey population were identified by

laboratory representatives who applied the selection criteria we specified

(see p. 4) to a preliminary subprogram list we compiled from the

institutional plans and sent to the laboratories. This approach was

followed by Idaho’s representatives, who identified 10 subprograms. We

reduced the number of Idaho subprograms to nine during the editing and

coding process. The $98.7-million value in the “R&D Budget” column is the

total of nine research subprogram budgets reported by Idaho program

managers on part I of the national laboratory inventory data collection

instrument.

Page 80 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XI 

Comments From Lawrence Berkeley

Laboratory

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See comment 1.

See comment 2.

See comment 3.

Page 81 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XI 

Comments From Lawrence Berkeley

Laboratory

Page 82 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XI 

Comments From Lawrence Berkeley

Laboratory

The following are GAO’s comments on the June 17, 1994, letter from

Lawrence Berkeley Laboratory.

GAO Comments Although Lawrence Berkeley agreed with the study’s analytic framework

and with the need for studies of this type to inform congressional

policymakers, the laboratory raised an issue about the relationship of the

national laboratories’ role in commercial product development to the

broader needs of industry or the nation, which was not addressed in the

report. This omission warrants clarification.

General Comments The relationship of the national laboratories’ role in commercial product

development to the broader needs of industry is an issue being discussed

in the public debate about the laboratories’ missions and structure, but

one that falls outside of the study scope. The purpose of this study was to

examine the extent to which the national laboratories are engaged in basic

and applied research or research related to commercial product

development. Scientific and technical infrastructure, which Lawrence

Berkeley gives as an example of industry need, while important to the

considerations of laboratory mission and structure that serve as the

study’s policy context, was not addressed. It was our expectation that the

findings of this study would serve as an empirical base for designing a

study to address this and other institutional issues.

Specific Comments 1. The statement “interest in the current balance of the research effort in

the 10 laboratories’ research programs . . . . (and) the extent to which the

national laboratories are engaged now in basic and applied research or in

research related to commercial product development. . . .” implies that

commercial product development is an indirect consequence of laboratory

R&D-related activities, rather than a subset of the broader research and

development role of the national laboratories. That is the meaning of the

phrase “research related to (emphasis added) commercial product

development,” and the use of the phrase “outputs related to (emphasis

added) commercial product development” elsewhere in the report. The

explicit definitions and discussions of CRADAs in the report make it clear

that the laboratories’ involvement in commercial product development is

limited to collaboration with industry partners in R&D-related activities that

produce innovations with market potential and that move these

technologies beyond the laboratories’ walls. These definitions assume that

the industry partner performs the subsequent research, testing, and

Page 83 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XI 

Comments From Lawrence Berkeley

Laboratory

marketing activity that accomplish commercial application. (See pp. 6, 12,

and 15.)

2. This study was not designed as a broad assessment of the national

laboratories’ roles, but to examine the balance of the laboratories’

R&D-related activities in two major areas: basic and applied research and

research related to commercial product development. We looked at these

activities with three types of measures, and our conclusions interpret our

findings for each type. The conclusion focuses on research related to

commercial product development because we found slightly more activity

in this area. We amplified this conclusion with an interpretation of findings

for the other two types of measures. A discussion of the noncommercial

product output of nuclear weapons research was not relevant.

3. We have added a discussion of these limitations to the Methodology

section. (See pp. 7-8.)

Page 84 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XII 

Comments From Lawrence Livermore

National Laboratory

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Now page 26.

Page 85 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XII 

Comments From Lawrence Livermore

National Laboratory

The following are GAO’s comments on the July 6, 1994, letter from

Lawrence Livermore National Laboratory.

GAO Comments We have added the revised text describing Lawrence Livermore

Laboratory to appendix I.

Page 86 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XIII 

Comments From Los Alamos National

Laboratory

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 87 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XIII 

Comments From Los Alamos National

Laboratory

The following are GAO’s comments on the June 23, 1994, letter from Los

Alamos National Laboratory.

GAO Comments We have added a footnote to the Background section discussing the legal

division of Department of Defense and civilian responsibility for nuclear

weapons research and development. We also expanded the phrase on page

2 from “weapons development” to “nuclear weapons research and

development.”

Page 88 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XIV 

Comments From National Renewable

Energy Laboratory

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See comment 1.

Now p. 2.

See comment 2.

Now p. 8.

See comment 3.

Now p. 10.

See comment 4.

Now p. 15.

See comment 5.

Now p. 28.

See comment 6.

Page 89 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XIV 

Comments From National Renewable

Energy Laboratory

The following are GAO’s comments on the June 22, 1994, letter from

National Renewable Energy Laboratory.

GAO Comments 1. One future report will provide a descriptive statistical analysis of the

technical and operating characteristics of the national laboratories’ major

research facilities. Other topics are yet to be determined.

2. We have made this correction to the text.

3. Graphs and tables are presented in the section on Principal Findings.

4. The aggregation in figure 1 is intentional. The graph is designed to

illustrate the balance between the two major areas of R&D-related activity

we examined. The last column of table IV.1, labeled “All Labs,” presents

percentages for development, technical assistance, and technology

transfer for the 10 laboratories.

5. See table VI.3 in appendix VI. Table VI.4 presents these percentages by

program area.

6. We have made this correction to the text.

Page 90 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XV 

Comments From Oak Ridge National

Laboratory

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 91 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XV 

Comments From Oak Ridge National

Laboratory

We did not receive Oak Ridge’s written comments from DOE. We did

discuss Oak Ridge’s views with laboratory representatives by telephone on

June 22 and July 13 and 19, 1994, and we spoke with a representative of

DOE’s Oak Ridge Operations Office on July 7, 1994. We also received new

output data for Oak Ridge’s subprograms by facsimile from

representatives of both organizations. A summary of their comments and

our response follows.

GAO Comments Oak Ridge raised two general issues. One was the effect of the study

sampling methodology on findings for the laboratory’s outputs related to

commercial product development. Oak Ridge took the position that most

of the laboratory’s outputs related to commercial product development

were produced by subprograms not selected in the study sample and,

consequently, expressed the concern that GAO’s findings for outputs

related to commercial product development based on the sampled

subprograms may not be representative because of this distribution of

outputs among all laboratory subprograms. Most of these outputs, they

explained, are produced by programs that fall below the $10-million

threshold for inclusion in the survey. In fact, according to tabulations they

had performed, the sampled programs, while representing 73 percent of

the overall budget, produce only 7 percent of the outputs in question.

Secondly, Oak Ridge thought that the report’s definitions and analyses

equate development work with commercial product development and that

the conclusion based on this definition is not supported by the data. We

address these issues separately.

First, Oak Ridge actually had identified two sources of potential

underreporting: (1) data for outputs of sampled subprograms that were

not available at the time of the survey and (2) data for outputs of

unsampled programs. We agreed that additional data for sampled

subprograms should be added to findings for Oak Ridge. We requested and

received from Oak Ridge the new data for the sampled subprograms, and

we added them to our database and report tables.

We did not add to our database and report tables the summary data Oak

Ridge tabulated as total outputs (including unsampled subprograms) for

the entire laboratory. To have incorporated these data would have violated

the sampling methodology. Moreover, without more detailed information

at the subprogram level, we could not judge to what extent these totals

represented outputs of research and development programs. This was a

Page 92 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XV 

Comments From Oak Ridge National

Laboratory

matter of some concern to us, particularly in light of the large number of

outputs Oak Ridge ascribed to the unsampled programs.

Second, we disagreed that the report definitions and analyses equate

development work with commercial product development. Our

definitions, analyses, and conclusions make it clear that the laboratories

were not expected to produce commercial products. We defined

development as having “some type of product as the output goal

(emphasis added),” but concluding “with a prototype rather than a usable

good.” Further, we point out that “Additional time, research, and testing

are required to convert the prototype to a weapon or commercially viable

product.” The definitions of outputs related to commercial product

development, including those for precompetitive commercial products and

processes, state that these outputs tend (emphasis added) to arise from

development work, but that “they will require a substantial additional

investment before they are ready to market.” The conclusion, moreover,

reiterates that these outputs are “precursors to marketable goods,” and

that, for this reason, “it is too early to determine whether this activity will

produce technologies with commercial uses.”

We also examined the assumption that R&D is a linear process, with all

commercial product-related outputs arising from development, and found

that our data did not support it. We included this segment of the analysis

to emphasize the uncertainty associated with current understanding of the

operation of the R&D process, and the origin of technologies with

commercial potential. The conclusion we reached concerning the

uncertain prospects of the laboratories’ commercial product-related

outputs is an interpretation of this finding as well as our definitions for

outputs related to commercial product development.

Page 93 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XVI 

Comments From Pacific Northwest

Laboratory

A representative of Pacific Northwest, called us on June 22, 1994, to

comment on the draft report by telephone. A summary of the laboratory’s

comments is included in our response, which follows.

GAO Comments Pacific Northwest offered one general comment and several comments

and questions about specific items in the text. We address the general

comment first and then the specific comments.

General Comment Pacific Northwest suggested that a section be added to the report

describing the major commercial product-related initiatives the national

laboratories have undertaken since the end of fiscal year 1992. Partnership

for a New Generation of Vehicles (PNGV) and American Textile Partnership

(AMTEX), two consortia for R&D targeted on commercial applications in

which several laboratories are participating, were mentioned as examples.

We are aware that the laboratories have been active in technology transfer

activities of many types since the end of fiscal year 1992. This activity will

be captured in any follow-up study that is performed in the next few years

to determine if progress has been made since fiscal years 1989-92, the time

period measured in this report.

Specific Comments 1. Pacific Northwest thought that the word “primarily” in the sentence

beginning on draft line 10, page 4 (now line 12, p. 3), should be deleted

because it implies that the laboratories have only one primary mission. We

have modified this sentence.

2. Pacific Northwest said that the output data in table 5 (now table V.1) not

reported for the laboratory are available and will be submitted to us. We

received and reviewed the data, and we added it to table V.1.

3. Pacific Northwest said that information on CRADA formation for the

laboratory as a whole was submitted to us during survey implementation.

We confirmed that this information had been received and added it to

table VI.1.

Page 94 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XVII 

Comments From Sandia National

Laboratories

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 95 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XVII 

Comments From Sandia National

Laboratories

The following are GAO’s comments on the July 1, 1994, letter from Sandia

National Laboratories.

GAO Comments Sandia agreed with the report’s objective, methodology, and conclusion,

but made two general comments. First, Sandia suggested that the report

include a description of the national laboratories’ expanded efforts in

technology transfer during fiscal years 1993-94. Second, Sandia suggested

that we review the substantial variation in the percentage of laboratory

funds not expended for critical technologies reported for Lawrence

Livermore, Los Alamos, and Sandia in table VI.3. Sandia expected this

percentage to be very similar for all three laboratories.

We are aware that the national laboratories have been active in technology

transfer activities of many types during fiscal years 1993-94, including

participation in large-scale R&D consortia such as PNGV and AMTEX. These

activities will be captured in any follow-up study that is performed during

the next few years to determine if progress has been made since fiscal

years 1989-92, the time period measured in this report.

We reviewed all responses by Lawrence Livermore, Los Alamos, and

Sandia concerning percent of expenditures for critical technologies and

funds not expended for R&D in these areas. We found that Lawrence

Livermore program managers allocated a percentage of funds expended to

the “other” category to a much greater extent than did program managers

at Sandia or Los Alamos. We also found considerable variation among all

laboratories in the proportion of expenditures allocated to this category.

R&D activities specified in the “other” category included items such as

robotics, special nuclear materials, environmental R&D, and detector

technology. Allocations to this category, and to the energy technologies

category, accounted for most of the difference in proportion of funds not

expended for critical technologies by Lawrence Livermore, Los Alamos,

and Sandia.

Page 96 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Appendix XVIII 

Major Contributors to This Report

Program Evaluation

and Methodology

Division

Robert E. White, Assistant Director

Sara E. Edmondson, Project Manager

Dale W. Harrison, Computer Analyst

Venkareddy Chennareddy, Referencer

Richard R. Scott, Project Adviser

Gerald L. Dillingham, Project Adviser

Eric M. Larson, Project Adviser

Nancy A. Briggs, Project Adviser

Denver Regional

Office

Miguel A. Lujan, Project Adviser

(973356) Page 97 GAO/PEMD-95-2 National Laboratories’ R&D Activities
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. Orders for 100 or more copies to be mailed to a

single address are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (301) 258-4097 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Mail

Postage & Fees Paid

GAO

Permit No. G100
GAO/PEMD-95-2 National Laboratories’ R&D Activities