Strengthening the Army

R&D Program

John W. Lyons, Joseph N. Mait, and Dennis R. Schmidt 

Center for Technology and National Security Policy 

National Defense University

March 2005 
The views expressed in these articles are those of the authors and do not reflect the official 

policy or position of the National Defense University, the Department of Defense, or the U.S. 

Government. All information and sources for these two papers were drawn from unclassified 

materials.

John W. Lyons is a Distinguished Research Professor at the Center for Technology and 

National Security Policy (CTNSP), National Defense University, and former director of the 

Army Research Laboratory. 

Joseph N. Mait is a senior research physicist at the Army Research Laboratory. During 2001-

2004 he served as a Senior Research Fellow at the CTNSP. 

Dennis R. Schmidt is Director, Technology Transition, Office of the Deputy Assistant Secretary 

of the Army for Research and Technology, Headquarters, Department of the Army. During 2004 

he served as Senior Research Fellow at the CTNSP 

Defense & Technology Papers are published by the National Defense University Center for Technology and 

National Security Policy, Fort Lesley J. McNair, Washington, DC. CTNSP publications are available online at 

http://www.ndu.edu/ctnsp/publications.html.

ii
Contents 

Executive Summary ......................................................................................................v 

1. A Strategy for Improving Army Research and Development Laboratories .....1 

Summary .......................................................................................................................1 

History of Army Laboratories........................................................................................4 

Factors Affecting Army R&D Laboratories..................................................................6 

 Organization and Budget of Army R&D Laboratories ......................................6 

The Army Laboratories..................................................................................................9 

 Indications of Success in R&D............................................................................9 

 Ways to Measure Laboratory Effectiveness .....................................................11 

Assessment of the Army Laboratories.........................................................................14 

 Factors in Retaining Excellent Laboratory Technical Staff..............................16 

 The Laboratory Director ..................................................................................17 

 Ways to Empower the Director.........................................................................18 

 Alternate Operating Options ............................................................................18 

 Selection of a Model .........................................................................................21 

Conclusions..................................................................................................................25

Recommendations........................................................................................................26 

2. Assessing and Predicting for Army Science and Technology ...........................28 

Summary .....................................................................................................................28

Introduction: The Challenge ........................................................................................30 

 Science, Engineering, and Technology: How They Work ..............................30 

 Technology Time Lines ...................................................................................31 

Processes for Tracking and Predicting Technology.....................................................34 

 Current Practice: Internal Sources.....................................................................34 

 Current Practice: External Sources ...................................................................35 

 The National Research Council ........................................................................36 

 The Defense Science Board (DSB) and Army Science Board (ASB) ..............37 

iii
 Other Sources of Expertise ...............................................................................39 

 Current Practice: Databases ..............................................................................40 

Analysis........................................................................................................................42 

Conclusion ...................................................................................................................44

 Recommendations...............................................................................................44 

Appendix: Examples of Reviews in the S&T Literature .............................................46 

iv
Executive Summary 

The U.S. Army is undergoing transformation in several areas. It is acquiring lighter, more 

agile, and more lethal weapons with survivability at least equal to current systems, improving 

communications to link units horizontally and vertically into a system of systems, and 

restructuring fighting units. The importance of technology in these changes raises the question, 

should the Army’s science and technology (S&T) program also be transformed? If so, what 

changes are needed? These two papers consider models for managing the Army laboratories, 

including the mode of operations (in-house, extramural, collaborations) and the means by which 

the Army can be assured that its technical enterprise is state of the art. 

The first paper discusses how to evaluate the level of excellence in the laboratories and 

how to address technical challenges in areas where the laboratories lack the necessary expertise. 

The paper recommends steps to strengthen the laboratories, including more extensive formal 

technology collaborations with the private sector. 

The second paper recommends ways to assure that the Army is fully aware of 

developments in science and technology, not only in areas where the Army has considerable 

expertise, but also where the Army is not expert and where technology is advancing rapidly. 

Taken together, the papers present opportunities to move the Army S&T program ahead 

without disrupting the current operations of the laboratories. 

v
vi
A Strategy for Improving Army Research and 

Development Laboratories 

By John W. Lyons, Joseph N. Mait and Dennis R. Schmidt 

Summary1

Change is ever with us. The Army is transforming itself, and the Army science and 

technology (S&T)2

 program is changing faster with each passing year. How should the 

Army S&T establishment keep pace with change? Should it, too, be transformed? This 

paper discusses these questions in terms of the changing environment, the laboratories’ 

history, and several possible models for managing the Army’s technical enterprise. 

The Army laboratories have a long and distinguished history in research and 

development (R&D), from devising new ways to manufacture large gun tubes, to 

fostering the development of the first electronic digital computer. This paper first 

considers these roots of the Army laboratories then looks at the current role of the labs, 

criteria for effectiveness, and techniques for evaluating the labs. It summarizes current 

assessments of the laboratories by various groups from inside and outside the 

government. The paper then notes improvements in how the laboratories are managed, 

especially in light of trends in the Federal Government’s conduct of research. Federal 

agencies are now actively encouraged to foster more collaboration between and among 

Federal labs, universities, and industrial labs. Considering these trends and recognizing 

the practical limitations for radical restructuring, the paper presents and discusses a 

number of options for operating the laboratories. These range from simply enhancing the 

current management approach to more substantive changes such as contracting out the 

research operation to the private sector in ways similar to the Department of Energy 

National Laboratories. 

The paper’s authors conclude that the current posture of the Army labs is strongly 

positive; however, they do have problems that should be addressed. These problems can 

be overcome and the authors recommend a number of steps that should be taken. 

Specifically, the Army should adopt a program of formal collaborations with universities 

and industry along the lines of the Army Research Laboratory’s Collaborative 

Technology Alliances. In these alliances, when the Army has some expertise, but the 

private sector has more, the Army forms consortia with the private sector. Under such an 

agreement, the Army is able to maintain management of day-to-day operations of the 

consortia and also rotate Army staff members into the consortia and consortia staff 

members into the Army labs. Formalizing this approach should improve mutual 

understanding and greatly shorten the time needed to move new concepts from the 

 

1 This paper is based on a study sponsored by the Office of the Deputy Assistant Secretary of the Army for

Research and Technology. 2

 S&T is the formal Army designation for the early portion of the research and development process

encompassing basic, applied, and advanced development (6.1, 6.2, and 6.3). The more general term used in

the technical community is research and development (R&D). 

1
consortia to the Army labs.3

 The Army can have the best of both worlds—the Army labs’ 

close relation to the warfighters and its understanding of their needs as well as the private 

sector’s technical expertise and capabilities. 

The Army laboratories play a key role in transformation by providing new

technologies and unbiased critiques of technologies coming from the private sector. This 

contribution continues a long and distinguished history of technical developments, 

starting from the earliest work in armories and arsenals before the Civil War. Some good 

examples include Thomas Rodman’s work at the Watertown arsenal on processes to 

manufacture very large gun tubes and, during World War II, the Army Ordnance and the 

Ballistics Research Laboratory at the Aberdeen Proving Ground contracting for and 

overseeing the development of the ENIAC (electronic numerical integrator and 

computer), the first electronic digital computer. 

In recent years, however, despite their contributions, the laboratories have been 

criticized for being uncreative, slow to produce results, and ineffective in transitioning

new products and processes to the warfighter. These, and other similar comments, have 

been leveled at all the military’s in-house labs despite a 40-year history of studies

performed by numerous groups. Many of these studies have found the labs to be in 

relatively good condition and to have a history of significant accomplishments. These 

successes include not only the Army role in developing ENIAC, but also the Navy’s role 

in developing the Global Positioning System (GPS). Recommendations for 

improvements, however, are many. Examples include those focused on improving and 

strengthening the personnel system for scientists and engineers—who are the foundation 

of quality research –implementing outside peer review, and empowering the laboratory 

directors in all phases of management.4

 To the detriment of the labs, most 

recommendations have not been put into effect for a variety of reasons, which, for the 

most part, have been primarily political. 

The criticisms involving the long time it has taken to move technologies from the 

laboratories to the field arise in part because of the compartmentalization of the technical 

work. In the DOD S&T budget, research work is categorized into basic (6.1), applied 

(6.2), and advanced development (6.3). Projects move from category to category with 

some difficulty, especially if one needs to move backwards to address a certain problem.

When the work is ready to move to the next category it often faces an organizational 

barrier. For example, the next organization in line to accept a new technology from the 

originating laboratory must usually be persuaded to accept the work. This barrier is a 

problem of internal technology transfer that is not much different from that of transfer 

from an external group—whether it is from the Defense Advanced Research Project 

Agency (DARPA) or from industry—into the Army. Later, the work must move to the 

program manager or program executive officer realm, which involves another transfer 

problem. The slowness of this process is not surprising given the many possible barriers 

to be encountered throughout the process. 

 

3

 Note that this approach only works when the Army has some expertise but lags behind the private sector. 

When the Army has the technical lead the traditional management mode is appropriate. When the Army has 

no expertise in a newly emerging area, then the Army should maintain cognizance of technical work in the 

private sector until a partnership can be formed. 4

 Report of the White House Science Council Federal Laboratory Review Panel (Washington, DC: The 

White House, May 1983).

2
These problems are further compounded by the different cultures within the Army

community. Consider the peculiar situation in which the staff members of military 

laboratories find themselves. They are charged with driving change while being 

embedded in a highly conservative structure. Creative people often chafe under the 

discipline found in the military; they like to pursue interesting leads that are sometimes a 

little off the main track. They like to publish their results but may find that the military 

prevents them from doing so. Managing a change-oriented staff is a challenge in the best 

of environments. In the military, it is even more difficult. 

What is needed to address this problem is a new paradigm for managing the 

research and development process; a new way of looking at both the creation of new 

concepts and the movement of these ideas into practice. If the Army does not find a way 

to improve and speed up its R&D process, then both the Army and the country will suffer 

from lost opportunities, increased budget costs, and warfighting that is more difficult than 

it needs to be. This paper addresses the problem from the perspective of the laboratories, 

rather than the entire development cycle. However, the discussion also contains lessons 

for the whole process. 

To put lab management in perspective, we start with a brief history of the labs. 

Next, we discuss issues pertaining to the labs, including their role within the Army, 

characteristics that define laboratory excellence, how one can evaluate lab performance, 

and factors in retaining excellent staff members. We follow this discussion with a look at 

alternative operating options that might benefit the Army laboratories, including 

enhancing the present structure or changing management to governmentowned/contractor-operated (GOCO) entities, federally funded research and development 

centers (FFRDCs), or government corporations. Finally, we draw some conclusions and 

offer recommendations for improving the laboratories. These proposed changes should 

enhance both the quality and the pace of the work. They include consideration of 

differences in governance and internal improvements in management of quality, 

relevance, and timeliness. Finally, we consider a relatively new model of program

management, specifically, formal, integrated collaborations between the labs and centers 

of excellence in the private sector. This concept solves the transfer problem in cases 

where the private sector leads in the technology and the Army needs help in 

strengthening its internal expertise. 

3
History of Army Laboratories 

The Army and its contractors have performed scientific and technical work since the 

Army was first created. Workers at the early arsenals were either directly involved in 

technical work or were involved in overseeing it. Examples include Thomas Rodman’s 

work in the 1850s and 1860s on devising new alloys and processes for making large gun 

tubes. Rodman was employed at, and commanded, the Watertown (MA) and Rock Island 

(IL) Arsenals. In the early days, innovation came from individual inventors who were 

often working from home because research laboratories did not exist as they do today. 

The first research facilities may have been the Naval Observatory and parts of the Coastal 

Survey (now the U.S. Coast and Geodetic Survey, part of the National Oceanographic 

and Atmospheric Administration, U.S. Department of Commerce) in the 1840s. The 

large, multi-program labs that are part of the current Army infrastructure primarily find

their roots in industry. For example, Edison’s lab at Menlo Park had been established 

well before the 20th century, and Bell Laboratories, DuPont, and General Electric had all 

established research facilities by the turn of the 20th century. The National Bureau of 

Standards was established by an Act of Congress in 1901, the Naval Research Laboratory 

(NRL) was founded in 1923, and the seeds of the National Institutes of Health (NIH) 

were planted in the 1930s. 

In World War II, R&D, as well as R&D modes of operation mushroomed. 

Although existing Army and Navy laboratories worked hard, much of the work had to be 

executed externally through large contracts to GOCO plants, such as the new nuclear 

laboratories at Los Alamos, New Mexico, and Oak Ridge, Tennessee; through large 

grants to universities such as the Radiation Laboratory at MIT; and through a great 

number of smaller awards to universities. The Office of Naval Research (ONR), created 

in 1946, reflected the military’s recognition of the importance of engaging the private 

sector by setting up a formal office for sponsoring R&D both inside and outside the 

Navy. ONR served as the model for the National Science Foundation (NSF), which was 

established in 1950. 

The history of the Army laboratories is no less distinguished. Marconi’s 

pioneering work in long-distance radio communication was done near what is now Fort 

Monmouth, NJ and led to the formation of the Army Signal Corps’ facility there in 1916. 

The Ballistics Research Laboratory, established in 1935 at the Aberdeen Proving Ground 

in Maryland, not only performed outstanding work in armaments and armor, but also 

oversaw the building—and was the first user—of the ENIAC. The Jet Propulsion 

Laboratory grew out of contracts in World War II from Army Ordnance based at 

Aberdeen, and the space age in America began in earnest just after the end of WWII at 

White Sands Proving Ground, New Mexico, and later at Redstone Arsenal, Alabama. 

Obviously, the Army does not lack for accomplishments in R&D. 

Yet criticism continues. Some criticism has even gone so far as to propose the 

elimination of in-house facilities altogether. Why? One reason is a perceived lack of 

vision or leadership in the Army R&D program. Unfortunately, this criticism is not new. 

The success of the Soviet Union’s Sputnik program came as a shock to many who had 

assumed that the United States held a clear lead in all military and civilian technologies. 

The U.S. military laboratories took some of the blame for this falling behind in the space 

4
race. One response to this blame was the creation of the Defense Advanced Research 

Projects Agency (DARPA) in 1958. 

A second possible explanation for the criticism may lie in the cutbacks in military 

R&D funding, which were severe in the post–Cold War years of the 1990s. The reduction 

in procurement, in particular, left many industries with idle lab and plant facilities that 

were eager to secure some of the funds and work that previously had been allocated to the 

military labs. The same lack of appropriations has left the military labs in even worse 

condition. Further, the new technologies affecting the battlefield are those in which the 

private sector has the technical lead, for example, communications, computers, and 

related information technologies. 

In addition, the Government’s personnel practices have not helped matters. The 

average age of staff members in the labs has been rising (into the mid-40s) for a number 

of years. One of the reasons for this is that in the immediate aftermath of the end of the 

Cold War the military was down-sizing and hiring few to no young, fresh graduates. 

Finally, the labs themselves are partly to blame for failing to communicate fully 

with the leadership of the Army. Very few Army officers have any experience doing 

research in laboratories. Although some may have technical degrees, their lack of handson experience in a laboratory setting makes it difficult for them to understand R&D. This 

inexperience makes effective communication between lab management and senior 

military leaders difficult, creating fertile ground for commercial vendors to sell their 

products, processes, and services, including research capabilities. As part of the private 

sector’s marketing plan to market their services in research and development, they have 

criticized the military laboratories’ competence. To counter these criticisms and provide a 

background, the next section discusses factors affecting Army laboratories. In particular, 

it describes the role of the Army labs and provides objective measures that indicate 

excellence in these roles.

5
Factors Affecting Army R&D Laboratories 

In 1991, during the Base Realignment and Closure (BRAC 91) process, a special 

commission was established by the Secretary of Defense to review the plans of the three 

departments of the armed services for lab closings and consolidation.5

 As part of this 

activity, the commission defined a lab as any activity that performs at least 10 percent of 

its work years in science and technology (S&T budget activities 6.1, 6.2, and 6.3) and at 

least 50 percent of all work in research, development, testing, and evaluation (RDT&E 

budget activities 6.1 through 6.6). Given this definition, the Army labs then include,

among others, the Army Materiel Command’s (AMC) Army Research Laboratory (ARL) 

and the Research, Development, and Engineering Centers (RDECs); the Walter Reed

Army Institute for Research and the other laboratories of the Army Medical Research and 

Materiel Command (MRMC); and the laboratories of the Corps of Engineers. 

Organization and Budget of Army R&D Laboratories 

The bulk of the Army R&D laboratories are in the Army Materiel Command. The Army 

Research Laboratory conducts basic and applied research and provides technologies to 

the Research, Development, and Engineering Centers (RDECs). The Army Research 

Office sponsors university research, which is mostly composed of basic research 

activities The RDECs are also involved in small amounts of basic research, however, 

they primarily engage in applied research and advanced development work. The RDECs 

are collocated with the AMC commodity commands. Until recently they were an integral 

part of these commands and they continue remain tightly coordinated with them. The 

following are the AMC RDECs: 

• Communications and Electronics RDEC (CERDEC) 

• Armaments RDEC (ARDEC)

• Aviation and Missile RDEC (AMRDEC) 

• Tank-Automotive RDEC (TARDEC) 

• Soldier, Chemical, and Biological RDEC (SCBRDEC) 

The Space and Missile Defense Command, the Medical Research and Materiel 

Command, the Corps of Engineers, and the Personnel Command have their own separate 

R&D groups. The Office of the Deputy Assistant Secretary for Research and Technology 

oversees all of this activity and plays a major role in formulating the various budgets. 

 

5

 Federal Advisory Commission on Consolidation and Conversion of Defense Research and Development

Laboratories, Report to the Secretary of Defense (Washington, DC: Department of Defense, September 

1991). 

6
FIGURE 1. Organization of the Army research and development laboratories. 

Source: U.S. Army, Army Science and Technology Master Plan (Washington, 

D.C, 2003). 

7
FY04 Army S&T Investment Perspective 

AMC RDECOM

$1.37B 

MRMC

ERDC 

SMDC

PEOs 

Army Research

Institute 

Army Research

Laboratory

STRICOM

SBCCOM

TARDEC 

(Tank & Automotive) 

CERDEC

DARPA 

ARDEC 

AVRDEC 

MRDEC

LAB FY04

MRMC $135M

DARPA (FCS) $114M

ERDC $94M

PEOs $28M

SMDC $26M

ARI $23M

STRICOM $21M

SBCCOM $101M

ARDEC $107M

AVRDEC $115M

MRDEC $178M

TARDEC $188M

CERDEC $246M

ARL (ARO $191M) $413M

Total $1,783M

$1.8B FY04 

FIGURE 2. Army S&T (6.1, 6.2, 6.3) funding for FY 2004 

Notes: SMDC = Space and Missile Defense Command; PEOs = Program 

Executive Officers; ERDC = the Corps of Engineers Research and Development 

Center; STRICOM = PEO for Simulation, Training, and Instrumentation; 

AVERDEC and MRDEC constitute the AMRDEC. Source: Office of the Deputy 

Assistant Secretary of the Army for Research and Technology. 

Of the total budget shown in figure 2, about three quarters goes to the AMC. It is 

important to note that this budget represents only 6.1–6.3 funding; the remainder of the 

acquisition funding, 6.4 and beyond, is not included. Typically, funding for the latter 

categories is much larger. 

8
The Army Laboratories 

The report of the BRAC 91 commission defined the role of the Army labs to include at 

least the following functions: 

• Performing laboratory work—theory, modeling, and experiment 

• Exploring new concepts and developing new knowledge 

• Ferreting out new S&T outside the labs 

• Applying new knowledge to solve enduring Army problems 

• Conducting developmental testing of new products or processes 

• Conducting engineering research to aid in scale-up 

• Facilitating transfer of technology to customers and users 

• Providing technical advice to Army senior leadership, thereby enabling the Army

to be a “smart buyer” 

In fact, the commission concluded that the last-listed function is the most important. 

That function arises from the other eight and requires that they be present. The same 

commission listed the attributes of an effective Army lab: 

• Clear and substantive mission 

• Critical mass of assigned work 

• Highly competent and dedicated workforce 

• Inspired, empowered, highly qualified leadership 

• State-of-the-art facilities and equipment 

• Effective, two-way relationship with the war fighters 

• Strong foundation in research 

• Management authority and flexibility 

• Strong linkage to universities, industry, and other government labs 

Indications of Success in R&D 

Bell Laboratories, which is generally regarded as the best example of an industrial lab 

because of its record in basic science and the plethora of products it has produced, has 

won 11 Nobel Prizes. Less recognized are the 10 Nobel Prizes in Chemistry and Physics 

that have been awarded to all government laboratory staff members: 

• Naval Research Laboratory—one Nobel Prize 

• National Institute of Standards and Technology (formerly the National Bureau of 

Standards)—two Nobel Prizes 

• National Institutes of Health—five Nobel prizes 

• National laboratories of the Department of Energy—two Nobel prizes 

Additional characteristics of a good lab include speed, agility in programs, and 

stability. Speed and agility are essential, especially in war time. Solving problems as they 

arise in the combat zone takes precedence. In such times, the leadership’s evaluation of

lab performance may be heavily dependent on specifics delivered to the warfighter. 

9
However, the ability to react quickly and effectively depends on having laboratories 

with stable personnel, funding, and organization. In these regards, the Army labs have not 

been stable for a long period of time. The budget went into steep decline after the end of 

the Cold War when the number of lab personnel was slashed, increasing the average age 

of the lab workforce. The organizational structure of the laboratories has since been 

characterized as chaotic.

Stability of lab organization in the Army has not been a strong point, especially when 

compared with other Service laboratories. At the AMC, the following organizations have 

been associated with running the electronics R&D programs:6

• Army Electronics Command—1962 

• Electronics R&D Command—1978 

• Laboratory Command—1985 

• ARL and the RDEC at Fort Monmouth—1992 

• Research, Development and Engineering Command (a new Command much like 

the old Laboratory Command)—2003 

When this kind of continual reorganization reaches down into the branch level the 

effect on staff morale can be considerable. 

Given that there have been five different governing entities in the past 40 years, one 

could get the impression that the Army is unsure how to organize its R&D program. In 

contrast, for example, is how the Navy has managed the Naval Research Laboratory 

(NRL). Established in 1923, the NRL is the Navy’s central laboratory and is funded 

through the headquarters of the Secretary of the Navy (HQSecNav). NRL continues to 

remain the central laboratory of the Navy and its funding continues to flow down through 

the Office of Naval Research from the HQSecNav. 

Further, with the constantly changing organization, titles, and related missions of the 

Army labs, it is difficult for Army leaders to know where new developments originate. 

Research laboratories, particularly central or corporate laboratories, rarely receive credit 

for new concepts because the work passes through many different organizations on the 

way to production and fielding. In the process, the identity of the source for the various 

innovations in the product is frequently lost. For example, it may be true that the 

weaponry and armor for the M–1 Abrams tank were developed at the Ballistics Research 

Laboratory (BRL) in Aberdeen. However, the BRL name is no longer used. The 

organization was subsumed by ARL in 1992 as the Weapons Directorate and renamed the 

Weapons and Materials Research Directorate in 1995 after an ARL reorganization. For 

those who know, the genealogy is transparent. For others, BRL has simply vanished. A 

helpful approach would be for senior management to let the system settle down and 

demonstrate its value without continual tinkering. 

 

6

 G. Adams and John M. Logsdon, “The Contributions of Department of Defense Laboratories to U.S.

Warfighting Capabilities, Case Studies of Twelve Laboratories” (Washington, D.C.: Center for 

International Science and Technology Policy and Security Policy Studies Program, Elliot School of 

International Affairs, The George Washington University, August 2002), 2ff.

10
Ways to Measure Laboratory Effectiveness 

In the face of criticism, it is important to be able to measure the effectiveness of a 

laboratory. Although the BRAC 91 commission7

 did not discuss methods of assessment 

explicitly, it did so implicitly through its comments on the three Services’ proposals. 

Measurement can be performed in two ways: (1) through a retrospective 

evaluation of results over many years or (2) through a parametric approach that looks at 

the current picture. It turns out, however, that simply reciting a laboratories results over 

time is not as persuasive as illuminating a labs current and near-term activities. This is 

especially true in the current state of affairs with the Army’s full commitment to fighting 

the global war on terrorism. Commanders have critical requirements in both Afghanistan 

and Iraq and need the labs to produce technology for these requirements in the immediate 

future. In this respect, Army labs have responded admirably to current needs. For 

example, the Night Vision Laboratory recently transitioned a compact, integrated laser 

for use in the Future Force Warrior program, either as a handheld or rifle-mounted 

designator. In the past year, ARL field-tested a vehicle-mounted acoustic sniper detection 

system in Iraq. The system indicates the direction of fire to passengers in the vehicle so 

they can exit to safety and not into the line of fire. Further, a combination of scientists 

and engineers from ARL’s Weapons and Materials Research Directorate (WMRD), the 

Tank and Automotive Command (TACOM), and the Anniston Army Depot created and 

developed new armor for Humvees in Iraq in record time. 

With respect to objective measures, many different kinds of metrics exist that one 

can use as parameters, including the number of papers published, the number of 

postdoctoral fellows, the percentage of staff members with Ph.D. degrees, the amount of 

continuing education completed, the number of patents disclosed and issued, and the 

number of awards received. However, these are indirect measures and not direct 

indications of effectiveness for users and customers. At least three kinds of more direct 

measurements are available8

: (1) peer review of quality, (2) immediate customer review 

of relevance and timely delivery of results, and (3) stakeholder reviews of overall 

program priorities. 

At ARL, a special Technology Assessment Board of the National Research 

Council performs reviews. These evaluations tell Army management how the quality of 

work, staff members, equipment, and facilities stack up against expectations and against 

peer institutions elsewhere.9 For ARL, the immediate customers are the Army RDECs. 

The RDEC directors sit on the ARL Board of Directors and participate in a full ARL 

review annually. They also respond to an annual questionnaire about their satisfaction 

with ARL performance. If ARL receives poor marks on a question, that laboratory is 

required to take corrective actions. 

ARL also established the concept of stakeholders—senior people within the Army

who have a broad field of interest that allows them to see the overall picture. One group, 

 

7

 Federal Advisory Commission on Consolidation and Conversion of Defense Research and Development

Laboratories, Report to the Secretary of Defense (Washington, DC: Department of Defense, September 

1991). 8

 E. A. Brown, “Reinventing Government Research and Development: A Status Report on Management

Initiatives and Reinvention Efforts at the Army Research Laboratory” (ARL, August 1998). 9

 Brown, “Reinventing Government Research and Development”; see also “Assessment of the Army

Research Laboratory” (Washington, DC: National Academy Press, 1996). 

11
composed of, among others, a four-star AMC commander and some of his staff members, 

as well as functional three-star generals, including Deputy Chiefs of Staff of the Army, 

were asked to consider major policy and budget questions that affected the ability of the 

laboratory to carry out its mission. They were not asked to complete a technical review of 

the programs. The reactions and comments of the stakeholders were of great value to 

both the laboratory managers and to the AMC Commander and his staff. Unfortunately, 

with changes in laboratory and AMC management, the group fell into disuse. It is very 

important that the Army continue to utilize the concept of stakeholders. A separate, highlevel group inside the Army could be very helpful when looking at the strategic 

directions, available resources, and potential barriers to success. One possibility for 

reconstituting an Army stakeholders group is for AMC to establish a single group of 

stakeholders jointly for ARL and the RDECs, with members from each of the major 

Army commands and from the three-star level at the Chief of Staff offices. Alternatively, 

each Army lab can have its own stakeholders group. 

A recent example of a challenge that a stakeholders group could have analyzed is 

the balance in the R&D portfolio between the Future Force and the Current Force. Until 

this past year the laboratories were told to address only problems related to the Future

Force. This approach was taken to accelerate the development cycle as much as possible. 

This decision left the Current Force without programs of technical upgrade. It is likely 

that the stakeholders would have been sensitive to this outcome and would have 

recommended against it.

The stakeholders might have also looked into the effect of the establishment of

lead systems integrators (LSIs) for the Future Force’s Future Combat System. The LSIs 

are empowered to make decisions that have traditionally been handled by the Program 

Executive Office (PEO)/Program Manager (PM) community in concert with the Army

laboratories. This has been an issue of great interest to senior leadership and questions 

exist as to the impact of this approach on the entire Army acquisition community. 

The RDECs are rated by their immediate customers: the PEOs and the PMs. The 

end users, represented by the Training and Doctrine Command (TRADOC), also rate the 

RDECs. Furthermore, the RDECs also make extensive use of external peer review. For 

example, the Communications and Electronics RDEC (CERDEC) charters a Senior 

Advisory Group of former Government civilian and military—as well as industrial—

leaders to review and comment on the relevance of its strategic technology objectives. 

Further, as part of CERDEC participation in the Army Strategic Readiness Systems, 

metrics are collected quarterly to assess innovation, prototyping, and delivery. CERDEC 

collects metrics on everything from the percentage of technology endeavors transitioned 

to advanced technology demonstrations, to advanced concept technology demonstrations, 

and to acquisition programs as well as the number of programs leveraged through 

partnerships with outside organizations. Picatinny Arsenal’s Armament RDEC (ARDEC) 

has an outside peer review committee made up of retired and active Army and private 

sector experts. The ARDEC peer review committee meets annually. 

Two commands outside AMC with significant laboratory investments in Army

research are the Army Medical Research and Materiel Command (MRMC) and the Corps 

of Engineers’ labs, or ERDC. Both have technical peer review processes. Review 

committees review concepts for the MRMC intramural research before the research 

begins. Proposals for extramural research are handled through reviews under a contract 

12
with the American Institute of Biological Sciences. At the technical program level, 

MRMC policy is to have “blue-ribbon” external panels review every separate research 

program at least once every three years. Similarly, all of the labs within the ERDC 

conduct peer reviews by external, independent panels of subject matter experts in order to 

assess the quality of scientific and technical work, staff members, equipment, and 

facilities relative to peer organizations. Lab directors have the responsibility to conduct 

these reviews at least every three years, consistent with “corporate” guidelines. 

13
Assessment of Army Laboratories 

The report of the Federal Commission10 only obliquely touches on evaluations, but does 

offer, through its critique of the Services’ reorganization plans, comments on their 

strengths and weaknesses. In 2000, Congress mandated a review of the military 

laboratories11 to judge the future relevance of work that was being conducted. The 

National Defense University (NDU) assembled a group of retired flag officers from the 

Services as well as retired R&D directors from across the Department of Defense (DOD). 

Beginning in the summer of 2001, the group of experts heard briefings on R&D from the 

Services and from the Office of the Secretary of Defense. They also visited 11 Service 

laboratories for briefings and lab tours. Their review of the labs is perhaps the most 

recent and comprehensive one available. 

As part of this review, the group of experts considered Army activities in sensors, 

information technology, and weapons. The group reviewed sensors work at the ARL 

Sensors and Electron Devices Directorate (SEDD) in Adelphi, Maryland, and at the 

Communications and Electronics Command RDEC (CERDEC) Night Vision Laboratory 

at Fort Belvoir, Virginia. Information technology was reviewed at CERDEC at Fort 

Monmouth and weapons work was reviewed at the Weapons and Materials Research 

Directorate (WMRD) at Aberdeen Proving Ground, Maryland. 

Overall, the commission’s findings were positive. The results are detailed in three 

reports: one on sensors, one on information technology, and one on weapons science and 

technology.12 The commission found the reviewed work to be relevant to the needs of the 

transformed forces of the future. However, the team did find that a heavy emphasis on the 

envisioned transformed force—referred to as the Objective Force (now termed the Future 

Force)—and strong top-down management of the work from the Department of the Army 

Headquarters (HQDA) was seriously reducing flexibility on the part of research staff 

members and first-line managers. In discussing the work and environment at the Night 

Vision Laboratory, the report on sensors notes: “Some members of the study group were 

somewhat uneasy over a perceived growing myopia in Army research focused only on 

the leadership’s direction toward the Objective Force.”13

 

10 Federal Advisory Commission on Consolidation and Conversion of Defense Research and Development

Laboratories, Report to the Secretary of Defense (Washington, DC: Department of Defense, September 

1991). 11 Section 913 of the National Defense Authorization Act for FY2000. The charge came in two parts: to

take a retrospective look at past accomplishments and relevance of the labs and to take a forward look to 

consider relevance of current programs. The latter was carried out by the Center for Technology and 

National Security Policy, National Defense University. The study results are presented in four reports that 

are available from the Center. 12 Please see the three NDU published reports: Section 913 Report, #1. “Sensors Science and Technology

and the Department of Defense Laboratories”, (Washington, DC: National Defense University, March, 

2002); Section 913 Report #2, “Information Science and Technology and the Department of Defense 

Laboratories,” (July, 2002); and Section 913 Report #3, “Weapons Science and Technology and the 

Department of Defense Laboratories,” (December, 2002). 13 “Sensors,” Section 913 Report 1 (Washington, DC: National Defense University, March, 2002), 17; see 

also page 9 of the Sensors report. 

14
The Night Vision staff members had, in effect, been told not to work on anything 

that was not related to the Objective Force. Another criticism, directed not so much at the 

labs, but rather at the Office of the Secretary of Defense (OSD), was on the noticeable 

lack of programs specially designed for joint forces and the lack of acquisition processes 

dedicated to joint work. 

Focus on the Objective Force/Future Force has been moderated somewhat by 

efforts of Army Chief of Staff Schoomaker to accelerate the transition to the Current 

Force those technologies that best meet the needs of troops in Afghanistan and Iraq. 

However, the lack of flexibility in setting direction remains. 

The review of the Information Technology programs at CECOM RDEC, Fort 

Monmouth, also rated the programs relevant to the future needs of the Army. Direct 

conversations with customers showed that they were highly to moderately satisfied with 

the work and its relevance. The study evinced concerns about the relative lack of 6.1 

basic research funding and concerns that most of the S&T funding was constrained by the 

Army process of tying down work through Science and Technology Objectives (STOs) 

and Advanced Technology Demonstrations (ATDs). The report on information 

technology stated that “the STO process freezes about 90 percent of the S&T work.”14 (In 

fact, this percentage does not apply to the 6.1 portion.) 

The report is positive about ARL programs and its focus on a fairly distant time 

horizon. The sensors report expressed concerns about the tight, top-down management of 

the lab by the HQDA with respect to its insistence on working only on the Objective 

Force and the relative “lack of adequate laboratory autonomy.”15 The group of experts 

liked the formal long-term partnerships established with the private sector. ARL 

customers wanted more effort on their problems but rated the relevance as very high or 

extremely high. 

At the site visit to WMRD in Aberdeen, the study group was pleased to have 

present for the review not only representatives of the ARL program, but also 

representatives of the work done at the TACOM Picatinny Arsenal (ARDEC) and at the 

Aviation and Missile Command’s Aviation and Missile Research and Development 

Center (AMRDEC). No only did the integrated presentations and demonstrations display 

very close working relationships between the laboratories, but the work was found to be 

relevant and “seems to be pushing the edge of weapons technologies.”16 The NDU study 

group was impressed with the intellectual climate at ARL, ARDEC, and AMRDEC. 

There was some concern again about the tight focus on the Objective Force, about the 

large number of weapons concepts under study (e.g., whether the Army could afford to 

field all of them), and concern about insufficient funding for basic research in the 

weapons area. 

With the creation of ARL, 6.1 funding in the RDECs was taken down to almost 

zero, except for modest amounts of Independent Laboratory Innovative Research (ILIR). 

Although considerable amounts of new ILIR funds have now been made available to the 

 

14 “Information Technology,” Section 913 Report 2, (Washington, DC: National Defense University, July, 

2002), 17. 15 “Sensors,” Section 913 Report 1, (Washington, DC: National Defense University, March, 2002), 9; see 

also page 17 of the Sensors report. 16 “Weapons,” Section 913 Report 3, (Washington, DC: National Defense University, December, 2002),

15. 

15
RDECs, ILIR 6.1 funding differs from ordinary 6.1 funding in that it is managed out of 

the office of the Deputy Assistant Secretary of the Army for Science and Technology 

(DASRT) and must be competed for annually. Because basic research is often the source 

of new projects, taking away local management control inhibits directors from guiding 

new starts. 

In summary, the three NDU reports are strongly positive in all of the topical areas 

of work related to sensors, information technology, and weapons. The criticisms in these 

reports focus sharply on two key areas: too much top-down management that reduces 

flexibility, and too little effort on joint technology development. However, the relevance 

of the programs presented was favorably rated. 

Factors in Retaining Excellent Laboratory Technical Staff 

The most important factor for laboratory excellence is a highly competent and dedicated 

work force. Much has been written and legislated about methods and processes for 

attaining high-quality technical people in the Federal Government. A variety of personnel 

demonstration systems have been designed with the goal of speeding up the hiring 

process and improving retention by a series of salary concepts that are based on 

performance. At the time of this writing, the Department of Defense has obtained 

authority from Congress to develop a DOD personnel system that is based in part on 

these earlier demonstration systems. Presumably, these new developments are taking the 

most effective components from the earlier systems, for example, pay banding and payfor-performance approaches. Details still need to be worked out concerning how to put 

into effect the pay-for-performance approaches and what effect pay banding in high grade 

levels will have on motivation and advancement. 

Other agencies have set up special personnel systems that have created elite groups, 

usually with respect to salary exceptions. However, in the following list of factors that we 

have determined are related to gaining and retaining excellent staff members, note that 

salary is not a major motivator for research staff members (so long as the system is 

viewed as being fairly administered): 

 

• Important mission 

• Opportunity to do exciting work 

• Stimulating colleagues 

• Excellent facilities and equipment 

• Management that understands R&D 

• Opportunities to propose new work 

• Freedom to publish and to gain recognition from outside peers 

• Reasonable salary 

Research staff members are motivated more by the nature and challenge of the 

work and less by salary. To carry on their work, they must have both access to the 

equipment and facilities they need and the flexibility and freedom to explore new 

avenues of research. Publication allows them to “leave tracks” and to build a scientific

16
reputation among their peers. Careers in academia come close to meeting these criteria. 

However, government labs can offer more continuity because of stable staffing.17

Some agencies have gotten around the difficulties in using civil service billets 

(e.g., slowness in hiring, salary rigidity, and citizenship requirements) by taking 

advantage of the Intergovernmental Personnel Act (IPA)18. Essentially, under the IPA an 

individual can physically work at a Government agency while remaining under the 

administrative authority of the home organization. The National Science Foundation 

(NSF) routinely rotates technical staff members from universities under the IPA 

authority. Sometimes these people hold jobs at NSF at the level of assistant director and 

are responsible for an entire area of science or technology. DARPA similarly rotates 

technical staff members under the IPA, and does so for a large number of staff members, 

but prefers that these people stay no longer than four years to ensure the infusion of new 

ideas and program interests. However, neither agency is actually operating laboratories. 

Various studies by the Defense Science Board19 (DSB) indicate that, in terms of 

laboratory governance (see the later section in this paper, “Alternative Operating 

Options”), the defense labs should consider a model wherein senior management is part 

of the regular civil service and the remainder of the technical staff is under the IPA. To

our knowledge, no such approach is being implemented in a laboratory. 

The Laboratory Director 

Although the senior laboratory leaders and the laboratory staff members are important in 

determining the quality and value of the labs, the key individual responsible for the 

success of any lab is the director. The Army lab director sets the tone and culture of the 

laboratory and guides the technical programs’ strategic direction. He or she is responsible 

for ensuring the quality and timeliness of the work products and for establishing and 

maintaining good relations with other labs and other parts of the Army. 

The task of selecting a director should not be taken lightly. The director should be 

appointed only after a nationwide search and a careful winnowing of candidates by a 

high-level panel of military and private-sector experts experienced in managing R&D. 

The director’s required qualifications will vary somewhat depending on the laboratory’s 

mission, but in any case, the candidate must be experienced in managing R&D, hold an 

advanced degree (usually a Ph.D.), be good at personal relations, and be well recognized 

nationally. If a change in laboratory direction is desired, then the most desirable 

candidate will likely come from outside the laboratory. The director should occupy the 

position for a term of not less than four years and usually not more than six. The usual 

two-year term of an Army officer is not long enough to have a substantial impact on 

program or culture. To make the position attractive to top candidates, the Army must 

address some of the empowerment issues raised in the Federal Laboratory Review 

Panel’s report and in the next section. 

 

17 A recent Nobel laureate in physics has said that he stayed at his government laboratory for just this 

reason, despite active recruitment by academia. 18 Intergovernmental Personnel Act of 1970 as amended, Public Law 91-648, 5 C.F.R. Part 334. 19 Report on the 1987 Summer Study on Technology Base Management (Washington, DC: Department of

Defense, December 1987); Defense Science Board Task Force on Defense Laboratory Management, 

Interim Report (Washington, DC: Department of Defense, April 1994); Defense Science Board Summer 

Study on Defense Science and Technology (Washington, DC: Department of Defense, May 2002).

17
Ways to Empower the Director 

A number of studies of the labs have noted that directors have lost control of the 

laboratories, both administratively and programmatically. In response to this trend, the 

Federal Laboratory Review Panel’s report (the first Packard Report) recommended that 

each director be given discretion over up to 10 percent of the laboratory’s funding—to be 

used to explore new areas and to seed new starts. At present, the Army labs have no such 

authority, despite a presidential directive in 1983 for this recommendation to be carried 

out. Other agencies provide directors with some sort of discretion of this type. For 

example, DOE National Laboratories implemented the 1983 recommendation. 

Furthermore, the National Institute of Standards and Technology has a special 

appropriation element for projects to be selected by the director after the budget is 

allocated. 

Further affecting a director’s influence over operations is the trend toward

centralizing administrative services by region. Financial services were centralized in the 

Defense Finance and Accounting Service (DFAS), followed by personnel, procurement, 

and more recently, facilities services. This centralizing approach leaves local 

management with no control over support to the staff. If a director wants to conduct 

alterations to the physical plant in any way, he or she must apply to a central agency, and 

because this agency is responsible for many entities, priority will not likely be given to 

the lab director’s needs. Although this centralizing trend continues, several reports have 

recommended that greater empowerment be provided to laboratory management. 

Alternative Operating Options 

Many people assume that the difficulties the labs are experiencing can somehow be fixed 

by a new and different mode of operating. They have proposed recommendations ranging 

from minor upgrades in local managers’ authorities to sweeping changes in status inside 

or outside the government. The following table lists some of these proposals. 

18
Proposal By Whom Date Details

Enhanced current BAST 1994 Increase or restore local 

authorities 

NIST model BAST 1994 Similar but somewhat bolder 

than the enhanced model 

More temporary 

staff members

DSB Various Massive use of IPAs 

Split model DSB 1987 Transfer 6.4 and above to 

acquisition community; move 

6.1 to 6.3 to university 

management

Government owned, 

collaborator assisted 

DSB and USAF 1998 Government owned facilities; 

split staff between 

government and private sector 

Multicenters Army 1996 Formal collaborations with 

large, dedicated external 

centers of excellence 

Public-private 

partnership 

British MOD 1990s Semi-private entity working 

for profit for government and 

private clients 

GOCO USG 1940s Government owned,

contractor operated (DOE 

labs) 

FFRDCs USG 1940s Privately owned, government 

funded (FFRDCs) 

Government Owned 

Corporation 

USG 1930s Independent governmentowned entities (e.g., TVA, 

U.S. Postal Service, State 

government-owned 

universities) 

TABLE 1. Suggestions for operating changes made by outside study groups. Notes: 

BAST = the NRC’s Board on Army Science and Technology; DSB = Defense 

Science Board; USAF = U.S. Air Force; MOD = Ministry of Defense; USG = U.S. 

The enhanced current approach involves continuing the existing structure, 

budgeting etc., but also carrying out the many constructive suggestions that have been 

made over the years to restore local authorities. Second, the National Institute of 

Standards and Technology (NIST) option is similar but a little more extensive. NIST 

(formerly the National Bureau of Standards) differs from the Army laboratories in many 

important respects. It is part of the Department of Commerce, but its customers are 

primarily in academia and industry. NIST is relatively independent, having its own 

19
charter, authorization, and appropriations, and is headed by a presidential appointee who 

is confirmed by the Senate. The possibility that the Army laboratories could be 

transformed into NIST-like entities seems highly unlikely. 

The Defense Science Board20 has made many useful suggestions along the lines 

of (a) more private sector involvement (the third proposal in the table) and (b) 

empowerment of the laboratories (the fourth proposal). The DSB model in the table 

suggests a very large use of IPAs (employees detailed to the government from private 

sector employers) who can be rotated in and out of the government. The DSB also 

suggests breaking up the current lab structure into (a) a piece in which work goes into the 

acquisition structure (the PEO/PM community) and (b) a piece in which the basic and

applied research as well as the advanced development work go under university 

management similar to the DOE GOCO model. The arguments against the GOCO apply 

also to this proposal. 

The DSB model in the table suggests a very large use of IPAs (employees detailed 

to the government from private sector employers) who can be rotated in and out of the 

government. The DSB also suggests breaking up the current lab structure into (a) a piece 

in which work goes into the acquisition structure (the PEO/PM community) and (b) a 

piece in which the basic and applied research as well as the advanced development work 

go under university management similar to the DOE GOCO model. The arguments 

against the GOCO apply also to this proposal. 

Generally, the options proposed in the table range from maintaining the same

overall governance and structures with fine-tuning and enhancements to moving the 

laboratories entirely out of the government. They present both advantages and 

disadvantages. Keeping the laboratories inside the Army and its major commands makes 

it possible to have very tight working relations between the laboratories and their 

customers. But staying in this position means the likely continuance of the 

micromanagement and lack of local authority that now prevails. At the other extreme, 

being located outside the government has at least the potential of eliminating the 

management problems. However, moving outside the Army would likely mean less 

effective contact and involvement with the customers and a greater barrier to technology 

transfer from the laboratories back into the Army. 

Dr. Timothy Coffey21, a former director of NRL, has reviewed some of these systems, 

including the following: 

• GOCO 

• FFRDC 

• Public-private partnerships 

• Government-owned corporations 

 

20 Report on the 1987 Summer Study on Technology Base Management; Defense Science Board Task

Force on Defense Laboratory Management, Interim Report; Defense Science Board Summer Study on

Defense Science and Technology. 21 T. Coffey, K. Lackie, and M. Marshall, “Alternative Governance: Tool for Military Laboratory Reform,” 

Defense Horizons 34 (Washington, DC: Center for Technology and National Security Policy, November 

2003).

20
In the United States, only the GOCO model has been used for government-owned 

laboratories, most notably DOE National Laboratories. However, in a separate study by 

the NRC, the Board on Army Science and Technology found that, although this choice 

undoubtedly would support the laboratory’s ability to hire outstanding research staff 

members, it would place the laboratory outside the Army and the Federal Government. 

Consequently, communications and technology transfers between the lab and its clients 

would be more difficult to process. Clients of the Army laboratories would include the 

warfighters, represented by the TRADOC. Further, a 1995 external review of the DOE 

National Laboratories found that the highly touted independence and flexibility of the 

GOCOs had been eroded by bureaucratic interference from Washington. That external 

review decided that this trend had gone too far and recommended that the DOE labs’ 

model be revisited, using “a clean sheet of paper.”22

Dr. Coffey is favorably inclined toward the government corporation or an analog of 

state universities. He would move the staff members under U.S. Code Title 10 rather than 

have them remain under the civil service system (Title 5). 

In 1993, the National Research Council (NRC) Board on Army Science and 

Technology (BAST) studied the Army Research Laboratory to see what new 

organizational and management options might be incorporated. Reporting in 199423, 

BAST considered four cases including the status quo. The multicenter option turns out to 

be the choice of greatest interest today because of intervening trends and developments in 

technology and organizations. In this option, 30 percent of ARL programs could be 

retained in “internal laboratories of excellence,” and the balance could be contracted out 

to a set of external centers with long-term sizable contracts. As the next section makes 

clear, this option, with some changes, appears to hold considerable interest for ARL and, 

perhaps, for Army labs in general. 

Selection of a Model 

The advantages of privatizing the Army laboratories seem to be more than offset by the 

loss of close contact with the customers. As Coffey24 points out, however, privatizing the 

laboratories also involves substantial up-front costs. Thus, the question that needs to be 

addressed is how to improve the effectiveness of the laboratories within the current Army 

structure. Some of the Defense Science Board’s suggestions (e.g., greater use of IPAs) 

deserve further study. A key challenge will be to strengthen Army expertise in those 

areas where the private sector has drawn ahead but where the Army wants to maintain a 

capability in-house. A close combination of in-house capabilities and private sector 

abilities is needed. Reviewing the models in the table led us to consider the climate for 

public-private R&D alliances. 

In recent years, the strong trend, both in and out of government, has been to 

collaborate in R&D rather than to start a new effort from scratch. “Going it alone” has 

become just too expensive and slow. There are now many examples of technology-based 

alliances. One of the best-known is the International Sematech Corporation, which 

 

22 Secretary of Energy Advisory Board, report of the Task Force on Alternative Futures for the Department

of Energy National Laboratories (Washington, DC: Department of Energy, 1995). 23 “The Army Research Laboratory: Alternative Organizational and Management Options” (Washington,

DC: Board on Army Science and Technology, National Research Council, 1994). 24 See ref. 21.

21
conducts R&D on semiconductor processing technology. The Sematech Corporation is a 

consortium that was formed in 1986 by 14 companies and funded on a matching basis by 

the companies and the Federal Government. The consortium has enjoyed much success 

and has become international in membership and scope. The reasoning behind this 

consortium and many others rests on two key arguments: (1) competitive forces among 

nations required the United States to find ways to improve the performance of its 

manufacturers by pooling knowledge and staff members, and (2) the costs of doing R&D 

and of building new plants made sharing risks and expenses an economic imperative. 

The Army has always contracted out work to the private sector. The RDECs, in 

particular, contract out much of their work. The Army Research Office runs a large 

program of grants to universities. However, these efforts have been carried out at arm’s 

length; once the awards were made, relatively little interaction occurred. Not much 

happened in the way of day-to-day collaboration among Army researchers and private 

sector investigators. Military R&D maintained a tradition of “going it alone” during the 

Cold War, with tight security and a lot of the work was classified. As the Cold War 

wound down, the Army laboratories began to initiate closer working relations with the 

private sector. 

Because contracts and grants were carried out at arms length technology transfer 

into the Army was more difficult than it needed to be. Army R&D managers devised a 

new approach to collaboration that has been highly successful. In this paper we call this 

approach the multi-center concept. The following discussion considers this concept 

further. 

For the purpose of discussion, we divide Army research and technology into three 

categories:

1. Research areas in which the Army is the dominant player 

2. Research areas in which the Army has special needs and internal expertise but the 

private sector has more expertise 

3. Areas in which the Army has relatively little or no expertise 

One can argue about which topics belong in which category, but the general 

considerations below apply. 

Where the Army is the dominant player (e.g., in ballistics, night vision, and 

rotorcraft), it should maintain full capability at in-house laboratories. Where the Army

has little or no expertise, it should not try to build internal capabilities but should 

maintain a window into external developments in the technologies. Also, it should make 

certain that the private-sector firms remain competent suppliers by maintaining their own 

laboratory systems. Finally, where the Army has some expertise but the private sector has 

more, the Army should find ways to draw in that private expertise. One way is a variant 

on the ARL experience, with external centers of excellence closely tied to ARL internal 

excellence. 

For the most part, military-funded external centers of excellence have been set up in 

an ad hoc fashion, often not as the result of a formal assessment of internal needs. 

Because these centers are set up without well-understood connections to the military labs, 

too often they are left without efficient channels into the service laboratories, making the 

eventual transition difficult. This problem has challenged some DARPA programs over 

22
the years. The centers of excellence proposed for ARL in the mid-1990s addressed this 

concern. 

The ARL external centers of excellence were formed in response to the Army Chief 

of Staff’s focus on digitizing the combat forces. The following were key criteria in 

establishing the centers: 

• The research areas must have been of top priority and the ARL neither had all the 

relationship with out-of-house experts. 

• Proposed operators of contracted centers of excellence needed to be willing to 

fully share new knowledge as it is generated and to allow guest workers from the 

sponsors into the private labs. 

• The program of the external centers needed to be planned and executed under the 

management of an internal, laboratory-designated manager. The internal and 

external programs should operate as a single R&D program. 

We note that single management is not possible under the usual contracting or grant 

authorities. Congress has solved this problem by authorizing Cooperative Agreements, 

which should not be confused with Cooperative R&D Agreements (CRDAs). 

Cooperative agreements are designed to allow just the sort of interactions described 

above. 

The original three ARL external centers, established as industry-led consortia, were 

established in 1996 at $10 million a year each for five years. The centers focused on 

sensors, communications, and displays. In 2001, the contracts were re-competed, and five 

centers were funded in the areas of sensors, communications, power and energy, robotics, 

and decision aids. In the original contracts, the government and the contractors agreed to 

put about 20 percent of their staff into rotation each year. This rotation was put in place to 

ensure the rapid transfer of new knowledge into the Army as well as the transfer of needs 

and requirements back to the contractors. Staff rotation was found to be difficult at the 

level conceived; much lower rates of movement occurred. Still, there has been movement 

of people back and forth between the labs and the external centers. Consortium funding 

has also since been changed from purely 6.1 to a mix of 6.1 and 6.2, and the period of 

performance has provided for an optional additional 3 years. 

It is just a short step from the centers model to a virtual laboratory construct. A virtual 

laboratory is one in which the components may be distributed geographically but are 

connected robustly by computer and communications networks. Some scientists and 

engineers have been working in this mode for many years, especially those working on 

computer modeling. However, this type of participation in a virtual manner has been by 

individuals working on individual projects. We suggest developing the concept formally 

for either large components of programs or for entire programs. 

The virtual laboratory should bring the academic and industrial communities not only 

to Army research, but also to the Army operational community. Although integrated 

project teams exist, their mission, structure, and mode of execution are traditional. The 

virtual laboratory would create a collaborative community using network technology. 

The Army’s enterprise intranet, Army Knowledge On-Line, provides an initial 

infrastructure to create this collaborative community. 

23
One benefit of the networked virtual laboratory is that it provides a test bed for 

network behavior under various conditions. Given the Army’s, and indeed the military’s 

focus on network centric warfare, it would seem beneficial to assess the social behavior, 

in particular the development of trust among participants, vis-à-vis metrics for 

productivity in a controlled and safe environment. 

24
Conclusions and Recommendations 

Although the following conclusions and recommendations are based on study of Army 

laboratories, specifically, the laboratories of the Army Materiel Command, the authors 

believe that many of them apply to all military laboratories and, indeed, to most 

government laboratories. Those that are directed at quality improvement and at 

collaboration with other centers of excellence are especially pertinent. 

Conclusions 

The overarching conclusion is that the role of the Army laboratories in science and 

technology cannot be understated. The Army laboratories are charged with safeguarding 

combatants and ensuring the reliability and security of their equipment. Failure to do so 

leaves vulnerable those on whom we depend the most to protect U.S. interests. This point 

should not be lost as proposals are considered in acquisition reform (e.g., movement 

toward lead systems integrators), and new reforms should consider their impact on the 

labs’ role to ensure that the Army is a smart buyer. 

Our study has led us to the following main conclusions: 

• Nothing is wrong with Army laboratories that application of well-known 

corrective measures cannot remedy. 

• These measures include more reviews by peers, by customers, and by senior 

stakeholders. They also include restoring authorities recently taken away. 

• The laboratories should remain within the Army. 

• Close collaboration with experts in the private sector in areas where the Army is 

not dominant would improve the effectiveness of the laboratories. 

The following items represent additional conclusions and details: 

• The Army laboratories, along with the other service laboratories, have been the 

repeated focus of studies and have been found to be effective and relevant. 

• Many managerial elements create problems in the way the labs are treated, for 

example, personnel rules, salary restraints, procurement red tape, and too often, 

too much micromanagement from above. 

• The most important factor in achieving excellence in a laboratory is the 

excellence of its staff members. 

• Well-tested criteria have been identified for excellence in a laboratory and for the 

hiring and retention of unusually well qualified staff members. 

• Properly qualified laboratory directors set the tone and culture of the laboratories 

and guide the evolution of the technical programs. Selection of the directors 

should be done with utmost care. 

• Lab evaluation should include external peer review, customer ratings, and senior 

management overview (stakeholders committee). 

• Adequate basic research is important for the health of a laboratory. Recently, 

substantial levels of ILIR funds have been added to the RDECs, but ILIR is not 

quite the same as a regular line for 6.1. 

25
• Strengthening the labs requires that they be able to enumerate their 

accomplishments and strengths as well as to assess their quality and relevance in 

ways that are understandable to senior military leaders. 

• There are several ways to operate the labs, for example, by means of GOCO, 

FFRDC, and government corporations. The NRC BAST and the DSB have made 

suggestions that could be carried out within the current governmentowned/government-operated model. These possibilities offer interesting 

alternatives and require further study and exploration. Some examples include 

much wider use of the IPA authority, exploration of the government corporation 

concept, and more extensive use of a multi-center (external) option. 

• The Army role in science and technology cannot be overstated. The Army 

laboratories are charged with providing technology to win the battle, safeguarding 

combatants, and ensuring the reliability and security of their equipment. Failure to 

do so leaves vulnerable those on whom we depend the most to protect U.S. 

interests at home and abroad. This point should not be lost as proposals are 

considered in acquisition reform (e.g., movement toward lead systems 

integrators). New reforms should consider their effect on the labs’ role to ensure 

that the Army is a smart buyer. 

Recommendations 

By adopting the following recommendations, the Army can improve the laboratories’ 

quality, make technology transfer from the private sector quicker, and raise the level of 

the laboratories’ effectiveness to that of its external collaborators: 

• Leave the laboratories within the Army. 

• Emphasize, in areas where the Army is not dominant but must have internal 

expertise, creation of tightly managed external collaborations along the lines of 

the multi-center model. 

• Insist on regular reviews for quality, relevance, and timeliness by external peers, 

stakeholders, and customers. 

• Recreate the stakeholder board to have cognizance, at the policy level, with 

respect to the functioning of the Army laboratories. 

• Consider the recent recommendations of the Defense Science Board, including 

the increased use of rotating IPAs as well as other visiting scientists and 

engineers). 

In particular, we would like to note that leaving the laboratories within the Army

maintains the current close relationship between the laboratories and the acquisition 

community as well as the ultimate user, the warfighters. 

Creation of new, integrated collaborations should be based on formal assessments 

of need, laboratory by laboratory, and on the existence of internal expertise adequate to 

participate with the private partners and to grow and strengthen the internal expertise. In 

addition, all collaborative activities should be required to be integrated with and managed 

by representatives of the all the partners under Army control. 

Finally, we make further recommendations about laboratory operating options: 

26
The Army should: 

• Require formal assessment of needs before funding external centers. 

• Require that internal laboratory expertise be maintained at levels sufficient to 

collaborate with the external centers. 

• Require that external centers be an integral part of the lab in planning and control 

and be managed accordingly. 

• Provide a clear pathway from the external center to internal Army labs and 

acquisition functions. The Army should manage the technical staff members by 

providing better personnel rules, empowerment, equipment, facilities, and so 

forth. 

• Establish formal regular internal and external review procedures that include 

external peer review of quality, customer surveys, and performance against 

metrics. 

• Audit each laboratory according to the criteria for excellence.

• Make each of the criteria part of a measurable set of metrics and use them in 

performance evaluation of managers. 

• Take every opportunity to empower the directors. 

• Convert the current ILIR lines to regular 6.1 funding activities and turn over 

management to the RDEC directors. 

• Keep records of accomplishments and be prepared at any time to present them. 

27
Assessing and Predicting for Army Science and 

Technology 

By John W. Lyons 

Summary 

An imperative for senior Army leaders is to be aware of new technical developments in 

fields pertinent to Army warfare. An immense amount of technical information is found 

in the public record and more is held in the collective memories of Army technical staff. 

In addition, other information about new innovative developments is being generated, but 

has not yet made its way into the published literature. The question is how to gather and 

interpret technical information for the Army leadership in a manner that is as 

comprehensive and as understandable as possible. What we are currently doing is 

satisfactory, but our techniques could benefit from some improvement. 

The results of basic scientific research have been and continue to be widely 

available through publications, talks and presentations at public meetings, and, 

increasingly, through interactions over the Internet. Army research staff members are 

expected to keep up with new developments in their fields, and, for the most part, 

experience tells us that this is often the case. However, difficulties arise in collecting and 

aggregating this knowledge, analyzing it for effects on the current R&D portfolio, and 

transmitting the results to Army leadership. Research in applied science and engineering 

is more difficult to access because much of it is in the private sector and is proprietary. It 

is often released only after patent protection has been secured and firms have achieved a 

satisfactory lead over their competition. In the military, both in the United States and 

abroad, some of this work is either sensitive or classified. Nonetheless, our research 

engineers keep up with their fields by traveling to foreign laboratories, attending 

international meetings, and accessing the informal networks that exist. The Army has 

difficulty learning about and keeping up with developments that are not part of existing 

Army R&D programs because they have few, if any, experts to cover them. One 

exception is the staff of the Army Research Office (ARO), which monitors a broad 

number of fields and sponsors private sector work in new fields, primarily at universities. 

Historically, many technical developments that have played a key role in Army

operations generally arose in fields that, at the particular time, were outside Army

technical communities. An example is the Global Positioning System (GPS). This key 

technology, widely used today on the battlefield, arose from a combination of 

developments in atomic clocks, space capabilities, and computers. In turn, these were 

made possible by advances in solid state physics. Other examples include new advances 

in neuroscience, biotechnology, quantum physics-based applications, and intelligent 

machines. Consequently, it is dangerous to circumscribe fields of interest. Rather, the 

Army should cast its net broadly, staying cognizant of a variety of subjects not currently 

in its development programs. 

This paper expands on these topics and reviews the large amount of technical 

information and public reviews of science and technology that are readily available 

28
online. The paper then considers a construct as to how the Army can digest all this 

information in basic research, applied research, and development and put it to use in 

Army programs. Improving the flow of technical information requires new organizational 

structures, additional staff members dedicated to collecting, analyzing, and disseminating 

this information flow, and new funding to make it happen. Special consideration is given 

to devising new techniques for staying in touch with developments in the venture capital 

markets and in the new, small, entrepreneurial companies. The job is too great to handle 

by simply adding it to the duties of existing personnel. Finally, the paper presents and 

discusses techniques for strengthening the Army’s competence in its approach to 

technical awareness. 

29
Introduction: The Challenge 

“How do I know what I don’t know?” This question, asked by a recent Chief of Staff of 

the Army, was about unknown (at least to him) technology developments with the 

potential to alter the military advantage on the battlefield. Another way to state the 

question is: What new technologies exist that could catch us by surprise? For the U.S. 

Army, technology is but one of many critical factors in readiness and the ability to win 

any engagement. But technology may be least understood by the leadership. The 

difficulty that Army senior leadership has in dealing with technology lies in the fact that 

technologists’ roots are in a culture very different from the military arts. The occasional 

exception only proves the general rule. This difficulty is experienced less in the Air Force 

where warfighting is centered on the technology of airframes and aerodynamics. 

Years ago, the Chief Executive Officers (CEOs) of technology-based companies 

were technologists themselves, and they understood these questions intuitively. But in 

recent times, the CEO of a tech based company is more likely to be a business school 

graduate or a marketing specialist than a scientist or technologist. A similar problem

exists for Army leaders. In the Army, the vocabularies are different –the operational style 

of the soldier is very unlike that of research scientists and engineers. Translation is 

required before understanding can develop. If the Chief of Staff is not a scientist or 

technologist with some experience in the practice of research and development (which is 

often the case), then he will be haunted by the question “How do I know what I don’t 

know?” This will usually take the form of two further questions: (1) what should we be 

doing to reduce the chances of technological surprise on the battlefield, and (2) are we 

getting our money’s worth out of the current deployment of our research and 

development (R&D) funds? All recent Chiefs of Staff have wrestled with these questions. 

Science, Engineering, and Technology: How They Work 

First, consider science, engineering, and technology and, in particular, the R&D activities 

needed for major advances. Scientific research is the systematic acquisition and 

classification of new knowledge formulated in relation to new truths or general laws. 

New knowledge most often evolves sequentially and incrementally. That is, new results 

rest on and elaborate on work that has already been done. Isaac Newton expressed this 

characteristic best when he acknowledged his predecessors by saying, “If I have seen 

further, it is by standing on the shoulders of giants.”25 Scientific research is often 

characterized as being curiosity driven rather than problem driven. DOD classifies this 

kind of work as basic research (6.1)26; much of it is sponsored in academia with the 

results published in the open literature. 

Communication among researchers in science is open, rapid, and often conducted 

in advance of formal publication (e.g., through e-mail, telephone, discussions at scientific 

meetings and seminars). Competent research staff members keep current on what others 

are doing in their field and should be able to tell a visitor not only what is going on in his 

 

25 Sir Isaac Newton in a letter to Robert Hooke, February 5, 1676. 26 The Department of Defense separates science and technology into budget activities 6.1 = basic research; 

6.2 = applied research and 6.3 = advanced development.

30
or her laboratory but also what is new worldwide. However, specific breakthroughs in 

science are difficult to predict. Even though one can know what areas are being 

addressed, one cannot specifically predict a new conjecture leading to a new discovery. A 

case in point was the sudden insight by Francis Crick and James Watson of the details of 

the structure of the DNA double helix. This insight came as a flash of inspiration after 

weeks of looking at X-ray data and molecular models. The community had been working 

on this problem and was certain that an answer would be found soon. Only the details 

were a surprise.

Engineering27 is that activity by which the properties of matter and the sources of 

energy in nature are converted into useful structures, machines, and products. 

Technology—the application of knowledge to practical purpose—is often used 

interchangeably with engineering. Engineering describes an activity—research, design, 

pilot plant operation, production, product testing, and evolution—whereas technology 

usually refers to a body of knowledge rather than to an activity. Technology is something 

that can be bought and sold or licensed, and also includes patents, and, importantly, 

know-how. Research in engineering, or technology, is similar in many ways to that in 

science. In fact, it is sometimes described as applied science. A major difference is that 

some of the research work in engineering is proprietary; publication usually has to await 

the filing of patents and international communication about current status is more 

restricted. In the military, this type of research is usually in the applied research (6.2) and 

early advanced development (6.3) categories and might be classified or have access to it 

otherwise controlled. Nonetheless, research staff members should be current on 

developments worldwide, though they will often rely on more informal means and on 

intelligence sources. 

Technology Time Lines 

The non-technical person often has difficulty realizing how long it takes to go from a new 

scientific discovery to a marketable product. He or she may also have difficulty realizing 

how severely the odds are stacked against any single discovery ending up as a market 

success. Most disclosures of new ideas for potential filing as patent applications do not 

yield patents, and most patents neither yield royalties nor are practiced successfully by

the holder. 

The odds against success are better than nine to one. Those odds keep on growing 

as one moves from lab discovery through applied research to development. By the time 

successful marketing occurs the odds are fewer than one in a hundred. So many things 

can go wrong after the laboratory demonstrations seem to indicate success. Customer

reactions, engineering design problems, and plant start-up are all areas that can block 

progress. At the same time, costs rise dramatically. The initial lab research phases are 

actually the least expensive and take the least amount of time. Developing the 

applications for a new chemical compound or developing a new product from new 

information in materials science takes a significant amount of time. Development is 

delayed by necessary iterations and recycling, false starts, adaptations to customer 

feedback, and so on. If all goes well, the decision to go to market will then require the 

design, construction, and start-up of a manufacturing facility with all the attendant costs. 

 

27 Webster’s Third New International Dictionary, 1986, s.v. “engineering.”

31
Timelines for going to market, or what the Bell Laboratories used to call 

“productization,” are usually measured in years and sometimes even decades. A recent,

well-publicized example is that of high-temperature superconductivity (the absence of 

electrical resistance in a material). Beginning early in the 20th century, both scientists and 

technologists were working with superconductivity that was obtained only in certain 

materials at liquid helium temperature (4 kelvin or -266 degrees Celsius). Such a low 

temperature is very difficult and expensive to attain in field applications. In 1986, two 

IBM materials scientists in Zurich discovered certain crystals that exhibit 

superconductivity at much higher temperatures, thereby allowing liquid nitrogen to be 

used as the coolant.28 Scientists then speculated that superconductivity could be achieved 

at or above room temperature. A lot of hype ensued about solving the world’s energy 

problems or dramatically reducing energy costs. Immediately, the field exploded in a 

frenzied scramble of research in the materials, physics, and electrical engineering 

communities to see whether these new materials could be put to use in a variety of ways 

on applications that had been frustrated by the earlier need for liquid helium.

These studies were variously directed at developing new theories about 

superconductivity, evaluating the electrical properties such as the maximum current that 

could be carried, the effect of magnetic fields, and so on. Most of this early work was 

widely shared because it was mostly fundamental science. However, a lot of early 

application work was also done. Some work was directed toward replacing the materials 

used at low temperature but continuing the same applications. Some examples include 

laboratory devices for voltage standards, superconducting magnets for medical devices 

(e.g., MRI), power transmission cables, and superconducting motors (a Navy interest). In 

1988, Congress established a National Commission on Superconductivity to review all 

major policy issues and to advise Congress on devising a national strategy to ensure U.S. 

leadership in the area of superconductivity. In 199029, the Commission reported with a 

review of the technical status and problems and assessed the potential for a number of 

applications.

However, it turned out that a number of technical barriers, some very challenging, 

blocked the use of these new materials. Most applications required very high current 

densities in the presence of magnetic fields-the new superconductor materials did not 

reach these densities. Now, 15 years after the discovery, the field has still not achieved

large applications of the phenomenon. Some small specialty uses have been introduced, 

but the world still awaits the dramatic product breakthroughs so excitedly predicted a 

decade and a half ago. For example, we do not know the time line for electric power 

transmission lines based on high-temperature superconductivity or whether, in fact, we 

shall ever see such a use. An interesting point to note is that, as the research community 

realized just how tough the technical problems were, the worldwide communications 

channels remained open. Thus, although much of the work was competitive, it did not 

become proprietary because of the need for new data from all sources. 

The key point for this discussion is that even though the occasional breakthrough 

will occur in the course of scientific research, the process to achieving practical 

applications in either military or civilian markets will almost always take a long time. 

 

28 The 1986 Nobel Prize in Physics was awarded to J.G. Bednorz and K.A. Muller for the discovery. 

29 “Report of the National Commission on Superconductivity,” Report to the President (Washington, DC: 

The White House, Office of Science and Technology Policy, 1990).

32
The scientific community will quickly learn of the breakthrough and many laboratories 

will be working on it within a very short time. The surprises, which many times cause 

delays, will often come in the details rather than in the underlying phenomena, but 

progress will then slow. That said, the example of the atomic bomb, which was based on 

the scientific discovery in the 1930s of nuclear fission, runs counter to research 

experience. The surprise was not in the capture of this energy in a bomb (that 

achievement had been speculated on) but in the incredibly short time the United States 

took to accomplish it. Nevertheless, this achievement is the exception of the rule. The 

United States did the job by pursuing many parallel paths simultaneously and at great 

expense—not a normal R&D approach. The United States also had the luxury of being 

able to assemble all the necessary scientists and technologists and managing them in 

extraordinary ways. 

33
Tracking and Predicting Technology 

Tracking, assessing, and making predictions are functions that are carried out by those 

inside the military, by contractors, and by other segments of society. The following 

section first considers the existing techniques with respect to their application to Army

needs and then turns to new concepts in learning about technical innovations. 

Current Practice: Internal Sources 

The best and most important source of information about technologies in which the Army

is already reasonably proficient is the internal technical staff. The military has tens of

thousands of highly trained people in its laboratories, technical and engineering centers, 

and test and evaluation entities. Each of these individuals is an expert in some area of 

science or engineering. The best are equal to peers outside the military sphere. An 

integral part of their assignment is to keep up with work in their fields around the world. 

This expectation is especially true of research staff members and those who administer 

research grants. Many information services are available to assist in this effort. The oldest 

of these are the various abstracts publications. These publications use a cadre of 

professionals, often volunteers, to read the literature and then prepare concise, oneparagraph abstracts on the material. The user may subscribe to a particular portion of the 

abstracts. A good source for patent information is the Official Gazette of the U.S. Patent 

and Trademark Office. One can also have pages of specific journals or all articles of 

specific topics routed to one’s desk. Thus, continual scanning of the world for new 

information is made relatively easy; keeping up with current information is part of the 

culture of S&T. By far the greatest source of S&T intelligence on areas of interest to 

Army laboratories is, therefore, internal staff. 

In a special study of the military laboratories, conducted in concert with the 1991 

Base Realignment and Closure (BRAC) activity, a special commission appointed by then 

Defense Secretary Richard Cheney reported that the first mission of the defense labs was 

to enable the services to be “smart buyers.”30 That is, through staff knowledge as to what 

is technically possible, what the barriers are to developing successful military products, 

and what the capabilities of various parties are, the laboratories are able to advise the 

acquisition community on the pros and cons of a given proposition and help create the 

most effective approach to a given problem. The advice given is thus free of commercial 

pressure and bias because the laboratories are considered to be neutral third parties. This 

“smart buyer” role was actually listed ahead of the labs’ R&D mission to conceive and 

develop new or improved military products and processes. 

The issue here is one of organization and process. Given that the knowledge exists 

somewhere in the S&T community, how do we efficiently and effectively gather that 

information, assess it, and provide the results to senior leadership? The present process 

relies on an informal “bubbling up” phenomenon whereby middle managers informally 

contact senior managers with status summaries of what they are hearing from the bench 

staff members. When this process is not working well the senior managers either 

 

30 “Report to the Secretary of Defense, Federal Advisory Commission on Consolidation and Conversion of

Defense Research and Development Laboratories” (Washington, DC: Department of Defense, 1991). 

34
complete an informal survey of their own or rely on members of their immediate personal 

staff to do so. They then pass their findings on to top leadership. This process tends to 

rely on too few people and leaves out the opinions of the majority of the technical staff

members. What is needed is a more organized approach to internal “intelligence” 

gathering and assessment. 

Furthermore, the current process does not apply well to learning about and 

possibly entering entirely new fields. These fields include biotechnology, 

nanotechnology, new applications of quantum physics, and many aspects of information 

technology. The Army Research Office (ARO) keeps up with developments in science 

through its sponsorship of basic research in universities. This information flows into the 

Army through ARO use of Army technologists to review proposals and subsequent 

progress. The transfer of ARO information into the Army Research Laboratory has 

helped speed the transfer of new ideas into the Army laboratory system. 

Current Practice: External Sources 

There are many different kinds of published surveys of technical developments with 

accompanying projections and predictions. For example, some are Government 

sponsored or executed while some are private sector. Some are narrow, some broad, and 

some are published on a regular basis, while others are occasional or one-time only. In 

the aggregate, a nearly overwhelming number of these compilations exist. No one can 

keep up with all of them. This is why we must rely on Army technical staff members to 

read and digest them and why we need mechanisms to pull together all of this 

information from the staff members with help from outside groups. 

Each of the scientific disciplines and many of the sub disciplines produce regular 

publications of key advances. This is often done on an annual basis. The following are 

just three examples, taken off of the Internet: 

• Advances in Chemical Physics31

• Materials Science and Technology: Challenges for the Chemical Sciences in the 

21st century32

• Applied Solid State Physics33

See appendix A for a more extensive list that is still just a small sample of what is 

available on the Internet. These publications are published in the private sector and are 

authored by experts in the field and sometimes include published proceedings of 

conferences or symposia in the field. Readership comprises the professionals in the field - 

they are not intended for the lay person. 

 

31 Advances in Chemical Physics (New York, N.Y.: John Wiley). 32 Board on Chemical Science and Technology, National Research Council, “Materials Science and 

Technology: Challenges for the Chemical Sciences in the 21st Century” (Washington, D.C.: National 

Academy Press, 2003). 

33 B. Kramer, ed., “Advances in Solid State Physics” (New York: Springer Verlag, 2002).

35
The Government characteristically relies heavily on outside advisors for 

assessment and analysis. In 1863, Congress chartered The National Academy of Science 

(NAS) as both an honorary society and a source of advice to the Government.34 The NAS 

created the National Research Council (NRC) in 1916 to institutionalize the process of 

outside assessment and analysis. The NAS is now the largest and most comprehensive 

resource we have. The military departments make good use of the NRC although it is 

likely that they could do better. An interesting point to note is that the second and third 

committees established by the NAS in 1863 were created on request from the Navy 

Department: the Committee on Protecting the Bottoms of Iron Vessels and the 

Committee on Magnetic Deviation in Iron Ships. 

The National Research Council 

The National Research Council of the National Academies is organized so that each of 

the various disciplines is covered by one or more committees. From time to time, these

committees will publish reports on exciting new areas and the research needed for them. 

One example is a group on the environmental sciences that lists what they call “the grand 

challenges” in their field. Examples of NRC work include: 

• Photonics: Maintaining Competitiveness in the Information Era35 is a report that 

summarizes opportunities in telecommunications, information processing, 

storage, display, and sensors. It reviews policy issues and makes 

recommendations to industry and to the Federal Government and is an example of 

a review of a fairly narrow slice of science and technology. 

• Materials Science and Technology: Challenges for the Chemical Sciences in the 

21st Century36 is a report by the National Materials Advisory Board. It is a 

sweeping review of the field of materials, with discussions of topics ranging from

educational needs to research opportunities with comparisons of efforts in the 

United States with those in other countries. 

• Physics Through the 1990s37 is one of a series of reviews that is published 

approximately every decade. The next one is in progress and is titled Physics in a 

New Era. It will survey the field broadly and discuss opportunities and 

recommendations. 

 

34 The National Academy of Engineering and the Institute of Medicine were established in 1964 and 1970 

respectively and were conjoined recently to the NAS in what is now called the National Academies. 35 Commission on the Physical Sciences, Mathematics, and Applications, National Research Council, 

“Photonics—Maintaining Competitiveness in the Information Era” (Washington, DC National Academy 

Press, 1988). 36 Board on Chemical Science and Technology, Materials Science and Technology. “Challenges for the 

Chemical Sciences in the 21st Century,” National Research Council (Washington, DC: National Academy 

Press). 37 Board on Physics and Astronomy, National Research Council, “Physics Through the 1990s”

(Washington, DC: National Academy Press, 1986). 

36
• Frontiers in Chemical Engineering: Research Needs and Opportunities38 is 

another periodic review of status and opportunities with suggestions for the 

Federal Government and the community. 

In addition, the NRC occasionally performs very broad reviews, for example, 

Directions in Engineering Research.39 This study is similar to those mentioned above, but 

its scope includes all of engineering. The study was done by organizing a set of panels 

that each focused on various subdivisions of engineering. Two examples that are 

particularly relevant to the present inquiry are: 

• STAR 21: Strategic Technologies for the Army of the Twenty-First Century40

comprises nine sub-volumes published in 1992–1994 on topics such as 

electronics, health and medical systems, lethal systems, airborne systems, and 

mobility systems. The NRC Board on Army Science and Technology (BAST) has 

been considering updating these studies. 

• Technology for the United States Navy and the Marine Corps: 2000–203541

comprises a summary volume and eight specialized volumes, including ones on 

information in warfare, weapons, platforms, undersea warfare, logistics, and 

modeling and simulation. 

These two very broad studies directly address the kind of question that the Army

Chief is asking. The Army study is not current and no automatic mechanism for updating 

it exists. Cost is a problem in that large numbers of people are required to cover the 

spectrum of technologies the Army needs. Time is also another problem. For example, a 

study such as STAR 21 takes a long time, both in the doing and in the NRC review and 

editing process. The National Academies also conduct annual symposia featuring young 

(ages 30–45) investigators speaking about their research. These sessions are called (a) 

Frontiers in Science and (b) Frontiers in Engineering. The proceedings are published by

the National Academy Press. 

The Defense Science Board (DSB) and the Army Science Board (ASB) 

The Defense Science Board (DSB) and the Army Science Board (ASB) were set up by 

the Department of Defense to provide technical advice to the Secretary of Defense and 

the Secretary of the Army, respectively. They are advisory committees and fall under the 

Federal Advisory Committee Act. Members are chosen from the private sector and 

include numerous representatives from industry who have experience in developing 

 

38 Commission on Physical Sciences, Mathematics, and Applications, National Research Council, 

“Frontiers in Chemical Engineering: Research Needs and Opportunities” (Washington, DC: National 

Academy Press, 1988). 

39 Engineering Research Board, National Research Council,” Directions in Engineering Research”

(Washington, DC: National Academy Press, 1987). 40 Board on Army Science and Technology, and National Research Council, “STAR 21—Strategic 

Technologies for the Army of the Twenty-First Century” (Washington, DC: National Academy Press, 

1992). 41 Naval Studies Board, National Research Council, “Technology for the United States Navy and the 

Marine Corps” (Washington, DC: National Academy Press, 1997). 

37
military systems, various academics and independent consultants, and a number of retired 

military officers. In the early days, the DSB membership comprised, inter alia, the 

directors of the National Science Foundation, the National Bureau of Standards, and the 

National Advisory Committee on Aeronautics; the president of the National Academy of 

Sciences; and the chairman of the President’s Science Advisory Committee. The current 

membership has none of those offices represented and, indeed, includes no current 

Government officials, even though the charter does allow officials of non-DOD 

Government agencies to serve. 

The current Army entity corresponding to the DSB is the Army Science Board 

(ASB). The ASB was chartered in 1977 to make recommendations to the Secretary of the 

Army, the Chief of Staff of the Army, and the Assistant Secretary of the Army (for 

Acquisition, Logistics, and Technology). The study list42 shows that the work of the ASB 

is largely limited to battlefield technology, leaving assessments of fundamental science to 

others. 

The technology reviews conducted by the DSB and the ASB would appear to be 

just what the Army Chief is looking for. Some recent examples include: 

• 21st Century Technology Strategies43

• Defense Science and Technology Base in the 21st Century44

• Tactics and Technologies for 21st Century Military Superiority45

Recent examples of reports by the ASB include: 

• Technical and Tactical Opportunities for Revolutionary Advances (report in 

review) 

• Concepts and Technologies for the Army Beyond 201046

• A System for Soliciting and Processing New Ideas/Concepts/Technologies 47

Clearly, these broad technology reviews are not done routinely. Perhaps that 

observation suggests there should be more such studies. From scanning the lists of 

publications, it is fairly obvious that the two science boards are not dealing much with 

science but mostly with technology and technology policy. Given the large number of 

members from companies that are associated with defense contracting, one must be 

careful in interpreting their recommendations. 

 

42 Army Science Board, accessed at <http://webportal.saalt.army.mil/SARD-ASB/default.htm>. 43 Defense Science Board, “21st Century Technology Strategies,” Defense Science Board Summer Study

(Washington, DC: Office of the Secretary of Defense, 1999). 44 Defense Science Board Task Force, “Defense Science and Technology Base,” Report of the Defense 

Science Board Task Force (Washington, DC: Office of the Secretary of Defense, 1998). 45 Defense Science Board Summer Study Task Force, “Tactics and Technologies for 21st Century Military

Superiority,” Vols. I–III (Washington, DC: Office of the Secretary of Defense, 1996). 

46 Army Science Board, “Concepts and Technologies for the Army Beyond 2010” (Washington, DC:

Office of the Assistant Secretary for Acquisition, Logistics, and Technology, 1999). 47 Army Science Board, “A System for Soliciting and Processing New Ideas/Concepts/Technologies” 

(Washington, DC: Office of the Assistant Secretary for Acquisition, Logistics, and Technology, 1994).

38
Other Sources of Expertise 

In 1986, the U.S. Department of Commerce asked the National Bureau of Standards (now 

the National Institute of Standards and Technology, or NIST) to review the new and 

emerging technologies that would be of economic importance in the years ahead. The 

Bureau responded with a report in June 1987 titled The Status of Emerging Technologies: 

An Economic/Technological Assessment to the Year 2000.48 The 20 page report discusses 

roughly a dozen technical areas, how the technical areas will be better than technology 

existing at the time, and reviews the barriers to successful competition in the international 

marketplace. In 1990, a more elaborate and ambitious version of the report was issued by 

the Department of Commerce.49 In that version, the emphasis was on the posture of the 

United States relative to both Japan and Western Europe. The technology list is similar to 

the earlier report but is updated to include digital imaging, superconductors, and sensors. 

The newer version also includes a comparison of the technologies list to the DOD critical 

technologies list (see next paragraph). A report format similar to the 1990 report would 

be valuable in any new survey involving the position of Army technology relative to that 

of potential adversaries. 

In the late 1980s, Congress instructed DOD to prepare an assessment of those 

technologies deemed critical for the military. The result was “The Department of Defense 

Critical Technologies Plan” of May 1989.50 Much of the list matched the emerging 

technologies list contained in the Department of Commerce report but with the addition 

of military items such as radar, phased arrays, signature control, and the like. Neither list 

focused on developments in science, developments that would presage much later 

technological innovations. In this sense, they are both relatively short-term studies. 

The critical technologies concept turned out to have some staying power. 

Congress gave the President’s Office of Science and Technology Policy the job of 

producing the list. In 1991, the Report of the National Critical Technologies Panel51 was 

published, which covered 22 technologies. The range of the report varied from the 

narrow topics of ceramics, photonics, and high-definition imaging to the broad subjects 

of systems management, software, medical technology, and energy technologies. A 

difficulty that surfaced in this study was that members representing different agencies felt 

obliged to put on the list at least one topic from their agency’s interests. This situation is 

often a problem with small study groups in that the members tend to “ride their own 

hobby horses.” Hence, we need to base these types of studies on the widest possible 

sampling of expertise. 

Congress institutionalized the concept of critical technologies by assigning 

permanent responsibility to the National Science Foundation (NSF) for doing more such 

studies. Subsequently, the NSF contracted with the RAND Corporation where the task 

now resides. The entity is now called the RAND Science and Technology Policy Institute 

 

48 National Bureau of Standards, “The Status of Emerging Technologies: An Economic/Technological

Assessment to the Year 2000,” NBS IR 67-3671 (Gaithersburg, MD: National Bureau of Standards, 1987). 49 Technology Administration, U.S. Department of Commerce, “Emerging Technologies: A Survey of

Technical and Economic Opportunities” (Washington, D.C.: U.S. Department of Commerce, 1990). 50 “Military Critical Technologies List,” The Department of Defense Defense Technical Information

Center, Fort Belvoir, VA, Accessed at: http://www.dtic.mil/mctl/. 51 National Critical Technologies Panel, “Report of the National Critical Technologies Panel,” National 

Technical Information Service Report No. PB91-1556869 (Springfield, VA: U.S. Department of

Commerce, 1991). 

39
and is a federally funded research and development center for the White House Office of 

Science and Technology Policy. Among other services, it maintains a database called 

RaDiUS on federally sponsored and performed R&D. This database is very useful for 

tracking Government research but is not very useful for tracking developments in 

industry. However, the concept of having the Federal Government periodically list 

critical or emerging technologies seems to have been lost. Critics claim that the exercise 

is equivalent to the Government’s picking winners and losers; they feel this is not a 

government role. 

Other studies have also been done, mostly by the Federal Government. In 1988, 

Congress established a Competitiveness Policy Council which reported annually for a 

short time in the early 1990s.52 It was concerned with critical technical areas and made 

policy recommendations to the Government. Since then, a private sector group called the 

Council on Competitiveness, formed by a group of CEO-level people from American 

industry in the 1990s, has published and continues to publish studies which look at where 

we stand with respect to international trading partners and the means by which that 

posture might be improved.53

The groups most involved in developing new and advanced technologies, as 

opposed to new science, are found in industry or in industry-university collaborations. 

Often, the industries involved consist of high-technology start-ups that are small and 

entrepreneurial. They are supported by venture capital firms and other investors. A 

window into this activity would be very useful. Currently, the Army’s only window is 

through the Small Business Innovative Research (SBIR) funding program. SBIR was 

established by Congress and is supported by a tax on extramural research budgets of all 

the Federal agencies. In Fiscal Year 2001, the funding level was approximately $1.5 

billion.

Of the non-DOD-related studies and study groups discussed in the above 

paragraphs, most are concerned with economic, rather than military security. 

Consequently, they are peripheral to the Army Chief’s question “How do I know what I 

don’t know?” The exception is the National Research Council, which will be discussed in 

the section on Recommendations.

Current Practice: Databases 

The only guaranteed way to deal with all of this information is through a set of linked 

databases designed for data mining. The Rand Corporation’s RaDiUS covers federally 

funded R&D but does not cover privately supported work in industry and universities. 

Organizations such as the Industrial Research Institute (IRI) collect portions of the 

needed information, but IRI is not a part of an integrated whole. No set of linked 

databases gives assurance that as much information as possible is contained therein. It is 

unlikely that this type of database will ever exist given our penchant for secrecy in 

pursuit of profits or security. 

 

52 Competitiveness Policy Council, “Pursuing a New Technology Policy” (Washington, DC: 

Competitiveness Policy Council, 1994); R.M. White, “U.S. Technology Policy: The Federal Government’s 

Role” (Washington, DC: Competitiveness Policy Council, 1995). 53 Council on Competitiveness, “Going Global: The New Shape of American Innovation” (Washington,

D.C.: Council on Competitiveness, 1998); Council on Competitiveness, “Imperatives for Innovation”

(Washington, DC: Council on Competitiveness, 2001). 

40
Still, we could do better. All relevant and available data sets will need to be linked 

together, which is no small task. Questions of security and access as well as related 

problems will continue to arise. Industry will not reveal its proprietary work; it will have 

to be inferred from available information. Inspection of public records such as the patent 

indexes and proceedings of conferences, public seminars, and publications will at least 

give indications of work in progress. 

Simple tools such as counting the number of references to a given subject 

(citation indices) will often point to new directions. Thus, if the number of references to 

devices based on quantum physics (e.g., quantum dots and quantum computing) is 

increasing rapidly, then that fact alone is indicative. Initial interpretation could be done 

by the staff of a new Army office (see the Recommendations section) and the emerging 

areas should then be subjected to further study. Potential for the Army can be decided by 

current mechanisms such as the Army Science and Technology Advisory Group, which 

draws on technical directors from all major commands, from the Training and Doctrine 

Command, and from the acquisition community (Program Executive Officers and 

Program Managers). Senior leaders should be briefed regularly about distilled sets of new 

topics. 

The databases could easily be set up at the Office of the Director, Defense 

Research and Engineering (DDR&E) and thereby serve all of the DOD. However, the 

queries and analysis should be done the Office of the Deputy Assistant Secretary of the 

Army for Research and Technology, because Army interests are best determined by the 

Army. 

41
Analysis 

The brief review above clearly shows that many credible sources of information 

regarding the status and likely evolution of science and technology can be found, with 

science being the easier of the two to survey because of its openness, even though it is 

harder to predict. What is needed is a process that reliably enables the Army to draw on 

these resources in an efficient and effective way. Consider first the internal resources.

The internal resources available to the Army comprise the Army’s own staff of 

scientists and technologists—nearly 10,000 strong if all categories of work are counted. 

In the R&D enterprise there are, at last count, 1,116 Ph.D.s and 7,920 total scientists and 

engineers. These scientists and engineers have the needed knowledge of the current state 

of the art in their areas of expertise, and they have the capability to make predictions. 

However, the only mechanism for “pulsing” this group is the informal review and 

summary performed by supervisors and managers, which is reinforced by occasional 

briefings by bench staff members to senior leaders. Clearly, the range of topics covered 

by this method would be limited by the topics that are actively pursued by the Army. We 

would not be assured of covering all significant new developments. 

If they are lacking a full knowledge of developments in technology, Army leaders 

may be subject to biased claims by vendors with something to sell. Consider the 

following case in point. A challenging category of technology is information technology 

(IT) and its supporting technologies, namely, electronics and computing. These 

technologies develop extremely rapidly, with new products appearing faster than one 

person can follow in any depth. Many of the products such as computer chips embody 

incremental changes, and the user only has to substitute a new chip for an old one or 

substitute a new motherboard inside an existing computer. The user cost is minimal and 

the rapid changes produce fast turnover in the marketplace. We can compare this scenario 

with the capital intensive production of a new platform or an upgrade thereof. 

Assessing and predicting in this IT arena is harder and requires some special 

techniques lest one be left behind, but the standard military acquisition process is too 

bureaucratic and slow for the Services to keep up. Therefore, we need not only new 

technical surveillance approaches but also a new acquisition process. 

Keeping up with IT requires that we have windows into developments ranging 

from Web-based portals to opportunities to actually participate in venture capital 

activities.54 Another approach is to place Army technologists into selected companies and 

universities where they can see first hand the future trends that are being developed. 

Recent efforts to set up external centers of excellence that are co-managed by Army

personnel and tightly coupled to in-house labs has resulted in the internal and external 

programs appearing as single, integrated efforts. Combining this approach with staff 

rotation back and forth provides effective early knowledge of new developments. The 

Army currently takes this approach in a few selected areas with U.S. industry, but more 

still needs to be done. Formal exchange programs with other countries give the Army

additional windows. In an area such as armor and armaments, given the Army’s dominant 

 

54 Participation in venture capital efforts was authorized by Congress in Section 845 of the National

Defense Authorization Act of 1994, as amended in Section 804 in 1997. 

42
position, extensive exchanges, and the improved relations with Russia, it is unlikely that 

radical new technology will appear without our foreknowledge. 

In regards to the acquisition process, it is much too complex and overburdened to 

handle such a fast-changing technology. The process will have to be reinvented for IT 

and, perhaps, for other areas. 

43
Conclusion 

A tremendous amount of technical information is available in the public domain that, if 

properly collected and analyzed, should enable senior leaders to keep up-to-date. This job 

requires both tools and staff members to refine and prepare the information. Current tools 

are incomplete because only the federally sponsored R&D work is well covered. 

However, even this work is not analyzed in a form suitable for briefing. 

A great deal of technology of interest to the Army is known to various Army 

technical staff members because it is their job to know what is going on in their fields of 

expertise. In cases where the Army plays a dominant role in developing technologies, 

leadership can and should assume that its technologists are well aware of new 

developments wherever they arise. 

No comprehensive tools are available with which to gather information from the 

nearly 8,000 scientists and engineers in the Army. The problem is acute in new fields 

where the Army does not have in-house expertise. These tools are typically being 

developed in academia and in leading industry laboratories—sometimes in partnerships 

between the two. These tools must also enable the gathering of information from new 

scientific and technical fields. 

Recommendations 

The Army should develop a formal method of collecting information on developments in 

science and technology. Toward this end, the Army should establish a new group within 

the Office of the Deputy Assistant Secretary for Research and Technology. This office 

should be made responsible for collecting, organizing, and analyzing technical 

information from all sources and for briefing Army leadership about the results on a 

regular basis. The office should be staffed by experienced professionals and should be 

sufficiently funded. In addition, the office should be authorized to work not only 

with/through internal Army scientists and engineers in the major commands, but also 

with sources of expertise in other Government agencies and in the private sector. To 

cover all possible topics of interest, the office should assign its staff members’ 

responsibility for covering specific abstract services, patent summaries, and annual 

scientific and technical reviews of recent progress in research. These assignments can

draw on selected members of the S&T staff throughout the Army. 

This new office should also specially assign members within the Army technical 

staff to serve as ferrets. These staff members would actively search the private sector for 

new developments. 

The Army should use the authority granted by Congress to engage in funding new 

concepts typically supported by the venture capital community to gain access into new 

high-technology firms. The Army should set up more external centers of excellence that 

are tightly coupled to internal laboratories. It should then require rotation of staff 

members to make certain that information moves rapidly into the system. 

The new office should assign coverage of most if not all of the National Research 

Council’s boards. Similarly, staff members should be assigned to cover new 

developments at the other services and other agencies of the Federal Government. 

44
The Army should seek to have the DSB and the ASB do more tracking, assessing, 

and predicting of developments in broad areas of science and technology and should 

require them to publish their findings on a regular basis. 

The Army should make better use of NRC capabilities. The Army should fund the 

NRC Board on Army Science and Technology (BAST) to repeat its STAR 21 exercise 

every five years. The Army should also review its use of the BAST with an eye to giving 

the BAST more critical assignments. By paying close attention to the reporting of the 

other NRC committees, the Army could also identify areas it needs to know more about 

in the various fields of science. A system for tracking and reporting on these other NRC 

reports ought to be a part of the new office discussed above. An efficient approach would 

be to assign the various Boards to specific units within the service to the task of looking 

for reports that directly relate to Army needs and programs. 

Finally, the Army should assign more well-trained technical Army personnel as 

technical liaisons to the staff members serving under the Chief of Staff, the Vice Chief, 

and other senior positions. 

Should these recommendations be adopted the chances of surprise would be 

greatly reduced. In addition, by expanding the horizons of the research staff, new 

opportunities would be uncovered. The improved technical awareness of the staff should 

lead to improved effectiveness of the research and development process in the Army. 

45
Appendix 

Examples of Reviews in the S &T Literature 

The entries in this list were obtained from a technical library catalog and are listed here 

only as examples of the many reviews available. To obtain details for a given title, 

consult a technical library, such as the NIST Virtual Library at http://nvl.nist.gov, and use 

the search engines located there. 

Advances in Artificial Intelligence Research 

Advances in Astronomy and Astrophysics 

Advances in Atomic Molecular and Optical Physics 

55Advances in Automation and Robotics 

Advances in Ceramic-Matrix Composites 

Advances in Electronic Materials 

Advances in Fibre-Optic Technology in Communications and for Guidance and Control 

Advances in Focal Plane Technology 

Advances in High-TC Superconductors 

Advances in Hypervelocity Techniques 

Advances in Image Processing and Pattern Recognition 

Advances in Image Transmission Techniques 

Advances in Information Systems Science 

Advances in Information Technology 

Advances in Infrared Fibers II 

Advances in Machine Vision 

Advances in Man-Machine Systems Research 

Advances in Materials Research 

Advances in Optical Fiber Sensors 

Advances in Optical Materials 

Advances in Physics 

Advances in Quantum Electronics 

Advances in Solid State Physics 

Advances in Telecommunications Networks 

Advancing Fronts in Chemistry 

Advancing Technology in Materials and Processes 

Chemical Reviews 

Reviews in Computational Chemistry 

Reviews in Macromolecular Chemistry 

Reviews in Modern Astronomy 

Reviews in Molecular Biotechnology 

Reviews in Polymer Technology 

Reviews on High-Temperature Materials 

Reviews of Infrared and Millimeter Waves 

 

46
Reviews of Modern Physics 

Review of Progress in Quantitative Nondestructive Evaluation 

Reviews of Pure and Applied Chemistry 

Review of Radio Science 

Review of Recent Advances in X-Ray Analysis 

Review of the Electrical Communication Laboratories 

Review of the Science of Fiber-Reinforced Plastics

47