Report to Congressional Committees

United States Government Accountability Office

GAO

June 2005 DEFENSE 

TECHNOLOGY 

DEVELOPMENT 

Management Process 

Can Be Strengthened 

for New Technology 

Transition Programs 

GAO-05-480 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-480.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Mike Sullivan at 

(937) 258-7915 or sullivanm@gao.gov. 

Highlights of GAO-05-480, a report to 

Congressional Committees

June 2005

DEFENSE TECHNOLOGY DEVELOPMENT

Management Process Can Be 

Strengthened for New Technology 

Transition Programs 

The ability to spur and leverage technological advances is vital to sustaining 

DOD’s ability to maintain its superiority over others and to improve and even 

transform how military operations are conducted. The three new transition 

programs we reviewed are all appropriately targeted on what has been a 

critical problem in this regard—quickly moving promising technologies from 

the laboratory and commercial environment into actual use. Moreover, by 

tailoring processes and criteria to focus on different objectives, whether that 

may be saving time or money or broadening the industrial base, DOD has 

had an opportunity to experiment with a variety of management approaches 

and criteria that can be used to help solve transition problems affecting the 

approximately $69 billion spent over the past 3 years on later stages of 

technology development. 

However, it is too soon for us to determine the impact the three new DOD 

technology transition programs are having. At the time of our review, the 

programs—the TTI, DACP, and Quick Reaction Fund—had completed only 

11 of 68 projects funded in fiscal years 2003 and 2004; of those, only 4 were 

providing full capability to users. Additionally, the programs have limited 

measures to gauge success of individual projects and return on investment. 

Nonetheless, reports from the programs have pointed to an array of benefits, 

including quicker fielding of technological improvements, cost savings, and 

the opportunity for DOD to tap into innovative technologies from firms that 

are new to defense work. Some sponsored technologies are bringing benefits 

to warfighters, such as a small, unmanned aircraft that can detect chemical 

and biological agents, and a device the size of an ink pen that can be used to 

purify water on the battlefield or in disaster areas. Furthermore, DOD 

officials credit the programs with giving senior leaders the flexibility to 

rapidly address current warfighter needs and for highlighting smaller 

technology projects that might otherwise be ignored. 

Long-term success for the programs likely will depend on how well the 

programs are managed and overseen. The programs must have effective 

processes for selecting the best projects, and management and oversight 

processes that will catch potential problems early. Thus far, of the three 

programs, the DACP has adopted the most disciplined and structured 

process for selecting and managing projects, and has encountered few 

problems managing projects. However, the program has had some 

difficulties processing the large number of proposals it receives. The TTI 

has also established selection criteria and processes, but it is unclear the 

extent to which it is reaching its intended audience and has had less success 

in tracking its projects. The Quick Reaction Fund has the least structured 

processes of the three programs—a deliberate approach seen as providing 

the flexibility needed to field innovations rapidly. It has had some difficulty 

selecting, managing and tracking projects. 

The Department of Defense (DOD) 

and Congress both recognize that 

Defense technology innovations 

sometimes move too slowly from 

the lab to the field. Three new 

programs have been recently 

created in DOD to help speed and 

enhance the transition of new 

technologies. A report 

accompanying the fiscal year 2003 

National Defense Authorization Act 

required GAO to review two of 

these programs—the Technology 

Transition Initiative (TTI) and 

Defense Acquisition Challenge 

Program (DACP). The first is 

designed to speed transition of 

technologies from DOD labs to 

acquisition programs and the 

second is designed to introduce 

cost-saving technologies from 

inside and outside DOD. We were 

also asked to review the Quick 

Reaction Fund, which is focused on 

rapidly field testing promising new 

technology prototypes. We 

assessed the impact the programs 

had on technology transition and 

the programs’ selection, 

management and oversight, and 

assessment practices. 

What GAO Recommends

GAO recommends that DOD 

develop data and measures that can 

be used to assess short- and longterm impacts of the programs and 

take other actions to strengthen 

selection, management and 

oversight. DOD agreed with our 

recommendations as they related 

to the DACP and TTI programs, but 

does not believe they apply to the 

Quick Reaction Fund program. 
Page i GAO-05-480 Defense Technology 

Letter 1

Results in Brief 3 

Background 4 

Technology Transition Programs Offering Benefits, but It Is too 

Early to Determine Their Impact 9 

Selection, Management and Oversight, and Assessment Processes 

Could Be Improved by Adopting Additional Practices 13 

Conclusions 23 

Recommendations for Executive Action 23 

Agency Comments and Our Evaluation 24 

Appendix I Comments from the Department of Defense 28 

Tables 

Table 1: The Office of the Secretary of Defense Sponsored 

 Technology Transition Programs 6 

Table 2: Technology Transition Programs 8 

Table 3: Examples of Projects 8 

Table 4: Projects Completed 12 

Table 5: Details of Selection Process 16 

Table 6: Details on Management and Oversight 20 

Figures 

Figure 1: Funding for TTI, DACP, and Quick Reaction Fund 7 

Figure 2: RAM Optic Assembly and Missile Launch 10 

Figure 3: DARPA Water Purification System 11 

Contents 
Page ii GAO-05-480 Defense Technology 

Abbreviations 

ACTD Advanced Concept Technology Demonstration 

DACP Defense Acquisition Challenge Program 

DARPA Defense Advanced Research Projects Agency 

DOD Department of Defense 

FCT Foreign Comparative Testing 

MANTECH Manufacturing Technology Program 

S&T science and technology 

TRL technology readiness level 

TTI Technology Transition Initiative 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
DRAFT 

Page 1 GAO-05-480 Defense Technology 

June 17, 2005 

The Honorable John Warner 

Chairman 

The Honorable Carl Levin 

Ranking Minority Member 

Committee on Armed Services 

United States Senate 

The Honorable Duncan L. Hunter 

Chairman 

The Honorable Ike Skelton 

Ranking Minority Member 

Committee on Armed Services 

House of Representatives 

Over the past 3 years, the Department of Defense (DOD) has invested 

almost $69 billion1

 on a wide range of advanced technology development 

projects—from detecting and neutralizing deeply buried facilities that 

protect weapons of mass destruction, to miniaturizing power and energy 

technologies to reduce the weight of gear soldiers carry into battle, to 

improving access to space. As we have previously reported, the majority of 

these dollars are spent within large weapons programs that have taken too 

long to get to the warfighter, in large part because these programs often 

attempt to incorporate technology advances that have not been proven. 

Invariably, this practice has resulted not only in large cost and schedule 

increases for large programs, but less available funding and commitment 

for small-scale development. 

Both DOD and Congress recognize these as well as other problems in 

transitioning technology and have initiated a number of programs over the 

past decade aimed at spurring and fielding smaller-scale technology 

advances. Recently, the fiscal year 2003 National Defense Authorization 

Act2

 required DOD to establish two new programs, the Technology 

 1

 This represents funding for applied research, advanced technology development projects, 

and advanced component and development prototypes. It does not include basic research. 

2

 Bob Stump National Defense Authorization Act for Fiscal Year 2003, P.L. 107-314, §§ 242 

and 243, Dec. 2, 2002. 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-05-480 Defense Technology 

Transition Initiative (TTI) and the Defense Acquisition Challenge Program 

(DACP)—the first designed to speed transition of technologies from 

DOD’s science and technology (S&T) programs to acquisition programs 

and the second designed to introduce innovative and cost-saving 

technologies from inside and outside DOD. In addition, DOD established 

the Quick Reaction Fund to rapidly field-test promising new technology 

prototypes. All three programs target relatively small-scale projects and 

their total combined annual budget is about $64 million in fiscal year 

2005—a very small portion of the overall dollars dedicated to technology 

research and development. We were required by the conference report 

accompanying the act to assess the implementation of the first two 

programs and were later asked by your offices to also review the Quick 

Reaction Fund. As discussed with your offices, our specific objectives 

were to assess (1) the impact the programs have had on technology 

transition and (2) practices that have helped or hindered the selection, 

management and oversight, and assessment of the programs. 

In conducting our work, we interviewed the managers of the three 

programs we reviewed and analyzed documents that detailed how the 

programs have been implemented, expected project benefits, and the 

types of performance metrics they used to gauge individual project and 

overall program success. We met with several high-level DOD officials 

who were appointed to a council to oversee the Technology Transition 

Initiative, and we observed meetings of a working group in charge of 

supporting the same initiative. In addition, we judgmentally selected 24 of 

68 projects funded by the three transition programs for more in-depth 

review, targeting those that had already been completed or were in the 

completion stages as well as assuring we had projects representing all of 

the military services and key Defense agencies. We conducted interviews 

with the managers of these 24 projects to discuss implementation, 

management, and oversight activities. We analyzed pertinent project 

documents, including original proposals and quarterly status reports. We 

also analyzed documents relating to the transition programs as well as 

broader transition issues and conducted interviews with officials in DOD, 

the Army, the Navy, the Air Force, the Marine Corps, the Defense 

Advanced Research Projects Agency, the Missile Defense Agency, the 

Defense Threat Reduction Agency, Defense Information Systems Agency, 

Special Operations Command, Joint Chiefs of Staff, and military 

department research laboratories. We relied on our previous best 

practices work, other transition program guidance, and general 

management practices as criteria when assessing the program offices’ 

efforts to select, manage and oversee, and assess projects. We also 

analyzed DOD budget documents to identify program funding, as well as 
Page 3 GAO-05-480 Defense Technology 

applied research and advanced technology development funding for fiscal 

years 2003 to 2009 and confirmed with DOD officials responsible for 

maintaining this information that our analysis was correct. Our review was 

conducted from June 2004 to April 2005 in accordance with generally 

accepted government auditing standards. 

It is too soon for us to determine the impact the three programs are having 

on technology transition within DOD because, at the time we selected 

projects for review, only 11of the 68 projects that were funded in fiscal 

years 2003 and 2004 had been completed, and, of those, only 4 were 

actually providing full capability to users. Moreover, the three transition 

programs have limited measures to gauge individual project success and 

return on investment. However, the Technology Transition Initiative and 

the Defense Acquisition Challenge Program, which are being implemented 

consistent with congressional intent and the Quick Reaction Fund are 

expected to result in several benefits. For example, each tout benefits 

such as quicker fielding of new or improved technologies, cost savings, 

and in the case of the Defense Acquisition Challenge Program, the 

opportunity for DOD to use new, innovative technologies from smaller 

firms and companies that have not done business with DOD in the past. In 

addition, some sponsored technologies are providing valued capabilities, 

such as the ability to detect chemical and biological agents via small 

unmanned aircraft and new ways of purifying water on the battlefield or 

disaster zones. DOD officials also believe the programs are worthwhile 

from the standpoint of giving senior leaders the flexibility to rapidly 

address current warfighter needs and by providing visibility for smaller 

technology projects that may not be high enough on an acquisition 

program’s priority list to receive funding. As a result, DOD plans to 

increase funding collectively for these programs from about $24 million in 

fiscal year 2003 to about $94 million by fiscal year 2009. 

Whether the three programs will be successful over the long run will likely 

hinge on how well they are managed and overseen, particularly if DOD’s 

investment increases as planned. Specifically, it will be important for the 

programs to have effective processes for selecting projects, to ensure that 

the best possible candidates are chosen and that the technologies 

themselves are ready for final testing and evaluation stages. It will also be 

vital that they instill effective management and oversight processes, so 

that they can identify and correct problems before they throw projects off 

track and so that they can sustain acquisition program commitment. In 

addition, given the importance of enhancing technology transition within 

DOD and the expectation that the investment in these programs will grow, 

Results in Brief 
Page 4 GAO-05-480 Defense Technology 

it will be important for all three programs to demonstrate to others that 

they are providing a worthwhile return on investment. To date, the DACP 

has adopted a fairly disciplined and structured process for selecting and 

managing projects. While its selection process has been slowed due to an 

overwhelming response and it has not yet completed a project, the 

program has encountered few problems in managing projects. The TTI has 

also established criteria and processes, but had less success in marketing 

the program and in tracking progress of individual projects. The Quick 

Reaction Fund had the least structured process and criteria, believing that 

a high degree of flexibility is needed in order to get technology prototypes 

quickly out to the field, where they can immediately impact military 

operations. At the same time, it had some difficulty in selecting, managing, 

and tracking the status of projects. 

All three programs are continuing to strengthen their management 

processes. This report recommends that DOD develop data and measures 

that can be used to assess short- and long-term impacts of the programs 

and take other actions, as appropriate, to further strengthen selection, 

management, and oversight as investments increase. DOD agreed with our 

recommendations as they related to the DACP and TTI programs. 

However, since the Quick Reaction Fund is meant to quickly test a new 

technology, DOD did not believe the recommendations should apply to 

that program. 

DOD relies on its research laboratories and test facilities as well as 

industry and academia to develop new technologies and systems that 

improve and enhance military operations and ensure technological 

superiority over adversaries. Yet, historically, DOD has experienced 

problems in bringing technologies out of the lab environment and into real 

use. At times, technologies do not leave the lab because their potential has 

not been adequately demonstrated or recognized. In other cases, 

acquisition programs—which receive the bulk of DOD’s funding in 

research, development, testing and evaluation of technology—are simply 

unwilling to fund final stages of development of a promising technology, 

preferring to invest in other aspects of the program that are viewed as 

more vital to success. Other times, they choose to develop the 

technologies themselves, rather than rely on DOD labs to do so—a 

practice that brings cost and schedule risk since programs may well find 

themselves addressing problems related to technology immaturity that 

hamper other aspects of the acquisition process. And often, DOD’s 

budgeting process, which requires investments to be targeted at least 2 

years in advance of their activation, makes it difficult for DOD to seize 

Background 
Page 5 GAO-05-480 Defense Technology 

opportunities to introduce technological advances into acquisition 

programs. In addition, it is challenging just to identify and pursue 

technologies that could be used to enhance military operations given the 

very wide range of organizations inside and outside of DOD that are 

focused on technology development and the wide range of capabilities 

that DOD is interested in advancing. 

In recognizing this array of challenges, DOD and Congress have 

established a number of “technology transition” programs, each with a 

particular focus. (See table 1.) The Advanced Concept Technology 

Demonstration (ACTD) program, for example, was initiated by DOD in 

1994 as a way to get technologies that meet critical military needs into the 

hands of users faster and at less cost than the traditional acquisition 

process. Under this program, military operators test prototypes that have 

already been developed and matured in realistic settings. If they find the 

items to have military utility, DOD may choose to buy additional quantities 

or just use the items remaining after the demonstration. In 1980, DOD 

established the Foreign Comparative Testing (FCT) Program to identify, 

evaluate, and procure technologies that have already been developed and 

tested in other countries—saving DOD the costly burden of maturing the 

technology itself. Other programs include those that seek to quickly 

identify and solve production problems associated with technology 

transition (the Manufacturing Technology Program—MANTECH) and to 

partner with the commercial sector in completing projects that are useful 

to both military and industry (the Dual Use Science and Technology 

program). Even taken together, however, these programs represent a very 

small portion of DOD dollars spent on applied research and advanced 

technology development—about $9 billion annually—and considerably 

less of total money spent on the later stages of technology development, 

which includes an additional $60 billion spent on advanced component 

development and prototypes, largely within weapons acquisition 

programs. As such, they cannot single-handedly overcome transition 

problems, but rather demonstrate various ways to ease transition and 

broaden participation from the industrial base. 
Page 6 GAO-05-480 Defense Technology 

Table 1: The Office of the Secretary of Defense Sponsored Technology Transition Programs 

Program Purpose 

Fiscal year 2004 

funding 

Manufacturing Technology 

(MANTECH) Program 

Mature and validate emerging manufacturing technologies to facilitate 

production of new affordable and sustainable technologies 

$256 million 

Advanced Concept Technology 

Demonstration (ACTD) 

Expedite the transition of maturing technologies from the developers to the 

users by having military operators and users test technology prototypes 

 223 million 

Defense Production Act Title III 

Program 

Create, maintain, modernize, or expand the productive capacities of domestic 

sources for critical components, technology items and industrial resources 

essential for national defense. 

 78 million 

Foreign Comparative Testing 

(FCT) Program 

Identify, test, and evaluate mature foreign technologies that DOD can adopt 

rather than having to sponsor development itself 

 34 million 

Defense Acquisition Challenge 

Program (DACP) 

Identify and introduce innovative and cost-saving technology or products from 

within DOD’s science and technology community as well as externally into 

existing DOD acquisition programs 

 17 million 

Quick Reaction Fund Identify and rapidly field-test promising new technologies within DOD’s 

budget execution years 

 15 million 

Technology Transition Initiative 

(TTI) 

Facilitate the rapid transition of new technologies from DOD science and 

technology programs into acquisition programs 

 13 million 

Joint Advanced Warfighting 

Program 

Assist in developing the capabilities envisioned in Joint Vision 2010 by 

leveraging advanced technology, innovative operational concepts, and new 

organizational structures 

 10 million 

Dual Use Science and 

Technology 

Partner with industry to jointly fund the development of dual use technologies 

needed to maintain technological superiority on the battlefield 

 0a

Tech Link Mission is threefold: (1) integrate advanced commercial-sector technologies 

into DOD systems, particularly from nontraditional defense contractors; (2) 

spin-off DOD-developed technologies to the commercial sector to make these 

technologies more affordable for military acquisition; and (3) establish 

collaborative research and development projects with the private sector for 

cost-sharing of new dual-use technology development 

 3 million 

Independent Research and 

Development 

Increase efficiency and productivity of contractor independent research and 

development activities 

 0b

Source: DOD. 

a

The Air Force provided $10 million for Dual Use Science and Technology projects in fiscal year 2004. 

b

OSD does not spend any money on this program. Rather, DOD compiles data on the amount of 

independent research and development spending by major defense contractors and reports this 

information annually. 

Three of the more recent initiatives include the TTI and DACP, both 

established by Congress in fiscal year 2003, and the Quick Reaction Fund, 

established by DOD the same year. TTI is focused on speeding the 

transition of technologies developed by DOD’s S&T programs into 

acquisition programs, while DACP is focused on introducing innovative 

and cost-saving technologies developed inside and outside DOD. The 

Quick Reaction Fund is focused on field testing technology prototypes. 
Page 7 GAO-05-480 Defense Technology 

All three programs are managed by DOD’s Office of Defense Research and 

Engineering, which reports to the Under Secretary of Defense for 

Acquisition, Technology and Logistics. 

Together, these three programs received about $64 million in fiscal year 

2005–a fraction of the $9.2 billion DOD invested in applied research and 

advanced technology development the same year and a relatively small 

budget compared to some of the other transition programs. Nevertheless, 

DOD has been increasing its investment in these programs and plans to 

further increase it over the next few years. (See figure 1.) 

Figure 1: Funding for TTI, DACP, and Quick Reaction Fund 

Table 2 highlights similarities and differences between DACP, TTI, and 

Quick Reaction Fund. Table 3 provides examples of projects that have 

already been funded. 

0

10

20

30

40

50

60

70

80

90

100

2003 2004 2005 2006 2007 2008 2009

Dollars in millions

Source: DOD.

Fiscal year

Planned funding
Page 8 GAO-05-480 Defense Technology 

Table 2: Technology Transition Programs 

Program Objective Scope 

Completion time 

frames 

Total fiscal year 

2003 and 2004 

funding 

Projects funded to 

date 

DACP Introduce innovative and 

cost-saving technologies or 

products into existing 

acquisition programs 

Other objectives of the 

program include expansion 

of opportunities for 

emerging defense suppliers 

and widening the U.S. 

defense industrial base 

Any person or 

activity inside or 

outside DOD 

Up to 3 years; 12 

to 24 months is 

preferred 

$29.83 million 30 projects ranging 

in value from 

$100,000 to $2.54 

million 

TTI Speed transition of 

technologies from DOD 

science and technology 

programs to acquisition 

programs 

DOD science and 

technology 

programs 

Up to 4 years; up 

to 24 months is 

preferred 

$19.27 million 21 projects ranging 

from $323,000 to 

$5.33 million 

Quick Reaction Fund Field and test prototypes 

that respond to immediate 

and emerging warfighter 

needs 

DOD science and 

technology 

programs 

6 to 12 months $21.54 million 17 projects ranging 

from $240,000 to 

$3.5 million 

Source: DOD. 

Table 3: Examples of Projects 

DACP TTI Quick Reaction Fund 

• A collapsible stock for the M-4 rifle that 

makes it more effective in an urban 

environment and ergonomic 

• An enhanced optics system for the Rolling 

Airframe Missile that will permit allweather operational capability 

• A new process for repairing titanium 

cracks in the B-2 aircraft that will improve 

mission readiness rates and has the 

potential to save DOD millions of dollars 

in replacement costs 

• A new welding process for naval 

propellers that will improve weld repair 

techniques and increase the strength and 

quality of the casting in repaired areas 

• An ink-pen size device that purifies water 

• A custom-design earplug that provides 

blast noise protection and increased 

communication capabilities 

• A low-cost, flame resistant coverall for 

infantry soldiers 

• A new coating for H-46 helicopter engine 

blades that reduces the rate of premature 

engine removals related to desert 

operations 

• Extending the field of view for a sensor 

targeting system that is capable of 

detecting, classifying, and locating 

weapons firing in real time 

• A greaseless M-4 rifle that is more 

suitable for desert operations 

• A special material that strengthens the 

physical structure of an unmanned aerial 

vehicle 

• A chemical and biological detection 

capability for small unmanned aerial 

vehicle 

• An accuracy enhancement for unguided 

rockets 

Source: DOD. 
Page 9 GAO-05-480 Defense Technology 

The three transition programs, which are being implemented consistent 

with congressional intent, reported that benefits can already be seen in 

many projects, including improvements to performance, affordability, 

manufacturability, and operational capability for the warfighter. While 

such benefits may have eventually been achieved through normal 

processes, program officials believe the three transition programs enabled 

DOD to realize them sooner due to the immediate funding that was 

provided to complete testing and evaluation as well as attention received 

from senior managers. DOD officials also emphasized that these programs 

are calling attention to emerging technologies that have the potential to 

offer important performance gains and cost savings but, due to their size 

and relative obscurity, may otherwise be overlooked when competing 

against other, larger-scaled technologies and/or technologies already 

deemed as vital to a particular acquisition program’s success. Another 

benefit cited with the DACP is an expansion of the Defense industrial 

base, because the program invites participation from companies and 

individuals that have not been traditional business partners with DOD. 

Nevertheless, it is too early for us to determine the impact that these 

programs have had on technology transition. At the time we selected 

projects to review, few projects had been completed. In addition, the 

programs had limited performance measures to gauge success of 

individual projects or track return on investment over time. 

The following examples highlight some of the reported benefits of 

individual projects. 

• Host Weapons Shock Profile Database—DOD spends a significant 

amount of time and resources to test new accessories (e.g., night vision 

scopes) for Special Operations Forces weapons. Currently, when new 

accessories are added, they must go through live fire testing to 

determine if they work properly and will meet reliability standards. 

This process could take several months to complete as the acquisition 

office must schedule time at a test range to complete the testing. 

Program officials must also identify and pay for an expert to conduct 

the testing and pay for ammunition that will be used in the test. The 

DACP is funding the test and evaluation of a database that will simulate 

the vibration or shock of various machine guns in order to test new 

accessories for that gun. This will eliminate almost all of the testing 

costs mentioned above and greatly reduce the amount of time needed 

for testing. The project office estimates that it will save almost $780,000 

per year in ammunition costs alone. 

Technology Transition 

Programs Offering 

Benefits, but It Is too 

Early to Determine 

Their Impact 
Page 10 GAO-05-480 Defense Technology 

• Enhanced Optics for the Rolling Airframe Missile—The Rolling 

Airframe Missile is part of the Navy’s ship self-defense system to 

counter attacks from missiles and aircraft. However, the missile 

experiences operational deficiencies in certain weather conditions, and 

the program has had problems producing components for the optics. 

The DACP is providing funding to a small business to test and evaluate 

a new sapphire dome and optics for the missile to resolve these 

problems. Program officials estimate that program funding will 

accelerate the development of a solution 1 to 2 years earlier than 

anticipated. If the DACP project is successful, an added benefit will be 

that the dome material will be readily available from manufacturers in 

the United States instead of a single overseas supplier, as is currently 

the case. 

Figure 2: RAM Optic Assembly and Missile Launch 

• Water Purification System— For tactical situations in which deployed 

troops do not have quick and easy access to potable water, the pen will 

allow soldiers to treat up to 300 liters of any available, non-brackish 

water source on one set of lithium camera batteries and common table 

salt. The pen eliminates the risk of the soldiers’ exposure to diseases 

and bio-chemical pollutants. TTI funding was used to purchase 

approximately 6,600 water pens for distribution to the military services. 

In addition, TTI funding enabled this item to be placed on a General 

Source: Raytheon Missile Systems. Source: Raytheon Missile Systems.
Page 11 GAO-05-480 Defense Technology 

Services Administration schedule, where approximately 8,600 

additional water pens have been purchased by DOD customers. DOD 

and the company that produces the pen donated hundreds of these 

systems to the tsunami relief effort in Southeast Asia. 

Figure 3: DARPA Water Purification System 

• Dragon Eye—The Dragon Eye is a small, unmanned aerial vehicle with 

video surveillance capabilities used by the marines. To address the 

concerns over a chemical and biological threat to troops in Iraq, the 

Quick Reaction Fund funded the integration of a small chemical 

detection and biological collection device on the Dragon Eye. The lowflying Dragon Eye can tell troops in real time where and when it is 

collecting samples, and in cases where a plume is detected, it can 

determine the direction the plume is moving. According to program 

officials, Quick Reaction funding allowed the chemical and biological 

detection capability to be developed 2 years ahead of schedule. The 

technology was available to a limited number of Special Operations 

Forces at the beginning of the Iraqi conflict. 

Source: Defense Advanced Research Projects Agency.
Page 12 GAO-05-480 Defense Technology 

Despite the evident benefits of certain projects, it is too early to determine 

the programs’ impact on technology transition. At the time we selected 

projects for review, only 11 of 68 projects started in fiscal years 2003 and 

2004 had been completed, and, of those, only 4 were currently available to 

warfighters. These include one TTI project—a miniaturized water 

purification system that is now being offered through a General Services 

Administration schedule to the warfighter—and three projects under the 

Quick Reaction Fund, including the Dragon Eye chemical and biological 

sensor, planning software used by Combatant Commanders dealing with 

weapons of mass destruction targets, and special materials that strengthen 

unmanned aerial vehicles. 

Since the time we selected projects, 20 have been reported as completed 

and 13 have been reported as available to warfighters. The latest project 

completion information by program is shown in Table 4. 

Table 4: Projects Completed 

Program 

Projects funded since 

the program began 

Projects completed

 as of 2/28/2005

TTI 21 11

DACP 30 0

Quick Reaction Program 17 9

Total 68 20

Source: DOD. 

It is important to note that, even though 20 TTI and Quick Reaction Fund 

projects are considered to be complete, not all of the capabilities have 

reached the warfighter. For example: 

• The T58 Titanium Nitride Erosion Protection is a TTI project that has 

transitioned to an acquisition program but has not yet reached the 

warfighter. The project is being developed to improve the reliability of 

T-58-16A helicopter engines used in Iraq. While the compressor blades 

are designed for 3000 operating hours, the Marine Corps has had to 

remove engines with fewer than 150 operational hours due to sand 

ingestion. The project received funding from the TTI in fiscal years 

2003 and 2004 to develop a titanium nitride coating for engine blades 

that would significantly mitigate erosion problems in a desert 

environment. According to program documents, blades with the new 

coating will be included in future production lots beginning in July 

2005. Modification kits will also be developed for retrofitting engines 

already produced. Program officials expect the project will double the 
Page 13 GAO-05-480 Defense Technology 

compressor life of the engine in a sand environment and save about $12 

million in life-cycle costs through fiscal year 2012. 

• The Ping project, funded by the Quick Reaction Fund, is an example of 

a project that is considered complete, but a prototype was never field 

tested by the warfighter. The Air Force had hoped to broaden the 

capability of the microwave technology it used to identify large objects 

such as tanks or cars to also detect concealed weapons or explosives—

such as suicide vests. However, the project was cancelled after some 

initial testing revealed that the technology was not accurate enough to 

determine the microwave signatures of small arms or suicide vests that 

could have numerous configurations and materials. DOD officials 

stated that, even though the project was unsuccessful, they gained a 

better understanding of microwave technologies and are continuing to 

develop these technologies for other applications. 

The long-term impact of the programs will also be difficult to determine 

because the technology transition programs have a limited set of metrics 

to gauge project success or the impact of program funding over time. 

While each funded project had to identify potential impact in terms of 

dollar savings, performance improvements, or acceleration to the field as 

part of the proposal process, actual impact of specific projects as well as 

the transition programs as a whole is not being tracked consistently. The 

value of having performance measures as well as DOD’s progress in 

adopting them for these transition programs is discussed in the next 

section of this report. 

To ensure that new technologies can be effectively transitioned and 

integrated into acquisitions, transition programs need to establish effective 

selection, management and oversight, and assessment processes. For 

example, programs must assure that proposals being accepted have 

established a sound business case, that is, technologies being transitioned 

are fairly mature and in demand and schedules and cost for transition fit 

within the program’s criteria. Once projects are selected, there needs to be 

continual and effective communication between labs and acquisition 

programs so that commitment can be sustained even when problems arise. 

To assure that the return on investment is being maximized, the impact of 

programs must be tracked, including cost and time savings as well as 

performance enhancements. Our work over the past 7 years has found that 

high-performing organizations adopt these basic practices as a means for 

successfully transitioning technologies into acquisitions. Moreover, several 

larger DOD technology transition programs, such as the ACTD program 

Selection, 

Management and 

Oversight, and 

Assessment Processes 

Could Be Improved by 

Adopting Additional 

Practices 
Page 14 GAO-05-480 Defense Technology 

and some Defense Advanced Research Projects Agency (DARPA) projects, 

embrace similar practices and have already developed tools to help 

sustain commitment, such as memorandums of agreement between 

technology developers and acquirers. Both DARPA and ACTD manage 

budgets that are considerably larger than the programs included in this 

review. As such, the level of detail and rigor associated with their 

management processes may not be appropriate for TTI, DACP, or Quick 

Reaction Fund. However, the concepts and basic ingredients of their 

criteria and guidance could serve as a useful starting point for the smaller 

programs to strengthen their own processes. 

The three programs we reviewed adopted these practices to varying 

degrees. Overall, the DACP had disciplined and well-defined processes for 

selecting and managing, and overseeing projects. The TTI had disciplined 

and well-defined processes for selecting projects, but less formal 

processes for management and oversight. The Quick Reaction Fund was 

the least formal and disciplined of all three, believing that success was 

being achieved through flexibility and a high degree of senior management 

attention. All three programs had limited performance measures to gauge 

progress and return on investment. Generally, we found that the more the 

programs adopted structured and disciplined management processes, the 

fewer problems they encountered with individual efforts. 

Success in transitioning technologies from a lab to the field or an 

acquisition program hinges on a transition program’s ability to choose the 

most promising technology projects. This includes technologies that can 

substantially enhance an existing or new system either through better 

performance or cost savings and those with technologies at a fairly mature 

stage, in other words, suitable for final stages of testing and evaluation. A 

program can only do this, however, if it is able to clearly communicate its 

purpose and reach the right audience to submit proposals in the first 

place. It is also essential that a program have a systematic process for 

determining the relative technical maturity of the project as well as for 

evaluating other aspects of the project, such as its potential to benefit 

specific acquisition programs. Involving individuals in the selection 

process from various functions within an organization—e.g., technical, 

business, and acquisition—further helps to assure that the right projects 

are being chosen and that they will have interested customers. An 

analytical tool that can be particularly useful in selecting projects is a 

technology readiness level (TRL) assessment, which assesses the maturity 

level of a technology ranging from paper studies (level 1), to prototypes 

that can be tested in a realistic environment (level 7), to an actual system 

that has proven itself in mission operations (level 9). Our prior work has 

Selection 
Page 15 GAO-05-480 Defense Technology 

found TRLs to be a valuable decision-making tool because it can presage 

the likely consequences of incorporating a technology at a given level of 

maturity into a product development. 

As further detailed in table 5, the DACP program has a fairly robust 

selection process. The program relies on internet-based tools to 

communicate its goals and announce its selection process and ensure a 

broad audience is targeted. As a result, it receives a wide array of 

proposals from which the program office assesses their potential for 

generating improvements to existing programs as well as actual interest 

from the acquisition community. The DACP also solicits technical experts 

from inside and outside DOD to assess potential benefits and risks. Once 

the number of projects is whittled down, the program takes extra steps to 

secure commitments from acquisition program managers as well as 

program executive officers. The program’s popularity, however, has had 

some drawbacks. For example, the sheer number of proposals have 

tended to overwhelm DACP staff and slowed down the selection process, 

particularly in the first year. In addition, while technology benefits and 

risks are assessed in making selection decisions, DACP does not formally 

confirm the technology readiness levels being reported. The TTI program 

also has a fairly rigorous selection process, with specific criteria for 

selection, including technology readiness, and a team of representatives of 

higher-level DOD S&T officials in charge of disseminating information 

about the program in their organization, assessing their organization’s 

proposals based on TTI criteria as well as other criteria they developed, 

and ranking their top proposals. The program, which is focused on 

reaching DOD’s S&T community rather than outside industry, had been 

communicating in a relatively informal manner and it was unclear during 

our review the extent to which the TTI was reaching its intended audience. 

The program, however, has been taking steps to strengthen its ability to 

reach out to the S&T community. In addition, TTI does not confirm TRLs. 

At the time of our review, the Quick Reaction Program selection process 

was much less structured and disciplined than DACP and TTI. This was by 

design, because the program wants to select projects quickly and get them 

out to the field where they can be of use in military operations in Iraq, 

Afghanistan, and elsewhere. However, the program experienced problems 

related to selection and as a result—for example, significant gaps in 

knowledge about technology readiness led to the cancellation of one 

project. To program officials, the risk associated with less formal selection 

is worth the benefit of being able to move rapidly evolving technologies 

into an environment where they can begin to immediately enhance 

military operations and potentially save lives. Nevertheless, the program 

is now taking steps to strengthen selection processes. 
Page 16 GAO-05-480 Defense Technology 

Table 5: Details of Selection Process 

 DACP TTI Quick Reaction Fund 

How program is 

communicated 

Each year, the program office issues 

an electronic broad agency 

announcement to communicate 

program goals and funding availability 

to both internal and external 

technology developers. Program 

officials are satisfied that this 

approach has helped them reach their 

target audience as evidenced by the 

large number of proposals received 

overall and also by the number 

received by small and non-traditional 

businesses. For example, almost 400 

proposals were received during the 

first and second solicitation periods 

combined. Of those, about 52 percent 

were submitted by small and nontraditional businesses. 

TTI established a working group 

comprised of senior-level S&T 

representatives from each of the military 

services to help disseminate information 

about program goals and funding 

availability to their respective 

organizations. 

It is unclear whether the TTI is reaching 

its intended audience. At the time of our 

review, six project managers we met with 

stated they found out about the TTI by 

happenstance and not through any 

formal mechanism. Furthermore, even 

though they are now aware of the 

program, many of these project 

managers still have not seen the 

program formally advertised throughout 

their respective organizations. The 

program office has been working to 

improve its ability to reach its target 

audience and assure that it is 

considering the best projects for funding 

by including presentations about the 

program at S&T forums and seminars 

and developing a program brochure. 

Communication was informal. 

Each military service and 

selected defense agency was 

asked to submit its top 10 

proposals. 

A new website was recently 

developed to automatically 

distribute the proposal data call 

and collect proposals. 
Page 17 GAO-05-480 Defense Technology 

 DACP TTI Quick Reaction Fund 

How projects are 

evaluated and 

selected 

DACP’s selection process is modeled 

after DOD’s FCT program, which 

already has a well-defined selection 

process. The process begins with 

program officials performing an initial 

review of all proposals to filter out the 

ones that are not relevant or feasible. 

Next, proposals are reviewed by a 

panel of technical experts. If the 

acquisition program office and prime 

contractor are interested in the 

proposed technology, the acquisition 

program office must develop a final, 

more in-depth proposal for the 

program office to consider for funding. 

Overall, projects are assessed against 

24 criteria, including acquisition 

funding commitment, technology 

readiness, and whether the project 

addresses a valid requirement, which 

are identified in the broad area 

announcement used to solicit 

proposals. 

However, the program office has had 

some problems processing the large 

volume of proposals received with a 

small staff of about 3.5 full-time 

equivalent positions. Currently, it 

takes about 9 months for a project to 

be selected for funding. According to 

the program manager, the expert 

technical review is the most timeconsuming part of the process. To 

date, the DACP has had to solicit the 

services of over 1,000 experts from 

within the Department, private 

industry, and academia nationwide to 

review proposals. Program officials 

are now considering identifying thrust 

areas for prioritizing proposals as a 

way to reduce cycle time. This was 

permitted by the legislation that set up 

DACP. 

TTI proposals are judged against a set of 

eight criteria, including time needed to 

transition, technology readiness, whether 

the project satisfies a critical 

requirement, and willingness on the part 

of an acquisition program to share costs 

of transition. Working group members 

play a key role in the selection process 

by reviewing proposals from their 

organization and ranking their top 

proposals for program office 

consideration. Program officials believe 

this approach expedites the selection 

process because it reduces the number 

of proposals the program office needs to 

review and also gives the officials an 

idea of which proposals are the most 

important to the service or agency. 

Selection was not based on well 

defined criteria at the time of 

our review. Instead, 2-page 

proposals were evaluated by 

the office running the Quick 

Reaction Fund with an eye 

toward whether they could bring 

immediate benefits to ongoing 

military operations. It usually 

takes 30 to 45 days between 

the time the solicitation is 

issued to the time projects are 

selected for funding. 
Page 18 GAO-05-480 Defense Technology 

 DACP TTI Quick Reaction Fund 

How TRLs are 

used 

DACP guidance specifically states 

that technology must have been 

demonstrated using a representative 

model or prototype system in a 

relevant environment, the equivalent 

of a technology readiness level 6 on a 

scale of 1-9, to be considered for 

funding. Technology readiness is also 

considered by technical experts 

reviewing proposals. However, DACP 

does not have a formal mechanism in 

place to ensure that the technology 

readiness levels have actually been 

achieved. 

One of the four criteria program officials 

consider during the selection process is 

whether the proposed new technology is 

mature. However, TTI does not have a 

formal mechanism to confirm technology 

readiness levels identified in the 

proposals. 

Program officials have limited 

information about maturity 

levels. Proposals are required 

to disclose technical risks to 

assist in this evaluation, but do 

not identify specific technology 

readiness levels. We identified 

one project where the Quick 

Reaction Fund invested $1.5 

million but later needed to stop 

the project because technology 

was not as mature as the 

project manager originally 

thought. 

According to the program 

manager, TRL maturity will be 

included as an evaluation factor 

in the fiscal year 2006 

assessment process. 

Source: GAO analysis. 

Selecting promising projects for funding is not enough to ensure 

successful transition. Program managers must also actively oversee 

implementation to make sure that project goals are being met and the 

program is working as intended and to identify potential barriers to 

transition. They must also sustain commitment from acquirers. Moreover, 

the transition program as a whole must have good visibility over progress 

and be positioned to shift attention and resources to problems as they 

arise. 

A tool that has proven particularly useful for other established DOD 

technology transition programs is designating individuals, preferably with 

experience in acquisitions or operations and/or the S&T world, as “deal 

brokers” or agents to facilitate communication between the lab and the 

acquisition program and to resolve problems as they arise. DARPA 

employs such individuals, for example, as well as some Navy-specific 

transition programs. Both have found that these agents have been integral 

to transition success. Another tool that is useful for sustaining 

commitment from the acquirers is a formal agreement. Our previous work 

found that best practice companies develop agreements with cost and 

schedule targets to achieve and sustain buy-in and that the agreements are 

modified as a project progresses to reflect more specific terms for 

accepting or rejecting a technology. DARPA develops similar agreements 

that describe how projects will be executed and funded as well as how 

projects will be terminated if the need arises. The agreements are signed 

Management and Oversight 
Page 19 GAO-05-480 Defense Technology 

by high-level officials, including the director of DARPA and senior-level 

representatives of the organizations DARPA is working with. The ACTD 

program develops “implementation directives” that clarify roles and 

responsibilities of parties executing an ACTD, time frames, funding, and 

the operational parameters by which military effectiveness is to be 

evaluated. The agreements are also signed by high-level officials. 

DACP has fairly robust management and oversight mechanisms. Status is 

monitored via formal quarterly reporting as well as interim meetings 

which, at a minimum, involve the customer, the developer, and the DACP 

project manager. The meetings provide an opportunity to ensure the 

acquisition program is still committed to the project and to resolve 

problems. Though formal memoranda of agreements are not usually 

employed, the program establishes test and evaluation plans that detail 

pass/fail criteria so that funding does not continue on projects that 

experience insurmountable problems. TTI also employs periodic status 

reports and meetings; however, communication has not been as open. In 

two cases, projects ran into significant problems, such as loss of 

acquisition program office support in one case and logistics issues that 

had not been addressed to transition a technology smoothly in the other, 

which had not come to the attention of the TTI program office. As a result, 

the TTI office thought the projects had transitioned when in actuality, 

significant problems still needed to be addressed. Per legislation, TTI had 

also established a formal council comprised of high-level DOD officials to 

help oversee the program; however, the Council has only met once in 

2 years, while the act requires that it meet at least semiannually. In 

addition, there is some confusion among Council members and others we 

spoke with as to what the purpose of the Council should be—that is, 

focused on TTI only or broader transition issues. Congressional officials 

expressed that they intended for the Council to focus on broader 

transition issues and how best to solve them. Although the Quick 

Reaction Fund does not require status reports to assess progress, project 

managers are required to submit after-action reports. However, these were 

not regularly reviewed by the office. We identified several problems that 

arose during transition that were not known to the Quick Reaction Fund 

program manager. The program manager is currently taking steps to 

improve the management and oversight of projects. For example, a 

website has been developed to help monitor and execute the program. 

Among other things, the website will allow for the automatic collection of 

monthly status reports. 
Page 20 GAO-05-480 Defense Technology 

Table 6: Details on Management and Oversight 

 DACP TTI Quick Reaction Fund 

How commitment is 

secured 

DACP employs a formal commitment 

process that includes pass/fail criteria 

that will either sustain commitment or 

result in the cancellation of a project 

based upon testing outcomes. 

DACP guidance includes practices that 

help the project manager achieve and 

maintain buy-in until a technology has 

been successfully transitioned. As part of 

the selection criteria, both the acquisition 

program office and prime contractor must 

agree to accept a technology if it is 

successfully demonstrated. For example, 

one of the projects we reviewed 

established pass/fail criteria for a new 

technology to repair titanium alloy cracks 

on the aft section of the B-2 aircraft. 

Further, selected projects are expected to 

use integrated product teams (which 

include at least the vendor, developer, 

and DACP manager) to exchange 

information and deal with potential 

problems very quickly. 

TTI employs a formal commitment process; 

it has not used formal agreements. 

Initially, all projects must show the acquirer 

has included funding for the technology in a 

future budget if the project is to be 

considered for funding. For the most part, 

once this happens, there is no requirement 

for additional interaction through methods 

like formal agreements. We identified one 

project that was not able to sustain support 

from the acquisition program—the IROS3 

Spartan project, which is intended to 

enhance the Navy’s shipboard protection. 

TTI officials believed it had transitioned to 

an acquisition program, when, in fact, the 

customer had dropped its support. 

According to the project manager, the 

Naval Surface Warfare Center at Crane 

was supposed to develop the first block of 

software for the project and a contractor 

would be selected to develop later blocks. 

However, after successful field-testing, the 

acquisition program office decided that it 

would be less risky to select a contractor to 

develop all three software blocks than to 

hand off the government-developed 

software to the contractor to maintain and 

upgrade. 

No formal commitment 

process used because the 

program goal is to 

demonstrate the military 

utility of emerging new 

technologies through field 

testing, and not necessarily 

to transition those 

technologies at this time. 
Page 21 GAO-05-480 Defense Technology 

 DACP TTI Quick Reaction Fund 

How status is 

tracked 

Project managers are required to submit 

quarterly progress reports and final 

reports once a project has been 

completed. In addition to these reports, 

periodic meetings are held with various 

participants to assess progress. Reports 

contain data on project accomplishments, 

planned actions for the next quarter, 

issues that need to be addressed, the 

transition strategy, and the current 

funding plan. All quarterly reports are 

submitted and maintained on an 

electronic system that can be accessed 

by program officials and the project 

manager. 

Periodic status reports are required and 

interim progress meetings are held, though 

not with the same range of representation 

as DACP. 

We identified one project—the Terminal 

Attack Communications Earplug System—

where problems had arisen during transition 

that had not come to the attention of TTI 

officials. Specifically, problems related to 

how the earplug’s use could be actualized 

(for example, training technicians to pour 

the new, custom ear molds and what 

organization would pay for the training) had 

not been brought to the attention of the TTI 

office. In fact, the office had thought the 

transition had gone smoothly. 

After-action reports are 

required, though not always 

reviewed for purposes of 

tracking progress. 

At the time of our review, 

the Quick Reaction Fund 

manager had problems 

addressing concerns 

identified in status reports 

and obtaining reports from 

some project managers. For 

example, the program 

manager was unaware that 

one project, thought to have 

been successfully field 

tested, actually ran out of 

funding prior to field testing. 

Although the project 

manager reported that the 

project had not been 

completed because it ran 

out of funding, the Quick 

Reaction Fund manager did 

not have enough time to 

thoroughly review the 

report. 

The program manager 

expects the newly 

established web-based 

system to improve his ability 

to track project progress. 

Use of deal 

brokers 

No formal use of “deal brokers”, though 

involvement of vendors, developers, and 

customers in status discussions have 

helped to identify and resolve transition 

problems. 

No formal use of “deal brokers.” No formal use of “deal 

brokers.” 

Source: GAO analysis. 

Though the transition programs we reviewed are relatively small in scale 

compared to other transition programs in DOD, the government’s 

investment is still considerable and it will continue to grow if DOD’s 

funding plans for the programs are approved. As a result, it is important 

that these programs demonstrate that they are generating a worthwhile 

return on investment—whether through cost savings to acquisition 

programs, reduced times for completing testing and evaluation and 

integrating technologies into programs, and/or enhanced performance or 

new capabilities. Developing such information can enable transition 

Assessment 
Page 22 GAO-05-480 Defense Technology 

program managers to identify what is or is not working well within a 

program; how well the program is measuring up to its goals, as well as to 

make trade-off decisions between individual projects. On a broader level, 

it can enable senior managers and oversight officials to compare and 

contrast the performance of transition programs across DOD. 

Finding the right measures to use for this purpose is challenging, however, 

given the wide range of projects being pursued, the different environments 

to which they are being applied, and difficulties associated with measuring 

certain aspects of return on investment. For example, measuring long-term 

cost savings could be problematical because some projects could have 

impacts on platforms and systems that were not part of the immediate 

transition effort. As a result, the best place to start may be with high-level 

or broad metrics or narratives that focus on the spectrum of benefits and 

cost savings being achieved through the program, complemented by more 

specific quantifiable metrics that do not require enormous efforts to 

develop and support, such as time saved in transition or short-term cost 

savings. At this time, however, the transition programs have limited 

measures to gauge individual project success and program impact or 

return on investment in the long term. At best, they are collecting after 

action reports that describe the results of transition projects, and 

occasionally identify some cost savings, but not in a consistent manner. In 

addition, there are inconsistencies in how the reports are being prepared, 

reviewed, and used. The Quick Reaction Fund program manager, in fact, 

had trouble just getting projects to submit after action reports. 

Officials from all three transition programs we reviewed as well as higher 

level officials agreed that they should be doing more to capture 

information regarding return on investments for the programs. They also 

agreed that there may already be readily available starting points within 

DOD. For example, the Foreign Comparative Testing Program has 

established metrics to measure the health, success, and cost-effectiveness 

of the program and has developed a database to facilitate return on 

investment analyses. The program also captures general performance 

enhancements in written narratives. The program has refined and 

improved its metrics over time and used them to develop annual reports. 

The specific metrics established by the FCT program may not be readily 

transferable to DACP, TTI, or the Quick Reaction Fund because the nature 

of FCT projects is quite different—technologies themselves are more 

mature and costs savings are achieved by virtue of the fact that DOD is 

essentially avoiding the cost of developing the technologies rather than 

applying the technologies to improve larger development efforts. 

However, the process by which the program came to identify useful 
Page 23 GAO-05-480 Defense Technology 

metrics as well as the automated tools it uses could be valuable to the 

other transition programs. In addition, DOD has asked the Naval Post 

Graduate School to study metrics that would be useful for assessing the 

ACTD program. The results of this study may also serve as a starting point 

for the transition programs in developing their own ways to assess return 

on investment. 

The ability to spur and leverage technological advances is vital to 

sustaining DOD’s ability to maintain its superiority over others and to 

improve and even transform how military operations are conducted. The 

three new transition programs are all appropriately targeted on what has 

been a critical problem in this regard—quickly moving promising 

technologies from the laboratory and commercial environment into actual 

use. Moreover, by tailoring processes and criteria to focus on different 

objectives, whether that may be saving time or money or broadening the 

industrial base, DOD has had an opportunity to experiment with a variety 

of management approaches and criteria that can be used to help solve 

transition problems affecting the approximately $69 billion spent annually 

on advanced stages of technology development. Already, it is evident that 

an element missing from all three programs is good performance 

measurement. Without having this capability, DOD will not be able to 

effectively assess which approaches are working best and whether the 

programs individually or as a whole are truly worthwhile. In addition, it is 

evident that having well-established tools for selecting and managing 

projects as well as communicating with technology developers and 

acquisitions helps programs to reduce risk and achieve success, and that 

there are opportunities for all three programs for strengthening their 

capabilities in this regard. In light of its plans to increase funding for the 

three programs, DOD should consider actions to strengthen selection and 

management capabilities, while taking into account resources needed for 

implementing them as well as their impact on the ability of the programs 

to maintain flexibility. 

We recommend that the Secretary of Defense take the following five 

actions: 

To optimize DOD’s growing investment in the Technology Transition 

Initiative, the Defense Acquisition Challenge Program, and the Quick 

Reaction Fund, we recommend that the Secretary of Defense direct the 

Under Secretary of Defense (Acquisition, Technology, and Logistics) to 

develop data and measures that can be used to support assessments of the 

Conclusions 

Recommendations for 

Executive Action 
Page 24 GAO-05-480 Defense Technology 

performance of the three transition programs as well as broader 

assessments of the return on investment that would track the long-term 

impact of the programs. DOD could use measures already developed by 

other transition programs, such as FCT, as a starting point as well as the 

results of its study on performance measurement being conducted by the 

Naval Post Graduate School. To complement this effort, we recommend 

that DOD develop formal feedback mechanisms, consisting of interim and 

after action reporting, as well as project reviews if major deviations occur 

in a project. Deviations include, but are not limited to, changes in the 

technology developer, acquirer, or user, or an inability for the technology 

developer to meet cost, schedule, or performance parameters at key points 

in time. 

We also recommend that the Secretary of Defense direct the Under 

Secretary of Defense (Acquisition, Technology, and Logistics) to 

implement the following, as appropriate, for each of the transition 

programs: (1) formal agreements to solidify up-front technology 

development agreements related to cost, schedule, and performance 

parameters that must be met at key points in time and (2) confirmation of 

technology readiness levels as part of the proposal acceptance process. 

In addition, we recommend that DOD identify and implement mechanisms 

to ensure that transition program managers, developers, and acquirers are 

able to better communicate to collectively identify and resolve problems 

that could hinder technology transition. There may be opportunities to 

strengthen communication by improving the structure and content of 

interim progress meetings and possibly even designating individuals to act 

as deal brokers. 

Lastly, as DOD considers solutions to broader technology transition 

problems, we recommend that Secretary of Defense direct the Under 

Secretary of Defense (Acquisition, Technology, and Logistics) to assess 

how the Technology Transition Council can be better used. 

DOD provided us with written comments on a draft of this report. DOD 

partially concurred with four of the five recommendations and concurred 

with one recommendation. The reason DOD only partially concurred with 

four of the recommendations is because it does not believe the Quick 

Reaction Fund fits the definition of a transition program. However, we 

continue to believe it is important for DOD to institute better management 

controls and have better visibility of the Quick Reaction Fund as it 

Agency Comments 

and Our Evaluation 
Page 25 GAO-05-480 Defense Technology 

increases its investment in this program over the next several years. DOD 

comments appear in appendix I. 

DOD partially concurred with our recommendation that the programs 

develop data and measures that can be used to support assessments of the 

performance of the three transition programs as well as broader 

assessments of return on investment that would track the long term 

impact of the programs. DOD agreed that performance measures for the 

DACP and TTI programs could be improved but does not believe that 

measuring the impact of the Quick Reaction Fund is necessary because it 

does not technically fit the definition of a transition program. We 

disagree. DOD should track the progress of its various programs to 

determine if the programs are worthwhile and should be continued, if the 

program should receive additional funding, or if changes should be made 

in the selection or implementation process that could result in better 

outcomes. Further, failure to track even the most basic information, such 

as the number of projects completed, could result in a lack of ability to 

manage the program properly and poor stewardship of taxpayer money. 

DOD partially concurred with our recommendation that the three 

programs develop formal feedback mechanisms consisting of interim and 

after action reporting, as well as project reviews if major deviations occur 

in a project. DOD agrees that the TTI and DACP can be improved and has 

recently taken steps in this regard. However, DOD believes that due to the 

limited scope and duration of Quick Reaction Fund projects, formal 

feedback mechanisms may not be necessary for this program. We believe 

that regular feedback on the progress of each program is important to help 

program managers mitigate risk. As stated in the report, the Quick 

Reaction Fund program manager was unaware that one project ran out of 

funding prior to field testing the technology. Had the program manager 

been aware of the problem, money that had not yet been allocated could 

have been used to finish the project. In addition, based upon our 

discussions with the current program manager, DOD is planning to require 

monthly status reports for funded projects. 

DOD partially concurred with our recommendation that the programs 

implement, as appropriate: (1) formal agreements to solidify up-front 

technology development agreements related to cost, schedule, and 

performance parameters that must be met at key points in time and (2) 

confirmation of technology readiness levels as part of the proposal 

acceptance process. DOD indicated that it recently implemented 

Technology Transition Agreements for the TTI, and the DACP program 

also uses formal agreements. However, DOD does not believe formal 
Page 26 GAO-05-480 Defense Technology 

agreements are necessary for the Quick Reaction Fund because it is not 

intended to be a transition program. Also, it does not believe TRLs should 

be a factor in the proposal acceptance process. As stated in the report, we 

agree that formal agreements may not be appropriate for Quick Reaction 

Fund projects. However, TRLs should be considered during the selection 

process. Since the goal of this particular program is to prototype a new 

technology in 12 months or less, it is important that DOD has some 

assurance that the technology is ready to be field tested. As discussed in 

the report, the Quick Reaction Fund had to cancel one project, after $1.5 

million had already been spent, because it had only achieved a TRL 3. Had 

the selecting official known the TRLs of each proposed project during the 

selection phase, he may have decided to fund another, more mature 

project instead. 

DOD also partially concurred with our recommendation that the programs 

identify and implement mechanisms to ensure that transition program 

managers, developers, and acquirers better communicate and collectively 

identify and resolve problems that could hinder technology transition. 

DOD established a Transition Overarching Integrated Product Team 

earlier this year to provide the necessary oversight structure to address 

this issue, but does not believe this recommendation applies to the Quick 

Reaction Fund program. We believe that if DOD receives monthly status 

reports on the Quick Reaction Fund, as planned by the program manager, 

it should be in a good position to identify and resolve problems that could 

hinder the testing of new technology prototypes. 

DOD concurred with our recommendation that the Under Secretary of 

Defense (Acquisition, Technology and Logistics) assess how the 

Technology Transition Council can be better used as DOD considers 

solutions to broader technology transition problems. Although DOD did 

not indicate how it plans to do this, the Deputy Under Secretary of 

Defense (Advanced Systems and Concepts) has a goal that the Council not 

limit itself to just the TTI program, but look at broader technology 

transition issues across DOD. 

We are sending copies of this report to the Secretary of Defense, the 

Director of the Office of Management and Budget, and interested 

congressional committees. We will also make copies available at no charge 

on the GAO Web site at http://www.gao.gov. 
Page 27 GAO-05-480 Defense Technology 

If you or your staff have any questions concerning this report, please 

contact me at (937) 258-7915. Key contributors to this report were Cristina 

Chaplain, Cheryl Andrew, Art Cobb, Gary Middleton, and Sean D. Merrill. 

Michael J. Sullivan 

Director 

Acquisition and Sourcing Management 
Page 28 GAO-05-480 Defense Technology 

Appendix I: Comments from the Department 

of Defense 
Page 29 GAO-05-480 Defense Technology 
Page 30 GAO-05-480 Defense Technology 
Page 31 GAO-05-480 Defense Technology (120408) 
The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER