United States General Accounting Office

GAO Report to the Chairman, Subcommittee

on Military Research and Development,

Committee on National Security, House

of Representatives

October 1998 DEFENSE

ACQUISITION

Advanced Concept

Technology

Demonstration

Program Can Be

Improved

GAO/NSIAD-99-4

GAO United States

General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-278429 

October 15, 1998

The Honorable Curt Weldon

Chairman, Subcommittee on Military

 Research and Development

Committee on National Security

House of Representatives

Dear Mr. Chairman:

The Advanced Concept Technology Demonstration (ACTD) program was

initiated by the Department of Defense (DOD) to determine the extent to

which a given mature technology will improve military capabilities before

entering the normal acquisition process and, by using mature technology,

reduce the length of time to develop and acquire weapon systems.

According to the ACTD program documentation, the ACTD program was

established in 1994 in response to recommendations of the 1986 Packard

Commission1

 and a 1991 Defense Science Board study.2 The Packard

Commission concluded that major improvements could be made in

defense acquisition by emulating the practices of successful commercial

companies. Accordingly, the Commission recommended, among other

things, building and testing prototypes to assess military utility and

provide a basis for realistic cost estimates before committing to

acquisition. The Defense Science Board’s study discussed the need for

early dialogue between the potential system’s user and producer for a

proper analysis of cost, risk, and operational capability.

ACTDs have been the subject of congressional interest since the program’s

inception. Congressional committees have expressed concerns about the

validity of several technologies selected for the ACTD program and the

number of assets procured for the projects. At your request, we assessed

the current ACTD program. Specifically, we determined whether (1) the

selection process includes criteria that are adequate to ensure that only

mature technologies are selected for ACTD prototypes and (2) guidance on

transitioning to the normal acquisition process ensures that a prototype

appropriately completes product and concept development and testing

before entering production. Finally, we assessed DOD’s current practice of

1

The Packard Commission was created by former President Reagan in 1985 to review the defense

acquisition system to determine how military acquisitions could be made quicker and at lower costs.

2

Report of the Defense Science Board, 1991 Summer Study on Weapon Development and Production

Technology, November 1991.

Page 1 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

procuring more ACTD prototypes than needed to assess the military utility

of a mature technology.

Background Through fiscal year 1998, about $172 million has been allocated to the ACTD

program and 48 projects have been approved. DOD’s budget request for

fiscal year 1999 for the ACTD program is $116.4 million. An additional 10 to

15 projects are expected to be funded in fiscal year 1999.

Under the current ACTD program, DOD builds prototypes to assess the

military utility of mature technologies, which are used to reduce or avoid

the time and effort usually devoted to technology development.

Demonstrations that assess a prototype’s military utility are structured to

be completed within 2 to 4 years and require the participation of field

users (war fighters).

ACTD projects are not acquisition programs. The ACTD program seeks to

provide the war fighter with the opportunity to assess a prototype’s

capability in realistic operational scenarios. From this demonstration, the

war fighter can refine operational requirements, develop an initial concept

of operation, and make a determination of the military utility of the

technology before DOD decides whether the technology should enter into

the normal acquisition process.

Not all projects will be selected for transition into the normal acquisition

process. The user can conclude that the technology (1) does not have

sufficient military utility and that acquisition is not warranted or (2) has

sufficient utility but that additional procurement is not necessary. Of the

11 ACTD projects completed as of August 1998, 2 were found to have

insufficient utility to proceed further, 8 were found to have military utility

but no further procurement was found to be needed at the time,3

 and 1

was found to have utility and has transitioned to the normal acquisition

process.

ACTD funding is to be used to procure enough prototypes to conduct the

basic demonstration of military utility. At the conclusion of the basic

demonstration, ACTD projects are expected to provide a residual

operational capability for the war fighter. Under the current practice, ACTD

funding is also to be available to support continued use of ACTD prototypes

that have military utility for a 2-year, post-demonstration period. The

3

Three of the projects in this category are primarily software, for which production would not be

appropriate.

Page 2 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

2 years of funding is to support continued use by an operational unit and

provide the time needed to separately budget for the acquisition of

additional systems. Further, if the ACTD prototypes—such as missiles—will

be consumed during the basic demonstration, additional prototypes are to

be procured.

As stated in the ACTD guidance, a key to successfully exploiting the results

of the demonstration is to enter the appropriate phase of acquisition

without loss of momentum. ACTDs are intended to shorten the acquisition

cycle by reducing or eliminating technology development and maturation

activities during the normal acquisition process. Further, DOD can

concentrate more on technology integration and demonstration activities.

Time and effort usually devoted to technology development can be

significantly reduced or avoided and the subsequent acquisition process

reduced accordingly, if the project is deemed to have sufficient military

utility.

ACTD candidates are nominated from a variety of sources within the

defense community, including the Commanders in Chief, the Joint Chiefs

of Staff, the Office of the Secretary of Defense agencies, the services, and

the research and development laboratories. The candidates are then

reviewed and assessed by staff from the Office of the Deputy Under

Secretary of Defense (Advanced Technology). After this initial screening,

the remaining candidates are further assessed by a panel of technology

experts. The best candidates are then submitted to the Joint Requirements

Oversight Council, which assesses their priority. The final determination

of the candidates to be funded is made within the Office of the Deputy

Under Secretary of Defense (Advanced Technology), with final approval

by the Under Secretary of Defense (Acquisition and Technology).

Results in Brief Through the determination of military value of mature technologies and

their use in the acquisition process, ACTDs have the potential to reduce the

time to develop and acquire weapon systems. However, several aspects of

the ACTD program can be improved.

DOD’s process for selecting ACTD candidates does not include adequate

criteria for assessing the maturity of the proposed technology and has

resulted in the approval of ACTD projects that included immature

technology. DOD has improved its guidance on the maturity of the

technologies to be used in ACTD projects but the revised guidance

describes several types of exceptions under which immature technologies

Page 3 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

may be used. Where DOD approves immature technologies as ACTD program

candidates and time is spent conducting developmental activities, the goal

of reduced acquisition cycle time will not be realized.

Further, guidance on entering technologies into the normal acquisition

process is not sufficient to ensure that a prototype completes product and

concept development and testing before entering production. According to

the ACTD guidance, if the prototype is found to have military utility, ACTD

can directly enter production. The guidance does not mention the

circumstances when transition to development may be appropriate or the

kinds of developmental activities that may be appropriate. While

commercial items that do not require any further development could

proceed directly to production, many ACTDs may still need to enter the

engineering and manufacturing development phase to proceed with

product and concept development and testing before production begins.

Through the ACTD early user demonstration, DOD is expected to obtain

more detailed knowledge about its technologies before entering into the

acquisition process. However, in the one case in which an ACTD has

proceeded into production, DOD made that decision before completing

product and concept development and testing, thereby accepting

programmatic risks that could offset the schedule and other benefits

gained through the ACTD process.

DOD’s current practice of procuring prototypes beyond those needed for

the basic ACTD demonstration and before completing product and concept

development and testing is unnecessarily risky. This practice risks wasting

resources on the procurement of items that may not work as expected or

may not have sufficient military utility—as would be determined in the

basic demonstration—and risks a premature and excessive commitment to

production.

Inadequate Guidance

for Assessing Maturity

of Technology

By limiting consideration to prototypes that feature mature technology,

the ACTD program avoids the time and risks associated with technology

development, concentrating instead on technology integration and

demonstration activities. The information gained through the

demonstration of the mature technology could provide a good jump start

to the normal acquisition process, if the demonstration shows that the

technology has sufficient military value. Time and effort usually devoted to

technology development could be reduced or avoided and the acquisition

process shortened accordingly.

Page 4 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

Program officials stated that they have a mechanism in place to ensure

that only those projects using mature technology are allowed to become

ACTDs. These officials explained that an ACTD candidate’s technology is

assessed by high-ranking representatives from the services and the DOD

science and technology community before candidates are selected.

Program personnel stated that determining technology maturity is

important before a candidate is selected because ACTD program funding is

not intended to be used for technology development. According to

program guidance, the ACTD funding is to be used for (1) costs incurred

when existing technology programs are reoriented to support ACTD,

(2) costs to procure additional assets for the basic ACTD demonstration,

and (3) costs for technical support for 2 years of field operations following

the basic ACTD demonstration. We were told that no ACTD money was to be

used for technology development activities.

However, the project selection process does not ensure that only mature

technologies enter the ACTD program. We found examples where immature

technologies were selected and technology development was taking place

after the approval and start of the ACTD program. The current operations

manager of the Combat Identification project, which began in fiscal year

1996, told us that one of his major concerns has been that some of the ACTD

funding was being used for technology development, and not exclusively

used for designing and implementing the assessment. However, during the

ACTD project, technical or laboratory testing was still necessary to evaluate

the acceptability of many of the 12 technologies included in the initial

project. Eventually, 6 of the 12 technologies had to be terminated.

According to the demonstration manager, 2 of the 6 technologies were

terminated because they were immature. According to the manager, that is

one of the reasons the project is currently behind schedule.

Another example of the inclusion of immature technology occurred in the

Outrider Unmanned Aerial Vehicle project. According to the management

plan for the project, one of the individual technologies to be incorporated

into the vehicle was a heavy fuel engine. According to a program official, it

was later deemed that this individual technology was too immature and an

alternate technology had to be used. However, trying to use this immature

technology has already caused schedule slippage and cost overruns in the

ACTD project.

Page 5 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

Insufficient Guidance

for Transitioning

ACTDs to the Normal

Acquisition Process

To complete the basic demonstration within the prescribed 2 to 4 year

period, ACTDs typically use early prototypes. If the demonstrated

technology is deemed to have sufficient military utility, many ACTD projects

will still need to enter the normal acquisition process to complete product

and concept development and testing to determine, for example, whether

the system is producible and can meet the user’s suitability needs.4

 These

attributes of a system go beyond the ACTD’s demonstration of military

utility to address whether the item can meet the full military requirement.

Commercial items that do not require any further development could

proceed directly to production. However, other non-software related ACTDs

should enter the engineering and manufacturing development phase to

proceed with product and concept development and testing.

According to ACTD guidance, if further significant development is needed, a

system might enter the development portion of the engineering and

manufacturing development phase. However, the guidance states that, if

the capability is adequate, the ACTD can directly enter production. The

guidance does not specifically define what is considered an “adequate

capability” to allow an ACTD system to enter low-rate production.

In 1994, we reported on numerous instances of weapon systems that began

production prematurely and later experienced significant operational

effectiveness or suitability problems.5

 In our best practices report, we

reported that typically DOD programs allowed much more technology

development to continue into the product development phase than is the

case in commercial practices.6

 Turbulence in program outcomes—in the

form of production problems and associated cost and schedule

increases—was the predictable consequence of DOD’s actions. In contrast,

commercial firms gained more knowledge about a product’s technology,

performance, and producibility much earlier in the product development

process. Commercial firms consider not having this type of knowledge

early in the acquisition process an unacceptable risk. In responding to that

report, the Secretary of Defense stated that DOD is vigorously pursuing the

adoption of such business practices. Specifically, he stated that DOD has

taken steps to separate technology development from product

development through the use of ACTDs. The ACTD guidance and DOD’s

current practice do not appear to reflect this emphasis.

4

“Suitability” involves factors such as maintainability, reliability, safety, and supportability.

5

Weapons Acquisition: Low-Rate Initial Production Used to Buy Weapon Systems Prematurely

(GAO/NSIAD-95-18, Nov. 21, 1994).

6

Best Practices: Successful Application to Weapon Acquisitions Requires Changes in DOD’s

Environment (GAO/NSIAD-98-56, Feb. 24, 1998).

Page 6 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

In the case of the Predator ACTD, the one ACTD that has proceeded into

production, DOD decided to enter the technology into production before

proceeding with product and concept development and testing, thereby

accepting programmatic risks that could offset the schedule and other

benefits gained through the ACTD process. In the early operational

assessment of the Predator’s ACTD demonstration, the Director,

Operational Test and Evaluation, did not make a determination of the

system’s potential operational effectiveness or suitability. However, the

system was found to be deficient in several areas, including mission

reliability, documentation, and pilot training. The assessment also noted

that the ACTD demonstration was not designed to evaluate several other

areas such as system survivability, supportability, target location accuracy,

training, and staffing requirements.

The basic ACTD demonstration may have clarified the Predator’s military

utility but it did not demonstrate its system requirements or its suitability.

Thus, instead of using the knowledge acquired during the demonstration

to complete the Predator’s development through the product and concept

development and testing stages of acquisition, DOD allowed it to directly

enter production.

Procuring ACTD

Prototypes Beyond

Those Needed for

Basic Demonstration

Is Unnecessarily

Risky

DOD’s practice is to procure sufficient ACTD prototypes to provide a 2-year

residual capability. When it determines that the original prototypes will be

consumed during the basic demonstration, additional prototypes are

procured for potential use after the basic ACTD demonstration. However,

these additional assets—like the basic demonstration prototypes—have

not been independently tested to determine their effectiveness and

suitability. Procuring additional ACTD prototypes before product and

concept development and testing is completed risks wasting resources on

the procurement of items that may not work as expected or may not have

sufficient military utility. Representatives from the service test agencies

did not support this practice and agreed that it had the potential for

problems. Without a meaningful independent assessment of a product’s

suitability, effectiveness, and survivability, users cannot be assured that it

will operate as intended and is supportable.

Congress has expressed concern about the amount of equipment being

procured beyond what is needed to conduct the basic ACTD demonstration.

Its concern is that DOD is making an excessive commitment to production

before military utility is demonstrated and before appropriate concepts of

operation are developed. For example, DOD plans to procure 192 Enhanced

Page 7 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

Fiber Optic Guided missiles at an estimated cost of $27 million and 144

Line-of-Sight Anti-Tank missiles at an estimated cost of $28 million beyond

the quantities of missiles required for the ACTD demonstrations—64 and 30

missiles, respectively. The production of these additional missiles will

follow the production of the missiles needed for the basic demonstration

and will continue on a regular basis throughout the 2-year,

post-demonstration period. If the prototypes are deemed to have sufficient

military utility, the service involved will be expected to fund the

production of additional missiles beyond these quantities. By establishing

a regular pattern of procurement in this way, DOD risks committing to a

continuing production program before a determination is made about the

technology’s military utility and before there is assurance that the system

will meet validated requirements and be supportable.

Conclusions The strength of the ACTD program is in conducting basic demonstrations of

mature technology in military applications before entering the normal

acquisition process. This practice could significantly reduce or eliminate

the time and effort needed for technology development from the

acquisition process. For this to occur, it is essential that DOD use only

mature technology in its ACTDs. DOD’s criteria for selecting technologies for

ACTD candidates should be clarified to ensure the selection of mature

technology with few, if any, exceptions.

Further, ACTDs may not, by themselves, result in an effective and safe

deployment of military capability. It is important that product and concept

development as well as test and evaluation processes be allowed to

proceed before the service commits to the production of the demonstrated

technology. If an ACTD project is shown to have military value, the normal

acquisition processes can and should be tailored—but not

bypassed—before DOD begins production. Lastly, emphasizing the need to

complete concept and product development and testing before procuring

more items than needed for the basic demonstration would reduce the risk

of prematurely starting production.

Recommendations We recommend that the Secretary of Defense clarify the ACTD program

guidance to (1) ensure the use of mature technology with few, if any,

exceptions and (2) describe when transition to the development phase of

the acquisition cycle is necessary and the types of development activity

that may be appropriate. Further, we recommend that the Secretary of

Defense limit the number of prototypes to be procured to the quantities

Page 8 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

needed for early user demonstrations of mature technology until the item’s

product and concept development and testing have been completed.

Agency Comments

and Our Evaluation

In commenting on a draft of this report, DOD partially concurred with each

of our recommendations. DOD said that it continues to refine the ACTD

selection and implementation process. DOD agreed that the ACTD program

should focus on mature technologies and stated that it had improved its

definitions of mature technology. DOD’s new guidance on the maturity of

technology to be used in ACTDs states

“. . . new technologies proposed for incorporation into an ACTD should not be in the 6.1

(basic research) or 6.2 (applied research) budget categories. Furthermore, the technologies

must have been successfully demonstrated at the subsystem or component level and at the

required performance level prior to the start of the ACTD.”

While this guidance is improved over previous versions, the new guidance

permits the selection of immature technology—even as the primary or

core technology—provided that it is demonstrated prior to the ACTD

demonstration. Also, some recent ACTD projects have been approved

without the technologies having been identified. Moreover, the new

guidance goes on to describe several types of exceptions under which

immature technologies may be permitted to be used in an ACTD. As our

report states, the use of immature technologies has delayed programs and

we continue to believe DOD needs to focus the ACTD program on the use of

mature technology with few, if any exceptions.

DOD also agreed that some but not all ACTDs may require additional product

and concept development before proceeding into production. DOD states

that a mandatory engineering and manufacturing development phase

would not be appropriate for all ACTD projects. We agree, however, the

existing ACTD guidance focuses on the transition directly to production and

provides too little guidance concerning a possible transition to

development. As stated in our recommendation, the guidance should

specify when a transition to development may be appropriate and the

kinds of developmental activities that may be appropriate.

Finally, DOD agreed that the number of ACTD prototypes to be procured

should be limited until the Under Secretary can confirm that sufficient

testing has been satisfactorily completed to support any additional

procurement. We agree with DOD that test results should form the basis for

starting limited procurement. However, DOD’s equating a determination of

Page 9 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

military utility (based on an ACTD demonstration) with a determination of a

system’s readiness to begin production is inappropriate because

production decisions require more testing data. We have long held the

view and have consistently recommended that DOD use extreme caution to

avoid premature commitments to production.

Scope and

Methodology

To determine the adequacy of the ACTD program’s selection criteria in

assessing technology maturity and guidance for transitioning to the normal

acquisition process, we reviewed existing program guidance, published

reports, the Office of the Inspector General’s April 1997 ACTD report, and

the recommendations of the 1986 Packard commission and the 1996

Defense Science Board. We discussed selection criteria, transitioning to

the acquisition process, and all 34 of the individual ACTD programs

approved through fiscal year 1997 with representatives from the Office of

the Deputy Under Secretary of Defense (Advanced Technology),

Washington, D.C.; the Army’s Deputy Chief of Staff for Operations and

Plans, Office of Science and Technology Programs, Washington, D.C.; the

Air Force’s Director for Operational Requirements, Rosslyn, Virginia; the

Navy’s Requirements and Acquisition Support Branch, Washington, D.C.;

the Marine Corps’ Combat Development Command Office of Science and

Innovation, Quantico, Virginia; the Joint Staff’s Acquisition and

Technology Division and Requirements Assessment Integration Division,

Washington, D.C.; and the Office of the Commander in Chief, U.S. Atlantic

Command, Norfolk, Virginia.

We discussed the issue of procuring additional residual assets for early

deployment with representatives from DOD’s Office of the Director,

Operational Test and Evaluation, Washington, D.C.; the Army’s Test and

Evaluation Management Agency, Washington, D.C.; the Army’s Operational

Test and Evaluation Command, Alexandria, Virginia; the Marine Corps’

Operational Test and Evaluation Activity, Quantico, Virginia; the Air

Force’s Test and Evaluation Directorate, Washington, D.C.; and the Navy’s

Commander, Operational Test and Evaluation Force, Norfolk, Virginia.

We conducted our review from September 1997 to July 1998 in accordance

with generally accepted government auditing standards.

As agreed with your office, unless you publicly announce the contents of

this report earlier, we plan no further distribution of this report until

30 days from its issue date. At that time, we will send copies to other

Page 10 GAO/NSIAD-99-4 Defense Acquisition
B-278429 

interested congressional committees; the Secretaries of Defense, the

Army, the Air Force, and the Navy; the Commandant of the Marine Corps;

the Director, Office of Management and Budget; and other interested

parties. We will also make copies available to others upon request.

Please contact me at (202) 512-4841, if you or your staff have any questions

concerning this report. The major contributors to this report were 

Bill Graveline, Laura Durland, and John Randall.

Sincerely yours,

Louis J. Rodrigues

Director, Defense Acquisitions Issues

Page 11 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

Page 12 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 1.

Page 13 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 2.

See comment 2.

See comment 3.

Page 14 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 4.

Page 15 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 5.

See comment 6.

See comment 7.

See comment 8.

See comment 9.

See comment 10.

Page 16 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 11.

See comment 12.

Page 17 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 1.

See comment 8.

Page 18 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

See comment 12.

Page 19 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

The following are GAO’s comments on the Department of Defense’s (DOD)

letter, dated August 31, 1998.

GAO Comments 1. DOD’s guidance is improved over previous versions. DOD’s revised

guidance on the maturity of technology to be used in the Advanced

Concept Technology Demonstration (ACTD) states that

“. . . new technologies proposed for incorporation into an ACTD should not be in the 6.1

(basic research) or 6.2 (applied research) budget categories. Furthermore, the technologies

must have been successfully demonstrated at the subsystem or component level and at the

required performance level prior to the start of the ACTD.”

The new guidance, however, permits the selection of immature

technology—even as the primary or core technology—provided that it is

demonstrated prior to the ACTD demonstration. The guidance goes on to

describe several types of exceptions under which immature technologies

may be permitted to be used in an ACTD. Moreover, the revised guidance

would permit—as has occurred in some cases—the approval of ACTD

projects without the technologies having been identified. Our point is that

DOD should continue to focus the ACTD program on the use of mature

technology with few, if any, exceptions.

2. For the Combat Identification ACTD, the management plan for the project

shows that the technologies in question were integral parts of the

capabilities being evaluated. The demonstration manager told us that the

involvement of immature technologies was one of the reasons that the

project was behind schedule.

3. According to the Outrider project’s management plan, the heavy fuel

engine was the primary technology to be used, not the alternative. Also,

the involvement of this immature technology was one of the causes of

schedule slippage and cost overruns on the project.

4. DOD’s most current ACTD guidance states

“. . .Strategies and approaches are described to facilitate transitioning from an ACTD to the

acquisition process as defined in DOD 5000.2R. The suggested approaches are based on

lessons learned. The focus of the suggestions are ACTDs that are planned—if successful—to

enter the acquisition process at the start of LRIP.”

Page 20 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

Although there is a basic recognition that the transition to development

may be possible, the bulk of the guidance is on how and when to transition

to production. As pointed out in the report, the guidance does not describe

when a transition to development or what types of development activity

may be appropriate. In our view, the guidance needs to be more balanced

between the possibility of transition to development and the transition of

ACTD projects directly to production.

5. As discussed in the report, the independent operational testing agencies

are observers in the ACTD demonstrations and not active participants.

While the Office of the Director of Operational Test and Evaluation was an

observer during the Predator demonstration, a determination was not

made that Predator was potentially effective and suitable.

6. We agree that ACTDs address the technology’s suitability. However, the

ACTD focus on suitability is in a very general sense and extensive data is

not collected on the system’s reliability, maintainability, and other aspects

of suitability needed to support production decisions.

7. As our report states, the Predator was rushed into low-rate initial

production prematurely given the limited amount of testing conducted at

that time and the problems that were uncovered during that limited

testing.

8. DOD’s equating a determination of military utility (based on an ACTD

demonstration) with a determination of a system’s readiness to begin

production is inappropriate because production decisions require more

testing data. During our review, we noted that sufficient information was

not obtained from an ACTD demonstration to make a commitment to

limited production. Commercial practice would dictate that much more

information be obtained about a product’s effectiveness, suitability,

producibility, or supportability before such a commitment is made.1

 We

believe the ACTD guidance needs to be more balanced and should

anticipate that ACTD prototypes may need to conduct more product and

concept development and testing prior to production. We have long held

the view and have consistently recommended that DOD use extreme

caution to avoid premature commitments to production.

9. We are not suggesting that a lengthy development phase be conducted

on all ACTD products nor, as DOD appears to suggest, that an ACTD prototype

1

Best Practices: Successful Application to Weapon Acquisitions Requires Changes in DOD’s

Environment (GAO/NSIAD-98-56, Feb. 24, 1998).

Page 21 GAO/NSIAD-99-4 Defense Acquisition
Appendix I 

Comments From the Department of Defense

may be ready to start limited production immediately after its basic

demonstration. As DOD stated in its intent to establish the ACTD program,

we believe the benefit of the ACTD process is in eliminating or reducing

technology development, not in making early commitments to production

or in postponing product and concept development and testing activities

until after production starts.

10. While ACTD demonstrations are performed in operational environments,

they are not operational tests. During the course of our work, we held

several discussions with officials from the operational test community.

Those officials were in favor of the user demonstrations featured in the

ACTD program, but none considered those demonstrations as substitutes

for operational testing because of their informality, lack of structure, and

the lack of a defined requirement by which to measure performance.

11. DOD appears not to recognize the very real possibility that the ACTD

demonstration may find the technology in question to have little or no

military utility or to be unaffordable in today’s budgetary and security

environment. In fact, due to budget constraints, the Army was forced to

prioritize its procurement programs, and the planned procurement funding

for Enhanced Fiber Optic Guided missiles has been reallocated.

12. While we agree with DOD that test results should form the basis for

starting limited procurement, the testing needed goes beyond the basic

demonstration of military utility provided by the ACTD program.

(707299) Page 22 GAO/NSIAD-99-4 Defense Acquisition
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100