GAO

United States General Accounting Office

Report to the Chairman, Subcommittee

on Military Research and Development,

Committee on Armed Services, House of

Representatives

September 2000 UNMANNED AERIAL

VEHICLES

Questionable Basis for

Revisions to Shadow

200 Acquisition

Strategy

GAO/NSIAD-00-204

Page 1 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Contents

Letter 3

Appendixes Appendix I: The Army Is Incorporating Lessons Learned

From Previous Experience Into Its

Shadow 200 Strategy 14

Appendix II: Comments From the Department of Defense 16

Appendix III: GAO Contacts and Staff Acknowledgments 18

Figures Figure 1: Shadow 200 Unmanned Aerial Vehicle System 6

Figure 2: Revised Schedule for Shadow 200 Acquisition Strategy 7
Page 2 GAO/NSIAD-00-204 Unmanned Aerial Vehicle
Page 3 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

United States General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-285860 Leter

September 26, 2000

The Honorable Curt Weldon, Chairman

Subcommittee on Military Research

and Development

Committee on Armed Services

House of Representatives

Dear Mr. Chairman:

The Army plans to buy 44 Shadow 200 tactical unmanned aerial vehicle

systems. Each system includes three unmanned aircraft (equipped with an

imagery sensor1

); a vehicle to carry the aircraft; two ground control

stations mounted on vehicles; and launch, recovery, and support equipment

pulled on trailers behind the vehicles. From inside the ground control

station, soldiers operating the unmanned aircraft will fly them over hostile

or contested territory, collecting imagery of areas of interest for Army

commanders so they can detect, identify, and locate enemy forces. The

acquisition cost, including research and development costs, for the

44 systems is an estimated $430 million through 2004.

Because the Army does not currently have enough unmanned aerial vehicle

systems to meet its requirements, it devised an acquisition strategy in 1999

focused on procuring and fielding the Shadow 200 system as quickly as

possible. The Army’s strategy called for acquiring a system that

incorporated mature technologies,2 which will be integrated and

1 The sensor is the payload carried by the aircraft to accomplish its mission. Imagery sensors

are generally electro-optical (for collecting television-type images) or infrared

(for collecting images based on detected heat radiating from objects). Infrared capability is

especially useful at night and in other low-light conditions. Combined

electro-optical/infrared sensors are available.

2 Mature technologies are those that have been developed to the point where they can be

integrated into a new product and counted on to meet product performance requirements.
B-285860

Page 4 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

demonstrated before the Army commits to full-rate production.3 The

Army’s acquisition strategy also included the low-rate initial production of

four Shadow 200 systems to be used in developmental and operational

testing.4 After evaluating the developmental and operational test results,

the Army plans to decide in September 2001 whether to begin full-rate

production of the Shadow 200 system.

To field systems more quickly, the Army revised its Shadow 200 acquisition

strategy in March 2000 by planning to increase the number of low-rate

initial production systems from four to eight. The decision to produce these

four additional systems would be made in February 2001–2 months before

operational testing and 7 months ahead of the scheduled full-rate

production decision. As you requested, we have assessed whether the

Army made a sound decision in revising its acquisition strategy. You also

expressed an interest in the extent to which the Army has incorporated

lessons learned from previous experience and from Kosovo/Operation

Allied Force into its Shadow 200 program.5 This information is provided in

appendix I.

Results in Brief The Army has a questionable basis for revising its acquisition strategy to

procure four additional Shadow 200 systems in February 2001 before

operational testing is conducted. In contrast, the Army’s original strategy,

which would prove system capabilities before producing additional

systems, was sound. Among its reasons to justify the revision, the Army

contends that accelerating the program will enable it to field a much

3 In DOD’s formal acquisition process, the decision to enter full-rate production does not

occur until a system has been operationally tested in an environment that realistically

simulates the system’s expected combat environment. However, before operational testing,

low-rate initial production can take place to produce articles for testing, to prove

manufacturing and production processes, or to build-up to planned higher rates of

production.

4 Developmental testing is a technical test conducted on components, subsystems, and

system-level configurations of hardware and software to provide data on the achievability of

critical system performance parameters. Operational testing refers to testing conducted in a

realistic combat environment on production or production representative articles to

support the decision to proceed beyond low-rate initial production. The purpose of this test

is to provide a valid estimate of expected system operational effectiveness and suitability.

5 Operation Allied Force refers to a military operation conducted from March to June 1999

by the U.S. and our North Atlantic Treaty Organization (commonly referred to as NATO)

allies to bring an end to Serbian atrocities in Kosovo.
B-285860

Page 5 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

needed capability sooner. Also, the Army believes that risk associated with

additional production prior to operational testing is substantially mitigated

by significant developmental and other testing that is planned. We are

concerned that the Army cannot know whether the Shadow 200 system will

be operationally effective before operational testing takes place. Our

previous reviews of other unmanned aerial vehicle programs have shown

that buying systems before successfully completing testing has repeatedly

led to defective systems that were later terminated or required costly

redesign and retrofit to achieve satisfactory performance.

This report recommends that the Army not buy four additional systems

until after operational testing is completed. The Department of Defense

disagreed with us and stated that the risk associated with procuring these

additional systems prior to operational testing is minimal. We continue to

believe that the Army should not buy the additional systems because only

operational testing of the system in a realistic combat environment can

show whether the overall system will meet the Army’s operational needs. If

the Army does not implement our recommendation, we believe the

Congress should consider directing it to do so.

Background The Shadow 200 unmanned aerial vehicle system is expected to provide the

Army with day or night reconnaissance, surveillance, and target acquisition

capability. The Shadow 200 aircraft will allow Army commanders a view

into heavily protected battlespace that cannot be penetrated by other

intelligence assets or one that presents a high risk to manned aircraft. The

Shadow 200 unmanned aerial vehicle system, formally designated the

RQ-7A, is shown in figure 1. The aircraft weighs approximately 325 pounds,

has a wingspan of 13 feet, and measures 11 feet from nose to tail.
B-285860

Page 6 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Figure 1: Shadow 200 Unmanned Aerial Vehicle System

Source: U.S. Army

Personnel/equipment

transport and trailer

Remote video terminal

and antenna

X4

Ground control stations

Ground data terminal

Portable ground

control station and

data terminal

X1

Air vehicles with payloads

X3

Air vehicle transport

and launch

trailer
B-285860

Page 7 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

On December 27, 1999, the U.S. Army awarded a contract to AAI

Corporation for the engineering and manufacturing development of the

Shadow 200 system. During the planned 16-month engineering and

manufacturing development phase, the primary objectives are to translate

the most promising design approach into a stable, interoperable,

producible, supportable, and cost-effective design; validate the

manufacturing and production process; and demonstrate system

capabilities through testing. Because the Army believes the basic Shadow

200 system design is mature, low-rate initial production of four systems

was started at the beginning of the engineering and manufacturing

development phase. The Army expects to receive the first four systems

between December 2000 and May 2001, when the engineering and

manufacturing development phase is scheduled to be completed. These

four systems will be used for developmental and operational testing,

contractor and government performance testing, training, and equipping

the first Army unit. In March 2000, the Army revised this acquisition

strategy and now plans to exercise a second contract option in February

2001, buying four additional systems prior to operational testing at a cost of

$31.8 million (see fig. 2).

Figure 2: Revised Schedule for Shadow 200 Acquisition Strategy

Source: Data from U.S. Army.

Decision to enter development and produce

4 systems

Deliveries of first 4 systems

Decision to produce 4 additional systems

Operational testing

Analysis of operational test and report writing

Full-rate production decision

O N D J F M A M J J A S O N D J F M A M J J A S

Fiscal year

2000 2001
B-285860

Page 8 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Justification for Buying

More Low-rate Initial

Production Systems Is

Questionable

The Army has a questionable justification for revising its acquisition

strategy to procure the four additional Shadow 200 systems before

operational testing begins. Prior to this decision, the Army’s acquisition

strategy for the Shadow 200 system was sound because it minimized risk by

(1) using mature technologies, (2) completing engineering and

manufacturing development, and (3) demonstrating system capabilities

through operational testing before buying any additional systems.

Original Acquisition

Strategy for Shadow 200

Was Sound

Ensuring the maturity of components included in a system’s design is a key

to establishing a sound acquisition strategy. The Army’s program office

considers all five of the technologies critical to the Shadow 200 system’s

basic design to be mature—that is, at a level considered acceptable for

programs entering the engineering and manufacturing development phase

and ready to be integrated into a single system.6 Our past work has shown

the soundness of this approach; programs using more mature technology at

the program start are more likely to succeed in meeting their objectives. In

contrast to this lower-risk approach, the Department of Defense has often

allowed immature technologies to be incorporated into its programs,

thereby increasing technical risks.7 For example, in the Hunter unmanned

aerial vehicle program, the contractor used a motorcycle engine that was

unproven as an airplane engine. Subsequently, in the flying environment,

the engines overheated and valves seized, leading to a redesign of the

engine to eliminate the problem.8

The original Shadow 200 acquisition strategy was also sound because it

committed the Shadow 200 contractor to complete all its engineering and

manufacturing development activities by May 2001, well before the next

planned production decision scheduled for September 2001. Included in

these activities were all the critical engineering steps necessary to ensure

that the Shadow 200 design is cost-effective, stable, supportable, and

producible. The original strategy also envisioned that the manufacturing

6 The critical technologies are those incorporated into the airframe, engine, sensor payload,

datalink, and ground control station.

7 Best Practices: Better Management of Technology Development Can Improve Weapon

System Outcomes (GAO/NSIAD-99-162, July 30, 1999).

8 Unmanned Aerial Vehicles: No More Hunter Systems Should Be Bought Until Problems Are

Fixed (GAO/NSIAD-95-52, Mar. 1995).
B-285860

Page 9 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

and production processes for the system would be validated by May 2001.

In contrast, at the time the Department committed to acquiring the Hunter

unmanned aerial vehicle system in January 1993, the system design was

neither stable nor supportable. Subsequently, a number of Hunter aircraft

crashed during testing due to design flaws, costing the U.S. government

millions of dollars and the program time and credibility.

Further, the original Shadow 200 acquisition strategy was sound because it

committed to producing only four systems prior to operational testing,

which is scheduled to begin in April 2001. Operational testing is the primary

means of evaluating weapon system performance in a realistic combat

environment. Although the Shadow 200 design incorporates mature

technologies, operational testing will provide the Army with knowledge

about whether these technologies, when integrated into a single system,

will meet the Army’s needs before it commits itself to the system’s full-rate

production in September 2001. Moreover, because operational testing was

scheduled to take place in April 2001 and additional production was not

planned to be approved until September 2001, the original strategy

provided the Army with ample opportunity to fully analyze the operational

test results.

Army Has Questionable

Basis for Revising

Acquisition Strategy

In December 1999, the Army’s acquisition executive authorized the Shadow

200 program to enter the engineering and manufacturing development

phase and also authorized building four low-rate initial production systems

to be used in developmental and operational testing. At that time, the

program manager was also directed to develop a plan to accelerate the

fielding of the Shadow 200 system. Subsequently, in March 2000, the Army

revised its original strategy. The revised plan will achieve accelerated

fielding by exercising an option to buy an additional four low-rate initial

production systems in February 2001 before engineering and

manufacturing development is completed and operational testing is

conducted (see fig. 2). These four additional systems would be fielded with

operational Army units; they are not needed for testing.

The intent behind this revision is to enable the Army to field Shadow 200

systems to operational Army units earlier than called for in the original

plan. We asked Army officials, including the program manager, to explain

the basis for this decision. They told us that recent reductions in the Army’s

force structure have put a premium on surveillance and reconnaissance to

provide greater situational awareness and knowledge, and acceleration of

the program would allow early fielding of the much needed Shadow 200’s
B-285860

Page 10 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

capability. They also stated that risk is substantially mitigated by significant

developmental and other testing. Further, acceleration only places a

portion of the $31.8-million cost for the four additional systems at risk

because most of the hardware could still be used if problems are later

identified during operational testing.

Although the Army might be able to deploy the Shadow 200 systems sooner

if more systems are produced beginning in February 2001, we are

concerned that the Army cannot guarantee their operational effectiveness

at the time it plans to make that decision. Only operational testing can

ensure that the components—even if mature—will work together in a

realistic combat environment. Our previous reviews of other unmanned

aerial vehicle programs have shown that buying systems before

successfully completing operational testing has repeatedly led to defective

systems that were later terminated or required costly redesign and retrofit

to achieve satisfactory performance. For example, because predecessors to

the Pioneer unmanned aerial vehicle system had been used successfully by

Israeli forces, the Navy procured nine Pioneer systems in 1985 without

testing and deployed the system to operational forces. As we reported in

1990, numerous problems ensued that led the Navy to redesign and modify

virtually the entire system at a cost of about $50 million.9 The redesign and

modification costs about matched the Navy’s cost of $56 million to initially

procure its nine systems.

Conclusions The Army’s initial acquisition strategy for the Shadow 200 system

appropriately minimized risk by scheduling operational testing before

committing to additional production. The Army’s revised—and riskier—

plan to procure four additional systems before operational testing has a

questionable rationale. Although the Army designed the Shadow 200

system using mature technologies, it has no assurance that its components

will work well together in a realistic combat environment until operational

testing is completed. We believe that the Army should not risk procuring

systems before proving that they will meet its needs and will not require

costly and time-consuming retrofits.

9 Unmanned Aerial Vehicles: Realistic Testing Needed Before Production of Short-Range

System (GAO/NSIAD-90-234, Sept. 28, 1990).
B-285860

Page 11 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Recommendations for

Executive Action

We recommend that the Secretary of Defense direct the Army not to

exercise the option to procure four more Shadow 200 unmanned aerial

vehicle systems until operational testing has been successfully completed

and shown that the systems meet the Army’s needs.

Matters for

Congressional

Consideration

If the Army does not implement our recommendation, we believe the

Congress should consider directing the Army to do so.

Agency Comments In its written comments on a draft of this report, the Department stated

that the Army should have the option to procure four more Shadow 200

systems before successfully completing operational testing. It stated that

the risk associated with procuring these additional systems is minimal

given the mature technology used in the program and the extensive

developmental and operational tempo testing10 planned before the

February 2001 scheduled decision. The Department added that the Army

does not need to wait for the results of operational testing before

exercising an additional production option because the Army will have

tested all of the system’s critical technical performance parameters during

developmental and other system testing. The Department also stated that

this minimal risk is outweighed by the benefits associated with accelerating

delivery of the Shadow 200 system.

We continue to believe that the Army should wait until after completion of

planned operational testing of the Shadow 200 system in May 2001. The

ongoing developmental and operational tempo testing will provide the

Army with valuable information on critical technical performance

parameters such as range and endurance, but it will not provide the data

about the overall performance of the system in a realistic combat

environment that will be obtained during operational testing. According to

10 Operational tempo testing differs from operational testing in that it attempts to increase

the pace of system operations to increase stress on the system, but its main purpose is to

identify problem areas in design that need correction rather than to determine if a system

will be effective and suitable in a realistic combat environment when operated by Army

units.
B-285860

Page 12 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

the Department of Defense guidance for its major system acquisitions,11

attainment of individual critical technical performance parameters does

not guarantee that overall system performance will meet operational

needs. Although the Shadow 200 system has not been designated a major

system, we believe this concept is equally applicable to the performance of

the Shadow 200 system.

The Department of Defense’s written comments are reprinted in

appendix II. The Army also provided technical comments, which we

incorporated as appropriate.

Scope and

Methodology

To assess the Shadow 200 program’s acquisition strategy and plans, we met

with Department of Defense, Army, and contractor officials, and analyzed

Shadow 200 system cost data, assessment reports, requirements

documents, and program plans. To assess Shadow 200 system maturity, we

asked program officials to identify the system’s critical technologies and

apply an analytical tool developed by the National Aeronautics and Space

Administration to assess the technical maturity of each critical technology

at the time the program entered the engineering and manufacturing design

phase.

To assess the extent to which lessons learned in Operation Allied Force

were used to improve the performance of the tactical unmanned aerial

vehicles (see app. I), we reviewed the Department of Defense’s lessons

learned report,12 reviewed program documentation, and held discussions

with program officials. We conducted our work at the Directorate for

Intelligence, Surveillance, and Reconnaissance Systems, Office of the

Secretary of Defense, Washington, D.C.; Tactical Unmanned Aerial Vehicle

Project Office, Huntsville, Alabama; Training and Doctrine Command

System Manager for Unmanned Aerial Vehicles, Fort Huachuca, Arizona;

and AAI Corporation, Hunt Valley, Maryland.

11 Department of Defense Regulation 5000.2-R, Mandatory Procedures for Major Defense

Acquisition Programs and Major Automated Information System Acquisition Programs

(Mar. 15, 1996), appendix III, page III-7.

12 Report to the Congress: Kosovo/Operation Allied Force After-Action Report, Department

of Defense (Jan. 31, 2000).
B-285860

Page 13 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

We conducted our work from August 1999 through September 2000 in

accordance with generally accepted government auditing standards.

We will send copies of this report to interested congressional committees;

the Honorable William Cohen, Secretary of Defense; the Honorable

Louis Caldera, Secretary of the Army; and the Honorable Jacob Lew,

Director, Office of Management and Budget. Copies will also be made

available to others upon request.

If you have any questions regarding this report, please contact me at

(202) 512-4841. Other contacts and key contributors to this report are listed

in appendix III.

Sincerely yours,

Louis J. Rodrigues

Director
Page 14 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Appendix I

The Army Is Incorporating Lessons Learned Apendxies

From Previous Experience Into Its Shadow

200 Strategy ApendixI

In 1999, the Army, the Navy, and the Air Force operated RQ-5A Hunter,

RQ-2A Pioneer, and RQ-1A Predator unmanned aerial vehicle systems,

respectively, in support of Operation Allied Force in Kosovo.1 According to

the Department of Defense, these unmanned aerial vehicles were used to

an unprecedented degree and played an unprecedented role in Operation

Allied Force. They were used extensively for surveillance and

reconnaissance missions, and their ability to loiter over hostile territory

enabled them to provide information that was otherwise unavailable while

avoiding the risk of losing aircrews. Nevertheless, in its January 2000

Kosovo/Operation Allied Force report, the Department of Defense

identified a number of technical improvements it believes unmanned aerial

vehicles need to attain their full promise on the battlefield. These include

the need for (1) improved tactics, techniques, and procedures; (2) more

capable sensor payloads (e.g., advanced imagery, radar, and laser); and

(3) air vehicles that can operate in all types of weather conditions. Based

on its previous experience operating Hunter unmanned aerial vehicles in

peacetime, however, the Army had already learned these lessons and has

been incorporating them into its acquisition strategy for its new tactical

unmanned aerial vehicle system, the Shadow 200.

In January 1999, to improve the Army’s understanding of tactics and of

planning considerations for using unmanned aerial vehicles in future

operations, the Army removed a Hunter system from storage and approved

its use for joint readiness training at Fort Polk, Louisiana. Several Army

units have rotated through the training center, and Army operational

concepts are being validated and new lessons learned. One critical lesson

learned is that the Army will be sending its unmanned aerial vehicles into

high-risk areas to conduct surveillance where they are likely to be lost in

battle. Thus, Army unmanned aerial vehicles must be inexpensive enough

to be considered expendable. Hence, the Army’s objective cost for a

Shadow 200 air vehicle is about $452,000, which is a fraction of the cost of a

manned observation helicopter.

To address the need for more capable sensor payloads in its unmanned

aerial vehicles, the Army began a yearlong payload study in March 2000.

During the study period, the Army is exploring alternatives and benefits

that synthetic aperture radar, chemical weapons detection, laser range

finders, signals intelligence, and other payloads could provide

1 See Report to the Congress: Kosovo/Operation Allied Force After-Action Report,

Department of Defense (Jan. 31, 2000).
Appendix I

The Army Is Incorporating Lessons Learned

From Previous Experience Into Its Shadow

200 Strategy

Page 15 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

commanders.2 Until the study is complete, the only payload that will be

integrated with the Shadow 200 is an imagery sensor to provide television

and infrared pictures. The study will take into account that the Shadow 200

must remain inexpensive and expendable. Therefore, the more complex

and expensive a particular payload is, the more benefits it must provide to

the commander if it is to be incorporated into the Shadow air vehicle.

The Army also recognizes the value of unmanned aerial vehicles with

all-weather capability. Before Operation Allied Force, the Army understood

that unmanned aerial vehicle flight and sensor performance are adversely

affected by the elements—especially icing. However, the Army’s belief that

tactical unmanned aerial vehicles must be inexpensive enough to lose and

the Shadow 200’s small size make it impractical to give the Shadow 200

aircraft all-weather capability. The Army plans to integrate a sensor on the

Shadow 200 aircraft to detect icing; once icing is detected, the aircraft

operator can fly the aircraft out of the icing environment. Other than this

sensor, the Shadow 200 aircraft will not have all-weather capability

because adding additional weight in the form of deicing equipment to a

vehicle of this size would substantially limit its range and sensor payload

capacity.

2 A synthetic aperture radar uses radar signals and the complex information processing

capability of modern digital electronics to provide high resolution imagery that is used by

the aircraft operators on the ground for reconnaissance and targeting.
Page 16 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Appendix II

Comments From the Department of Defense ApendixI
Appendix II

Comments From the Department of Defense

Page 17 GAO/NSIAD-00-204 Unmanned Aerial Vehicle
Page 18 GAO/NSIAD-00-204 Unmanned Aerial Vehicle

Appendix III

GAO Contacts and Staff Acknowledgments ApendxiI

GAO Contacts Robert E. Levin (202) 512-4841

Charles A. Ward (202) 512-4343

Acknowledgments In addition to those named above, Thomas L. Gordon, Danny G. Owens,

Jose A. Ramos, Jr., and Susan Woodward made key contributions to this

report.

(707439) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00