United States General Accounting Office

GAO Report to the Honorable

William V. Roth, Jr., U.S. Senate

June 1996 ELECTRONIC

WARFARE

Navy’s New Radar

Warning Receiver

Needs More Testing

G O A

years

1921 - 1996

GAO/NSIAD-96-68

GAO United States

General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-260441 

June 20, 1996

The Honorable William V. Roth, Jr.

United States Senate

Dear Senator Roth:

At your request, we examined the ALR-67(V)3 radar warning receiver

(RWR) program. The ALR-67(V)3 is intended to help protect Navy aircraft

and the program is expected to cost about $1 billion. We are issuing this

report to bring to your attention certain aspects of the Navy’s acquisition

plans that we believe will unnecessarily increase the Department of

Defense’s (DOD) risk on the program.

Background RWRs are electronic warfare devices that help protect aircraft against

radar-controlled weapons. RWRs sense the signals from hostile radars,

provide an audio warning to the pilot, and display the warning information

on a video screen in the cockpit. The display identifies the threats,

provides their location or relative bearings, and ranks the threats in order

of danger to the aircraft. Based on the warning, the pilot chooses from

various options for defeating the threat, such as employing electronic

jamming to interfere with the radar.

The ALR-67(V)3 is intended to be the Navy’s next generation RWR for use

on its future F/A-18 E/F aircraft. The ALR-67(V)3 is also planned for use on

some current F/A-18 C/D aircraft, and with minor antenna modifications,

can also be retrofitted as the ALR-67(V)4 into the F-14 and AV-8B aircraft.

The Navy expects the ALR-67(V)3 to significantly outperform the current

RWR, the ALR-67(V)2. The ALR-67(V)3 is expected to be capable of

detecting threats at a greater range and identifying more types of threat

radars than its predecessor.

Results in Brief Despite numerous performance problems that surfaced in developmental

testing of the ALR-67(V)3, the adverse consequences from the premature

procurement of the original ALR-67 and the ALR-67(V)2, and having

already produced sufficient test articles for all operational testing, the

Navy plans to commit to low-rate production of the ALR-67(V)3 before

determining through completion of operational testing that the system is

Page 1 GAO/NSIAD-96-68 Electronic Warfare
B-260441 

operationally effective and operationally suitable.1 As a result, the Navy

risks procuring a deficient system that may require expensive

modifications and retrofit to achieve adequate performance.

Adverse Impact of

Premature Production

of ALR-67 RWRs

Despite the importance of operational testing2 as a management control to

ensure adequate system performance, DOD started production of the

original ALR-67 and its successor, the ALR-67(V)2, before they were

operationally tested. The adverse consequences included deploying an

unsatisfactory system to the operational forces and placing newly

produced systems in storage rather than having them enhance the Navy’s

combat posture.

The Navy began producing the original ALR-67 in the early 1980s before

proving it was operationally effective. Subsequently, the system performed

so poorly during operational testing that the testing had to be curtailed.

However, the Navy continued production while the system’s performance

problems remained unresolved.

As a result, the Navy installed the unsatisfactory system in operational

aircraft for use by the combat forces. Subsequently, some Navy pilots

during Operation Desert Storm distrusted the system to the extent that

they stopped using it to detect threat radars and relied instead on other

means that DOD considers classified.

To solve the ALR-67’s problems, the Navy acquired the ALR-67(V)2. Once

again, however, the Navy started production before correcting

performance deficiencies. After operational testing showed that the

ALR-67(V)2 did not meet the Navy requirement, some 230 production

units, costing $75 million, were placed in storage pending modifications to

achieve satisfactory performance.

1

Operational effectiveness refers to the ability of a system to accomplish its mission in the planned

operational environment. Suitability is the degree to which a system can be placed satisfactorily in

field use considering such factors as reliability and maintainability.

2

Operational testing is DOD’s primary means of evaluating weapon system performance. It is also a key

internal control to ensure that decisionmakers have objective information available on a weapon

system’s performance to minimize risks of procuring costly and ineffective systems.

Page 2 GAO/NSIAD-96-68 Electronic Warfare
B-260441 

Developmental Tests

Reveal Problems With

ALR-67(V)3 Prototype

The Navy is developing the ALR-67(V)3 to overcome the problems with the

previous versions of the ALR-67. However, developmental testing of

ALR-67(V)3 prototypes conducted so far has disclosed numerous

deficiencies with its performance. According to the System Anomaly

Reports3

 the Navy provided to us, some of the deficiencies include

• misidentification of threat radars or identification of friendly radar as a

threat,

• warnings of multiple threats when only one is present or vice-versa, or

• failure to detect radars tracking the aircraft.

Moreover, developmental tests show that it has not been proven

compatible with the F/A-18 aircraft’s radar. For example, normal operation

of the F/A-18 radar can cause the ALR-67(V)3 to warn of nonexistent

threats.

The System Anomaly Reports indicate that some of the reported

deficiencies were identified as early as mid-1993, and the contractor’s

recommended action for many of the reported deficiencies was not to

proceed to operational evaluation until corrected. Without operational

evaluation, however, the Navy cannot have complete assurance that any

planned fixes for those deficiencies will work.

In addition to the developmental tests of effectiveness conducted earlier,

the Navy conducted an operational assessment of a developmental model

of the ALR-67(V)3 in September 1994. However, because the system was

not production representative, that is, it was not of the final design

planned to be produced, the Navy deferred testing of operational

suitability. Thus, the Navy has no assurance yet that suitability

considerations, such as reliability and maintainability, will be adequate.

Legislative

Requirement Imposed

on ALR-67(V)3

Program

Because of concerns that the Navy was proceeding with a high-risk

acquisition plan for the ALR-67(V)3, the Congress included, among other

requirements in the fiscal year 1995 Department of Defense Appropriation

Act, a requirement that DOD certify, based on the results of joint

developmental tests and operational test flights, that the ALR-67(V)3 was

potentially operationally effective and potentially operationally suitable

before low-rate production. This requirement was applicable only during

fiscal year 1995. In its comments on a draft of this report, however, the

3

System Anomaly Reports are official records created to document instances when a system fails to

perform as expected.

Page 3 GAO/NSIAD-96-68 Electronic Warfare
B-260441 

Navy states that it will adhere to this and other fiscal year 1995

requirements pertaining to the ALR-67(V)3.

The Navy has defined an operational test program for the ALR-67(V)3

calling for a first phase of joint developmental tests with 

1 month of operational test flights, which it calls OT-IIA. OT-IIA will occur

primarily in the Navy’s indoor chamber facility at Patuxent River,

Maryland, and is planned to be completed in August 1996. This is to be

followed by a 4-month phase of operational flight testing called OT-IIB that

is planned for the Navy’s open air range at China Lake, California. The

Navy’s program schedule indicates that OT-IIB is scheduled to be

completed during the spring of 1997. According to the program office,

potential fixes for the developmental problems described above will be

tested and confirmed using 20 production representative assets procured

specifically for the operational tests.

ALR-67(V)3 Schedule

Contains Unnecessary

Risk

Notwithstanding the unresolved deficiencies in the ALR-67(V)3’s

effectiveness, the unknown nature of its suitability, and the Navy’s

experiences with earlier versions of the ALR-67, the Navy plans to commit

to the ALR-67(V)3’s low-rate initial production before completing the

second phase of operational testing (OT-IIB) in the spring of 1997.

According to the program schedule, the Navy plans to award a low-rate

production contract in the first quarter of fiscal year 1997 for the

manufacture of ALR-67(V)3 systems for deployment on F/A-18 aircraft

after the first phase (OT-IIA) of joint developmental and operational

testing. However, while OT-IIA may demonstrate potentialities, the Navy

will not be able to conclude from this testing phase alone that the system

is operationally effective and suitable; such a conclusion must be based on

a large number of flight hours in a realistic environment that only OT-IIB

can provide. Furthermore, because the Navy intends to conduct both

operational test phases using ALR-67(V)3 production representative

systems previously procured as test assets, low-rate production of

additional new systems is not needed to complete operational testing.

While the Navy’s stated intent to adhere to an earlier requirement that the

ALR-67(V)3 be potentially effective and suitable may provide some

assurance of future success, we do not believe that this goes far enough.

Past failures with earlier versions, unresolved deficiencies with the

ALR-67(V)3, and the availability of test assets for all operational tests,

leads us to conclude completion of OT-IIB rather than an early

Page 4 GAO/NSIAD-96-68 Electronic Warfare
B-260441 

commitment to low-rate initial production after OT-IIA would be a far

more prudent acquisition strategy.

Recommendation To minimize the risk of procuring another deficient RWR, and because the

Navy does not need additional ALR-67(V)3 units to complete operational

testing, we recommend that the Secretary of Defense require that the

ALR-67(V)3 complete both phases of operational testing (OT-IIA and

OT-IIB) to determine its effectiveness and suitability, and that the

deficiencies identified during developmental testing have been resolved

before committing to low-rate production.

Matter for

Congressional

Consideration

Because the Navy has not modified what we believe is a high-risk

acquisition strategy, the Congress may wish to restrict DOD from spending

any funds on ALR-67(V)3 production until both phases of operational

testing (OT-IIA and OT-IIB) are complete and the system is certified

effective and suitable by DOD.

Agency Comments DOD generally concurred with this report, but indicated it did not agree

with our draft report recommendation that the Secretary of Defense

require the Navy to do sufficient testing before production to ensure that

developmental deficiencies have been resolved. DOD maintained that

because of the congressional reporting requirements, it saw no need to

place any additional requirements on the Navy. However, when the

requirements were enacted, the deficiencies identified during

developmental testing were not generally known. As we stated in this

report, the Navy’s intent to adhere to the earlier requirements may provide

some assurance of future success, but in light of the unresolved

deficiencies, the Navy’s continued adherence to its acquisition strategy is

not reassuring. We have, therefore, modified our report and

recommendation to address the congressional reporting requirements and

restate our concern that the Navy continues to pursue a high-risk

acquisition strategy by not completing both phases of operational testing

before low-rate production and by planning to acquire additional systems

not needed for testing. DOD’s comments are presented in their entirety in

appendix I along with our evaluation of them.

Scope and

Methodology

We performed our work at the Naval Air Systems Command, Washington,

D.C.; the Offices of the Director of Electronic Combat and Director of

Page 5 GAO/NSIAD-96-68 Electronic Warfare
B-260441 

Operational Test and Evaluation (DOT&E) and the Office of the Secretary of

Defense (OSD), Washington, D.C. In evaluating ALR-67(V)3 performance,

we reviewed developmental test results, including System Anomaly

Reports and operational assessment results. We also discussed the test

results and potential performance issues with DOT&E, OSD, and Navy

officials responsible for oversight of electronic warfare systems’

acquisition.

Our review was performed from January 1995 through April 1996 in

accordance with generally accepted government auditing standards.

We are sending copies of this report to other appropriate congressional

committees; the Secretaries of Defense and the Navy; and the Director,

Office of Management and Budget. We will make copies available to others

upon request.

Please contact me at (202) 512-4841 if you or your staff have any questions

concerning this report. Major contributors to this report were Jackie B.

Guin, Charles A. Ward, and A. Delores Cohen.

Sincerely yours,

Louis J. Rodrigues

Director, Defense Acquisition Issues

Page 6 GAO/NSIAD-96-68 Electronic Warfare
Page 7 GAO/NSIAD-96-68 Electronic Warfare
Appendix I 

Comments From the Department of Defense

Note: GAO comments

supplementing those in the

report text appear at the

end of this appendix.

See comment 1.

See comment 2.

Page 8 GAO/NSIAD-96-68 Electronic Warfare
Appendix I 

Comments From the Department of Defense

See comment 1.

See comment 3.

See comment 4.

Page 9 GAO/NSIAD-96-68 Electronic Warfare
Appendix I 

Comments From the Department of Defense

Page 10 GAO/NSIAD-96-68 Electronic Warfare
Appendix I 

Comments From the Department of Defense

The following are GAO’s comments on the Department of Defense’s (DOD)

letter dated March 7, 1996.

GAO Comments 1. According to the ALR-67(V)3 program schedule provided to us by the

program office, the Navy plans to begin low-rate production of the system

after the first of two phases of testing (OT-IIA). DOD considers OT-IIA to be

joint developmental tests and operational flight testing. However, as

discussed in DOD’s letter, the first phase will be conducted primarily

indoors in a chamber facility at Patuxent River, Maryland. Thus, most of

OT-IIA will not be done in a realistic operational environment as required

by DOD policy and should therefore not be considered to constitute

realistic operational testing. The Navy’s program schedule further

indicates that the second phase of testing (OT-IIB), which is planned for a

4-month period at the Navy’s open air range at China Lake, California, is to

take place after low-rate production begins.

2. Requiring the Navy to comply with our recommendation would better

ensure that the Navy does not commit to production of a deficient system,

as it did with earlier versions of the ALR-67. DOD’s compliance with the

congressional reporting requirements, including mandatory certification

from the Director of Operational Test and Evaluation that the ALR-67(V)3

is potentially operationally effective and potentially operationally suitable,

may provide some assurance of future successful performance. However,

DOD can only be certain that the ALR-67(V)3’s problems have been solved

by satisfactorily completing both phases of testing (OT-IIA and OT-IIB)

and determining that it is both effective and suitable.

3. The majority of the flight testing, lasting 4 months according to the

Navy’s schedule, is to be conducted after the low-rate production decision.

4. We believe that the intent to comply with congressional reporting

requirements will ensure that the ALR-67(V)3 is better tested prior to

beginning production than its predecessor ALR-67 models. However,

assurance of operational effectiveness and suitability can only be achieved

through successful completion of both test phases (OT-IIA and B). Despite

the various congressional reporting requirements, the Navy may still

commit to the ALR-67(V)3’s low-rate production before demonstrating in a

realistic operational environment that the system’s performance is

satisfactory by completing both phases of testing. Compliance with our

recommendation, in addition to the reporting requirements, could prevent

premature commitment to production.

(707104) Page 11 GAO/NSIAD-96-68 Electronic Warfare
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100