The armed services have a long-standing shortfall in their capability to adequately test electronic combat systems on aircraft and ships. From August 1989 through July 1991, we issued a series of reports identifying each service's problems with their test equipment for electronic combat systems. To address these problems, in June 1993, the Air Force and Navy approved a Joint Mission Need Statement for a flight-line electronic combat systems tester to improve aircrafts' electronic combat test capability. The Department of Defense designated the Air Force as the lead service, and the Air Force and Navy entered into a memorandum of agreement in December 1994 to establish a joint tester program. Following a concept development phase, an engineering and manufacturing development contract was awarded in March 1996. The tester has been developed to provide the Air Force and Navy with a flight-line test capability for aircraft electronic combat systems, to include both on-board systems and those mounted outside the aircraft in pods. The contractor for the tester, AAI Corporation, has developed a basic core test set that can be used with various aircraft. The basic core test set is supplemented by subsidiary test program sets and related software for each aircraft type and its specific systems. The tester provides an end-to-end test capability for electronic combat systems, including jammers, radar warning receivers, and other subsystems and their associated wiring. The tester inputs radio frequency signals into the aircraft's antennae and then measures whether the signals were correctly received and the appropriate responses generated by the electronic combat systems. The tester can identify faulty wiring and also isolate the faulty system component to make the maintenance task easier. Developmental testing of the basic core test set and the test program set for the F-15C was completed in October 2000 and for the F/A-18C test program set in December 2000. Additional test program sets are to be developed for most of the current Air Force and Navy fighter aircraft equipped with electronic combat systems, and there will be growth potential for adapting the system for future aircraft. Quantities to be procured include 56 Air Force and 40 Navy basic core test sets with test program sets for the F-15C and F/A-18C, respectively. The total planned procurement for the basic core test set is 121 for the Air Force and 188 for the Navy. Test program sets for other aircraft are to be subsequently developed and procured. Schedule slippage and cost growth have occurred in the tester program. However, the Air Force's and Navy's use of the new tester indicates that performance goals are being met and that a useful capability is likely to be achieved. The development schedule for the new tester has slipped about 2 years from the original plan's schedule because the difficulty in designing the system was underestimated. This delayed the production decision for the tester until April 2001. Prior to the production decision, the services completed developmental testing but did not undertake operational testing of the tester as planned. Operational testing was deferred because the lead test agency--the Air Force Operational Test and Evaluation Command-- was concerned that the tester contractor was still making design changes to the system and that operational testing should utilize articles that represent the final design to be produced. Consequently, additional developmental testing using available prototypes was substituted for operational testing to provide test data to support the production decision. If operational testing of the tester's final design identifies a need for further design changes, the testers procured would require retrofit. Regarding program cost, the cost under the initial development contract for the basic core test set and the F-15C and F/A-18C test program sets was originally estimated to be about $12 million. As of January 2001, the cost of the contract had increased to $28.9 million. Ultimately, the program's total cost will be a function of future decisions regarding the extent to which other aircraft and electronic combat systems, such as the radar warning receivers and radar jammers on the Air Force's F-15E and the Navy's F/A-18E/F, will use the new tester. These aircraft and their electronic combat systems will require the development and procurement of customized test program sets, as well as additional quantities of the basic core test set. According to the Air Force, the tester has performed effectively in testing. Developmental testing of the basic test set and the F-15C test program set was performed at Eglin Air Force Base from March through October 2000. According to the Air Force's developmental test organization, the tester met or exceeded expectations for all test objectives. For the key performance parameter of demonstrating at least 90-percent success in fault detection, the tester detected and isolated all faults. The testing disclosed that 29 of 31 F-15Cs actually had one or more faults in their electronic combat systems. The faults detected ranged from the identification of parts needing to be replaced inside the electronic combat systems (so-called Group B) to the wiring, antennae, and control units that connect the systems to the aircraft (so-called Group A). According to program officials, no existing tester has previously been able to test the Group A equipment as well as the Group B systems. Moreover, the new tester provides an ability to augment an electronic combat system's internal system check (referred to as Built in Test, or BIT). In the past, if a system's BIT indicated a fault, maintenance technicians were forced to remove the system components from an aircraft to retest them in the maintenance shop--a time-consuming and cumbersome process. The new tester provides a check against the BIT without the system's removal from the aircraft. The Air Force used the tester to test operational 33rd Fighter Wing F-15C aircraft at Eglin about to be deployed to Operation Southern Watch in Iraq. After successful testing at the 33rd, it was then used to test F-15C aircraft at the 1st Fighter Wing at Langley Air Force Base. These aircraft are regularly deployed to Operation Northern Watch in Iraq. At Langley, 12 of 13 F-15Cs thought to be fully mission capable actually had one or more faults in their electronic combat systems. The potential effects of some of these faults could have been that these aircraft would have entered combat with partially functioning protective systems; some of these faults would have left the systems nonfunctional. Navy test officials advised us that the tester also performed well with their F/A-18C aircraft, identifying faults that the Navy's current test equipment had been unable to identify. The Navy performed developmental testing of the basic test set and the F/A-18C test program set at Naval Air Warfare Center, Weapons Division, Point Mugu, California; Miramar Marine Corps Air Station, California; Lemoore Naval Air Station, California; and Oceana Naval Air Station, Virginia, from September 1999 through January 2001. The Navy tested 16 aircraft in California, 14 of which had faults identified by the tester. Subsequently, 10 F/A-18C aircraft were tested at Oceana, and all were found to have unknown faults in their electronic combat systems. Each of the 10 aircraft had at least 3 faults disclosed by the new tester, and 1 aircraft had 12 faults. Because the tester works so well at disclosing faults, the services plan to expand its use to other electronic combat systems on other fighter aircraft. The Air Force intends to use the tester also on its F-16s and the Navy, on its F-14s. Because the tester has a much greater ability to identify electronic combat system problems, it can identify faults that the currently used test equipment is not able to find. The disclosure of these problems could have significant implications for readiness levels, logistics, and maintenance. Additionally, the failure to address problems with the electronic combat systems could encourage pilots to rely less on their electronic combat systems and more on other specialized aircraft designed to suppress enemy air defenses, such as the EA-6B. The test results for the F-15C and F/A-18C have implications for readiness levels not only for those types of aircraft, but also for other aircraft using either the same or similar electronic combat systems (such as the F-15E and F/A-18E.) Readiness levels are lower than the services previously believed, since the F-15C and F/A-18C aircraft, which were previously (and reasonably) reported by the services as fully mission capable, actually have electronic combat systems with previously unknown faults. During our review, we found this to be true as a result of our direct observation of the new tester in use at Eglin. We observed four aircraft being tested for an upcoming Southern Watch deployment. In the testing that we observed, all four aircraft, which were believed to be fully mission capable, were found to have unknown faults that had to be repaired. The Air Force has a criterion that its F-15 fighter wings seek to maintain an 81-percent fully mission capable rate. However, combining the statistics for using the new tester in 2000 at the Eglin wing (29 of 31 aircraft had unknown faults) and the Langley wing (12 of 13 had unknown faults), the Air Force found that 41 of 44 F-15Cs tested were not fully mission capable. Likewise, since all 10 of the Navy's F/A-18C aircraft tested at Oceana Naval Air Station with the new tester had three or more unknown faults, the Navy also could face unacceptably low readiness levels. Once the services introduce the new tester for widespread usage, they are likely to find, as they did during testing, that the reliability of their electronic combat systems is much lower than previously thought. Consequently, more logistics support in the form of additional spare parts to fix previously undiagnosed faults will be required in the future. According to Air Force officials, on the basis of the new tester's use on the F-15C aircraft at Eglin and Langley, the Air Force will experience a requirement for more frequent repairs and an added logistics problem. At Warner Robbins Air Logistics Center, we were advised that spare parts shortages already exist for F-15 electronic combat systems. Maintenance officials at both Eglin and Langley stated that these shortages cause them to use cannibalization--i.e., removing a working part from one aircraft to install it on another aircraft--to meet the wing's flying schedule. For example, while we observed the new tester being used on operational aircraft at Eglin, several cannibalizations of electronic combat system parts were required before the testing could be completed. Maintenance officials told us that because spare parts were in limited supply, it was common for aircraft being tested to use cannibalized parts from another aircraft in order to be repaired. Although the scope of our review did not include an assessment of the impact of using the new tester on logistics for the Navy's F/A-18C fleet, we believe that using the new tester could also reveal a significant future problem for F/A-18C operational deployments. Generally, even if the Navy does not have a spare parts shortage as serious as the Air Force's, maintaining the readiness of deployed aircraft on carriers is more difficult because of the quantity limitations on spare parts storage aboard ship. A Navy maintenance person advised us that on his carrier's recent deployment to Southern Watch, the spare parts for the electronic combat systems used on the F/A-18C were completely exhausted and maintenance personnel had to resort to cannibalization to maintain flight operations. This situation existed without the Navy's having access to the new tester, which would likely identify even more parts needing to be replaced. Our review indicates that, in addition to the potential for heightened readiness and logistics concerns, the introduction of the new tester could increase the maintenance burden on the services because the new tester could identify many more repairs that have to be made. This could intensify existing pressures on maintenance personnel to resort to cannibalization. As we stated in our recent testimony for the Congress, making repairs via cannibalization requires at least twice the maintenance time as making repairs using new spare parts. Moreover, if use of the new tester results in further increases to the maintenance burden, it could also affect the Air Force's problem in retaining skilled technicians. Reinforcing this, both Eglin and Langley maintenance officials advised us that there are already shortages of trained maintenance personnel at the 33rd and 1st wings. In fact, the Air Force Posture Statement 2000 cites low retention of maintenance technicians as one of four factors resulting in the 99-percent drop in the mission-capable rates of Air Force aircraft since 1994. Furthermore, given the test results associated with the use of the new tester on the F/A-18C, the Navy could expect a significant increase in its maintenance burden. However, we were not made aware of any particular retention problem associated with the maintenance burden being experienced by Navy personnel during this review. The new tester's use could cause pilots of Air Force and Navy combat aircraft to be reluctant to rely solely on their electronic combat systems for self-protection from enemy air defenses. Recognizing reduced readiness and reliability of their self-protection systems, pilots could look for greater support from other specialized aircraft designed to suppress enemy air defenses, such as the EA-6B. We recently reported that current suppression capabilities are not adequate. To the extent that the new tester discloses reliability problems with existing electronic combat self- protection systems, the need to improve suppression capabilities would only be that much greater. Given the experience from using the new tester on the F-15C and F/A-18C, it is likely that using the new tester will find a number of undisclosed faults in electronic combat systems. Many of the electronic combat systems on current aircraft are older systems that are already experiencing obsolescence problems, such as difficulty in acquiring spares due to vendors that go out of business or are no longer producing old technology equipment (referred to as "vanishing vendors"). The Air Force's special test program, called Combat Shield, is used periodically to test a variety of types of operational aircraft for readiness. Typically, even without using the new tester, testing via Combat Shield has found that some aircraft in every wing tested have faults in their electronic combat systems, regardless of the aircraft type. For example, Combat Shield found undisclosed faults when testing was conducted at wings equipped with the F-16. In fact, Air Force and Navy officials have already identified emerging problems regarding readiness, logistics, and maintenance for other electronic combat systems. This applies to systems both internally carried or externally mounted on an aircraft. For example, the ALQ-131 jammer system, externally carried by several Air Force aircraft, is projected to have a mission capable rate of 30 to 40 percent by 2006 because of obsolescence and the lack of spares. Furthermore, according to Air Force officials at Warner Robbins Air Logistics Center, funding priorities have constrained both spare parts acquisition and sustaining the engineering needed to address the obsolescent parts issue. The armed services have had problems for years with their ability to adequately test their electronic combat systems. The success of the new tester in providing improved test capability is a positive development. Because the tester has identified many more faults in the F-15C and F/A-18C electronic combat systems than the current test equipment was identifying, existing readiness, logistics, and maintenance problems with such systems could worsen. However, pilots would at least have greater knowledge about the readiness and reliability of their self-protection systems and their need for support from specialized aircraft designed to suppress enemy air defenses. On balance, we believe it makes sense for the Air Force and Navy to consider using the new test equipment on their nonfighter aircraft. Because the new tester's use provides the ability to identify previously unknown faults in electronic combat systems, we recommend that the Secretary of Defense direct the Air Force and the Navy to consider expanding the new tester's use beyond fighter aircraft to other types of aircraft. In written comments on a draft of this report, the Department of Defense agreed with our finding that the new tester provides a much better capability to assess electronic combat systems than the services' existing testers. It also agreed that once the services introduce the new tester for use on a widespread basis, they are likely to find that the reliability of the electronic combat systems is lower than previously thought. Consequently, more logistics support may be required in the future, and the maintenance burden may increase. The Department concurred with our recommendation. We reviewed the results of the Joint Service Electronic Combat Systems Tester development testing and determined program status through discussions with program office officials and a review of appropriate documentation. We discussed the status of the Air Force's aircraft electronic combat systems with Air Combat Command officials responsible for these systems on all Air Force operational aircraft. We held discussions regarding logistics support and maintenance with officials at Warner Robbins Air Logistics Center responsible for Air Force electronic combat systems. We held similar discussions with officials at Jacksonville Naval Air Station regarding Navy aircraft electronic combat systems. We also observed and discussed the testing of operational F-15C aircraft with officials at the 33rd Wing at Eglin Air Force Base and discussed the results of similar tests with officials of the 1st Wing at Langley Air Force Base. These two Wings have about 40 percent of the Air Force's F-15C aircraft. We also relied on our previous reviews of electronic warfare for background information on the existing logistics and maintenance problems with electronic combat systems. We conducted our review from August 2000 to August 2001 in accordance with generally accepted government auditing standards. We are sending copies of this letter to the Secretaries of the Air Force and Navy; to interested congressional committees; and to the Director, Office of Management and Budget. If you have any questions, please contact me on (202) 512-4841. Major contributors to this report were Michael Aiken, Terry Parker, and Charles Ward.

Summary:
The armed services have had problems for years with their ability to adequately test their electronic combat systems. The success of the new Joint Service Electronic Combat Systems Tester Program in providing improved test capability is a positive development. Because the tester has identified many more faults in the F-15C and F/A-18C electronic combat systems than has the current test equipment, existing readiness, logistics, and maintenance problems with such systems could worsen. However, pilots would at least have greater knowledge about the readiness and reliability of their self-protection systems and their need for support from specialized aircraft designed to suppress enemy air defenses. GAO believes that it makes sense for the Air Force and Navy to consider using the new test equipment on their non-fighter aircraft.