Tactical Intelligence: Accelerated Joint STARS Ground Station Acquisition Strategy Is Risky
Released: 1996-05-23T00:00:00

Letter

Background

Joint STARS is a multiservice, multimode radar system that is to provide the capability to locate, track, and classify wheeled and track vehicles beyond ground line of sight, during day and night, under most weather conditions. It is to provide Army Corps and Division commanders an “electronic high-ground” from which to observe enemy forces across the forward line of their own troops into an enemy’s first and second echelons. The Joint STARS radar is mounted on an Air Force E-8 aircraft, a Boeing 707 variant. It is to provide real-time information simultaneously to operators in the aircraft and operators in Army GSMs. These GSMs are to have the ability to supplement this radar data with unmanned aerial vehicle imagery and electronic intelligence reports. Through fiscal year 2001, the total cost of the Army’s Joint STARS development and acquisition is estimated at $1.4 billion.

Since the Joint STARS program inception, four versions of GSMs have been developed prior to the CGS. They are the Limited Procurement Urgent, the Interim GSM, the Medium GSM, and the Light GSM. Descriptions of the various GSMs are provided in appendix I. Production quantities by fiscal year and GSM variant are detailed in table 1.

The CGS Acquisition Plan

The Army recently issued a solicitation for the CGS system and selected a contractor to produce the system. It awarded an 8-year production contract on December 14, 1995, and made a fiscal year 1996 commitment to the production of 18 systems, the maximum production allowed by the solicitation. The CGS system is to provide the same functionality as the Light GSM with an initial enhancement of the integration of secondary imagery data, and planned additional enhancements provided by post-award contract modifications.

The CGS acquisition strategy provides for 2 years of LRIP, during which the Army anticipated buying 22 CGS systems at an estimated cost of about $138 million, though it received approval from DOD to procure up to an additional 16 CGS systems to accommodate other service and allied requirements. The Army’s first year commitment to 18 systems and current plan to acquire 16 systems in the second year raises the estimated 2-year LRIP cost to over $153 million. Regarding program cost, DOD stated that the CGS LRIP quantity includes not only the number needed for testing purposes, but considers production rate efficiencies and cost factors. It believes that producing only four prior to test would require the stop and restart of production, resulting in loss of skilled people, inefficient use of contractor resources, and higher costs. The CGS LRIP quantity does not, however, reflect consideration of production rate efficiencies and cost factors because under the CGS contract’s pricing structure, the planned second LRIP year acquisitions can be purchased in later years at lower cost. In sum, under the CGS contract, the Army can save millions of dollars by lowering future CGS LRIP acquisitions to the minimum quantity necessary to maintain the contract and then contracting for those systems in the post-LRIP years.

The Light GSM and the Medium GSM were scheduled to be operationally tested during a Joint STARS multiservice OT&E. That test was delayed and then altered because of the deployment of Joint STARS assets to the European theater to support Bosnian operations. The Army now plans to evaluate the Medium and Light GSMs during that deployment and follow-on tests, if needed. It also plans to conduct an initial OT&E of the CGS system in the first quarter of fiscal year 1998. The degree and length of that initial OT&E will depend on how similar the CGS system is to its predecessors, which will be a function of the approach that the CGS contractor follows.

The CGS solicitation provided functional specifications such that the proposals received may or may not represent significant hardware and software differences from already procured GSMs. The degree of technological difference between the CGS system and its predecessor systems, the Light GSM and Medium GSM, depends on the approach taken by the contractor. That difference will, in turn, influence the degree to which the Light and Medium GSM’s performance during any OT&E can and should be relied upon as an indicator of the CGS’s maturity to continue production. Furthermore, the more similar the CGS system is to its predecessors, the less extensive its initial OT&E will need to be.

The Army Has Not Justified Accelerated Production

The Army began procuring CGS systems prior to the completion of an OT&E by any GSM. However, the Army did not perform any risk analyses demonstrating that there was (1) an urgent need for the added capabilities of the CGS system or (2) any significant benefit to be derived from its accelerated procurement. According to DOD, the revised CGS development and production schedule fields ground stations in synch with E-8C aircraft deliveries. Under the prior development schedule, the Army planned to continue to buy pre-CGS model ground stations—presumably also in synch with E-8C aircraft deliveries. Furthermore, an Army official in the program executive office that has oversight of the Army’s Joint STARS program stated that the Air Force is behind in its E-8C delivery schedule and that, as a result, GSM acquisition is currently scheduled ahead of aircraft fieldings.

LRIP Acquisitions Prior to OT&E Raise Program Risks

Over the years, we have reported on numerous instances in which production of both major and nonmajor systems were optimistically permitted to begin under LRIP and continue based on factors other than the systems’ technical maturity. In our November 1994 report on the use of LRIP in the acquisition process, we detailed a number of examples of systems that entered LRIP before operational tests were conducted and that later experienced significant problems. For example, a year into the LRIP of the Navy T-45A aircraft, OT&E demonstrated that the T-45A was not effective in a carrier environment and was not operationally suitable because of safety deficiencies. Subsequent major design changes included a new engine, new wings, and a modified rudder.

DOD believes that, unlike the Navy T-45A aircraft, the CGS is not a new, immature system. It has stated that the CGS system uses 100 percent of the Light GSM mechanical design, rack structure, power distribution, lighting, ventilation, and air conditioning. It has also stated that the Light GSM software baseline is the CGS baseline and that the CGS system represents the Light GSM functional baseline with the addition of product improvements. However, the CGS contractor may make configuration changes that could represent significant hardware and software differences from already procured GSMs. Furthermore, DOD’s position is also contradicted by the 2-year delay of the GSM full-rate production decision to follow a CGS OT&E and by the Joint STARS integrated product team’s call for an independent assessment of the CGS’s testing risk, given the nature and extent of the configuration changes that the selected contractor may make.

The risks of systems starting production before operational tests are conducted are numerous. They include reliability that is significantly less than expectations, systems that cannot meet current specifications, systems that are never fielded and/or retired after fielding because of poor performance, and systems that require significant and expensive post-fielding repairs for faults identified during OT&E. While there is an operational need for Joint STARS, and despite the desire of operational commanders to have more capable systems as soon as possible, the fact remains that the Army has not adequately justified the urgency or benefits to be derived from accelerated fielding of the CGS in 1998 versus the originally planned fielding in fiscal year 2002.

Prior Test Results Further Indicate Risk

The Army’s CGS acquisition strategy seems to ignore the fact that to date the GSMs have undergone limited testing and demonstrated disappointing results in those tests. That acquisition strategy allowed the Army to begin procuring CGS systems without demonstrating resolution of issues raised as a result of prior tests and will allow it to continue procuring systems without demonstrating resolution of those issues.

In December 1991, a decision was made that the Medium GSM would undergo a limited user test rather than a traditional initial OT&E. The absence of important functionality, including an unmanned aerial vehicle interface, a production representative data link, Defense Mapping Agency electronic map databases, and trained military operators, prompted this decision. Based on the results of this test, which occurred in early 1993, the Army Operational Test and Evaluation Command provided an overall assessment of the Medium GSM’s performance. It stated that the Medium GSM “consistently demonstrated potential to be operationally effective” and that the Medium GSM “demonstrated potential to be operationally suitable” (emphasis added). However, this was not a finding that the Medium GSM was operationally effective or suitable. The Command also noted that the “current software lacks robustness and reliability, and limits mission performance.” One of the Command’s recommendations was that prior to LRIP fielding, the Medium GSM “must successfully complete an independently evaluated operational demonstration including simultaneous employment of all software, interface, and tactics, techniques, and procedures corrections.” The Medium GSM has yet to successfully complete an independently evaluated operational test. Its initial OT&E was to be the multiservice OT&E.

The Medium GSM follow-on system, the Light GSM, was also to participate in the multiservice OT&E. Like the Medium GSM, the Light GSM has yet to complete an OT&E. The Light GSM has, however, undergone other tests, including a Force Development Test and Evaluation (FDT&E) in September 1994; reliability confidence testing from October through December 1994; and a follow-on demonstration at Eglin Air Force Base in January 1995. In May 1995, we reported to the Secretary of Defense that based on a preliminary review of those test results, it was clear that the Light GSM had not met the DOD-set LRIP exit criteria and that our preliminary analysis indicated that, at best, the Light GSM had only passed 2 of the 12 Light GSM performance-related LRIP exit criteria. At the same time, the DOD Director of OT&E concluded that the Light GSM had only passed 1 of the 12 Light GSM performance-related LRIP exit criteria. The Director recommended a formal review of the program to identify the causes of problems, solutions, and appropriate tests to demonstrate the solutions. In a June 30, 1995, memorandum, the Director, commenting on efforts to resolve 55 specific problems identified in the Light GSM testing, stated that his goal “was to see that the Army had identified the key problems and was working effective fixes for those problems.” He added that he wanted the Joint STARS multiservice OT&E “to have a reasonable chance of success.” According to an OT&E official, the Director’s assessment of the Light GSM’s performance during those tests has not changed. The issue of the 55 specific problems was resolved based on the Director’s satisfaction “that the Army has identified a process to fix the various problems that have been identified . . . .”

In some instances, problems were attributed to shortfalls in operator training or another non-materiel cause. The majority of deficiencies involved software fixes, not major hardware redesign. The Army has also gained experience operating the GSMs assigned to the III Corps and XVIII Airborne Corps and in training and preparation for multi-service OT&E. In November 1995, the Program Executive Officer for Joint STARS certified the system ready for OT&E, which attests to the developer’s confidence in system maturity.

DOD believes that the GSMs’ prior test results indicate only prudent program risk. It states that the series of tests used in development of the GSMs, including a limited user test, FDT&E, reliability confidence testing, and other demonstrations, have been a continuous fix-test-fix process, which has identified shortfalls, determined fixes, and verified or tested the results. It also notes that during the current deployment of Joint STARS to the European Theater (Bosnia-Herzegovina), members of the Army and the Air Force test commands will conduct an operational evaluation of Joint STARS performance. Although the Army and the Air Force plan to operationally evaluate Joint STARS during that deployment, how well the Army’s process has worked remains to be demonstrated through the Light GSM’s performance during an OT&E.

Delaying Commitments Would Lower Cost

The Army’s commitment to its currently planned second year LRIP buy of 16 CGS systems prior to the completion of the CGS OT&E would raise not only the program’s risk but also its cost. The CGS contract provides decreasing unit costs over its 8-year life. Furthermore, a program official stated and our review of the contract indicates that the Army needs to commit to only one CGS system in the second LRIP year to maintain the contract. If the Army buys one system in fiscal year 1997 and 37 systems in the third and fourth years of the contract, it could save over $5 million while obtaining the same 4-year buy of 56 systems currently anticipated given its fiscal year 1997 budget request and approved acquisition strategy. These savings can be seen in a comparison of tables 2 and 3. Table 2 details the first 4 years of the contract’s variable CGS acquisition costs under the Army’s anticipated future buy schedule. Table 3 details the first 4 years of those costs under a plan that minimizes the size of the second year LRIP commitment.

Recommendation

The Army lacks an analysis justifying a need to accelerate the fielding of the CGS system and can save millions of dollars by minimizing production in its second year of CGS production. Furthermore, there are inherent risks in procuring systems prior to their successful completion of an OT&E and the benefits of the Army’s acquisition strategy do not clearly outweigh the associated risks. We therefore recommend that the Secretary of Defense direct the Secretary of the Army to limit the future system procurement to the minimum quantity necessary to maintain the CGS contract (i.e. one system in each contract option year) until the CGS has successfully completed an OT&E.

Agency Comments and Our Evaluation

In commenting on a draft of this report, DOD disagreed with our conclusion that the Army’s CGS acquisition strategy was unnecessarily risky and our recommendation to reduce that risk. DOD took the position that the acquisition strategy espouses prudent risk in balance with program cost, schedule, and technical requirements.

DOD’s comments are reprinted in their entirety in appendix II.

Matters for Congressional Consideration

In light of DOD’s unwillingness to have the Army revise its acquisition strategy for the CGS, Congress may wish to take the actions necessary to limit the number of CGS systems to be procured under LRIP prior to the CGS successfully completing operational testing.

Scope and Methodology

During this review, we interviewed officials at and reviewed documents from the offices of the Under Secretary of Defense for Acquisition and Technology and the Director for Operational Test and Evaluation in Washington, D.C. We also visited officials and reviewed documents from the U.S. Army Materiel Systems Analysis Activity, Aberdeen, Maryland, and the U.S. Army Communications and Electronics Command, Office of the Program Manager for Joint STARS, Fort Monmouth, New Jersey.

We conducted this review from August 1995 to April 1996 in accordance with generally accepted government auditing standards.

We are sending copies of this report to other appropriate congressional committees; the Director, Office of Management and Budget; and the Secretaries of Defense, the Army, and the Air Force. We will also make copies available to other interested parties upon request.

Please contact me at (202) 512-4841 if you or your staff have any questions concerning this report. Major contributors to this report were Thomas J. Schulz, Charles F. Rey, Bruce H. Thomas, and Gregory K. Harmon.

Ground Station Module Descriptions

Limited Procurement Urgent (LPU). The LPU GSMs were produced and deployed as replacements to the AN/UPD-7 Ground Station Terminal. They receive data from the Mohawk Side Looking Airborne Radar and do not receive/process data from Joint Surveillance Target Attack Radar System (Joint STARS) E8 aircraft. The Army acquired nine LPU GSMs. They are expected to be decommissioned no later than fiscal year 1997.

Interim Ground Station Module (GSM). The Interim GSM receives and processes data from both the Joint STARS E8 aircraft and the Mohawk Side Looking Airborne Radar. Eight engineering and manufacturing development Interim GSMs were developed and fielded to the XVIII Airborne. These systems represent the current GSM contingency force. The Interim GSM was deployed to Operation Desert Storm/Desert Shield. No production is planned.

Medium GSM. This module provides enhancements to the Interim GSM capability. Its development stemmed from a Department of Defense (DOD) decision that was made in fiscal year 1989 to restructure the Army Joint STARS GSM program. The Medium GSM enhancements include a downsized electronic suite, an enhanced man/machine interface with extensive Built In Test/Built In Test Equipment capabilities, and the ability to simultaneously display and analyze data from multiple sensors. The Army acquired 12 Medium GSMs.

Light GSM. This module is housed in a light weight multipurpose shelter, a standard integrated command post shelter variant, mounted on a High Mobility Multi-Purpose Wheeled Vehicle. It is to provide the light/contingency forces a C130 Drive-on/Drive-off Joint STARS capability. The Light GSM has a prime and support vehicle, each with a trailer/generator in tow. It is supposed to be able to operate on the move, receive unmanned aerial vehicle imagery and intelligence reports, and incorporate electronic map backgrounds. The Army plans to acquire a total of 10 Light GSMs.

Common Ground Station (CGS). The CGS system is to provide Light GSM functionality with the addition of the integration of secondary imagery data. Further enhancements are expected and are to be achieved through post-award modifications to the contract. Two versions of this ground station are being contemplated (i.e., a light and heavy CGS). The Light CGS will be patterned on the Light GSM two-vehicle configuration. The heavy CGS is to be a track-mounted system, intended to provide the heavy forces a high speed, cross-country/off-road GSM. It is to be integrated into a Bradley Fighting Vehicle variant. Integration of the CGS capability into a tracked vehicle is part of the preplanned product improvement initiatives and will not be included in the fiscal year 1996 CGS contract award. Initial CGS fielding is planned for fiscal year 1998. The Army currently anticipates the acquisition of 73 CGS systems.

Comments From the Department of Defense

The following are GAO’s comments on DOD’s letter dated January 24, 1996.

GAO Comments

1. While the CGS contractor has prior experience developing and producing ground stations, those ground stations have undergone limited testing and demonstrated disappointing results. Among its previous work, the CGS contractor developed and produced the two immediate predecessor GSMs to the CGS, the Medium and Light GSMs. As we stated in our report, based on the results of a limited user test of the Medium GSM, the Army Operational Test and Evaluation Command stated that the Medium GSM consistently demonstrated the potential to be operationally effective and the potential to be operationally suitable. It noted that the “current software lacks robustness and reliability, and limits mission performance.” It recommended, among other things, that prior to LRIP fielding the Medium GSM “must successfully complete an independently evaluated operational demonstration including simultaneous employment of all software, interface, and tactics, techniques, and procedures corrections.” Furthermore, the Light GSM passed only 1 of 12 performance-related criteria during developmental testing, and neither the Medium nor the Light GSM has yet successfully completed an OT&E. 2. We continue to believe that the CGS acquisition strategy risks millions of dollars on systems that have not yet been demonstrated operationally effective and suitable. We have, however, revised the report to reflect the Army’s apparent commitment to evaluate the operation of the Joint STARS system during deployment to Bosnia-Herzegovina. 3. We have revised our recommendation to allow the Army to maintain its CGS contract in effect and thus avoid a break in production. Because the contract provides decreasing unit costs over its life, and since the Army has already committed to 18 first-year LRIP systems, we want to further limit LRIP pending successful completion of an OT&E.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.