Enhanced Fiber Optic Guided Missile: Need to Define Requirements and Establish Criteria to Assess Performance
Released: 1995-10-17T00:00:00

Letter

Background

The EFOG-M is being designed to engage armored combat vehicles, other high value ground targets (such as command, control, and communication centers), and helicopters beyond the line of sight at ranges up to 15 kilometers. The system will consist of a gunner’s station and eight missiles mounted on a High Mobility Multipurpose Wheeled Vehicle. The missiles are launched toward a target area based upon forward intelligence information. After missile launch, the gunner can intervene at any time to lock on and engage detected targets. The gunner views the flight path and the target via a seeker (located in the missile) that is linked to the gunner’s video console by fiber optic cable. Figures 1 and 2 show the EFOG-M fire unit and missile and the potential EFOG-M deployment concept, respectively.

According to an Army official, the EFOG-M uses the same concept and some of the same technology as three previously terminated efforts costing more than $440 million—the Fiber Optic Guided Missile (FOG-M), the Non-Line-of-Sight Missile (NLOS), and the NLOS-Combined Arms (CA). The Army began development work in 1978 to demonstrate fiber optics guidance and conducted flight tests in 1984 to demonstrate the technology as an antitank missile (FOG-M). However, in late 1986, the Office of the Secretary of Defense (OSD) approved development not primarily as an antitank weapon but to provide defense against helicopters (NLOS). Although the Army had planned to produce NLOS, OSD decided to terminate the program once its development was completed because other programs had higher priority and other systems could accomplish the intended mission. However, the Army then terminated the program in January 1991 before completing development because of excessive cost growth. The Army restarted the effort as NLOS-CA in mid-1991, performed concept analyses, explored alternative acquisition strategies, and sought approval for engineering and manufacturing development. But its development was not approved.

The Army is now developing the EFOG-M and plans to acquire limited quantities under an advanced technology demonstration program designed to demonstrate potential technology enhancements; and the Army will provide the system and support it for the RFPI ACTD. RFPI is exploring new approaches to provide an early entry force that is significantly more capable against a heavy armored threat.

The primary objective of an ACTD is to accelerate the application of new technology to solve military problems. ACTDs are to (1) evaluate military utility before committing to acquisition, (2) develop operational concepts, and (3) rapidly provide operational capability. During this process, ACTD programs require much more early user involvement than expected during normal acquisition program phases. Department of Defense (DOD) officials believe ACTD programs will shorten the acquisition process.

Under the demonstration program, the Army plans to procure 12 fire units, 3 platoon vehicles, 300 missiles, and associated equipment at an estimated cost of about $280 million. According to Army officials, the development, demonstrations, and evaluations could result in one of the following actions: terminating the effort before building the system hardware (not a likely option); purchasing only the limited quantities and making a decision as to whether to leave the residual quantities in the field; procuring much larger quantities of the EFOG-M currently being developed (3,126 missiles and 120 fire units are being examined from an affordability standpoint); or substantially modifying the system and procuring larger quantities.

The Army plans to demonstrate EFOG-M performance and military utility through (1) simulations, (2) contractor-conducted missile performance tests, (3) a force-on-force demonstration along with other early entry systems and potential systems, (4) government check-out missile firings, and (5) a 2-year user fielding and evaluation of a residual force. Table 1 shows the schedule for these events.

The Requirement for EFOG-M Has Not Been Established

The Army does not have an agreed-upon requirement for the EFOG-M. It has not completed the documentation nor analyses for the EFOG-M program required for most acquisition programs. For example, the Army has not (1) prepared a mission need statement documenting the mission deficiency, (2) analyzed other alternatives to satisfy the need, (3) defined the system’s operational and performance requirements, nor (4) comprehensively compared EFOG-M’s cost and operational effectiveness to other existing or developmental systems.

According to Army officials, that type of documentation, analysis, and evaluation is not required for ACTD programs. They said these changes resulted from defense acquisition reform efforts. However, at the current time, U. S. Army Training and Doctrine Command (responsible for determining requirements) officials state (1) the system is needed for use with early entry forces and (2) the requirement will be defined during the ACTD. “NLOS-CA has struggled in budget competition within the Army because it is such a revolutionary concept. It simply doesn’t fit well anywhere within the Army’s branch structure and has been passed around among air defense (anti-helicopter version), artillery, and infantry branches.”

Because requirements and/or support for predecessor systems have disappeared after considerable effort and expenditure of funds, we believe that the EFOG-M requirement should be agreed upon and formally documented. In addition, we believe the system’s cost and operational effectiveness should be comprehensively compared to other alternatives for satisfying that requirement.

In its report (104-131, June 1, 1995) on the National Defense Authorization Act for Fiscal Year 1996, the House National Security Committee expressed concern that the Army is pursuing a weapon system that provides questionable value and possesses known fiscal risk. The committee recommended a provision (sec. 215) that would (1) require the Secretary of the Army to certify by December 1, 1995, that a requirement exists for the EFOG-M and whether there is a cost-effectiveness analysis supporting such requirement and (2) limit the expenditure of funds for the EFOG-M program to that identified in the current program plan only ($280 million, based on fiscal year 1995 constant dollars) and deny continuation of the program beyond fiscal year 1998 if contract obligations are not met.

Some Criteria for Evaluating Performance Are Not Specific

Army guidance for advanced technology demonstration programs require establishment of criteria to be met and the RFPI ACTD management plan recognizes that criteria as the technical goals for the system. A DOD instruction states that, to be effective, the criteria must be specific and quantitative. Since the ACTD’s objective is to judge the military value of the system, it appears reasonable and prudent to establish specific measurable standards as a basis for making the judgment.

The Army’s EFOG-M Advanced Technology Demonstration Plan establishes exit criteria for evaluating EFOG-M performance (see app. I). Some of these criteria are specific and easily measurable. For example, the plan establishes specific minimum criteria that must be accomplished by mid-1996 for missile reload time, the number of missiles mounted on each fire unit, and the system response time for missile launch. It also provides specific minimum criteria that must be accomplished by mid-1999 for missile range and set-up time for system operation.

However, the criteria for some other operational issues that project officials consider critical do not provide the specific values to be attained—a standard to measure against to determine success. For example, to demonstrate successful identification of targets, the minimum criterion to be accomplished in 1996 is “gunner recognition without diverting the missile and obtain in-flight intelligence.” However, the plan does not identify the minimum required probabilities of correctly identifying the target—a performance issue very critical to the effectiveness of the weapon system—either in 1996 or at the end of the technology demonstration.

Another criterion extremely important to the basic role and need for the system is demonstrating that targets can be engaged even though they are not within the gunner’s view. The criterion states that the Army is to demonstrate engaging targets not in the line of sight by mid-1996. But the criterion does not address the required probability for engaging each target correctly identified—a key determinant of the success of the system—either in 1996 or at the end of advanced technology demonstration in 1999.

In addition, the minimum criteria for warhead lethality is to “defeat existing threat tanks and helicopters.” But it does not establish and provide for measuring specific minimum required probabilities of defeating the tanks or helicopters with a single shot. However, the probability of killing a target with a single shot is critical to determining whether the system is cost-effective and, consequently, whether it should be procured.

We believe that in order to accomplish an evaluation of the system, the criteria for determining a success must be (1) specific and measurable and (2) representative of the capability needed rather than the capability available. In our opinion, if the military value of the program is to be judged, the criteria for measuring that value, including specific performance of the missile, should be established in advance of the tests rather than relying on subjective judgment of success afterward.

Future EFOG-M Acquisition Could Be Shortened

ACTD programs are designed to shorten the time required to obtain operating capability. But, when asked where EFOG-M would enter the acquisition process if a larger procurement is desired, the Under Secretary of Defense for Advanced Technology said that it depends upon the quality of the ACTD—it could enter at production or it could go back to the beginning of engineering and manufacturing development. However, since the ACTD is scheduled for 6 years, it appears to us that, unless engineering and manufacturing development is greatly abbreviated, entering the process at that phase would accomplish little toward shortening the acquisition process. One shortening strategy could involve conducting tests and evaluations during the limited acquisition in such a fashion to prevent duplication during a larger procurement.

For normal Army acquisition programs, development testers (Army Test and Evaluation Command) plan and conduct developmental testing and provide safety release of all systems; independent evaluators or assessors (Army Materiel Systems Analysis Activity or Test and Evaluation Command) determine the degree to which the technical parameters of the system have been achieved; and operational testers and independent operational evaluators (Army Operational Test and Evaluation Command) conduct operational tests and address the operational effectiveness and suitability of the system. However, the roles of development testers, independent evaluators, and operational testers and evaluators in the RFPI demonstration and EFOG-M tests and evaluations are not well defined at this time.

The RFPI ACTD Management Plan is endorsed by the Test and Evaluation Command but the plan does not specify the Command’s role nor the role of other independent testers in the demonstration. More detailed draft plans for conducting EFOG-M tests, conducting the demonstrations, and acquiring the EFOG-M limited quantities also do not identify the specific roles. And discussions with independent testers and evaluators and with EFOG-M management officials provided little additional definitive information about the role of the independent testers and evaluators.

According to EFOG-M management officials, the contractor has prepared a draft master test plan for the limited acquisition, and the contractor will be responsible for the tests. Project test officials have sent the plan to the independent testers and evaluators for comment, but their approval is not required. The project manager will approve the test plan, and will consider the independent comments. Project management officials said that the testers and evaluators would be invited to observe the tests, but not control them.

However, there are no formal agreements with independent testers and evaluators as to (1) their role in the testing and evaluation of EFOG-M or (2) the amount of testing and independent tester and evaluator involvement required to prevent retesting and reevaluating the system if a larger quantity is desired. All acknowledge receiving the contractor’s master test plan. However, the Army Materiel Systems Analysis Activity, for example, is only currently attempting to define its role in ACTD programs. Its representatives have participated in RFPI and EFOG-M discussions, and they plan to provide some informal evaluation. Army Test and Evaluation Command representatives have been informed they will be responsible for safety tests, and they are actively attempting to define their involvement. Operational Test and Evaluation Command officials are aware of the RFPI and EFOG-M programs, but they have not yet defined their role in the programs. They believe they will be involved at the appropriate time. One RFPI ACTD manager has begun efforts to provide coordinated evaluation for the virtual prototype evaluation If, in order to accomplish the ACTD objective, the Army initiates strategies to ensure that the ACTD reduces the time required to acquire a larger quantity of systems, we believe there should be assurances that required tests and evaluations of the system are conducted in such a fashion during the ACTD program to preclude the need to repeat the tests and evaluations to support a larger procurement.

Resources for Fielding System Beyond the ACTD Are Not Ensured

Because of the early stage of the ACTD program, the Army has not yet planned for the personnel and funds to support, operate, and maintain the EFOG-M beyond the ACTD program. In addition, the Army has not yet determined whether a deployment of the residual equipment would be cost-effective. According to Army officials, the ACTD could result in (1) leaving the EFOG-M residual equipment deployed with a combat unit but not purchasing additional systems or (2) purchasing a much larger quantity of EFOG-Ms—possibly to equip the entire early entry force. Before making decisions regarding retaining the residual deployment or a larger deployment, the Army should ensure that it has the force structure and funding needed to operate, support, and maintain EFOG-M beyond the ACTD program and that the deployment is cost-effective.

For the extended user evaluation, the EFOG-M will be assigned to a company consisting of 3 platoons with a total of 58 personnel. Each platoon will have 1 platoon leader vehicle and 4 EFOG-M fire units (12 per company), and the company will be assigned support vehicles for resupply of ammunition and fuel. The EFOG-M contractor will support and maintain the system during the period.

Training and Doctrine Command officials informed us that the company will perform its normal activities during the evaluation. For example, if the unit went to training, it would train with the EFOG-M. If the unit were deployed for a military contingency, it would deploy with the EFOG-M as a part of the force.

The Army Forces Command will provide the personnel to operate and support the systems during the user evaluation, and the RFPI program management office will fund the supporting contractor. However, Training and Doctrine Command officials informed us that funding or support beyond the 2-year extended user evaluation period has not been planned for the residual quantity or for a larger procurement. They said such plans would be premature since decisions have not been made regarding retaining the residual quantity or procuring a larger amount.

In addition, although retaining the residual quantity without a larger procurement is an option, at this time the Army has not examined the cost-effectiveness of such a deployment. For example, we found no evidence the Army has compared (1) the cost of personnel to operate the system and the cost to establish or contract for maintenance and logistics support with (2) the cost to accomplish the mission with other alternatives. An Army official said the Army plans to make these comparisons during the ACTD.

We believe the Army should ensure that such cost-effectiveness studies are performed as well as ensure that a supporting/operating force is available before making decisions regarding retaining the residual deployment. In addition, before making decisions regarding a larger deployment, DOD should ensure that the Army has the force structure and funding planned to operate, support, and maintain the larger procurement.

Recommendations

We recommend that, before deciding to either acquire more EFOG-Ms or retain the limited quantity beyond the user evaluation, the Secretary of Defense require the Army to prepare (1) a formal EFOG-M requirements document and (2) analyses comparing EFOG-M’s cost and operational effectiveness with other alternatives for satisfying the requirement, including the weapons of other services if appropriate.

We recommend that the Secretary of Defense establish measurable exit criteria regarding the most critical EFOG-M performance issues before beginning the tests, demonstrations, and evaluations.

We also recommend that the Secretary of Defense evaluate the feasibility and costs of performing the tests and evaluations to be conducted during the limited procurement in such a fashion to preclude the need to repeat them if a larger procurement is desired.

We further recommend that, before requesting appropriations to support and operate the EFOG-M equipment beyond the extended user evaluation period, the Secretary of Defense require the Army to provide evidence that such a deployment would be cost-effective. In addition, before requesting funds for a larger procurement, we recommend that the Secretary of Defense ensure that the Army has planned sufficient funding and personnel to support, operate, and maintain the larger procurement.

Agency Comments and Our Evaluation

In commenting on a draft of this report, DOD said the report contained many useful comments and observations and it partially agreed with the recommendations. However, it did not agree with the findings because it believes the report treats EFOG-M as a normal acquisition program instead of as part of the RFPI ACTD. We disagree. The report is directed toward improving DOD’s management of acquiring EFOG-M for the RFPI ACTD, demonstrating EFOG-M’s utility, and evaluating its military value.

DOD partially agreed with our draft recommendation to prepare a formal requirements document and conduct analyses comparing EFOG-M cost- and operational effectiveness with other alternatives by the end of the force-on-force demonstration. DOD stated that it would prepare a formal cost- and operational effectiveness analysis and statement of requirement if the results of the ACTD indicates that a larger quantity of EFOG-M should be acquired. However, it believed that the timing should be keyed to the transition decision. Based on DOD’s comments, we modified the recommendation to provide more flexibility in the timing of establishing requirements and conducting a cost- and operational effectiveness analysis. DOD agreed with the modified recommendation.

DOD did not agree with our draft recommendation to establish measurable exit criteria regarding the most critical EFOG-M performance issues. DOD stated that exit criteria are not appropriate for use with an ACTD. It further stated that appropriate testing would be performed to characterize performance and required levels of performance will be established at the conclusion of the ACTD. We disagree with DOD. The Army has already established exit criteria for EFOG-M and the RFPI ACTD management plan recognizes that most of the systems (including the EFOG-M) have approved exit criteria that describe the technical goals for each system. Our recommendation is directed toward making some of these technical goals more specific and measurable. We continue to believe that measurable critical levels of performance should be established before beginning the tests, demonstrations, and evaluations.

Because of a misinterpretation, DOD partially agreed with our draft recommendation to evaluate the feasibility and costs of performing sufficient tests and evaluations during the limited procurement to preclude the need to duplicate them during a larger procurement. DOD concluded that we wanted them to expand the testing program. However, our intent was to preclude the need to repeat tests to support a larger procurement. Therefore, we modified the recommendation to prevent any misunderstanding.

DOD agreed to provide evidence that the deployment of EFOG-M would be cost-effective before requesting appropriations to support and operate the EFOG-M equipment beyond the extended user evaluation period. DOD stated that the results of the RFPI ACTD would include an analysis of the cost-effectiveness of limited fielding with the inventory procured for the ACTD as well as for an expanded deployment and that any decision to procure additional units would include full consideration of funding and personnel levels required to operate and support the expanded deployment.

The DOD response and our comments are included in appendix III.

We are sending copies of the report to the Secretaries of Defense and the Army and the Director, Office of Management and Budget. Copies will be made available to others upon request.

Please contact me at (202) 512-4841 if you or your staff have any questions concerning this report. Major contributors to this report are listed in appen dix IV.

EFOG-M Exit Criteria

Engage targets not in line of sight (continued)

Sling transportable by CH-47D helicopter in a march order configuration Sling transportable by UH-60 helicopter (2 lifts)

Scope and Methodology

We obtained information regarding the purposes of the Rapid Force Projection Initiative (RFPI) Advanced Concept Technology Demonstration (ACTD) by (1) reviewing the RFPI ACTD management plan and (2) discussing the matter with the Deputy Under Secretary of Defense for Advanced Technology; the Director of Technology, Office of the Assistant Secretary of the Army for Research, Development, and Acquisition; and officials from the RFPI Program Office, U.S. Army Missile Command.

We obtained information regarding the Enhanced Fiber Optic Guided Missile (EFOG-M) system’s exit criteria by reviewing the EFOG-M Advanced Technology Plan and interviewing officials from the Non-Line of Sight Project Office (responsible for managing the EFOG-M program), Program Executive Office for Tactical Missiles. In addition, we obtained information regarding demonstration, test, and evaluation plans from discussions with RFPI and EFOG-M project officials and officials from the (1) Army Materiel Systems Analysis Activity, Aberdeen Proving Ground, Maryland; (2) Army Test and Evaluation Command, Aberdeen Proving Ground and Redstone Arsenal, Alabama; and (3) Operational Test and Evaluation Command, Alexandria, Virginia.

We also obtained information regarding EFOG-M system requirements, force structure requirements, and fielding plans from the U. S. Army Training and Doctrine Command’s System Manager for Antitank Missiles and the Dismounted Battlespace Battle Laboratory, Fort Benning, Georgia, and the Early Entry Lethality and Survivability Battle Laboratory, Fort Monroe, Virginia.

We conducted our review from September 1994 through July 1995 in accordance with generally accepted government auditing standards.

Comments From the Department of Defense

The following are GAO’s comments on the Department of Defense’s (DOD) letter dated September 15, 1995.

GAO Comments

1. The report does not focus on the EFOG-M program as a normal acquisition program. The report is directed toward improving DOD’s management of acquiring a limited number of EFOG-Ms for the RFPI ACTD. For example, we believe that the recommendation regarding the formal agreed-upon requirement is appropriate because requirements and/or support for three EFOG-M predecessors have disappeared after considerable effort and expenditure of funds. 2. The report does not ignore the primary thrust of ACTDs. The draft recommendation was directed at establishing an EFOG-M requirement by the end of the force-on-force demonstration in mid-1998, or nearly 4 years into the ACTD program, not at its inception. Our intent was to ensure that the Army validated its requirement for EFOG-M before deciding whether to either acquire a larger quantity of EFOG-Ms or retain the residual ACTD quantity after the 2-year evaluation. Based on DOD’s comments, we modified our recommendation to permit more flexibility in the timing and even greater user evaluation. 3. We disagree that requirements, exit criteria, and cost-effectiveness analyses must be products of an ACTD. We addressed the importance of exit criteria in the agency comments and evaluation section of the report and the importance of requirements in comment 1. A cost-effectiveness analysis can be performed at any time, not at just the end of the ACTD. 4. The report does not recommend force structure planning at this time; however, it does recommend that such planning occur before a decision is made to either acquire a larger quantity or retain the limited quantity after the 2-year evaluation. DOD agreed with the recommendation. 5. The report neither addresses changes in threat nor prohibits exploring EFOG-M’s effectiveness under early entry conditions. However, as modified, it recommends an agreed-upon requirement before making a decision to either procure a larger quantity or retain the limited quantity. 6. We do not judge EFOG-M because of its history; but, at the same time, we believe that history should be used to assist in making good management decisions. 7. Our review was not designed to evaluate the ACTD process, but rather to examine selected aspects of the acquisition of the Army’s EFOG-M system. Therefore, we cannot comment on the benefits of ACTD programs. 8. Regarding critical decisions, we modified our recommendations to permit more flexibility in establishing the requirement; however, we still believe that a requirement should be established before decisions are made regarding a larger procurement or retaining a limited quantity. We also believe that specific measurable exit criteria, or standards for performance, should be established before tests, evaluations, and demonstrations. 9. DOD’s comments and our evaluation are included in the body of the report.

Major Contributors to This Report

National Security and International Affairs Division, Washington, D.C.

Atlanta Field Office

Thomas W. Gilliam, Evaluator-in-Charge Erin B. Baker, Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (301) 258-4097 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Address Correction Requested