Defense Acquisition: Comanche Program Objectives Need to Be Revised to More Achievable Levels
Released: 2001-06-13T00:00:00

Letter

Background

The Comanche helicopter program began in 1983 to provide a family of high technology, low-cost aircraft that would replace the Army’s light helicopter fleet, which includes the AH-1 Cobra, OH-58 Kiowa, OH-6 Cayuse, and the UH-1 Iroquois (Huey). The Army subsequently decided to develop only a single Comanche aircraft capable of conducting either armed reconnaissance or attack missions. The Army intends for the Comanche to be part of its future or “objective” force.

The Comanche is designed to have improved speed, agility, aircrew visibility, reliability, availability, and maintainability over current reconnaissance and attack helicopters. The helicopter is also designed for low observability (stealth) and is expected to be capable of deploying over long ranges without refueling. Lastly, the Comanche is being designed to provide enemy information to force commanders at all levels.

Critical to achieving the Comanche’s desired capabilities is the successful development and integration of advanced technologies, especially for the mission equipment package. The mission equipment package includes an integrated communication system, piloting system, target acquisition system, navigation system, helmet-mounted display, survivability and early warning equipment, mission computer, and weapon management system.

The Comanche program started in 1983 and is currently projected to continue through fiscal year 2028. A timeline of the Comanche’s acquisition history and schedule is provided below.

The Comanche Program Continues to Experience Cost Increases, Schedule Delays, and Performance Shortfalls

Since our August 1999 review, the Comanche program’s estimated cost has increased significantly—from $43.3 billion to $48.1 billion—and costs are expected to increase further. In addition, the Comanche continues to experience scheduling delays and performance risks. These problems are due to a range of factors, such as understated acquisition program cost estimates; ambitious flight test schedules with substantial concurrency in test events; delays in another DOD program which had been counted on to develop a critical component of the aircraft; inadequate facilities to fully test and integrate system hardware and software; and considerable growth in aircraft weight. The Army has not updated the Comanche’s cost or schedule estimates since April 2000 and does not plan such an update until its in-progress program review in January 2003.

Cost Estimate Increases

The Comanche program’s latest cost estimate, in April 2000, shows estimated costs have increased by almost $4.8 billion—from $43.3 billion to $48.1 billion—since our last report. Table 2 identifies where the cost estimate has changed.

The $75.3 million increase in research, development, testing, and evaluation resulted from added testing for the Comanche program. During the Milestone II decision process, the Defense Acquisition Executive directed that the Comanche testing program be expanded by adding more testing to fully demonstrate the aircraft’s reliability before completion of its engineering and manufacturing development phase.

The $4.777 billion increase in estimated production cost was to address DOD concerns about the long-term affordability and stability of the Comanche program. Specifically, DOD directed the Army to add 10 percent to Comanche’s production unit cost estimate in order to ensure that annual planned procurement funding would be sufficient to cover planned procurement quantities. To reduce the annual funding increase resulting from this directive, the Army reduced Comanche’s peak annual production rate from 72 aircraft per year to 62 per year, which extended the planned delivery schedule by 3 years.

The $67.5 million reduction in estimated military construction costs reflects changes in anticipated needs for operating and maintenance facilities.

Additional Funds Recently Provided by DOD

In January 2001, DOD added about $504 million in funding to the Comanche program over the next few years. About $84 million of the additional funds are earmarked for research, development, test, and evaluation, and the remaining $420 million for production. These additional funds have not yet been reflected in the program’s official cost estimates. The program office plans to use the additional development funding to at least partially address what had been unfunded requirements in three areas considered to be high risk: (1) developing and integrating the mission equipment package; (2) developing the technology to detect and isolate equipment problems (automatic fault isolation); and (3) developing and integrating satellite communication capabilities. The section on performance discusses these areas in more detail.

Additional Cost Growth Likely

The Comanche’s most recent cost estimate was made in April 2000, when DOD approved the program for entry into the engineering and manufacturing development phase. At that time, DOD’s Cost Analysis Improvement Group estimated that the Comanche program would need an additional $180 million for its engineering and manufacturing development phase. However, the higher costs estimated by the Cost Analysis Group were not included in the cost estimate when the program office established a new baseline for the Comanche program in April 2000. The Comanche program is scheduled for an in-progress program review in January 2003 to review, among other things, its cost estimate. DOD believes that this January 2003 review, along with other major program reviews and oversight processes will permit successful management of program risks. The Deputy Program Manager acknowledged that the Army’s cost estimate for the Comanche may need to be revised at this point.

The Comanche program office also maintains a list of unfunded requirements. The additional funds recently added to the program have reduced these funding requirements, but the revised list still has unfunded requirements in the amount of $68 million. The program office acknowledges that, unless additional funds are obtained, some yet-to-be- determined program performance requirements could be impacted.

Highly Compressed and Concurrent Schedule May Lead To Additional Schedule Slippage and Higher Costs

We have reported that when development work and low-rate initial production are done concurrently, significant schedule delays that cause cost increases and other problems are not uncommon in early production. Also, production processes are often not able to consistently yield output of high quality when full-rate production begins. DOD’s guidance also states that programs in which development work and low-rate initial production are done concurrently typically have a higher risk of production items having to be retrofitted to make them work properly and of system design not being thoroughly tested. We have also reported that the discovery of problems in testing conducted late in development is a fairly common occurrence on DOD programs, as is the attendant “late cycle churn”, that is, the unanticipated effort that must be invested to overcome such problems. Further, these problems could be exacerbated if the program plans to produce a significant number of systems during the low-rate initial production period, before design and testing are completed.

In August 1999, we reported that the Army would experience a 19-month delay in testing because the first pre-production aircraft for testing were expected to be delivered 19 months later than planned. We noted that, by retaining the December 2006 initial operating capability date, the delay in acquiring test aircraft would compress the majority of Comanche’s flight- test schedule into the last 3 years of development. The compressed flight- test schedule would, in turn, shorten the available time for completing all test events and taking necessary corrective actions before the full-rate production decision.

Since our last report, the first pre-production aircraft to be used for development testing is now scheduled for delivery in January 2004, adding an additional 3-month delay to the 19-month delay we reported in August 1999. As shown in figure 1 below, the delivery of pre-production Comanche aircraft has been delayed and, because the Army has retained the December 2006 full-rate production decision, the time available for testing, assessing, and correcting problems has been reduced.

Many critical test events are now scheduled late in the development stages—during the low-rate initial production phase of the program—and, as shown in figure 2, many developmental and operational test events are scheduled to be conducted concurrently.

The combination of compressing the development schedule and undertaking developmental and operational testing activities concurrently leaves the Army with little room to accommodate any delays that may result from assessing, correcting, and retesting problems found during testing. In Comanche’s case, several critical subsystems—to be included in the mission equipment package—may not be available until the development flight-testing is well underway. These subsystems are very complex, state-of-the-art systems that have not been demonstrated on a helicopter platform like Comanche. As testing proceeds, any problems found will need to be analyzed, fixed, and retested. However, with the ambitious test schedule, there may not be time available between test events to correct problems and prepare properly for the next event.

Further, the Army’s schedule for developing and testing software for the Comanche may not be completed prior to the full-rate production decision. The contractor is experiencing a shortage of software engineers available to work on the Comanche contract. In addition, only about 1.4 million of the projected 1.9 million lines of computer code for the Comanche’s mission equipment package will be completed by the time the package is to be tested on the initial pre-production aircraft. Additional segments of computer code for the mission equipment package will be introduced as developmental testing is underway. At this point, it is uncertain if all of the computer code for the full mission equipment package will be completed by the time the Army is scheduled to make a full-rate production decision for Comanche in late 2006.

Finally, the Army plans to use pre-production aircraft that it considers production-representative for operational flight-testing. Before this testing is complete, the Army plans to begin producing a total of 84 low-rate initial production aircraft. These aircraft are to be used to equip Army helicopter units and to ramp-up production. To produce that many aircraft during low-rate initial production, the Army will have to ramp-up its production capabilities rapidly and at a time when the aircraft design is still evolving as new subsystems are introduced and test results are evaluated. Specifically, the Army does not plan to freeze Comanche’s design configuration until January 2006, or six months after the low-rate initial production decision point. Making design changes and retrofits to a large number of aircraft already produced could be costly.

Performance Degradation Is Still A Risk

In our last report, we noted that the Army was making modifications to the Comanche that would adversely impact some of the Comanche’s planned performance capabilities; for example, some modifications have added weight and drag to the aircraft. While their exact impacts are still unknown, these changes increase the risk that the Comanche’s planned performance goals may not be achieved. The Comanche continues to have several areas of high technical risk that jeopardize the achievement of several critical performance requirements.

Weight Growth Could Adversely Impact Vertical Rate of Climb

The Comanche’s ability to climb at a rate of 500 feet per minute is a key performance requirement for the aircraft. Since we last reported on the Comanche program, the aircraft’s projected empty weight has increased by 653 pounds—from 8,822 pounds to 9,475 pounds. At the current projected design weight of 9,475 pounds, the Comanche program office has acknowledged that the helicopter cannot achieve the required vertical rate of climb of 500 feet per minute without increasing the horsepower of the current engine. Consequently, the program office has assessed its achievement of the weight requirement as high risk.

The Army offered its prime contractor for Comanche’s development, Boeing-Sirkosky, an award fee of $1.4 million to reduce its projected weight to 9,250 pounds. However, the contractor did not achieve the first iteration of weight reduction in December 2000. The program office is considering increasing the incentive fee to $5 million for the contractor to reduce the projected weight to 9,300 pounds in December 2001.

The program office believes that it can achieve its vertical rate of climb, even with the increase in Comanche’s weight, by increasing the horsepower of Comanche’s T-801 engine from its current horsepower rating of 1131 to 1201. The program office estimates that the increase in the engine’s power can be obtained at a cost of about $13 million, and this approach will be less costly than other weight reduction efforts. However, an increase in engine performance could adversely affect the expected life of the engine since it will have to perform about 47 degrees hotter than is normally required. According to the program office, this increased performance may not have an appreciable impact on the engine’s life.

Scheduled Integration of Critical Mission Equipment Package Still High Risk

As noted earlier in this report, the successful development and integration of the mission equipment package is critical to meeting Comanche’s performance requirements. This package includes an integrated communications system, piloting system, target acquisition system, navigation system, helmet-mounted display, survivability and early warning equipment, mission computer, and weapons management system. The program office has assessed the achievement of this portion of its development effort as high risk.

In order to reduce this risk, the Army had planned to develop a mobile integration laboratory, called a hotbench, which simulates Comanche’s hardware, to integrate and test mission equipment package software before installing the software on the flight test aircraft. However, due to a shortage of development funds, the Army had listed the hotbench as an unfunded requirement. DOD recently provided additional funding to the Comanche program, which the program office plans to use to fully fund the hotbench. Despite the additional funding for the hotbench, the program office continues to acknowledge that integration of Comanche’s mission equipment package as an area of high technical risk.

Fully Capable On-Board Fault Isolation Requirement May Not Be Achieved Until 2 Years After Initial Fielding

A critical Comanche requirement is an on-board fault detection system that can rapidly and accurately provide information about equipment problems. With an on-board fault isolation system, the Army would be able to promptly identify and correct potential problems in advance, according to the Comanche’s operational requirements document. Additionally, without the system, the time and cost of maintaining the aircraft will likely increase. According to the Army, this system needs to be 75 to 95 percent accurate—75 percent for mechanical and electrical equipment and 95 percent for avionics and electronics equipment. The Comanche program office has concluded that this requirement will be difficult to achieve within the current cost, weight, and packaging constraints, and does not expect to achieve a mature fault detection and fault isolation capability until 2 years after initial fielding. According to the program office, this system depends, in part, on a database built on flight data and equipment failure experience; therefore, the system becomes better with additional flight hours. The program office anticipates that after 2 years of flight testing, the system should meet the full level of predictability required. Although some of the recently provided development funding will be used by the Army in this area, the Comanche program has identified an additional $20 million unfunded requirement for the fault isolation capability.

Critical Comanche Beyond-Line-of-Sight Communications Requirements May Not Be Achievable

In some battle situations, the Army plans to use Comanche as a deep reconnaissance aircraft to provide critical information and situational awareness to joint forces. Satellite communication technology is necessary for the helicopter to be able to achieve the “beyond-line-of- sight” capability needed to carry out this function, according to the Comanche operational requirement document. To meet this need, the Army was planning to rely on satellite communication technology being developed and miniaturized as part of the Joint Strike Fighter program, which is being developed jointly by the Air Force, Navy, and Marines. However, in May 2000, Congress provided that the Joint Strike Fighter program could not enter into the engineering and manufacturing development phase until the Secretary of Defense certified the technological maturity of its critical technologies. This has delayed the Joint Strike Fighter program’s schedule for beginning its engineering and manufacturing development phase.

When assessing the risk of its dependency on the Joint Strike Fighter’s program, the Comanche program office concluded that the helicopters in low-rate initial production would not have the beyond-line-of-sight communication capability if the Joint Strike Fighter program was delayed. The program office now believes that it must develop its own satellite communication capability. However, the development schedule remains high-risk for the timely inclusion of this capability on the initially fielded Comanche helicopters. The Army has estimated that it will require about $58 million to develop this capability and plans to fully fund this effort with additional funds recently provided by DOD.

The Army Plans to Begin Comanche Production Despite Numerous Uncertainties

Our work on best practices has found that product development in successful commercial firms is a clearly defined undertaking for which firms insist on having in hand the technology that meets customers’ needs before starting. The firms demand—and receive—specific knowledge about a new product before production begins. And, they do not go forward unless a strong business case on which the program was originally justified continues to hold true. Such a knowledge-based process is essential to commercial firms getting better cost, schedule, and performance outcomes. It enables decision-makers to be reasonably certain about critical facets of the product under development when they need it.

At the point of going into production, successful firms will already know that (1) technologies match customer requirements, that is, they can fit onto a product and function as expected, (2) the product’s design meets performance requirements, and (3) the product can be produced within cost, schedule, and quality targets. The Comanche program does not yet have this knowledge and is not likely to have this knowledge when it plans to begin low-rate initial production in June 2005.

First, the Army does not yet know and it will not know until well after its low-rate initial production decision whether certain technologies being developed will fit on the helicopter and function as expected. Our reporton incorporating new technologies into programs indicated that demonstrating a high level of maturity before new technologies are incorporated into product development programs puts those programs into a better position to succeed. Further, technologies that were included in a product development before they were mature later contributed to cost increases and schedule delays to those products. While the Comanche program has made progress in the technology readiness level of its critical components, integration of those components into subsystems, such as the mission equipment package, and the helicopter as a whole remains high-risk. In addition, the integration, development, and configuration of key satellite communication technology for inclusion in the integrated communication, navigation, and identification avionics has also been assessed as high risk. Finally, some of the technologies have not been developed to meet Comanche’s specific configuration requirements. For instance, the Comanche’s second generation forward-looking infrared sensor has been tested and proven on the Black Hawk helicopter by the Army’s night vision laboratory but not on the Comanche itself. Such testing needs to be done to ensure that the system can work together with other unique systems being developed for the Comanche, including the piloting, target acquisition, and navigation systems, which work as one unit. Comanche’s contractor has maintained that its mission equipment package technology is challenging because some key components have not been developed and configured in the required manner for the helicopter’s intended mission.

Second, as discussed earlier, the Army does not yet know and may not know until well after the start of low-rate initial production, whether performance requirements can be met—including vertical rate of climb, on-board fault isolation, and beyond-line-of-sight communication requirements. The Army plans to conduct a limited user test before it begins low-rate initial production but it is a rudimentary test and not a complete operational test that fully demonstrates the aircraft’s capabilities. By compressing many key events late in the development schedule and conducting developmental and operational testing activities concurrently, the Army is running the risk of not fully demonstrating many of its critical capabilities before its planned full-rate production decision. Under current plans, for example, the Army will not complete a full demonstration of its integrated mission equipment package until December 2006—over a full year after its low-rate initial production decision and within the same month that the Army plans to make its decision on Comanche full-rate production.

Third, as noted earlier, it is still uncertain whether the Comanche can be developed within cost and scheduling estimates. Although additional costs have been identified for the Comanche since it was last restructured, the full development cost will not be known until critical technology is fully developed, integrated, and tested. This will not occur until well after a low- rate initial production decision has been made in June 2005. The program office believes that it will know the cost of the initial production aircraft, which will have been negotiated prior to the low-rate initial production decision. However, at that time, the program office and the contractor will have limited experience and data relative to producing the fully developed Comanche helicopter. Until more experience and data is available, there is not a high level of confidence in the Army’s production cost estimate.

Further, the Director of Operational Test and Evaluation in assessing the results of the Comanche milestone II test data indicated that it is highly unlikely that the Army can deliver the expected system performance within the current budget and schedule. The Director's assessment revealed that, without an operational assessment of an integrated system, it is difficult to predict with any degree of confidence whether (1) the individual subsystems can be successfully integrated, (2) the subsystems will function properly in an operational environment, or (3) the subsystems, in concert, will provide the anticipated benefits in operational performance.

Conclusion

In 1999, we reported that the Army started the Comanche’s program development too early in terms of technology readiness, which is contrary to best commercial practices. Further, in approving the program for engineering and manufacturing development, the Army accelerated the development of some components, reduced the number of test aircraft, and compressed the test schedule. Two years later, the program is confronted with rising development costs, a compressed development schedule, and several major areas of high technical risk. The Army plans to proceed to low-rate initial production in June 2005 and full-rate production in December 2006, both of which could be well in advance of attaining sufficient knowledge of the helicopter’s technical maturity, demonstrated performance capabilities, and production costs. With such a scenario, the potential for adverse program outcomes is high—higher than expected costs, longer than expected schedules, and uncertain performance. DOD and Army officials acknowledge that the current program cost and schedule objectives are not achievable and should be changed to reflect more realistic objectives, but they believe that the planned January 2003 review for the Comanche program is the appropriate time to address such changes. Such a delay in revising the program’s cost and schedule estimate limits the visibility and knowledge that Army and DOD management as well as the Congress needs to (1) provide program oversight and direction; (2) make effective cost, schedule, and performance trade-off decisions; and (3) assess affordability and annual funding requirements.

Recommendations for Executive Action

To improve management oversight and direction and achieve more favorable program outcomes, this report recommends that the Secretary of the Army reassess the program’s cost, schedule, and performance objectives, and revise those objectives to more achievable levels prior to submitting its next fiscal year budget.

Agency Comments and Our Evaluation

In commenting on a draft of this report, DOD partially concurred with our recommendation. DOD noted that it agrees with some of our concerns and recognizes there are risks in the currently planned Comanche engineering and manufacturing development program. DOD noted that these risks were understood during the Comanche milestone II review. At that time, the Defense Acquisition Executive directed that the program proceed as planned, but that interim decision reviews be conducted in January 2003 and June 2005 to review program status. DOD stated that these reviews, along with other major program review and oversight processes, will permit successful management of program risks. Nevertheless, DOD stated that it is currently examining whether any of Comanche's requirements should be deferred, in order to reduce the risk of not meeting cost and schedule objectives. DOD's examination of Comanche's requirements is consistent with our recommendation. We continue to believe that DOD should report on the results of this examination and any revisions to the program’s objectives to the defense committees of the Congress with its next budget request.

DOD disagreed with a reference to our previous Comanche report stating that current program risks are caused by, among other things, the program being allowed to enter engineering and manufacturing development prior to maturation of key technologies. DOD maintains that the Comanche program successfully demonstrated its exit criteria prior to entering engineering and manufacturing development. However, the Comanche program’s demonstration of its exit criteria was not sufficient as a basis to move forward in the acquisition process. For example, the exit criteria did not require that the technologies used in Comanche be at or above specific levels of demonstrated readiness. As we previously reported, the Army's own assessments clearly indicated that several key areas of technology were not at those levels called for in commercial best practices guidelines.

DOD's comments are reprinted in appendix I. Other comments provided by DOD were incorporated in the report as appropriate.

Scope and Methodology

To evaluate changes in the Comanche’s status with regard to cost, schedule, and performance and assess whether the Army has the certainty it needs to proceed with beginning production, we examined and compared program schedules, pertinent cost documents, and acquisition strategies, and discussed potential changes and causative factors with cognizant Comanche program officials. We analyzed flight-test plans, schedules, and reports and discussed significant issues with program officials. We reviewed program documents related to risk and analyzed program risks and development problems by comparing them with various test schedules and plans. To assess performance capabilities before beginning with production, we analyzed required and projected performance and compared it with the Comanche’s operational requirements. We relied on previous GAO best practices work to examine Comanche’s technological readiness levels for key program technologies.

Our analyses focused on the impact of Comanche’s cost, schedule, and performance on the Army’s ability to field a Comanche helicopter that would meet its requirements and incorporate technological upgrades in its helicopter fleet.

In performing our work, we obtained pertinent program documents and interviewed officials from the offices of the Secretary of Defense and the Army, Washington, D.C.; the Program Executive Office-Aviation and Comanche Program Office, Redstone Arsenal, Alabama; the U.S. Army Training and Doctrine Command, Fort Rucker, Alabama; the Comanche Joint Project Office, Huntsville, Alabama; and the Aviation Test and Evaluation Command, Alexandria, Virginia. We conducted our review from September 2000 through March 2001 in accordance with generally accepted government auditing standards.

As agreed with your office, unless you publicly announce the contents of this report earlier, we will not distribute this report until 5 days from its date. At that time, we will send copies of this report to the Honorable Donald H. Rumsfeld, Secretary of Defense; the Honorable Thomas White, Secretary of the Army; Director, Office of Management and Budget; and other interested congressional committees and parties. We will also make copies available to others upon request.

If you have any questions regarding this report, please contact me on (202) 512-4530. GAO contacts and major contributors to this report are listed in appendix II.

Appendix I: Comments From the Department of Defense

Appendix II: GAO Contacts and Acknowledgments

GAO Contacts

Acknowledgments

In addition to those named above, Leon S. Gill, Wendy Smythe, Gary Middleton, and Cristina Chaplain made key contributions to this report.