Defense Acquisitions: Steps to Improve the Crusader Program's Investment Decisions
Published: 2002-02-25T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Army wants an artillery system with greater firepower, range, and mobility than its current self-propelled howitzer. In 1994, the Army began to develop the Crusader, an advanced artillery system consisting of a self-propelled 155-millimeter howitzer and a resupply vehicle. The Department of Defense (DOD) will decide next year whether the Crusader program should enter its system development and demonstration stage, which will require the commitment of major resources.
section_title: What GAO Found
paragraphs: GAO found that the Crusader program has made considerable progress in developing key technologies and reducing its size and weight. However, more progress and knowledge is needed to minimize the risk of cost overruns, schedule delays, and performance shortfalls. The Crusader program will likely enter product development with most of its critical technologies less mature than best practices recommend. Most of the Crusader's critical technologies have been demonstrated in a relevant environment but not in the more demanding operational environment. Although the Army is reducing the Crusader's weight so that two vehicles can be deployed on a C-17 aircraft, the deployability advantage gained does not appear significant. The reduction in the Crusader system's weight would only decrease the number of C-17 flights needed to transport two complete systems and support equipment from five to four flights. A lighter system offers several other benefits, and knowing the magnitude of the deployability advantage of reduced weight would allow the Army to make better decisions on trade offs. An apparent overlap exists between the Crusader's and the Future Combat Systems' capabilities and schedules. The Army expects the Future Combat Systems to meet the same artillery missions as the Crusader and eventually replace it. The current schedules for initial fielding of the Future Combat Systems and the Crusader system occur in the same year, 2008. The extent of this apparent overlap depends more on the Future Combat Systems than the Crusader because less is known about the Future Combat Systems' technologies.

section_title: Letter
paragraphs: To address future threats, the Army has identified a requirement for an  artillery system that has greater firepower, range, and mobility than its  current self-propelled howitzer—the Paladin. Operation Desert Storm  demonstrated that current howitzers were unable to keep up with our  tanks and fighting vehicles. In 1994, the Army began to develop the  Crusader, an advanced artillery system consisting of a self-propelled 155- millimeter howitzer and a resupply vehicle. The Army’s total acquisition  cost in the Crusader program is projected to be about $11 billion. In 2000,  the Army changed its requirements and restructured the Crusader program  to make the system lighter and more deployable. This change was in  response to the Army’s planned transformation to a future force, which will  also be lighter and more deployable. The Army expects to use the Crusader  until it is eventually replaced by the main component of the future force,  known as the Future Combat Systems.
In April 2003, Department of Defense (DOD) will decide whether the  Crusader program should enter its system development and  demonstration—or product development—stage, which will require the  commitment of major resources to develop and design the Crusader system  and to demonstrate its integration, interoperability, and utility. The  opportunity to take actions that can put the program in a better position to  succeed and, thus, minimize future cost and schedule increases is now,  before the start of product development. As the Army approaches that  decision point, we examined three major aspects of the program: (1) the  progress in developing Crusader’s technology and software, (2) the  Crusader’s requirement for improved deployability, and (3) the Army’s  timetables for developing the Crusader and the Future Combat Systems.
subsections: section_title: Background
paragraphs: The Army plans to invest about $11 billion developing and procuring the  Crusader, an automated, next generation field artillery system. To date, the  program has spent about $1.7 billion in development costs. It plans to  procure 482 Crusader systems—each system consisting of a self-propelled  155-millimeter howitzer and a resupply vehicle. The Army is developing 2  different resupply vehicles—1 with tracks and 1 with wheels—and plans to  procure 241 of each type. The purpose of the Crusader system is to  overcome threats from enemy artillery and reconnaissance or surveillance  systems as well as have the mobility needed to keep up with Army tanks  and fighting vehicles. Figure 1 shows the planned Crusader howitzer, figure  2 the planned tracked resupply vehicle, and figure 3 the planned wheeled  resupply vehicle.
The Army restructured the Crusader program in January 2000 to align  Crusader’s design with the Army’s transformation to a lighter force. The  Army’s transformation will affect all aspects of Army organization, training,  doctrine, leadership, and strategic plans as well as the types of equipment  and technology the Army acquires. The Army expects the transformation to  be at least a 30-year process and has not estimated its full cost. The  centerpiece of the lighter, more deployable future force is the Future  Combat Systems. The Future Combat Systems concept is a system of  ground and air, manned and unmanned weapon systems, each under 20  tons that is planned to replace most, if not all, of the Army’s ground combat  systems without a loss in lethality and survivability. Artillery systems are  among those to be replaced.
The Army expects the Crusader system to fill the existing gap in artillery  capabilities until it is replaced by the Future Combat Systems. In keeping  with the transformation philosophy of lightweight vehicles and ease of  deployability, the Army is redesigning Crusader to make it lighter and more  deployable, with the goal of reducing the weight of the self-propelled  howitzer and tracked resupply vehicle from about 60 tons to about 40 tons  each. Program officials said that a lighter system would enhance  operational flexibility in employing Crusader in support of any operation.
The Crusader is currently in the program definition and risk reduction  phase of its development program. In April 2003, the program is scheduled  for a milestone B review to determine whether it is ready to enter its  system development and demonstration phase. Milestone B is the point at  which DOD decides whether to commit major resources to develop and  design the system and to demonstrate its integration, interoperability, and  utility. The milestone marks the start of the program’s product  development. The Army plans to deliver the first full Crusader prototype  system in October 2004, followed by a low-rate initial production decision  in February 2006, and initial system fielding in April 2008.
section_title: Critical Technologies Need Additional Maturity to Better Assure Low-risk Product Development
paragraphs: Based on current Army plans, the Army will begin the Crusader’s product  development in April 2003 but before maturing critical Crusader  technologies to a level considered low risk relative to best practices. These  risks relate less to whether these technologies can be matured, but more to  how much time and cost it will take to mature them. If, after starting  product development, the Crusader technologies do not mature on  schedule and instead cause delays, the Army may spend more and take  longer to develop, produce, and field the Crusader system. Crusader  performance goals may also be at risk. On the other hand, the Army has  made improvements to the management of the Crusader software  development process.
subsections: section_title: Assessing Technology Readiness Provides Opportunities to Improve Outcomes
paragraphs: The maturity of a program’s technologies at the start of product  development is a good predictor of that program’s future performance. Our  past reviews of programs incorporating technologies into new products  and weapon systems showed that they were more likely to meet product objectives when the technologies were matured before product  development started. For example, the Ford Motor Company’s practice of  demonstrating new technologies in driving conditions before they are  included in a new product is essential to ensuring that the new product can  be developed on time and within budget. Similarly, we have found that the  early demonstration of propulsion and water-planing technologies,  essential to the performance of the Marine Corps’ Advance Amphibious  Assault Vehicle, has been instrumental to that program’s staying within  15 percent of cost and schedule estimates.
Conversely, cost, schedule, and performance problems were more likely to  occur when programs started with technologies at lower readiness levels.For example, the enabling technologies for the Army’s Brilliant Anti-Armor  Submunition program were very immature at the start of the program, and  their delays became major contributors to the program’s subsequent  88-percent cost growth and 62-percent schedule slippage. Separating  technology development from product development into two distinct  program phases is a best practice of both successful commercial and  defense programs. This entails demonstrating all critical technologies at  the component or subsystem level in an operational environment during  technology development, prior to committing major funding to product  development. Under this practice, the critical technologies would be  demonstrated in component or subsystem prototypes that are nearly the  right size, weight, and configuration needed for the intended product. Such  demonstrations need not require a full system prototype of a Crusader  vehicle, but can be done using surrogate vehicles.
Technology readiness levels (TRL) are a good way to gauge the maturity of  technologies. TRLs were pioneered by the National Aeronautics and Space  Administration to determine the readiness of technologies to be  incorporated into products such as weapon systems. Readiness levels are  measured along a scale of one to nine, starting with paper studies of the  basic concept, proceeding with laboratory demonstrations, and ending  with a technology that has proven itself on the intended product. TRLs are  based on actual demonstrations of how well specific technologies perform  in the intended application. For example, a technology that has been  demonstrated in an operational environment using subsystem prototype  hardware (such as a complete cannon system) that is at or near the final  system design would be rated as a TRL 7. The individual TRL descriptions  can be found in appendix I.
DOD has agreed that technology readiness assessments are important and  necessary in assisting officials who decide when and where to insert new  technologies into weapon system programs. In January 2001, DOD issued a  new acquisition instruction that redefined the phases in the defense  acquisition cycle and emphasized the role of technology development in  the acquisition process. Under the instruction, programs use the concept  and technology development phase, which precedes the system  development and demonstration phase, for developing components and  subsystems that must be demonstrated before integration into the system.  The first portion of system development and demonstration phase is  dedicated to integrating the components and subsystems into the system.  The instruction states that DOD prefers that technology be demonstrated in  an operational environment but must be demonstrated in a relevant  environment to be considered mature enough for product development in  the system development and demonstration phase. According to the TRL  descriptions, technology demonstrated in an operational environment is  TRL 7 and technology demonstrated in a relevant environment is TRL 6.
Maturing technology from a TRL 6 to a TRL 7 represents a major step up in  maturity. A technology at the TRL 6 maturity level needs only to be  demonstrated as a subsystem prototype or model in a laboratory or  simulated operational environment. A technology at the TRL 7 maturity  level must be demonstrated as a subsystem prototype at or near the size of  the required subsystem outside the laboratory in an actual operational  environment. For example, operating a prototype engine on a laboratory  test stand that simulates the effects of the vehicle’s weight on the engine  would be a TRL 6 level demonstration while operating an engine in a  surrogate vehicle or actual prototype that weighed 50 tons, on roads and  cross country, would be a TRL 7 demonstration.
In June 2001, DOD issued a new acquisition regulation. It stated that  technology maturity is a principal element of program risk and directed  technology readiness assessments for critical technologies sufficiently  prior to selected milestone decision points—including milestone B--to  provide useful technology maturity information to the acquisition review  process. Although the new regulation recognizes that TRLs enable  consistent, uniform discussions of technical maturity across different types  of technologies and provides the definitions of TRLs used in this report, it  permits the use of TRLs or “some equivalent assessment” when performing  a technology readiness assessment.
section_title: Technology Maturation Will Continue into Product Development
paragraphs: In June 2001, Crusader program office engineers and we assessed the  maturity of 16 critical Crusader technologies using TRLs. This joint  assessment determined that 10 of the 16 critical Crusader technologies  were below TRL 7. Since the Crusader program is not scheduled to commit  to product development until April 2003, the Army still has time to mature  the 10 critical technologies to a TRL 7 level—demonstrate them in a  component or subsystem prototype in an operational environment.  However, the Army’s Crusader plans will result in 10 of the critical  Crusader technologies remaining below TRL 7 at the milestone B decision  and in technology development continuing into the product development  phase. As a result, the Crusader program would not reach the low levels of  risk that best practices show is needed for meeting product development  cost and schedule commitments. Table 1 shows the results of our joint  technology readiness assessment.
As shown in table 1, if technology develops as planned, eight critical  technologies will be at a TRL 6 level of maturity and two will be at a TRL 5  level of maturity at milestone B. While some technologies may embody  some risk in meeting requirements, for the most part, the risk in the  Crusader technologies involves the amount of time and effort needed to  reach maturity. The planned technology maturity levels for the Crusader  program at milestone B increase the probability that technical problems, if  they occur, will need to be resolved in the higher cost environment of  system development and demonstration. Confining delays in maturing  technology to a time prior to the start of product development—in an  environment where small teams of technologists work in laboratories and  are dedicated to perfecting the technology—is critical to saving time and  money. Conversely, if delays occur in product development when a large  engineering force is in place to design and manufacture the product, delays  would be much more costly. In fact, industry experts estimate that a delay  during product development costs several times more than a similar delay  that occurs before product development.
Under the current Crusader acquisition plans, the critical technologies  would be demonstrated in two steps after milestone B. Program officials  are planning to demonstrate mobility component technologies first and  then the remaining critical technologies. They recognize a risk in  integrating the Crusader’s mobility components—track, suspension,  engine, and transmission—and plan to produce a mobility test rig to  demonstrate that integration and to start accumulating reliability data on  the mobility components. The mobility test rig would have the additional  advantage of demonstrating the maturity of those technologies in an  operational environment. The contractor is scheduled to deliver the  mobility test rig in December 2003. The test rig would later be rebuilt as a  Crusader prototype.
The remaining critical technologies would not be demonstrated until after  the contractor delivers the Crusader prototypes. The first Crusader system  prototype is scheduled for delivery in October 2004 and is to enter testing  the same month. Other prototypes would enter testing as they are  delivered. The Army plans to award contracts for low-rate initial  production long-lead items in March 2005—less than a fourth of the way  through the prototype-testing schedule. This leaves little time in the  Crusader’s projected system development and demonstration schedule for  solving unanticipated problems before the Army awards contracts for long- lead production items.
The Army’s approach to readying the Crusader for milestone B is to  demonstrate progress toward achieving five of the system’s requirements,  two of which are key performance parameters—the cannon rate of fire  and the ability to resupply the self-propelled howitzer. For example, a  Crusader key performance parameter is that the Crusader cannon be able  to fire 10 to 12 rounds per minute; however, the program only needs to  demonstrate the ability to fire 6 rounds per minute before milestone B. The  demonstrations, called exit criteria, were approved by both the Army and  DOD. Among the demonstrations required by the exit criteria, only the  cannon system is expected to be demonstrated in an operational  environment; the other critical technologies are expected to be  demonstrated in a laboratory environment.
Moreover, like many other DOD programs, the Crusader program is using  risk management plans and engineering judgment, without the benefit of  TRLs, to assess technological maturity and mitigate program risk. Risk  management plans and engineering judgment are necessary to manage risk  in any major development effort like the Crusader. However, we have found  in our reviews that without an underpinning, such as TRLs, that allows  transparency into program decisions, significant technical unknowns may  be judged acceptable risks because a plan exists for resolving them. For  example, we recently reported that while DOD judged the technical risks  facing the Joint Strike Fighter as acceptable for starting product  development, an analysis of TRLs showed that eight critical technologies  were below TRL 7, with six technologies at TRL 4 or 5. When problems are  encountered in resolving these unknowns, programs often fail to meet  promised outcomes, as noted above with the Brilliant Anti-Armor  Submunition program.
section_title: Army Has Improved the Crusader’s Software Development Management
paragraphs: The Army has made improvements to its management of the software  development process. Program officials stated that they would continue to  aggressively manage the software development program to achieve and  sustain the software process improvements.
The automated Crusader system will be a software intensive program,  projected to use about 1.9 million lines of code. Unlike any previous ground  vehicle, all of the major functions of the Crusader are automated, including  aiming, loading, and firing the cannon; managing inventory (projectiles and  propellant); and resupplying the howitzer with ammunition and fuel. The  crew compartment consists of a digital command center, with flat panel  displays and re-configurable crew stations that give the crew real-time  situation awareness, targeting information, integrated electronic technical  manuals, decision aids, and diagnostic information.
In 1998, the program began to experience software problems before  meeting the software’s preliminary design milestone. In June 1999, the  Army decided that there were incomplete areas of the preliminary design  and that the software team was not resolving design issues in a timely  manner. Additionally, the software engineering team lacked disciplined  quality assurance and configuration management practices, which led to  some of the problems.
In response, the program office tasked a software action team to identify  problems and recommend improvements. The team drafted a recovery plan  and recommended a number of process improvements for the prime  contractor to implement. Program officials used the Software Development  Capability Maturity ModelSM to define and determine the software  development process maturity. The Software Engineering Institute, part of  Carnegie Mellon University, developed the model to measure and rank an  organization’s software development and acquisition process. The  contractor agreed to mature its software engineering processes to a level  where the standard processes for software development, such as project  and risk management, are documented and enforced across the  organization. According to the Software Engineering Institute, increasing  the maturity level of an organization’s software engineering process puts  the organization in better position to successfully develop software.
As a result of these efforts, the Army and its prime contractor have made  improvements to their management of the Crusader software engineering  process. Improved areas include requirements generation and validation,  quality assurance, configuration management, risk management, schedule  and cost estimation, project tracking and control, and peer reviews of  software engineering products such as design documents, code, and test  plans. In addition, outside experts assisted in software analysis and design.  Others were brought in to independently assess the software recovery plan.  The contractor implemented a number of changes in the software design  process, including the establishment of a common set of software  development and management tools shared by all software teams and  improved software testing. The program office has also revised the  Crusader contract to provide the contractor monetary incentives to  produce high-quality software on schedule. Software teams are also  tracking progress and reporting it to management on a weekly or biweekly  basis and have greatly improved their processes for estimating the size and  schedule of the software. As a result of these improvements, the contractor  has made more timely deliveries of software.
Army officials will need to continue their aggressive management approach  because significant amounts of software remain to be developed before the  Crusader is fully operational. Program officials stated that they would  continue to manage the program to achieve and sustain the software  process improvements.
section_title: Lighter-weight Crusader May Not Significantly Improve Strategic Deployability
paragraphs: The Army has made considerable progress over the past 2 years in  redesigning the Crusader to substantially reduce its size and weight. In  general, a lighter system offers a number of advantages, such as lower fuel  consumption and easier transportation by truck and rail. However, it is  uncertain that the requirement to deploy two Crusader howitzers on a C-17  aircraft provides a significant improvement in strategic deployability.  Efforts to meet the deployability requirement will be a challenge and may  require costly design changes and/or performance tradeoffs.
According to an Army official, in October 1999, the Chief of Staff of the  Army directed that the Crusader system become lighter and more  deployable to better fit in with the Army’s transformation to lighter forces.  The Army subsequently revised the Crusader's Operational Requirements  Documents to reflect new deployability requirements. Specifically, the  documents state that the Crusader vehicles must not exceed 42 tons at curb weight and 50  tons at combat weight; any combination of two Crusader vehicles, at curb weight, must be air  transportable on both a C-5 and a C-17 aircraft; and both the C-5 and C-17 aircraft must be able to transport a single  Crusader vehicle at combat weight.
subsections: section_title: Crusader’s Reduced Size and Weight May Not Provide a Significant Improvement in Deployability
paragraphs: The main reason for the decision in January 2000 to restructure the  program and redesign the Crusader weapon system was to reduce the  system’s weight and to improve its strategic deployability by air. However,  the Army expects to rarely airlift the Crusader system—only during  extreme emergencies—and that, in those circumstances, it would be likely  that only small numbers of Crusader systems would be airlifted. Sealift  would be the primary means of moving the Crusader system over long  distances. In February 1999, the Army reported to Congress that the  fielding of a lighter-weight Crusader would provide little in improved  strategic deployability over a heavier version. In May 2000, the DOD’s  Office of Program Analysis and Evaluation questioned the need to improve  the Crusader’s deployability, stating that it is unclear whether airlifting a  small force of the heavier Crusaders, when needed, would be a severe  burden on airlift.
A limited Army analysis comparing the deployability by air of small  numbers of the original heavier Crusader with that of the lighter-weight  Crusader showed that the lighter-weight Crusader system might not  significantly improve the system’s strategic deployability. For example, this  analysis showed that the lighter-weight Crusader system would reduce the  number of sorties required to carry two Crusader systems and support  equipment by 20 percent—one aircraft sortie—over the system’s original,  heavier design. The study showed that it would take four C-17 sorties to  airlift two of the lighter-weight Crusader systems and support equipment  while it would take five sorties to airlift two of the original heavier systems  and support equipment. In addition, the heavier Crusader howitzers and  both resupply vehicles would arrive loaded for combat while the lighter  Crusader howitzers and only one resupply vehicle would arrive loaded for  combat. The other resupply vehicle would have to be manually loaded  upon arrival.
The recent analysis was done with inputs from various Army officials but  has not been officially reviewed by the Air Force. Prior to our request, the  Army had not formally analyzed the improvements in strategic  deployability offered by a 40-ton Crusader over the earlier 60-ton Crusader.
section_title: Army Faces Risks in Meeting Crusader’s Deployability Requirement
paragraphs: Meeting the requirement for carrying two Crusader howitzers on a C-17  aircraft will be challenging. According to the Air Force, the C-17 aircraft is a  more versatile aircraft and smaller than the C-5 aircraft. The C-5 is normally  used for strategic deployments—into and out of the combat theater—while  the C-17 aircraft can be used for both strategic deployments and tactical  missions within a combat theater. According to Army and Air Force  officials responsible for aircraft loading plans, the only possible way to  load two Crusader howitzers on a C-17 aircraft would be back to back.  However, they have concerns about this loading method. First, it will be a  very tight fit with one howitzer’s cannon barrel expected to be 20 inches  from the forward bulkhead (on the edge of a crew safety zone) and the  other howitzer’s barrel expected to be within 3 inches of the stowed aft  loading ramp. Second, according to an Air Force official, the 59 inches  separating the two howitzers may not be enough room to properly restrain  the vehicles with heavy chains.
In October 2001, the Army performed a preliminary computer analysis of  loading two Crusader howitzers on a C-17. It indicated that, if the vehicles  dimensions remain the same through redesign, development, testing,  production, and fielding, the two howitzers may fit. This analysis also  showed that the loading plan would be a very tight fit and does not address  the issue of restraining the howitzers during flight. Air Force officials have  not reviewed this analysis. Army and Air Force officials told us that it is  unlikely they will know if the Crusader can actually be loaded and carried  until two lighter-weight prototypes are produced and tested in a C-17  aircraft.
Army officials told us that, if carrying two Crusaders on a C-17 aircraft is  not feasible, they will still accept the Crusader system because it is a much  more capable system than the current self-propelled howitzer system, the  Paladin. Program officials also told us that reducing the system’s weight is  desirable because it reduced the logistics needed to support the system and  improves, among other things, ground transportability and mobility.
section_title: Cost and/or Performance Trade-offs May Be Needed to Meet Weight Requirement
paragraphs: According to the DOD and the Army, achieving the Crusader’s reduced  weight requirement and meeting the 42-ton limit will be a difficult challenge  and will require aggressive weight management to mitigate the risks  involved with system weight. As of November 2001, the Crusader howitzer  is projected to weigh 41.2 tons, which is close to the upper limit of the  42-ton curb weight requirement. This projection, however, is based on  computer modeling that is still evolving. The projected weight could  change considerably as specific components are fabricated and tested.  Program office officials told us that, at this point in time, they have an  80-percent confidence level in the model’s weight projection.
The Army has already made significant changes to the Crusader system  design to reduce the curb weights of the system’s vehicles. The curb weight  of the howitzer is expected to go from 60 tons to a projected weight of  below 42 tons. To achieve this weight reduction, the program office is  redesigning the Crusader system by reducing the size and payload of the  Crusader vehicles, substituting lighter weight materials for some  components, and developing, with the Abrams tank program, a lighter  weight engine. Additionally, the team plans to remove the heavy armor for  top attack and road wheel protection and make it into kits that can be  applied when needed in combat situations. To help reduce the overall  weight of the Crusader system, the team decided to use a Palletized Load  System truck carrying a newly designed resupply module as a second type  of Crusader resupply vehicle—a wheeled resupply vehicle.
Although the Army has not made vehicle weight a key performance  parameter for the Crusader program, it has instituted an aggressive weight  management program designed to mitigate the risks associated with  maintaining the 42-ton per vehicle weight limit. As part of the weight  management program, the Army may have to consider the trade-offs  between the system’s weight and the program’s cost, schedule, and  performance requirements in order to achieve the required curb and  combat weights. The program is also in the position of not being allowed  any weight growth during development, production, fielding, and service.  Before the Crusader redesign, the program had a 17-percent weight growth  expectation for the Crusader vehicles. According to an Army official, if a  new capability is added to the Crusader that increases its weight, the Army  will have to find a way to reduce the weight of the Crusader by an  equivalent amount.
section_title: Apparent Overlap of Crusader and Future Combat Systems Programs Creates Uncertainties
paragraphs: The Army’s current schedule to begin fielding the Crusader system and its  replacement, the Future Combat Systems, in the same fiscal year—2008—  represents a potential risk of investing in duplicative systems to fulfill the  same missions. However, at this time it is uncertain that the initial versions  of the Future Combat Systems will have the capabilities to meet the  Crusader’s missions.
The Future Combat Systems are expected to be revolutionary, lightweight  weapon systems—20 tons or less—that involve manned and unmanned,  ground and air systems, all of which would be digitally networked together.  All the vehicles in the system are being designed for transport on a C-130 or  similar aircraft—which are smaller aircraft than the C-17. Future Combat  Systems vehicles may include command and control systems,  reconnaissance systems, direct- and indirect-fire guns, rockets, and  antitank missiles.
The Future Combat Systems program is in an earlier stage of development  than the Crusader—it is still in its initial 2-year concept design. Although  the Future Combat Systems is a complex system of systems and the Army  is still developing system concepts and technologies, the Army expects that  the Future Combat Systems can be developed and produced in much  shorter time frames than other weapons programs. Under the current Army  schedule, the initial versions of the Future Combat Systems might enter the  system development and demonstration phase as early as fiscal year 2003  and the first combat unit is scheduled to be equipped in 2008. Once fully  fielded, the Future Combat Systems are intended to replace all of the  Army’s heavy weapon systems including the Crusader. Current Army plans  show the Crusader to be in the force until 2032 or later.
Because all the technologies needed for the Future Combat Systems may  not be mature enough to be put into systems, the Army is planning to  develop the initial version of the Future Combat Systems with less than its  full capabilities and then upgrade it in a number of steps, called blocks, as  the required technologies mature. The Army has not defined the  capabilities that it can develop in the initial version of the Future Combat  Systems, which it hopes will enter product development in 2003. As early as  February 2002, the Army plans to award a contract to define these initial  capabilities based on technologies that are mature enough to enter system  development and demonstration in 2003.
Eventually, the Army expects the Future Combat Systems to meet, using  advanced technologies, the same artillery missions as the Crusader and  eventually replace the Crusader system. While the final weapon  technologies have not been selected for the Future Combat Systems,  technologies that could provide the systems with capabilities to perform  artillery missions similar to or greater than the Crusader include a multi- role armament system. This possible system could feature a 105-mm  cannon that may have a non-line-of-sight capability out to a range of about  50 kilometers. Also, the Army is considering an advanced missile system  that could be comprised of small-containerized missiles, known as  NetFires, which are projected to have a range of 50 to 100 kilometers. A  high-level Army official told us that he believes, based on recent technical  briefings, that the initial version of the Future Combat Systems will not  have the capabilities to meet the same artillery missions as the Crusader.
section_title: Conclusions
paragraphs: Moving into product development without demonstrating critical  technologies in an operational environment increases the risk of cost  overruns, schedule delays, and performance shortfalls. As currently  planned, the majority of the critical Crusader technologies will have been  demonstrated in a relevant environment but not the important operational  environment. If the Crusader program follows the approach of moving into  product development with less mature technologies, the program will need  to continue to develop and demonstrate those technologies while  concentrating on integrating subsystems into the system, testing at the  subsystem and system levels, and preparing for production. As a result,  technical problems, if they occur, will need to be resolved in the higher cost  environment of system development and demonstration. On the other  hand, demonstrating the critical technologies in an operational  environment before entering system development and demonstration could  necessitate more time and money than currently planned before the  milestone B decision, but such investments would be relatively small  compared to solving technical problems after the decision.
The Army restructured the Crusader program to improve the system’s  strategic deployability by reducing the system’s weight. The lighter-weight  system, however, may not provide a significant improvement to strategic  deployability. At this time, the Army is making design trade-offs to meet its  weight requirement and it is not clear whether the Army can maintain its  lighter weight goals throughout the development, production, and fielding  of the Crusader system. Given the uncertainty, the Army risks making  unnecessary cost, schedule, and performance trade-offs to meet  deployability requirements that may not be clearly justified.
The Army has not ruled out the possibility that it will field the Future  Combat Systems with the ability to meet the same artillery mission as the  Crusader in the same year the Crusader is fielded. However, the extent of  this apparent overlap will not be clear until the potential capabilities and  schedule of the initial version of the Future Combat Systems are  determined. Therefore, it is important that the Army ensure that the  projected capabilities and schedule for the initial Future Combat Systems  are considered in the Crusader milestone decision.
section_title: Recommendations for Executive Action
paragraphs: To reduce the risk of schedule delays and increased costs in the product  development phase of the Crusader program, we recommend that the  secretary of defense direct the secretary of the army to dedicate the  resources necessary to ensure that the critical Crusader technologies are  demonstrated, at the component and subsystem level, in an operational  environment before the program commits to product development at  milestone B.
To confirm the value and usefulness of the Crusader program’s  deployability requirement, we recommend that the secretary of defense  direct the secretary of the army to conduct an analysis, before the decision  to enter product development, to determine how important it is to deploy  two Crusaders howitzers on a single C-17 aircraft. If it is important to the  Army, we recommend that the secretary of defense direct the secretary of  the army to establish, as a key performance parameter, the maximum per  vehicle weight that would allow the C-17 aircraft to carry two Crusader  howitzers. If the analysis determines that the redesigned Crusader does not  significantly improve the system’s military utility, we recommend that the  secretary of defense direct the secretary of the army to reduce the priority  placed on attaining the 42-ton weight limit.
Finally, to ensure the Army does not invest in two weapon systems that will  meet the same artillery missions at the same time, we recommend that the  secretary of defense direct the secretary of the army to determine, based  on available data, the potential capabilities and schedule of the initial  version of the Future Combat Systems and the implication of those  capabilities and schedule on the Crusader’s utility to the Army before  making the decision on beginning the Crusader’s system development and  demonstration—currently scheduled for April 2003.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, the director of strategic and  tactical systems, within the Office of the Under Secretary of Defense for  Acquisition, Technology, and Logistics, said that DOD did not agree with  our recommendation that the Crusader technologies be demonstrated in an  operational environment before the program commits to product  development. DOD said that the Crusader program was a simulation-based  acquisition program and, as such, evaluates system, component, and  subsystem performance and technology readiness using modeling and  simulation validated with test stands, integration laboratories, and  subsystem prototypes. DOD questioned our definition of critical Crusader  technologies and said that the track, for example, was selected by us as a  critical technology and assessed as a TRL 5 despite the Army’s many years  of expertise in track development. DOD also said that the Crusader is  currently demonstrating performance equal to or in excess of threshold  requirements for the final system. Finally, DOD said that changing the  Crusader’s acquisition strategy to accommodate building system level  prototypes required to demonstrate TRL 7 for all critical technologies  would add significantly to the development time and expense without  significantly reducing risk or improving performance. The full text of  DOD’s comments is included in appendix II.
We agree that modeling and simulation is a key and accepted practice in  any modern development program. However, we have found that programs  need to demonstrate a high level of technology maturity before committing  to product development. As shown by our past reviews, the best practice  standard is that technology must be demonstrated, at the component or  subsystem level, in an operational environment to be considered mature  enough for entering product development. We believe that a program  should use this best practice to assure success in meeting its cost and  schedule goals.
The determination of the critical Crusader technologies was a joint effort  between the Crusader program office and us. We defined critical Crusader  technologies as those required to meet the Crusader’s key performance  parameters and developed the initial critical technology list. Crusader  program office engineers reviewed the initial list, suggested revisions, and  agreed that the revised critical technologies list was complete and  appropriate. Also, our analysts and program office engineers jointly arrived  at the appropriate TRL for each critical technology. In addition, DOD’s  statement that track should not be a critical Crusader technology or should  have been assessed at a higher TRL because of the Army’s many years of  expertise in track development underscores the value of the TRL  methodology. Track was included as a critical Crusader technology  because the Crusader cannot meet its mobility key performance parameter  without track. The track was assessed at TRL 5 because the Crusader  program was developing a new lighter-weight track. The Army plans to  demonstrate it in an operational environment after milestone B. TRLs  measure whether sufficient knowledge has been accumulated with respect  to each application of a technology, not the development difficulty of the  technology or whether the technology has been previously used in another  application. The issue is not whether a technology like the newly developed  track will ever work, but how much time and effort will be needed to  demonstrate its maturity in this application. The Crusader system  development and demonstration phase does not have much time between  prototype testing and procurement of long-lead items for production to  adjust for any delays or problems in prototype testing caused by  technology problems. Such delays or problems could either delay the long- lead item procurement or reduce the amount of information available when  committing to the procurement.
DOD’s assessment that the Crusader system is currently demonstrating  performance equal to or in excess of threshold requirements for the final  system is based mainly on modeling, simulations, and laboratory tests  because the program has not produced the final system. As mentioned  above, best practice calls for critical technologies to be demonstrated in an  operational environment not in models, simulations, or laboratory  environments before entering product development.
DOD stated that building the full system prototype required to demonstrate  TRL 7 would add significant time and expense to the program. However,  demonstrating at TRL 7 does not require a full system prototype but only a  prototype of the component or subsystem that contains a new technology.  The demonstration can be accomplished by putting the new component or  subsystem, such as an engine, on a surrogate vehicle; that is, a vehicle that  already exists. The report’s point is that using full system prototypes to  demonstrate the maturity of critical technologies during the product  development phase, as planned in the Crusader program, is potentially  more costly than using component or subsystem prototypes to do so during  the technology development phase. Problems that occur during required  demonstrations may cause program delays in either phase, but as noted in  the report, the delay is more expensive during the product development  phase.
DOD stated that it partially agreed with our recommendation to conduct an  analysis to determine the importance of the deployability requirement and  said that the current requirement is not considered a key performance  parameter and, as a result, the Army is allowed to make trade-offs between  the requirement and system cost and performance. DOD further stated that  the Army plans to review the Crusader's requirements prior to the 2003  milestone B decision as required by regulations. We believe that an analysis  to determine the importance of deploying two Crusader howitzers on a C- 17 aircraft should be conducted as soon as possible to provide the Army  greater flexibility and knowledge in considering its ongoing trade-off  decisions needed to meet weight requirements.
DOD stated that it partially agreed with our recommendation to determine  the potential capabilities and schedule of the initial version of the Future  Combat Systems before making the decision to begin Crusader product  development and stated that the Crusader’s capabilities are intended to  complement rather than compete with or be redundant to the capabilities  of the Future Combat Systems. We continue to believe that DOD cannot  determine whether the two systems will be complementary or redundant  without knowledge of the initial Future Combat Systems capabilities and  fielding schedule. DOD does not have this knowledge. We continue to  believe that this knowledge needs to be considered as part of the decision  to allow the Crusader program to enter product development. We have  rewritten the recommendation to clarify its intent.
section_title: Scope and Methodology
paragraphs: To determine the readiness of the Crusader program to enter the system  development and demonstration phase, we assessed, along with engineers from the Crusader Project Office, the current maturity of the critical  Crusader technologies using the technology readiness level tool. We  identified the Crusader technologies we believed were critical to meeting  the Crusader system key performance parameters. Program engineers  reviewed our list, suggested revisions, and agreed that the revised critical  technologies list was complete and appropriate. After considering the  program’s plans for maturing the critical technologies before milestone B,  we jointly determined the probable TRL levels of each of the critical  technologies at the milestone. This determination assumed that the  program office would successfully execute its existing plans for  demonstrating some of the technologies before the milestone.
To assess the status of the Crusader software development, we used  project management criteria derived from Software Engineering Institute's  Software Development Capability Maturity Model. We visited the Crusader  prime contractor, met with Army and contractor officials, observed  software development and test facilities, and examined project  information. We also obtained and reviewed project documentation from  the prime contractor and the Army program office.
To assess the Crusader program’s ability to meet the Crusader reduced  weight requirements and improve the Crusader system’s strategic  deployability, we analyzed the Army’s plans and requirements for reducing  the weight of the Crusader and requested that the Army perform an  analysis of the improvement in strategic deployability that the reduced  weight Crusader system would provide compared to the original weight  Crusader system. For this analysis, at our request, the Army determined the  number of Crusader systems to be deployed, the other equipment and  supplies that were required to be deployed with the Crusader systems, and  the range of the aircraft used for the deployment. We reviewed the results  of the Army’s Crusader deployment analysis.
To determine whether the Army is developing the Crusader and the Future  Combat Systems to be fielded at the same time and to meet the same  artillery missions, we analyzed and compared the Crusader and Future  Combat Systems schedules and reviewed the Crusader system operational  requirements documents. The Future Combat Systems do not have  operational requirements documents at this stage of development. Also, we  discussed with appropriate officials in the Army’s Objective Force Task  Force, the Army’s artillery school, and the Crusader and the Future Combat  Systems programs (1) the probability that the two programs would meet  their individual schedules and (2) the potential technologies that might be  used in the Future Combat Systems to provide it with artillery capabilities.
In performing our work, we obtained documents and interviewed officials  involved in the Crusader and the Future Combat Systems programs in the  Office of the Deputy Chief of Staff for Operations, Washington, D.C.; U.S.  Army Training and Doctrine Command, Fort Monroe, Virginia; U.S. Army  Field Artillery School and Center, Fort Sill, Oklahoma; the Defense  Advanced Research Projects Agency, Arlington, Virginia; the Military  Traffic Management Command, Newport News, Virginia; the U.S. Air Force,  Air Mobility Command, St. Louis, Missouri; the U.S. Air Force Aeronautical  Systems Command, Dayton, Ohio; the Crusader Project Office, Picatinny  Arsenal, New Jersey; and the prime contractor’s Minneapolis, Minnesota,  facility.
We conducted our review between March 2001 and October 2001 in  accordance with generally accepted government auditing standards.
We also are sending copies of this report to the appropriate congressional committees; the director, Office of Management and Budget; and the  secretaries of defense and the army. We will also provide copies to others upon request.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or William R. Graveline at (256) 650-1414. Key  contributors are listed in appendix III.
section_title: Technology Readiness Levels and Their Descriptions
subsections: section_title: 1. Basic principles observed and reported.
paragraphs: Lowest level of technology readiness. Scientific research begins to be translated into  technology’s basic properties.
Invention begins. Once basic principles are observed, practical applications can be  invented. The application is speculative and there is no proof or detailed analysis to  support the assumptions. Examples are still limited to paper studies.
Active research and development is initiated. This includes analytical studies and  laboratory studies to physically validate analytical predictions of separate elements of the  technology. Examples include components that are not yet integrated or representative.
4. Component and/or breadboard validation  in laboratory environment.
Basic technological components are integrated to establish that the pieces will work  together. This is relatively “low fidelity” compared to the eventual system. Examples  include integration of “ad hoc” hardware in a laboratory.  5. Component and /or breadboard validation  Fidelity of breadboard technology increases significantly. The basic technological  in relevant environment.  components are integrated with reasonably realistic supporting elements so that the  technology can be tested in a simulated environment. Examples include “high fidelity”  laboratory integration of components.
Representative model or prototype system, which is well beyond the breadboard tested  for level 5, is tested in a relevant environment. Represents a major step up in technology’s  demonstrated readiness. Examples include testing a prototype in a high fidelity laboratory  environment or in a simulated operational environment.
Prototype near or at planned operational system. Represents a major step up from level  6, requiring the demonstration of an actual system prototype in an operational  environment. Examples include testing the prototype in a test bed aircraft.
Technology has been proven to work in its final form and under expected conditions. In  almost all cases, this level represents the end of true system development. Examples  include developmental test and evaluation of the system in its intended weapon system to  determine if it meets design specifications.  9. Actual system proven through successful  Actual application of technology in its final form and under mission conditions, such as  mission operations.  those encountered in operational test and evaluation. Examples include using the system  under operational mission conditions.
section_title: Comments from the Department of Defense
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Staff Acknowledgments
paragraphs: In addition to those named above, the following individuals made  significant contributions to this report: Robert L. Ackley; Nabajyoti Barkakati; Paul L. Francis; Lawrence D. Gaston, Jr.; Matthew B. Lea;  Gary L. Middleton; Madhav S. Panwar; Robert J. Stolba; and John P. Swain.
section_title: GAO’s Mission
paragraphs: The General Accounting Office, the investigative arm of Congress, exists to  support Congress in meeting its constitutional responsibilities and to help improve  the performance and accountability of the federal government for the American  people. GAO examines the use of public funds; evaluates federal programs and  policies; and provides analyses, recommendations, and other assistance to help  Congress make informed oversight, policy, and funding decisions. GAO’s  commitment to good government is reflected in its core values of accountability,  integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents is through the  Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-text files of  current reports and testimony and an expanding archive of older products. The  Web site features a search engine to help you locate documents using key words  and phrases. You can print these documents in their entirety, including charts and  other graphics.
Each day, GAO issues a list of newly released reports, testimony, and  correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site  daily. The list contains links to the full-text document files. To have GAO E-mail  this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily  e-mail alert for newly released products” under the GAO Reports heading.
subsections: section_title: Order by Mail or Phone
section_title: Visit GAO’s Document Distribution Center
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
paragraphs: Web site: www.gao.gov/fraudnet/fraudnet.htm, E-mail: fraudnet@gao.gov, or 1-800-424-5454 or (202) 512-7470 (automated answering system).
section_title: Public Affairs