Clearly, the acquisition process has produced superior weapons, but it does so at a high price. Weapon systems routinely take much longer time to field, cost more to buy, and require more support than investment plans provide for. These consequences reduce the buying power of the defense dollar, delay capabilities for the war fighter, and force unplanned—and possibly unnecessary—trade-offs in desired acquisition quantities and an adverse ripple effect among other weapons programs or defense needs. Because of the lengthy time to develop new weapons, many enter the field with outdated technologies and a diminished supply base needed for system support. Frequently, this requires upgrades to the capability as soon as the new system is fielded. As previously noted, these inefficiencies have often led to reduced quantities of new systems. In turn, legacy systems remain in the inventory for longer periods, requiring greater operations and support cost that pull funds from other accounts, including modernization. DOD is facing these problems with its tactical air force assets now. We believe DOD can learn lessons from the experiences with the F/A-22 program as it frames the acquisition environment for its many transformational investments. DOD recognizes the need to get better weapon system outcomes, and its newest acquisition policy emphasizes the use of evolutionary, knowledge- based acquisition concepts proven to produce more effective and efficient outcomes in developing new products. It incorporates the elements of a knowledge-based acquisition model for developing new products, which we have recommended in our reviews of commercial best practices. Our body of work focuses on how DOD can better leverage its investments by shortening the time it takes to field new capabilities at a more predictable cost and schedule. However, policy changes alone will not guarantee success. Unless written policies are consistently implemented in practice through timely and informed decisions on individual programs, outcomes will not change. This requires sustained leadership and commitment and attention to the capture and use of key product knowledge at critical decision points to avoid the problems of the past. A key enabler to the success of commercial firms is using an approach that evolves a product to its ultimate capabilities on the basis of mature technologies and available resources. This approach allows commercial companies to develop and produce more sophisticated products faster and less expensively than their predecessors. Commercial companies have found that trying to capture the knowledge required to stabilize the design of a product that requires significant amounts of new technical content is an unmanageable task, especially if the goal is to reduce development cycle times and get the product to the marketplace as quickly as possible. Therefore, product features and capabilities not achievable in the initial development are planned for subsequent development efforts in future generations of the product, but only when technologies are proven to be mature and other resources are available. DOD’s new policy embraces the idea of evolutionary acquisition. Figure 1 compares evolutionary and single step (“big bang”) acquisitions. An evolutionary environment for developing and delivering new products reduces risks and makes cost more predictable. While the customer may not receive an ultimate capability initially, the product is available sooner, with higher quality and reliability, and at lower, more predictable cost. Improvements are planned for future generations of the product. Leading commercial firms expect that their program managers will deliver high-quality products on time and within budgets. Doing otherwise could result in losing a customer in the short term and losing the company in the longer term. Thus, in addition to creating an evolutionary product development environment that brings risk in control, these firms have adopted practices that put their individual program managers in a good position to succeed in meeting these expectations on individual products. Collectively, these practices ensure that a high level of knowledge exists about critical facets of the product at key junctures during its development. Such a knowledge-based process enables decision makers to be reasonably certain about critical facets of the product under development when they need to be. The knowledge-based process followed by leading firms is shown in detail in table 1, but in general can be broken down into three knowledge points. First, a match must be made between the customer’s needs and the available resources—technology, engineering knowledge, time, and funding—before a program is launched. Second, a product’s design must demonstrate its ability to meet performance requirements and be stable about midway through development. Third, the developer must show that the product can be manufactured within cost, schedule, and quality targets and is demonstrated to be reliable before production begins. The following table illustrates more specifically what we have learned about how successful programs gather knowledge as they move through product development. DOD programs often do not employ these practices. We found that if the evolutionary, knowledge-based acquisition concepts were not applied, a cascade of negative effects became magnified in the product development and production phases of an acquisition program. These led to acquisition outcomes that included significant cost increases and schedule delays, poor product quality and reliability, and delays in getting new capability to the war fighter. This is often the case in DOD programs as shown in our past work on systems like F/A-22 fighter, C-17 airlifter, V-22 tiltrotor aircraft, PAC-3 missile, BAT antitank munition, and others. We did find some DOD programs that employed best practice concepts and have had more successful program outcomes to date. These included the Global Hawk unmanned vehicle, AIM-9X missile, and Joint Direct Attack Munitions guided bomb. Figure 3 shows a notional illustration of the different paths and effects of a product development. It is clear that knowledge about the product’s technology, design, and processes captured at the right time can reduce development cycle times and deliver a more cost effective, reliable product to the customer sooner than programs that do not capture this knowledge. In applying the knowledge-based approach, the most leveraged decision point of the three, is matching the customer’s needs with the developer’s resources—technology, design, timing, and funding. This initial decision sets the stage for the eventual outcome—desirable or problematic. The match is ultimately achieved in every development program, but in successful development programs, it occurs prior to program launch. In successful programs, negotiations and trade-offs occur before a product development is launched to ensure that a match exists between customer expectations and developer resources. The results achieved from this match are balanced and achievable requirements, sufficient investment to complete the development, and a firm commitment to deliver the product. Commercial companies we have visited usually limit product development cycle-time to less than 5 years. In DOD, this match is seldom achieved. It is not unusual for DOD to bypass early trade-offs and negotiations, instead planning to develop a product based on a rigid set of requirements that are unachievable within a reasonable development time frame. This results in cost and schedule commitments that are unrealistic. Although a program can take as long as 15 years in DOD, the program manager is expected to develop and be accountable for precise cost and schedule estimates made at the start of the program. Because of their short tenures, it normally takes several program managers to complete product development. Consequently, the program manager that commits to the cost and schedule estimate at the beginning of the program is not the same person responsible for achieving it. Therefore, program accountability is problematic. Ironically, this outcome is rational in the traditional acquisition environment. The pressures put on program managers to get programs approved encourage promising more than can be delivered for the time and money allotted. They are not put in a position to succeed. The differences in the practices employed by successful commercial firms and DOD reflect the different demands imposed on programs by the environments in which they are managed. Specific practices take root and are sustained because they help a program succeed in its environment. The way success and failure are defined for commercial and defense product developments differs considerably, which creates a different set of incentives and evokes different behaviors from managers. Attempts at reforming weapon system acquisitions have not succeeded because they did not change these incentives. All of the participants in the acquisition process play a part in creating incentives. The F/A-22 program, advertised as a flagship of acquisition reform in its early days, failed to establish this match before program launch and today we are discussing the resulting outcomes to-date. The F/A-22 provides an excellent example of what can happen when a major acquisition program is not guided by the principles of evolutionary, knowledge-based acquisition. The program failed to match requirements with resources and make early trade-offs and took on a number of new and unproven technologies. Instead of fielding early capability and then evolving the product to get new capabilities to the war fighter sooner, the Air Force chose a “big bang” product development approach that is now planned to take about 19 years. This created a challenging and risky acquisition environment that has delayed the war fighter the capabilities expected from this new aircraft. Program leaders did not capture the specific knowledge identified as key for each of the three critical knowledge points in product development. Instead, program managers proceeded through the F/A-22’s development without the requisite knowledge necessary for reducing program risk and achieving more successful program outcomes. Now the optimism underlying these decisions has resulted in significant cost increases, schedule delays, trade- offs—making do with less than half the number of originally desired aircraft—and concerns about the capability to be delivered. Since the F/A-22 acquisition program was started in October 1986, the F/A- 22 cost and schedule estimates have grown significantly to where, today, the Air Force estimates the total acquisition unit cost of a single aircraft is $257.5 million. This represents a 74 percent increase from the estimate at the start of development and a commensurate loss in the buying power of the defense dollar. Intended to replace the aging F-15 fighter, the F/A-22 program is now scheduled to reach its initial operational capability in December 2005—making its development cycle about 19 years. During this cycle, the planned buy quantity has been reduced 63 percent from 750 to 276 aircraft.