The Department of Defense (DOD) spends close to $100 billion annually to research, develop, and acquire weapon systems, and this investment is expected to grow substantially. Over the next 5 years, starting in fiscal year 2003, DOD’s request for weapon system development and acquisition funds is estimated to be $700 billion (see fig. 1). How effectively DOD manages these funds will determine whether it receives a good return on its investment. Our reviews over the past 20 years have consistently found that DOD’s weapon system acquisitions take much longer and cost much more than originally anticipated, causing disruptions to the department’s overall investment strategy and significantly reducing its buying power. Because such disruptions can limit DOD’s ability to effectively execute war-fighting operations, it is critical to find better ways of doing business. In view of the importance of DOD’s investment in weapon systems, we have undertaken an extensive body of work that examines DOD’s acquisition issues from a different, more cross-cutting perspective—one that draws lessons learned from the best commercial product development efforts to see if they apply to weapon system acquisitions. This report looks at the core of the acquisition process, specifically product development and ways to successfully design and manufacture the product. Our previous reports looked at such issues as how companies matched customer needs and resources, tested products, assured quality, and managed suppliers and are listed in related GAO products at the end of the report. Leading commercial companies expect their program managers to deliver high-quality products on time and within budget. Doing otherwise could result in the customer walking away. Thus, the companies have created an environment and adopted practices that put their program managers in a good position to succeed in meeting these expectations. Collectively, these practices ensure that a high level of knowledge exists about critical facets of the product at key junctures during development. Such a knowledge- based process enables decision makers to be reasonably certain about critical facets of the product under development when they need this knowledge. To ensure the right level of knowledge at each key decision point in product development, leading commercial companies separate technology from product development and take steps to ensure the product design is stabilized early so product performance and producibility can be demonstrated before production. The process followed by leading companies, illustrated in figure 2, can be broken down into the following three knowledge points. Knowledge point 1 occurs when a match is made between the customer’s needs and the available resources—technology, design, time, and funding. To achieve this match, technologies needed to meet essential product requirements must be demonstrated to work in their intended environment. In addition, the product developer must complete a preliminary product design using systems engineering to balance customer desires with available resources. Knowledge point 2 occurs when the product’s design demonstrates its ability to meet performance requirements. Program officials are confident that the design is stable and will perform acceptably when at least 90 percent of engineering drawings are complete. Engineering drawings reflect the results of testing and simulation and describe how the product should be built. Knowledge point 3 occurs when the product can be manufactured within cost, schedule, and quality targets and is reliable. An important indicator of this is when critical manufacturing processes are in control and consistently producing items within quality standards and tolerances. Another indicator is when a product’s reliability is demonstrated through iterative testing that identifies and corrects design problems. This report focuses on best practices for achieving knowledge points 2 and 3, particularly at how successful companies design and manufacture a product within established cost, schedule, and quality targets. The concepts discussed build on our previous reports, which looked at the earlier phases of an acquisition, including matching customer needs and available resources. A key success factor evident in all our work is the ability to obtain the right knowledge at the right time and to build knowledge to the point that decision makers can make informed decisions about moving ahead to the next phase. Programs that do this typically have successful cost and schedule outcomes. Programs that do not typically encounter problems that eventually cascade and become magnified through the product development and production phases. As shown in figure 3, the effects of not following a knowledge-based process can be debilitating. DOD has historically developed new weapon systems in a highly concurrent environment that usually forces acquisition programs to manage technology, design, and manufacturing risk at the same time. This environment has made it difficult for either DOD or congressional decision makers to make informed decisions because appropriate knowledge has not been available at key decision points in product development. DOD’s common practice for managing this environment has been to create aggressive risk reduction efforts in its programs. Cost reduction initiatives that typically arise after a program is experiencing problems are common tools used to manage these risks. Figure 4 shows the overlapping and concurrent approach that DOD uses to develop its weapon systems. This figure shows that DOD continues to capture technology, design, and manufacturing knowledge long after a program passes through each of the three knowledge points when this knowledge should have been available for program decisions. More important, the problems created by this concurrent approach on individual programs can profoundly affect DOD’s overall modernization plans. It is difficult to prioritize and allocate limited budgets among needed requirements when acquisition programs’ cost and schedule are always in question. Programs that are managed without the knowledge-based process are more likely to have surprises in the form of cost and schedule increases that are accommodated by disrupting the funding of other programs. Because of these disruptions, decision makers are not able to focus on a balanced investment strategy. DOD is taking steps to change the culture of the acquisition community with actions aimed at reducing product development cycle times and improving the predictability of cost and schedule outcomes. DOD recently made constructive changes to its acquisition policy that embrace best practices. These changes focused primarily on (1) ensuring technologies are demonstrated to a high level of maturity before beginning a weapon system program and (2) taking an evolutionary, or phased, approach to developing new weapon systems. Because these changes occurred in 2000 and 2001, it is too early to determine how effectively they will be put into practice. While these are good first steps, further use of best practices in product development would provide a greater opportunity to improve weapon system cost and schedule outcomes. Our overall objective was to determine whether best practices offer methods to improve the way DOD ensures that the design is stable early in the development process and whether having manufacturing processes in control before production results in better cost, schedule, and quality outcomes in DOD major acquisition programs. Specifically, we identified best practices that have led to more successful product development and production outcomes, compared the best practices to those used in DOD programs, and analyzed current weapon system acquisition guidance for applicability of best practices. To determine the best practices for ensuring product design and manufacturing maturity from the commercial sector, we conducted general literature searches. On the basis of our literature searches and discussions with experts, we identified a number of commercial companies as having innovative development processes and practices that resulted in successful product development. We visited the following commercial companies: Caterpillar designs and manufactures construction and mining equipment, diesel and natural gas engines, and industrial gas turbines. In 2001, it reported sales and revenues totaling $20.45 billion. We visited its offices in Peoria, Illinois. Cummins Inc. (Engine Business group) designs and manufactures diesel and natural gas engines ranging in size from 60 to 3,500 horsepower for mining, construction, agriculture, rail, oil and gas, heavy and medium- duty trucks, buses, and motor homes. In 2001, the Engine Business Group reported sales of $3.1 billion. We visited its offices in Columbus, Indiana. General Electric Aircraft Engines designs and manufactures jet engines for civil and military aircraft and gas turbines, derived from its successful jet engine programs, for marine and industrial applications. In 2001, it reported earnings totaling $11.4 billion. We visited its offices in Evendale, Ohio. Hewlett Packard designs and manufactures computing systems and imaging and printing systems for individual and business use. In 2001, it reported revenues totaling $45.2 billion. We visited its offices involved in the design and manufacturing of complex ink jet imaging equipment in Corvallis, Oregon. Xerox Corporation designs and manufactures office equipment, including color and black and white printers, digital presses, multifunction devices, and digital copiers designed for offices and production-printing environments. In 2001, it reported revenues totaling $16.5 billion. We visited its offices in Rochester, New York. At each of the five companies, we conducted structured interviews with representatives to gather uniform and consistent information about each company’s new product development processes and best practices. During meetings with these representatives, we obtained a detailed description of the processes and practices they believed necessary and vital to mature a product design and get manufacturing processes under control. We met with design engineers, program managers, manufacturing and quality engineers, and developers of the knowledge-based processes and policies. During the past 5 years, we have gathered information on product development practices from such companies as 3M, Boeing Commercial Airplane Group, Chrysler Corporation, Bombardier Aerospace, Ford Motor Company, Hughes Space and Communications, and Motorola Corporation. This information enabled us to develop an overall model to describe the general approach leading commercial companies take to develop new products. Our report highlights several best practices in product development based on our fieldwork. As such, they are not intended to describe all practices or suggest that commercial companies are without flaws. Representatives from the commercial companies visited told us that the development of their best practices has evolved over many years and that the practices continue to be improved based on lessons learned and new ideas and information. They admit that the application and use of these have not always been consistent or without error. However, they strongly suggested that the probability of success in developing new products is greatly enhanced by the use of these practices.