DOD Acquisition Outcomes: A Case for Change
Published: 2005-11-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) is shepherding a portfolio of major weapon systems valued at about $1.3 trillion. How DOD is managing this investment has been a matter of concern for some time. Since 1990, GAO has designated DOD's weapon system acquisitions as a high-risk area for fraud, waste, abuse, and mismanagement. DOD has experienced cost overruns, missed deadlines, performance shortfalls, and persistent management problems. In light of the serious budget pressures facing the nation, such problems are especially troubling. GAO has issued hundreds of reports addressing broad-based issues, such as best practices, as well as reports focusing on individual acquisitions. These reports have included many recommendations. Congress asked GAO to testify on possible problems with and improvements to defense acquisition policy. In doing so, we highlight the risks of conducting business as usual and identify some of the solutions we have found in successful acquisition programs and organizations.
section_title: What GAO Found
paragraphs: DOD is facing a cascading number of problems in managing its acquisitions. Cost increases incurred while developing new weapon systems mean DOD cannot produce as many of those weapons as intended nor can it be relied on to deliver to the warfighter when promised. Military operations in Afghanistan and Iraq are consuming a large share of DOD resources and causing the department to invest more money sooner than expected to replace or fix existing weapons. Meanwhile, DOD is intent on transforming military operations and has its eye on multiple megasystems that are expected to be the most expensive and complex ever. These costly conditions are running head-on into the nation's unsustainable fiscal path. DOD knows what to do to achieve more successful outcomes but finds it difficult to apply the necessary discipline and controls or assign much-needed accountability. DOD has written into policy an approach that emphasizes attaining a certain level of knowledge at critical junctures before managers agree to invest more money in the next phase of weapon system development. This knowledge-based approach results in evolutionary--that is, incremental, manageable, predictable--development and inserts several controls to help managers gauge progress in meeting cost, schedule, and performance goals. But DOD is not employing the knowledge-based approach, discipline is lacking, and business cases are weak. Persistent practices show a decided lack of restraint. DOD's requirements process generates more demand for new programs than fiscal resources can support. DOD compounds the problem by approving so many highly complex and interdependent programs. Once too many programs are approved to start, the budgeting process exacerbates problems. Because programs are funded annually and departmentwide, cross-portfolio priorities have not been established, competition for funding continues over time, forcing programs to view success as the ability to secure the next funding increment rather than delivering capabilities when and as promised. Improving this condition requires discipline in the requirements and budgetary processes. Determining who should be held accountable for deviations and what penalties are needed is crucial. If DOD cannot discipline itself now to execute programs within fiscal realities, then draconian, budget-driven decisions may have to be made later.

section_title: Letter
paragraphs: Mr. Chairman and members of the subcommittee:  I am pleased to be here today to discuss why and how to get a better  return from the Department of Defense’s (DOD) weapon system  investments. U.S. weapons are the best in the world, but the programs to  acquire them frequently take significantly longer and cost more money  than promised and often deliver fewer quantities and other capabilities  than planned. It is not unusual for estimates of time and money to be off  by 20 to 50 percent. When costs and schedules increase, quantities are cut,  and the value for the warfighter—as well as the value of the investment  dollar—is reduced.
DOD’s planned investment in research, development, and procurement of  major weapon systems is approximately $1.3 trillion for its current  portfolio, with over $800 billion of that investment yet to be made. The  planned annual investment is expected to rise from around $149 billion in  fiscal year 2005 to $178 billion in fiscal year 2011. Marquee programs  include the Army’s Future Combat Systems; the Missile Defense Agency’s  suite of land, sea, air, and space systems; the Navy’s advanced ships, such  as the DD(X) Destroyer; the Air Force’s Transformational Satellite  Communications System; and the Joint Strike Fighter. Programs like  these—and the Global Information Grid that is designed to interconnect  them—are likely to dominate the budget and doctrinal debate well into the  next decade. Not only do these programs represent huge technological  leaps over their predecessors, DOD is proposing to deliver them faster.
The persistent nature of acquisition problems has perhaps made decision  makers complacent about cost growth, schedule delays, and quantity  reductions in weapon system programs. But fiscal realities, coupled with  the larger scale of acquisitions, will not allow budgets to accommodate the  typical margins of error. Thus, we must either make tough decisions now  to increase the chances for programs to be executable within fiscal  realities or brace ourselves for more draconian decisions later driven by  those fiscal realities. The means to make the thoughtful decisions are  known.
My statement today highlights the risks of conducting business as usual  and identifies some of the solutions we have found in successful  acquisition programs and organizations.
subsections: section_title: The Case for Change
paragraphs: The way DOD develops and produces its major weapons systems has had  disappointing outcomes. There is a vast difference between DOD’s  budgeting plans and the reality of the cost of its systems. Performance, if it  is defined as the capability that actually reaches the warfighter, often falls  short, as cost increases result in fewer quantities of produced systems and  schedule slips. Performance, if it is defined as an acceptable return on  investment, has not lived up to promises.
Table 1 illustrates seven programs with a significant reduction in buying  power; we have reported similar outcomes in many more programs. For  example, the Air Force initially planned to buy 648 F/A-22 Raptor tactical  aircraft at a program acquisition unit cost of about $125 million (fiscal year  2006 dollars). Technology and design components matured late in the  development of the aircraft, which contributed to cost growth and  schedule delays. Now, the Air Force plans to buy 181 aircraft at a program  acquisition unit cost of about $361 million, an almost 189 percent increase.
Furthermore, the conventional acquisition process is not agile enough for  today’s demands. Congress has expressed concern that urgent warfighting  requirements are not being met in the most expeditious manner and has  put in place several authorities for rapid acquisition to work around the  process. The U.S. Joint Forces Command’s Limited Acquisition Authority  and the Secretary of Defense’s Rapid Acquisition Authority seek the ability  to get warfighting capability to the field faster. According to U.S. Joint  Forces Command officials, it is only through Limited Acquisition Authority  that the command has the authority to satisfy the unanticipated,  unbudgeted, urgent mission needs of other combatant commands. With a  formal process that requires as many as 5, 10, or 15 years to get from  program start to production, such experiments are needed to meet the  warfighters’ needs.
Today we are at a crossroad. Our nation is on an unsustainable fiscal path.  Long-term budget simulations by GAO, the Congressional Budget Office,  and others show that, over the long term, we face a large and growing  structural deficit due primarily to known demographic trends and rising  health care costs. Continuing on this unsustainable fiscal path will  gradually erode, if not suddenly damage, our economy, our standard of  living, and ultimately our national security. Federal discretionary  spending, along with other federal policies and programs, will face serious  budget pressures in the coming years stemming from new budgetary  demands and demographic trends. Defense spending falls within the  discretionary spending accounts. Further, current military operations,  such as those in Afghanistan and Iraq, consume a large share of DOD  resources and are causing faster wear on existing weapons. Refurbishment  or replacement sooner than planned is putting further pressure on DOD’s  investment accounts.
At the same time DOD is facing these problems, programs are  commanding larger budgets. DOD is undertaking new efforts that are  expected to be the most expensive and complex ever and on which DOD is  heavily relying to fundamentally transform military operations. And it is  giving contractors increased program management responsibilities to  develop requirements, design products, and select major system and  subsystem contractors. Table 2 shows that just 5 years ago, the top five  weapon systems cost about $291 billion combined; today, the top five  weapon systems cost about $550 billion.
If these megasystems are managed with traditional margins of error, the  financial consequences can be dire, especially in light of a constrained  discretionary budget.
Success for acquisitions means making sound decisions to ensure that  program investments are getting promised returns. In the commercial  world, successful companies have no choice but to adopt processes and  cultures that emphasize basing decisions on knowledge, reducing risks  prior to undertaking new efforts, producing realistic cost and schedule  estimates, and building-in quality in order to deliver products to customers  at the right price, the right time, and the right cost. At first blush, it would  seem DOD’s definition of success would be very similar: deliver capability  to the warfighter at the right price, the right time, and the right cost.  However, this is not happening within DOD. In an important sense,  success has come to mean starting and continuing programs even when  cost, schedule, and quantities must be sacrificed.
DOD knows what to do to improve acquisitions but finds it difficult to  apply the controls or assign the accountability necessary for successful  outcomes. To understand why these problems persist, we must look not  just at the product development process but at the underlying  requirements and budgeting processes to define problems and find  solutions.
section_title: A Knowledge-Based Process Can Lead to Better Outcomes
paragraphs: Over the last several years, we have undertaken a body of work that  examines weapon acquisition issues from a perspective that draws upon  lessons learned from best product development practices. Leading  commercial firms expect that their program managers will deliver high- quality products on time and within budget. Doing otherwise could result  in the customer walking away. Thus, those firms have created an  environment and adopted practices that put their program managers in a  good position to succeed in meeting these expectations. Collectively, these  practices comprise a process that is anchored in knowledge. It is a process  in which technology development and product development are treated  differently and managed separately. The process of developing technology  culminates in discovery—the gathering of knowledge—and must, by its  nature, allow room for unexpected results and delays. Leading firms do  not ask their product managers to develop technology. Successful  programs give responsibility for maturing technologies to a science and  technology organizations, rather than the program or product  development managers. The process of developing a product culminates in  delivery, and, therefore, gives great weight to design and production. The  firms demand—and receive—specific knowledge about a new product  before production begins. A program does not go forward unless a strong  business case on which the program was originally justified continues to  hold true.
Successful product developers ensure a high level of knowledge is  achieved at key junctures in development. We characterize these junctures  as knowledge points. These knowledge points and associated indicators  are defined as follows:
Knowledge point 1: Resources and needs match. This point occurs  when a sound business case is made for the product—that is, a match  is made between the customer’s requirements and the product  developer’s available resources in terms of knowledge, time, money,  and capacity. Achieving a high level of technology maturity at the start  of system development is an important indicator of whether this match  has been made. This means that the technologies needed to meet  essential product requirements have been demonstrated to work in  their intended environment.
Knowledge point 2: Product design is stable. This point occurs when  a program determines that a product’s design is stable— that is, it will  meet customer requirements, as well as cost, schedule and reliability  targets. A best practice is to achieve design stability at the system-level  critical design review, usually held midway through development.  Completion of at least 90 percent of engineering drawings at the  system design review provides tangible evidence that the design is  stable.
Knowledge point 3: Production processes are mature. This point is  achieved when it has been demonstrated that the company can  manufacture the product within cost, schedule, and quality targets. A  best practice is to ensure that all key manufacturing processes are in  statistical control—that is, they are repeatable, sustainable, and  capable of consistently producing parts within the product’s quality  tolerances and standards—at the start of production.
A result of this knowledge-based process is evolutionary product  development, an incremental approach that enables developers to rely  more on available resources rather than making promises about unproven  technologies. Predictability is a key to success as successful product  developers know that invention cannot be scheduled and its cost is  difficult to estimate. They do not bring technology into new product  development unless that technology has been demonstrated to meet the  user’s requirements. Allowing technology development to spill over into  product development puts an extra burden on decision makers and  provides a weak foundation for making product development estimates.  While the user may not initially receive the ultimate capability under this  approach, the initial product is available sooner and at a lower, more  predictable cost.
There is a synergy in this process, as the attainment of each successive  knowledge point builds on the preceding one. Metrics gauge when the  requisite level of knowledge has been attained. Controls are used to attain  a high level of knowledge before making additional significant  investments. Controls are considered effective if they are backed by  measurable criteria and if decision makers are required to consider them  before deciding to advance a program to the next level. Effective controls  help decision makers gauge progress in meeting cost, schedule, and  performance goals and ensure that managers will (1) conduct activities to  capture relevant product development knowledge, (2) provide evidence  that knowledge was captured, and (3) hold decision reviews to determine  that appropriate knowledge was captured to move to the next phase. The  result is a product development process that holds decision makers  accountable and delivers excellent results in a predictable manner.
A hallmark of an executable program is shorter development cycle times,  which allow more systems to enter production more quickly. DOD itself  suggests that product development should be limited to about 5 years.  Time constraints, such as this, are important because they serve to limit  the initial product’s requirements. Limiting product development cycle  times to 5 years or less would allow for more frequent assimilation of new  technologies into weapon systems, speeding new technology to the  warfighter, hold program managers accountable, as well as make more  frequent and predictable work in production, where contractors and the  industrial base can profit by being efficient.
section_title: Despite Policy, DOD Is Not Employing a Knowledge-Based Process
paragraphs: DOD’s policy adopts the knowledge-based, evolutionary approach used by  leading commercial companies that enables developers to rely more on  available resources rather than making promises about unproven  technologies. The policy provides a framework for developers to ask  themselves at key decision points whether they have the knowledge they  need to move to the next phase of acquisition. For example, DOD  Directive 5000.1 states that program managers “shall provide knowledge  about key aspects of a system at key points in the acquisition process,”  such as demonstrating “technologies in a relevant environment … prior to  program initiation.” This knowledge-based framework can help managers  gain the confidence they need to make significant and sound investment  decisions for major weapon systems. In placing greater emphasis on  evolutionary product development, the policy sets up a more manageable  environment for achieving knowledge.
However, the longstanding problem of programs beginning development  with immature technologies is continuing to be seen on even the newest  programs. Several programs approved to begin product development  within only the last few years began with most of their technologies  immature and have already experienced significant development cost  increases. In the case of the Army’s Future Combat Systems, nearly 2 years  after program launch and with $4.6 billion invested, only 1 out of more  than 50 critical technologies is considered mature and the research and  development cost estimate has grown by 48 percent.
In March 2005, we reported that very few programs—15 percent of the  programs we assessed—began development having demonstrated high  levels of technology maturity. Acquisition unit costs for programs  leveraging mature technologies increased by less than 1 percent, whereas  programs that started development with immature technologies  experienced an average acquisition unit cost increase of nearly 21 percent  over the first full estimate.
section_title: Establishing a Sound Business Case Depends on Disciplined Requirements and Funding Process
paragraphs: The decision to start a new program is the most highly leveraged point in  the product development process. Establishing a sound business case for  individual programs depends on disciplined requirements and funding  processes. Our work has shown that DOD’s requirements process  generates more demand for new programs than fiscal resources can  support. DOD compounds the problem by approving so many highly  complex and interdependent programs. Moreover, once a program is  approved, requirements can be added along the way that increases costs  and risks.
Once too many programs are approved to start, the budgeting process  exacerbates problems. Because programs are funded annually and  department wide, cross-portfolio priorities have not been established,  competition for funding continues over time, forcing programs to view  success as the ability to secure the next funding increment rather than  delivering capabilities when and as promised. As a result, there is pressure  to suppress bad news about programs, which could endanger funding and  support, as well as to skip testing because of its high cost. Concurrently,  when faced with budget constraints, senior officials tend to make across- the-board cuts to all programs rather than make the hard decisions as to  which ones to keep and which ones to cancel or cut back. In many cases,  the system delivers less performance than promised when initial  investment decisions were made.
So, the condition we encounter time after time describes a predictable  outcome. The acquisition environment encourages launching product  developments that embody more technical unknowns and less knowledge  about the performance and production risks they entail. A new weapon  system is encouraged to possess performance features that significantly  distinguish it from other systems and promises the best capability. A new  program will not be approved unless its costs fall within forecasts of  available funds and, therefore, looks affordable. Because cost and  schedule estimates are comparatively soft at the time, successfully  competing for funds encourages the program’s estimates to be squeezed  into the funds available. Consequently, DOD program managers have  incentives to promote performance features and design characteristics  that rely on immature technologies and decision makers lack the  knowledge they need to make good decisions.
section_title: The Path to Better Decisions
paragraphs: A path can be laid out to make decisions that will lead to better program  choices and better outcomes. Much of this is known and has been  recommended by one study or another. GAO itself has issued hundreds of  reports. The key recommendations we have made have been focused on  the product development process: constraining individual program requirements by working within  available resources and by leveraging systems engineering; establishing clear business cases for each individual investment; enabling science and technology organizations to shoulder the ensuring that the workforce is capable of managing requirements  trades, source selection, and knowledge-based acquisition strategies;  and establishing and enforcing controls to ensure that appropriate  knowledge is captured and used at critical junctures before moving  programs forward and investing more money.
As I have outlined above, however, setting the right conditions for  successful acquisitions outcomes goes beyond product development. We  are currently examining how to bring discipline to the Department’s  requirements and budgetary process and the role played by the program  manager.
As we conduct this work, we will be asking who is currently accountable for acquisition decisions; who should be held accountable; how much deviation from the original business case is allowed before  the entire program investment is reconsidered; and what is the penalty when investments do not result in meeting  promised warfighter needs?
We can make hard, but thoughtful, decisions now or postpone them,  allowing budgetary realities to force draconian decisions later.
Mr. Chairman, this concludes my prepared statement. I would be happy to  respond to any questions that you or other members of the subcommittee  may have.
section_title: Contacts and Staff Acknowledgments
paragraphs: For further information regarding this testimony, please contact   Katherine V. Schinasi at (202) 512-4841 or schinasik@gao.gov. Individuals  making key contributions to this testimony included Paul L. Francis,   David B. Best, David J. Hand, Alan R. Frazier, Adam Vodraska, and   Lily J. Chin.
section_title: Related GAO Products
paragraphs: Space Acquisitions: Stronger Development Practices and Investment  Planning Needed to Address Continuing Problems. GAO-05-891T.  Washington, D.C.: July 12, 2005.
Air Force Procurement: Protests Challenging Role of Biased Official  Sustained. GAO-05-436T. Washington, D.C.: April 14, 2005.
Tactical Aircraft: F/A-22 and JSF Acquisition Plans and Implications  for Tactical Aircraft Modernization. GAO-05-591T. Washington, D.C.:  April 6, 2005.
Defense Acquisitions: Assessments of Selected Major Weapon Programs.  GAO-05-301. Washington, D.C.: March 31, 2005.
Defense Acquisitions: Future Combat Systems Challenges and Prospects  for Success. GAO-05-428T. Washington, D.C.: March 16, 2005.
Defense Acquisitions: Stronger Management Practices Are Needed to  Improve DOD’s Software-Intensive Weapon Acquisitions. GAO-04-393.  Washington, D.C.: March 1, 2004.
Defense Acquisitions: DOD’s Revised Policy Emphasizes Best Practices,  but More Controls Are Needed. GAO-04-53. Washington, D.C.: November  10, 2003.
Best Practices: Setting Requirements Differently Could Reduce Weapon  Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.: February  11, 2003.
Best Practices: Capturing Design and Manufacturing Knowledge Early  Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15,  2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best  Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to  Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8,  2001.
Best Practices: A More Constructive Test Approach Is Key to Better  Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31,  2000.
Defense Acquisition: Employing Best Practices Can Shape Better  Weapon System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April  26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon System  Program Implement Best Practices. GAO/NSIAD-99-206. Washington,  D.C.: August 16, 1999.
Best Practices: Better Management of Technology Development Can  Improve Weapon System Outcomes. GAO/NSIAD-99-162. Washington,  D.C.: July 30, 1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program  Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.
Defense Acquisition: Improved Program Outcomes Are Possible.   GAO/T-NSIAD-98-123. Washington, D.C.: March 18, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires  Changes in DOD’s Environment. GAO/NSIAD-98-56. Washington, D.C.:  February 24, 1998.
Major Acquisitions: Significant Changes Underway in DOD’s Earned  Value Management Process. GAO/NSIAD-97-108. Washington, D.C.: May 5,  1997.
Best Practices: Commercial Quality Assurance Practices Offer  Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26,  1996.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.