Defense Acquisitions: Where Should Reform Aim Next?
Released: 2013-10-29T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: DOD's acquisition of major weapon systems has been on GAO's high risk list since 1990. Over the past 50 years, Congress and DOD have continually explored ways to improve acquisition outcomes, including reforms that have championed sound management practices, such as realistic cost estimating, prototyping, and systems engineering. Too often, GAO reports on the same kinds of problems today that it did over 20 years ago.

The topic of today's hearing is: "25 Years of Acquisition Reform: Where Do We Go From Here?" To that end, this testimony discusses (1) the performance of DOD's major defense acquisition program portfolio; (2) the management policies and processes currently in place to guide those acquisitions; (3) the incentives to deviate from otherwise sound acquisition practices; and (4) suggestions to temper these incentives. This statement draws from GAO's extensive body of work on DOD's acquisition of weapon systems.

section_title: What GAO Found
paragraphs: The Department of Defense (DOD) must get better outcomes from its weapon system investments, which in recent years have totaled around $1.5 trillion or more. Recently, there have been some improvements, owing in part to reforms. For example, cost growth declined between 2011 and 2012 and a number of programs also improved their buying power by finding efficiencies in development or production and requirements changes. Still, cost and schedule growth remain significant; 39 percent of fiscal 2012 programs have had unit cost growth of 25 percent or more.

DOD's acquisition policy provides a methodological framework for developers to gather knowledge that confirms that their technologies are mature, their designs stable, and their production processes are in control. The Weapon Systems Acquisition Reform Act of 2009 and DOD's recent "Better Buying Power" initiatives introduced significant changes that, when fully implemented, should further strengthen practices that can lead to successful acquisitions. GAO has also made numerous recommendations to improve the acquisition process, based on its extensive work in the area. While recent reforms have benefited individual programs, it is premature to say there is a trend or a corner has been turned. The reforms still face implementation challenges and have not yet been institutionalized within the services.

Reforms that focus on the methodological procedures of the acquisition process are only partial remedies because they do not address incentives to deviate from sound practices. Weapons acquisition is a complicated enterprise, complete with unintended incentives that encourage moving programs forward by delaying testing and employing other problematic practices. These incentives stem from several factors. For example, the different participants in the acquisition process impose conflicting demands on weapon programs so that their purpose transcends just filling voids in military capability. Also, the budget process forces funding decisions to be made well in advance of program decisions, which encourages undue optimism about program risks and costs. Finally, DOD program managers' short tenures and limitations in experience and training can foster a short-term focus and put them at a disadvantage with their industry counterparts.

Drawing on its extensive body of work in weapon systems acquisition, GAO sees several areas of focus regarding where to go from here:

at the start of new programs, using funding decisions to reinforce desirable principles such as well-informed acquisition strategies;

identifying significant risks up front and resourcing them;

exploring ways to align budget decisions and program decisions more closely; and

attracting, training, and retaining acquisition staff and managers so that they are both empowered and accountable for program outcomes.

These areas are not intended to be all-encompassing, but rather, practical places to start the hard work of realigning incentives with desired results.

section_title: Letter
paragraphs: I am pleased to be here today to discuss weapon systems acquisition and  where reform should focus next. Weapon systems acquisition has been  on GAO’s high risk list since 1990. Over the past 50 years, Congress  and the Department of Defense (DOD) have explored ways to improve  acquisition outcomes, including recent actions like the Weapon Systems  Acquisition Reform Act and the department’s own “Better Buying Power”  initiatives. These and other reforms have championed sound  management practices, such as realistic cost estimating, prototyping, and  systems engineering. DOD’s declining budgets and the impact of  sequestration have lent additional impetus to reduce the costs of  weapons. While some progress has been made on this front, too often we  report on the same kinds of problems today that we did over 20 years  ago. The cost growth of DOD’s 2012 portfolio of weapon systems about  $411 billion and schedule delays average more than 2 years. To get  better results the focus should not be on adding to or discarding  acquisition policies, but on the incentives that work against them.

Today, I will (1) provide summary cost and schedule information on  DOD’s portfolio of major weapon systems; (2) describe the policies and  processes in place to guide those acquisitions; (3) discuss incentives to  deviate from otherwise sound acquisition practices; and (4) suggest ways  to temper these incentives. This statement draws from our extensive body  of work on DOD’s acquisition of weapon systems and the numerous  recommendations we have made both on individual weapons and  systemic improvements to the acquisition process. The work on which  this testimony is based was conducted in accordance with generally  accepted government auditing standards. Those standards require that  we plan and perform the audit to obtain sufficient, appropriate evidence to  provide a reasonable basis for our findings and conclusions based on our  audit objectives. We believe the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.
subsections: section_title: Trends in DOD’s Portfolio of Major Acquisitions
paragraphs: There can be little doubt that we can—and must—get better outcomes  from our weapon system investments. As seen in table 1, the value of  these investments in recent years has been on the order of $1.5 trillion or  more, making them a significant part of the federal discretionary budget.

Large programs have an outsized impact on the aggregate portfolio. For  example, Joint Strike Fighter costs have now consumed nearly a quarter  of the entire portfolio. Yet, as indicated in table 1, 39 percent of programs  have had unit cost growth of 25 percent or more. Recently, we have seen  some modest improvements. For example, cost growth has declined  between 2011 and 2012.programs have improved their buying power by finding efficiencies in   We have also observed that a number of  development or production, and requirements changes. On the other  hand, cost and schedule growth remain significant when measured  against programs’ first full estimates. The performance of some very large  programs are no longer reflected in the latest data as they are no longer  acquisition programs. For example, the Future Combat Systems program  was canceled in 2009 after an investment of about $18 billion and the F- 22 Raptor program has completed aircraft procurement. In addition, the  Ballistic Missile Defense System are not included in any of the analysis as  those investments have proceeded without a baseline of original  estimates, so the many difficulties experienced in the roughly $130 billion  program are not quantifiable.

The enormity of the investment in acquisitions of weapon systems and its  role in making U.S. fighting forces capable, warrant continued attention  and reform. The potential for savings and for better serving the warfighter  argue against complacency.

section_title: One Side of the Acquisition Process: Stated Policy
paragraphs: When one thinks of the weapon systems acquisition process, the image  that comes to mind is that of the methodological procedure depicted on  paper and in flow charts. DOD’s acquisition policy takes the perspective  that the goal of acquisition is to obtain quality products that satisfy user  needs at a fair and reasonable price. The sequence of events that  comprise the process defined in policy reflects principles from disciplines  such as systems engineering, as well as lessons learned, and past  reforms. The body of work we have done on benchmarking best practices   Recent, significant changes  has also been reflected in acquisition policy.to the policy include those introduced by the Weapon Systems Acquisition  Reform Act of 2009 and the department’s own “Better Buying Power”  initiatives which, when fully implemented, should further strengthen  practices that can lead to successful acquisitions. The policy provides a  framework for developers of new weapons to gather knowledge that  confirms that their technologies are mature, their designs are stable, and  their production processes are in control. These steps are intended to  ensure that a program will deliver the capabilities required utilizing the  resources—cost, schedule, technology, and personnel—available.  Successful product developers ensure a high level of knowledge is  achieved at key junctures in development. We characterize these  junctures as knowledge points. While there can be differences of opinion  over some of the specifics of the process, I do not believe there is much  debate about the soundness of the basic steps. It is a clear picture of  “what to do.”

Table 2 summarizes these steps and best practices, organized around  three key knowledge points in a weapon system acquisition.

Our work over the last year shows that, to the extent reforms like the  Weapon Systems Acquisition Reform Act and DOD’s Better Buying  Power initiatives are being implemented, they are having a positive effect  on individual programs. For example, several programs we have  reviewed are: making early trade-offs among cost, schedule, and technical developing more realistic cost and schedule estimates; increasing the amount of testing during development; and placing greater emphasis on reliability.

These improvements do not yet signify a trend or suggest that a corner  has been turned. The reforms themselves still face implementation  challenges such as staffing and clarity of guidance and will doubtless  need refining as experience is gained. We have made a number of  recommendations on how DOD can improve implementation of the  Weapon Systems Acquisition Reform Act.

To a large extent, the improvements we have seen tend to result from  external pressure exerted by higher level offices within DOD on individual  programs. In other words, the reforms have not yet been institutionalized  within the services. We still see employment of other practices—that are  not prescribed in policy—such as concurrent testing and production,  optimistic assumptions, and delayed testing. These are the same kinds of  practices that perpetuate the unsatisfactory results that have persisted in  acquisitions through the decades, such as significant cost growth and  schedule delays. They share a common dynamic: moving forward with  programs before the knowledge needed to make decisions is sufficient.

We have reported that most programs still proceed through the critical  design review without having a stable design, even though we have made  a number of recommendations on the importance of this review and how  to prepare for it. Also, programs proceed with operational testing before  they are ready. Other programs are significantly at odds with the  acquisition process. Among these I would number Ballistic Missile  Defense System, Future Combat Systems (since canceled), Littoral  Combat Ship, and airships. We recently reported on the Unmanned  Carrier-Launched Airborne Surveillance and Strike program which  proposes to complete the main acquisition steps of design, development,  testing, manufacturing, and initial fielding before it formally enters the  acquisition process.

The fact that programs adopt practices that run counter to what policy and  reform call for is evidence of the other pressures and incentives that  significantly influence program practices and outcomes. I will turn to these  next.

section_title: Another Side of Acquisition: Incentives
paragraphs: An oft-cited quote of David Packard, former Deputy Secretary of Defense,  is: “We all know what needs to be done. The question is why aren’t we  doing it?” To that point, reforms have been aimed mainly at the “what”  versus the “why.” They have championed sound management practices,  such as realistic estimating, thorough testing, and accurate reporting.  Today, these practices are well known. We need to consider that they  mainly address the mechanisms of weapon acquisitions. Seen this way,  the practices prescribed in policy are only partial remedies. The  acquisition of weapons is much more complex than policy describes and  involves very basic and strongly reinforced incentives to field weapons.  Accordingly, rival practices, not normally viewed as good management  techniques, comprise an effective stratagem for fielding a weapon  because they reduce the risk that the program will be interrupted or called  into question.

I will now discuss several factors that illustrate the pressures that create  incentives to deviate from sound acquisition management practices.
subsections: section_title: Several Factors Create Incentives to Deviate from Sound Acquisition Practices
subsections: section_title: Conflicting Demands
paragraphs: The process of acquiring new weapons is (1) shaped by its different  participants and (2) far more complex than the seemingly straightforward  purchase of equipment to defeat an enemy threat. Collectively, as  participants’ needs are translated into actions on weapon programs, the  purpose of such programs transcends efficiently filling voids in military  capability. Weapons have become integral to policy decisions, definitions  of roles and functions, justifications of budget levels and shares, service  reputations, influence of oversight organizations, defense spending in  localities, the industrial base, and individual careers. Thus, the reasons  “why” a weapon acquisition program is started are manifold and  acquisitions do not merely provide technical solutions.

While individual participants see their needs as rational and aligned with  the national interest, collectively, these needs create incentives for  pushing programs and encouraging undue optimism, parochialism, and  other compromises of good judgment. Under these circumstances,  persistent performance problems, cost growth, schedule slippage, and  difficulties with production and field support cannot all be attributed to  errors, lack of expertise, or unforeseeable events. Rather, a level of these  problems is embedded as the undesirable, but apparently acceptable,  consequence of the process. These problems persist not because they  are overlooked or under-regulated, but because they enable more  programs to survive and thus more needs to be met. The problems are  not the fault of any single participant; they are the collective responsibility  of all participants. Thus, the various pressures that accompany the  reasons why a program is started can also affect and compromise the  practices employed in its acquisition.

I would like to highlight three characteristics about program funding that  create incentives in decision making that can run counter to sound  acquisition practices. First, there is an important difference between what  investments in new products represent for a private firm and for DOD. In  a private firm, a decision to invest in a new product, like a new car design,  represents an expense. Company funds must be expended that will not  provide a revenue return until the product is developed, produced, and  sold. In DOD, new products, in the form of budget line items, can  represent revenue. An agency may be able to justify a larger budget if it  can win approval for more programs. Thus, weapon system programs can  be viewed both as expenditures and revenue generators.

Second, budgets to support major program commitments must be  approved well ahead of when the information needed to support the  decision to commit is available. Take, for example, a decision to start a  new program scheduled for August 2016. Funding for that decision would  have to be included in the fiscal year 2016 budget. This budget would be  submitted to Congress in February 2015—18 months before the program  decision review is actually held. DOD would have committed to the  funding before the budget request went to Congress. It is likely that the  requirements, technologies, and cost estimates for the new program— essential to successful execution—may not be very solid at the time of  funding approval. Once the hard-fought budget debates put money on the  table for a program, it is very hard to take it away later, when the actual  program decision point is reached.

Third, to the extent a program wins funding, the principles and practices it  embodies are thus endorsed. So, if a program is funded despite having  an unrealistic schedule or requirements, that decision reinforces those  characteristics, not sound acquisition processes. Pressure to make  exceptions for programs that do not measure up are rationalized in a  number of ways: an urgent threat needs to be met; a production capability  needs to be preserved; despite shortfalls, the new system is more  capable than the one it is replacing; or the new system’s problems will be  fixed in the future. It is the funding approvals that ultimately define  acquisition policy.

DOD has a unique relationship with the defense industry that differs from  the commercial marketplace. The combination of a single buyer (DOD), a  few very large prime contractors in each segment of the industry, and a  limited number of weapon programs constitutes a structure for doing  business that is altogether different from a classic free market. For  instance, there is less competition, more regulation, and once a contract  is awarded, the contractor has considerable power. Moreover, in the  defense marketplace, the firm and the customer have jointly developed  the product and, as we have reported previously, the closer the product  comes to production the more the customer becomes invested and the  less likely they are to walk away from that investment. While a defense  firm and a military customer may share some of the same goals,  important goals are different. Defense firms are accountable to their  shareholders and can also build constituencies outside the direct  business relationship between them and their customers. This  relationship does not fit easily into a contract.

J. Ronald Fox, author of Defense Acquisition Reform 1960-2009: An  Elusive Goal, sums up the situation as follows. “Many defense acquisition  problems are rooted in the mistaken belief that the defense industry and  the government-industry relationship in defense acquisition fit naturally  into the free enterprise model. Most Americans believe that the defense  industry, as a part of private industry, is equipped to handle any kind of  development or production program. They also by and large distrust  government ‘interference’ in private enterprise. Government and industry  defense managers often go to great lengths to preserve the myth that  large defense programs are developed and produced through the free  enterprise system.” But neither the defense industry nor defense  programs are governed by the free market; “major defense acquisition  programs rarely offer incentives resembling those of the commercial  marketplace.”

Dr. Fox also points out that in private industry, the program manager  concept works well because the managers have genuine decision-making  authority, years of training and experience, and understand the roles and  tactics within government and industry. In contrast, Dr. Fox concludes  that DOD program managers lack the training, experience, and stature of  their private sector counterparts, and are influenced by others in their  service, DOD, and Congress. managers indicated to us that the acquisition process does not enable  them to succeed because it does not empower them to make decisions  on whether the program is ready to proceed forward or even to make  relatively small trade-offs between resources and requirements as  unexpected problems are encountered. Program managers said that they  are also not able to shift personnel resources to respond to changes  affecting the program.

Fox, Defense Acquisition Reform. for their position or forced into the near-term perspective of their tenures.  In this environment, the effectiveness of management can rise and fall on  the strength of individuals; accountability for long-term results is, at best,  elusive.

section_title: Where Do We Go from Here?
paragraphs: In my more than 30 years in the area, I do not know of a time or era when  weapon system programs did not exhibit the same symptoms that they do  today. Similarly, I do not subscribe to the view that the acquisition process  is too rigid and cumbersome. Clearly, this could be the case if every  acquisition followed the same process and strategy without exception. But  they do not. We repeatedly report on programs approved to modify policy  and follow their own process. DOD refers to this as tailoring, and we see  plenty of it.

At this point, we should build on existing reforms—not necessarily by  revisiting the process itself but by augmenting it by tackling incentives. To  do this, we need to look differently at the familiar outcomes of weapon  systems acquisition—such as cost growth, schedule delays, large support  burdens, and reduced buying power. Some of these undesirable  outcomes are clearly due to honest mistakes and unforeseen obstacles.  However, they also occur not because they are inadvertent but because  they are encouraged by the incentive structure. I do not think it is  sufficient to define the problem as an objective process that is broken.  Rather, it is more accurate to view the problem as a sophisticated  process whose consistent results are indicative of its being in equilibrium.  The rules and policies are clear about what to do, but other incentives  force compromises. The persistence of undesirable outcomes such as  cost growth and schedule delays suggests that these are consequences  that participants in the process have been willing to accept.

Drawing on our extensive body of work in weapon systems acquisition, I  have four areas of focus regarding where to go from here. These are not  intended to be all-encompassing, but rather, practical places to start the  hard work of realigning incentives with desired results.

Reinforce desirable principles at the start of new programs: The principles  and practices programs embrace are determined not by policy, but by  decisions. These decisions involve more than the program at hand: they  send signals on what is acceptable. If programs that do not abide by  sound acquisition principles win funding, then seeds of poor outcomes  are planted. The highest point of leverage is at the start of a new  program. Decision makers must ensure that new programs exhibit  desirable principles before they are approved and funded. Programs that  present well-informed acquisition strategies with reasonable and  incremental requirements and reasonable assumptions about available  funding should be given credit for a good business case. As an example,  the Presidential Helicopter, the Armored Multi Purpose Vehicle, the  Enhanced Polar System, and the Ground Combat Vehicle are all  acquisitions estimated to cost at least a billion dollars, in some cases  several billions of dollars, and slated to start in 2014. These could be  viewed as a “freshman” class of acquisitions. There is such a class every  year, and it would be beneficial for DOD and Congress to assess them as  a group to ensure that they embody the right principles and practices.

Identify significant program risks upfront and resource them: Weapon  acquisition programs by their nature involve risks, some much more than  others. The desired state is not zero risk or elimination of all cost growth.  But we can do better than we do now. The primary consequences of risk  are often the need for additional time and money. Yet, when significant  risks are taken, they are often taken under the guise that they are  manageable and that risk mitigation plans are in place. In my experience,  such plans do not set aside time and money to account for the risks  taken. Yet in today’s climate, it is understandable—any sign of weakness  in a program can doom its funding. This needs to change. If programs are  to take significant risks, whether they are technical in nature or related to  an accelerated schedule, these risks should be declared and the resource  consequences acknowledged. Less risky options and potential off-ramps  should be presented as alternatives. Decisions can then be made with full  information, including decisions to accept the risks identified. If the risks  are acknowledged and accepted by DOD and Congress, the program  should be supported.

More closely align budget decisions and program decisions: Because  budget decisions are often made years ahead of program decisions, they  depend on the promises and projections of program sponsors.  Contentious budget battles create incentives for sponsors to be optimistic  and make it hard to change course as projections fade in the face of  information. This is not about bad actors; rather, optimism is a rational  response to the way money flows to programs. Aside from these  consequences, planning ahead to make sure money is available in the  future is a sound practice. I am not sure there is an obvious remedy for  this. But I believe ways to have budget decisions follow program  decisions should be explored, without sacrificing the discipline of  establishing long-term affordability.

Attract, train, and retain acquisition staff and management: Dr. Fox’s book  does an excellent job of laying out the flaws in the current ways DOD  selects, trains, and provides a career path for program managers. I refer  you to these, as they are sound criticisms. We must also think about  supporting people below the program manager who are also instrumental  to program outcomes, including engineers, contracting officers, cost  analysts, testers, and logisticians. There have been initiatives to support  these people, but they have not been consistent over time. The tenure for  acquisition executives is a more challenging prospect in that they  arguably are at the top of their profession and already expert. What can  be done to keep good people in these jobs longer? I am not sure of the  answer, but I believe part of the problem is that the contentious  environment of acquisition grinds good people down at all levels. In top  commercial firms, a new product development is launched with a strong  team, corporate funding support, and a time frame of 5 to 6 years or less.  In DOD, new weapon system developments can take twice as long, have  turnover in key positions, and every year must contend for funding. This  does not necessarily make for an attractive career.

Mr. Chairman, this concludes my statement and I would be happy to  answer any questions.

This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.


Question-summary annotations: questions: question: Have the weapon system investments of the DOD had good outcomes for their price?
answer: The Department of Defense (DOD) must get better outcomes from its weapon system investments, which in recent years have totaled around $1.5 trillion or more.
child_questions: question: How have the outcomes of the investments changed recently?
answer: Recently, there have been some improvements, owing in part to reforms.

question: What aspects of the investments have improved the recent outcomes?
answer: For example, cost growth declined between 2011 and 2012 and a number of programs also improved their buying power by finding efficiencies in development or production and requirements changes.

question: How can the DOD further improve its outcomes?
answer: Still, cost and schedule growth remain significant; 39 percent of fiscal 2012 programs have had unit cost growth of 25 percent or more.

questions: question: How does the acquisition policy of the DOD ensure stability and control of technologies?
answer: DOD's acquisition policy provides a methodological framework for developers to gather knowledge that confirms that their technologies are mature, their designs stable, and their production processes are in control.
child_questions: question: How have acquisition practices been further strengthened?
answer: The Weapon Systems Acquisition Reform Act of 2009 and DOD's recent "Better Buying Power" initiatives introduced significant changes that, when fully implemented, should further strengthen practices that can lead to successful acquisitions. GAO has also made numerous recommendations to improve the acquisition process, based on its extensive work in the area.

question: Have the recent reforms had any significant impact yet?
answer: While recent reforms have benefited individual programs, it is premature to say there is a trend or a corner has been turned.
child_questions: question: To what extent have the reforms been implemented?
answer: The reforms still face implementation challenges and have not yet been institutionalized within the services.

questions: question: Why are procedural overhauls not effective solutions?
answer: Reforms that focus on the methodological procedures of the acquisition process are only partial remedies because they do not address incentives to deviate from sound practices.
child_questions: question: How are sound practices unintentionally negative?
answer: Weapons acquisition is a complicated enterprise, complete with unintended incentives that encourage moving programs forward by delaying testing and employing other problematic practices.

question: How are these unintended incentives fostered?
answer: These incentives stem from several factors. For example, the different participants in the acquisition process impose conflicting demands on weapon programs so that their purpose transcends just filling voids in military capability. Also, the budget process forces funding decisions to be made well in advance of program decisions, which encourages undue optimism about program risks and costs.

question: To what extent does the quick turnover of DOD project management affect the acquisition process?
answer: Finally, DOD program managers' short tenures and limitations in experience and training can foster a short-term focus and put them at a disadvantage with their industry counterparts.

questions: question: To what extent is GAO concerned about the DOD's acquisition of weapon systems?
answer: DOD's acquisition of major weapon systems has been on GAO's high risk list since 1990.

question: How is the acquisition process reviewed?
answer: Over the past 50 years, Congress and DOD have continually explored ways to improve acquisition outcomes, including reforms that have championed sound management practices, such as realistic cost estimating, prototyping, and systems engineering.
child_questions: question: How efficient is the review process at fixing problems in the acquisition process?
answer: Too often, GAO reports on the same kinds of problems today that it did over 20 years ago.

questions: question: What is the topic of the hearing?
answer: The topic of today's hearing is: "25 Years of Acquisition Reform: Where Do We Go From Here?"

question: How does the hearing aim to review DOD performance?
answer: To that end, this testimony discusses (1) the performance of DOD's major defense acquisition program portfolio; (2) the management policies and processes currently in place to guide those acquisitions; (3) the incentives to deviate from otherwise sound acquisition practices; and (4) suggestions to temper these incentives.
child_questions: question: How will the hearing analyze the DOD's acquisition of weapon systems?
answer: This statement draws from GAO's extensive body of work on DOD's acquisition of weapon systems.