Defense Acquisitions: Addressing Incentives is Key to Further Reform Efforts
Released: 2014-04-30T00:00:00

Letter

Trends in DOD’s Portfolio of Major Acquisitions

There can be little doubt that we can—and must—get better outcomes from our weapon system investments. As seen in table 1, the value of these investments in recent years has been on the order of $1.5 trillion or more, making them a significant part of the Federal discretionary budget.

As one can see, cost and schedule growth for DOD’s aggregate portfolio remain significant. For example, when measured against programs’ first full estimates, the total cost of the portfolio has increased by nearly $448 billion with an average delay of 28 months in initial operating capability. Also, as indicated in table 1, 42 percent of programs have had unit cost growth of 25 percent or more. On the other hand, we have recently seen some modest improvements in a large number of programs. For example, 50 of the 80 programs in the portfolio reduced their total acquisition costs over the past year. A number of these programs have improved their buying power by finding efficiencies.

While these modest improvements are encouraging, the enormity of the investment in acquisitions of weapon systems and its role in making U.S. fighting forces capable, warrant continued attention and reform. The potential for savings and for better serving the warfighter argue against complacency.

One Side of Acquisitions: Stated Policy and Process

When one thinks of the weapon system acquisition process, the image that comes to mind is that of the methodological procedure depicted on paper and in flow charts. It is the “how to” side of acquisitions. DOD’s acquisition policy takes the perspective that the goal of acquisition is to obtain quality products that satisfy user needs in a timely manner at a fair and reasonable price. The sequence of events that comprise the process defined in policy reflects principles from disciplines such as systems engineering, as well as lessons learned and past reforms. The body of work we have done on benchmarking best practices has also been reflected in acquisition policy. Recent, significant changes to the policy include those introduced by the Weapon Systems Acquisition Reform Act of 2009 and the Department’s own “Better Buying Power” initiatives which, when fully implemented, should further strengthen practices that can lead to successful acquisitions. The policy provides a framework for developers of new weapons to gather knowledge at appropriate stages that confirms that their technologies are mature, their designs are stable, and their production processes are in control. These steps are intended to ensure that a program will deliver the capabilities required utilizing the resources—cost, schedule, technology, and personnel—available. Successful product developers ensure a high level of knowledge is achieved at key junctures in development. We characterize these junctures as knowledge points. While there can be differences of opinion over some of the specifics of the process, I do not believe there is much debate about the soundness of the basic steps. It is a clear picture of “what to do.”

Table 2 summarizes these steps and best practices, organized around three key knowledge points in a weapon system acquisition.

Our work over the last few years shows that, to the extent reforms like the Weapon Systems Acquisition Reform Act and DOD’s Better Buying Power initiatives are being implemented, they are having a positive effect on individual programs. For example, we found that over 80 percent of the 38 programs included in our annual assessment of weapon programs this year had conducted a “should-cost” analysis—one of DOD’s Better Buying Power initiatives—and reported an anticipated savings of approximately $24 billion, with more than half of this amount to be reallocated to meet other DOD priorities. In addition, we recently reviewed several programs to determine the impact of the Weapon Systems Acquisition Reform Act and found that the programs are: making early tradeoffs among cost, schedule, and technical developing more realistic cost and schedule estimates, increasing the amount of testing during development, and placing greater emphasis on reliability.

These improvements do not yet signify a trend or suggest that a corner has been turned and, in fact, we found in our annual assessment of programs that most are not yet fully following a knowledge-based acquisition approach. The reforms themselves still face implementation challenges, such as staffing and clarity of guidance and will doubtless need refining as experience is gained. We have made a number of recommendations on how DOD can improve implementation of the Weapon Systems Acquisition Reform Act.

To a large extent, the improvements we have seen tend to result from external pressure exerted by higher level offices within DOD on individual programs. In other words, the reforms have not yet been institutionalized within the services. We still see employment of other practices—not prescribed in policy—such as concurrent testing and production, optimistic assumptions, and delayed testing. These are the same kinds of practices that perpetuate the significant cost growth and schedule delays that have persisted in acquisitions through the decades. They share a common dynamic: moving forward with programs before the knowledge needed to reduce risk and make those decisions is sufficient.

We have found that programs proceed through the critical design review without having a stable design, although we have made recommendations on the importance of this review and how to prepare for it. Programs also proceed with testing and production before they are ready. The F-35 Joint Strike Fighter program is a classic example of how concurrency can erode the cost and schedule of an acquisition. Further, some programs are significantly at odds with the acquisition process. Among these I would number the Ballistic Missile Defense System, Littoral Combat Ship, and airships. We also recently reported on the Unmanned Carrier-Launched Airborne Surveillance and Strike program which proposes to complete the main acquisition steps of design, development, testing, manufacturing, and initial fielding before it formally enters the acquisition process.

The fact that programs adopt practices that run counter to what policy and reform call for is evidence of the other pressures and incentives that significantly influence program practices and outcomes. I will turn to these next.

Another Side of Acquisitions: Incentives to Deviate from Sound Practices

An oft-cited quote of David Packard, former Deputy Secretary of Defense, is: “We all know what needs to be done. The question is why aren’t we doing it?” To that point, reforms have been aimed mainly at the “what” versus the “why.” They have championed sound management practices, such as realistic estimating, thorough testing, and accurate reporting. Reforms have also added program decision points, reviews, and reporting requirements to help ensure these practices are used. We need to consider that these reforms mainly address the mechanisms of weapon acquisitions. Seen this way, the practices prescribed in policy are only partial remedies. The acquisition of weapons is much more complex than this and involves very basic and strongly reinforced incentives to pursue weapons that are not always feasible and affordable. Accordingly, rival practices, not normally viewed as good management techniques, comprise an effective stratagem for fielding a weapon because they reduce the risk that the program will be interrupted or called into question.

I will now discuss several factors that illustrate the pressures that create incentives to deviate from sound acquisition management practices.

Mismatch between Requirements and Resources

A key cause of poor acquisition outcomes is the mismatch between the validated capability requirements for a new weapon system and the appropriate systems engineering knowledge, funding, and time that is planned to develop that new system. DOD’s three key decision making processes for acquiring weapon systems—requirements determination, resource allocation, and the acquisition management system—are fragmented, making it difficult for the department to achieve a balanced mix of weapon systems that are achievable and affordable and provide the best military value to the warfighter when the warfighter needs them. In addition, these processes are led by different organizations, making it difficult to hold any one person or organization accountable for saying “no” to an unrealistic requirement or for tempering optimistic cost and schedule estimates. While the department has worked hard to overcome this fragmented decision making paradigm and policies have been written to force more integrated decisions and more accountability, we continue to see programs that have experienced cost and schedule growth. This is because weapon system programs often begin with validated requirements that have not been informed by solid systems engineering practices, often do not represent true “needs” as much as “desires,” have optimistic cost and schedule estimates, and, all too often, are unachievable. Program managers are handed a business case that can be fatally flawed, and usually have no recourse other than to execute it as best they can and therefore cannot be held accountable.

Conflicting Demands

The process of planning and executing the program is (1) shaped by many different participants and (2) far more complex than the seemingly straightforward purchase of equipment to defeat an enemy threat. Collectively, as participants’ needs are translated into actions on weapon programs, the purpose of such programs transcends efficiently filling voids in military capability. Weapons have become integral to policy decisions, definitions of roles and functions, justifications of budget levels and shares, service reputations, influence of oversight organizations, defense spending in localities, the industrial base, and to individual careers. Consequently, the reasons “why” a weapon acquisition program is started are manifold and thus acquisitions do not merely provide technical solutions.

While individual participants see their needs as rational and aligned with the national interest, collectively, these needs create incentives for pushing programs and encouraging undue optimism, parochialism, and other compromises of good judgment. Under these circumstances, persistent performance problems, cost growth, schedule slippage, and difficulties with production and field support cannot all be attributed to errors, lack of expertise, or unforeseeable events. Rather, a level of these problems is embedded as the undesirable, but apparently acceptable, consequence of the process. These problems persist not because they are overlooked or under-regulated, but because they enable more programs to survive and thus more needs to be met. The problems are not the fault of any single participant; they are the collective responsibility of all participants. Thus, the various pressures that accompany the reasons why a program is started can also affect and compromise the practices employed in its acquisition.

Funding Dynamics

There are several characteristics about the way programs are funded that create incentives in decision-making that can run counter to sound acquisition practices. First, there is an important difference between what investments in new products represent for a private firm and for DOD. In a private firm, a decision to invest in a new product, like a new car design, represents an expense. Company funds must be expended that will not provide a revenue return until the product is developed, produced, and sold. Thus, leading companies have an incentive to follow a disciplined approach and acquire requisite knowledge to facilitate successful product development. To do otherwise could have serious economic consequences. In DOD, there can be few consequences if funds are not used efficiently. For example, as has often been the case in the past, agency budgets generally do not fluctuate much year to year and, programs that experience problems tend to eventually receive more funding to get well. Also, in DOD, new products in the form of budget line items can represent revenue. An agency may be able to justify a larger budget if it can win approval for more programs. Thus, weapon system programs can be viewed both as expenditures and revenue generators.

Second, budgets to support major program commitments must be approved well ahead of when the information needed to support the decision is available. Take, for example, a decision to start a new program scheduled for August 2016. Funding for that decision would have to be included in the Fiscal Year 2016 budget. This budget would be submitted to Congress in February 2015—18 months before the program decision review is actually held. DOD would have committed to the funding before the budget request went to Congress. It is likely that the requirements, technologies, and cost estimates for the new program— essential to successful execution—may not be very solid at the time of funding approval. Once the hard-fought budget debates put money on the table for a program, it is very hard to take it away later, when the actual program decision point is reached.

Third, to the extent a program wins funding, the principles and practices it embodies are thus endorsed. So, if a program is funded despite having an unrealistic schedule or requirements, that decision reinforces those characteristics instead of sound acquisition practices. Pressure to make exceptions for programs that do not measure up are rationalized in a number of ways: an urgent threat needs to be met; a production capability needs to be preserved; despite shortfalls, the new system is more capable than the one it is replacing; and the new system’s problems will be fixed in the future. It is the funding approvals that ultimately define acquisition policy.

Industry Relationship

DOD has a unique relationship with the Defense industry that differs from the commercial marketplace. The combination of a single buyer (DOD), a few very large prime contractors in each segment of the industry, and a limited number of weapon programs constitute a structure for doing business that is altogether different from a classic free market. For instance, there is less competition, more regulation, and once a contract is awarded, the contractor has considerable power. Moreover, in the Defense marketplace, the firm and the customer have jointly developed the product and, as we have reported previously, the closer the product comes to production the more the customer becomes invested and the less likely they are to walk away from that investment. While a Defense firm and a military customer may share some of the same goals, certain key goals are different. Defense firms are accountable to their shareholders and can also build constituencies outside the direct business relationship between them and their customers. This relationship does not fit easily into a contract.

J. Ronald Fox, author of Defense Acquisition Reform 1960-2009: An Elusive Goal, sums up the situation as follows. “Many defense acquisition problems are rooted in the mistaken belief that the defense industry and the government-industry relationship in defense acquisition fit naturally into the free enterprise model. Most Americans believe that the defense industry, as a part of private industry, is equipped to handle any kind of development or production program. They also by and large distrust government ‘interference’ in private enterprise. Government and industry defense managers often go to great lengths to preserve the myth that large defense programs are developed and produced through the free enterprise system.” But neither the defense industry nor defense programs are governed by the free market; “major defense acquisition programs rarely offer incentives resembling those of the commercial marketplace.”

The Right People

Dr. Fox also points out that in private industry, the program manager concept works well because the managers have genuine decision-making authority, years of training and experience, and understand the roles and tactics within government and industry. In contrast, Dr. Fox concludes that DOD program managers often lack the training, experience, and stature of their private sector counterparts, and are influenced by others in their service, DOD, and Congress. Other acquisition reform studies over the past decade have highlighted this issue as well. The studies highlight the need for a more professional program manager cadre within each of the military services, and new incentives and improved career opportunities for acquisition personnel. In 2006, we reported that program managers indicated to us that the acquisition process does not enable them to succeed because it does not empower them to make decisions on whether the program is ready to proceed forward or even to make relatively small trade-offs between resources and requirements as unexpected problems are encountered. Program managers said that they are also not able to make personnel shifts to respond to changes affecting the program.

We have also reported on the lack of continuity in the tenure of key acquisition leaders across the timeframe of individual programs. A major acquisition can have multiple program managers during product development. For example, the F-35 Joint Strike Fighter program has had six different program managers since it was approved to start development in 2001. Other key positions throughout the acquisition chain of command also turn over frequently. For example, the average tenure of the Under Secretary of Defense for Acquisition, Technology and Logistics since the position was established in 1986 has been only about 22 months. Consequently, DOD acquisition executives do not necessarily stay in their positions long enough to develop the needed long-term perspective or to effectively change traditional incentives. Moreover, their decisions can be overruled through the cooperative actions of other acquisition participants. The effectiveness of reforms to the acquisition process depends in large measure on a cadre of good people who may be inadequately prepared for their position or forced into the near-term perspective of their tenures. In this environment, the effectiveness of management can rise and fall on the strength of individuals; accountability for long term results is, at best, elusive.

Where Do We Go From Here?

I do not necessarily subscribe to the view that the acquisition process is too rigid and cumbersome. Clearly, this could be the case if every acquisition followed the same process and strategy without exception, but they do not. We repeatedly report on programs where modifications of the process are approved. DOD refers to this as tailoring, and we see plenty of it.

While one should always be looking to improve the process and make it more efficient, at this point, the focus should be to build on existing reforms by holding decision makers more accountable, tackling existing incentives, and providing new ones. To do this, we need to look differently at the familiar outcomes of weapon system acquisitions—such as cost growth, schedule delays, large support burdens, and reduced buying power. Some of these undesirable outcomes are clearly due to honest mistakes and unforeseen obstacles. However, they also occur not because they are inadvertent but because they are encouraged by the incentive structure. I do not think it is sufficient to define the problem as an objective process that is broken. Rather, it is more accurate to view the problem as a sophisticated process whose consistent results are indicative of its being in equilibrium. The rules and policies are clear about what to do, but other incentives force compromises. The persistence of undesirable program outcomes suggests that these are consequences that participants in the process have been willing to accept.

Drawing on our extensive body of work in weapon system acquisition, there are six areas of focus regarding where to go from here. These are not intended to be all-encompassing, but rather, practical places to start the hard work of realigning incentives with desired results.

Hold decision makers accountable from top to bottom: Our work over the years benchmarking best practices at leading commercial product developers and manufacturers has yielded a wide range of best practices for efficiently and quickly developing new products to meet market needs. Firms we visited described an integrated process for establishing product requirements, making tradeoffs among cost and product performance well ahead of a decision to begin product development, and ensuring that all decision makers—requirements setters, product developers, and finance—agree to and are held accountable for the business case presented to the program manager for execution of a new product’s development. These firms had trained professionals as program managers with backgrounds in technical fields such as engineering and various aspects of project management. Once empowered with an achievable, executable business case, they were in charge of product development from beginning to end. Therefore, they could be held accountable for meeting product development cost, schedule, and performance targets.

Today, getting managers to make hard decisions, when necessary, and say no to those that push unrealistic or unaffordable plans continues to be a challenge because the critical processes to acquire a new weapon system are segregated, independent, and have different goals. DOD must be open to examining best practices and implementing new rules to really integrate the processes into one and holding all communities accountable for decisions. I do not pretend to have all the answers on how to change the current environment, but it is clear that top decision makers cannot be held accountable to work in concert on such large and critical investments unless they begin with an executable business case. Congressional and DOD leadership must be in concert on this.

Attract, train, and keep acquisition staff and management: Dr. Fox’s book does an excellent job of laying out the flaws in the current way DOD selects, trains, and provides a career path for program managers. I refer you to this book, as it provides sound criticisms. We must also think about supporting people below the program manager who are also instrumental to program outcomes, including engineers, contracting officers, cost analysts, testers, and logisticians. There have been initiatives aimed at program managers and acquisition personnel, but they have not been consistent over time. RAND, for example, recently analyzed program manager tenure in DOD and found that the intent of policies designed to lengthen tenure may not have been achieved and no enforcement mechanism has been readily apparent over time. RAND indicates this could be because of the fundamental conflict that exists between what military officers need to do to be promoted and their tenure as program managers. Unless these two things are aligned, such that experience and tenure in an acquisition program can be advantageous for promotion, then it appears unlikely that tenure policies will consistently yield positive results. The tenure for acquisition executives is a more challenging prospect in that they arguably are at the top of their profession and already expert. What can be done to keep good people in these jobs longer?

I am not sure of the answer, but I believe part of the problem is that the contentious environment of acquisition grinds good people down at all levels. In top commercial firms, a new product development is launched with a strong team, corporate funding support, and a timeframe of 5 to 6 years or less. In DOD, new weapon system development can take twice as long, have turnover in key positions, and every year must contend for funding. This does not necessarily make for an attractive career. Several years ago, the Defense Acquisition Performance Assessment Panel recommended establishing the military department’s service acquisition executives as a five-year, fixed-term position to add leadership continuity and stability to the acquisition process. I believe something like this recommendation is worth considering. And perhaps the military services should examine the current career track for acquisition officers to ensure it provides appropriate training, rewards, and opportunities for advancement.

Reinforce desirable principles at the start of new programs: The principles and practices programs embrace are determined not by policy, but by decisions. These decisions involve more than the program at hand: they send signals as to what is acceptable. If programs that do not abide by sound acquisition principles win funding, then seeds of poor outcomes are planted. The highest point of leverage is at the start of a new program. Decision makers must ensure that new programs exhibit desirable principles before they are approved and funded. Programs that present well informed acquisition strategies with reasonable and incremental requirements and reasonable assumptions about available funding should be given credit for a good business case. As an example, the Presidential Helicopter, Armored Multi Purpose Vehicle, and Enhanced Polar System are all acquisitions slated to start in 2014, with development estimates currently ranging from nearly $1 billion to over $2.5 billion. These and other programs expected to begin system development in 2014 could be viewed as a “freshman” class of acquisitions. It would be beneficial for DOD and Congress to assess them as a group to ensure that they embody the right principles and practices. Recent action by DOD to terminate the Army’s Ground Combat Vehicle program, which was slated to start this year, and instead focus efforts on selected science and technology activities reinforces sound principles. On the other hand, approving the Unmanned Carrier-Launched Airborne Surveillance and Strike program despite its running counter to sound principles sends a conflicting message.

Identify significant program risks upfront and resource them: Weapon acquisition programs by their nature involve risks, some much more than others. The desired state is not zero risk or elimination of all cost growth. But we can do better than we do now. The primary consequences of risk are often the need for additional time and money. Yet, when significant risks are taken, they are often taken under the guise that they are manageable and that risk mitigation plans are in place. In my experience, such plans do not set aside time and money to account for the risks taken. Yet in today’s climate, it is understandable—any sign of weakness in a program can doom its funding. This needs to change. If programs are to take significant risks, whether they be technical in nature or related to an accelerated schedule, these risks should be declared and the resource consequences acknowledged. Less risky options and potential off ramps should be presented as alternatives. Decisions can then be made with full information, including decisions to accept the risks identified. If the risks are acknowledged and accepted by DOD and Congress, the program should be supported.

A potential way to reduce the risks taken in acquisition programs is to address the way in which DOD leverages its science and technology enterprise. Leading commercial companies save time and money by separating technology development from product development and fully developing technologies before introducing them into the design of a system. These companies develop technology to a high level of maturity in a science and technology environment which is more conducive to the ups and downs normally associated with the discovery process. This affords the opportunity to gain significant knowledge before committing to product development and has helped companies reduce costs and time from product launch to fielding. Although DOD’s science and technology enterprise is engaged in developing technology, there are organizational, funding, and process impediments which make it difficult to bring technologies into acquisition programs. For example, it is easier to move immature technologies into weapon system programs because they tend to attract bigger budgets than science and technology projects. Creating stronger and more uniform incentives that encourage the development of technologies in the right environment to reduce the cost of later changes, and encourage the technology and acquisition communities to work more closely together to deliver the right technologies at the right time would be beneficial.

More closely align budget decisions and program decisions: Because budget decisions are often made years ahead of program decisions, they depend on the promises and projections of program sponsors. Contentious budget battles create incentives for sponsors to be optimistic and make it hard to change course as projections fade in the face of information. This is not about bad actors; rather, optimism is a rational response to the way money flows to programs. Aside from these consequences, planning ahead to make sure money is available in the future is a sound practice. I am not sure there is an obvious remedy for this. But, I believe ways to have budget decisions follow program decisions should be explored, without sacrificing the discipline of establishing long-term affordability.

Investigate other tools to improve program outcomes: There are ways to structure an acquisition program that would create opportunities for better outcomes. Key among these are: limits on development time (time certain development of 5 years), which limits the scope of the development task; evolutionary or incremental product development, wherein the initial increment of a new weapon system adds value for the warfighter, is delivered to the field faster, and can be followed with block upgrades as technologies and funding present themselves; and strategies that focus more on incentivizing overall cost reduction over profit limitation. DOD should investigate the potential of these concepts as it structures and manages programs moving forward. Central to opening an environment for these tools is the need to focus on requirements that are well understood and manageable. This would allow the department to offer contracts that place more cost risk on the contractor and less on the government. A prime example of this is the KC-46 Tanker program that is being developed under a fixed-price development contract with incentives for holding cost down. The government and industry felt comfortable with that arrangement specifically because it was an incremental program based on a commercial airframe. The first development program is to militarize a commercial aircraft to replace a portion of the existing KC-135 fleet. Future increments may be approved to replace the rest of the KC- 135 fleet and the KC-10 fleet and provides DOD an opportunity to include the new technologies. Also, the contractor had significant systems engineering knowledge about the design and the ability to meet the requirements. A word of caution: if time certain development (e.g., 5 years), incremental acquisition strategies, and contracts that incentivize cost reduction over profit limitations are to be explored, the government will need to examine whether they have the contract management and negotiation expertise to do this. DOD has begun to examine ways to strengthen contract incentives and restructure profit regulations through its Better Buying Power initiatives; however, it is too soon to tell whether these efforts will lead to needed improvements.

Mr. Chairman, this concludes my statement and I would be happy to answer any questions.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.