Defense Acquisitions: DOD Must Prioritize Its Weapon System Acquisitions and Balance Them with Available Resources
Released: 2009-03-18T00:00:00

Letter

Background

For almost two decades, we have reported on pervasive and long-standing weaknesses in DOD’s business operations. In January 2009, we released our high-risk series update for the 111th Congress. This series emphasizes federal programs and operations that are at high risk because of vulnerabilities to fraud, waste, abuse, and mismanagement and has also evolved to draw attention to areas associated with broad-based transformation needed to achieve greater efficiency, effectiveness, and sustainability. Solutions to high-risk problems offer the potential to save billions of dollars, dramatically improve service to the public, strengthen confidence and trust in the performance and accountability of the U.S. government, and ensure the ability of government to deliver on its promises. Since our high-risk program began, the government has taken these problems seriously and has made progress toward correcting them. Of the 30 high-risk areas identified by GAO across the government, DOD bears sole responsibility for 8 high-risk areas, including weapon systems acquisition, and shares responsibility for 7 other high-risk areas (see table 1).

In addition to monitoring these high-risk areas, we also monitor actions that DOD has taken in response to our findings, conclusions, and recommendations. During fiscal years 2001 through 2007, we issued 637 reports to DOD that included a total of 2,726 recommendations. In December 2008, we reported to this committee on the implementation status of these recommendations and related financial accomplishments. As of October 2008, 1,682 or 62 percent of the recommendations we made were reported as were closed and implemented, 758 or 28 percent were open, and 286 or 10 percent were closed, but not implemented for a variety of reasons. Consistent with past experience that shows it takes agencies some time to implement recommendations, we found most recommendations from fiscal year 2001 have been implemented while most recommendations from fiscal year 2007 remain open. During this same period, we recorded over $89 billion in financial benefits associated with our work involving DOD. Besides financial accomplishments, our recommendations also produce many nonfinancial benefits and accomplishments, such as DOD actions taken to improve operations or management oversight. Both types of benefits result from our efforts to provide information to the Congress that helped to (1) change laws and regulations, (2) improve services to the public, and (3) promote sound agency and governmentwide management.

For fiscal year 2007, 74 of our 313 recommendations to DOD were related to improving weapon system acquisition programs. In addition, for fiscal year 2007, we reported $2.6 billion in financial benefits related to weapon system acquisition programs. The financial benefits claimed result from the actions taken by Congress or DOD that are based on findings, conclusions, or recommendations contained in our products. Such actions include congressional reductions to the President’s annual budget requests, cost reductions due to greater efficiency, or cost reductions due to program cancellations or program delays. For example, the fiscal year 2007 budget request for the Army’s Future Combat System was reduced by $254 million based in part on our testimony about the program’s development risks. Over the next 5 years, DOD plans to spend more than $357 billion on the development and procurement of major defense acquisition programs. We will continue to seek to improve the efficiency and effectiveness of DOD’s weapon system investments through our work on individual programs and crosscutting areas that affect acquisition outcomes.

Failure to Match Requirements with Technology and Other Resources Underlie Poor Weapon Program Outcomes and Undermine Accountability

Over the past several years our work has highlighted a number of underlying systemic causes for cost growth and schedule delays at both the strategic and program levels. At the strategic level, DOD’s processes for identifying warfighter needs, allocating resources, and developing and procuring weapon systems—which together define DOD’s overall weapon system investment strategy—are fragmented. As a result, DOD fails to effectively address joint warfighting needs and commits to more programs than it has resources for, thus creating unhealthy competition for funding. At the program level, a military service typically establishes and DOD approves a business case containing requirements that are not fully understood and cost and schedule estimates that are based on overly optimistic assumptions rather than on sufficient knowledge. Once a program begins, it too often moves forward with inadequate technology, design, testing, and manufacturing knowledge, making it impossible to successfully execute the program within established cost, schedule, and performance targets. Furthermore, DOD officials are rarely held accountable for poor decisions or poor program outcomes.

DOD Lacks an Integrated Approach to Balance Weapon System Investments

At the strategic level, DOD largely continues to define warfighting needs and make investment decisions on a service-by-service and individual platform basis, using fragmented decision-making processes. This approach makes it difficult for the department to achieve a balanced mix of weapon systems that are affordable and feasible and that provide the best military value to the joint warfighter. In contrast, we have found that successful commercial enterprises use an integrated portfolio management approach to focus early investment decisions on products collectively at the enterprise level and ensure that there is a sound basis to justify the commitment of resources. By following a disciplined, integrated process—during which the relative pros and cons of competing product proposals are assessed based on strategic objectives, customer needs, and available resources, and where tough decisions about which investments to pursue and not to pursue are made—companies minimize duplication between business units, move away from organizational stovepipes, and effectively support each new development program. To be effective, integrated portfolio management must have strong, committed leadership; empowered portfolio managers; and accountability at all levels of the organization.

DOD determines its capability needs through the Joint Capabilities and Integration Development System (JCIDS). While JCIDS provides a framework for reviewing and validating needs, it does not adequately prioritize those needs from a joint, departmentwide perspective and lacks the agility to meet changing warfighter demands. We recently reviewed JCIDS documentation related to new capability proposals and found that almost 70 percent were sponsored by the military services with little involvement from the joint community, including the combatant commands, which are responsible for planning and carrying out military operations. By continuing to rely on capability needs defined primarily by the services, DOD may be losing opportunities for improving joint warfighting capabilities and reducing the duplication of capabilities in some areas. The JCIDS process has also proven to be lengthy and cumbersome—taking on average up to 10 months to validate a need—thus undermining the department’s efforts to effectively respond to the needs of the warfighter, especially those needs that are near term. Furthermore, the vast majority of capability proposals that enter the JCIDS process are validated or approved without accounting for the resources or technologies that will be needed to acquire the desired capabilities. Ultimately, the process produces more demand for new weapon system programs than available resources can support.

The funding of proposed programs takes place through a separate process, the department’s Planning, Programming, Budgeting, and Execution (PPBE) system, which is not fully synchronized with JCIDS. While JCIDS is a continuous, need-driven process that unfolds in response to capability proposals as they are submitted by sponsors, PPBE is a calendar-driven process comprising phases occurring over a 2-year cycle, which can lead to resource decisions for proposed programs that may occur several years later. We recently reviewed the effect of the PPBE process on major defense acquisition programs and found that the process does not produce an accurate picture of the department’s resource needs for weapon system programs. The cost of many of the programs we reviewed exceeded the funding levels planned for and reflected in the Future Years Defense Program (FYDP)—the department’s long-term investment strategy (see fig. 1). Rather than limit the number and size of programs or adjust requirements, DOD opts to push the real costs of programs to the future. With too many programs under way for the available resources and high cost growth occurring in many programs, the department must make up for funding shortfalls by shifting funds from one program to pay for another, reducing system capabilities, cutting procurement quantities, or in rare cases terminating programs. Such actions not only create instability in DOD’s weapon system portfolio, they further obscure the true future costs of current commitments, making it difficult to make informed investment decisions.

Initiating Programs with Inadequate Knowledge of Requirements and Resources Often Results in Poor Outcomes

At the program level, the key cause of poor outcomes is the approval of programs with business cases that contain inadequate knowledge about requirements and the resources—funding, time, technologies, and people—needed to execute them. Our work in best practices has found that an executable business case for a program demonstrated evidence that (1) the identified needs are real and necessary and that they can best be met with the chosen concept and (2) the chosen concept can be developed and produced within existing resources. Over the past several years, we have found no evidence of the widespread adoption of such an approach for major acquisition programs in the department. Our annual assessments of major weapon systems have consistently found that the vast majority of programs began system development without mature technologies and moved into system demonstration without design stability.

The chief reason for these problems is the encouragement within the acquisition environment of overly ambitious and lengthy product developments that embody too many technical unknowns and not enough knowledge about the performance and production risks they entail. The knowledge gaps are largely the result of a lack of early and disciplined systems engineering analysis of a weapon system’s requirements prior to beginning system development. Systems engineering translates customer needs into specific product requirements for which requisite technological, software, engineering, and production capabilities can be identified through requirements analysis, design, and testing. Early systems engineering provides the knowledge a product developer needs to identify and resolve performance and resource gaps before product development begins by either reducing requirements, deferring them to the future, or increasing the estimated cost for the weapon system’s development. Because the government often does not perform the proper up-front requirements analysis to determine whether the program will meet its needs, significant contract cost increases can and do occur as the scope of the requirements changes or becomes better understood by the government and contractor. Not only does DOD not conduct disciplined systems engineering prior to the beginning of system development, it has allowed new requirements to be added well into the acquisition cycle. We have reported on the negative effect that poor systems engineering practices have had on several programs, such as the Global Hawk Unmanned Aircraft System, F-22A, Expeditionary Fighting Vehicle, and Joint Air-to-Surface Standoff Missile.

With high levels of uncertainty about requirements, technologies, and design, program cost estimates and related funding needs are often understated, effectively setting programs up for cost and schedule growth. We recently assessed the service and independent cost estimates for 20 major weapon system programs and found that while the independent estimates were somewhat higher, both estimates were too low in most cases. In some of the programs we reviewed, cost estimates have been off by billions of dollars. For example, the Army’s initial cost estimate for the development of the Future Combat System (FCS) was about $20 billion, while DOD’s Cost Analysis and Improvement Group’s estimate was $27 billion. The department began the program using the $20 billion estimate, but development costs for the FCS are now estimated to be $28 billion and the program is still dealing with significant technical risk. Estimates this far off the mark do not provide the necessary foundation for sufficient funding commitments and realistic long-term planning.

The programs we reviewed frequently lacked the knowledge needed to develop realistic cost estimates. For example, program Cost Analysis Requirements Description documents—used to build the program cost estimate—often lack sufficient detail about planned program content for developing sound cost estimates. Without this knowledge, cost estimators must rely heavily on parametric analysis and assumptions about system requirements, technologies, design maturity, and the time and funding needed. A cost estimate is then usually presented to decision makers as a single, or point, estimate that is expected to represent the most likely cost of the program but provides no information about the range of risk and uncertainty or level of confidence associated with the estimate.

Lack of Accountability for Making Weapon System Decisions Hinders Achieving Successful Outcomes

DOD’s requirements, resource allocation, and acquisition processes are led by different organizations, thus making it difficult to hold any one person or organization accountable for saying no to a proposed program or for ensuring that the department’s portfolio of programs is balanced. DOD’s 2006 Defense Acquisition Performance Assessment study observed that these processes are not connected organizationally at any level below the Deputy Secretary of Defense and concluded that this weak structure induces instability and inhibits accountability. Frequent turnover in leadership positions in the department exacerbates the problem. The average tenure, for example, of the Under Secretary of Defense for Acquisition, Technology and Logistics over the past 22 years has been only about 20 months.

When DOD’s strategic processes fail to balance needs with resources and allow unsound, unexecutable programs to move forward, program managers cannot be held accountable when the programs they are handed already have a low probability of success. Program managers are also not empowered to make go or no-go decisions, have little control over funding, cannot veto new requirements, and have little authority over staffing. At the same time, program managers frequently change during a program’s development, making it difficult to hold them accountable for the business cases that they are entrusted to manage and deliver.

Recent DOD Policy Changes Could Improve Future Performance of Weapon System Programs

DOD understands many of the problems that affect acquisition programs and has recently taken steps to remedy them. It has revised its acquisition policy and introduced several initiatives based in part on direction from Congress and recommendations from GAO that could provide a foundation for establishing sound, knowledge-based business cases for individual acquisition programs. However, to improve outcomes, DOD must ensure that its policy changes are consistently implemented and reflected in decisions on individual programs—not only new program starts but also ongoing programs. In the past, inconsistent implementation of existing policy has hindered DOD’s efforts to execute acquisition programs effectively. Moreover, while policy improvements are necessary, they may be insufficient unless the broader strategic issues associated with the department’s fragmented approach to managing its portfolio of weapon system investments are also addressed.

In December 2008, DOD revised its policy governing major defense acquisition programs in ways intended to provide key department leaders with the knowledge needed to make informed decisions before a program starts and to maintain disciplined development once it begins. The revised policy recommends the completion of key systems engineering activities before the start of development, includes a requirement for early prototyping, and establishes review boards to evaluate the effect of potential requirements changes on ongoing programs. The policy also establishes early reviews for programs going through the pre–systems acquisition phase. In the past, DOD’s acquisition policy may have encouraged programs to rush into systems development without sufficient knowledge, in part because no formal milestone reviews were required before system development. If implemented, these policy changes could help programs replace risk with knowledge, thereby increasing the chances of developing weapon systems within cost and schedule targets while meeting user needs.

As part of its strategy for enhancing the roles of program managers in major weapon system acquisitions, DOD has established a policy that requires formal agreements among program managers, their acquisition executives, and the user community setting forth common program goals. According to DOD, these agreements are intended to be binding and to detail the progress the program is expected to make during the year and the resources the program will be provided to reach these goals. DOD also requires program managers to sign tenure agreements so that their tenure will correspond to the next major milestone review closest to 4 years. DOD acknowledges that any actions taken to improve accountability must be based on a foundation whereby program managers can launch and manage programs toward successful performance, rather than focusing on maintaining support and funding for individual programs. DOD acquisition leaders have also stated that any improvements to program managers’ performance depend on the department’s ability to promote requirements and resource stability over weapon system investments.

Over the past few years, DOD has also been testing portfolio management approaches in selected capability areas—command and control, net- centric operations, battlespace awareness, and logistics—to facilitate more strategic choices for resource allocation across programs. The department recently formalized the concept of capability portfolio management, issuing a directive in 2008 that established policy and assigned responsibilities for portfolio management. The directive established nine joint capability-area portfolios, each to be managed by civilian and military coleads. While the portfolios have no independent decision-making authority over requirements determination and resource allocation, according to some DOD officials, they provided key input and recommendations in this year’s budget process. However, without portfolios in which managers have authority and control over resources, the department is at risk of continuing to develop and acquire systems in a stovepiped manner and of not knowing if its systems are being developed within available resources.

Concluding Observations on What Remains to Be Done

A broad consensus exists that weapon system problems are serious and that their resolution is overdue. With the federal budget under increasing strain from the nation’s economic crisis and long-term fiscal challenges looming, the time for change is now. Achieving successful and lasting improvements in weapon program outcomes will require changes to the overall acquisition environment and the incentives that drive it. Acquisition problems are likely to persist until DOD’s approach to managing its weapon system portfolio (1) prioritizes needs with available resources, thus eliminating unhealthy competition for funding and the incentives for making programs look affordable when they are not; (2) ensures that programs that are started can be executed by matching requirements with resources; and (3) balances the near-term needs of the joint warfighter with the long-term need to modernize the force. Establishing a single point of accountability for managing DOD’s weapon system portfolio could help the department make these changes. Congress can also support change though its own decisions about whether to authorize and appropriate funds for individual weapon programs.

From an acquisition policy perspective, DOD is off to a good start with its recent policy revisions. However, DOD could do more in this regard too by requiring new programs to have manageable development cycles, requiring programs to establish knowledge-based cost and schedule estimates, and requiring contractors to perform detailed systems engineering analysis before proceeding to system development. Limiting the length of development cycles would make it easier to more accurately estimate costs, predict the future funding needs, effectively allocate resources, and hold decision makers accountable. DOD’s conventional acquisition process often requires as many as 10 or 15 years to get from program start to production as programs strive to provide revolutionary capability. Constraining cycle times to 5 or 6 years would force programs to adopt more realistic requirements and lend itself to fully funding programs to completion, thereby increasing stability and the likelihood that capability can be delivered to the warfighter within established time frames and available resources.

Recently proposed acquisition reform legislation addresses some of these areas. Provisions increasing the emphasis on systems engineering, requiring early preliminary design reviews, and strengthening independent cost estimates and technology readiness assessments should make the critical front end of the acquisition process more disciplined. Establishing a termination criterion for critical cost breaches could help prevent the acceptance of unrealistic cost estimates at program initiation. Having greater combatant command involvement in determining requirements and greater consultation between the requirements, budget, and acquisition processes could help improve the department’s efforts to balance its portfolio of weapon system programs. However, while legislation and policy revisions may lead to improvements, they will not be effective without changes to the overall acquisition environment. The department has tough decisions to make about its weapon systems and portfolio, and stakeholders, including the DOD Comptroller, military services, industry, and Congress, have to play a constructive role in the process toward change. It will also require strong leadership and accountability within the department.

Mr. Chairman, this concludes my prepared statement. I would be happy to answer any questions you may have at this time.

Contacts and Acknowledgments

For further information about this statement, please contact Michael J. Sullivan at (202) 512-4841 or sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. Individuals who made key contributions to this statement include Ann Borseth, Dayna Foster, Matt Lea, Susan Neill, John Oppenheim, Ken Patton, Sharon Pickup, Ron Schwenn, Charlie Shivers, Bruce Thomas, and Alyssa Weir.

Related GAO Products

Defense Acquisitions: DOD Must Balance Its Needs with Available Resources and Follow an Incremental Approach to Acquiring Weapon Systems. GAO-09-413T. Washington, D.C.: March 3, 2009.

Defense Acquisitions: Perspectives on Potential Changes to DOD’s Acquisition Management Framework. GAO-09-295R. Washington, D.C.: February 27, 2009.

Defense Management: Actions Needed to Overcome Long-standing Challenges with Weapon Systems Acquisition and Service Contract Management. GAO-09-362T. Washington, D.C.: February 11, 2009.

Status of Recommendations to the Department of Defense (Fiscal Years 2001-2007). GAO-09-201R. Washington, D.C.: December 11, 2008.

Defense Acquisitions: Fundamental Changes Are Needed to Improve Weapon Program Outcomes. GAO-08-1159T. Washington, D.C.: September 25, 2008.

Defense Acquisitions: DOD’s Requirements Determination Process Has Not Been Effective in Prioritizing Joint Capabilities. GAO-08-1060. Washington, D.C.: September 25, 2008.

Defense Acquisitions: A Knowledge-Based Funding Approach Could Improve Major Weapon System Program Outcomes. GAO-08-619. Washington, D.C.: July 2, 2008.

Defense Acquisitions: Better Weapon Program Outcomes Require Discipline, Accountability, and Fundamental Changes in the Acquisition Environment. GAO-08-782T. Washington, D.C.: June 3, 2008.

Defense Acquisitions: Assessments of Selected Weapon Programs. GAO-08-467SP. Washington, D.C.: March 31, 2008.

Best Practices: Increased Focus on Requirements and Oversight Needed to Improve DOD’s Acquisition Environment and Weapon System Quality. GAO-08-294. Washington, D.C.: February 1, 2008.

Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD’s Acquisition Outcomes. GAO-07-388. Washington, D.C.: March 30, 2007.

Defense Acquisitions: Major Weapon Systems Continue to Experience Cost and Schedule Problems under DOD’s Revised Policy. GAO-06-368. Washington, D.C.: April 13, 2006.

Best Practices: Better Support of Weapon System Program Managers Needed to Improve Outcomes. GAO-06-110. Washington, D.C.: November 1, 2005.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.