Defense Acquisitions: DOD Must Balance Its Needs with Available Resources and Follow an Incremental Approach to Acquiring Weapon Systems
Published: 2009-03-03T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since 1990, GAO has consistently designated the Department of Defense's (DOD) management of its major weapon acquisitions as a high-risk area. A broad consensus exists that weapon system problems are serious, but efforts at reform have had limited impact. Last year, GAO reported that DOD's portfolio of weapon programs experienced cost growth of $295 billion from first estimates, were delayed by an average of 21 months, and delivered fewer quantities and capabilities to the warfighter than originally planned. At a time when DOD faces increased fiscal pressures from ongoing operations in Iraq and Afghanistan, and the federal budget is strained by a growing number of priorities, it is critical that the department effectively manage its substantial investment in weapon system programs. Every dollar wasted or used inefficiently on acquiring weapon systems means that less money is available for the government's other important budgetary demands. This testimony describes the systemic problems that contribute to the cost, schedule, and performance problems in weapon system programs, recent actions that DOD has taken to address these problems, proposed reform legislation that the committee recently introduced, and additional steps needed to improve future performance of acquisition programs. The testimony is drawn from GAO's body of work on DOD's acquisition, requirements, and funding processes.
section_title: What GAO Found
paragraphs: For several years, GAO's work has highlighted a number of strategic- and program-level causes for cost, schedule, and performance problems in DOD's weapon system programs. At the strategic level, DOD's processes for identifying warfighter needs, allocating resources, and developing and procuring weapon systems, which together define the department's overall weapon system investment strategy, are fragmented. As a result, DOD fails to balance the competing needs of the services with those of the joint warfighter and commits to more programs than resources can support. At the program level, DOD allows programs to begin development without a full understanding of requirements and the resources needed to execute them. The lack of early systems engineering, acceptance of unreliable cost estimates based on overly optimistic assumptions, failure to commit full funding, and the addition of new requirements well into the acquisition cycle all contribute to poor outcomes. Moreover, DOD officials are rarely held accountable for poor decisions or poor program outcomes. Recognizing the need for more discipline in weapon systems acquisition and to implement Congressional direction, DOD recently revised its policy and introduced several initiatives. The revised policy, if implemented properly, could provide a foundation for developing individual acquisition programs with sound, knowledge-based business cases. The policy recommends the completion of key systems engineering activities, establishes early milestone reviews, requires competitive prototyping, and establishes review boards to manage potential requirements changes to ongoing programs. The committee's proposed reform legislation should lead to further improvements in outcomes. Improved systems engineering, early preliminary design reviews, and strengthened independent cost estimates and technology readiness assessments should make the critical front end of the acquisition process more disciplined. Establishing a termination criterion for critical cost breaches could help prevent the acceptance of unrealistic cost estimates at program initiation. Having greater combatant command involvement in determining requirements and greater consultation between the requirements, budget, and acquisition processes could help improve the department's efforts to balance its portfolio of weapon system programs. Legislation and policy revisions may lead to improvements but cannot work effectively without changes to the overall acquisition environment and the incentives that drive it. Resisting the urge to achieve revolutionary but unachievable capabilities, allowing technologies to mature in the technology base before bringing them onto programs, ensuring requirements are well-defined and doable, and instituting shorter development cycles would all make it easier to estimate costs accurately, and then predict funding needs and allocate resources effectively. These measures will only succeed if the department balances its portfolio and adopts an incremental approach to developing and procuring weapon systems.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the Department of Defense’s  (DOD) management of its major weapon system acquisitions—an area that  has been on GAO’s high-risk list since 1990. Prior to and since that time,  Congress and DOD have continually explored ways to improve acquisition  outcomes without significant results. While the technological  sophistication of DOD weapon systems is unparalleled, major weapon  programs continue to cost more, take longer to complete, and deliver  fewer quantities and capabilities than originally planned. Last year we  reported that the cumulative cost growth in DOD’s portfolio of 95 major  defense acquisition programs was $295 billion from first estimates and the  average delay in delivering promised capabilities to the warfighter was 21  months. Clearly, some problems are to be expected in developing weapon  systems given the technical risks and complexities involved. However, all  too often we have found that cost and schedule problems are rooted in  poor planning, execution, and oversight.
Investment in weapon systems is now at its highest level in two decades,  and DOD plans to spend more than $357 billion over the next 5 years on  major defense acquisition programs. Effective management of this  substantial investment is critical as competition for funding has increased  dramatically within the department and across the government. DOD faces  a number of fiscal pressures: the ongoing military campaigns in  Afghanistan and Iraq, rising personnel costs, and the rebuilding and  modernization of the force. In addition, the economic and fiscal crises now  facing the nation have required unprecedented spending by the federal  government, and budget deficits are projected to remain high for many  years to come. At a time when the federal budget is strained by spending  needs for a growing number of national priorities, it is imperative that  DOD get the best value for every dollar it invests in weapon system  programs. Every dollar wasted during the development and acquisition of  weapon systems is money not available for other priorities within DOD  and elsewhere in the government.
Today, I will discuss (1) the systemic problems that have contributed to  cost, schedule, and performance problems in DOD’s acquisition of major  weapon systems; (2) recent actions the department has taken to address  these problems; (3) our observations on the committee’s proposed  acquisition reform legislation; and (4) steps that Congress and the  department need to take to improve the future performance of acquisition  programs. The statement includes findings from our July 2008 report on a  knowledge-based funding approach and February 2009 report on potential  changes to DOD’s acquisition management framework. It also draws from  our extensive body of work on DOD’s acquisition of weapon systems. A  list of our key products is provided at the end of this statement. This work  was conducted in accordance with generally accepted government  auditing standards. Those standards require that we plan and perform the  audit to obtain sufficient, appropriate evidence to provide a reasonable  basis for our findings and conclusions based on our audit objectives. We  believe that the evidence obtained provides a reasonable basis for our  findings and conclusions based on our audit objectives.
subsections: section_title: Fragmented Investment Decision Making, Unexecutable Programs, and Lack of Accountability Underlie Poor Acquisition Outcomes
paragraphs: Over the past several years our work has highlighted a number of  underlying systemic causes for cost growth and schedule delays at both  the strategic and program levels. At the strategic level, DOD’s processes  for identifying warfighter needs, allocating resources, and developing and  procuring weapon systems—which together define DOD’s overall weapon  system investment strategy—are fragmented. As a result, DOD fails to  effectively address joint warfighting needs and commits to more programs  than it has resources for, thus creating unhealthy competition for funding.  At the program level, a military service typically establishes and DOD  approves a business case containing requirements that are not fully  understood and cost and schedule estimates that are based on overly  optimistic assumptions rather than on sufficient knowledge. Once a  program begins, it too often moves forward with inadequate technology,  design, testing, and manufacturing knowledge, making it impossible to  successfully execute the program within established cost, schedule, and  performance targets. Furthermore, DOD officials are rarely held  accountable for poor decisions or poor program outcomes.
subsections: section_title: DOD Lacks an Integrated Approach to Balance Weapon System Investments
paragraphs: At the strategic level, DOD largely continues to define warfighting needs  and make investment decisions on a service-by-service and individual  platform basis, using fragmented decision-making processes. This  approach makes it difficult for the department to achieve a balanced mix  of weapon systems that are affordable and feasible and that provide the  best military value to the joint warfighter. In contrast, we have found that  successful commercial enterprises use an integrated portfolio  management approach to focus early investment decisions on products  collectively at the enterprise level and ensure that there is a sound basis to  justify the commitment of resources. By following a disciplined,  integrated process—during which the relative pros and cons of competing  product proposals are assessed based on strategic objectives, customer  needs, and available resources, and where tough decisions about which  investments to pursue and not to pursue are made—companies minimize  duplication between business units, move away from organizational  stovepipes, and effectively support each new development program they  commit to. To be effective, integrated portfolio management must have  strong, committed leadership; empowered portfolio managers; and  accountability at all levels of the organization.
DOD determines its capability needs through the Joint Capabilities and  Integration Development System (JCIDS). While JCIDS provides a  framework for reviewing and validating needs, it does not adequately  prioritize those needs from a joint, departmentwide perspective; lacks the  agility to meet changing warfighter demands; and validates almost all of  the capability proposals that are submitted. We recently reviewed JCIDS  documentation related to new capability proposals and found that most— almost 70 percent—were sponsored by the military services with little  involvement from the joint community, including the combatant  commands, which are responsible for planning and carrying out military  operations. Because DOD also lacks an analytic approach to determining  the relative importance of the capabilities needed for joint warfighting, all  proposals appear to be treated as equal priorities within the JCIDS  process. By continuing to rely on capability needs defined primarily by the  services, DOD may be losing opportunities for improving joint warfighting  capabilities and reducing the duplication of capabilities in some areas. The  JCIDS process has also proven to be lengthy and cumbersome—taking on  average up to 10 months to validate a need—thus undermining the  department’s efforts to effectively respond to the needs of the warfighter,  especially those needs that are near term. Furthermore, the vast majority  of capability proposals that enter the JCIDS process are validated or  approved without accounting for the resources or technologies that will be  needed to acquire the desired capabilities. Ultimately, the process  produces more demand for new weapon system programs than available  resources can support.
The funding of proposed programs takes place through a separate process,  the department’s Planning, Programming, Budgeting, and Execution  (PPBE) system, which is not synchronized with JCIDS. While JCIDS is a  continuous, need-driven process that unfolds in response to capability  proposals as they are submitted by sponsors, PPBE is a calendar-driven  process comprising phases occurring over a 2-year cycle, which can lead  to resource decisions for proposed programs that may occur several years  later. In addition, because PPBE is structured by military service and  defense programs and not by the joint capability areas being used in  JCIDS, it is difficult to link resources to capabilities. The PPBE process  also largely allocates resources based on historical trends rather than on a  strategic basis. Service shares of the overall budget have remained  relatively static for decades, even though DOD’s strategic environment and  warfighting needs have changed dramatically in recent years. Because  DOD’s programming and budgeting reviews occur at the back end of the  PPBE process—after the services have developed their budgets—it is  difficult and disruptive to make changes, such as terminating programs to  pay for new, higher-priority programs.
We recently reviewed the impact of the PPBE process on major defense  acquisition programs and found that the process does not produce an  accurate picture of the department’s resource needs for weapon system  programs, in large part because it allows too many programs to go forward  with unreliable cost estimates and without a commitment to fully fund  them. The cost of many of the programs we reviewed exceeded the  funding levels planned for and reflected in the Future Years Defense  Program (FYDP)—the department’s long-term investment strategy (see fig.  1). DOD’s failure to balance its needs with available resources promotes  an unhealthy competition for funding that encourages sponsors of weapon  system programs to pursue overly ambitious capabilities and  underestimate costs to appear affordable. Rather than limit the number  and size of programs or adjust requirements, DOD opts to push the real  costs of programs to the future. With too many programs under way for  the available resources and high cost growth occurring in many programs,  the department must make up for funding shortfalls by shifting funds from  one program to pay for another, reducing system capabilities, cutting  procurement quantities, or in rare cases terminating programs. Such  actions not only create instability in DOD’s weapon system portfolio, they  further obscure the true future costs of current commitments, making it  difficult to make informed investment decisions.
section_title: Initiating Programs with Inadequate Knowledge of Requirements and Resources Often Results in Poor Outcomes
paragraphs: At the program level, the key cause of poor outcomes is the approval of  programs with business cases that contain inadequate knowledge about  requirements and the resources—funding, time, technologies, and  people—needed to execute them. Our work in best practices has found  that an executable business case for a program demonstrated evidence  that (1) the identified needs are real and necessary and that they can best  be met with the chosen concept and (2) the chosen concept can be  developed and produced within existing resources. Over the past several  years, we have found no evidence of the widespread adoption of such an  approach for major acquisition programs in the department. Our annual  assessments of major weapon systems have consistently found that the  vast majority of programs began system development without mature  technologies and moved into system demonstration without design  stability.
The chief reason for these problems is the encouragement within the  acquisition environment of overly ambitious and lengthy product  developments—sometimes referred to as revolutionary or big bang  acquisition programs—that embody too many technical unknowns and not  enough knowledge about the performance and production risks they  entail. The knowledge gaps are largely the result of a lack of early and  disciplined systems engineering analysis of a weapon system’s  requirements prior to beginning system development. Systems engineering  translates customer needs into specific product requirements for which  requisite technological, software, engineering, and production capabilities  can be identified through requirements analysis, design, and testing. Early  systems engineering provides the knowledge a product developer needs to  identify and resolve performance and resource gaps before product  development begins by either reducing requirements, deferring them to  the future, or increasing the estimated cost for the weapon system’s  development. Because the government often does not perform the proper  up-front requirements analysis to determine whether the program will  meet its needs, significant contract cost increases can and do occur as the  scope of the requirements changes or becomes better understood by the  government and contractor. Not only does DOD not conduct disciplined  systems engineering prior to the beginning of system development, it has  allowed new requirements to be added well into the acquisition cycle. We  have reported on the negative impact that poor systems engineering  practices have had on several programs, such as the Global Hawk  Unmanned Aircraft System, F-22A, Expeditionary Fighting Vehicle, and  Joint Air-to-Surface Standoff Missile.
With high levels of uncertainty about requirements, technologies, and  design, program cost estimates and related funding needs are often  understated, effectively setting programs up for cost and schedule growth.  We recently assessed the service and independent cost estimates for 20  major weapon system programs and found that while the independent  estimates were somewhat higher, both estimates were too low in most  cases. In some of the programs we reviewed, cost estimates have been off  by billions of dollars. For example, the initial service estimate for the  development of the Marines’ Expeditionary Fighting Vehicle was about  $1.1 billion. The department’s Cost Analysis and Improvement Group  (CAIG) estimated the development cost of the program to be $1.4 billion,  but development costs for the program are now expected to be close to  $3.6 billion. In the case of the Future Combat System (FCS), the Army’s  initial estimate for the development cost was about $20 billion, while  CAIG’s estimate was $27 billion. The department began the program using  the program office’s estimate of $20 billion, but development costs for the  FCS are now estimated to be $28 billion and the program is still dealing  with significant technical risk. Estimates this far off the mark do not  provide the necessary foundation for sufficient funding commitments and  realistic long-term planning.
The programs we reviewed frequently lacked the knowledge needed to  develop realistic cost estimates. For example, program Cost Analysis  Requirements Description documents—used to build the program cost  estimate—often lack sufficient detail about planned program content for  developing sound cost estimates. Without this knowledge, cost estimators  must rely heavily on parametric analysis and assumptions about system  requirements, technologies, design maturity, and the time and funding  needed. A cost estimate is then usually presented to decision makers as a  single, or point, estimate that is expected to represent the most likely cost  of the program but provides no information about the range of risk and  uncertainty or level of confidence associated with the estimate.
section_title: Lack of Accountability for Making Weapon System Decisions Hinders Achieving Successful Outcomes
paragraphs: DOD’s requirements, resource allocation, and acquisition processes are led  by different organizations, thus making it difficult to hold any one person  or organization accountable for saying no to a proposed program or for  ensuring that the department’s portfolio of programs is balanced. DOD’s  2006 Defense Acquisition Performance Assessment study observed that  these processes are not connected organizationally at any level below the  Deputy Secretary of Defense and concluded that this weak structure  induces instability and inhibits accountability. Furthermore, a former  Under Secretary of Defense for Acquisitions, Technology and Logistics has  stated that weapon system investment decisions are a shared  responsibility in the department and, therefore, no one individual is  accountable for these decisions. Frequent turnover in leadership positions  in the department exacerbates the problem. The average tenure, for  example, of the Under Secretary of Defense for Acquisition, Technology  and Logistics over the past 22 years has been only about 20 months.
When DOD’s strategic processes fail to balance needs with resources and  allow unsound, unexecutable programs to move forward, program  managers cannot be held accountable when the programs they are handed  already have a low probability of success. Program managers are also not  empowered to make go or no-go decisions, have little control over  funding, cannot veto new requirements, and have little authority over  staffing. At the same time, program managers frequently change during a  program’s development, making it difficult to hold them accountable for  the business cases that they are entrusted to manage and deliver.
The government’s lack of control over and accountability for decision  making is further complicated by DOD’s growing reliance on technical,  business, and procurement expertise supplied by contractors. This  reliance may reach the point where the foundation upon which decisions  are based may be largely crafted by individuals who are not employed by  the government, who are not bound by the same rules governing their  conduct, and who are not required to disclose any financial or other  personal interests they may have that conflict with the responsibilities  they have performing contract tasks for DOD. For example, while the total  planned commitments to major acquisition programs have doubled over  recent years, the size of the department’s systems engineering workforce  has remained relatively stable, leading program offices to rely more on  contractors for systems engineering support. Further, in systems  development, DOD typically uses cost-reimbursement contracts in which  it generally pays the reasonable, allocable, and allowable costs incurred  for the contractor’s best efforts, to the extent provided by the contract.  The use of these contracts may contribute to the perpetuation of an  acquisition environment that lacks incentives for contractors to follow  best practices and keep costs and schedules in check.
section_title: Recent DOD Policy Changes Could Improve Future Performance of Weapon System Programs
paragraphs: The department understands many of the problems that affect acquisition  programs and has recently taken steps to remedy them. It has revised its  acquisition policy and introduced several initiatives based in part on  direction from Congress and recommendations from GAO that could  provide a foundation for establishing sound, knowledge-based business  cases for individual acquisition programs. However, to improve outcomes,  DOD must ensure that its policy changes are consistently implemented  and reflected in decisions on individual programs—not only new program  starts but also ongoing programs as well. In the past, inconsistent  implementation of existing policy has hindered DOD’s efforts to execute  acquisition programs effectively. Moreover, while policy improvements are  necessary, they may be insufficient unless the broader strategic issues  associated with the department’s fragmented approach to managing its  portfolio of weapon system investments are also addressed.
In December 2008, DOD revised its policy governing major defense  acquisition programs in ways intended to provide key department leaders  with the knowledge needed to make informed decisions before a program  starts and to maintain disciplined development once it begins. The revised  policy recommends the completion of key systems engineering activities  before the start of development, includes a requirement for early  prototyping, establishes review boards to identify and mitigate technical  risks and evaluate the impact of potential requirements changes on  ongoing programs, and incorporates program manager agreements to  increase leadership stability and management accountability. The policy  also establishes early milestone reviews for programs going through the  pre–systems acquisition phase. In the past, DOD’s acquisition policy may  have encouraged programs to rush into systems development without  sufficient knowledge, in part, because no formal milestone reviews were  required before system development. If implemented, these policy changes  could help programs replace risk with knowledge, thereby increasing the  chances of developing weapon systems within cost and schedule targets  while meeting user needs. Some aspects of the policy were first pilot- tested on selected programs, such as the Joint Light Tactical Vehicle  program, and indications are that these programs are in the process of  acquiring the requisite knowledge before the start of systems  development. Some key elements of the department’s new acquisition  policy include    a new materiel development decision as a starting point for all programs  regardless of where they are intended to enter the acquisition process,    a more robust Analysis of Alternatives (AOA) to assess potential materiel  solutions that address a capability need validated through JCIDS,    a cost estimate for the proposed solution identified by the AOA,    early program support reviews by systems engineering teams,    competitive prototyping of the proposed system or key system elements as  part of the technology development phase,    certifications for entry into the technology development and system  development phases (as required by congressional legislation),    preliminary design review that may be conducted before the start of    configuration steering boards to review all requirements and technical  changes that have potential to affect cost and schedule.
As part of its strategy for enhancing the roles of program managers in  major weapon system acquisitions, the department has established a  policy that requires formal agreements among program managers, their  acquisition executives, and the user community setting forth common  program goals. These agreements are intended to be binding and to detail  the progress the program is expected to make during the year and the  resources the program will be provided to reach these goals. DOD also  requires program managers to sign tenure agreements so that their tenure  will correspond to the next major milestone review closest to 4 years. The  department acknowledges that any actions taken to improve  accountability must be based on a foundation whereby program managers  can launch and manage programs toward successful performance, rather  than focusing on maintaining support and funding for individual programs.  DOD acquisition leaders have also stated that any improvements to  program managers’ performance depend on the department’s ability to  promote requirements and resource stability over weapon system  investments.
Over the past few years, DOD has also been testing portfolio management  approaches in selected capability areas—command and control, net- centric operations, battlespace awareness, and logistics—to facilitate  more strategic choices for resource allocation across programs. The  department recently formalized the concept of capability portfolio  management, issuing a directive in 2008 that established policy and  assigned responsibilities for portfolio management. The directive  established nine joint capability area portfolios, each to be managed by  civilian and military co-leads. While the portfolios have no independent  decision-making authority over requirements determination and resource  allocation, according to some DOD officials, they provided key input and  recommendations in this year’s budget process. However, without  portfolios in which managers have authority and control over resources,  the department is at risk of continuing to develop and acquire systems in a  stovepiped manner and of not knowing if its systems are being developed  within available resources.
section_title: Observations on Proposed Acquisition Reform Legislation
paragraphs: Overall, we believe that the legislative initiatives being proposed by the  committee have the potential, if implemented, to lead to significant  improvements in DOD’s management of weapon system programs. Several  of the initiatives—including the increased emphasis on systems  engineering and developmental testing, the requirement for earlier  preliminary design reviews, and the strengthening of independent cost  estimates and technology readiness assessments—could instill more  discipline into the front end of the acquisition process when it is critical  for programs to gain knowledge. Establishing a termination criterion for  Nunn-McCurdy cost breaches could help prevent the acceptance of  unrealistic cost estimates as a foundation for starting programs. Having  greater involvement by the combatant commands in determining  requirements and requiring greater consultation between the  requirements, budget, and acquisition processes could help improve the  department’s efforts to balance its portfolio of weapon system programs.  In addition, several of the proposals as currently drafted will codify what  DOD policy already calls for, but are not being implemented consistently  in weapon programs.
subsections: section_title: Section 101: Systems Engineering Capabilities
paragraphs: Requires DOD to (1) assess the extent to which the department has in  place the systems engineering capabilities needed to ensure that key  acquisition decisions are supported by a rigorous systems analysis and  systems engineering process and (2) establish organizations and  develop skilled employees to fill any gaps in such capabilities.
The lack of disciplined systems engineering analysis conducted prior to  starting system development has been a key factor contributing to poor  acquisition outcomes. Systems engineering activities—requirements  analysis, design, and testing—are needed to ensure that a weapon system  program’s requirements are achievable and designable given available  resources, such as technologies. In recent years, DOD has taken steps to  improve its systems engineering capabilities by establishing a Systems and  Software Engineering Center of Excellence and publishing guidance to  assist the acquisition workforce in the development of systems  engineering plans, education, and training. However, as the National  Research Council recently reported, DOD’s systems engineering  capabilities have declined over time and shifted increasingly to outside  contractors. A comprehensive assessment to determine what systems  engineering capabilities are in place and what capabilities are needed, as  recommended in the proposed legislation, is a critical first step in  enhancing the function of systems engineering in DOD acquisitions. At the  same time, it will be important for DOD to implement steps to ensure  systems engineering is applied in the right way and at the right time.
section_title: Section 102: Developmental Testing
paragraphs: Requires DOD to reestablish the position of Director of Developmental  Test and Evaluation and requires the services to assess and address any  shortcomings in their developmental testing organizations and  personnel.
Robust developmental testing efforts are an integral part of the systems  development process. They help to identify, evaluate, and reduce technical  risks, and indicate whether the design solution is on track to satisfy the  desired capabilities. As the Defense Science Board reported in 2008,  developmental testing in weapon system programs needs to be improved.  We believe that developmental testing would be strengthened by a formal  elevation of its role in the acquisition process and the reestablishment of a  Director of Developmental Test and Evaluation position. Furthermore,  requiring the Director to prepare an annual report for Congress  summarizing DOD’s developmental test and evaluation activities would  provide more accountability. We also agree that the military services  should be required to assess their respective developmental testing  entities and address any shortcomings. This action would help ensure that  the services have the knowledge and capacity for effective developmental  test efforts.
section_title: Section 103: Technological Maturity Assessments
paragraphs: Makes it the responsibility of the Director of Defense Research and  Engineering (DDR&E) to periodically review and assess the  technological maturity of critical technologies used in major defense  acquisition programs.
Ensuring that programs have mature technology before starting systems  development is critical to avoiding cost and schedule problems, yet for  many years we have reported that a majority of programs go forward with  immature technologies and experience significant cost growth. Legislation  enacted by Congress in 2006, requiring DOD to certify that the technology  in a program has been demonstrated in a relevant environment before it  receives approval to start system development, has begun to help address  this problem. Since the legislation was enacted, DOD has asked the  DDR&E to conduct independent reviews of technology readiness  assessments for system development milestone decisions. Although  DDR&E reviews are advisory in nature, we have seen reviews that have  pushed programs to do more to demonstrate technology maturity. The  improvements that this proposed legislation, as currently written, is  intended to bring about may already be occurring in DOD. Congress,  however, may wish to consider requiring the DDR&E to conduct  technology readiness reviews not just periodically, but for all major  defense acquisition programs, and whether or not DDR&E has the  capacity and resources to effectively conduct technology assessments.
section_title: Section 104: Independent Cost Assessment
paragraphs: Establish a Director of Independent Cost Assessment to ensure that cost  estimates for major defense acquisition programs are fair, reliable, and  unbiased.
Within DOD, the Cost Analysis Improvement Group (CAIG) is the  organization responsible for conducting independent costs estimates for  major defense acquisition programs. The CAIG reports to the department’s  Director of Program Analysis and Evaluation, but its principal customer is  the Under Secretary of Defense for Acquisition, Technology and Logistics.  We believe that establishing an independent assessment office that reports  directly to the Secretary or Deputy Secretary of Defense and to  Congress—similar to the Office of the Director of Operation Test and  Evaluation—would more fully integrate cost estimating with the  acquisition management framework and provide an increased level of  accountability. We see no reason why CAIG should not form the basis of  the proposed organization. Congress may also wish to consider appointing  the Director for a time-certain term and making the Director responsible  for prescribing cost-estimating policy and guidance and for preparing an  annual report summarizing cost estimates for major acquisition programs.  Ultimately, however, improved cost estimating will only occur if there is a  better foundation for planning and acquiring weapon system programs— one that promotes well-defined requirements, is knowledge-based and  informed by disciplined systems engineering, requires mature technology,  and adheres to shorter development cycle times.
section_title: Section 105: Role of Combatant Commanders
paragraphs: Requires the Joint Requirements Oversight Council (JROC) to seek and  consider input from the commanders of the combatant commands in  identifying joint military requirements.
Requirements determination in DOD, particularly for major weapon  system programs, continues to be driven largely by the military services.  Studies by the Defense Science Board, Center for Strategic and  International Studies, and others have revealed that although the  combatant commands—which are responsible for planning and executing  military missions—are the principal joint warfighting customer in DOD,  they have played a limited role in determining requirements. Currently, the  JROC is doing more to seek out and consider input from the combatant  commands through regular trips and meetings to discuss capability needs  and resourcing issues. However, many of the combatant commands do not  believe that their needs, which are reflected through the Integrated  Priority List process, are sufficiently addressed through the department’s  JCIDS process. For the combatant commands to meet this proposed  legislative mandate and have more influence in establishing requirements,  DOD should consider providing the combatant commands with additional  resources to establish robust analytical capabilities for identifying and  assessing their capability needs. Ultimately, the department must better  prioritize and balance the needs of the military services, combatant  commands, and other defense components, and be more agile in  responding to near-term capability needs.
section_title: Section 201: Trade-offs of Cost, Schedule, and Performance
paragraphs: Requires consultation between the budget, requirements, and  acquisition processes to ensure the consideration of trade-offs between  cost, schedule, and performance early in the process of developing major  weapon systems.
As currently structured, DOD’s budget, requirements, and acquisition  processes do not operate in an integrated manner. The function and timing  of the processes are not sufficiently synchronized, and the decision  makers for each process are motivated by different incentives. These  weaknesses have contributed to the development of a portfolio with more  programs than available resources can support and programs that launch  into system development without executable business cases. We have  recommended that the department establish an enterprisewide portfolio  management approach to weapon system investment decisions that  integrates the determination of joint warfighting needs with the allocation  of resources, and cuts across the services by functional or capability  area. To ensure the success of such an approach, we believe that the  department should establish a single point of accountability with the  authority, responsibility, and tools to implement portfolio management  effectively.
section_title: Section 202: Preliminary Design Review
paragraphs: Require the completion of a Preliminary Design Review (PDR) and a  formal post-PDR assessment before a major defense acquisition program  receives approval to start system development.
We have found that a key deliverable in a knowledge-based acquisition  process is the preliminary design of the proposed solution based on a  robust systems engineering assessment prior to making a large financial  commitment to system development. Early systems engineering provides  the knowledge needed by a developer to identify and resolve gaps, such as  overly optimistic requirements that cannot be met with current resources,  before product development begins. Consequently, DOD would have more  confidence that a particular system could successfully proceed into a  detailed system development phase and meet stated performance  requirements within cost, schedule, risk, and other relevant constraints.  The recently revised DOD acquisition policy places an increased emphasis  on programs planning for preliminary design review prior to the start of  system development but does not go as far as making it a requirement to  do so. We support any effort to add controls to the acquisition process to  ensure that timely and robust systems engineering is conducted before  major investment decisions, such as the approval to start system  development, are made.
section_title: Section 203: Life-Cycle Competition
paragraphs: Require DOD to adopt measures recommended by the 2008 Defense  Science Board Task Force on Defense Industrial Structure for  Transformation—such as competitive prototyping, dual sourcing, open  architectures, periodic competitions for subsystem upgrades, and  licensing of additional suppliers—to maximize competition throughout  the life of a program.
We have reported in the past on the problem of diminishing competition  and the potential benefits of more competition. In discussing the  environment that leads to poor acquisition outcomes, we have noted that  changes within the defense supplier base have added pressure to this  environment. We noted that in 2006, a DOD-commissioned study found  that the number of fully competent prime contractors competing for  programs had fallen from more than 20 in 1985 to only 6, and that this has  limited DOD’s ability to maximize competition in order to reduce costs  and encourage innovation. However, avenues exist for reducing costs  through competition. For example, we reported that although continuing  an alternate engine program for the Joint Strike Fighter would cost  significantly more in development costs than a sole-source program, it  could, in the long run, reduce overall life cycle costs and bring other  benefits.
section_title: Section 204: Nunn- McCurdy Breaches
paragraphs: Requires that a major defense acquisition program that experiences a  critical Nunn-McCurdy cost breach be terminated unless the Secretary of  Defense certifies that (1) continuing the program is essential to  national security and the program can be modified to proceed in a cost- effective manner and (2) the program receives a new milestone approval  prior to the award of any new or modified contract extending the scope  of the program.
In order for DOD to improve its program outcomes, realistic cost  estimates must be required when programs are approved for development  initiation. DOD often underestimates costs in large part because of a lack  of knowledge and overly optimistic assumptions about requirements and  critical technologies. This underestimation is also influenced by DOD’s  continuing failure to balance its needs with available resources, which  promotes unhealthy competition among programs and encourages  programs to overpromise on performance capabilities and underestimate  cost. This false optimism is reinforced by an acquisition environment in  which there are few ramifications for cost growth and delays. Only in very  rare instances have programs been terminated for poor performance.  When DOD consistently allows unsound, unexecutable programs to begin  with few negative ramifications for poor outcomes, accountability suffers.  As section 204 proposes, the strengthening of the Nunn-McCurdy  provision—by including the potential termination of programs that  experience critical cost growth—could facilitate a change in DOD’s  behavior by preventing the acceptance of unrealistic cost estimates as a  foundation for program initiation and placing more accountability on  senior DOD leadership for justifying program continuation. Programs may  thus be forced to be more candid and up front about potential costs, risks,  and funding needs, and the likelihood of delivering a successful capability  to the warfighter at the cost and in the time promised may grow.
section_title: Section 205: Organizational Conflicts of Interest
paragraphs: Prohibits systems engineering contractors from participating in the  development or construction of major weapon systems on which they are  advising DOD, and requires tightened oversight of organizational  conflicts of interest by contractors in the acquisition of major weapon  systems.
The defense industry has undergone significant consolidation in recent  years which has resulted in a few large, vertically integrated prime  contractors. This consolidation creates the potential for organizational  conflicts of interest where, for example, one business unit of a large  company may be asked to provide systems engineering work on a system  being produced by another unit of the same company. As the Defense  Science Board has recognized, these conflicts of interest may lead to  impaired objectivity, which may not be mitigated effectively through  techniques such as erecting a firewall between the employees of the two  units. While the Federal Acquisition Regulation currently covers some  cases of potential organizational conflicts of interest involving the systems  engineering function, there may be a need for additional coverage in this  area. In general, we would support efforts to enhance the oversight of  potential organizational conflicts of interest, particularly in the current  environment of a heavily consolidated defense industry.
section_title: Section 206: Acquisition Excellence
paragraphs: Establishes an annual awards program to recognize individuals and  teams that make significant contributions to the improved cost,  schedule, and performance of defense acquisition programs  We support the creation of an annual awards program to recognize  individuals and teams for improving the cost, schedule, and performance  of defense acquisition programs. We have reported that meaningful and  lasting reform will not be achieved until the right incentives are  established and accountability is bolstered at all levels of the acquisition  process. The need for incentives emerged as a significant issue in our  recent discussions with acquisition experts examining potential changes  to the acquisition processes enumerated in last year’s defense  authorization act. The discussions revealed that those changes may not  achieve the desired improvement in acquisition outcomes unless they are  accompanied by changes in the overall acquisition environment and  culture, and the incentives they provide for success.
section_title: Concluding Observations on What Remains to Be Done
paragraphs: A broad consensus exists that weapon system problems are serious and  that their resolution is overdue. With the federal budget under increasing  strain from the nation’s economic crisis, the time for change is now. DOD  is off to a good start with the recent revisions to its acquisition policy,  which, if implemented properly, should provide a foundation for  establishing sound, knowledge-based business cases before launching into  development and for maintaining discipline after initiation. The new policy  will not work effectively, however, without changes to the overall  acquisition environment. Resisting the urge to achieve the revolutionary  but unachievable capability, allowing technologies to mature in the  science and technology base before bringing them onto programs,  ensuring that requirements are well-defined and doable, and instituting  shorter development cycles would all make it easier to estimate costs  accurately, and then predict funding needs and allocate resources  effectively. But these measures will succeed only if the department uses an  incremental approach. Constraining development cycle times to 5 or 6  years will force more manageable commitments, make costs and  schedules more predictable, and facilitate the delivery of capabilities in a  timely manner.
Acquisition problems are also likely to continue until DOD’s approach to  managing its weapon system portfolio (1) prioritizes needs with available  resources, thus eliminating unhealthy competition for funding and the  incentives for making programs look affordable when they are not;   (2) facilitates better decisions about which programs to pursue and which  not to pursue given existing and expected funding; and (3) balances the  near-term needs of the joint warfighter with the long-term need to  modernize the force. Achieving this affordable portfolio will require strong  leadership and accountability. Establishing a single point of accountability  could help the department align competing needs with available resources.
The department has tough decisions to make about its weapon systems  and portfolio, and stakeholders, including military services, industry, and  Congress, have to play a constructive role in the process toward change.  Reform will not be achieved until DOD changes its acquisition  environment and the incentives that drive the behavior of its decision  makers, the military services, program managers, and the defense  industry.
Mr. Chairman, this concludes my prepared statement. I would be happy to  answer any questions you may have at this time.
section_title: Contacts and Acknowledgements
paragraphs: For further information about this statement, please contact Michael J.  Sullivan (202) 512-4841 or sullivanm@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on the  last page of this statement. Individuals who made key contributions to this  statement include John Oppenheim, Charlie Shivers, Dayna Foster, Matt  Lea, Susan Neill, Ron Schwenn, and Bruce Thomas.
section_title: Related GAO Products
paragraphs: Defense Acquisitions: Perspectives on Potential Changes to DOD’s  Acquisition Management Framework. GAO-09-295R. Washington, D.C.:  February 27, 2009.
Defense Management: Actions Needed to Overcome Long-standing  Challenges with Weapon Systems Acquisition and Service Contract  Management. GAO-09-362T. Washington, D.C.: February 11, 2009.
Defense Acquisitions: Fundamental Changes Are Needed to Improve  Weapon Program Outcomes. GAO-08-1159T. Washington, D.C.: September  25, 2008.
Defense Acquisitions: DOD’s Requirements Determination Process Has  Not Been Effective in Prioritizing Joint Capabilities. GAO-08-1060.  Washington, D.C.: September 25, 2008.
Defense Acquisitions: A Knowledge-Based Funding Approach Could  Improve Major Weapon System Program Outcomes. GAO-08-619.  Washington, D.C.: July 2, 2008.
Defense Acquisitions: Better Weapon Program Outcomes Require  Discipline, Accountability, and Fundamental Changes in the  Acquisition Environment. GAO-08-782T. Washington, D.C.: June 3, 2008.
Defense Acquisitions: Assessments of Selected Weapon Programs.  GAO-08-467SP. Washington, D.C.: March 31, 2008.
Best Practices: Increased Focus on Requirements and Oversight Needed  to Improve DOD’s Acquisition Environment and Weapon System  Quality. GAO-08-294. Washington, D.C.: February 1, 2008.
Cost Assessment Guide: Best Practices for Estimating and Managing  Program Costs. GAO-07-1134SP. Washington, D.C.: July 2007.
Defense Acquisitions: Assessments of Selected Weapon Programs.  GAO-07-406SP. Washington, D.C.: March 30, 2007.
Best Practices: An Integrated Portfolio Management Approach to Weapon  System Investments Could Improve DOD’s Acquisition Outcomes.  GAO-07-388. Washington, D.C.: March 30, 2007.
Defense Acquisitions: Major Weapon Systems Continue to Experience  Cost and Schedule Problems under DOD’s Revised Policy. GAO-06-368.  Washington, D.C.: April 13, 2006.
DOD Acquisition Outcomes: A Case for Change. GAO-06-257T.  Washington, D.C.: November 15, 2005.
Best Practices: Better Support of Weapon System Program Managers  Needed to Improve Outcomes. GAO-06-110. Washington, D.C.: November 1,  2005.
Defense Acquisitions: Stronger Management Practices Are Needed to  Improve DOD’s Software-Intensive Weapon Acquisitions. GAO-04-393.  Washington, D.C.: March 1, 2004.
Best Practices: Setting Requirements Differently Could Reduce Weapon  Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.: February  11, 2003.
Defense Acquisitions: Factors Affecting Outcomes of Advanced Concept  Technology Demonstration. GAO-03-52. Washington, D.C.: December 2,  2002.
Best Practices: Capturing Design and Manufacturing Knowledge Early  Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15,  2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best  Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.