Defense Acquisitions: DOD Can Improve Its Management of Configuration Steering Boards
Published: 2011-07-07T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: GAO has previously reported that requirements changes are factors in poor cost and schedule outcomes on Department of Defense (DOD) weapon programs. In 2007, DOD introduced Configuration Steering Boards (CSBs) to review requirement and configuration changes that could adversely affect programs. In 2008, Congress made annual CSB meetings a requirement for all of the military departments' major defense acquisition programs. In response to the Senate report accompanying the bill for the Ike Skelton National Defense Authorization Act for Fiscal Year 2011, GAO assessed (1) the extent to which DOD has complied with the statutory requirements for CSBs, and (2) the extent to which CSBs have been effective in controlling requirements and mitigating cost and schedule risks. To conduct this work, GAO surveyed DOD's major defense acquisition programs, reviewed CSB documentation, and interviewed relevant military service and program officials.
section_title: What GAO Found
paragraphs: The military departments varied in their compliance with the CSB requirements in statute. The Air Force and Navy did not fully comply with the requirement to hold annual CSB meetings for all major defense acquisition programs in 2010, while the Army did. In total, the military departments held an annual CSB meeting for 74 of 96 major defense acquisition programs they managed in 2010. According to GAO's survey results, when the military departments held CSB meetings, 19 programs endorsed requirements or configuration changes. In most of these cases, strategies were developed to mitigate the effects of these changes--a key provision in the statute and DOD policy. However, key acquisition and requirements personnel were often absent from Air Force and Navy CSB meetings when these issues were discussed. Two major defense acquisition programs--the Ballistic Missile Defense System and the Chemical Demilitarization-Assembled Chemical Weapons Alternatives programs--are not subject to the CSB provisions in statute because the statute only applies to programs overseen by military departments; the programs are managed by other DOD components. These programs are subject to DOD's CSB policy, which differs from the statute in that it only requires major defense acquisition programs that are in development to hold annual CSB reviews. Individual programs varied in the extent to which they utilized CSBs to control requirements and mitigate cost and schedule risks. According to GAO's survey results, the majority of CSB meetings neither reviewed requirement changes nor discussed options to moderate requirements or reduce the scope of programs. There were a number of specific instances where CSB meetings were effective in mitigating the effect of necessary changes, rejecting other changes, facilitating discussion of requirements, and endorsing "descoping" options with the potential to improve or preserve cost or schedule. However, in response to a survey, program officials cast some doubts about the effectiveness of CSBs, and in interviews, acquisition officials indicated that program managers may be reluctant to recommend descoping options due to cultural biases that encourage meeting warfighters' stated needs rather than achieving cost savings, a preference not to elevate decisions to higher levels of review, and concerns that future funding may be cut if potential savings are identified. In response, the Army and Air Force have issued additional descoping guidance and set savings or budget targets. The types of discussions for which CSBs were useful changed based on whether programs were in development or production. Development programs found them more useful to consider requirements changes and descoping options, and production programs found CSBs more useful to prevent changes. In an effort to further increase effectiveness and efficiency of CSBs, some of the military departments have taken steps to coordinate CSB meetings among programs that provide similar capabilities and align CSB meetings with other significant reviews. Among GAO's recommendations for DOD components are that they amend their CSB policies to be consistent with statute and align CSBs with other reviews when possible. In comments on a draft of this report, DOD concurred or partially concurred with all seven of GAO's recommendations and agreed to take action to address six of them.

section_title: Letter
paragraphs: The Department of Defense’s (DOD) major defense acquisition programs  have historically cost more and taken longer to field capabilities to the  warfighter than initially planned. The total acquisition cost of DOD’s  portfolio of major programs has increased by $135 billion since 2008, and  the average delay in delivering initial capability is now 22 months. We  have previously reported that requirements changes and the inability of  program managers to defer requirements that could not be completed  under existing cost and schedule targets are factors in poor acquisition  program outcomes. To address this issue, the Under Secretary of  Defense (USD) for Acquisition, Technology and Logistics (AT&L)  introduced Configuration Steering Boards (CSBs) in 2007 to review  requirement and configuration changes that could adversely affect cost  and schedule for major programs in development. Congress has also  identified the CSB as a way to enable this process. In the Duncan Hunter  National Defense Authorization Act for Fiscal Year 2009, Congress made  annual CSB meetings a requirement for all major defense acquisition  programs.
In our 2010 assessment of selected weapons programs, we found that  few programs reported holding CSB meetings in 2009. In response, the  Senate report that accompanied the bill for the Ike Skelton National  Defense Authorization Act for Fiscal Year 2011 asked us to review DOD’s  use of CSBs in fiscal year 2010. This report assesses: (1) the extent to  which DOD has complied with the statutory requirements for CSBs and  (2) the extent to which CSBs have been effective in controlling  requirements and mitigating cost and schedule risks.
To determine the extent to which DOD complied with the statutory  requirement to hold annual CSB meetings, we identified 98 active major  defense acquisition programs using the DOD’s Defense Acquisition  Management Information Retrieval System. We defined an active  program as one that issued a selected acquisition report in December  2009. For each program, we asked the acquisition organization  overseeing it—the Army, Navy, Air Force, Missile Defense Agency, and  the Assistant Secretary of Defense for Nuclear, Chemical, and Biological  Defense Programs—to provide the minutes and lists of attendees from  the CSB meetings held in calendar year 2010.
To determine the extent to which CSBs have been effective in controlling  requirements and mitigating cost and schedule risks, we surveyed all 98  program offices to gather information on their programs, on the CSB  meetings their programs held in fiscal year 2010, and on the utility of  these meetings. We collected fiscal-year data in our survey because the  Senate report language that contained our mandate focused on fiscal- year 2010. All 98 programs completed the survey. We also conducted  interviews with 17 programs to collect more information about how  requirements changes were reviewed, how they mitigated cost and  schedule changes, and what made CSB meetings effective or ineffective.  We selected these programs based on the types of activities that took  place during their CSB meetings as reported in their survey responses  and minutes. Appendix I contains more information regarding our scope  and methodology.
We conducted this performance audit from September 2010 to July 2011  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: In July 2007, the USD (AT&L) established CSBs for every current and  future major defense acquisition program in development as a measure to  limit requirements change and avoid cost increases. The CSBs were to  have a broad membership, including senior representatives from the  offices of USD (AT&L) and Joint Staff. CSBs were intended to review all  requirements and significant technical configuration changes with the  potential to adversely affect the program. The USD (AT&L) directed that  these changes should generally be rejected or deferred unless funds and  schedule adjustments could be identified to mitigate their effects. In  addition, program managers were asked to identify options to reduce  program cost or moderate requirements, referred to as “descoping”  options, on a roughly annual basis. USD (AT&L) also instructed that,  while policy would be to keep within planned costs as much as possible  even at the expense of scope and content, all expected increases in  program costs must be budgeted at the absolute earliest opportunity.  USD (AT&L) incorporated CSBs into DOD’s primary acquisition policy— DOD Instruction 5000.02—in December 2008.
In October 2008, Congress enacted the Duncan Hunter National Defense  Authorization Act for Fiscal Year 2009, which required the establishment  of CSBs for the major defense acquisition programs of the military  departments. According to the statute, a CSB must meet at least once  each year for each of these programs. The statute also provided  direction on CSB membership and responsibilities. It requires CSBs to  include the appropriate service acquisition executive as chair and  include representatives from USD (AT&L), the Chief of Staff for the  armed forces, representatives from other armed forces as  appropriate, the Joint Staff, the comptroller of the military department,  the military deputy to the service acquisition executive, the program  executive officer for the program concerned, and others as  appropriate;    prevent unnecessary changes to programs that could have an  adverse impact on program cost or schedule, mitigate adverse cost  and schedule effects of changes that may be required, and ensure  that each program delivers as much planned capability as possible at  or below the planned cost and schedule;  review and approve or disapprove any proposed changes to program  requirements or system configuration with the potential to adversely  affect cost and schedule; and  review and recommend proposals that could reduce requirements and  improve cost and schedule.
In addition, the statute provided program managers the authority to    object to adding new requirements that would be inconsistent with  previously established parameters unless approved by the CSB and    propose opportunities to reduce program requirements to improve  cost and schedule consistent with program objectives.
In our March 2010 assessment of selected weapon programs, we  reported that only 7 of the 42 programs we assessed held CSB meetings  in 2009. As a result, in the Senate report accompanying the bill for the  Ike Skelton National Defense Authorization Act for Fiscal Year 2011, the  Senate Armed Services Committee directed USD (AT&L) to take  appropriate steps to ensure that CSBs meet at least once a year to  consider the full range of proposed changes to program requirements or  system configuration for each major defense acquisition program.
section_title: Compliance with the CSB Provisions in Statute Varied by Military Department
paragraphs: The military departments’ compliance with statutory CSB requirements  varied. The Air Force and Navy did not fully comply with the requirement  to hold annual CSB meetings for all major defense acquisition programs  in 2010; the Army did comply. In total, the military departments held an  annual CSB meeting for 74 of 96 major defense acquisition programs  they managed in 2010. According to our survey results, when the military  departments held CSB meetings, 19 programs endorsed requirements or  configuration changes. In most of these cases, strategies were developed  to mitigate any effect on a program’s cost and schedule—a key provision  in the statute and DOD policy. However, key acquisition and requirements  personnel were often absent from Air Force and Navy CSB meetings  when these issues were discussed. Two major defense acquisition  programs—the Ballistic Missile Defense System (BMDS) and the  Chemical Demilitarization-Assembled Chemical Weapons Alternatives  programs, which are managed by DOD components rather than military  departments—are not subject to the CSB provisions in statute, but rather  to DOD policy, because the statute only applies to programs overseen by  military departments. This policy differs from the statute in that it only  requires major defense acquisition programs in development to hold  annual CSB reviews and does not require the same members, including  the comptroller of the military department.
subsections: section_title: The Air Force and Navy Did Not Hold CSB Meetings for All Programs, While the Army Did
paragraphs: The Air Force and Navy did not hold CSB meetings for all of their major  defense acquisition programs in 2010. The Air Force did not hold CSB  meetings for 13 of 31 programs, and the Navy did not hold CSB meetings  for 9 of 37 programs. The Army held a CSB meeting for each of its 28  major defense acquisition programs. Of the 96 major defense acquisition  programs managed by the military departments, 74 held CSB meetings in  2010 and 22 failed to do so. Table 1 shows how many programs had CSB  meetings by military department.
Of the 22 programs that did not have CSB meetings in 2010, 9 programs  had meetings in early 2011. In addition, according to the Air Force and  Navy, 8 other programs were in the process of being completed or  cancelled. Table 2 includes explanations from the Air Force and Navy  about why CSB meetings were not held for individual programs.
For each of the military departments, when a CSB meeting reviewed  requirements or configuration changes, most were endorsed and  strategies to mitigate the effects on a program’s cost and schedule were  developed and discussed. However, most of the programs we surveyed  did not present requirements or configuration changes to be approved or  rejected at their fiscal-year-2010 CSB meetings. Specifically, our survey  showed the following results:    Air Force: 6 CSB meetings reviewed requirements or configuration  changes, 5 of these meetings endorsed changes, and 4 discussed the  cost and schedule effects and ways to mitigate them.    Army: 6 CSB meetings reviewed and endorsed requirements or  configuration changes, and 4 of these discussed the cost and  schedule effects and ways to mitigate them.    Navy: 10 CSB meetings reviewed requirements or configuration  changes; 8 meetings endorsed changes, and 7 of these discussed the  cost and schedule effects and ways to mitigate them.
The Navy did not hold CSB reviews for all programs that experienced  requirements changes in fiscal-year 2010. According to our survey  results, three Navy programs changed system requirements or  specifications yet did not hold a CSB meeting. Two of these programs,  the Advanced Anti-Radiation Guided Missile and the Remote Minehunting  System, held other high-level reviews during this period—two program  management reviews and a critical Nunn-McCurdy breach review,  respectively—and officials reported that a third program, the  Expeditionary Fighting Vehicle, did not conduct its CSB meeting because  DOD proposed canceling the program.
section_title: Key Acquisition and Requirements Personnel Were Absent from Many CSB Meetings
paragraphs: Key acquisition and requirements personnel were absent from many of  the CSB meetings held by the Air Force and Navy in 2010. The CSB  provision in the Duncan Hunter National Defense Authorization Act for  Fiscal Year 2009 lists seven officials or offices that should be part of a  CSB, including the service acquisition executive who should serve as the  chairperson of the CSB; representatives from the acquisition,  requirements, and funding communities; and others as appropriate. Army  CSB meetings held in 2010 included the full array of board members in all  but one case. Although USD (AT&L) was invited to the meeting in this  case, Army officials reported that the office did not send a representative.  The medium of CSB board members’ participation also varied among the  military departments. The Army conducted all its CSB meetings in person,  whereas both the Air Force and the Navy conducted virtual, otherwise  known as paper, CSB meetings for certain programs in 2010 and early  2011.
The Air Force held all of its 2010 CSB meetings without key acquisition  participants listed in the CSB statute. According to Air Force officials, their  CSB meetings may be chaired by either the service acquisition executive  or the principal military deputy to provide for flexibility in scheduling  meetings. Generally, the principal military deputy acts as chair in the  place of the service acquisition executive and does not attend those  meetings that the service acquisition executive chairs. According to the  attendee lists provided by the Air Force, only 2 of the 18 CSB meetings  held were attended and chaired by the service acquisition executive. At  one of those meetings neither the principal military deputy nor a  representative of the comptroller was in attendance although officials  report that both had been invited. The CSB meetings the service  acquisition executive did not attend included numerous discussions of  changes that could affect programs’ costs and schedules, including  requirements and configuration changes or descoping opportunities. For  example, one meeting discussed changes to the Space Based Infrared  System’s architecture that could accelerate the program’s delivery of  initial capability by 2 years but would cost an additional $45 million.
The Air Force also allows paper CSBs to fulfill the requirement for an  annual CSB for programs it believes are stable. A program is eligible to  conduct paper CSB meetings if (1) it has a Probability of Program  Success score of greater than 80; (2) it has made no requirements  and/or significant technical configuration changes since the last CSB that  have the potential to affect the cost and schedule of the program; (3)  when in production, it is in steady state production but has not reached 90  percent of planned expenditures completed or 90 percent of quantities  delivered; and (4) descoping options will not yield any real cost savings.  The Air Force did not conduct any paper CSBs in 2010; however, 6 of the  13 Air Force programs that did not hold a CSB meeting in 2010  conducted paper reviews in January 2011. According to Air Force  officials, the process for these paper reviews began in December 2010.
The Navy held most of its 2010 CSB meetings without key acquisition and  requirements personnel. The Navy has incorporated CSB meetings into  the Navy’s gate review process and uses the gate 6 review, with the  service acquisition executive or his designee acting as chair, to fulfill the  requirement for an annual CSB. However, the Navy’s policy on gate  reviews does not include the Joint Staff—a key player in the requirements  process and a participant required by statute and DOD policy—as a  participant, and at least 22 of the 28 CSB meetings held in 2010 lacked a  representative of the Joint Staff. As a result of our review, Navy officials  reported that they are revising their policy and procedures for CSBs to  ensure the Joint Staff is invited to future CSB meetings.
Navy policy allows the service acquisition executive to delegate the chair  to another official within the Navy’s acquisition office, which officials  stated provides flexibility in scheduling CSBs. In practice, this resulted in  meetings where required members of the CSB did not participate in  discussions of requirements, configuration, or descoping. In 2010, the  Navy service acquisition executive chaired and attended 12 of the 28  CSB meetings and participated in at least 2 others, both CSBs conducted  via paper. According to our review of CSB documentation, six CSB  meetings clearly discussed descoping options, and the service acquisition  executive did not attend any of the five held in person. The sixth meeting  was a paper CSB and it is unclear whether the service acquisition  executive participated. When the Navy service acquisition executive or  others chair the CSB meeting, the principal military deputy typically does  not attend. In addition, at least three CSB meetings in 2010 did not  include a representative from USD (AT&L).
The Navy also allows paper CSBs to fulfill the requirement for an annual  CSB. In four cases, the Navy used paper CSBs to review requirement  and configuration changes sometimes requiring millions of dollars or tens  of millions of dollars in additional funding. According to Navy officials,  Navy policy allows CSB members to reach decisions on issues of  requirements and configuration by circulating briefing slides and  memoranda rather than holding an actual meeting; however, there are not  clear criteria specifying the circumstances under which a program may  hold a paper CSB. Multiple Navy program managers stated that they do  not understand which programs are eligible or when and how to request a  paper CSB. In one case, a program manager stated that although the  program was planning for and preferred a CSB meeting in person, Navy  officials changed the format to a paper CSB a few days before the  scheduled meeting time.
section_title: Two Major Defense Acquisition Programs Are Not Covered By The CSB Statute
paragraphs: Two major defense acquisition programs—the Ballistic Missile Defense  System (BMDS) and the Chemical Demilitarization-Assembled Chemical  Weapons Alternatives programs, which are managed by DOD  components rather than military departments—are not subject to the CSB  provisions in statute because the statute only applies to major defense  acquisition programs overseen by the military departments. However,  DOD acquisition policy, which requires CSBs for all major defense  acquisition programs in development, applies to these programs.
The Missile Defense Agency (MDA), which is responsible for the  management of BMDS, did not hold a CSB for the system in 2010;  however, it did conduct reviews that discussed many of the same issues  and included some of the same participants as those required for CSBs.  The Program Change Board manages the development, fielding, and  integration of BMDS through separate program elements and ensures the  integrity of the system as a whole. This board, which is the primary forum  for discussing and mitigating changes to program elements’ requirements  and configuration, met 42 times in 2010. The Program Change Board is  chaired by the equivalent of a service acquisition executive—the director  of MDA—and, according to an MDA official, includes the equivalent of the  comptroller, the program executive officer, and the program manager.  MDA policy also requires USD (AT&L) to be invited to Program Change  Boards, and allows for the military services’ participation when deemed  appropriate, but does not include the Joint Staff. The Missile Defense  Executive Board oversees implementation of strategic plans and reviews  the priorities and budget for BMDS as a whole. The Missile Defense  Executive Board includes the Joint Staff as well as the MDA director and  an array of Office of the Secretary of Defense (OSD) and military service  representatives, but according to DOD it does not generally discuss  requirements and configuration at the element level. The executive board  met seven times in 2010.
The Assistant Secretary of Defense for Nuclear, Chemical, and Biological  Defense Programs, who is responsible for the management of the  Chemical Demilitarization-Assembled Chemical Weapons Alternatives  program, also did not hold a CSB in 2010. However, a similar board—the  Chemical Demilitarization Program Strategic Governance Board—met  three times in 2010 to discuss program progress, including how it is  performing against its requirements and funding issues, including those  related to significant cost and schedule growth. In 2010, the Assistant  Secretary acted as the chair for this board which also includes  representatives from the OSD comptroller, the Joint Staff, and the Army.
section_title: DOD Policy Is Not Consistent with the CSB Provisions in Statute
paragraphs: The CSB requirements in DOD’s primary acquisition instruction are not  fully consistent with the provisions in statute. Most significantly, the  instruction only requires CSB meetings for major defense acquisition  programs in development, rather than major defense acquisition  programs in development and production. Additionally, the instruction  does not include the comptroller as a CSB member. According to USD  (AT&L) officials, the CSB provisions in statute may not have been fully  incorporated into USD (AT&L)’s December 2008 revision of DOD’s  acquisition instruction because the statute was enacted in October 2008  and there was not enough time to reconcile them. USD (AT&L) is in the  process of updating the instruction and is considering changes to the  CSB requirements. USD (AT&L), according to officials, has also not  consistently tracked whether programs are fulfilling the current  requirements in DOD policy because the statute makes CSBs a military  department responsibility.
section_title: CSB Meetings Had Some Positive Effects on Programs’ Efforts to Control Requirements and Costs
paragraphs: Individual programs varied in the extent to which they utilized CSBs to  control requirements and mitigate cost and schedule risks. According to  our survey results, the majority of CSB meetings neither reviewed  requirement changes nor discussed options to reduce requirements or  the scope of programs. We found a number of instances in which CSB  meetings were effective in mitigating the effect of necessary changes,  rejecting other changes, facilitating discussion of requirements, and  endorsing descoping options with the potential to improve or preserve  cost or schedule. Program managers, however, may be reluctant to  recommend descoping options because of cultural biases about the role  of a program manager, a preference not to elevate decisions to higher  levels of review, and concerns that future funding will be cut. In an effort  to increase descoping proposals, the Army and Air Force have issued  additional descoping guidance and set savings or budget targets. The  perceived effectiveness of the CSB meetings also varied based on the  acquisition phase of a program and which CSB members participated. To  further increase effectiveness and efficiency of CSBs, some of the military  departments have taken steps to coordinate CSB meetings among  programs that provide similar capabilities and align CSB meetings with  other significant reviews.
subsections: section_title: Programs Have Had Some Success in Using CSBs to Control and Reduce Requirements
paragraphs: We identified individual examples from each military department in which  CSB meetings were used to prevent or reject requirements or  configuration changes, mitigate the cost and schedule effects of endorsed  changes, facilitate the prioritization of requirements, and provide program  managers with opportunities to reduce requirements or suggest other  programmatic changes to lower costs and field systems faster. However,  most of the program officials who held CSB meetings and responded to  our survey reported that CSB meetings were not useful for preventing  changes to requirements or configuration, mitigating the potential effects  on cost and schedule when changes were endorsed, or recommending  ways to improve a program’s cost and schedule by moderating  requirements. In interviews with program officials, some explained that  they did not utilize the CSB meetings to control requirements because  they addressed requirement issues as they arose within the program  rather than waiting for their program’s scheduled CSB meeting to occur.  Others stated that their program was stable and that there were no  requirement changes or descoping options to discuss. According to our  survey results, reviews of CSB documentation, and interviews:    26 percent of the programs in our survey with CSB meetings reported  that these meetings were useful forums to prevent changes to  requirements. Moreover, 35 percent reported that the meetings were  useful to make necessary changes to requirements. In an interview,  several program officials stated that the mere suggestion of convening  a CSB meeting to discuss a new requirement was enough to deter  changes.    25 percent of the programs in our survey with CSB meetings reported  that these meetings were useful forums to prevent changes to  technical configuration. Conversely, 23 percent reported that the  meetings were useful to make necessary changes to technical  configurations. Our review of minutes and presentations also show at  least one CSB meeting that rejected a change that had the potential to  adversely affect program cost; the August 2010 CSB review for the  LPD 17 amphibious ship program rejected a proposed configuration  change that would have added new equipment to the ship at an  estimated cost of $26 million.    Some CSB meetings also included discussions of how to prioritize  requirements. For example, according to officials, the Air Force used a  June 2010 CSB meeting for the Global Hawk—an unmanned  surveillance aircraft—to prioritize joint urgent operational needs.  According to program officials, the Global Hawk program has received  numerous requests to add new capabilities to the platform due to its  use in current operations. The program manager stated that the CSB  meeting provided the opportunity to present the costs and benefits of  those requests to decision makers and receive guidance from them on  which ones to pursue or defer.    28 percent of the programs in our survey with CSB meetings reported  that these meetings were useful forums to mitigate the potential cost  and schedule effects of changes brought to the CSB for consideration.  Moreover, 18 percent of programs reported CSB meetings were useful  forums to mitigate the potential cost and schedule effects of changes  made as a result of the CSB. The Vertical Take Off and Landing  Tactical Unmanned Aerial Vehicle program used a CSB meeting to  discuss ways to restructure the program in response to cost growth. At  the meeting, the members of the CSB encouraged the program  manager to go beyond his proposals and investigate changes to  program quantities, contract strategy, and operational plans when  restructuring the program, in order to reduce cost.    CSB meetings seem to have been effective in mitigating the cost and  schedule effects of changes or only endorsing changes that would not  affect costs and schedules. Of the 19 programs in our survey in which  a CSB meeting endorsed changes to requirements or technical  configuration, 1 reported an increase in program cost and 2 reported a  delay in the delivery of an initial operational capability.    30 percent of programs in our survey with CSB meetings reported that  these meetings were useful forums to offer options to lower costs and  field systems faster. Survey results show that descoping options were  presented for 19 programs and those options were endorsed for 8 of  them. For example, at the December 2009 CSB meeting for the Air  Force’s Joint Air-to-Surface Standoff Missile, the program office  recommended adopting the extended range version’s lower reliability  requirement for the baseline missile. The program office stated the  existing baseline requirement, which was 5 percent higher, had the  potential to become a cost driver in testing for the program. The CSB  endorsed the program office’s recommendation.    Program officials also reported that the exercise of formulating  descoping options, regardless of whether or not they were endorsed,  helped their office identify and develop mitigation strategies in the  event costs increased.
Table 3 provides examples of programs across the military departments  that used CSB meetings to endorse requirement, configuration, or other  programmatic changes to improve or preserve cost or schedule.
Program managers may be reluctant to recommend descoping options to  moderate requirements during a CSB meeting because of cultural biases  about the role of a program manager, a preference not to elevate  decisions to higher levels of review, and concerns that future funding will  be cut. According to several acquisition officials, there is a cultural bias  throughout DOD that the role of the program manager is to meet the  requirements handed to them, not to seek to reduce them to achieve cost  savings. In this context, if a program manager recommends reducing  requirements, it may suggest the person is not managing the program or  serving the warfighter well. Still others preferred to reduce requirements  that were within their span of control through their program’s internal  change-management process rather than waiting for a CSB meeting to  ask permission. For example, the DDG-51 program office proposed  changes to the ships’ configuration to reduce cost by removing or  relocating equipment and the CH-53K program avoided cost by relaxing a  requirement for self-sealing fuel tanks. Our interviews with program  officials also suggest that there may be a reluctance to present descoping  options at a CSB meeting because it could be interpreted as an  opportunity to reduce the program’s budget.
section_title: Army and Air Force Are Encouraging More Descoping Options to Reduce Costs
paragraphs: The Army and Air Force have both taken steps to encourage or require  program managers to seek options to lower costs by reducing scope.  Acquisition officials noted that the presentation of descoping options and  the focus on reducing costs has increased in importance since CSBs  were first established, as the budget environment has become more  constrained. In a November 2010 memorandum, the Army emphasized  the need for program officials to aggressively seek descoping  opportunities with the goal of reducing per-unit or total program costs by 5  percent. Army officials stated that the memorandum was signed by senior  leaders from the requirements, acquisition, and budgeting communities  specifically to address the bias that reducing requirements is  unacceptable.
According to officials, the Air Force amended its guidance for CSB  meetings to require programs to present three to four descoping options  along with the effect of those options on performance and program  execution, the dollar amount already invested, and the estimated savings  likely to result. Program managers are instructed to treat the descoping  options as a budgeting exercise and to present the decisions that would  need to be made if the program’s current budget were reduced by 10, 20,  and 30 percent. Several program offices told us that forcing programs to  present options to reduce requirements or scope led them to spend time  preparing options that were not viable or that they would have to  recommend against implementing.
section_title: CSBs’ Effects Differ for Programs in Development and Production
paragraphs: The types of discussions for which CSBs were useful changed based on  whether programs were in development or production. According to our  survey results, programs in development found CSB meetings to be more  useful than programs in production for making necessary changes to  requirements or technical configuration, mitigating the potential cost and  schedule effects of changes, and recommending proposals to improve  program costs and schedule. Table 4 presents our survey results of  program officials’ opinions on the usefulness of CSB meetings.
Programs in development also proposed changes to requirements or  configuration, presented options for reducing scope, and had those  options endorsed at a higher rate than those in production. Even so, an  official for one program in development stated that its CSB meeting was  not effective because the program was meeting cost and schedule targets  and its requirements were narrowly defined, which decreased  opportunities for reducing scope.
According to our survey results, a higher percentage of programs in  production reported that CSBs were useful in preventing changes  compared to programs in development. We have previously reported that  stabilizing a program’s requirements and design well before production is  important because changes have increasingly negative effects on cost  and schedule the further a program progresses. Program officials were  wary about using CSB meetings to try to reduce costs for programs in  production either through requirements changes or reductions in scope  because the configuration should be locked, the available trade space is  probably limited, and potential changes could be disruptive. For instance,  the E-2D program reported in its April 2010 CSB meeting that its  configuration was extremely stable and, with development and  demonstration almost complete, reducing the scope of the programs  could prove detrimental because it could lead to redesigns or decreases  in capability. Changes at this stage of a program can still have a positive  effect on cost if they do not require extensive design changes. For  example, the program manager for the Family of Medium Tactical  Vehicles—which is well into production with over 40,000 vehicles  fielded—recommended removing the self-recovery winch from some  vehicles, resulting in savings of $9,535 per vehicle.
section_title: Broad Senior Level Participation in CSB Meetings Facilitates Decision Making
paragraphs: CSBs provide a unique opportunity for program managers to address  programmatic issues in front of a broad group of high-level decision  makers that includes the acquisition, requirements, and funding  communities. In some cases, the makeup of the CSB helped to  accelerate the resolution of issues and facilitate decision making. For  example, the Grey Eagle program utilized its CSB meeting to endorse an  increase in the number of active units from 13 to 17. The program office  reported that this decision, which otherwise may have taken years to  approve and fund, was made and implemented quickly by the CSB  because of the senior leadership present. Other program offices stated  that the broad membership on CSBs, which includes key stakeholders  and other interested parties, helps to create institutional buy-in for  programmatic changes. CSB meetings also raised stakeholders’  awareness of cost increases. Specifically, CSB meetings provided the  Joint Staff with its first knowledge of cost growth on at least four programs  and triggered separate reviews by the Joint Requirements Oversight  Council.
When critical stakeholders are absent, the decision-making ability of the  CSB may be limited. In particular, some programs with users from across  the military services and organizations external to DOD reported that the  utility of CSBs was limited when those users were not represented. For  example, the primary users of the Air Force’s Global Positioning System  IIIA program include the Army, Navy, and other organizations external to  DOD. The September 2010 CSB meeting for the system did not include  these stakeholders, and program officials stated that as a result, the CSB  was not empowered to make significant changes to the program.
section_title: Aligning CSB Meetings with Other Reviews May Increase Effectiveness and Efficiency
paragraphs: The decisions made at CSB meetings can affect complementary  programs, as well as the funding required for programs. As a result,  acquisition and program officials told us there is value in aligning CSB  meetings so they are held together with reviews of similar programs or  sequencing them to occur before key funding decisions are made. For  example, in 2010, the Army grouped programs into capability portfolios,  such as aviation or precision fires capabilities, and held one CSB meeting  to discuss requirement changes and descoping options for all the  programs. These CSB meetings generally occurred after the Army’s  capability portfolio reviews—which revalidate, modify, or terminate  requirements and ensure the proper allocation of funds across  programs—and reviewed, endorsed, and implemented the  recommendations coming from them. Holding CSB meetings for  capability portfolios can facilitate discussions about interoperability and  interdependency and promote an examination of requirements and  capabilities across programs, including potential redundancies. Officials  also stated that if two well-executed, high-performing programs within the  same portfolio were reviewed independently, those discussions might not  take place. For example, the Army’s Excalibur—a precision-guided  munition—and Guided Multiple Launch Rocket System were both  relatively stable programs in production. However, according to officials,  during a capability portfolio review, the Army identified an overlap in the  two programs’ capabilities and missions and recommended reducing the  number of Excalibur munitions to be procured. At the subsequent April  2010 CSB meeting, the Army reviewed and implemented the proposal,  which reduced the cost of the Excalibur program by $893.5 million.  According to acquisition officials, grouping programs in this manner can  also ease the difficulty of scheduling a large number of meetings that  require senior leadership participation.
According to program officials, when CSB meetings were aligned with  budget deliberations, it enabled an informed discussion of funding issues  and rapid changes to program budgets. USD (AT&L)’s 2007  memorandum establishing CSBs stressed the importance of making  necessary budget adjustments, especially those involving expected  increases in program costs, at the earliest opportunity. In one example,  the Army’s November 2009 CSB for the Patriot and Medium Extended Air  Defense System programs corresponded with the service’s fiscal-year- 2011 budget-formulation process. Program officials stated that this helped  facilitate the transfer of funds and efforts among the two programs, which  had been endorsed by senior leaders from the acquisition and funding  communities at the CSB. However, it may be functionally challenging to  align CSB meetings with the budget formulation process in all cases, as  CSB meetings in some cases must be event driven while the budget  process is calendar driven.
section_title: Conclusions
paragraphs: With the prospect of slowly growing or flat defense budgets for years to  come, DOD must get better returns on its weapon system investments  than it has in the past. CSBs, which are intended to ensure that a  program delivers as much planned capability as possible at or below the  expected cost, can be a key tool in furthering this goal. They represent a  unique forum that brings together a broad range of high-level decision  makers from the acquisition, requirements, and funding communities, who  can make and implement decisions quickly.
DOD’s experience with CSBs to date has already demonstrated their  potential value—costly new requirements have been rejected, and  options to moderate requirements and reduce program costs by millions  of dollars have been endorsed. However, the efficiency and effectiveness  of CSBs can still be improved. Ensuring key CSB members from the  acquisition and requirements community are present at meetings could  help build consensus more quickly and make decisions more efficiently.  Similarly, while the law is silent on whether paper CSB meetings may be  used to meet the annual requirement, holding in-person meetings may be  more effective because a paper meeting may not provide the opportunity  for in-depth discussion or proper oversight. Holding CSBs in conjunction  with capability portfolio reviews and other similar meetings has the  potential to expand opportunities to review and rationalize requirements  across programs. Improving the connection between CSBs and the  budget process and other reviews can help further efforts to match  weapon system requirements with funding resources. Reviewing  programs at CSBs on a case-by-case basis well into production would  help decision makers identify cost savings and shift funding as warfighter  needs and funding priorities change. Taken together, these steps have  the potential to improve not only the efficiency and effectiveness of CSBs  but also the affordability and execution of DOD’s major defense  acquisition programs.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the Secretary of Defense take the following seven  actions directing:  the Navy to amend its policy on CSBs to ensure that all statutorily  required participants, particularly the Joint Staff, are included;  the MDA to amend its policy to ensure that all statutorily required  participants for military department CSBs are included in MDA’s  Program Change Board, particularly the Joint Staff, if it is to serve as  an equivalent review;    USD (AT&L) to amend its acquisition instruction to:    ensure that all statutorily required participants, in particular the  comptroller, are included on CSBs;  require CSB meetings for major defense acquisition programs in  production as well as development but also coordinate with the  military departments and the Congress to evaluate the  effectiveness of CSB meetings for programs well into production;  and    develop the means to better track CSBs and ensure compliance  with the requirement that CSBs hold a meeting at least once each  year;    USD (AT&L) to work with DOD components to determine whether  paper CSBs are as effective as in-person meetings and, if not, amend  the acquisition instruction accordingly; and    DOD components to amend their policies to encourage alignment  between CSB meetings and other complementary reviews whenever  possible.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided us with written comments on a draft of this report. In its  comments, DOD concurred or partially concurred with all seven of our  recommendations and agreed to take action to address six of them. The  comments are reprinted in appendix II. DOD also provided technical  comments, which we addressed in the report, as appropriate.
In concurring with our recommendation that the Navy amend its policy on  CSBs to include all statutorily required participants, DOD stated that the  Navy has already issued two policy memorandums that do so. DOD also  stated that the Navy will continue to issue policy guidance consistent with  our recommendation. This will be particularly important as the Navy is  currently in the process of revising its primary acquisition instruction.
DOD also concurred with our recommendations to amend its acquisition  instruction to ensure that all statutorily required participants are included  in CSBs and that meetings occur for programs in development as well as  those in production. DOD did not address the portion of our  recommendation to coordinate with the military departments and the  Congress to evaluate the effectiveness of CSB meetings for programs  well into production. Given our mixed findings on the utility of CSB  meetings late in production, we continue to believe it would be in the  interest of the department to study this issue.
DOD partially concurred with our recommendation that MDA amend its  policy to ensure that all statutorily required participants for military  department CSBs, in particular the Joint Staff, are included in MDA’s  Program Change Board, if it is to serve as an equivalent review. In its  comments, DOD stated that Joint Staff participation would provide little  value because of the role of the Joint Staff in the acquisition of BMDS. In  addition, DOD pointed out that the Joint Staff participates in the Missile  Defense Executive Board, a forum in which strategic direction and  funding priorities are established. However, we continue to believe that if  the Program Change Board is to act as the forum for discussing  configuration and requirements changes, it is important that the user  communities, as represented by the Joint Staff, participate in these  discussions.
DOD partially concurred with our recommendations on improving the  tracking of CSB meetings, determining the effectiveness of paper CSBs,  and aligning complimentary reviews with CSB meetings, when possible.  In its comments, DOD stated that it would address these issues in “best  practices” guidance to the military departments. With regard to developing  the means to better track CSB meetings and compliance with the  requirement to hold a meeting at least once each year, DOD stated the  best practices guidance will direct the military departments to ensure  adequate tracking vehicles are in place. We continue to believe that USD  (AT&L) should play a role in tracking compliance and holding the military  departments accountable, given our findings that the military departments  did not hold CSBs for all the required programs.
We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Army, Navy, and Air Force; USD (AT&L); and the  Director of the Office of Management and Budget. In addition, the report  will be made available at no charge on the GAO Web site at  http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841. Contact points for our offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. Staff members making key contributions to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
subsections: section_title: Selection and Classification of Major Defense Acquisition Programs
paragraphs: This report presents information on the Department of Defense’s (DOD)  use of Configuration Steering Boards (CSB) for the major defense  acquisition program portfolio in 2010. We used the Defense Acquisition  Management Information Retrieval system to identify 98 active major  defense acquisition programs. We defined an active program as one that  issued a selected acquisition report in December 2009. This report  presents information on all of these programs. One program, the Ballistic  Missile Defense System, is managed by the Missile Defense Agency  (MDA), which reports acquisition information on the system by functional  elements. We reviewed nine elements and analyzed them separately  from the rest of the major programs.
We categorized programs by the five acquisition organizations designated  as having oversight—Army, Navy, Air Force, MDA, and the Assistant  Secretary of Defense for Nuclear, Chemical, and Biological Defense  programs—to assess trends in the use of CSBs. The selected acquisition  report for each program designates the program’s acquisition  organization. As the lead authority for joint programs rotates among the  acquisition organizations as determined by the Office of the Secretary of  Defense, we categorized all joint programs according to the service that  was designated as the lead authority in the December 2009 selected  acquisition report.
All of the programs in our audit fall into one of two phases: engineering  manufacturing and development (referred to as development) or  production and sustainment (referred to as production). Development  generally begins with the initiation of an acquisition program as well as  the start of engineering and manufacturing development and generally  ends with entry into production. Production generally begins with the  decision to enter low-rate initial production. For most programs in our  assessment, the placement of programs in one of these two phases was  determined by the dates of their Milestone B/II and Milestone C/III  decisions. For instance, we categorized programs that have held a  Milestone B/II decision but not a Milestone C/III as in the development  phase and those that have held a Milestone C/III decision as in the  production phase. The dates of milestone decisions for the programs  used in the audit were determined through use of the Defense Acquisition  Management Information Retrieval system.
Due to the nature of individual programs, select programs were not  classified by milestone decision because they either have multiple  increments that may begin production in advance of the notional  Milestone C/III date, or the programs do not report milestone dates. In  these cases, we used the program’s selected acquisition reports to  determine the appropriate phase. The Navy often authorizes shipbuilding  programs to begin production of the lead ship at Milestone B/II. We  classified these programs as in the production phase. As the MDA  programs develop systems’ capabilities incrementally instead of following  the standard DOD acquisition model, we did not identify acquisition  phases for Ballistic Missile Defense System elements.
section_title: Compliance with Statutory Requirements
paragraphs: To assess the extent that DOD has complied with the statutory  requirements for CSB meetings in 2010, we compared CSB execution to  provisions in the statute that call for annual CSB meetings and discussion  of specific content. To determine the extent to which DOD complied with  the requirement to hold an annual CSB for each program, we analyzed  CSB records provided by the acquisition organization we reviewed and,  using these records, calculated the number of CSBs held for each  program in calendar-year 2010. To determine whether the components  established boards that included the statutorily required participants, we  analyzed policy and procedure documentation from each of the  components as well as attendance lists of CSBs held in calendar-year  2010, provided by the acquisition organizations we reviewed. To identify  issues discussed at CSBs and actions resulting from these CSBs, we  reviewed CSB documents and questionnaire data and interviewed  acquisition officials. We also reviewed and analyzed current and draft  documentation related to department and service-level CSB policies,  directives, guidance, and instructions to determine if they establish a  structure that would facilitate compliance with the statute; examples of  these documents include Department of Defense Instruction 5000.02,  Department of the Army Pamphlet 70-3 regarding Army Acquisition  Procedures, SECNAV Instruction 5000.2D, Air Force Instruction 63-101,  and Missile Defense Agency Directive 5010.18 regarding Acquisition  Management.
We also interviewed officials representing organizations that participate in  CSBs or their equivalents including the Office of the Under Secretary of  Defense for Acquisition, Technology and Logistics, Joint Staff, military  service and MDA offices, program offices, and capabilities and  requirements offices to address department, military service, and MDA  policies and execution.
section_title: Effectiveness of Configuration Steering Boards
paragraphs: To assess how effective CSBs have been controlling requirements and  mitigating cost and schedule risks on programs, we analyzed CSB  documentation to identify actions proposed and actions taken as a result  of the CSB and their effect on cost, schedule, performance, and system  configuration. We also asked program officials in our questionnaire to  identify requirement changes or descoping options discussed at the CSB,  the impact of decisions made, perceived effectiveness of the CSB, and  explanations for not conducting a CSB, if applicable. To further analyze  the effectiveness, challenges, and benefits of holding CSBs, we selected  17 programs for interviews. We based our selection on answers to our  questionnaire, discussions with officials, and programmatic factors such  as acquisition organization and phase. Specifically, we met with program  officials at Wright Patterson Air Force Base, Ohio; Redstone Arsenal,  Alabama; Washington Navy Yard in Washington DC; the Naval Air  Station Patuxent River in Patuxent River, Maryland; and conducted video  teleconferences with program officials at Picatinny Arsenal in New Jersey  and at Los Angeles Air Force Base in El Segundo, California. We also  interviewed acquisition officials, reviewed selected acquisition reports,  and examined documentation related to service-level CSB policies,  directives, guidance, and instructions to determine whether other reviews  or acquisition processes influenced the effectiveness of CSBs.
section_title: DOD Major Defense Acquisition Programs Questionnaire
paragraphs: To collect information about DOD’s use of CSBs in fiscal year 2010, we  developed and administered a Web-based questionnaire to the program  offices of all 98 programs. Fiscal-year data was collected in our survey to  be consistent with the Senate report language that contained our  mandate. We administered separate questionnaires to nine Ballistic  Missile Defense System elements and analyzed the results separately  from the rest of the programs in our review. We fielded the survey from  October 2010 to December 2010, and after extensive follow-up, we  received responses from all 98 programs.
Our questionnaire of the 98 program offices, was not a sample  questionnaire, so it has no sampling errors. However, the practical  difficulties of conducting any questionnaire may introduce errors,  commonly referred to as nonsampling errors. For example, difficulties in  interpreting a particular question or limitations in the sources of  information available to respondents can introduce unwanted variability  into the questionnaire results. We took steps in developing the  questionnaire, collecting the data, and analyzing the responses to  minimize such nonsampling errors. For example, social science survey  specialists designed the questionnaire in collaboration with GAO’s  subject-matter experts. We conducted pretests with program managers to  check that (1) the questions were clear and unambiguous, (2) terminology  was used correctly, (3) the questionnaire did not place an undue burden  on agency officials, (4) the information could feasibly be obtained, and (5)  the questionnaire was comprehensive and unbiased. For the pretests, we  selected programs from each military department and from various  phases of the acquisition life cycle. We conducted four pretests. We  made changes to the content and format of the questionnaire after each  pretest, based on the feedback received. When we analyzed the data, an  independent analyst checked all computer programs to reduce risk of  error. Since this was a Web-based questionnaire, respondents entered  their answers directly into the electronic questionnaire, eliminating the  need to key data into a database, minimizing error.
We did not validate the data provided by the program offices, but  reviewed the data and performed various checks to determine that the  data were reliable enough for our purposes. Where we discovered  discrepancies from reviewing responses and interviewing program  offices, we clarified the data with the program office and made changes to  the questionnaire data accordingly.
section_title: Appendix II: Comments from the Department of Defense
section_title: Appendix III: GAO Contact and Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, Ronald E. Schwenn, Assistant  Director; Noah B. Bleicher; MacKenzie Cooper; Morgan Delaney  Ramaker; J. Kristopher Keener; Jean McSween; Kenneth E. Patton; and  Brian Schwartz made key contributions to this report.