Quadrennial Defense Review: Future Reviews Could Benefit from Improved Department of Defense Analyses and Changes to Legislative Requirements
Published: 2007-09-14T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Department of Defense (DOD) is required by law to conduct a comprehensive examination of the national defense strategy, force structure, modernization plans, infrastructure, and budget every 4 years including an assessment of the force structure best suited to implement the defense strategy at low-to-moderate level of risk. The 2006 Quadrennial Defense Review (QDR), completed in February 2006, represents the first comprehensive review that DOD had undertaken since the military forces have been engaged in operations in Iraq and Afghanistan. GAO was asked to assess (1) the strengths and weaknesses of DOD's approach and methodology for the 2006 QDR and (2) what changes, if any, in the QDR legislation could improve the usefulness of the report, including any changes that would better reflect 21st century security conditions. To conduct its review, GAO analyzed DOD's methodology, QDR study guidance, and results from key analyses and also obtained views of defense analysts within and outside of DOD.
section_title: What GAO Found
paragraphs: DOD's approach and methodology for the 2006 QDR had several strengths, but several weaknesses significantly limited the review's usefulness in addressing force structure, personnel requirements, and risk associated with executing the national defense strategy. Key strengths of the QDR included sustained involvement of senior DOD officials, extensive collaboration with interagency partners and allied countries, and a database to track implementation of initiatives. However, GAO found weaknesses in three key areas. First, DOD did not conduct a comprehensive, integrated assessment of different options for organizing and sizing its forces to provide needed capabilities. Without such an assessment, DOD is not well positioned to balance capability needs and risks within future budgets, given the nation's fiscal challenges. Second, DOD did not provide a clear analytical basis for its conclusion that it had the appropriate number of personnel to meet current and projected demands. During its review, DOD did not consider changing personnel levels and instead focused on altering the skill mix. However, a year after the QDR report was issued, DOD announced plans to increase Army and Marine Corps personnel by 92,000. Without performing a comprehensive analysis of the number of personnel it needs, DOD cannot provide an analytical basis that its military and civilian personnel levels reflect the number of personnel needed to execute the defense strategy. Third, the risk assessments conducted by the Secretary of Defense and the Chairman of the Joint Chiefs of Staff, which are required by the QDR legislation, did not fully apply DOD's risk management framework because DOD had not developed assessment tools to measure risk. Without a sound analytical approach to assessing risk, DOD may not be able to demonstrate how it will manage risk within current and expected resource levels. As a result, DOD is not in the best position to demonstrate that it has identified the force structure best suited to implement the defense strategy at low-to-moderate risk. Through discussions with DOD officials and defense analysts, GAO has identified several options for refining the QDR legislative language that Congress could consider to improve the usefulness of future QDRs, including changes to encourage DOD to focus on high priority strategic issues and better reflect security conditions of the 21st century. Congress could consider options to clarify its expectations regarding what budget information DOD should include in the QDR and eliminate reporting elements for issues that could be addressed in different reports. For example, the requirement to assess revisions to the unified command plan is also required and reported under other legislation. Further, some reporting elements such as how resources would be shifted between two conflicts could be eliminated in light of DOD's new planning approach that focuses on capabilities to meet a range of threats rather than on the allocation of forces for specific adversaries. GAO also presents an option to have an advisory group work with DOD prior to and during the QDR to provide DOD with alternative perspectives and analyses.

section_title: Letter
paragraphs: Among the 21st century challenges facing the Department of Defense  (DOD) and the nation are difficult decisions concerning how to strike an  affordable balance between national security and domestic needs. Aided  by annual and supplemental funding of over $400 billion per year since  fiscal year 2003, DOD has been maintaining a high pace of operations  while simultaneously transforming its military forces to meet emerging  threats of the new security environment. However, as we have emphasized  in previous reports, the federal government now faces increasing fiscal  challenges, and DOD may face increasing competition for federal dollars.  The 2006 Quadrennial Defense Review (QDR) represents the first  comprehensive review of the national defense strategy that DOD has  undertaken since military forces have been engaged in operations in Iraq  and Afghanistan. Further, the 2006 QDR provided an opportunity for DOD  to move beyond its long-standing approaches and methods and identify  the capabilities required to meet current, emerging, and future threats.
The QDR is a key component of national security planning. To ensure that  the country’s defense needs are reviewed periodically, Congress directed  DOD to conduct comprehensive QDRs every 4 years to examine elements  of the defense program and policies of the United States including the  national defense strategy, force structure, modernization, infrastructure,  and budget plan. Key assessments required during the review that relate  to national security planning include: (1) the force structure best suited to  implement the defense strategy at low-to-moderate level of risk; (2) the  budget plan that would be required to provide sufficient resources to  execute successfully the full range of missions called for in the national  defense strategy at a low-to-moderate level of risk; (3) the Secretary of  Defense’s assessment of the nature and magnitude of the political,  strategic, and military risks associated with executing the missions called  for under the national defense strategy; and (4) the Chairman of the Joint  Chiefs of Staff’s assessment of risk.
DOD submitted its report on the third quadrennial review to Congress on  February 6, 2006. In the report, DOD concluded that “For the foreseeable  future, steady-state operations including operations as part of a long war  against terrorist networks and associated rotation base and sustainment  requirements will be the main determinant for sizing U.S. forces.” DOD  also confirmed that for the long term it must size and shape U.S. forces for  three main types of missions: homeland defense, the war on  terrorism/irregular warfare, and conventional campaigns. In addition,  DOD acknowledges that it must implement departmentwide change to  ensure that organizational structures, processes, and procedures  effectively support its strategic priorities. For example, DOD created the  Defense Business Transformation Agency to integrate and oversee  corporate-level business systems and initiatives in areas such as  acquisition and logistics. Further, DOD reached several key decisions in  the 2006 QDR that emphasized the need to continue changing the mix of  joint capabilities and forces, such as stabilizing Army and Marine Corps  active duty personnel at fiscal year 2006 congressionally authorized levels  while increasing special operations forces in areas such as civil affairs  units and special forces battalions, and military personnel for sea, air, and  land teams.
DOD viewed the 2006 QDR as a refinement of the concepts it introduced in  its 2001 QDR report, such as shifting the basis of force planning from  focusing on specific adversaries and geographic locations to capabilities- based planning that identifies the capabilities the military will need to  prepare for a range of potential military operations against unknown  enemies. In the 2001 QDR report, DOD introduced a risk management  framework designed to help address the tension between preparing for  future threats and meeting the demands of the present with finite  resources and to size, shape, and manage the department to accomplish its  strategic priorities. DOD planned to use the framework in conducting the  2006 QDR.
Our work on past QDRs has shown long-standing weaknesses in DOD’s  assessment of force structure requirements. In past QDRs, DOD has not  focused on longer-term threats and requirements for support capabilities,  and its QDR reports have provided little information on some required  issues, such as assumptions used in its analyses. Moreover, we have  reported that force structure decisions were not clearly supported by  analysis and linked to strategic plans. Further, in November 2005, we  reported that DOD has not fully implemented a risk management approach  and it planned to refine its risk management framework during the 2006  QDR.
In 2006, Congress passed legislation which added new reporting elements  that will apply to the next QDR in 2010 as well as future QDRs. For  example, the Secretary of Defense must establish an independent panel to  conduct a postreview assessment of the QDR including the  recommendations, assumptions used, and vulnerabilities of the strategy  and force structure underlying the review. The new legislation also  required that the Secretary of Defense submit to the Senate and House  Committees on Armed Services quarterly reports on the status of the  department’s implementation of the 2006 QDR decisions, beginning in  January 2007. DOD submitted its first and second quarterly reports to  Congress on January 31, 2007, and June 14, 2007, respectively.
You asked us to evaluate DOD’s overall approach and supporting analysis  in preparing the 2006 QDR and assess whether the QDR legislative  requirements could be reevaluated to improve the usefulness of the report,  including any changes needed to better reflect the security conditions of  the 21st century. Accordingly, this report assesses (1) the strengths and  weaknesses of DOD’s approach and methodology for the 2006 QDR and  (2) what changes, if any, in the QDR legislation could improve the  usefulness of future QDRs.
To assess the 2006 QDR’s strengths and weaknesses, we reviewed DOD’s  study guidance for the QDR and discussed the advantages and  disadvantages of the review’s process with DOD’s officials. We also  examined the methodology and results of the QDR key analyses and  assessed how capabilities-based planning principles were applied during  the assessments. To understand how the Secretary of Defense and the  Chairman of the Joint Chiefs of Staff conducted their risk assessments, we  reviewed the QDR’s study guidance on assessing risk and held discussions  with officials responsible for conducting risk assessments during the QDR.  We reviewed DOD’s quarterly report to Congress on the status of  implementation for the 2006 QDR and post-QDR study team reports and  implementation plans to review the processes that DOD has to implement  QDR initiatives. Further, we held discussions with Office of the Secretary  of Defense (OSD) officials responsible for monitoring the status of  initiatives related to the QDR and ongoing work in the post-QDR study  teams. To determine whether changes to the QDR legislation could  improve the usefulness of future QDRs, we identified potential options  from our analyses of prior QDRs and obtained the views of DOD civilian  and military leaders who participated in the 2006 QDR as well as  nongovernmental defense analysts, many of whom had played key roles in  previous QDRs or in prior defense strategy reviews. We performed our  review from May 2006 through May 2007 in accordance with generally  accepted government auditing standards. Further information on our  scope and methodology appears in appendix I.
subsections: section_title: Background
paragraphs: In May 1995, the Commission on Roles and Missions of the Armed Forces  proposed the idea of a comprehensive quadrennial review by DOD of the  country’s defense strategy and force structure. In August 1995, the  Secretary of Defense endorsed the idea, and the following year legislation  directed DOD to conduct the 1997 QDR.
Congress created a permanent requirement for DOD to conduct a QDR  every 4 years in the National Defense Authorization Act for Fiscal Year  2000, passed in 1999. According to this legislation, DOD was to conduct a  comprehensive examination of the national defense strategy, force  structure, force modernization plans, infrastructure, budget plan, and  other elements of the country’s defense program and policies with a view  toward determining and expressing the nation’s defense strategy and  establishing a defense program for the next 20 years. Originally the  legislation identified 14 specific issues for DOD to address, such as a  comprehensive discussion of the national defense strategy of the United  States and the force structure best suited to implement that strategy at a  low-to-moderate level of risk. In addition, it allowed the Secretary of  Defense to review any other issues he considers appropriate. The  legislation in effect during the 2006 QDR reflected several amendments to  the original legislation, for example, requiring DOD to assess the national  defense mission of the Coast Guard. (See app. II for the legislation in  effect during the 2006 QDR.)
Among other requirements, the 1999 QDR legislation required that the  Secretary of Defense assess the nature and magnitude of the political,  strategic, and military risks associated with executing the missions called  for under the national defense strategy. In the 2001 QDR report, DOD  introduced a new risk management framework that identified four areas of  risk—operational, force management, future challenges, and institutional.  According to the 2001 QDR report, the framework would enable DOD to  address the tension between preparing for future threats and meeting the  demands of the present with finite resources. Further, the framework was  intended to ensure that DOD was sized, shaped, postured, committed, and  managed with a view toward accomplishing the strategic priorities of the  2001 QDR.
Future QDRs will be affected by the new reporting elements added to the  QDR legislation by the John Warner National Defense Authorization Act  for Fiscal Year 2007. Specifically, the legislation requires DOD to  establish an independent review panel to conduct an assessment of the  QDR no later than 6 months before the date that DOD’s report on the QDR  is submitted to Congress. The panel is required to submit, within 3 months  after the date on which the QDR is submitted, an assessment of the  review, including its recommendations, the stated and implied  assumptions incorporated in the review, and the vulnerabilities of the  strategy and force structure underlying the review. The legislation also  specifies that the QDR review should not be constrained to comply with  the budget submitted to Congress by the President. In addition, the  legislation added several specific issues that DOD is required to address  such as providing the specific capabilities, including the general number  and type of specific military platforms, needed to achieve the strategic and  warfighting objectives. Lastly, the authorization act directs DOD to submit  to the Senate and House Armed Services Committees a report on the  implementation of recommendations identified in the 2006 QDR report no  later than 30 days after the end of each fiscal year quarter. (See app. III  for a summary of additions to the QDR legislation, 10 U.S.C. §118 as a  result of the John Warner National Defense Authorization Act for Fiscal  Year 2007.)
DOD considers the 2006 QDR a refinement of its predecessor 2001 QDR,  which detailed the department’s intent to shift the basis of defense  planning from the long-standing “threat-based” model, which focused on  specific adversaries and geographic locations, to a “capabilities-based”  construct that seeks to prepare for a range of potential military operations  against unknown enemies. According to the 2001 QDR report, the  capabilities-based model focuses on how an adversary might fight rather  than specifically who the adversary might be or where the war might  occur.
The Under Secretary of Defense (Policy) had the lead role in conducting  the 2006 QDR. The Joint Staff played a supporting role in the process and  had primary responsibility for leading the analytical work to support the  Chairman of the Joint Chief of Staff’s risk assessment. In March 2005, the  Secretary of Defense approved guidance, called the Terms of Reference,  for the review. The Terms of Reference identified four focus areas and  provided guidance to senior officials to develop capabilities and make  investment decisions to shape the future force and reduce risks in these  areas. The four focus areas were 1) defeating terrorist networks,   2) defending the homeland in depth, 3) shaping the choices of countries at  strategic crossroads, and 4) preventing hostile states and nonstate actors  from acquiring or using weapons of mass destruction. During the spring of  2005, DOD senior leaders held meetings on the focus areas with  interagency partners from across the federal government and international  allies to identify the potential threats and the types of capabilities needed  to address the challenges associated with the focus areas. Officials from  the intelligence community, such as the Defense Intelligence Agency,  provided threat assessments for each of the focus areas.
The Terms of Reference also established six study teams to assess  capabilities associated with the QDR focus areas and directed the teams to  develop options to reduce risk in these areas. Top-level civilian and  military leaders from OSD and Joint Staff led the study teams, which  included officials from the services and Combatant Commands. The  Deputy Secretary of Defense and the Vice Chairman of the Joint Chiefs of  Staff co-chaired a senior level group, which was eventually referred to as  the Deputy’s Advisory Working Group, and this group reviewed the work  of the study teams during the summer and fall of 2005. Other members of  the review group included the Under Secretaries of Defense, the services’  Under Secretaries, the services’ Vice Chief of Staffs, and the Deputy  Commander, U.S. Special Operations Command. The Deputy Secretary  and his working group determined what information each study team  would provide to the senior-level review group, which was led by the  Secretary of Defense. Figure 1 shows the structure that OSD established to  conduct the QDR.
According to the 2006 QDR report, the foundation of this QDR is the  National Defense Strategy, published in March 2005. The Secretary of  Defense’s National Defense Strategy is implemented through the National  Military Strategy, which is developed by the Chairman of the Joint Chiefs  of Staff. The National Military Strategy provides focus for military  activities by defining a set of interrelated military objectives from which  the service chiefs and combatant commanders identify desired capabilities  and against which the Chairman of the Joint Chiefs of Staff assesses risk.
section_title: The QDR’s Approach Had Several Strengths but Analytical Weaknesses Limited Its Usefulness in Assessing Force Structure, Personnel Requirements, and Risk
paragraphs: While DOD’s approach and methodology for the 2006 QDR had several  strengths, several weaknesses significantly limited the review’s usefulness  in addressing force structure, personnel requirements, and risk associated  with executing the national defense strategy. On the positive side, the 2006  QDR benefited from the sustained involvement of key senior DOD  officials, interagency and allied participation, and internal collaboration  among the QDR’s participants. However, weaknesses in the assessment of  three key areas—force structure, personnel requirements, and risk— hampered DOD’s ability to undertake a fundamental reassessment of the  national defense strategy and U.S. military forces. As a result of these  weaknesses, Congress lacks assurance that DOD has conducted the  analysis needed to determine the force best suited to implement the  defense strategy. Further, DOD is not well positioned to demonstrate to  Congress how it considered risks and made difficult trade-offs among its  capabilities to balance investments within future budgets, given the  nation’s fiscal challenges.
subsections: section_title: Senior Leadership Involvement, Interagency and Allied Participation, Internal Collaboration, and a System to Monitor Implementation Provided Benefits
paragraphs: DOD’s approach for the 2006 QDR benefited from several strengths. First,  key senior DOD leaders maintained sustained involvement throughout the  review. As we have noted in previous reports, best practices clearly  indicate that top-level leadership is crucial for engineering major changes  in an organization. Top leaders establish the framework for change and  provide guidance and direction to others to achieve change. During the  2006 QDR process, the Deputy Secretary of Defense and the Vice  Chairman of the Joint Chiefs of Staff co-chaired a senior level review  group, now referred to as the Deputy’s Advisory Working Group, to review  and approve initiatives of varying complexity presented by the six study  team leaders and leaders of specialized issue areas, such as special  operations forces. According to an official in the Office of the Secretary of  Defense, during most of the QDR process, this senior level group met  several times a week to review the study teams’ options and provide  guidance to the teams to ensure that the QDR’s strategic priorities were  addressed. Since the QDR report was issued in February 2006, the  Deputy’s Advisory Working Group continues to meet regularly to oversee  implementation of the QDR’s strategic priorities, such as improving DOD’s  management structures and business processes to support effective  decision making.
Second, DOD collaborated with interagency partners, such as the  Department of Homeland Security, and U.S. international allies, such as  the United Kingdom, to discuss potential strategic challenges and  determine capabilities that are required to meet current and future  challenges. According to DOD officials, senior officials from the  Department of Homeland Security including the U.S. Coast Guard and the  Departments of Energy, State, and other federal agencies participated in  DOD’s discussions establishing the strategic direction of the QDR during  the spring of 2005. U.S. agency officials discussed with DOD officials the  types of capabilities and investments needed to reduce risk in the QDR’s  four focus areas—defeating terrorist networks, defending the homeland in  depth, shaping the choices of countries at strategic crossroads, and  preventing hostile states and nonstate actors from acquiring or using  weapons of mass destruction. For example, DOD officials who  coordinated the QDR stated that U.S. Coast Guard officials identified  current and planned maritime defense capabilities as part of DOD’s  discussion on combating weapons of mass destruction. Further, officials  from U.S. allies, such as the United Kingdom, participated in the  discussions to share their perspectives about how DOD, its allies, and  global partners could address the nontraditional, asymmetric warfighting  challenges of the 21st century, such as preventing the acquisition or use of  weapons of mass destruction by nonstate actors. As a result of  contributions from the interagency partners and allies, DOD was in a  better position to identify and develop the four focus areas that eventually  shaped the scope of the QDR.
Third, leaders of the six study teams collaborated with each other to avoid  duplication of work as they developed options to address challenges  associated with the focus areas. The study team leaders held weekly  meetings to discuss whether their issues could be better addressed by  another study team, the progress of their work plans, and whether they  could provide each other with mutually supporting analysis. Further, a  group of senior officials, led by an official in the Office of the Secretary of  Defense for Policy, attended the study teams’ weekly meetings to ensure  that the options addressed the capabilities associated with the four focus  areas and helped identify overlaps or gaps in the development of options.  For example, three study teams, which developed and identified options  related to force structure, personnel requirements, and roles and missions  respectively, coordinated their work to minimize any overlap and identify  any gaps in the development of options to increase the number of military  and civilian personnel proficient in key languages such as Arabic, Farsi,  and Chinese.
Fourth, following the release of the 2006 QDR, the Deputy Secretary of  Defense requested that officials in OSD establish procedures to track the  implementation of the 2006 QDR initiatives which encompassed a range of  military capabilities, from implementing its new personnel management  system to developing a new land-based, penetrating long-range strike  capability by 2018. Senior officials from the Office of the Director,  Administration and Management created a departmentwide database and  established criteria to categorize the implementation status of each  initiative. Specifically, implementation of an initiative was categorized as  “completed” if the initiative was fully implemented or if DOD had taken  actions that officials determined as having met the intent of the initiative,  even though the initiative may take years to fully implement. OSD officials  have provided periodic briefings on the status of QDR initiatives to the  Deputy Secretary of Defense and his advisory group since the publication  of the 2006 QDR report. DOD reported to Congress in January 2007 that it  had completed implementation of about 90, or 70 percent, of the 130  initiatives. Further, in January 2006 at the end of the QDR process, the  Deputy Secretary of Defense identified eight study areas and established a  process to continue developing DOD’s approaches for the issues  associated with these study areas. According to senior DOD officials,  these areas identified for post-QDR study were generally complex and  involved multiple organizations, such as developing interoperable strategic  communications. The Deputy Secretary provided guidance for the teams  that included requirements to (1) define objectives, timelines, and  performance metrics and (2) establish an oversight process as part of an  implementation plan to ensure the decisions made during the QDR were  achieved. According to DOD officials, DOD plans to provide Congress with  information about the status of the post-QDR study teams’ implementation  in its quarterly reports. For example, in DOD’s January 2007 report to  Congress, DOD reported that one of the Institutional Reform and  Governance study team’s objectives is to continue developing concepts  and overseeing initiatives related to reforming governance and  management functions such as capabilities-based planning.
section_title: Weaknesses in Assessment of Force Structure, Personnel Requirements, and Risk Limited the QDR’s Usefulness in Linking Force Structure to the Defense Strategy and Addressing Affordability Challenges
paragraphs: Weaknesses in the assessment of three key areas—force structure,  personnel requirements, and risk—significantly limited the review’s  usefulness in reassessing the force structure best suited to implement the  defense strategy at low-to-moderate level of risk, which is a key  requirement of the review. Our previous reporting on DOD’s prior QDRs  and other work has shown that weaknesses in establishing a substantive  basis for force structure, personnel requirements, and risk have been long- standing issues for the department. Further, until DOD can demonstrate an  analytical basis for its force structure and personnel requirements, it will  not be well-positioned to balance capability needs within budgets that are  likely to be constrained in the future, given the nation’s fiscal challenges.
subsections: section_title: DOD Did Not Conduct a Comprehensive, Integrated Analysis of Alternative Force Structures Using a Capabilities- Based Approach
paragraphs: Although the 2006 QDR study guidance emphasized that DOD would use  capabilities-based planning to focus on how a range of potential enemies  might fight, DOD did not conduct a comprehensive, integrated assessment  of alternative force structures during the QDR using a capabilities-based  approach. Based on our discussions with DOD officials and our review of  DOD documents and non-DOD published studies, a capabilities-based  approach requires a common understanding of how a capability will be  used, who will use it, when it is needed, and why it is needed. Further,  each capability should be assessed based on the effects it seeks to  generate and the associated operational risk of not having the capability.  A capabilities-based approach also seeks to identify capability gaps or  redundancies and make trade-offs among the capabilities in order to  efficiently use fiscal resources. In table 1 we identify several key elements  of a capabilities-based planning approach and provide descriptions of  these elements.
DOD’s primary basis for assessing the overall force structure best suited to  implement the national defense strategy, according to several DOD  officials, was a Joint Staff-led study known as Operational Availability 06.  The study compared the number and types of units in DOD’s planned force  structure to the operational requirements for potential scenarios to  determine whether and to what extent the planned force structure   would experience shortages. However, the Joint Staff’s Operational  Availability 06 Study did not assess alternatives to planned force  structures and evaluate trade-offs among capabilities.
In conducting the Operational Availability 06 Study, the Joint Staff  completed two different analyses. The first analysis, referred to as the  base case, relied on a set of operational scenarios that created  requirements for air, ground, maritime, and special operations forces.  During this study, the Joint Staff examined requirements for a broad range  of military operations over a 7-year time frame. Two overlapping  conventional campaigns served as the primary demand for forces with  additional operational demands created by 23 lesser contingency  operations, some of which represented the types of operations that  military forces would encounter while defending the homeland and  executing the war on terrorism. The Joint Staff then compared the number  of military units in DOD’s planned air, ground, maritime, and special  operations forces to the operational demands of the scenarios. The Joint  Staff made two key assumptions during the analysis. First, the Joint Staff  assumed that reserve component units could not deploy more than once in  6 years. Second, the Joint Staff assumed that while forces within each  service could be reassigned or retrained to meet shortfalls within the force  structure, forces could not be substituted across the services. Results of  the Joint Staff’s first analysis showed that maritime forces were capable of  meeting operational demands and air, ground, and special operations  forces experienced some shortages.
In response to a tasking from top-level officials the Joint Staff performed a  second analysis that developed a different set of operational demands   reflecting the high pace of operations in Iraq. In this analysis, the Joint  Staff used the same 2012 planned force structure that was examined in the  first analysis. When it compared the operational demands that were  similar to those experienced in Iraq with DOD’s planned force structure,  the Joint Staff found that the air, ground, maritime, and special operations  forces experienced shortages and they could only meet operational  demands for a security environment similar to Iraq, one conventional  campaign, and 11 of the 23 lesser contingency scenarios.
While the Operational Availability 06 Study had some benefits, several  weaknesses significantly limited the study’s usefulness for integrating a  capabilities-based approach that assessed force structure options. On the  positive side, top leaders maintained sustained involvement in the  Operational Availability Study; for example, based on their guidance, the  Joint Staff conducted a second analysis that depicted operational  demands, which more accurately represented the current security  environment. That study demonstrated that significant shortages in  military forces exist when forces are not retrained or reassigned to meet  operational demands. However, weaknesses in the study’s methodology to  assess different levels of force structure and use a capabilities-based  planning approach limited the study’s usefulness in reassessing the  fundamental relationship between the national defense strategy and the  force structure best suited to implement the strategy. First, the Joint Staff  did not vary the number and types of units to demonstrate that it assessed  different levels or mixes of air, ground, maritime, and special operations  force structure in its second analysis. Second, the Joint Staff did not  identify capabilities of the force structure and make recommendations  about trade-offs among capabilities.
Further, concurrent with the Operational Availability 06 Study, DOD  conducted separate assessments of some segments of its force structure to  inform decisions about investments for capabilities. For example, DOD  conducted a departmentwide study that assessed options about different  levels and types of tactical air assets, such as the Joint Strike Fighter.  However, in this study DOD did not fully address whether and to what  extent future investment plans are affordable within projected funding  levels, and in April 2007, we reported that DOD does not have a single,  comprehensive, and integrated investment plan for recapitalizing and  modernizing fighter and attack aircraft. In another example, DOD also  conducted a study to determine whether ground forces in the Army,  Marine Corps, and Special Operations Command could meet operational  demands for a broad range of scenarios without relying extensively on  reserve personnel. However, options to increase ground forces were not  part of the study’s scope, and the implications of the ongoing operations in  Iraq, such as the number of active brigade combat teams that would be  needed and their length of time in theater, were not fully considered.
A key reason why DOD did not use an integrated capabilities-based  approach to assessing force structure options is that DOD did not have a  unified management approach to implement capabilities-based planning  principles into the QDR assessment. At the time of the QDR, no one  individual or office had been assigned overall responsibility and authority  necessary for implementing an integrated capabilities-based planning  approach. Further, DOD had not provided comprehensive written  guidance to implement departmentwide methods for capabilities-based  planning that specifies the need to identify capabilities at the appropriate  level of detail, identify redundant or excess capabilities that could be  eliminated, facilitate trades among capabilities, assess and manage risk,  and balance decisions about trade-offs with near- and long-term costs.  Currently, DOD is undertaking some initiatives related to capabilities- based planning. However, these select initiatives do not represent the type  of comprehensive, unified management approach needed to assess the  force structure requirements to address a range of potential military  operations against unknown enemies. For example:    The Joint Staff initiated the Joint Capabilities Integration and  Development System in 2003 to assess gaps in joint capabilities and  recommend solutions to resolve those gaps. Under this system, boards  comprised of high-level DOD civilians and military officials are convened  to identify future capabilities needed in key functional areas, such as  battle space awareness, and to make recommendations about trade-offs  among air, space, land, and sea platforms. While this process may be  important to assess gaps in joint warfighting capabilities, we have reported  that its focus is to review and validate the initial need for proposed  capabilities. However, we have also reported that the process is not yet  functioning as envisioned to define gaps and redundancies in existing and  future military capabilities across the department and to identify solutions  to improve joint capabilities. Further, we reported that programs  assessed by the Joint Staff’s process build momentum and move toward  starting product development with little if any early department-level  assessment of the costs and feasibility. According to senior DOD officials,  the Joint Staff’s process does not thoroughly link capabilities to the  strategic priorities identified in the QDR.
The Deputy Secretary of Defense tasked the Institutional Reform and  Governance post-QDR study team to develop departmentwide approaches  that would allow DOD to integrate and facilitate its capabilities-based  planning initiatives. Based on the study team’s work, in March 2007 the  Deputy Secretary of Defense tasked several DOD organizations to develop  plans to facilitate a capabilities-based planning approach. For example, the  Joint Requirements Oversight Council is tasked with developing a process  for identifying capability priorities and gaps at the appropriate level of  detail and ranking all capabilities from high to low priority by October  2007. Further, the Deputy Secretary of Defense has reaffirmed the  department’s commitment to portfolio management and expanded the  scope of responsibility for the four capability portfolio test case  managers. Among their new responsibilities, each portfolio manager is  required to provide the Deputy’s Advisory Working Group with an  independent portfolio assessment to inform investment decisions during  DOD’s fiscal year 2009 program review. DOD may establish more  portfolios as the roles and responsibilities of the existing managers evolve  and operate in DOD’s existing decision processes, such as the Deputy’s  Advisory Working Group.
DOD made some changes to the current force structure to address  perceived gaps in capabilities based on the QDR review, although these  did not represent major changes to the composition of the existing force  structure. For example, among the key force-structure-related decisions  highlighted in the QDR were to (1) increase Special Operations forces by  15 percent and the number of Special Forces Battalions by one-third;   (2) expand Psychological Operations and Civil Affairs units by 3,700  personnel, a 33 percent increase; (3) develop a new land-based penetrating  long-strike capability to be fielded by 2018 and fully modernize the current  bomber force (B-52s, B-1s, and B-2s); and (4) decrease the number of  active component brigade combat teams from 43 to 42 and the number of  planned Army National Guard brigade combat teams from 34 to 28. In  January 2007—about a year after the QDR was completed—DOD  approved the Army’s plan to increase the number of active component  brigade combat teams to 48. Since DOD did not conduct a comprehensive,  data-driven assessment of force structure alternatives during the QDR, it is  not in the best position to assure itself or Congress that it has identified  the force best suited to execute the national defense strategy.
section_title: DOD Did Not Conduct a Thorough Review of Personnel Requirements
paragraphs: Although DOD concluded in the 2006 QDR report that the size of today’s  forces—both the active and reserve components across all four military  services—was appropriate to meet current and projected operational  demands, it did not provide a clear analytical basis for its conclusion. In  January 2007, the Secretary of Defense announced plans to permanently  increase the size of the active component Army and the Marine Corps by a  total of 92,000 troops over the next 5 years. But again, DOD did not  identify the analysis that it used to determine the size of the increase. In  February 2005, we recommended that DOD review active personnel  requirements as part of the QDR, and in doing so, discuss its conclusions  about the appropriate personnel levels for each of the services and  describe the key assumptions guiding the department’s analysis, the  methodology used to evaluate requirements, and how the risks associated  with various alternative personnel force levels were evaluated. While DOD  agreed with our recommendation, it did not perform a comprehensive,  data-driven analysis of the number of personnel needed to implement the  defense strategy as part of its 2006 QDR. Until DOD performs a  comprehensive review of personnel requirements, it cannot effectively  demonstrate to Congress a sound basis for the level of military and civilian  personnel it requests.
Our prior work has shown that valid and reliable data about the number of  personnel required to meet an agency’s needs are critical because human  capital shortfalls can threaten an organization’s ability to perform missions  efficiently and effectively. Data-driven decision making is one of the  critical factors in successful strategic workforce management. High- performing organizations routinely use current, valid, and reliable data to  inform decisions about current and future workforce needs, stay alert to  emerging mission demands, and remain open to reevaluating their human  capital practices. Further, federal agencies have a responsibility to provide  thorough analytical support over significant decisions affecting  requirements for federal dollars so that Congress can effectively evaluate  the benefits, costs, and risks.
Rather than conducting a comprehensive assessment of its personnel  requirements, DOD’s approach to active and reserve military personnel  and civilian personnel levels was to limit growth and initiate efforts to use  current personnel levels more efficiently. Consequently, the study team  that was assigned to review issues related to manning and balancing the  force took the existing force size as a given.  From that basis, the study  team identified alternative courses of action for changing the mix of  specific skills, such as civil affairs, in the active and reserve components to  meet future operational requirements. The team also considered whether  changes in the mix of skills would require more military and civilian  personnel at headquarters staffs. While these reviews are important for  understanding how to use the force more efficiently, they cannot be used  to determine whether U.S. forces have enough personnel to accomplish  missions successfully because these reviews did not systematically assess  the extent to which different levels of end strength could fill DOD’s  combat force structure and provide institutional support at an acceptable  level of risk.
Although DOD’s 2006 QDR concluded that the Army and Marine Corps  should plan to stabilize their personnel levels at 482,400 and 175,000 active  personnel respectively, by 2012, in February 2007 the President’s fiscal  year 2008 budget submission documented a plan to permanently increase  the size of the active components of the Army by 65,000 to 547,400 and the  Marine Corps by 27,000 to 202,000 over the next 5 years; and the Army  National Guard by 8,200 to 358,200 and the U.S. Army Reserve by 6,000 to  206,000 by 2013. Shortly after the increase was announced, we testified  before Congress that DOD’s record in providing an analytically driven  basis for requested military personnel levels needs to be improved and  suggested that Congress should carefully weigh the long-term costs and  benefits in evaluating DOD’s proposal for the increases. Both the Army  and Marine Corps are coping with additional demands that were not fully  reflected in the QDR. For example, the Marine Corps decided to initiate a  new study to assess active military personnel requirements shortly after  the 2006 QDR was completed due to its high pace of operations and the  QDR-directed changes in force structure, such as establishing a Special  Operations Command requiring about 2,600 military personnel. Without  performing a comprehensive analysis of the number of personnel it needs,  DOD cannot ensure that its military and civilian personnel levels reflect  the number of personnel needed to execute the defense strategy. Further it  cannot ensure that it has a sufficient basis for understanding the risks  associated with different levels of military and civilian personnel. For  example, while too many active military personnel could be inefficient and  costly, having too few could result in other negative consequences, such as  the inability to provide the capabilities that the military forces need to  deter and defeat adversaries.
section_title: DOD Did Not Conduct an Analytically Sound Risk Assessment of Its Proposed Force Structure
paragraphs: During the 2006 QDR, the risk assessments conducted by the Secretary of  Defense and the Chairman of the Joint Chiefs of Staff did not fully apply  DOD’s risk management framework to demonstrate how risks associated  with its proposed force structure were evaluated. DOD introduced its risk  management approach in 2001; however, we have reported that it has  faced difficulty implementing this approach. For example, we found that  DOD faced challenges in integrating its risk management framework and  reform initiatives into a unified management approach. We have reported  that an emerging challenge for the federal government involves the need  for completion of comprehensive national threat and risk assessments in a  variety of areas. For example, evolving requirements from the changing  security environment, coupled with increasingly limited fiscal resources  across the federal government, emphasize the need for agencies to adopt a  sound approach to establishing resource decisions. We have advocated  that the federal government, including DOD, adopt a comprehensive risk  management approach as a framework for decision making that fully links  strategic goals to plans and budgets, assesses values and risks of various  courses of actions as a tool for setting priorities and allocating resources,  and provides for the use of performance measures to assess outcomes. A  risk management approach represents a series of analytical and  managerial steps that can be used to assess risk, evaluate alternatives for  reducing risks, choose among those alternatives, implement the  alternatives, monitor their implementation, and that incorporate new  information to adjust and revise the assessments and actions, as needed.  Further, such a data-driven risk assessment can provide a guide to help  shape, focus, and prioritize investment decisions to develop capabilities.
A key reason why DOD did not apply its risk framework during the QDR is  that it had difficulty in developing department-level measures that would  be necessary to assess risk and as a result, the assessment tools were not  available for use during the QDR. The QDR’s study guidance tasked the  QDR coordination group, led by officials in the Office of the Under  Secretary of Defense (Policy), to review the QDR risk management  guidelines and provide these guidelines to the QDR’s study teams for  review. The guidelines were to provide some examples of how to measure  performance related to DOD’s key areas identified in its framework— operational, force management, institutional, and future challenges. The  QDR coordination group was to incorporate the study teams’ feedback  about recommended changes. Lastly, the QDR coordination group was to  issue the guidelines and monitor the application of performance measures  during the QDR. According to an official in the Office of the Under  Secretary of Defense (Policy), the QDR coordination group had difficulty  developing the measures and thus did not issue guidelines. As a result, the  study teams did not have the assessment tools to assess risk during the  QDR.
Since department-level measures for assessing risk were not available  during the 2006 QDR, several of the study teams relied primarily on  professional judgment to assess the risks of not investing in various  capabilities. For example, the study team responsible for developing  capabilities told us that they examined information about potential future  threats and determined that DOD needed medical countermeasures to  address the threat of genetically engineered biological agents. Members of  the study team discussed the consequences of not developing the medical  procedures and treatments that would be needed to increase survival rates  if U.S. military personnel were to encounter the highly advanced genetic  material.  Further, the Chairman of the Joint Chiefs of Staff was not tasked  to use the risk management framework in assessing risks and did not  choose to use it in his assessment. Rather, the Chairman’s assessment  examined the extent to which the 2006 QDR initiatives would address  combatant commanders’ operational needs for potential future  requirements.
Without a sound analytical approach to assess risk during future QDRs,  DOD will not have a sufficient basis to demonstrate how the risks  associated with the capabilities of its proposed force structure were  evaluated. Further, DOD may be unable to demonstrate how it will manage  risk within current and expected resource levels. Without an analytically  based risk assessment, DOD may not be able to prioritize and focus the  nation’s investments to combat 21st century security threats efficiently  and wisely.
section_title: Options for Modifying Some Legislative Requirements Could Improve Usefulness of Future QDRs
paragraphs: The security environment of the 21st century has been characterized by  conflicts that are very different from traditional wars among states. This  environment has created the need for DOD to reexamine the fundamental  operations of the department and the capabilities needed to continue to  execute its missions. In addition, DOD has created new organizations,  such as the U.S. Northern Command and the Assistant Secretary of  Defense for Homeland Defense, to counter new threats to the homeland  and support the federal response to any potential catastrophic event,  natural, or man-made. Through our discussions with defense analysts, we  have identified options for modifying several QDR legislative requirements  that could be considered in light of the changed security environment, to  make the QDR process and report more useful to Congress and DOD. The  QDR legislation contains numerous issues for DOD to address, some that  require reporting on broad issues, such as the national defense strategy  and the force structure needed to execute that strategy, and some that are  more detailed, such as the requirement that DOD examine the appropriate  ratio of combat forces to support forces under the national defense  strategy. Many defense analysts we spoke with thought some of the  strategic issues are of great importance and should remain for future  QDRs. Further, they believe DOD should focus its efforts on providing  more information on the analytic basis for its key assumptions and  strategic planning decisions. However, they also asserted that several of  the QDR’s detailed reporting elements detract attention from strategic  issues, are already required and reported under other laws, or are no  longer relevant in the new security environment. Options to improve the  usefulness of future QDRs include (1) clarifying expectations for how the  QDR should address the budget plan, (2) eliminating some reporting  elements for the QDR legislation that could be addressed in different  reports, (3) eliminating some reporting elements in the QDR legislation for  issues that may no longer be as relevant due to changes in the security  environment, and (4) establishing an independent advisory group to work  with DOD prior to and during the QDR to provide alternative perspectives  and analyses.
subsections: section_title: Strategic Focus of Some QDR Legislative Requirements Is Considered Useful for Establishing Defense Plans and Programs
paragraphs: Several defense analysts we spoke with asserted that the permanent  requirement for DOD to conduct a comprehensive strategic review of the  defense program every 4 years is important and that Congress should  continue to require that DOD conduct future QDRs. Moreover, several  defense analysts acknowledge that certain key requirements remain  critical to the QDR’s purpose of fundamentally reassessing the defense  strategy and program. Specifically, the requirements that task the  Secretary of Defense to (1) delineate a defense strategy and (2) define  sufficient force structure, force modernization, budget plan, and other  elements of a defense program that could successfully execute the full  range of missions called for by the defense strategy at low to medium risk  over 20 years were seen as critical elements needed to ensure that  Congress understands DOD’s strategies and plans. Several defense  analysts told us that it is in the national interest to ensure that DOD  conducts the kind of long-range strategic planning that can provide  meaningful recommendations for meeting future national security  challenges and that enables debate on the costs and benefits of  requirements for future military and capabilities as well as risks in  capability gaps in light of national fiscal challenges.
The QDR legislation also directs DOD to define the nature and magnitude  of the political, strategic, and military risks associated with executing the  missions called for under the national defense strategy in the QDR and  include a comprehensive discussion of the force structure best suited to  implement that strategy at low-to-moderate level of risk. Analysts saw  these areas as important for DOD to provide Congress the assurance that  there is a sound analytical basis for its risk assessment that includes how  DOD identified risks and evaluated alternatives for reducing risks.  Additionally, analysts viewed this discussion as important in assuring that  the department has incorporated a variety of perspectives in its risk  assessments. Some analysts stated that the requirements to discuss the  assumed or defined national security interests, the threats to the assumed  or defined national security interests, and the scenarios developed in the  examination of those threats are several key elements that should remain  to enable the department to demonstrate that principles of risk assessment  have been addressed. Similarly, analysts suggest that the Chairman of the  Joint Chiefs of Staff’s requirement to assess the results of the QDR review,  including an independent assessment of risk, is helpful to provide another  assessment that DOD and Congress can use to understand the risks  associated with the force structure and consider the courses of actions the  department might want to take to reduce risks.
section_title: Legislative Options Are Available to Improve Usefulness of Future QDRs
paragraphs: Some DOD defense analysts told us that the QDR legislation includes  numerous detailed requirements that may impede DOD’s focus on high- priority areas. Based on our discussions with analysts, we identified  several options that Congress should consider to enhance the focus of  future QDRs on high-priority issues and improve the thoroughness of  DOD’s analysis:    Clarify expectations for how the QDR should address the budget  plan that supports the national defense strategy. The QDR  legislation has several reporting elements that relate to budget planning to  support the defense strategy. First, the QDR legislation requires DOD “to  delineate a national defense strategy…” and “to identify the budget plan  that would be required to provide sufficient resources to execute  successfully the full range of missions called for in that national defense  strategy at a low-to-moderate level of risk.” Second, the legislation  requires DOD “to conduct a comprehensive examination…of the national  defense strategy…with a view toward establishing a defense program for  the next 20 years.” Third, based on recent changes to the legislation that  will apply to the next QDR in 2010 as well as future QDRs, DOD is required  to “make recommendations that are not constrained to comply with the  budget submitted to Congress by the President.”
Some defense analysts raised concerns about whether these reporting  requirements provide sufficient and clear guidance for DOD to use in  conducting QDRs. For example, they questioned whether the planning  time frame of 20 years established by the QDR legislation is most useful in  providing Congress with information to perform its oversight of the  defense program. Although DOD officials and defense analysts  acknowledged the benefits of forecasting threats and capabilities for a   20-year period, they stated it would be difficult to develop a detailed  budget plan for a 20-year period given the uncertain nature of threats in  the new security environment. Further, analysts asserted that rather than  enabling DOD to set strategic priorities without regard to current budgets,  the requirement to “make recommendations that are not constrained to  comply with the budget..,” could lead the services and the capability  portfolio managers to push for inclusion of every program in their plans.  This could make it more difficult for DOD to prioritize investments to meet  key capability needs and assess the affordability of new capabilities across  the department.
Moreover, DOD’s three QDR reports since 1997 have not fully described  DOD’s methodology or approach for assessing its budget needs or budget  plans that explained how DOD intended to fund the full range of missions  called for in the national defense strategy. For the 2006 QDR, DOD  included several QDR initiatives in the President’s fiscal year 2007 budget  that was submitted to Congress at the same time as the QDR report but  stated that it would continue to define a budget plan for the QDR by  identifying the funding details in DOD’s future years defense program for  fiscal years 2008 through 2013. In addition, the report did not provide  information about the extent to which DOD considered the long-term  affordability of the overall defense program. We have emphasized in  previous reports that the federal government now faces increasing fiscal  challenges, and DOD may face increasing competition for federal dollars.  Further, in November 2005, we reported that DOD has not demonstrated  discipline in its requirements and budgeting processes, and its costly plans  for transforming military operations and expensive acquisitions may not  be affordable in light of the serious budget pressures facing the nation.  For example, we reported that DOD’s planned annual investment in  acquisition programs it has already begun is expected to rise from   $149 billion in fiscal year 2005 to $178 billion in fiscal year 2011. Given  these pressures, Congress may want a clearer view of how DOD should  budget for the capabilities associated with the proposed force structure,  and how it evaluated the trade-offs in capabilities to maximize the  effectiveness of future investments. If Congress decides that it needs  additional budget-related information to carry out its oversight of future  QDRs, then it might consider clarifying the reporting element relating to  the required budget plan to specify what information DOD should include  in the QDR. Further, Congress may want to consider clarifying its  expectations for the information DOD provides in the QDR as to how it  has addressed the long-term affordability challenges of transforming  military operations.
Eliminate some reporting elements in the QDR legislation for  issues that could be addressed in different reports. According to  some defense analysts, some requirements contained in the QDR  legislation are not essential to the strategic purpose of the QDR and may  divert DOD’s focus from that strategic purpose. While important, some  reporting elements are already examined in other DOD reviews, and  Congress has access to the results of these periodic reviews. These  reporting elements include the following:   An evaluation of “the strategic and tactical airlift, sealift, and ground  transportation capabilities required to support the national defense  strategy.” In November 2002 we reported that the QDR may not be the  appropriate venue for addressing mobility issues because examination  of this issue requires detailed analysis that can best be conducted after  DOD decides on a defense strategy, identifies a range of planning  scenarios consistent with the new strategy, and completes its detailed  analysis of requirements for combat forces. Furthermore, DOD  routinely conducts analyses of its mobility requirements outside of the  QDR process, according to DOD officials. Since 1992, DOD has issued  four major analyses of the U.S. military strategic lift requirements: the  1992 Mobility Requirements Study, Bottom Up Review; the 1995  Bottom Up Review Update; the 2001 Mobility Requirements Study— 2005, issued in 2001; and the Mobility Capability Study, issued in 2005.
An assessment of the “advisability of revisions to the Unified Command  Plan as a result of the national defense strategy.” DOD has a process  for assessing the Unified Command Plan and is required to report  changes to the plan to Congress under other legislation. Specifically,  the Chairman of the Joint Chiefs of Staff is required to review  periodically and not less than every 2 years the missions,  responsibilities, and forces of each combatant command and  recommend any changes to the President, through the Secretary of  Defense. This legislation also requires that, except during times of  hostilities or the imminent threat of hostilities, the President notify  Congress not more than 60 days after either establishing a new  combatant command or significantly revising the missions,  responsibilities, or force structure of an existing command. As such, a  major event or change in the political or security landscape could  trigger the need for a change in the plan. For example, in the spring of  2007, the President announced that DOD intends to establish a U.S.  Africa Command to oversee military operations on the African  continent. According to an OSD official, DOD will revise the 2002  Unified Command Plan and report on the changes in the military  command structure after plans for U.S. Africa Command are more fully  developed.
Eliminate some reporting elements in the QDR legislation for  issues that may no longer be as relevant due to changes in the  security environment. As we reported in our assessment of DOD’s 2001  QDR, a DOD official and some defense analysts said that two reporting  elements should be eliminated because they are related to the allocation of  forces under the old two-major-theater-war construct, which is more  limited than DOD’s current force planning construct that includes a  broader range of threats. These reporting elements include the following:   A discussion of the “appropriate ratio of combat forces to support  forces (commonly referred to as the ‘tooth-to-tail ratio’) under the  national defense strategy.” DOD’s goal has been to reduce the number  of personnel and costs associated with the support forces, or “tail.”  However, during the 2006 QDR process and report DOD did not  identify which units should be considered support and which should be  considered combat. Given rapidly changing technologies,  differentiating between support and combat troops has become  increasingly irrelevant and difficult to measure. For example, as the  United States moves toward acquiring greater numbers of unmanned  aircraft piloted from remote computer terminals and relies increasingly  on space-based assets operated by personnel in the United States, it  will be more difficult to distinguish between combat and support  personnel.
Assessments of “the extent to which resources must be shifted among  two or more theaters under the national defense strategy in the event  of conflict in such theaters,” and the assumptions used regarding  “warning time.” Both these reporting elements relate to the allocation  of forces under the old two-major-theater-war planning construct.  Under this construct, the amount of time that was assumed available  for warning and the separation time between major theater wars were  critical factors in planning the size and composition of U.S. forces and  assessing operational risk, particularly for assets that might need to be  shifted between theaters. However, under the new defense strategy,  along with DOD’s new force planning construct, DOD assumes that it  will continue to be involved in a wide range of military operations  around the world. Given the full spectrum of threats that DOD is  planning to address, it may be more useful for DOD’s force structure  assessments to be tied to requirements for a broad range of potential  threats.
Establish an independent advisory group to work with DOD prior  to or during the QDR to provide alternative perspectives and  analyses. As part of our assessment of the 1997 QDR, we suggested that a  congressionally mandated panel, such as the 1997 National Defense Panel,  could be used to encourage DOD to consider a wider range of strategy,  force structure, and modernization options. Specifically, we noted that  such a review panel, if it preceded the QDR, could be important because it  is extremely challenging for DOD to conduct a fundamental reexamination  of defense needs, given that its culture rewards consensus building and  often makes it difficult to gain support for alternatives that challenge the  status quo. One of the recent additions to the QDR legislation requires the  establishment of an independent panel to conduct an assessment of future  QDRs after the process is completed; however, most defense analysts we  spoke with agreed that an independent analysis of key issues for the  Secretary of Defense either prior to or during the next review would  complement a post-QDR assessment and strengthen DOD’s ability to  develop its strategic priorities and conduct a comprehensive force  structure and capabilities analysis. The analysts agreed that an advisory  group established before or during the QDR process could function as an  independent analytical team to challenge DOD’s thinking, recommend  issues for DOD to review and review assumptions, and provide alternative  perspectives in activities such as identifying alternative force structures  and capabilities, and performing risk assessments. An independent group’s  assessments could be useful to DOD in future QDRs to identify the  capabilities of the nation’s current and future adversaries because  potential enemies will likely be more difficult to target than the  adversaries of the Cold War era.
section_title: Conclusions
paragraphs: The 2006 QDR represented an opportunity for DOD to perform a  comprehensive review of the national defense strategy for the first time  since military forces have been engaged in the Global War on Terrorism.  Sustained DOD leadership facilitated decision making, and extensive  collaboration with interagency partners and allies provided a range of  perspectives on threats and capabilities. However, weaknesses in DOD’s  analysis of force structure, personnel requirements, and risk limited its  reassessment of the national defense strategy and U.S. military forces. For  example, by not fully incorporating capabilities-based planning into a  comprehensive assessment of alternative force structures, DOD could not  comprehensively identify capabilities gaps, associated operational risks,  and trade-offs that must be made to efficiently use limited fiscal resources.  Therefore, DOD was not in a good position to assure Congress that it  identified the force best suited to execute the national defense strategy.  Moreover, the Secretary of Defense’s announcement of plans to increase  the sizes of the Army and Marine Corps in January 2007 calls into question  the analytical basis of the QDR conclusion that the number of personnel  and the size of the force structure for the services were appropriate to  meet current and future requirements. Further, without a comprehensive  approach to assessing risk, DOD’s 2006 QDR did not provide a sufficient  basis to demonstrate how risks associated with its proposed force  structure were evaluated. Unless DOD takes steps to provide  comprehensive analytical support for significant decisions in future QDRs,  the department will not be in the best position to distinguish between the  capabilities it needs to execute the defense strategy versus those  capabilities it wants but may not be able to afford at a time when the  nation’s fiscal challenges are growing. Moreover, Congress will be unable  to effectively evaluate the benefits, costs, and risks associated with  decisions flowing from future QDRs.
Opportunities exist for Congress to consider further changes to the  QDR legislation that may encourage DOD to concentrate its efforts on  high-priority matters such as developing a defense strategy and identifying  the force structure best suited to execute the strategy. Unless Congress  clearly identifies its expectations for DOD to develop a budget plan that  supports the strategy, DOD may not thoroughly address the challenges it  will face as it competes with other federal agencies and programs for  taxpayers’ dollars and may spend considerable effort assessing options for  capabilities that could be unaffordable given our nation’s fiscal challenges.  Moreover, the large number of reporting elements in the QDR legislation  presents DOD with a challenge in conducting data-driven comprehensive  analyses of many significant complex issues. A reassessment of the QDR’s  scope could provide greater assurances that DOD will thoroughly assess  and report on the most critical security issues that the nation faces and  could help it decide what actions it needs to take to establish the most  effective military force to counter 21st century threats. Lastly, although  Congress has established a new legislative requirement for an independent  panel to conduct a post-QDR review, there is currently no mechanism for  Congress and the Secretary of Defense to obtain an independent  perspective prior to and during the QDR. Without an independent group of  advisors that could provide comprehensive data-driven analyses to DOD  prior to and during future QDR reviews, DOD may not consider a wider  range of perspectives, such as force structure options, thus limiting the  analytic basis of its QDR decisions.
section_title: Recommendations for Executive Action
paragraphs: To enhance the usefulness of future QDRs and assist congressional  oversight, we recommend that the Secretary of Defense take the following  two actions:    Develop appropriate methods for the department to use in a  comprehensive, data-driven capabilities-based assessments of alternative  force structures and personnel requirements during future QDRs.
Develop appropriate methods for the department to use in conducting a  comprehensive, data-driven approach to assess the risks associated with  capabilities of its proposed force structure during future QDRs.
section_title: Matters for Congressional Consideration
paragraphs: To improve the usefulness of future QDRs, Congress should consider  revisions to the QDR legislation, including (1) clarifying expectations on  how the QDR should address the budget plan that supports the national  defense strategy, (2) eliminating some detailed reporting elements that  could be addressed in different reports and may no longer be relevant due  to changes in the security environment, and (3) requiring an independent  panel to provide advice and alternatives to the Secretary of Defense before  and during the QDR process.
section_title: Agency Comments and Our Evaluation
paragraphs: The Principal Deputy Under Secretary of Defense for Policy provided  written comments on a draft of this report. The department partially  agreed with our recommendations and agreed with the matters we raised  for congressional consideration regarding possible changes to the QDR  legislative language. In addition, the comments provided information  about steps the department is taking to update its methodologies for  analyzing force structure requirements and assessing risks. DOD’s  comments are reprinted in their entirety in appendix IV. DOD also  provided technical comments which we incorporated as appropriate.
In its comments, the department partially agreed with our  recommendation that the Secretary of Defense develop appropriate  methods for conducting comprehensive, data-driven capabilities-based  assessments of alternative force structures and personnel requirements.  DOD agreed with our conclusion that the 2006 QDR did not  comprehensively assess alternatives to planned structure; rather, its  analysis was limited to identifying shortfalls in current structure when  compared to various illustrative operational scenarios. However, in its  comments, the department noted that it has developed or is developing  new illustrative security environments to use to demonstrate the demands  associated with force structures and personnel requirements for each  strategic environment. The department also pointed out the difficulty of  undertaking an evaluation of the defense strategy and producing a defense  program within the QDR process, as required under current QDR  legislation. It said that as the department further develops the underlying  assumptions for the force planning construct and refreshes the illustrative  scenarios available for analysis, it will be in a better position to analyze  overall needed capabilities, including personnel requirements. Finally, the  department noted that the 2006 QDR was based on information available  in 2005, which included a different demand than what military forces face  today. At that time, the department’s collective decision, approved by the  then Secretary of Defense, was that the size of the force was about right,  although the force mix should be adjusted. As a result of this change in  demand since the 2006 QDR, according to DOD’s comments, DOD has  responded by increasing Army and Marine Corps end strength. We believe  that the steps DOD outlined in its comments, such as revising the  illustrative scenarios and developing force demands for new security  environments, will help DOD to improve its force structure analyses.  However, we believe that a comprehensive assessment that identifies and  documents the basis for trade-off decisions across capability areas is  critical to developing the force structure best suited to execute the  defense strategy. Until DOD undertakes a comprehensive assessment of  alternative force structure options that clearly documents how the  department reached its force structure decisions, it will not be in the best  position to determine the force structure best suited to execute the  missions called for in the defense strategy at low-to-moderate risk.
DOD also partially concurred with our recommendation to develop  appropriate methods for conducting comprehensive, data-driven  assessments of the risks associated with the capabilities of its proposed  force structure during future QDRs. In its comments, the department  agreed that improving the department’s risk methodology is necessary to  appropriately assess risk. It noted that in addition to risks associated with  capabilities, strategic, operational, force management, and institutional  risks need to be addressed in a risk assessment methodology. The  department cited several post-QDR initiatives the department is  undertaking to improve how the department assesses risk, including new  measures to help link strategic goals to plans and budgets and develop  performance metrics. Also in its comments, the department described  efforts to strengthen and integrate existing assessments to allow decision  makers to better set priorities, allocate resources, and assess outcomes  and risks and stated its intent to improve risk assessment methods to  inform risk measurement in future QDRs. We agree that assessing risk  associated with capabilities is only one type of risk facing the department  and that the initiatives the department is undertaking to link strategic  goals with plans and budgets and improve its risk assessment  methodology can, when implemented, help it improve its ability to identify  and manage risks. Until the department’s risk management framework is  sufficiently developed that it can support comprehensive assessments of  risk across domains, assess progress toward accomplishing strategic  goals, and provide senior leaders reliable analysis to inform decisions  among alternative actions, DOD will not be in the best position to identify  or assess risks to establish investment priorities.
DOD also provided its views on matters we raised for congressional  consideration in a draft of this review regarding possible revisions to the  QDR legislation. Specifically, DOD agreed with clarifying expectations for  addressing the budget plan and eliminating some reporting requirements.  In a draft of this report, we originally raised as a matter for congressional  consideration broadening the QDR legislation by requiring the legislatively  required independent advisory panel, which would provide a post-QDR  critique of the results of the process, to provide DOD with alternative  perspectives and analysis prior to or during the QDR. The department  stated that having an independent panel that could provide advice and  alternatives to the Secretary of Defense before and during the QDR  process would be useful. However, it raised the concern that tasking the  same independent panel that is required to provide a post-QDR critique to  also perform an advisory function before and during the review could  create mistrust between the department leadership and the independent  advisory panel. To address DOD’s concerns we have modified the matter  for consideration to suggest that an independent panel be required to  provide advice and alternatives to the Secretary of Defense before and  during the QDR. This change is intended to provide Congress with the  flexibility to establish separate independent panels to provide advice prior  to and following the next QDR.
We are sending copies of this report to other appropriate congressional  committees and the Secretary of Defense. We will also make copies  available to other interested parties upon request. In addition, the report  will be available at no charge on the GAO Web site at http://www.gao.gov.  If you have any questions about this report, please contact me at   (202) 512-4402. Contact points for our Offices of Congressional Relations  and Public Affairs may be found on the last page of this report. Major  contributors to this report are listed in appendix V.
section_title: Appendix I: Scope and Methodology
paragraphs: To assess the strengths and weaknesses of the Department of Defense’s  (DOD) approach and methodology for the 2006 Quadrennial Defense  Review (QDR), we examined the relevant documentation including the  John Warner National Defense Authorization Act for Fiscal Year 2007; the  National Defense Strategy of the United States of America (March 2005);  the 1997, 2001, and 2006 QDRs; the QDR Terms of Reference (March 2005);  the Under Secretary of Defense (Policy) issue papers for the QDR’s focus  areas; and the 2006 QDR’s study teams’ briefings and other documentation  for the DOD’s senior-level review group, as well as our reports on aspects  of previous QDRs. We also examined documents identifying the  methodology and results of the QDR’s key force structure analyses and  risk assessments. We reviewed studies on capabilities-based planning and  compared the key elements of capabilities-based planning identified in the  studies to the QDR’s Terms of Reference and DOD’s documented  methodology for the Operational Availability 06 Study to assess the extent  to which capabilities-based planning concepts were used during the QDR.  We also discussed these issues with officials from the Office of the Under  Secretary of Defense (Policy); the Office of Program Analysis and  Evaluation; the Joint Chiefs of Staff Directorate for Force Assessment;  U.S. Special Operations Command; and officials from the Army, Air Force,  and Marine Corps who participated in the QDR process. To understand  how DOD established processes to ensure that QDR initiatives are  implemented, we examined internal DOD documents, DOD’s January 2007  quarterly report to Congress on the status of implementation of the 2006  QDR, and post-QDR study teams’ reports to understand the methodology  that was developed to oversee implementation. We discussed the  implementation status of the QDR initiatives with officials from the Office  of the Director, Administration and Management and the Under Secretary  of Defense (Policy). We did not undertake an assessment of the  effectiveness of implementation of the QDR initiatives because it was  outside of scope of our review. We obtained and examined documents  from the Deputy Secretary of Defense and the post-QDR study teams and  discussed the status of the teams’ work with officials from the Under  Secretary of Defense (Policy), the Institutional Reform and Governance  team, and the Joint Command and Control and Battlespace Awareness  capability portfolios. Moreover, we reviewed the internal controls on  DOD’s tracking system for QDR initiatives and evaluated the reliability of  that data for DOD’s use. We applied evidence standards from the generally  accepted government auditing standards in our evaluation of DOD’s  database. As a result, we determined the information we used meets these  evidence standards and is sufficiently reliable for our purposes.
To determine whether changes to the QDR legislation could improve the  usefulness of future reviews including any changes needed to better reflect  the security conditions of the 21st century, we examined a wide variety of  studies that discussed the strengths and weaknesses of DOD’s 2006 QDR  and prior reviews. Our review included studies from the RAND  Corporation, the National Defense University, and the Center for Strategic  and Budgetary Assessments. To obtain opinions and develop options to  improve the usefulness of future QDRs, we interviewed several DOD  officials who participated in the 2006 QDR from the services and the Joint  Staff. Further, we met with 11 defense analysts who had detailed  knowledge of DOD’s QDR process and/or participated in DOD’s 1997,  2001, or 2006 QDRs. We used a standard set of questions to interview each  of these analysts to ensure we consistently discussed the reporting  elements of the QDR legislation and DOD’s approach and methods for its  three QDRs. To develop the questions, we reviewed the QDR legislation,  DOD’s QDR reports, and our prior work on DOD’s strategic reviews. One  of the defense analysts served in various positions within and outside of  DOD such as the former Chairman of the Defense Science Board and the  Chairman of the 1997 National Defense Panel. Other defense analysts were  senior officials from the following organizations: the American Enterprise  Institute, the Center for American Progress, the Center for Naval Analysis,  the Center for a New American Security, the Center for Strategic and  Budgetary Assessments, the Center for Strategic and International Studies,  the Lexington Institute, the National Defense University’s Institute for  National Strategic Studies, the RAND Corporation, and the Heritage  Foundation. Based on our review of QDR literature and our discussions  with DOD analysts, we developed a matrix summarizing these individuals’  concerns regarding the QDR legislative requirements and their views on  the options to address them.
Our work was conducted in the Washington, D.C., metropolitan area and  Tampa, Florida. We performed our review from May 2006 through May  2007 in accordance with generally accepted government auditing  standards.
section_title: Appendix II: Quadrennial Defense Review Legislation in Effect for the 2006 Quadrennial Defense Review
paragraphs: TITLE 10 U.S.C. §118. Quadrennial Defense Review  (a) Review required.—The Secretary of Defense shall every four years,  during a year following a year evenly divisible by four, conduct a  comprehensive examination (to be known as a “quadrennial defense  review”) of the national defense strategy, force structure, force  modernization plans, infrastructure, budget plan, and other elements of  the defense program and policies of the United States with a view toward  determining and expressing the defense strategy of the United States and  establishing a defense program for the next 20 years. Each such  quadrennial defense review shall be conducted in consultation with the  Chairman of the Joint Chiefs of Staff.  (b) Conduct of review.—Each quadrennial defense review shall be  conducted so as—  (1) to delineate a national defense strategy consistent with the most  recent National Security Strategy prescribed by the President pursuant  to section 108 of the National Security Act of 1947 (50 U.S.C.404a);  (2) to define sufficient force structure, force modernization plans,  infrastructure, budget plan, and other elements of the defense program  of the United States associated with that national defense strategy that  would be required to execute successfully the full range of missions  called for in that national defense strategy;  (3) to identify (A) the budget plan that would be required to provide  sufficient resources to execute successfully the full range of missions  called for in that national defense strategy at a low-to-moderate level  of risk, and (B) any additional resources (beyond those programmed in  the current future-years defense program) required to achieve such a  level of risk; and  (c) Assessment of risk.—The assessment of risk for the purposes of  subsection (b) shall be undertaken by the Secretary of Defense in  consultation with the Chairman of the Joint Chiefs of Staff. That  assessment shall define the nature and magnitude of the political,  strategic, and military risks associated with executing the missions called  for under the national defense strategy.  (d) Submission of QDR to Congressional committees.—The Secretary  shall submit a report on each quadrennial defense review to the  Committees on Armed Services of the Senate and the House of  Representatives. The report shall be submitted in the year following the  year in which the review is conducted, but not later than the date on which  the President submits the budget for the next fiscal year to Congress under  section 1105(a) of title 31. The report shall include the following:  (1) The results of the review, including a comprehensive discussion of  the national defense strategy of the United States, the strategic  planning guidance, and the force structure best suited to implement  that strategy at a low-to-moderate level of risk.  (2) The assumed or defined national security interests of the United  States that inform the national defense strategy defined in the review.  (3) The threats to the assumed or defined national security interests of  the United States that were examined for the purposes of the review  and the scenarios developed in the examination of those threats.  (4) The assumptions used in the review, including assumptions relating  to— (A) the status of readiness of United States forces; (B) the  cooperation of allies, mission-sharing and additional benefits to and  burdens on United States forces resulting from coalition operations;  (C) warning times; (D) levels of engagement in operations other than  war and smaller-scale contingencies and withdrawal from such  operations and contingencies; and (E) the intensity, duration, and  military and political end-states of conflicts and smaller-scale  contingencies.  (5) The effect on the force structure and on readiness for high-intensity  combat of preparations for and participation in operations other than  war and smaller-scale contingencies.  (6) The manpower and sustainment policies required under the  national defense strategy to support engagement in conflicts lasting  longer than 120 days.  (7) The anticipated roles and missions of the reserve components in  the national defense strategy and the strength, capabilities, and  equipment necessary to assure that the reserve components can  capably discharge those roles and missions.  (8) The appropriate ratio of combat forces to support forces  (commonly referred to as the ‘tooth-to-tail’ ratio) under the national  defense strategy, including, in particular, the appropriate number and  size of headquarters units and Defense Agencies for that purpose.  (9) The strategic and tactical air-lift, sea-lift, and ground transportation  capabilities required to support the national defense strategy.  (10) The forward presence, pre-positioning, and other anticipatory  deployments necessary under the national defense strategy for conflict  deterrence and adequate military response to anticipated conflicts.  (11) The extent to which resources must be shifted among two or  more theaters under the national defense strategy in the event of  conflict in such theaters.  (12) The advisability of revisions to the Unified Command Plan as a  result of the national defense strategy.  (13) The effect on force structure of the use by the armed forces of  technologies anticipated to be available for the ensuing 20 years.  (14) The national defense mission of the Coast Guard.  (15) Any other matter the Secretary considers appropriate.  (e) CJCS review.—(1) Upon the completion of each review under  subsection (a), the Chairman of the Joint Chiefs of Staff shall prepare and  submit to the Secretary of Defense the Chairman’s assessment of the  review, including the Chairman’s assessment of risk.  (2) The Chairman shall include as part of that assessment the Chairman’s  assessment of the assignment of functions (or roles and missions) to the  armed forces, together with any recommendations for changes in  assignment that the Chairman considers necessary to achieve maximum  efficiency of the armed forces. In preparing the assessment under this  paragraph, the Chairman shall consider (among other matters) the  following:  (A) unnecessary duplication of efforts among the armed forces.  (B) changes in technology that can be applied effectively to warfare.  (3) The Chairman’s assessment shall be submitted to the Secretary in time  for the inclusion of the assessment in the report. The Secretary shall  include the Chairman’s assessment, together with the Secretary’s  comments, in the report in its entirety.
section_title: Appendix III: Summary of New Changes in 10 U.S.C. §118 for Future Quadrennial Defense Reviews
paragraphs: This appendix provides a summary of changes to the Quadrennial Defense  Review (QDR) legislation (10 U.S.C. §118) as a result of the John Warner  National Defense Authorization Act for Fiscal Year 2007. The new  requirements will be in effect when the Department of Defense issues its  next quadrennial review in 2010.
The QDR should make recommendations that are not constrained to  comply with the budget submitted to Congress by the President.
The review shall include the following new reporting elements:  the specific capabilities, including the general number and type of  specific military platforms, needed to achieve the strategic and  warfighting objectives identified in the review; and the homeland defense and support to civil authority missions of the  active and reserve components, including the organization and  capabilities required for the active and reserve components to  discharge each such mission.
The Chairman shall describe the capabilities needed to address the risk  that he identified in his risk assessment.
The Secretary of Defense shall establish an independent panel to conduct  an assessment of the QDR not later than 6 months before the date on  which the QDR will be submitted.
Not later than 3 months after the date on which the QDR is submitted,  the panel shall submit an assessment of the review, including the  review’s recommendations, the stated and implied assumptions  incorporated in the review, and the vulnerabilities of the strategy and  force structure underlying the review.
The panel’s assessment shall include analyses of the trends,  asymmetries, and concepts of operations that characterize the military  balance with potential adversaries, focusing on the strategic  approaches of possible opposing forces.
section_title: Appendix IV: DOD Comments
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact name above, Margaret Morgan, Assistant  Director; Deborah Colantonio; Alissa Czyz; Nicole Harms; Elizabeth  Morris; Brian Pegram; Rebecca Shea; and John Townes made major  contributions to this report.
section_title: Related GAO Products
paragraphs: Tactical Aircraft: DOD Needs a Joint and Integrated Investment  Strategy. GAO-07-415. Washington, D.C.: April 2, 2007.
Best Practices: An Integrated Portfolio Management Approach to   Weapon System Investments Could Improve DOD’s Acquisition  Outcomes. GAO-07-388. Washington, D.C.: March 30, 2007.
United States Government Accountability Office: Supporting the  Congress through Oversight, Insight, and Foresight. GAO-07-644T.  Washington, D.C.: March 21, 2007.
Fiscal Stewardship and Defense Transformation. GAO-07-600CG.  Washington, D.C.: March 8, 2007.
Homeland Security: Applying Risk Management Principles to Guide  Federal Investments. GAO-07-386T. Washington, D.C.: February 7, 2007.
Military Personnel: DOD Needs to Provide a Better Link between Its  Defense Strategy and Military Personnel Requirements. GAO-07-397T.  Washington, D.C.: January 30, 2007.
Force Structure: Joint Seabasing Would Benefit from a Comprehensive  Management Approach and Rigorous Experimentation before Service  Spend Billions on New Capabilities. GAO-07-211. Washington, D.C.:  January 26, 2007.
Force Structure: Army Needs to Provide DOD and Congress More  Visibility Regarding Modular Force Capabilities and Implementation  Plans. GAO-06-745.Washington, D.C.: September 6, 2006.
Force Structure: DOD Needs to Integrate Data into Its Force  Identification Process and Examine Options to Meet Requirements for  High-Demand Support Forces. GAO-06-962. Washington, D.C.:   September 5, 2006.
DOD Acquisition Outcomes: A Case for Change. GAO-06-257T.  Washington, D.C.: November 15, 2005.
Defense Management: Additional Actions Needed to Enhance DOD’s  Risk-Based Approach for Making Resource Decisions. GAO-06-13.  Washington, D.C.: November 15, 2005.
DOD’s High-Risk Areas: Successful Business Transformation Requires  Sound Strategic Planning and Sustained Leadership. GAO-05-520T.  Washington, D.C.: April 13, 2005.
Military Personnel: DOD Needs to Conduct a Data-Driven Analysis of  Active Military Personnel Levels Required to Implement the Defense  Strategy. GAO-05-200. Washington, D.C.: February 1, 2005.  21st Century Challenges: Reexamining the Base of the Federal  Government. GAO-05-325SP. Washington, D.C.: February 1, 2005.
High-Risk Series: An Update. GAO-05-207. Washington, D.C.:  January 1, 2005.
Results-Oriented Cultures: Implementation Steps to Assist Mergers and  Organizational Transformations. GAO-03-669. Washington, D.C.:   July 2, 2003.
Quadrennial Defense Review: Future Reviews Can Benefit from Better  Analysis and Changes in Timing and Scope. GAO-03-13. Washington,  D.C.: November 4, 2002.
A Model of Strategic Human Capital Management. GAO-02-373SP.  Washington, D.C.: March 15, 2002.
Quadrennial Defense Review: Opportunities to Improve the Next  Review.GAO/NSIAD-98-155. Washington, D.C.: June 25, 1998.
Quadrennial Defense Review: Some Personnel Cuts and Associated  Savings May Not Be Achieved. GAO/NSIAD-98-100. Washington, D.C.:  April 30, 1998.
Combating Terrorism: Threat and Risk Assessments Can Help Prioritize  and Target Program Investments. GAO/NSIAD-98-74. Washington, D.C.:  April 9, 1998.
Bottom-Up Review: Analysis of DOD War Game to Test Key  Assumptions. GAO/NSIAD-96-170. Washington, D.C.: June 21, 1996.
Bottom-Up Review: Analysis of Key DOD Assumptions.   GAO/NSIAD-95-56. Washington, D.C.: Jan. 31, 1995.