Weapon System Acquisitions: Opportunities Exist to Improve the Department of Defense's Portfolio Management
Published: 2015-08-27T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: DOD's weapon system acquisition programs have a total estimated acquisition cost of over $1.4 trillion. Portfolio management is an approach used by organizations to evaluate, select, prioritize, and allocate resources to projects that best accomplish strategic or organizational goals. In March 2007, GAO recommended that DOD implement a department-wide portfolio management approach for weapon system investments.
Senate Report 113-44 accompanying the National Defense Authorization Act for Fiscal Year 2014 included a provision that GAO review DOD processes for identifying duplicative and inefficient acquisitions. This report assesses the extent to which (1) DOD uses portfolio management to optimize weapon system investments; (2) DOD conducts integrated portfolio reviews; and (3) the military services conduct portfolio reviews. GAO compared DOD and military service policies and portfolio reviews with best practices and standards for portfolio management.
section_title: What GAO Found
paragraphs: The Department of Defense (DOD) is not effectively using portfolio management to optimize its weapon system investments, as evidenced by affordability challenges in areas such as shipbuilding and potential duplication among some of its programs. Best practices recommend assessing investments collectively from an enterprise-wide perspective and integrating requirements, acquisition, and budget information, but several factors inhibit DOD's ability to do so.
Fragmented governance: DOD has numerous processes, organizations, and decision makers to oversee weapon system investments that operate in stove-pipes, not as an integrated whole. The requirements and acquisition processes also focus on individual programs rather than assessing investments collectively, as best practices recommend.
Lack of sustained leadership and policy: DOD stopped implementing its portfolio management efforts and policy, in part due to changes in leadership. DOD's policy is also dated, does not fully reflect best practices, and does not identify an office with sufficient authority to implement it.
Perceived lack of decision-making authority: Enterprise-level involvement is key to optimizing investments across DOD because the military services prioritize needs and optimize investments within their services rather than across the military. Title 10, which gives the services responsibility over equipping the force, does not preclude enterprise-level influence over service investment decisions, but some DOD officials said it limits their influence.
DOD's enterprise-level requirements, acquisition, and budgeting communities, meaning those at the at the Office of the Secretary of Defense, Joint Chiefs of Staff, and Joint Staff level, are not consistently conducting portfolio reviews or collaborating to integrate key information. As a result, DOD may be missing opportunities to better leverage its resources and identify investment priorities that best reflect DOD-wide needs. Best practices and portfolio management standards state that organizations should conduct regular reviews to adjust to strategic changes, among other reasons. The Joint Staff, which is responsible for validating warfighting needs, has taken the most concrete actions to conduct portfolio reviews, but even these efforts did not integrate key requirements, acquisition, and budget information. Requirements and acquisition officials said they lacked the resources, readily accessible data, and analytical tools to effectively conduct reviews. For example, the Joint Staff lacks a database that pulls together current information to help it manage its portfolios and has to rely on repeated data calls, which are inefficient and time consuming.
The military services have conducted reviews more consistently than the enterprise level and their experiences at the service level may offer lessons for DOD. They have used the reviews to reduce redundancies, plan for budget uncertainty, and realign resources. Nevertheless, their reviews lack some of the key information needed to provide an integrated assessment of needs, investments, and resources and are limited to the services' own programs. A more integrated approach to portfolio reviews at both the enterprise and military-service levels would better position DOD to conduct sound investment planning.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD update its portfolio management policy, designate a senior official responsible for its implementation; conduct annual portfolio reviews that integrate key information from the requirements, acquisition, and budget processes; and invest in analytical tools to support its portfolio management efforts. DOD partially concurred with the recommendations. However, as discussed in the report, DOD's planned actions will not fully address the issues GAO identified.

section_title: Letter
paragraphs: The Department of Defense (DOD) has 78 major weapon system  programs under way with a total estimated acquisition cost of over $1.4  trillion. These include some of the most advanced weapons in the world.  However, DOD has not consistently managed its weapon system  investments as a portfolio to ensure they are strategy-driven, affordable,  and balance near- and long-term needs. Rather, DOD and the military  services plan to acquire more weapons than they can afford given  anticipated levels of funding. Furthermore, when multiple services have  weapon system needs in common, they sometimes develop separate,  rather than common solutions, that result in inefficient, and in some  cases, potentially duplicative investments.
We and others have found that leading commercial companies use  portfolio management—a disciplined process that helps optimize  investments by ensuring organizations have the right mix of new products  that meet customer needs within available resources. Portfolio  management focuses on products collectively at an enterprise level and  involves evaluating, selecting, prioritizing, and allocating limited resources  to projects that best accomplish strategic or organizational goals. It is also  a vehicle to make a wide variety of decisions, including capability and  funding trade-offs, to achieve the optimal capability mix for a given  investment. In 2007, we recommended that DOD implement a  department-wide portfolio management approach to making weapon  system investments. DOD concurred with this recommendation.
Senate Report 113-44 accompanying the National Defense Authorization  Act for Fiscal Year 2014 included a provision that GAO review DOD’s  processes and procedures for identifying duplicative and inefficient major  development and procurement programs. We examined ways to reduce  potential duplication and increase the efficiency and effectiveness of  weapon system investments through portfolio management. This report  assesses the extent to which (1) DOD uses portfolio management to  optimize its weapon system investments, (2) DOD conducts integrated  portfolio reviews at the enterprise level, and (3) the military services  conduct integrated portfolio reviews and what, if any, lessons their  experiences offer DOD.
To assess the extent to which DOD uses portfolio management to  optimize its weapon system investments, we compared DOD’s weapon  system investment policies and decision-making processes, which  include the requirements, acquisition, and budget processes—known  respectively as the Joint Capabilities Integration and Development  System (JCIDS), Defense Acquisition System, and Planning,  Programming, Budgeting, and Execution (PPBE) processes—to portfolio  management best practices. We identified these best practices in prior  GAO reports and conducted a literature review to ensure they were still  current. To determine the extent to which DOD conducts integrated  portfolio reviews at the enterprise level—meaning at the Office of the  Secretary of Defense, Joint Chiefs of Staff, and Joint Staff level—we  analyzed enterprise-level portfolio reviews and other portfolio-level  analyses. We compared portfolio review content with key portfolio  elements that we identified based on the Project Management Institute’s  The Standard for Portfolio Management—Third Edition and best  practices, which we adapted to the DOD environment. We also  interviewed Joint Staff; Office of the Under Secretary of Defense for  Acquisition, Technology and Logistics (AT&L); Cost Assessment and  Program Evaluation (CAPE); and Office of the Under Secretary of  Defense (Comptroller) officials. To assess the extent to which the military  services conduct integrated portfolio reviews and identify potential  lessons for DOD, we reviewed Army, Air Force, and Navy requirements,  acquisition, and budget policies and interviewed officials from all of those  communities. We also compared Army, Air Force, and Navy portfolio  reviews with key portfolio review elements.
We conducted this performance audit from November 2013 to August  2015 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Weapon system investment decision-making in DOD is highly complex. It  involves numerous entities, levels, and policies at the military service and  enterprise level. The four military services—the Army, Air Force, Navy,  and Marine Corps—by law have authority to organize, train, and equip  their services. They make initial decisions regarding what to buy, how to  buy it, and how much it will cost. Those decisions are made by their  requirements, acquisition, and budget communities and go through multi- layered review processes within the services. Each service has its own  requirements, acquisition, and budget policies, which are based in part on  enterprise-level policies.
For major investments, the military services’ decisions regarding what to  buy, how to buy it, and how much to buy may be reviewed and approved  or disapproved at the enterprise level. At the enterprise level, JCIDS  assesses, validates, and prioritizes warfighter needs; the Defense  Acquisition System oversees product development and procurement;  and PPBE allocates resources taking into account capability needs, risk,  and affordability. Each of those processes is governed by its own set of  policies (See appendix II for a list of relevant policies).
subsections: section_title: Weapon System Investment Challenges
paragraphs: DOD faces several weapon system investment planning challenges. First,  DOD plans more weapons acquisition programs than it can afford. In the  Congressional Budget Office’s (CBO) most recent analysis of DOD’s 5- year Future Years Defense Program (FYDP), CBO projected acquisition  costs for weapons and other major equipment to increase by 21 percent  between fiscal years 2015-2019, and to peak in the early 2020s at about  43 percent higher than in 2015. CBO explained that the steep increase in  acquisition costs it projects beyond the FYDP timeframe shows that the  classic bow wave has returned—meaning that DOD is pushing out  funding needs assuming it will receive higher acquisition funding levels in  the future to complete its programs. Similarly, CBO estimated that the  total costs of carrying out the Navy’s fiscal year 2015 shipbuilding plan  would require one-third more funding on an annual basis than the Navy  has received in recent decades. CBO reported that, as a result, the  Navy will likely have to purchase substantially fewer ships than under its  current plans or make other trade-offs. In April 2015, we also raised  affordability issues with the Joint Strike Fighter—DOD’s largest weapon  acquisition program. We estimated the program will require an average of  $12.4 billion per year, which represents around one-quarter of DOD’s  annual funding for major weapons programs, over the next 5 years. We  reported that it is unlikely the program will be able to sustain such a high  level of annual funding and if required funding levels are not reached, the  program’s procurement plan may not be affordable.
In addition, the military services have a history of tailoring their planned  investments to their own needs rather than trying to find common ground,  which has led to inefficiencies and, in some cases, duplication. More  specifically, the military services sometimes generate unique system  requirements that justify multiple acquisition programs to meet similar  needs. Our work has concluded that redundancy or overlapping  capabilities in some areas may be desirable or necessary, but acquisition  efforts to develop similar capabilities can also result in the same capability  gap being filled twice or more, lead to inefficient use of resources, and  contribute to other warfighting needs going unfilled. In 2012, we found  potential duplication among electronic warfare systems, some of which  DOD has canceled. In 2013, we found additional areas of overlap in the  roles that electronic warfare systems were intended to perform and that  new areas of overlap and potential duplication could emerge as  investment plans evolved. More recently, we concluded that when  developing a new ground radar system, the Air Force focused on what  made its requirements unique from a similar Marine Corps’ system,  instead of looking for ways to leverage the Marine Corps’ program. We  found these programs to be potentially duplicative.
section_title: Portfolio Management
paragraphs: Leading commercial companies use a portfolio management approach to  optimize their investments. They begin with an enterprise-level  identification and definition of market opportunities—in other words,  potential customer wants or needs—and then prioritize those  opportunities while taking into account resource constraints. Once these  companies have prioritized opportunities, they draft initial business cases  for alternative product ideas that could be developed to exploit each of  the highest priority opportunities. Each alternative product proposal enters  a gated review process. At each review, product proposals are assessed  against others in the portfolio weighing corporate resources, established  criteria, competing products, and the goals and objectives of the  organization as a whole. As alternatives pass through each review, the  number is expected to decrease, until only those alternatives with the  greatest potential to succeed make it into the product portfolio. In effect,  new programs are born from portfolio analysis and management. Over  time, as potential new products are identified or if products under  development experience problems, companies review and rebalance their  portfolio of proposed and existing product investments based on those  products that add the most value and are the best strategic fit.
More generally, rather than defaulting to optimizing individual programs,  portfolio management focuses on selecting the optimum mix of programs  and modifying that mix as needed over time. Take a hypothetical example  in which DOD starts with 10 programs and $50 billion to invest. Without  portfolio management, program managers will seek to get the most that  they can out of each of the 10 programs, without assessing their  aggregate contributions to defense. Using portfolio management, DOD  executives would look at different combinations of and approaches to the  10 programs to see what, collectively, would provide the best capabilities  for $50 billion. This would enable executives to decide, for example,  whether it is better to concentrate more investment in 7 programs rather  than fund all 10 as best as possible. In another example, if a program  began to have cost or performance problems, portfolio management  would consider whether the other programs in the portfolio could address  the requirements of the problematic program rather than just putting more  money into it.
Portfolio management literature has identified characteristics and effects  of investment planning processes that are not based on portfolio  management. These characteristics and effects, which are shown in  figure 1, are consistent with some of our observations on DOD’s weapon  system investment challenges.
section_title: Portfolio Management Best Practices
paragraphs: In 2007, we identified several best practices for portfolio management,  which form the basis for our assessments in this report. Organizations  that follow these practices assess product investments collectively from an enterprise level,  rather than as independent and unrelated initiatives; continually make go/no-go decisions through a gated review process  to rebalance portfolios based on investments that add the most value; use an integrated approach to prioritize needs and allocate resources  in accordance with strategic goals;  rank and select investments using a disciplined process to assess the  costs, benefits, and risks of alternative products; empower leadership to make investment decisions and hold  leadership accountable for investment outcomes; and provide sustained leadership for portfolio management.
section_title: Portfolio Reviews
paragraphs: Portfolio management best practices and the Project Management  Institute’s portfolio management standards also state that organizations  should conduct regular reviews to adjust to strategic changes or changes  in the mix of products within a portfolio, among other reasons. These  reviews are tools that organizations use to plan and manage their  investments. From a DOD perspective, portfolio reviews can help  increase return on taxpayers’ investments in weapon systems in a  number of ways, such as: helping to ensure investments align with  national security and military strategies; prioritizing the most important  investments; selecting the optimum mix of investments; identifying and  eliminating unwarranted duplication; monitoring programs’ health to  determine whether changes to the portfolio are warranted; and  determining whether investments are affordable. Although there is not  one model for portfolio reviews that fits all situations, we identified key  portfolio elements, which—if included in a review—would integrate  relevant information from the requirements, acquisition, and budget  communities to achieve some of the benefits described above (see table  1).
section_title: DOD Is Not Using an Integrated Portfolio Management Approach to Optimize Its Weapon System Investments at the Enterprise Level
paragraphs: DOD is not using an integrated portfolio management approach to  optimize its weapon system investments at the enterprise level, as called  for in best practices. DOD lacks the governance structure, sustained  leadership, and policy to do so. In particular, DOD’s governance structure  includes decision-making processes and responsibilities that are divided  among its stove-piped requirements, acquisition, and budget communities  and focus largely on optimizing individual investments. This fragmentation  does not allow for an integrated portfolio management approach to  making investment decisions across the department. In addition, the  military services operate in stove-pipes of their own and tend to optimize  investments for their service, rather than for the military as a whole. A  perceived lack of enterprise-level influence over the services hinders  DOD’s ability to address this behavior and optimize investments across  the department.
subsections: section_title: DOD’s Stove-Piped Governance Structure Is an Impediment to Portfolio Management
paragraphs: DOD’s stove-piped governance structure is an impediment to using an  integrated portfolio management approach to optimize weapon system  investments. DOD has numerous enterprise-level processes,  organizations, and decision makers to oversee its weapon system  investments, which generally operate as stove-pipes, not as an integrated  whole. In addition, the acquisition process and to some extent, the  requirements process, are geared toward making decisions about  individual programs rather than assessing investments collectively from  an enterprise level, as best practices suggest. DOD has taken some  steps to integrate its requirements, acquisition, and budget processes.  However, these steps primarily include sending representatives to  participate in the other communities’ decision-making forums. As a result,  most investment decisions get made on a piecemeal basis within the  requirements, acquisition, and budget processes. DOD officials stated  that decision-making forums, such as the Deputy Secretary of Defense’s  Management Action Group, serve an integrating function and make  departmental management decisions on a more cross-cutting basis, but  these forums primarily address issues on an ad hoc basis. Table 2  describes the key enterprise-level governance structures that are  responsible for DOD weapon system investment decision-making.
Other aspects of DOD’s governance structure for making weapon system  investment decisions are also fragmented and inconsistent with an  integrated portfolio management approach.
Decision-making responsibilities: DOD’s enterprise-level decision- making responsibilities are fragmented, which makes it difficult to  identify who is responsible for investment decisions and to hold them  accountable. According to portfolio management best practices,  leadership should be clearly defined and held accountable for  outcomes. In DOD, weapon system investment decisions are  dispersed among multiple decision makers. This fragmentation could  lead to conflicting decisions or decisions that align with the goals of  one community, but not those of another. Several DOD officials said  weapon system investment decision-making is dispersed by design  and that natural tension between and different perspectives among  decision makers are beneficial to the process. Ultimately, though,  these diffuse decision-making responsibilities make it difficult to  determine who is empowered to make enterprise-level weapon  system investment decisions and who can be considered to be  portfolio managers. In our enterprise- and service-level interviews,  some officials stated that no one at the enterprise level is empowered  to make investment decisions, and others responded that only a  strong Secretary of Defense or Deputy Secretary Defense can take  actions required for effective portfolio management, such as making  trades across services or canceling programs.
Investment planning assumptions: DOD is not using an integrated,  common set of planning assumptions, such as warfighting scenarios,  to inform investment decisions across the department. Best practices  state that organizations should rank and select investments using a  disciplined process to assess the costs, benefits, and risks of  alternative products. Former senior and current DOD officials  explained that it is difficult to analyze investments across the services’  portfolios without a common analytical baseline because  organizations use scenarios that favor their specific interests. In a  November 2014 memo, the Deputy Secretary of Defense outlined  steps to reinvigorate DOD’s support for strategic analysis efforts and  move toward a common starting point for departmental analysis.  These steps included Deputy Secretary and Vice Chairman of the  Joint Chiefs of Staff level review of scenario selection, key  assumptions, and concepts of operations, among others areas.
Portfolio constructs: To the extent that the requirements,  acquisition, and budget communities group investments by portfolio,  they do so using different portfolio constructs. According to portfolio  management best practices, organizations should use an integrated  approach to prioritize needs and allocate resources in accordance  with strategic goals. Using different portfolio constructs is a barrier to  taking an integrated approach. At the enterprise level, the  requirements community uses nine joint capability areas for  examining warfighting needs, acquisition portfolios vary by military  service, and budget data is organized into eleven major force  programs for the FYDP. (Table 3 lists these portfolios.)
DOD attempted to standardize portfolios in the 2006 to 2008 time period.  However, a former senior official who was involved in that effort said the  mapping was “impossible” and that there was organizational resistance  because the portfolios did not align with many decision makers’ areas of  responsibility. Many of the enterprise- and service-level officials we  interviewed said using a wide variety of constructs is necessary and  sometimes beneficial given the different roles and perspectives of the  organizations involved. However, when they want to analyze their  portfolios from another perspective—for example, examining funding  associated with joint capability areas—they have to go through extensive  mapping exercises, as notionally illustrated in figure 2.
section_title: DOD Lacks Sustained Leadership to Implement Portfolio Management
paragraphs: DOD lacks sustained leadership for portfolio management, and as a  result, it has been difficult for DOD to maintain momentum for portfolio  management-related initiatives. Our prior work found that effective  portfolio management requires a governance structure with sustained  leadership. However, changing leadership priorities pose challenges to  portfolio management in DOD. In the mid-to-late 2000s, DOD made a  concerted effort to implement capability portfolio management. DOD  initiated portfolio management pilots in select capability areas to improve  strategic decisions on resource allocation across programs in 2006 and  issued a directive on capability portfolio management in 2008. Soon  afterwards, leadership priorities shifted away from portfolio management.  DOD eliminated the pilots, did not document the results of the pilots, and  stopped implementing the directive. This was attributed to a variety of  factors, including the lack of a senior level champion for the effort as  political leadership changed, which can happen every few years in DOD.  Officials from AT&L and the Joint Staff said that DOD no longer has a  champion for portfolio management.
section_title: DOD Lacks a Portfolio Management Policy That Reflects Best Practices
paragraphs: DOD does not have a policy to guide portfolio management across the  department that fully reflects key best practices. The policy is also not  current and DOD is not implementing it, but it has not been rescinded.  DOD Directive 7045.20, Capability Portfolio Management, states that  DOD shall use capability portfolio management to advise the Deputy  Secretary of Defense and the heads of the DOD components on how to  optimize investments across DOD and minimize risk in meeting capability  needs. Portfolios are to be managed by civilian and military capability  portfolio managers who are to develop capability portfolio strategic plans,  provide cross-component perspectives, recommend capability priorities to  senior decision makers, and take other actions, such as developing  portfolio governance forums. In addition, the policy attempts to  standardize DOD’s portfolio constructs by aligning portfolios with joint  capability areas.
However, under the policy, portfolio managers do not have sufficient  authority to effectively influence weapon system investment decisions.  They do not have any independent decision-making authority, nor are  they to “infringe on any existing statutory or regulatory authorities” such  as the military services’ authority under title 10 to organize, train, and  equip their forces. Instead, their influence is to be through the Deputy  Secretary’s Management Action Group, which is to use information from  the portfolio managers to inform the Deputy Secretary of Defense. More  generally, the policy is intended to operate within DOD’s existing  fragmented organizational and process framework—which includes  multiple barriers to integrated portfolio management. Table 4 includes our  assessment of how well DOD’s current portfolio management policy  addresses key best practices.
Officials we met with throughout DOD explained that they are not  implementing DOD’s capability portfolio management policy. A few  officials stated that DOD is implementing aspects of portfolio  management, but in a less integrated fashion. Others appeared to be  unaware of the policy. Several enterprise-level officials who were familiar  with the policy stated that DOD is not implementing the directive because  elements, such as using a single set of portfolios, are impractical and  leadership priorities changed. In the policy, DOD also did not identify an  office to be responsible for its overall implementation. Federal internal  control standards require that organizations establish objectives and  clearly define key areas of authority and responsibility. The Office of the  Under Secretary of Defense for Policy (OUSD (Policy)) issued DOD’s  capability portfolio management directive, however, that office does not  have direct line management authority over the organizations responsible  for implementing it. DOD officials explained that the only officials with  authority over those organizations are the Deputy Secretary of Defense  and Secretary of Defense. An official from OUSD (Policy) stated that they  are aware that the policy is not being followed. OUSD (Policy) is  reviewing the policy and may rescind or revise it in the future.
section_title: Perceived Lack of Enterprise-Level Decision- Making Authority over the Services Is an Impediment to Portfolio Management
paragraphs: Enterprise-level involvement is important to optimize investments  collectively across the military because the military services tend to focus  on optimizing their own investments. For DOD to effectively manage  investments at the enterprise level, it would need to exercise its authority  and influence service-level investment decisions and make cross-service  trades. Three senior enterprise-level officials explained that the military  services focus on maintaining their portion of the investment budget and  tend to optimize investments for their own services, rather than for the  military as a whole.
A perceived lack of authority over the military services is an impediment  to managing investments collectively at the enterprise level. According to  best practices, portfolio managers should be empowered to make  investment decisions. Under title 10 the military services have the  responsibility for equipping their forces—in other words, buying weapons.  Enterprise-level officials we interviewed stated that they think this  provides primary control over setting weapon system investment priorities  to the services. While title 10 does not preclude enterprise-level influence  over investment decisions, this influence currently comes in the form of  oversight reviews after the services have developed investment plans,  weapon system requirements, and acquisition plans. Joint Staff officials  said they have portfolio management responsibilities, such as prioritizing  requirements, but their influence is limited because the services control  the money. AT&L officials stated that they have an oversight role, but  cannot change the military services’ acquisition priorities. Similarly, DOD  Comptroller officials explained that enterprise-level involvement to  optimize investments across the department is limited by title 10.
Experiences with ground radar systems show some of the challenges the  enterprise level has experienced when trying to optimize investments  across services. In 2008, the Undersecretary of Defense for AT&L  established a Radar Joint Analysis Team to develop a DOD-wide  roadmap for radar technology, development, and procurement and  consider overlap and commonality among programs. In two cases, the  team encouraged the military services to collaborate on developing a  single system to address their needs. However, the services had different  budget priorities and acquisition timelines and therefore decided to  pursue separate acquisition programs. More recently, some Office of the  Secretary of Defense officials noted there were similarities between the  Air Force’s Three-Dimensional Expeditionary Long-Range Radar and the  Marine Corps’ Ground/Air Task Oriented Radar requirements, and  questioned the need for two separate programs. An AT&L official said  that the discussions that resulted from the questions were beneficial, but  the Air Force and the Marine Corps are going forward with separate  programs, in part, because they own their own resources.
section_title: DOD Does Not Conduct Integrated Enterprise-Level Portfolio Reviews
paragraphs: DOD does not conduct enterprise-level portfolio reviews that integrate the  requirements, acquisition, and budget communities. Instead, the Joint  Staff, AT&L, and CAPE each have their own limited portfolio review  efforts, some of which are still being developed. Conducting portfolio  reviews on a regular basis can help optimize portfolios. For example,  reviews can serve as a tool to identify potentially duplicative capabilities  or systems and evaluate trade-offs to address those inefficiencies. DOD  officials said that limited resources and a lack of analytical tools have  hampered their ability to conduct integrated portfolio reviews and  portfolio-based analyses.
subsections: section_title: DOD Does Not Conduct Integrated Portfolio Reviews
paragraphs: DOD does not conduct integrated enterprise-level portfolio reviews.  Instead, the Joint Staff, AT&L, and CAPE have a fragmented approach  that involves separate, but limited, portfolio reviews and analyses using  different portfolio constructs. None of these reviews and analyses have  been consistently conducted. Portfolio management best practices state  that organizations should conduct regular reviews to adjust to factors,  such as strategic changes. Consistently conducting portfolio reviews  provides opportunities to identify and address inefficiencies within  portfolios, and to adjust and realign priorities to meet current needs.
In addition to not being conducted in an integrated manner, the reviews  we assessed did not consistently incorporate key portfolio review  elements, which would result in a more integrated portfolio review. Best  practices state that taking an integrated approach to conducting reviews  helps avoid pursuing more investments than are affordable, balance near-  and long-term needs, and maximize return on investment. In DOD’s case,  an integrated review for weapon system investments would involve the  requirements, acquisition, and budget communities or information from  those communities (see figure 3).
section_title: Joint Staff Reviews to Date Have Varied in Content and Quality
paragraphs: We assessed five out of the six Joint Staff portfolio reviews conducted in  2014 and found that these reviews varied widely in content and quality. A  Joint Staff official who coordinated the reviews said they varied because it  was the Joint Staff’s first year conducting them. The Joint Staff initiated  the portfolio reviews to inform its position on DOD’s fiscal year 2016  budget submission. The goal of these reviews was to identify  redundancies, possible trade-offs, investment opportunities, outdated  requirements, and impactful issues for each portfolio. A Joint Staff official  who led one of the reviews said that he found great value in conducting  the review, which allowed his office to evaluate how the cost of a  capability impacts whether other capability needs within the portfolio  could be met. In comparing the reviews against the stated goals, all five  included impactful issues, such as cost drivers within a particular portfolio.  We found that four reviews included information on investment  opportunities and redundancies; two addressed outdated requirements;  and one identified possible trade-offs. Joint Staff officials said they plan to  continue conducting portfolio reviews to inform the budget and to develop  guidance for the next round of reviews to try to ensure they are more  consistent.
The Joint Staff portfolio reviews did not integrate information from the  requirements, acquisition, and budget communities. For example, none of  the reviews included information on program cost, schedule, or  performance, and only one included a timeline of when systems in the  portfolio would be fielded. In addition, none of the reviews considered  funding scenarios and only two reviews included information on portfolio  affordability, such as details on funding for the portfolio over the next 5  years. By not incorporating this information into the reviews, the Joint  Staff may be ill-equipped to shape decisions about balancing capability  needs and portfolio affordability and offer recommendations on portfolio  modifications to address potential budget changes.
The Joint Staff recently revised its JCIDS policy and guidance to provide  more emphasis on portfolio management. The revised JCIDS manual  states that Functional Capabilities Board (FCB) chairs need to monitor  activities that could affect their portfolios throughout the year and  reassess capability requirements portfolios as needed. For example,  they may assess their portfolios at the request of Joint Staff senior  leadership, to inform annual reviews such as the budget review, and to  reassess the content or priorities of the portfolio due to changes in  strategic guidance. In addition, the FCBs are to track acquisition  programs’ progress toward satisfying capability requirements. Tracking  progress will help ensure that the FCBs can assess changes to programs  or timelines for their effect on their portfolio’s capability needs.
section_title: AT&L’s Portfolio Review Initiative Has Resulted in Few Reviews
paragraphs: In DOD’s first iteration of the Better Buying Power initiatives, which seek  to address affordability challenges by increasing the efficiency of defense  spending, AT&L directed that portfolio reviews be conducted to identify  potential redundancies; however few have been conducted (see table  5). AT&L’s 2010 guidance called for portfolio reviews at the enterprise  level and for the military services to conduct similar reviews for lower  dollar acquisition programs, known as acquisition category (ACAT) II and  III programs.  (Appendix III assesses additional portfolio-based analyses  called for under Better Buying Power.) The Better Buying Power initiatives  have evolved over the past few years. The most recent version of the  guidance still contains the goal of eliminating redundancy within  warfighter portfolios, but it is no longer an area of emphasis.
In 2013, AT&L began conducting portfolio reviews, called mission area  portfolio assessments, but they have had a limited focus and do not  include some key information suggested by portfolio management  standards and best practices. These reviews focused largely on  identifying science and technology investments needed to keep up with  future threats. An AT&L official said that the first set of reviews included  some requirements, acquisition, and budget information such as  identification of capability gaps, science and technology needs, and  affordability analyses. However, the reviews did not include other key  requirements and budget information, such as prioritized lists of capability  gaps, outdated requirements, and funding scenarios that would allow  them to look at the portfolio in an integrated manner as suggested by  portfolio management standards and best practices. An AT&L official  responsible for the reviews said that AT&L is in the beginning stages of  developing these reviews and it is still determining how to break down  larger portfolios to review, how they will complement other portfolio  reviews, and what roles other Office of the Secretary of Defense  stakeholders and the military services might play in the review process.  AT&L initially planned to conduct these reviews on an annual basis, but it  has not been able to do so due to time and resource constraints and a  decision has yet to be made on whether the reviews will continue.
section_title: CAPE Does Not Conduct Regular Portfolio Reviews
paragraphs: At the direction of Deputy Secretary of Defense, CAPE coordinates  strategic portfolio reviews for select portfolios or issue areas to inform  budget decisions; however, these reviews have evolved over time and the  topics covered vary from year-to-year. The original purpose of the  strategic portfolio reviews was to: (1) look across service weapon system  portfolios to identify redundancies and capability gaps, and (2) provide  DOD components with earlier decisions on capability needs, including  which component would include funding in their budget submission for the  highest priority capabilities. CAPE has generally not used the reviews in  the way that they were originally intended. The reviews are not  necessarily organized by portfolio unlike the Joint Staff and AT&L  reviews—they may sometimes address just one system. In addition, they  address more specific, targeted questions that vary from year-to-year  depending on what issues the Deputy Secretary of Defense identifies as  important. CAPE officials also said that the reviews are not designed to  specifically address duplication of weapon system capabilities. However,  if CAPE notices potential duplication in the course of its reviews, it may  propose changes to programs during the budget review process.  According to the Assistant Secretary of Defense for Acquisition, the topics  for last year’s reviews included space, strategic offense, protection forces,  innovation, infrastructure, and readiness. A CAPE official stated that  some reviews are more consistent with the stated purpose of strategic  portfolio reviews than others. For example, a recent review of the nuclear  enterprise helped prioritize capability needs and provided the context for  evaluating investment plans during the program budget review process.
section_title: Lack of Resources and Analytical Tools Limits DOD’s Ability to Effectively Conduct Portfolio Reviews
paragraphs: The Joint Staff, AT&L, and CAPE reported that resource limitations have  affected their ability to regularly conduct portfolio reviews. Despite these  resource constraints, they have not coordinated their portfolio reviews to  integrate and streamline them, although they have had conversations  about their reviews and the Joint Staff and AT&L have provided input to  strategic portfolio reviews. We have previously found that one way to  better manage fragmented activities is to improve collaboration and  coordination. This includes engaging in key practices such as defining  and articulating common outcomes, agreeing on roles and  responsibilities, and identifying and addressing needs by leveraging  resources. Without better coordination and collaboration, DOD will  struggle both from an information and resource perspective to implement  integrated portfolio reviews.
Both AT&L and the Joint Staff said that a lack of readily accessible data  and analytical tools also hampered their ability to effectively conduct  portfolio reviews. AT&L officials said that more accessible data and better  tools would be helpful for identifying redundancies and for the mission  area portfolio assessments. Having readily accessible data and tools is  essential for these analyses. For example, in the past, lack of visibility into  certain programs, such as special access programs, urgent operational  needs, and lower ACAT level programs contributed to potential  unnecessary duplication. Joint Staff officials said that past efforts relied  on repeated data calls, which were a drain on resources and time  consuming. In addition, one of the officials in charge of a Joint Staff  portfolio review said that the Joint Staff does not have the authority to  compel other DOD components to provide the necessary information and  data to conduct the reviews. Finally, Joint Staff officials report that they do  not have the analytical tools and dynamic databases to effectively  conduct portfolio reviews, assess potential redundancy, and prioritize  capabilities.
The Joint Staff is currently developing a database to provide it with a  better analytical tool to support portfolio management. The database is to  track capability gaps to enable the Joint Staff to more easily identify  potential redundancies, prioritize capability gaps and close out capability  gaps that have been addressed or that no longer need to be addressed  because the threats have evolved. Joint Staff officials said that assessing  duplication is challenging because as investment plans and capability  needs evolve, new areas of overlap and potential duplication can emerge.  They expect the database will help them identify which programs are  addressing particular capability gaps, which gaps remain open, which  gaps are closed, and which gaps are outdated. Knowing if there are  multiple systems with requirements addressing the same capability gaps  could help identify potential duplication.
The Joint Staff has not yet developed a formal plan to address some of  the database’s known limitations. First, the database has been difficult to  populate and is potentially missing information because it is dependent on  another system that has limited functionality. Second, the Joint Staff has  had limited coordination outside of the immediate office developing the  database to determine its utility to others, although the Joint Staff and  AT&L are funding cross-cutting studies to help refine the tool. Third, the  Joint Staff has plans to improve database usability and functionality, but  has not developed a formal implementation plan to ensure that it  continues to improve and meet the needs of its current and potential  users. Developing such plans is a key project management practice.  Without establishing this planning foundation, the Joint Staff will not be in  a sound position to effectively monitor and evaluate the implementation of  its efforts to provide quality information for reviews.
section_title: Military Service Portfolio Reviews Offer Lessons for DOD, but Could Be Improved
paragraphs: Most military services have investment planning processes that review  weapon system portfolios and their experiences may offer potential  lessons for DOD. The Army’s capability portfolio reviews have been held  up as a model by DOD and, in the past, were used by the Army to re- examine requirements and realign resources. The Air Force’s investment  plans take into account budget uncertainty by examining multiple funding  scenarios. The Navy conducts assessments that include portfolio-based  analysis to provide an analytical basis for evaluating budget proposals  against capability needs. However, these reviews also lacked some of  the key acquisition, requirements, and funding information that would  better position the services to conduct sound investment planning.
subsections: section_title: Army Re-examined Requirements in its Initial Reviews to Inform Trade- offs, but Stopped Conducting Regular Portfolio Reviews
paragraphs: consistently conduct portfolio reviews;  regularly include a majority of key portfolio  review elements in its reviews; and  incorporate review guidance into policy.
The Army has conducted portfolio reviews, which could offer lessons for  DOD, especially in the area of re-examining capability needs from a  portfolio perspective. The Army’s portfolios are generally aligned with the  U.S. Army Training and Doctrine Command’s Centers of Excellence, such  as aviation, although the portfolio groupings have changed over time and  increased in number from nine to eleven.  The Secretary of the Army  initiated capability portfolio reviews in December 2009. The purpose of  the reviews is to, among other things, ensure that funds are programmed,  budgeted, and executed against valid requirements and informed  alternatives and to revalidate portfolios based on strategy, Combatant  Commanders needs, and affordability. The reviews are led by the  requirements community and involve the acquisition and budget  communities. The process involves multiple working group meetings and  then a series of briefings involving increasingly higher level officials.  These culminate in a session with Army senior leadership to identify  portfolio priorities and other recommendations. A former senior official  who spearheaded the reviews stated that the Army originally planned to  review each portfolio individually on an annual basis.
Some Army capability portfolio reviews resulted in changes to weapon  system investments, including program cancellations, as a result of re- examining requirements and capabilities from a portfolio perspective. For  example, a 2010 Precision Fires review, which examined the Army’s  balance of high-end precision munitions and lower-end near-precision  munitions, led to several changes. The review determined that a  proposed precision munitions system—the Non-Line-of-Sight Launch  System—would not provide cost-effective capabilities. The Army  canceled the program and relied on another system to provide similar  capabilities at lower cost. Based on that review, the Army also decided to  reduce the number of certain munitions rounds due to cost and other  factors. AT&L held up this review as a model in its first iteration of the  Better Buying Power initiatives, which stated that enterprise-level and  component-level portfolio reviews were to be conducted with an eye  toward identifying redundancies and savings.
Although the Army’s capability portfolio reviews have been repeatedly  cited as a potential model for DOD, they did not consistently include key  requirements, acquisition, and budget portfolio review elements that could  have yielded more comprehensive reviews. For example, most of these  reviews did not prioritize capability gaps, incorporate information on  science and technology needs, or assess investment options for multiple  potential funding scenarios. As a result, the reviews did not position the  Army, for example, to proactively identify strategies to deal with varying  funding scenarios. Results of our analysis of how well the Army’s six  portfolio reviews from 2012 incorporated key review elements are  included in table 6.
Furthermore, the Army’s more recent portfolio review efforts have not  been consistent, in part because of a lack of current guidance. A 2010  memorandum issued by the Secretary of the Army provided direction for  the reviews, although it has not been incorporated into Army guidance.  The memo was to be valid for 1 year after issuance and called for high  level Army officials to make recommendations during that year regarding  whether to continue conducting the reviews, incorporate them into other  Army requirements and acquisition processes, develop new processes  that combine requirements validation with portfolio reviews, or terminate  the reviews. The Standard for Internal Control in the Federal  Government—Third Edition, notes that agency policies should be clearly  documented. Despite the instructions in the memo, Army officials stated  that memo has not been updated, nor have the details of the  memorandum been incorporated into Army guidance, although the Army  has continued to conduct the reviews.
In the past two years, the Army has only conducted reviews for 4 of its 11  portfolios. Some of these more recent reviews were less comprehensive  than earlier reviews. For example, both the March 2014 Movement  Maneuver and the September 2013 Combating Weapons of Mass  Destruction capability portfolio reviews do not include key budget  elements, such as funding scenarios and charts showing the relative  costs of portfolio components over time. Without such information, review  participants cannot readily determine whether programs are affordable in  their broader portfolio context and what changes to the portfolio would be  warranted given varying potential funding scenarios.
The Army budget community has recently focused efforts on a different  review, the Long-Range Investment Requirements Analysis (LIRA). This  analysis produces an assessment of planned spending for weapon  systems by portfolio over 30 years. The Army conducts the analysis  annually to inform the Army’s Program Objective Memorandum, the final  product of the programming phase of the PPBE process. The analysis  includes funding information spanning key weapon systems’ lifecycles  from initial research and development to divestiture. According to Army  budget officials, the LIRA process enables the Army to assess levels of  risks and analyze if the investment decisions they make now will be  affordable in the long term, and determine the resources required for  each portfolio. Army requirements officials said that the LIRA is a broader,  longer-term examination of the Army’s budget than the portfolio reviews.  The Army’s portfolio reviews provide some information to inform the  LIRAs although the two are conducted separately.
section_title: Air Force Portfolio Reviews Address Most Key Review Elements, but Lack Certain Acquisition Information
paragraphs: The Air Force also conducts portfolio reviews that could offer lessons for  DOD due to their comprehensiveness and consideration of multiple risk  and funding scenarios. The Air Force began conducting these reviews,  called core function support plans, in 2010. The Air Force conducts the  reviews for 12 Air Force “core functions.” The purpose of the reviews is to  establish the best investment options for capabilities in the near- to long- term. Core function teams, which are led by “core function lead  integrators,” perform the reviews. The teams identify and prioritize top- level gaps and requirements within their portfolios and the plans help  inform capability development and the planning and programming phases  of the PPBE process. The teams may include requirements and budget  personnel, but membership varies. The acquisition community is involved  when it comes time to integrate the plans. The Air Force has a policy that  describes its strategic planning system, but it is dated and does not  provide guidance on the core function support planning process. The Air  Force is in the process of updating this policy. develop guidance specific to the reviews.
One strength of the Air Force reviews, which was different from the Army  and Navy reviews, was their assessment of priorities and needs given  multiple funding and risk scenarios. The core function teams assess  operational risk using multiple warfighting scenarios. Based on these  analyses, they identify needs and priorities given different funding  scenarios, which can be a good technique for evaluating trade-offs. In the  plans we reviewed, one of these funding scenarios reflected “zero real  growth,” a flat budget that accepted whatever risk level would result from  that funding level. The other funding level was relatively unconstrained  and reflected the funding needed to conduct operations with moderate  risk. Air Force core function team officials also said they have used  additional funding levels to develop core function support plans, such as  zero real growth minus 5 or 10 percent. Based on these scenarios, the  team creates prioritized lists of capabilities, and sometimes potential cuts  or offsets, to inform budget decisions. The core function support planning  process has led to decisions to reduce tactical airlift capacity, minimize  facilities modernization and renovation, and increase munitions  procurement. An Air Force official responsible for integrating the plans  explained that trades have been made both within and across portfolios.  In order to reduce risk in one core function it is often necessary to accept  additional risk in another.
The Air Force’s core function support plans could potentially serve as a  model for DOD, but they lack a few key elements that could contribute to  a more integrated review. We found that the Air Force’s reviews included  a wide variety of key portfolio review elements, but lacked discussions of  potential redundancies and included limited information on the health or  status of key acquisition programs. The assessments’ content was also  relatively consistent, in part, because the Air Force provides a template  for the plans. Results of our analysis of portfolio reviews from 2012 are  included in table 7.
section_title: Navy Conducts Analytical Assessments to Inform Navy-wide Investment Plans
paragraphs: conduct in-depth reviews for its individual  portfolios and  include several key portfolio review  elements in its reviews.
The Navy does not conduct in-depth reviews for its individual portfolios,  but it does conduct annual analytical assessments at the Navy enterprise- level to inform the budget process that could offer lessons for DOD.  These analytical assessments, called for under the Navy’s Warfighting  Capability, Capacity, and Wholeness policy, evaluate proposed budgets  in the context of capability priorities and risks and provide  recommendations to leadership to inform resource-allocation decisions.  They identify near-, mid-, and long-term capability and capacity gaps and  provide quantitative and qualitative information to support trades among  Navy programs. The Navy’s analytical arm in the Office of the Chief of  Naval Operations leads these assessments and is to integrate information  from the requirements and budget communities within the Navy when it  conducts them. The acquisition community is generally not involved in  these reviews. The process includes four major assessments (see figure  4).
From a Navy-wide perspective, the Navy has used its assessments to  balance its available resources against its warfighting needs by making  trades to fit its priorities within its overall top line. The assessments we  reviewed included recommendations regarding program cancellations,  truncations, and deferrals. For example, as a result of a more recent  assessment, the Navy decided to end production of the Joint Standoff  Weapon. Navy officials said that its analytical arm determined that the  Navy could truncate production because it had produced enough of the  weapon and had another system in development that could provide the  capability in the future. The Navy’s assessments also included  information on capability gaps and the timeline of systems to meet those  gaps by portfolio or capability area. This information is used to develop  the front-end assessment, which includes areas where the Navy should  invest additional resources to address priority gaps. According to Navy  officials, the front-end assessment is one of the most important products  of the assessment because it identifies areas of focus for the  requirements community and may identify areas of overlap or duplication.  The officials said that they can identify when more than one requirements  office has proposed a solution for the same capability gap.
While the Navy’s assessments are intended to provide an analytical basis  for evaluating budget proposals against capability needs; they do not  include an in-depth review of each portfolio and do not include key  information from the requirements, acquisition, and budget communities.  We assessed the Navy’s Warfighting Capability, Capacity, and  Wholeness assessments conducted to develop its fiscal year 2014  budget request and found that the assessments reflected some key  portfolio review elements (see table 8). However, the assessments also  did not include a number of other key portfolio review elements and, for  some elements, the information was not portfolio-based. By not including  such information, the Navy is missing an opportunity to focus on portfolio- specific challenges, such as the trade-offs that might be needed to  address the affordability of its shipbuilding portfolio.
section_title: Conclusions
paragraphs: DOD operates as a joint force on the battlefield, yet still does not use an  integrated approach to optimizing its weapon system investments. The  military services develop investment plans that reflect their own priorities.  Enterprise-level reviews of these plans and acquisition programs are  stove-piped within the requirements, acquisition, and budgeting  communities. Further, oversight processes largely focus on individual  programs. As a result, DOD continues to craft investment plans that  reflect military service preferences and are not affordable over the long- term and default to optimizing for and addressing problems in individual  programs instead of focusing on portfolios of programs that might provide  greater military capability at lower risk or cost.
Addressing these challenges through strategic investment planning and  portfolio management is important today given the constrained budget  environment, quickly evolving threats, and uncertainty about the types of  conflicts the United States may face in the future. But DOD currently lacks  the governance structure, policy, and decision-making authorities to  implement the more strategic and integrated approach to weapon system  investment embodied in portfolio management best practices. Completely  breaking down the stove pipes between the requirements, acquisition,  and budget communities; standardizing portfolio constructs; and  empowering enterprise-level portfolio managers to drive investment  decisions would require a fundamental restructuring of how DOD does  business—this is unrealistic in the short run and would be challenging  even in the long run. Nevertheless, DOD can lay the foundation for better  portfolio management by taking some incremental steps that have the  potential to improve the efficiency and effectiveness of its weapon system  investment planning and management processes.
DOD has an opportunity to make meaningful changes to its portfolio  management efforts if senior leadership makes a commitment and invests  the resources to do so. Several elements stand out as candidates for  immediate action. First, DOD’s current capability portfolio management  policy is dated, senior leadership commitment has waned, and, as a  result, the policy is not being implemented. Reforming this policy to better  reflect best practices is essential to bridging the stovepipes in DOD’s  requirements, acquisition, and budget processes and moving DOD  toward a more integrated approach to planning and managing its weapon  system investments. Senior leadership with direct line management  authority over the organizations responsible for implementing any policy  will also be needed if the outcomes from any renewed portfolio  management effort will be different from those in the past. Next, if DOD is  to move toward a weapon system investment plan that is affordable,  strategy-driven, and balanced between near-term and long-term needs,  then portfolio managers will need better data and analytical tools to  assess threats and capability needs and make trade-offs between groups  of acquisition programs that can address them. Finally, as DOD has  recognized in its Better Buying Power initiatives, the military services offer  potential lessons for DOD’s portfolio management efforts. The military  services have used portfolio reviews and other assessments to reduce  redundancies, realign resources, produce more analytically rigorous  budget proposals, and plan for budget uncertainty. However, even these  efforts can be improved by making sure they reflect key portfolio review  elements from across the requirements, acquisition, and budget  communities.
section_title: Recommendations for Executive Action
paragraphs: To improve DOD’s use of portfolio management for its weapon system  investments and ensure that its investment plans are affordable, strategy- driven, balance near- and long-term needs, and leverage efforts across  the military services, we are making two recommendations for executive  action.
First, to provide a solid foundation for future portfolio management efforts  at the enterprise-level, we recommend that the Secretary of Defense  revise DOD Directive 7045.2 on Capability Portfolio Management in  accordance with best practices and promote the development of better  tools to enable more integrated portfolio reviews and analyses of weapon  system investments. Key elements of this recommendation would include designating the Deputy Secretary of Defense or some appropriate  delegate responsibility for implementing the policy and overseeing  portfolio management in DOD;  requiring annual enterprise-level portfolio reviews that incorporate key  portfolio review elements, including information from the  requirements, acquisition, and budget processes; directing the Joint Staff, AT&L, and CAPE to collaborate on their data  needs and develop a formal implementation plan for meeting those  needs either by building on the database the Joint Staff is developing  for its analysis or investing in new analytical tools; and  incorporating lessons learned from military service portfolio reviews  and portfolio management activities, such as using multiple risk and  funding scenarios to assess needs and re-evaluate priorities.
Second, to help ensure the military services’ portfolio reviews are  conducted regularly and effectively integrate information from the  requirements, acquisition, and budget communities, the Secretary of  Defense should direct the Secretaries of the Army, Navy, and Air Force to  update or develop policies that require them to conduct annual portfolio  reviews that incorporate key portfolio review elements, including  information from the requirements, acquisition, and budget processes.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to DOD for comment. In its written  comments, which are reprinted in full in appendix IV, DOD partially  concurred with our two recommendations. However, as discussed below,  DOD’s planned actions will not fully address the issues we raised in this  report.
DOD partially concurred with our first recommendation to revise DOD  Directive 7045.20 on Capability Portfolio Management in accordance with  best practices and promote the development of better tools to enable  more integrated portfolio reviews and analyses of weapon system  investments. In its response, DOD agreed with the need to further  develop portfolio management tools, ensure access to authoritative data,  and incorporate lessons learned by others performing portfolio  management. However, DOD stated that other aspects of our  recommendation were redundant to, and would conflict with other  processes and activities in place to perform portfolio management. DOD’s  response presumes that its existing portfolio management processes and  activities have been effective and fails to acknowledge that they have not  successfully addressed weapon system investment challenges, such as  affordability issues in certain portfolios and potential duplication among  some programs. We disagree that existing processes and activities have  been effective. Further, we continue to believe that our recommendation  to revamp DOD’s portfolio management policy to align with best  practices, establish clear responsibility for its implementation, and  conduct annual portfolio reviews would provide the foundation for  improved weapon system investment planning and management. DOD’s  specific comments on portfolio management policy, leadership, and  annual portfolio reviews follow.
Policy: DOD does not plan to revise its capability portfolio  management policy in accordance with best practices, as we  recommended, but rather stated the policy should be rescinded and  stakeholders directed to participate in portfolio management through  its requirements, acquisition, and budget processes. In addition, DOD  stated that recent revisions to its requirements and acquisition policies  vastly improve the integration of these processes. We continue to  believe that having an enterprise-level portfolio management policy  that is consistent with best practices is necessary to provide a solid  foundation for future portfolio management efforts. Rescinding, rather  than revising, its enterprise-level portfolio management policy and  relying on its requirements, acquisition, and budgeting process to fill- in the policy gaps could reinforce the stove-piped governance  structure that we found to be an impediment to integrated portfolio  management.
Sustained leadership: DOD does not plan to designate the Deputy  Secretary of Defense or an appropriate delegate responsibility for  overseeing portfolio management, as we recommended. DOD  responded that the Deputy Secretary of Defense already makes  portfolio management decisions in the context of DOD’s budget  process and that, as long as these decisions are informed by the  requirements and acquisition communities, they are sufficient to  adjust portfolios to meet the military’s needs. We continue to believe  that sustained enterprise-level leadership for implementing portfolio  management processes is needed to support sound portfolio  management decisions by the Deputy Secretary of Defense or others.  Our review found that key DOD portfolio management-related efforts,  including some designed to inform the budget process and some  called for in guidance, have lacked sustained leadership and been  inconsistently implemented. If the Deputy Secretary of Defense or an  appropriate delegate assumed leadership for portfolio management, it  could help ensure portfolio management policy as well as other  portfolio management efforts are more consistently implemented  throughout the department.
Annual portfolio reviews: DOD does not plan to require annual  enterprise-level portfolio reviews that integrate key portfolio review  elements from the requirements, acquisition, and budget processes,  as we recommended. DOD responded that portfolio reviews would be  redundant to annual reviews already conducted through existing  departmental processes. For example, DOD responded that the  annual Capability Gap Assessment process called for under the  requirements process serves as an annual portfolio review and that  AT&L leads reviews of long-term science and technology plans to  help frame future capability portfolios. DOD also stated that it uses  Capability Gap Assessments during the budget process to ensure that  military service budget submissions are consistent with the  warfighters’ capability needs. We continue to believe that annual  portfolio reviews that incorporate key portfolio review elements would  provide DOD a more integrated and comprehensive perspective on its  weapon system investment plans than its current processes provide.  The processes DOD described in its response largely reflect its stove- piped governance structures where the responsibilities for aspects of  portfolio management are dispersed among the requirements,  acquisition, and budget communities. Further, our review found that  portfolio reviews were inconsistently conducted, did not address all  portfolios, or only involved parts of the organization. As a result, we  continue to believe that updating DOD’s portfolio management policy  to require annual integrated enterprise-level portfolio reviews is the  best way to ensure DOD has the information it needs to craft weapon  system investment plans that are affordable, balance near- and long- term needs, and maximize return on investment.
DOD also partially concurred with our second recommendation to update  or develop policies that require the military services to conduct their own  annual portfolio reviews that incorporate key portfolio review elements  from the requirements, acquisition, and budget processes. However,  DOD did not indicate that it would take any action to address it. Instead,  DOD responded that the services’ budget processes and Office of the  Secretary of Defense’s review of the services’ budgets meet the intent of  our recommendation. We disagree. Our review found that military service  level portfolio reviews, including Air Force and Navy reviews closely  linked to their budget processes, did not always contain key portfolio  review elements or were not consistently conducted. We continue to  believe that updating or developing policies to require the services to  conduct annual portfolio reviews incorporating information from the  requirements, acquisition, and budget processes would help ensure the  services conduct regular, integrated reviews, which in turn should yield  benefits, such as prioritizing the most important investments and selecting  the optimum investment mix.
We are sending copies of this report to the appropriate congressional  committees; the Secretary of Defense; the Secretaries of the Army, Navy,  and Air Force; the Commandant of the Marine Corps; and other  interested parties. This report will also be available at no charge on  GAO’s website at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or by e-mail at sullivanm@gao.gov.  Contact points for our Offices of Congressional Relations and Public  Affairs may be found on the last page of this report. Key contributors to  this report are listed in appendix V.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: Our objectives were to assess the extent to which (1) the Department of  Defense (DOD) uses portfolio management to optimize its weapon  system investments, (2) DOD conducts integrated portfolio reviews at the  enterprise level to optimize their weapon system investments, and (3) the  military services conduct integrated portfolio reviews to optimize their  weapon system investments and their experiences offer potential lessons  for DOD.
To address our first objective, we reviewed commercial portfolio  management best practices from GAO-07-388 Best Practices: An  Integrated Portfolio Management Approach to Weapon System  Investments Could Improve DOD’s Acquisition Outcomes and reviewed  portfolio management-related literature from peer reviewed journals,  books, and trade publications to determine which, if any, additional  practices should be added. Based on this review, we determined that no  changes to the previously identified practices were needed. We  compared these best practices with DOD’s weapon system investment  processes, which include the requirements, acquisition, and budget  processes. We analyzed relevant DOD policies and statutes, including:  DOD’s portfolio management policy, DOD Directive 7045.20 Capability  Portfolio Management; requirements policies, including Chairman of the  Joint Chiefs of Staff Instruction 3170.01I Joint Capabilities Integration and  Development System and the Manual for the Operation of the Joint  Capabilities Integration and Development System; acquisition policies,  including DOD Directive 5000.01 The Defense Acquisition System and  DOD Instruction 5000.02 Operation of the Defense Acquisition System;  the budget policy, DOD Directive 7045.14 The Planning, Programming,  Budgeting, and Execution (PPBE) Process; and relevant sections of title  10 of the United States Code. We reviewed relevant prior GAO reports  and DOD memoranda, including the November 2014 memorandum from  the Deputy Secretary of Defense on the support for strategic analysis  process and the July 2014 memorandum from the Vice Chairman of the  Joint Chiefs of Staff on the support for strategic analysis process. We also  compared DOD’s capability portfolio management policy with portfolio  management best practices to evaluate the extent to which it reflects  those practices. To describe previous capability portfolio management  efforts in the mid-to-late 2000s and the extent they are being  implemented today, we reviewed relevant documents, including a  September 2006 memorandum from the Deputy Secretary of Defense on  capability portfolio management test cases and military service  requirements, acquisition, and budget policies. Based on our assessment  of these policies and additional documentation, we also identified  potential impediments to DOD’s use of portfolio management to optimize  investments across the department. To inform all assessments for this  objective, we interviewed officials from DOD enterprise- and military  service-level requirements, acquisition, and budget communities,  including:
Requirements: Force Structure, Resource and Assessment  Directorate, J-8, and Directorate for Intelligence, J-2 for the enterprise  level; Army Capabilities Integration Center and Headquarters,  Department of the Army G-3/5/7 Capability Portfolio Review and  Integration; Directorate of Resources; Air Force Office of the Director  of Operational Capability Requirements, A5R; the Air Combat  Command; Office of the Chief of Naval Operations, N803, Joint &  Urgent Requirements; Office of the Chief of Naval Operations, N96,  Surface Warfare; and Marine Corps, Office of the Deputy  Commandant, Combat Development and Integration.
Acquisition: Office of the Under Secretary of Defense for Acquisition,  Technology and Logistics (AT&L) for the enterprise level; Office of the  Assistant Secretary of the Army for Acquisition, Logistics, and  Technology; Office of the Assistant Secretary of the Air Force for  Acquisition; Office of the Assistant Secretary of the Navy for  Research, Development, and Acquisition; and Marine Corps Systems  Command.
Budget: Office of the Secretary of Defense, Cost Assessment and  Program Evaluation (CAPE) and Office of the Under Secretary of  Defense Comptroller  for the enterprise level; Directorate of  Resources, Army G-8, Force Development; Air Force Division Chief,  Planning, Programming, and Requirements Division, Air Combat  Command; the Office of the Chief of Naval Operations, N81,  Integration and Force Structure and the Office of the Chief of Naval  Operations, Fiscal Management, N82 and Office of the Assistant  Secretary of the Navy for Financial Management and Comptroller; and  Marine Corps Office of Programs and Resources.
To address our second objective, we reviewed relevant guidance and  documents to identify enterprise-level portfolio reviews and other  portfolio-level analyses that DOD components conduct and interviewed  DOD’s enterprise-level requirements, acquisition, and budget  communities. We analyzed standards in the Project Management  Institute’s The Standard for Portfolio Management—Third Edition and  GAO best practices and compared them with DOD policies. Based on this  analysis, we developed a list of key portfolio elements which, if included  in portfolio reviews, could help integrate key portfolio management  information from the requirements, acquisition, and budget  communities—information that is needed for an integrated approach to  prioritizing needs and allocating resources in accordance with strategic  goals. We compared the Joint Staff Functional Capability Board portfolio  reviews’ content with these key portfolio review elements. To assess  whether DOD has conducted affordability analyses in accordance with  DOD Instruction 5000.02 and DOD’s Better Buying Power guidance, we  reviewed six Defense Acquisition Board briefings for programs that held  an acquisition or milestone review from January to June 2014. To assess  whether DOD has conducted portfolio reviews and other portfolio-based  analyses to identify and eliminate potential redundancies in accordance  with DOD’s Better Buying Power guidance, we interviewed Joint Staff,  AT&L, and military service officials, and reviewed relevant guidance. We  also interviewed Joint Staff, AT&L, and CAPE officials to determine the  extent to which the three offices have collaborated on these portfolio  reviews and portfolio-based analyses, and to determine whether they had  the necessary data, resources, and analytical tools to effectively conduct  reviews.
To address our third objective, we reviewed Army, Air Force, and Navy  requirements, acquisition, and budget policies to determine the extent to  which they required portfolio reviews. We analyzed Army and Air Force  portfolio reviews, and the Navy’s analytical assessments by comparing  their contents with key portfolio review elements. The military service  assessments we evaluated include:
Army: the six capability portfolio reviews the Army conducted in 2012,  including Combat Vehicles; Precision Fires; Sustainment  (Transportation); Aviation; Assured Mobility and Protection; and  Mission Command; and the two most recent unclassified reviews  provided by the Army, including Combating Weapons of Mass  Destruction from 2013 and Movement and Maneuver from 2014;
Air Force: of its 12 core function support plans from 2012 that  informed the Air Force’s fiscal year 2015 president’s budget request,  we reviewed the nine that had weapon system centric portfolios,  including Global Precision Attack; Space Superiority; Air Superiority;  Global Integrated Intelligence, Surveillance, and Reconnaissance;  Command and Control; Rapid Global Mobility; Nuclear Deterrence  Operations; Personnel Recovery; and Special Operations;
Navy: four assessments that were part of the Navy’s Warfighting  Capability, Capacity, and Wholeness process to inform the Navy’s  Program Objective Memorandum for fiscal year 2014, including  Capability Assessments; Front-end Assessments; Warfighting  Support and Capability Plan; and Integrated Program Assessment.
We also interviewed officials from all four military services’ requirements,  acquisition, and budget communities to determine how they conduct  these assessments, the role of the assessments in their weapon system  investment decision-making, and the results of the assessments. Based  on our assessment of these policies, reviews, and interviews with service  officials, we identified the strengths and weaknesses of these reviews and  the potential lessons they may offer DOD.
section_title: Appendix II: Key DOD Enterprise- and Military Service-Level Policies Governing the Requirements, Acquisition, and Budget Processes for Weapon System Investments
subsections: section_title: Enterprise-Level Policies
paragraphs: Department of Defense, Manual for the Operation of the Joint Capabilities  Integration and Development System (Feb. 12, 2015).
Chairman of the Joint Chiefs of Staff Instruction 3170.01I, Joint  Capabilities Integration and Development System (JCIDS) (Jan. 23,  2015).
Chairman of the Joint Chiefs of Staff Instruction 5123.01G, Charter of the  Joint Requirements Oversight Council (JROC) (Feb. 12, 2015).
Department of Defense Directive 5000.01, The Defense Acquisition  System (May 12, 2003).
Department of Defense Instruction 5000.02, Operation of the Defense  Acquisition System (Jan. 7, 2015).
Department of Defense Directive 7045.14, The Planning, Programming,  Budgeting, and Execution (PPBE) Process (Jan. 25, 2013).
Chairman of the Joint Chiefs of Staff Instruction 8501.01B, Chairman of  the Joint Chiefs of Staff, Combatant Commanders, Chief, National Guard  Bureau, and Joint Staff Participation in the Planning, Programming,  Budgeting and Execution Process (Aug. 21, 2012).
section_title: Military Service-Level Policies
subsections: section_title: Army
paragraphs: Army Regulation 1-1, Planning, Programming, Budgeting and Execution  System (Jan. 30, 1994).
Army Training and Doctrine Command Regulation 71-20, Concept  Development, Capabilities Determination, and Capabilities Integration  (Jun. 28, 2013).
Army Regulation 71-9, Warfighting Capabilities Determination (Dec. 28,  2009).
Army Regulation 70-1, Army Acquisition Policy (Jul. 22, 2011).
Department of the Army Pamphlet 70-3, Army Acquisition Procedures  (Apr. 1, 2009).
section_title: Air Force
paragraphs: Air Force Policy Directive 10-6, Capability Requirements Development  (Nov. 6, 2013).
Air Force Instruction 10-601, Operational Capability Requirements  Development (Nov. 6, 2013).
Air Force Policy Directive 63-1, Integrated Life Cycle Management (Jul. 3,  2012).
Air Force Instruction 63-101/20-101, Integrated Life Cycle Management  (Mar. 7, 2013).
Air Force Policy Directive 65-6, Financial Management: Budget (May 1,  1998).
Air Force Policy Directive 90-11, Strategic Planning System (Mar. 26,  2009).
section_title: Navy
paragraphs: Secretary of the Navy Manual M-5000.2, Department of the Navy:  Acquisition and Capabilities Guidebook (May 9, 2012).
Secretary of the Navy Instruction 5000.2E, Department of the Navy  Implementation and Operation of the Defense Acquisition System and the  Joint Capabilities Integration and Development System (Sept. 1, 2011).
Chief of Naval Operations Instruction 3050.25, Warfighting Capability,  Capacity, and Wholeness Assessments (Jul. 30, 2012).
section_title: Appendix III: Implementation of Portfolio- Based Analyses to Support Individual Acquisition Program Decisions
paragraphs: In addition to its portfolio review efforts, the Department of Defense  (DOD) has called for other portfolio-based analyses to support acquisition  decisions for individual acquisition programs. These analyses are to  review the program in the context of its corresponding portfolio. The  Office of the Under Secretary of Defense for Acquisition, Technology and  Logistics (AT&L) called for these portfolio-based analyses to ensure the  affordability of its weapon systems programs and to eliminate programs  that are potentially duplicative of other systems—two areas where we  have identified challenges. However, the Assistant Secretary of Defense  for Acquisition said that AT&L has not implemented the latter of these  efforts because it lacks readily accessible data, resources, and tools to do  so.
AT&L requires the military services and other DOD components to  conduct portfolio-based affordability analyses under Better Buying Power  and in its acquisition instruction; however, AT&L has not reviewed the  quality of the analyses or used them to inform acquisition decisions. We  also found that these efforts are only partially being implemented (see  table 9). Better Buying Power guidance from 2010 and 2012 recommends  portfolio-based affordability analyses to inform acquisition program  milestone decisions. Subsequently, AT&L required affordability analysis  for acquisition programs under its primary acquisition policy—DOD  Instruction 5000.02—as a tool to promote responsible and sustainable  investment decisions. This policy states that the purpose of the  affordability analysis is to avoid starting or continuing programs that DOD  cannot produce and support within reasonable expectations for future  budgets. The military services are to consider the long-term affordability  of a program within the context of its portfolio to inform the program’s  milestone decisions. The portfolios to be used for these analyses were  provided by the services and are generally different than the portfolios  used for requirements and budget purposes at the enterprise-wide level.  AT&L officials said they have not monitored the military services’  compliance with conducting the analyses or reviewed the quality of the  analyses because the responsibility for conducting them falls to service  leadership.
In our review of defense acquisition board briefings from January through  June 2014, we found that 5 of 6 programs that held reviews included an  affordability analysis. However, two of these analyses did not look at  affordability over the recommended 30- to 40-year period, and for the  three in which the portfolio exceeded the total obligation authority in any  given year, there was no discussion of possible trades to make the  portfolio affordable. According to the Assistant Secretary of Defense for  Acquisition, statute charges the military services with equipping the force,  so it is the services that have to make trade-offs. It is not an AT&L role.
Better Buying Power guidance from 2012 also called for portfolio-based  analyses to identify potential redundancy and eliminate unnecessary  duplication; however these analyses have not been conducted (see table  10). AT&L was to conduct these analyses for ACAT I programs in support  of all acquisition program decisions, including at requirements decision  points, and during annual program and budget review processes. The  military services were to conduct these analyses for lower dollar  acquisition programs. The most recent version of the Better Buying Power  guidance still contains the goal of eliminating redundancy within  warfighter portfolios, but it is no longer an area of emphasis.
section_title: Appendix IV: Comments from the Department of Defense
section_title: Appendix V: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
paragraphs: Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov.
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Ron Schwenn, Assistant Director;  Laura Holliday, Analyst-in-Charge; Jenny Chanley; Dani Greene; Kristine  Hassinger; Sean Manzano; Carol Petersen; and Erin Preston made key  contributions to this report.