Weapon System Acquisitions: Opportunities Exist to Improve the Department of Defense's Portfolio Management
Released: 2015-08-27T00:00:00

Letter

Background

Weapon system investment decision-making in DOD is highly complex. It involves numerous entities, levels, and policies at the military service and enterprise level. The four military services—the Army, Air Force, Navy, and Marine Corps—by law have authority to organize, train, and equip their services. They make initial decisions regarding what to buy, how to buy it, and how much it will cost. Those decisions are made by their requirements, acquisition, and budget communities and go through multi- layered review processes within the services. Each service has its own requirements, acquisition, and budget policies, which are based in part on enterprise-level policies.

For major investments, the military services’ decisions regarding what to buy, how to buy it, and how much to buy may be reviewed and approved or disapproved at the enterprise level. At the enterprise level, JCIDS assesses, validates, and prioritizes warfighter needs; the Defense Acquisition System oversees product development and procurement; and PPBE allocates resources taking into account capability needs, risk, and affordability. Each of those processes is governed by its own set of policies (See appendix II for a list of relevant policies).

Weapon System Investment Challenges

DOD faces several weapon system investment planning challenges. First, DOD plans more weapons acquisition programs than it can afford. In the Congressional Budget Office’s (CBO) most recent analysis of DOD’s 5- year Future Years Defense Program (FYDP), CBO projected acquisition costs for weapons and other major equipment to increase by 21 percent between fiscal years 2015-2019, and to peak in the early 2020s at about 43 percent higher than in 2015. CBO explained that the steep increase in acquisition costs it projects beyond the FYDP timeframe shows that the classic bow wave has returned—meaning that DOD is pushing out funding needs assuming it will receive higher acquisition funding levels in the future to complete its programs. Similarly, CBO estimated that the total costs of carrying out the Navy’s fiscal year 2015 shipbuilding plan would require one-third more funding on an annual basis than the Navy has received in recent decades. CBO reported that, as a result, the Navy will likely have to purchase substantially fewer ships than under its current plans or make other trade-offs. In April 2015, we also raised affordability issues with the Joint Strike Fighter—DOD’s largest weapon acquisition program. We estimated the program will require an average of $12.4 billion per year, which represents around one-quarter of DOD’s annual funding for major weapons programs, over the next 5 years. We reported that it is unlikely the program will be able to sustain such a high level of annual funding and if required funding levels are not reached, the program’s procurement plan may not be affordable.

In addition, the military services have a history of tailoring their planned investments to their own needs rather than trying to find common ground, which has led to inefficiencies and, in some cases, duplication. More specifically, the military services sometimes generate unique system requirements that justify multiple acquisition programs to meet similar needs. Our work has concluded that redundancy or overlapping capabilities in some areas may be desirable or necessary, but acquisition efforts to develop similar capabilities can also result in the same capability gap being filled twice or more, lead to inefficient use of resources, and contribute to other warfighting needs going unfilled. In 2012, we found potential duplication among electronic warfare systems, some of which DOD has canceled. In 2013, we found additional areas of overlap in the roles that electronic warfare systems were intended to perform and that new areas of overlap and potential duplication could emerge as investment plans evolved. More recently, we concluded that when developing a new ground radar system, the Air Force focused on what made its requirements unique from a similar Marine Corps’ system, instead of looking for ways to leverage the Marine Corps’ program. We found these programs to be potentially duplicative.

Portfolio Management

Leading commercial companies use a portfolio management approach to optimize their investments. They begin with an enterprise-level identification and definition of market opportunities—in other words, potential customer wants or needs—and then prioritize those opportunities while taking into account resource constraints. Once these companies have prioritized opportunities, they draft initial business cases for alternative product ideas that could be developed to exploit each of the highest priority opportunities. Each alternative product proposal enters a gated review process. At each review, product proposals are assessed against others in the portfolio weighing corporate resources, established criteria, competing products, and the goals and objectives of the organization as a whole. As alternatives pass through each review, the number is expected to decrease, until only those alternatives with the greatest potential to succeed make it into the product portfolio. In effect, new programs are born from portfolio analysis and management. Over time, as potential new products are identified or if products under development experience problems, companies review and rebalance their portfolio of proposed and existing product investments based on those products that add the most value and are the best strategic fit.

More generally, rather than defaulting to optimizing individual programs, portfolio management focuses on selecting the optimum mix of programs and modifying that mix as needed over time. Take a hypothetical example in which DOD starts with 10 programs and $50 billion to invest. Without portfolio management, program managers will seek to get the most that they can out of each of the 10 programs, without assessing their aggregate contributions to defense. Using portfolio management, DOD executives would look at different combinations of and approaches to the 10 programs to see what, collectively, would provide the best capabilities for $50 billion. This would enable executives to decide, for example, whether it is better to concentrate more investment in 7 programs rather than fund all 10 as best as possible. In another example, if a program began to have cost or performance problems, portfolio management would consider whether the other programs in the portfolio could address the requirements of the problematic program rather than just putting more money into it.

Portfolio management literature has identified characteristics and effects of investment planning processes that are not based on portfolio management. These characteristics and effects, which are shown in figure 1, are consistent with some of our observations on DOD’s weapon system investment challenges.

Portfolio Management Best Practices

In 2007, we identified several best practices for portfolio management, which form the basis for our assessments in this report. Organizations that follow these practices assess product investments collectively from an enterprise level, rather than as independent and unrelated initiatives; continually make go/no-go decisions through a gated review process to rebalance portfolios based on investments that add the most value; use an integrated approach to prioritize needs and allocate resources in accordance with strategic goals; rank and select investments using a disciplined process to assess the costs, benefits, and risks of alternative products; empower leadership to make investment decisions and hold leadership accountable for investment outcomes; and provide sustained leadership for portfolio management.

Portfolio Reviews

Portfolio management best practices and the Project Management Institute’s portfolio management standards also state that organizations should conduct regular reviews to adjust to strategic changes or changes in the mix of products within a portfolio, among other reasons. These reviews are tools that organizations use to plan and manage their investments. From a DOD perspective, portfolio reviews can help increase return on taxpayers’ investments in weapon systems in a number of ways, such as: helping to ensure investments align with national security and military strategies; prioritizing the most important investments; selecting the optimum mix of investments; identifying and eliminating unwarranted duplication; monitoring programs’ health to determine whether changes to the portfolio are warranted; and determining whether investments are affordable. Although there is not one model for portfolio reviews that fits all situations, we identified key portfolio elements, which—if included in a review—would integrate relevant information from the requirements, acquisition, and budget communities to achieve some of the benefits described above (see table 1).

DOD Is Not Using an Integrated Portfolio Management Approach to Optimize Its Weapon System Investments at the Enterprise Level

DOD is not using an integrated portfolio management approach to optimize its weapon system investments at the enterprise level, as called for in best practices. DOD lacks the governance structure, sustained leadership, and policy to do so. In particular, DOD’s governance structure includes decision-making processes and responsibilities that are divided among its stove-piped requirements, acquisition, and budget communities and focus largely on optimizing individual investments. This fragmentation does not allow for an integrated portfolio management approach to making investment decisions across the department. In addition, the military services operate in stove-pipes of their own and tend to optimize investments for their service, rather than for the military as a whole. A perceived lack of enterprise-level influence over the services hinders DOD’s ability to address this behavior and optimize investments across the department.

DOD’s Stove-Piped Governance Structure Is an Impediment to Portfolio Management

DOD’s stove-piped governance structure is an impediment to using an integrated portfolio management approach to optimize weapon system investments. DOD has numerous enterprise-level processes, organizations, and decision makers to oversee its weapon system investments, which generally operate as stove-pipes, not as an integrated whole. In addition, the acquisition process and to some extent, the requirements process, are geared toward making decisions about individual programs rather than assessing investments collectively from an enterprise level, as best practices suggest. DOD has taken some steps to integrate its requirements, acquisition, and budget processes. However, these steps primarily include sending representatives to participate in the other communities’ decision-making forums. As a result, most investment decisions get made on a piecemeal basis within the requirements, acquisition, and budget processes. DOD officials stated that decision-making forums, such as the Deputy Secretary of Defense’s Management Action Group, serve an integrating function and make departmental management decisions on a more cross-cutting basis, but these forums primarily address issues on an ad hoc basis. Table 2 describes the key enterprise-level governance structures that are responsible for DOD weapon system investment decision-making.

Other aspects of DOD’s governance structure for making weapon system investment decisions are also fragmented and inconsistent with an integrated portfolio management approach.

Decision-making responsibilities: DOD’s enterprise-level decision- making responsibilities are fragmented, which makes it difficult to identify who is responsible for investment decisions and to hold them accountable. According to portfolio management best practices, leadership should be clearly defined and held accountable for outcomes. In DOD, weapon system investment decisions are dispersed among multiple decision makers. This fragmentation could lead to conflicting decisions or decisions that align with the goals of one community, but not those of another. Several DOD officials said weapon system investment decision-making is dispersed by design and that natural tension between and different perspectives among decision makers are beneficial to the process. Ultimately, though, these diffuse decision-making responsibilities make it difficult to determine who is empowered to make enterprise-level weapon system investment decisions and who can be considered to be portfolio managers. In our enterprise- and service-level interviews, some officials stated that no one at the enterprise level is empowered to make investment decisions, and others responded that only a strong Secretary of Defense or Deputy Secretary Defense can take actions required for effective portfolio management, such as making trades across services or canceling programs.

Investment planning assumptions: DOD is not using an integrated, common set of planning assumptions, such as warfighting scenarios, to inform investment decisions across the department. Best practices state that organizations should rank and select investments using a disciplined process to assess the costs, benefits, and risks of alternative products. Former senior and current DOD officials explained that it is difficult to analyze investments across the services’ portfolios without a common analytical baseline because organizations use scenarios that favor their specific interests. In a November 2014 memo, the Deputy Secretary of Defense outlined steps to reinvigorate DOD’s support for strategic analysis efforts and move toward a common starting point for departmental analysis. These steps included Deputy Secretary and Vice Chairman of the Joint Chiefs of Staff level review of scenario selection, key assumptions, and concepts of operations, among others areas.

Portfolio constructs: To the extent that the requirements, acquisition, and budget communities group investments by portfolio, they do so using different portfolio constructs. According to portfolio management best practices, organizations should use an integrated approach to prioritize needs and allocate resources in accordance with strategic goals. Using different portfolio constructs is a barrier to taking an integrated approach. At the enterprise level, the requirements community uses nine joint capability areas for examining warfighting needs, acquisition portfolios vary by military service, and budget data is organized into eleven major force programs for the FYDP. (Table 3 lists these portfolios.)

DOD attempted to standardize portfolios in the 2006 to 2008 time period. However, a former senior official who was involved in that effort said the mapping was “impossible” and that there was organizational resistance because the portfolios did not align with many decision makers’ areas of responsibility. Many of the enterprise- and service-level officials we interviewed said using a wide variety of constructs is necessary and sometimes beneficial given the different roles and perspectives of the organizations involved. However, when they want to analyze their portfolios from another perspective—for example, examining funding associated with joint capability areas—they have to go through extensive mapping exercises, as notionally illustrated in figure 2.

DOD Lacks Sustained Leadership to Implement Portfolio Management

DOD lacks sustained leadership for portfolio management, and as a result, it has been difficult for DOD to maintain momentum for portfolio management-related initiatives. Our prior work found that effective portfolio management requires a governance structure with sustained leadership. However, changing leadership priorities pose challenges to portfolio management in DOD. In the mid-to-late 2000s, DOD made a concerted effort to implement capability portfolio management. DOD initiated portfolio management pilots in select capability areas to improve strategic decisions on resource allocation across programs in 2006 and issued a directive on capability portfolio management in 2008. Soon afterwards, leadership priorities shifted away from portfolio management. DOD eliminated the pilots, did not document the results of the pilots, and stopped implementing the directive. This was attributed to a variety of factors, including the lack of a senior level champion for the effort as political leadership changed, which can happen every few years in DOD. Officials from AT&L and the Joint Staff said that DOD no longer has a champion for portfolio management.

DOD Lacks a Portfolio Management Policy That Reflects Best Practices

DOD does not have a policy to guide portfolio management across the department that fully reflects key best practices. The policy is also not current and DOD is not implementing it, but it has not been rescinded. DOD Directive 7045.20, Capability Portfolio Management, states that DOD shall use capability portfolio management to advise the Deputy Secretary of Defense and the heads of the DOD components on how to optimize investments across DOD and minimize risk in meeting capability needs. Portfolios are to be managed by civilian and military capability portfolio managers who are to develop capability portfolio strategic plans, provide cross-component perspectives, recommend capability priorities to senior decision makers, and take other actions, such as developing portfolio governance forums. In addition, the policy attempts to standardize DOD’s portfolio constructs by aligning portfolios with joint capability areas.

However, under the policy, portfolio managers do not have sufficient authority to effectively influence weapon system investment decisions. They do not have any independent decision-making authority, nor are they to “infringe on any existing statutory or regulatory authorities” such as the military services’ authority under title 10 to organize, train, and equip their forces. Instead, their influence is to be through the Deputy Secretary’s Management Action Group, which is to use information from the portfolio managers to inform the Deputy Secretary of Defense. More generally, the policy is intended to operate within DOD’s existing fragmented organizational and process framework—which includes multiple barriers to integrated portfolio management. Table 4 includes our assessment of how well DOD’s current portfolio management policy addresses key best practices.

Officials we met with throughout DOD explained that they are not implementing DOD’s capability portfolio management policy. A few officials stated that DOD is implementing aspects of portfolio management, but in a less integrated fashion. Others appeared to be unaware of the policy. Several enterprise-level officials who were familiar with the policy stated that DOD is not implementing the directive because elements, such as using a single set of portfolios, are impractical and leadership priorities changed. In the policy, DOD also did not identify an office to be responsible for its overall implementation. Federal internal control standards require that organizations establish objectives and clearly define key areas of authority and responsibility. The Office of the Under Secretary of Defense for Policy (OUSD (Policy)) issued DOD’s capability portfolio management directive, however, that office does not have direct line management authority over the organizations responsible for implementing it. DOD officials explained that the only officials with authority over those organizations are the Deputy Secretary of Defense and Secretary of Defense. An official from OUSD (Policy) stated that they are aware that the policy is not being followed. OUSD (Policy) is reviewing the policy and may rescind or revise it in the future.

Perceived Lack of Enterprise-Level Decision- Making Authority over the Services Is an Impediment to Portfolio Management

Enterprise-level involvement is important to optimize investments collectively across the military because the military services tend to focus on optimizing their own investments. For DOD to effectively manage investments at the enterprise level, it would need to exercise its authority and influence service-level investment decisions and make cross-service trades. Three senior enterprise-level officials explained that the military services focus on maintaining their portion of the investment budget and tend to optimize investments for their own services, rather than for the military as a whole.

A perceived lack of authority over the military services is an impediment to managing investments collectively at the enterprise level. According to best practices, portfolio managers should be empowered to make investment decisions. Under title 10 the military services have the responsibility for equipping their forces—in other words, buying weapons. Enterprise-level officials we interviewed stated that they think this provides primary control over setting weapon system investment priorities to the services. While title 10 does not preclude enterprise-level influence over investment decisions, this influence currently comes in the form of oversight reviews after the services have developed investment plans, weapon system requirements, and acquisition plans. Joint Staff officials said they have portfolio management responsibilities, such as prioritizing requirements, but their influence is limited because the services control the money. AT&L officials stated that they have an oversight role, but cannot change the military services’ acquisition priorities. Similarly, DOD Comptroller officials explained that enterprise-level involvement to optimize investments across the department is limited by title 10.

Experiences with ground radar systems show some of the challenges the enterprise level has experienced when trying to optimize investments across services. In 2008, the Undersecretary of Defense for AT&L established a Radar Joint Analysis Team to develop a DOD-wide roadmap for radar technology, development, and procurement and consider overlap and commonality among programs. In two cases, the team encouraged the military services to collaborate on developing a single system to address their needs. However, the services had different budget priorities and acquisition timelines and therefore decided to pursue separate acquisition programs. More recently, some Office of the Secretary of Defense officials noted there were similarities between the Air Force’s Three-Dimensional Expeditionary Long-Range Radar and the Marine Corps’ Ground/Air Task Oriented Radar requirements, and questioned the need for two separate programs. An AT&L official said that the discussions that resulted from the questions were beneficial, but the Air Force and the Marine Corps are going forward with separate programs, in part, because they own their own resources.

DOD Does Not Conduct Integrated Enterprise-Level Portfolio Reviews

DOD does not conduct enterprise-level portfolio reviews that integrate the requirements, acquisition, and budget communities. Instead, the Joint Staff, AT&L, and CAPE each have their own limited portfolio review efforts, some of which are still being developed. Conducting portfolio reviews on a regular basis can help optimize portfolios. For example, reviews can serve as a tool to identify potentially duplicative capabilities or systems and evaluate trade-offs to address those inefficiencies. DOD officials said that limited resources and a lack of analytical tools have hampered their ability to conduct integrated portfolio reviews and portfolio-based analyses.

DOD Does Not Conduct Integrated Portfolio Reviews

DOD does not conduct integrated enterprise-level portfolio reviews. Instead, the Joint Staff, AT&L, and CAPE have a fragmented approach that involves separate, but limited, portfolio reviews and analyses using different portfolio constructs. None of these reviews and analyses have been consistently conducted. Portfolio management best practices state that organizations should conduct regular reviews to adjust to factors, such as strategic changes. Consistently conducting portfolio reviews provides opportunities to identify and address inefficiencies within portfolios, and to adjust and realign priorities to meet current needs.

In addition to not being conducted in an integrated manner, the reviews we assessed did not consistently incorporate key portfolio review elements, which would result in a more integrated portfolio review. Best practices state that taking an integrated approach to conducting reviews helps avoid pursuing more investments than are affordable, balance near- and long-term needs, and maximize return on investment. In DOD’s case, an integrated review for weapon system investments would involve the requirements, acquisition, and budget communities or information from those communities (see figure 3).

Joint Staff Reviews to Date Have Varied in Content and Quality

We assessed five out of the six Joint Staff portfolio reviews conducted in 2014 and found that these reviews varied widely in content and quality. A Joint Staff official who coordinated the reviews said they varied because it was the Joint Staff’s first year conducting them. The Joint Staff initiated the portfolio reviews to inform its position on DOD’s fiscal year 2016 budget submission. The goal of these reviews was to identify redundancies, possible trade-offs, investment opportunities, outdated requirements, and impactful issues for each portfolio. A Joint Staff official who led one of the reviews said that he found great value in conducting the review, which allowed his office to evaluate how the cost of a capability impacts whether other capability needs within the portfolio could be met. In comparing the reviews against the stated goals, all five included impactful issues, such as cost drivers within a particular portfolio. We found that four reviews included information on investment opportunities and redundancies; two addressed outdated requirements; and one identified possible trade-offs. Joint Staff officials said they plan to continue conducting portfolio reviews to inform the budget and to develop guidance for the next round of reviews to try to ensure they are more consistent.

The Joint Staff portfolio reviews did not integrate information from the requirements, acquisition, and budget communities. For example, none of the reviews included information on program cost, schedule, or performance, and only one included a timeline of when systems in the portfolio would be fielded. In addition, none of the reviews considered funding scenarios and only two reviews included information on portfolio affordability, such as details on funding for the portfolio over the next 5 years. By not incorporating this information into the reviews, the Joint Staff may be ill-equipped to shape decisions about balancing capability needs and portfolio affordability and offer recommendations on portfolio modifications to address potential budget changes.

The Joint Staff recently revised its JCIDS policy and guidance to provide more emphasis on portfolio management. The revised JCIDS manual states that Functional Capabilities Board (FCB) chairs need to monitor activities that could affect their portfolios throughout the year and reassess capability requirements portfolios as needed. For example, they may assess their portfolios at the request of Joint Staff senior leadership, to inform annual reviews such as the budget review, and to reassess the content or priorities of the portfolio due to changes in strategic guidance. In addition, the FCBs are to track acquisition programs’ progress toward satisfying capability requirements. Tracking progress will help ensure that the FCBs can assess changes to programs or timelines for their effect on their portfolio’s capability needs.

AT&L’s Portfolio Review Initiative Has Resulted in Few Reviews

In DOD’s first iteration of the Better Buying Power initiatives, which seek to address affordability challenges by increasing the efficiency of defense spending, AT&L directed that portfolio reviews be conducted to identify potential redundancies; however few have been conducted (see table 5). AT&L’s 2010 guidance called for portfolio reviews at the enterprise level and for the military services to conduct similar reviews for lower dollar acquisition programs, known as acquisition category (ACAT) II and III programs. (Appendix III assesses additional portfolio-based analyses called for under Better Buying Power.) The Better Buying Power initiatives have evolved over the past few years. The most recent version of the guidance still contains the goal of eliminating redundancy within warfighter portfolios, but it is no longer an area of emphasis.

In 2013, AT&L began conducting portfolio reviews, called mission area portfolio assessments, but they have had a limited focus and do not include some key information suggested by portfolio management standards and best practices. These reviews focused largely on identifying science and technology investments needed to keep up with future threats. An AT&L official said that the first set of reviews included some requirements, acquisition, and budget information such as identification of capability gaps, science and technology needs, and affordability analyses. However, the reviews did not include other key requirements and budget information, such as prioritized lists of capability gaps, outdated requirements, and funding scenarios that would allow them to look at the portfolio in an integrated manner as suggested by portfolio management standards and best practices. An AT&L official responsible for the reviews said that AT&L is in the beginning stages of developing these reviews and it is still determining how to break down larger portfolios to review, how they will complement other portfolio reviews, and what roles other Office of the Secretary of Defense stakeholders and the military services might play in the review process. AT&L initially planned to conduct these reviews on an annual basis, but it has not been able to do so due to time and resource constraints and a decision has yet to be made on whether the reviews will continue.

CAPE Does Not Conduct Regular Portfolio Reviews

At the direction of Deputy Secretary of Defense, CAPE coordinates strategic portfolio reviews for select portfolios or issue areas to inform budget decisions; however, these reviews have evolved over time and the topics covered vary from year-to-year. The original purpose of the strategic portfolio reviews was to: (1) look across service weapon system portfolios to identify redundancies and capability gaps, and (2) provide DOD components with earlier decisions on capability needs, including which component would include funding in their budget submission for the highest priority capabilities. CAPE has generally not used the reviews in the way that they were originally intended. The reviews are not necessarily organized by portfolio unlike the Joint Staff and AT&L reviews—they may sometimes address just one system. In addition, they address more specific, targeted questions that vary from year-to-year depending on what issues the Deputy Secretary of Defense identifies as important. CAPE officials also said that the reviews are not designed to specifically address duplication of weapon system capabilities. However, if CAPE notices potential duplication in the course of its reviews, it may propose changes to programs during the budget review process. According to the Assistant Secretary of Defense for Acquisition, the topics for last year’s reviews included space, strategic offense, protection forces, innovation, infrastructure, and readiness. A CAPE official stated that some reviews are more consistent with the stated purpose of strategic portfolio reviews than others. For example, a recent review of the nuclear enterprise helped prioritize capability needs and provided the context for evaluating investment plans during the program budget review process.

Lack of Resources and Analytical Tools Limits DOD’s Ability to Effectively Conduct Portfolio Reviews

The Joint Staff, AT&L, and CAPE reported that resource limitations have affected their ability to regularly conduct portfolio reviews. Despite these resource constraints, they have not coordinated their portfolio reviews to integrate and streamline them, although they have had conversations about their reviews and the Joint Staff and AT&L have provided input to strategic portfolio reviews. We have previously found that one way to better manage fragmented activities is to improve collaboration and coordination. This includes engaging in key practices such as defining and articulating common outcomes, agreeing on roles and responsibilities, and identifying and addressing needs by leveraging resources. Without better coordination and collaboration, DOD will struggle both from an information and resource perspective to implement integrated portfolio reviews.

Both AT&L and the Joint Staff said that a lack of readily accessible data and analytical tools also hampered their ability to effectively conduct portfolio reviews. AT&L officials said that more accessible data and better tools would be helpful for identifying redundancies and for the mission area portfolio assessments. Having readily accessible data and tools is essential for these analyses. For example, in the past, lack of visibility into certain programs, such as special access programs, urgent operational needs, and lower ACAT level programs contributed to potential unnecessary duplication. Joint Staff officials said that past efforts relied on repeated data calls, which were a drain on resources and time consuming. In addition, one of the officials in charge of a Joint Staff portfolio review said that the Joint Staff does not have the authority to compel other DOD components to provide the necessary information and data to conduct the reviews. Finally, Joint Staff officials report that they do not have the analytical tools and dynamic databases to effectively conduct portfolio reviews, assess potential redundancy, and prioritize capabilities.

The Joint Staff is currently developing a database to provide it with a better analytical tool to support portfolio management. The database is to track capability gaps to enable the Joint Staff to more easily identify potential redundancies, prioritize capability gaps and close out capability gaps that have been addressed or that no longer need to be addressed because the threats have evolved. Joint Staff officials said that assessing duplication is challenging because as investment plans and capability needs evolve, new areas of overlap and potential duplication can emerge. They expect the database will help them identify which programs are addressing particular capability gaps, which gaps remain open, which gaps are closed, and which gaps are outdated. Knowing if there are multiple systems with requirements addressing the same capability gaps could help identify potential duplication.

The Joint Staff has not yet developed a formal plan to address some of the database’s known limitations. First, the database has been difficult to populate and is potentially missing information because it is dependent on another system that has limited functionality. Second, the Joint Staff has had limited coordination outside of the immediate office developing the database to determine its utility to others, although the Joint Staff and AT&L are funding cross-cutting studies to help refine the tool. Third, the Joint Staff has plans to improve database usability and functionality, but has not developed a formal implementation plan to ensure that it continues to improve and meet the needs of its current and potential users. Developing such plans is a key project management practice. Without establishing this planning foundation, the Joint Staff will not be in a sound position to effectively monitor and evaluate the implementation of its efforts to provide quality information for reviews.

Military Service Portfolio Reviews Offer Lessons for DOD, but Could Be Improved

Most military services have investment planning processes that review weapon system portfolios and their experiences may offer potential lessons for DOD. The Army’s capability portfolio reviews have been held up as a model by DOD and, in the past, were used by the Army to re- examine requirements and realign resources. The Air Force’s investment plans take into account budget uncertainty by examining multiple funding scenarios. The Navy conducts assessments that include portfolio-based analysis to provide an analytical basis for evaluating budget proposals against capability needs. However, these reviews also lacked some of the key acquisition, requirements, and funding information that would better position the services to conduct sound investment planning.

Army Re-examined Requirements in its Initial Reviews to Inform Trade- offs, but Stopped Conducting Regular Portfolio Reviews

consistently conduct portfolio reviews; regularly include a majority of key portfolio review elements in its reviews; and incorporate review guidance into policy.

The Army has conducted portfolio reviews, which could offer lessons for DOD, especially in the area of re-examining capability needs from a portfolio perspective. The Army’s portfolios are generally aligned with the U.S. Army Training and Doctrine Command’s Centers of Excellence, such as aviation, although the portfolio groupings have changed over time and increased in number from nine to eleven. The Secretary of the Army initiated capability portfolio reviews in December 2009. The purpose of the reviews is to, among other things, ensure that funds are programmed, budgeted, and executed against valid requirements and informed alternatives and to revalidate portfolios based on strategy, Combatant Commanders needs, and affordability. The reviews are led by the requirements community and involve the acquisition and budget communities. The process involves multiple working group meetings and then a series of briefings involving increasingly higher level officials. These culminate in a session with Army senior leadership to identify portfolio priorities and other recommendations. A former senior official who spearheaded the reviews stated that the Army originally planned to review each portfolio individually on an annual basis.

Some Army capability portfolio reviews resulted in changes to weapon system investments, including program cancellations, as a result of re- examining requirements and capabilities from a portfolio perspective. For example, a 2010 Precision Fires review, which examined the Army’s balance of high-end precision munitions and lower-end near-precision munitions, led to several changes. The review determined that a proposed precision munitions system—the Non-Line-of-Sight Launch System—would not provide cost-effective capabilities. The Army canceled the program and relied on another system to provide similar capabilities at lower cost. Based on that review, the Army also decided to reduce the number of certain munitions rounds due to cost and other factors. AT&L held up this review as a model in its first iteration of the Better Buying Power initiatives, which stated that enterprise-level and component-level portfolio reviews were to be conducted with an eye toward identifying redundancies and savings.

Although the Army’s capability portfolio reviews have been repeatedly cited as a potential model for DOD, they did not consistently include key requirements, acquisition, and budget portfolio review elements that could have yielded more comprehensive reviews. For example, most of these reviews did not prioritize capability gaps, incorporate information on science and technology needs, or assess investment options for multiple potential funding scenarios. As a result, the reviews did not position the Army, for example, to proactively identify strategies to deal with varying funding scenarios. Results of our analysis of how well the Army’s six portfolio reviews from 2012 incorporated key review elements are included in table 6.

Furthermore, the Army’s more recent portfolio review efforts have not been consistent, in part because of a lack of current guidance. A 2010 memorandum issued by the Secretary of the Army provided direction for the reviews, although it has not been incorporated into Army guidance. The memo was to be valid for 1 year after issuance and called for high level Army officials to make recommendations during that year regarding whether to continue conducting the reviews, incorporate them into other Army requirements and acquisition processes, develop new processes that combine requirements validation with portfolio reviews, or terminate the reviews. The Standard for Internal Control in the Federal Government—Third Edition, notes that agency policies should be clearly documented. Despite the instructions in the memo, Army officials stated that memo has not been updated, nor have the details of the memorandum been incorporated into Army guidance, although the Army has continued to conduct the reviews.

In the past two years, the Army has only conducted reviews for 4 of its 11 portfolios. Some of these more recent reviews were less comprehensive than earlier reviews. For example, both the March 2014 Movement Maneuver and the September 2013 Combating Weapons of Mass Destruction capability portfolio reviews do not include key budget elements, such as funding scenarios and charts showing the relative costs of portfolio components over time. Without such information, review participants cannot readily determine whether programs are affordable in their broader portfolio context and what changes to the portfolio would be warranted given varying potential funding scenarios.

The Army budget community has recently focused efforts on a different review, the Long-Range Investment Requirements Analysis (LIRA). This analysis produces an assessment of planned spending for weapon systems by portfolio over 30 years. The Army conducts the analysis annually to inform the Army’s Program Objective Memorandum, the final product of the programming phase of the PPBE process. The analysis includes funding information spanning key weapon systems’ lifecycles from initial research and development to divestiture. According to Army budget officials, the LIRA process enables the Army to assess levels of risks and analyze if the investment decisions they make now will be affordable in the long term, and determine the resources required for each portfolio. Army requirements officials said that the LIRA is a broader, longer-term examination of the Army’s budget than the portfolio reviews. The Army’s portfolio reviews provide some information to inform the LIRAs although the two are conducted separately.

Air Force Portfolio Reviews Address Most Key Review Elements, but Lack Certain Acquisition Information

The Air Force also conducts portfolio reviews that could offer lessons for DOD due to their comprehensiveness and consideration of multiple risk and funding scenarios. The Air Force began conducting these reviews, called core function support plans, in 2010. The Air Force conducts the reviews for 12 Air Force “core functions.” The purpose of the reviews is to establish the best investment options for capabilities in the near- to long- term. Core function teams, which are led by “core function lead integrators,” perform the reviews. The teams identify and prioritize top- level gaps and requirements within their portfolios and the plans help inform capability development and the planning and programming phases of the PPBE process. The teams may include requirements and budget personnel, but membership varies. The acquisition community is involved when it comes time to integrate the plans. The Air Force has a policy that describes its strategic planning system, but it is dated and does not provide guidance on the core function support planning process. The Air Force is in the process of updating this policy. develop guidance specific to the reviews.

One strength of the Air Force reviews, which was different from the Army and Navy reviews, was their assessment of priorities and needs given multiple funding and risk scenarios. The core function teams assess operational risk using multiple warfighting scenarios. Based on these analyses, they identify needs and priorities given different funding scenarios, which can be a good technique for evaluating trade-offs. In the plans we reviewed, one of these funding scenarios reflected “zero real growth,” a flat budget that accepted whatever risk level would result from that funding level. The other funding level was relatively unconstrained and reflected the funding needed to conduct operations with moderate risk. Air Force core function team officials also said they have used additional funding levels to develop core function support plans, such as zero real growth minus 5 or 10 percent. Based on these scenarios, the team creates prioritized lists of capabilities, and sometimes potential cuts or offsets, to inform budget decisions. The core function support planning process has led to decisions to reduce tactical airlift capacity, minimize facilities modernization and renovation, and increase munitions procurement. An Air Force official responsible for integrating the plans explained that trades have been made both within and across portfolios. In order to reduce risk in one core function it is often necessary to accept additional risk in another.

The Air Force’s core function support plans could potentially serve as a model for DOD, but they lack a few key elements that could contribute to a more integrated review. We found that the Air Force’s reviews included a wide variety of key portfolio review elements, but lacked discussions of potential redundancies and included limited information on the health or status of key acquisition programs. The assessments’ content was also relatively consistent, in part, because the Air Force provides a template for the plans. Results of our analysis of portfolio reviews from 2012 are included in table 7.

Navy Conducts Analytical Assessments to Inform Navy-wide Investment Plans

conduct in-depth reviews for its individual portfolios and include several key portfolio review elements in its reviews.

The Navy does not conduct in-depth reviews for its individual portfolios, but it does conduct annual analytical assessments at the Navy enterprise- level to inform the budget process that could offer lessons for DOD. These analytical assessments, called for under the Navy’s Warfighting Capability, Capacity, and Wholeness policy, evaluate proposed budgets in the context of capability priorities and risks and provide recommendations to leadership to inform resource-allocation decisions. They identify near-, mid-, and long-term capability and capacity gaps and provide quantitative and qualitative information to support trades among Navy programs. The Navy’s analytical arm in the Office of the Chief of Naval Operations leads these assessments and is to integrate information from the requirements and budget communities within the Navy when it conducts them. The acquisition community is generally not involved in these reviews. The process includes four major assessments (see figure 4).

From a Navy-wide perspective, the Navy has used its assessments to balance its available resources against its warfighting needs by making trades to fit its priorities within its overall top line. The assessments we reviewed included recommendations regarding program cancellations, truncations, and deferrals. For example, as a result of a more recent assessment, the Navy decided to end production of the Joint Standoff Weapon. Navy officials said that its analytical arm determined that the Navy could truncate production because it had produced enough of the weapon and had another system in development that could provide the capability in the future. The Navy’s assessments also included information on capability gaps and the timeline of systems to meet those gaps by portfolio or capability area. This information is used to develop the front-end assessment, which includes areas where the Navy should invest additional resources to address priority gaps. According to Navy officials, the front-end assessment is one of the most important products of the assessment because it identifies areas of focus for the requirements community and may identify areas of overlap or duplication. The officials said that they can identify when more than one requirements office has proposed a solution for the same capability gap.

While the Navy’s assessments are intended to provide an analytical basis for evaluating budget proposals against capability needs; they do not include an in-depth review of each portfolio and do not include key information from the requirements, acquisition, and budget communities. We assessed the Navy’s Warfighting Capability, Capacity, and Wholeness assessments conducted to develop its fiscal year 2014 budget request and found that the assessments reflected some key portfolio review elements (see table 8). However, the assessments also did not include a number of other key portfolio review elements and, for some elements, the information was not portfolio-based. By not including such information, the Navy is missing an opportunity to focus on portfolio- specific challenges, such as the trade-offs that might be needed to address the affordability of its shipbuilding portfolio.

Conclusions

DOD operates as a joint force on the battlefield, yet still does not use an integrated approach to optimizing its weapon system investments. The military services develop investment plans that reflect their own priorities. Enterprise-level reviews of these plans and acquisition programs are stove-piped within the requirements, acquisition, and budgeting communities. Further, oversight processes largely focus on individual programs. As a result, DOD continues to craft investment plans that reflect military service preferences and are not affordable over the long- term and default to optimizing for and addressing problems in individual programs instead of focusing on portfolios of programs that might provide greater military capability at lower risk or cost.

Addressing these challenges through strategic investment planning and portfolio management is important today given the constrained budget environment, quickly evolving threats, and uncertainty about the types of conflicts the United States may face in the future. But DOD currently lacks the governance structure, policy, and decision-making authorities to implement the more strategic and integrated approach to weapon system investment embodied in portfolio management best practices. Completely breaking down the stove pipes between the requirements, acquisition, and budget communities; standardizing portfolio constructs; and empowering enterprise-level portfolio managers to drive investment decisions would require a fundamental restructuring of how DOD does business—this is unrealistic in the short run and would be challenging even in the long run. Nevertheless, DOD can lay the foundation for better portfolio management by taking some incremental steps that have the potential to improve the efficiency and effectiveness of its weapon system investment planning and management processes.

DOD has an opportunity to make meaningful changes to its portfolio management efforts if senior leadership makes a commitment and invests the resources to do so. Several elements stand out as candidates for immediate action. First, DOD’s current capability portfolio management policy is dated, senior leadership commitment has waned, and, as a result, the policy is not being implemented. Reforming this policy to better reflect best practices is essential to bridging the stovepipes in DOD’s requirements, acquisition, and budget processes and moving DOD toward a more integrated approach to planning and managing its weapon system investments. Senior leadership with direct line management authority over the organizations responsible for implementing any policy will also be needed if the outcomes from any renewed portfolio management effort will be different from those in the past. Next, if DOD is to move toward a weapon system investment plan that is affordable, strategy-driven, and balanced between near-term and long-term needs, then portfolio managers will need better data and analytical tools to assess threats and capability needs and make trade-offs between groups of acquisition programs that can address them. Finally, as DOD has recognized in its Better Buying Power initiatives, the military services offer potential lessons for DOD’s portfolio management efforts. The military services have used portfolio reviews and other assessments to reduce redundancies, realign resources, produce more analytically rigorous budget proposals, and plan for budget uncertainty. However, even these efforts can be improved by making sure they reflect key portfolio review elements from across the requirements, acquisition, and budget communities.

Recommendations for Executive Action

To improve DOD’s use of portfolio management for its weapon system investments and ensure that its investment plans are affordable, strategy- driven, balance near- and long-term needs, and leverage efforts across the military services, we are making two recommendations for executive action.

First, to provide a solid foundation for future portfolio management efforts at the enterprise-level, we recommend that the Secretary of Defense revise DOD Directive 7045.2 on Capability Portfolio Management in accordance with best practices and promote the development of better tools to enable more integrated portfolio reviews and analyses of weapon system investments. Key elements of this recommendation would include designating the Deputy Secretary of Defense or some appropriate delegate responsibility for implementing the policy and overseeing portfolio management in DOD; requiring annual enterprise-level portfolio reviews that incorporate key portfolio review elements, including information from the requirements, acquisition, and budget processes; directing the Joint Staff, AT&L, and CAPE to collaborate on their data needs and develop a formal implementation plan for meeting those needs either by building on the database the Joint Staff is developing for its analysis or investing in new analytical tools; and incorporating lessons learned from military service portfolio reviews and portfolio management activities, such as using multiple risk and funding scenarios to assess needs and re-evaluate priorities.

Second, to help ensure the military services’ portfolio reviews are conducted regularly and effectively integrate information from the requirements, acquisition, and budget communities, the Secretary of Defense should direct the Secretaries of the Army, Navy, and Air Force to update or develop policies that require them to conduct annual portfolio reviews that incorporate key portfolio review elements, including information from the requirements, acquisition, and budget processes.

Agency Comments and Our Evaluation

We provided a draft of this report to DOD for comment. In its written comments, which are reprinted in full in appendix IV, DOD partially concurred with our two recommendations. However, as discussed below, DOD’s planned actions will not fully address the issues we raised in this report.

DOD partially concurred with our first recommendation to revise DOD Directive 7045.20 on Capability Portfolio Management in accordance with best practices and promote the development of better tools to enable more integrated portfolio reviews and analyses of weapon system investments. In its response, DOD agreed with the need to further develop portfolio management tools, ensure access to authoritative data, and incorporate lessons learned by others performing portfolio management. However, DOD stated that other aspects of our recommendation were redundant to, and would conflict with other processes and activities in place to perform portfolio management. DOD’s response presumes that its existing portfolio management processes and activities have been effective and fails to acknowledge that they have not successfully addressed weapon system investment challenges, such as affordability issues in certain portfolios and potential duplication among some programs. We disagree that existing processes and activities have been effective. Further, we continue to believe that our recommendation to revamp DOD’s portfolio management policy to align with best practices, establish clear responsibility for its implementation, and conduct annual portfolio reviews would provide the foundation for improved weapon system investment planning and management. DOD’s specific comments on portfolio management policy, leadership, and annual portfolio reviews follow.

Policy: DOD does not plan to revise its capability portfolio management policy in accordance with best practices, as we recommended, but rather stated the policy should be rescinded and stakeholders directed to participate in portfolio management through its requirements, acquisition, and budget processes. In addition, DOD stated that recent revisions to its requirements and acquisition policies vastly improve the integration of these processes. We continue to believe that having an enterprise-level portfolio management policy that is consistent with best practices is necessary to provide a solid foundation for future portfolio management efforts. Rescinding, rather than revising, its enterprise-level portfolio management policy and relying on its requirements, acquisition, and budgeting process to fill- in the policy gaps could reinforce the stove-piped governance structure that we found to be an impediment to integrated portfolio management.

Sustained leadership: DOD does not plan to designate the Deputy Secretary of Defense or an appropriate delegate responsibility for overseeing portfolio management, as we recommended. DOD responded that the Deputy Secretary of Defense already makes portfolio management decisions in the context of DOD’s budget process and that, as long as these decisions are informed by the requirements and acquisition communities, they are sufficient to adjust portfolios to meet the military’s needs. We continue to believe that sustained enterprise-level leadership for implementing portfolio management processes is needed to support sound portfolio management decisions by the Deputy Secretary of Defense or others. Our review found that key DOD portfolio management-related efforts, including some designed to inform the budget process and some called for in guidance, have lacked sustained leadership and been inconsistently implemented. If the Deputy Secretary of Defense or an appropriate delegate assumed leadership for portfolio management, it could help ensure portfolio management policy as well as other portfolio management efforts are more consistently implemented throughout the department.

Annual portfolio reviews: DOD does not plan to require annual enterprise-level portfolio reviews that integrate key portfolio review elements from the requirements, acquisition, and budget processes, as we recommended. DOD responded that portfolio reviews would be redundant to annual reviews already conducted through existing departmental processes. For example, DOD responded that the annual Capability Gap Assessment process called for under the requirements process serves as an annual portfolio review and that AT&L leads reviews of long-term science and technology plans to help frame future capability portfolios. DOD also stated that it uses Capability Gap Assessments during the budget process to ensure that military service budget submissions are consistent with the warfighters’ capability needs. We continue to believe that annual portfolio reviews that incorporate key portfolio review elements would provide DOD a more integrated and comprehensive perspective on its weapon system investment plans than its current processes provide. The processes DOD described in its response largely reflect its stove- piped governance structures where the responsibilities for aspects of portfolio management are dispersed among the requirements, acquisition, and budget communities. Further, our review found that portfolio reviews were inconsistently conducted, did not address all portfolios, or only involved parts of the organization. As a result, we continue to believe that updating DOD’s portfolio management policy to require annual integrated enterprise-level portfolio reviews is the best way to ensure DOD has the information it needs to craft weapon system investment plans that are affordable, balance near- and long- term needs, and maximize return on investment.

DOD also partially concurred with our second recommendation to update or develop policies that require the military services to conduct their own annual portfolio reviews that incorporate key portfolio review elements from the requirements, acquisition, and budget processes. However, DOD did not indicate that it would take any action to address it. Instead, DOD responded that the services’ budget processes and Office of the Secretary of Defense’s review of the services’ budgets meet the intent of our recommendation. We disagree. Our review found that military service level portfolio reviews, including Air Force and Navy reviews closely linked to their budget processes, did not always contain key portfolio review elements or were not consistently conducted. We continue to believe that updating or developing policies to require the services to conduct annual portfolio reviews incorporating information from the requirements, acquisition, and budget processes would help ensure the services conduct regular, integrated reviews, which in turn should yield benefits, such as prioritizing the most important investments and selecting the optimum investment mix.

We are sending copies of this report to the appropriate congressional committees; the Secretary of Defense; the Secretaries of the Army, Navy, and Air Force; the Commandant of the Marine Corps; and other interested parties. This report will also be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please contact me at (202) 512-4841 or by e-mail at sullivanm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to assess the extent to which (1) the Department of Defense (DOD) uses portfolio management to optimize its weapon system investments, (2) DOD conducts integrated portfolio reviews at the enterprise level to optimize their weapon system investments, and (3) the military services conduct integrated portfolio reviews to optimize their weapon system investments and their experiences offer potential lessons for DOD.

To address our first objective, we reviewed commercial portfolio management best practices from GAO-07-388 Best Practices: An Integrated Portfolio Management Approach to Weapon System Investments Could Improve DOD’s Acquisition Outcomes and reviewed portfolio management-related literature from peer reviewed journals, books, and trade publications to determine which, if any, additional practices should be added. Based on this review, we determined that no changes to the previously identified practices were needed. We compared these best practices with DOD’s weapon system investment processes, which include the requirements, acquisition, and budget processes. We analyzed relevant DOD policies and statutes, including: DOD’s portfolio management policy, DOD Directive 7045.20 Capability Portfolio Management; requirements policies, including Chairman of the Joint Chiefs of Staff Instruction 3170.01I Joint Capabilities Integration and Development System and the Manual for the Operation of the Joint Capabilities Integration and Development System; acquisition policies, including DOD Directive 5000.01 The Defense Acquisition System and DOD Instruction 5000.02 Operation of the Defense Acquisition System; the budget policy, DOD Directive 7045.14 The Planning, Programming, Budgeting, and Execution (PPBE) Process; and relevant sections of title 10 of the United States Code. We reviewed relevant prior GAO reports and DOD memoranda, including the November 2014 memorandum from the Deputy Secretary of Defense on the support for strategic analysis process and the July 2014 memorandum from the Vice Chairman of the Joint Chiefs of Staff on the support for strategic analysis process. We also compared DOD’s capability portfolio management policy with portfolio management best practices to evaluate the extent to which it reflects those practices. To describe previous capability portfolio management efforts in the mid-to-late 2000s and the extent they are being implemented today, we reviewed relevant documents, including a September 2006 memorandum from the Deputy Secretary of Defense on capability portfolio management test cases and military service requirements, acquisition, and budget policies. Based on our assessment of these policies and additional documentation, we also identified potential impediments to DOD’s use of portfolio management to optimize investments across the department. To inform all assessments for this objective, we interviewed officials from DOD enterprise- and military service-level requirements, acquisition, and budget communities, including:

Requirements: Force Structure, Resource and Assessment Directorate, J-8, and Directorate for Intelligence, J-2 for the enterprise level; Army Capabilities Integration Center and Headquarters, Department of the Army G-3/5/7 Capability Portfolio Review and Integration; Directorate of Resources; Air Force Office of the Director of Operational Capability Requirements, A5R; the Air Combat Command; Office of the Chief of Naval Operations, N803, Joint & Urgent Requirements; Office of the Chief of Naval Operations, N96, Surface Warfare; and Marine Corps, Office of the Deputy Commandant, Combat Development and Integration.

Acquisition: Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (AT&L) for the enterprise level; Office of the Assistant Secretary of the Army for Acquisition, Logistics, and Technology; Office of the Assistant Secretary of the Air Force for Acquisition; Office of the Assistant Secretary of the Navy for Research, Development, and Acquisition; and Marine Corps Systems Command.

Budget: Office of the Secretary of Defense, Cost Assessment and Program Evaluation (CAPE) and Office of the Under Secretary of Defense Comptroller for the enterprise level; Directorate of Resources, Army G-8, Force Development; Air Force Division Chief, Planning, Programming, and Requirements Division, Air Combat Command; the Office of the Chief of Naval Operations, N81, Integration and Force Structure and the Office of the Chief of Naval Operations, Fiscal Management, N82 and Office of the Assistant Secretary of the Navy for Financial Management and Comptroller; and Marine Corps Office of Programs and Resources.

To address our second objective, we reviewed relevant guidance and documents to identify enterprise-level portfolio reviews and other portfolio-level analyses that DOD components conduct and interviewed DOD’s enterprise-level requirements, acquisition, and budget communities. We analyzed standards in the Project Management Institute’s The Standard for Portfolio Management—Third Edition and GAO best practices and compared them with DOD policies. Based on this analysis, we developed a list of key portfolio elements which, if included in portfolio reviews, could help integrate key portfolio management information from the requirements, acquisition, and budget communities—information that is needed for an integrated approach to prioritizing needs and allocating resources in accordance with strategic goals. We compared the Joint Staff Functional Capability Board portfolio reviews’ content with these key portfolio review elements. To assess whether DOD has conducted affordability analyses in accordance with DOD Instruction 5000.02 and DOD’s Better Buying Power guidance, we reviewed six Defense Acquisition Board briefings for programs that held an acquisition or milestone review from January to June 2014. To assess whether DOD has conducted portfolio reviews and other portfolio-based analyses to identify and eliminate potential redundancies in accordance with DOD’s Better Buying Power guidance, we interviewed Joint Staff, AT&L, and military service officials, and reviewed relevant guidance. We also interviewed Joint Staff, AT&L, and CAPE officials to determine the extent to which the three offices have collaborated on these portfolio reviews and portfolio-based analyses, and to determine whether they had the necessary data, resources, and analytical tools to effectively conduct reviews.

To address our third objective, we reviewed Army, Air Force, and Navy requirements, acquisition, and budget policies to determine the extent to which they required portfolio reviews. We analyzed Army and Air Force portfolio reviews, and the Navy’s analytical assessments by comparing their contents with key portfolio review elements. The military service assessments we evaluated include:

Army: the six capability portfolio reviews the Army conducted in 2012, including Combat Vehicles; Precision Fires; Sustainment (Transportation); Aviation; Assured Mobility and Protection; and Mission Command; and the two most recent unclassified reviews provided by the Army, including Combating Weapons of Mass Destruction from 2013 and Movement and Maneuver from 2014;

Air Force: of its 12 core function support plans from 2012 that informed the Air Force’s fiscal year 2015 president’s budget request, we reviewed the nine that had weapon system centric portfolios, including Global Precision Attack; Space Superiority; Air Superiority; Global Integrated Intelligence, Surveillance, and Reconnaissance; Command and Control; Rapid Global Mobility; Nuclear Deterrence Operations; Personnel Recovery; and Special Operations;

Navy: four assessments that were part of the Navy’s Warfighting Capability, Capacity, and Wholeness process to inform the Navy’s Program Objective Memorandum for fiscal year 2014, including Capability Assessments; Front-end Assessments; Warfighting Support and Capability Plan; and Integrated Program Assessment.

We also interviewed officials from all four military services’ requirements, acquisition, and budget communities to determine how they conduct these assessments, the role of the assessments in their weapon system investment decision-making, and the results of the assessments. Based on our assessment of these policies, reviews, and interviews with service officials, we identified the strengths and weaknesses of these reviews and the potential lessons they may offer DOD.

Appendix II: Key DOD Enterprise- and Military Service-Level Policies Governing the Requirements, Acquisition, and Budget Processes for Weapon System Investments

Enterprise-Level Policies

Department of Defense, Manual for the Operation of the Joint Capabilities Integration and Development System (Feb. 12, 2015).

Chairman of the Joint Chiefs of Staff Instruction 3170.01I, Joint Capabilities Integration and Development System (JCIDS) (Jan. 23, 2015).

Chairman of the Joint Chiefs of Staff Instruction 5123.01G, Charter of the Joint Requirements Oversight Council (JROC) (Feb. 12, 2015).

Department of Defense Directive 5000.01, The Defense Acquisition System (May 12, 2003).

Department of Defense Instruction 5000.02, Operation of the Defense Acquisition System (Jan. 7, 2015).

Department of Defense Directive 7045.14, The Planning, Programming, Budgeting, and Execution (PPBE) Process (Jan. 25, 2013).

Chairman of the Joint Chiefs of Staff Instruction 8501.01B, Chairman of the Joint Chiefs of Staff, Combatant Commanders, Chief, National Guard Bureau, and Joint Staff Participation in the Planning, Programming, Budgeting and Execution Process (Aug. 21, 2012).

Military Service-Level Policies

Army

Army Regulation 1-1, Planning, Programming, Budgeting and Execution System (Jan. 30, 1994).

Army Training and Doctrine Command Regulation 71-20, Concept Development, Capabilities Determination, and Capabilities Integration (Jun. 28, 2013).

Army Regulation 71-9, Warfighting Capabilities Determination (Dec. 28, 2009).

Army Regulation 70-1, Army Acquisition Policy (Jul. 22, 2011).

Department of the Army Pamphlet 70-3, Army Acquisition Procedures (Apr. 1, 2009).

Air Force

Air Force Policy Directive 10-6, Capability Requirements Development (Nov. 6, 2013).

Air Force Instruction 10-601, Operational Capability Requirements Development (Nov. 6, 2013).

Air Force Policy Directive 63-1, Integrated Life Cycle Management (Jul. 3, 2012).

Air Force Instruction 63-101/20-101, Integrated Life Cycle Management (Mar. 7, 2013).

Air Force Policy Directive 65-6, Financial Management: Budget (May 1, 1998).

Air Force Policy Directive 90-11, Strategic Planning System (Mar. 26, 2009).

Navy

Secretary of the Navy Manual M-5000.2, Department of the Navy: Acquisition and Capabilities Guidebook (May 9, 2012).

Secretary of the Navy Instruction 5000.2E, Department of the Navy Implementation and Operation of the Defense Acquisition System and the Joint Capabilities Integration and Development System (Sept. 1, 2011).

Chief of Naval Operations Instruction 3050.25, Warfighting Capability, Capacity, and Wholeness Assessments (Jul. 30, 2012).

Appendix III: Implementation of Portfolio- Based Analyses to Support Individual Acquisition Program Decisions

In addition to its portfolio review efforts, the Department of Defense (DOD) has called for other portfolio-based analyses to support acquisition decisions for individual acquisition programs. These analyses are to review the program in the context of its corresponding portfolio. The Office of the Under Secretary of Defense for Acquisition, Technology and Logistics (AT&L) called for these portfolio-based analyses to ensure the affordability of its weapon systems programs and to eliminate programs that are potentially duplicative of other systems—two areas where we have identified challenges. However, the Assistant Secretary of Defense for Acquisition said that AT&L has not implemented the latter of these efforts because it lacks readily accessible data, resources, and tools to do so.

AT&L requires the military services and other DOD components to conduct portfolio-based affordability analyses under Better Buying Power and in its acquisition instruction; however, AT&L has not reviewed the quality of the analyses or used them to inform acquisition decisions. We also found that these efforts are only partially being implemented (see table 9). Better Buying Power guidance from 2010 and 2012 recommends portfolio-based affordability analyses to inform acquisition program milestone decisions. Subsequently, AT&L required affordability analysis for acquisition programs under its primary acquisition policy—DOD Instruction 5000.02—as a tool to promote responsible and sustainable investment decisions. This policy states that the purpose of the affordability analysis is to avoid starting or continuing programs that DOD cannot produce and support within reasonable expectations for future budgets. The military services are to consider the long-term affordability of a program within the context of its portfolio to inform the program’s milestone decisions. The portfolios to be used for these analyses were provided by the services and are generally different than the portfolios used for requirements and budget purposes at the enterprise-wide level. AT&L officials said they have not monitored the military services’ compliance with conducting the analyses or reviewed the quality of the analyses because the responsibility for conducting them falls to service leadership.

In our review of defense acquisition board briefings from January through June 2014, we found that 5 of 6 programs that held reviews included an affordability analysis. However, two of these analyses did not look at affordability over the recommended 30- to 40-year period, and for the three in which the portfolio exceeded the total obligation authority in any given year, there was no discussion of possible trades to make the portfolio affordable. According to the Assistant Secretary of Defense for Acquisition, statute charges the military services with equipping the force, so it is the services that have to make trade-offs. It is not an AT&L role.

Better Buying Power guidance from 2012 also called for portfolio-based analyses to identify potential redundancy and eliminate unnecessary duplication; however these analyses have not been conducted (see table 10). AT&L was to conduct these analyses for ACAT I programs in support of all acquisition program decisions, including at requirements decision points, and during annual program and budget review processes. The military services were to conduct these analyses for lower dollar acquisition programs. The most recent version of the Better Buying Power guidance still contains the goal of eliminating redundancy within warfighter portfolios, but it is no longer an area of emphasis.

Appendix IV: Comments from the Department of Defense

Appendix V: GAO Contact and Staff Acknowledgments

GAO Contact

Michael J. Sullivan, (202) 512-4841 or sullivanm@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Ron Schwenn, Assistant Director; Laura Holliday, Analyst-in-Charge; Jenny Chanley; Dani Greene; Kristine Hassinger; Sean Manzano; Carol Petersen; and Erin Preston made key contributions to this report.