United States Government Accountability Office

GAO Report to the Committee on Armed 

Services, U.S. Senate 

BEST PRACTICES

An Integrated 

Portfolio Management 

Approach to Weapon 

System Investments 

Could Improve DOD’s 

Acquisition Outcomes 

March 2007 

 

GAO-07-388 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

March 2007

BEST PRACTICES

An Integrated Portfolio Management 

Approach to Weapon System 

Investments Could Improve DOD’s 

Acquisition Outcomes 

Highlights of GAO-07-388, a report to the 

Committee on Armed Services, U.S. 

Senate 

Over the next several years, the

Department of Defense (DOD) 

plans to invest $1.4 trillion in major 

weapons programs. While DOD 

produces superior weapons, GAO 

has found that the department has 

failed to deliver weapon systems 

on time, within budget, and with

desired capabilities. While recent 

changes to DOD’s acquisition 

policy held the potential to improve

outcomes, programs continue to 

experience significant cost and 

schedule overruns. 

GAO was asked to examine how 

DOD’s processes for determining 

needs and allocating resources can 

better support weapon system 

program stability. Specifically, 

GAO compared DOD’s processes 

for investing in weapon systems to 

the best practices that successful 

commercial companies use to

achieve a balanced mix of new 

products, and identified areas 

where DOD can do better. In 

conducting its work, GAO 

identified the best practices of: 

Caterpillar, Eli Lilly, IBM, Motorola, 

and Procter and Gamble.

What GAO Recommends

GAO is making several 

recommendations for DOD to 

implement an integrated portfolio 

management approach to weapon

system investments. DOD stated 

that it is undertaking several pilot 

efforts to improve the department’s

approach and that implementation

of any new business rules will be

contingent upon the outcomes of 

these efforts. 

To achieve a balanced mix of executable development programs and ensure 

a good return on their investments, the successful commercial companies 

GAO reviewed take an integrated, portfolio management approach to 

product development. Through this approach, companies assess product 

investments collectively from an enterprise level, rather than as independent 

and unrelated initiatives. They weigh the relative costs, benefits, and risks of 

proposed products using established criteria and methods, and select those 

products that can exploit promising market opportunities within resource 

constraints and move the company toward meeting its strategic goals and 

objectives. Investment decisions are frequently revisited, and if a product 

falls short of expectations, companies make tough go/no-go decisions. The 

companies GAO reviewed have found that effective portfolio management 

requires a governance structure with committed leadership, clearly aligned 

roles and responsibilities, portfolio managers who are empowered to make 

investment decisions, and accountability at all levels of the organization. 

In contrast, DOD approves proposed programs with much less consideration 

of its overall portfolio and commits to them earlier and with less knowledge 

of cost and feasibility. Although the military services fight together on the 

battlefield as a joint force, they identify needs and allocate resources

separately, using fragmented decision-making processes that do not allow 

for an integrated, portfolio management approach like that used by 

successful commercial companies. Consequently, DOD has less assurance 

that its investment decisions address the right mix of warfighting needs, and, 

as seen in the figure below, it starts more programs than current and likely 

future resources can support, a practice that has created a fiscal bow wave.

If this trend goes unchecked, Congress will be faced with a difficult choice: 

pull dollars from other high-priority federal programs to fund DOD’s 

acquisitions or accept gaps in warfighting capabilities. 

Figure: Costs Remaining Versus Annual Appropriations for Major Defense Acquisitions 

Dollars (in billions)

Fiscal year

0

200

400

600

800

1,000

1,200

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Costs remaining for major defense acquisitions

Annual appropriations for major defense acquisitions

Source: DOD (data); GAO (analysis and presentation).

www.gao.gov/cgi-bin/getrpt?GAO-07-388.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Michael J. 

Sullivan at (202) 512-4841 or 

sullivanm@gao.gov. 
Contents 

Letter 1

Results in Brief 3 

Background 5

Successful Companies Take a Disciplined, Integrated Approach to 

Prioritize Market Needs and Initiate a Balanced Mix of 

Executable Development Programs 7 

Lacking an Integrated, Portfolio-Based Approach, DOD Has Too 

Many Programs Competing for Limited Resources 18 

Conclusions 32

Recommendations 32

Agency Comments and Our Evaluation 33 

Appendix I Objectives, Scope, and Methodology 35 

Appendix II Comments from the Department of Defense 38 

Related GAO Products 42 

Table 

Table 1: Cost and Cycle Time Growth for 27 Weapon Systems 5 

Figures 

Figure 1: DOD’s Weapon System Investment Process 7 

Figure 2: Portfolio Management Approach to Product Investments 9 

Figure 3: IBM Market Segmentation 11 

Figure 4: Risk Versus Rewards Matrix 16 

Figure 5: Service Allocations of DOD’s Investment Budget 22 

Figure 6: Governance of DOD’s Investment Process 24 

Figure 7: Costs Remaining Versus Annual Appropriations for Major 

Defense Acquisitions 26 

Page i GAO-07-388 Best Practices
Abbreviations 

ACAT Acquisition Category 

AOA Analysis of Alternatives 

AT&L Acquisition, Technology, and Logistics 

DAS Defense Acquisition System 

DOD Department of Defense 

FY fiscal year 

GAO U.S. Government Accountability Office 

IBM International Business Machines 

ICD Initial Capabilities Document 

IPD Integrated Product Development 

JCIDS Joint Capabilities Integration and Development System 

JCS Joint Chiefs of Staff 

JROC Joint Requirements Oversight Council 

JTRS Joint Tactical Radio System 

MS Milestone 

NPV Net Present Value 

OSD Office of the Secretary of Defense 

PA&E Program Analysis and Evaluation 

PPBE Planning, Programming, Budgeting, and Execution

RDT&E research, development, test, and evaluation 

USD Under Secretary of Defense 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page ii GAO-07-388 Best Practices
United States Government Accountability Office

Washington, DC 20548 

March 30, 2007 

The Honorable Carl Levin 

Chairman 

The Honorable John McCain 

Ranking Member 

Committee on Armed Services 

United States Senate 

Although the Department of Defense (DOD) produces the best weapons in 

the world, it has not been able to deliver planned systems on time and 

within budget. It is not unusual to see cost increases that add up to tens or 

hundreds of millions of dollars, schedule delays that add up to years, and 

large and expensive programs being scrapped after years of failing to 

achieve promised capabilities.1

 While recent changes to DOD’s acquisition

policy held the potential to improve such outcomes, programs have 

continued to experience significant cost and schedule overruns and 

performance shortfalls.2

 Over the next several years, DOD plans to invest 

$1.4 trillion in major weapon system programs—doubling what it planned 

to spend on such programs 5 years ago. Continued failure to deliver 

weapon systems on time and within budget not only delays providing 

critical capabilities to the warfighter, but results in less funding being 

available for other DOD and federal priorities. 

In the commercial market, effectively developing and marketing new 

products is fundamental to the continued growth and success of 

companies. Without a steady stream of product innovations to meet 

evolving market needs, companies are likely to see their sales and profits 

fall. At the same time, if the products in development outstrip their 

resources or do not meet customer needs, companies can face financial 

ruin. Several recent studies issued by leading thinkers in the area of 

product innovation and development have reported that leading 

commercial companies achieve success in product development by 

 1

GAO, Defense Acquisitions: Actions Needed to Get Better Results on Weapons Systems 

Investments, GAO-06-585T (Washington, D.C.: Apr. 5, 2006).

2

GAO, Defense Acquisitions: Major Weapon Systems Continue to Experience Cost and

Schedule Problems under DOD’s Revised Policy, GAO-06-368 (Washington, D.C.: Apr. 13, 

2006). 

Page 1 GAO-07-388 Best Practices
following a disciplined process for ensuring they have the right mix of new 

products that meet customer needs within available resources. 

In fiscal year 2006, the Senate Armed Services Committee raised concerns

that DOD’s poor track record with acquisition programs was linked not 

only to the department’s Defense Acquisition System (DAS) for managing 

product development, but also to the department’s Joint Capabilities 

Integration and Development System (JCIDS) for identifying the 

warfighters’ needs and the Planning, Programming, Budgeting and 

Execution (PPBE) process for allocating resources. Consequently, the 

Committee directed GAO to examine how DOD’s needs identification and 

resource allocation processes can better support program stability in 

major weapon systems acquisition.3

 This report (1) identifies best 

practices of successful commercial companies for ensuring that they 

pursue the right mix of programs to meet the needs of their customers 

within resource constraints and (2) compares DOD’s enterprise-level

processes for investing in weapon systems to these practices.4

To identify best practices of successful companies, we reviewed related

professional and academic publications, and interviewed knowledgeable 

officials from five successful commercial companies: Caterpillar, Eli Lilly, 

IBM, Motorola, and Procter & Gamble. While the products developed by

these companies range from heavy construction equipment and high-end 

electronics to pharmaceuticals and household items, each of the 

companies manages a large diversified portfolio of products, spends 

billions of dollars annually on research and development, and has 

thousands of employees worldwide. To examine DOD’s processes for 

making investment decisions, we reviewed related legislation and DOD 

directives, instructions, and guidance; conducted interviews with and 

received briefings from relevant Joint Staff, Office of the Secretary of 

Defense (Comptroller, Program Analysis and Evaluation, and Acquisition, 

Technology and Logistics), and other government officials; reviewed 

current literature assessing DOD’s decision-making processes; and 

analyzed DOD requirements documents. We compared DOD’s enterpriselevel practices to commercial best practices to identify potential areas for

 3

S.Rep.No. 109-69 at 343-346 (2005). 

4

The Senate mandate specifically asked GAO to review the requirements and budgeting 

portions of DOD weapon system acquisition. Therefore, we do not focus on the DAS in this

report. GAO has, however, produced a body of best practices work focusing on the DAS; 

many of those reports are listed in the related GAO products section of this report. 

Page 2 GAO-07-388 Best Practices
improvement. For additional details on how we performed our review, see 

appendix I. Our work was conducted between March 2006 and February

2007 in accordance with generally accepted government auditing 

standards. 

Successful commercial companies use an integrated portfolio 

management approach to prioritize market needs and allocate resources; 

thus, they avoid pursuing more products than their resources can support 

and optimize the return on their investment. Through portfolio 

management, all of a company’s product investments are addressed 

collectively from an enterprise level, rather than as independent and 

unrelated initiatives. Potential product developments are identified and 

assessed through a systematic and disciplined screening process. 

Companies weigh the relative costs, benefits, and risks of each proposed

product using established criteria and methods to select the best mix of 

products to develop. They not only select those products that have a 

sound business case to warrant further investment, but also those that 

help the company balance near- and future-term market opportunities, 

different product lines, and available resources against the demand for 

product investments. Once initial investment decisions are made, they are 

revisited at multiple stages throughout product development in a gated 

review and assessment process to ensure products are still of high value. 

If not, companies make tough decisions to defer or terminate investments 

and rebalance their product portfolios. To be effective, portfolio 

management is enabled by strong governance with committed leadership, 

clearly aligned organizational roles and responsibilities, empowered 

portfolio managers who determine the best way to invest resources, and 

accountability at all levels of the organization. 

Results in Brief 

Although the military services fight together on the battlefield as a joint 

force, they do not identify warfighting needs and make weapon system 

investment decisions together in an integrated manner. DOD has taken 

steps to identify warfighting needs through a joint requirements process, 

but its service-centric structure and fragmented decision-making 

processes do not allow for the portfolio management approach used by 

successful commercial companies to make investment decisions that 

benefit the organization as a whole. DOD largely continues to define 

warfighting needs and make investment decisions on a service-by-service

basis, an approach that has contributed to duplication in programs and 

equipment that does not operate effectively together. Also, DOD assesses 

warfighting needs and their funding implications under separate decisionmaking processes, impeding its ability to prioritize warfighting needs so 

Page 3 GAO-07-388 Best Practices
that it pursues not only the ones that are most important but also the ones 

it can afford. While DOD’s JCIDS process provides a framework for 

reviewing and validating the initial need for proposed capabilities, it does 

not focus on the cost and feasibility of acquiring the capability to be 

developed and fielded. Instead, these considerations are addressed 

through separate budgeting and acquisition processes. Moreover, although 

DOD policy provides for a series of early reviews—focused on the concept 

refinement and technology development phases of proposed weapon 

system programs—in prior work we found that the reviews are often 

skipped or are not fully implemented. Consequently, proposed programs 

build momentum and move toward starting product development with 

little if any early department-level assessment of the costs and feasibility. 

Committing to programs before they have this knowledge contributes to 

poor cost, schedule, and performance outcomes and destabilizes 

acquisition programs as the department attempts to pay for poorly 

performing programs by taking funds from others. 

The department has begun to pilot-test several interrelated initiatives 

intended to address shortfalls in its existing approach to investment 

decisions. These initiatives include a new approach to an early decision 

gate for reviewing proposed programs at the concept stage, testing 

portfolio management approaches in selected capability areas, and setting 

up capital budgeting accounts for programs in development. However, as 

currently structured, the initiatives are intended to operate within DOD’s 

existing organizational and process framework and may not allow for 

sufficient authority and control over resources to effectively influence 

weapon system investments. 

To improve DOD’s ability to deliver a balanced mix of weapon system 

programs at the right time and right cost, we are recommending the 

department establish an integrated, portfolio-based approach to

investments that incorporates best practices of successful commercial 

companies. To ensure the success of such an approach, we are also 

recommending that DOD establish a single point of accountability at the 

department level with the responsibility, authority, and accountability for 

ensuring that portfolio management for weapon system investments is 

effectively implemented across the department. DOD concurred with the 

majority of our recommendations and partially concurred with two. 

Generally, in responding to these recommendations, DOD stated that it is 

undertaking several initiatives and pilot efforts to improve the 

department’s approach to investment and program decision making, and

that implementation of any new business rules will be contingent upon the 

outcome of these initiatives. However, we believe the department’s 

Page 4 GAO-07-388 Best Practices
current initiatives do not fundamentally change DOD’s service-centric 

framework or sufficiently integrate its decision-making processes. DOD 

did not provide comments regarding our recommendation that the 

Secretary establish a single point of accountability. 

DOD’s programs for acquiring major weapon systems have taken longer, 

cost more, and delivered fewer quantities and capabilities than planned. 

We have documented these problems for decades. Most recently, we 

reported that 27 major weapon programs we have assessed since they 

began product development have experienced cost increases of nearly 

34 percent over their original research, development, test, and evaluation

(RDT&E) estimates, and increases of almost 24 percent in acquisition 

cycle time (see table 1).5

Background 

Table 1: Cost and Cycle Time Growth for 27 Weapon Systems

Billions of constant 2007 dollars 

First full estimate Latest full estimate Percentage change

Total cost $506.4 $603.1 19.1

RDT&E cost $104.7 $139.7 33.5

Weighted average acquisition cycle timea137.9 months 170.2 months 23.5

Source: GAO analysis of DOD data. 

a

This is a weighted estimate of average acquisition cycle time for the 27 programs based on total 

program costs at the first full and latest estimates. The simple average for these two estimates was 

98.9 months for the first full estimate and 124.6 months for the latest estimate resulting in a 

26.1 percent change. 

When cost and schedule problems occur in one program, DOD often 

attempts to pay for the poorly performing program by taking funds from 

others. Doing so has destabilized other programs and reduced the overall 

buying power of the defense dollar as DOD and the military services are 

forced to cut back on planned quantities or capabilities to stay within 

budget limitations. The F-22A Raptor program is a case in point: As costs 

escalated in the program, the number of aircraft the Air Force planned to

buy was drastically reduced from 648 to 183. Similarly, as the Joint 

Tactical Radio System (JTRS) encountered development problems, the 

number of requirements was reduced or deferred by about one-third. As a 

result, several programs that were dependent on JTRS also had to make 

 5

GAO, Defense Acquisitions: Assessments of Selected Major Weapon Programs, 

GAO-07-406SP (Washington, D.C.: Mar. 30, 2007). 

Page 5 GAO-07-388 Best Practices
adjustments and go forward with alternative, less capable solutions. 

DOD’s approach to managing weapon system investments ultimately 

results in less funding being available for other competing needs in DOD 

as well as other federal priorities, as the expenditure of tax dollars within

DOD reduces the amount of funding available for those priorities. 

Taking into account the differences between commercial product 

development and weapons acquisitions, we have recommended that DOD 

adopt a knowledge-based, incremental approach to developing and 

producing weapon systems. This type of an approach requires program 

officials to demonstrate that critical technologies are mature, product 

designs are stable, and production processes are in control at key 

junctures in the acquisition process.6

DOD has three major processes involved in making weapon system 

investment decisions. These processes, depicted in figure 1, are the Joint 

Capabilities Integration and Development System (JCIDS), for identifying 

warfighting needs; the Planning, Programming, Budgeting and Execution 

(PPBE) system, for allocating resources; and the Defense Acquisition 

System (DAS), for managing product development and procurement. 

Much of our prior work has focused on identifying commercial best 

practices that could be used to improve the Defense Acquisition System—

from the point just before product development starts onward. In this 

report, however, we look at earlier stages in DOD’s investment process—

from the point where gaps in warfighting capability are assessed in JCIDS 

through the point where alternative solutions to resolve those gaps are 

analyzed under the DAS (see fig. 1). 

 6

GAO, Best Practices: Capturing Design and Manufacturing Knowledge Early Improves 

Acquisition Outcomes, GAO-02-701 (Washington, D.C.: July 15, 2002); Best Practices: 

Better Matching of Needs and Resources Will Lead to Better Weapon System Outcomes, 

GAO-01-288 (Washington, D.C.: Mar. 8, 2001); and Best Practices: Better Management of

Technology Development Can Improve Weapon System Outcomes, GAO/NSIAD-99-162 

(Washington, D.C.: July 30, 1999). 

Page 6 GAO-07-388 Best Practices
Figure 1: DOD’s Weapon System Investment Process 

Source: DOD (data); GAO (analysis and presentation).

Product

developed

Technology

developed

Concept

refined

Defense Acquisition System

Alternative solutions

identified 

Capability gaps

assessed 

Future capabilities

identified 

Joint Capabilities Integration and Development System

Planning Programming and Budgeting Execution

Planning, Programming, Budgeting, and Execution System

Scope of our review

To ensure they achieve a balanced mix of executable development 

programs, the successful commercial companies we reviewed use a 

disciplined and integrated approach to prioritize market needs and 

allocate resources. This approach, known as portfolio management, 

requires companies to view each of their investments from an enterprise 

level as contributing to the collective whole, rather than as independent 

and unrelated. With this enterprise viewpoint, companies can effectively 

(1) identify and prioritize market opportunities and (2) apply available 

resources to potential products to select the best mix of products to 

exploit the highest priority—or most promising—opportunities. 

Ultimately, each of the companies we reviewed seeks to achieve a 

balanced portfolio that maximizes the return on investments and moves 

the company toward achieving its strategic goals and objectives. This type

of approach depends on strong governance with committed leadership, 

clearly aligned responsibility, and effective accountability at all levels of 

the organization. 

Successful Companies 

Take a Disciplined, 

Integrated Approach 

to Prioritize Market 

Needs and Initiate a 

Balanced Mix of 

Executable 

Development 

Programs 

Page 7 GAO-07-388 Best Practices
As depicted in figure 2, a portfolio management approach begins with an 

enterprise-level identification and definition of market opportunities and 

then the prioritization of those opportunities within resource constraints. 

Once opportunities have been prioritized, companies draft initial business 

cases for alternative product ideas that could be developed to exploit each 

of the highest priority opportunities. Each alternative product proposal—

represented by a black dot—enters a gated review process. At each review 

gate, product proposals are assessed against corporate resources, 

established criteria, competing products, and the goals and objectives of 

the company as a whole. As alternatives pass through each review gate, 

the number is expected to decrease, until only those alternatives with the 

greatest potential to succeed make it into the product portfolio. 

Page 8 GAO-07-388 Best Practices
Figure 2: Portfolio Management Approach to Product Investments Market opportunities identified and defined Market opportunities prioritized Initial business cases drafted for competing product ideas

Review

gate

Phase 1 Phase 2 Phase 3 Product development phase

Review

gate

Review

gate

Product

portfolio

Review

gate

Available resources

Corporate strategic goals and objectives

Source: GAO analysis and presentation of commercial best practices.

Identifying and Prioritizing 

Market Opportunities Lays 

the Foundation for 

Achieving the Right Mix of 

Products

To make informed decisions about what market opportunities to target, 

the companies we reviewed first establish a strategy that lays out the 

overall goals, objectives, and direction for the company. As part of their 

strategy, companies identify enterprise-level sales and profit targets, 

strategic business areas they want to focus on, the extent to which current 

products and new development efforts will support their growth 

objectives, and how they will allocate resources across business units and 

functional areas. This strategy provides a framework for the companies’ 

investment decisions. Within this framework, companies conduct a series

of market analyses to develop a comprehensive understanding of the 

Page 9 GAO-07-388 Best Practices
market environment, including product trends, technology trends, and 

customer needs. 

IBM for example, follows a structured market planning process to identify, 

prioritize, and target attractive market segments. The first phase of this 

process, called Market Definition, focuses on understanding the 

marketplace, including identifying potential customers and their needs.7

During this phase, IBM examines the marketplace and technology 

environments and identifies attractive market segments that contain 

potential market opportunities—where customer wants or needs exist. 

Each segment is categorized into one of four areas based on needs of the

customers and the company’s product offerings (see fig. 3): “strike zone,” 

“traditional,” “pushing the envelope,” and “white space.” The strike zone 

represents IBM’s core business—market segments where IBM has an 

established customer base that it is successfully serving with existing 

product offerings. In contrast, white space represents market segments of 

new customers with wants and needs that are new and different for IBM. 

White space opportunities often require discovery and innovation. The 

traditional and pushing the envelope areas fall between the strike zone and 

white space. Traditional opportunities exist when new customers could be 

attracted to an existing market—one IBM is already active in—by 

modifying or enhancing existing products or services. Pushing the 

envelope opportunities exist where the needs of current customer groups 

move them into a new market segment. These attractive market segments 

are prioritized during the next phase of IBM’s process, known as the 

Capability Assessment phase. During this phase each segment’s overall 

attractiveness and potential profitability are assessed, along with IBM’s 

available resources—like capital, cash, and current products—and its 

competitive position within each segment. This analysis leads to the 

selection of targeted market segments. 

 7

IBM focuses on providing its customers with what it calls “industry integrated solutions.” 

As a result, portfolios within IBM are structured around the customers’ and buyers’ 

behaviors and needs and not on specific product offerings. In other words, the company

focuses on providing functional solutions to the customers, which could be done with a

number of product offerings and services, and not simply limiting themselves to narrowly

focused, single product solutions that the customer can take or leave, an approach that

IBM believes caused problems for the company in the past. 

Page 10 GAO-07-388 Best Practices
Figure 3: IBM Market Segmentation

Source: IBM’s IPD process (data); GAO (analysis and presentation).

MARKETS

OFFERINGS

Current 

business

Enhancement to 

current business

New business New to industry

New customer 

wants and needs

Customers in new 

segments

New customers in 

existing segments

Current customers Strike

Zone

White space opportunity

Pushing the envelope

Traditional

Motorola emphasizes the importance of targeting the right market 

segments at the enterprise level to ensure that a balanced mix of project 

and resource investments is maintained. Officials noted that excessively 

focusing on segments that require new and innovative products can result 

in long cycle times, wasted money, and lost opportunities elsewhere. 

Likewise, critical opportunities can be lost when too much emphasis is 

placed on simply continuing to invest in old markets with old products. 

According to the officials we spoke with, the current investment mix for 

Motorola’s Government and Enterprise Mobility Solutions business unit is 

roughly 70-20-10, where 70 percent of its projects and resources are 

dedicated to maintaining its core business, while 20 percent are invested 

in pursuing new markets with existing products or introducing new or 

enhanced products into existing markets, and the remaining 10 percent are 

dedicated to discovering new markets and new products. 

As part of their market analyses, companies increasingly refine their 

understanding of who their customers are and what they need. For several

of the companies we met with, determining the needs of their customers is 

Page 11 GAO-07-388 Best Practices
complex because they have multiple groups of customers to consider. For 

example, Eli Lilly has four customer groups with diverse needs: patients, 

doctors, insurance companies, and government regulators. This 

complexity is compounded when considering that success in a worldwide 

market is critically dependent on a company’s ability to operate within 

different governmental systems, laws, and regulations; and regional 

markets. Several of the companies we reviewed use a variety of methods—

including interviews, surveys, focus groups, and concept tests—to actively 

engage their customers and help determine what they need. Some 

companies also observe customer behaviors to identify unstated wants 

and needs that if met—assuming corporate knowledge and resources 

allow—could actually exceed customer expectations. While companies 

actively seek customer input to identify products that show the most 

promise and satisfy customer needs, customers generally do not identify 

specific products to be developed. 

Companies Follow a 

Disciplined Process to 

Identify New Products and 

Achieve a Balanced 

Portfolio 

Once companies have identified and prioritized their market opportunities, 

they follow a disciplined process to assess the costs, benefits, and risks of 

potential product alternatives and allocate resources to achieve a balanced 

portfolio that spreads risk across products, aligns with the company’s 

strategic goals and objectives, and maximizes the company’s return on 

investment. At an early stage, each alternative product is expected to be

accompanied by an initial business case that contains knowledge-based 

information on strategic relevance and estimates of cost, technology 

maturity, and the cycle time for getting the product from concept to 

market. To ensure comparability across alternatives, companies require 

initial business case information to be developed in a transparent manner, 

to use specific standards, and to report estimates within certain levels of 

confidence or allowable deviations. Each of the companies we reviewed 

also stressed the importance of having multiple management review 

points, or gates, at early phases to assess and prioritize alternative 

products. As products move through review gates, from ideas, to more 

concrete concepts, to the start of development where a final business case 

is made, companies expect uncertainties—which are typically inherent in

the early phases—to be addressed and estimates to become more precise. 

Consequently, the number of viable alternatives tends to decrease at each

review gate as those with the lowest potential for success and least value

are terminated or deferred, while those that are poised to succeed and 

providing the best value are approved to proceed (illustrated in fig. 2). 

Companies emphasized that making tough go/no-go decisions is critical to

keeping a balanced portfolio. Over time, as potential new products are 

identified, companies review them against other product investments 

Page 12 GAO-07-388 Best Practices
(proposed and existing) and rebalance their portfolios based on those that 

add the most value. 

The companies we visited each follow a disciplined, gated review process 

to ensure that they commit to development programs that help balance the 

portfolio and that are executable given available corporate resources. This 

allows companies to avoid committing to more programs than their 

resources can support and ensure stability in the programs they invest in. 

Although the number of review gates prior to the start of full-scale product 

development varied between companies—ranging from four at Procter & 

Gamble, to eight at Motorola—they all required potential products to 

follow an established, disciplined process and meet specified criteria at 

each review point. For example, Caterpillar assesses product alternatives 

at four review gates prior to the start of development—three of which 

were recently added to enhance the rigor of its investment decision 

making. Each alternative must be supported by a draft business case that 

includes quantifiable data that can be compared with specific standards 

and used to determine if the related product can move past that gate. At 

each gate, alternatives are reviewed to ensure that knowledge about 

critical technologies, life-cycle costs, product reliability, and product 

affordability is being acquired and that the product contributes to 

achieving the company’s strategic goals and objectives. 

Because developing a new drug is costly and time consuming, Eli Lilly 

requires that the data supporting potential new drugs must meet high 

standards to ensure that managers are informed to make sound investment 

decisions.8

 Each potential new drug must be supported by an initial 

business case that contains information about safety and efficacy; 

forecasted revenue; expected unit demand; capital, medical, supply and 

material, development, and selling and marketing expenses; and general 

administrative costs. The initial business case must also identify critical 

success factors, state the probability of technical success, and provide a 

timeline that details when major milestone events are expected and how 

long it will take to get the associated drug to market. Eli Lilly assesses, 

approves, and funds proposed new drugs incrementally. At each milestone 

review a contract is established between the project team and a 

gatekeeper committee, which contains deliverables, time frames, and the 

 8

According to Eli Lilly representatives, the average cost to discover and develop a new drug

ranges from $800 million to $1 billion, and the average length of time to get a new drug

from discovery to the market is 10 to 15 years.

Page 13 GAO-07-388 Best Practices
costs to get to the next milestone. Once this contractual agreement is 

reached the budget is allocated for the entire phase. The gatekeeper 

committees expect each new drug proposal to achieve an 80-percent 

confidence level in their cost and schedule estimates for the next phase. 

This high level of confidence is achievable in large part because final 

budget estimates are not developed by project teams until 2 months prior 

to the milestone review. Projects are terminated at early points in the 

review process when it is determined that their critical success factors 

cannot be achieved. Because Eli Lilly’s projects typically have a high 

degree of technical risk, only about 1 percent of those that start early 

development actually make it to the marketplace. 

Motorola officials also emphasized the importance of having sound 

information when assessing potential new products. They noted that a 

process without sound information will not produce good outcomes. 

Therefore, Motorola’s Government and Enterprise Mobility Solutions 

business unit expects potential products to be supported by initial 

business cases containing data that meet specific standards and levels of

confidence at each review gate. For example, cost estimates for potential

products are developed in several phases and are expected to increase in 

confidence with each successive phase. Early in the investment planning,

when an initial business case is first drafted, the confidence parameters 

are generous, ranging from as much as 75 percent higher to as much as 

25 percent lower than what the project will likely end up costing. By the 

time a product alternative reaches the beginning of product development,

when a final business case is made, Motorola expects the cost estimates to 

be at confidence levels of 10 percent higher and 5 percent lower. Proposed 

products that fail to meet the specified criteria at early review gates are 

either terminated or sent back to further mature and reenter the review 

process from the beginning. 

The companies we reviewed use a variety of portfolio management tools 

and methods to inform the investment and resource allocation decisions 

they make at each review gate. Some companies employ scoring methods, 

using experts to rate products based on a number of factors—such as 

strategic fit, risk, and economic value—and use that information to 

prioritize alternative products. Another common tool plots alternative 

products on a decision matrix that compares factors such as costs and 

benefits, or risks and rewards of competing alternatives. Using this type of 

matrix, alternative products are often represented by circles, where the 

Page 14 GAO-07-388 Best Practices
size of the circles provides information about key constraints such as 

available annual resources or the estimated annual costs for each 

alternative. For example, figure 4 compares risk and expected rewards9

 by

plotting competing alternatives on a matrix. Alternatives that fall into the 

upper left quadrant are high risk and low reward, while alternatives that 

fall into the lower right quadrant are low risk and high reward. By 

weighing risk against rewards and considering constraints such as annual 

resources or annual cost, this tool provides critical information and a 

structured means to help managers make informed decisions. Company 

officials at Procter & Gamble emphasized the importance of selecting a 

balanced mix of products to pursue. They noted that pursuing only lowrisk and high-reward products at the expense of more innovative, higherrisk products could cause the company to miss out on opportunities to 

improve their competitive standing in the marketplace. Likewise, 

excessive pursuit of higher-risk products with the potential for high 

returns could also result in lost opportunities to elsewhere. 

 9

Companies commonly measure rewards and benefits using cash flow analysis known as

net present value (NPV) analysis. NPV techniques can show, in today’s dollars, the relative 

net cash flow of various alternatives over a long period of time. Simply stated, net cash 

flow is the amount of dollars that is left after sales and revenues have offset expenses. In

general, the greater the net cash flow for a particular investment, the greater the return on 

the investment. 

Page 15 GAO-07-388 Best Practices
Figure 4: Risk Versus Rewards Matrix Ri

s

k

Rewards

low

high

high

Source: GAO.

Recognizing the inherent risks in pursuing a new development program—

that overruns or underruns in one business case result in lost opportunity 

to invest resources in another worthwhile project—IBM permits products 

to deviate from their original business case estimates as long as the 

deviation is within established limits. These limits are specified in a 

contractual document resulting from negotiations between senior 

management and project managers and signed at the beginning of product

development. Product development teams are expected to execute 

according to the contract; if established thresholds are breached, action is 

taken immediately to reassess the product within the context of the 

portfolio and determine whether it is still a relevant and affordable 

investment to pursue. 

Page 16 GAO-07-388 Best Practices
Successful portfolio management requires strong governance with 

committed leadership that empowers portfolio managers to make 

decisions about the best way to invest resources and holds those 

managers accountable for the outcomes they achieve. The companies we 

reviewed indicated that it is critical to have commitment from the top 

leaders of the organization and recognition at all levels that what is best 

for the company must be a priority, and not simply what is best for a 

particular business unit or product line. In addition, the companies 

emphasized that roles and responsibilities for implementing portfolio 

management, including the designation of who is responsible for product 

investment decisions and oversight, must be clearly defined. Because 

portfolio managers are on the front line, the companies we reviewed 

empower these managers to make product investment decisions and hold 

them accountable for outcomes, not just for individual products but also 

for the overall performance of their portfolios. To support their portfolio 

managers, the companies encourage collaboration and communication, 

including sharing bad news early. Several companies also emphasized the

importance of supporting their portfolio managers with cross-functional 

teams, composed of representatives from the key functional areas within 

the company—such as science and technology, marketing, engineering,

and finance—to ensure that they are adequately informed when making 

investment decisions. To ensure accountability, companies often use 

incentives and disincentives, including promotion and termination. We 

have previously reported that high-performing organizations have 

monetary and other rewards that clearly link employee knowledge, skills, 

and contributions to achieving the organization’s goals and objectives.10

These organizations underscore the importance of holding individuals 

accountable and aligning performance expectations with organizational 

goals and cascade those expectations down to lower levels. Companies 

stressed that the transformation to portfolio management takes time and 

requires not only process changes but cultural changes throughout the 

company. 

Successful Portfolio 

Management Requires 

Strong Governance with 

Committed Leadership, 

Empowered Decision 

Makers, and Effective 

Accountability 

Eli Lilly emphasized that a key to making its portfolio management 

process work is having a single committee with a high-level official in 

charge responsible for making product investment decisions. Previously, 

the company had a multi-layered committee structure in place, and 

decisions were made based on reaching a consensus—an approach that 

 10GAO, Results-Oriented Cultures: Creating a Clear Linkage between Individual

Performance and Organizational Success, GAO-03-488 (Washington, D.C.: Mar. 14, 2003).

Page 17 GAO-07-388 Best Practices
was viewed as cumbersome and lengthy. Eli Lilly also ensures 

accountability by directly linking management and employee bonuses to

the overall success of the company. Individual employee performance 

objectives are aligned with specific company objectives, such as meeting

budgetary goals, time frames, and data quality levels for a given project. 

Achievement of individual employee objectives is measured periodically to

provide feedback to the employee. Eli Lilly officials stressed that having

the right performance metrics in place is important because ultimately you 

get what you measure; therefore, be sure to measure the right things. 

Motorola considers accountability to be the critical factor in making its 

portfolio management process successful. In addition, Motorola’s culture

is not averse to reporting bad news to management. Project managers are 

encouraged to report problems early so that they can be addressed before 

they get out of control. Senior managers, however, are not intimately 

involved in the day-to-day decision making for individual products. That 

responsibility, in nearly every case, is delegated to the business unit 

general manager. The general manager of a business unit is held 

accountable for ensuring that the products within his unit succeed at all 

levels. The general manager is responsible for holding product managers 

accountable for the attainment of critical knowledge at key points and the 

performance of their individual products overall. General managers and

product managers can be fired for not meeting objectives. Motorola 

believes that if managers are held accountable for results, then they have 

more desire to get it right. 

Although the military services fight together on the battlefield as a joint 

force, they do not identify warfighting needs and make weapon system 

investment decisions together. DOD has taken steps to identify warfighting 

needs through a more joint requirements process, but the department’s 

service-centric structure and fragmented decision-making processes are at 

odds with the integrated, portfolio management approach used by 

successful commercial companies to make enterprise-level investment 

decisions. Consequently, DOD has less assurance that its weapon system 

investment decisions address its most important warfighting needs and are 

affordable in the context of its overall fiscal resources. In addition, DOD 

commits to products earlier than the companies we reviewed and with far 

less knowledge about their cost and feasibility. This leads to poor program 

outcomes and funding instability, as the department attempts to fix 

troubled programs by taking funds from others. 

Lacking an Integrated, 

Portfolio-Based 

Approach, DOD Has 

Too Many Programs 

Competing for 

Limited Resources 

Page 18 GAO-07-388 Best Practices
Although recent DOD policy emphasizes a more joint approach to 

identifying and prioritizing warfighting needs,11 DOD’s service-centric 

structure and fragmented decision-making processes hinder the policy’s 

successful implementation. This policy, which introduced the JCIDS 

process, calls for a wider range of stakeholders than before, including 

more customer (i.e., combatant command) involvement; introduces new 

methodologies intended to foster jointness; and groups warfighting needs 

into eight functional areas based on warfighting capabilities—such as 

netcentric, force application, and battlespace awareness12—that cut across 

the military services and defense agencies. The JCIDS process emphasizes 

early attention to the fiscal implications of newly identified needs, 

including identifying ways to pay for new capabilities by divesting the 

department of lower priority or redundant capabilities. Despite these 

provisions, assessments of warfighting needs continue to be driven by the 

services and to be based on investment decision-making processes that do 

not function together to ensure that DOD pursues needs that its resources 

can support.

Service-centric Structure 

and Fragmented Decisionmaking Processes Impede 

DOD’s Ability to Prioritize 

Warfighting Needs 

The military services identify warfighting needs individually, and 

department-level organizations are not optimized to integrate the services’

results or evaluate their fiscal implications early on. Historically, this 

approach has contributed to duplication in weapon systems and 

equipment that does not interoperate. At the department level, Functional

Capability Boards oversee each of the eight functional areas, reviewing the 

services’ assessments, and providing recommendations to the Joint 

Requirements Oversight Council (JROC), which leads the JCIDS process. 

However, defense experts and DOD officials report that the Functional 

Capability Boards do not have the staff or analytical resources required to 

effectively evaluate service assessments within the context of the broader 

capability portfolio and assess whether the department can afford to 

address a particular capability gap. Several recent studies have 

recommended that DOD increase joint analytical resources for a less 

 11Chairman of the Joint Chiefs of Staff Instruction, Joint Capabilities Integration and 

Development System, CJCSI 3170.01E (May 11, 2005). The original instruction was CJCSI

3170.01C (June 24, 2003). 

12The other capability areas are command and control, focused logistics, force 

management, force protection, and joint training. 

Page 19 GAO-07-388 Best Practices
stovepiped understanding of warfighting needs.13 In addition, the boards 

lack the authority to allocate resources and to make or enforce decisions 

to divest their capability area of existing programs to pay for new ones—

authority successful companies provide to their portfolio managers. 

Finally, some defense experts contend that the service ties of JROC’s 

members—that is, the services’ Vice Chiefs and the Assistant Commandant 

of the Marine Corps—reinforce service stovepipes. To better ensure a 

more joint perspective, they recommend a more diverse JROC, with 

representatives from other department-level organizations and the 

combatant commands.14

Resource allocation decisions are made through a separate process—the 

Planning, Programming, Budgeting, and Execution system (PPBE)—which 

hinders the department’s ability to weigh the relative costs, benefits, and 

risks of investing in new weapon systems early on. Within the PPBE 

system, the individual military services are responsible for budgeting and 

allocating resources under authority that is commonly understood to be 

based on Title 10 of the United States Code.15 PPBE is structured by 

military service and defense program, although the department integrates 

data on the services’ current and projected budget requests under 

11 crosscutting mission areas called Major Force Programs. The crosscutting view provided by the Major Force Program structure is intended to

facilitate a strategic basis for resource allocation, allowing the Secretary of 

Defense to more easily see where the greatest mission needs are and to

re-allocate funds to meet those needs regardless of which service stands to

gain or lose. However, we have reported in the past that the Major Force 

 13Institute for Defense Analyses, Improving Integration of Department of Defense 

Processes for Capabilities Development and Planning (Sept. 2006); Center for Strategic 

and International Studies, Beyond Goldwater-Nichols: Defense Reform for a New Strategic 

Era, Phase 1 Report (Mar. 2004); and Joint Defense Capabilities Study Team (DOD), Joint 

Defense Capabilities Study: Improving DoD Strategic Planning, Resourcing and 

Execution to Satisfy Joint Capabilities. Final Report (Jan. 2004). 

14Assessment Panel of the Defense Acquisition Performance Assessment Project for the

Deputy Secretary of Defense, Defense Acquisition Performance Assessment Report (Jan. 

2006); Center for Strategic and International Studies, Beyond Goldwater-Nichols: U.S.

Government and Defense Reform for a New Strategic Era, Phase 2 Report (July 2005); and 

M. Thomas Davis, “The JROC: Doing What? Going Where?” National Security Studies 

Quarterly (Summer 1998). 

15 Sections 3013, 5013, and 8013 of Title 10 grant authority to the Secretaries of the Army, 

the Navy, and the Air Force, respectively, to conduct all affairs of their departments, 

including recruiting, organizing, supplying, equipping, training, servicing, mobilizing,

demobilizing, administering, maintaining, and military construction and maintenance. 

Page 20 GAO-07-388 Best Practices
Program structure has not provided sufficient visibility in certain mission 

areas.16 Moreover, although they cut across the services, the program 

mission areas are not consistent with the more recently established 

capability areas used in the JCIDS process,17 and as a result, it is difficult to 

relate resources to capabilities. For example, in prior work, we observed 

that the Major Force Programs contain large numbers of programs with 

varied capabilities, complicating comparisons needed to understand 

defense capabilities and associated trade-off decisions.18 We have 

recommended that DOD report funding levels for defense capabilities in 

its Future Years Defense Program report to the Congress, which is 

currently organized by the Major Force Programs. 

In addition, our analysis of DOD’s investment accounts—which pay for 

developing, testing, and buying weapon systems and other equipment—

indicates that DOD generally does not allocate resources on a strategic 

basis. Figure 5 illustrates that the service allocations as a percentage of the 

department’s overall investment budget have remained relatively static for 

the 25-year period we examined, even though DOD’s strategic environment 

and warfighting needs have changed dramatically during that time, with 

the demise of the cold war and the emergence of the global war on terror.19

In contrast, successful commercial companies using portfolio management 

would expect to see their resource allocations across business areas to 

reflect changes in the marketplace and the competitive environment. 

 16GAO, Military Transformation: Actions Needed by DOD to More Clearly Identify Triad 

Spending and Develop a Long-term Investment Approach, GAO-05-540 (Washington, D.C.: 

June 30, 2005). 

17The Major Force Programs are as follows: Strategic Forces; General Purpose Forces; 

Command, Control, Communications, and Intelligence; Mobility Forces; Guard and Reserve

Forces; Research and Development; Central Supply and Maintenance; Training, Medical, 

and other General Personnel Activities; Administration and Associated Activities; Support

of Other Nations; and Special Operations Forces. 

18GAO, Future Years Defense Program: Actions Needed to Improve Transparency of 

DOD’s Projected Resource Needs, GAO-04-514 (Washington, D.C.: May 7, 2004). 

19The fiscal framework of the federal government constrains DOD’s ability to totally control 

its investment allocations. Ultimately resource allocation decisions are made by the

Congress in the annual authorization and appropriations process.

Page 21 GAO-07-388 Best Practices
Figure 5: Service Allocations of DOD’s Investment Budget (FY1986 through FY2011) 

Percentage

Fiscal year

0

10

20

30

40

50

60

70

80

90

100

1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Source: DOD (data); GAO (analysis and presentation).

Defense wide

Navy

Army

Air Force

PPBE and JCIDS are led by different organizations (see fig. 6), as is the 

third of the three processes involved in DOD’s weapon system investment 

decisions, the Defense Acquisition System (DAS), making it difficult to 

hold any one person or organization accountable for investment 

outcomes. The 2006 Quadrennial Defense Review highlighted the need for 

governance reforms,20 and a 2006 study commissioned by DOD observed 

that the budget, acquisition, and requirements processes are not 

connected organizationally at any level below the Deputy Secretary of 

Defense, concluding that this structure induces instability and erodes 

accountability.21 The Under Secretary of Defense/Acquisitions, 

Technology, and Logistics (USD/AT&L) has stated that weapon system 

investment decisions are a shared responsibility, and, therefore, no one 

 20The 2006 Quadrennial Defense Review led to the Institutional Reform and Governance 

project, which is focusing on (1) integrating core decision processes, (2) aligning and 

focusing the department’s governance and management functions under an integrated 

enterprise model, as well as (3) establishing a common and authoritative analytical 

framework to link strategic decisions to execution. 

21Assessment Panel of the Defense Acquisition Performance Assessment Project for the

Deputy Secretary of Defense, Defense Acquisition Performance Assessment Report 

(January 2006). 

Page 22 GAO-07-388 Best Practices
individual is accountable for these decisions. At a broader, strategic level, 

we have stated in prior work that DOD has lacked sustained leadership 

and accountability for various department-wide management reform 

efforts,22 including the establishment of an effective risk management 

approach as a framework for decision making.23 This approach would link 

strategic goals to plans and budgets, assess the value and risks of various

courses of action as a tool for setting investment priorities and allocating 

resources at the department level, and use performance measures to 

assess outcomes. To address the lack of sustained leadership, we have 

supported legislation to create a chief management official at DOD.24

 22GAO, Department of Defense: Sustained Leadership Is Critical to Effective Financial 

and Business Transformation, GAO-06-1000T (Washington, D.C.: Aug. 3, 2006); and

Department of Defense: Further Actions Are Needed to Effectively Address Business 

Management Problems and Overcome Key Business Transformation Challenges, 

GAO-05-140T (Washington, D.C.: Nov. 18, 2004). 

23GAO, Defense Management: Additional Actions Needed to Enhance DOD’s Risk-Based 

Approach for Making Resource Decisions, GAO-06-13 (Washington, D.C.: Nov. 15, 2005). 

24S. 780, 109th Cong. §1 (2005). 

Page 23 GAO-07-388 Best Practices
Figure 6: Governance of DOD’s Investment Process 

Source: DOD (data); GAO (analysis and presentation).

Product

developed

Technology

developed

Concept

refined

Defense Acquisition System (event-driven)

Alternative solutions

identified 

Capability gaps

assessed 

Future capabilities

identified 

Joint Capabilities Integration and Development System (need-driven)

JCS (JROC) responsibility USD (AT&L) Responsibility

Planning Programming and Budgeting Execution

Planning, Programming, Budgeting, and Execution System (calendar-driven)

OSD (Policy, PA&E, and Comptroller) responsibility

JCS (JROC) — Joint Chiefs of Staff (Joint Requirements Oversight Council)

OSD (Policy, PA&E, and Comptroller) — OSD (Policy, Program Analysis and Evaluation, and Comptroller)

OSD (AT&L) — OSD (Acquisition, Technology and Logistics)

The Office of the Secretary of Defense (OSD) does not assess the funding 

implications of a proposed program at the front end of the investment 

process, when it is initially validated by JROC. JCIDS is a continuous, 

need-driven process that unfolds in response to warfighting needs as they 

are identified. However, PPBE is a calendar-driven process comprised of 

phases that occur over a 2-year cycle, thus OSD’s formal review of a 

proposed program is not often synchronized with JROC’s, and can occur 

several years later.25 Nevertheless, according to Joint Staff and AT&L 

officials we met with, proposed programs begin to gain momentum when 

they are validated by JROC, and they become very difficult to stop. These 

officials indicated that momentum begins to gather because the services 

 25DAS is an event-driven process structured into discrete phases separated by major 

decision points called milestones that can be tailored to individual programs. 

Page 24 GAO-07-388 Best Practices
start programming and budgeting for the proposed capability right away to 

secure funding, generally several years before actual product development 

begins and before OSD formally reviews the services’ programming and 

budgeting proposals. In the interim, the services have not only budgeted 

for their proposed programs, but established a program office, conducted 

their Analysis of Alternatives, and identified specific user requirements. 

OSD’s programming and budgeting review occurs at the back end of the 

investment process, when it is difficult and disruptive to make changes, 

such as terminating existing programs to pay for new, higher priority 

programs. 

These practices have contributed to the department starting more 

programs than its resources can support. DOD defers much of the 

additional cost of its programs into the future, resulting in what some have 

characterized as a fiscal bow wave (illustrated in fig. 7). This bow wave 

has grown at a pace that greatly exceeds DOD’s annual funding increases. 

The cost remaining for DOD’s major weapons programs increased almost 

135 percent between 1992 and 2006, while the department’s annual funding

level only increased 57 percent over that same time period. If this trend 

goes unchecked, Congress will likely be faced with a difficult choice: pull 

funds from other high-priority federal programs to support DOD’s 

acquisitions or accept less warfighting capability than originally promised. 

Page 25 GAO-07-388 Best Practices
Figure 7: Costs Remaining Versus Annual Appropriations for Major Defense Acquisitions 

Dollars (in billions)

Fiscal year

0

200

400

600

800

1,000

1,200

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Source: DOD (data); GAO (analysis and presentation).

Costs remaining for major defense acquisitions

Annual appropriations for major defense acquisitions

DOD Commits to a 

Solution Earlier and with 

Less Knowledge 

DOD commits to a solution to address a warfighting need earlier in the 

investment process than commercial companies do and before it has 

adequate knowledge about cost and technical feasibility. Proposed options 

for resolving a gap in military capability are submitted in an Initial 

Capabilities Document (ICD). DOD guidance states that this document 

should contain a range of approaches based in part on the cost and 

technological feasibility posed by the approaches, laying the foundation 

for a more detailed Analysis of Alternatives to be conducted under the 

Defense Acquisition System. In addition, JROC is to receive a briefing on 

the ICD that follows a standard format and addresses such issues as 

• linkage of the proposal to strategic guidance; 

• the time frame within which the capability is needed; 

• the threat/operational environment; 

• risks and assumptions (including the risk associated with 

proceeding and not proceeding with solutions to each); and 

• a description of the best materiel and non-materiel approaches 

based upon cost, efficacy, performance, technology maturity, and 

risk. 

Page 26 GAO-07-388 Best Practices
Although DOD guidance calls for the analysis of a solution’s cost and 

feasibility, we found that ICDs contained little of this type of information. 

Several DOD officials we met with, who are directly involved in the JCIDS 

process, did not believe cost and feasibility information was mandated at 

this point. In our review of 14 unclassified ICDs approved by JROC from 

2003-06, we found that 11 did not contain acquisition cost estimates and 

12 did not contain estimates of the technical feasibility of proposed 

solutions.

26 We also found that JCIDS guidance does not specify the level 

of accuracy sought in cost and feasibility estimates, and a white paper that 

does provide recommendations in this regard is advisory.27

We found that ICDs generally focused on the strategic, or operational, 

relevance of proposed solutions, but a lack of guidance and an evolving 

methodology have raised questions about the accuracy of data supporting 

those assessments. JCIDS uses new joint warfighting concepts28 to 

translate top-level military strategy into the capabilities a commander 

might need on the battlefield. The joint concepts underpin a capabilitiesbased approach29 to identifying requirements, in which analyses are 

expected to focus on broad military capabilities rather than servicespecific platforms. However, the joint concepts and capabilities-based 

assessments are works in progress. The concepts are being updated due to

concerns about their scope, and guidance on conducting a capabilities-

 26The 14 ICDs we reviewed are related to weapon systems and were finalized after June 24, 

2003 (the publication date for the initial JCIDS instruction). They are unclassified ACAT I, 

II, or III ICDs contained in the Joint Staff’s Knowledge Management Decision Support Tool 

database, which serves in part as a repository for JCIDS documents. 

27The white paper suggests roughly characterizing the 20-year-lifecycle costs of proposed

solutions in terms of developmental costs, facility or infrastructure costs, per-unit and

rough force-level acquisition costs, and recurring operating costs. It states that rough

estimates of the technical feasibility of proposed solutions should be developed, not at the 

engineering level because of the broad range of possibilities, but at the least to characterize

as no risk, very low risk, low risk, medium risk, and high risk using legitimate technology 

experts. “Whitepaper on Conducting a Capabilities-Based Assessment Under the Joint 

Capabilities Integration and Development System,” Joint Chiefs of Staff J-8/Force 

Application Assessment Division (Jan. 2006). 

28Joint future concepts are visualizations of future operations that describe how a 

commander might employ warfighting capabilities to achieve effects and objectives. 

29The 2001 Quadrennial Defense Review directed DOD to implement a capabilities-based 

planning approach. Capabilities-based planning has been described as a framework for 

defense planning and decision making in a strategic environment characterized by 

uncertainty and a fiscal environment characterized by limited resources. See Paul K. Davis,

Analytic Architectures for Capabilities-Based Planning, Mission System Analysis, and 

Transformation (Santa Monica, Calif.: RAND, 2002). 

Page 27 GAO-07-388 Best Practices
based analysis has been lacking. Several DOD officials we met with stated 

that assessments vary in their rigor, and a senior Joint Staff official said 

that training on requirements development is one of three central 

challenges at present. In January 2007, we reported that DOD officials 

described concerns about the analytical framework for a capabilitiesbased assessment on joint seabasing, which could lead to inaccurately 

identifying gaps in implementing the concept.30 Enhancing a seabasing 

capability is expected to be costly and could be the source of billions of 

dollars of investment if DOD chooses an option involving the development 

of new ships. 

DOD does not consistently follow a disciplined review process to ensure 

that proposed solutions are making progress toward an executable 

development program, although DOD policy emphasizes that such reviews 

are necessary.31 DOD’s policy identifies several key decision points prior to 

starting a new weapon system development program: 

• an initial decision point, where the Initial Capabilities Document is 

reviewed, validated, and approved by the JROC; 

• a Concept Decision review, where entry into the concept refinement 

phase of the Defense Acquisition System should be authorized; and 

• a Milestone A decision point, where a preferred solution and a 

technology development strategy should be reviewed and 

approved.32

Since Initial Capabilities Documents generally do not contain information 

on cost and technical feasibility, the JROC does not have a sufficient basis 

 30GAO, Force Structure: Joint Seabasing Would Benefit from a Comprehensive 

Management Approach and Rigorous Experimentation before Services Spend Billions on

New Capabilities, GAO-07-211 (Washington, D.C.: Jan. 26, 2007). Joint seabasing is one of 

several evolving concepts describing how commanders in the future will project and 

sustain forces for conducting joint military operations without relying on immediate access 

to nearby land bases.

31CJCSI 3170.01E , Joint Capabilities Integration and Development System; DODI 5000.2, 

Operation of the Defense Acquisition System. 

32The JROC has another decision point just prior to program initiation, when it reviews,

validates, and approves a Capabilities Development Document. Approval and validation of 

a Capabilities Development Document is a key entrance criteria for initiating a new 

development program at Milestone B. We did not include the Capabilities Development

Document decision point in our current review because it is closely associated with 

Milestone B, the focus of many of our former reviews. See the list of related GAO products

on the last pages of this report. 

Page 28 GAO-07-388 Best Practices
for making go/no-go decisions at the initial decision point. In the 4 years 

since JCIDS was implemented, nearly all of the warfighting needs 

identified by the services and submitted for review in an ICD have been 

validated and sent into the acquisition pipeline for further analysis as 

potential programs, which calls into question whether go/no-go decisions

are the point of this first key gate. Information on cost and feasibility is 

generally developed after the ICD is approved and proposed solutions 

undergo further refinement through an Analysis of Alternatives (AOA). An 

AOA should compare alternative solutions in terms of life-cycle cost, 

schedule, and operational effectiveness, leading up to the identification of 

a preferred alternative. However, officials from PA&E and the Joint Staff 

indicate that AOAs often make a case for a single preferred solution. 

Several of them indicated other concerns about AOAs, such as not setting 

up trade-off discussions, lack of analytical rigor, length, and timeliness. 

In any case, the next review points—the Concept Decision and Milestone 

A—are often skipped; thus, the opportunity to review an evolving business 

case and to make go/no-go decisions is bypassed. In prior work, we found 

that 80 percent of the programs we reviewed entered the Defense 

Acquisition System at Milestone B without holding any prior major 

reviews, such as a Milestone A review.

33 Such reviews are intended to 

provide acquisition officials with an opportunity to assess whether 

program officials had the knowledge needed to develop an executable 

business case. Senior officials with OSD confirmed that this is a common 

practice among defense acquisition programs. We concluded that this 

practice eliminates a key opportunity for decision makers to assess the 

early product knowledge needed to establish a business case that is based 

on realistic cost, schedule, and performance expectations. In addition, we

found that programs are regularly approved to begin development even 

though officials reported levels of knowledge below the criteria suggested 

in DOD’s acquisition policy. 

There is, then, generally little department-level oversight between the 

point at which an ICD is approved and when system-level requirements are 

validated and product development is initiated. At this point, as we 

indicated earlier, there is generally no turning back, because the services 

have invested considerable time and money, established a budget, and 

 33GAO, Defense Acquisitions: Major Weapon Systems Continue to Experience Cost and

Schedule Problems under DOD’s Revised Policy, GAO-06-368 (Washington, D.C.: Apr. 13, 

2006). Our review focused on the Concept Decision and Milestone A reviews points and did 

not assess the extent to which JROC reviews were held. 

Page 29 GAO-07-388 Best Practices
formed a constituency for a proposed program, and decision makers 

become reluctant to terminate a program or send it back for further study. 

DOD Is Piloting Several 

Initiatives to Address 

Disconnects in Investment 

Decision-making 

In response to the 2006 Quadrennial Defense Review and other recent 

acquisition reform studies,34 DOD has undertaken several key, interrelated 

initiatives intended to strengthen the department’s approach to investment 

decisions. The initiatives include (1) taking a new approach to reviewing 

proposed concepts that will provide decision makers with an early 

opportunity to evaluate trade-offs among alternative approaches to 

meeting a capability need, (2) testing portfolio management approaches in

selected capability areas to facilitate more strategic choices about how to 

allocate resources across programs, and (3) using capital budgeting as a 

potential means to stabilize program funding. While promising, these 

initiatives do not fundamentally change DOD’s existing service-centric

framework for making weapon system investment decisions. 

To address a perceived gap between DOD’s major decision-making 

processes and provide a department-level means to assess potential 

solutions (materiel and non-materiel) to fill a validated capability need, 

DOD is testing a new approach to a Concept Decision review, which will

take place after a warfighting need is validated by the JROC. This new 

approach is intended to focus attention on the affordability and feasibility 

of potential solutions and generate early cost, schedule, and performance 

trade-offs prior to the point of a significant investment commitment. As 

currently proposed, the Concept Decision will be informed by a newly 

required Evaluation of Alternatives that will integrate the Functional 

Solutions Analysis conducted under JCIDS with the Analysis of 

Alternatives conducted under the acquisition system and lay out the 

relative merits and limitations of potential solutions. Furthermore, concept 

decision reviews will be implemented by a tri-chair board consisting of 

lead decision makers from the JCIDS, PPBE, and DAS processes. While 

promising, the Concept Decision review largely reinstitutes a review point 

that already existed but was only intermittingly used. For Concept 

Decision reviews to be effective, DOD will have to establish enforcement 

and accountability mechanisms to ensure the reviews are actually 

implemented. In addition, the extent to which the concept reviews can 

 34Defense Acquisition Performance Assessment (January 2006); Defense Science Board

Summer Study on Transformation: A Progress Assessment (February 2006); Beyond 

Goldwater-Nichols: U.S. Government and Defense Reform for a New Strategic Era, Phase 

2 Report (July 2005); and Joint Defense Capabilities Study (January 2004).

Page 30 GAO-07-388 Best Practices
achieve desired effects will depend on what authority Concept Decisions 

carry and who will be held accountable, particularly in light of the servicedominated investment structure that currently exists. 

The department has also begun to pilot-test capability-based portfolio 

management, selecting four joint capability areas to focus on—joint 

command and control, joint net-centric operations, battlespace awareness, 

and joint logistics. The intent is to enable the department to develop and 

manage capabilities, as opposed to simply individual programs, and 

enhance the integration and interoperability within and across sets of 

capabilities. Each portfolio is being structured somewhat differently to 

help the department determine how best to proceed with portfolio 

management. All, however, are intended to focus initially on existing 

programs and to operate within DOD’s existing decision-making 

framework. The portfolios are largely advisory and will, as a first step, 

provide input to decisions made through the JCIDS, PPBE, and DAS 

processes. At this point, the capability portfolio managers have not been 

given direct authority to manage fiscal resources and make investment 

decisions. Without portfolios in which managers have authority and 

control over resources, DOD is at risk of continuing to develop and 

acquire systems in a stovepiped manner and of not knowing whether its 

systems are being developed within available resources. 

DOD is also examining the use of capital accounts as a potential means of 

stabilizing program funding, which has long been cited as a significant 

issue in program management. This capital budgeting pilot initiative is in 

the early stages of planning, and the specifics of how such accounts will 

be implemented are being developed, but the intent is for DOD to commit

a set amount of funding for the development portion of a project and hold

to that commitment by not adjusting funding up or down until the product 

is delivered. In addition to resource constraints, programs would be given 

a fixed amount of time to get from one milestone to the next. If successful, 

this initiative could represent a step toward stabilizing long-term costs 

within major defense acquisition programs, as well as a strengthening of

the ability of program managers to conduct long-term planning and control 

costs. However, for this initiative to be effective, DOD will need to 

overcome long-standing problems it has had in starting programs without 

sufficient knowledge of the costs, requirements, and technologies needed 

to develop proposed weapon systems. Unless this changes, it is unlikely 

that capital accounts will lead to increased program stability. 

Page 31 GAO-07-388 Best Practices
While DOD has increasingly strengthened its ability to operate as a joint 

force on the battlefield, the department’s organizational structures, 

processes, and practices for planning and acquiring weapon systems are 

not similarly joint. Put simply, DOD largely continues to base its 

investment decisions on service-driven analyses that do not provide an 

enterprise-level understanding of overall warfighting needs and on 

individual platforms rather than broader sets of capabilities. In contrast, 

successful commercial companies use an integrated portfolio management 

approach to focus early investment decisions on products collectively at 

an enterprise level and to ensure there is a sound basis to justify the 

commitment of resources. By following a disciplined, integrated process—

where the relative pros and cons of market opportunities and competing 

product proposals are assessed based on available resources and 

customer needs, and where tough decisions about which investments to 

pursue are made—companies are able to reduce duplication between 

business units, move away from organizational stovepipes, and effectively

support each new development program they commit to. Until DOD takes 

a joint, portfolio management approach to weapon system acquisition—

with functionally aligned entities that have the requisite responsibility, 

authority, and control over resources—it will continue to struggle to 

effectively prioritize warfighting needs, make informed trade-offs, and 

achieve a balanced mix of weapon systems that are affordable, feasible, 

and provide the best military value to the warfighter. Committing to more 

programs than the budget can support and approving programs based on 

insufficient knowledge to effectively manage risks will further delay 

providing critical capabilities to the warfighter and lead to lost 

opportunities to address other current and emerging needs. 

We recommend that the Secretary of Defense implement an enterprisewide portfolio management approach to making weapon system 

investments that integrates the assessment and determination of 

warfighting needs with available resources and cuts across the services by 

functional or capability area. To ensure the success of such an approach, 

the Secretary should establish a single point of accountability at the 

department level with the authority, responsibility, and tools to ensure 

that portfolio management for weapon system investments is effectively 

implemented across the department. 

Conclusions 

Recommendations 

In addition, the Secretary should ensure that the following commercial 

best practices, identified in this report, are incorporated: 

Page 32 GAO-07-388 Best Practices
• implement a review process in which needs and resources are 

integrated early and in which resources are committed 

incrementally based on the achievement of specific levels of 

knowledge at established decision points; 

• prioritize programs based on the relative costs, benefits, and risks 

of each investment to ensure a balanced portfolio; 

• require increasingly precise cost, schedule, and performance 

information for each alternative that meets specified levels of 

confidence and allowable deviations at each decision point leading 

up to the start of product development; 

• establish portfolio managers who are empowered to prioritize 

needs, make early go/no-go decisions about alternative solutions, 

and allocate resources within fiscal constraints; and 

• hold officials at all levels accountable for achieving and maintaining 

a balanced, joint portfolio of weapon system investments that meet 

the needs of the warfighter within resource constraints. 

We also recommend that the Secretary take steps to support departmentlevel decision makers and portfolio managers by developing a stronger 

joint analytical capability to assess and prioritize warfighting needs. 

DOD provided us with written comments on a draft of this report. The 

comments appear in appendix II. 

DOD concurred with the majority of our recommendations and partially 

concurred with two. Generally, in responding to these recommendations, 

DOD stated that it is undertaking several initiatives and pilot efforts to 

improve the department’s approach to investment and program decision 

making, and that implementation of any new business rules will be 

contingent upon the outcome of these initiatives. The department also 

stated that it is experimenting with portfolio management, related 

authorities and organizational constructs, and integrated decision-making 

processes. 

Agency Comments 

and Our Evaluation 

We believe that these initiatives and pilot efforts may be steps in the right

direction, but we are concerned that they do not go far enough to address 

the systemic cultural and structural problems identified in this report. 

DOD has attempted many similar acquisition reform efforts over the past 3 

Page 33 GAO-07-388 Best Practices
decades, including significant revisions to both defense requirements and 

acquisition policy. However, despite these efforts, weapon system 

acquisition programs continued to experience cost overruns, schedule 

slips, and performance shortfalls. The department’s current initiatives are 

likely to face the same fate because they do not fundamentally change 

DOD’s service-centric framework or sufficiently integrate its decisionmaking processes for making weapon system investments. 

DOD did not provide comments regarding our recommendation that the 

Secretary establish a single point of accountability at the department level 

with the authority, responsibility, and tools to ensure that portfolio 

management for weapon system investments is effectively implemented 

across the department. We believe that a single point of accountability is

necessary to successfully implement a portfolio management approach 

and integrate DOD’s fragmented decision-making processes under one 

senior official who is accountable for weapon system investment 

outcomes. We further believe that our recommendations would better 

position DOD to make tough, knowledge-based choices among potential

weapon system investments. 

We are sending copies of this report to the Secretary of Defense; the 

Secretaries of the Air Force, Army, and Navy; and the Director of the 

Office of Management and Budget. We will provide copies to others on 

request. This report will also be available at no charge on GAO’s Web site 

at http://www.gao.gov. 

If you have any questions about this report or need additional information, 

please call me at (202) 512-4841 (sullivanm@gao.gov). Key contributors to 

this report were John Oppenheim, Assistant Director; Lily Chin; 

John Krump; Matthew Lea; Travis Masters; Sean Seales; Karen Sloan; 

Susan Woodward; and Rebecca Yurman. 

Michael J. Sullivan 

Director, Acquisition 

 and Sourcing Management 

Page 34 GAO-07-388 Best Practices
Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology 

Methodology 

This report examines the Department of Defense’s (DOD) requirements 

identification and resource allocation processes for major weapons 

systems. The primary focus is on identifying successful private-sector 

principles and practices that could be adopted by DOD to help improve 

stability in weapon system acquisition programs. Specifically, our 

objectives were to (1) identify best practices of successful commercial 

companies for ensuring that they pursue the right mix of programs to meet

the needs of their customers within resource constraints and (2) compare 

DOD’s enterprise-level processes for investing in weapon systems to those 

practices. Our work was conducted between March 2006 and February 

2007, in accordance with generally accepted government auditing 

standards. 

We analyzed the outputs of DOD’s investment decision-making support 

processes—the requirements determination process known as JCIDS and 

the resource allocation process known as PPBE—using criteria 

established in DOD policy and in previous GAO reports. We identified 

impacts of the existing processes by analyzing quantitative and qualitative

data on DOD spending trends, conducting interviews with DOD officials, 

and reviewing previous reports by GAO and by other knowledgeable audit 

and research organizations. In addition, we met with officials representing 

the Office of the Secretary of Defense, Joint Staff, and military services. At 

each of these locations, we conducted interviews that helped us describe 

the current condition of DOD’s requirements identification and resource 

allocation processes. We also reviewed DOD and military service policies 

and funding documents pertaining to the DOD requirements identification 

process and resource allocation decisions for major weapons systems. 

Specifically, we reviewed the contents of 14 unclassified Initial Capability 

Documents that were finalized after June 24, 2003—the publication date

for the JCIDS instruction—to assess the extent to which they contained 

cost and technical feasibility information. Those 14 ICDs were 

unclassified, weapon system-related ACAT I, II, or III ICDs that were 

contained in the Joint Staff requirements database. We relied on previous 

GAO reports that highlight both the symptoms and causes of unstable 

requirements and funding in DOD weapons acquisition programs. A list of

these reports can be found at the end of this report. In addition, we 

reviewed recent key studies and reports addressing acquisition reform 

issues by the Center for Strategic International Studies, Institute for 

Defense Analysis, the U.S. Naval War College, the Defense Acquisition 

Performance Assessment Project, the Joint Defense Capabilities Study 

Team, the Joint C4ISR Decision Support Center, the Defense Science 

Board, and the 2001 and 2005 Quadrennial Defense Reviews. 

Page 35 GAO-07-388 Best Practices
Appendix I: Objectives, Scope, and 

Methodology 

We also reviewed pertinent literature from authoritative corporate, 

academic, and professional organizations, to identify commercial best 

practices and processes that could be used by DOD to improve its weapon

system investment decision-making processes. In addition we conducted 

case studies of five leading commercial companies. In selecting them, we 

sought to identify companies that were recognized in the literature for best 

practices, had large and diversified portfolios of products, and make 

significant investments in the development and production of new 

products. For each of the companies, we interviewed management 

officials knowledgeable about their requirements identification and 

resource allocation activities, to gather consistent information about 

processes, practices, and metrics the companies use to help achieve 

successful product development outcomes. Below are descriptions of the 

five companies featured in this report: 

Motorola 

Motorola is a Fortune 100 global communications leader that provides 

seamless mobility products and solutions across broadband, embedded 

systems, and wireless networks. According to Motorola’s 2005 Corporate 

Profile, the company is the market leader in mission critical wireless 

communication systems, two-way radios, embedded telematics systems, 

digital set-top shipments, cable modem shipments, digital head-ends, 

embedded computer systems for communication applications, CDMA 

infrastructure sales (excluding the United States), and second in world 

wide wireless handsets. Motorola achieved net sales of $31.323 billion and 

spent $3.060 billion on research and development in 2004. The corporation 

has approximately 68,000 employees, in 320 facilities, spanning 

73 countries. We met with the management of Motorola’s Government & 

Enterprise Mobility Solutions and Global Telecom Solutions sectors in 

Schaumburg, Illinois. 

International Business Machines (IBM) 

IBM is one of the world’s largest technological companies, spending about 

$3 billion annually on research and development activities. It is the largest

supplier of hardware, software, and information technology services. With

3,248 U.S. patents, IBM earned more patents than any other company for 

the 12th consecutive year in 2004. In the past 4 years, IBM inventors 

received more than 13,000 patents—approximately 5,400 more than any 

other patent recipient. IBM has over 329,000 employees worldwide. We 

met with managers from IBM Integrated Product Development (IPD) in 

Somers, New York. 

Page 36 GAO-07-388 Best Practices
Appendix I: Objectives, Scope, and 

Methodology 

Procter & Gamble (P&G) 

Procter & Gamble Corp. (P&G) is a leading producer of consumer goods. 

It currently leads in global sales and marketshare among all fabric care,

baby care, feminine care, and hair care products. It currently has over 

130,000 employees in 80+ countries. Twenty-two of its brands have annual

gross sales exceeding $1 billion each. In fiscal year 2005/2006, P&G 

invested $2.075 billion or 3 percent of net sales in research and 

development (R&D). This ranks them as one of the top 20 largest research 

& development investors among U.S.-based companies. P&G has more 

Ph.D.s working in labs around the world than the combined science and 

engineering faculties of Harvard, MIT, and Berkeley. We met with the 

management of P&G’s New Initiative Delivery team in Cincinnati, Ohio. 

Eli Lilly Corporation 

Eli Lilly is a global pharmaceutical company and one of the world’s largest 

corporations. It was founded over 130 years ago and currently employs 

approximately 42,000 people worldwide, including 13,991 employed at its 

headquarters in Indianapolis, Ind. Approximately 8,336 employees 

(19 percent of the total work force) are engaged in research and 

development (R&D); clinical research is conducted in over 50 countries; 

there are R&D facilities in 9 countries; and manufacturing plants in 

13 countries. Its products are marketed in 143 countries. Lilly’s net sales in

2005 were $14.6 billion. Eli Lilly strives to grow sales by 6 percent to 

7 percent each year. In 2005, $3 billion was spent on R&D, a $334.4 million 

increase from the previous year. Currently, R&D represents 20.7 percent of 

sales. Lilly’s total R&D investment in the last 5 years from continuing 

operations was $12.5 billion. We met with managers from Eli Lilly’s 

Corporate Headquarters in Indianapolis, Ind. 

Caterpillar Corporation 

Caterpillar is a technology leader and the world’s leading manufacturer of 

construction and mining equipment, diesel and natural gas engines, and 

industrial gas turbines. In 2005, its total sales and revenues were 

$36.3 billion, and its total R&D expenditures exceeded $1 billion, 

compared with $20.5 billion sales and $696 million R&D in 2001. Between

2001 and 2005, the average return on equity of its stockholders’ shares 

more than doubled. Caterpillar has over 85,000 employees, and over 

105,000 people are employed by Caterpillar’s dealers worldwide. We met 

with managers responsible for Caterpillar’s New Product Introduction 

(NPI) process in Peoria, Illinois. 

Page 37 GAO-07-388 Best Practices
Appendix II: Comments from the Department of Defense Appendix II: Comments from the Department 

of Defense 

Page 38 GAO-07-388 Best Practices
Appendix II: Comments from the Department of Defense 

Page 39 GAO-07-388 Best Practices
Appendix II: Comments from the Department of Defense 

Page 40 GAO-07-388 Best Practices
Appendix II: Comments from the Department of Defense 

Page 41 GAO-07-388 Best Practices
Related GAO Products Related GAO Products 

Best Practices: Stronger Practices Needed to Improve DOD Technology 

Transition Processes. GAO-06-883. Washington, D.C.: September 14, 2006. 

Defense Acquisitions: Major Weapon Systems Continue to Experience 

Cost and Schedule Problems under DOD’s Revised Policy. GAO-06-368. 

Washington, D.C.: April 13, 2006. 

DOD Acquisition Outcomes: A Case for Change. GAO-06-257T. 

Washington, D.C.: November 15, 2005. 

Defense Acquisitions: Stronger Management Practices Are Needed to 

Improve DOD’s Software-Intensive Weapon Acquisitions. GAO-04-393. 

Washington, D.C.: March 1, 2004. 

Best Practices: Setting Requirements Differently Could Reduce Weapon 

Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.: February 

11, 2003. 

Best Practices: Capturing Design and Manufacturing Knowledge Early 

Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15,

2002. 

Defense Acquisitions: DOD Faces Challenges in Implementing Best 

Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.

Best Practices: Better Matching of Needs and Resources Will Lead to 

Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 

2001. 

Best Practices: A More Constructive Test Approach Is Key to Better 

Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31, 

2000. 

Defense Acquisition: Employing Best Practices Can Shape Better 

Weapon System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 

26, 2000. 

Best Practices: DOD Training Can Do More to Help Weapon System 

Programs Implement Best Practices. GAO/NSIAD-99-206. Washington,

D.C.: August 16, 1999. 

Page 42 GAO-07-388 Best Practices
Related GAO Products 

Best Practices: Better Management of Technology Development Can 

Improve Weapon System Outcomes. GAO/NSIAD-99-162. Washington, 

D.C.: July 30, 1999. 

Defense Acquisitions: Best Commercial Practices Can Improve Program 

Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999. 

Defense Acquisition: Improved Program Outcomes Are Possible. 

GAO/T-NSIAD-98-123. Washington, D.C.: March 17, 1998. 

Best Practices: DOD Can Help Suppliers Contribute More to Weapon 

System Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998.

Best Practices: Successful Application to Weapon Acquisition Requires

Changes in DOD’s Environment. GAO/NSIAD-98-56. Washington, D.C.: 

February 24, 1998. 

Best Practices: Commercial Quality Assurance Practices Offer 

Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26, 

1996. 

(120516) Page 43 GAO-07-388 Best Practices
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER